{"id":4278,"date":"2018-09-18T18:47:04","date_gmt":"2018-09-18T22:47:04","guid":{"rendered":"https:\/\/www.amyork.ca\/academic\/zz\/?p=4278"},"modified":"2018-09-18T18:57:52","modified_gmt":"2018-09-18T22:57:52","slug":"reliability-and-empirical-estimates","status":"publish","type":"post","link":"https:\/\/www.amyork.ca\/academic\/zz\/psychological-measurement\/reliability-and-empirical-estimates\/","title":{"rendered":"Reliability and empirical estimates"},"content":{"rendered":"<p><strong>Validity:<\/strong><\/p>\n<ul>\n<li>Represents the degree to which the observed scores from a test actually represent the attribute or construct of interest<\/li>\n<li>Reliability doesn\u2019t guarantee validity, but without reliability you have no hope of validity<\/li>\n<\/ul>\n<p><strong>Alternate forms reliability:<\/strong><\/p>\n<ul>\n<li>Similar to parallel tests, argues that it is effectively impossible in practice as:\n<ul>\n<li>We can never be sure the true scores associated with the two tests are in fact measuring the same construct<\/li>\n<li>The two tests are not based on the same items<\/li>\n<\/ul>\n<\/li>\n<li>The only way alternate forms reliability would truly work is if people\u2019s memories were erased after completing the first test<\/li>\n<\/ul>\n<p><strong>Carry-over effects:<\/strong><\/p>\n<ul>\n<li>Different responses on different tests result due to external effects e.g. changes in selfconfidence, illness<\/li>\n<\/ul>\n<p><strong>Test- retest reliability:<\/strong><\/p>\n<ul>\n<li>Next best option is to create only one test, but administer it on two different occasions<\/li>\n<li>In this case, the same exact items are presented on both occasions, so the true score should represent the same construct<\/li>\n<li>Unsure of the true score variance<\/li>\n<li>Expect the construct of interest to be stable (e.g. intelligence)<\/li>\n<li>Some constructs in psychology are relatively stable, and differences between time 1 and time<\/li>\n<\/ul>\n<p>2 are due to measurement error o Despite being stable, carry over effects may result as some people may improve more than others<\/p>\n<p><strong>Equal error variances:<\/strong><\/p>\n<ul>\n<li>Assumption is tough to meet, but not totally impossible<\/li>\n<li>Assume error is equal between times 1- time 2.<\/li>\n<\/ul>\n<p><strong>Test-retest interval:<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p>All other things being equal, the magnitude of the interval between the two testing sessions will affect the magnitude of the correlation between the scores<\/p>\n<ul>\n<li>g. greater differences between testing an 18 yo and again 40 years later, rather than testing them the next day.<\/li>\n<li>Developmental changes need to be considered, particularly in children<\/li>\n<\/ul>\n<p><strong>Internal-consistency reliability:<\/strong><\/p>\n<ul>\n<li>Internal consistency offers a practical alternative to the alternative forms procedure and the test-retest procedure<\/li>\n<li>The internal consistency reliability procedure requires respondents to complete one form of the test<\/li>\n<li>It only needs to be completed on one occasion<\/li>\n<li>The \u2018trick\u2019 is to treat different parts or items within the same test as different forms of the test<\/li>\n<li>There are two fundamental factors that affect internal consistency reliability:\n<ul>\n<li>The degree of consistency between parts\/items in the test<\/li>\n<li>The length of the test (all other things being equal a longer test will yield more reliable results)<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><strong>Cronbach\u2019s Alpha:<\/strong><\/p>\n<ul>\n<li>Represents the reliability of all possible split-halves<\/li>\n<li>Instead of splitting up the test into two halves, calculate the correlation between all of the items<\/li>\n<li>Each individual item was a portion (\u2018half\u2019) of the test<\/li>\n<li>Represents the ratio of true score variance to total variance, which is how reliability has been defined at the theoretical level<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Validity: Represents the degree to which the observed scores from a test actually represent the attribute or construct of interest Reliability doesn\u2019t guarantee validity, but without reliability you have&hellip; <a class=\"continue\" href=\"https:\/\/www.amyork.ca\/academic\/zz\/psychological-measurement\/reliability-and-empirical-estimates\/\">Continue Reading<span> Reliability and empirical estimates<\/span><\/a><\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[107],"tags":[],"_links":{"self":[{"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/posts\/4278"}],"collection":[{"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/comments?post=4278"}],"version-history":[{"count":0,"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/posts\/4278\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/media?parent=4278"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/categories?post=4278"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.amyork.ca\/academic\/zz\/wp-json\/wp\/v2\/tags?post=4278"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}