Validity:

  • Represents the degree to which the observed scores from a test actually represent the attribute or construct of interest
  • Reliability doesn’t guarantee validity, but without reliability you have no hope of validity

Alternate forms reliability:

  • Similar to parallel tests, argues that it is effectively impossible in practice as:
    • We can never be sure the true scores associated with the two tests are in fact measuring the same construct
    • The two tests are not based on the same items
  • The only way alternate forms reliability would truly work is if people’s memories were erased after completing the first test

Carry-over effects:

  • Different responses on different tests result due to external effects e.g. changes in selfconfidence, illness

Test- retest reliability:

  • Next best option is to create only one test, but administer it on two different occasions
  • In this case, the same exact items are presented on both occasions, so the true score should represent the same construct
  • Unsure of the true score variance
  • Expect the construct of interest to be stable (e.g. intelligence)
  • Some constructs in psychology are relatively stable, and differences between time 1 and time

2 are due to measurement error o Despite being stable, carry over effects may result as some people may improve more than others

Equal error variances:

  • Assumption is tough to meet, but not totally impossible
  • Assume error is equal between times 1- time 2.

Test-retest interval:

 

All other things being equal, the magnitude of the interval between the two testing sessions will affect the magnitude of the correlation between the scores

  • g. greater differences between testing an 18 yo and again 40 years later, rather than testing them the next day.
  • Developmental changes need to be considered, particularly in children

Internal-consistency reliability:

  • Internal consistency offers a practical alternative to the alternative forms procedure and the test-retest procedure
  • The internal consistency reliability procedure requires respondents to complete one form of the test
  • It only needs to be completed on one occasion
  • The ‘trick’ is to treat different parts or items within the same test as different forms of the test
  • There are two fundamental factors that affect internal consistency reliability:
    • The degree of consistency between parts/items in the test
    • The length of the test (all other things being equal a longer test will yield more reliable results)

Cronbach’s Alpha:

  • Represents the reliability of all possible split-halves
  • Instead of splitting up the test into two halves, calculate the correlation between all of the items
  • Each individual item was a portion (‘half’) of the test
  • Represents the ratio of true score variance to total variance, which is how reliability has been defined at the theoretical level