Test-retest reliability is a psychometric index expressed as the correlation coefficient (typically Pearson's r or intraclass correlation coefficient ICC) between scores obtained when the same individual completes the same test at different time points. Values approaching 1.0 indicate stable measurement that reflects true individual ability rather than random error. For cognitive tests, r = 0.70 is considered the minimum for practical utility, 0.80+ is good, and 0.90+ is excellent. Practice effects and diurnal variation are among the factors that can reduce apparent reliability in cognitive assessment.
Statistical Foundation
Test-retest reliability corresponds to the reliability definition in Classical Test Theory (CTT). CTT formalizes observed scores as the sum of true score plus error: X = T + E. The reliability coefficient represents the proportion of observed score variance attributable to true score variance. In the test-retest method, the correlation between two administrations estimates this reliability coefficient. However, measurement interval introduces complications: too short an interval inflates correlation through memory effects, while too long an interval deflates it through genuine ability changes.
Factors Affecting Reliability
Multiple factors reduce cognitive test reliability. Practice effects are one such factor: in a study that repeatedly tested healthy adults at short intervals, most tests showed gains that were largest early on. These gains are thought to reflect familiarity with the task and its strategies. Diurnal variation (circadian arousal fluctuations), day-to-day variation (differences in sleep quality, stress, caffeine intake), and motivational fluctuations all introduce error variance. Insufficient trial counts increase measurement error through inadequate sampling of the performance distribution. The Spearman-Brown prophecy formula predicts how reliability changes when test length (the number of trials) is increased.
Interpreting Score Changes
Single-trial scores have low reliability and poorly represent individual ability. To determine whether a score change represents genuine improvement versus measurement noise, the standard error of measurement (SEM) derived from the reliability coefficient is used as a reference.