Definition and Mechanism

Selection bias is a systematic error that arises when the sample studied is not representative of the target population. It occurs during participant recruitment, data collection, or analysis when certain individuals are more likely to be included than others. The resulting conclusions may be valid for the sample but misleading when generalized.

Common Forms

Self-selection bias occurs when participants volunteer, attracting those with stronger opinions or motivation. Non-response bias emerges when certain groups systematically decline to participate. Survivorship bias focuses only on those who "survived" a process, ignoring dropouts.

In online surveys and ranking tools, the users who engage are not necessarily representative of the general population, and a reference group built from those users is skewed in the same way.

Impact on Ranking Accuracy

If a ranking's underlying data suffers from selection bias, your percentile may not reflect your true position in the broader population. For instance, if a fitness app's user base skews toward active individuals, being "average" among its users could mean you are above average in the general population.

Mitigation Strategies

Researchers address selection bias through methods such as random sampling, stratification, and weighting adjustments. For readers of ranking data, checking the data source and who was surveyed can help reveal selection bias.