Reliability Is Not Validity

Reliability asks whether a score is consistent or precise enough. Validity asks whether evidence supports the meaning and use claimed for that score. They are connected but not interchangeable.

Imagine a bathroom scale that always reads ten pounds too high. Its readings may be consistent, but the interpretation is wrong. Personality measurement is more complex because the construct itself is not directly visible.

An assessment can increase internal consistency by asking the same idea repeatedly in slightly different words. That may create an impressive statistic while narrowing the construct until important facets disappear. This is why 702it begins with broad facet definitions and will not choose questions only to maximize coefficient alpha.

Reliability itself has forms. Internal consistency concerns relationships among items at one administration. Test–retest evidence concerns stability across time. Model-based precision may vary across the score range. Each estimate carries assumptions and uncertainty.

Validity is an accumulated argument using content, response process, internal structure, relationships with other variables, fairness, and consequences for the intended use. Evidence for reflection and communication does not automatically support hiring prediction.

The methodology should report multiple appropriate indicators and the decisions they support. “Validated” without a version, population, interpretation, and use is too broad.

Try this: For any assessment statistic, ask, “What question does this number answer—and what does it not answer?”

Related terms: Reliability · Internal consistency · Test–retest reliability · Validity · Precision

Research starting points: McNeish on reliability estimates, https://pubmed.ncbi.nlm.nih.gov/28557467/; Standards for Educational and Psychological Testing, https://www.testingstandards.net/