Definition
The Replication Crisis is an ongoing methodological crisis in which many scientific studies are difficult or impossible to replicate. Because independent replication is the ultimate arbiter of what is real in science, the failure to reproduce results undermines the reliability of entire bodies of literature, especially in the social and medical sciences.
Why It Matters
If science cannot be reproduced, it is noise, not knowledge. The replication crisis is a warning that our knowledge factory is leaking false positives, leading to wasted billions in research and flawed public policy based on mirages.
Core Concepts
- The Ultimate Filter: In science, any one study can be a fluke or biased. Only phenomena that are real will consistently show up in the data of independent researchers using the same methods.
- The 2015 Benchmark: An attempt to replicate 100 psychological studies found that only 39 could be successfully reproduced.
- Causes of Failure:
- p-Value Hacking and researcher degrees of freedom.
- Publication Bias: The “File Drawer Problem,” where negative results are not published, leaving only the lucky false positives in the literature.
- Lack of Power: Small sample sizes that are too “noisy” to reliably detect small effects.
- Innocent Cheating: Most failure is not due to fraud, but to researchers unconsciously exploiting their degrees of freedom to reach publishable results.