Definition
Publication bias occurs when the outcome of an experiment or research study influences the decision whether to publish or otherwise distribute it. It most commonly manifests as the “file drawer problem,” where statistically significant positive results are published while null or negative results are hidden.
Why It Matters
Publication bias creates a “hall of mirrors” in science. If we only see the “wins,” we build medical treatments and social policies on false premises. The cost is the “Replication Crisis”—a world where we “know” many things that aren’t actually true, leading to wasted lives and billions in dead-end research. It is a fundamental corruption of the scientific record.
Core Concepts
- The File Drawer Problem: Unpublished negative studies sitting in researchers’ file drawers, skewing the available literature.
- p-Hacking: Manipulating data analysis to achieve a statistically significant -value () to ensure publication.
- How to read: “The p is less than 0.05.”
- Meaning: Journals favor “significant” results—researchers manipulate analysis until crosses the 0.05 threshold.
- Meta-analysis Distortion: Systemic overestimation of an effect size because the meta-analysis only includes published, positive studies.