- Over 70% of researchers have failed to reproduce others' experiments.
- Publication bias and p-hacking are primary drivers of irreproducible results.
- Only 36% of 100 landmark psychology studies replicated successfully.
The replication crisis is the widespread failure to reproduce published scientific results when independent researchers repeat experiments, casting doubt on findings across psychology, medicine, and other fields.
Why It Matters
Key figure
70%
of researchers have tried and failed to reproduce another scientist's results (Nature, 2016)
Science depends on a simple contract: if one researcher publishes a finding, another researcher following the same method should get the same result. When that contract breaks down at scale, the consequences reach beyond laboratories. Medical treatments built on irreproducible results may not work. Policy decisions grounded in shaky evidence may do harm.
The scale of the problem became measurable in 2016, when the journal Nature surveyed 1,576 researchers. More than 70% reported they had tried and failed to reproduce another scientist's experiments. The breakdown varied by field: 87% of chemists, 77% of biologists, 69% of physicists and engineers, and 67% of medical researchers reported replication failures. More than half had failed to reproduce their own work.
These numbers confirmed what individual replication projects had already shown. The crisis is not confined to one discipline. It touches every field that relies on statistical inference from experimental data.
How It Works
Several structural forces drive replication failures. Publication bias is among the most powerful: journals preferentially publish positive, novel results, creating an incentive to find effects whether or not they are real. Researchers may engage in p-hacking, the practice of testing multiple statistical comparisons and reporting only those that reach significance thresholds. Small sample sizes amplify both problems, making results more susceptible to random noise.
Key figure
36%
of 100 psychology studies replicated successfully (Open Science Collaboration, 2015)
Selective reporting compounds the issue. A researcher who tests twenty hypotheses and publishes the one that reaches p < 0.05 has not demonstrated a real effect. The result is statistically expected by chance. When combined with pressure to publish frequently ("publish or perish"), these incentives create a system that produces unreliable results at industrial scale.
Insufficient methodological detail also plays a role. Many published papers lack the specificity needed for another lab to replicate the work precisely. Missing information about reagent sources, software versions, or environmental conditions can make exact replication impossible, even when the original finding is genuine.
Key Context
Statistician John Ioannidis, then at the University of Ioannina in Greece and Tufts University, published an essay in PLOS Medicine in 2005 arguing mathematically that most published research findings are false. His reasoning: when studies are small, effects are subtle, and researchers have analytical flexibility, the odds tilt toward false positives. The paper has been cited over 10,000 times and is among the most accessed articles in the history of scientific publishing.
In 2015, the Open Science Collaboration, coordinated by University of Virginia psychologist Brian Nosek, attempted to replicate 100 studies from three leading psychology journals. Only 36 of 97 studies with originally significant results replicated successfully, and the average effect size in the replications was roughly half the original. The project generated widespread media attention and made "replication crisis" a mainstream term.
Responses have included the rise of preregistration, where researchers publicly declare their hypotheses and methods before collecting data, and registered reports, where journals peer-review study designs before results exist. The Center for Open Science, founded by Nosek in 2013, hosts hundreds of thousands of preregistrations on its Open Science Framework platform. Funding agencies including the NIH have begun requiring data-sharing plans, and some researchers have called for dedicating at least 0.1% of the NIH budget (roughly $48 million) to funding replication studies specifically.
FAQ
Is the replication crisis limited to psychology?
No. While psychology's Reproducibility Project drew the most attention, replication failures have been documented in medicine, biology, chemistry, economics, and other fields. The 2016 Nature survey showed the problem spans every discipline that relies on statistical methods.
What is the difference between reproducibility and replicability?
Reproducibility means re-analyzing the original data and getting the same results. Replicability means running the same experiment from scratch with new data and getting the same results. The crisis involves both, though replicability failures are generally more concerning because they suggest the original finding may not reflect a real phenomenon.
Does a failure to replicate mean the original study was wrong?
Not necessarily. Replication failures can result from differences in populations, conditions, or methods between the original and replication studies. However, when multiple well-powered replication attempts fail, the original finding becomes substantially less credible.
What is preregistration and how does it help?
Preregistration is the practice of publicly documenting a study's hypotheses, methods, and analysis plan before data collection begins. It reduces p-hacking and selective reporting by making it transparent when researchers deviate from their original plan. Hundreds of thousands of studies have been preregistered through the Center for Open Science.
Related Reading
Sources
- Primary Research:
- Baker, M. (2016). 1,500 scientists lift the lid on reproducibility. Nature, 533, 452-454.
- Open Science Collaboration (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716.
- Additional Context:
- Ioannidis, J.P.A. (2005). Why Most Published Research Findings Are False. PLOS Medicine, 2(8), e124.
- Stanford Encyclopedia of Philosophy. Reproducibility of Scientific Results.
- Chemical & Engineering News (2025). Amid White House claims of a research 'replication crisis,' scientists offer solutions.
Fact Check: Claim-by-Claim Verification Verified
All major claims verified against primary sources. Key statistics (70% replication failure rate, 36% replication success in psychology) confirmed by Nature and Science publications respectively.
Sources used for verification
- 1,500 scientists lift the lid on reproducibility - nature.com
- Estimating the reproducibility of psychological science - science.org
- Why Most Published Research Findings Are False - plos.org
- Reproducibility of Scientific Results - Stanford Encyclopedia of Philosophy
