Written by Dackohn; last reviewed November 2026.
P-hacking means trying analyses until one reaches p < .05 and reporting only that one. It inflates false positives far above 5%: 20 independent tests have a 64% chance of at least one significant result by luck, and checking an experiment repeatedly raises the rate to about 19%. Choosing the analysis in advance prevents it.
Why it matters
The 5% false-positive rate holds only if you run one planned test. Each extra attempt is another chance for noise to look like a finding. P-hacking is often unintentional: each decision seems reasonable at the time.
Common forms
- Testing many outcomes and reporting the one that worked.
- Peeking: checking the p-value as data arrive and stopping when it is significant.
- Trying subgroups (men, women, under 30…) until one shows an effect.
- Flexible cleaning: removing outliers only when that helps.
- Switching tests or adding covariates until the result crosses .05.
- HARKing: Hypothesising After the Results are Known, then presenting the idea as planned.
The numbers: many tests
With k independent tests of true null hypotheses, the chance of at least one false positive is 1 − 0.95k. Five tests give 22.6%. Twenty tests give 64.2%.
The numbers: peeking
We simulated 20,000 experiments with no real effect. In each one, we ran a t-test after every 10 participants per group, up to 100 per group, and stopped at the first p < .05. 19.4% of them ended “significant”, nearly four times the promised 5%, and every one of those findings is false.
How to spot it
- Many p-values just under .05 across a set of results.
- Unusual subgroups or outcomes that were not part of the original question.
- Sample sizes with no stated rationale.
- Large effects from small studies that fail to replicate.
How to avoid it
- Preregister your hypothesis, outcome, sample size and analysis before collecting data.
- Fix the sample size in advance with a power calculation.
- Correct for multiple tests with the multiple comparisons calculator.
- Use a sequential test if you need to monitor results as they arrive.
- Report everything you tried, and label exploratory analyses as exploratory. Exploring is fine; the problem is presenting exploration as a planned test.
Try it yourself
False Positive Risk Calculator →See how the risk grows with the number of tests. Multiple Comparisons Correction →Adjust p-values with Bonferroni, Holm or Benjamini–Hochberg.Frequently asked questions
Is p-hacking always deliberate?
No. It often comes from reasonable-looking decisions made after seeing the data, which is why committing to a plan in advance helps.
Can I look at my data before the study ends?
Yes, if you use a method designed for it, such as a sequential test or a pre-planned interim analysis with an adjusted threshold.
Does correcting for multiple tests fix p-hacking?
Only if you count every test you actually ran. Corrections cannot account for analyses that go unreported.
Related reading: How to interpret a p-value · Type I and Type II errors · Statistical power explained
Sources & further reading: NIST/SEMATECH e-Handbook of Statistical Methods · OpenStax Introductory Statistics. How we compute & validate: methodology. Written & reviewed by Dackohn; last reviewed August 2026.