When you run a hypothesis test, you make a yes/no decision about the null hypothesis — and any decision can be wrong. There are exactly two kinds of mistake, and statisticians gave them famously unmemorable names: Type I and Type II. Here's how to keep them straight for good.
The two errors
| H₀ is really true | H₀ is really false | |
|---|---|---|
| You reject H₀ | ❌ Type I error (false positive) | ✅ Correct |
| You keep H₀ | ✅ Correct | ❌ Type II error (false negative) |
Type I error — the false positive
You conclude there's an effect when there really isn't one. You "cried wolf." The probability of this is α (alpha) — the significance level you choose, usually 0.05. Setting α = 0.05 literally means "I accept a 5% chance of a false positive."
Type II error — the false negative
You miss a real effect — it was there, but your test didn't detect it. The probability of this is β (beta). The flip side, 1 − β, is the test's power: the probability of correctly detecting a real effect.
A memory hook: a Type I error convicts an Innocent null (false alarm). A Type II error lets a guilty null go free (missed detection).
The trade-off
The two errors pull against each other. If you make α very strict (say 0.001) to avoid false positives, you make it harder to reach significance — so you miss more real effects (β rises). Loosen α and you catch more real effects but raise your false-positive rate. You can't drive both to zero at once with a fixed sample.
The escape hatch: sample size
The one lever that reduces both error rates simultaneously is a bigger sample. More data sharpens your estimate, so you can keep α low and still have high power. This is why studies are "powered" — you choose a sample size that gives you, typically, 80% power (β = 0.20) to detect the smallest effect you care about.
Sample Size Calculator →Find the n you need for a target confidence and margin of error.Which error is worse?
It depends entirely on context:
- Medical screening: a Type II error (missing a disease) can be fatal, so tests are often tuned to minimise false negatives even at the cost of more false positives.
- Spam filters / justice: a Type I error (flagging a legitimate email, convicting an innocent person) is the costlier one, so those systems favour caution.
Deciding which error you can least afford — before you collect data — is part of good study design.
Related reading: Interpreting p-values · Null & alternative hypothesis · Effect size explained · Confidence intervals.