What statistical power means
Power is the probability that a study detects an effect that really exists. A study with 80% power finds a true effect of the assumed size four times out of five, and misses it the fifth time. Low power is one of the most common reasons studies fail to replicate. The ideas behind it are explained in statistical power explained.
Worked example
You plan to compare two independent groups and expect a medium effect, d = 0.5. At α = .05 (two-sided) and 80% power you need 64 per group, 128 in total. With only 30 per group, power falls to about 48%, worse than a coin flip. Smaller effects cost far more: d = 0.3 needs 176 per group and d = 0.2 needs 394.
How to choose the effect size
- Use the smallest effect that would matter in practice. This is usually the best choice.
- Use estimates from earlier studies with care, because published effects tend to be inflated.
- Use 0.2, 0.5 and 0.8 only as a last resort, as these are rough conventions and not properties of your field.
How it is calculated
Power is computed exactly from the noncentral t distribution with SciPy, not from a normal approximation. Results were checked against the statsmodels power module in 240 scenarios: power agreed to 15 decimal places and every required sample size matched.
For proportions, use the A/B test sample size calculator. For a margin of error rather than a test, use the sample size calculator.
Frequently asked questions
What is a good level of statistical power?
80% is the usual minimum, and 90% is common in clinical trials and when a false negative would be costly.
How many participants do I need for d = 0.5?
For two independent groups at alpha = .05 two-sided and 80% power, you need 64 per group, or 128 in total.
Should I calculate power after the study?
No. Post-hoc power computed from the observed effect is just a restatement of the p-value. Report a confidence interval instead.
Why do paired designs need fewer participants?
Here d is standardised by the SD of the differences. When the two measurements are correlated, that SD is smaller, so the same raw change gives a larger d.