A t-test compares two group means. So what do you do with three or more groups? The tempting answer — "just run a t-test on every pair" — is exactly the mistake ANOVA exists to prevent.
Why is ANOVA better than running multiple t-tests? Every t-test at α = 0.05 carries a 5% risk of a false positive, and those risks add up. Comparing 3 groups pairwise takes 3 t-tests, which pushes the chance of at least one false positive to about 14%; 4 groups need 6 tests (about 26%) and 5 groups need 10 (about 40%), treating the tests as independent. One-way ANOVA tests all the groups at once and keeps the overall false-positive rate at 5%. If ANOVA is significant, a post-hoc test such as Tukey’s HSD (see the Tukey q table) then finds which groups differ.
The multiple-comparisons problem
Every statistical test carries a false-positive risk. At α = 0.05, each t-test has a 5% chance of declaring a difference that isn't real. Run several and those risks compound:
- 3 groups → 3 pairwise t-tests → chance of at least one false positive ≈ 14%
- 4 groups → 6 t-tests → ≈ 26%
- 5 groups → 10 t-tests → ≈ 40%
By the time you have five groups, you're more likely than not to find a "significant" difference by chance alone. This is the family-wise error rate problem.
ANOVA's job: it performs a single test of the null hypothesis "all group means are equal," holding the overall false-positive rate at 5% no matter how many groups you have.
How ANOVA works, briefly
One-way ANOVA (analysis of variance) compares the variation between group means to the variation within groups. If the between-group variation is large relative to within-group noise, the F-statistic is large and the p-value small — evidence that at least one group differs. Critical F-values come from the F-distribution table.
| T-Test | One-Way ANOVA | |
|---|---|---|
| Number of groups | Exactly 2 | 3 or more |
| Test statistic | t | F |
| Controls false positives across groups | — | Yes |
| Tells you which groups differ | Yes (only 2) | No — needs a post-hoc test |
ANOVA is significant — now what?
A significant ANOVA tells you at least one group differs, but not which. To find the specific pairs, you run a post-hoc test (Tukey's HSD, Bonferroni) that compares pairs while still controlling the overall error rate. Never go back to plain uncorrected t-tests — that reintroduces the very problem ANOVA solved.
Quick decision
- Two groups → t-test
- Three or more groups → one-way ANOVA, then a post-hoc test if significant
- Non-normal data → non-parametric equivalents (Mann-Whitney for two groups; Kruskal-Wallis for more)
Related: Choosing a statistical test · Interpreting p-values · F-table.
See also: Two-way ANOVA explained