When to use a t-test
A t-test compares means when the population standard deviation is unknown and must be estimated from the sample — which is almost always the case with real data. There are three variants, and choosing correctly is the most common sticking point:
- One-sample t-test — compare one group's mean against a known or hypothesized value. Example: “Is this factory's mean fill volume different from the 500 ml target?”
- Two-sample (independent) t-test — compare the means of two separate, unrelated groups, where each observation belongs to exactly one group. Example: treatment vs. control, version A vs. version B.
- Paired t-test — compare two measurements on the same subjects or matched pairs. Example: blood pressure before and after a drug. Pairing removes between-subject variability and usually gives more statistical power.
With three or more groups, do not run multiple t-tests — that inflates the false-positive rate. Use ANOVA instead. Unsure which test fits? The which-test wizard walks you through it, and the guide on t-test vs z-test covers the closest alternative.
Assumptions, and how to check them
The t-test rests on four assumptions. It tolerates mild violations well, but large ones change your conclusions:
- Independence. Observations must not influence one another. This is a design property — random sampling, no unit measured twice within a group. There is no after-the-fact fix.
- Normality. The sampling distribution of the mean should be roughly normal. By the Central Limit Theorem, with n ≳ 30 per group this holds even for skewed data. For small samples, inspect a histogram or Q–Q plot, or run a Shapiro–Wilk test with the normality test.
- Equal variances (two-sample only). Student's t-test assumes the groups share one variance; Welch's t-test does not. This calculator applies Welch's correction by default — the modern recommendation, since it costs almost nothing when variances are equal and protects you when they are not.
- Continuous, interval data. A t-test compares means, so the outcome must be numeric. Ordinal or ranked data call for the Mann–Whitney U test instead.
Worked example
A teacher compares two study methods. Group A scores 85, 88, 90, 79, 92 (mean 86.8) and Group B scores 78, 82, 80, 75, 85 (mean 80.0). A two-sample Welch's t-test on these gives about t ≈ 2.40, df ≈ 7.4, and p ≈ 0.046. Because p is just under 0.05, the difference is statistically significant at α = 0.05 — but only barely, and with five students per group the estimate is noisy. This is exactly when you should report an effect size (here Cohen's d ≈ 1.5, a large effect) and, ideally, collect more data. Paste these numbers into the calculator above to see the full breakdown.
Common mistakes
- Running many pairwise t-tests across several groups instead of ANOVA, which inflates the Type I error rate.
- Using an independent test on paired data. If the same subjects are measured twice, the paired test is both correct and more powerful.
- Reading the p-value as the probability the null is true. It is not — see how to interpret a p-value.
- Confusing statistical with practical significance. With large n, trivial differences turn “significant.” Always pair the p-value with an effect size.
- Defaulting to one-tailed. Only use a one-sided alternative if you committed to the direction before seeing the data.
What to do if assumptions fail
If normality is clearly violated and the sample is small, switch to the non-parametric Mann–Whitney U test (independent groups) or the Wilcoxon signed-rank test (paired). If variances differ sharply, Welch's t-test — used here — already handles it. If observations are not independent (clustered or repeated measures), you need a mixed-effects model, which is beyond a simple t-test. For the full computation details and how results are validated against SciPy and cross-checked with NIST reference datasets, see our methodology page.
Frequently asked questions
A t-test is a statistical test that determines whether there is a significant difference between means. It compares the observed difference to what we would expect by chance, accounting for sample size and variability.
The result is a t-statistic and a p-value. If the p-value is below your chosen significance level (usually 0.05), you conclude the difference is unlikely due to chance.
One-sample t-test: You have one group and want to compare its mean to a known reference value (e.g. “Is this class's average exam score different from the national mean of 70?”)
Two-sample independent t-test: You have two separate, unrelated groups (e.g. control vs treatment, men vs women). Each participant appears in only one group.
Paired t-test: You have one group measured twice — before and after an intervention, or matched pairs. The same subjects appear in both conditions.
The p-value is the probability of observing a result as extreme as yours (or more extreme) if the null hypothesis were true. It is NOT the probability that the null hypothesis is true.
A p-value of 0.03 means: “If the groups were truly equal, we would see a difference this large or larger only 3% of the time by chance.” With α = 0.05, this is considered significant — we reject H₀.
- Normality: The data (or sampling distribution of the mean) should be approximately normal. With n > 30, the Central Limit Theorem means this is usually satisfied regardless of the data distribution.
- Independence: Observations must be independent of each other.
- Equal variances (two-sample): The Welch's t-test (default in most software) relaxes this assumption.
- Random sampling: Data should be a random sample from the population of interest.
If normality is violated and your sample is small (<30), consider the Mann-Whitney U test.
A result can be statistically significant (p < 0.05) but practically meaningless, especially with large samples. Always report an effect size alongside your p-value. Use the Effect Size Calculator to compute Cohen's d and see how large the difference actually is.