What a normality test tells you
Many common tests, including the t-test and ANOVA, assume the data (or the residuals) come from a roughly normal distribution. A normality test checks that assumption. Its null hypothesis is that the data are normal, so a small p-value is evidence that they are not, while a large p-value only means no departure was detected.
Which test to use
- Shapiro–Wilk is the most powerful general-purpose test for small and moderate samples and is the one most journals expect. It is the main result shown here.
- Anderson–Darling gives extra weight to the tails, so it is good at detecting outliers and heavy tails.
- D’Agostino–Pearson (K²) combines skewness and kurtosis, which helps you see how the data depart from normality. It needs at least 8 values.
Worked example
The example data are 20 reaction times in milliseconds. Most fall between 390 and 540 ms, but two slow responses (620 and 702 ms) create a long right tail. Shapiro–Wilk gives W = .81, p = .001, so the data are clearly not normal, and the skewness of 1.75 shows strong positive skew. The Anderson–Darling and D’Agostino tests agree (p = .002 and p < .001). On the Q-Q plot, the points follow the line in the middle but bend sharply upward at the right. Reaction times are almost always right-skewed, so a log transformation or a nonparametric test is the usual remedy.
How to read a Q-Q plot
- Points on the line: the data are consistent with a normal distribution.
- Curving upward at the right end (and often below the line at the left): right skew, a long upper tail.
- An S shape with both ends moving away from the line: heavier tails than normal; the reverse S means lighter tails.
- A few isolated points far off the line: outliers worth checking.
Sample size changes what the test means
With small samples (under about 20), normality tests rarely reject even when the data are skewed, so rely on the plot and on what you know about the variable. With large samples (several hundred), they reject for departures too small to matter, and the Central Limit Theorem makes t-tests and ANOVA robust anyway. For that reason, use the test together with the Q-Q plot and the skewness, not on its own. Testing the residuals of a model, rather than the raw data of each group, is often the more relevant check.
Computation
Shapiro–Wilk and D’Agostino–Pearson use SciPy’s shapiro and normaltest; Shapiro–Wilk uses the same algorithm as R’s shapiro.test. The Anderson–Darling p-value uses the Stephens/D’Agostino approximation for an estimated mean and standard deviation, and matches statsmodels’ normal_ad. Skewness and excess kurtosis are the bias-corrected sample estimates.
Frequently asked questions
What p-value means my data are normal?
A p-value above your significance level (usually 0.05) means the test found no evidence against normality. It does not prove the data are normal, especially with small samples.
Should I use Shapiro-Wilk or Kolmogorov-Smirnov?
Shapiro-Wilk. The Kolmogorov-Smirnov test has low power for normality when the mean and standard deviation are estimated from the data, and its Lilliefors-corrected version is still usually weaker than Shapiro-Wilk.
My sample is large and the test says not normal. Can I still use a t-test?
Often yes. With large samples the t-test is robust to moderate non-normality because of the Central Limit Theorem, and normality tests flag tiny departures. Check the Q-Q plot and the skewness before deciding.
Do I test each group or the residuals?
For a t-test or ANOVA, check normality within each group or, equivalently, the residuals from the model. Pooling groups with different means can make the combined data look non-normal even when each group is normal.
How do I report a Shapiro-Wilk test in APA style?
Give W, the sample size and the p-value, for example: a Shapiro-Wilk test indicated that the data were not normally distributed, W(20) = .81, p = .001.