What the sign test does
The sign test uses only the direction of each paired difference. If there is no real change, pluses and minuses should be about equally likely, so the number of pluses follows a Binomial(n, 0.5) distribution. It assumes almost nothing about the data, so it works when you only know that something got better or worse.
Worked example
Ten people’s sleep scores change by +3, −1, +5, +6, −2, +8, +4, +9, +7 and +10: 8 up and 2 down. The exact two-sided p-value is .109. The Wilcoxon signed-rank test on the same data gives p = .010, because it also uses the fact that the two falls were the smallest changes. See sign test vs Wilcoxon.
When to use it
- You only have directions: better or worse, preferred A or preferred B.
- The differences are strongly skewed, so Wilcoxon’s symmetry assumption is doubtful.
- You want a simple exact test that is easy to explain.
Accuracy
p-values are exact binomial probabilities and match SciPy’s binomtest with p = 0.5. A two-sided test needs at least 6 non-zero pairs to reach p < .05: 6 out of 6 gives p = .031, while 5 out of 5 gives p = .063.
Frequently asked questions
How do you calculate a sign test?
Count the positive and negative differences and drop zeros. The p-value is the binomial probability of a split at least this extreme with p = 0.5.
What is the minimum sample size for a sign test?
For a two-sided test at the 5% level, at least 6 non-zero pairs. With 5, even a clean sweep gives p = .063.
Is the sign test less powerful than Wilcoxon?
Usually, because it ignores the size of the differences. It can be better when differences are heavy-tailed or strongly skewed.