Two-Proportions Z-Test

Test whether two sample proportions are significantly different — ideal for A/B tests and clinical trials.

A two-proportion z-test checks whether two groups differ in the rate of a yes/no outcome — conversion in an A/B test, recovery rate between treatments, pass rate between cohorts. You enter the successes and totals for each group; it returns the difference in proportions, a z-statistic, and a p-value. It assumes independent groups and large enough counts (at least about 10 successes and 10 failures per group). Results are validated against SciPy to full precision.

Enter Counts

Group 1

Group 2

When to use it

This is the standard test for comparing two rates — most commonly an A/B test where you compare the conversion rate of version A against version B. Each observation is a yes/no outcome and each subject belongs to exactly one group.

Assumptions

Worked example

Version A converts 120 of 2,000 visitors (6.0%); version B converts 150 of 2,000 (7.5%). The z-test returns z ≈ 2.0 and p ≈ 0.046 — a statistically significant lift at α = 0.05, though modest. Enter your counts above for the exact result.

Common mistakes

Guide: Two-Proportions Z-Test

The two-proportions z-test tests whether the proportion of successes in two independent groups is statistically different. The null hypothesis is H₀: p₁ = p₂ — the proportions are the same in both groups.

Example: you run an A/B test where Version A converted 45/100 visitors and Version B converted 35/100. Is this 10-percentage-point difference statistically significant, or could it be due to random chance?

A "success" is just the event you're counting — whatever you want to compare between the two groups:

  • A/B test: number of users who clicked / converted
  • Clinical trial: number of patients who recovered
  • Survey: number of respondents who answered "yes"
  • Quality control: number of defective items

The 95% CI for (p₁ − p₂) gives you a range of plausible values for the true difference in proportions. If this interval does not contain 0, the difference is statistically significant at α = 0.05.

For example, a CI of [0.02, 0.18] means the true difference is likely between 2 and 18 percentage points, with Group 1 being higher — a meaningful and significant result.

Cohen's h is the effect size measure for proportions. Unlike the raw difference p₁ − p₂, Cohen's h accounts for the non-linearity of proportions near 0 and 1.

The formula is: h = 2·arcsin(√p₁) − 2·arcsin(√p₂)

Cohen's benchmarks: small (0.2), medium (0.5), large (0.8). A statistically significant result with a small Cohen's h may not be practically meaningful — you need both a significant p-value and a meaningful effect size before acting on the result.

For the normal approximation to be valid, each group should satisfy: np ≥ 5 and n(1−p) ≥ 5. With very small proportions (like 1-2% conversion rates), you may need hundreds or thousands of observations per group to detect meaningful differences.

Use the Sample Size Calculator to find the required n for a given power and minimum detectable difference.

Related Tools

Chi-Square Test Confidence Interval (Proportion) Sample Size Calculator Binomial Distribution