Question 1

What is a two-proportion z-test used for?

Accepted Answer

It tests whether two independent groups have the same proportion of successes. Common applications: A/B testing (comparing conversion rates for two webpage designs), clinical trials (comparing cure rates for two treatments), surveys (comparing agreement rates for two groups), quality control (comparing defect rates for two production lines).

Question 2

What assumptions does the two-proportion z-test require?

Accepted Answer

Three assumptions: (1) Independent samples — the two groups must be independent of each other. (2) Large enough samples — each group must have at least 10 successes and 10 failures: n₁p̂₁ ≥ 10, n₁(1−p̂₁) ≥ 10, and similarly for group 2. (3) Random sampling — or at least a representative sampling process. If the large-sample condition fails, use Fisher's exact test instead.

Question 3

What is Cohen's h for proportions?

Accepted Answer

Cohen's h = 2arcsin(√p₁) − 2arcsin(√p₂) is the standardized effect size for proportions. The arcsine transformation stabilizes variance across the full proportion range. Benchmarks: |h| = 0.2 (small), 0.5 (medium), 0.8 (large). Unlike the raw difference p₁−p₂, Cohen's h accounts for the fact that a difference of 0.05 near 0.5 is very different statistically from a difference of 0.05 near 0.01.

Question 4

What is the difference between a pooled and unpooled standard error?

Accepted Answer

For testing H₀: p₁=p₂ (the null hypothesis test), use the pooled SE: √[p̂(1−p̂)(1/n₁+1/n₂)] where p̂ is the combined proportion. For the confidence interval for p₁−p₂ (estimation), use the unpooled SE: √[p̂₁(1−p̂₁)/n₁ + p̂₂(1−p̂₂)/n₂]. This calculator uses pooled for the z-test and unpooled for the CI — the statistically correct approach.

Question 5

Can I use this for A/B testing?

Accepted Answer

Yes. Enter conversions and visitors for each variant. Note: this test assumes fixed sample sizes. Sequential A/B testing (peeking at results while collecting data) inflates false positive rates. For proper A/B testing with sequential data, use sequential probability ratio tests or Bayesian A/B testing. Also ensure the two-proportion z-test assumptions hold: at least 10 conversions per group.

Question 6

When should I use Fisher's exact test instead?

Accepted Answer

Use Fisher's exact test when: (1) any expected cell count is less than 5 in a 2×2 table, or (2) either group has fewer than 10 successes or 10 failures. Fisher's exact test makes no large-sample approximation and is exact for any sample size. The two-proportion z-test is preferred for large samples because it is more powerful and produces CIs more easily.

Symbol	Meaning
x₁, x₂	Number of successes in Group 1 and Group 2
n₁, n₂	Sample sizes for each group
p̂₁ = x₁/n₁, p̂₂ = x₂/n₂	Sample proportions
p̂	Pooled proportion under H₀: p₁ = p₂
Cohen's h	2arcsin(√p̂₁) − 2arcsin(√p̂₂) — effect size on arcsine scale

Two-Proportion Z-Test Calculator

Formula

Enter Data

Two-Proportion Z-Test: Formulas

Worked Example: A/B Conversion Test

Assumptions & Alternatives

This test requires:

Use a different test when:

Related Calculators

Disclaimer

Frequently Asked Questions