T-Test vs Z-Test — When to Use Each

The Fundamental Difference: Estimating σ

When you perform a hypothesis test on a mean, you need to know how much sampling variability to expect under H₀. This requires knowing (or estimating) the population standard deviation σ.

In almost all real analyses, σ is unknown — you only have the sample standard deviation s. Substituting s for σ introduces additional uncertainty because s itself is an estimate. The t-distribution accounts for this extra uncertainty by having heavier tails than the normal distribution. The smaller the sample size (degrees of freedom), the heavier the tails, and the larger the critical value needed for significance.

The z-test assumes σ is known. This is realistic in two scenarios: (1) historical data or regulatory standards establish σ with certainty — for example, a production line with decades of records where σ is precisely known; (2) the sample is large enough (n ≥ 30–50) that s is a reliable estimate of σ, making the difference between t and z negligible in practice.

The practical implication: for any sample where you don't know the true population σ (virtually every real study), use the t-test. The t-test is conservative — it is harder to achieve significance — which is appropriate given the extra estimation uncertainty.

When to Use the T-Test

Use the t-test for: (1) One-sample t-test — comparing a sample mean to a known reference value (H₀: μ = μ₀) when σ is unknown. (2) Independent two-sample t-test — comparing means of two independent groups (e.g., treatment vs control) when σ₁ and σ₂ are unknown. Use Welch's t-test (unequal variances assumed) as the default. (3) Paired t-test — comparing before/after measurements on the same subjects, or matched pairs.

The t-test is appropriate regardless of sample size when σ is unknown. For very small samples (n < 10), the t-test also requires the data to be approximately normally distributed — with non-normal small samples, use non-parametric alternatives (Wilcoxon signed-rank, Mann-Whitney U).

Worked example: A new sleep therapy is tested on 18 patients. Mean sleep improvement = 1.4 hours (s = 0.8 hours). Test if the improvement is significant against H₀: μ = 0. t = (1.4 − 0) / (0.8/√18) = 1.4 / 0.189 = 7.41. df = 17. p < 0.0001. Reject H₀. Note: we use t, not z, because σ is unknown — we only have s = 0.8 from this specific sample.

When to Use the Z-Test

Use the z-test for: (1) Proportions — comparing a sample proportion to a reference value (one-proportion z-test) or comparing two proportions (two-proportion z-test). The z-test is the standard test for proportions because proportions have a known variance structure: σ² = p(1−p)/n. (2) Large-sample means — when n is very large (n ≥ 50–100) and σ is known from extensive prior data. In practice, with large n the t and z critical values converge and the distinction is moot.

Proportion z-test formula: z = (p̂ − p₀) / √(p₀(1−p₀)/n) for one proportion; z = (p̂₁ − p̂₂) / √(p̂(1−p̂)(1/n₁ + 1/n₂)) for two proportions, where p̂ is the pooled proportion under H₀.

Worked example: An e-commerce site has a historical conversion rate of 3.5%. After a redesign, 48 out of 1,200 visitors convert (4.0%). Is this increase significant? z = (0.040 − 0.035) / √(0.035×0.965/1200) = 0.005 / 0.00531 = 0.942. p = 0.346 (two-tailed). Fail to reject H₀ — the improvement is not yet statistically significant. Use z (not t) for this proportion test.

What Happens If You Use the Wrong Test?

Using z when you should use t: you underestimate the critical value needed for significance. For small samples, the t-distribution has substantially heavier tails than the normal. At df = 5, the two-tailed critical value at α = 0.05 is t* = 2.571 vs z* = 1.960. If you use z*, you inflate your Type I error rate — you reject H₀ too often, producing false positives. The actual α is higher than you set it.

The error is most severe for small samples: at n = 6 (df = 5), using z instead of t makes your effective α about 10% instead of 5%. At n = 30 (df = 29), the critical values are t* = 2.045 vs z* = 1.960 — a 4% difference that affects borderline results. At n = 100, t* = 1.984 vs z* = 1.960 — less than 2% difference, rarely consequential.

Using t when you should use z (for proportions): this is technically incorrect (the t-test is designed for means, not proportions) but with large samples the results will be very similar. The more important error with proportions is using any parametric test when np < 5 or n(1−p) < 5, in which case use the exact binomial test.