Question 1

What is a t-test and when should I use it?

Accepted Answer

A t-test tests whether a mean is significantly different from a reference value or from another mean. Use a one-sample t-test when comparing a sample mean to a known constant (e.g., "Is our product weight significantly different from the stated 500g?"). Use a two-sample t-test when comparing two independent groups (e.g., "Do drug and placebo groups have different mean outcomes?"). Use a paired t-test for before/after measurements on the same subjects.

Question 2

What is the p-value in a t-test?

Accepted Answer

The p-value is the probability of observing a t-statistic as extreme as yours (or more extreme) if the null hypothesis were true. A small p-value (typically p < 0.05) means the result is unlikely under the null hypothesis, providing evidence against it. p = 0.03 means: if H₀ were true, there is only a 3% chance of seeing this result by chance. Important: p-value is not the probability that H₀ is true.

Question 3

What is the difference between one-tailed and two-tailed t-tests?

Accepted Answer

Two-tailed test asks: "Is the mean different from the reference?" (either direction). One-tailed test asks: "Is the mean greater than (or less than) the reference?" One-tailed tests have more power but require you to specify the direction before collecting data. For most research, two-tailed tests are standard and more conservative. Use one-tailed only when you have a strong a priori directional hypothesis.

Question 4

What does "degrees of freedom" mean?

Accepted Answer

Degrees of freedom (df) represents the number of independent pieces of information used to estimate a parameter. For a one-sample t-test, df = n−1. For a two-sample test (Welch), df is approximated using both sample sizes and variances. Higher df gives a t-distribution closer to normal. The t-distribution has heavier tails than normal for small df, which accounts for the additional uncertainty of estimating σ from a small sample.

Question 5

When should I use t-test vs z-test?

Accepted Answer

Use a z-test when: (1) population standard deviation is known, or (2) sample size is large (n ≥ 30) and CLT applies. Use a t-test when: population std dev is unknown (the usual case in practice) and/or sample is small. In practice, t-tests are almost always preferred because population σ is rarely known. For large samples, t and z give virtually identical results.

Question 6

What are the assumptions of a t-test?

Accepted Answer

1) Independence: observations are independent of each other. 2) Normality: data follows a normal distribution (can be relaxed with large samples via CLT). 3) Equal variance (for equal-variance two-sample test): both groups have similar variance. Use the Welch t-test (unequal variance) when group variances differ substantially — it is safer and is the default in most software. The t-test is fairly robust to modest violations of normality for n ≥ 15.

Test	Use when	Example
One-sample	Comparing a sample mean to a known constant	Is our product weight significantly different from the stated 500g?
Two-sample (independent)	Comparing means of two unrelated groups	Do male and female students score differently on the same test?
Paired (not here)	Same subjects measured twice (before/after)	Did blood pressure change after treatment in the same patients?

Condition	t-Test	z-Test	Mann-Whitney U
Population σ known	—	✓ Preferred	—
Population σ unknown (typical)	✓ Preferred	Acceptable (n≥30)	—
Small sample (n < 30)	✓ Yes	Not ideal	✓ If non-normal
Data is normally distributed	✓ Yes	✓ Yes	Not required
Data is heavily skewed / non-normal	Robust (n≥15)	Not ideal	✓ Preferred
Ordinal data (rankings, Likert)	No	No	✓ Yes

T-Test Calculator

Formula

When to Use Each t-Test Type

Common Mistakes in t-Tests

t-Test vs z-Test vs Mann-Whitney — Which to Use?

Case Study: Comparing Pain Reduction Scores in a Clinical Trial

Related Statistics Tools

Related Calculators

Disclaimer

Frequently Asked Questions