Probability Rules Explained

The Three Axioms of Probability

All of probability theory rests on three axioms (Kolmogorov, 1933): (1) Non-negativity: P(A) ≥ 0 for any event A. (2) Normalisation: P(sample space) = 1 — the probability of all possible outcomes is 1. (3) Additivity: For mutually exclusive events A and B, P(A ∪ B) = P(A) + P(B).

From these three axioms, every probability rule can be derived. They are the bedrock of the entire field. A consequence: if you know P(A), you automatically know P(not A) = 1 − P(A). And all probabilities must be between 0 and 1 inclusive.

Rule 1: The Complement Rule

Formula: P(Aᶜ) = 1 − P(A), where Aᶜ (read "A complement") is the event that A does NOT occur.

The complement rule is your most-used tool for "at least one" and "not A" calculations. Instead of computing the complex event directly, compute the probability of its complement and subtract from 1.

Worked example: A machine has a 2% failure rate per component. A system uses 5 independent components. What is the probability of at least one failure? P(at least one fails) = 1 − P(none fail) = 1 − (0.98)⁵ = 1 − 0.904 = 0.096 (9.6%). Computing the complement (zero failures) is far easier than summing the probability of exactly 1, exactly 2, ..., exactly 5 failures.

When to use: Whenever the phrase "at least one," "at least once," "not all," or "not zero" appears. The complement approach almost always simplifies the calculation.

Rule 2: The Addition Rule

The addition rule calculates the probability that at least one of two events occurs: P(A or B) = P(A ∪ B).

For mutually exclusive events (events that cannot both occur simultaneously): P(A ∪ B) = P(A) + P(B). Example: drawing a King or a Queen from a deck: P(King) = 4/52, P(Queen) = 4/52. Since you cannot draw both at once: P(King or Queen) = 4/52 + 4/52 = 8/52 ≈ 15.4%.

For non-mutually exclusive events (events that can overlap): P(A ∪ B) = P(A) + P(B) − P(A ∩ B). The subtraction removes the double-counting of outcomes where both events occur. Example: drawing a red card or a face card. P(red) = 26/52, P(face card) = 12/52, P(red face card) = 6/52. P(red or face card) = 26/52 + 12/52 − 6/52 = 32/52 ≈ 61.5%.

Key question: Can both events happen at the same time? If yes, use the general formula with the intersection subtracted. If no, use the simple addition.

Rule 3: The Multiplication Rule

The multiplication rule calculates the probability that two events both occur: P(A and B) = P(A ∩ B).

For independent events (one event does not affect the probability of the other): P(A ∩ B) = P(A) × P(B). Example: flipping two coins. P(head on coin 1) = 0.5. P(head on coin 2) = 0.5. Since the coins are independent: P(two heads) = 0.5 × 0.5 = 0.25.

For dependent events (one event changes the probability of the other): P(A ∩ B) = P(A) × P(B|A), where P(B|A) is the conditional probability of B given A has occurred. Example: drawing two Aces from a deck without replacement. P(first Ace) = 4/52. Given the first was an Ace, P(second Ace) = 3/51 (only 3 Aces remain). P(both Aces) = (4/52) × (3/51) ≈ 0.0045.

Independence test: Events A and B are independent if and only if P(A ∩ B) = P(A) × P(B). If the product does not equal the joint probability, the events are dependent.

Rule 4: Conditional Probability

Formula: P(B|A) = P(A ∩ B) / P(A), read as "the probability of B given that A has already occurred."

Conditional probability updates your probability estimate when you receive new information. The denominator P(A) restricts the sample space to only those outcomes where A occurred; the numerator P(A ∩ B) counts outcomes where both occurred.

Worked example: 40% of students studied hard (S). Of those who studied hard, 90% passed (P|S). Of those who did not study, 30% passed (P|Sᶜ). P(S and P) = P(S) × P(P|S) = 0.40 × 0.90 = 0.36. P(Sᶜ and P) = 0.60 × 0.30 = 0.18. P(pass overall) = 0.36 + 0.18 = 0.54.

Common error — confusing P(A|B) with P(B|A): P(positive test | has disease) is NOT the same as P(has disease | positive test). This confusion, called the base rate neglect or the prosecutors' fallacy, leads to systematically wrong conclusions in medical testing, legal reasoning, and everyday probability estimation.

Rule 5: Bayes' Theorem

Formula: P(A|B) = P(B|A) × P(A) / P(B), where P(B) = P(B|A) × P(A) + P(B|Aᶜ) × P(Aᶜ).

Bayes' theorem reverses the direction of conditional probability. It answers: given that I observed B, what is the probability that A caused it? It formally combines prior knowledge P(A) — your belief before seeing the evidence — with the likelihood P(B|A) — how probable the evidence is if A is true — to produce the posterior P(A|B) — your updated belief after seeing the evidence.

Medical testing example: A disease affects 1% of the population. A test is 99% sensitive (P(positive | disease) = 0.99) and 95% specific (P(negative | no disease) = 0.95, so P(false positive) = 0.05). You test positive. What is the probability you actually have the disease? P(disease | positive) = P(positive | disease) × P(disease) / P(positive) = (0.99 × 0.01) / (0.99×0.01 + 0.05×0.99) = 0.0099 / (0.0099 + 0.0495) = 0.0099 / 0.0594 ≈ 16.7%. Despite the test being 99% accurate, you only have a 16.7% chance of actually being ill — because the disease is rare (base rate = 1%).

This counterintuitive result is why medical tests are repeated and why understanding the base rate is critical. Bayes' theorem is the foundation of Bayesian statistics, spam filters, machine learning classifiers, and diagnostic AI systems.