Conditional Probability and Independence

In this lesson you’ll learn what conditional probability means, how to compute it, how to use tree diagrams and two-way tables, how to test whether two events are independent, and why $P(A \mid B)$ and $P(B \mid A)$ are not the same thing.

Conditional probability is the probability of an event given that you already know something else happened.

$$P(A \mid B) = \frac{P(A \cap B)}{P(B)}$$

Read the bar as “given.” $P(A \mid B)$ is “the probability of $A$ given $B$.”

The intuition: knowing $B$ happened shrinks your world. Instead of the whole sample space, you’re now working inside $B$ only. So you ask what fraction of $B$ also contains $A$. That’s exactly what the formula computes: the overlap divided by the new, smaller universe.

A quick example. Roll a die. $P(\text{roll is } 2) = \frac{1}{6}$. But given that the roll is even, the world shrinks to $\{2,4,6\}$ and

$$P(2 \mid \text{even}) = \frac{1}{3}$$

The information doubled the probability.

Note that conditioning can move a probability in either direction. Given that the roll is odd, $P(2 \mid \text{odd}) = 0$.

Multiply both sides by $P(B)$:

$$P(A \cap B) = P(B) \times P(A \mid B)$$

This is the general multiplication rule from the last lesson. The two formulas are the same statement rearranged.

Events $A$ and $B$ are independent if knowing one changes nothing:

$$P(A \mid B) = P(A)$$

Equivalently, and often easier to check:

$$P(A \cap B) = P(A) \times P(B)$$

If that equation holds, they’re independent. If it fails, they’re dependent. This is a test you can actually run on data, not just a description.

This is the most important idea in the lesson, and the one that causes real-world harm.

$$P(A \mid B) \neq P(B \mid A)$$

These are different questions with different answers. Confusing them is called the prosecutor’s fallacy or, more generally, confusion of the inverse.

An example that makes the gap obvious:

• $P(\text{is a dog} \mid \text{has four legs})$ is fairly low. Plenty of four-legged things aren’t dogs.
• $P(\text{has four legs} \mid \text{is a dog})$ is nearly 1.

Same two properties, wildly different probabilities depending on which you condition on.

The consequential version:

• $P(\text{positive test} \mid \text{has disease})$ is usually high. That’s the test being good.
• $P(\text{has disease} \mid \text{positive test})$ can be surprisingly low for a rare disease.

Getting these mixed up is why a positive screening result is far less alarming than most people assume. The next lesson, on Bayes’ theorem, is entirely about converting one direction into the other.

For sequential events, a tree diagram organizes everything.

The rules for reading a tree:

• Probabilities on branches leaving any single node sum to 1.
• Multiply along a path to get the probability of that full sequence.
• Add across paths to get the probability of an event that several paths satisfy.
• Second-level branches show conditional probabilities, given the first-level outcome.

Trees are the most reliable way to avoid errors in multi-step problems, because they force you to write down every case.

When you have counts in two categories, a contingency table makes conditional probabilities easy to read.

	Passed	Failed	Total
Studied	42	8	50
Did not study	12	18	30
Total	54	26	80

Everything comes from reading the right row, column, or cell:

• $P(\text{passed}) = \frac{54}{80} = 0.675$ (use the grand total)
• $P(\text{passed} \mid \text{studied}) = \frac{42}{50} = 0.84$ (restrict to the “studied” row)
• $P(\text{studied} \mid \text{passed}) = \frac{42}{54} \approx 0.778$ (restrict to the “passed” column)

Notice those last two are both about studying and passing, and they differ (0.84 versus 0.778) because the conditioning direction differs. The denominator is what changes. That’s the whole trick: conditioning tells you which total to divide by.

Are studying and passing independent here? Check:

$$P(\text{passed}) = 0.675 \qquad P(\text{passed} \mid \text{studied}) = 0.84$$

Not equal, so they’re dependent. Studying is associated with passing, which is reassuring.

Example 1: Straight from the formula.

$P(A) = 0.6$, $P(B) = 0.5$, $P(A \cap B) = 0.3$. Find $P(A \mid B)$ and determine independence.

Solution.

$$P(A \mid B) = \frac{0.3}{0.5} = 0.6$$

Since $P(A \mid B) = 0.6 = P(A)$, the events are independent.

Confirm the other way: $P(A) \times P(B) = 0.6 \times 0.5 = 0.3 = P(A \cap B)$ ✓

Example 2: Cards without replacement.

Two cards are drawn without replacement. Given the first is a heart, find the probability the second is also a heart.

Solution. After removing one heart, 51 cards remain with 12 hearts.

$$P(\text{2nd heart} \mid \text{1st heart}) = \frac{12}{51} = \frac{4}{17} \approx 0.235$$

Lower than the unconditional $\frac{13}{52} = 0.25$, because a heart is now missing from the deck.

Example 3: Tree diagram.

Factory A makes 60% of a company’s widgets with a 2% defect rate. Factory B makes 40% with a 5% defect rate. A widget is chosen at random. Find the probability it’s defective.

Solution. Four paths through the tree:

$$P(A \cap \text{def}) = 0.60 \times 0.02 = 0.012$$ $$P(A \cap \text{good}) = 0.60 \times 0.98 = 0.588$$ $$P(B \cap \text{def}) = 0.40 \times 0.05 = 0.020$$ $$P(B \cap \text{good}) = 0.40 \times 0.95 = 0.380$$

Check: $0.012+0.588+0.020+0.380 = 1.000$ ✓

Defective can come from either factory, so add those paths:

$$P(\text{defective}) = 0.012 + 0.020 = 0.032 = 3.2\%$$

Example 4: Reversing the condition.

Using Example 3, a widget is found defective. What’s the probability it came from Factory B?

Solution. Now condition on defectiveness, so the denominator is $P(\text{defective})$.

$$P(B \mid \text{def}) = \frac{P(B \cap \text{def})}{P(\text{def})} = \frac{0.020}{0.032} = 0.625$$

So 62.5% of defective widgets come from Factory B, even though B makes only 40% of all widgets. Its higher defect rate makes it overrepresented among failures.

Compare the two directions: $P(\text{def} \mid B) = 0.05$ but $P(B \mid \text{def}) = 0.625$. Same two events, and the probabilities differ by more than tenfold. This calculation is Bayes’ theorem, which the next lesson formalizes.

Example 5: Two-way table.

Of 200 people, 90 own a car, 70 own a bike, and 40 own both. Find $P(\text{bike} \mid \text{car})$ and test independence.

Solution.

$$P(\text{bike} \mid \text{car}) = \frac{40/200}{90/200} = \frac{40}{90} \approx 0.444$$

For independence, compare to $P(\text{bike}) = \frac{70}{200} = 0.35$.

Since $0.444 \neq 0.35$, they’re dependent. Car owners are somewhat more likely to own bikes than the general group.

Equivalently: $P(\text{car}) \times P(\text{bike}) = 0.45 \times 0.35 = 0.1575$, but $P(\text{both}) = \frac{40}{200} = 0.20$. Not equal, so dependent.

Example 6: The prosecutor’s fallacy.

A forensic match occurs in 1 in 10,000 innocent people. A defendant matches. A prosecutor argues there’s a 1 in 10,000 chance the defendant is innocent. What’s wrong?

Solution. The prosecutor swapped the conditioning.

The evidence gives $P(\text{match} \mid \text{innocent}) = 0.0001$.

The claim made is about $P(\text{innocent} \mid \text{match})$, which is a different quantity entirely.

Suppose the match came from searching a database of 500,000 people. Among innocent people you’d expect roughly $500{,}000 \times 0.0001 = 50$ matches by chance. If one guilty person also matches, then given a match, the probability of innocence is about $\frac{50}{51} \approx 98\%$, not 0.01%.

The prior probability of guilt, and how the suspect was found, both matter enormously. This error has contributed to real wrongful convictions.

Medical diagnosis is the highest-stakes application. Clinicians must convert test sensitivity, $P(\text{positive} \mid \text{disease})$, into the thing the patient wants to know, $P(\text{disease} \mid \text{positive})$. Studies have repeatedly shown that many physicians get this conversion wrong, systematically overestimating disease probability after a positive screening test.

Spam filters compute $P(\text{spam} \mid \text{contains these words})$ from the more easily measured $P(\text{these words} \mid \text{spam})$. The classic Naive Bayes filter is exactly this lesson plus an independence assumption.

Legal reasoning suffers from the prosecutor’s fallacy, and courts in several countries have overturned convictions specifically over misused conditional probability. The Sally Clark case in the UK is the best-known example.

Machine learning classifiers output conditional probabilities, and calibration (making sure a stated 70% confidence really corresponds to being right 70% of the time) is an active engineering concern.

Insurance underwriting conditions on everything it can legally observe: age, location, history. Each variable that changes the conditional probability of a claim changes the premium.

In A/B testing, conditioning matters for interpretation. The conversion rate among users who saw a feature is a conditional probability, and it differs from the rate among all users assigned to that group.