Mathematics & Logic · Probability & statistics

Base rates and what a test result means

Count true and false alarms in a made-up factory to see why the meaning of a test result depends on how common the problem was before testing.

4 short lessons · about 17 minutes · 3 sources · updated November 2026

What you will learn

  • Calculate true and false positives from stated group sizes and test rates
  • Explain why a detection rate isn't the probability of a defect after a positive result
  • Calculate how a higher base rate changes the share of positives that are true
  • Explain why a test's performance alone can't tell you what a positive result means

Lesson 1 of 4 · 4 min

Why most alarms from a good detector can be false

Before you readIn a 2007 study, 160 gynecologists were given a screening problem about a test that catches 90% of cancers. What share picked the correct chance that a positive result means cancer?

Correct.Not quite. The answer is Only 21%.

Only 21% chose the right answer, and most said 81% or 90%. This lesson shows why a test's catch rate is not the chance that a positive result is real.

Picture a factory that screens 10,000 parts. One in a hundred is defective, so 100 are defective and 9,900 are good. Its detector catches 90% of defects and wrongly flags 1% of good parts.

The alarm goes off for one part. How likely is that part to be defective? The tempting answer is 90%. The real one is under half, and you can see why by counting both ways a part can set off the alarm.

The detector catches 90% of the 100 defective parts, so expect 90 alarms. Those are true positives. It wrongly flags 1% of the 9,900 good parts, which makes 99 more alarms. Those are false positives.

That's 189 alarms, or positive results, and only 90 come from defective parts. So the chance that a flagged part is defective is 90 out of 189, about 47.6%. More of the alarms are false than true.

Check yourselfSay a batch has 1,000 parts, and 100 of them are defective. A detector catches 90% of defects and flags 1% of good parts. How many true and false positives should you expect?

Correct.Not quite. The answer is 90 true positives and 9 false positives.

There are 900 good parts, and the detector flags 1% of them, which gives 9 false positives. It catches 90% of the 100 defective parts, which gives 90 true positives. Getting 10 false positives means taking 1% of all 1,000 parts, defective ones included.

The 90% is the detector's sensitivity, the share of defective parts it catches. It starts from the defective parts and asks how many set off the alarm. Your question starts from the alarms and asks how many are defective. Swapping the direction changes the group you're counting in.

The swap fools experts. In 2007, 160 gynecologists were given a screening problem of the same shape. In it, 1% of women had breast cancer, and the test caught 90% of cancers and falsely flagged 9% of healthy women. About 1 in 10 positives is a real cancer. Only 21% chose that answer, and most said 81% or 90%.

How common a problem is before any testing is called the base rate. Here it's 1%. That makes the pile of good parts 99 times the size of the defective pile. So a 1% error rate on the big pile produces about as many alarms as a 90% catch rate on the small one.

The detector still earns its keep. An alarm lifts a part's chance of being defective from 1% to 47.6%. That's a big jump, just a smaller one than the 90% suggests.

Try to recallWhy is a detector's 90% sensitivity not the same as a 90% chance that a part that tests positive is defective?

Sensitivity counts positives among the defective parts. The chance of a defect after a positive counts defective parts among all the positives, false alarms included, and it depends on the base rate.

After a training session on turning the percentages into counts of women out of 1,000, 87% of those 160 gynecologists got the screening problem right.

To read a positive result, count every route into the positive group. The false alarms belong in the total too.

Lessons in this topic

  1. Why most alarms from a good detector can be false

    Count true and false positives in a made-up factory to see why a detector that catches 90% of defects can still be wrong about most of the parts it flags.

    Essentials · 4 min

  2. Why the same alarm means more in a worse batch

    Run the same detector on a batch with ten times as many defects, and watch the meaning of a positive result jump from under half to over 90%.

    Essentials · 4 min

  3. When two positives don't mean twice the evidence

    Compare a repeat test that makes its own errors with one that repeats the first test's mistake, and see why only the first kind counts as new evidence.

    Deeper · 4 min

  4. How much a negative result should reassure you

    Count the defects a detector misses to see when a negative result is reassuring and when it isn't.

    Deeper · 5 min

Three quick stories from this topic

When a negative result reassures

A negative test result is most trustworthy when the problem it looks for is rare

Take a made-up detector that misses 10% of defects. If 1% of parts are defective, a pass is wrong about once in a thousand. If half are defective, the same pass is wrong about once in eleven, because there are far more defects to miss.

Count where negatives come from

Negatives come from good parts the detector clears and defective parts it misses. At 1% defective, the detector misses 10 parts and clears 9,801, so 10 of 9,811 negatives are defective. At 50% it misses 500 and clears 4,950, so 500 of 5,450 negatives are defective. The detector never changed.

It's the mirror image of the positive case

A rare problem makes positives less reliable and negatives more reliable, and a common problem does the reverse. The rates here are invented and assume the detector performs the same on every batch, which a real test may not.

Doctors and positive mammograms

Given the numbers, only 21% of 160 gynecologists knew what a positive mammogram means

The 2007 problem said 1% of women had breast cancer, and the test caught 90% of cancers and falsely flagged 9% of healthy women. About 1 in 10 positives is then a real cancer. Most of the doctors answered 81% or 90%.

Count it out

Of 1,000 women, 10 have cancer and 9 of them test positive. Of the 990 without cancer, about 89 also test positive. So 9 of 98 positives are real.

Counts fixed it

After a training session on turning the percentages into counts like these, 87% of the same doctors chose the right answer.

A repeat test can add nothing

A second positive test adds nothing if it repeats the first test's mistake

With a made-up detector, one alarm means a 48% chance of a real defect. A second alarm from an independent test lifts that to about 99%. A second alarm from a scanner fooled by the same harmless mark leaves it at 48%.

A perfect repeat adds nothing new

If the second result always matches the first, it tells you nothing more once you know the first.

Check the assumption before multiplying

You can treat two false alarms as independent only if there's evidence their errors are independent among the good parts.

Sources

This topic is part of From everyday numbers to proof, a guided path through related topics.

  • What an expected value promises

    An expected value is the average you'd get over many repetitions. See how to calculate one, why it can be a result that never happens, and what it can't tell you about a single try.

    4 lessons · 20 min

  • The rules of probability

    Probability puts a number between 0 and 1 on how likely something is. Learn to count outcomes, when to add chances and when to multiply them, and why 'at least one' is easiest to find by working out 'none'.

    4 lessons · 16 min

  • The normal distribution and the bell curve

    Measurement errors, dice totals and test scores often pile up in the same bell shape. See why many small effects added together produce it, how to read it with the 68-95-99.7 rule, and when assuming it goes wrong.

    4 lessons · 16 min

  • How calculus finds speed at a single instant

    Build a speedometer reading from average speeds over shorter and shorter intervals. See why the method never divides by zero, why a sharp corner breaks it, and how one rate predicts the next moment.

    4 lessons · 17 min

  • Why examples are not a proof

    Watch a formula produce 40 primes in a row and then fail, then learn to prove a claim for every case by direct argument, by induction and by the contrapositive.

    4 lessons · 17 min

  • The Pythagorean theorem

    In a right triangle, the squares on the two short sides add up to the square on the long side. See why that's true, how builders use it, and how Babylon, India and China knew it before Pythagoras.

    4 lessons · 17 min

All mathematics & logic topics · Take the quiz