The positive test paradox

A 99%-accurate test says yes. Are you sick?

Of the 98 people who test positive, only 9 actually have it.

A positive test means a 9% chance you have it.A negative test means a 99.8% chance you don't.

Has it, tests positive9
Healthy, tests positive (false alarm)89
Has it, tests negative (missed)1
Healthy, tests negative901

How common it is prevalence1%

Share of the 1,000 who actually have the condition.

Catches the sick sensitivity90%

Of people who have it, how many test positive.

Clears the healthy specificity91%

Of healthy people, how many test negative.

Show only the positives

The screening example. Take 1,000 people. 1% of them (10 people) have the condition. The test catches 90% of people who have it, and correctly clears 91% of healthy people.

That gives 98 positive results: 9 people who really have it, and 89 healthy people who got a false alarm. So a positive test here means about a 9% chance you have it.

has it, positive healthy, positive has it, missed healthy, negative

Turn on JavaScript to move the sliders and try other numbers.

What's going on

The missing ingredient is the base rate: how common the condition is before anyone is tested. In the screening example, only 10 of the 1,000 people have it, and the test finds 9 of them. But there are 990 healthy people, and a specificity of 91% means 9% of them still test positive. That is about 89 false alarms, almost ten for every real case. Bayes' theorem says the same thing in symbols: the chance you have it given a positive result is sensitivity × prevalence, divided by all the positives, true and false.

This is why screening for rare conditions produces mostly false alarms. When almost everyone is healthy, even a small false-positive rate applied to that huge healthy group outnumbers the few true cases. Push prevalence up to 20% and the same test becomes far more convincing. Gerd Gigerenzer, in Calculated Risks, found that doctors given figures like these as percentages often badly overestimated the chance of disease. When the same facts were shown as natural frequencies ("9 out of 98"), far more of them got it right, which is why this page counts people instead of quoting probabilities.

It is also why clinicians treat a positive screen as a reason to look closer, not as a diagnosis. Among people who screened positive, the condition is now about 9 times more likely than it was, so a different, more specific confirmatory test starts from a much higher base rate, and its result means much more. Symptoms, risk factors and history change the starting probability in the same way. The numbers here are illustrative, not those of any real test, and this is not medical advice: if you have a result to understand, talk to the clinician who ordered it.

Source: Bayes' theorem; Gigerenzer, Calculated Risks (2002)