ViewWiki

Lesson 3 of 4

Base rates.

About 25 minutes · 5 practice questions

Why a test that is $90\%$ accurate is wrong about most of its positives when the disease is rare: count the false positives, find when a positive is half and half, and see what a better test does and does not do.

Step 1 of 5 · A rare disease

Step 1 of 5

A rare disease

In the last Lesson 10%10\%⁠ of the hall were ill. Many diseases are much rarer. Here How common the disease is is 1%1\%⁠: a hall of 10001000⁠ holds only 1010⁠ ill people. This rate, how common the disease is before anyone is tested, is called the base rate.

The test is still 90%90\%⁠ accurate. Predict, then the board will sort the positives.

To the question

Step 2 of 5

Counting false alarms

The right-hand mat is far more crowded than the left. Everyone on it is a false alarm: a healthy person the test flagged.

To the question

Step 3 of 5

Half and half

At 1%1\%⁠, only about 8%8\%⁠ of the positives are ill. At 10%10\%⁠, in the last Lesson, half. The answer depends on the base rate as much as on the test.

To the question

Step 4 of 5

A better test

If the base rate matters so much, perhaps a better test solves the problem? The test is 90%90\%⁠ accurate now, and it is about to be swapped for a 99%99\%⁠ one.

To the question

Step 5 of 5

Rarer still

The same excellent test, 99%99\%⁠, for a disease that affects one person in a thousand. Doctors meet exactly this in screening. Work it out before you open the board.

The mistake of reading the accuracy as the chance of being ill is called base-rate neglect. It is very common, doctors included.

To the question

Still of the Bayesian reasoning and evidence 3D scene
Interactive scene

Loading the interactive version…

Each step sets up the Scene for you, and a control marked “fixed” stays put for that step.

To the experiment

Practice · Predict, then look

A disease affects 1%1\%⁠ of people, and the test is 90%90\%⁠ accurate. Someone has tested positive. What is the chance they are really ill?

To the experiment

Practice · Calculate

In the same hall, 1%1\%⁠ ill and a 90%90\%⁠ accurate test: how many of the positives are healthy?

To the experiment

Practice · Reach the target

The test stays 90%90\%⁠ accurate. Change How common the disease is until half the positives are really ill, then press Check.

To the experiment

Practice · Predict, then look

Back to 1%1\%⁠ ill. The test is replaced by one that is 99%99\%⁠ accurate. What is the chance now that a positive is ill?

To the experiment

Practice · Calculate

And a rarer disease still, 11⁠ in 10001000⁠ (0.1%0.1\%⁠), with the same 99%99\%⁠ test? What is the chance a positive is ill? Give it as a per cent, the number only, to one decimal place.