July 11, 2010 at 22:13Errors in the interpretation of experiments
A piece from Peter Norwig's excellent article on experimental errors .
Cerebral vasculitis affects one person in a million, and seasonal colds affect one in ten. Suppose we have developed tests for these diseases that give the correct answer in 99% of cases (both for patients and for healthy ones).
What conclusion should the doctor make, having the following data:
Nikolai's test for cerebral vasculitis gave a positive result (the result is correct in 99% of cases).
In Sergei, a cold test gave a positive result (the result is correct in 99% of cases).
The doctor will probably decide that both are likely to be ill. If you also think so, then your intuition has failed. The chance that Nikolai is sick with vasculitis is 0.01%. It may seem unbelievable, but it is. Perhaps the following example will help to understand why.
Suppose we have a device for determining Martians, and it gives the correct answer in 99% of cases. We scan, for example, deputies one by one, the result is invariably negative. But when it comes to Vladimir Volfovich, the inscription lights up that he is a Martian. Will you believe the device? Probably not, this is just that 1% of cases when the device gives an erroneous answer.
In the case of vasculitis, our test due to a 1% error will give a positive result for 10,000 out of a million people, but only one of these 10,000 people will actually be sick.
Cerebral vasculitis affects one person in a million, and seasonal colds affect one in ten. Suppose we have developed tests for these diseases that give the correct answer in 99% of cases (both for patients and for healthy ones).
What conclusion should the doctor make, having the following data:
Nikolai's test for cerebral vasculitis gave a positive result (the result is correct in 99% of cases).
In Sergei, a cold test gave a positive result (the result is correct in 99% of cases).
The doctor will probably decide that both are likely to be ill. If you also think so, then your intuition has failed. The chance that Nikolai is sick with vasculitis is 0.01%. It may seem unbelievable, but it is. Perhaps the following example will help to understand why.
Suppose we have a device for determining Martians, and it gives the correct answer in 99% of cases. We scan, for example, deputies one by one, the result is invariably negative. But when it comes to Vladimir Volfovich, the inscription lights up that he is a Martian. Will you believe the device? Probably not, this is just that 1% of cases when the device gives an erroneous answer.
In the case of vasculitis, our test due to a 1% error will give a positive result for 10,000 out of a million people, but only one of these 10,000 people will actually be sick.