In the wake of Jade Goody’s young death from cervical cancer, there are already calls for all twenty-something women to go get themselves tested for the disease. After all, if more women are tested, more cancer will be caught, and that can only be a good thing, right?

Not necessarily, for reasons which are outlined in full in the book Reckoning with Risk by Gerd Gigerenzer, and which I’ll try to recap here.

Imagine what seems like a very good test for a disease: for every 100 people who suffer from it, this test will correctly identify 99 of them as doing so. This seems like a pretty good success rate, but there’s a catch.

No test is perfect, so also imagine that for every 100,000 people who do not have the disease, the test gets it wrong once, and incorrectly identifies one of these people as having the disease. It can be for any reason: perhaps one in 100,000 times a test gets mixed up with another woman’s, or perhaps a benign lump of cells is mistaken for a potentially dangerous one. However, one false positive in every 100,000 seems quite low, so there doesn’t seem to be too much of a problem.

But, now imagine that the disease is quite rare within a particular population, and so that only one person in every 100,000 people actually has it. Remember that 99 out of 100 such people will be identified as such by the test, and as such you would expect the test to identify this 1 person.

Remember though, that the test also incorrectly identifies one in every 100,000 healthy people as having the disease. As there are also 99,999 healthy people also receiving this test, you would expect 1 person who does not have the disease to be incorrectly identified as a sufferer of it.

So, if these 100,000 people are all tested, you would expect 2 people in total to be identified as having the disease: 1 of which will actually have it, and 1 of which who will not. In other words, a person who receives a positive test has only a 1 in 2 chance of having the disease! However, these two people (and perhaps their doctors) will not know that, and both may go through the stress and potentially invasive treatments that come from this diagnosis.

The problem comes from the 1 in 100,000 ‘base rate’ of the disease: it’s so rare that even a seemingly low false positive rate will become significant when testing many, mostly healthy people. This is why blanket screening for cervical cancer of all twenty-something women would not necessarily be a good thing: the base rate for this population is likely to be so low that this false positive effect will be significant, and the more young women not in the risk group who are encouraged to be tested, the more young women there will be who unnecessarily go through the mental anguish or brutal treatment associated with the disease.

This argument is not as important for people who are in the risk groups for cervical cancer (e.g. the elderly), for whom the base rate in their population will be higher, and so the significance of false positives lower. If the base rate in our hypothetical test was higher, say 100 in 100,000, this 1 in 100,000 false positive rate becomes less significant, and this problem goes away to some extent. It is with blanket screening of a non-risk group, such as women in their twenties for cervical cancer, where this problem becomes particularly acute.

If you think this sounds too abstract and mathematical, and my numbers too made-up, I point you to a recent Ben Goldacre article on this effect in the real-life example of screening for prostate cancer in men between the ages of 55 and 69:

1410 men would need to be screened to prevent one death. For each death prevented, 48 people would need to be treated: and prostate cancer treatment has a high risk of very serious side effects like impotence and incontinence.

I don’t have the actual numbers for cervical cancer and young women, but the point is the same: screening does not just have benefits. Instead, each life saved also incurs a cost in terms of the unnecessary treatment and anguish of the healthy.

Decisions on screening should be made by balancing these benefits and costs, not on the basis of the unfortunate and well-publicised death of a single young woman.