What do breast cancer tests and airport screens for terrorists have in common? Answer: Both involve huge uncertainties. Nothing in life is for sure. Nothing in life comes for free. There really are no free lunches in life. This is as true of testing as of any other area in life. Whenever, and wherever, we run tests there are always two errors that float around. The first, which is called a Type I error, occurs when the test fails to detect a problem. For example, one might have an inspector looking for a gas leak in a house. If the machine fails to detect a leak, when one really does exist, then one has a Type I error. For the homeowner in question, this can be very inconvenient if not plain downright distressing. On the other hand, the world is full of natural gas, so if the machine is set at a level that is too sensitive it will detect natural gases in the air of a house that does not have an actual gas leak. The result for that homeowner might be a wholly unnecessary and very expensive repair job. This kind of error is called a Type II error. It results in “false positives.” All scientific testing involves a balance between these two errors.
Take the recent brouhaha about testing of breast cancer. Of course, we all want to detect malignant cancers early. The problem is that if the test is set at too fine a level then any number of women who do not have malignant tumors will be identified as being cancer patients. I read recently that the numbers ran something like 1,000 to 1. That is, one thousand women were being incorrectly diagnosed as having cancer in the process of finding the one woman who did have cancer. Society, therefore, is presented with a tradeoff. How many women should be inconvenienced to save the life of one of them? I don’t have an answer to that tough question, but it is a question that needs to be addressed by a rational public — but never was.
Now we have a new bomb scare which will pose the same question. How many airline passengers must be inconvenienced and to what degree to ensure that one never has a single incident. Putting it another way, how many Type II errors must we have to ensure that we don’t have a single Type I error. To go to the extreme, are we proposing that everyone should be strip searched and have their body cavities probed as if they were criminals entering prison, so that we can detect the one person who has hidden an explosive in their mouth, for instance? If not, then we have to face the fact that there is always the possibility of a terrorist slipping through. This has nothing whatsoever to do with a catastrophic failure of the system as the press wants to make it out to be, but the simple result of all testing procedures. In fact, looked at it another way, that we have not had a terror attack on an airline for a decade seems to suggest that the current system has worked extremely well. So, should we have full body X-rays of little children and elderly women? I don’t have an answer to that tough question either, but it is a question, like that of breast cancer detection, that needs to be addressed by a rational public — but is not.