False positives

What do breast cancer tests and airport screens for terrorists have in common? Answer: Both involve huge uncertainties. Nothing in life is for sure. Nothing in life comes for free. There really are no free lunches in life. This is as true of testing as of any other area in life. Whenever, and wherever, we run tests there are always two errors that float around. The first, which is called a Type I error, occurs when the test fails to detect a problem. For example, one might have an inspector looking for a gas leak in a house. If the machine fails to detect a leak, when one really does exist, then one has a Type I error. For the homeowner in question, this can be very inconvenient if not plain downright distressing. On the other hand, the world is full of natural gas, so if the machine is set at a level that is too sensitive it will detect natural gases in the air of a house that does not have an actual gas leak. The result for that homeowner might be a wholly unnecessary and very expensive repair job. This kind of error is called a Type II error. It results in “false positives.” All scientific testing involves a balance between these two errors. Continue reading