contact@onbelief.orgThe Severe Practical Effects of a Small Amount of Uncertainty
The moment we no longer pretend to be perfect in our analysis and instead admit to the possibility of error we need to have due regard to that error. This is what the 'laws of probability' require.
Take situations where the underlying probability of detecting a disease is very low because very few of those being screened for it actually suffer from the condition. In that type of situation the effect of test limitations in terms of spurious results is surprisingly high. If there is only a 1 in 500 chance of the condition being found but you can only detect a true positive in 99% of cases and you make a mistake of artificially producing a false positive result in 1 in 100 cases your application of the test will be completely useless in the real world because your chances of making a serious mistake are much higher than the underlying probability of finding the real event. In other words the quantitative effect of errors on belief varies with the underlying probability of the events.
In worked examples shown in Wikipedia the probability of a positive test identifying a true positive case is much lower than one would intuitively guess. Take for example use of a test for a disease with 99% sensitivity and specificity where only 1% of those tested actually have the disease. In that situation there is only a 50% probability that a positive test result will be true. If the frequency of the disease drops to 0.5% in the population there is only 33% probability of a single positive test result being correct. (For further explanation and calculated examples see the discussion of drug testing at Bayes' theorem and the medical screening example given in the article on Conditional probability at wikipedia). Another way to think of this scenario is that when the prevalence or scale of the entity under test is less than sor even approaching that of the experimental error (or variation) the test is very seriously compromised.
In these situations the concept of 'probably true' can have life changing consequences for the persons concerned and moral implications for those who practice diagnostic testing. Indeed the tendency to run tests for little reason, in highly litigious countries such as the U.S.A. where there is a tendency for doctors to test due to fear of medical litigation, is counter productive. Where doctors go on 'fishing expeditions' by running large amounts of unnecessary tests on large numbers of patients the same result applies. This is due to the fact that if the underlying frequency of 'true positive' in the test population declines substantially the probabilistic meaning of a positive test result also declines in the group of patients tested by that same doctor. It is better practice for the doctor to test when there is a good reason ( or belief) for doing so. It would not be good practice, for example, to routinely carry out tests for prostate cancer in young men, without a good reason for doing so on an individual basis, because young men tend not to suffer from that form of cancer. In that situation a larger proportion of false positive tests would arise
Consider the more extreme position of the investigator who is searching for misuse of drugs amongst athletes or employees. In a well run athletics federation with law abiding athletes very few of the tests should actually be for 'true positive' results. In this situation there is no particular reason for testing one individual, just the possibility that drugs could be being misused by a few individuals within the whole population of athletes. If the incidence of 'true positive' results was less than 1 in 100 for a test with 99% 'efficiency' there would be less than a 50% chance of a single positive test result being real. The investigator must then decide on an appropriate course of action. The worst thing he or she could do would be to reach a definite conclusion based on one positive test if justice were the deciding factor.
The situation has increased difficulty for the technologist who needs to take action in the world. The doctor who is not only faced with uncertainty of diagnosis (testing) also has the additional uncertainty of treatment efficacy (action). The interference of placebo and other confounding effects, and problems with reports of patient compliance with treatment (accuracy or truth) becomes very important. The compounding or cumulative effects of these probabilities must be very high.
On the Nature of Belief
www.onbelief.org
Scotland, 12th October 2007 and thereafter
Copyright 2007 onwards
