contact(at)onbelief.orgList of Articles
How to Interpret Tests of Beliefs
As indicated previously, Bayes' theorem is useful in the real world where we use less than perfect tests of belief i.e. where there is the possibility of error. The power of science and applied science is in part to do with its ability to define error. Any belief systems which claims to be 100% correct or error free should be avoided. Laboratory tests of scientifically based medical beliefs are a very clear example of where error can arise. If errors of belief arise in such defined circumstances consider the consequences of this observation for economics, politics and religion.
False Positive and False Negative
In complex situations such as biological systems every parameter is precisely controllable when they are examined by experimental procedures. There is therefore an inbuilt probability of error and so generate different rates of false positive and false negative results. Such considerations also apply in many other avenues of life where any degree of complexity is observed.
In medicine for example the more sensitive a test is for detecting a disease the more 'true positive' cases you will find. However this generally results in an increased number of 'false positive' results. In practice a small number of false positives is often accepted in diagnostic tests so that the number of false negatives is reduced. This strategy is adopted to reduce the number of serious cases of disease that are 'missed' by the doctor (i.e. false negatives).
Test Outcome
Category Total Type of Result
Positive
Negative
Positive
a = true positive
b = false positive
a + b
Negative
c = false negative
d = true negative
c + d
Total (n)
a + c
b + d
a + b + c + d
1) The positive predictive value is the percentage of positive tests that are correct a/ [ a + b ] x100
2) The negative predictive value is the percentage of negative tests that are correct d/ [ c + d ]x100
3) The percentage of false positives is [b/a] x100
4) The conditional probability of a true positive test is called the 'sensitivity' and is calculated as the very simple ratio : a/ [ a + c ]
5) The conditional probability of a true-negative test is the 'specificity' and is calculated as the very simple ratio: d/ [ b + d ]
[Adapted from The Polygraph and Lie Detection (2003) by the National Academies of the U.S.A.]
In more general terms there are enormous philosophical implications when we acknowledge the possibility of error in our thinking. It is my contention that we are then compelled to adopt pragmatic relativism rather than truth seeking as our way of trying to understand the world.
The Receiver Operator (ROC) Curve
For the empiricist there seems little value in holding a belief unless there is testable evidence to do so. In addition there is the more stringent requirement that conclusions drawn from the test are potentially refutable given another type of test. An idea that cannot possibly be refuted under any circumstances can only be of indeterminate value. The idea of the existence of gods falls into this category. Tests of belief are therefore of crucial importance. To imagine that no test of strong belief is required, as the devout communist or religionist might do, is folly.
Scientists refer to tests of belief as experiments. Doctors refer to such tests as clinical trials. Any one who applies a test of belief needs to have some understanding of how the test performs under a variety of circumstances. When new tests of belief are being developed in medicine a very precise understanding of how the test performs should be arrived at. A detailed analysis of a test's performance pushes the test to various levels of sensitivity and then observes the number of false positives. The data can then be shown as a graph of true positive vs. false positive results. This graph is known as receiver operator curve (ROC) and represents a description of the performance of a practical test (for an explanation see this source).
True certainty would not be a line on the graph but one point that was equal to 100% true positives and 0% false positives. This would be a point at the top left of the graph. One testing strategy is to use the point on the graph that comes closest to the top left corner (i.e. closest to true certainty). In the test shown on the right hand graph notice how after a test sensitivity of between 80% and 90% is reached on the vertical axis the graph almost flattens off so that for a very small improvement in detection of true positives there is a large increase in the number of false positives. As stated above an increase in sensitivity leads to a loss of specificity. To achieve certainty of a true positive result we end up with all of the negative results becoming false positives. Under these circumstances the performance of the test of belief becomes so bad that it becomes meaningless. In short under the circumstances shown: certainty of detection = nonsense.
supplier of Med Roc SoftwareUsed with permission of : Oliver Sander and Tobias Sing
http://addictedtor.free.fr/graphiques/RGraphGallery_fr.php?graph=100Fuzziness
The performance of tests for our beliefs with regard to living things cannot be represented as series of simple true certainty points on a large number of ROC graphs. The tests might be more like a large number of ' fussy' ROC graphs, like those of Sander and Sing above where each line represents the characteristics of a single test for each belief. Even these 'test characteristic' lines are not discrete but have the fussiness of variability associated with them. Not even the amount of variability or the amount of error can be certain.
On the Philosophy of Belief
www.onbelief.org
Scotland, 12th October 2007 and thereafter
Copyright 2007 onwardscontact (at) onbelief.org
