contact(at)onbelief.orgList of Articles
Testing Beliefs
Faced with the amoral nature of science, the corruption of politics, misuse of law and the incoherence and errors of religion should we abandon all beliefs? Certainly not.
Belief's, in the sense being used on the internet site, are necessary for change and our sense of well-being. We should however treat our beliefs as provisional rather than truthful and act accordingly. In matters of scientific, political or religious, belief we should first of all ask what is the observational basis of those beliefs. Then we should formulate questions to test our beliefs and continue by making predictions. Having made those predictions we should devise evidential tests. If we can neither ask questions nor devise predictions or tests we should reformulate our beliefs to make those steps possible. Any belief systems that prevents us from undertaking these actions should be abandoned or at the very least be doubted. If having tested our beliefs for evidence and no supporting evidence emerges they should either be contemplated for their beauty or be abandoned in favour of something else. That is why it is appropriate for us to change the nature of our beliefs during the course of our lifetimes and why science evolves as a belief system.
The major difficulty that we encounter is to devise useful tests of belief or to put the ideas differently to devise test that demonstrate that our beliefs have some utility. Rigourous belief testing involves either life experience or specific actions ( experiments, clinical trials, detailed historical analysis, enactment of economic policies and laws, etc.). The world is too complex and too unpredictable to yield anything but trivial results from the philosopher's arm chair in an a priori fashion. Many useful tests could of course of course be devised in the philosopher's arm chair.
Natural Law vs real-world situations
The probabilistic approach to certainty might seem unsatisfactory to those who wish to promulgate 'natural laws'. It also provides practical difficulties for those who develop new technologies in computer software, engineering or medicine. This is partly due to the fact that a Bayesian approach is more suitable for situations where the actual, real or underlying probability is neither 0 nor 1 or in other words there is no logical omniscience. Nevertheless there still needs to be some definitive or factual observation, at least on a provisional basis.
Where a theory, or the test of a theory, is imprecise we can consider for practical purposes a blurring of the distinction between strength of belief (subjective probability) based on a belief of prevalence of the condition under investigation and improbability due to inherent randomness. If the doctor is to take action on the basis of a diagnostic test she must have strengths of belief associated with the previously reported anterior probabilities of the test's performance and the underlying frequency of disease in her local community. For example a perfectly rational Scottish doctor would be expected to have a lower strength of belief in the positive result, of a test for ebola virus on someone who had no African contacts and lived on the remote Scottish island of Unst, than a similar doctor practicing in Zaire. If the Scottish result turned out to be a true positive after the death of the patient, the Scottish doctor would be advised to revise her strength of belief in the probability associated with the prevalence of that disease in her island community. The same could not be said for her Zairian equivalent. In other words where there can only be a lack of certainty due to incompleteness of observational data it is natural for us to vacillate in our opinions or beliefs or give them a changing or unequal weighting rather than act as a logical machine. It is in such situations that we clearly see the subjective nature of probability.
Tests of Belief in Medical Research
Given the presence of such effects we can see that it is extremely difficult to make definitive progress of belief in medicine and very easy for the charlatan to make false claims of more than random effects. For that reason very rigorous carefully designed predictive tests of belief, known in medicine as clinical trials, need to be carried out. For such trials it is now common place and even an ethical prerequisite to carry out a priori 'power calculations' using assumed probabilities. The researcher uses assumptions or highly speculative beliefs about how effective the drug might be when tested. Using those assumptions a study design is devised which would involve enough people. If the expected difference between 2 drug treatments is small then a larger number of people are required to demonstrate the effect by research. If the researcher designs and executes a research study that is shown retrospectively to have been 'underpowered' due to a lack of sufficient patients, for example, he or she will have wasted time and resources.
Medicine however has now gone beyond analysis of the single predictive test of belief. Meta-analysis is used instead to combine a variety of trials undertaken in different circumstances and so produce a more predictively accurate analysis of the likely out come on specific populations. In that sense the researchers are deriving a meta-probability. Nowhere in all of this action does the 'experimentalist believer' expect to find certainty or universality. No falsification of a universally predictive proposition is attempted. Indeed, it would not be rational do so.
Just recently the Bayesian approach has gained more prominence in molecular genetics and medicine and a new world of analytical opportunities seems to be opening up based on this understanding. Although the meta-analysis is important in clinical trials it is not possible to ignore the subjective basis on which the data is arrived at for the probability calculation in the first instance. Indeed we now have the situation where at least 2 authors are wisely suggesting that by looking at prior evidence in medical research a range of different outcomes can be arrived at depending on how accepting or skeptical a practitioner is of previous evidence using a Bayesian approach (see source >). Some might be resistant to this way of thinking because they see it as subjective. However in the classical or frequentist view the underlying assumptions are usually ignored and so these analyses operate with hidden subjective components. For example criteria might be adopted for exclusion of specific studies from a meta-analysis of multiple trials. In such an eventuality there may be very good reasons for adopting these criteria however a subjective judgment or choice is being introduced without any probabilistic weighting.
Although medicine as a whole seeks to achieve the most benefit to the largest number of people, 'evidence-based' decision making in a specific case is different from the pursuit of research evidence of qualified general applicability. The application of research to clinical practice needs to be tempered with of an understanding of how a treatment might be relevant in an individual case. An individual doctor might have very good reasons for questioning whether or not the data obtained in a carefully selected group of patients in a certain double-blind placebo-controlled randomised clinical trial applies to her particular patient. The individual patient might not have met the inclusion criteria or could have been rejected by the exclusion criteria that had been applied during the research study. Alternatively the patient might be at one end of the distribution for more or one of the characteristics of the study group such as age and so be much younger or older than the average of those in the study. Many drug trials exclude women of reproductive age because of the possibility of causing harm to an unborn child. If the doctor then chooses to use that drug on a young non-pregnant female patient they are making a subjective judgment. In that case they might take into consideration the results of studies based on non-human animals. In other words a weighting of the evidence might be appropriate based on independent information. If the doctor ignores the applicability of research trial to a specific case then they are inevitably making a subjective judgment in doing so. Simply saying 'this is the published evidence' is over simplistic because medical research does not result in universally valid predictions.
In Cognitive Behaviour Therapy (CBT)
Strength of belief analysis also has practical value in clinical psychology and so is used by cognitive behaviour therapists to help patients in depressive or anxiety states. If the anxious patients can be taught a strategy to reduce the strength or frequency of dysfunctional beliefs about themselves from a pseudo-stochastic (or a pseudo-quantitative) strength of 97% for example to 3% then their overall or average well-being will improve. The mind might be considered to be flipping between different belief states or between functional and dysfunctional beliefs. The very depressed patient might have dysfunctional self-beliefs 97% of the time and 3% of the time in functional states. If those figures could be swapped around through therapy the patient would feel better. We would then be rationalising that form of therapy and qualitative analysis by construing the strength of belief as a temporal change of state.
Optometry, Objectivity and Judgment
When people think they or their children need spectacles they visit an optometrist to have their eyes tested. The optometrist examines the health of the eyes and produces an optical prescription designed to allow the optician to manufacture the correct lenses to correct for vision defects. During an eye examination the optometrist might begin the examination with an automated refractor or retinoscope to give an objective estimate of the prescription required. She will then proceed to the use of a phoropter where she can change lenses and other settings and ask the patient to perform objective reading tasks using letters of different sizes. She will also ask the patient to provide subjective feedback about whether or not they can see better in particular tests, such as in the examination of astigmatism with a 'Dot Pattern' (Try an online demo of a manual phoropter > )
Consider the position of the optometrist when patients give inconsistent subjective responses. Clearly the optometrist will repeat the test. However if no definitive result is obtained after repetition they have a problem of uncertainty. They have some degree of objective knowledge at the completion of the examination and some degree of uncertainty. In this case the skilled and experienced optometrist will use professional judgment to reach a decision about what they consider to be the best prescription for the patient. In this profession and in many others there can be cases where there is no escape from the need for judgment in the face of uncertainty arising from subjectivity or assumption.
The Dynamics of Belief in Medicine
A terminally ill columnist once impressively wrote for a British broadsheet newspaper "there are only two kinds of medicine ..... ones that work and ones that don't" (As I am perhaps paraphrasing his words for lack of recall see also a quotation from Dawkins.) . The implication was that it does not matter whether or not you label your medicine, mainstream, conventional, alternative, complementary or holistic. You need to address the simple question does it work? The relevant questions to the seriously ill person are; is the medication effective; is there good observational evidence about its efficacy and side-effect profile? The ill person should also ask what is the probability that it will work on me and how good will the effect be? It is also highly likely that where no side effects have ever been recorded there will be no beneficial effects either. I would therefore add to that journalist's words by saying 'work to a greater or lesser extent on an individual basis'.
A limitation of today's medical practice is that it is tested on groups of people. One group is treated with the approach under test and one receives a placebo or a treatment that is currently thought to be effective. The average response between groups is usually then compared despite the fact that we can predict individual variability of effect. Clearly it is a limitation of 'conventional' medicine that the 'gold standard' in research is presently the statistically-verifiable double-blind placebo-controlled randomised trial, which operates on defined population groups rather than the individuals. However the discipline of modern 'evidence based medicine' is such that during the coming century we expect practice of this technology to change toward a more individualised approach. Rather than breaking the test group into 2 (one with the treatment and one without) it will be possible to define sub-groups of people based on genetic differences. Making such progress will of course require more discoveries in genetics and biochemistry and a better understanding of how our individual genotype affects our phenotype. Furthermore we will need to see a very radical shift in the design of medical trials and the basis on which they are analysed. Even then one must ask will those that refuse to see the value of reductionism and the power of identifying distinct disease entities be satisfied by anything other than the mysticism of holistic metaphors.
By evolving a critique of current beliefs and practices we can have grounds for optimism both in medicine and elsewhere. If, like metaphysics, the conventional becomes educationally supplanted by another branch of natural philosophy with greater explanatory power we should welcome this change. We should not think like the religious fundamentalist and be threatened by adaptation. We can have much to gain by accepting change. If by some unlikely course of events a 'holistic therapist' should make some genuinely important medical observation then that too should be welcomed. The question still remains: what is the evidence?
(For further reading see Randomised, placebo-controlled, double-blind trials, The gold standard in medical testing, The Placebo Effect and related articles at www.skeptics.org.uk/health.php)
On the Philosophy of Belief
www.onbelief.org
Scotland, 12th October 2007 and thereafter
Copyright 2007 onwardscontact (at) onbelief.org
