Evidence Based Medicine and Prior Plausibility
Want to “Randomize, Double-Blind, Control” this?
In my last post I mentioned that prior plausibility is important. I’d like to touch here on what prior plausibility means.
If basic science (the principles of which are based on physics and chemistry phenomena and have been proven beyond reasonable doubt) states that the likelihood of something being true is close to zero, it does not matter what clinical studies can show . In fact, if further clinical studies show a non-null outcome of a treatment which contradicts basics laws of physics and chemistry, we must suspect something unwholesome happening. Furthermore, conducting clinical trials or meta-analyses may be a waste of (already) thin healthcare resources, or be downright dangerous .
This may be counter-intuitive for those who have been heavily indoctrinated by evidence based medicine (EBM). It’s counter-intuitive because EBM states that you may make conclusions based on the outcomes of one publication (be it a randomized controlled trial or a meta-analysis). While EBM principles allow you to judge about the internal validity of one study (how well was the study designed), they say precious little about how you can incorporate the results and conclusions of a study into practice (which is what we care about at the end of the day).
By relegating all evidence except clinical trials, RCTs and meta-analyses so low on the ladder that one may simply altogether ignore it evidence based medicine opens itself to exploitation by those looking for pseudo-scientific ways to promote their products/work.
Now, I am not arguing that EBM in itself is flawed. It is not. But it can only be applied in its pure form to those studies which have irrefutable underlying scientific evidence.
As biased as pharmaceutical companies are made out to be, their products are largely based on a solid base of scientific work. Usually a drug is tested in-vitro. The drug is then tested in animals. Only after these two steps which eliminate a good 95% or more of drug candidates does a drug molecule make it to human trials . These human trials (usually phase II or phase III) are the ones being assessed by evidence based medicine methods.
The reason why EBM has to be applied at the large human trial stage is because it serves as a form of check and balance. Sometimes despite all the rigorous science behind drugs, clinicians and researchers still don’t know what happens in the human body. So having invested upwards to $1 billion dollars into a drug, the bias formed is immense (not because the companies are evil, that’s just human nature) . This is certainly not an excuse but that’s why we have EBM methods to assess validity of trials based on good fundamental science.
Now, you might say, why conduct all the basic research if these drugs might still fail at treating diseases in humans?
That’s because the alternative is that we synthesize millions of chemicals/molecules and blindly give them to clinical trial participants. Without knowing prior pharmacology, how they may affect a living organism, what their degradation products are, what the predicted mechanism or action might be and how it may translate into side effects and much more.
So, if we didn’t do any studies before hand and just gave random molecules to people, would that be ethical? In fact, how could we recruit clinical trial participants and have them signed informed consent forms if we have nothing to inform them of? Would you sign up for a study which said “This consent form is to waive any legal responsibility that absolutely anything can happen to you, from mild diarrhea to death AND we don’t have a single intelligent way of even remotely informing you of what the likelihoods of these might be”. Of course you wouldn’t.
So, in a nutshell, this is prior probability (or plausability). Given all we know about a treatment AND judging from the results of THIS particular clinical trial, what can we conclude?
EBM which currently does not take prior plausibility into account say “Given THIS clinical trial, what can we conclude?” Well, I’m not sure what we can conclude. The only thing we can reliably assess is how well THIS particular study (internal validity) was conducted. But conclusions, these we cannot possibly begin to form without assessing prior knowledge about the subject matter.
It is conceivable to design a rigorous clinical study on the effects of unicorn dust on preventing heart disease and possibly even show some margin of clinical benefit. But would you in your right mind believe it?
So why then believe all the other nonsense, albeit much better disguised, as valid? What is the sense of assessing how technically sound a study was conducted if its underlying premise was flawed?
The same applies to EBM. Why spend a tremendous amount of time (and in a busy clinical practice, 1 hour of time IS a tremendous amount of time) analysing the internal validity of a study when regardless of what you conclude will be inapplicable?
1. Smith GC, Pell JP. Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials. BMJ. 2003 Dec 20;327(7429):1459-61. [LINK]
2. SISCRP. 2013. Clinical Trials Facts & Figures. http://www.ciscrp.org/professional/facts_pat.html