Science Based-Evidence Ship (Replacing the EBM Pyramid)

It is enough to do a quick Google Images search for the words “evidence based medicine” and you get a whole bunch of pyramids (aside from other irrelevant stuff).

Of course, when looking at a pyramid you can get a sense that you can look at only the top 2 layers and disregard the rest. In fact, sometimes, you can even get a sense that you could simply discard the rest. Conceptually this makes sense when you look at something like a pyramid. Although the top layers build on the bottom layers, the layers appear largely independent.

In fact, the relationship is very bizarre. For example, if you place a meta-analysis at the top of a pyramid you can understand that you couldn’t have a meta-analysis without some clinical trials, such as randomized controlled trials (RCT). But a good meta analysis will usually critically appraise the studies included and strive to include only high quality RCTs. You can certainly include case-control and other cohort studies in a meta-analysis, but this mixing of studies would usually weaken it. It is also obvious that we cannot skip from basic science to a meta-analysis (because there would be no patient data to combine) [1].

Aside from this connection of clinical trials to a meta-analysis, it is quite unclear how other types of evidence in the EBM hierarchy of evidence are linked to each other. Is a case-control conducted to look for side effects of a drug linked to an RCT of the drug’s efficacy in any way? How does animal research fit in? How does basic, fundamental science, link to the top of the pyramid? Can we simply skip the “rungs” in the pyramid and hop directly from basic science to an RCT? And while the definition of EBM calls for the use of clinical experience and patient values, it is quite unclear how these interplay [2].

When you look at things like homeopathy it becomes clear, that they simply skip the basic science and jump straight into clinical trials. Sometimes they make a crude attempt at basic science but that is flawed and rarely makes the leap to construct higher levels of evidence [3,4].

There have also been proposals to utilized lower levels of evidence on the pyramid when higher levels are absent. This makes sense, but only for those treatments that have a solid scientific base. But what if there is no scientific base, what can we move down to? Once one can assure a solid scientific base, one can move on and think on a higher level of clinical trials and meta-analyses. This does not work for treatments that lack any background evidence and do not stand on a good base.

There are also proponents of the approach “if it’s stupid and it works, it’s not stupid”. In other words, if we took a homeopathic treatment and ran it through a whole bunch of clinical trials and got good results, we could then accept it as valid. Never mind the fact that we haven’t the slightest clue about how it works. The end justifies the means? I think not. The scary thing is that it’s already happening – we are chasing our own shadow. Every time the results are no better than placebo and yet every time we see a new shadow we chase it again, expecting different results.

“Insanity: doing the same thing over and over and expecting different results” [5]

Looking back at things like mathematics and physics where you are always urged to “show your solution” it becomes clear that simply guessing the answer is not a good approach. It’s not a good approach because you don’t really understand what has been done and quite possibly could not repeat the process. Since EBM stands firmly on frequentist statistics (hence the meta-analysis being the king of the pyramid) it implies that we need to be able to replicate every step in the process of evidence building. Not just repeat the experiment, but repeat every step. A treatment that was pulled out of the ether however can only be expected to work on one premise – blind luck. We could not hope to improve it or refine it. Imagine if the Wright brothers, instead of building an airplane, found an already built model. They could have then flown it and shown that it flies, but since they didn’t build it, they would have had no idea how it works. Without this knowledge of inner workings we’d still be toying around with flimsy cardboard planes.

This is where the pure EBM-ists rush in and criticize that seemingly “scientific” theories have failed in practice. They are of course right in pointing out that certain theories haven’t panned out in practice. Unfortunately they are victims to the “science god complex”. They assume that if things are based on basic science, then they must be 100% true. Perhaps there’s some misconception and disillusionment in science. Perhaps an expectation that science must always produce amazing positive results.

Unfortunately, by driving this divide between science and clinical testing, EBM opens up to pseudoscience. Both EBM and pseudoscience can say “since basic science has been shown to be false before, we should not rely on it”. But without basic science, clinical research would have nothing to direct, it would be based on blind prodding and poking with occasional success in RCTs (if you’re blindly stumbling around, blind luck can occasionally still give good results).

I would like to propose a different conceptual structure for viewing evidence in health care. This structure would have to include a base without which the entire structure collapses. The base would include all the basic science, animal research and essentially any research required to validate the reality of a drug. On top of this base we can build the levels of evidence proposed by EBM in essentially the same way as they are in the pyramid. The point is that without the base, the entire structure collapses.

Here is what I would like to propose:

SBM Structure

(Click to enlarge)

Unlike the EBM pyramid this structure makes basic and clinical sciences connected but interdependent.

1) The ship: it is the foundation consisting of physics, chemistry, biology, physiology, biochemistry, animal research, etc. Without basic science everything else drowns.

2) The sails: these are clinical sciences (roughly in the same order as the EBM pyramid puts them). Without clinical studies and clinical practice basic science on its own is immobile. It stagnates, generating knowledge without ever applying it. Without clinical science, basic science cannot be moved and directed.

Expert opinion and clinician experience can also direct science, but since they only make up part of the directing force, they would be insufficient on their own. I also placed retrospective/case-control studies on the same level as RCTs because they often study fundamentally different things. For example, RCTs are suitable for efficacy studies and can never be employed for safety studies.

This then is the interdependency of basic science and clinical science. It is not that either one is subservient to the other. Notice also the “good quality” prefix assigned to all the clinical studies. That is because “low quality” clinical studies can be rejected based on the internal validity flaws. Low quality studies do not contribute anything. They are like ripped sails, the only thing that can be concluded from them is that they should be replaced (i.e. conducted with a higher quality).

References

1. CEBM. March 2009. Levels of Evidence. http://www.cebm.net/?O=1025

2. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ. 1996 Jan 13;312(7023):71-2.

3. https://sciencepharmer.wordpress.com/2013/06/06/homeopathy-or-how-to-misapply-evidence-based-medicine-part-2/

4. https://sciencepharmer.wordpress.com/2013/06/03/homeopathy-or-how-to-misapply-evidence-based-medicine-part-1/

5. http://en.wikiquote.org/wiki/Albert_Einstein

Advertisements

Tags:

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: