Are there crises in EBM (Evidence-Based Medicine)?
If so, what are the crises and what can we do to make things better?
John wrote this in the comments to We all rely on evidence. The important difference is __________.
I wonder what you think about this:
The article, The crisis in evidence-based medicine, references a BMJ article that I will try to write about soon, since it provides an excellent description of what EBM (Evidence-Based Medicine) is and how EBM is misused.
What many of the opponents of EBM (I am not including Bill Gardner) seem to ignore is that the problems that affect the application of high quality evidence to patient care are even greater with the application of
the lowest quality evidence anecdotal experience to patient care. For example, multimorbidity makes it even more difficult to draw conclusions based on anecdotes, than based on evidence prospectively collected in a way that is designed to minimize the complications produced by the many conditions not being studied, while anecdotalists rely on experience, that is more likely to be irrelevant than the research and they rely on their memories. One thing that objective evidence shows us is that our memories are not accurate.
However, Bill Gardner is looking at the ways we can avoid making mistakes with EBM, rather than making excuses for ignoring EBM. I recommend reading the BMJ article first, then going back and reading his article and the other articles at The Incidental Economist on EBM.
There is a weak point in Bill Gardner’s solution, and he is aware of it. At what point does the quality of the data become so low that the massive quantity is not able to provide useful information?
The NTDB® (National Trauma Data Bank®) is a great idea. We objectively collect as much data on as many trauma patients as possible and mine that data for signals that stand out from the noise. I have pointed out some of the problems with the application of this approach several times.,,
Eventually, we should become much better at acquiring data, but so far we have demonstrated that even data prospectively collected by the experts can be GIGO (Garbage In = Garbage Out). The same is true for EBM. If we use low quality data, we should expect low quality results. Eventually, we should find better ways to filter out the noise, but we are not there, yet.
The Framingham Heart Study is an example of using massive amounts of data to find evidence that several risk factors lead to significantly shorter lives.
As with anything else, the problems are not a reason to abandon hope, but a reason to use caution. We will learn as we go, while the acolytes of anecdotalism will continue to promote misunderstanding and continue to discourage progress.
We need to learn more about how to use science and how not to be misled by science, rather than to abandon science. Here is just a start to learning how not to get fooled by bad science.
Image credit. Click on the image for a much larger version (1754×1280).
While the perfect is often the enemy of the good, that does not mean that good is not good, or that we should reject anything that is not perfect in favor of the alternative (not good or bad or much worse). It means that we need to keep improving. Science does keep improving.
 Evidence based medicine: a movement in crisis?
Greenhalgh T, Howick J, Maskrey N; Evidence Based Medicine Renaissance Group.
BMJ. 2014 Jun 13;348:g3725. doi: 10.1136/bmj.g3725. No abstract available.
PMID: 24927763 [PubMed - in process]
Or listen to the audio version in MP3 format by Dr. Mark Crislip –
121: Who you gonna believe, me or your lying eyes