Without evidence of benefit, an intervention should not be presumed to be beneficial or safe.

- Rogue Medic

How did this happen? – Research

Part of the problem with research is the same as the problem with all prediction. We are not good at it. We remember the things we were right about, but we conveniently forget the things we were wrong about.

Nassim Nicholas Taleb[1] describes part of the problem by reversing the situation. Do not try to predict the future, but try to predict the past. Imagine an ice cube placed on a table and try to imagine the way that it will melt. What will the result look like? The result will be a puddle of water.

Now, don’t imagine the situation as one of predicting the future. Imagine you are faced with a puddle of water like the one from the first example. Now, try to imagine what that puddle came from. Was it an ice cube? Was it condensation from a cold glass? Was it something entirely different that produced this puddle? If it did come from a piece of ice, was it an ice cube, a small ice sculpture containing the same amount of water, did some of the water evaporate before you saw the puddle? Is the substance that forms this apparently clear puddle actually water?

When performing research we need to try to control variables, so that we know as much as possible, what happened at each step of the experiment. If we put an ice cube down on a table, then leave, and come back and see a puddle, do we know that the puddle is the result of the ice cube melting? 

No, we do not. Is it reasonable to assume this? Yes, it is a reasonable thing to assume, but research is not about assuming things that are likely. Research is about controlling for all variables, especially those that can easily be controlled for.

When performing research, we need to control everything that we can reasonably control. If we are going to see what happens to an ice cube, we need to sit and watch the ice cube melt. Or we can record the events, so that we can examine the events later. If we are sitting there, watching the ice cube, and somebody comes along, puts the ice cube in a glass with a lot of other ice, puts the glass in the same spot, and the condensation from the glass forms the puddle, we need to know this. This would completely change our results. 

Is this far fetched? It does seem to be, but how do we know until we perform the experiment? If we assume things because we think we already know what is going to happen, then we are fooling ourselves. We can assume all sorts of things, just because the seem like common sense. That is not research. Anybody claiming that it is research is wrong. Unfortunately, this kind of carelessness is not uncommon in EMS research.

Do we really care about an ice cube? No. At least, I do not. This is just a very simple example of how we can assume things in research that will lead to a result that is worse than worthless. Why worse than worthless? Because the resources that could have been used to perform a valid experiment have now been wasted. Because the patients who have been exposed to the experimental treatment will never know if they might have benefited from the study – and neither does anyone else.

Of course the investigators will claim that they were able to demonstrate all sorts of useful information, but this is only because they are incompetent. We should not encourage them.

Actually, we should punish them.

Let’s look at the biggest problem of EMS research – quality.

All sorts of criteria are examined, when performing EMS research. Rarely examined is the quality of the providers participating in the study. Are they typical for the organization? A large enough study can take care of that. Are the providers in the organization representative of excellent, or even just competent, EMS ability? 

To many people, just asking that question is an insult. This should give you a hint of what the answer is for that organization. If they are not constantly questioning their quality, how do they have any idea?

Just because an EMS organization is questioning their quality, does not mean that they are asking the right questions, but it is a good start.

What should we ask?

Was this result from an EMS organization with aggressive medical oversight? Requirements for OLMC (On Line Medical Command) permission are not an example of aggressive oversight. This would better be compared to the Wizard of Oz pulling all kinds of levers and making loud noises to create an impressive spectacle. A spectacle that does not have any substance and is supposed to disguise the reality. We need to avoid the smoke, the mirrors, the man behind the curtain, and look for real indicators of quality.

What can we look for to indicate that an EMS organization is able to provide the kind of quality oversight that would recommend them as a site for evaluation of trauma triage criteria?

If the question is – Can EMS safely triage trauma patients by physical assessment, rather than by mechanism criteria? – then these are some of the questions we should be asking – 

Do they have feedback from the trauma center about patients transported to the trauma center?

If not why not?

HIPAA does not forbid this. 

Do they have feedback from the local hospital about patients transported to the local hospital? 

If not why not?

Again, there is no HIPAA problem, here. If anybody is claiming HIPAA, they are telling you a lie. 

Is the medical director following up on all of the trauma alerts, potential trauma alerts, mechanism alerts, patients who should have gone to a trauma center (which is a huge can of worms on its own – does an ICU admission mean a sick patient or a clueless ED?), . . . ?

If not why not?

Are there continuing education classes available?

If not why not? 

Are opportunities available to spend time in the trauma center performing assessments?

If not why not?

If you believe that Dr. Scalea is correct, that we should not be frugal when it comes to people’s lives (I do agree with this), then why aren’t we making sure that we have excellent EMS providers taking care of these patients? 

Are only trauma center patients deserving of excellent care?

Are only trauma patients deserving of excellent care?

Medical patients require much more critical judgment by EMS. They will only be flown to Shock Trauma when they also have a trauma complaint. Are they unimportant?

Certainly not.

What about feedback on medical patients?

How many pneumonia patients receive furosemide from EMS?

How many pneumonia patients receive furosemide from the hospital?

All that crackles is not CHF.

CHF should not be treated by EMS with furosemide anyway.

This is another way of finding the organizations that not only should not be participating in research, but should not be participating in EMS. 

But this is not trauma and we are only interested in trauma.

Not true. CHF patients are victims of trauma, too. Medical decision making does not limit itself to trauma, or medical, or IV skills, or intubation skills, or . . . . We need to look at the capabilities of those we seek to use as our example of EMS providers. The mediocre need not apply.

If the assessment skills are not common, we need to improve them, so that they become more common. We cannot throw our arms up and whine about it being impractical. This high quality is demonstrated in some EMS organization. 

High quality is not too expensive. If it is considered too expensive, then the organization should be limited to BLS care only.

Narrative Fallacy –

Narrative Fallacy I

How did this happen? – Research

Narrative Fallacy II

CAST and Narrative Fallacy

C A S T and Narrative Fallacy comment from Shaggy

Some Research Podcasting Comments

Shaggy Comments on Some Research Podcasting Comments.

Spine Immobilization in Penetrating Trauma: More Harm Than Good?

EMS EdUCast – Journal Club 2: Episode 43

Education Problems, Autism, and Vaccines

Footnotes:

[1] The Black Swan: The Impact of the Highly Improbable
By Nassim Nicholas Taleb
A must read book. If you have anything to do with risk management, then uncertainty/randomness/the unexpected are important parts of what you do. He deals with them better than anyone else. Too many misunderstand his writing, perhaps because they cannot abandon their own biases and accept their lack of control of events. While I find his prose to be awkward (perhaps he does not appear to be awkward, when compared to my writing, so maybe it is just me), his conclusions are essential to the understanding of risk management. Risk management people include any of us who treat patients.
Article about The Black Swan.

.

Narrative Fallacy I

In looking at the research on the ability of mechanism to predict actual severity of injury, we first have to look at some of the problems with research. One of these problems is the use of narrative fallacy to try to explain things that the research has no ability to explain.

What is narrative fallacy?

Narrative fallacy is the need to put information into a narrative, or story, to explain the unknown. Although, if we are explaining the unknown, it cannot be unknown. The problem is that we delude ourselves into believing that we understand what we are explaining. We convince ourselves that we know the unknown. A study comes out that shows that Drug X has a positive effect on Disease Y, so somebody feels the need to give explanation Z.

Why?

To satisfy that need for an explanation. We need to ignore the explanations as much as possible.

But we need to know why we are giving a treatment!

We give a treatment because research shows that it is effective for a specific condition. Everything that has not been shown to be effective is experimental. The explanation is not important. The explanation is often wrong and will probably be revised significantly, if not completely reversed.

The creation of a hypothesis is a necessary part of the scientific method. It is a way of generating testable ideas. We should not apply it to the results of experimentation, except to create more hypotheses to be tested.

Without the scientific method, we have magical thinking. The scientific method is the best way we have of protecting ourselves from the mistakes of magical thinking. Magical thinking is believing that something is happening by some mechanism that sounds good, may even make a lot of sense, but it is untested.

Explanations make it so that we feel better about what is happening. We feel that we have some ability to control things. Stories also help us to remember information. Try remembering a bunch of random information. It is difficult to remember random information. Put that information into a story and you will find that your ability to remember the information increases dramatically. The story doesn’t even have to be relevant. This is why we use mnemonics. They are memory aids.

One of the old mnemonics for ACLS (Advanced Cardiac Life Support) was Shock, shock, shock; Everybody shock; Little shock, Big shock. This was a way of memorizing the beginning of the Ventricular Fibrillation/Pulseless Ventricular Tachycardia algorithm. Shock, shock, shock = shock 3 times, or defibrillate at 200 joules, 200 – 300 joules, then 360 joules. Everybody shock = the E was a reminder that Epinephrine was the first drug after the initial shocks, then shock once again. Little shock = the L was a reminder that the first antiarrhythmic drug was Lidocaine, then we shocked again. Then you needed to remember to repeat epinephrine and a shock. You might give more lidocaine, or you might move on to the next antiarrhythmic – Big shock = the B was a reminder that the next antiarrhythmic drug was Bretylium.

These were just something that somebody thought up. They were just to help some people remember the order of drugs, but not the doses. The drugs from the mnemonic are mostly gone from the treatment of Ventricular Fibrillation/Pulseless Ventricular Tachycardia.

Why?

They weren’t based on real science, just expert opinion. For the time, expert opinion was the way to go. Now we have done quite a bit of research on these drugs and bretylium is not even used any more. Lidocaine is only used by places that do not use amiodarone. Epinephrine is still used, but not in the high doses that used to be encouraged. None of the drugs are supported by research that shows more patients surviving with a functioning brain, but we are so caught up in the narrative fallacy of epinephrine to get the heart going and antiarrhythmics to make the bad rhythms go away, that we continue to use these ineffective drugs. As we become more aware of the gaping holes in the narrative fallacy, we modify, or remove, the errors. This is how science progresses. We stumble along from one experiment to the next, with the occasional major revision.

The drugs are no longer the main focus, since they do not work. Still, in science politics does have a role. Some refuse to let go of the idea of a wonder drug of resuscitation. But this is a great example of the narrative fallacy.

How?

The drugs do not work, but there are wonderful scientific explanations of the way they work. Since they do not work, this is just wishful thinking. It may be great to help you remember the order of treatments, but you will have to learn some other mnemonic later, because they will eventually accept that the research is just not there to support using these drugs in cardiac arrest.

If you want to read the current narrative fallacy, the link at the bottom will give you all of the current explanation of how the drugs work. The older versions of ACLS explanations carry the narrative fallacy from that time period. The next revision will carry the new and improved narrative fallacy. At least the AHA (American Heart Association) starts out by admitting this lack of understanding in the ACLS text. Too many of the instructors just seem to use the algorithm charts, which do not include the following statement.

For victims of witnessed VF arrest, prompt bystander CPR and early defibrillation can significantly increase the chance for survival to hospital discharge. In comparison, typical ACLS therapies, such as insertion of advanced airways and pharmacologic support of the circulation, have not been shown to increase rate of survival to hospital discharge.[1]

Stories are great – as long as we realize that they are fiction. Even if printed in the most reputable scientific journal, they are not the truth. They are an approximation of how the current research might be explained. Science is not etched in stone. Science is constantly changing. Not every change is progress, but cumulatively there is tremendous progress.

Narrative Fallacy –

Narrative Fallacy I

How did this happen? – Research

Narrative Fallacy II

CAST and Narrative Fallacy

C A S T and Narrative Fallacy comment from Shaggy

Some Research Podcasting Comments

Shaggy Comments on Some Research Podcasting Comments.

Spine Immobilization in Penetrating Trauma: More Harm Than Good?

EMS EdUCast – Journal Club 2: Episode 43

Education Problems, Autism, and Vaccines

Footnotes:

[1] (Circulation. 2005;112:IV-58 – IV-66.)
© 2005 American Heart Association, Inc.
2005 American Heart Association Guidelines for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care
Part 7.2: Management of Cardiac Arrest

Introduction

.