Without evidence of benefit, an intervention should not be presumed to be beneficial or safe.

- Rogue Medic

Dr. Kudenchuk is Misrepresenting ALPS as ‘Significant’

ResearchBlogging.org
 

The results of ALPS (Amiodarone, Lidocaine, Placebo Study) are clear. There is no statistically significant difference in cardiac arrest outcomes with amiodarone or lidocaine, when compared with placebo.
 

Conclusions Overall, neither amiodarone nor lidocaine resulted in a significantly higher rate of survival or favorable neurologic outcome than the rate with placebo among patients with out-of-hospital cardiac arrest due to initial shock-refractory ventricular fibrillation or pulseless ventricular tachycardia.[1]

 

This study was very well done, but it was not designed to provide valid information about the effects of amiodarone or lidocaine on witnessed arrests or on EMS Witnessed arrests. Maybe the authors were overconfident.

In resuscitation research, we have abundant evidence that overconfidence is much more common than improvements in outcomes. There is no study that has shown an improvement in neurologically intact survival to discharge with any drug. Leaving the hospital with a working brain is the result that matters most to patients. We give drugs because we have too much confidence in the drugs and we are treating our confidence, not because we are doing anything to benefit the patients.
 

I WANT TO BE DECEIVED version of Domenichino, Virgin and Unicorn 1 copy
 

In ALPS there was a subgroup that might have reached statistical significance, but the researchers never determined what would be statistically significant when setting up the study, so these results are merely post hoc data mining (fitting the numbers to allow for a positive spin).

This is the Texas sharpshooter fallacy. The Texas sharpshooter shoots at the side of a barn, then draws targets around the bullet holes so that the the bullet holes are in the bull’s eyes.
 


 

The Texas sharpshooter didn’t shoot at any target, but he went back later and made it look like he hit the center of the target, because he drew the target around the bullet holes. Science requires that we state our hypotheses ahead of time, so that scientists are kept honest. Science requires that we calculate statistical significance ahead of time, especially for secondary outcomes/subgroup analysis, which may mean decreasing the p value to less than 0.03, or to less than 0.01, or even lower to reach statistical significance, so that scientists are kept honest. You are not permitted to bet on the outcome of a horse race that is already in progress for the same reason.

Why do we need to keep scientists honest? Because, as Dr. Peter Kudenchuk unintentionally demonstrates, scientists are just as biased as everyone else. Scientists need to follow the rules of science to minimize the influence of prejudices, such as overconfidence. When scientists do not follow these rules, they are just as easily fooled as everyone else and they may use that self-delusion, and their reputation, to fool others. Dr. Oz makes a fortune telling people what they want to hear about treatments that do not work.

I don’t claim that Dr. Kudenchuk, or even Dr. Oz, is deliberately fooling others, only that they have fooled themselves and are trying to convince others that their prejudices are accurate representations of reality. Here is what Dr. Kudenchuk has been telling people –
 

Researchers have confirmed that certain heart rhythm medications, when given by paramedics to patients with out-of-hospital cardiac arrest who had failed electrical shock treatment, improved likelihood of patients surviving transport to the hospital.[2]

 

The researchers have not confirmed any such thing.

If Dr. Kudenchuk wants to study whether amiodarone or lidocaine or both improve outcomes for witnessed cardiac arrest patients, or for EMS witnessed cardiac arrest patients, he needs to set up a study with all of the criteria for a positive result specified before the start of the study, because this study did not. The study explicitly states this, so Dr. Kudenchuk should be able to just read the study and see that he is wrong. Here is another statement that contradicts the information that was published.
 

Two groups of patients were pre-specified by the study as likely to respond differently to treatment: those with a witnessed cardiac arrest and those with an unwitnessed arrest. When it was originally designed, the study predicted that because patients with witnessed cardiac arrest are recognized and treated sooner, they would more likely be responsive to effective treatments than unwitnessed arrests. When first discovered, patients with an unwitnessed arrest are more likely to have already sustained irreversible organ damage resulting from a longer “down time” and less likely to respond to any treatment. This is precisely what was seen in the study – a statistically significant 5% improvement in survival to hospital discharge in witnessed arrests, and no effect from the drugs in unwitnessed arrests.[3]

 

Why does the published version of the paper contradict Dr. Kudenchuk? One of our biases is to remember things differently from the way things really happened. This is why eyewitness testimony is so often wrong. Here is what the published paper states about the witnessed arrest results.
 

We observed an interaction of treatment with the witnessed status of out-of-hospital cardiac arrest, which is often taken as a surrogate for early recognition of cardiac arrest, a short interval between the patient’s collapse from cardiac arrest and the initiation of treatment, and a greater likelihood of therapeutic responsiveness. Though prespecified, this subgroup analysis was performed in the context of an insignificant difference for the overall analysis, and the P value for heterogeneity in this subgroup analysis was not adjusted for the number of subgroup comparisons. Nonetheless, the suggestion that survival was improved by drug treatment in patients with witnessed out-of-hospital cardiac arrest, without evidence of harm in those with unwitnessed arrest, merits thoughtful consideration.[1]

 

The authors did not adjust the p value, so the authors do not claim that the witnessed cardiac arrest results are statistically significant. They only state that these results merit thoughtful consideration. In other words, if we want to claim this hypothesis is true, we need to set up a study to actually examine this hypothesis.

One earlier study (also by ROC – the Resuscitation Outcomes Consortium) even has similar results.[4],[5] These results are also not statistically significant, but suggest that with larger numbers the results might be significant. So why did the authors set up such a small study? Overconfidence and an apparent lack of familiarity with their own research.
 


 

The Seattle phenomenon (they claim that their resuscitation rate is the highest in America) seems to be due to excellent bystander CPR rates (apparently the highest in America), but that is only good enough for them to be experts on improving bystander CPR rates. The rest is probably due to defibrillation and chest compressions, which are the only prehospital interventions demonstrated to improve neurologically intact survival.

Why does a bystander CPR specialist focus on drugs? Overconfidence and an apparent lack of understanding of the resuscitation research. Dr. Kudenchuk preaches like Timothy Leary about the benefits of drugs and with just as little evidence. We should give appropriate credit for Dr. Kudenchuk’s work on CPR, but we should not mistake that for a thorough understanding of the resuscitation research, even the research with his name attached.
 

A new podcast reviews ALPS. Dominick Walenczak does not notice the mistakes of Dr. Kudenchuk, but he is not one of the researchers, so that is easy to overlook. The rest of the podcast is excellent. Listen to it here.
 

Episode 8: Conquering the ALPS (Study)
CritMedic – Critical Care Paramedicine Podcast
Dominick Walenczak
April 7, 2016
Podcast page
 

Footnotes:

[1] Amiodarone, Lidocaine, or Placebo in Out-of-Hospital Cardiac Arrest.
Kudenchuk PJ, Brown SP, Daya M, Rea T, Nichol G, Morrison LJ, Leroux B, Vaillancourt C, Wittwer L, Callaway CW, Christenson J, Egan D, Ornato JP, Weisfeldt ML, Stiell IG, Idris AH, Aufderheide TP, Dunford JV, Colella MR, Vilke GM, Brienza AM, Desvigne-Nickens P, Gray PC, Gray R, Seals N, Straight R, Dorian P; Resuscitation Outcomes Consortium Investigators.
N Engl J Med. 2016 Apr 4. [Epub ahead of print]
PMID: 27043165

Free Full Text from NEJM

[2] Antiarrhythmic drugs found beneficial when used by EMS treating cardiac arrest
Press release
For Immediate Release:April 4, 2016
NHLBI (National Heart Lung and Blood Institute)
Press release

[3] Dr. Kudenchuk: Study reveals exciting news about cardiac arrest treatment
Lindsay Bosslet
18 hours ago
Public Health Insider
Article

[4] Wide variability in drug use in out-of-hospital cardiac arrest: A report from the resuscitation outcomes consortium.
Glover BM, Brown SP, Morrison L, Davis D, Kudenchuk PJ, Van Ottingham L, Vaillancourt C, Cheskes S, Atkins DL, Dorian P; Resuscitation Outcomes Consortium Investigators.
Resuscitation. 2012 Nov;83(11):1324-30. doi: 10.1016/j.resuscitation.2012.07.008. Epub 2012 Jul 31.
PMID: 22858552 [PubMed – indexed for MEDLINE]

Free Full Text from PubMed Central.

[5] Wide variability in drug use in out-of-hospital cardiac arrest: A report from the resuscitation outcomes consortium – Part I
Mon, 17 Sep 2012
Rogue Medic
Article

 
Kudenchuk, P., Brown, S., Daya, M., Rea, T., Nichol, G., Morrison, L., Leroux, B., Vaillancourt, C., Wittwer, L., Callaway, C., Christenson, J., Egan, D., Ornato, J., Weisfeldt, M., Stiell, I., Idris, A., Aufderheide, T., Dunford, J., Colella, M., Vilke, G., Brienza, A., Desvigne-Nickens, P., Gray, P., Gray, R., Seals, N., Straight, R., & Dorian, P. (2016). Amiodarone, Lidocaine, or Placebo in Out-of-Hospital Cardiac Arrest New England Journal of Medicine DOI: 10.1056/NEJMoa1514204

 

Glover BM, Brown SP, Morrison L, Davis D, Kudenchuk PJ, Van Ottingham L, Vaillancourt C, Cheskes S, Atkins DL, Dorian P, & the Resuscitation Outcomes Consortium Investigators (2012). Wide variability in drug use in out-of-hospital cardiac arrest: A report from the resuscitation outcomes consortium. Resuscitation PMID: 22858552

.

Comments

  1. Well, the study itself is significant – in that it reinforces the fact that we should not be using these drugs in cardiac arrest, because there is still no significant evidence of benefit.

    It is also significant in that it points to some good avenues for more research.

    I would, however, like to see this study repeated, with at least the following changes:

    – using the most common formulation of amiodarone in use as of the beginning of the study, rather than a new and not widely used formulation;
    – designed from the start to determine if amiodarone or lidocaine may be beneficial in witnessed arrests, as this study suggests, and;
    – powered sufficiently to determine if the less than expected increase in survival to discharge with these antiarrhythmics is real or just statistical noise.

Trackbacks

  1. […] [3] Dr. Kudenchuk is Misrepresenting ALPS as ‘Significant’ Tue, 12 Apr 2016 Rogue Medic Article […]

  2. […] Dr. Kudenchuk is Misrepresenting ALPS as ‘Significant’ […]