Without evidence of benefit, an intervention should not be presumed to be beneficial or safe.

- Rogue Medic

How TO Respond to Negative Research

Also posted over at Paramedicine 101 (now at EMS Blogs). Go check out the rest of the excellent material at these sites. There is a new research podcast specifically for EMS – EMS Research Podcast. On episode 2 we discuss several topics, including the research on the RAD-57 non-invasive monitor.

In the comments to How Not to Respond to Negative Research is this excellent example from daedalus2u of a response to negative research that is much better than the response by Dr. O’Reilly.

I suspect that the problem is in the factory calibration of the device (the algorithms for calculating COHb from spectral information) which I think derives from a mistaken idea of what is important to determine.

The error seems to be due to the way the RAD-57 is calibrated.

I suspect the factory calibration is being driven by marketing considerations, i.e. because the marketing people want to put “accurate to X%” on their glossy marketing brochures, rather than listen to those involved in treating patients as to what information they actually need to treat those patients.

I would be cynical, if I were to conclude that Dr. O’Reilly might not have considered this, and that he had a lot of input into the calibration marketing of the RAD-57. That is, if daedalus2u is correct in his analysis and if I am cynical. Clearly, these are impossibilities.

The only purpose of measurements like this (and for any diagnostic test) is differential diagnosis which is only useful for differential treatment. A test does not need resolution better than the resolution necessary to provide differential treatment.

Why calibrate the RAD-57 to identify dangerous levels of carboxyhemoglobin, when the alternative is to calibrate the RAD-57 for only the appearance of accuracy?

What is important to determine in a patient with COHb is not a low percentage deviation from the “true” amount of COHb, but rather what is the absolute amount of COHb within a certain range.

And here is the explanation –

If the “true” COHb level is 20%, getting a reading of 30% or 15% isn’t going to change the treatment, O2 and getting to a HBO2 facility (my guess, but IANAD).

Actually, since the RAD-57 is a non-invasive portable device advertised as capable of ruling out the possibility of unrecognized carbon monoxide toxicity, the choice is between identifying a need for transport to the hospital.

I think the “problem” is that the device is tuned to give the most precise readings in the low COHb range where precision isn’t important because low COHb readings are the most common. Then when those measurements are compared and averaged, the precision at the low end dominates the average, but the precision at the low end isn’t at all important. What is important is the rare go/no go values at the high end.

At the hospital, a blood test will be done to confirm any elevated RAD-57 reading.

All that is needed is a Go/No Go decision. Go to the hospital vs. don’t go to the hospital.

I don’t know anything about the details of how the device works, other than it uses some kind of spectral stuff, but it is inconceivable to me that the technique couldn’t be made to technically work, and technically work just by changing the software. I think the problem there is that because the device has to be FDA approved, changing the software turns it into a new device that needs new FDA approval even if it works better. I suspect that business considerations are preventing that from happening.

Sometimes knowing too much about the specifics may distract us from the relevant points.

With the way the FDA does things – occasionally too affected by politics – changing the software should be much more easily resolved than paying out some large settlements for unrecognized toxicity, but I don’t mean to criticize, if paying out large settlements is Masimo’s marketing strategy.

If this explanation is correct, I think that Masimo owes daedalus2u a lot of money.

PS – Steve comments –

In this crazy world of online advertising, immediately after your damning post is… an ad for the Masimo Rad-57!! “Quick and Noninvasive Assessment of CO Levels in the Blood” it reads, and features a photo of a Fireman and a Medic. The Fireman has had a rough day; he’s seated, no helmet, with the Rad-57 clip attached to his finger. But you know in just a moment the Medic is going to tell him he’s A-OK and send him back to work – because the Rad-57 says he’s great! (Maybe…maybe not!)

Steve’s comment about the message communicated by the ad is correct.

Steve’s comment about the placement of the ad is not correct. I copied the ad into my post to make a point. The image of the Magic 8-Ball is right next to the RAD-57 image to make a counterpoint.


.

Trackbacks

  1. […] This post was mentioned on Twitter by ALCO Medics & EMT's, Chronicles of EMS. Chronicles of EMS said: How TO Respond to Negative Research http://bit.ly/eZucG3 Via @EMSblogs #CoEMS […]