Without evidence of benefit, an intervention should not be presumed to be beneficial or safe.

- Rogue Medic

Educated to Kill: How EMS Education Preprograms Medical Errors in Future Clinicians

 

 

Jamie Davis (the man behind the camera for this show and the man behind the MedicCast), Chris Montera (the man behind the EMS Garage), Kyle David Bates (Pedi-U and First Few Moments), Kelly Grayson (A Day in the Life of an Ambulance Driver and Confessions of an EMS Newbie) and I were on a video podcast from the 2012 EMS Expo in New Orleans.

None of the participants seem to feel that this discussion topic is a form of personal attack and the discussion is calm and thoughtful.
 

Chris Montera Assembles a Crew of EMS Leaders at EMS World Expo
November 19th, 2012

You can watch the video podcast at the ProMed Network page here.

Advance the video to the 10 minute mark for the beginning of the discussion.
 


 

Before the podcast I was at David Page’s presentation on how we educate EMTs and medics to kill. I had been trying to hear Dr. Keith Wesley’s presentation on the harms of oxygen, but the room was overcrowded, the crowd was spilling out into the hallway, and the microphone was not working. Why was Dr. Wesley put in a small room? I don’t know, but I could not hear, so I left.

I was very disappointed – until I saw the title of David Page’s presentation. That is a great topic. When I heard his presentation, I was even happier.

If you are an educator and you get a chance to attend this presentation – do not miss it.[1]

In our talk, we discussed the one thing on which I did disagree with David Page – the value of multiple choice test questions on a certification exam.

Multiple choice questions provide the person taking the test with limited information and no opportunity to obtain further information. Then the examinee is supposed to choose, from among several selections, the one best answer.

The idea of one best answer is a fraud.

Limited information.

No opportunity to gather more information.

Must choose one of the selections.

This just teaches bad decision making skills.

The multiple choice question on a certification exam is an excellent example of Educated to Kill: How EMS Education Preprograms Medical Errors in Future Clinicians.[1]

That was the only disagreement I had with David’s presentation.

We come up with ridiculous ways to artificially limit the learning of our students. For example – why not have the students create their own scenarios after reading the material for the class? No lecture. No guidance from the instructors. The students run it with only occasional feedback from the instructors.

The objections came from the instructors –

What do you mean, we are not necessary?!?!?

They were not thinking of the students and how this may be better for the students. They were only thinking of how this devalued their input and overturned their dogma.

If you are big on dogma, you are probably more harmful than beneficial.

This is another excellent example of Educated to Kill: How EMS Education Preprograms Medical Errors in Future Clinicians.[1]

We discuss several problems with education.

Why do we assume that the amount of time we spend on education is just right?

If our certification exams are any good at assessing for adequate education, why do we have any kind of classroom attendance requirements to get in?

I think that we just figure the completion of the class means that it doesn’t matter who passes the test, since the certification test is not a valid assessment.

Here is an example of the problem with multiple choice questions –

Worst test question ever! – Maybe
 

You can watch the video podcast at the ProMed Network page here.
 

The first 10 minutes is an advertisement, but it is for an interesting product that looks as if it can dramatically cut down on the risk of needlestick injuries and cut down on the cost of disposal of needles.

If you regularly have problems getting needles into the sharps container, because people who inexplicably passed a multiple choice exam esteemed colleagues have stuffed bandages and other non-sharps into the sharps container, the Sharps Terminator by Medical Safety Solutions should prevent that dangerous situation. I do not receive anything from the company.

Footnotes:

[1] Educated to Kill: How EMS Education Preprograms Medical Errors in Future Clinicians
Nov 1 2012 11:00AM
David Page, MS, NREMT-P
Thursday Schedule

This controversial look at safety and medical errors in EMS explores the role of education in preprogramming future clinicians to make deadly mistakes. Is it time for EMS education to embrace its role in creating cultures of safety? Or is it all up to the employers?

.

Comments

  1. This presentation was great, except for the annoying guy in the back that kept asking about multiple choice questions. I’m not sure who that was….

    Seriously, this was a fantastic presentation that hopefully opened the eyes of some of the instructors there. Practice like you play should be the mantra in every EMS classroom.

  2. I don’t think multiple-choice exams are inherently bad or that they necessarily teach poor decision-making. I would agree that they are not ideal – there certainly are better ways to evaluate understanding, if you have the time and resources to use them – but multiple-choice exams (MPE’s) provide a practical, convenient way to evaluate knowledge of simple core concepts and facts.

    MPE’s have limitations, for sure. They aren’t going to evaluate a student’s ability to integrate lots of data and process complex concepts, such as a realistic patient assessment scenario. Things like simulation, essays, and oral boards are probably better for that. But those are also more time consuming and are harder to evaluate objectively.

    Medical schools and other advanced clinical programs rely heavily on MPE’s quite successfully. The program I’m in now uses mostly MPE’s (as well as lots of simulation), and while I’m not in love with them, I think they’re pretty valid for lots of things, as long as they are well written and as long as they don’t try to evaluate things that just can’t be evaluated in such a simple format.

    IMO, the big issues with EMS education aren’t the examination format. The big problems are that most paramedic programs are too brief, not at all intellectually rigorous, and taught by people who (usually) have no advanced training in a clinical discipline or in education.

    You can’t take people with little or no science background and:
    – put them in a program that is only 9 months long,
    – and is based on textbooks written at a 10th-grade reading comprehension level,
    – and is taught by instructors who have little or no advanced education themselves,
    – and only requires a couple hundred hours of (very often completely useless) clinical experience,
    – and evaluates the students solely using poorly-written exams,
    – and expect to produce a clinician who is supposed to be able to function at the level of a physician within a very narrow scope of practice.

    If we want paramedics to be viewed as qualified clinicians, then our educational programs need to have a solid foundation in the basic sciences (either by including it in the program or requiring them as pre-reqs), they need to be long enough so that the courses can cover material in some depth and with some rigor, the clinical experiences need to be structured and focused and lengthy and aimed at developing clinical-decision making skills, and the importance of clinical research and evidence-based practice needs to be stressed every step of the way.

  3. I agree with you, but I also agree with Allan. I’ve seen a fair number of EMS educators blatantly teach their students to pass “the test”. Deplorable! But I also bit the bullet a couple of years ago and signed up to do an item writing committee for NREMT, and will probably do another in the near future. I spent many hours crafting questions to try and elicit not the “right” answer, but an answer that would demonstrate knowledge of the curriculum and the ability to think, to apply that knowledge and choose the best answer. And then the Registry locked 10 of us in a conference room for a weekend to make sure that all the questions really did exactly that. It’s ironic….a number of EMS instructors “teach” their students to pass a test, and the folks writing the test are trying to make sure that the instructors taught their students to think.

  4. Let’s not forget, the NREMT multiple choice exam is not supposed to be the only way to evaluate if a person is good enough to be a paramedic. A prospective medic can’t even take the exam unless he proves he has completed a certified paramedic course that includes a certain number of clinical hours and a certain number of field hours under a qualified preceptor. The NREMT is simply a check to ensure that the paramedic school is a real school and not simply selling grades.

    Because of this requirement to pass a paramedic program first, the multiple choice exam is not the only measure of whether a student is good enough; it’s merely the last check. The student should have already been evaluated for hundreds of hours in real-world situations (clinical and internship).

    For comparison, look at college atheletics. To ensure that a student athlete is not merely being handed passing grades to keep eligibility, the athlete must get a minimum score on the SATs (a rather notorious multiple choice exam). The SATs do not make a person a college student or not; they merely establish whether the grades from high school are credible. Similarly, the NREMT should simply establish whether a student’s passing grade is credible.

    • mpatk

      Let’s not forget, the NREMT multiple choice exam is not supposed to be the only way to evaluate if a person is good enough to be a paramedic. A prospective medic can’t even take the exam unless he proves he has completed a certified paramedic course that includes a certain number of clinical hours and a certain number of field hours under a qualified preceptor. The NREMT is simply a check to ensure that the paramedic school is a real school and not simply selling grades.

      And yet the NR test is the difference between being able to work and not being able to work for maybe most of the people in EMS.

      It does not really evaluate the knowledge of the student.

      It may evaluate the ability of the school to prepare people to take a test to evaluate the ability of the school at teaching to the test.

      We need valid ways of evaluating potential paramedics, not excuses for the shortcomings of the tests we use.

      We need valid ways of evaluating paramedic programs, not excuses for the shortcomings of the tests we use.

      Because of this requirement to pass a paramedic program first, the multiple choice exam is not the only measure of whether a student is good enough; it’s merely the last check. The student should have already been evaluated for hundreds of hours in real-world situations (clinical and internship).

      I think you are not taking that far enough.

      Because of this requirement to pass a paramedic program first, the multiple choice exam is just window dressing.

      Does the test in any way decrease the weaknesses of the other methods of screening?

      If the test contributes nothing, why use it?

      .