Just how bad is the idea of using a multiple choice exam to evaluate competence? The faulty attempts at logic used to justify the continuing use multiple choice exams as the final knowledge exam give us an idea.
CCC usually makes more sense, but in response to Do EMS Exams Really Gauge Provider Competency? he writes –
The NREMT is forced to evaluate applicants by testing, since they are a certifying agency.
The NREMT (National Registry of EMTs or National Registry) chooses to compete in the for profit testing market.
-
Part of the NREMT’s accreditation requires they ensure the competence of their applicants.
Do they actually claim to ensure the competence of applicants?
No. We only ensure that there is documentation of jumping through hoops that we pretend represent competence.
A multiple choice exam does not even come close to ensuring competence.
We come up with more complicated questions, in an effort to evaluate knowledge, but we are using the wrong method. I could chant very complicated magic spells, but magic doesn’t work. A simple magic spell doesn’t work. A complicated magic spell doesn’t work.
It doesn’t matter how ornate the test. Multiple choice is the wrong choice.
-
Until there is a better way to do that than a multiple choice test, the NREMT will continue to administer that test.
Why not use trial by combat or trial by ordeal to demonstrate competence?
Is there any reason to believe that a multiple choice test, which discriminates in favor of those who are good at avoiding the answers designed to be misleading, is any better at assessing competence?
The continuing reliance on the multiple choice format demonstrates that the National Registry does not understand assessment of ability. More weight is given to the appearance of objectivity, than to the result.
-
Without a test of some sort, there is no NREMT.
EMS does not exist for the NREMT. The National Registry exists for EMS. We are responsible for our own dissolution.
Consider the multiple choice test as the food we eat.
Humans need energy, so I am forced to eat.
I am forced to consume calories by eating, since humans are calorie-burning creatures.
Part of my metabolism requires that I ensure the adequacy of my food intake.
Until there is a better better food than Twinkies, I will continue to live on Twinkies.
This is a more obvious non sequitur than CCC’s, but the premise does not support the conclusion in his example, either.
I am being reasonable. I make some introductory nutrition-related comments, then I just claim that Twinkies are the best and I ignore the possibility that Twinkies are not the best, therefore Twinkies are the best.
Case closed. Twinkies Rule!
-
I think the claim that there is no better food than Twinkies is just as absurd as the claim that there is no better way to evaluate competence than a multiple choice test.
In what universe is a multiple choice test the best of all possible ways of evaluating competence?
The same universe in which Twinkies are the best of all possible foods.
.
NREMT is a non-profit organization, so I’m not sure how that makes them in the “for-profit testing market.” Besides, nobody is forcing states to use NREMT as their tester. States use NREMT because of the ease. Petition your state EMS office to drop NREMT and allow you to administer the exam. There’s potential for you to make money, and ensure the competence of your applicants. But you should be prepared to.spend.hundreds of thousands of dollars to defend your exam methodology.
Would putting an applicant in front of an examiner to field questions be completely objective? I dont think so. As it is now, the applicant either knows the material, or they don’t. That seems pretty objective to me.
If I missed your post on how you.would fix this perceived.problem, I apologize. If there isn’t one, perhaps you could explain how you would fix this perceived problem rather than compare the test to twinkies.
CCC,
I never expressed any interest in starting my own testing organization. I am just pointing out that multiple choice is completely inappropriate for this task.
-
Who cares?
Patients don’t care how objective the test is. They care about how competent the paramedic is.
A coin flip is much more objective, so why not use a coin flip?
Heads you pass. Tails you fail and take the course again.
-
As it is now, the test evaluates their test taking skills, not their knowledge.
That is discrimination against those not good at taking multiple choice tests. Even the goal of objectivity is not met.
-
In other words, this worst of all possible tests is perfect, as long as I don’t have a better solution.
Anything else would be better.
An essay test would be much better.
A Socratic test would be best, although it has as much of an objectivity problem as the NREMT’s tests.
.
Just to clarify one point, my home state had to switch to using the NREMT test for initial licensing because someone attempted to sue the state EMS board using the Sunshine law to open the test bank. Instead of taking their chances in court, the board voted to get out of the testing business. While the tests were still multiple guess, since they were maintained by the State, they were easier to update and keep current with new research and treatments.
I do agree with RM that the current system is not just lacking, but is completely inadequate. I would prefer some combination of written test (maybe a SOAP note based on a given scenario) and an oral board in conjunction with an internship period. Wishful thinking I know.
Regarding ensuring competence, the NCCA requires agencies that wish to be accredited as a.certifying agency to “verify competence of members in an objective manner.”
I could give you much more info on the NREMT, but I would hate to bore your readers. The 6 people who read my blog are probably bored to tears about now. Perhaps we could email about it.
Thanks for saying I usually make sense. “Usually” being the operative word.
CCC,
Then they screwed up with the NREMT.
-
Well, you do.
.
I agree and disagree. There should be some kind of finial testing. The skills testing and all the different test we take to become certified should count into the final grade. I think the final test should only contain material the EMT or EMT-P or anyone else will do or use. For an EMT, don’t put Para questions in the test.
Interesting that there has been no mention of the practical exam in this discussion. I know the focus has been on how inadequate the written exam is at assessing a candidate’s knowledge base, but the practical, for the most part, isn’t a whole lot better. Basically, in any given station a candidate recites a script showing that he/she can say the right thing at the right time and lay hands on a mannequin in the same way, sometimes with or without devices to be placed in or on the mannequin.
Does that show knowledge?
In my view, it shows that anyone can memorize a script.