Without evidence of benefit, an intervention should not be presumed to be beneficial or safe.

- Rogue Medic

Worst test question ever! – Maybe

 

Thank you to David Baumrind of EMS 12 Lead for linking to this here. It probably is not the worst test question ever, but it is very bad.

Read the question, figure out what your response would be, then scroll down for my explanation.
 

You are dispatched emergency traffic to the scene of a 24 yo F with “palpitations.” You arrive to find her pale, sweaty and lethargic. You palpate a radial pulse with an extreme rate. You hook her up to the monitor and find the following rhythm? You have a 45 minute transport time. Which of the following is the most appropriate initial treatment for this condition?

1.) Nitroglycerin 0.4mg SL
2.) Immediate synchronized cardioversion
3.) Adenosine 12mg Rapid IV push followed by 20cc NS bolus
4.) Epinephrine 1mg 1:10000 q-3-5m IVP

-Admin Paul

The original posting was from Exhausted Medic Students ‘R’ Us here.

Go read the original with its hundreds of comments.

 

*

 

*

 

*

 

*

 

*

 

*

 

*

 

*

 

*

 

*

 

*

 

All of the answers are completely wrong.
 

ST (Sinus Tachycardia) is the rhythm.

There are clear P waves with consistent PR intervals. It is faster than what some people expect to see from ST, but that is because many of us do not think about what we are learning in EMS.

It is true that the cardiology part of paramedic school is probably the toughest for most people, and we are overwhelmed with new information, but we should be very familiar with this rhythm.

Carry a patient up/down a flight of steps and you may have significant ST – maybe even faster than what is on this strip. If your heart rate is over 150, so what?

Before you have a chance to recover, use the pulse oximeter to measure your heart rate after carrying a patient. You are just checking the accuracy of the machine before applying it to the patient, or before reconnecting it to the patient.
 

1. Nitroglycerin is NOT indicated for palpitations.

NTG is not indicated even for a lot of palpitations. Do you have a protocol for NTG for palpitations?

Ask your medical director how much NTG should be given for palpitations, but don’t be surprised if you are expected to go through some scenarios to demonstrate that you would not really give NTG for palpitations.
 

2. Cardioversion is NOT indicated for sinus tachycardia.

Cardioversion is supposed to cause asystole. During that asystole, it is hoped that the sinus node will become the pacemaker for the patient’s rhythm.

SINUS tachycardia means that the sinus node is already the pacemaker.
 

Cardioversion of sinus tachycardia can only make things worse.
 

Cardioversion of sinus bradycardia can only make things worse.

Cardioversion of any sinus rhythm can only make things worse.
 

3. Adenosine is NOT indicated for sinus tachycardia.

The dose does not matter. The drug is not indicated.

No matter how wrong NTG is for palpitations, adenosine is worse.
 

4. Epinephrine is NOT indicated for sinus tachycardia with a pulse.

How much faster do we want this ST to be? Epinephrine can make it faster.
 

Maybe some people think that the choices should include a vagal maneuver.

No. That would also be wrong.

Calcium channel blocker?

Another wrong.

Beta blocker?

Wrong again.
 

No competent paramedic should attempt to justify any of these answers.

Maybe this is a question to find out just how incompetent people will be to satisfy an authority figure.

One horrible answer is –
 

As a paramedic instructor and a evaluator for National Registry…if my student didn’t cardiovert…I’m failing them.

 

Does the National Registry hire people this ignorant as evaluators?

Yes, but so does every other testing organization. Maybe this guy is lying about being an instructor and evaluator, but this is EMS and we like low standards.

A defender of cardioversion posted the ACLS tachycardia cheat sheet.
 

Click on image to make it larger.

 

Unfortunately, the cheat sheet does not state that we should not shock sinus tachycardia.

If all we know is the cheat sheet, we should consider a career change to explore the exciting world of fast food order fulfillment.

The text of the 2010 ACLS guidelines states –
 

ACLS professionals should be able to recognize and differentiate between sinus tachycardia, narrow-complex supraventricular tachycardia (SVT), and wide-complex tachycardia.[1]

 

A lot of people could not recognize an obvious sinus tachycardia.

Is that the fault of their instructors?

Yes and No.
 

Sinus tachycardia is among the rhythms listed that we are expected to be able to identify.
 

Synchronized cardioversion is recommended to treat (1) unstable SVT, (2) unstable atrial fibrillation, (3) unstable atrial flutter, and (4) unstable monomorphic (regular) VT. Shock can terminate these tachyarrhythmias by interrupting the underlying reentrant pathway that is responsible for them.[1]

 

Sinus tachycardia is not listed among the rhythms that should be shocked.

Here is the important part –
 

If judged to be sinus tachycardia, no specific drug treatment is required. Instead, therapy is directed toward identification and treatment of the underlying cause. When cardiac function is poor, cardiac output can be dependent on a rapid heart rate. In such compensatory tachycardias, stroke volume is limited, so “normalizing” the heart rate can be detrimental.[1]

 

We treat sinus tachycardia by treating the cause.

The cause of sinus tachycardia is never lack of cardioversion.
 

A good test near the end of the cardiology section of paramedic school might include this question to find out if the students have learned anything.

All of the choices are wrong.
 

In medicine, there is not one best answer for all patients.
 

Anyone who says differently is selling something.

Footnotes:

[1] Tachycardia
2010 American Heart Association Guidelines for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care Science
Part 8: Adult Advanced Cardiovascular Life Support
Part 8.3: Management of Symptomatic Bradycardia and Tachycardia
Cardioversion and Regular Narrow-Complex Tachycardia

.

What can I give you as a present for 2012? FOAM

 

 

This is a time when many of us are giving gifts to others.

This is not my gift, it comes from Life in the Fast Lane. I am only letting you know about it.
 

This is FREE.
 

F – Free
O – Open
A – Access
M – Meducation

Meducation is Medical Education. There was Guinness involved. The quantity of Guinness was not specified.
 

Then he got an idea. An awful idea. The Grinch got a wonderful, *awful* idea![1]

 

Is this awful?

No, but I am not fond of the term meducation. 😉
 

 

What is the most important part of medical education?

Medical education is expensive!

FOAM is –

Free.

No cost to us.

Open Access.

We do not have to subscribe to anything.

Medical education.

Better understanding of medicine.
 

We have access to free, up to date, medical education anywhere we have internet access, or we can download it for later use.
 

If you want to know how we practiced medicine 5 years ago, read a textbook.
If you want to know how we practiced medicine 2 years ago, read a journal.
If you want to know how we practice medicine now, go to a (good) conference.
If you want to know how we will practice medicine in the future, listen in the hallways and use FOAM.
— from International EM Education Efforts & E-Learning by Joe Lex 2012[2]

 

Dr. Joe Lex of Temple University Hospital has had the Free Emergency Medicine Talks web site for years. There are over 2,000 free mp3 downloads of presentations from many of the top medical conferences. These are usually without the slides, but some do include the slides.

I disagree with What Dr. Cadogan says about anonymity. My identity is no secret. If you don’y know who I am, you either don’t care or you don’t know how to use a search engine.

Would knowledge of my identity change anything about what I write?

No.

Would I need to pretend that bad ideas deserve more respect?

Not even a little bit. Bad ideas do not deserve respect.

We need to be able to determine whether information is valid without having to rely on the authority of the name attached to the information. If we need names, then we do not know enough for people to put their lives in our hands.[3]

Some people may need to maintain anonymity because of the archaic rules of their employers, so anonymity can be important, but anonymity is not bad.
 

Check out FOAM.

Check out GMEP.org

Check out Free Emergency Medicine Talks

We can learn from people we may never have heard of, speaking/writing in places we would never be able to travel to.

We can give ourselves the vital gifts of education and understanding.

Footnotes:

[1] Grinch (1966) quotes
IMDb.com
Quote page

[2] International EM Education Efforts & E-Learning
Dr. Joe Lex
Free Emergency Medicine Talks
Web page with link to mp3 download

[3] Blogging and Anonymity
Rogue Medic
Sun, 07 Oct 2012
Article

.

Do the wrong standards improve EMS



This week on EMS Office Hours, Jim Hoffman, Josh Knapp, Bob Sullivan, and I discuss what we need to do to move EMS forward as a profession.

National Curriculum, EMS Titles and Hurdles
 

Is this the way your medical director, chief, CEO, ALS coordinator treats you?
 


Image credit.

We do seem to agree that our standards are too low, but we do not agree on what we should do to raise our standards and what are our obstacles to better standards.

We spend a lot of time worrying about the amount of classroom hours to complete each part of a merit badge, rather than how we should assess competence.

We avoid discretion as if it were evil, but we are exercising discretion every time we make any decision.

Every time we drive, we decide when and how hard to press on the throttle, when and how hard to press on the brake pedal, when and how much to turn the wheel, when and for how long to indicate a turn. Rather than assess competence behind the wheel, many organizations just require an EVOC (Emergency Vehicle Operator Course) completion cared.

As with all of the other merit badges that organizations require as an alternative to assessing competence themselves, the quality of these courses varies tremendously. It is like calling medical command for orders – more depends on who answers the phone than on anything else, but we pretend that this is some objective protection for patients.

We tell ourselves what we want to hear.

How much of what is taught in a merit badge course is based on the course materials and how much is based on the instructors opinions?

How much of what is in the course materials is based on good evidence?

We have a bunch of people trying to keep the standards low. Those who think that every seat should be filled with a medics will not have an easy time filling all of those seats with people wearing paramedic medic badge patches if the standards are high.

If being a paramedic is a participation award, will the patients really want the proud owner of a participation prize to be caring for them, or will patients want someone who is being held to standards that matter? Will patients want a paramedic who is treating the serious patients, rather than driving half of the serious patients to the hospital?

If medical directors, chiefs, CEOs, ALS coordinators, and others oppose improvements in standards, we need to ridicule them.

Medical directors who keep standards low do not deserve respect.

Chiefs who keep standards low do not deserve respect.

CEOs who keep standards low do not deserve respect.

ALS coordinators who keep standards low do not deserve respect.

If we are concerned about our image, we need to stop cooperating with the clowns running the circus.

Just because someone has a title does not mean they deserve respect. Leaders need to demonstrate that they deserve respect.

Those who don’t deserve respect should not be defended by us.

Go listen to the podcast.

.

Comments on Do EMS Exams Really Gauge Provider Competency?

Just how bad is the idea of using a multiple choice exam to evaluate competence? The faulty attempts at logic used to justify the continuing use multiple choice exams as the final knowledge exam give us an idea.

CCC usually makes more sense, but in response to Do EMS Exams Really Gauge Provider Competency? he writes –

The NREMT is forced to evaluate applicants by testing, since they are a certifying agency.

The NREMT (National Registry of EMTs or National Registry) chooses to compete in the for profit testing market.

Part of the NREMT’s accreditation requires they ensure the competence of their applicants.

Do they actually claim to ensure the competence of applicants?

No. We only ensure that there is documentation of jumping through hoops that we pretend represent competence.

A multiple choice exam does not even come close to ensuring competence.

We come up with more complicated questions, in an effort to evaluate knowledge, but we are using the wrong method. I could chant very complicated magic spells, but magic doesn’t work. A simple magic spell doesn’t work. A complicated magic spell doesn’t work.

It doesn’t matter how ornate the test. Multiple choice is the wrong choice.

Until there is a better way to do that than a multiple choice test, the NREMT will continue to administer that test.

Why not use trial by combat or trial by ordeal to demonstrate competence?

Is there any reason to believe that a multiple choice test, which discriminates in favor of those who are good at avoiding the answers designed to be misleading, is any better at assessing competence?

The continuing reliance on the multiple choice format demonstrates that the National Registry does not understand assessment of ability. More weight is given to the appearance of objectivity, than to the result.

Without a test of some sort, there is no NREMT.

EMS does not exist for the NREMT. The National Registry exists for EMS. We are responsible for our own dissolution.

Consider the multiple choice test as the food we eat.

Humans need energy, so I am forced to eat.

I am forced to consume calories by eating, since humans are calorie-burning creatures.

Part of my metabolism requires that I ensure the adequacy of my food intake.

Until there is a better better food than Twinkies, I will continue to live on Twinkies.

This is a more obvious non sequitur than CCC’s, but the premise does not support the conclusion in his example, either.
 


Image credit.
 

I am being reasonable. I make some introductory nutrition-related comments, then I just claim that Twinkies are the best and I ignore the possibility that Twinkies are not the best, therefore Twinkies are the best.

Case closed. Twinkies Rule!

I think the claim that there is no better food than Twinkies is just as absurd as the claim that there is no better way to evaluate competence than a multiple choice test.

In what universe is a multiple choice test the best of all possible ways of evaluating competence?

The same universe in which Twinkies are the best of all possible foods.

.

Interfacility Transport vs. 911 EMS



This week on EMS Office Hours, Jim Hoffman, Josh Knapp, Joe Paczkowski, and I discuss whether 911 medics are better, interfacility medics are better, or if it is more complicated than that.

Interfacility Transport vs. 911 EMS

However, Josh starts out by returning to an earlier discussion of education evaluation standards and whether a multiple choice exam is the best method of evaluating knowledge.

Anyone who understands education assessment should realize that this is not any kind of reasonable suggestion. Josh does agree with this, to some extent, but then he states.

It may not be the best way, but we don’t have another way.

The only reason we don’t have another way is that so many of us have gone along with the status quo.

I will continue to point out how ridiculously inappropriate it is to use a multiple choice test to evaluate knowledge.

Being able to select the right answer (from a tiny group of choices worded to be misleading) is not the way to find out what people understand.

The way we find out what people know is by asking open-ended questions – and waiting for the answer, then asking for more information – and waiting for the answer, then asking for more information – and waiting for the answer, then asking for more information – ad infinitum.

This is not anything new.

Socrates, if he existed, was doing this thousands of years ago. This applies to medicine as much as to philosophy.

How do we learn about the ways a person will respond to a mistake he makes, if we give him a mistake we thought up, rather than waiting for him to make a mistake himself – then asking questions about how he deals with that?

This is what we deal with in real patient care – basic EMT, medic, nurse, doctor, . . . everyone who provides patient care. We are human. We make mistakes. We have to be able to deal with those mistakes.

In the real world, we do not have a multiple choice selection of responses.

We have to figure out what to do based on our understanding.

We can write complex multiple choice questions, but multiple choice tests can be passed by people who are good at taking tests. These tests discriminate in favor of those who are skilled at taking tests, which is not important in patient care settings.

Pick one answer and remember your answer. This is a test. The answer is at the end of the earlier post on this.

The topic under discussion for the rest of the show is one that will never be solved, but it depends on the patient. Some 911 patients require excellent patient care. Some interfacility patients require excellent care. They are different, but not necessarily in a way that one is better than the other.

Go listen to the podcast.

.

2 Paramedics’ Licenses Yanked For Refusing To Treat Patient

What a dramatic headline!

Yanked?

They had their licenses suspended, but apparently this suspension is just a way of warning them not to do this (depending on what this is) again.

suspended because they are accused of failing to treat a patient.[1]

That does not read the same as the headline.

The paramedics are also accused of failing to document ambulance runs, which the EMS director says is a regular pattern of substandard, incomplete and/or lack of documentation.[1]

But we are trained to do that.

I cannot count the number of times I have been told –

If you didn’t document it, it didn’t happen.

Obviously, these medics were listening. They did not document doing anything, so how can they be accused of doing something wrong? We taught them to document this way.

True. It was not what we intended, but is ignorance an acceptable excuse for an educator?

It isn’t my fault that I don’t know what I am talking about.

It is true that this is not what is meant by if you did not document it, it did not happen, but we need to stop using bumper sticker slogans. We need to make it clear what a patient is, what a patient contact is, how patient contacts need to be documented, and how non-patient contacts need to be documented.

What is a non-patient?

Someone who does not meet criteria to be a patient.

What is a patient?

“A patient is any person who, upon contact with an EMS system, presents with a complaint, circumstance, and/or condition that might require further assessment or treatment.”[2]

 

?
 

Who decides what that means?

“The standard of judgment is that of a reasonable and prudent medic.”[2]

Do reasonable and prudent medics exist?

What about the definition?

A patient is any person who, . . .

This could be absolutely anyone, but there are some limitations –

upon contact with an EMS system, . . .

demonstrates at least one of the following to EMS –

presents with a complaint,

The patient is complaining of something that appears to be medical, or –

presents with a circumstance,

The circumstances suggest that there is something medical going on, even if the patient is not complaining of anything, –

and/or presents with a condition

that might require further assessment or treatment.

Depending on the person doing the interpretation, this could describe anyone. 😳
 


 

There is contact with an EMS system.

That is a person (a poor drawing of a person, but meant to represent a person).

That is a circumstance and/or a condition that might require further assessment or treatment.

This could temporarily solve some EMS budget shortfalls. 🙄

Footnotes:

[1] 2 Paramedics’ Licenses Yanked For Refusing To Treat Patient
July 18, 2012 5:34 AM
Steve Miller
WBBM CBS Chicago
Article with link to short broadcast

[2] “What Is A Patient?”
EMS Medical Directors of Texas
Medical Directors Committee
Page with link to document in PDF format

.

90% of Everything in EMS is Best Described as Bad – or Worse

I didn’t do my homework. No, this is not a confession of misspent youth. There was a homework assignment from A Day in the Life of an Ambulance Driver that I intended to complete, but never did.

We EMS bloggers were supposed to point out the strengths and weaknesses of the following statement –

Nobody in EMS is paid what they’re worth. 25% are paid far less than what they’re worth, and 75% are paid far more than what they’re worth.[1]

Yes.

No.

Sometimes.

And, since the Bible tells us that everything is relative – Compared to what?

I could give many examples, but here are just a few –

I know some well paid and dangerous EMS personnel, but how do we define pay?
 


Image credit.

Money? Benefits? Which benefits? Authority? Professional courtesy (a free pass on many laws that affect our neighbors, but that we feel should not affect us)?

Should competent management be considered a benefit?

Wouldn’t competent management change the ratio for their organization?

If competent management improves the quality, or value, of the EMS providers at an organization, what is the effect of bad management?

What does this tell us about the 75/25 ratio, or the 90/10 ratio?

Management is not the only source of good EMS personnel, of bad EMS personnel, or of indifferent EMS personnel.

We are responsible for our own actions. We can be tempted to be better by good management. We can be tempted to be worse by bad management.

Who we are determines how we respond to the people who say that it is a waste of time trying to change things. For all of us, there are more things that are not worth our time, than there are things that are worth our time. We can use this to convince ourselves that insignificance is a worthy goal. Or we can try to change the things we think are wrong.

Then there are the people who say that it is OK to change things, but claim that there is only one way to change things. That is absurd.

That is similar to EMS educators claiming that there is just one right answer to a question. For example, a trick multiple-choice question. This points out the inability of the educators to come up with good questions.

It is unlikely that there will be one, and only one, right way to treat EMS patients, unless that way is very flexible.

One, and only one, right answer requires very limited information and almost unlimited ignorance.
 

One right answer = one bad examination.
 

Trying for one right way of doing things ignores one of the biggest flaws of EMS.

 

Many of our patients survive to the hospital in spite of us, not because of us.
 

Do we try to improve our understanding of what we do to our patients?

do we try to minimize the harm that we do to our patients and maximize the ways we help our patients?

Or do we continue to rely on our patients being too healthy for us to be able to kill them?

I don’t teach nearly so many EMS classes as I used to.[1]

This is one important way to change EMS for the better, but too much of EMS education is about maintaining the status quo, rather than teaching people to think. This is changing, but not quickly enough.

What is one important sign of the wrong attitude for EMS students?

Will this be on the test?

What is one important sign of the wrong attitude for EMS educators?

We have to teach this, because it will be on the test.

It isn’t about teaching people to be good at patient care, but about teaching people to be good at test-taking skills. The tests are strongly biased toward those with good test-taking skills, not toward those good at patient care.

How important are test-taking skills (the ability to recognize the intentionally misleading choices that are designed to distract from the answer the writers has determined is the one best answer) in patient care settings?

Not important at all. If we look at the information we gather about patients as if there are several misleading possibilities and one best answer, we are not dealing with the real world.

What won’t be on the test?

A patient who has an ambiguous presentation that will be best served by reassessment and benign neglect.

What is our least valued treatment?
 

Benign neglect.
 

Is there one way to answer the original question of being paid too much or too little?

Is an EMS union representative asking the question?

Is an employer asking the question?

Is Medicare asking the question?

Is a loan officer asking the question?

Is family asking the question?

Are we asking the question?

I almost never intubate, but I am insulted at the suggestion I should practice, because I am too good to practice on mannequins, and my pay is my over-inflated ego.

This is one classic example of the over-paid, less-than-competent medic, but there are many others.

Go read what Kelly wrote and the comments. Add your own comment.

Footnotes:

[1] Occupy EMS
Kelly Grayson
A Day in the Life of an Ambulance Driver
Article

.

Do EMS Exams Really Gauge Provider Competency?



Does an exam evaluate competence?

Absolutely not, but we spend this week’s podcast making excuses for using exams, especially the worst exams (multiple choice exams), to do create the illusion of evaluating competence.

Jim Hoffman, Josh Knapp, Bob Sullivan, and I discuss testing on EMS Office Hours.

Do EMS Exams Really Gauge Provider Competency?

Josh is attending the EMT-CC (EMT- Critical Care) course at UMBC (University of Maryland Baltimore County), so part of the discussion is why we need to have this as an add-on

The recent article by Kelly Grayson – Occupy EMS is discussed. That article will be my topic tomorrow.

We can come up with more complex multiple choice questions, but the problem is that a multiple choice test is the wrong way to evaluate understanding.

Multiple choice is about recognizing the answer that was written with the intention of being the correct answer. Other choices are written with the intent of misleading us so that we choose one of them, because of some similarity to the correct answer.

We cannot ask the person why he chose that answer, because one of the purposes of multiple choice questions is to prevent interaction between the person being tested and the examiner(s).

It is pointed out that not everything is gray, as if this is a justification for applying a black and white guess test to medicine, which is almost never black and white.

It is suggested that since doctors will take multiple choice tests as part of medical school, multiple choice tests are valid.

Doctors also complete years of residency after completing medical school. If the multiple choice exam were enough to evaluate competence, doctors would not need to continue to be supervised and evaluated during years of residency.

If a doctor fails to complete his residency successfully, can he claim that he has passed a multiple choice test and that the residency clearly is missing the competence that he demonstrated on the multiple choice test? 😳

What about nurses? They take a multiple guess test.

Nurses generally start in med/surg and only progress out as they demonstrate the interest in doing more, but nurses will be precepted in these more acute care settings.

It is also suggested that the hospital and ambulance time during paramedic school justifies using an irrelevant test, because this has already been evaluated.

If that is the case, we need to throw out the irrelevant test.

If the test is useless, no matter how much we think it is a part of a larger theme.

If the test does not do what it is supposed to do, it is useless.

Multiple choice requires that the one (as if there could be just one) correct answer be presented as one of three, or four, or five, or mix and match guesses. The correct answer is always among the guesses presented (even if the correct answer is none of the above).

Pick one answer and remember your answer. This is a test. The answer is at the end.

Is it improbable that anyone would be able to pass one of these multiple guess tests by guessing?

The evidence is abundant in the bad EMS providers that we see passing these tests.

Ask them some questions and they will demonstrate their lack of understanding of patient care.

Do this as they leave the test.

Do this a day later.

Do this a week later.

Do this a month later.

Do this a year later.

There will be a lot of people who passed the multiple guess test, but who are not able to demonstrate understanding of the medical topics that the multiple guess test claims to be evaluating.

We all know this, so why do we pretend otherwise?

We don’t know what else to do, because we believe in the test.

Does the test prevent dangerous people from becoming paramedics?

Of course not.

Does the test work?

Of course not.

The test is purely a ceremony of passage that should be viewed exclusively as the superstitious ritual that it is.

The answer to the multiple choice question above is at the end.

How many of you knew the answer?

How many of you were able to figure it out?

We can automatically rule out Eeeny, Meeny, and Miny, because they are not relevant to EMS.

Larry and Moe are critical to good patient care, but not the best choices.

Shemp came before Curly. Shemp came after Curly. A couple of good reasons for choosing Shemp, but –

It’s always Curly.

You can never go wrong with Curly.

There is no such thing as too much Curly.

This is EMS lore and will stand up to rigorous validation.

Ask a hundred EMS educators.

It’s Curly.

Ask anyone.

It’s Curly.

However, if the patient is not a Three Stooges fan, or if the patient is a Three Stooges fan, but one of those adherents of a deviant sect that does not acknowledge the greatness of Curly, then this information may not help the patient.

Validation of EMS exams has more to do with popular opinion, than with medical evidence. The lack of a well designed, randomized, placebo-controlled trial comparing Stooges is not a fatal flaw for this test question.

The odds of guessing the correct answer are one in seven. It would be impossible for a million monkeys taking this test to come up with the correct answers by chance, at least that makes sense if we do not understand coincidence and probability.

A 14% chance of coming up with the correct answer by chance, but I would be surprised if any group scored less than 90% correct on this question.

 

Image credit.
 

If the questions do not evaluate what they are supposed to evaluate, does the test matter?

Go listen to the podcast.

.