Science-Based Medicine, has a post by Dr. Val Jones – 2009’s Top 5 Threats To Science In Medicine.

I do not disagree with the list except, and what would one of my posts be without an except, the number one threat to science in medicine is much more of a problem. Our science education in grade school is where we fail our children. Before they even become adults, they are exposed to all sorts of magical thinking.

Full moons, speaking about something bad increasing the chances it will occur (a jinx), believing that something natural is safer than something manufactured – just because it is not man made, or just a belief in the stereotypical mad scientist bringing about horrors by using the scientific method of inquiry.

That is what science is. Science is a method of inquiry. Science is a tool we use to find out how the world works.

We see something that makes us think. Most people may form an opinion, but not look at this with a method designed to minimize the effect of our biases. And we all have biases.

It seems that there are more patients during a full moon. With a full moon falling at 19:15 GMT (Greenwich Mean Time) this past New Years Eve, this must have been a horrible night of death and destruction. Or was it?

We formulate a hypothesis.

The full moon causes accidents and/or illnesses, or makes accidents and/or illnesses even worse than they would be if there were no full moon.

We figure out what we need to control for to limit our variable to just the possible influence of the full moon. So, let’s look at a study that investigated the effect of a full moon on something that would be very difficult to misinterpret.
 

We postulated that on full moon days there would be more available moonlight, thus influencing individuals’ activities, and in turn, the propensity for cardiac arrest.^[1]

Interesting. They are not really assuming that the cause of an increase in cardiac arrests would be due to some mystical property of the moon, but that it would be due to more moonlight. fortunately, it does not matter what the actual cause would be for an increase in cardiac arrests, if they set the experiment up properly.

What do they need to do?
 

This study was a retrospective analysis of a computerized billing database of ED visits.^[1]

The study population consisted of CPR (CardioPulmonary Resuscitation) occurring daily at a cohort of seven hospital ED in northern New Jersey, USA, during the period of 1 January 1988 to 31 December 1998, comprising 4018 days over 11 years. Consecutive patients seen by an emergency physician were included. Emergency physicians see 80–95% of all ED patient visits and the vast majority of cardiac arrest patients. Private physicians see the remainder of the patients.^[1]

Their theory was that the increased moonlight would lead to more activity; more activity would lead to more cardiac arrests; thus there would be more cardiac arrests during a full moon.

Did the investigators prove their hypothesis?

According to their table, which does not reproduce well, there is no increase in cardiac arrest incidence during a full moon. They actually recorded a decrease, but the difference is not statistically significant. The reported statistically significant difference in incidence of cardiac arrest is this. During the new moon, there is less likely to be a cardiac arrest treated by an emergency physician.
 

There were 2370233 patient visits in the database during the 4018-day (11year) period of study, with 6827 having the primary ICD-9 diagnosis of cardiac arrest.^[2] Table 2 contains the time series regression results. Full moon days were not significantly different from other days (P=0.97). We had an 80% power to identify a difference of 4.5%. However, on average 0.12 fewer CPR occurred on new moon days than on other days (P=0.02). This translates into an average of 6.5% fewer CPR (95% confidence interval 1.3–11.7%) on new moon days than other days. In addition, the results for the potentially confounding variables are presented in Table 2.^[1]

I left part of their original hypothesis off of the initial quote. The stated objective of the study is –
 

Objective
To determine the effect of the phase of the full and new moon on the variation in the number of daily cardiopulmonary resuscitations.^[1]

In the discussion, they elaborate on their purpose –
 

Our results show a small but statistically significant decrease in the incidence of CPR with new moon days. We speculate that this may be secondary to a decrease in activity because of less available light on these days, as it has been shown that increased activity is a risk factor for sudden death[39]. Our initial rationale sought to identify and determine the size of any effect on the occurrence of cardiac arrest and its attempted resuscitation (‘CPR’) by lunar influence as a potential insight into an aspect of the occurrence of cardiac arrest. In addition, we sought to identify patient volume variation by lunar cycle potentially to allow for staffing modifications; however, the effect identified did not warrant this.^[1]

This –
 

We postulated that on full moon days there would be more available moonlight, thus influencing individuals’ activities, and in turn, the propensity for cardiac arrest.^[1]

Becomes –
 

We speculate that this may be secondary to a decrease in activity because of less available light on these days, as it has been shown that increased activity is a risk factor for sudden death[39] ^[1]

They have found a way to stick with their initial hypothesis by reversing it.

More moonlight does not appear to lead to more cardiac arrests. Why this lack of correlation does not need to be explained is not in the paper. However, the correlation between fewer deaths during a new moon is something that they feel needs to be explained. Haven’t they just misappropriated a Willy Wonka quote? Strike that. Reverse it. Willy Wonka was reversing the meaning of what he was saying. That was the reason he needed to reverse the order.

The authors have not really changed the meaning, only the way they express it. More light/less light leads to more activity/less activity. This leads to more/fewer cardiac arrests.

While I do not dispute the results of this study, I do have a problem with the way they get from Point A to Point C. They seem to travel there by way of a study that shows that more activity leads to more cardiac arrest. Actually the study is of vigorous exertion, not just more activity, but the authors seem to have interpreted the study as couch potatoes live longer. The vigorous exertion study did show –
 

As expected, the base-line level of habitual exercise significantly attenuated the increase in the risk of sudden death that was associated with an episode of vigorous exertion in both the primary analysis and the three sensitivity analyses. Habitually active men had a much lower risk of sudden death in association with an episode of vigorous exertion than men who exercised less than once a week; however, the most active men’s risk remained significantly elevated during and after vigorous exertion in all analyses.^[3]

There is no suggestion that these episodes of vigorous exertion occurred less frequently during the time of the new moon. According to the hypothesis of the full new moon study, the effect of the new moon should only be at night, when it would make a difference in the amount of available light. This might make more difference in rural areas, than in the suburbs, and more of a difference in suburbs, than in cities, due to the wonders of electrical lighting.

Is there any evidence to support this string of conclusions? I don’t think so.

That does not mean that this hypothesis is incorrect, just that their way of getting there is not supported by the information provided.

One very nice part of this study is the brief review of previous studies and whether they seemed to support, or refute, a connection between various activities and the full moon, but this post is already too long. I will write about other full moon research elsewhere.

Another problem is the way they define a full moon –
 

We identified full and new moon days that occurred during the study period from the United States National Oceanographic and Aeronautic Administration website. Using this information we created variables for full and new moon days to be used in a regression model of daily CPR, described below.^[1]

So. What is their definition of a new moon, or a full moon? They have not made that clear, but it appears to be limited to one specific day during each lunar cycle.

Using NOAA’s (United States National Oceanographic and Aeronautic Administration’s) website, I found a page that identifies new moons, full moons, and other phases by entering the time period I want to look at. The problem I see is that there is not much detail about how they used this information, or if they were using the same part of NOAA’s website.

Why no discussion of this?

Why no discussion of their definition of new moon and full moon?

If their hypothesis is one that depends on the available light, why does the day before a new moon not count, or the day after, or two days before (or after)?

However, if they are looking for a mystical connection between the full moon and bad events, a case can be made that the full power of the full moon would be on one specific date.

If they are claiming that they are examining the effects of the amount of moonlight, how much difference is there in the amount of moonlight from one day to the next? Where is your cut-off? Why?

I think that their conclusion should be that they find no apparent mystical connection between the full moon and cardiac arrests treated by emergency physicians. Their attempts at explaining their results wander into narrative fallacy, which I have written about here, here, here, here, here, here, here, here, here, and here.

As I have stated before about the way we should look at explanations for scientific results –

Bet that the explanation is wrong.

This is one of the failures of our basic education of students. We do not make this clear to them. Yes, this was where the post started, with the failure of science education in the grade schools.

The media report that a certain study means X, even though the authors of the study may not have suggested that this is true. It is later found that the conclusion popularized by the media is wrong. The blame goes, not to the reporters misrepresenting the science, but to the science. We need to avoid creating explanations that are unsupportable and likely to be found to be in error. We need to stop telling fairy tales. We need to stop talking to media members, who spin research results with misleading explanations.

We have people graduating from high school, but unable to recognize the difference between good science and bad science. Unable to look at a study and determine if there is something there that is meaningful. This continues through college, and even medical school. Number 3 on the list was – Academic Medical Centers, so I am not the only one critical of these ivory towers. The top threat to science in medicine is the lack of understanding of what science is. We fail before the students ever get started. This lack of understanding is due to a lack of education in grade school.

We need to change how we teach science. Some do it well, but the debates on scientific topics in the media suggest that few have been well taught. The ignorant mobs are trying to keep themselves in scientific debates for which they are not even remotely qualified. We need to make it so that more are qualified.

We need to improve our basic science education, because we cannot rely on people from other countries coming here to do the science that we have become too ignorant to handle. Eventually, the destination of smart foreign-born scientists will not be the United States. We do not develop enough of our talent. We remain a scientific power because import talent. We do the same thing with grape pickers and day laborers. These seem to be the jobs that we will no longer do for ourselves. Scientist and day laborer are both looked at as undesirable. We need to change this.

It is in the earlier grades that the problems of bad science education are established. After that point it is a much more difficult task to correct this creeping indoctrination in magical thinking.

I think that this is the number one threat to science in medicine. All of the others only contribute to this.

Magical thinking is a form of corruption. As with other types of corruption, it usually does not start with a cannonball into the deep end, but a gradual acclimatization to more and more corruption. At some point, critical judgment is only a fond memory.

There must have been a time, in the beginning, when we could have said – no. But somehow we missed it. – Tom Stoppard.

We can still fix that problem in education for those in grade school.

–

Footnotes:

–

^[1] Effect of lunar cycle on temporal variation in cardiopulmonary arrest in seven emergency departments during 11 years.
Alves DW, Allegra JR, Cochrane DG, Cable G.
Eur J Emerg Med. 2003 Sep;10(3):225-8.
PMID: 12972900 [PubMed – indexed for MEDLINE]

–

^[2] ICD-9 code definition from the study above

The physicians’ billing department assigns codes according to the International Classification of Diseases, Ninth Revision, and Clinical Modification [International Classification of Disease (ICD)-9 codes]. Patients were included as CPR if they contained any of the ICD-9 codes listed in Table 1 as one of their three primary diagnoses.

Table 1
International Classification of Disease 9 codes
427.4       Ventricular fibrillation
427.41     Ventricular fibrillation
427.5       Cardiac arrest
798.1        Death instantaneous
798.2      Death occurring less than 24h from onset of symptoms
798.9       Death unattended

–

^[3] Triggering of sudden death from cardiac causes by vigorous exertion.
Albert CM, Mittleman MA, Chae CU, Lee IM, Hennekens CH, Manson JE.
N Engl J Med. 2000 Nov 9;343(19):1355-61.
PMID: 11070099 [PubMed – indexed for MEDLINE]

Free Full Text from NEJM.

–

Alves, D., Allegra, J., Cochrane, D., & Cable, G. (2003). Effect of lunar cycle on temporal variation in cardiopulmonary arrest in seven emergency departments during 11 years European Journal of Emergency Medicine, 10 (3), 225-228 DOI: 10.1097/00063110-200309000-00013

.