Without evidence of benefit, an intervention should not be presumed to be beneficial or safe.

- Rogue Medic

More Examples of Errors Due to Confirmation Bias

The government is often a great source of logical fallacies, since politicians pander to the biases of the electorate. Here is one example.

“The increases in youth drug use reflected in the Monitoring the Future Study are disappointing,” said Gil Kerlikowske, director of the White House Office of National Drug Control Policy. “Mixed messages about drug legalization, particularly marijuana, may be to blame. Such messages certainly don’t help parents who are trying to prevent kids from using drugs.[1]

If mixed messages (as opposed to chanting the party line?) are the explanation, or a large part of the explanation, then marijuana use should be increasing faster than the use of other drugs measured.

The MTF survey also showed a significant increase in the reported use of MDMA, or Ecstasy, with 2.4 percent of eighth-graders citing past-year use, compared to 1.3 percent in 2009. Similarly, past-year MDMA use among 10th-graders increased from 3.7 percent to 4.7 percent in 2010.[1]

What were the changes in the rates of marijuana use?

Most measures of marijuana use increased among eighth-graders, and daily marijuana use increased significantly among all three grades. The 2010 use rates were 6.1 percent of high school seniors, 3.3 percent of 10th -graders, and 1.2 percent of eighth-graders compared to 2009 rates of 5.2 percent, 2.8 percent, and 1.0 percent, respectively.[1]

Notice the data are comparing daily use with past year use. This is not a useful way to look at data. Probably the most relevant number to look at, if we are interested in the effect of anything, is past 30 day use. The daily user is not likely to alter his behavior because of much and the same is true for someone who only rarely uses a drug. Those using more frequently are the ones we are most interested in discouraging from increasing their use to daily use.

If we look at these data, what can we conclude?


Green = Marijuana use 2009
Purple = Marijuana use 2010
Orange = Ecstasy use 2009
Maroon = Ecstasy use 2010

The year to year increase is small. The year to year increase of marijuana is not much more than the year to year increase of ecstasy. Are there Mixed messages about drug legalization affecting ecstasy that might explain this?

Not that I am aware of, but if there are, please let me know.

What explains the increase in ecstasy use if the increase in marijuana use is significantly due to Mixed messages about drug legalization? If the effect of Mixed messages about drug legalization are insignificant, why mention this?

Because it confirms the bias of the people selling this idea. The War on Drugs is tens of billions of dollars of our money each year. That is a strong incentive to keep us hooked on our addiction to the War on Drugs.

As long as we are keeping kids from getting hooked on drugs, getting them hooked on lies may not be a bad trade off.

Did you notice that the quote only mentioned use of ecstasy in the 8th grade and 10th grade? Why not mention the 12th grade?

Because the use of ecstasy in the 12th grade decreased and not by just a tiny percentage of use.

Maybe if we look at the rate of change, things will show that there has been a greater change with marijuana this year than there was last year. This could be evidence of Mixed messages about drug legalization.


Green = Marijuana increase/decrease in use from 2008 to 2009
Purple = Marijuana increase/decrease in use from 2009 to 2010
Orange = Ecstasy increase/decrease in use from 2008 to 2009
Maroon = Ecstasy increase/decrease in use from 2009 to 2010

Let’s look at the percentage changes this year (changes from the 2009 numbers to the 2010 numbers) and compare them with the percentage changes last year (changes from the 2008 numbers to the 2009 numbers).

For 8th grade, the percentage change this year was close to twice as much as the percentage change the previous year. That is significant.

On the other hand, for 10th grade, the percentage change this year was only one third as much as the percentage change the previous year. That is probably also significant, but in the opposite direction.

For the 12th grade, the percentage change this year was only five eights as much as the percentage change the previous year. That is probably not significant, and in the opposite direction, but the reality is that these numbers are not demonstrating any significant trend. The numbers go up. The numbers go down. They go in different directions in the same time period. This is not a pattern. This is confirmation bias, which is just a way of fooling ourselves and fooling others with nonsense.

The ecstasy numbers were also supposed to have increased significantly. Let’s look at the change in use during the same time periods. Do they show the significant increase that was claimed?

For 8th grade, the percentage change this year was a huge increase. The percentage change the previous year was a decrease. That must be significant.

For 10th grade, the percentage change this year was almost three times as much as the percentage change the previous year. That also must be significant.

For 12th grade, the percentage change this year was negative by over 20% from the previous year. The percentage change the previous year was zero. If that is significant, then what about the other numbers that are significant in the opposite direction?

We are finding patterns that support our biases. We do not appear to be finding data that has any predictive potential.

If anything, this year, there appears to be an unusually large increase in the use of marijuana and ecstasy among 8th grade students; A split on the use among 108th grade students, with the rate of ecstasy use increase rising by 3 times as much, while the rate of marijuana use increase rises by only 1/3 as much; And 128th grade students have a smaller increase in the use of marijuana, but no increase in the use of ecstasy – a large drop in the use of ecstasy.

How should we interpret any of this? If we were to accept the notion that the potential legalization of marijuana is a major influence on the data, and there is no good reason for someone not on the receiving end of money from the War on Drugs to accept that –

Then 8th grade students are much more influenced by the political considerations than 10th grade students.

10th grade students are much more influenced by the political considerations than 12th grade students.

12th grade students – the only ones who might be old enough to vote – are not at all influenced by the political considerations of the potential legalization of marijuana.

Just how high were the people who came up with this interpretation?

Red is 12th grade. Blue is 10th grade. Green is 8th grade.

The perception of regular marijuana use as a great risk has been dropping since about 1990. It is foolish to suggest that efforts to legalize marijuana have produced some sudden change.

The perception of marijuana being easy to obtain rose from about 1990 to 1996, then has dropped down to where it had been. No significant change with efforts to legalize marijuana.

Since the 12th grade students have overwhelmingly considered marijuana easy to obtain for as long as this data has been tracked, what does that say about the effectiveness of those tens of billions of dollars spent every year?

Ineffective. Maybe even counterproductive.

Footnotes:

[1] Teen marijuana use increases, especially among eighth-graders
NIDA’s Monitoring the Future Survey shows increases in Ecstasy use and continued high levels of prescription drug abuse
NIDA
Tuesday, December 14, 2010
NIH News Release

[2] Monitoring The Future
Drug and Alcohol Press Release: Text, Figures, & Tables
Web page with link to PDFs of the various data provided

Free PDF of the data on past 30 day use of all drugs from monitoringthefuture.org

.

Trackbacks

  1. […] This post was mentioned on Twitter by Chronicles of EMS. Chronicles of EMS said: More Examples of Errors Due to Confirmation Bias http://bit.ly/hMF2qr Via @EMSblogs #CoEMS […]

  2. […] admin wrote an interesting post today Here’s a quick excerpt The year to year increase is small. The year to year increase of marijuana is not much more than the year to year increase of ecstasy. Are there Mixed messages about drug legalization affecting ecstasy that might explain this? […]

  3. […] More Examples of Errors Due to Confirmation Bias | Rogue Medic […]

  4. […] to the chart I posted in More Examples of Errors Due to Confirmation Bias, almost all 12th grade students consider it fairly easy or very easy to obtain […]