I’m a little late to the party, since my blog-perusing time has been stolen from me of late. I will not begin heroics on our equine friend with lividity, I just have one comment to your statement,

That is the purpose of the 95% Confidence Interval. Unless it is calculated incorrectly, due to a misunderstanding of what is being measured, or a misunderstanding of what variables are relevant, the research should only have a one-in-twenty chance of misrepresenting theconclusion.

I assume you were just typing at 100 miles/hr but it is a point worth clearing up.

Well, typing faster than I think, perhaps 100 millimeters per hour (one of the reasons *I* don’t post more often), but I did ponder over the use of the word *conclusion*. I could not come up with a more appropriate word at the time and I never returned to it, although I should have.

Confidence Intervals are not any type of guarantee of conclusions at all! As I am sure you are aware, CI represents the probability that the data are somehow correlated to a phenomena and not just the result of “statistical randomness”, inasmuch as the selected group will represent (statistically) the larger population.

You are correct. I was trying to qualify that by excluding the things that would lead to incorrectly drawn conclusions. I was giving a bit too much importance to the conclusion. I was trying to make a point about one thing that is not well understood. I was trying to simplify things. I did a poor job.

BY NO MEANS does a CI of 95% (or 99% for that matter) guarantee anything about VALID CONCLUSIONS.*

Unfortunately, many studies with >95% CI’s have 100% chance of misrepresenting conclusions!

This is true. This is one area of research that provides me with so much to write about. When researchers are measuring the wrong thing, or trying to measure the right thing, but not effectively controlling for variables, the results probably will have no more than a random chance of producing information about anything, except *How not to do research*.

Several factors play a role in the ability to made a valid conclusion from any collection of data. Not least of these are the actual study design, inclusion(exclusion) criteria, experimental methodology,..etc etc ….but just to clarify- Confidence Interval is merely a testament to the chances that the data are related to an observed pattern and not just random- and speaks to how the measured group potentially scales up.

Yes. I tried to work that into my description of conclusions, but I only ended up complicating things. The researchers may claim that the data gives a 95% CI, but they may not be measuring the right things, so, their claim of a 95% CI may be the weakest point of a study. If the researchers do not adequately control for **all** relevant variables, it may not matter how much data they have, they are not any more likely to be measuring relevant data than would some conspiracy theorist.

To quote one of my favorite curmudgeons,

“There are Lies, Damn Lies, and Statistics.”

Was that Jenny Killer McCarthy?

The important thing to point out about the 95% CI is that it does mean that, if the research is properly designed, there is **less than a 5% chance that the data are due to chance**.

This does **not** mean that 5% of research will come to the opposite conclusion. This does not even mean that 5% of the research will have data that differs from what accurately represents the observable data. It means that, if the variables are understood, the research is properly designed, the study is carried out without significant deviations from the study protocol, . . . , then the data have less than a 5% chance of coming out as they do *purely by chance*.

The differences in the size of studies mean that a study can have greater than 95% CI, but be several times smaller than another greater than 95% CI. The idea that 1 in 20 studies will come to the opposite conclusion, is a misunderstanding of the meaning of the Confidence Interval.

* [As an aside, if it is the last thing I am going to do I am getting a “Correlation does not equal Causation” tattoo]

You are old enough to make these decisions on your own. I would recommend using this at the appropriate time. Maybe as a bet. And there are several science tattoo blogs that might be interested in having you show a bit of skin. Don’t be offended if they are only interested in the skin with the tattoo.

.

## Subscribe to RogueMedic.com