Without evidence of benefit, an intervention should not be presumed to be beneficial or safe.

- Rogue Medic

Methodology of tPA Studies – Comment from Joshua


In response to my criticism of the ACEP (American College of Emergency Physicians) polict statement on Genentech’s r-tPA (recombinant tissue Plasminogen Activator) – alteplase (Activase®),[1] Joshua responded with these comments on what I cited/quoted. I think this is a first for me. –

First of all, IST-3 is a very large trial, involving more than 3000 subjects. Nortin Hadler, in his book Rethinking Aging, wisely suggests that whenever a very large trial is required to show statistical benefit, it means that the purported benefit cannot be clinically important.19 [2]

I disagree. When we are unable isolate a single variable (in this case administration of tPA) a large scale study may be the only way to identify outcomes attributable to the intervention (good or bad). The further from treatment to identifiable outcome and the more variables that can’t be (or simply are not) accounted for the larger the sample size should be.


There are several reasons I did not use that quote. I would have used an earlier book[3] as the source of that idea.

You do raise an important point. If few variables have been controlled for, are we measuring what we think we are measuring?

You appear to be making one of the same arguments that Dr. Lyden made. That enough data makes the flawed methods irrelevant.

I do not agree with that.

Dr. Lyden wrote –

The study leadership invoke a concept known as “large simple” trials, meaning trials that seek power by simplifying protocols and reducing data collection to a bare minimum in hopes of enhanced recruitment. Large simple trials may be an answer to the continuing difficulty getting clinical trials finished on budget and in time.[4]


And –

When reorganized, the trial became an unblinded trial with a design that sounds rigorous: “prospective, randomized, open, blinded end point” (PROBE). The investigators have written that their end point is blinded because a centralized researcher, unaware of the patient’s treatment group, contacts the patient or family member 6 months after stroke and attempts to derive a modified Rankin Scale from a questionnaire or telephone call.[4]


While this does make things more simple, it does not appear to be capable of producing blinded data, but it may blind some people to the fatal flaws of the study methods.

If we have only one variable that is the same among thousands of patients, then we might draw some conclusions, but only tentatively. There could easily be other variables which are not universally present and are not being controlled for, yet may have much greater influence on the outcomes. We would not know what is being measured.


There’s a lot more that goes into the treatment of stroke patient than just whether or not they received fibrinolytic therapy. How many patients received rehabilitative care? How many were treated for hypertension and if so with what? How was patient weight calculated with an administration dose that’s weight dependent? If you only have 100 patients 1 or 2 nurses screwing up their drug calculations (of course that never happens and it ALWAYS gets reported ;-)) is a difference of 1-2% when evaluating final outcomes. What about 10 patients receiving anti hypertensive therapy in a sample size of 100? If 8 out of those ten patients have a negative outcome what’s to say that the combination of therapies and not TPA itself is responsible for this 8% increase in patients with increased neurological deficit at the 90 day mark? The larger the sample size the more you can negate the weight of these variables. While it’s not unreasonable to assume that 1 or 2 nurses may inadvertently administer an incorrect dose assuming that 100-200 nurses in a sample size of 1000 would do the same would suggest that tPA is the last thing we should be worried about when evaluating our patient’s outcomes.


The variability in competence and variability in methods are some of the problems that become more difficult to address in larger studies, but they may reflect the diversity in treatment that exists across the country, or around the world.

The substantial increased rate of symptomatic intracerebral hemorrhage among tPA-treated patients has tempered enthusiasm for the rapid adoption of tPA as routine care, in part because of the concern that treatment may be less safe in routine clinical practice than in the highly monitored setting of a clinical trial.[5]


This raises a serious problem.

In theory, there is no difference between theory and practice. But in practice, there is.
― Yogi Berra

If tPA works tolerably only during studies, but does a lot damage in use outside of controlled studies, is tPA too dangerous to use outside of controlled trials?


Three out of eight guideline writers directly involved with the pharmaceutical manufacturer. As far as indirect support, however, if they wanted to be more transparent, Dr. Edlow, Dr. Jagoda, Dr. Stead, Dr. Wears, and Dr. Decker also ought to have disclosed their association with the Foundation for Education and Research in Neurologic Emergencies – supported by multitudinous pharmaceutical manufacturers, including Genentech.

If you’re irritated that pharmaceutical manufacturers are helping write our clinical guidelines, make your voice heard.[3] [6]

Why are we suprised that pharmacuetical manufacturers are helping write our clinical guidelines?
(http://www.manhattan-institute.org/html/fda_05.htm)(http://www.sciencebasedmedicine.org/index.php/what-does-a-new-drug-cost/ <– read both parts)

It takes a lot of money to conduct phase 2 and 3 drug trials, though as the second article points out the actual cost is probably significantly skewed intentionally. But even at a conservate estimate of $43.4 million that's a lot of money to be gambling. So it's no wonder that they want the best chance at a possible outcome and hire people to make it happen.


Genentech should not have any control over who writes policy statements.

It is not a surprise that a drug company would attempt to influence policy statements, but it should not be so easy for any drug company to succeed.


More so than full financial disclosure (because given the cost there's only one entity that can afford it so it's usually safe to assume who's paying to make it happen) I'd like to see a better explanation of how patients are treated in their entirety. Give me ALL the data and not just what you want me to see or what magical mathematical formula you think justifies what you've written in your abstract and let me be the one to interpret it. It's kind of like EPI in cardiac arrest. We may find that the drug is not beneficial for everyone but should instead be reserved for a particular subset of patients.


The quality of the research that is published does leave a lot to be desired. How much would it cost to add an on-line data file supplement, or several file supplements, of the raw data to the paper?

Not much.

How much do some of the researchers want to hide this information?

Probably very much.

Along with requiring registering the study with ClinicalTrials.gov, this is something that should be simple to achieve high compliance on. The reality is only about half of trials were registered prior to enrolling patients.[7] Improvements in research quality take time, even though they seem as if they should be no-brainers.

One of the big problems with NINDS was the refusal of the NINDS investigators to release the raw data.[8],[9]

The biggest problem may be the expectation that we will successfully develop drugs in laboratories, when this is not the way we have historically come up with useful drugs.

We would like a drug better than warfarin (Coumadin), but the designer drugs do not seem to be better. The designer anti-clotting drugs only seem to improve surrogate endpoints over warfarin, but we spend a lot on them and that may increase dramatically.

A lot of patients studied, but no important difference in outcomes. Sometimes, when a very large trial is required to show statistical benefit, it means that the purported benefit cannot be is not clinically important.

Image credit.

Where is Waldo?

Is Waldo’s location clinically significant?

Click on the image to make the image larger, but making it larger won’t make Waldo any more clinically significant. 😉

There he is.

Image credit.

“Spinal immobilization” is one area where the results will require huge numbers, the number of affected patients will be tiny, but the importance of the outcomes will be very important clinically.

The sad thing is that so many people do not want to know how much harm we may be causing or if we could improve care.


[1] Can we trust drug companies to provide accurate information about their products? Part II
Mon, 28 Jan 2013
Rogue Medic

[2] How is more negative evidence being used to support claims of benefit: The curious case of the third international stroke trial (IST-3).
Hoffman JR, Cooper RJ.
Emerg Med Australas. 2012 Oct;24(5):473-6. doi: 10.1111/j.1742-6723.2012.01604.x. No abstract available.
PMID: 23039286 [PubMed – in process]

[3] Fiction And Fantasy In Medical Research: The large scale randomised trial
By James Penston
Published by London Press, 2003
ISBN 0954463617, 9780954463618
144 pages
GoogleBooks link

[4] In anticipation of International Stroke Trial-3 (IST-3).
Lyden PD.
Stroke. 2012 Jun;43(6):1691-4. Epub 2012 May 3. No abstract available.
PMID: 22556196 [PubMed – indexed for MEDLINE]

[5] Clinical Policy: Use of Intravenous tPA for the Management of Acute Ischemic Stroke in the Emergency Department.
This clinical policy is the result of a collaborative project of the American College of Emergency Physicians and the American Academy of Neurology.
Ann Emerg Med. 2013 Feb;61(2):225-43. doi: 10.1016/j.annemergmed.2012.11.005. No abstract available.
PMID: 23331647 [PubMed – in process]

Free Full Text Download in PDF format from Elsevier.

[6] New ACEP tPA Clinical Policy
Wednesday, January 23, 2013
EM Literature of Note

[7] Characteristics of clinical trials registered in ClinicalTrials.gov, 2007-2010.
Califf RM, Zarin DA, Kramer JM, Sherman RE, Aberle LH, Tasneem A.
JAMA. 2012 May 2;307(17):1838-47. doi: 10.1001/jama.2012.3424.
PMID: 22550198 [PubMed – indexed for MEDLINE]

Free Full Text from JAMA.

The proportion of trials registered before beginning participant enrollment increased over the 2 time periods: from 33% (9041/27 667) in October 2004–September 2007 to 48% (19 347/40 333) in October 2007–September 2010.

[8] Tissue plasminogen activator for acute ischemic stroke. The National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group.
[No authors listed]
N Engl J Med. 1995 Dec 14;333(24):1581-7.
PMID: 7477192 [PubMed – indexed for MEDLINE]

Free Full Text from New England Journal of Medicine.

[9] The raw data of the NINDS trial should be made public
Dr. Jeffrey Mann
Rapid response letter

Unfortunately, it has been years since Dr. Mann discontinued his EM Guidemaps site, where he posted the raw data that the NINDS investigators finally sent him in 2003, and I no longer have a copy of what he posted.



  1. […] as well as new recommendations by the Third US Preventive Services Task Force are also included.Clinical Guidelines in Family Practice This up-to-date medical text (4th Edition) contains the lates…ical and useful book for the primary care clinician. The book also contains a focus on development […]