Normally, comparing a drug's efficacy data to a historical control would be something investors should take with a grain of salt.
But it might be just what Sarepta Therapeutics (SRPT +1.08%) needs to get its Duchenne muscular dystrophy drug eteplirsen across the Food and Drug Administration's finish line.
Sarepta has been running a phase 2 trial testing eteplirsen for more than three years. The trial wasn't really set up as a pivotal trial to gain FDA approval. The design was more of a typical proof-of-concept trial used to justify going into a larger phase 3 trial.
Fortunately -- or unfortunately, depending on how you look at it -- eteplirsen worked much better than anyone could have expected. Twelve Duchenne muscular dystrophy patients were split into two groups: One group got the drug immediately and the other group served as the control group for the first 25 weeks, but then received the drug.
The initial results showed a difference between the two treatment groups. But the protocol called for both groups to continued receiving drug. Sarepta kept measuring the patients' ability to walk, and there continued to be a difference between the initial treatment group and those who had a delayed treatment. However, at some point, that comparison becomes somewhat meaningless because both groups are receiving the drug.
What you'd really like to see is the treated patients compared with patients receiving placebo. But Sarepta doesn't have that data. And realistically, given the unmet need, it would have been hard for Sarepta to run a trial designed that way even if it had realized that eteplirsen was going to work as well as it did. Imagine asking parents to have their child treated with placebo for three years while they watched their child lose the ability to walk in a clinical trial for a drug that appears to help the treatment group.
The next best thing
Sarepta, apparently at the FDA's request, made the only comparison it could: all of the treated patients versus a matched group of patients.
Source: Sarepta Therapeutics.
On the surface, the comparison looks pretty good, with the treatment group walking substantially further than the historical untreated group.
But the comparison is far from perfect. It isn't exactly comparing apples to oranges, but if a double-blind placebo-controlled prospective study is a freshly picked apple, a retrospective historical comparison is a bruised apple fit for the pigs.
There are issues that are a problem for every retrospective historical comparison. First, historical controls rely on historical standard of care that tends to improve over time. It's easy to argue that isn't as big of an issue for Duchenne muscular dystrophy as it is for something like cancer treatments, but it doesn't make the issue go away.
The other obvious issue is that a historical comparison is inherently biased. In a prospective study, none of the people analyzing the data know which patients got which treatment. In a retrospective study, that obviously isn't possible.
And then, specifically for this comparison, there are only 13 patients in the historical controls. Obviously the company can only compare its data to matched patients that have the same characteristics -- starting walking distance and age, for instance -- as the patients in the treatment group. But I'd feel more comfortable if there were two or three matched historical patients compared to each patient. That would shrink the error bars -- shown as the white lines above and below the curves -- giving a little more confidence that the decline of the historical control is what should have been expected.
Enough is enough
While there are obvious issues with historical comparisons, at some point the data has to be convincing enough on its own as I asked nearly two years ago.
If $SRPT refused to run a P3 trial and just kept reporting P2 data, how long until the FDA would accept it?
— Brian Orelli (@BiologyFool) January 15, 2014
Since my tweet, Sarepta has started a phase 3 trial, but based on the phase 2 data -- now three years old -- it's hard to believe that the minuscule decline the patients have experience isn't due to the drug. Anyone with experience with Duchenne muscular dystrophy should be able to recognize the three-year treatment data isn't what you'd normally expect.
Even with all its flaws, the new historical comparison shouldn't take away from the generally positive data and should be enough to get the FDA to approve the drug.
