Vertex Pharmaceuticals' (VRTX 1.72%) cystic fibrosis drug combination might work, but investors should realize this data is far from clean.
First let's look at what very clean data looks like. This is Vertex's Kalydeco in the subset of patients that have a mutation that makes them responsive to the drug. The graph reports increase in FEV1, a measure of lung function. You can see FEV1 immediately jumps up in patients taking Kalydeco. And at 48 weeks, when the patients that got placebo entered the extension study, their FEV1 immediately jumped up as well.
You can stare at your monitor from the other side of the room and see that there's an effect. It's the kind of data that brings a tear of joy to scientists' eyes. Data doesn't get much cleaner than this.
Crying in a different way
Here's the data presented last week for Vertex's combination of Kalydeco, labeled as its generic name ivacaftor, and VX-809 at different concentrations.
If you just compare the 600mg dose (blue line) to the placebo (black line), the data looks fairly good. But there are some issues, here.
First, the two lower doses don't seem to be showing a dose response; the 200mg dose (tan line) performs better than the 400mg (green line). There are explanations for why you wouldn't see a dose response, including that you need to get over a certain threshold to respond, but even if you buy that argument, the data shows the high variation in FEV1 measurements.
The results could have been affected by the use of antibiotics, such as Gilead Sciences' (GILD +0.39%) Cayston and Novartis' (NVS 0.53%) Tobi, to rid patients lungs of bacteria that causes the lung function problems in cystic fibrosis patients, although the investigator presenting the data said they couldn't find a correlation.
The points on the graph are averages, and those lines that extend upward from each point are confidence intervals. They also extend an equal distance below the point, and there's a 95% chance that the true value falls somewhere in there. Anyone want to hazard a guess why they didn't draw them in both directions like was done on the Kalydeco along slide above? It would make for a very busy slide and point out just how dirty the data really is. Small patient numbers will do that.
You'll notice that the data starts at day 28, which is when the combination treatment began. For the first 28 days, patients got VX-809 or placebo. Let's zoom out and look at the full trial. The data was presented on two slides (I'll let you guess why).
If you assume that the monotherapy isn't doing anything, the first 28 days show just how variable the data are given the small number of patients. How can we trust the 2.5 point decline in the placebo from day 28 to day 56 when the different treatment groups in the first 28 days varied from around a 3 point decline to a slight increase? If you look at the change for any given two-week period, there's even more variability.
Moving forward
Now admittedly, the 600mg dose showed a 3.4 point increase even if you measure from day one, and we didn't see any kind of increase like that over the first 28 days, so it seems reasonable to assume the drug combination is doing something. What worries me is that Vertex won't make the right assumptions when setting up the phase 3 trials to prove that the effect is real.
For an effect to be statistically significant, enough patients have to be enrolled in the trial. The exact number of patients required for a trial to be successful is determined by the expected difference between the treatment and the control groups as well as the expected variation within the treatment groups (those confidence intervals). Smaller treatment effects and the larger variations require more patients to prove a drug works.
There are a lot of drugs that probably work, but failing to show statistical significance in a clinical trial has kept them off the market. InterMune's (ITMN +0.00%) Esbriet, for instance, only passed one of its two clinical trials.
Worrying for nothing?
Pushing forward with limited phase 2 data is kind of dangerous, as investors in Aeterna Zentaris (AEZS 7.33%) and Keryx Biopharmaceuticals (KERX +0.00%) know all too well. But Vetrex has a distinct advantage because the phase 2 trial only looked at the combination treatment for four weeks. The phase 3 trial will likely measure lung function over a longer timeframe, perhaps 48 weeks like for Kalydeco. If the combination is working, the curves may continue to separate, making the variation of the individual data points less of an issue.
