UPFRONT

Playing the Numbers

Peter K. Kaiser, MD

“Statistics: the only science that enables different experts using the same figures to draw different conclusions.”
— Evan Esar

At ARVO, we learned the CATT and IVAN outcomes. As Dr. Martin noted, each side viewed the results to its own benefit. Lucentis advocates say they “conclusively” prove Lucentis dries the retina better and more importantly is safer, but IVAN showed Lucentis had a “significantly” higher rate of arteriothrombolic events (P=.03).

Avastin users say the two drugs were noninferior in terms of visual outcomes in CATT but omit that Avastin failed the noninferiority hypothesis and was very close to being inferior in IVAN. Using statistics, either side could easily argue its point.

The IVAN presentation required almost 10 minutes to explain the statistics used; this confusion has extended throughout the study results. Some believe the studies support using PRN treatment, but CATT showed a significant difference with monthly winning. In contrast, IVAN showed they were equivalent.

The “meta-analysis” of the two studies also showed a significant difference in favor of monthly. Meta-analysis usually trumps individual studies, but with only two studies and CATT's size dwarfing IVAN, CATT will drive the results of this analysis. Only with more comparison studies being added will any outcome be clinically meaningful. So we still don't know whether the huge difference in number of injections in the monthly regimen equates with better long-term results.

The problem with this statistical morass is that insurance companies are also bending the statistics to their benefit. A friend told me that the National Health Service is considering “approving” Avastin (which they can't do because NHS is an insurance carrier), to which I replied, tongue firmly in cheek, that this is excellent news as now they can also “approve” other drugs that barely missed their noninferiority endpoints, including high-dose Lucentis (HARBOR), anecortave acetate (AART), and combination therapy with PDT (DENALI), which missed their primary endpoints by a few letters, similar to the “clinically insignificant” differences in CATT and IVAN, but failed to meet the noninferiority endpoint required by regulatory agencies.

American insurance carriers have even sent letters requiring the use of Avastin, taking the choice way from doctors and patients, but the number needed to harm (in terms of hospitalizations, not ATEs) for Avastin right now is considerably higher than Vioxx when it was taken off the market. By requiring Avastin, insurers may be setting themselves up for problems if the safety trend continues in other comparison studies.

The poet Andrew Lang said an unsophisticated forecaster uses statistics as a drunken man uses lamp posts: for support rather than illumination. This describes perfectly where we stand now. We have two drugs that are pretty similar, except in price. If we made the same amount of money using them, we could really see which is better. Maybe the best conclusion is for everyone to have another round…