A recent report in the American Journal of Ophthalmology describes the results from a prospective, randomized, double-masked clinical comparison of bevacizumab and ranibizumab for treatment of patients with age-related macular degeneration. The abstract results state that a t test failed to demonstrate statistical significance between the 2 groups, and the conclusion states that “early results … show no difference in efficacy.” The reader is led to believe that the 2 interventions are equivalent. Unfortunately, the data presented do not support that conclusion. In the study, 13 patients received bevacizumab and 7 received ranibizumab. The authors report mean differences in change in visual acuity and central macular thickness. The statistical analysis reveals overlapping confidence intervals and a P value that does not reach statistical significance. No statistical power is provided.

When a trial fails to detect a statistical difference between interventions, it is possible that the study was underpowered. The appropriate test statistic in this situation is statistical power and not a P value. The authors note that a sample size of 135 patients (not 20) would be needed to detect a “moderate sized effect” with power of 0.79; they do not report the statistical power of their trial. We respectfully ask that the authors report the likelihood that a study with this sample size would have been able to detect a 10% difference in acuity and macular thickness outcomes between the groups.

The public health and economic consequences of a well-designed, appropriately powered study comparing these 2 treatments are enormous. As the authors note, there is a large, ongoing, multicenter, randomized clinical trial (Comparison of ARMD Treatments Trial) designed to evaluate the questions posed by these authors in which 1200 patients are being enrolled so that the study has adequate power to answer the questions posed by these authors. Unfortunately, the only valid conclusion of this current report is that a study enrolling at least 135 patients in needed to compare these treatments adequately. No guidance regarding the relative efficacy of these 2 medications can be obtained from this study. Other methodologic criticisms aside (e.g., nonrepresentative patient demographics, change in inclusion criteria, short follow-up), the conveyed message of an equivalent treatment effect based on an insufficient sample is dangerous. Although the authors do mention that sample size is a limitation of the study, the conclusion that “reliance on data … such as this one, will have to suffice” in the interim, is simply irresponsible.