Flawed Statistical Analysis

Amparo and associates “included patients 18 years and older with a previous diagnosis of DED, who had continued presence of dry eye symptoms and corneal fluorescein staining,” yet did not include tear osmolarity as part of the criteria. With staining and symptoms as the only inclusion criteria, there is a considerable risk of mean regression in these signs. Put simply, since the investigators are selecting for patients with high degrees of corneal staining and symptoms, probability dictates that on retesting, these metrics will test lower as a reflection of natural variation. Owing to the suboptimal study design and lack of a control group, the investigators incorrectly assign this lowering trend to a treatment effect. Measures not used as inclusion criteria are not subject to this constraint and do not show the same bias. If true, the conclusions from Amparo and associates are based entirely on artifact, and the claims regarding the utility of tear osmolarity are specious.

To test this hypothesis, we retrospectively examined longitudinal data from 52 subjects who were “eligible if they were older than 17 years and had been given an ocular examination indicating the presence of dry eye disease within the 2 years before the study.” The population was a true clinical representation of the spectrum of dry eye seen in a cornea clinic, and the inclusion criteria were unbiased—no threshold was applied to a subset of signs. Patients were followed for 3 months, allowing correlations of changes to be observed in each of the 1- and 2-month intervals.

The unbiased data revealed that without the mean regression pressure, there was a statistically significant, though modest, correlation between the changes in tear osmolarity and ocular surface disease index (OSDI) at both 1-month ( r = 0.256; P < .034) and 2-month ( r = 0.243; P < .042) intervals. However, there was no statistically significant correlation between the recorded change in corneal fluorescein staining and OSDI at any of the time points ( r = 0.094, r = −0.134).

By the logic of Amparo and associates, these data would suggest that corneal staining has no clinical utility and significant limitations in prospective follow-up of patients as a marker of whether patients are doing “clinically better” or not. Of course, we know this not to be true. Corneal staining has proven its utility in assisting physicians to manage dry eye disease, as has tear osmolarity. Rather, the methods of Amparo and associates were fundamentally flawed and entirely driven by artifact. There are no clinically relevant correlations between any of the measures of dry eye at equilibrium. DED markers fluctuate independently of each other, and unless there is a specific, prospective therapeutic influence (or artifactual statistical influence), the markers do not move in synchrony.

Flawed Study Design and Overly Strong Conclusions

Tear osmolarity and the other dry eye signs are well known to be statistically independent. As a result, the relatively low osmolarity and broad variance are expected when biased selection criteria are applied. However, this has absolutely no bearing on the utility of tear osmolarity in monitoring the response of dry eye patients to therapy; if tear osmolarity were perfectly correlated with corneal staining or OSDI then it would offer no additional value, as the information would already be available.

Unlike Amparo and associates’ study, well-controlled prospective studies have repeatedly shown that tear osmolarity is capable at monitoring therapy. Oftentimes improvements in tear osmolarity precede that of staining and symptoms. Given their narrow definition of dry eye disease, it is not surprising that the authors failed to take into account the relative sensitivities of the markers, instead randomly mixing therapeutic duration and initiation. Instead of a prospectively controlled, randomized trial to assess efficacy, the investigators analyzed disparate treatments initiated at different times, with considerable variation between visits (137 ± 95 days). These choices effectively masked the predictive ability of tear osmolarity.

Finally, notwithstanding the impacts of poor trial design, the reported correlation between the changes in staining and OSDI in this study were r ² = 0.09, meaning that only 9% of the proportion of the total variation of the outcomes was predicted by corneal staining, hardly a clinically relevant number. These data inform the very strong conclusions made by Amparo and associates and lead them to portray a very different version of reality than the data support.

Flawed Study Design and Overly Strong Conclusions

Tear osmolarity and the other dry eye signs are well known to be statistically independent. As a result, the relatively low osmolarity and broad variance are expected when biased selection criteria are applied. However, this has absolutely no bearing on the utility of tear osmolarity in monitoring the response of dry eye patients to therapy; if tear osmolarity were perfectly correlated with corneal staining or OSDI then it would offer no additional value, as the information would already be available.

Unlike Amparo and associates’ study, well-controlled prospective studies have repeatedly shown that tear osmolarity is capable at monitoring therapy. Oftentimes improvements in tear osmolarity precede that of staining and symptoms. Given their narrow definition of dry eye disease, it is not surprising that the authors failed to take into account the relative sensitivities of the markers, instead randomly mixing therapeutic duration and initiation. Instead of a prospectively controlled, randomized trial to assess efficacy, the investigators analyzed disparate treatments initiated at different times, with considerable variation between visits (137 ± 95 days). These choices effectively masked the predictive ability of tear osmolarity.

Finally, notwithstanding the impacts of poor trial design, the reported correlation between the changes in staining and OSDI in this study were r ² = 0.09, meaning that only 9% of the proportion of the total variation of the outcomes was predicted by corneal staining, hardly a clinically relevant number. These data inform the very strong conclusions made by Amparo and associates and lead them to portray a very different version of reality than the data support.