Purpose
To present a methodology for calculating likelihood ratios for glaucoma diagnosis for continuous retinal nerve fiber layer (RNFL) thickness measurements from spectral-domain optical coherence tomography (spectral-domain OCT).
Design
Observational cohort study.
Methods
A total of 262 eyes of 187 patients with glaucoma and 190 eyes of 100 control subjects were included in the study. Subjects were recruited from the Diagnostic Innovations Glaucoma Study. Eyes with preperimetric and perimetric glaucomatous damage were included in the glaucoma group. The control group was composed of healthy eyes with normal visual fields from subjects recruited from the general population. All eyes underwent RNFL imaging with Spectralis spectral-domain OCT. Likelihood ratios for glaucoma diagnosis were estimated for specific global RNFL thickness measurements using a methodology based on estimating the tangents to the receiver operating characteristic (ROC) curve.
Results
Likelihood ratios could be determined for continuous values of average RNFL thickness. Average RNFL thickness values lower than 86 μm were associated with positive likelihood ratios (ie, likelihood ratios greater than 1), whereas RNFL thickness values higher than 86 μm were associated with negative likelihood ratios (ie, likelihood ratios smaller than 1). A modified Fagan nomogram was provided to assist calculation of posttest probability of disease from the calculated likelihood ratios and pretest probability of disease.
Conclusion
The methodology allowed calculation of likelihood ratios for specific RNFL thickness values. By avoiding arbitrary categorization of test results, it potentially allows for an improved integration of test results into diagnostic clinical decision making.
Several imaging technologies have been introduced with the purpose of objectively quantifying structural damage to the optic disc and retinal nerve fiber layer (RNFL) in glaucoma. As the number of available technologies increases, difficult decisions arise about which tests should be used and how to incorporate their results in clinical practice. Sensitivity, specificity, and the area under the receiver operating characteristic (ROC) curve have traditionally been used as measures of diagnostic accuracy. Although well established, these measures have some limitations in clinical use, which arise mainly from the fact that they are population measures; that is, they summarize the characteristics of the test over a population. However, in the diagnostic process, clinicians want measures that can be directly incorporated into the decision-making process for individual patients.
During the diagnostic process, a clinician combines medical history and clinical examination to estimate the probability that the disease is present. If there is still uncertainty with regard to the presence of disease, additional diagnostic tests can be performed. The purpose of the diagnostic test will be to modify the pretest probability of disease into a more conclusive posttest probability of disease. Diagnostic likelihood ratios can be directly used to modify the initial suspicion for disease (pretest probability) into a new probability of disease (posttest probability). The use of likelihood ratios has been proposed as the best way of incorporating results of diagnostic tests into clinical decision making according to the principles of evidence-based medicine. Moreover, like sensitivity and specificity, likelihood ratios have the property of not being affected by the prevalence of the disease.
A few previous studies have reported likelihood ratios for glaucoma diagnosis using optic disc and RNFL analysis from imaging instruments. In these studies, results of the diagnostic tests were arbitrarily categorized into normal vs abnormal according to specific cut-offs. The dichotomization of test results with continuous outcomes may result in loss of information, because results that are markedly abnormal are grouped with results that are only mildly abnormal. The loss of information is even more important when the patient’s test result is close to the established cut-off.
In the present study, we apply a methodology for estimating likelihood ratios for glaucoma diagnosis that does not require arbitrary categorization of test results. This methodology allows the calculation of likelihood ratios for all values of the test with direct application to the calculation of posttest probability of disease. We demonstrate the methodology using RNFL thickness analysis with spectral-domain optical coherence tomography (spectral-domain OCT). Finally, we propose a modified Fagan nomogram that can be used to facilitate the use of likelihood ratios for continuous RNFL thickness measurements into the diagnostic process at the clinical practice.
Methods
This was an observational cohort study. Patients were recruited from the Diagnostic Innovations in Glaucoma Study conducted at the Hamilton Glaucoma Center (University of California, San Diego). The Diagnostic Innovations in Glaucoma Study is registered at http://www.clinicaltrials.gov under the identifier NCT00221897 . Participants were longitudinally evaluated according to a protocol that included visits with a comprehensive clinical examination and several imaging and functional tests. All participants who met the inclusion criteria described below were enrolled and all data were entered in a computer database. Informed consent was obtained from all participants. The University of California San Diego Human Subjects Committee approved all protocols, and methods adhered to the Declaration of Helsinki.
Each subject underwent a comprehensive ophthalmic examination, including review of medical history, best-corrected visual acuity, slit-lamp biomicroscopy, intraocular pressure measurement using Goldmann applanation tonometry, gonioscopy, dilated fundoscopy examination using a 78-diopter (D) lens, stereoscopic optic disc photography, and standard automated perimetry with 24-2 Swedish Interactive Threshold Algorithm (Carl Zeiss Meditec, Inc, Dublin, California, USA). To be included, subjects had to have best-corrected visual acuity of 20/40 or better, spherical refraction within ±5.0 D, cylinder correction less than 3.0 D, and open angle with gonioscopy. Subjects with coexisting retinal disease, uveitis, or nonglaucomatous optic disc neuropathy were excluded from the study.
Eyes were classified as glaucomatous if they had repeatable (at least 3 consecutive) abnormal visual field test results, defined as pattern standard deviation with a P value <5% and/or a glaucoma hemifield test result outside normal limits, regardless of the appearance of the optic disc. Visual fields with more than 33% fixation losses, more than 33% false-negative errors, or more than 15% false-positive errors were excluded. The only exception was the inclusion of patients with more than 33% of false-negative errors and advanced disease stage (mean deviation lower than minus 12 dB). In addition, all visual fields were reviewed by the Visual Field Assessment Center (University of California at San Diego) and masked to the results of other tests. Visual fields were reviewed and excluded in the presence of artifacts such as eyelid or rim artifacts, fatigue effects, inattention, or inappropriate fixation. Visual fields were also reviewed for the presence of abnormalities that could indicate diseases other than glaucoma, such as homonymous hemianopia.
We also included in the glaucoma group patients with preperimetric glaucomatous damage. The inclusion of these eyes required longitudinal follow-up to determine their ultimate diagnosis, as described in previous publications. These eyes had normal visual field examinations at the time of imaging, but had history of documented evidence of progressive glaucomatous change in the appearance of the optic disc, as assessed by simultaneous stereoscopic photographs (Topcon Instrument Corporation of America, Paramus, New Jersey, USA). The evidence of progressive glaucomatous damage had to be present before the imaging test date. For evaluation of progressive optic disc damage, stereoscopic sets were examined using a stereoscopic viewer (Asahi; Pentax, Tokyo, Japan). All stereophotographs were evaluated by 2 experienced graders masked to the subject’s identity, chronological sequence of the stereophotographs, and other test results. For inclusion, stereophotographs needed to be graded adequate or better. Definition of change was based on focal or diffuse thinning of the neuroretinal rim, increased excavation, or enlargement of the RNFL defects. Changes on rim color, presence of disc hemorrhages, or progressive peripapillary atrophy were not sufficient for characterization of progression. Discrepancies between the 2 graders were resolved by either consensus or adjudication of a third experienced grader. The use of this composite reference standard for glaucoma diagnosis allowed us to evaluate likelihood ratios for continuous RNFL thickness measurements in patients with a broad spectrum of disease severity, as patients with visual field loss and patients with normal visual fields but confirmed progressive glaucomatous damage were included.
Healthy eyes had intraocular pressures of 21 mm Hg or less with no history of increased intraocular pressure, a normal clinical examination, and at least 2 reliable baseline normal visual field tests. A normal visual field was defined as a mean deviation and pattern standard deviation within 95% confidence limits and a glaucoma hemifield test result within normal limits. Healthy subjects were recruited from the general population or from spouses and relatives of patients.
Spectralis Spectral-Domain OCT
RNFL thickness measurements were obtained using the Spectralis spectral-domain OCT (software version 5.3.3.0; Heidelberg Engineering, Carlsbad, California, USA). Details of its operation have been published elsewhere. The high-resolution protocol was used, obtaining 1536 A-scans from a 3.45-mm circle centered at the optic disc, providing an axial resolution of 3.9 μm and a lateral resolution of 6.0 μm. To increase the image quality, the Spectralis spectral-domain OCT includes an automatic real-time function that gathers multiple frames (B-scans). On average, 20.0 B-scans per eye were used to generate the peripapillary RNFL curve. The images were then averaged for noise reduction. The quality score ranges from 0 dB (poor) to 40 dB (excellent). All images were reviewed by experienced technicians at the Imaging Data Evaluation and Assessment Center. To be included, images had to be centered, with accurate segmentation, and had to have a signal strength >15 dB. For this study we estimated likelihood ratios for the global RNFL thickness (360-degree measure). We used this parameter because it represents a global measure of structural damage that has been shown to have good diagnostic performance in previous studies.
Statistical Analysis
Descriptive statistics included mean and standard deviation for normally distributed variables and mean, median, first quartile, and third quartile for non-normally distributed variables (determined by the Kolmogorov-Smirnov normality test). Normally distributed variables were compared between groups using the 2-sample t test, while non-normally distributed variables were compared using the Wilcoxon rank sum test. The χ 2 test was used to test for differences in categorical variables between groups. Generalized estimating equations with robust standard errors were used to adjust for potential correlations between both eyes of the same individual.
The application of likelihood ratios in the interpretation of results of imaging instruments for glaucoma diagnosis has been detailed previously. Briefly, likelihood ratio is the ratio between 2 probabilities: the probability of a given test result when the disease is present (true-positive ratio) divided by the probability of the same test result when the disease is absent (false-positive ratio). Once determined, a likelihood ratio can be directly incorporated into the calculation of posttest odds of disease by using a formulation of the Bayes theorem, which uses odds of disease.
The first step in the calculation of the posttest probability of disease is to calculate the pretest odds of disease from the pretest probability of the disease using the following equation:
odds = probability / ( 1 − probability )
posttest odds = ( pretest odds ) × likelihood ratio
Finally, we can calculate the posttest probability of disease from posttest odds of disease using the following equation:
Supplemental Figure 1 (available at AJO.com ) shows the steps necessary for the calculation of the posttest probability of disease and how the modified Fagan nomogram (described below) simplifies the process.
In the present study, likelihood ratios for continuous RNFL thickness values were estimated using the method described by Gu and Pepe. Formulas used to derive likelihood ratios for continuous RNFL thickness measurements are included in the Appendix . Supplemental Figure 2 (available at AJO.com ) illustrates that the likelihood ratio can be interpreted as the tangent of the ROC curve at a threshold c . Adjustments for the presence of both eyes and for differences in confounding variables (age and sex) between cases and controls were performed as described by Alonzo and Pepe.
All statistical analyses were performed with commercially available software (Stata version 11; StataCorp, College Station, Texas, USA). The alpha level (type I error) was set at 0.05.