Color-Code Agreement Among Stratus, Cirrus, and Spectralis Optical Coherence Tomography in Relapsing-Remitting Multiple Sclerosis With and Without Prior Optic Neuritis




Purpose


To evaluate the agreement of retinal nerve fiber layer (RNFL) color codes among Stratus, Cirrus, and Spectralis optical coherence tomography (OCT) in patients with relapsing-remitting multiple sclerosis.


Design


Prospective cohort study.


Methods


In 140 eyes from 70 patients having relapsing-remitting multiple sclerosis from January 2011 to September 2011, peripapillary RNFL thickness was measured using the fast RNFL program by Stratus, the optic disc cube protocol by Cirrus, and the N-site axonal analysis by Spectralis.


Results


Overall, a moderate to good RNFL color code agreement was found (0.435-0.884), except for the nasal quadrant. The temporal quadrant was the most abnormal color coding by both Cirrus (64.7%) and Spectralis (61.7%) in both the optic neuritis (ON) and non-ON group and by Stratus (58.8%) in the ON group. Abnormal temporal RNFL color-code rate was significantly higher in ON eyes than non-ON eyes by Cirrus ( P < .001), Stratus ( P < .001), and Spectralis ( P = .030). Overall, Cirrus significantly displayed abnormal findings while both Stratus and Spectralis displayed normal results for the inferior quadrant ( P < .05). On the other hand, Spectralis OCT showed a significantly higher rate of abnormal findings while Cirrus displayed normal results for the temporal quadrant in non-ON eyes ( P < .001).


Conclusions


We found a substantial color-code disagreement among devices in patients with relapsing-remitting multiple sclerosis regarding the ON antecedent. In non-ON eyes, Spectralis yielded a significantly higher thinning for temporal quadrant than Cirrus, suggesting that N-site axonal analysis could define axonal damage in relapsing-remitting multiple sclerosis patients earlier than conventional RNFL analysis.


Relapsing-remitting multiple sclerosis is a chronic immune-mediated disease of the central nervous system and the visual system is frequently involved, often in the form of visual loss from optic neuritis (ON). Optical coherence tomography (OCT) is a very valuable tool for evaluating patients with ON and relapsing-remitting multiple sclerosis and has evolved into an important primary and secondary outcome measure for relapsing-remitting multiple sclerosis clinical trials.


Newer-generation spectral-domain (SD) OCT devices offer advantages over time-domain (TD) OCT techniques, such as, for example, faster scanning speeds and higher-resolution retinal nerve fiber layer (RNFL) imaging.


Many patients with multiple sclerosis who have been scanned with Stratus TD-OCT (Carl-Zeiss Meditec Inc, Dublin, California, USA) can now be scanned with SD-OCT devices. There are significant differences among commercially available OCT devices in RNFL thickness measurements in patients with ON and/or relapsing-remitting multiple sclerosis, so that RNFL measurements cannot be directly compared. Clinicians should be aware that measurements are generally higher with Stratus than with Cirrus, except at thinner RNFL values, and slightly higher on Spectralis compared to Cirrus OCT.


OCT provides information on probable RNFL defects as 4 color codes through comparison of the RNFL profile with an internal age-matched normative database. Recently, Kim and associates evaluated the color-code agreement between Cirrus and Stratus OCT in patients with glaucoma, and they concluded that color codes were not interchangeable. Besides that, the pattern of RNFL loss in multiple sclerosis patients with or without ON differs from that of the glaucoma patients.


The purpose of this study was to evaluate and compare the agreement of RNFL color codes in relapsing-remitting multiple sclerosis patients scanned with 3 OCT devices—1 TD-OCT (Stratus) and the 2 most available SD-OCT devices (Cirrus and Spectralis)—according to the ON antecedent.


Methods


Subjects


A prospective transversal study was performed. The study was approved by the Research Ethics Committee in the Ramón y Cajal University Hospital. All research complied with the tenets of the Declaration of Helsinki, and all patients participating in the study gave their informed consent. Confidentiality of participating patients was protected throughout the study. Seventy patients with a diagnosis of relapsing-remitting multiple sclerosis ( Table 1 ) were enrolled consecutively from the Neuro-Ophthalmology Department from January 1, 2011 to September 30, 2011. Diagnosis was made by a neurologist (L.C.) based on McDonald criteria. None of the included patients had a diagnosis of secondary progressive multiple sclerosis.



Table 1

Clinical and Demographic Characteristics of the 140 Eyes of 70 Patients With Relapsing-Remitting Multiple Sclerosis Included in the Study a




























































































ON Eyes (N = 34) Non-ON Eyes (N = 106) P MD ≤ −1.83 (N = 70) MD > −1.83 (N = 70) P
Age (y) 42 (10) 39 (11) .170 41 (11) 39 (10) .352
Female (%) 19 (56) 67 (65) .370 38 (54) 49 (70) .040
Mean VA 0.9 (0.3) 1.0 (0.2) .003 0.9 (0.2) 1.0 (0.2) .004
EDSS 2.5 (1.7) 2.5 (1.8) .810 2.9 (2.1) 2.0 (1.1) .006
Duration (mo) 7.3 (6.4) 6.7 (7.3) .676 8.7 (7.7) 5.1 (5.7) .002
Mean MD (dB) −3.8 (5.3) −2.3 (3.1) .047 −4.7 (4.5) −0.7 (0.8) <.001
Mean PSD (dB) 2.6 (1.9) 1.97 (1.4) .029 2.6 (1.9) 1.6 (0.7) <.001
Mean RNFL thickness by Stratus (μm) 85 (15) 95 (13) .001 88 (13) 97 (14) .001
Mean RNFL thickness by Cirrus (μm) 79 (13) 87 (12) .001 81 (11) 89 (3) <.001
Mean RNFL thickness by Spectralis (μm) 80 (14) 90 (13) .000 83 (13) 91 (14) .001

EDSS = Expanded Disability Status Scale; MD = Humphrey visual field 24-2, SITA standard mean deviation; ON = optic neuritis antecedent; PSD = Humphrey visual field 24-2, SITA pattern standard deviation; RNFL = retinal nerve fiber layer; VA = visual acuity.

a Data are shown as mean (standard deviation).



All underwent a complete neuro-ophthalmic evaluation that included pupillary, anterior segment, and funduscopic examinations; assessment of best-corrected visual acuity (BCVA) relative to the Snellen scale; and visual field examination, and they were scanned after pupillary dilation with Stratus, Cirrus (Carl Zeiss Meditec AG, Jena, Germany), and Spectralis OCT (Heidelberg Engineering GmbH, Heidelberg, Germany) on the same day.


Both eyes of each subject were included. Exclusion criteria were intraocular pressure of 21 mm Hg or higher, a refractive error greater than 5.0 diopters (D) of spherical equivalent or 3.0 D of astigmatism in either eye, systemic conditions that could affect the visual system, a history of ocular trauma, or concomitant ocular diseases.


Related medical records were carefully reviewed, including the duration of the disease, the Expanded Disability Status Scale (EDSS) scored by a neurologist (L.C.), and the presence of prior episodes of ON as reported by the treating neurologist and the patient.


The visual field (VF) was tested using a Humphrey Field Analyzer (Carl Zeiss Meditec) and the SITA Standard strategy (program 24-2). It was considered reliable when fixation losses were less than 20% and false-positive and false-negative errors were less than 15%.


Optical Coherence Tomography Measurements


A single well-trained optometrist (N.O.) performed all OCT examinations in random order to prevent any fatigue bias. All poor-quality scans were rejected, defined as those with signal strength of <6 by Stratus and Cirrus. For Spectralis OCT only those images with a score higher than 25 were analyzed. Scans with misalignment, segmentation failure, or decentration of the measurement circle were excluded from the analysis.


Cross-sectional imaging of the peripapillary area was performed using Stratus OCT (software version 4.0). The RNFL thickness was measured with the Fast RNFL program, which consisted of 3 sets of 3.46-mm-diameter circular scans (256 A-scans) centered on the optic disc.


On Cirrus OCT, peripapillary RNFL thickness was determined using the optic disc cube protocol (software version 5.1.1.6) that generates a cube of data through a 6-mm-square grid. A 3.46-mm-diameter calculation circle was automatically positioned around the optic disc.


Both OCTs (Stratus and Cirrus) provide average RNFL thickness and maps with 4 quadrants (superior, inferior, nasal, and temporal) and 12 clock hours, including classification from an internal normative database. Spectralis OCT (software version 5.2.0.3) simultaneously captures infrared fundus and SD-OCT images at 40 000 A-scans per second. A real-time eye-tracking system measures eye movements and provides feedback to the scanning mechanism, to stabilize the retinal position of the B-scan. The instrument uses 1024 A-scan points from a 3.45-mm circle centered on the optic disc. The examiner is required to manually place the scan around the optic disc. RNFL measurements were obtained using the N-site axonal protocol, which differs from the standard RNFL scan because it starts and terminates in the nasal side of optic nerve. The RNFL Spectralis protocol generates a map showing the average thickness, maps with 4 quadrants (superior, inferior, nasal, and temporal), and maps with 6 sector thicknesses (superonasal, nasal, inferonasal, inferotemporal, temporal, and superotemporal).


The RNFL thicknesses in the normal range are represented by green backgrounds. Those that are abnormal at the 5% and at the 1% level are represented by yellow and red backgrounds, respectively. The hypernormal (95th to 100th percentiles) RNFL thicknesses are presented by a white color in Stratus and Cirrus, and by a blue/purple color in Spectralis.


Disagreement among instruments was defined as when an abnormal sector (yellow or red color) in one OCT was displayed as normal (green, white, or blue) in another OCT or vice versa.


Statistical Analysis


A Bland-Altman scatterplot was used to assess agreement among devices in RNFL measurements. The agreement of RNFL color codes was evaluated with a Cohen kappa test. After defining red or yellow color codes as abnormal results, we quantified the frequency of color-code disagreement among OCTs according to the history of ON. Differences in means between subgroups were tested with Student t test for independent samples, or with Mann-Whitney U test when the variables did not follow a Gaussian distribution. Association between qualitative variables was tested using the χ 2 test or Fisher exact test when small samples were present.


Data were analyzed using Stata/SE 12.0 for Unix and IBM SPSS Version 20 for Unix. A P value of less than .05 was considered statistically significant.




Results


A total of 140 eyes from 70 patients with relapsing-remitting multiple sclerosis were enrolled. Median EDSS was 1.5 (range, 1-7). Twenty-three patients (32.9%) had no treatment and 47 (67.1%) received 1 drug for multiple sclerosis, the most common therapy being interferon beta 1a (Rebif or Avonex) (70.2%).


Table 1 shows the epidemiologic and clinical characteristics of the included eyes. Considering the median mean deviation (MD), eyes were categorized in 2 groups. Eyes with worse visual field (MD ≤ −1.83 dB) had a worse mean visual acuity (VA) ( P = .004), a longer-lasting disease ( P = .002), a higher EDSS score ( P = .006), and a significantly thinner RNFL with all the OCT instruments than eyes with less visual field damage (MD > −1.83 dB).


According to the ON antecedent, the ON group (N = 34) had a significantly worse mean VA ( P = .003), MD ( P = .047) and pattern standard deviation (PSD) ( P = .09) than the non-ON group (N = 106). The average RNFL thickness was significantly thinner in the ON group with all devices, Stratus ( P = .000), Cirrus ( P < .001), and Spectralis ( P = .001), than in the non-ON group.


RNFL Thickness Measurement Agreement Among Devices


Bland-Altman plots showed an important discrepancy between RNFL thickness measurements obtained by the 3 instruments. For average thicker RNFLs, Cirrus measurements tended to be thinner than those of the Stratus ( Figure 1 ). Similar findings were observed in all the quadrants. A similar slope was found comparing Spectralis and Cirrus. The Spectralis measurements tended to be thicker than those of the Cirrus for thicker RNFLs ( Figure 2 ).




Figure 1


Agreement between retinal nerve fiber layer (RNFL) measurements with Stratus and Cirrus optical coherence tomography in patients with relapsing-remitting multiple sclerosis. Bland-Altman plot shows that Stratus RNFL measurements are larger than Cirrus, but differences tend to be smaller when average RNFL thickness is very thin. The slope is −0.293, P = .001. Mean difference (Cirrus minus Stratus) is −7.499 (95% confidence interval −8.550 to −6.448); and the limits of agreement ranged from −19.783 to 4.785.



Figure 2


Bland-Altman plot comparing the measurements displayed by Cirrus and Spectralis optical coherence tomography in patients with relapsing-remitting multiple sclerosis. The difference between both measurements is plotted against the average of both measurements. Slope was −0.255 ( P = .007). Mean difference (Cirrus minus Spectralis) was −2.913 (95% confidence interval −3.862 to −1.965); and the limits of agreement ranged from −13.497 to 7.670.


Table 2 shows the measured difference of RNFL thickness (Cirrus minus Stratus and Cirrus minus Spectralis) for average and quadrants. Overall, the Cirrus minus Stratus differences were higher compared with the Cirrus minus Spectralis differences. The limits of agreement were also narrower between both SD devices than between Stratus and Cirrus ( Figures 1 and 2 ). Average RNFL thickness measurement for Cirrus minus Spectralis was statistically greater in the non-ON group than in the ON group ( P = .004) ( Table 2 ).



Table 2

Differences in Retinal Nerve Fiber Layer Thickness Measurements (μm) Among Eyes With and Without Optic Neuritis, in Patients With Relapsing-Remitting Multiple Sclerosis































































ON eyes Non-ON eyes P
Stratus Minus Cirrus
Average 6.40 (6.50) 7.89 (6.23) .234
Temporal 5.65 (11.24) 6.20 (8.97) .769
Superior 1.44 (12.70) 3.42 (14.91) .489
Nasal 5.85 (19.67) 8.65 (16.05) .407
Inferior 12.59 (12.62) 12.58 (11.58) .998
Spectralis Minus Cirrus
Average 0.56 (7.32) 3.77 (4.37) .004
Temporal 2.44 (7.42) 5.00 (8.62) .136
Superior −0.37 (9.3) 1.40 (10.19) .387
Nasal −1.28 (8.9) −0.18 (8.73) .543
Inferior 5.31 (7.62) 7.59 (7.03) .124

ON = optic neuritis.

Data are shown as mean (standard deviation).


Color Code Agreement Among Devices


Table 3 shows the Cohen kappa of the color code agreement between Cirrus and Stratus and between Cirrus and Spectralis. The agreement of RNFL color code was moderate to good in all subgroups (0.435-0.882), except for nasal quadrant, which showed the lowest kappa value. Cohen kappa values were higher in ON than non-ON eyes.



Table 3

Cohen Kappa of Color-Code Agreement Between Cirrus and Stratus and Between Cirrus and Spectralis Overall and Regarding Optic Neuritis History in Patients With Relapsing-Remitting Multiple Sclerosis























































Cirrus vs Stratus Cirrus vs Spectralis
Overall With ON Without ON Overall With ON Without ON
Average 0.746 0.882 0.676 0.683 0.751 0.617
Temporal 0.705 0.752 0.597 0.697 0.826 0.601
Superior 0.580 0.810 0.492 0.695 0.788 0.646
Nasal 0.311 0.242 0.364 0.374 0.536 0.183
Inferior 0.527 0.435 0.538 0.706 0.669 0.701

Only gold members can continue reading. Log In or Register to continue

Stay updated, free articles. Join our Telegram channel

Jan 9, 2017 | Posted by in OPHTHALMOLOGY | Comments Off on Color-Code Agreement Among Stratus, Cirrus, and Spectralis Optical Coherence Tomography in Relapsing-Remitting Multiple Sclerosis With and Without Prior Optic Neuritis

Full access? Get Clinical Tree

Get Clinical Tree app for offline access