Reproducibility of Retinal Thickness Measurements on Normal and Pathologic Eyes by Different Optical Coherence Tomography Instruments




Purpose


To compare retinal thickness measurements produced by different time-domain and spectral-domain optical coherence tomography (TD-OCT and SD-OCT) devices when imaging normal and pathologic eyes.


Design


Prospective, observational study in an academic institutional setting.


Methods


A total of 110 eyes were imaged by 6 different OCT devices: Stratus and Cirrus (Carl Zeiss Meditec Inc), Spectralis HRA+OCT (Heidelberg Engineering), RTVue-100 (Optovue Inc), SDOCT Copernicus HR (Optopol Technology S.A.), and 3D OCT-1000 (Topcon Corporation). Eyes were normal or affected by different pathologies of the retina, including exudative and nonexudative age-related macular degeneration, epiretinal membrane, cystoid macular edema, and macular hole. For each instrument we used standard analysis protocols for macular thickness evaluation. Mean retinal thickness values between the instruments in the ETDRS central circular 1000-μm-diameter areas and in the ETDRS midperipheral circular 3000-μm-diameter areas were compared.


Results


The 6 different devices produced measurements that differ in variance (Bartlett test, P = .006), and mean values (Friedman test, P < .001). Bland-Altman analysis revealed that the limits of agreement for all the comparisons were not acceptable. Regression was calculated and it was elaborated into a conversion table, despite a high standard error for both intercepts and slope conversion values.


Conclusions


This study suggests that retinal thickness measurements obtained with various OCT devices are different beyond clinical practice tolerance, according to Bland-Altman analysis. Furthermore, regression analysis reveals high standard error values. These differences appear to be primarily attributable to the analysis algorithms used to set retinal inner and outer boundaries.


Optical coherence tomography (OCT) allows noninvasive capturing of cross-sectional high-resolution images of the retina in various pathologies. Since its introduction, OCT has allowed for the measurement of retinal thickness over time with good reliability and reproducibility. This feature has improved the clinical efficiency of diagnosis and follow-up of both normal retinas and those with pathologic conditions. Recent improvements in OCT technology have led to the development of a new generation of machines, based on spectral-domain principles (SD-OCT). These instruments differ from the previous time-domain OCT (TD-OCT) machines, by permitting faster image acquisition speeds and higher axial resolution. The introduction of these new commercially available OCT devices offers many advantages, including hardware and software improvements with potential economic convenience because of major competition between a greater number of manufacturing companies. However, such variety in the marketplace for OCT technology has led to variability in retinal measurements and subsequent confusion in the interpretation of examinations. This has further complicated the delineation of the guidelines and procedures from the different reading centers in multicenter clinical trials.


In this study we used the most commonly utilized TD-OCT (Stratus OCT; Carl Zeiss Meditec Inc, Dublin, California, USA) and 5 new SD-OCTs to examine the same patients’ eyes, in the same visit, to evaluate possible variability in retinal thickness measurements with the different machines.


Methods


This prospective study was performed on consecutive patients with pathologic retinal diseases and also normal retinas at the Eye Clinic, Department of Clinical Science, Luigi Sacco Hospital, University of Milan after obtaining informed written consent. All patients underwent OCT examination on the same clinic visit using 6 different OCT instruments: Stratus (Version 4.0.2; Carl Zeiss Meditec Inc), Cirrus (Version 2.0.0.54; Carl Zeiss Meditec Inc), Spectralis HRA+OCT (Version 3.1.4; Heidelberg Engineering, Heidelberg, Germany), RTVue-100 (Version 2.5; Optovue Inc, Fremont, California, USA), SDOCT Copernicus HR (Version 2.01; Optopol Technology SA, Zawiercie, Poland), and 3D OCT-1000 (Version 2.12; Topcon Corporation, Tokyo, Japan).


The order of OCT examinations was randomly chosen for each patient and a minimum of 15 minutes elapsed between each examination. Expert and trained operators performed all the OCT examinations according to the analysis protocol and variables for each machine ( Table 1 ). For each machine, the best achievable image quality was obtained by providing artificial tears/lubricants before each scanning session and accounting for the best-corrected visual acuity when adjusting the focus. Following each examination, the images were deemed acceptable if the retina was clearly visible and distinguishable in every B-scan, no eye movement or blinking artifacts occurred during the examination, and the full depth and extent of the retina was visualized in each B-scan image.



TABLE 1

Optical Coherence Tompgraphy Instruments and Macular Thickness Protocols Used

















































Instrument Protocol Area Scan Lines A-scans per B-scan
Zeiss Stratus Fast macular thickness map 6-mm lines, equally spaced 30 degrees apart centered at the fovea 6 lines 128
Zeiss Cirrus 512 × 128 cube 6 × 6 mm 128 horizontal lines 512
Heidelberg Spectralis Volume 19 horizontal, consecutive parallel lines on area of 30 degrees (length) per 15 degrees (height) Real-time mean image reconstruction on 20 single frames 1536
Optovue RTVue-100 MM5 5 × 5 mm centered on the fovea 11 horizontal and 11 vertical lines (central 3 × 3-mm area) 668 on 3 × 3-mm area
6 horizontal and 6 vertical lines (central 5 × 5-mm area) 400 on 5 × 5-mm area
Optopol Copernicus 3D scan 7 × 7 mm 50 horizontal lines 743
Topcon 3D OCT-1000 3D acquisition 6 × 6 mm 128 horizontal lines 512

MM5 = macular map 5.


All of the instruments provided a retinal thickness map based on the ETDRS model: 3 concentric circular areas are centered on the fixation point. The diameter of each area is 1, 3, and 6 mm for all the instruments with the exception of the RTVue-100, which, according to the protocol used in this study, produced a map of 1-, 3-, and 5-mm concentric areas. The midperipheral concentric areas are divided into 4 subfields: superior, inferior, nasal, and temporal. In each one of these areas, including the central area, the software elaborates the mean value from all the thickness measurements on each single A-scan. In this study the comparative analysis was assessed on the mean thickness values of the central and midperipheral areas. This is because the external peripheral area extension was not uniform between the different instruments. Expert observers evaluated the artifacts of each single B-scan, produced during automatic retinal segmentation. Errors were defined by evidence of incongruence greater than 5 μm between real retinal inner and outer limit and the automatic positioning by the software of the OCTs. The presence of at least 1 error in any one B-scan was considered sufficient to classify the examination as affected by artifacts. Three different expert examiners performed this analysis.


Statistical Analysis


Statgraphics ver.5.1 (Statistical Graphics Corp, Herndon, Virginia, USA) and R language (available at www.R-project.org ) statistics software were used.


Data were analyzed using Shapiro-Wilk test to evaluate the normality of sample distribution, Bartlett test to evaluate variance homogeneity, and Friedman test to compare mean values. A P value > .05 was considered significant. We also used Bland-Altman analysis, setting the limits of agreement (LOA) to 2 standard deviations, and we calculated the Pearson correlation coefficient between means and differences on Bland-Altman graphs (R value). Regression analysis was executed to elaborate conversion values between the instruments.




Results


Image analysis was performed on a total of 110 eyes (63 patients; 23 male, 40 female). The mean patient age was 65.7 (range: 20–90); the mean corrected visual acuity was 20/32 (range: 20/320–20/20). Thirty-one of the eyes (28.2%) were normal, 25 (22.7%) were affected by exudative age-related macular degeneration, 17 (15.5%) by epiretinal membrane without foveal contour alterations, 13 (11.8%) by cystoid macular edema from diabetic retinopathy, and 12 (10.9%) by nonexudative age-related macular degeneration. The 12 remaining eyes (10.9%) were affected by macular hole, branch retinal vein occlusion, Stargardt disease, and central serous chorioretinopathy.


During the acquisition of images and data analysis there was some minor loss of data: results from 5 eyes in superior, temporal, and nasal peripheral areas and from 9 eyes in inferior peripheral areas were lost.


The analysis of retinal segmentation errors revealed that for the whole study population the total percentage of examinations showing at least 1 error was 25.4% (168/660); the percentage in normal eyes was 6.9% (13/186) and in pathologic eyes was 32.7% (155/474).


Retinal Segmentation


All the OCT instruments identified the inner retinal boundary as the first interferometric signal after the vitreous hyporeflective space, which corresponds to the internal limiting membrane. However, there were important differences between positioning of the outer retinal boundaries ( Figure 1 ). The Zeiss Stratus identifies the outer boundary at the inner-outer photoreceptor junctional interface. The Topcon 3D OCT and the Optopol Copernicus identify the outer boundary at the inner limit of the retinal pigment epithelium (RPE) layer, the Zeiss Cirrus at the middle of the RPE layer, the Optovue RTVue-100 at the external limit of the RPE, and the Heidelberg Spectralis at the Bruch membrane.




FIGURE 1


Different positioning of retinal boundaries in the optical coherence tomography (OCT) devices. (Top left) Zeiss Stratus. (Top middle) Topcon 3D OCT. (Top right) Optovue RTVue-100. (Bottom left) Optopol Copernicus. (Bottom middle) Zeiss Cirrus. (Bottom right) Heidelberg Spectralis. The red arrows indicate the distance used by the different algorithms to calculate retinal thickness. The only time-domain OCT (TD-OCT), the Zeiss Stratus (Top left), sets the outer limit more internal compared to all the other instruments, on the hyperreflective band corresponding to the interface between the inner and the outer segment of photoreceptors. All the spectral-domain OCT (SD-OCT) devices set the outer limit at level of the retinal pigment epithelium (RPE), even if in different position. The Heidelberg Spectralis (Bottom right) sets the outer limit more external than all the other instruments, at the level of the Bruch membrane, even if this layer is not distinguishable to RPE in this image. On the bottom left of each panel is reported the corresponding mean retinal thickness value from the central circular 1000-μm area. CMRT = central mean retinal thickness.


These differences are very relevant in pathologies that affect the RPE–Bruch membrane complex, particularly in cases of choroidal neovascularization ( Figure 2 , Table 2 ). In these cases the measurements are very different, especially when comparing the Zeiss Cirrus and the Heidelberg Spectralis.




FIGURE 2


Fibrotic choroidal neovascularization analyzed with the Zeiss Cirrus (Top) and the Heidelberg Spectralis (Bottom). The different algorithms of the 2 devices identify the outer retinal boundary at different positions: the Zeiss Cirrus, trying to identify the RPE shape, sets the retinal limit within the scar, while the Heidelberg Spectralis sets the outer boundary at the level of the Bruch membrane, leading to a big difference in retinal thickness measurement. On the top left of each panel, after the instrument label, is reported the corresponding mean retinal thickness value from the central circular 1000-μm area. CMRT = central mean retinal thickness.


TABLE 2

Mean Retinal Thickness Measurements ± SD (μm) in Different Pathologies in the Central Circular (1000-μm-Diameter) Early Treatment Diabetic Retinopathy Study














































Instrument Exudative AMD Epiretinal Membrane CME in DR Nonexudative AMD
Zeiss Cirrus 315 ± 77 318 ± 78 354 ± 131 278 ± 34
Optopol Copernicus 189 ± 116 224 ± 92 262 ± 129 176 ± 33
Optovue RTvue-100 288 ± 95 313 ± 81 353 ± 118 273 ± 34
Heidelberg Spectralis 384 ± 135 331 ± 82 390 ± 135 292 ± 34
Zeiss Stratus 243 ± 73 256 ± 77 320 ± 101 220 ± 25
Topcon 3D OCT-1000 259 ± 99 283 ± 86 348 ± 122 234 ± 35

AMD = age-related macular degeneration; CME = cystoid macular edema; DR = diabetic retinopathy.


Correlation Analysis


Thickness measurements of the entire sample of patients did not follow a normal distribution (Shapiro-Wilk test, P < .0001). Most of the pathologic conditions showed increased macular thickness, and this subset of measurements did not follow a normal distribution (Shapiro-Wilk test, P < .0001). Retinal thickness measurements from healthy eyes, however, had a normal distribution (Shapiro-Wilk test, P = .15).


The 6 different devices produced measurements without equal variances (Bartlett test, P = .006) and also with different mean values (Friedman test, P < .001) ( Figure 3 , Table 3 ). We obtained the same result: 1) excluding examinations with at least 1 artifact in retinal segmentation (Bartlett test, P = .012; Friedman, P < .001); 2) analyzing normal subjects only (Bartlett test, P < .001; Friedman, P < .001); 3) analyzing normal subjects and excluding examinations with at least 1 artifact (Bartlett test, P < .001; Friedman, P < .001); 4) analyzing retinal thickness mean values in the peripheral areas (Bartlett test, P < .02, Friedman, P < .001); and 5) analyzing retinal thickness mean values in the peripheral areas on normal subjects only (Bartlett test, P < .045, Friedman, P < .001).




FIGURE 3


Box-and-whisker graphs showing retinal thickness measurements with different optical coherence tomography instruments. (Top) All eyes. (Bottom) Normal eyes. In both analyses variances and mean values are different. The Optopol Copernicus and the Zeiss Stratus produced the lowest retinal thickness values, while the Heidelberg Spectralis produced the highest values.


TABLE 3

Evaluation of the Differences in Variance and in Mean Values for Thickness Measurements in the Evaluated Early Treatment Diabetic Retinopathy Study Subfields, Between the Different Instruments





































































Group (Area) a n Bartlett Test P Friedman Test P
All subjects (C) 110 .006 <.001
All subjects, no artifacts (C) 33 .012 <.001
Normal subjects (C) 31 <.001 <.001
Normal subjects, no artifacts (C) 15 <.001 <.001
All subjects (I) 101 b .02 <.001
All subjects (T) 105 b <.001 <.001
All subjects (S) 105 b <.001 <.001
All subjects (N) 105 b .01 <.001
Normal subjects (I) 27 b <.001 <.001
Normal subjects (T) 29 <.001 <.001
Normal subjects (S) 29 .045 <.001
Normal subjects (N) 29 <.001 <.001

C = central; I = inferior; N = nasal; S = superior; T = temporal.

a C is central circular area, 1000 μm in diameter; I, T, S, and N are peripheral circular areas, 3000 μm in diameter.


b Minor data loose during analysis.



Table 4 describes Bland-Altman analysis of the 15 possible comparisons for the 6 instruments. The Pearson correlation coefficient showed that 8 comparisons had a significant relationship between different mean and mean value. R values were both positive and negative. Figure 4 illustrates 3 examples of Bland-Altman analysis graphs, showing the 3 possible outcomes that were obtained: the Optopol Copernicus compared to the Zeiss Stratus is an example of positive R value ( Figure 4 , Top); the Topcon 3D OCT compared to the Zeiss Stratus is an example of negative R value ( Figure 4 , Middle); the Heidelberg Spectralis compared to the Zeiss Stratus is a typical case in which, despite a positive R value, the dispersion in the differences between values increases with the mean thickness values ( Figure 4 , Bottom).



TABLE 4

Mean Retinal Thickness Measurements Comparison Using Bland-Altman Analysis in the Central Circular (1000-μm-Diameter) Early Treatment Diabetic Retinopathy Study Subfield, Between Instruments




































































































































OCT Comparison MD (μm) LOA (μm) Range (μm) R Values P Value
Zeiss Cirrus Optopol Copernicus 101.2 −29.6/232.1 261.7 −0.2 .01
Zeiss Cirrus Optovue RTVue-100 10.1 −59.3/79.6 138.9 NS NS
Zeiss Cirrus Heidelberg Spectralis −30.3 −141.8/81.2 223 −0.433 <.001
Zeiss Cirrus Zeiss Stratus 60.5 −25.3/146.4 171.7 NS NS
Zeiss Cirrus Topcon 3D OCT 39.8 −41.7/121.4 163.1 NS NS
Optopol Copernicus RTVue-100 −91 −217.2/35.1 252.3 0.199 .04
Optopol Copernicus Heidelberg Spectralis −131.5 −305.9/42.8 348.7 NS NS
Optopol Copernicus Zeiss Stratus −40.6 −175.4/94.1 269.5 0.295 .002
Optopol Copernicus Topcon 3D OCT −61.3 −181.4/58.7 240.1 NS NS
Optovue RTVue-100 Heidelberg Spectralis −40.4 −163.8/82.8 246.6 −0.353 .001
Optovue RTVue-100 Zeiss Stratus 50.4 −29.1/129.9 159 NS NS
Optovue RTVue-100 Topcon 3D OCT 29.6 −51.1/110.5 161.6 NS NS
Heidelberg Spectralis Zeiss Stratus 90.8 −37.4/219.2 256.6 0.449 <.001
Heidelberg Spectralis Topcon 3D OCT 70.1 −54.1/194.4 248.5 0.272 .004
Zeiss Stratus Topcon 3D OCT −20.1 −110.5/69 179.5 −0.257 .007

Only gold members can continue reading. Log In or Register to continue

Related posts:

  1. Oxygen Saturation in Central Retinal Vein Occlusion
  2. Reply
  3. Spontaneous Reattachment of Descemet Stripping Automated Endothelial Keratoplasty Lenticles: A Case Series of 12 Patients
  4. Associations of Birth Weight With Ocular Biometry, Refraction, and Glaucomatous Endophenotypes: The Australian Twins Eye Study

Stay updated, free articles. Join our Telegram channel
Tags:
Jan 17, 2017 | Posted by in OPHTHALMOLOGY | Comments Off on Reproducibility of Retinal Thickness Measurements on Normal and Pathologic Eyes by Different Optical Coherence Tomography Instruments

Full access? Get Clinical Tree
Get Clinical Tree app for offline access