PARTICIPANTS

The participants were selected from an ongoing Asan Glaucoma Progression Study. All participants initially underwent a complete ophthalmic examination, including a review of medical history, measurement of best-corrected visual acuity, manifest refraction, slit lamp biomicroscopy, Goldmann applanation tonometry, gonioscopy, funduscopic examination, stereoscopic optic disc photography, retinal nerve fiber layer photography, and a VF test (Humphrey field analyzer; Swedish Interactive Threshold Algorithm 24-2; Carl Zeiss Meditec). All participants were diagnosed with primary open-angle glaucoma, which was defined as follows: normal anterior chamber with open angle on gonioscopic examination; glaucomatous optic nerve head with diffuse or focal neural rim thinning; a difference in the vertical cup-to-disk ratio >0.2 between both eyes; and retinal nerve fiber layer defects of disc hemorrhage accompanied by a glaucomatous VF defect in the corresponding area. The following glaucomatous VF defects were defined: (1) a cluster of 3 points with a probability <5%, of which at least 1 point is required to have a probability <1% on a pattern deviation map; (2) a glaucoma hemifield test result outside 99% of the age-specific normal limit; and (3) a pattern standard deviation (PSD) outside 95% of the normal limit confirmed by at least 2 reliable consecutive VF tests (false positive error <15%; a false negative error <15%; and a fixation loss <20%). We selected patients who had regularly undergone reliable standard 24-2 perimetry tests with the Swedish Interactive Threshold Algorithm using Humphrey Field Analyzer (Carl Zeiss Meditec) at least 7 times for at least 4 years between May 1998 and May 2020. After excluding patients aged 30 years or younger at the time of the first test, our pool of VFs included 96,542 tests from 9212 eyes of 6047 patients with an average follow-up of 9.5 years. All procedures conformed to the Declaration of Helsinki, and this study was approved by the institutional review board of the Asan Medical Center, University of Ulsan, Seoul, Korea.

PREPARING DATA FOR DEEP LEARNING

Before constructing the CNN model, we prepared data in a stepwise manner:

• 1.
  

  From a total of 9212 eyes, 20% of the eyes (1843 eyes from 1210 patients) were randomly assigned to the test group, whereas the remaining 80% (7369 eyes from 4837 patients) were assigned to the training group. We have used clustered sampling to ensure that both eyes of the patient are assigned to the same group. The training group was used exclusively to train the CNN model, whereas the test group was used exclusively to measure the performance of the model.

  
  
• 2.
  

  For the input, we converted VFs to 8 × 9 tensors by padding with zeros to preserve spatial characteristics. In addition, to incorporating chronological features, we stacked 3 consecutive VFs in the shape of 8 × 9 × 3 while maintaining sequential information, which was then fed to the CNN. We later named this 3D tensor as a “VF-block.” VF-blocks were allowed to overlap, hence, for the eye having N distinct consecutive VFs, N − 2 VF-blocks have been constructed.

  
  
• 3.
  

  In supervised NNs, a goal of training, a “label,” should be defined by the researcher. In our case, the primary label is a binary value with 1 indicating glaucomatous VF progression and 0 indicating no VF progression. The labeling process is described in detail in the following section.

  
  
• 4.
  

  From the last test of the VF-block, a subsequent 3-year period was defined as a “surveillance period,” while the period after the surveillance period up to 6 years was defined as a “confirmatory period.” Only VF-blocks with at least 2 VFs in the surveillance period and at least 1 VF in the confirmatory period were included in the analysis. For all VF-blocks, VFs beyond 6 years from the last test of the VF-block were dropped.

  
  
• 5.
  

  A VF-block was labeled as 1 when the progression happened within the surveillance period and was sustained in the confirmatory period.

  
  
• 6.
  

  Given that the average follow-up time is much longer than the surveillance period (9.5 vs 3 years), a large portion of VFs would be unused if only 1 VF-block per eye were used in the analysis. Hence, for each eye, we used all unique VF-blocks with at least 2 VFs in the surveillance period and 1 VF in the confirmatory period. As a result, 43,260 labeled VF-blocks were created: 34,467 in the training group and 8793 in the test group ( Figure 1 ).

  
  

  
  

  
  

  

  
  

  
  

  Defining VF-blocks in the examination eye. Three VF-blocks based on 8 × 9 tensor: VF-block #1, VF-block #2, and VF-block #3. Blocks #1, 2, and 3 were valid (A-C), whereas the fourth VF block (D) was excluded because there was no VF to confirm the progression. The construction of a 3D tensor from individual VFs constituting VF-block #1 (A) is shown in E. VF = visual field.

  
  

DETERMINATION OF GLAUCOMATOUS VISUAL FIELD PROGRESSION USING 3 DIFFERENT CRITERIA

• A.
  

  The following trend-based criteria were used to determine glaucomatous VF progression:

  
  
  
  
  • 1.
    

    First, we constructed a “standard dataset” by randomly selecting 2000 eyes from 7369 eyes of the training group. After excluding eyes with a minimum value of MD of ≤ −16 dB throughout the entire follow-up, the standard group was found to have a baseline age of 54.3 years; an MD of −3.27 dB; a PSD of 3.81 dB; and a VFI of 92.34%.

    
    
  • 2.
    

    From the calculated rate of deterioration of the global indices (MD and VFI) in the standard dataset, we derived cutoff values (a statistically significant slope <0 at P < .05 for a 2-tailed test) of −0.49 dB/year for MD and −1.55%/year for VFI. ^,

    
    
  • 3.
    

    In NN models, a VF-block was labeled as “progressed” if deterioration exceeding the cut-off values was observed during the surveillance period (3 years) and if the deterioration was not reversed in any of the VFs during the confirmation period.

  
  
  
  
• B.
  

  Event-based criteria were used to determine VF progression as follows:

  
  
  
  
  • 1.
    

    From the standard dataset, we calculated the probability of a change in glaucoma pattern for all points excluding the blind spot. As intertest variation is known to be associated with the initial deviation from normal age-corrected sensitivity, we divided all points of all eyes into multiple levels based on the initial age-corrected sensitivity from 0 dB to −30 dB in 2-dB steps. The average variability was calculated for all 52 VF points and each initial sensitivity level. If the average variability of a specific point-level was unobtainable, imputation was performed with an exponentially weighted moving average (EWMA) with a window of 5 sensitivity levels (2 above and 2 below). The average variability of all 52 points was smoothed with EWMA along the sensitivity ax is with a window of 3.

    
    
  • 2.
    

    A probability map of the pattern deviation-based changes was calculated for all VFs in the surveillance and confirmation periods of a given VF-block. Progression was determined based on modified EMGT (Early Manifest Glaucoma Trial) criteria: at least 3 significantly progressing points ( P < .05) at the same location in at least 2 consecutive test results during the surveillance period and in at least 1 test result during the confirmation period. If the progression was reversed in any of VFs in the confirmation period, the VF-block was labeled as “nonprogressed.”

  
  
  
  
• C.
  

  The combined criterion was defined as one fulfilling both trend-based and event-based criteria for glaucomatous VF progression.

“BASIC” NEURAL NETWORK MODEL

We constructed a basic NN model with 3 VF-blocks of a convolutional network composed of a 3D convolution layer, a batch normalization layer, and rectified linear unit activation layer followed by 2 fully connected layers with the final activation function being a sigmoid function. Binary crossentropy was used as a loss function, and stochastic gradient descent with momentum was used as an optimization algorithm. For the validation, 10-fold crossvalidation was used with the training dataset divided into 10 subsets at the eye level.

To improve the performance of the model, we examined the accuracy of the models by adding the age of the patient for each test, an interim from the first test and laterality to the VF-blocks. Variables were concatenated to the flattened tensor before the second fully connected layer. We examined all possible combinations of additional inputs; however, none of them improved the accuracy of the model ( Figure 2 , A).

Schematic diagram shows the neural network models. The basic model consists of convolution layers followed by a fully connected layer and outputs a binary value. A. The enhanced model is a 2-stage model: the first stage includes 3 preliminary models consisting of convolution layers followed by a fully connected layer and deconvolution layers to output an 8 × 9 tensor. In a second stage, outputs from preliminary models are concatenated to the original VF-block to form an 8 × 9 × 6 tensor, which is passed through convolution layers followed by a fully connected layer to output a binary value (B).

“ENHANCED” NEURAL NETWORK MODEL

We explored another strategy to improve the model by incorporating more interim information into the basic model. We created 3 separate NN models (“preliminary” models) aimed at predicting future VF defects from the VF-blocks ( Figure 2 , B).

For each eye in the training group, we built every possible combination of a VF-block and a single VF in the surveillance period divided into 3 periods of equivalent length: a VF-block paired with a VF examination performed in the first year of the surveillance period (44,008 pairs comprising the “first-year set”); the second year of the surveillance period (54,447 pairs comprising the “second-year set”); and the third year of the surveillance period (46,095 pairs comprising the “third-year set”). Single VFs paired to VF-blocks were used as labels for training preliminary NN models.

Preliminary NN models have the same structure as the basic model with additional transposing layers (deconvolution) and the linear final activation function to match the different shape and values of a different label, an 8 × 9 tensor and values in decibels. Furthermore, instead of binary crossentropy, the mean absolute error was used as a loss function. The output from 3 preliminary NN models are predictions of glaucomatous VF progression derived from a given VF-block. The model trained with the first-year set predicts glaucomatous VF progression after ~0.5 year; the model trained with the second-year set predicts glaucomatous VF progression after ~1.5 years; and the model trained with the third-year set predicts glaucomatous VF progression after ~2.5 years.

The enhanced NN model has fundamentally the same structure as the basic model but takes as an input a concatenated tensor of VF-block and outputs from 3 preliminary models, forming an 8 × 9 × 6 tensor. As in the basic model, 3 blocks of 3D convolution, batch normalization, and rectified linear unit activation layer were followed by 2 fully connected layers.

LINEAR MODELS

We constructed 2 linear regression models using ordinary least squares with VF-blocks from the test group: 1 model with linear regression on the global indices and the other on PLR. From the global indices model, we predicted MD and VFI at 3 years, and progression was defined as deterioration in MD or VFI exceeding cutoff values (−0.49 dB/year for MD and −1.55%/year for VFI). Progression in the PLR model was defined as at least 1 point progressing faster than −1.0 dB/year with P ≤ .05. ^, For the convenience of comparison, AUROC values were calculated from DOR (diagnostic odds ratio), assuming homogeneous dependence of the test accuracy on the threshold.

STATISTICAL ANALYSIS

Tensor flow version 2.3 ( https://www.tensorflow.org/ ) and Keras (version 2.4: https://keras.io/ ), running in the Python (version 3.7: https://www.python.org/ ) environment were used for NN training. During the training of NNs, a class-weight attribute of fit method in the Keras models application programming interface. All other statistical analyses were performed with SAS version 9.4 software (SAS Institute). For statistical comparison, the χ ² test was used for categorical variables, and the independent Student t -test was used for continuous variables. Comparisons between AUROC values were performed using a method proposed by DeLong and associates.