Clinic-Based Eye Disease Screening Using Non-Expert Fundus Photo Graders at the Point of Screening: Diagnostic Validity and Yield





Purpose


The intent of this study was to determine the diagnostic accuracy of several diagnostic tests for age-related macular degeneration (AMD), diabetic retinopathy (DR), glaucoma, and cataract, as well as the proportions of patients with eye disease from each of 3 enrolling clinics.


Design


Diagnostic accuracy study.


Methods


Patients ≥50 years old in a diabetes, thyroid, and general medicine clinic were screened using visual acuity, tonometry, and fundus photography. Photographs were graded at the point-of-screening by non-ophthalmic personnel. Participants with positive screening test results in either eye and a 10% random sample with negative results in both eyes were referred for an in-person, reference-standard ophthalmology examination.


Results


Of 889 participants enrolled, 229 participants failed at least 1 test in either eye, of which 189 presented for an ophthalmic examination. An additional 76 participants with completely normal screening test results were referred for examination, of which 50 attended. Fundus photography screening had the highest yield for DR (sensitivity: 67%; 95% confidence interval [CI]: 39%-87%), visual acuity screening for cataract (sensitivity: 89%; 95% CI: 86%-92%), and intraocular pressure screening for glaucoma or suspected glaucoma (sensitivity: 25%; 95% CI: 14%-40%). The burden of disease was relatively high in all 3 clinics, with at least 1 of the diseases of interest (ie, AMD, DR, glaucoma or suspected glaucoma, or cataract) detected in 25% of participants (95% CI: 17-35%) from the diabeteses clinic, 34% (95% CI: 22%-49%) from the thyroid clinic, and 21% (95% CI: 13%-32%) from the general clinic.


Conclusions


Non-expert eye disease screening in health clinics may be a useful model for detection of eye disease in resource-limited settings.


An estimated 338 million individuals worldwide have moderate-to-severe visual impairment or blindness resulting from cataract, glaucoma, undercorrected refractive error, age-related macular degeneration (AMD), or diabetic retinopathy (DR). , Most of these visual impairments occur in the developing world, where non-communicable eye diseases such as AMD, DR, and glaucoma are becoming increasingly more prevalent causes of blindness.


A shortage of eye care providers in resource-poor settings limits access to eye care and delays diagnoses of eye disease. However, recent advances in portable, non-invasive imaging and diagnostic technologies have made it possible for non-ophthalmologist technicians to screen for eye disease. Such task-shifting, in which lower-level personnel perform simpler tasks and ophthalmologists are reserved for high-level medical decision making and surgery, is often cited as a crucial mechanism for prevention of blindness in low- and middle-income countries, but relatively few studies have assessed its implementation in a rigorous way.


Fundus photographs captured by non-ophthalmologists could be used to detect a host of diseases including AMD, glaucoma, and DR. However, fundus photographs require interpretation. Such interpretation would usually be done remotely at a grading center but could also be done at the point-of-care by an automated software program or by the non-expert performing the test. , Each of those approaches has advantages and disadvantages. The quality control systems in place at a remote reading center enhance the validity of diagnoses, but the diagnosis is delivered after the screening visit has ended, creating challenges for patient notification. Automated algorithms for DR are fast and consistent, but readily available software is lacking for AMD and glaucoma. , Non-expert image interpretation can be done quickly but is likely subject to more interexaminer variability than a reading center.


Despite the promising advances in the field of mobile retinal imaging, the optimal methods and setting for an eye screening program have not been established. Ideally, an accurate, inexpensive, and rapid screening test would be used in a population at relatively high risk for disease. However, few studies have assessed the validity of screening tests in a real-world setting or have compared the diagnostic yield of screening in different populations, especially in a resource-limited setting. , To begin to address those gaps in knowledge, this study instituted an eye disease screening program at a general medical clinic, a diabetes clinic, and a thyroid clinic at Chiang Mai University (CMU) Hospital in northern Thailand. This study assessed the diagnostic accuracy of the screening tests and the burden of eye disease discovered by screening in each of the clinics.


METHODS


Ethical Approval


This prospective diagnostic accuracy study was conducted with approval from the Committee on Human Research at the University of California San Francisco and the CMU Faculty of Medicine Research Ethics Committee. The study adhered to the tenets of the Declaration of Helsinki. All participants provided written informed consent.


General Study Design


Participants waiting for their routine outpatient medical clinic appointments were invited to participate in an eye disease screening program consisting of visual acuity assessment, intraocular pressure (IOP) measurement, and fundus photography. Participants meeting prespecified referral criteria were referred for a comprehensive eye examination, as were a random sample of participants with normal test results.


Study Setting


The study took place at outpatient medical and ophthalmology clinics of Maharaj Nakorn CMU, a university-based tertiary medical center in northern Thailand. No screening programs for AMD, DR, or glaucoma were in place at the hospital or in the community at the time of the study.


Study Participants


A consecutive series of patients ≥50 years old presenting for routine care at a diabetes clinic, a thyroid clinic, and a general medicine clinic at CMU hospital were approached by nursing staff and offered eye screening examinations while they waited to see their doctor. Clinic nurses determined eligibility from the list of patients who had checked in to clinic that day and offered eligible patients the ability to participate during the check-in process. Because the main goal of the study was to identify undiagnosed eye disease, patients who had received an eye examination within the past 1 year were excluded.


Screening Tests


Three screening tests were performed. First, visual acuity (VA) testing was performed. Participants read a single 5-character line of numbers from an Early Treatment Diabetic Retinopathy Study (ETDRS) VA chart, with the optotype size and testing distance corresponding to 20/60 vision. Each eye was tested separately with distance correction if available. If more than 1 number was read incorrectly, then pinhole VA was elicited. Second, IOP was estimated for each eye by using an iCare tonometer (ic100 model; iCare Tonometers, Finland). Finally, nonmydriatic fundus photography of each eye was performed using a nonmydriatic fundus camera (Nonmyd 7; Kowa Co., Tokyo, Japan), using the internal fixation target to capture separate images of the macula and optic nerve. Fundus photographs were graded at the time of screening by a trained research assistant for a simple metric of image quality (ie, gradable vs. ungradable), as well as 4 key features, including 1) vertical cup-to-disk ratio (VCDR) in 0.05-step increments; 2) disk abnormality, defined as a notch, hemorrhage, or retinal nerve fiber layer defect; 3) the presence of any DR according to the International Clinical Diabetic Retinopathy Severity Scale ; and 4) the presence of AMD, defined according to a modification of the AREDS (Age-Related Eye Disease Study) system as ≥1 small druse within 1 disk diameter of the fovea, geographic atrophy, or choroidal neovascularization. In addition to those 4 features, the operators could also note the presence of other retinal pathologies. Participants were not charged for the screening visit.


Referral Criteria


Participants were referred for a comprehensive ophthalmologic examination if any of the screening test results were abnormal for either eye according to predefined referral criteria ( Figure 1 ). Fundus photographs deemed ungradable for any of the 4 key features described above were considered to show abnormal results, which triggered referral. In addition, a 10% random sample of participants with normal screening test results for both eyes was referred, implemented through the randomization module in a Research Electronic Data Capture (REDCap) tool using a random sequence generated using the rand function in Excel software (Microsoft, Redmond, Washington, USA). Participants referred for an eye examination were given an appointment date before being discharged from the screening visit.




Figure 1


Referral criteria for screening tests. Participants meeting any of these criteria in either eye were referred for a reference standard ophthalmologic examination.


Referral Examinations


An experienced ophthalmologist masked to the screening test results performed a comprehensive eye examination for all referred participants by using a standardized form. The examination consisted of VA assessment, IOP assessment, slit lamp biomicroscopy, and indirect ophthalmoscopy through a dilated pupil. Those in whom glaucoma was suspected, defined as an IOP >21 mm Hg by Goldman applanation tonometry or a suspicious optic nerve on examination (ie, VCDR >0.6, disk notch, disk hemorrhage, or retinal nerve fiber layer defect) underwent visual visual field assessment with a Humphrey field analyzer (Zeiss, Jena, Switzerland)u \using the 24-2 SITA fast algorithm. Other ancillary testing was performed at the discretion of the examining ophthalmologist. The participant was not charged for the referral examination or for any ancillary testing performed at the time of the referral examination. Participants who received a diagnosis of disease received treatment at the same subsidized cost as was standard for any Thai citizen.


Quality Control of Fundus Photography Examination


The research assistants who captured the fundus images performed the photo grading in real time at the point of screening. One of the assistants was a medical student who learned fundus photography for this study, and the other assistant had experience capturing but not interpreting fundus photographs for previous studies. , Each assistant underwent the same 2-day training program designed to allow differentiation of a normal fundus photograph from an abnormal one and identification of the most common findings of DR (eg, microaneurysms, retinal hemorrhages, cotton wool spots, neovascularization) and AMD (eg, drusen, geographic atrophy, choroidal neovascularization). All photographs classified as showing abnormal findings and a 10% random sample of photographs classified as showing normal findings were graded by an ophthalmologist for the first 2 months of the study. The ophthalmologist communicated results as well as remedial education by email. Agreement among the research assistants and the ophthalmologist was believed to be sufficient after 2 months (Cohen’s κ = 0.63 for the outcome of any referable condition), although agreement for specific conditions was slightly lower (eg, Cohen’s κ = 0.59 for DR; Cohen’s κ = 0.53 for AMD).


Statistical Analysis


The sensitivity, specificity, negative predictive value (NPV), and positive predictive value (PPV) of each screening test (ie, VA, IOP, and fundus photograph interpretation) was estimated relative to an ophthalmologic diagnosis of AMD, DR, or glaucoma, with the glaucoma designation including both suspected glaucoma and perimetrically confirmed glaucoma. Because only a random sample of patients passing all tests were referred for an ophthalmic examination, the sensitivity and specificity could not be directly computed. Instead, the data were treated as survey data, weighted by the product of sampling weights and nonresponse weights. Sampling weights were based on the numbers of participants in the 2 sampling strata (ie, those whose vision failed any of the screening tests, 100% of whom were referred vs. those whose vision passed all tests, 10% of whom were referred); nonresponse propensity score weights were derived from a logistic regression assessing whether a referred participant followed up with the ophthalmic examination, using all factors in Supplemental Table 1 as explanatory variables. The main analyses were performed at the person level, assessing diagnostic metrics for a positive test result or ophthalmic diagnosis in either eye. A similar eye-level analysis was performed to assess the diagnostic accuracy of specific photo grades for the different eye diseases. Statistical analysis was performed using R version 4.0 software (R Foundation for Statistical Computing, Vienna, Austria).


RESULTS


From September 2017 to July 2019, clinic logs listed 15,479 encounters with female patients and 7,354 encounters with male patients. From those encounters, 889 participants were enrolled in the study, including 253 from the diabetes clinic, 317 from the thyroid clinic, and 319 from the general medical clinic. The median age was 61 years old (interquartile range [IQR]: 55, 67 years), and 641 were female (72%); clinic-stratified characteristics are shown in Supplemental Table 1.


Participant flow is shown in Supplemental Figure 1. In total, 229 of 889 participants (26%) had vision that failed at least 1 test. The highest number came from the diabetes clinic (84 of 253; 33%), followed by the thyroid clinic (82 of 317; 26%), and then the general medical clinic (63 of 319; 20%). Fundus photography was the most likely test to result in referral, with 135 participants (15%) judged to have 1 or more abnormalities. A Venn diagram shows the distribution of participants failing each of the tests (Supplemental Figure 2). Participants whose vision failed a test tended to fail only a single test, with the most overlap coming from the 47 participants (ie, 21% of the 229 participants with a positive test result) who were referred based on both vision test and fundus photographs. Eye-level results of the screening tests are summarized in Table 1 . One participant from the general medicine clinic had an ocular prosthesis in 1 eye; that eye was excluded from eye-level analyses, resulting in a total of 1,777 eyes across all 3 clinics.



Table 1

Eye-Level Results of Screening Tests


















































































































Diabetes Clinic 506 Eyes Thyroid Clinic 634 Eyes General Clinic 637 Eyes
Any screen positive 139.0 (27) 126.0 (20) 96.0 (15)
Worse than 20/60 a 70.0 (14) 64.0 (10) 52.0 (8)
IOP >22 mm Hg 14.0 (3) 23.0 (4) 3.0 (0.5)
Fundus photography
Disk notch/heme
Present 4.0 (1) 10.0 (2) 7.0 (1)
Ungradable 16.0 (3) 6.0 (1) 8.0 (1)
VCDR > 0.6
Present 11.0 (2) 14.0 (2) 5.0 (1)
Ungradable 20.0 (4) 9.0 (1) 4.0 (1)
AMD
Present 5.0 (1) 6.0 (1) 14.0 (2)
Ungradable 27 (5) 8 (1) 15 (2)
DR
Present 38.0 (8) 23.0 (4) 12.0 (2)
Ungradable 25.0 (5) 8.0 (1) 12.0 (2)
Other retinal pathology
Present 6.0 (1) 10.0 (2) 9.0 (1)
Any photo abnormality
Present 59.0 (12) 52.0 (8) 43.0 (7)
Ungradable 29.0 (6) 10.0 (2) 13.0 (2)

Only gold members can continue reading. Log In or Register to continue

Stay updated, free articles. Join our Telegram channel

Jul 10, 2021 | Posted by in OPHTHALMOLOGY | Comments Off on Clinic-Based Eye Disease Screening Using Non-Expert Fundus Photo Graders at the Point of Screening: Diagnostic Validity and Yield

Full access? Get Clinical Tree

Get Clinical Tree app for offline access