Since Helmholtz’s pioneering invention of the ophthalmoscope in the 19th century, direct and indirect ophthalmoscopy serve as the standard methodology for the diagnostic assessment and management of retinal diseases.¹ As image acquisition instrumentation evolved, technologies such as echography and ultrawidefield fundus photography have emerged to assist or, in some cases, supplant standard funduscopic examination. Concurrently, ongoing advances in artificial intelligence (AI) have prompted great interest in automated retinal image analysis in clinical and research settings. Ophthalmology, and specifically the field of retina, has the opportunity to capitalize on the offerings of AI given the myriad clinical data and multimodal imaging routinely performed. This brief review will address how AI has been implemented in the field of retina and its potential applications in the screening and management of retinal diseases.

ARTIFICIAL INTELLIGENCE AND DEEP LEARNING

Artificial intelligence broadly refers to the field of computer science concerned with a computer’s ability to carry out complex tasks synonymous with human performance, including visual processing, pattern recognition, and decision making.² Machine learning (ML), a subset of AI, includes the computer system “learning” associations between the input and output data provided and subsequently editing its coding to make enhanced predictions about new data.^2,3

Deep learning is a subset of machine learning that distinguishes itself from traditional ML by the type of data it uses and methodology the computer system learns. Deep learning eliminates a considerable amount of the predefined data that is commonly employed in machine learning, which in turn permits the usage of unstructured data and limits the dependency of human input.⁴ Additionally, in deep learning, “hidden layers” are added between the input and output layers that permit a more intricate evaluation of the input data. Thus, the deep learning system can independently formulate decisions or associations, making decisions often invisible to human experts. Through the combination of input data and different weights and biases determined by the system, deep learning neural networks make novel yet accurate perceptions, classifications, and delineations of various data sets.⁴

DIABETIC RETINOPATHY

Given the increasing burden of diabetic retinopathy (DR) and its far-reaching public health and societal implications, DR screening has been a major focus of AI efforts. Despite being the leading cause of vision loss among working-age adults in the United States, only 40% of diabetic patients obtain their annual recommended screening.⁵ Artificial intelligence screening tools may come to address existing access/resource gaps and to better delineate patients who have referrable disease and require treatment from the wider population pool. A 2016 pivotal study by Google Inc. highlighted the potential of deep learning in DR screening by showing that referable DR could be identified from a single fundus photo with a sensitivity of 97.5% and specificity of 98.5%.⁶ Since then, autonomous AI systems have been developed and approved for DR screening. In April 2018, the IDx-DR (IDx) became the first US Food and Drug Administration approved fully autonomous AI-based medical device to detect “more than mild diabetic retinopathy” (mtmDR).⁷ This system uses 45-degree fundus photos, which are uploaded and analyzed using cloud-based software for the detection of mtmDR. The images can be taken during the patient’s primary care appointment and the patient referred to an ophthalmologist if mtmDR is detected. In the pivotal prospective study using this system, sensitivity was 87.2% and specificity was 90.7% for the detection of mtmDR.⁸

Additional systems including the iGradingM (Medalytix/Emis Health), EyeArt (EyeNuk Inc.), and Retmarker (Retmarker Ltd.) have been approved in Europe for the automated screening of DR using fundus photography. In the assessment of 102,856 fundus photos of 20,258 patients, the EyeArt and Retmarker systems achieved sensitivities of 93.8% and 85.0%, respectively, for referable retinopathy, and sensitivities of 99.6% and 97.9% for proliferative disease, respectively.⁹ Both the Retmarker and EyeArt systems have been validated to have acceptable sensitivity to capture referable retinopathy when compared to human graders, potentially making them a cost-effective alternative to manual grading alone.⁹

In the pivotal clinical study using the EyeArt system, a sensitivity of 96% and specificity of 88% were found for detection of mtmDR.¹⁰ This system gained FDA clearance in the United States for automated DR screening in August 2020.

Other imaging modalities, such as optical coherence tomography (OCT) and OCT angiography (OCTA), are actively being investigated for DR screening and management.¹¹ Prognostic models have been developed to predict retinal response to anti-VEGF treatments in patients with macular edema through the analysis of OCT images.¹²

RETINOPATHY OF PREMATURITY

Substantial challenges exist in the screening and diagnosis of retinopathy of prematurity given the disease’s clinical variability and limitations to access to trained screening specialists.^13-15 Management decisions hinge on the location, stage of vascular findings, as well as the presence of plus disease. Given that the Early Treatment of ROP study identified plus disease as one of the most important parameters for identifying treatment-level ROP, great emphasis has been placed on identifying it on screening examinations.¹⁶ Creating a standardized ROP screening system that is both reliable and repeatable has become a major goal, and computer-based image analysis stands to make a significant impact.

Multiple algorithms have demonstrated promise for detecting plus or pre-plus disease with achieved accuracies of 95% while outperforming human ROP experts evaluating the same data set.^17,18 The DeepROP system incorporates both ROP zone and stage into its classification model and grades images as either normal, minor ROP, or severe ROP.¹⁹ The i-ROP system is capable of categorizing fundus photos into type 1, type 2, and pre-plus ROP with probability scores of 0.96 and 0.91 for detecting type 1 ROP and clinically significant ROP, respectively.²⁰ The i-ROP score has been shown to be noninferior to human diagnosis when identifying vascular changes in pre-plus and plus disease.¹⁷ These programs demonstrate that deep learning may minimize the interobserver variability that has challenged ROP screening and play a continued role in the screening, particularly in resource-limited settings.

AGE-RELATED MACULAR DEGENERATION

Timely detection and treatment of AMD, specifically neovascular AMD, often leads to better visual outcomes.²¹ In-office examinations, as well as home monitoring tools such as the Amsler grid and portable devices (Foresee Preferential Hyperacuity Perimeter; Reichert Technologies), have customarily been employed for detecting AMD progression. Although AI DR screening relies primarily on fundus photography, AI systems in the context of AMD have focused heavily on OCT images. Trained neural networks have demonstrated strong accuracy in the differentiation of OCT images of normal and AMD patients, with a sensitivity and specificity of 92.64% in normal patients and 93.69% in AMD patients.²² In the context of diagnosing exudative AMD, AI had upwards of 91.0% accuracy and 95.5% accuracy for predicting the need for injection treatment.^22,23 Software has also reliably detected subretinal and sub-RPE fluid with high accuracy.²⁴ Fundus image models have also demonstrated promising results. The DeepSeeNet program performed better than retina specialists in the accuracy (0.671 vs 0.599), sensitivity (0.590 vs 0.512), and specificity (0.930 vs 0.916) in classifying eyes based on the AREDS severity score.²⁵ The program has also demonstrated the capability to identify geographic atrophy with an accuracy comparable to human graders.²⁶

Artificial intelligence models have also been used in a prognostic context in AMD. A predictive model using OCT features and demographic factors of 495 fellow eyes from the HARBOR trial, differentiated converting vs nonconverting eyes with a performance of 0.68 and 0.80 for the development of choroidal neovascularization and geographic atrophy, respectively.²⁷ Models have also been developed with the purpose of predicting visual acuity response to treatment with anti-VEGF therapy using baseline OCT images.^27-29

OTHER DISEASES

Sickle cell disease is one of the most common inherited genetic diseases with various ocular manifestations in the retina, warranting careful monitoring and treatment. OCTA features such as blood vessel diameter and tortuosity, vessel perimeter index, foveal avascular zone area, contour irregularity, and parafoveal avascular density have been used to train algorithms to identify retinopathy with an average accuracy of 95%.^30,31 Algorithms have also been able to differentiate between mild sickle-cell retinopathy (stage II) and severe sickle-cell retinopathy (stage III) with an accuracy rate of 97%.³¹

Machine learning tools have also been applied to screen for systemic risk factors and disease. Models have accurately determined a patient’s age, sex, smoking status, and systolic blood pressure from a single fundus photograph.³² The same algorithm may also predict a patient’s 5-year risk of developing a major adverse cardiac event.³²

LIMITATIONS AND CONCLUSION

Although the diagnostic accuracy of AI programs is impressive, algorithms may have a relatively high false-positive rate of detection, which may necessitate unneeded referrals. However, since these result in clinical examinations, no unnecessary treatments would occur. Screening programs must have high sensitivity in order to be clinically safe. The specificity should be high enough to be clinically useful. Furthermore, development of functional algorithms rely on the quality and the abundance of source data. Homogenous data may lead to biases in the models, particularly when they are generalized and utilized in conjunction with underrepresented populations.³³ Thus, the training set used must be diverse and include various subsets of the population at large to develop an algorithm with widespread applicability. Additionally, despite the common goal of accurately identifying diseases, there currently is no standardized methodology/protocols for image capturing and image analysis algorithms, inherently resulting in variability and usability. Furthermore, the importance of obtaining images with sufficient quality for grading is paramount because systems are unable to access images if below a certain threshold.^8,32,33 The National Institutes of Health through its collaborative community projects is working on these areas of unmet needs for various ophthalmic conditions. Collaboration across countries and organizations as well as extensive data sharing and open-source algorithms will ensure relevant and useful AI systems in the future for screening and determining of treatment prognosis for ophthalmic conditions.

As the capabilities of AI evolve, more commercially available products for not only DR but other diseases will begin to appear. Together with current routine clinical practice, AI and deep learning offer potential avenues to improve clinical efficiency, expand access to care, and ultimately improve the overall quality of care. RP

REFERENCES

1. Keeler CR. The ophthalmoscope in the lifetime of Hermann von Helmholtz. Arch Ophthalmol. 2002;120(2):194-201. doi:10.1001/archopht.120.2.194
2. Armstrong GW, Lorch AC. A(eye): A review of current applications of artificial intelligence and machine learning in ophthalmology. Int Ophthalmol Clin. 2020;60(1):57-71. doi:10.1097/IIO.0000000000000298
3. Consejo A, Melcer T, Rozema JJ. Introduction to machine learning for ophthalmologists. Semin Ophthalmol. 2019;34(1):19-41. doi:10.1080/08820538.2018.1551496
4. IBM Cloud Education. What is deep learning? 2020. Accessed September 27, 2021. www.ibm.com/cloud/learn/deep-learning
5. Flaxel CJ, Adelman RA, Bailey ST, et al. Diabetic Retinopathy Preferred Practice Pattern® [published correction appears in Ophthalmology. 2020 Sep;127(9):1279]. Ophthalmology. 2020;127(1):P66-P145. doi:10.1016/j.ophtha.2019.09.025
6. Gulshan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316(22):2402-2410. doi:10.1001/jama.2016.17216
7. US Food and Drug Administration. FDA permits marketing of artificial intelligence-based device to detect certain diabetes-related eye problems. Press release. 2018. Accessed September 27, 2021. https://www.fda.gov/news-events/press-announcements/fda-permits-marketing-artificial-intelligence-based-device-detect-certain-diabetes-related-eye
8. Abramoff M, Lavin P, Birch M, Sha N, Folk J. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. Npj Digital Medicine. 2018;39.
9. Tufail A, Rudisill C, Egan C, et al. Automated diabetic retinopathy image assessment software: diagnostic accuracy and cost-effectiveness compared with human graders. Ophthalmology. 2017;124(3):343-351. doi:10.1016/j.ophtha.2016.11.014
10. Ipp E, Liljenquist D, Bode B, et al; EyeArt Study Group. Efficacy and safety of the EyeArt artificial intelligence system for detecting diabetic retinopathy: results from a pivotal clinical study. JAMA Open. 2021; In press.
11. Sandhu HS, Eladawi N, Elmogy M, et al. Automated diabetic retinopathy detection using optical coherence tomography angiography: a pilot study. Br J Ophthalmol. 2018;102(11):1564-1569. doi:10.1136/bjophthalmol-2017-311489
12. Rasti R, Allingham MJ, Mettu PS, et al. Deep learning-based single-shot prediction of differential effects of anti-VEGF treatment in patients with diabetic macular edema. Biomed Opt Express. 2020;11(2):1139-1152. Published 2020 Jan 28. doi:10.1364/BOE.379150
13. Fierson WM; American Academy of Pediatrics Section on Ophthalmology; American Academy of Ophthalmology; American Association for Pediatric Ophthalmology and Strabismus; American Association of Certified Orthoptists. Screening examination of premature infants for retinopathy of prematurity [published correction appears in Pediatrics. 2019 Mar;143(3):]. Pediatrics. 2018;142(6):e20183061. doi:10.1542/peds.2018-3061
14. Reynolds JD, Dobson V, Quinn GE, et al. Evidence-based screening criteria for retinopathy of prematurity: natural history data from the CRYO-ROP and LIGHT-ROP studies. Arch Ophthalmol. 2002;120(11):1470-1476. doi:10.1001/archopht.120.11.1470
15. Chiang MF, Jiang L, Gelman R, Du YE, Flynn JT. Interexpert agreement of plus disease diagnosis in retinopathy of prematurity. Arch Ophthalmol. 2007;125(7):875-880. doi:10.1001/archopht.125.7.875
16. Good WV, Hardy RJ. The multicenter study of Early Treatment for Retinopathy of Prematurity (ETROP). Ophthalmology. 2001;108(6):1013-1014. doi:10.1016/s0161-6420(01)00540-1
17. Ataer-Cansizoglu E, Bolon-Canedo V, Campbell JP, et al. Computer-Based Image Analysis for Plus Disease Diagnosis in Retinopathy of Prematurity: Performance of the “i-ROP” System and Image Features Associated With Expert Diagnosis. Transl Vis Sci Technol. 2015;4(6):5. Published 2015 Nov 30. doi:10.1167/tvst.4.6.5
18. Brown JM, Campbell JP, Beers A, et al. Automated Diagnosis of Plus Disease in Retinopathy of Prematurity Using Deep Convolutional Neural Networks. JAMA Ophthalmol. 2018;136(7):803-810. doi:10.1001/jamaophthalmol.2018.1934
19. Wang J, Ju R, Chen Y, et al. Automated retinopathy of prematurity screening using deep neural networks. EBioMedicine. 2018;35:361-368. doi:10.1016/j.ebiom.2018.08.033
20. Redd TK, Campbell JP, Brown JM, et al. Evaluation of a deep learning image assessment system for detecting severe retinopathy of prematurity [published online ahead of print, 2018 Nov 23]. Br J Ophthalmol. 2018;bjophthalmol-2018-313156. doi:10.1136/bjophthalmol-2018-313156
21. Ho AC, Albini TA, Brown DM, Boyer DS, Regillo CD, Heier JS. The potential importance of detection of neovascular age-related macular degeneration when visual acuity is relatively good. JAMA Ophthalmol. 2017;135(3):268-273. doi:10.1001/jamaophthalmol.2016.5314
22. Lee CS, Baughman DM, Lee AY. Deep learning is effective for the classification of OCT images of normal versus Age-related Macular Degeneration. Ophthalmol Retina. 2017;1(4):322-327. doi:10.1016/j.oret.2016.12.009
23. Prahs P, Radeck V, Mayer C, et al. OCT-based deep learning algorithm for the evaluation of treatment indication with anti-vascular endothelial growth factor medications. Graefes Arch Clin Exp Ophthalmol. 2018;256(1):91-98. doi:10.1007/s00417-017-3839-y
24. Chakravarthy U, Goldenberg D, Young G, et al. Automated identification of lesion activity in neovascular age-related macular degeneration. Ophthalmology. 2016;123(8):1731-1736. doi:10.1016/j.ophtha.2016.04.005
25. Peng Y, Dharssi S, Chen Q, et al. DeepSeeNet: A Deep Learning Model for Automated Classification of Patient-based Age-related Macular Degeneration Severity from Color Fundus Photographs. Ophthalmology. 2019;126(4):565-575. doi:10.1016/j.ophtha.2018.11.015
26. Keenan TD, Dharssi S, Peng Y, et al. A deep learning approach for automated detection of geographic atrophy from color fundus photographs. Ophthalmology. 2019;126(11):1533-1540. doi:10.1016/j.ophtha.2019.06.005
27. Schmidt-Erfurth U, Waldstein SM, Klimscha S, et al. Prediction of individual disease conversion in early AMD using artificial intelligence. Invest Ophthalmol Vis Sci. 2018;59(8):3199-3208. doi:10.1167/iovs.18-24106
28. Schmidt-Erfurth U, Bogunovic H, Sadeghipour A, et al. Machine learning to analyze the prognostic value of current imaging biomarkers in neovascular age-related macular degeneration. Ophthalmol Retina. 2018;2(1):24-30. doi:10.1016/j.oret.2017.03.015
29. Aslam TM, Zaki HR, Mahmood S, et al. Use of a neural net to model the impact of optical coherence tomography abnormalities on vision in age-related macular degeneration. Am J Ophthalmol. 2018;185:94-100. doi:10.1016/j.ajo.2017.10.015
30. Alam M, Le D, Lim JI, Chan RVP, Yao X. Supervised machine learning based multi-task artificial intelligence classification of retinopathies. J Clin Med. 2019;8(6):872. Published 2019 Jun 18. doi:10.3390/jcm8060872
31. Alam M, Thapa D, Lim JI, Cao D, Yao X. Computer-aided classification of sickle cell retinopathy using quantitative features in optical coherence tomography angiography. Biomed Opt Express. 2017;8(9):4206-4216. Published 2017 Aug 25. doi:10.1364/BOE.8.004206
32. Poplin R, Varadarajan AV, Blumer K, et al. Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning. Nat Biomed Eng. 2018;2(3):158-164. doi:10.1038/s41551-018-0195-0
33. Beil M, Proft I, van Heerden D, Sviri S, van Heerden PV. Ethical considerations about artificial intelligence for prognostication in intensive care. Intensive Care Med Exp. 2019;7(1):70. Published 2019 Dec 10. doi:10.1186/s40635-019-0286-6
34. van der Heijden AA, Abramoff MD, Verbraak F, van Hecke MV, Liem A, Nijpels G. Validation of automated screening for referable diabetic retinopathy with the IDx-DR device in the Hoorn Diabetes Care System. Acta Ophthalmol. 2018;96(1):63-68. doi:10.1111/aos.13613
35. Verbraak FD, Abramoff MD, Bausch GCF, et al. Diagnostic accuracy of a device for the automated detection of diabetic retinopathy in a primary care setting. Diabetes Care. 2019;42(4):651-656. doi:10.2337/dc18-0148