National use of artificial intelligence for eye screening in Singapore

• Main Authors: GUNASEKERAN, Dinesh Visva, MILLER, Steven, HSU, Wynne, LI, Mong, WONG, Tym Hon, LEE, Mun Tuck, LAMOUREAU, Ecosse, TING, Daniel Shu Wei, TAN, Gavin Siew Wei, WONG, Tien-Yin
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2024
• Subjects: Analytical, Diagnostic and Therapeutic Techniques and Equipment; Artificial Intelligence and Robotics; Health Information Technology
• Online Access: https://ink.library.smu.edu.sg/sis_research/9566; https://ai.nejm.org/stoken/default+domain/VU74HZSFIAKDE3DQH6HI/full?redirectUri=doi/full/10.1056/AIcs2400404
• Institution: Singapore Management University

Diabetes is a major health care challenge, affecting 10% of the global population. One third of patients with diabetes have an ocular complication known as diabetic retinopathy (DR). DR progression to manifestations such as vision-threatening diabetic retinopathy (VTDR) remains the leading cause of blindness in working-aged adults. Yearly DR screening is a universally recommended practice in primary care settings for patients with diabetes, but it is often difficult to implement due to a lack of staffing and screening capacity in primary care. This case study highlights our experience with developing a medical artificial intelligence (AI) software-as-a-medical-device (SaMD) solution for DR screening and implementing it at a national level to provide the capacity needed for DR screening in Singapore. Our approach involved two broad phases. First, we established a national telemedicine screening program, Singapore Integrated Diabetic Retinopathy Program (SiDRP), for population screening of DR in primary care run by trained, nonclinician human graders. Second, we deployed a deep learning–based AI solution, Singapore Eye Lesion Analyzer (SELENA+), into the SiDRP to scale-up the DR screening process by the human graders. We demonstrated the cost-effectiveness of this solution, and obtained medical device regulatory approval for clinical use in health care settings. We report the prospective evaluation of SELENA+ in SiDRP using real-world pilot data from the first 1712 patients consecutively recruited. Sensitivity and specificity of SELENA+ in detection of referable DR cases were 94.7% (95% confidence interval [CI] 88.0% to 98.3%) and 82.2% (95% CI 80.8% to 83.5%), respectively. In comparison, sensitivity and specificity of human graders were 98.9% (95% CI 94.0% to 99.9%) and 97.2% (95% CI 96.6–97.8%), respectively. For patients with VTDR, SELENA+ demonstrated a substantial advantage of higher sensitivity compared with human performance, reflecting the benefit of the fine-tuning of SELENA+ that we performed to enhance the AI solution’s ability to detect VTDR. We outline the clinical, technical, operational, regulatory, and governance challenges encountered as well as the lessons learnt in this AI algorithm implementation journey. We also present a conceptual framework with considerations and strategies for the broader adoption of medical AI SaMD solutions in the field of ophthalmology and beyond.