Miss Alexa Nord Bronzyk1, Dr. Micheal Dunn, Prof. Jerry Menikoff, Prof. Julian Savulescu, Dr Angela Ballantyne, Dr Pavitra Krishnaswamy, Dr Marcus Ong, Dr Liu Nan, Dr Mayli Mertens, Dr Sumytra Menon1
1National University Of Singapore, , Singapore
Abstract:
Risk prediction in emergency medicine holds unique challenges due to issues surrounding urgency and the high-pressure environment in emergency departments (ED). AI risk prediction tools have been developed with the aim of streamlining triaging processes and mitigating perennial issues affecting EDs globally, such as overcrowding and delays. The implementation of these tools is complicated by the potential risks associated with over-triage and under-triage, as well as the potential for the biases of healthcare professionals toward technology leading to the incorrect usage of such tools. As AI becomes more ubiquitous in medicine, learning new implementation strategies that align with ethical and regulatory frameworks will be essential. This paper explores risk surrounding these issues by analysing a case study in Singapore involving a machine learning triage tool called Score for Emergency Risk Prediction (SERP) used for estimating mortality after emergency admissions. After two successful retrospective studies demonstrating SERP’s predictive accuracy, the traditional pre-implementation randomised controlled trial (RCT) may not be feasible due to how the tool interacts with clinical judgment, complicating the blind arm of the trial. This has led researchers to consider other methods of testing SERP’s real-world capabilities such as ongoing-evaluation type studies. A risk-benefit analysis is carried out in order to determine if the proposed implementation strategy is ethically appropriate and aligned with systematic approaches to implementation, especially the Learning Health Systems framework (LHS) to ensure safety, efficacy, and ongoing learning.