The oesophageal cancer multi-disciplinary tool: a co-designed, externally validated, machine learning clinical decision support system

Abstract

BACKGROUND: The oesophageal cancer (OC) multi-disciplinary team (MDT) operates under significant pressures, handling complex decision-making. Machine learning (ML) can learn complex decision-making paradigms to improve efficiency, consistency, and cost if trained and deployed responsibly. We present an externally validated ML-based clinical decision support system (CDSS) designed to predict OC MDT treatment decisions and prognosticate palliative scenarios, co-designed using Responsible Research and Innovation (RRI) principles.

METHODS: Clinicopathological data collected from 1931 patients between 4th September 2009, and 8th November 2022 were used to test and validate models trained through four ML algorithms to predict curative and palliative treatment pathways along with palliative prognosis. 953 OC cases treated at University Hospitals Southampton (UHS) were used to train ML models which were externally validated on 978 OC cases from Oxford University Hospitals (OUH). Model performance was evaluated using Area Under Curve (AUC) for treatment classifiers and calibration curves for survival models. A parallel RRI program at the University of Southampton (United Kingdom) combining clinician interviews and inter-disciplinary workshops was conducted between 16.3.23 and 23.5.24. The RRI program comprised a group of 17 domain experts comprising programmers, computer scientists, clinicians and patient representatives to allow end-users to contribute towards the co-design of the CDSS user interface.

FINDINGS: Cohorts differed in baseline characteristics, with the external cohort (OUH) being younger, having better performance status, and a higher prevalence of pulmonary and vascular disease. Despite these differences, on internal validation (UHS cohort) mean AUCs for the primary treatment model were: MLR 0.905 ± 0.048, XGB 0.909 ± 0.044 and RF 0.883 ± 0.059 (k = 5 cross-validation) and MLR 0.866 (95% CI 0.866-0.867), XGB 0.863 (0.862-0.864), RF 0.863 (0.867-0.868) on bootstrapped resampling. For the palliative classifier, mean AUCs were: MLR 0.805 ± 0.096, XGB 0.815 ± 0.081 and RF 0.793 ± 0.083 (k = 5 cross-validation) and MLR 0.736 (95% CI 0.734-0.737), XGB 0.799 (0.798-0.800), RF 0.781 (0.778-0.782) on bootstrapped resampling. On external validation (OUH cohort), AUCs were MLR 0.894, XGB 0.887 and RF 0.891 for the primary treatment model and MLR 0.711, XGB 0.742 and RF 0.730 for the palliative treatment classifier. Predicted survival probability from the palliative survival model was well calibrated over the first 12 months post-diagnosis in both cohorts. The RRI program provided a collaborative environment leading to valuable modifications to the CDSS including prediction explanations, visual aids for survival and integrated education for users producing a user-friendly and quick to use tool.

INTERPRETATION: We present a novel, responsibly developed, externally validated AI CDSS trained to predict oesophageal cancer MDT decisions. It represents the foundations of a transformative application of ML, personalised, consistent and efficient MDT decision-support within OC which aligns to RRI principles.

FUNDING: Doctoral Studentship for NT (Institute for Life Sciences (University of Southampton) & University Hospital Southampton), UKRI TAS Pump-Priming Grant (TAS_PP_00167).

More information

Identifiers

Catalogue record

Export record

ASCII CitationAtomBibTeXData Cite XMLDublin CoreDublin CoreEP3 XMLEndNoteHTML CitationHTML CitationHTML ListJSONMETSMODSMPEG-21 DIDLOpenURL ContextObjectOpenURL ContextObject in SpanRDF+N-TriplesRDF+N3RDF+XMLRIOXX2 XMLReferReference ManagerSimple Metadata

Altmetrics