Development and validation of the ISARIC 4C deterioration model for adults hospitalised with COVID-19: a prospective cohort study
Development and validation of the ISARIC 4C deterioration model for adults hospitalised with COVID-19: a prospective cohort study
BACKGROUND: Prognostic models to predict the risk of clinical deterioration in acute COVID-19 cases are urgently required to inform clinical management decisions.
METHODS: We developed and validated a multivariable logistic regression model for in-hospital clinical deterioration (defined as any requirement of ventilatory support or critical care, or death) among consecutively hospitalised adults with highly suspected or confirmed COVID-19 who were prospectively recruited to the International Severe Acute Respiratory and Emerging Infections Consortium Coronavirus Clinical Characterisation Consortium (ISARIC4C) study across 260 hospitals in England, Scotland, and Wales. Candidate predictors that were specified a priori were considered for inclusion in the model on the basis of previous prognostic scores and emerging literature describing routinely measured biomarkers associated with COVID-19 prognosis. We used internal-external cross-validation to evaluate discrimination, calibration, and clinical utility across eight National Health Service (NHS) regions in the development cohort. We further validated the final model in held-out data from an additional NHS region (London).
FINDINGS: 74 944 participants (recruited between Feb 6 and Aug 26, 2020) were included, of whom 31 924 (43·2%) of 73 948 with available outcomes met the composite clinical deterioration outcome. In internal-external cross-validation in the development cohort of 66 705 participants, the selected model (comprising 11 predictors routinely measured at the point of hospital admission) showed consistent discrimination, calibration, and clinical utility across all eight NHS regions. In held-out data from London (n=8239), the model showed a similarly consistent performance (C-statistic 0·77 [95% CI 0·76 to 0·78]; calibration-in-the-large 0·00 [-0·05 to 0·05]); calibration slope 0·96 [0·91 to 1·01]), and greater net benefit than any other reproducible prognostic model.
INTERPRETATION: The 4C Deterioration model has strong potential for clinical utility and generalisability to predict clinical deterioration and inform decision making among adults hospitalised with COVID-19.
FUNDING: National Institute for Health Research (NIHR), UK Medical Research Council, Wellcome Trust, Department for International Development, Bill & Melinda Gates Foundation, EU Platform for European Preparedness Against (Re-)emerging Epidemics, NIHR Health Protection Research Unit (HPRU) in Emerging and Zoonotic Infections at University of Liverpool, NIHR HPRU in Respiratory Infections at Imperial College London.
349-359
Gupta, Rishi K
29f2e5a6-86bc-442c-ae8f-35791e299e6c
Harrison, Ewen M
6e9fdc8e-928d-4135-b8d1-9547783025f1
Ho, Antonia
6b78d676-c51e-4faa-9802-98e5826f7dde
Docherty, Annemarie B
0b579461-1b7a-456a-aa13-09f229c7a9d8
Knight, Stephen R
55d26f24-3867-469b-9961-806d0393e7c1
van Smeden, Maarten
9d821332-2c57-457b-9619-d7e21a2df7d9
Abubakar, Ibrahim
7a6eaaa0-90f8-42cd-b52f-e408a3961e4a
Lipman, Marc
5a10162e-e8d5-442b-a740-599848e9cdcd
Quartagno, Matteo
76d416ed-14e5-4550-88d6-da0b4a1ed5a9
Pius, Riinu
7d1b224a-f607-425f-b2e0-a39a60c7feab
Buchan, Iain
fe0050fa-9b04-40c5-8ef4-6574742b18e8
Carson, Gail
56178b9a-2f1e-4823-b6d0-0ee1b2eef830
Drake, Thomas M
059a4d12-8b2d-449a-87d6-f59e00786557
Dunning, Jake
a311288a-ef83-4475-bf56-b36d9a3525b3
Fairfield, Cameron J
17091fca-beba-417a-874a-a25a78959302
Gamble, Carrol
6d685bb2-1ec5-4e38-a8d4-3cf6f6e625ac
Green, Christopher A
91b1dbcb-2a84-4348-b4db-39223853e3e1
Halpin, Sophie
22481154-e349-46d5-a754-2a86bd224be2
Hardwick, Hayley E
5f11e28f-495e-43ff-bf3c-7388f0619684
Holden, Karl A
ec5c9047-c8a7-4b23-ad41-52c8424e250b
Horby, Peter W
8f921e75-4605-4fb8-bb2c-be66fac6bb96
Jackson, Clare
86c2028c-8150-4cf6-87f7-be9aa0adc220
Mclean, Kenneth A
6df7eed9-ddb7-4d69-9f16-9d9adbdce677
Merson, Laura
56ce4715-df23-4879-b039-b3c5c1f98661
Nguyen-Van-Tam, Jonathan S
1aac2164-f614-4c82-87f9-e123817bf128
Norman, Lisa
e5c55403-401e-41b7-bcc0-5f02ad79d893
Olliaro, Piero L
b81416d3-55fc-4303-9307-ecf79e8ab0cb
Pritchard, Mark G
c606a7e6-14fc-460b-aa2e-b433bd449fbe
Russell, Clark D
80352f5e-cda1-4635-b0ca-c3c920c67590
Scott-Brown, James
357ad995-51dc-4334-893c-80c44106f240
Shaw, Catherine A
1c06c617-da0e-4cee-8930-f5b8e9a38d44
Sheikh, Aziz
5b7eb24b-0820-4bd6-8304-cc46bbe561a8
Solomon, Tom
3311a8fe-fc3d-42ab-8c99-cacc179f1f77
Sudlow, Cathie
c1884a16-0661-477c-92c9-006dcca2e712
Swann, Olivia V
4af0b2eb-e6e7-4936-9eb1-418eb80e2677
Turtle, Lance
46a79918-7fd1-4002-84d5-bc3bd0b26632
Openshaw, Peter J M
4e1ec99b-8f41-4740-be78-a57b2361d483
Baillie, J Kenneth
a08f7c7f-62a3-44ea-af0e-aa5f1ffde00c
Semple, Malcolm G
15b207df-e044-4078-9b40-1af17b800ab1
Noursadeghi, Mahdad
c86534b2-c59d-4f43-8af4-48ecba03b5ee
Dushianthan, Ahilanandan
013692a2-cf26-4278-80bd-9d8fcdb17751
April 2021
Gupta, Rishi K
29f2e5a6-86bc-442c-ae8f-35791e299e6c
Harrison, Ewen M
6e9fdc8e-928d-4135-b8d1-9547783025f1
Ho, Antonia
6b78d676-c51e-4faa-9802-98e5826f7dde
Docherty, Annemarie B
0b579461-1b7a-456a-aa13-09f229c7a9d8
Knight, Stephen R
55d26f24-3867-469b-9961-806d0393e7c1
van Smeden, Maarten
9d821332-2c57-457b-9619-d7e21a2df7d9
Abubakar, Ibrahim
7a6eaaa0-90f8-42cd-b52f-e408a3961e4a
Lipman, Marc
5a10162e-e8d5-442b-a740-599848e9cdcd
Quartagno, Matteo
76d416ed-14e5-4550-88d6-da0b4a1ed5a9
Pius, Riinu
7d1b224a-f607-425f-b2e0-a39a60c7feab
Buchan, Iain
fe0050fa-9b04-40c5-8ef4-6574742b18e8
Carson, Gail
56178b9a-2f1e-4823-b6d0-0ee1b2eef830
Drake, Thomas M
059a4d12-8b2d-449a-87d6-f59e00786557
Dunning, Jake
a311288a-ef83-4475-bf56-b36d9a3525b3
Fairfield, Cameron J
17091fca-beba-417a-874a-a25a78959302
Gamble, Carrol
6d685bb2-1ec5-4e38-a8d4-3cf6f6e625ac
Green, Christopher A
91b1dbcb-2a84-4348-b4db-39223853e3e1
Halpin, Sophie
22481154-e349-46d5-a754-2a86bd224be2
Hardwick, Hayley E
5f11e28f-495e-43ff-bf3c-7388f0619684
Holden, Karl A
ec5c9047-c8a7-4b23-ad41-52c8424e250b
Horby, Peter W
8f921e75-4605-4fb8-bb2c-be66fac6bb96
Jackson, Clare
86c2028c-8150-4cf6-87f7-be9aa0adc220
Mclean, Kenneth A
6df7eed9-ddb7-4d69-9f16-9d9adbdce677
Merson, Laura
56ce4715-df23-4879-b039-b3c5c1f98661
Nguyen-Van-Tam, Jonathan S
1aac2164-f614-4c82-87f9-e123817bf128
Norman, Lisa
e5c55403-401e-41b7-bcc0-5f02ad79d893
Olliaro, Piero L
b81416d3-55fc-4303-9307-ecf79e8ab0cb
Pritchard, Mark G
c606a7e6-14fc-460b-aa2e-b433bd449fbe
Russell, Clark D
80352f5e-cda1-4635-b0ca-c3c920c67590
Scott-Brown, James
357ad995-51dc-4334-893c-80c44106f240
Shaw, Catherine A
1c06c617-da0e-4cee-8930-f5b8e9a38d44
Sheikh, Aziz
5b7eb24b-0820-4bd6-8304-cc46bbe561a8
Solomon, Tom
3311a8fe-fc3d-42ab-8c99-cacc179f1f77
Sudlow, Cathie
c1884a16-0661-477c-92c9-006dcca2e712
Swann, Olivia V
4af0b2eb-e6e7-4936-9eb1-418eb80e2677
Turtle, Lance
46a79918-7fd1-4002-84d5-bc3bd0b26632
Openshaw, Peter J M
4e1ec99b-8f41-4740-be78-a57b2361d483
Baillie, J Kenneth
a08f7c7f-62a3-44ea-af0e-aa5f1ffde00c
Semple, Malcolm G
15b207df-e044-4078-9b40-1af17b800ab1
Noursadeghi, Mahdad
c86534b2-c59d-4f43-8af4-48ecba03b5ee
Dushianthan, Ahilanandan
013692a2-cf26-4278-80bd-9d8fcdb17751
Gupta, Rishi K, Harrison, Ewen M, Ho, Antonia, Docherty, Annemarie B, Knight, Stephen R, van Smeden, Maarten, Abubakar, Ibrahim, Lipman, Marc, Quartagno, Matteo, Pius, Riinu, Buchan, Iain, Carson, Gail, Drake, Thomas M, Dunning, Jake, Fairfield, Cameron J, Gamble, Carrol, Green, Christopher A, Halpin, Sophie, Hardwick, Hayley E, Holden, Karl A, Horby, Peter W, Jackson, Clare, Mclean, Kenneth A, Merson, Laura, Nguyen-Van-Tam, Jonathan S, Norman, Lisa, Olliaro, Piero L, Pritchard, Mark G, Russell, Clark D, Scott-Brown, James, Shaw, Catherine A, Sheikh, Aziz, Solomon, Tom, Sudlow, Cathie, Swann, Olivia V, Turtle, Lance, Openshaw, Peter J M, Baillie, J Kenneth, Semple, Malcolm G and Noursadeghi, Mahdad
,
ISARIC4C Investigators
(2021)
Development and validation of the ISARIC 4C deterioration model for adults hospitalised with COVID-19: a prospective cohort study.
The Lancet Respiratory Medicine, 9 (4), .
(doi:10.1016/S2213-2600(20)30559-2).
Abstract
BACKGROUND: Prognostic models to predict the risk of clinical deterioration in acute COVID-19 cases are urgently required to inform clinical management decisions.
METHODS: We developed and validated a multivariable logistic regression model for in-hospital clinical deterioration (defined as any requirement of ventilatory support or critical care, or death) among consecutively hospitalised adults with highly suspected or confirmed COVID-19 who were prospectively recruited to the International Severe Acute Respiratory and Emerging Infections Consortium Coronavirus Clinical Characterisation Consortium (ISARIC4C) study across 260 hospitals in England, Scotland, and Wales. Candidate predictors that were specified a priori were considered for inclusion in the model on the basis of previous prognostic scores and emerging literature describing routinely measured biomarkers associated with COVID-19 prognosis. We used internal-external cross-validation to evaluate discrimination, calibration, and clinical utility across eight National Health Service (NHS) regions in the development cohort. We further validated the final model in held-out data from an additional NHS region (London).
FINDINGS: 74 944 participants (recruited between Feb 6 and Aug 26, 2020) were included, of whom 31 924 (43·2%) of 73 948 with available outcomes met the composite clinical deterioration outcome. In internal-external cross-validation in the development cohort of 66 705 participants, the selected model (comprising 11 predictors routinely measured at the point of hospital admission) showed consistent discrimination, calibration, and clinical utility across all eight NHS regions. In held-out data from London (n=8239), the model showed a similarly consistent performance (C-statistic 0·77 [95% CI 0·76 to 0·78]; calibration-in-the-large 0·00 [-0·05 to 0·05]); calibration slope 0·96 [0·91 to 1·01]), and greater net benefit than any other reproducible prognostic model.
INTERPRETATION: The 4C Deterioration model has strong potential for clinical utility and generalisability to predict clinical deterioration and inform decision making among adults hospitalised with COVID-19.
FUNDING: National Institute for Health Research (NIHR), UK Medical Research Council, Wellcome Trust, Department for International Development, Bill & Melinda Gates Foundation, EU Platform for European Preparedness Against (Re-)emerging Epidemics, NIHR Health Protection Research Unit (HPRU) in Emerging and Zoonotic Infections at University of Liverpool, NIHR HPRU in Respiratory Infections at Imperial College London.
This record has no associated files available for download.
More information
e-pub ahead of print date: 11 January 2021
Published date: April 2021
Additional Information:
Funding Information:
This work uses data provided by patients and collected by the NHS as part of their care and support. We are extremely grateful to the 2648 front-line NHS clinical and research staff and volunteer medical students who collected this data in challenging circumstances; and the generosity of the participants and their families for their individual contributions in these difficult times. We also acknowledge the support of Jeremy J Farrar (Wellcome Trust) and Nahoko Shindo (WHO). This work is supported by grants from the NIHR (award CO-CIN-01); the MRC (grant MC_PC_19059); the NIHR HPRU in Emerging and Zoonotic Infections at the University of Liverpool, in partnership with Public Health England (PHE), in collaboration with the Liverpool School of Tropical Medicine and the University of Oxford (award 200907); the NIHR HPRU in Respiratory Infections at Imperial College London with PHE (award 200927); the Wellcome Trust and Department for International Development (215091/Z/18/Z); the Bill & Melinda Gates Foundation (OPP1209135); the Liverpool Experimental Cancer Medicine Centre (grant reference C18616/A25153); the NIHR BRC at Imperial College London (IS-BRC-1215–20013); the EU Platform for European Preparedness Against (Re-)emerging Epidemics (FP7 project 602525); and the NIHR Clinical Research Network, who provided infrastructure support for this research. RKG is funded by the NIHR (DRF-2018-11-ST2-004). IA acknowledges NIHR Senior Investigator Award (NF-SI-0616-10037) funding. ML acknowledges NIHR funding (HTA 16/88/06). PJMO is supported by an NIHR senior investigator award (201385). LT is supported by the Wellcome Trust (award 205228/Z/16/Z). MN is funded by a Wellcome Trust investigator award (207511/Z/17/Z) and the NIHR University College London Hospitals BRC. The views expressed in this Article are those of the authors and not necessarily those of the DHSC, Department for International Development, NIHR, MRC, Wellcome Trust, or PHE.
Funding Information:
ABD reports grants from the UK Department of Health and Social Care (DHSC) during the conduct of the study and grants from Wellcome Trust outside of the submitted work. CAG reports grants from the DHSC NIHR during the conduct of the study. PWH reports grants from the Wellcome Trust, Department for International Development, Bill & Melinda Gates Foundation, and NIHR during the conduct of the study. JSN-V-T reports grants from DHSC, during the conduct of the study, and is seconded to DHSC. MN is supported by a Wellcome Trust investigator award and the NIHR University College London Hospitals Biomedical Research Centre (BRC). PJMO reports personal fees from consultancies and from the European Respiratory Society, grants from the UK Medical Research Council (MRC), MRC Global Challenge Research Fund, EU, NIHR BRC, MRC/GSK, Wellcome Trust, NIHR (Health Protection Research Unit [HPRU] in Respiratory Infection), and is an NIHR senior investigator outside of the submitted work; his role as President of the British Society for Immunology was unpaid but travel and accommodation at some meetings was provided by the Society. JKB reports grants from the MRC. MGS reports grants from the DHSC NIHR, MRC, and the HPRU in Emerging and Zoonotic Infections, University of Liverpool, during the conduct of the study; and is chair of the scientific advisory board and a minority share holder at Integrum Scientific (Greensboro, NC, USA) outside of the submitted work. LT reports grants from the HPRU in Emerging and Zoonotic Infections, University of Liverpool, during the conduct of the study, and grants from Wellcome Trust outside of the submitted work. All other authors declare no competing interests.
Funding Information:
This work uses data provided by patients and collected by the NHS as part of their care and support. We are extremely grateful to the 2648 front-line NHS clinical and research staff and volunteer medical students who collected this data in challenging circumstances; and the generosity of the participants and their families for their individual contributions in these difficult times. We also acknowledge the support of Jeremy J Farrar (Wellcome Trust) and Nahoko Shindo (WHO). This work is supported by grants from the NIHR (award CO-CIN-01); the MRC (grant MC_PC_19059); the NIHR HPRU in Emerging and Zoonotic Infections at the University of Liverpool, in partnership with Public Health England (PHE), in collaboration with the Liverpool School of Tropical Medicine and the University of Oxford (award 200907); the NIHR HPRU in Respiratory Infections at Imperial College London with PHE (award 200927); the Wellcome Trust and Department for International Development (215091/Z/18/Z); the Bill & Melinda Gates Foundation (OPP1209135); the Liverpool Experimental Cancer Medicine Centre (grant reference C18616/A25153); the NIHR BRC at Imperial College London (IS-BRC-1215?20013); the EU Platform for European Preparedness Against (Re-)emerging Epidemics (FP7 project 602525); and the NIHR Clinical Research Network, who provided infrastructure support for this research. RKG is funded by the NIHR (DRF-2018-11-ST2-004). IA acknowledges NIHR Senior Investigator Award (NF-SI-0616-10037) funding. ML acknowledges NIHR funding (HTA 16/88/06). PJMO is supported by an NIHR senior investigator award (201385). LT is supported by the Wellcome Trust (award 205228/Z/16/Z). MN is funded by a Wellcome Trust investigator award (207511/Z/17/Z) and the NIHR University College London Hospitals BRC. The views expressed in this Article are those of the authors and not necessarily those of the DHSC, Department for International Development, NIHR, MRC, Wellcome Trust, or PHE.
Publisher Copyright:
© 2021 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 license
Copyright:
Copyright 2021 Elsevier B.V., All rights reserved.
Identifiers
Local EPrints ID: 449148
URI: http://eprints.soton.ac.uk/id/eprint/449148
ISSN: 2213-2600
PURE UUID: a28d3d32-2040-4bab-8cc1-761c86c3b1ea
Catalogue record
Date deposited: 18 May 2021 16:31
Last modified: 16 Apr 2024 01:54
Export record
Altmetrics
Contributors
Author:
Rishi K Gupta
Author:
Ewen M Harrison
Author:
Antonia Ho
Author:
Annemarie B Docherty
Author:
Stephen R Knight
Author:
Maarten van Smeden
Author:
Ibrahim Abubakar
Author:
Marc Lipman
Author:
Matteo Quartagno
Author:
Riinu Pius
Author:
Iain Buchan
Author:
Gail Carson
Author:
Thomas M Drake
Author:
Jake Dunning
Author:
Cameron J Fairfield
Author:
Carrol Gamble
Author:
Christopher A Green
Author:
Sophie Halpin
Author:
Hayley E Hardwick
Author:
Karl A Holden
Author:
Peter W Horby
Author:
Clare Jackson
Author:
Kenneth A Mclean
Author:
Laura Merson
Author:
Jonathan S Nguyen-Van-Tam
Author:
Lisa Norman
Author:
Piero L Olliaro
Author:
Mark G Pritchard
Author:
Clark D Russell
Author:
James Scott-Brown
Author:
Catherine A Shaw
Author:
Aziz Sheikh
Author:
Tom Solomon
Author:
Cathie Sudlow
Author:
Olivia V Swann
Author:
Lance Turtle
Author:
Peter J M Openshaw
Author:
J Kenneth Baillie
Author:
Malcolm G Semple
Author:
Mahdad Noursadeghi
Author:
Ahilanandan Dushianthan
Corporate Author: ISARIC4C Investigators
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics