Full Text View
ISARIC-4C Mortality Score overestimates risk of death due to COVID-19 in Australian ICU patients: a validation cohort study
Matthew L Durie, Ary Serpa Neto, Aidan JC Burrell, D Jamie Cooper, Andrew A Udy, for the SPRINT-SARI Australia Investigators
Crit Care Resusc 2021; 23 (4): 403-413
- Matthew L Durie 1, 2
- Ary Serpa Neto 3
- Aidan JC Burrell 1, 3
- D Jamie Cooper 1, 3
- Andrew A Udy 1
- for the SPRINT-SARI Australia Investigators 4
All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf. Ary Serpa Neto reports personal fees from Drager, outside the submitted work. Matthew Durie, Aidan JC Burrell, D Jamie Cooper and Andrew A Udy declare no financial relationships with any organisations that might have an interest in the submitted work in the previous 3 years. All authors declare that they have no relationships or activities that could appear to have influenced the submitted work.
OBJECTIVE: To assess the performance of the UK International Severe Acute Respiratory and Emerging Infections Consortium (ISARIC) Coronavirus Clinical Characterisation Consortium (4C) Mortality Score for predicting mortality in Australian patients with coronavirus disease 2019 (COVID-19) requiring intensive care unit (ICU) admission.
DESIGN: Multicentre, prospective, observational cohort study.
SETTING: 78 Australian ICUs participating in the SPRINT-SARI (Short Period Incidence Study of Severe Acute Respiratory Infection) Australia study of COVID-19.
PARTICIPANTS: Patients aged 16 years or older admitted to participating Australian ICUs with polymerase chain reaction (PCR)-confirmed COVID-19 between 27 February and 10 October 2020.
MAIN OUTCOME MEASURES: ISARIC-4C Mortality Score, calculated at the time of ICU admission. The primary outcome was observed versus predicted in-hospital mortality (by 4C Mortality and APACHE II).
RESULTS: 461 patients admitted to a participating ICU were included. 149 (32%) had complete data to calculate a 4C Mortality Score without imputation. Overall, 61/461 patients (13.2%) died, 16.9% lower than the comparable ISARIC-4C cohort in the United Kingdom. In patients with complete data, the median (interquartile range [IQR]) 4C Mortality Score was 10.0 (IQR, 8.0–13.0) and the observed mortality was 16.1% (24/149) versus 22.9% median predicted risk of death. The 4C Mortality Score discriminatory performance measured by the area under the receiver operating characteristic curve (AUROC) was 0.79 (95% CI, 0.68–0.90), similar to its performance in the original ISARIC-4C UK cohort (0.77) and not superior to APACHE II (AUROC, 0.81; 95% CI, 0.75–0.87).
CONCLUSIONS: When calculated at the time of ICU admission, the 4C Mortality Score consistently overestimated the risk of death for Australian ICU patients with COVID-19. The 4C Mortality Score may need to be individually recalibrated for use outside the UK and in different hospital settings.
- Armstrong R, Kane A, Cook T. Outcomes from intensive care in patients with COVID-19: a systematic review and meta-analysis of observational studies. Anaesthesia 2020; 75: 1340-9.
- Armstrong RA, Kane AD, Cook TM. Decreasing mortality rates in ICU during the COVID-19 pandemic. Anaesthesia 2021; 76 (Suppl): 10.
- Tan E, Song J, Deane AM, Plummer MP. Global impact of coronavirus disease 2019 infection requiring admission to the ICU: a systematic review and meta-analysis. Chest 2021; 159: 524-36.
- Wynants L, Van Calster B, Collins GS, et al. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ 2020; 369: m1328.
- Gupta R, Marks M, Samuels THA, et al. Systematic evaluation and external validation of 22 prognostic models among hospitalised adults with COVID-19: an observational cohort study. Eur Respir J 2020; 56: 2003498.
- Levy T, Richardson S, Coppa K, et al. Development and validation of a survival calculator for hospitalized patients with COVID-19 [preprint]. medRxiv 20075416; 2 June 2020. doi: 10.1101/2020.04.22.20075416 (viewed Oct 2021).
- Dong YM, Sun J, Li YX, et al. Development and validation of a nomogram for assessing survival in patients with COVID-19 pneumonia. Clin Infect Dis 2021; 72: 652-60.
- Liang W, Liang H, Ou L, et al. Development and validation of a clinical risk score to predict the occurrence of critical illness in hospitalized patients with COVID-19. JAMA Intern Med 2020; 180: 1081-9.
- Knight SR, Ho A, Pius R, et al. Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score. BMJ 2020; 370: m3339.
- Wellbelove Z, Walsh C, Perinpanathan T, et al. Comparing the 4C mortality score for COVID-19 to established scores (CURB65, CRB65, qSOFA, NEWS) for respiratory infection patients. J Infect 2021; 82: 414-51.
- van Dam P, Zelis N, van Kuijk SMJ, et al. Performance of prediction models for short-term outcome in COVID-19 patients in the emergency department: a retrospective study. Ann Med 2021; 53: 402-9.
- Yildiz H, Castanares-Zapatero D, Hannesse C, et al. Prospective validation and comparison of COVID-GRAM, NEWS2, 4C mortality score, CURB-65 for the prediction of critical illness in COVID-19 patients. Infect Dis (Lond) 2021; 53: 640-2.
- Covino M, De Matteis G, Burzo ML, et al. Predicting in-hospital mortality in COVID-19 older patients with specifically developed scores. J Am Geriatr Soc 2021; 69: 37-43.
- Burrell AJ, Pellegrini B, Salimi F, et al. Outcomes for patients with COVID-19 admitted to Australian intensive care units during the first four months of the pandemic. Med J Aust 2021; 214: 23-30.
- Intensive Care National Audit and Research Centre. ICNARC report on COVID-19 in critical care: England, Wales and Northern Ireland — 25 September 2020. London: ICNARC, 2020. https://www.icnarc.org/DataServices/Attachments/Download/baa7de02-3f00-eb11-912b-00505601089b (viewed Oct 2020).
- Knaus WA, Wagner DP, Draper EA, et al. The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest 1991; 100: 1619-36.
- Vincent JL, Moreno R, Takala J, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med 1996; 22: 707-10.
- Paul E, Bailey M, Pilcher D. Risk prediction of hospital mortality for adult patients admitted to Australian and New Zealand intensive care units: development and validation of the Australian and New Zealand Risk of Death model. J Crit Care 2013; 28: 935-41.
- ISARIC-4C Consortium. 4C Mortality Score 2020. https://isaric4c.net/risk/ (viewed Oct 2020).
- Grasselli G, Greco M, Zanella A, et al. Risk factors associated with mortality among patients with COVID-19 in intensive care units in Lombardy, Italy. JAMA Intern Med 2020; 180: 1345-55.
- Zangrillo A, Beretta L, Scandroglio AM, et al. Characteristics, treatment, outcomes and cause of death of invasively ventilated patients with COVID-19 ARDS in Milan, Italy. Crit Care Resusc 2020; 22: 200-11.
- Grasselli G, Zangrillo A, Zanella A, et al. Baseline characteristics and outcomes of 1591 patients infected with SARS-CoV-2 admitted to ICUs of the Lombardy Region, Italy. JAMA 2020; 323: 1574-81.
- Mårtensson J, Engerström L, Walther S, et al. COVID-19 critical illness in Sweden: characteristics and outcomes at a national population level. Crit Care Resusc 2020; 22: 312-20.
- Monti G, Cremona G, Zangrillo A, et al. Home ventilators for invasive ventilation of patients with COVID-19. Crit Care Resusc 2020; 22: 266-70.
- Ling L, So C, Shum HP, et al. Critically ill patients with COVID-19 in Hong Kong: a multicentre retrospective observational cohort study. Crit Care Resusc 2020; 22: 119-25.
- Docherty AB, Harrison EM, Green CA, et al. Features of 20 133 UK patients in hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: prospective observational cohort study. BMJ 2020; 369: m1985.
- Intensive Care National Audit and Research Centre. ICNARC report on COVID-19 in critical care: England, Wales and Northern Ireland — 3 June 2021. London: ICNARC, 2020. https://www.icnarc.org/DataServices/Attachments/Download/483d4e84-3fc5-eb11-9131-00505601089b (viewed Oct 2020).
- RECOVERY Collaborative Group; Horby P, Lim WS, Emberson JR, et al. Dexamethasone in hospitalized patients with COVID-19. N Engl J Med 2020; 384: 693-704.
- Horby P, Lim WS, Emberson J, et al. Effect of dexamethasone in hospitalized patients with COVID-19: preliminary report [preprint]. medRxiv 20137273; 22 June 2020. doi: 10.1101/2020.06.22.20137273.
- Burrell AJ, Serpa Neto A, Trapani T, et al. Rapid translation of COVID-19 preprint data into critical care practice. Am J Respir Crit Care Med 2021; 203: 368-71.
- Gupta RK, Harrison EM, Ho A, et al. Development and validation of the ISARIC 4C deterioration model for adults hospitalised with COVID-19: a prospective cohort study. Lancet Respir Med 2021; 9: 349-59.
- Paul E, Bailey M, Van Lint A, Pilcher V. Performance of APACHE III over time in Australia and New Zealand: a retrospective cohort study. Anaesth Intensive Care 2012; 40: 980-94.
- Coombes ID, Dooley MJ, Townsend S, et al. Physician drug prescribing preferences and availability for ventilation of patients with COVID-19. Crit Care Resusc 2020; 22: 271-4.
- COVID-19 National Incident Room Surveillance Team. COVID-19 Australia: epidemiology report 28: fortnightly reporting period ending 25 October 2020. Commun Dis Intell (2018) 2020; 44. doi: 10.33321/cdi.2020.44.84.
- Millar JE, Busse R, Fraser JF, et al. Apples and oranges: international comparisons of COVID-19 observational studies in ICUs. Lancet Respir Med 2020; 8: 952-3.
- Nguyen T, Gupta S, Raman J, et al. Geolocated Twitter-based population mobility in Victoria, Australia, during the staged COVID-19 restrictions. Crit Care Resusc 2020; 22: 293-4.
- Landoni G, Moro M, Belletti A, et al. Recent exposure to smoking and COVID-19. Crit Care Resusc 2020; 22: 253-6.
- Plummer MP, Pellegrini B, Burrell AJ, et al. Smoking in critically ill patients with COVID-19: the Australian experience. Crit Care Resusc 2020; 22: 281-3.
- Ma X, Vervoort D. Critical care capacity during the COVID-19 pandemic: global availability of intensive care beds. J Crit Care 2020; 58: 96-7.