Full Text View

Original Article

Assessment of a novel marker of ICU strain, the ICU Activity Index, during the COVID-19 pandemic in Victoria, Australia

David V Pilcher , Graeme Duke , Melissa Rosenow, Nicholas Coatsworth , Genevieve O'Neill , Tracey A Tobias, Steven McGloughlin, Anthony Holley , Steven Warrillow , Claire Cattigan, Sue Huckson, Gian Sberna, Jason McClure

Crit Care Resusc 2021; 23 (3): 300-307

Correspondence:d.pilcher@alfred.org.au

• Author Details
• Competing Interests
None declared
• Abstract
OBJECTIVES: To validate a real-time Intensive Care Unit (ICU) Activity Index as a marker of ICU strain from daily data available from the Critical Health Resource Information System (CHRIS), and to investigate the association between this Index and the need to transfer critically ill patients during the coronavirus disease 2019 (COVID-19) pandemic in Victoria, Australia.
DESIGN: Retrospective observational cohort study.
SETTING: All 45 hospitals with an ICU in Victoria, Australia.
PARTICIPANTS: Patients in all Victorian ICUs and all critically ill patients transferred between Victorian hospitals from 27 June to 6 September 2020.
MAIN OUTCOME MEASURE: Acute interhospital transfer of one or more critically ill patients per day from one site to an ICU in another hospital.
RESULTS: 150 patients were transported over 61 days from 29 hospitals (64%). ICU Activity Index scores were higher on days when critical care transfers occurred (median, 1.0 [IQR, 0.4–1.7] v 0.6 [IQR, 0.3–1.2]; P < 0.001). Transfers were more common on days of higher ICU occupancy, higher numbers of ventilated or COVID-19 patients, and when more critical care staff were unavailable. The highest ICU Activity Index scores were observed at hospitals in north-western Melbourne, where the COVID-19 disease burden was greatest. After adjusting for confounding factors, including occupancy and lack of available ICU staff, a rising ICU Activity Index score was associated with an increased risk of a critical care transfer (odds ratio, 4.10; 95% CI, 2.34–7.18; P < 0.001).
CONCLUSIONS: The ICU Activity Index appeared to be a valid marker of ICU strain during the COVID-19 pandemic. It may be useful as a real-time clinical indicator of ICU activity and predict the need for redistribution of critical ill patients.
• References
1. Kahn JM. What we talk about when we talk about intensive care unit strain. Ann Am Thorac Soc 2014; 11: 219-20
2. Harris S, Singer M, Rowan K, Sanderson C. Delay to admission to critical care and mortality among deteriorating ward patients in UK hospitals: a multicentre, prospective, observational cohort study. Lancet 2015; 385 (Suppl): S40
3. Mathews KS, Durst MS, Vargas-Torres C, et al. Effect of emergency department and ICU occupancy on admission decisions and outcomes for critically ill patients. Crit Care Med 2018; 46: 720-7
4. Robert R, Coudroy R, Ragot S, et al. Influence of ICU-bed availability on ICU admission decisions. Ann Intensive Care 2015; 5: 55
5. Anesi GL, Chowdhury M, Small DS, et al. Association of a novel index of hospital capacity strain with admission to intensive care units. Ann Am Thorac Soc 2020; 17: 1440-7
6. Rewa OG, Stelfox HT, Ingolfsson A, et al. Indicators of intensive care unit capacity strain: a systematic review. Crit Care 2018; 22: 86
7. Yang X, Yu Y, Xu J, et al. Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: a single-centered, retrospective, observational study. Lancet Respir Med 2020; 8: 475-81
8. Grasselli G, Pesenti A, Cecconi M. Critical care utilization for the COVID-19 outbreak in Lombardy, Italy: early experience and forecast during an emergency response. JAMA 2020; 323: 1545-6
9. Doidge JC, Gould DW, Ferrando-Vivas P, et al. Trends in intensive care for patients with COVID-19 in England, Wales and Northern Ireland. Am J Respir Crit Care Med 2020; 203: 565-74
10. Richardson S, Hirsch JS, Narasimhan M, et al. Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York City area. JAMA 2020; 323: 2052-9
11. Bauer J, Brüggmann D, Klingelhöfer D, et al. Access to intensive care in 14 European countries: a spatial analysis of intensive care need and capacity in the light of COVID-19. Intensive Care Med 2020; 46: 2026-34
12. Rimmelé T, Pascal L, Polazzi S, Duclos A. Organizational aspects of care associated with mortality in critically ill COVID-19 patients. Intensive Care Med 2021; 47: 119-21
13. O’Reilly GM, Mitchell RD, Mitra B, et al. Epidemiology and clinical features of emergency department patients with suspected and confirmed COVID-19: a multisite report from the COVED Quality Improvement Project for July 2020 (COVED-3). Emerg Med Australas 2020; 33: 114-24
14. Australian and New Zealand Intensive Care Society. ANZICS Centre for Outcome and Resource Evaluation. 2018 report. https://www.anzics.com.au/wp-content/uploads/2019/10/2018-ANZICS-CORE-Report.pdf (viewed July 2021)
15. Victorian Department of Health and Human Services. Victorian coronavirus (COVID-19) data. https://www.dhhs.vic.gov.au/victorian-coronavirus-covid-19-data (viewed July 2021)
16. Moses T, Holland PW. A comparison of statistical selection strategies for univariate and bivariate log-linear models. Br J Math Stat Psychol 2010; 63: 557-74
17. Martínez-Camblor P, Pardo-Fernández JC. The Youden Index in the generalized receiver operating characteristic curve context. Int J Biostat 2019; 15: /j/ijb.2019.15.issue-1/ijb-2018-0060/ijb-2018-0060.xml
18. Halpern SD. ICU capacity strain and the quality and allocation of critical care. Curr Opin Crit Care 2011; 17: 648-57
19. Mukherjee V, Toth AT, Fenianos M, et al. Clinical outcomes in critically ill coronavirus disease 2019 patients: a unique New York City public hospital experience. Crit Care Explor 2020; 2: e0188
20. Burrell AJ, Pellegrini B, Salimi F, et al. Outcomes for patients with COVID-19 admitted to Australian intensive care units during the first four months of the pandemic. Med J Aust 2021; 214: 23-30
21. Serpa Neto A, Burrell AJ, Bailey M, et al. Comparison of critical care occupancy and outcomes of critically ill patients during the 2020 COVID-19 winter surge and 2009 H1N1 influenza pandemic in Australia. Ann Am Thorac Soc 2020; doi: 10.1513/AnnalsATS.202010-1311OC [Epub ahead of print]
22. Kontis V, Bennett JE, Rashid T, et al. Magnitude, demographics and dynamics of the effect of the first wave of the COVID-19 pandemic on all-cause mortality in 21 industrialized countries. Nat Med 2020; 26: 1919-28
23. Australian and New Zealand Intensive Care Society (ANZICS) Centre for Outcome and Resource Evaluation. Report on COVID-19 admissions to intensive care in Victoria (01 January to 30 September 2020). https://www.anzics.com.au/wp-content/uploads/2020/10/ANZICS-CORE-COVID-19-Report_VIC_01-Jan_30-Sept.pdf (viewed July 2021)
24. Iwashyna TJ, Kramer AA, Kahn JM. Intensive care unit occupancy and patient outcomes. Crit Care Med 2009; 37: 1545-57
25. Gabler NB, Ratcliffe SJ, Wagner J, et al. Mortality among patients admitted to strained intensive care units. Am J Respir Crit Care Med 2013; 188: 800-6
26. Wilcox ME, Harrison DA, Patel A, Rowan KM. Higher ICU capacity strain is associated with increased acute mortality in closed ICUs. Crit Care Med 2020; 48: 709-16
27. Neuraz A, Guérin C, Payet C, et al. Patient mortality is associated with staff resources and workload in the ICU: a multicenter observational study. Crit Care Med 2015; 43: 1587-94
28. Vardoulakis S, Jalaludin BB, Morgan GG, et al. Bushfire smoke: urgent need for a national health protection strategy. Med J Aust 2020; 212: 349-53.e1
29. Darvall JN, Durie M, Pilcher D, et al. Intensive care implications of epidemic thunderstorm asthma. Crit Care Resusc 2018; 20: 294-303
Limited access to, or excess demand for, available intensive care unit (ICU) resources, also known as “ICU strain”, may be associated with adverse patient outcomes. 1 A fully occupied ICU may delay the definitive management of a patient who requires critical care, 2 or compromise the care of those in ICU. It may result in some patients being denied admission to an ICU. 3 Both are associated with increased mortality risk. 4 Although several individual indices of hospital and ICU strain have been investigated, 5, 6 none is ideal, and composite indices to measure strain have rarely been examined. 6

The coronavirus disease 2019 (COVID-19) pandemic placed a marked increase in demand on health care resources throughout the world, with requirement for ICU services exceeding capacity in many countries, including China, 7 Italy, 8 the United Kingdom 9 and the United States. 10 Lack of access due to the strain on critical care services was associated with adverse patient outcomes. 11, 12

In June 2020, in response to rising notifications of COVID-19 within Victoria, Australia, 13 ICU clinical directors representing each of the nine state health care service clusters commenced a daily morning meeting with representatives from Ambulance Victoria, Safer Care Victoria and the Victorian Department of Health and Human Services. The intention of the daily meeting was to quantify clinical demand, identify and support ICUs under strain and, where necessary, proactively facilitate transfers to another hospital’s ICU to address excess local demand. This situation provided an opportunity to develop and validate a de novo real-time composite measure of ICU strain.

Our hypothesis was that strain within an ICU would be associated with the need to transfer a critically ill patient to another hospital with greater ICU capacity. Our objectives were to measure daily levels of ICU strain using an ICU Activity Index and to assess its association with the acute interhospital transfer of one or more critically ill patients to another hospital’s ICU.

Methods

This project was approved as a low risk project by the Human Research Ethics Committee of The Alfred Hospital (HREC: 480/20).

Data sources

De-identified data on all adult critically ill patients transferred from an ICU or from the emergency department to another hospital’s ICU (in the same state) between 27 June and 6 September 2020 were extracted from the Ambulance Victoria’s Adult Retrieval Victoria Information System. We excluded transfers for specialist (neurosurgical, cardiac or trauma) services not available at the sending hospital, transfers from hospitals from another jurisdiction, and transfers from sending hospitals without an onsite ICU.

Real-time ICU activity data from all Victorian acute care hospitals were extracted from the Critical Health Resource Information System (CHRIS) for the same period. CHRIS is a real-time dashboard of specific ICU activity and acuity and resources, developed and implemented nationally by the Australian and New Zealand Intensive Care Society, 14 Ambulance Victoria, the Australian Government Department of Health, and Telstra Purple in response to the COVID-19 pandemic.

Data extracted from CHRIS were used to calculate the total daily number of available, staffed and equipped (“open available”) ICU beds, the number of additional ICU beds that were open above the baseline reported at the beginning of the study period, and current daily ICU occupancy — based on a staffing minimum of one critical care nurse for every ICU patient and one for every two high dependency (step-down) patients within the ICU. Individual patient length of stay and outcome data were not reported

The ICU Activity Index

A numerical Activity Index was calculated for each ICU as follows:

$$\text{ICU Activity Index} = {{\text{COVIDs} + \text{MV} + \text{RRT} + \text{ECMO} + \text{ICU}} \over \text{Total daily open available ICU beds}}$$

where COVIDs = number of COVID-19 patients within the ICU; MV = number of ICU patients receiving invasive ventilation; RRT = number of ICU patients receiving renal replacement therapy; ECMO = number of ICU patients receiving extracorporeal membrane oxygenation; ICU = number of patients requiring 1:1 nurse to patient ratio; and total daily open available ICU beds = total number of open, equipped, 1:1 staffed bed spaces in the ICU declared daily on CHRIS. Thus, the minimum value was “0” if all patients in the ICU were being nursed 1:2, none required renal replacement, invasive ventilation or ECMO and there were no COVID-19 patients. The maximum value was “5” if every patient in the ICU was a ventilated COVID-19 patient on renal replacement therapy and ECMO.

The ICU Activity Index was calculated daily at 8.30 am. If site information had not been updated during the previous 24 hours, the ICU Activity Index was considered as unavailable. There was no imputation of missing data.

Primary outcome

The primary outcome was the acute interhospital transfer of one or more critically ill patients to another hospital ICU during the 24-hour calendar day of the calculated ICU Activity Index. Decisions to transfer patients were based on the clinical judgement of the treating intensive care specialist communicated to their local cluster-lead representative. Transfers included, but were not restricted to, patients with COVID-19.

The ICU Activity Index value was not accessible to treating clinicians. Those with access to the ICU Activity Index were not engaged in the clinical decision to transfer a patient.

Statistical methods

For descriptive purposes, ICUs were categorised into one of three large geographic groups — north-western Melbourne, south-eastern Melbourne and regional Victoria — representing the regions with the highest, intermediate and lowest numbers of community COVID-19 notifications respectively. 15 Each hospital was classified as either a public tertiary, public metropolitan, public rural or private facility.

Data for each ICU site on each day (“site-day”) were reported either as number (proportion) or median (interquartile range [IQR]). Maximum, minimum and mean values were also provided to describe resource use and demand, where appropriate. Statistical comparisons employed χ2, Wilcoxon rank sum or Kruskal–Wallis tests depending on type and distribution of data.

Descriptive statistics, kernel density and graphical time series plots were used to show variation in the ICU Activity Index over time. Associations with outcome were investigated using mixed effects hierarchical logistic regression, with hospital site treated as a random intercept, and results reported as odds ratios (ORs) and 95% confidence interval (CI). Akaike and Bayesian information criteria were used for best model selection. 16 Cut points to maximise specificity and sensitivity of the ICU Activity Index were determined using the Youden Index. 17 A two-sided P < 0.05 was considered significant. All data were analysed using Stata version 16.1 (StataCorp, Texas, USA).

Results

All 45 public and private adult hospitals with an onsite ICU within the state of Victoria contributed data. There were 150 ICU patients (1.7%) transported on 61 study days (84.7%), equivalent to 143 site-days (4.4%), from 29 hospitals (64.4%). During this time, there were a total of 468 170 hospital separations, of which 7911 (1.7%) included an ICU admission, and 2697 (0.6%) confirmed COVID-19 hospital separations, of which 247 (9.2%) were admitted to an ICU.

Patterns of the ICU Activity Index varied between hospitals and over time. ICU Activity Index scores were higher at sites on days when critical care transfers occurred (median, 1.0; IQR, 0.4–1.7 compared with days when no transfers occurred (median, 0.6; IQR, 0.3–1.2; P < 0.001). Critical care transfers were more frequent from sites where there was higher occupancy; higher COVID-19 patient numbers in the ICU; higher proportion of patients requiring mechanical ventilation, renal replacement therapy or 1:1 critical care nursing; or when there were more critical care staff unavailable (Table 1).

ICU Activity Index scores were highest at hospitals in north-western Melbourne, the region with the highest burden of COVID-19. The highest proportion (51/864, 5.9%) of site-days with critical care transfers occurred at hospitals in this region, with three hospitals transferring more than one patient per day on seven occasions (Online Appendix, table 2). Patients transferred from these hospitals more commonly had respiratory diagnoses (33/58, 57%), required mechanical ventilation (47/58, 81%) and were more likely to be transferred directly from the ICU (41/58, 71%) (Online Appendix, table 3). An example of the daily ICU Activity Index scores from the four metropolitan hospitals in north-western Melbourne and associated days when critical care transfers occurred is shown in Figure 1, with figures for all hospitals presented in the Online Appendix, figures 1a–1c.

At higher ICU Activity Index levels, there were more critical care transfers, higher levels of ICU occupancy, greater numbers of patients requiring 1:1 nurse to patient ratios, and more critical care staff unavailable (Figure 2). In keeping with its calculation, the ICU Activity Index increased proportionally with the number of patients receiving mechanical ventilation, renal replacement therapy or extracorporeal membrane oxygenation (Online Appendix figure 2).

Overall, the point of maximal combined sensitivity (72%) and specificity (48%) for the prediction of a critical care transfer was at a low ICU Activity Index score of 0.6. A high ICU Activity Index score of 1.6 had a sensitivity of 28% and specificity of 90% for interhospital transfer. This was also the level that maximised overall sensitivity and specificity for public tertiary hospitals (Online Appendix, table 4). Kernel density plots for site-days when critical care transfers occurred compared with those when there was no transfer had greatest separation either below an ICU Activity Index score of 0.6 or above 1.6 (Online Appendix, figure 3). A critical care transfer occurred on 12% of occasions when the ICU Activity Index score was 1.6 or more, compared with 3% of occasions when it was below 0.6 (< 0.001) (Online Appendix, table 5).

After adjusting for confounding factors, such as hospital type, occupancy and number of unavailable critical care staff, a rising ICU Activity Index was associated with an increased risk of a critical care transfer (OR, 4.10; 95% CI, 2.34–7.18; < 0.001). Public metropolitan and rural hospitals were more likely to transfer patients than public tertiary or private hospitals. Of note, occupancy was not predictive of critical care transfer in the multivariable analysis (OR, 0.48; 95% CI, 0.14–1.64; = 0.25) (Table 2). These findings were consistent in multiple sensitivity analyses (Online Appendix, tables 6a–6d).

Discussion

Summary of findings

The number of critical care transfers in Victoria (during the second wave of the COVID-19 pandemic in Australia) averaged two per day and affected one in 60 ICU patients. A simple ICU Activity Index based on readily available real-time data was a better predictor of the need for a critical care transfer than ICU occupancy alone. The ICU Activity Index may be a pragmatic indicator of ICU strain.

Comparison to previous literature

The need for a marker which can identify ICUs where patient outcomes are at risk because of increased strain is well documented. 18 Strain on critical care services has been reported from many parts of the world during the COVID-19 pandemic. The lack of access to intensive care beds and ICU strain has been associated with higher case fatality rates for COVID-19 patients. 1, 11 Interhospital transport of critically ill patients has been recognised as a patient safety mechanism to redistribute demand to match ICU resources during the pandemic. 19

Due to successful public health measures and the localised nature of the second wave of infections in Victoria, 20, 21, 22, 23 the COVID-19 pandemic appeared to result in fewer attributable excess deaths in Australia compared with many countries. 22 Despite this, concerning levels of ICU strain were seen at hospitals, particularly in north-western Melbourne, which led to the transfer of patients to ICUs with greater capacity. The ICU Activity Index provided an objective measure of the level of ICU strain.

The ICU Activity Index is a composite metric of both ICU census and aggregate daily patient acuity. Commonly used measures of ICU strain such as high occupancy and daily census data may have variable relationships to patient outcomes. 6 Large studies have demonstrated either no relationship between high occupancy and adverse patient outcomes24 or only present in a closed medical staffing model with intensive care specialist-led services,25 the model used in Australia and New Zealand. A large study from the UK suggested that the relationship between markers of strain and mortality was strongest when patient acuity was also taken into account. 26 The ICU Activity Index may be a more robust marker of ICU strain because it includes a measure of complexity as well as occupancy.

Lower staffing ratios, staff shortages and high levels of procedural activity in addition to high patient turnover have been shown to be associated with adverse patient outcomes. 27 In our study, levels of occupancy, acuity and staff unavailability were all higher on days with greater ICU Activity Index values.

Implications of findings

This study supports the potential use of the ICU Activity Index as a marker of strain, at least in Australia and New Zealand. It has content validity: greater values were associated with higher proportions of patients requiring invasive ICU therapies (as expected from its calculation) but no one factor was more influential than others. The ICU Activity Index also had face validity: higher values were seen in ICUs with greater occupancy and nursing requirements, and in ICUs that were known to be under strain with high numbers of COVID-19 admissions or high levels of staff unavailability. In addition, the ICU Activity Index had greater predictive validity than other potential markers of strain, such as occupancy and staff shortage, for determining the need to transfer a critically ill patient.

Strengths and limitations

The strengths of the study include engagement with senior clinicians, high compliance of clinical staff with CHRIS data entry, and the quality of the data entered. These clinicians were aware that ICU demand and capacity were being monitored but were unaware of the ICU Activity Index value itself, so it is unlikely to have directly influenced clinician decision making. The ICU Activity Index has now been incorporated into the CHRIS dashboard.
The limitations of the study included absence of clinical data on individual patient characteristics. Patient outcomes were not available. No information was available on metrics such as after-hours or premature discharge from ICU, nor on overall staffing within each ICU. All hospitals were required by the state health department to reduce elective surgery during the study period. It was not possible to determine which transfers were triggered by rising COVID-19 presentations or non-COVID-19 demand, nor how many of the patients transferred were positive for COVID-19 infection. The value of the ICU Activity Index in other states of Australia has not been proven, nor has it been tested during periods of low COVID-19 prevalence.

Future studies

The potential of the ICU Activity Index to quantify and predict ICU strain in real time and act as a trigger for clinical intervention is attractive and warrants further prospective evaluation. Future studies should determine whether there is a relationship between the ICU Activity Index and the capacity to accept critical care transfers, in addition to investigating a direct effect on patient outcomes such length of stay, short and long term survival, and serious adverse events in the ICU population, both during seasonal fluctuations in demand as well as during a pandemic. The potential of the ICU Activity Index for monitoring system responses to changes in public health policy, process implementation and provision of intensive care resources throughout Australia and New Zealand warrants investigation. The Index may also have a role in routine monitoring of ICU capacity in response to public health emergencies such as mass casualty events, bushfires, 28 thunderstorm asthma, 29 or peaks in seasonal demand.

Conclusion

The ICU Activity Index appeared to be a valid marker of ICU strain and predictive of the transfer of critically ill patients during second wave of the COVID-19 pandemic in Victoria. It is an immediately available measure that can routinely monitor ICU demand and capacity in Australia and New Zealand.

Acknowledgements: We would like to thank and acknowledge the contribution of the members of the Victorian Intensive Care Cluster leads meeting: David Anderson, Timothy Chimunda, Jason Fletcher, Christopher MacIsaac, Jeff Presneill, Angus Richardson, Owen Roodenburg, Helen Rizzoli, Nilesh Shah, and Trevor Weston.