Perioperative anaemia and the requirement for allogeneic red blood cell transfusion are independently associated with increased morbidity and mortality following cardiac surgery.
1
Both nationally and internationally, surgery accounts for around a third of national blood product usage.
2
Cardiothoracic surgery patients received 5.6% of all blood products in Australia.
3
Although transfusion costs vary internationally, the burden on health service providers is universally significant.
4
Hung M, Ortmann E, Besser M, et al. A prospective observational cohort study to identify the causes of anaemia and association with outcome in cardiac surgical patients. Heart 2015; 101: 107-12
Wallis JP, Wells AW, Chapman CE. Changing indications for red cell transfusion from 2000 to 2004 in the North of England. Transfus Med 2006; 16: 411-7
Shortt J, Polizzotto MN, Waters N, et al. Assessment of the urgency and deferability of transfusion to inform emergency blood planning and triage: the Bloodhound prospective audit of red blood cell use. Transfusion 2009; 49: 2296-303
Amin M, Fergusson D, Aziz A, et al. The cost of allogeneic red blood cells — a systematic review. Transfus Med 2003; 13: 275-86
Various strategies to reduce overall blood product use by both preoperative optimisation of haemoglobin and blood conservation strategies have been developed and are grouped together under the term “patient blood management” (PBM). PBM programs incur a significant economic burden, and in order to best allocate resources, a number of scoring systems have been developed to identify patients at increased risk of perioperative transfusion. 5, 6, 7, 8
Alghamdi AA, Davis A, Brister S, et al. Development and validation of Transfusion Risk Understanding Scoring Tool (TRUST) to stratify cardiac surgery patients according to their blood transfusion needs. Transfusion 2006; 46: 1120-9
Goudie R, Sterne JAC, Verheyden V, et al. Risk scores to facilitate preoperative prediction of transfusion and large volume blood transfusion associated with adult cardiac surgery. Br J Anaesth 2015; 114: 757-66
Ranucci M, Castelvecchio S, Frigiola A, et al. Predicting transfusions in cardiac surgery: the easier, the better: the Transfusion Risk and Clinical Knowledge score. Vox Sang 2009; 96: 324-32
Vuylsteke A, Pagel C, Gerrard C, et al. The Papworth Bleeding Risk Score: a stratification scheme for identifying cardiac surgery patients at risk of excessive early postoperative bleeding. Eur J Cardiothorac Surg 2011; 39: 924-30
Klein AA, Collier T, Yeates J, et al. The ACTA PORT-score for predicting perioperative risk of blood transfusion for adult cardiac surgery. Br J Anaesth 2017; 119: 394-401
To date, all relevant red blood cell transfusion risks scores (including ACTA-PORT) have been derived from UK, US, Canadian or European populations. None have yet been developed or validated using data from Australia or New Zealand. One article described a score to predict requirement for platelet transfusion in cardiac surgery using Australian and New Zealand data.
10
Despite similarities in clinical practice, risk prediction tools derived from a UK population do not necessarily retain precision when directly applied to an antipodean population.
11
As noted in a systematic review, “risk prediction tools are developed and initially validated in a specific patient population, in a specific health care setting, at a specific point in time”.
12
This precludes direct translation of risk prediction models between countries and highlights the importance of validating such models with local data before applying them.
Flint AWJ, Bailey M, Reid CM, et al. Preoperative identification of cardiac surgery patients at risk of receiving a platelet transfusion: the Australian Cardiac Surgery Platelet Transfusion (ACSePT) risk prediction tool. Transfusion 2020; 60: 2272-83
Campbell D, Boyle L, Soakell‐Ho M, et al. National risk prediction model for perioperative mortality in non‐cardiac surgery. Br J Surg 2019; 106: 1549-57
Reilly JR, Gabbe BJ, Brown WA, et al. Systematic review of perioperative mortality risk prediction models for adults undergoing inpatient non-cardiac surgery. ANZ J Surg 2021; 91: 860-70
In this study, we determine the accuracy of the original ACTA-PORT score in a cohort of Australian cardiac surgical patients. We subsequently develop and validate a recalibrated version of this score (henceforth, the AntiPORT score) in our population.
Methods
We performed a retrospective validation study using the Australian and New Zealand Society of Cardiac and Thoracic Surgeons (ANZSCTS) database. We included data from all cardiac surgery procedures performed between 1 September 2016 and 31 December 2018. Ethics approval was obtained from the Monash Human Research Ethics Committee (project ID 16923). The ANZSCTS database currently collects perioperative data for all patients undergoing cardiac surgery across 26 public and 30 private hospitals in the region. Since mid-2016, data collection has routinely included preoperative haemoglobin. These dates were chosen due to the importance of haemoglobin data in the prediction of transfusion requirement and because of the availability of audited data at the time of analysis. The components and calculation of the ACTA-PORT score
9
are included in Table 1. With the exception of EuroSCORE I, all other data are collected routinely by the ANZSCTS database.
Klein AA, Collier T, Yeates J, et al. The ACTA PORT-score for predicting perioperative risk of blood transfusion for adult cardiac surgery. Br J Anaesth 2017; 119: 394-401
ACTA-PORT used EuroSCORE as a surrogate for operative mortality, which was not available in our dataset. Some of the EuroSCORE components differed slightly from components collected in the ANZSCTS database, and one (systolic pulmonary artery pressure) was not available at all. We therefore calculated EuroSCORE I (excluding pulmonary artery pressure) based on the closest available variables, as has been previously described.
13
We assessed the correlation between EuroSCORE and “all procedures” AusSCORE. To make future calculations easier using Australian data, we subsequently replaced the EuroSCORE mortality prediction ranges in ACTA-PORT with an equivalent Australian risk prediction score developed from the ANZSCTS database. This score was originally developed for coronary artery bypass grafting procedure only as AusSCORE
14
and subsequently developed to include all cardiac surgical procedures, known as the “all procedures” model.
15
This “all procedures” version was used in our analysis, which is referred to as AusSCORE-AP. In the absence of previously documented techniques of finding equivalent score ranges, we calculated the interquartile range (IQR) of the AusSCORE-AP in our population for each of the five possible categories of EuroSCORE included in the original ACTA-PORT score. Approximations of these IQRs of AusSCORE-AP replaced the EuroSCORE categories in the new model. These two techniques in turn yielded two scores: the original ACTA-PORT score using the calculated EuroSCORE (the ACTA-PORT-ES), and the original ACTA-PORT score using the AusSCORE-AP ranges to replace EuroSCORE (the ACTA-PORT-AS). The discrimination and calibration of each of these two scores were then assessed.
Yap CH, Reid C, Yii M, et al. Validation of the EuroSCORE model in Australia. Eur J Cardiothorac Surg 2006; 29: 441-6
Reid C, Billah B, Dinh D, et al. An Australian risk prediction model for 30-day mortality after isolated coronary artery bypass: the AusSCORE. J Thorac Cardiovasc Surg 2009; 138: 904-10
Billah B, Reid CM, Shardey GC, Smith JA. A preoperative risk prediction model for 30-day mortality following cardiac surgery in an Australian cohort. Eur J Cardiothorac Surg 2010; 37: 1086-92
Finally, given that it is likely transfusion practices will vary across the two populations, it is also likely that the calibration of the score will be affected. We determined a priori that we would recalibrate the ACTA-PORT score using data derived from the Australian population. In order to do this, the dataset was split randomly into two populations (by hospital) in an approximate 75:25 ratio, resulting in a training set and a validation set. Logistic regression was carried out in the training set using allogeneic red cell transfusion as the outcome and the ACTA-PORT-AS as the independent variable. Predictions based on this logistic regression were generated for the validation set using the ACTA-PORT-AS integer score. We termed this the “AntiPORT” (Antipodean Perioperative Risk of Blood Transfusion) recalibration. Discrimination and calibration were then assessed in the validation set.
Baseline characteristics were compared between patients transfused and those not transfused using χ2 tests for categorical data, Student t tests for normally distributed data, and Wilcoxon rank sum tests for non-normally distributed data. Correlation was assessed using the Pearson pairwise correlation. Discrimination of AntiPORT was assessed using area under the receiver operating characteristic (AUROC) curves. Calibration was assessed using calibration plots. Both were assessed using the Brier score. All analyses were carried out using Stata version 16.1.
Results
We analysed data from 30 393 patients from 37 hospitals, whose baseline characteristics are displayed in Table 2. Five patients (0.02%) had missing red cell transfusion data and were excluded, resulting in a final analysis cohort of 30 388 patients. Of these, a total of 10 030 patients (33%) were transfused. Age, female sex, New York Heart Association class 4 status, diabetes mellitus, hypertension, and elevated creatinine were positively associated with risk of transfusion following univariate analysis. In addition, patients undergoing combined surgery or emergency surgery were more likely to require transfusion. Body mass index and preoperative haemoglobin were both negatively associated with transfusion risk.
The EuroSCORE and AusSCORE-AP were strongly correlated (r = 0.83; P < 0.001). The range of EuroSCORE used in ACTA-PORT versus the corresponding AusSCORE-AP are shown in Figure 1. Due to the significant overlap between a EuroSCORE of 1 and 2, these two categories were combined. Assigned ACTA-PORT scores for AusSCORE-AP based on these equivalents are shown in Table 3. The resultant integer AntiPORT scoring system is shown in Table 4.
Discrimination in the Australian population was similar to the UK population, with an AUROC curve for ACTA-PORT-ES and red cell transfusion of 0.76 (95% CI, 0.75–0.76; n = 30 071), and for ACTA-PORT-AS of 0.76 (95% CI, 0.75–0.76; n = 29 487). Brier scores were 0.19 for both. Calibration was poorer, with overprediction of transfusion on calibration plots (Figure 2).
Splitting the dataset yielded a training set (n = 22 407, comprising 28 hospitals) and a validation set (n = 7981, comprising nine hospitals). The median hospital volume per year was 353 (IQR, 169–492) in the training set compared with 324 (IQR, 222–468) in the validation set. Logistic regression in the training set showed ACTA-PORT-AS was strongly associated with red cell transfusion (odds ratio, 1.22; 95% CI, 1.21–1.23; P < 0.001; n = 21 743). AUROC curve in the validation set was 0.76 (95% CI, 0.75–0.77; n = 7744) (Figure 3, A), and Brier score was 0.18. Calibration was improved (Figure 3, B). Sensitivity, specificity and percentage correctly classified are shown in the Online Appendix. The predicted rate of transfusion based on the AntiPORT recalibration is shown in Table 5, with the corresponding predicted rate of transfusion from the original ACTA-PORT score as reference.
Discussion
We performed a retrospective validation study of the ACTA-PORT score for patients undergoing cardiac surgery in Australia. We demonstrated that discrimination between UK and Australian populations for the original ACTA-PORT score was similar when either derived EuroSCORE or AusSCORE-AP was used, but calibration was inaccurate, with overprediction of transfusion based on observed rates. Recalibrating the ACTA-PORT score to the local population resulted in a score that was both discriminative and well calibrated.
Broadly speaking, discrimination represents the ability to predict individual patient outcomes, and calibration reflects the model’s ability to predict groups of patients at different levels of risk (or the agreement between estimated and actual risk). In this study, although the discrimination is acceptable (AUROC curve, 0.7–0.8), the calibration is good. This means that we can predict groups of patients at elevated or reduced risk of transfusion relatively well and potentially intervene in patients with higher risk (eg, > 50% transfusion risk). The ability to accurately predict transfusion risk allows for better directed and more cost-effective use of PBM strategies. Patients with a low score could proceed to surgery without further optimisation, reserving more intensive methods for those at higher risk. Many strategies to optimise anaemia require delays to be effective, and attempts to avoid such delays may necessitate expensive combination treatments such as intravenous iron, vitamin B12, folate and erythropoietin.
16
In addition to guiding the graduated use of PBM, and potentially improving resource allocation, this score could be used for benchmarking. Studies of transfusion in cardiac surgery regularly show that practices vary enormously between centres.
17,
18
It is possible that practices at units with lower risk-adjusted transfusion rates could be investigated and emulated at other hospitals. Finally, there are significant implications to future PBM research. When designing an international clinical trial, the capacity to screen and assess using a common “risk prediction language” can have major impacts on feasibility, as was recently shown in a study that validated multiple risk prediction tools as part of an international cohort study.
19
Furthermore, from the perspective of the bedside clinician, applying the results of clinical trials from other countries to the individual patient is both easier and more reliable if the risk prediction models used in the trial have been validated in the local population first. Consequently, the transformation of ACTA-PORT to “AntiPORT” could allow common enrolment of patients in Australia and the UK into trials that attempted to recruit patients at a high risk of receiving allogeneic blood transfusion as part of cardiac surgery. Alternatively, this could allow a clinician in the UK to apply the results of an Australian trial to their individual practice once the appropriate recalibration factor was considered. Critically, both ACTA-PORT and AntiPORT use data collected routinely for the purposes of clinical audit (including the EuroSCORE in the UK and AusSCORE-AP in Australia), raising the possibility of using these scores as automatically calculated metrics to screen patients for embedded clinical trials.
Spahn DR, Schoenrath F, Spahn GH, et al. Effect of ultra-short-term treatment of patients with iron deficiency or anaemia undergoing cardiac surgery: a prospective randomised trial. Lancet 2019; 393: 2201-12
Klein AA, Chau M, Yeates JA, et al. Preoperative intravenous iron before cardiac surgery: a prospective multicentre feasibility study. Br J Anaesth 2020; 124: 243-50
McQuilten ZK, Andrianopoulos N, Wood EM, et al. Transfusion practice varies widely in cardiac surgery: results from a national registry. J Thorac Cardiovasc Surg 2014; 147: 1684-90
Wong DJN, Harris S, Sahni A, et al. Developing and validating subjective and objective risk-assessment measures for predicting mortality after major surgery: an international prospective cohort study. PLoS Med 2020; 17: e1003253
Even though there are similarities between UK and antipodean cardiac surgical and transfusion practices, previous studies that have attempted to adapt risk prediction scores developed in the northern hemisphere to Australasian patients have shown that this process is not simple. A recent study that attempted to adapt a surgical risk prediction score from the UK to a New Zealand cohort showed that an unadjusted model underpredicted mortality by a factor of five.
11
In cardiac surgery, the commonly used EuroSCORE model was found to significantly overpredict mortality in a representative cohort of Australian patients drawn from six hospitals,
13
necessitating the development of a prediction model that was trialled and validated in a local patient cohort: the AusSCORE.
14
Campbell D, Boyle L, Soakell‐Ho M, et al. National risk prediction model for perioperative mortality in non‐cardiac surgery. Br J Surg 2019; 106: 1549-57
Yap CH, Reid C, Yii M, et al. Validation of the EuroSCORE model in Australia. Eur J Cardiothorac Surg 2006; 29: 441-6
Reid C, Billah B, Dinh D, et al. An Australian risk prediction model for 30-day mortality after isolated coronary artery bypass: the AusSCORE. J Thorac Cardiovasc Surg 2009; 138: 904-10
Multiple scoring systems have been developed to predict red blood cell transfusion risk in cardiac surgery with varying degrees of accuracy, simplicity and generalisability. The best known of these systems are the BRiSc,
8
TRACK,
7
TRUST5 and Goudie scores.
6
All of these scores have significant merits but also weaknesses, most driven by the inverse relationship between simplicity and accuracy. The ACTA-PORT score was chosen for our study as we feel it bests strikes the balance between simplicity, accuracy, and practicality for use by clinicians. As mentioned in the literature, these scores have little value to patients unless they have a positive impact on clinical management and patient outcomes, and to do this they must, above all, be usable.
20
Vuylsteke A, Pagel C, Gerrard C, et al. The Papworth Bleeding Risk Score: a stratification scheme for identifying cardiac surgery patients at risk of excessive early postoperative bleeding. Eur J Cardiothorac Surg 2011; 39: 924-30
Ranucci M, Castelvecchio S, Frigiola A, et al. Predicting transfusions in cardiac surgery: the easier, the better: the Transfusion Risk and Clinical Knowledge score. Vox Sang 2009; 96: 324-32
Goudie R, Sterne JAC, Verheyden V, et al. Risk scores to facilitate preoperative prediction of transfusion and large volume blood transfusion associated with adult cardiac surgery. Br J Anaesth 2015; 114: 757-66
Bartoszko J, Karkouti K. Can predicting transfusion in cardiac surgery help patients? Br J Anaesth 2017; 119: 350-2
The strengths and limitations of this study should be recognised. We used a large national database incorporating multiple surgical units. The split into development and validation cohorts was carried out by the surgical unit (randomly), rather than using individual patients. This suggests the score is valid across surgical units with differing practices. It is notable that PBM and transfusion practices have evolved significantly over the past decade, affecting the applicability of the score to current practice. Recent research regarding liberal versus restrictive transfusion practices, including the TRICS III trial,
21
suggests that restrictive transfusion practices are non-inferior to liberal ones. The publication of this trial towards the end of our study period could be argued to have influenced transfusion practices during our study period, and it is possible that transfusion practices could have changed since. However, a review of the ANZSCTS database between 2017 and 2020 (unpublished data) did not show any broad change in transfusion rates during that period. This is consistent with previous research demonstrating the significant lag between research and a change in clinical practice.
22
For this reason, we believe the data remain relevant. Nonetheless, it is likely that future recalibrations will be necessary as patient risk profile and medical practice change. It is possible that a more discriminative risk prediction tool could have been generated in this population by bespoke design, rather than recalibration of an existing tool. However, for reasons described previously, it was felt that the ability to apply a common risk score internationally was more important.
Mazer CD, Whitlock RP, Fergusson DA, et al. Restrictive or liberal red-cell transfusion for cardiac surgery. N Engl J Med 2017; 377: 2133-44
Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med 2011; 104: 510-20
Conclusions
To conclude, we have developed a local variation of a UK-based transfusion risk prediction score. The AntiPORT score is an accurate scoring system to predict perioperative blood transfusion in patients undergoing cardiac surgery in Australia. The score has a number of potential uses, including the effective allocation of PBM resources, as a quality control initiative as part of a program of comprehensive audit, and as a means of achieving consistent appreciation of risk in patients enrolled in international clinical trials of transfusion practice across different countries. Finally, the AntiPORT score is a clinically useful transfusion-prediction score that can be utilised easily at the bedside by clinicians using the QxMD software package, including an online calculator and the “Calculate by QxMD” application for mobile devices (https://qxmd.com). In addition to its academic value, we hope that this may become a practical and useful tool for perioperative physicians in Australia when caring for patients in preparation for cardiac surgery.
Acknowledgements: The Australian and New Zealand Society of Cardiac and Thoracic Surgeons (ANZSCTS) Cardiac Surgery Database Program is funded by the Department of Health (Victoria), the Clinical Excellence Commission (New South Wales), Queensland Health (Queensland), and by individual units. ANZSCTS Database Research activities are supported through a National Health and Medical Research Council Principal Research Fellowship (GNT 1136372) and Program Grant (GTN 1092642) awarded to Christopher Reid. We thank all the investigators, data managers, and institutions that participate in the program.