Full Text View
The psychometric properties and minimal clinically important difference for disability assessment using WHODAS 2.0 in critically ill patients
Alisa M Higgins, Ary Serpa Neto, Michael Bailey, Jonathan Barrett, Rinaldo Bellomo, D James Cooper, Belinda Gabbe, Natalie Linke, Paul S Myles, Michelle Paton, Steve Philpot, Mark Shulman, Meredith Young, Carol L Hodgson, On behalf of the PREDICT Study Investigators*
Crit Care Resusc 2021; 23 (1): 103-112
- Alisa M Higgins 1
- Ary Serpa Neto 1, 2, 3
- Michael Bailey 1, 2
- Jonathan Barrett 4, 5
- Rinaldo Bellomo 1, 2, 3
- D James Cooper 1, 6
- Belinda Gabbe 7
- Natalie Linke 1
- Paul S Myles 7, 8
- Michelle Paton 1, 9
- Steve Philpot 10
- Mark Shulman 7, 8
- Meredith Young 6
- Carol L Hodgson 1, 6
- On behalf of the PREDICT Study Investigators* 11
OBJECTIVES: The 12-item World Health Organization Disability Assessment Schedule 2.0 (WHODAS 2.0) provides a standardised method for measuring health and disability. This study aimed to determine its reliability, validity and responsiveness and to establish the minimum clinically important difference (MCID) in critically ill patients.
DESIGN: Prospective, multicentre cohort study.
SETTING: Intensive care units of six metropolitan hospitals.
PARTICIPANTS: Adults mechanically ventilated for > 24 hours.
MAIN OUTCOME MEASURES: Reliability was assessed by measuring internal consistency. Construct validity was assessed by comparing WHODAS 2.0 scores at 6 months with the EuroQoL visual analogue scale (EQ VAS) and Lawton Instrumental Activities of Daily Living (IADL) scale scores. Responsiveness was evaluated by assessing change over time, effect sizes, and percentage of patients showing no change. The MCID was calculated using both anchor and distribution-based methods with triangulation of results.
MAIN RESULTS: A baseline and 6-month WHODAS 2.0 score were available for 448 patients. The WHODAS 2.0 demonstrated good correlation between items with no evidence of item redundancy. Cronbach α coefficient was 0.91 and average split-half coefficient was 0.91. There was a moderate correlation between the WHODAS 2.0 and the EQ VAS scores (r = -0.72; P < 0.001) and between the WHODAS 2.0 and the Lawton IADL scores (r = -0.66; P < 0.001) at 6 months. The effect sizes for change in the WHODAS 2.0 score from baseline to 3 months and from 3 to 6 months were low. Ceiling effects were not present and floor effects were present at baseline only. The final MCID estimate was 10%.
CONCLUSION: The 12-item WHODAS 2.0 is a reliable, valid and responsive measure of disability in critically ill patients. A change in the total WHODAS 2.0 score of 10% represents the MCID.
- Wunsch H, Angus DC, Harrison DA, et al. Variation in critical care services across North America and Western Europe. Crit Care Med 2008; 36: 2787-93, e1-9.
- Kaukonen KM, Bailey M, Suzuki S, et al. Mortality related to severe sepsis and septic shock among critically ill patients in ANZ, 2000-2012. JAMA 2014; 311: 1308-16.
- Australian and New Zealand Intensive Care Society Centre for Outcome and Resource Evaluation. 2019 Report. Melbourne: ANZICS, 2020. https://www.anzics.com.au/wp-content/uploads/2020/11/2019-CORE-Report.pdf (viewed Dec 2020).
- Herridge MS, Chu LM, Matte A, et al. The RECOVER Program: disability risk groups and 1-year outcome after 7 or more days of mechanical ventilation. Am J Respir Crit Care Med 2016; 194: 831-44.
- Herridge MS, Tansey CM, Matte A, et al. Functional disability 5 years after acute respiratory distress syndrome. N Engl J Med 2011; 364: 1293-304.
- Hodgson CL, Udy AA, Bailey M, et al. The impact of disability in survivors of critical illness. Intensive Care Med 2017; 43: 992-1001.
- Iwashyna TJ, Cooke CR, Wunsch H, Kahn JM. Population burden of long-term survivorship after severe sepsis in older Americans. J Am Geriatr Soc 2012; 60: 1070-7.
- Needham DM, Davidson J, Cohen H, et al. Improving long-term outcomes after discharge from intensive care unit: report from a stakeholders’ conference. Crit Care Med 2012; 40: 502-9.
- Needham DM, Sepulveda KA, Dinglas VD, et al. Core outcome measures for clinical research in acute respiratory failure survivors: an international modified Delphi consensus study. Am J Respir Crit Care Med 2017; 196: 1122-30.
- Üstün TB, Kostanjesek N, Chatterji S, Rehm J; editors. Measuring health and disability: manual for WHO Disability Assessment Schedule (WHODAS 2.0). Geneva; 2010. https://apps.who.int/iris/handle/10665/43974 (viewed Dec 2020).
- Ustun TB, Chatterji S, Kostanjsek N, et al. Developing the World Health Organization Disability Assessment Schedule 2.0. Bull World Health Organ 2010; 88: 815-23.
- Luciano JV, Ayuso-Mateos JL, Fernández A, et al. Psychometric properties of the twelve item World Health Organization Disability Assessment Schedule II (WHO-DAS II) in Spanish primary care patients with a first major depressive episode. J Affect Disord 2010; 121: 52-8.
- Saltychev M, Barlund E, Mattie R, et al. A study of the psychometric properties of 12-item World Health Organization Disability Assessment Schedule 2.0 in a large population of people with chronic musculoskeletal pain. Clin Rehabil 2017; 31: 262-72.
- Shulman MA, Myles PS, Chan MT, et al. Measurement of disability-free survival after surgery. Anesthesiology 2015; 122: 524-36.
- Shulman MA, Kasza J, Myles PS. defining the minimal clinically important difference and patient-acceptable symptom state score for disability assessment in surgical patients. Anesthesiology 2020; 132: 1362-70.
- Herdman M, Gudex C, Lloyd A, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res 2011; 20: 1727-36.
- Lawton MP, Brody EM. Assessment of older people: self-maintaining and instrumental activities of daily living. Gerontologist 1969; 9: 179-86.
- Fan E, Ciesla ND, Truong AD, et al. Inter-rater reliability of manual muscle strength testing in ICU survivors and simulated patients. Intensive Care Med 2010; 36: 1038-43.
- Terwee CB, Bot SD, de Boer MR, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007; 60: 34-42.
- Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol 2008; 61: 102-9.
- Wells G, Beaton D, Shea B, et al. Minimal clinically important differences: review of methods. J Rheumatology 2001; 28: 406-12.
- Myles PS, Myles DB, Galagher W, et al. Minimal clinically important difference for three quality of recovery scales. Anesthesiology 2016; 125: 39-45.
- Federici S, Bracalenti M, Meloni F, Luciano JV. World Health Organization disability assessment schedule 2.0: an international systematic review. Disabil Rehabil 2017; 39: 2347-80.
- Myles PS, Bellomo R, Corcoran T, et al. Restrictive versus liberal fluid therapy for major abdominal surgery. N Engl J Med 2018; 378: 2263-74.
- Saltychev M, Katajapuu N, Bärlund E, Laimi K. Psychometric properties of 12-item self-administered World Health Organization disability assessment schedule 2.0 (WHODAS 2.0) among general population and people with non-acute physical causes of disability — systematic review. Disabil Rehabil 2019: 1-6.
- Hopkins RO, Suchyta MR, Kamdar BB, et al. Instrumental activities of daily living after critical illness: a systematic review. Ann Am Thorac Soc 2017; 14: 1332-43.
- Yost KJ, Eton DT. Combining distribution- and anchor-based approaches to determine minimally important differences: the FACIT experience. Eval Health Prof 2005; 28: 172-91.
- Andrews G, Kemp A, Sunderland M, et al. Normative data for the 12 item WHO Disability Assessment Schedule 2.0. PLoS One 2009; 4: e8343.
- Brummel NE, Bell SP, Girard TD, et al. Frailty and subsequent disability and mortality among patients with critical illness. Am J Respir Crit Care Med 2017; 196: 64-72.
- Davydow DS, Gifford JM, Desai SV, et al. Depression in general intensive care unit survivors: a systematic review. Intensive Care Med 2009; 35: 796-809.
- Davydow DS, Gifford JM, Desai SV, et al. Posttraumatic stress disorder in general intensive care unit survivors: a systematic review. Gen Hosp Psychiatry 2008; 30: 421-34.
- Hopkins RO, Herridge MS. Quality of life, emotional abnormalities, and cognitive dysfunction in survivors of acute lung injury/acute respiratory distress syndrome. Clin Chest Med 2006; 27: 679-89.
- Iwashyna TJ, Ely EW, Smith DM, Langa KM. Long-term cognitive impairment and functional disability among survivors of severe sepsis. JAMA 2010; 304: 1787-94.
- Jackson JC, Pandharipande PP, Girard TD, et al. Depression, post-traumatic stress disorder, and functional disability in survivors of critical illness in the BRAIN-ICU study: a longitudinal cohort study. Lancet Respir Med 2014; 2: 369-79.
- Pandharipande PP, Girard TD, Jackson JC, et al. Long-term cognitive impairment after critical illness. New Engl J Med 2013; 369: 1306-16.
- Elliott D, Davidson JE, Harvey MA, et al. Exploring the scope of post-intensive care syndrome therapy and care: engagement of non-critical care providers and survivors in a second stakeholders meeting. Crit Care Med 2014; 42: 2518-26.
- Denehy L, de Morton NA, Skinner EH, et al. A physical function test for use in the intensive care unit: validity, responsiveness, and predictive utility of the physical function ICU test (scored). Phys Ther 2013; 93: 1636-45.
- Hodgson CL, Needham D, Haines K, et al. Feasibility and inter-rater reliability of the ICU Mobility Scale. Heart Lung 2014; 43: 19-24.
- Parry SM, Granger CL, Berney S, et al. Assessment of impairment and activity limitations in the critically ill: a systematic review of measurement instruments and their clinimetric properties. Intensive Care Med 2015; 41: 744-62.
- Tipping CJ, Bailey MJ, Bellomo R, et al. The ICU Mobility Scale has construct and predictive validity and is responsive. A multicenter observational study. Ann Am Thorac Soc 2016; 13: 887-93.
The World Health Organization Disability Assessment Schedule 2.0 (WHODAS 2.0) is a generic assessment instrument developed by WHO to provide a standardised method for measuring health and disability according to the International Classification of Functioning, Disability and Health (ICF) framework. 10
The aim of the current study was to assess the reliability, validity and responsiveness of the 12-item WHODAS 2.0 in patients who have survived critical illness, and to determine the MCID using data collected in a large multicentre observational study.
Patient-reported long term outcomes were assessed at three pre-specified time points: baseline, 3 and 6 months. Baseline health and disability (defined as the status one month before ICU admission) were assessed retrospectively at the 3-month interview. All assessments were performed by telephone by trained central outcome assessors, who were blinded to the details of the patient’s hospital admission. Data were entered into an electronic data capture system (REDCap, Vanderbilt University, USA).
The EQ-5D-5L is a generic preference-based health status instrument which is comprised of five dimensions: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. It also incorporates the EuroQoL visual analogue scale (EQ VAS), which is a quantitative measure of the patient’s own judgement of their health status on a scale ranging from the worst imaginable health state (0) to the best imaginable health state (100). 16
The Lawton IADL scale is a valid and reliable functional assessment instrument that measures independent living skills across eight domains of function. 17
The construct validity of the WHODAS 2.0 at 6 months after ICU admission was evaluated by hypothesising that the WHODAS 2.0 would be more closely related to HRQoL (EQ VAS score) and functionality measures (Lawton IADL total score) and less correlated with weight. Spearman rank correlation coefficients were used to assess the correlation between the WHODAS 2.0 values and continuous variables.
Responsiveness was evaluated by assessing change over time, effect sizes, and percentage of patients showing no change. The WHODAS 2.0 values were compared to assess significance of change over time from baseline to 3 months and from 3 months to 6 months using a Wilcoxon signed rank test. The percentage of patients showing no change was calculated between each time point. The ability of WHODAS 2.0 to detect a meaningful change in the clinical state of a patient was quantified using the Cohen effect size. This is the mean difference in scores between two time points, divided by the average of their standard deviations. Floor and ceiling effects were determined by assessing the number and percentage of participants scoring the lowest value (0) and the highest value (100) at baseline, 3 and 6 months. Floor and ceiling effects below 15% were considered acceptable. 19
The minimal clinically important difference is the smallest change in score on a given scale that corresponds to a meaningful change in clinical state from the patient’s perspective. 20, 21
Anchor-based methods estimate the MCID by relating a change in patient-centred outcome score to a change in clinical scenario or a change on a patient-reported global rating scale. The MCID was estimated anchored to the patient-reported global disability rating scale at 6 months, with a worsening or improving of one step (eg, no disability to mild disability, or mild disability to moderate disability) in the global disability rating at 6 months compared with baseline being considered consistent with a change equivalent to the MCID. The distribution and anchor-based estimates were then averaged to reach a final MCID estimate. As the MCID may be sensitive to different population groups and clinical scenarios, the MCID was further compared in the following subgroups:
- gender (women v men);
- age (≤ 65 v > 65 years old);
- type of admission (medical v surgical);
- presence of sepsis (yes v no);
- cardiac arrest (yes v no);
- acute respiratory failure (yes v no); and
- severity of illness (Acute Physiology and Chronic Health Evaluation [APACHE] III score ≤ 64 v > 64).
Floor and ceiling effects
Minimal clinically important difference
Triangulating the average distribution-based MCID estimate (6%) and two anchor-based MCID (12%), a final MCID estimate of 10% was reached. Using this MCID value to quantify the change in disability from baseline to 6 months, 148 patients (36.4%) had a clinically important increase in disability, 106 patients (26.1%) had a clinically important decrease in disability, and 152 patients (37.4%) had no change in disability.
The WHODAS 2.0 is being used increasingly as an outcome measure in clinical research, with a recent systematic review finding that, of 811 publications using the WHODAS 2.0, more than 50% were published in the most recent 3 years. 23
The correlation between the WHODAS 2.0 and existing, commonly used critical care outcome measures (EQ VAS and Lawton IADL) was used to determine the validity of the WHODAS 2.0. 9, 26
The strengths of this study include its prospective, multicentre, longitudinal design, the inclusion of a heterogeneous group of critically ill patients, the collection of a range of detailed clinical parameters (allowing comparisons to other populations), and central assessment of long term outcome measures. Our sample size of over 400 patients is large compared with other validation studies in critically ill patients. 37, 38, 39, 40