Millions of patients survive critical illness each year. ¹ As survival rates have improved, ^{2, 3} the focus has shifted to improving the quality of recovery. Disability is common after critical illness, ^{4, 5, 6, 7} and while the new physical, psychological and cognitive impairments that occur following critical illness (collectively termed post-intensive care syndrome ⁸ ) are often assessed, a global measure of disability or impaired functioning may enable the overall burden of critical illness to be determined. Many current outcome assessment tools measure physical, psychological or cognitive impairments in isolation. As a result, core outcome sets often require a large number of measures to ensure different impairments are captured. ⁹ This increases the burden on patients, health care professionals, and researchers, and may increase withdrawal rates in research.

The World Health Organization Disability Assessment Schedule 2.0 (WHODAS 2.0) is a generic assessment instrument developed by WHO to provide a standardised method for measuring health and disability according to the International Classification of Functioning, Disability and Health (ICF) framework. ¹⁰ It provides a cross-cultural, standardised method for measuring limitations and restrictions on individuals’ activities and participation in their society. ¹¹ It captures the level of function across the six ICF activity and participation domains: cognition, mobility, self-care, interpersonal relationships, work and household roles, and participation in society. It has been used extensively and has been shown to be reliable and valid in numerous populations, including surgical patients, patients with chronic musculoskeletal pain, and patients with depression. ^{12, 13, 14} However, its psychometric properties have not been explored in a critically ill population. There are also limited data about a minimal clinically important difference (MCID), although this has recently been published for a surgical population. ¹⁵

The aim of the current study was to assess the reliability, validity and responsiveness of the 12-item WHODAS 2.0 in patients who have survived critical illness, and to determine the MCID using data collected in a large multicentre observational study.
 

This was a multicentre, prospective cohort study conducted at six metropolitan intensive care units (ICUs) in the state of Victoria, Australia. Three public tertiary teaching hospitals, one public metropolitan hospital and two private hospitals were included (Online Appendix, eTable 1). Ethics committee approval, including a waiver of consent for data collection, was obtained at each site.
 

Consecutive eligible patients were identified from the hospital clinical information system at each participating site. Patients were eligible if they had been admitted to the ICU and received more than 24 hours of mechanical ventilation. Patients who were aged less than 18 years, who had a proven or suspected acute primary brain process that was likely to result in global impairment of consciousness or cognition (eg, traumatic brain injury or stroke), or who did not speak English were excluded.
 

Demographic data were obtained from electronic health records for all eligible patients under a waiver of consent. Patients who survived the hospital admission were contacted initially by mail and invited to participate in telephone interviews at 3 and 6 months after ICU admission. Patients who were still in an acute hospital at 3 or 6 months after ICU admission were not called for the follow-up to reduce any burden on the patient while they were acutely ill.

Patient-reported long term outcomes were assessed at three pre-specified time points: baseline, 3 and 6 months. Baseline health and disability (defined as the status one month before ICU admission) were assessed retrospectively at the 3-month interview. All assessments were performed by telephone by trained central outcome assessors, who were blinded to the details of the patient’s hospital admission. Data were entered into an electronic data capture system (REDCap, Vanderbilt University, USA).
 

Outcome measures included the 12-item WHODAS 2.0, the EQ-5D-5L health-related quality of life (HRQoL) questionnaire, the Lawton Instrumental Activities of Daily Living (IADL) scale, and a global disability rating scale where patients rated their current disability on a 5-point scale with the options of 1, no disability; 2, mild disability; 3, moderate disability; 4, marked disability; or 5, severe disability. Each WHODAS 2.0 item has scores from 0 (no difficulty) to 4 (extreme difficulty) and a total WHODAS 2.0 score range from 0 to 48, with higher scores representing greater disability. ¹¹ The total score is divided by 48 and multiplied by 100 to convert it to a percentage of maximum disability. As outlined in the WHODAS 2.0 manual, where a single WHODAS 2.0 item was missing, the mean value of the remaining 11 items was assigned to the missed item. ¹⁰ The WHODAS 2.0 score was not calculated when more than one item was missed.

The EQ-5D-5L is a generic preference-based health status instrument which is comprised of five dimensions: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. It also incorporates the EuroQoL visual analogue scale (EQ VAS), which is a quantitative measure of the patient’s own judgement of their health status on a scale ranging from the worst imaginable health state (0) to the best imaginable health state (100). ¹⁶ The EQ-5D-5L is recommended as a core outcome measure for critically ill patients. ⁹

The Lawton IADL scale is a valid and reliable functional assessment instrument that measures independent living skills across eight domains of function. ¹⁷ A summary score ranges between 0 and 8, with higher scores indicating greater levels of independence.

Reliability was assessed by measuring internal consistency. An inter-item correlation matrix was visually inspected before measuring Cronbach α and split-half reliability coefficients. Correlations were defined as excellent (> 0.75), moderate (0.50–0.74), fair (0.25–0.49), and no meaningful correlation (< 0.25). ¹⁸ With a sample size of 450 patients, this study has a 99% power (two-sided P = 0.05) to detect a correlation coefficient of 0.2 or greater.

The construct validity of the WHODAS 2.0 at 6 months after ICU admission was evaluated by hypothesising that the WHODAS 2.0 would be more closely related to HRQoL (EQ VAS score) and functionality measures (Lawton IADL total score) and less correlated with weight. Spearman rank correlation coefficients were used to assess the correlation between the WHODAS 2.0 values and continuous variables.

Responsiveness was evaluated by assessing change over time, effect sizes, and percentage of patients showing no change. The WHODAS 2.0 values were compared to assess significance of change over time from baseline to 3 months and from 3 months to 6 months using a Wilcoxon signed rank test. The percentage of patients showing no change was calculated between each time point. The ability of WHODAS 2.0 to detect a meaningful change in the clinical state of a patient was quantified using the Cohen effect size. This is the mean difference in scores between two time points, divided by the average of their standard deviations. Floor and ceiling effects were determined by assessing the number and percentage of participants scoring the lowest value (0) and the highest value (100) at baseline, 3 and 6 months. Floor and ceiling effects below 15% were considered acceptable. ¹⁹

The minimal clinically important difference is the smallest change in score on a given scale that corresponds to a meaningful change in clinical state from the patient’s perspective. ^{20, 21} A combination of anchor and distribution-based methods with triangulation (averaging) of results was used to determine the MCID. Three distribution-based methods were used considering the 6-month WHODAS 2.0 scores: i) 0.3 times the standard deviation (SD); ii) the standard error of the measurement (SEM; calculated as the standard deviation multiplied by the square root of 1 minus the reliability estimate, calculated from the Cronbach α); and iii) 5% of the score range. These methods have been recommended and previously used to determine the WHODAS 2.0 MCID in surgical patients. ^{15, 22}

Anchor-based methods estimate the MCID by relating a change in patient-centred outcome score to a change in clinical scenario or a change on a patient-reported global rating scale. The MCID was estimated anchored to the patient-reported global disability rating scale at 6 months, with a worsening or improving of one step (eg, no disability to mild disability, or mild disability to moderate disability) in the global disability rating at 6 months compared with baseline being considered consistent with a change equivalent to the MCID. The distribution and anchor-based estimates were then averaged to reach a final MCID estimate. As the MCID may be sensitive to different population groups and clinical scenarios, the MCID was further compared in the following subgroups:

All analyses were performed considering a two-sided hypothesis test, with a significance level of 0.05. Analyses were performed using the software R v.4.0.3 (R Core Team, 2016, Austria).
 

A total of 1475 patients, admitted to the ICU between May 2017 and June 2018 were screened, of which 899 patients met the eligibility criteria. Eleven patients opted out of all data collection at the time of follow-up, resulting in 888 enrolled patients. Of these, 222 patients (25%) died by 6 months and 66 (7%) opted out of the 3- and/or 6-month telephone follow-up. The final sample for the current study includes 448 participants (67.3% of surviving participants at 6 months) who completed the baseline WHODAS 2.0 and a follow-up WHODAS 2.0 at 6 months (Figure 1). Baseline data for these patients are shown in Table 1. Characteristics of surviving patients who were included in the study and of the surviving patients who did not complete the follow-up assessment are shown in the Online Appendix, eTable 2. A higher proportion of non-responders were male, but there were no other baseline differences between responders and non-responders.
 

The WHODAS 2.0 demonstrated excellent reliability. The inter-item correlation matrix for the WHODAS 2.0 at 6 months (Online Appendix, eTable 3 and eFigure 1) demonstrated good correlation between items with no evidence of item redundancy, indicated by almost all inter-item correlations between 0.4 and 0.8. Cronbach α coefficient was 0.91 and average split-half coefficient was 0.91 (minimum and maximum split-half coefficients were 0.77 and 0.96 respectively).
 

Construct validity was demonstrated through a moderate correlation between the WHODAS 2.0 and the EQ VAS scores (r = -0.72; 95% CI, -0.77 to -0.67; P < 0.001), and between the WHODAS 2.0 and Lawton IADL scores (r = -0.66; 95% CI, -0.71 to -0.61; P < 0.001) at 6 months (Figure 2, A and B). In addition, no significant correlation was found between WHODAS 2.0 scores at 6 months and the unrelated variable patient body weight (r = 0.07; 95% CI, -0.03 to 0.17; P = 0.163) (Figure 2, C).
 

The effect sizes for change in the WHODAS 2.0 score from baseline (median, 12.5;  interquartile range [IQR], 0.0–39.6) to 3 months (median, 20.8; IQR, 6.2–41.7) and to 6 months (median 16.7; IQR, 4.2–35.4) were low (Figure 3). The WHODAS 2.0 score changed from baseline to 3 months (P = 0.001) and from 3 to 6 months (P < 0.001). At 6 months, the WHODAS 2.0 had a very good scaling property:  the 10th, 25th, 50th, 75th and 90th percentiles were 0.0, 4.2, 16.7, 35.4 and 52.7 respectively.
 

A floor effect was present only at baseline, with 29.1%, 12.1% and 15.8% of the patients at baseline, 3 and 6 months having a WHODAS 2.0 score of 0. A ceiling effect was low at all time points (0.4%, 0.6% and 0.4% at baseline, 3 and 6 months respectively) (Figure 4). The percentage of patients showing no change from baseline to 3 months was 14.2% and from 3 to 6 months it was 12.6%.

The distribution-based estimates of the MCID were similar for SD, SEM and 5% instrument range methods (Table 2). Sensitivity analysis found that MCID estimates were consistent in all but in one subgroup (a lower MCID was found in patients with cardiac arrest). Anchor-based estimates of the MCID, based on Global Disability Score, were higher than the distribution-based estimated (Table 3). The average of the mean differences from the two methods of anchor-based estimation was 12%.

Triangulating the average distribution-based MCID estimate (6%) and two anchor-based MCID (12%), a final MCID estimate of 10% was reached. Using this MCID value to quantify the change in disability from baseline to 6 months, 148 patients (36.4%) had a clinically important increase in disability, 106 patients (26.1%) had a clinically important decrease in disability, and 152 patients (37.4%) had no change in disability.
 

The 12-item WHODAS 2.0 provides a standardised method to assess health and disability according to the ICF framework. The current study has shown the WHODAS 2.0 has strong psychometric qualities in critically ill patients, including excellent reliability, validity and responsiveness. We estimate that the minimal clinically important difference for WHODAS 2.0 in critically ill patients is 10%, indicating that a change in WHODAS 2.0 score of 10% or more is consistent with a clinically meaningful increase or decrease in the level of disability.

The WHODAS 2.0 is being used increasingly as an outcome measure in clinical research, with a recent systematic review finding that, of 811 publications using the WHODAS 2.0, more than 50% were published in the most recent 3 years. ²³ The WHODAS 2.0 has been used to measure disability in critical care studies, ^{6, 24} and its use in ongoing registries and trials is increasing (eg, NCT03793257, NCT02735707, NCT04588311, NCT03671447). Despite this, we are unaware of any previous studies that have evaluated its psychometric properties in the critically ill. A systematic review of the psychometric properties of the 12-item WHODAS 2.0 in the general population and in people with non-acute physical causes of disability concluded that the WHODAS 2.0 is an internally consistent and reliable scale demonstrating overall good correlation with other measures of disability. ²⁵ A study in surgical patients also found that the WHODAS 2.0 had excellent reliability, criterion and convergent validity and responsiveness. ¹⁴ Our findings in a critically ill population are consistent with this research in other patient groups.

The correlation between the WHODAS 2.0 and existing, commonly used critical care outcome measures (EQ VAS and Lawton IADL) was used to determine the validity of the WHODAS 2.0. ^{9, 26} The WHODAS 2.0 showed a moderate correlation with both the EQ VAS (r = -0.72) and the Lawton IADL (r = -0.66), which is expected as the scales measure slightly different constructs. Divergent validity was shown through the lack of correlation between the WHODAS 2.0 and body weight (r = 0.07).
 

Consistent with recommendations, we used a number of alternative methods to estimate the MCID for the WHODAS 2.0, including distribution-based and anchor-based approaches. ²⁷ Based on these methods, our results suggest that the MCID for the WHODAS 2.0 is 10%, which is slightly higher than the 8% MCID in non-surgical populations, ^{10, 28} and higher than the 5% MCID recently reported for a surgical population. ¹⁵ The distribution of the WHODAS 2.0 differs in the surgical population, where higher proportions of patients had lower WHODAS 2.0 scores (75% of scores < 16.7%). ¹⁵ A change of 10% may be represented by larger changes across two items in the WHODAS 2.0 (eg, moving from no difficulty in walking for long distances and standing for long periods to severe difficulty walking for long distances and moderate difficulty standing for long periods) or by small changes across up to five items (eg, moving from no difficulty in washing, getting dressed, concentrating for 10 minutes, learning a new task, and maintaining a friendship to mild difficulties in each of these tasks). The MCID can be used by clinicians and researchers to determine sample size for future studies and to determine the clinical significance of changes in WHODAS 2.0 scores both in research and in clinical practice.
 

Recovery following critical illness is complex and physical, cognitive and psychological impairments are common. ^{4, 5, 6, 29, 30, 31, 32, 33, 34, 35} There is increasing recognition of the need for systematic and frequent assessment for impairments associated with post-intensive care syndrome. ³⁶ Identifying these impairments, including characterisation of patients at risk, is essential to enable development of appropriate preventive and treatment interventions. The core outcome set for evaluating post-discharge outcomes of acute respiratory failure survivors includes the EQ-5D (with the optional addition of SF-36 version 2), the Hospital Anxiety and Depression Scale, and the Impact of Events Scale–Revised. ⁹ Completion of multiple assessment measures may not be feasible on a large scale due to logistical and resource barriers. A simple measure that is able to identify patients with disability may be useful for both clinical practice and research. The 12-item WHODAS 2.0 may serve as a reliable and valid tool to screen patients for disability following critical illness, with subsequent referral of those identified as having disability to enable identification of specific impairments and development of management plans.

The strengths of this study include its prospective, multicentre, longitudinal design, the inclusion of a heterogeneous group of critically ill patients, the collection of a range of detailed clinical parameters (allowing comparisons to other populations), and central assessment of long term outcome measures. Our sample size of over 400 patients is large compared with other validation studies in critically ill patients. ^{37, 38, 39, 40} Our study also has several limitations. We only included patients who were ventilated for over 24 hours, and the psychometric properties of the WHODAS 2.0 in non-ventilated critically ill patients are unknown. Of the 666 survivors at 6 months, only 448 (67.3%) were included in the current study. While the only difference in the baseline characteristics of patients who were included and patients who were not included was that a higher proportion of responders were male, there may be important unmeasured characteristics that could influence our results. A further limitation is that our baseline WHODAS 2.0 score and the baseline global disability rating scale were assessed retrospectively at 3 months after ICU admission. While it is not possible to obtain a baseline measure from sedated, mechanical ventilated patients, the retrospective assessment may have introduced recall bias.
 

Our results demonstrate that the 12-item WHODAS 2.0 is a reliable, valid, and responsive measure of disability in critically ill patients, and that a change in the total WHODAS 2.0 score of 10% represents the minimal clinically important difference. The current study provides support to use the WHODAS 2.0 as a common, international, standardised instrument to measure disability following critical illness.