Susan Kheder, LMSW

Few today deny the importance of the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey.[1, 2] The Centers for Medicare and Medicaid Services' (CMS) Value Based Purchasing incentive, sympathy for the ill, and relationships between the patient experience and quality of care provide sufficient justification.[3, 4] How to improve the experience scores is not well understood. The national scores have improved only modestly over the past 3 years.[5, 6]

Clinicians may not typically compartmentalize what they do to improve outcomes versus the patient experience. A possible source for new improvement strategies is to understand the types of patients in which both adverse outcomes and suboptimal experiences are likely to occur, then redesign the multidisciplinary care processes to address both concurrently.[7] Previous studies support the existence of a relationship between a higher mortality risk on admission and subsequent worse outcomes, as well as a relationship between worse outcomes and lower HCAHPS scores.[8, 9, 10, 11, 12, 13] We hypothesized the mortality risk on admission, patient experience, and outcomes might share a triad relationship (Figure 1). In this article we explore the third edge of this triangle, the association between the mortality risk on admission and the subsequent patient experience.

Figure 1

METHODS

We studied HCAHPS from 5 midwestern US hospitals having 113, 136, 304, 443, and 537 licensed beds, affiliated with the same multistate healthcare system. HCAHPS telephone surveys were administered via a vendor to a random sample of inpatients 18 years of age or older discharged from January 1, 2012 through June 30, 2014. Per CMS guidelines, surveyed patients must have been discharged alive after a hospital stay of at least 1 night.[14] Patients ineligible to be surveyed included those discharged to skilled nursing facilities or hospice care.[14] Because not all study hospitals provided obstetrical services, we restricted the analyses to medical and surgical respondents. With the permission of the local institutional review board, subjects' survey responses were linked confidentially to their clinical data.

We focused on the 8 dimensions of the care experience used in the CMS Value Based Purchasing program: communication with doctors, communication with nurses, responsiveness of hospital staff, pain management, communication about medicines, discharge information, hospital environment, and an overall rating of the hospital.[2] Following the scoring convention for publicly reported results, we dichotomized the 4‐level Likert scales into the most favorable response possible (always) versus all other responses.[15] Similarly we dichotomized the hospital rating scale at 9 and above for the most favorable response.

Our unit of analysis was an individual hospitalization. Our primary outcome of interest was whether or not the respondent provided the most favorable response for all questions answered within a given domain. For example, for the physician communication domain, the patient must have answered always to each of the 3 questions answered within the domain. This approach is appropriate for learning which patient‐level factors influence the survey responses, but differs from that used for the publically reported domain scores for which the relative performance of hospitals is the focus.[16] For the latter, the hospital was the unit of analysis, and the domain score was basically the average of the percentages of top box scores for the questions within a domain. For example, if 90% respondents from a hospital provided a top box response for courtesy, 80% for listening, and 70% for explanation, the hospital's physician communication score would be (90 + 80 + 70)/3 = 80%.[17]

Our primary explanatory variable was a binary high versus low mortality‐risk status of the respondent on admission based on age, gender, prior hospitalizations, clinical laboratory values, and diagnoses present on admission.[12] The calculated mortality risk was then dichotomized prior to the analysis at a probability of dying equal to 0.07 or higher. This corresponded roughly to the top quintile of predicted risk found in prior studies.[12, 13] During the study period, only 2 of the hospitals had the capability of generating mortality scores in real time, so for this study the mortality risk was calculated retrospectively, using information deemed present on admission.[12]

To estimate the sample size, we assumed that the high‐risk strata contained approximately 13% of respondents, and that the true percent of top box responses from patients in the lower‐risk stratum was approximately 80% for each domain. A meaningful difference in the proportion of most favorable responses was considered as an odds ratio (OR) of 0.75 for high risk versus low risk. A significance level of P < 0.003 was set to control study‐wide type I error due to multiple comparisons. We determined that for each dimension, approximately 8583 survey responses would be required for low‐risk patients and approximately 1116 responses for high‐risk patients to achieve 80% power under these assumptions. We were able to accrue the target number of surveys for all but 3 domains (pain management, communication about medicines, and hospital environment) because of data availability, and because patients are allowed to skip questions that do not apply. Univariate relationships were examined with ², t test, and Fisher exact tests where indicated. Generalized linear mixed regression models with a logit link were fit to determine the association between patient mortality risk and the top box experience for each of the HCAHPS domains and for the overall rating. The patient's hospital was considered a random intercept to account for the patient‐hospital hierarchy and the unmeasured hospital‐specific practices. The multivariable models controlled for gender plus the HCAHPS patient‐mix adjustment variables of age, education, self‐rated health, language spoken at home, service line, and the number of days elapsed between the date of discharge and date of the survey.[18, 19, 20, 21] In keeping with the industry analyses, a second order interaction variable was included between surgery patients and age.[19] We considered the potential collinearity between the mortality risk status, age, and patient self‐reported health. We found the variance inflation factors were small, so we drew inference from the full multivariable model.

We also performed a post hoc sensitivity analysis to determine if our conclusions were biased due to missing patient responses for the risk‐adjustment variables. Accordingly, we imputed the response level most negatively associated with most HCAHPS domains as previously reported and reran the multivariable models.[19] We did not find a meaningful change in our conclusions (see Supporting Figure 1 in the online version of this article).

RESULTS

The hospitals discharged 152,333 patients during the study period, 39,905 of whom (26.2 %) had a predicted 30‐day mortality risk greater or equal to 0.07 (Table 1). Of the 36,280 high‐risk patients discharged alive, 5901 (16.3%) died in the ensuing 30 days, and 7951 (22%) were readmitted.

The high‐risk subset was under‐represented in those who completed the HCAHPS survey with 7% (2513/36,280) completing surveys compared to 13% of low‐risk patients (14,996/111,600) (P < 0.0001). Moreover, compared to high‐risk patients who were alive at discharge but did not complete surveys, high‐risk survey respondents were less likely to die within 30 days (16/2513 = 0.64% vs 5885/33,767 = 17.4%, P < 0.0001), and less likely to be readmitted (424/2513 = 16.9% vs 7527/33,767 = 22.3%, P < 0.0001).

On average, high‐risk respondents (compared to low risk) were slightly less likely to be female (52.4% vs 57.9%), less educated (30.6% with some college vs 42.3%), less likely to have been on a surgical service (13.8% vs 42.5%), and less likely to report good or better health (49.0% vs 68.0%, all P < 0.0001). High‐risk respondents were also older (76.6 vs 63.1 years), stayed in the hospital longer (4.6 vs 3.5 days), and received care from more specialties (2.2 vs 1.7 specialties) (all P < 0.0001). High‐risk respondents experienced more 30‐day readmissions (16.9% vs 8.1%) and deaths within 30 days (0.6 % vs 0.1 %, all P < 0.0001) than their low‐risk counterparts.

High‐risk respondents were less likely to provide the most favorable response (unadjusted) for all HCAHPS domains compared to low‐risk respondents, although the difference was not significant for discharge information (Table 1, Figure 2A). The gradient between high‐risk and low‐risk patients was seen for all domains within each hospital except for pain management, hospital environment, and overall rating (Figure 3).

Figure 2

Figure 3

The multivariable regression models examined whether the mortality risk on admission simply represented older medical patients and/or those who considered themselves unhealthy (Figure 2B) (see Supporting Table 1 in the online version of this article). Accounting for hospital, age, gender, language, self‐reported health, educational level, service line, and days elapsed from discharge, respondents in the high‐mortality‐risk stratum were still less likely to report an always experience for doctor communication (OR: 0.85; 95% confidence interval [CI]: 0.77‐0.94) and responsiveness of hospital staff (OR: 0.77; 95% CI: 0.69‐0.85). Higher‐risk patients also tended to have less favorable experiences with nursing communication, although the CI crossed 1 (OR: 0.91; 95% CI: 0.82‐1.01). In contrast, higher‐risk patients were more likely to provide top box responses for having received discharge information (OR: 1.30; 95% CI: 1.14‐1.48). We did not find independent associations between mortality risk and the other domains when the patient risk‐adjustment factors were considered.[18, 19, 20, 21]

DISCUSSION

The high‐mortality‐risk stratum on admission contained a subset of patients who provided less favorable responses for almost all incentivized HCAHPS domains when other risk‐adjustment variables were not taken into consideration (Figure 2A). These univariate relationships weakened when we controlled for gender, the standard HCAHPS risk‐adjustment variables, and individual hospital influences (Figure 2B).[18, 19, 20, 21] After multivariable adjustment, survey respondents in the high‐risk category remained less likely to report their physicians always communicated well and to experience hospital staff responding quickly, but were more likely to report receiving discharge information. We did not find an independent association between the underlying mortality risk and the other incentivized HCAHPS domains after risk adjustment.

We are cautious with initial interpretations of our findings in light of the relatively small number of hospitals studied and the substantial survey response bias of healthier patients. Undoubtedly, the CMS exclusions of patients discharged to hospice or skilled nursing facilities provide a partial explanation for the selection bias, but the experience of those at high risk who did not complete surveys remains conjecture at this point.[14] Previous evidence suggests sicker patients and those with worse experiences are less likely to respond to the HCAHPS survey.[18, 22] On the other hand, it is possible that high‐risk nonrespondents who died could have received better communication and staff responsiveness.[23, 24] We were unable to find a previous, patient‐level study that explicitly tested the association between the admission mortality risk and the subsequent patient experience, yet our findings are consistent with a previous single‐site study of a surgical population showing lower overall ratings from patients with higher Injury Severity Scores.[25]

Our findings provide evidence of complex relationships among admission mortality risk, the 3 domains of the patient experience, and adverse outcomes, at least within the study hospitals (Figure 1). The developing field of palliative care has found very ill patients have special communication needs regarding goals of care, as well as physical symptoms, anxiety, and depression that might prompt more calls for help.[26] If these needs were more important for high‐risk compared to low‐risk patients, and were either not recognized or adequately addressed by the clinical teams at the study hospitals, then the high‐risk patients may have been less likely to perceive their physicians listened and explained things well, or that staff responded promptly to their requests for help.[27] On the other hand, the higher ratings for discharge information suggest the needs of the high‐risk patients were relatively easier to address by current practices at these hospitals. The lack of association between the mortality risk and the other HCAHPS domains may reflect the relatively stronger influence of age, gender, educational level, provider variability, and other unmeasured influences within the study sites, or that the level of patient need was similar among high‐risk and low‐risk patients within these domains.[27]

There are several possible confounders of our observed relationship between mortality risk and HCAHPS scores. The first category of confounders represents patient level variables that might impact the communication scores, some of which are part of the formula of our mortality prediction rule, for example, cognitive impairment and emergent admission.[18, 22, 27] The effect of the mortality risk could also be confounded by unmeasured patient‐level factors such as lower socioeconomic status.[28] A second category of confounders pertains to clinical outcomes and processes of care associated with serious illness irrespective of the risk of dying. More physicians involved in the care of the seriously ill (Table 1) may impact the communication scores, due to the larger opportunity for conflicting or confusing information presented to patients and their families.[29] The longer hospital stays, readmissions, and adverse events of the seriously ill may also underlie the apparent association between mortality risk and HCAHPS scores.[8, 9, 10]

Even if we do not understand precisely if and how the mortality risk might be associated with suboptimal physician communication and staff responsiveness, there may still be some value in considering how these possible relationships could be leveraged to improve patient care. We recall Berwick's insight that every system is perfectly designed to achieve the results it achieves.[7] We have previously argued for the use of mortality‐risk strata to initiate concurrent, multidisciplinary care processes to reduce adverse outcomes.[12, 13] Others have used risk‐based approaches for anticipating clinical deterioration of surgical patients, and determining the intensity of individualized case management services.[30, 31] In this framework, all patients receive a standard set of care processes, but higher‐risk patients receive additional efforts to promote better outcomes. An efficient extension of this approach is to assume patients at risk for adverse outcomes also have additional needs for communication, coordination of specialty care, and timely response to the call button. The admission mortality risk could be used as a determinant for the level of nurse staffing to reduce deaths plus shorten response time to the call button.[32, 33] Hospitalists and specialists could work together on a standard way to conference among themselves for high‐risk patients above that needed for less‐complex cases. Patients in the high‐risk strata could be screened early to see if they might benefit from the involvement of the palliative care team.[26]

Our study has limitations in addition to those already noted. First, our use of the top box as the formulation of the outcome of interest could be challenged. We chose this to be relevant to the Value‐Based Purchasing environment, but other formulations or use of other survey instruments may be needed to tease out the complex relationships we hypothesize. Next, we do not know the extent to which the patients and care processes reflected in our study represent other settings. The literature indicates some hospitals are more effective in providing care for certain subgroups of patients than for others, and that there is substantial regional variation in care intensity that is in turn associated with the patient experience.[29, 34] The mortality‐risk experience relationship for nonstudy hospitals could be weaker or stronger than what we found. Third, many hospitals may not have the capability to generate mortality scores on admission, although more hospitals may be able to do so in the future.[35] Explicit risk strata have the benefit of providing members of the multidisciplinary team with a quick preview of the clinical needs and prognoses of patients in much the way that the term baroque alerts the audience to the genre of music. Still, clinicians in any hospital could attempt to improve outcomes and experience through the use of informal risk assessment during interdisciplinary care rounds or by simply asking the team if they would be surprised if this patient died in the next year.[30, 36] Finally, we do not know if awareness of an experience risk will identify remediable practices that actually improve the experience. Clearly, future studies are needed to answer all of these concerns.

We have provided evidence that a group of patients who were at elevated risk for dying at the time of admission were more likely to have issues with physician communication and staff responsiveness than their lower‐risk counterparts. While we await future studies to confirm these findings, clinical teams can consider whether or not their patients' HCAHPS scores reflect how their system of care addresses the needs of these vulnerable people.

Few today deny the importance of the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey.[1, 2] The Centers for Medicare and Medicaid Services' (CMS) Value Based Purchasing incentive, sympathy for the ill, and relationships between the patient experience and quality of care provide sufficient justification.[3, 4] How to improve the experience scores is not well understood. The national scores have improved only modestly over the past 3 years.[5, 6]

Clinicians may not typically compartmentalize what they do to improve outcomes versus the patient experience. A possible source for new improvement strategies is to understand the types of patients in which both adverse outcomes and suboptimal experiences are likely to occur, then redesign the multidisciplinary care processes to address both concurrently.[7] Previous studies support the existence of a relationship between a higher mortality risk on admission and subsequent worse outcomes, as well as a relationship between worse outcomes and lower HCAHPS scores.[8, 9, 10, 11, 12, 13] We hypothesized the mortality risk on admission, patient experience, and outcomes might share a triad relationship (Figure 1). In this article we explore the third edge of this triangle, the association between the mortality risk on admission and the subsequent patient experience.

Figure 1

METHODS

We studied HCAHPS from 5 midwestern US hospitals having 113, 136, 304, 443, and 537 licensed beds, affiliated with the same multistate healthcare system. HCAHPS telephone surveys were administered via a vendor to a random sample of inpatients 18 years of age or older discharged from January 1, 2012 through June 30, 2014. Per CMS guidelines, surveyed patients must have been discharged alive after a hospital stay of at least 1 night.[14] Patients ineligible to be surveyed included those discharged to skilled nursing facilities or hospice care.[14] Because not all study hospitals provided obstetrical services, we restricted the analyses to medical and surgical respondents. With the permission of the local institutional review board, subjects' survey responses were linked confidentially to their clinical data.

We focused on the 8 dimensions of the care experience used in the CMS Value Based Purchasing program: communication with doctors, communication with nurses, responsiveness of hospital staff, pain management, communication about medicines, discharge information, hospital environment, and an overall rating of the hospital.[2] Following the scoring convention for publicly reported results, we dichotomized the 4‐level Likert scales into the most favorable response possible (always) versus all other responses.[15] Similarly we dichotomized the hospital rating scale at 9 and above for the most favorable response.

Our unit of analysis was an individual hospitalization. Our primary outcome of interest was whether or not the respondent provided the most favorable response for all questions answered within a given domain. For example, for the physician communication domain, the patient must have answered always to each of the 3 questions answered within the domain. This approach is appropriate for learning which patient‐level factors influence the survey responses, but differs from that used for the publically reported domain scores for which the relative performance of hospitals is the focus.[16] For the latter, the hospital was the unit of analysis, and the domain score was basically the average of the percentages of top box scores for the questions within a domain. For example, if 90% respondents from a hospital provided a top box response for courtesy, 80% for listening, and 70% for explanation, the hospital's physician communication score would be (90 + 80 + 70)/3 = 80%.[17]

Our primary explanatory variable was a binary high versus low mortality‐risk status of the respondent on admission based on age, gender, prior hospitalizations, clinical laboratory values, and diagnoses present on admission.[12] The calculated mortality risk was then dichotomized prior to the analysis at a probability of dying equal to 0.07 or higher. This corresponded roughly to the top quintile of predicted risk found in prior studies.[12, 13] During the study period, only 2 of the hospitals had the capability of generating mortality scores in real time, so for this study the mortality risk was calculated retrospectively, using information deemed present on admission.[12]

To estimate the sample size, we assumed that the high‐risk strata contained approximately 13% of respondents, and that the true percent of top box responses from patients in the lower‐risk stratum was approximately 80% for each domain. A meaningful difference in the proportion of most favorable responses was considered as an odds ratio (OR) of 0.75 for high risk versus low risk. A significance level of P < 0.003 was set to control study‐wide type I error due to multiple comparisons. We determined that for each dimension, approximately 8583 survey responses would be required for low‐risk patients and approximately 1116 responses for high‐risk patients to achieve 80% power under these assumptions. We were able to accrue the target number of surveys for all but 3 domains (pain management, communication about medicines, and hospital environment) because of data availability, and because patients are allowed to skip questions that do not apply. Univariate relationships were examined with ², t test, and Fisher exact tests where indicated. Generalized linear mixed regression models with a logit link were fit to determine the association between patient mortality risk and the top box experience for each of the HCAHPS domains and for the overall rating. The patient's hospital was considered a random intercept to account for the patient‐hospital hierarchy and the unmeasured hospital‐specific practices. The multivariable models controlled for gender plus the HCAHPS patient‐mix adjustment variables of age, education, self‐rated health, language spoken at home, service line, and the number of days elapsed between the date of discharge and date of the survey.[18, 19, 20, 21] In keeping with the industry analyses, a second order interaction variable was included between surgery patients and age.[19] We considered the potential collinearity between the mortality risk status, age, and patient self‐reported health. We found the variance inflation factors were small, so we drew inference from the full multivariable model.

We also performed a post hoc sensitivity analysis to determine if our conclusions were biased due to missing patient responses for the risk‐adjustment variables. Accordingly, we imputed the response level most negatively associated with most HCAHPS domains as previously reported and reran the multivariable models.[19] We did not find a meaningful change in our conclusions (see Supporting Figure 1 in the online version of this article).

RESULTS

The hospitals discharged 152,333 patients during the study period, 39,905 of whom (26.2 %) had a predicted 30‐day mortality risk greater or equal to 0.07 (Table 1). Of the 36,280 high‐risk patients discharged alive, 5901 (16.3%) died in the ensuing 30 days, and 7951 (22%) were readmitted.

The high‐risk subset was under‐represented in those who completed the HCAHPS survey with 7% (2513/36,280) completing surveys compared to 13% of low‐risk patients (14,996/111,600) (P < 0.0001). Moreover, compared to high‐risk patients who were alive at discharge but did not complete surveys, high‐risk survey respondents were less likely to die within 30 days (16/2513 = 0.64% vs 5885/33,767 = 17.4%, P < 0.0001), and less likely to be readmitted (424/2513 = 16.9% vs 7527/33,767 = 22.3%, P < 0.0001).

On average, high‐risk respondents (compared to low risk) were slightly less likely to be female (52.4% vs 57.9%), less educated (30.6% with some college vs 42.3%), less likely to have been on a surgical service (13.8% vs 42.5%), and less likely to report good or better health (49.0% vs 68.0%, all P < 0.0001). High‐risk respondents were also older (76.6 vs 63.1 years), stayed in the hospital longer (4.6 vs 3.5 days), and received care from more specialties (2.2 vs 1.7 specialties) (all P < 0.0001). High‐risk respondents experienced more 30‐day readmissions (16.9% vs 8.1%) and deaths within 30 days (0.6 % vs 0.1 %, all P < 0.0001) than their low‐risk counterparts.

High‐risk respondents were less likely to provide the most favorable response (unadjusted) for all HCAHPS domains compared to low‐risk respondents, although the difference was not significant for discharge information (Table 1, Figure 2A). The gradient between high‐risk and low‐risk patients was seen for all domains within each hospital except for pain management, hospital environment, and overall rating (Figure 3).

Figure 2

Figure 3

The multivariable regression models examined whether the mortality risk on admission simply represented older medical patients and/or those who considered themselves unhealthy (Figure 2B) (see Supporting Table 1 in the online version of this article). Accounting for hospital, age, gender, language, self‐reported health, educational level, service line, and days elapsed from discharge, respondents in the high‐mortality‐risk stratum were still less likely to report an always experience for doctor communication (OR: 0.85; 95% confidence interval [CI]: 0.77‐0.94) and responsiveness of hospital staff (OR: 0.77; 95% CI: 0.69‐0.85). Higher‐risk patients also tended to have less favorable experiences with nursing communication, although the CI crossed 1 (OR: 0.91; 95% CI: 0.82‐1.01). In contrast, higher‐risk patients were more likely to provide top box responses for having received discharge information (OR: 1.30; 95% CI: 1.14‐1.48). We did not find independent associations between mortality risk and the other domains when the patient risk‐adjustment factors were considered.[18, 19, 20, 21]

DISCUSSION

The high‐mortality‐risk stratum on admission contained a subset of patients who provided less favorable responses for almost all incentivized HCAHPS domains when other risk‐adjustment variables were not taken into consideration (Figure 2A). These univariate relationships weakened when we controlled for gender, the standard HCAHPS risk‐adjustment variables, and individual hospital influences (Figure 2B).[18, 19, 20, 21] After multivariable adjustment, survey respondents in the high‐risk category remained less likely to report their physicians always communicated well and to experience hospital staff responding quickly, but were more likely to report receiving discharge information. We did not find an independent association between the underlying mortality risk and the other incentivized HCAHPS domains after risk adjustment.

We are cautious with initial interpretations of our findings in light of the relatively small number of hospitals studied and the substantial survey response bias of healthier patients. Undoubtedly, the CMS exclusions of patients discharged to hospice or skilled nursing facilities provide a partial explanation for the selection bias, but the experience of those at high risk who did not complete surveys remains conjecture at this point.[14] Previous evidence suggests sicker patients and those with worse experiences are less likely to respond to the HCAHPS survey.[18, 22] On the other hand, it is possible that high‐risk nonrespondents who died could have received better communication and staff responsiveness.[23, 24] We were unable to find a previous, patient‐level study that explicitly tested the association between the admission mortality risk and the subsequent patient experience, yet our findings are consistent with a previous single‐site study of a surgical population showing lower overall ratings from patients with higher Injury Severity Scores.[25]

Our findings provide evidence of complex relationships among admission mortality risk, the 3 domains of the patient experience, and adverse outcomes, at least within the study hospitals (Figure 1). The developing field of palliative care has found very ill patients have special communication needs regarding goals of care, as well as physical symptoms, anxiety, and depression that might prompt more calls for help.[26] If these needs were more important for high‐risk compared to low‐risk patients, and were either not recognized or adequately addressed by the clinical teams at the study hospitals, then the high‐risk patients may have been less likely to perceive their physicians listened and explained things well, or that staff responded promptly to their requests for help.[27] On the other hand, the higher ratings for discharge information suggest the needs of the high‐risk patients were relatively easier to address by current practices at these hospitals. The lack of association between the mortality risk and the other HCAHPS domains may reflect the relatively stronger influence of age, gender, educational level, provider variability, and other unmeasured influences within the study sites, or that the level of patient need was similar among high‐risk and low‐risk patients within these domains.[27]

There are several possible confounders of our observed relationship between mortality risk and HCAHPS scores. The first category of confounders represents patient level variables that might impact the communication scores, some of which are part of the formula of our mortality prediction rule, for example, cognitive impairment and emergent admission.[18, 22, 27] The effect of the mortality risk could also be confounded by unmeasured patient‐level factors such as lower socioeconomic status.[28] A second category of confounders pertains to clinical outcomes and processes of care associated with serious illness irrespective of the risk of dying. More physicians involved in the care of the seriously ill (Table 1) may impact the communication scores, due to the larger opportunity for conflicting or confusing information presented to patients and their families.[29] The longer hospital stays, readmissions, and adverse events of the seriously ill may also underlie the apparent association between mortality risk and HCAHPS scores.[8, 9, 10]

Even if we do not understand precisely if and how the mortality risk might be associated with suboptimal physician communication and staff responsiveness, there may still be some value in considering how these possible relationships could be leveraged to improve patient care. We recall Berwick's insight that every system is perfectly designed to achieve the results it achieves.[7] We have previously argued for the use of mortality‐risk strata to initiate concurrent, multidisciplinary care processes to reduce adverse outcomes.[12, 13] Others have used risk‐based approaches for anticipating clinical deterioration of surgical patients, and determining the intensity of individualized case management services.[30, 31] In this framework, all patients receive a standard set of care processes, but higher‐risk patients receive additional efforts to promote better outcomes. An efficient extension of this approach is to assume patients at risk for adverse outcomes also have additional needs for communication, coordination of specialty care, and timely response to the call button. The admission mortality risk could be used as a determinant for the level of nurse staffing to reduce deaths plus shorten response time to the call button.[32, 33] Hospitalists and specialists could work together on a standard way to conference among themselves for high‐risk patients above that needed for less‐complex cases. Patients in the high‐risk strata could be screened early to see if they might benefit from the involvement of the palliative care team.[26]

Our study has limitations in addition to those already noted. First, our use of the top box as the formulation of the outcome of interest could be challenged. We chose this to be relevant to the Value‐Based Purchasing environment, but other formulations or use of other survey instruments may be needed to tease out the complex relationships we hypothesize. Next, we do not know the extent to which the patients and care processes reflected in our study represent other settings. The literature indicates some hospitals are more effective in providing care for certain subgroups of patients than for others, and that there is substantial regional variation in care intensity that is in turn associated with the patient experience.[29, 34] The mortality‐risk experience relationship for nonstudy hospitals could be weaker or stronger than what we found. Third, many hospitals may not have the capability to generate mortality scores on admission, although more hospitals may be able to do so in the future.[35] Explicit risk strata have the benefit of providing members of the multidisciplinary team with a quick preview of the clinical needs and prognoses of patients in much the way that the term baroque alerts the audience to the genre of music. Still, clinicians in any hospital could attempt to improve outcomes and experience through the use of informal risk assessment during interdisciplinary care rounds or by simply asking the team if they would be surprised if this patient died in the next year.[30, 36] Finally, we do not know if awareness of an experience risk will identify remediable practices that actually improve the experience. Clearly, future studies are needed to answer all of these concerns.

We have provided evidence that a group of patients who were at elevated risk for dying at the time of admission were more likely to have issues with physician communication and staff responsiveness than their lower‐risk counterparts. While we await future studies to confirm these findings, clinical teams can consider whether or not their patients' HCAHPS scores reflect how their system of care addresses the needs of these vulnerable people.

Few today deny the importance of the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey.[1, 2] The Centers for Medicare and Medicaid Services' (CMS) Value Based Purchasing incentive, sympathy for the ill, and relationships between the patient experience and quality of care provide sufficient justification.[3, 4] How to improve the experience scores is not well understood. The national scores have improved only modestly over the past 3 years.[5, 6]

Clinicians may not typically compartmentalize what they do to improve outcomes versus the patient experience. A possible source for new improvement strategies is to understand the types of patients in which both adverse outcomes and suboptimal experiences are likely to occur, then redesign the multidisciplinary care processes to address both concurrently.[7] Previous studies support the existence of a relationship between a higher mortality risk on admission and subsequent worse outcomes, as well as a relationship between worse outcomes and lower HCAHPS scores.[8, 9, 10, 11, 12, 13] We hypothesized the mortality risk on admission, patient experience, and outcomes might share a triad relationship (Figure 1). In this article we explore the third edge of this triangle, the association between the mortality risk on admission and the subsequent patient experience.

Figure 1

METHODS

We studied HCAHPS from 5 midwestern US hospitals having 113, 136, 304, 443, and 537 licensed beds, affiliated with the same multistate healthcare system. HCAHPS telephone surveys were administered via a vendor to a random sample of inpatients 18 years of age or older discharged from January 1, 2012 through June 30, 2014. Per CMS guidelines, surveyed patients must have been discharged alive after a hospital stay of at least 1 night.[14] Patients ineligible to be surveyed included those discharged to skilled nursing facilities or hospice care.[14] Because not all study hospitals provided obstetrical services, we restricted the analyses to medical and surgical respondents. With the permission of the local institutional review board, subjects' survey responses were linked confidentially to their clinical data.

We focused on the 8 dimensions of the care experience used in the CMS Value Based Purchasing program: communication with doctors, communication with nurses, responsiveness of hospital staff, pain management, communication about medicines, discharge information, hospital environment, and an overall rating of the hospital.[2] Following the scoring convention for publicly reported results, we dichotomized the 4‐level Likert scales into the most favorable response possible (always) versus all other responses.[15] Similarly we dichotomized the hospital rating scale at 9 and above for the most favorable response.

Our unit of analysis was an individual hospitalization. Our primary outcome of interest was whether or not the respondent provided the most favorable response for all questions answered within a given domain. For example, for the physician communication domain, the patient must have answered always to each of the 3 questions answered within the domain. This approach is appropriate for learning which patient‐level factors influence the survey responses, but differs from that used for the publically reported domain scores for which the relative performance of hospitals is the focus.[16] For the latter, the hospital was the unit of analysis, and the domain score was basically the average of the percentages of top box scores for the questions within a domain. For example, if 90% respondents from a hospital provided a top box response for courtesy, 80% for listening, and 70% for explanation, the hospital's physician communication score would be (90 + 80 + 70)/3 = 80%.[17]

Our primary explanatory variable was a binary high versus low mortality‐risk status of the respondent on admission based on age, gender, prior hospitalizations, clinical laboratory values, and diagnoses present on admission.[12] The calculated mortality risk was then dichotomized prior to the analysis at a probability of dying equal to 0.07 or higher. This corresponded roughly to the top quintile of predicted risk found in prior studies.[12, 13] During the study period, only 2 of the hospitals had the capability of generating mortality scores in real time, so for this study the mortality risk was calculated retrospectively, using information deemed present on admission.[12]

To estimate the sample size, we assumed that the high‐risk strata contained approximately 13% of respondents, and that the true percent of top box responses from patients in the lower‐risk stratum was approximately 80% for each domain. A meaningful difference in the proportion of most favorable responses was considered as an odds ratio (OR) of 0.75 for high risk versus low risk. A significance level of P < 0.003 was set to control study‐wide type I error due to multiple comparisons. We determined that for each dimension, approximately 8583 survey responses would be required for low‐risk patients and approximately 1116 responses for high‐risk patients to achieve 80% power under these assumptions. We were able to accrue the target number of surveys for all but 3 domains (pain management, communication about medicines, and hospital environment) because of data availability, and because patients are allowed to skip questions that do not apply. Univariate relationships were examined with ², t test, and Fisher exact tests where indicated. Generalized linear mixed regression models with a logit link were fit to determine the association between patient mortality risk and the top box experience for each of the HCAHPS domains and for the overall rating. The patient's hospital was considered a random intercept to account for the patient‐hospital hierarchy and the unmeasured hospital‐specific practices. The multivariable models controlled for gender plus the HCAHPS patient‐mix adjustment variables of age, education, self‐rated health, language spoken at home, service line, and the number of days elapsed between the date of discharge and date of the survey.[18, 19, 20, 21] In keeping with the industry analyses, a second order interaction variable was included between surgery patients and age.[19] We considered the potential collinearity between the mortality risk status, age, and patient self‐reported health. We found the variance inflation factors were small, so we drew inference from the full multivariable model.

We also performed a post hoc sensitivity analysis to determine if our conclusions were biased due to missing patient responses for the risk‐adjustment variables. Accordingly, we imputed the response level most negatively associated with most HCAHPS domains as previously reported and reran the multivariable models.[19] We did not find a meaningful change in our conclusions (see Supporting Figure 1 in the online version of this article).

RESULTS

The hospitals discharged 152,333 patients during the study period, 39,905 of whom (26.2 %) had a predicted 30‐day mortality risk greater or equal to 0.07 (Table 1). Of the 36,280 high‐risk patients discharged alive, 5901 (16.3%) died in the ensuing 30 days, and 7951 (22%) were readmitted.

The high‐risk subset was under‐represented in those who completed the HCAHPS survey with 7% (2513/36,280) completing surveys compared to 13% of low‐risk patients (14,996/111,600) (P < 0.0001). Moreover, compared to high‐risk patients who were alive at discharge but did not complete surveys, high‐risk survey respondents were less likely to die within 30 days (16/2513 = 0.64% vs 5885/33,767 = 17.4%, P < 0.0001), and less likely to be readmitted (424/2513 = 16.9% vs 7527/33,767 = 22.3%, P < 0.0001).

On average, high‐risk respondents (compared to low risk) were slightly less likely to be female (52.4% vs 57.9%), less educated (30.6% with some college vs 42.3%), less likely to have been on a surgical service (13.8% vs 42.5%), and less likely to report good or better health (49.0% vs 68.0%, all P < 0.0001). High‐risk respondents were also older (76.6 vs 63.1 years), stayed in the hospital longer (4.6 vs 3.5 days), and received care from more specialties (2.2 vs 1.7 specialties) (all P < 0.0001). High‐risk respondents experienced more 30‐day readmissions (16.9% vs 8.1%) and deaths within 30 days (0.6 % vs 0.1 %, all P < 0.0001) than their low‐risk counterparts.

High‐risk respondents were less likely to provide the most favorable response (unadjusted) for all HCAHPS domains compared to low‐risk respondents, although the difference was not significant for discharge information (Table 1, Figure 2A). The gradient between high‐risk and low‐risk patients was seen for all domains within each hospital except for pain management, hospital environment, and overall rating (Figure 3).

Figure 2

Figure 3

The multivariable regression models examined whether the mortality risk on admission simply represented older medical patients and/or those who considered themselves unhealthy (Figure 2B) (see Supporting Table 1 in the online version of this article). Accounting for hospital, age, gender, language, self‐reported health, educational level, service line, and days elapsed from discharge, respondents in the high‐mortality‐risk stratum were still less likely to report an always experience for doctor communication (OR: 0.85; 95% confidence interval [CI]: 0.77‐0.94) and responsiveness of hospital staff (OR: 0.77; 95% CI: 0.69‐0.85). Higher‐risk patients also tended to have less favorable experiences with nursing communication, although the CI crossed 1 (OR: 0.91; 95% CI: 0.82‐1.01). In contrast, higher‐risk patients were more likely to provide top box responses for having received discharge information (OR: 1.30; 95% CI: 1.14‐1.48). We did not find independent associations between mortality risk and the other domains when the patient risk‐adjustment factors were considered.[18, 19, 20, 21]

DISCUSSION

The high‐mortality‐risk stratum on admission contained a subset of patients who provided less favorable responses for almost all incentivized HCAHPS domains when other risk‐adjustment variables were not taken into consideration (Figure 2A). These univariate relationships weakened when we controlled for gender, the standard HCAHPS risk‐adjustment variables, and individual hospital influences (Figure 2B).[18, 19, 20, 21] After multivariable adjustment, survey respondents in the high‐risk category remained less likely to report their physicians always communicated well and to experience hospital staff responding quickly, but were more likely to report receiving discharge information. We did not find an independent association between the underlying mortality risk and the other incentivized HCAHPS domains after risk adjustment.

We are cautious with initial interpretations of our findings in light of the relatively small number of hospitals studied and the substantial survey response bias of healthier patients. Undoubtedly, the CMS exclusions of patients discharged to hospice or skilled nursing facilities provide a partial explanation for the selection bias, but the experience of those at high risk who did not complete surveys remains conjecture at this point.[14] Previous evidence suggests sicker patients and those with worse experiences are less likely to respond to the HCAHPS survey.[18, 22] On the other hand, it is possible that high‐risk nonrespondents who died could have received better communication and staff responsiveness.[23, 24] We were unable to find a previous, patient‐level study that explicitly tested the association between the admission mortality risk and the subsequent patient experience, yet our findings are consistent with a previous single‐site study of a surgical population showing lower overall ratings from patients with higher Injury Severity Scores.[25]

Our findings provide evidence of complex relationships among admission mortality risk, the 3 domains of the patient experience, and adverse outcomes, at least within the study hospitals (Figure 1). The developing field of palliative care has found very ill patients have special communication needs regarding goals of care, as well as physical symptoms, anxiety, and depression that might prompt more calls for help.[26] If these needs were more important for high‐risk compared to low‐risk patients, and were either not recognized or adequately addressed by the clinical teams at the study hospitals, then the high‐risk patients may have been less likely to perceive their physicians listened and explained things well, or that staff responded promptly to their requests for help.[27] On the other hand, the higher ratings for discharge information suggest the needs of the high‐risk patients were relatively easier to address by current practices at these hospitals. The lack of association between the mortality risk and the other HCAHPS domains may reflect the relatively stronger influence of age, gender, educational level, provider variability, and other unmeasured influences within the study sites, or that the level of patient need was similar among high‐risk and low‐risk patients within these domains.[27]

There are several possible confounders of our observed relationship between mortality risk and HCAHPS scores. The first category of confounders represents patient level variables that might impact the communication scores, some of which are part of the formula of our mortality prediction rule, for example, cognitive impairment and emergent admission.[18, 22, 27] The effect of the mortality risk could also be confounded by unmeasured patient‐level factors such as lower socioeconomic status.[28] A second category of confounders pertains to clinical outcomes and processes of care associated with serious illness irrespective of the risk of dying. More physicians involved in the care of the seriously ill (Table 1) may impact the communication scores, due to the larger opportunity for conflicting or confusing information presented to patients and their families.[29] The longer hospital stays, readmissions, and adverse events of the seriously ill may also underlie the apparent association between mortality risk and HCAHPS scores.[8, 9, 10]

Even if we do not understand precisely if and how the mortality risk might be associated with suboptimal physician communication and staff responsiveness, there may still be some value in considering how these possible relationships could be leveraged to improve patient care. We recall Berwick's insight that every system is perfectly designed to achieve the results it achieves.[7] We have previously argued for the use of mortality‐risk strata to initiate concurrent, multidisciplinary care processes to reduce adverse outcomes.[12, 13] Others have used risk‐based approaches for anticipating clinical deterioration of surgical patients, and determining the intensity of individualized case management services.[30, 31] In this framework, all patients receive a standard set of care processes, but higher‐risk patients receive additional efforts to promote better outcomes. An efficient extension of this approach is to assume patients at risk for adverse outcomes also have additional needs for communication, coordination of specialty care, and timely response to the call button. The admission mortality risk could be used as a determinant for the level of nurse staffing to reduce deaths plus shorten response time to the call button.[32, 33] Hospitalists and specialists could work together on a standard way to conference among themselves for high‐risk patients above that needed for less‐complex cases. Patients in the high‐risk strata could be screened early to see if they might benefit from the involvement of the palliative care team.[26]

Our study has limitations in addition to those already noted. First, our use of the top box as the formulation of the outcome of interest could be challenged. We chose this to be relevant to the Value‐Based Purchasing environment, but other formulations or use of other survey instruments may be needed to tease out the complex relationships we hypothesize. Next, we do not know the extent to which the patients and care processes reflected in our study represent other settings. The literature indicates some hospitals are more effective in providing care for certain subgroups of patients than for others, and that there is substantial regional variation in care intensity that is in turn associated with the patient experience.[29, 34] The mortality‐risk experience relationship for nonstudy hospitals could be weaker or stronger than what we found. Third, many hospitals may not have the capability to generate mortality scores on admission, although more hospitals may be able to do so in the future.[35] Explicit risk strata have the benefit of providing members of the multidisciplinary team with a quick preview of the clinical needs and prognoses of patients in much the way that the term baroque alerts the audience to the genre of music. Still, clinicians in any hospital could attempt to improve outcomes and experience through the use of informal risk assessment during interdisciplinary care rounds or by simply asking the team if they would be surprised if this patient died in the next year.[30, 36] Finally, we do not know if awareness of an experience risk will identify remediable practices that actually improve the experience. Clearly, future studies are needed to answer all of these concerns.

We have provided evidence that a group of patients who were at elevated risk for dying at the time of admission were more likely to have issues with physician communication and staff responsiveness than their lower‐risk counterparts. While we await future studies to confirm these findings, clinical teams can consider whether or not their patients' HCAHPS scores reflect how their system of care addresses the needs of these vulnerable people.