Wasteful care may account for between 21% and 34% of the United States’ $3.2 trillion in annual healthcare expenditures, making it a prime target for cost-saving initiatives.^1,2 Telemetry is a target for value improvement strategies because telemetry is overutilized, rarely leads to a change in management, and has associated guidelines on appropriate use.^3-10 Telemetry use has been a focus of the Joint Commission’s National Patient Safety Goals since 2014, and it is also a focus of the Society of Hospital Medicine’s Choosing Wisely^® campaign.^11-13

Previous initiatives have evaluated how changes to telemetry orders or education and feedback affect telemetry use. Few studies have compared a system-wide electronic health record (EHR) approach to a multifaceted intervention. In seeking to address this gap, we adapted published guidelines from the American Heart Association (AHA) and incorporated them into our EHR ordering process.³ Simultaneously, we implemented a multifaceted quality improvement initiative and compared this combined program’s effectiveness to that of the EHR approach alone.

Study Design, Setting, and Population

We performed a 2-group observational pre- to postintervention study at University of Utah Health. Hospital encounters of patients 18 years and older who had at least 1 inpatient acute care, nonintensive care unit (ICU) room charge and an admission date between January 1, 2014, and July 31, 2016, were included. Patient encounters with missing encounter-level covariates, such as case mix index (CMI) or attending provider identification, were excluded. The Institutional Review Board classified this project as quality improvement and did not require review and oversight.

Intervention

On July 6, 2015, our Epic (Epic Systems Corporation, Madison, WI) EHR telemetry order was modified to discourage unnecessary telemetry monitoring. The new order required providers ordering telemetry to choose a clinical indication and select a duration for monitoring, after which the order would expire and require physician renewal or discontinuation. These were the only changes that occurred for nonhospitalist providers. The nonhospitalist group included all admitting providers who were not hospitalists. This group included neurology (6.98%); cardiology (8.13%); other medical specialties such as pulmonology, hematology, and oncology (21.30%); cardiothoracic surgery (3.72%); orthopedic surgery (14.84%); general surgery (11.11%); neurosurgery (11.07%); and other surgical specialties, including urology, transplant, vascular surgery, and plastics (16.68%).

Between January 2015 and June 2015, we implemented a multicomponent program among our hospitalist service. The hospitalist service is composed of 4 teams with internal medicine residents and 2 teams with advanced practice providers, all staffed by academic hospitalists. Our program was composed of 5 elements, all of which were made before the hospital-wide changes to electronic telemetry orders and maintained throughout the study period, as follows: (1) a single provider education session reviewing available evidence (eg, AHA guidelines, Choosing Wisely^® campaign), (2) removal of the telemetry order from hospitalist admission order set on March 23, 2015, (3) inclusion of telemetry discussion in the hospitalist group’s daily “Rounding Checklist,”¹⁴ (4) monthly feedback provided as part of hospitalist group meetings, and (5) a financial incentive, awarded to the division (no individual provider payment) if performance targets were met. See supplementary Appendix (“Implementation Manual”) for further details.

Data Source

We obtained data on patient age, gender, Medicare Severity-Diagnosis Related Group, Charlson comorbidity index (CCI), CMI, admitting unit, attending physician, admission and discharge dates, length of stay (LOS), 30-day readmission, bed charge (telemetry or nontelemetry), ICU stay, and inpatient mortality from the enterprise data warehouse. Telemetry days were determined through room billing charges, which are assigned based on the presence or absence of an active telemetry order at midnight. Code events came from a log kept by the hospital telephone operator, who is responsible for sending out all calls to the code team. Code event data were available starting July 19, 2014.

Measures

Our primary outcome was the percentage of hospital days that had telemetry charges for individual patients. All billed telemetry days on acute care floors were included regardless of admission status (inpatient vs observation), service, indication, or ordering provider. Secondary outcomes were inpatient mortality, escalation of care, code event rates, and appropriate telemetry utilization rates. Escalation of care was defined as transfer to an ICU after initially being admitted to an acute care floor. The code event rate was defined as the ratio of the number of code team activations to the number of patient days. Appropriate telemetry utilization rates were determined via chart review, as detailed below.

In order to evaluate changes in appropriateness of telemetry monitoring, 4 of the authors who are internal medicine physicians (KE, CC, JC, DG) performed chart reviews of 25 randomly selected patients in each group (hospitalist and nonhospitalist) before and after the intervention who received at least 1 day of telemetry monitoring. Each reviewer was provided a key based on AHA guidelines for monitoring indications and associated maximum allowable durations.³ Chart reviews were performed to determine the indication (if any) for monitoring, as well as the number of days that were indicated. The number of indicated days was compared to the number of telemetry days the patient received to determine the overall proportion of days that were indicated (“Telemetry appropriateness per visit”). Three reviewers (KE, AR, CC) also evaluated 100 patients on the hospitalist service after the intervention who did not receive any telemetry monitoring to evaluate whether patients with indications for telemetry monitoring were not receiving it after the intervention. For patients who had a possible indication, the indication was classified as Class I (“Cardiac monitoring is indicated in most, if not all, patients in this group”) or Class II (“Cardiac monitoring may be of benefit in some patients but is not considered essential for all patients”).³

Adjustment Variables

To account for differences in patient characteristics between hospitalist and nonhospitalist groups, we included age, gender, CMI, and CCI in statistical models. CCI was calculated according to the algorithm specified by Quan et al.¹⁵ using all patient diagnoses from previous visits and the index visit identified from the facility billing system.

Statistical Analysis

We computed descriptive statistics for study outcomes and visit characteristics for hospitalist and nonhospitalist visits for pre- and postintervention periods. Descriptive statistics were expressed as n (%) for categorical patient characteristics and outcome variables. For continuous patient characteristics, we expressed the variability of individual observations as the mean ± the standard deviation. For continuous outcomes, we expressed the precision of the mean estimates using standard error. Telemetry utilization per visit was weighted by the number of total acute care days per visit. Telemetry appropriateness per visit was weighted by the number of telemetry days per visit. Patients who did not receive any telemetry monitoring were included in the analysis and noted to have 0 telemetry days. All patients had at least 1 acute care day. Categorical variables were compared using χ² tests, and continuous variables were compared using t tests. Code event rates were compared using the binomial probability mid-p exact test for person-time data.¹⁶

We fitted generalized linear regression models using generalized estimating equations to evaluate the relative change in outcomes of interest in the postintervention period compared with the preintervention period after adjusting for study covariates. The models included study group (hospitalist and nonhospitalist), time period (pre- and postintervention), an interaction term between study group and time period, and study covariates (age, gender, CMI, and CCI). The models were defined using a binomial distributional assumption and logit link function for mortality, escalation of care, and whether patients had at least 1 telemetry day. A gamma distributional assumption and log link function were used for LOS, telemetry acute care days per visit, and total acute care days per visit. A negative binomial distributional assumption and log link function were used for telemetry utilization and telemetry appropriateness. We used the log of the acute care days as an offset for telemetry utilization and the log of the telemetry days per visit as an offset for telemetry appropriateness. An exchangeable working correlation matrix was used to account for physician-level clustering for all outcomes. Intervention effects, representing the difference in odds for categorical variables and in amount for continuous variables, were calculated as exponentiation of the beta parameters for the covariate minus 1.

P values <.05 were considered significant. We used SAS version 9.4 statistical software (SAS Institute Inc., Cary, NC) for data analysis.

The percent of patients who had any telemetry charges decreased from 36.2% to 15.9% (P < .001) in the hospitalist group and from 31.8% to 28.0% in the nonhospitalist group (P < .001; Table 1). Rates of code events did not change over time (P = .9).

In this study, implementing a change in the EHR telemetry order produced reductions in telemetry days. However, when combined with a multicomponent program including education, audit and feedback, financial incentives, and changes to remove telemetry orders from admission orders sets, an even more marked improvement was seen. Neither intervention reduced LOS, increased code event rates, or increased rates of escalation of care.

Prior studies have evaluated interventions to reduce unnecessary telemetry monitoring with varying degrees of success. The most successful EHR intervention to date, from Dressler et al.,¹⁸ achieved a 70% reduction in overall telemetry use by integrating the AHA guidelines into their EHR and incorporating nursing discontinuation guidelines to ensure that telemetry discontinuation was both safe and timely. Other studies using stewardship approaches and standardized protocols have been less successful.^19,20 One study utilizing a multidisciplinary approach but not including an EHR component showed modest improvements in telemetry.²¹

Although we are unable to differentiate the exact effect of each component of the intervention, we did note an immediate decrease in telemetry orders after removing the telemetry order from our admission order set, a trend that was magnified after the addition of broader EHR changes (Figure 1). Important additional contributors to our success seem to have been the standardization of rounds to include daily discussion of telemetry and the provision of routine feedback. We cannot discern whether other components of our program (such as the financial incentives) contributed more or less to our program, though the sum of these interventions produced an overall program that required substantial buy in and sustained focus from the hospitalist group. The importance of the hospitalist program is highlighted by the relatively large differences in improvement compared with the nonhospitalist group.

Our study has several limitations. First, the study was conducted at a single center, which may limit its generalizability. Second, the intervention was multifaceted, diminishing our ability to discern which aspects beyond the system-wide change in the telemetry order were most responsible for the observed effect among hospitalists. Third, we are unable to fully account for baseline differences in telemetry utilization between hospitalist and nonhospitalist groups. It is likely that different services utilize telemetry monitoring in different ways, and the hospitalist group may have been more aware of the existing guidelines for monitoring prior to the intervention. Furthermore, we had a limited sample size for the chart audits, which reduced the available statistical power for determining changes in the appropriateness of telemetry utilization. Additionally, because internal medicine residents rotate through various services, it is possible that the education they received on their hospitalist rotation as part of our intervention had a spillover effect in the nonhospitalist group. However, any effect should have decreased the difference between the groups. Lastly, although our postintervention time period was 1 year, we do not have data beyond that to monitor for sustainability of the results.

In this single-site study, combining EHR orders prompting physicians to choose a clinical indication and duration for monitoring with a broader program—including upstream changes in ordering as well as education, audit, and feedback—produced reductions in telemetry usage. Whether this reduction improves the appropriateness of telemetry utilization or reduces other effects of telemetry (eg, alert fatigue, calls for benign arrhythmias) cannot be discerned from our study. However, our results support the idea that multipronged approaches to telemetry use are most likely to produce improvements.

Acknowledgments

The authors thank Dr. Frank Thomas for his assistance with process engineering and Mr. Andrew Wood for his routine provision of data. The statistical analysis was supported by the University of Utah Study Design and Biostatistics Center, with funding in part from the National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health, through Grant 5UL1TR001067-05 (formerly 8UL1TR000105 and UL1RR025764).

Disclosure

The authors have no conflicts of interest to report.

Wasteful care may account for between 21% and 34% of the United States’ $3.2 trillion in annual healthcare expenditures, making it a prime target for cost-saving initiatives.^1,2 Telemetry is a target for value improvement strategies because telemetry is overutilized, rarely leads to a change in management, and has associated guidelines on appropriate use.^3-10 Telemetry use has been a focus of the Joint Commission’s National Patient Safety Goals since 2014, and it is also a focus of the Society of Hospital Medicine’s Choosing Wisely^® campaign.^11-13

Previous initiatives have evaluated how changes to telemetry orders or education and feedback affect telemetry use. Few studies have compared a system-wide electronic health record (EHR) approach to a multifaceted intervention. In seeking to address this gap, we adapted published guidelines from the American Heart Association (AHA) and incorporated them into our EHR ordering process.³ Simultaneously, we implemented a multifaceted quality improvement initiative and compared this combined program’s effectiveness to that of the EHR approach alone.

Study Design, Setting, and Population

We performed a 2-group observational pre- to postintervention study at University of Utah Health. Hospital encounters of patients 18 years and older who had at least 1 inpatient acute care, nonintensive care unit (ICU) room charge and an admission date between January 1, 2014, and July 31, 2016, were included. Patient encounters with missing encounter-level covariates, such as case mix index (CMI) or attending provider identification, were excluded. The Institutional Review Board classified this project as quality improvement and did not require review and oversight.

Intervention

On July 6, 2015, our Epic (Epic Systems Corporation, Madison, WI) EHR telemetry order was modified to discourage unnecessary telemetry monitoring. The new order required providers ordering telemetry to choose a clinical indication and select a duration for monitoring, after which the order would expire and require physician renewal or discontinuation. These were the only changes that occurred for nonhospitalist providers. The nonhospitalist group included all admitting providers who were not hospitalists. This group included neurology (6.98%); cardiology (8.13%); other medical specialties such as pulmonology, hematology, and oncology (21.30%); cardiothoracic surgery (3.72%); orthopedic surgery (14.84%); general surgery (11.11%); neurosurgery (11.07%); and other surgical specialties, including urology, transplant, vascular surgery, and plastics (16.68%).

Between January 2015 and June 2015, we implemented a multicomponent program among our hospitalist service. The hospitalist service is composed of 4 teams with internal medicine residents and 2 teams with advanced practice providers, all staffed by academic hospitalists. Our program was composed of 5 elements, all of which were made before the hospital-wide changes to electronic telemetry orders and maintained throughout the study period, as follows: (1) a single provider education session reviewing available evidence (eg, AHA guidelines, Choosing Wisely^® campaign), (2) removal of the telemetry order from hospitalist admission order set on March 23, 2015, (3) inclusion of telemetry discussion in the hospitalist group’s daily “Rounding Checklist,”¹⁴ (4) monthly feedback provided as part of hospitalist group meetings, and (5) a financial incentive, awarded to the division (no individual provider payment) if performance targets were met. See supplementary Appendix (“Implementation Manual”) for further details.

Data Source

We obtained data on patient age, gender, Medicare Severity-Diagnosis Related Group, Charlson comorbidity index (CCI), CMI, admitting unit, attending physician, admission and discharge dates, length of stay (LOS), 30-day readmission, bed charge (telemetry or nontelemetry), ICU stay, and inpatient mortality from the enterprise data warehouse. Telemetry days were determined through room billing charges, which are assigned based on the presence or absence of an active telemetry order at midnight. Code events came from a log kept by the hospital telephone operator, who is responsible for sending out all calls to the code team. Code event data were available starting July 19, 2014.

Measures

Our primary outcome was the percentage of hospital days that had telemetry charges for individual patients. All billed telemetry days on acute care floors were included regardless of admission status (inpatient vs observation), service, indication, or ordering provider. Secondary outcomes were inpatient mortality, escalation of care, code event rates, and appropriate telemetry utilization rates. Escalation of care was defined as transfer to an ICU after initially being admitted to an acute care floor. The code event rate was defined as the ratio of the number of code team activations to the number of patient days. Appropriate telemetry utilization rates were determined via chart review, as detailed below.

In order to evaluate changes in appropriateness of telemetry monitoring, 4 of the authors who are internal medicine physicians (KE, CC, JC, DG) performed chart reviews of 25 randomly selected patients in each group (hospitalist and nonhospitalist) before and after the intervention who received at least 1 day of telemetry monitoring. Each reviewer was provided a key based on AHA guidelines for monitoring indications and associated maximum allowable durations.³ Chart reviews were performed to determine the indication (if any) for monitoring, as well as the number of days that were indicated. The number of indicated days was compared to the number of telemetry days the patient received to determine the overall proportion of days that were indicated (“Telemetry appropriateness per visit”). Three reviewers (KE, AR, CC) also evaluated 100 patients on the hospitalist service after the intervention who did not receive any telemetry monitoring to evaluate whether patients with indications for telemetry monitoring were not receiving it after the intervention. For patients who had a possible indication, the indication was classified as Class I (“Cardiac monitoring is indicated in most, if not all, patients in this group”) or Class II (“Cardiac monitoring may be of benefit in some patients but is not considered essential for all patients”).³

Adjustment Variables

To account for differences in patient characteristics between hospitalist and nonhospitalist groups, we included age, gender, CMI, and CCI in statistical models. CCI was calculated according to the algorithm specified by Quan et al.¹⁵ using all patient diagnoses from previous visits and the index visit identified from the facility billing system.

Statistical Analysis

We computed descriptive statistics for study outcomes and visit characteristics for hospitalist and nonhospitalist visits for pre- and postintervention periods. Descriptive statistics were expressed as n (%) for categorical patient characteristics and outcome variables. For continuous patient characteristics, we expressed the variability of individual observations as the mean ± the standard deviation. For continuous outcomes, we expressed the precision of the mean estimates using standard error. Telemetry utilization per visit was weighted by the number of total acute care days per visit. Telemetry appropriateness per visit was weighted by the number of telemetry days per visit. Patients who did not receive any telemetry monitoring were included in the analysis and noted to have 0 telemetry days. All patients had at least 1 acute care day. Categorical variables were compared using χ² tests, and continuous variables were compared using t tests. Code event rates were compared using the binomial probability mid-p exact test for person-time data.¹⁶

We fitted generalized linear regression models using generalized estimating equations to evaluate the relative change in outcomes of interest in the postintervention period compared with the preintervention period after adjusting for study covariates. The models included study group (hospitalist and nonhospitalist), time period (pre- and postintervention), an interaction term between study group and time period, and study covariates (age, gender, CMI, and CCI). The models were defined using a binomial distributional assumption and logit link function for mortality, escalation of care, and whether patients had at least 1 telemetry day. A gamma distributional assumption and log link function were used for LOS, telemetry acute care days per visit, and total acute care days per visit. A negative binomial distributional assumption and log link function were used for telemetry utilization and telemetry appropriateness. We used the log of the acute care days as an offset for telemetry utilization and the log of the telemetry days per visit as an offset for telemetry appropriateness. An exchangeable working correlation matrix was used to account for physician-level clustering for all outcomes. Intervention effects, representing the difference in odds for categorical variables and in amount for continuous variables, were calculated as exponentiation of the beta parameters for the covariate minus 1.

P values <.05 were considered significant. We used SAS version 9.4 statistical software (SAS Institute Inc., Cary, NC) for data analysis.

The percent of patients who had any telemetry charges decreased from 36.2% to 15.9% (P < .001) in the hospitalist group and from 31.8% to 28.0% in the nonhospitalist group (P < .001; Table 1). Rates of code events did not change over time (P = .9).

In this study, implementing a change in the EHR telemetry order produced reductions in telemetry days. However, when combined with a multicomponent program including education, audit and feedback, financial incentives, and changes to remove telemetry orders from admission orders sets, an even more marked improvement was seen. Neither intervention reduced LOS, increased code event rates, or increased rates of escalation of care.

Prior studies have evaluated interventions to reduce unnecessary telemetry monitoring with varying degrees of success. The most successful EHR intervention to date, from Dressler et al.,¹⁸ achieved a 70% reduction in overall telemetry use by integrating the AHA guidelines into their EHR and incorporating nursing discontinuation guidelines to ensure that telemetry discontinuation was both safe and timely. Other studies using stewardship approaches and standardized protocols have been less successful.^19,20 One study utilizing a multidisciplinary approach but not including an EHR component showed modest improvements in telemetry.²¹

Although we are unable to differentiate the exact effect of each component of the intervention, we did note an immediate decrease in telemetry orders after removing the telemetry order from our admission order set, a trend that was magnified after the addition of broader EHR changes (Figure 1). Important additional contributors to our success seem to have been the standardization of rounds to include daily discussion of telemetry and the provision of routine feedback. We cannot discern whether other components of our program (such as the financial incentives) contributed more or less to our program, though the sum of these interventions produced an overall program that required substantial buy in and sustained focus from the hospitalist group. The importance of the hospitalist program is highlighted by the relatively large differences in improvement compared with the nonhospitalist group.

Our study has several limitations. First, the study was conducted at a single center, which may limit its generalizability. Second, the intervention was multifaceted, diminishing our ability to discern which aspects beyond the system-wide change in the telemetry order were most responsible for the observed effect among hospitalists. Third, we are unable to fully account for baseline differences in telemetry utilization between hospitalist and nonhospitalist groups. It is likely that different services utilize telemetry monitoring in different ways, and the hospitalist group may have been more aware of the existing guidelines for monitoring prior to the intervention. Furthermore, we had a limited sample size for the chart audits, which reduced the available statistical power for determining changes in the appropriateness of telemetry utilization. Additionally, because internal medicine residents rotate through various services, it is possible that the education they received on their hospitalist rotation as part of our intervention had a spillover effect in the nonhospitalist group. However, any effect should have decreased the difference between the groups. Lastly, although our postintervention time period was 1 year, we do not have data beyond that to monitor for sustainability of the results.

In this single-site study, combining EHR orders prompting physicians to choose a clinical indication and duration for monitoring with a broader program—including upstream changes in ordering as well as education, audit, and feedback—produced reductions in telemetry usage. Whether this reduction improves the appropriateness of telemetry utilization or reduces other effects of telemetry (eg, alert fatigue, calls for benign arrhythmias) cannot be discerned from our study. However, our results support the idea that multipronged approaches to telemetry use are most likely to produce improvements.

Acknowledgments

The authors thank Dr. Frank Thomas for his assistance with process engineering and Mr. Andrew Wood for his routine provision of data. The statistical analysis was supported by the University of Utah Study Design and Biostatistics Center, with funding in part from the National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health, through Grant 5UL1TR001067-05 (formerly 8UL1TR000105 and UL1RR025764).

Disclosure

The authors have no conflicts of interest to report.

Wasteful care may account for between 21% and 34% of the United States’ $3.2 trillion in annual healthcare expenditures, making it a prime target for cost-saving initiatives.^1,2 Telemetry is a target for value improvement strategies because telemetry is overutilized, rarely leads to a change in management, and has associated guidelines on appropriate use.^3-10 Telemetry use has been a focus of the Joint Commission’s National Patient Safety Goals since 2014, and it is also a focus of the Society of Hospital Medicine’s Choosing Wisely^® campaign.^11-13

Previous initiatives have evaluated how changes to telemetry orders or education and feedback affect telemetry use. Few studies have compared a system-wide electronic health record (EHR) approach to a multifaceted intervention. In seeking to address this gap, we adapted published guidelines from the American Heart Association (AHA) and incorporated them into our EHR ordering process.³ Simultaneously, we implemented a multifaceted quality improvement initiative and compared this combined program’s effectiveness to that of the EHR approach alone.

Study Design, Setting, and Population

We performed a 2-group observational pre- to postintervention study at University of Utah Health. Hospital encounters of patients 18 years and older who had at least 1 inpatient acute care, nonintensive care unit (ICU) room charge and an admission date between January 1, 2014, and July 31, 2016, were included. Patient encounters with missing encounter-level covariates, such as case mix index (CMI) or attending provider identification, were excluded. The Institutional Review Board classified this project as quality improvement and did not require review and oversight.

Intervention

On July 6, 2015, our Epic (Epic Systems Corporation, Madison, WI) EHR telemetry order was modified to discourage unnecessary telemetry monitoring. The new order required providers ordering telemetry to choose a clinical indication and select a duration for monitoring, after which the order would expire and require physician renewal or discontinuation. These were the only changes that occurred for nonhospitalist providers. The nonhospitalist group included all admitting providers who were not hospitalists. This group included neurology (6.98%); cardiology (8.13%); other medical specialties such as pulmonology, hematology, and oncology (21.30%); cardiothoracic surgery (3.72%); orthopedic surgery (14.84%); general surgery (11.11%); neurosurgery (11.07%); and other surgical specialties, including urology, transplant, vascular surgery, and plastics (16.68%).

Between January 2015 and June 2015, we implemented a multicomponent program among our hospitalist service. The hospitalist service is composed of 4 teams with internal medicine residents and 2 teams with advanced practice providers, all staffed by academic hospitalists. Our program was composed of 5 elements, all of which were made before the hospital-wide changes to electronic telemetry orders and maintained throughout the study period, as follows: (1) a single provider education session reviewing available evidence (eg, AHA guidelines, Choosing Wisely^® campaign), (2) removal of the telemetry order from hospitalist admission order set on March 23, 2015, (3) inclusion of telemetry discussion in the hospitalist group’s daily “Rounding Checklist,”¹⁴ (4) monthly feedback provided as part of hospitalist group meetings, and (5) a financial incentive, awarded to the division (no individual provider payment) if performance targets were met. See supplementary Appendix (“Implementation Manual”) for further details.

Data Source

We obtained data on patient age, gender, Medicare Severity-Diagnosis Related Group, Charlson comorbidity index (CCI), CMI, admitting unit, attending physician, admission and discharge dates, length of stay (LOS), 30-day readmission, bed charge (telemetry or nontelemetry), ICU stay, and inpatient mortality from the enterprise data warehouse. Telemetry days were determined through room billing charges, which are assigned based on the presence or absence of an active telemetry order at midnight. Code events came from a log kept by the hospital telephone operator, who is responsible for sending out all calls to the code team. Code event data were available starting July 19, 2014.

Measures

Our primary outcome was the percentage of hospital days that had telemetry charges for individual patients. All billed telemetry days on acute care floors were included regardless of admission status (inpatient vs observation), service, indication, or ordering provider. Secondary outcomes were inpatient mortality, escalation of care, code event rates, and appropriate telemetry utilization rates. Escalation of care was defined as transfer to an ICU after initially being admitted to an acute care floor. The code event rate was defined as the ratio of the number of code team activations to the number of patient days. Appropriate telemetry utilization rates were determined via chart review, as detailed below.

In order to evaluate changes in appropriateness of telemetry monitoring, 4 of the authors who are internal medicine physicians (KE, CC, JC, DG) performed chart reviews of 25 randomly selected patients in each group (hospitalist and nonhospitalist) before and after the intervention who received at least 1 day of telemetry monitoring. Each reviewer was provided a key based on AHA guidelines for monitoring indications and associated maximum allowable durations.³ Chart reviews were performed to determine the indication (if any) for monitoring, as well as the number of days that were indicated. The number of indicated days was compared to the number of telemetry days the patient received to determine the overall proportion of days that were indicated (“Telemetry appropriateness per visit”). Three reviewers (KE, AR, CC) also evaluated 100 patients on the hospitalist service after the intervention who did not receive any telemetry monitoring to evaluate whether patients with indications for telemetry monitoring were not receiving it after the intervention. For patients who had a possible indication, the indication was classified as Class I (“Cardiac monitoring is indicated in most, if not all, patients in this group”) or Class II (“Cardiac monitoring may be of benefit in some patients but is not considered essential for all patients”).³

Adjustment Variables

To account for differences in patient characteristics between hospitalist and nonhospitalist groups, we included age, gender, CMI, and CCI in statistical models. CCI was calculated according to the algorithm specified by Quan et al.¹⁵ using all patient diagnoses from previous visits and the index visit identified from the facility billing system.

Statistical Analysis

We computed descriptive statistics for study outcomes and visit characteristics for hospitalist and nonhospitalist visits for pre- and postintervention periods. Descriptive statistics were expressed as n (%) for categorical patient characteristics and outcome variables. For continuous patient characteristics, we expressed the variability of individual observations as the mean ± the standard deviation. For continuous outcomes, we expressed the precision of the mean estimates using standard error. Telemetry utilization per visit was weighted by the number of total acute care days per visit. Telemetry appropriateness per visit was weighted by the number of telemetry days per visit. Patients who did not receive any telemetry monitoring were included in the analysis and noted to have 0 telemetry days. All patients had at least 1 acute care day. Categorical variables were compared using χ² tests, and continuous variables were compared using t tests. Code event rates were compared using the binomial probability mid-p exact test for person-time data.¹⁶

We fitted generalized linear regression models using generalized estimating equations to evaluate the relative change in outcomes of interest in the postintervention period compared with the preintervention period after adjusting for study covariates. The models included study group (hospitalist and nonhospitalist), time period (pre- and postintervention), an interaction term between study group and time period, and study covariates (age, gender, CMI, and CCI). The models were defined using a binomial distributional assumption and logit link function for mortality, escalation of care, and whether patients had at least 1 telemetry day. A gamma distributional assumption and log link function were used for LOS, telemetry acute care days per visit, and total acute care days per visit. A negative binomial distributional assumption and log link function were used for telemetry utilization and telemetry appropriateness. We used the log of the acute care days as an offset for telemetry utilization and the log of the telemetry days per visit as an offset for telemetry appropriateness. An exchangeable working correlation matrix was used to account for physician-level clustering for all outcomes. Intervention effects, representing the difference in odds for categorical variables and in amount for continuous variables, were calculated as exponentiation of the beta parameters for the covariate minus 1.

P values <.05 were considered significant. We used SAS version 9.4 statistical software (SAS Institute Inc., Cary, NC) for data analysis.

The percent of patients who had any telemetry charges decreased from 36.2% to 15.9% (P < .001) in the hospitalist group and from 31.8% to 28.0% in the nonhospitalist group (P < .001; Table 1). Rates of code events did not change over time (P = .9).

In this study, implementing a change in the EHR telemetry order produced reductions in telemetry days. However, when combined with a multicomponent program including education, audit and feedback, financial incentives, and changes to remove telemetry orders from admission orders sets, an even more marked improvement was seen. Neither intervention reduced LOS, increased code event rates, or increased rates of escalation of care.

Prior studies have evaluated interventions to reduce unnecessary telemetry monitoring with varying degrees of success. The most successful EHR intervention to date, from Dressler et al.,¹⁸ achieved a 70% reduction in overall telemetry use by integrating the AHA guidelines into their EHR and incorporating nursing discontinuation guidelines to ensure that telemetry discontinuation was both safe and timely. Other studies using stewardship approaches and standardized protocols have been less successful.^19,20 One study utilizing a multidisciplinary approach but not including an EHR component showed modest improvements in telemetry.²¹

Although we are unable to differentiate the exact effect of each component of the intervention, we did note an immediate decrease in telemetry orders after removing the telemetry order from our admission order set, a trend that was magnified after the addition of broader EHR changes (Figure 1). Important additional contributors to our success seem to have been the standardization of rounds to include daily discussion of telemetry and the provision of routine feedback. We cannot discern whether other components of our program (such as the financial incentives) contributed more or less to our program, though the sum of these interventions produced an overall program that required substantial buy in and sustained focus from the hospitalist group. The importance of the hospitalist program is highlighted by the relatively large differences in improvement compared with the nonhospitalist group.

Our study has several limitations. First, the study was conducted at a single center, which may limit its generalizability. Second, the intervention was multifaceted, diminishing our ability to discern which aspects beyond the system-wide change in the telemetry order were most responsible for the observed effect among hospitalists. Third, we are unable to fully account for baseline differences in telemetry utilization between hospitalist and nonhospitalist groups. It is likely that different services utilize telemetry monitoring in different ways, and the hospitalist group may have been more aware of the existing guidelines for monitoring prior to the intervention. Furthermore, we had a limited sample size for the chart audits, which reduced the available statistical power for determining changes in the appropriateness of telemetry utilization. Additionally, because internal medicine residents rotate through various services, it is possible that the education they received on their hospitalist rotation as part of our intervention had a spillover effect in the nonhospitalist group. However, any effect should have decreased the difference between the groups. Lastly, although our postintervention time period was 1 year, we do not have data beyond that to monitor for sustainability of the results.

In this single-site study, combining EHR orders prompting physicians to choose a clinical indication and duration for monitoring with a broader program—including upstream changes in ordering as well as education, audit, and feedback—produced reductions in telemetry usage. Whether this reduction improves the appropriateness of telemetry utilization or reduces other effects of telemetry (eg, alert fatigue, calls for benign arrhythmias) cannot be discerned from our study. However, our results support the idea that multipronged approaches to telemetry use are most likely to produce improvements.

Acknowledgments

The authors thank Dr. Frank Thomas for his assistance with process engineering and Mr. Andrew Wood for his routine provision of data. The statistical analysis was supported by the University of Utah Study Design and Biostatistics Center, with funding in part from the National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health, through Grant 5UL1TR001067-05 (formerly 8UL1TR000105 and UL1RR025764).

Disclosure

The authors have no conflicts of interest to report.

Given the high cost of readmissions to the healthcare system, there has been a substantial push to reduce readmissions by policymakers.¹ Among these is the Hospital Readmissions Reduction Program (HRRP), in which hospitals with higher than expected readmission rates receive reduced payments from Medicare.² Recent evidence has suggested the success of such policy changes, with multiple reports demonstrating a decrease in 30-day readmission rates in the Medicare population starting in 2010.^3-8

Initiatives to reduce readmissions can also have an effect on total number of admissions.^9,10 Indeed, along with the recent reduction in readmission, there has been a reduction in all admissions among Medicare beneficiaries.^11,12 Some studies have found that as admissions have decreased, the burden of comorbidity has increased among hospitalized patients,^3,11 suggesting that hospitals may be increasingly filled with patients at high risk of readmission. However, whether readmission risk among hospitalized patients has changed remains unknown, and understanding changes in risk profile could help inform which patients to target with future interventions to reduce readmissions.

Hospital efforts to reduce readmissions may have differential effects on types of patients by risk. For instance, low-intensity, system-wide interventions such as standardized discharge instructions or medicine reconciliation may have a stronger effect on patients at relatively low risk of readmission who may have a few important drivers of readmission that are easily overcome. Alternatively, the impact of intensive care transitions management might be greatest for high-risk patients, who have the most need for postdischarge medications, follow-up, and self-care.

The purpose of this study was therefore twofold: (1) to observe changes in average monthly risk of readmission among hospitalized Medicare patients and (2) to examine changes in readmission rates for Medicare patients at various risk of readmission. We hypothesized that readmission risk in the Medicare population would increase in recent years, as overall number of admissions and readmissions have fallen.^7,11 Additionally, we hypothesized that standardized readmission rates would decline less in highest risk patients as compared with the lowest risk patients because transitional care interventions may not be able to mitigate the large burden of comorbidity and social issues present in many high-risk patients.^13,14

We performed a retrospective cohort study of hospitalizations to US nonfederal short-term acute care facilities by Medicare beneficiaries between January 2009 and June 2015. The design involved 4 steps. First, we estimated a predictive model for unplanned readmissions within 30 days of discharge. Second, we assigned each hospitalization a predicted risk of readmission based on the model. Third, we studied trends in mean predicted risk of readmission during the study period. Fourth, we examined trends in observed to expected (O/E) readmission for hospitalizations in the lowest, middle, and highest categories of predicted risk of readmission to determine whether reductions in readmissions were more substantial in certain risk groups than in others.

Data were obtained from the Centers for Medicare and Medicaid Services (CMS) Inpatient Standard Analytic File and the Medicare Enrollment Data Base. We included hospitalizations of fee-for-service Medicare beneficiaries age ≥65 with continuous enrollment in Part A Medicare fee-for-service for at least 1 year prior and 30 days after the hospitalization.¹⁵ Hospitalizations with a discharge disposition of death, transfer to another acute hospital, and left against medical advice (AMA) were excluded. We also excluded patients with enrollment in hospice care prior to hospitalization. We excluded hospitalizations in June 2012 because of an irregularity in data availability for that month.

Hospitalizations were categorized into 5 specialty cohorts according to service line. The 5 cohorts were those used for the CMS hospital-wide readmission measure and included surgery/gynecology, medicine, cardiovascular, cardiorespiratory, and neurology.¹⁵ Among the 3 clinical conditions tracked as part of HRRP, heart failure and pneumonia were a subset of the cardiorespiratory cohort, while acute myocardial infarction was a subset of the cardiovascular cohort. Our use of cohorts was threefold: first, the average risk of readmission differs substantially across these cohorts, so pooling them produces heterogeneous risk strata; second, risk variables perform differently in different cohorts, so one single model may not be as accurate for calculating risk; and, third, the use of disease cohorts makes our results comparable to the CMS model and similar to other readmission studies in Medicare.^7,8,15

For development of the risk model, the outcome was 30-day unplanned hospital readmission. Planned readmissions were excluded; these were defined by the CMS algorithm as readmissions in which a typically planned procedure occurred in a hospitalization with a nonacute principal diagnosis.¹⁶ Independent variables included age and comorbidities in the final hospital-wide readmission models for each of the 5 specialty cohorts.¹⁵ In order to produce the best possible individual risk prediction for each patient, we added additional independent variables that CMS avoids for hospital quality measurement purposes but that contribute to risk of readmission: sex, race, dual eligibility status, number of prior AMA discharges, intensive care unit stay during current hospitalization, coronary care unit stay during current hospitalization, and hospitalization in the prior 30, 90, and 180 days. We also included an indicator variable for hospitalizations with more than 9 discharge diagnosis codes on or after January 2011, the time at which Medicare allowed an increase of the number of International Classification of Diseases, 9th Revision-Clinical Modification diagnosis billing codes from 9 to 25.¹⁷ This indicator adjusts for the increased availability of comorbidity codes, which might otherwise inflate the predicted risk relative to hospitalizations prior to that date.

Based on the risk models, each hospitalization was assigned a predicted risk of readmission. For each specialty cohort, we pooled all hospitalizations across all study years and divided them into risk quintiles. We categorized hospitalizations as high risk if in the highest quintile, medium risk if in the middle 3 quintiles, and low risk if in the lowest quintile of predicted risk for all study hospitalizations in a given specialty cohort.

For our time trend analyses, we studied 2 outcomes: monthly mean predicted risk and monthly ratio of observed readmissions to expected readmissions for patients in the lowest, middle, and highest categories of predicted risk of readmission. We studied monthly predicted risk to determine whether the average readmission risk of patients was changing over time as admission and readmission rates were declining. We studied the ratio of O/E readmissions to determine whether the decline in overall readmissions was more substantial in particular risk strata; we used the ratio of O/E readmissions, which measures number of readmissions divided by number of readmissions predicted by the model, rather than crude observed readmissions, as O/E readmissions account for any changes in risk profiles over time within each risk stratum. Independent variables in our trend analyses were year—entered as a continuous variable—and indicators for postintroduction of the Affordable Care Act (ACA, March 2010) and for postintroduction of HRRP (October 2012); these time indicators were included because of prior studies demonstrating that the introduction of ACA was associated with a decrease from baseline in readmission rates, which leveled off after introduction of HRRP.⁷ We also included an indicator for calendar quarter to account for seasonal effects.

Statistical Analysis

We developed generalized estimating equation models to predict 30-day unplanned readmission for each of the 5 specialty cohorts. The 5 models were fit using all patients in each cohort for the included time period and were adjusted for clustering by hospital. We assessed discrimination by calculating area under the receiver operating characteristic curve (AUC) for the 5 models; the AUCs measured the models’ ability to distinguish patients who were readmitted versus those who were not.¹⁸ We also calculated AUCs for each year to examine model performance over time.

Using these models, we calculated predicted risk for each hospitalization and averaged these to obtain mean predicted risk for each specialty cohort for each month. To test for trends in mean risk, we estimated 5 time series models, one for each cohort, with the dependent variable of monthly mean predicted risk. For each cohort, we first estimated a series of 12 empty autoregressive models, each with a different autoregressive term (1, 2...12). For each model, we calculated χ² for the test that the autocorrelation was 0; based on a comparison of chi-squared values, we specified an autocorrelation of 1 month for all models. Accordingly, a 1-month lag was used to estimate one final model for each cohort. Independent variables included year and indicators for post-ACA and post-HRRP; these variables captured the effect of trends over time and the introduction of these policy changes, respectively.¹⁹

To determine whether changes in risk over time were a result of changes in particular risk groups, we categorized hospitalizations into risk strata based on quintiles of predicted risk for each specialty cohort for the entire study period. For each individual year, we calculated the proportion of hospitalizations in the highest, middle, and lowest readmission risk strata for each cohort.

We calculated the monthly ratio of O/E readmission for hospitalizations in the lowest 20%, middle 60%, and highest 20% of readmission risk by month; O/E reflects the excess or deficit observed events relative to the number predicted by the model. Using this monthly O/E as the dependent variable, we developed autoregressive time series models as above, again with a 1-month lag, for each of these 3 risk strata in each cohort. As before, independent variables were year as a continuous variable, indicator variables for post-ACA and post-HRRP, and a categorical variable for calendar quarter.

All analyses were done in SAS version 9.3 (SAS Institute Inc., Cary, NC) and Stata version 14.2 (StataCorp LLC, College Station, TX).

We included 47,288,961 hospitalizations in the study, of which 11,231,242 (23.8%) were in the surgery/gynecology cohort, 19,548,711 (41.3%) were in the medicine cohort, 5,433,125 (11.5%) were in the cardiovascular cohort, 8,179,691 (17.3%) were in the cardiorespiratory cohort, and 2,896,192 (6.1%) were in the neurology cohort. The readmission rate was 16.2% (n = 7,642,161) overall, with the highest rates observed in the cardiorespiratory (20.5%) and medicine (17.6%) cohorts and the lowest rates observed in the surgery/gynecology (11.8%) and neurology (13.8%) cohorts.

The final predictive models for each cohort ranged in number of parameters from 56 for the cardiorespiratory cohort to 264 for the surgery/gynecology cohort. The models had AUCs of 0.70, 0.65, 0.67, 0.65, and 0.63 for the surgery/gynecology, medicine, cardiovascular, cardiorespiratory, and neurology cohorts, respectively; AUCs remained fairly stable over time for all disease cohorts (Appendix Table 1).

A number of mechanisms may account for the across-the-board improvements in readmission reduction. Many hospitals have instituted system-wide interventions, including patient education, medicine reconciliation, and early postdischarge follow-up,²⁰ which may have reduced readmissions across all patient risk strata. Alternatively, hospitals may have implemented interventions that disproportionally benefited low-risk patients while simultaneously utilizing interventions that only benefited high-risk patients. For instance, increasing threshold for admission⁷ may have the greatest effect on low-risk patients who could be most easily managed at home, while many intensive transitional care interventions have been developed to target only high-risk patients.^21,22

With the introduction of HRRP, there have been a number of concerns about the readmission measure used to penalize hospitals for high readmission rates. One major concern has been that the readmission metric may be flawed in its ability to capture continued improvement related to readmission.²³ Some have suggested that with better population health management, admissions will decrease, patient risk of the remaining patients will increase, and hospitals will be increasingly filled with patients who have high likelihood of readmission. This potential for increased risk with HRRP was suggested by a recent study that found that comorbidities increased in hospitalized Medicare beneficiaries between 2010 and 2013.¹¹ Our results were mixed in supporting this potential phenomenon because we examined global risk of readmission and found that some of the cohorts had increased risk over time while others did not. Others have expressed concern that readmission measure does not account for socioeconomic status, which has been associated with readmission rates.^24-27 Although we did not directly examine socioeconomic status in our study, we found that hospitals have been able to reduce readmission across all levels of risk, which includes markers of socioeconomic status, including race and Medicaid eligibility status.

Although we hypothesized that readmission risk would increase as number of hospitalizations decreased over time, we found no increase in readmission risk among the cohorts with HRRP diagnoses that had the largest decrease in readmission rates.^7,8 Conversely, readmission risk did increase—with a concurrent increase in the proportion of high-risk hospitalizations—in the surgery/gynecology and neurology cohorts that were not subject to HRRP penalties. Nonetheless, rehospitalizations were reduced for all risk categories in these 2 cohorts. Notably, surgery/gynecology and neurology had the lowest readmission rates overall. These findings suggest that initiatives to prevent initial hospitalizations, such as increasing the threshold for postoperative admission, may have had a greater effect on low- versus high-risk patients in low-risk hospitalizations. However, once a patient is hospitalized, multidisciplinary strategies appear to be effective at reducing readmissions for all risk classes in these cohorts.

For the 3 cohorts in which we observed an increase in readmission risk among hospitalized patients, the risk appeared to increase in early 2011. This time was about 10 months after passage of ACA, the timing of which was previously associated with a drop in readmission rates,^7,8 but well before HRRP went into effect in October 2012. The increase in readmission risk coincided with an increase in the number of diagnostic codes that could be included on a hospital claim to Medicare.¹⁷ This increase in allowable codes allowed us to capture more diagnoses for some patients, potentially resulting in an increase in apparent predicted risk of readmissions. While we adjusted for this in our predictive models, we may not have fully accounted for differences in risk related to coding change. As a result, some of the observed differences in risk in our study may be attributable to coding differences. More broadly, studies demonstrating the success of HRRP have typically examined risk-adjusted rates of readmission.^3,7 It is possible that a small portion of the observed reduction in risk-adjusted readmission rates may be related to the increase in predicted risk of readmission observed in our study. Future assessment of trends in readmission during this period should consider accounting for change in the number of allowed billing codes.

Other limitations should be considered in the interpretation of this study. First, like many predictive models for readmission,¹⁴ ours had imperfect discrimination, which could affect our results. Second, our study was based on older Medicare patients, so findings may not be applicable to younger patients. Third, while we accounted for surrogates for socioeconomic status, including dual eligibility and race, our models lacked other socioeconomic and community factors that can influence readmission.^24-26 Nonetheless, 1 study suggested that easily measured socioeconomic factors may not have a strong influence on the readmission metric used by Medicare.²⁸ Fourth, while our study included over 47 million hospitalizations, our time trend analyses used calendar month as the primary independent variable. As our study included 77 months, we may not have had sufficient power to detect small changes in risk over time.

Medicare readmissions have declined steadily in recent years, presumably at least partly in response to policy changes including HRRP. We found that hospitals have been effective at reducing readmissions across a range of patient risk strata and clinical conditions. As a result, the overall risk of readmission for hospitalized patients has remained constant for some but not all conditions. Whether institutions can continue to reduce readmission rates for most types of patients remains to be seen.

Acknowledgments

This study was supported by the Agency for Healthcare Research and Quality (AHRQ) grant R01HS022882. Dr. Blecker was supported by the AHRQ grant K08HS23683. The authors would like to thank Shawn Hoke and Jane Padikkala for administrative support.

Disclosure

This study was supported by the Agency for Healthcare Research and Quality (AHRQ) grants R01HS022882 and K08HS23683. The authors have no conflicts to report.

Given the high cost of readmissions to the healthcare system, there has been a substantial push to reduce readmissions by policymakers.¹ Among these is the Hospital Readmissions Reduction Program (HRRP), in which hospitals with higher than expected readmission rates receive reduced payments from Medicare.² Recent evidence has suggested the success of such policy changes, with multiple reports demonstrating a decrease in 30-day readmission rates in the Medicare population starting in 2010.^3-8

Initiatives to reduce readmissions can also have an effect on total number of admissions.^9,10 Indeed, along with the recent reduction in readmission, there has been a reduction in all admissions among Medicare beneficiaries.^11,12 Some studies have found that as admissions have decreased, the burden of comorbidity has increased among hospitalized patients,^3,11 suggesting that hospitals may be increasingly filled with patients at high risk of readmission. However, whether readmission risk among hospitalized patients has changed remains unknown, and understanding changes in risk profile could help inform which patients to target with future interventions to reduce readmissions.

Hospital efforts to reduce readmissions may have differential effects on types of patients by risk. For instance, low-intensity, system-wide interventions such as standardized discharge instructions or medicine reconciliation may have a stronger effect on patients at relatively low risk of readmission who may have a few important drivers of readmission that are easily overcome. Alternatively, the impact of intensive care transitions management might be greatest for high-risk patients, who have the most need for postdischarge medications, follow-up, and self-care.

The purpose of this study was therefore twofold: (1) to observe changes in average monthly risk of readmission among hospitalized Medicare patients and (2) to examine changes in readmission rates for Medicare patients at various risk of readmission. We hypothesized that readmission risk in the Medicare population would increase in recent years, as overall number of admissions and readmissions have fallen.^7,11 Additionally, we hypothesized that standardized readmission rates would decline less in highest risk patients as compared with the lowest risk patients because transitional care interventions may not be able to mitigate the large burden of comorbidity and social issues present in many high-risk patients.^13,14

We performed a retrospective cohort study of hospitalizations to US nonfederal short-term acute care facilities by Medicare beneficiaries between January 2009 and June 2015. The design involved 4 steps. First, we estimated a predictive model for unplanned readmissions within 30 days of discharge. Second, we assigned each hospitalization a predicted risk of readmission based on the model. Third, we studied trends in mean predicted risk of readmission during the study period. Fourth, we examined trends in observed to expected (O/E) readmission for hospitalizations in the lowest, middle, and highest categories of predicted risk of readmission to determine whether reductions in readmissions were more substantial in certain risk groups than in others.

Data were obtained from the Centers for Medicare and Medicaid Services (CMS) Inpatient Standard Analytic File and the Medicare Enrollment Data Base. We included hospitalizations of fee-for-service Medicare beneficiaries age ≥65 with continuous enrollment in Part A Medicare fee-for-service for at least 1 year prior and 30 days after the hospitalization.¹⁵ Hospitalizations with a discharge disposition of death, transfer to another acute hospital, and left against medical advice (AMA) were excluded. We also excluded patients with enrollment in hospice care prior to hospitalization. We excluded hospitalizations in June 2012 because of an irregularity in data availability for that month.

Hospitalizations were categorized into 5 specialty cohorts according to service line. The 5 cohorts were those used for the CMS hospital-wide readmission measure and included surgery/gynecology, medicine, cardiovascular, cardiorespiratory, and neurology.¹⁵ Among the 3 clinical conditions tracked as part of HRRP, heart failure and pneumonia were a subset of the cardiorespiratory cohort, while acute myocardial infarction was a subset of the cardiovascular cohort. Our use of cohorts was threefold: first, the average risk of readmission differs substantially across these cohorts, so pooling them produces heterogeneous risk strata; second, risk variables perform differently in different cohorts, so one single model may not be as accurate for calculating risk; and, third, the use of disease cohorts makes our results comparable to the CMS model and similar to other readmission studies in Medicare.^7,8,15

For development of the risk model, the outcome was 30-day unplanned hospital readmission. Planned readmissions were excluded; these were defined by the CMS algorithm as readmissions in which a typically planned procedure occurred in a hospitalization with a nonacute principal diagnosis.¹⁶ Independent variables included age and comorbidities in the final hospital-wide readmission models for each of the 5 specialty cohorts.¹⁵ In order to produce the best possible individual risk prediction for each patient, we added additional independent variables that CMS avoids for hospital quality measurement purposes but that contribute to risk of readmission: sex, race, dual eligibility status, number of prior AMA discharges, intensive care unit stay during current hospitalization, coronary care unit stay during current hospitalization, and hospitalization in the prior 30, 90, and 180 days. We also included an indicator variable for hospitalizations with more than 9 discharge diagnosis codes on or after January 2011, the time at which Medicare allowed an increase of the number of International Classification of Diseases, 9th Revision-Clinical Modification diagnosis billing codes from 9 to 25.¹⁷ This indicator adjusts for the increased availability of comorbidity codes, which might otherwise inflate the predicted risk relative to hospitalizations prior to that date.

Based on the risk models, each hospitalization was assigned a predicted risk of readmission. For each specialty cohort, we pooled all hospitalizations across all study years and divided them into risk quintiles. We categorized hospitalizations as high risk if in the highest quintile, medium risk if in the middle 3 quintiles, and low risk if in the lowest quintile of predicted risk for all study hospitalizations in a given specialty cohort.

For our time trend analyses, we studied 2 outcomes: monthly mean predicted risk and monthly ratio of observed readmissions to expected readmissions for patients in the lowest, middle, and highest categories of predicted risk of readmission. We studied monthly predicted risk to determine whether the average readmission risk of patients was changing over time as admission and readmission rates were declining. We studied the ratio of O/E readmissions to determine whether the decline in overall readmissions was more substantial in particular risk strata; we used the ratio of O/E readmissions, which measures number of readmissions divided by number of readmissions predicted by the model, rather than crude observed readmissions, as O/E readmissions account for any changes in risk profiles over time within each risk stratum. Independent variables in our trend analyses were year—entered as a continuous variable—and indicators for postintroduction of the Affordable Care Act (ACA, March 2010) and for postintroduction of HRRP (October 2012); these time indicators were included because of prior studies demonstrating that the introduction of ACA was associated with a decrease from baseline in readmission rates, which leveled off after introduction of HRRP.⁷ We also included an indicator for calendar quarter to account for seasonal effects.

Statistical Analysis

We developed generalized estimating equation models to predict 30-day unplanned readmission for each of the 5 specialty cohorts. The 5 models were fit using all patients in each cohort for the included time period and were adjusted for clustering by hospital. We assessed discrimination by calculating area under the receiver operating characteristic curve (AUC) for the 5 models; the AUCs measured the models’ ability to distinguish patients who were readmitted versus those who were not.¹⁸ We also calculated AUCs for each year to examine model performance over time.

Using these models, we calculated predicted risk for each hospitalization and averaged these to obtain mean predicted risk for each specialty cohort for each month. To test for trends in mean risk, we estimated 5 time series models, one for each cohort, with the dependent variable of monthly mean predicted risk. For each cohort, we first estimated a series of 12 empty autoregressive models, each with a different autoregressive term (1, 2...12). For each model, we calculated χ² for the test that the autocorrelation was 0; based on a comparison of chi-squared values, we specified an autocorrelation of 1 month for all models. Accordingly, a 1-month lag was used to estimate one final model for each cohort. Independent variables included year and indicators for post-ACA and post-HRRP; these variables captured the effect of trends over time and the introduction of these policy changes, respectively.¹⁹

To determine whether changes in risk over time were a result of changes in particular risk groups, we categorized hospitalizations into risk strata based on quintiles of predicted risk for each specialty cohort for the entire study period. For each individual year, we calculated the proportion of hospitalizations in the highest, middle, and lowest readmission risk strata for each cohort.

We calculated the monthly ratio of O/E readmission for hospitalizations in the lowest 20%, middle 60%, and highest 20% of readmission risk by month; O/E reflects the excess or deficit observed events relative to the number predicted by the model. Using this monthly O/E as the dependent variable, we developed autoregressive time series models as above, again with a 1-month lag, for each of these 3 risk strata in each cohort. As before, independent variables were year as a continuous variable, indicator variables for post-ACA and post-HRRP, and a categorical variable for calendar quarter.

All analyses were done in SAS version 9.3 (SAS Institute Inc., Cary, NC) and Stata version 14.2 (StataCorp LLC, College Station, TX).

We included 47,288,961 hospitalizations in the study, of which 11,231,242 (23.8%) were in the surgery/gynecology cohort, 19,548,711 (41.3%) were in the medicine cohort, 5,433,125 (11.5%) were in the cardiovascular cohort, 8,179,691 (17.3%) were in the cardiorespiratory cohort, and 2,896,192 (6.1%) were in the neurology cohort. The readmission rate was 16.2% (n = 7,642,161) overall, with the highest rates observed in the cardiorespiratory (20.5%) and medicine (17.6%) cohorts and the lowest rates observed in the surgery/gynecology (11.8%) and neurology (13.8%) cohorts.

The final predictive models for each cohort ranged in number of parameters from 56 for the cardiorespiratory cohort to 264 for the surgery/gynecology cohort. The models had AUCs of 0.70, 0.65, 0.67, 0.65, and 0.63 for the surgery/gynecology, medicine, cardiovascular, cardiorespiratory, and neurology cohorts, respectively; AUCs remained fairly stable over time for all disease cohorts (Appendix Table 1).

A number of mechanisms may account for the across-the-board improvements in readmission reduction. Many hospitals have instituted system-wide interventions, including patient education, medicine reconciliation, and early postdischarge follow-up,²⁰ which may have reduced readmissions across all patient risk strata. Alternatively, hospitals may have implemented interventions that disproportionally benefited low-risk patients while simultaneously utilizing interventions that only benefited high-risk patients. For instance, increasing threshold for admission⁷ may have the greatest effect on low-risk patients who could be most easily managed at home, while many intensive transitional care interventions have been developed to target only high-risk patients.^21,22

With the introduction of HRRP, there have been a number of concerns about the readmission measure used to penalize hospitals for high readmission rates. One major concern has been that the readmission metric may be flawed in its ability to capture continued improvement related to readmission.²³ Some have suggested that with better population health management, admissions will decrease, patient risk of the remaining patients will increase, and hospitals will be increasingly filled with patients who have high likelihood of readmission. This potential for increased risk with HRRP was suggested by a recent study that found that comorbidities increased in hospitalized Medicare beneficiaries between 2010 and 2013.¹¹ Our results were mixed in supporting this potential phenomenon because we examined global risk of readmission and found that some of the cohorts had increased risk over time while others did not. Others have expressed concern that readmission measure does not account for socioeconomic status, which has been associated with readmission rates.^24-27 Although we did not directly examine socioeconomic status in our study, we found that hospitals have been able to reduce readmission across all levels of risk, which includes markers of socioeconomic status, including race and Medicaid eligibility status.

Although we hypothesized that readmission risk would increase as number of hospitalizations decreased over time, we found no increase in readmission risk among the cohorts with HRRP diagnoses that had the largest decrease in readmission rates.^7,8 Conversely, readmission risk did increase—with a concurrent increase in the proportion of high-risk hospitalizations—in the surgery/gynecology and neurology cohorts that were not subject to HRRP penalties. Nonetheless, rehospitalizations were reduced for all risk categories in these 2 cohorts. Notably, surgery/gynecology and neurology had the lowest readmission rates overall. These findings suggest that initiatives to prevent initial hospitalizations, such as increasing the threshold for postoperative admission, may have had a greater effect on low- versus high-risk patients in low-risk hospitalizations. However, once a patient is hospitalized, multidisciplinary strategies appear to be effective at reducing readmissions for all risk classes in these cohorts.

For the 3 cohorts in which we observed an increase in readmission risk among hospitalized patients, the risk appeared to increase in early 2011. This time was about 10 months after passage of ACA, the timing of which was previously associated with a drop in readmission rates,^7,8 but well before HRRP went into effect in October 2012. The increase in readmission risk coincided with an increase in the number of diagnostic codes that could be included on a hospital claim to Medicare.¹⁷ This increase in allowable codes allowed us to capture more diagnoses for some patients, potentially resulting in an increase in apparent predicted risk of readmissions. While we adjusted for this in our predictive models, we may not have fully accounted for differences in risk related to coding change. As a result, some of the observed differences in risk in our study may be attributable to coding differences. More broadly, studies demonstrating the success of HRRP have typically examined risk-adjusted rates of readmission.^3,7 It is possible that a small portion of the observed reduction in risk-adjusted readmission rates may be related to the increase in predicted risk of readmission observed in our study. Future assessment of trends in readmission during this period should consider accounting for change in the number of allowed billing codes.

Other limitations should be considered in the interpretation of this study. First, like many predictive models for readmission,¹⁴ ours had imperfect discrimination, which could affect our results. Second, our study was based on older Medicare patients, so findings may not be applicable to younger patients. Third, while we accounted for surrogates for socioeconomic status, including dual eligibility and race, our models lacked other socioeconomic and community factors that can influence readmission.^24-26 Nonetheless, 1 study suggested that easily measured socioeconomic factors may not have a strong influence on the readmission metric used by Medicare.²⁸ Fourth, while our study included over 47 million hospitalizations, our time trend analyses used calendar month as the primary independent variable. As our study included 77 months, we may not have had sufficient power to detect small changes in risk over time.

Medicare readmissions have declined steadily in recent years, presumably at least partly in response to policy changes including HRRP. We found that hospitals have been effective at reducing readmissions across a range of patient risk strata and clinical conditions. As a result, the overall risk of readmission for hospitalized patients has remained constant for some but not all conditions. Whether institutions can continue to reduce readmission rates for most types of patients remains to be seen.

Acknowledgments

This study was supported by the Agency for Healthcare Research and Quality (AHRQ) grant R01HS022882. Dr. Blecker was supported by the AHRQ grant K08HS23683. The authors would like to thank Shawn Hoke and Jane Padikkala for administrative support.

Disclosure

This study was supported by the Agency for Healthcare Research and Quality (AHRQ) grants R01HS022882 and K08HS23683. The authors have no conflicts to report.

Given the high cost of readmissions to the healthcare system, there has been a substantial push to reduce readmissions by policymakers.¹ Among these is the Hospital Readmissions Reduction Program (HRRP), in which hospitals with higher than expected readmission rates receive reduced payments from Medicare.² Recent evidence has suggested the success of such policy changes, with multiple reports demonstrating a decrease in 30-day readmission rates in the Medicare population starting in 2010.^3-8

Initiatives to reduce readmissions can also have an effect on total number of admissions.^9,10 Indeed, along with the recent reduction in readmission, there has been a reduction in all admissions among Medicare beneficiaries.^11,12 Some studies have found that as admissions have decreased, the burden of comorbidity has increased among hospitalized patients,^3,11 suggesting that hospitals may be increasingly filled with patients at high risk of readmission. However, whether readmission risk among hospitalized patients has changed remains unknown, and understanding changes in risk profile could help inform which patients to target with future interventions to reduce readmissions.

Hospital efforts to reduce readmissions may have differential effects on types of patients by risk. For instance, low-intensity, system-wide interventions such as standardized discharge instructions or medicine reconciliation may have a stronger effect on patients at relatively low risk of readmission who may have a few important drivers of readmission that are easily overcome. Alternatively, the impact of intensive care transitions management might be greatest for high-risk patients, who have the most need for postdischarge medications, follow-up, and self-care.

The purpose of this study was therefore twofold: (1) to observe changes in average monthly risk of readmission among hospitalized Medicare patients and (2) to examine changes in readmission rates for Medicare patients at various risk of readmission. We hypothesized that readmission risk in the Medicare population would increase in recent years, as overall number of admissions and readmissions have fallen.^7,11 Additionally, we hypothesized that standardized readmission rates would decline less in highest risk patients as compared with the lowest risk patients because transitional care interventions may not be able to mitigate the large burden of comorbidity and social issues present in many high-risk patients.^13,14

We performed a retrospective cohort study of hospitalizations to US nonfederal short-term acute care facilities by Medicare beneficiaries between January 2009 and June 2015. The design involved 4 steps. First, we estimated a predictive model for unplanned readmissions within 30 days of discharge. Second, we assigned each hospitalization a predicted risk of readmission based on the model. Third, we studied trends in mean predicted risk of readmission during the study period. Fourth, we examined trends in observed to expected (O/E) readmission for hospitalizations in the lowest, middle, and highest categories of predicted risk of readmission to determine whether reductions in readmissions were more substantial in certain risk groups than in others.

Data were obtained from the Centers for Medicare and Medicaid Services (CMS) Inpatient Standard Analytic File and the Medicare Enrollment Data Base. We included hospitalizations of fee-for-service Medicare beneficiaries age ≥65 with continuous enrollment in Part A Medicare fee-for-service for at least 1 year prior and 30 days after the hospitalization.¹⁵ Hospitalizations with a discharge disposition of death, transfer to another acute hospital, and left against medical advice (AMA) were excluded. We also excluded patients with enrollment in hospice care prior to hospitalization. We excluded hospitalizations in June 2012 because of an irregularity in data availability for that month.

Hospitalizations were categorized into 5 specialty cohorts according to service line. The 5 cohorts were those used for the CMS hospital-wide readmission measure and included surgery/gynecology, medicine, cardiovascular, cardiorespiratory, and neurology.¹⁵ Among the 3 clinical conditions tracked as part of HRRP, heart failure and pneumonia were a subset of the cardiorespiratory cohort, while acute myocardial infarction was a subset of the cardiovascular cohort. Our use of cohorts was threefold: first, the average risk of readmission differs substantially across these cohorts, so pooling them produces heterogeneous risk strata; second, risk variables perform differently in different cohorts, so one single model may not be as accurate for calculating risk; and, third, the use of disease cohorts makes our results comparable to the CMS model and similar to other readmission studies in Medicare.^7,8,15

For development of the risk model, the outcome was 30-day unplanned hospital readmission. Planned readmissions were excluded; these were defined by the CMS algorithm as readmissions in which a typically planned procedure occurred in a hospitalization with a nonacute principal diagnosis.¹⁶ Independent variables included age and comorbidities in the final hospital-wide readmission models for each of the 5 specialty cohorts.¹⁵ In order to produce the best possible individual risk prediction for each patient, we added additional independent variables that CMS avoids for hospital quality measurement purposes but that contribute to risk of readmission: sex, race, dual eligibility status, number of prior AMA discharges, intensive care unit stay during current hospitalization, coronary care unit stay during current hospitalization, and hospitalization in the prior 30, 90, and 180 days. We also included an indicator variable for hospitalizations with more than 9 discharge diagnosis codes on or after January 2011, the time at which Medicare allowed an increase of the number of International Classification of Diseases, 9th Revision-Clinical Modification diagnosis billing codes from 9 to 25.¹⁷ This indicator adjusts for the increased availability of comorbidity codes, which might otherwise inflate the predicted risk relative to hospitalizations prior to that date.

Based on the risk models, each hospitalization was assigned a predicted risk of readmission. For each specialty cohort, we pooled all hospitalizations across all study years and divided them into risk quintiles. We categorized hospitalizations as high risk if in the highest quintile, medium risk if in the middle 3 quintiles, and low risk if in the lowest quintile of predicted risk for all study hospitalizations in a given specialty cohort.

For our time trend analyses, we studied 2 outcomes: monthly mean predicted risk and monthly ratio of observed readmissions to expected readmissions for patients in the lowest, middle, and highest categories of predicted risk of readmission. We studied monthly predicted risk to determine whether the average readmission risk of patients was changing over time as admission and readmission rates were declining. We studied the ratio of O/E readmissions to determine whether the decline in overall readmissions was more substantial in particular risk strata; we used the ratio of O/E readmissions, which measures number of readmissions divided by number of readmissions predicted by the model, rather than crude observed readmissions, as O/E readmissions account for any changes in risk profiles over time within each risk stratum. Independent variables in our trend analyses were year—entered as a continuous variable—and indicators for postintroduction of the Affordable Care Act (ACA, March 2010) and for postintroduction of HRRP (October 2012); these time indicators were included because of prior studies demonstrating that the introduction of ACA was associated with a decrease from baseline in readmission rates, which leveled off after introduction of HRRP.⁷ We also included an indicator for calendar quarter to account for seasonal effects.

Statistical Analysis

We developed generalized estimating equation models to predict 30-day unplanned readmission for each of the 5 specialty cohorts. The 5 models were fit using all patients in each cohort for the included time period and were adjusted for clustering by hospital. We assessed discrimination by calculating area under the receiver operating characteristic curve (AUC) for the 5 models; the AUCs measured the models’ ability to distinguish patients who were readmitted versus those who were not.¹⁸ We also calculated AUCs for each year to examine model performance over time.

Using these models, we calculated predicted risk for each hospitalization and averaged these to obtain mean predicted risk for each specialty cohort for each month. To test for trends in mean risk, we estimated 5 time series models, one for each cohort, with the dependent variable of monthly mean predicted risk. For each cohort, we first estimated a series of 12 empty autoregressive models, each with a different autoregressive term (1, 2...12). For each model, we calculated χ² for the test that the autocorrelation was 0; based on a comparison of chi-squared values, we specified an autocorrelation of 1 month for all models. Accordingly, a 1-month lag was used to estimate one final model for each cohort. Independent variables included year and indicators for post-ACA and post-HRRP; these variables captured the effect of trends over time and the introduction of these policy changes, respectively.¹⁹

To determine whether changes in risk over time were a result of changes in particular risk groups, we categorized hospitalizations into risk strata based on quintiles of predicted risk for each specialty cohort for the entire study period. For each individual year, we calculated the proportion of hospitalizations in the highest, middle, and lowest readmission risk strata for each cohort.

We calculated the monthly ratio of O/E readmission for hospitalizations in the lowest 20%, middle 60%, and highest 20% of readmission risk by month; O/E reflects the excess or deficit observed events relative to the number predicted by the model. Using this monthly O/E as the dependent variable, we developed autoregressive time series models as above, again with a 1-month lag, for each of these 3 risk strata in each cohort. As before, independent variables were year as a continuous variable, indicator variables for post-ACA and post-HRRP, and a categorical variable for calendar quarter.

All analyses were done in SAS version 9.3 (SAS Institute Inc., Cary, NC) and Stata version 14.2 (StataCorp LLC, College Station, TX).

We included 47,288,961 hospitalizations in the study, of which 11,231,242 (23.8%) were in the surgery/gynecology cohort, 19,548,711 (41.3%) were in the medicine cohort, 5,433,125 (11.5%) were in the cardiovascular cohort, 8,179,691 (17.3%) were in the cardiorespiratory cohort, and 2,896,192 (6.1%) were in the neurology cohort. The readmission rate was 16.2% (n = 7,642,161) overall, with the highest rates observed in the cardiorespiratory (20.5%) and medicine (17.6%) cohorts and the lowest rates observed in the surgery/gynecology (11.8%) and neurology (13.8%) cohorts.

The final predictive models for each cohort ranged in number of parameters from 56 for the cardiorespiratory cohort to 264 for the surgery/gynecology cohort. The models had AUCs of 0.70, 0.65, 0.67, 0.65, and 0.63 for the surgery/gynecology, medicine, cardiovascular, cardiorespiratory, and neurology cohorts, respectively; AUCs remained fairly stable over time for all disease cohorts (Appendix Table 1).

A number of mechanisms may account for the across-the-board improvements in readmission reduction. Many hospitals have instituted system-wide interventions, including patient education, medicine reconciliation, and early postdischarge follow-up,²⁰ which may have reduced readmissions across all patient risk strata. Alternatively, hospitals may have implemented interventions that disproportionally benefited low-risk patients while simultaneously utilizing interventions that only benefited high-risk patients. For instance, increasing threshold for admission⁷ may have the greatest effect on low-risk patients who could be most easily managed at home, while many intensive transitional care interventions have been developed to target only high-risk patients.^21,22

With the introduction of HRRP, there have been a number of concerns about the readmission measure used to penalize hospitals for high readmission rates. One major concern has been that the readmission metric may be flawed in its ability to capture continued improvement related to readmission.²³ Some have suggested that with better population health management, admissions will decrease, patient risk of the remaining patients will increase, and hospitals will be increasingly filled with patients who have high likelihood of readmission. This potential for increased risk with HRRP was suggested by a recent study that found that comorbidities increased in hospitalized Medicare beneficiaries between 2010 and 2013.¹¹ Our results were mixed in supporting this potential phenomenon because we examined global risk of readmission and found that some of the cohorts had increased risk over time while others did not. Others have expressed concern that readmission measure does not account for socioeconomic status, which has been associated with readmission rates.^24-27 Although we did not directly examine socioeconomic status in our study, we found that hospitals have been able to reduce readmission across all levels of risk, which includes markers of socioeconomic status, including race and Medicaid eligibility status.

Although we hypothesized that readmission risk would increase as number of hospitalizations decreased over time, we found no increase in readmission risk among the cohorts with HRRP diagnoses that had the largest decrease in readmission rates.^7,8 Conversely, readmission risk did increase—with a concurrent increase in the proportion of high-risk hospitalizations—in the surgery/gynecology and neurology cohorts that were not subject to HRRP penalties. Nonetheless, rehospitalizations were reduced for all risk categories in these 2 cohorts. Notably, surgery/gynecology and neurology had the lowest readmission rates overall. These findings suggest that initiatives to prevent initial hospitalizations, such as increasing the threshold for postoperative admission, may have had a greater effect on low- versus high-risk patients in low-risk hospitalizations. However, once a patient is hospitalized, multidisciplinary strategies appear to be effective at reducing readmissions for all risk classes in these cohorts.

For the 3 cohorts in which we observed an increase in readmission risk among hospitalized patients, the risk appeared to increase in early 2011. This time was about 10 months after passage of ACA, the timing of which was previously associated with a drop in readmission rates,^7,8 but well before HRRP went into effect in October 2012. The increase in readmission risk coincided with an increase in the number of diagnostic codes that could be included on a hospital claim to Medicare.¹⁷ This increase in allowable codes allowed us to capture more diagnoses for some patients, potentially resulting in an increase in apparent predicted risk of readmissions. While we adjusted for this in our predictive models, we may not have fully accounted for differences in risk related to coding change. As a result, some of the observed differences in risk in our study may be attributable to coding differences. More broadly, studies demonstrating the success of HRRP have typically examined risk-adjusted rates of readmission.^3,7 It is possible that a small portion of the observed reduction in risk-adjusted readmission rates may be related to the increase in predicted risk of readmission observed in our study. Future assessment of trends in readmission during this period should consider accounting for change in the number of allowed billing codes.

Other limitations should be considered in the interpretation of this study. First, like many predictive models for readmission,¹⁴ ours had imperfect discrimination, which could affect our results. Second, our study was based on older Medicare patients, so findings may not be applicable to younger patients. Third, while we accounted for surrogates for socioeconomic status, including dual eligibility and race, our models lacked other socioeconomic and community factors that can influence readmission.^24-26 Nonetheless, 1 study suggested that easily measured socioeconomic factors may not have a strong influence on the readmission metric used by Medicare.²⁸ Fourth, while our study included over 47 million hospitalizations, our time trend analyses used calendar month as the primary independent variable. As our study included 77 months, we may not have had sufficient power to detect small changes in risk over time.

Medicare readmissions have declined steadily in recent years, presumably at least partly in response to policy changes including HRRP. We found that hospitals have been effective at reducing readmissions across a range of patient risk strata and clinical conditions. As a result, the overall risk of readmission for hospitalized patients has remained constant for some but not all conditions. Whether institutions can continue to reduce readmission rates for most types of patients remains to be seen.

Acknowledgments

This study was supported by the Agency for Healthcare Research and Quality (AHRQ) grant R01HS022882. Dr. Blecker was supported by the AHRQ grant K08HS23683. The authors would like to thank Shawn Hoke and Jane Padikkala for administrative support.

Disclosure

This study was supported by the Agency for Healthcare Research and Quality (AHRQ) grants R01HS022882 and K08HS23683. The authors have no conflicts to report.

Point-of-care ultrasound (POCUS) is a valuable tool to assist in the diagnosis and treatment of many common diseases.^1-11 Its use has increased in clinical settings over the years, primarily because of more portable, economical, high-quality devices and training availability.¹² POCUS improves procedural success and guides the diagnostic management of hospitalized patients.^2,9-12 Literature details the training of medical students,^13,14 residents,^15-21 and providers in emergency medicine²² and critical care,^23,24 as well as focused cardiac training with hospitalists.^25-27 However, no literature exists describing a comprehensive longitudinal training program for hospitalists or skills retention.

This document details the hospital medicine department’s ultrasound training program from Regions Hospital, part of HealthPartners in Saint Paul, Minnesota, a large tertiary care medical center. We describe the development and effectiveness of the Comprehensive Hospitalist Assessment and Mentorship with Portfolios (CHAMP) Ultrasound Program. This approach is intended to support the development of POCUS training programs at other organizations.

The aim of the program was to build a comprehensive bedside ultrasound training paradigm for hospitalists. The primary objective of the study was to assess the program’s effect on skills over time. Secondary objectives were confidence ratings in the use of ultrasound and with various patient care realms (volume management, quality of physical exam, and ability to narrow the differential diagnosis). We hypothesized there would be higher retention of ultrasound skills in those who completed portfolios and/or monthly scanning sessions as well as increased confidence through all secondary outcome measures (see below).

This was a retrospective descriptive report of hospitalists who entered the CHAMP Ultrasound Program. Study participants were providers from the 454-bed Regions Hospital in Saint Paul, Minnesota. The study was deemed exempt by the HealthPartners Institutional Review Board. Three discrete 3-day courses and two 1-day in-person courses held at the Regions Hospital Simulation Center (Saint Paul, Minnesota) were studied.

Program Description

In 2014, a working group was developed in the hospital medicine department to support the hospital-wide POCUS committee with a charter to provide standardized training for providers to complete credentialing.²⁸ The goal of the hospital medicine ultrasound program was to establish the use of ultrasound by credentialed hospitalists into well-defined applications integrated into the practice of hospital medicine. Two providers were selected to lead the efforts and completed additional training through the American College of Chest Physicians (CHEST) Certificate of Completion Program.²⁹ An overall director was designated with the responsibilities delineated in supplementary Appendix 1. This director provided leadership on group practice, protocols, and equipment, creating the organizational framework for success with the training program. The hospital medicine training program had a 3-day in-person component built off the CHEST Critical Care Ultrasonography Program.²⁴ The curriculum was adapted from the American College of Chest Physicians/Société de Réanimation de Langue Française Statement on Competence in Critical Care Ultrasonography.³⁰ See Table 1 for the components of the training program.

Online Modules

As a prerequisite to the 3-day introductory course, hospitalists were required to complete modules for precourse knowledge involving a set of focused-topic online reading and videos with quizzes (supplementary Appendix 3).

3-Day In-Person Course with Assessments

The 3-day course provided 6 hours of didactics, 8 hours of image interpretation, and 9 hours of hands-on instruction (supplementary Appendix 4). Hospitalists first attended a large group didactic, followed by divided groups in image interpretation and hands-on scanning.²⁴

Didactics were provided in a room with a 2-screen set up. Providers used 1 screen to present primary content and the other for simultaneously scanning a human model.

Image interpretation sessions were interactive smaller group learning forums in which participants reviewed high-yield images related to the care of hospital medicine patients and received feedback. Approximately 45 videos with normal and abnormal findings were reviewed during each session.

The hands-on scanning component was accomplished with human models and a faculty-to-participant ratio between 1:2 and 1:3. Human models for this course were paid community models. A variety of ultrasound machine platforms were provided for participants. Learning objectives were clearly delineated prior to each scanning session to ensure the coverage of required content.

Portfolios

Refresher Training: 1-Day In-Person Course with Assessments and Monthly Scanning Sessions (Optional)

Only hospitalists who completed the 3-day course were eligible to take the 1-day in-person refresher course (supplementary Appendix 5). The first half of the course incorporated scanning with live human models, while the second half of the course had scanning with hospitalized patients focusing on pathology (pleural effusion, hydronephrosis, reduced left ventricular function, etc.). The course was offered at 3, 6, and 12 months after the initial 3-day course.

Monthly scanning sessions occurred for 2 hours every third Friday and were also available prior to the 1-day refresher. The first 90 minutes had a hands-on scanning component with hospitalized patients with faculty supervision (1:2 ratio). The last 30 minutes had an image interpretation component.

Assessments

Knowledge and skills assessment were adapted from the CHEST model (supplementary Appendix 6).²⁴ Before and after the 3-day and 1-day in-person courses, the same hands-on skills assessment with a checklist was provided (supplementary Appendix 7). Before and after the 3-day course, a written knowledge assessment with case-based image interpretation was provided (supplementary Appendix 6). A final knowledge and skills assessment was given at either of the in-person courses to those who completed the required components of the training. Passing scores for the final knowledge assessment were established at 85% items correct by an expert panel by using the Angoff method.³¹ This same standard was applied to the final skills examination. Participants who do not pass the final assessments are provided opportunities for further training and allowed to reattempt the assessments. In this regard, there is a standard training outcome but variances in length of training time for each participant. Pre- and postcourse skills assessments used the same faculty, checklist, and ultrasound device. Raters received an orientation the day prior to each in-person course, reviewing common learner pitfalls, reviewing the checklist, and discussing specific examples.

Measurement

Participant demographic and clinical information was collected at the initial 3-day course for all participants, including age, gender, specialty, years of experience, and number and type of ultrasound procedures personally conducted or supervised in the past year. For skills assessment, a 20-item dichotomous checklist was developed and scored as done correctly or not done/done incorrectly. This same assessment was provided both before and after each of the 3-day and 1-day courses. A 20-question image-based knowledge assessment was also developed and administered both before and after the 3-day course only. The same 20-item checklist was used for the final skills examination. However, a new more detailed 50-question examination was written for the final examination after the portfolio of images was complete. Self-reported measures were confidence in the use of ultrasound, volume management, quality of physical exam, and ability to narrow the differential diagnosis. Confidence in ultrasound use, confidence in volume management, and quality of physical exam were assessed by using a questionnaire both before and after the 3-day course and 1-day course. Participants rated confidence and quality on a 5-point scale, 1 being least confident and 5 being most confident.

Statistical Analysis

Demographics of the included hospitalist population and pre and post 3-day assessments, including knowledge score, skills score, confidence in ultrasound use, confidence in volume management, and quality of physical exam, were summarized. Values for all assessment variables are presented as percentages. Confidence scores were reported as a percentage of the Likert scale (eg, 4/5 was reported as 80%). Skills and written examinations were expressed as percentages of items correct. Data were reported as median and interquartile range or means and standard deviation based on variable distributions. Differences between pre- and postvalues for 3-day course variables were assessed by using 2-sample paired Wilcoxon signed rank tests with a 95% confidence level.

For the subset of hospitalists who also completed the 1-day course, pre and post 1-day course assessments, including skills score, confidence in ultrasound use, confidence in volume management, and quality of physical exam, were summarized. Differences between pre- and postvalues for 1-day assessment variables were assessed by using 2-sample paired Wilcoxon signed rank tests with a 95% confidence level.

Demographics

3-Day In-Person Course

For the 53 hospitalists who completed skills-based assessments, performance increased significantly after the 3-day course. Knowledge scores also increased significantly from preassessment to postassessment. Self-reported confidence ratings for ultrasound use, confidence in volume management, and quality of physical exam all increased significantly from preassessment to postassessment (Table 2).

Refresher Training: 1-Day In-Person Course

Because the refresher training was encouraged but not required, only 25 of 53 hospitalists, 23 with complete data, completed the 1-day course. For the 23 hospitalists who completed skills-based assessments before and after the 1-day course, mean skills scores increased significantly (Table 2). Self-reported confidence ratings for ultrasound use, confidence in volume management, and quality of physical exam all increased significantly from preassessment to postassessment (Table 2).

Monthly Scanning Sessions and Portfolio Development

The skills retention from initial course to refresher course by portfolio completion and monthly scanning sessions is shown in Table 2. Multiple regression analysis showed that for every 10% increase in the percent of monthly sessions attended, the mean change in skills score was 3.7% (P = .017), and for every 10% increase in the percent of portfolio completed, the mean change in skills score was 2.5% (P = .04), showing that both monthly scanning session attendance and portfolio completion are significantly predictive of skills retention over time.

Final Assessments

Four providers met mastery at initial attempt. No providers to date have needed remediation. Many others are going through different stages of the process and are expected to attain mastery in a short period of time.

This is the first description of a successful longitudinal training program with assessments in POCUS for hospital medicine providers that shows an increase in skill retention with the use of a follow-up course and bedside scanning.

The CHAMP Ultrasound Program was developed to provide hospital medicine clinicians with a specialty focused in-house training pathway in POCUS and to assist in sustained skills acquisition by providing opportunities for regular feedback and practice. Practice with regular expert feedback is a critical aspect to develop and maintain skills in POCUS.^32,33 Arntfield³⁴ described the utility of remote supervision with feedback for ultrasound training in critical care, which demonstrated varying learning curves in the submission of portfolio images.^35,36 The CHAMP Ultrasound training program provided expert oversight, longitudinal supervision, and feedback for course participants. The educational method of mastery learning was employed by setting minimum standards and allowing learners to practice until they met that standard.^37-39

This unique program is made possible by the availability of expert-level faculty. Assessment scores improved with an initial 3-day course; however, they also decayed over time, most prominently with hospitalists that did not continue with POCUS scanning after their initial course. Ironically, those who performed more ultrasounds in the year prior to beginning the 3-day course had lower confidence ratings, likely explained by their awareness of their limitations and opportunities for improvement. The incorporation of refresher training to supplement the core 3-day course and portfolio development are key additions that differentiate this training program. These additions and the demonstration of successful training make this a durable pathway for other hospitalist programs. There are many workshops and short courses for medical students, residents, and practicing providers in POCUS.^40-43 However, without an opportunity for longitudinal supervision and feedback, there is a noted decrease in the skills for participants. The refresher training with its 2 components (1-day in-person course and monthly scanning sessions) provides evidence of the value of mentored training.

In the initial program development, refresher training was encouraged but optional. We intentionally tracked those that completed refresher training compared with those that did not. Based on the results showing significant skills retention among those attending some form of refresher training, the program is planned to change to make this a requirement. We recommend refresher training within 12 months of the initial introductory course. There were several hospitalists that were unable to accommodate taking a full-day refresher course and, therefore, monthly scanning sessions were provided as an alternative.

The main limitation of the study is that it was completed in a single hospital system with available training mentors in POCUS. This gave us the ability to perform longitudinal training but may make this less reproducible in other hospital systems. Another limitation is that our course participants did not complete the pre- and postknowledge assessments for the refresher training components of the program, though they did for the initial 3-day course. Our pre- and postassessments have not been externally shown to produce valid data, though they are based on the already validated CHEST ultrasound data.⁴⁴

Finally, our CHAMP Ultrasound Program required a significant time commitment by both faculty and learners. A relatively small percentage of hospitalists have completed the final assessments. The reasons are multifactorial, including program rigor, desire by certain hospitalists to know the basics but not pursue more expertise, and the challenges of developing a skillset that takes dedicated practice over time. We have aimed to address these barriers by providing additional hands-on scanning opportunities, giving timely feedback with portfolios, and obtaining more ultrasound machines. We expect more hospitalists to complete the final assessments in the coming year as evidenced by portfolio submissions to the shared online portal and many choosing to attend either the monthly scanning sessions and/or the 1-day course. We recognize that other institutions may need to adapt our program to suit their local environment.

A comprehensive longitudinal ultrasound training program including competency assessments significantly improved ultrasound acquisition skills with hospitalists. Those attending monthly scanning sessions and participating in the portfolio completion as well as a refresher course significantly retained and augmented their skills.

Acknowledgments

The authors would like to acknowledge Kelly Logue, Jason Robertson, MD, Jerome Siy, MD, Shauna Baer, and Jack Dressen for their support in the development and implementation of the POCUS program in hospital medicine.

Disclosure

The authors do not have any relevant financial disclosures to report.

Point-of-care ultrasound (POCUS) is a valuable tool to assist in the diagnosis and treatment of many common diseases.^1-11 Its use has increased in clinical settings over the years, primarily because of more portable, economical, high-quality devices and training availability.¹² POCUS improves procedural success and guides the diagnostic management of hospitalized patients.^2,9-12 Literature details the training of medical students,^13,14 residents,^15-21 and providers in emergency medicine²² and critical care,^23,24 as well as focused cardiac training with hospitalists.^25-27 However, no literature exists describing a comprehensive longitudinal training program for hospitalists or skills retention.

This document details the hospital medicine department’s ultrasound training program from Regions Hospital, part of HealthPartners in Saint Paul, Minnesota, a large tertiary care medical center. We describe the development and effectiveness of the Comprehensive Hospitalist Assessment and Mentorship with Portfolios (CHAMP) Ultrasound Program. This approach is intended to support the development of POCUS training programs at other organizations.

The aim of the program was to build a comprehensive bedside ultrasound training paradigm for hospitalists. The primary objective of the study was to assess the program’s effect on skills over time. Secondary objectives were confidence ratings in the use of ultrasound and with various patient care realms (volume management, quality of physical exam, and ability to narrow the differential diagnosis). We hypothesized there would be higher retention of ultrasound skills in those who completed portfolios and/or monthly scanning sessions as well as increased confidence through all secondary outcome measures (see below).

This was a retrospective descriptive report of hospitalists who entered the CHAMP Ultrasound Program. Study participants were providers from the 454-bed Regions Hospital in Saint Paul, Minnesota. The study was deemed exempt by the HealthPartners Institutional Review Board. Three discrete 3-day courses and two 1-day in-person courses held at the Regions Hospital Simulation Center (Saint Paul, Minnesota) were studied.

Program Description

In 2014, a working group was developed in the hospital medicine department to support the hospital-wide POCUS committee with a charter to provide standardized training for providers to complete credentialing.²⁸ The goal of the hospital medicine ultrasound program was to establish the use of ultrasound by credentialed hospitalists into well-defined applications integrated into the practice of hospital medicine. Two providers were selected to lead the efforts and completed additional training through the American College of Chest Physicians (CHEST) Certificate of Completion Program.²⁹ An overall director was designated with the responsibilities delineated in supplementary Appendix 1. This director provided leadership on group practice, protocols, and equipment, creating the organizational framework for success with the training program. The hospital medicine training program had a 3-day in-person component built off the CHEST Critical Care Ultrasonography Program.²⁴ The curriculum was adapted from the American College of Chest Physicians/Société de Réanimation de Langue Française Statement on Competence in Critical Care Ultrasonography.³⁰ See Table 1 for the components of the training program.

Online Modules

As a prerequisite to the 3-day introductory course, hospitalists were required to complete modules for precourse knowledge involving a set of focused-topic online reading and videos with quizzes (supplementary Appendix 3).

3-Day In-Person Course with Assessments

The 3-day course provided 6 hours of didactics, 8 hours of image interpretation, and 9 hours of hands-on instruction (supplementary Appendix 4). Hospitalists first attended a large group didactic, followed by divided groups in image interpretation and hands-on scanning.²⁴

Didactics were provided in a room with a 2-screen set up. Providers used 1 screen to present primary content and the other for simultaneously scanning a human model.

Image interpretation sessions were interactive smaller group learning forums in which participants reviewed high-yield images related to the care of hospital medicine patients and received feedback. Approximately 45 videos with normal and abnormal findings were reviewed during each session.

The hands-on scanning component was accomplished with human models and a faculty-to-participant ratio between 1:2 and 1:3. Human models for this course were paid community models. A variety of ultrasound machine platforms were provided for participants. Learning objectives were clearly delineated prior to each scanning session to ensure the coverage of required content.

Portfolios

Refresher Training: 1-Day In-Person Course with Assessments and Monthly Scanning Sessions (Optional)

Only hospitalists who completed the 3-day course were eligible to take the 1-day in-person refresher course (supplementary Appendix 5). The first half of the course incorporated scanning with live human models, while the second half of the course had scanning with hospitalized patients focusing on pathology (pleural effusion, hydronephrosis, reduced left ventricular function, etc.). The course was offered at 3, 6, and 12 months after the initial 3-day course.

Monthly scanning sessions occurred for 2 hours every third Friday and were also available prior to the 1-day refresher. The first 90 minutes had a hands-on scanning component with hospitalized patients with faculty supervision (1:2 ratio). The last 30 minutes had an image interpretation component.

Assessments

Knowledge and skills assessment were adapted from the CHEST model (supplementary Appendix 6).²⁴ Before and after the 3-day and 1-day in-person courses, the same hands-on skills assessment with a checklist was provided (supplementary Appendix 7). Before and after the 3-day course, a written knowledge assessment with case-based image interpretation was provided (supplementary Appendix 6). A final knowledge and skills assessment was given at either of the in-person courses to those who completed the required components of the training. Passing scores for the final knowledge assessment were established at 85% items correct by an expert panel by using the Angoff method.³¹ This same standard was applied to the final skills examination. Participants who do not pass the final assessments are provided opportunities for further training and allowed to reattempt the assessments. In this regard, there is a standard training outcome but variances in length of training time for each participant. Pre- and postcourse skills assessments used the same faculty, checklist, and ultrasound device. Raters received an orientation the day prior to each in-person course, reviewing common learner pitfalls, reviewing the checklist, and discussing specific examples.

Measurement

Participant demographic and clinical information was collected at the initial 3-day course for all participants, including age, gender, specialty, years of experience, and number and type of ultrasound procedures personally conducted or supervised in the past year. For skills assessment, a 20-item dichotomous checklist was developed and scored as done correctly or not done/done incorrectly. This same assessment was provided both before and after each of the 3-day and 1-day courses. A 20-question image-based knowledge assessment was also developed and administered both before and after the 3-day course only. The same 20-item checklist was used for the final skills examination. However, a new more detailed 50-question examination was written for the final examination after the portfolio of images was complete. Self-reported measures were confidence in the use of ultrasound, volume management, quality of physical exam, and ability to narrow the differential diagnosis. Confidence in ultrasound use, confidence in volume management, and quality of physical exam were assessed by using a questionnaire both before and after the 3-day course and 1-day course. Participants rated confidence and quality on a 5-point scale, 1 being least confident and 5 being most confident.

Statistical Analysis

Demographics of the included hospitalist population and pre and post 3-day assessments, including knowledge score, skills score, confidence in ultrasound use, confidence in volume management, and quality of physical exam, were summarized. Values for all assessment variables are presented as percentages. Confidence scores were reported as a percentage of the Likert scale (eg, 4/5 was reported as 80%). Skills and written examinations were expressed as percentages of items correct. Data were reported as median and interquartile range or means and standard deviation based on variable distributions. Differences between pre- and postvalues for 3-day course variables were assessed by using 2-sample paired Wilcoxon signed rank tests with a 95% confidence level.

For the subset of hospitalists who also completed the 1-day course, pre and post 1-day course assessments, including skills score, confidence in ultrasound use, confidence in volume management, and quality of physical exam, were summarized. Differences between pre- and postvalues for 1-day assessment variables were assessed by using 2-sample paired Wilcoxon signed rank tests with a 95% confidence level.

Demographics

3-Day In-Person Course

For the 53 hospitalists who completed skills-based assessments, performance increased significantly after the 3-day course. Knowledge scores also increased significantly from preassessment to postassessment. Self-reported confidence ratings for ultrasound use, confidence in volume management, and quality of physical exam all increased significantly from preassessment to postassessment (Table 2).

Refresher Training: 1-Day In-Person Course

Because the refresher training was encouraged but not required, only 25 of 53 hospitalists, 23 with complete data, completed the 1-day course. For the 23 hospitalists who completed skills-based assessments before and after the 1-day course, mean skills scores increased significantly (Table 2). Self-reported confidence ratings for ultrasound use, confidence in volume management, and quality of physical exam all increased significantly from preassessment to postassessment (Table 2).

Monthly Scanning Sessions and Portfolio Development

The skills retention from initial course to refresher course by portfolio completion and monthly scanning sessions is shown in Table 2. Multiple regression analysis showed that for every 10% increase in the percent of monthly sessions attended, the mean change in skills score was 3.7% (P = .017), and for every 10% increase in the percent of portfolio completed, the mean change in skills score was 2.5% (P = .04), showing that both monthly scanning session attendance and portfolio completion are significantly predictive of skills retention over time.

Final Assessments

Four providers met mastery at initial attempt. No providers to date have needed remediation. Many others are going through different stages of the process and are expected to attain mastery in a short period of time.

This is the first description of a successful longitudinal training program with assessments in POCUS for hospital medicine providers that shows an increase in skill retention with the use of a follow-up course and bedside scanning.

The CHAMP Ultrasound Program was developed to provide hospital medicine clinicians with a specialty focused in-house training pathway in POCUS and to assist in sustained skills acquisition by providing opportunities for regular feedback and practice. Practice with regular expert feedback is a critical aspect to develop and maintain skills in POCUS.^32,33 Arntfield³⁴ described the utility of remote supervision with feedback for ultrasound training in critical care, which demonstrated varying learning curves in the submission of portfolio images.^35,36 The CHAMP Ultrasound training program provided expert oversight, longitudinal supervision, and feedback for course participants. The educational method of mastery learning was employed by setting minimum standards and allowing learners to practice until they met that standard.^37-39

This unique program is made possible by the availability of expert-level faculty. Assessment scores improved with an initial 3-day course; however, they also decayed over time, most prominently with hospitalists that did not continue with POCUS scanning after their initial course. Ironically, those who performed more ultrasounds in the year prior to beginning the 3-day course had lower confidence ratings, likely explained by their awareness of their limitations and opportunities for improvement. The incorporation of refresher training to supplement the core 3-day course and portfolio development are key additions that differentiate this training program. These additions and the demonstration of successful training make this a durable pathway for other hospitalist programs. There are many workshops and short courses for medical students, residents, and practicing providers in POCUS.^40-43 However, without an opportunity for longitudinal supervision and feedback, there is a noted decrease in the skills for participants. The refresher training with its 2 components (1-day in-person course and monthly scanning sessions) provides evidence of the value of mentored training.

In the initial program development, refresher training was encouraged but optional. We intentionally tracked those that completed refresher training compared with those that did not. Based on the results showing significant skills retention among those attending some form of refresher training, the program is planned to change to make this a requirement. We recommend refresher training within 12 months of the initial introductory course. There were several hospitalists that were unable to accommodate taking a full-day refresher course and, therefore, monthly scanning sessions were provided as an alternative.

The main limitation of the study is that it was completed in a single hospital system with available training mentors in POCUS. This gave us the ability to perform longitudinal training but may make this less reproducible in other hospital systems. Another limitation is that our course participants did not complete the pre- and postknowledge assessments for the refresher training components of the program, though they did for the initial 3-day course. Our pre- and postassessments have not been externally shown to produce valid data, though they are based on the already validated CHEST ultrasound data.⁴⁴

Finally, our CHAMP Ultrasound Program required a significant time commitment by both faculty and learners. A relatively small percentage of hospitalists have completed the final assessments. The reasons are multifactorial, including program rigor, desire by certain hospitalists to know the basics but not pursue more expertise, and the challenges of developing a skillset that takes dedicated practice over time. We have aimed to address these barriers by providing additional hands-on scanning opportunities, giving timely feedback with portfolios, and obtaining more ultrasound machines. We expect more hospitalists to complete the final assessments in the coming year as evidenced by portfolio submissions to the shared online portal and many choosing to attend either the monthly scanning sessions and/or the 1-day course. We recognize that other institutions may need to adapt our program to suit their local environment.

A comprehensive longitudinal ultrasound training program including competency assessments significantly improved ultrasound acquisition skills with hospitalists. Those attending monthly scanning sessions and participating in the portfolio completion as well as a refresher course significantly retained and augmented their skills.

Acknowledgments

The authors would like to acknowledge Kelly Logue, Jason Robertson, MD, Jerome Siy, MD, Shauna Baer, and Jack Dressen for their support in the development and implementation of the POCUS program in hospital medicine.

Disclosure

The authors do not have any relevant financial disclosures to report.

Point-of-care ultrasound (POCUS) is a valuable tool to assist in the diagnosis and treatment of many common diseases.^1-11 Its use has increased in clinical settings over the years, primarily because of more portable, economical, high-quality devices and training availability.¹² POCUS improves procedural success and guides the diagnostic management of hospitalized patients.^2,9-12 Literature details the training of medical students,^13,14 residents,^15-21 and providers in emergency medicine²² and critical care,^23,24 as well as focused cardiac training with hospitalists.^25-27 However, no literature exists describing a comprehensive longitudinal training program for hospitalists or skills retention.

This document details the hospital medicine department’s ultrasound training program from Regions Hospital, part of HealthPartners in Saint Paul, Minnesota, a large tertiary care medical center. We describe the development and effectiveness of the Comprehensive Hospitalist Assessment and Mentorship with Portfolios (CHAMP) Ultrasound Program. This approach is intended to support the development of POCUS training programs at other organizations.

The aim of the program was to build a comprehensive bedside ultrasound training paradigm for hospitalists. The primary objective of the study was to assess the program’s effect on skills over time. Secondary objectives were confidence ratings in the use of ultrasound and with various patient care realms (volume management, quality of physical exam, and ability to narrow the differential diagnosis). We hypothesized there would be higher retention of ultrasound skills in those who completed portfolios and/or monthly scanning sessions as well as increased confidence through all secondary outcome measures (see below).

This was a retrospective descriptive report of hospitalists who entered the CHAMP Ultrasound Program. Study participants were providers from the 454-bed Regions Hospital in Saint Paul, Minnesota. The study was deemed exempt by the HealthPartners Institutional Review Board. Three discrete 3-day courses and two 1-day in-person courses held at the Regions Hospital Simulation Center (Saint Paul, Minnesota) were studied.

Program Description

In 2014, a working group was developed in the hospital medicine department to support the hospital-wide POCUS committee with a charter to provide standardized training for providers to complete credentialing.²⁸ The goal of the hospital medicine ultrasound program was to establish the use of ultrasound by credentialed hospitalists into well-defined applications integrated into the practice of hospital medicine. Two providers were selected to lead the efforts and completed additional training through the American College of Chest Physicians (CHEST) Certificate of Completion Program.²⁹ An overall director was designated with the responsibilities delineated in supplementary Appendix 1. This director provided leadership on group practice, protocols, and equipment, creating the organizational framework for success with the training program. The hospital medicine training program had a 3-day in-person component built off the CHEST Critical Care Ultrasonography Program.²⁴ The curriculum was adapted from the American College of Chest Physicians/Société de Réanimation de Langue Française Statement on Competence in Critical Care Ultrasonography.³⁰ See Table 1 for the components of the training program.

Online Modules

As a prerequisite to the 3-day introductory course, hospitalists were required to complete modules for precourse knowledge involving a set of focused-topic online reading and videos with quizzes (supplementary Appendix 3).

3-Day In-Person Course with Assessments

The 3-day course provided 6 hours of didactics, 8 hours of image interpretation, and 9 hours of hands-on instruction (supplementary Appendix 4). Hospitalists first attended a large group didactic, followed by divided groups in image interpretation and hands-on scanning.²⁴

Didactics were provided in a room with a 2-screen set up. Providers used 1 screen to present primary content and the other for simultaneously scanning a human model.

Image interpretation sessions were interactive smaller group learning forums in which participants reviewed high-yield images related to the care of hospital medicine patients and received feedback. Approximately 45 videos with normal and abnormal findings were reviewed during each session.

The hands-on scanning component was accomplished with human models and a faculty-to-participant ratio between 1:2 and 1:3. Human models for this course were paid community models. A variety of ultrasound machine platforms were provided for participants. Learning objectives were clearly delineated prior to each scanning session to ensure the coverage of required content.

Portfolios

Refresher Training: 1-Day In-Person Course with Assessments and Monthly Scanning Sessions (Optional)

Only hospitalists who completed the 3-day course were eligible to take the 1-day in-person refresher course (supplementary Appendix 5). The first half of the course incorporated scanning with live human models, while the second half of the course had scanning with hospitalized patients focusing on pathology (pleural effusion, hydronephrosis, reduced left ventricular function, etc.). The course was offered at 3, 6, and 12 months after the initial 3-day course.

Monthly scanning sessions occurred for 2 hours every third Friday and were also available prior to the 1-day refresher. The first 90 minutes had a hands-on scanning component with hospitalized patients with faculty supervision (1:2 ratio). The last 30 minutes had an image interpretation component.

Assessments

Knowledge and skills assessment were adapted from the CHEST model (supplementary Appendix 6).²⁴ Before and after the 3-day and 1-day in-person courses, the same hands-on skills assessment with a checklist was provided (supplementary Appendix 7). Before and after the 3-day course, a written knowledge assessment with case-based image interpretation was provided (supplementary Appendix 6). A final knowledge and skills assessment was given at either of the in-person courses to those who completed the required components of the training. Passing scores for the final knowledge assessment were established at 85% items correct by an expert panel by using the Angoff method.³¹ This same standard was applied to the final skills examination. Participants who do not pass the final assessments are provided opportunities for further training and allowed to reattempt the assessments. In this regard, there is a standard training outcome but variances in length of training time for each participant. Pre- and postcourse skills assessments used the same faculty, checklist, and ultrasound device. Raters received an orientation the day prior to each in-person course, reviewing common learner pitfalls, reviewing the checklist, and discussing specific examples.

Measurement

Participant demographic and clinical information was collected at the initial 3-day course for all participants, including age, gender, specialty, years of experience, and number and type of ultrasound procedures personally conducted or supervised in the past year. For skills assessment, a 20-item dichotomous checklist was developed and scored as done correctly or not done/done incorrectly. This same assessment was provided both before and after each of the 3-day and 1-day courses. A 20-question image-based knowledge assessment was also developed and administered both before and after the 3-day course only. The same 20-item checklist was used for the final skills examination. However, a new more detailed 50-question examination was written for the final examination after the portfolio of images was complete. Self-reported measures were confidence in the use of ultrasound, volume management, quality of physical exam, and ability to narrow the differential diagnosis. Confidence in ultrasound use, confidence in volume management, and quality of physical exam were assessed by using a questionnaire both before and after the 3-day course and 1-day course. Participants rated confidence and quality on a 5-point scale, 1 being least confident and 5 being most confident.

Statistical Analysis

Demographics of the included hospitalist population and pre and post 3-day assessments, including knowledge score, skills score, confidence in ultrasound use, confidence in volume management, and quality of physical exam, were summarized. Values for all assessment variables are presented as percentages. Confidence scores were reported as a percentage of the Likert scale (eg, 4/5 was reported as 80%). Skills and written examinations were expressed as percentages of items correct. Data were reported as median and interquartile range or means and standard deviation based on variable distributions. Differences between pre- and postvalues for 3-day course variables were assessed by using 2-sample paired Wilcoxon signed rank tests with a 95% confidence level.

For the subset of hospitalists who also completed the 1-day course, pre and post 1-day course assessments, including skills score, confidence in ultrasound use, confidence in volume management, and quality of physical exam, were summarized. Differences between pre- and postvalues for 1-day assessment variables were assessed by using 2-sample paired Wilcoxon signed rank tests with a 95% confidence level.

Demographics

3-Day In-Person Course

For the 53 hospitalists who completed skills-based assessments, performance increased significantly after the 3-day course. Knowledge scores also increased significantly from preassessment to postassessment. Self-reported confidence ratings for ultrasound use, confidence in volume management, and quality of physical exam all increased significantly from preassessment to postassessment (Table 2).

Refresher Training: 1-Day In-Person Course

Because the refresher training was encouraged but not required, only 25 of 53 hospitalists, 23 with complete data, completed the 1-day course. For the 23 hospitalists who completed skills-based assessments before and after the 1-day course, mean skills scores increased significantly (Table 2). Self-reported confidence ratings for ultrasound use, confidence in volume management, and quality of physical exam all increased significantly from preassessment to postassessment (Table 2).

Monthly Scanning Sessions and Portfolio Development

The skills retention from initial course to refresher course by portfolio completion and monthly scanning sessions is shown in Table 2. Multiple regression analysis showed that for every 10% increase in the percent of monthly sessions attended, the mean change in skills score was 3.7% (P = .017), and for every 10% increase in the percent of portfolio completed, the mean change in skills score was 2.5% (P = .04), showing that both monthly scanning session attendance and portfolio completion are significantly predictive of skills retention over time.

Final Assessments

Four providers met mastery at initial attempt. No providers to date have needed remediation. Many others are going through different stages of the process and are expected to attain mastery in a short period of time.

This is the first description of a successful longitudinal training program with assessments in POCUS for hospital medicine providers that shows an increase in skill retention with the use of a follow-up course and bedside scanning.

The CHAMP Ultrasound Program was developed to provide hospital medicine clinicians with a specialty focused in-house training pathway in POCUS and to assist in sustained skills acquisition by providing opportunities for regular feedback and practice. Practice with regular expert feedback is a critical aspect to develop and maintain skills in POCUS.^32,33 Arntfield³⁴ described the utility of remote supervision with feedback for ultrasound training in critical care, which demonstrated varying learning curves in the submission of portfolio images.^35,36 The CHAMP Ultrasound training program provided expert oversight, longitudinal supervision, and feedback for course participants. The educational method of mastery learning was employed by setting minimum standards and allowing learners to practice until they met that standard.^37-39

This unique program is made possible by the availability of expert-level faculty. Assessment scores improved with an initial 3-day course; however, they also decayed over time, most prominently with hospitalists that did not continue with POCUS scanning after their initial course. Ironically, those who performed more ultrasounds in the year prior to beginning the 3-day course had lower confidence ratings, likely explained by their awareness of their limitations and opportunities for improvement. The incorporation of refresher training to supplement the core 3-day course and portfolio development are key additions that differentiate this training program. These additions and the demonstration of successful training make this a durable pathway for other hospitalist programs. There are many workshops and short courses for medical students, residents, and practicing providers in POCUS.^40-43 However, without an opportunity for longitudinal supervision and feedback, there is a noted decrease in the skills for participants. The refresher training with its 2 components (1-day in-person course and monthly scanning sessions) provides evidence of the value of mentored training.

In the initial program development, refresher training was encouraged but optional. We intentionally tracked those that completed refresher training compared with those that did not. Based on the results showing significant skills retention among those attending some form of refresher training, the program is planned to change to make this a requirement. We recommend refresher training within 12 months of the initial introductory course. There were several hospitalists that were unable to accommodate taking a full-day refresher course and, therefore, monthly scanning sessions were provided as an alternative.

The main limitation of the study is that it was completed in a single hospital system with available training mentors in POCUS. This gave us the ability to perform longitudinal training but may make this less reproducible in other hospital systems. Another limitation is that our course participants did not complete the pre- and postknowledge assessments for the refresher training components of the program, though they did for the initial 3-day course. Our pre- and postassessments have not been externally shown to produce valid data, though they are based on the already validated CHEST ultrasound data.⁴⁴

Finally, our CHAMP Ultrasound Program required a significant time commitment by both faculty and learners. A relatively small percentage of hospitalists have completed the final assessments. The reasons are multifactorial, including program rigor, desire by certain hospitalists to know the basics but not pursue more expertise, and the challenges of developing a skillset that takes dedicated practice over time. We have aimed to address these barriers by providing additional hands-on scanning opportunities, giving timely feedback with portfolios, and obtaining more ultrasound machines. We expect more hospitalists to complete the final assessments in the coming year as evidenced by portfolio submissions to the shared online portal and many choosing to attend either the monthly scanning sessions and/or the 1-day course. We recognize that other institutions may need to adapt our program to suit their local environment.

A comprehensive longitudinal ultrasound training program including competency assessments significantly improved ultrasound acquisition skills with hospitalists. Those attending monthly scanning sessions and participating in the portfolio completion as well as a refresher course significantly retained and augmented their skills.

Acknowledgments

The authors would like to acknowledge Kelly Logue, Jason Robertson, MD, Jerome Siy, MD, Shauna Baer, and Jack Dressen for their support in the development and implementation of the POCUS program in hospital medicine.

Disclosure

The authors do not have any relevant financial disclosures to report.

With hospital reimbursement increasingly being linked to patient satisfaction,¹ about half of US hospitals have embraced arts programs as a means of humanizing clinical environments and improving the patient experience.^2,3 There is emerging evidence that integrating such programs into clinical settings is associated with less pain, stress, and anxiety^4-10 as well as improved mood,¹¹ greater levels of interaction,¹² and feeling less institutionalized.¹³ However, it has been observed that existing studies have been undertaken with variable methodological rigor,¹⁴ and few randomized controlled trials (RCTs) have linked specific design features or interventions directly to healthcare outcomes. We designed a RCT to test the hypotheses that (1) placing a painting by a local artist in the line of vision of hospitalized patients would improve psychological and clinical outcomes and patient satisfaction and (2) letting patients choose their own painting would offer even greater benefit in these areas.

Outcomes and Measures

The primary outcomes were psychological and included the following: anxiety, mood, depression, and sense of control and/or influence. These were measured using the validated State-Trait Anxiety Inventory (STAI)¹⁵ an emotional thermometer instrument (ETI)¹⁶, and a self-designed instrument measuring one’s sense of control and influence over the environment. Secondary outcomes were clinical, encompassing pain, quality of life (QOL), length of stay (LOS), and related to perceptions of the hospital environment. These were assessed using data extracted from the electronic medical record (EMR) as well as the Room Assessment (RA) survey, a validated instrument used in prior clinical studies to assess inpatient settings.¹⁷ The RA survey uses the Semantic Differential scale, a rating scale designed to measure emotional associations by using paired attributes.¹⁸ In our scale, we listed 17 paired and polar opposite attributes, with one descriptor reflecting a more positive impression than the other. Anxiety, emotional state, and control and/or influence were assessed at baseline and prior to discharge; emotional state was assessed every 1 to 2 days; and perceptions of the room and overall patient experience were measured once, prior to discharge, using the RA survey.

Data Analysis

A sample of 180 participants were chosen, with a 2:1 ratio of art group to no-art control group to provide at least 80% power to detect a difference in anxiety score of 4 units, for the comparisons of interest among the groups. The calculations assumed a 2-sided test with α = 0.05.

Comparisons were made between (1) those with paintings versus those without and (2) those with a choice of paintings versus those with no choice. For the primary psychological outcome, the average anxiety score at discharge was compared between groups of interest by using analysis of covariance, with adjustment for baseline score. Items measuring mood, depression, control, and influence that were collected more frequently were compared between groups by using repeated measures analysis of covariance, with adjustment for corresponding score at baseline. For clinical outcomes, median LOS was compared between groups by using the Wilcoxon rank sum test due to the skewed distribution of data, and QOL and pain were compared between groups by using repeated measures analysis of covariance. The model for patient-reported pain included covariates for pain medication received and level of pain tolerance. Outcomes measuring perceptions of hospital environment were collected at a single time point and compared between groups by using the 2-sample t-test. Results were reported in terms of means and 95% confidence intervals or medians and quartiles. Significance was defined by P < .05. All facets of this study were approved by the Pennsylvania State University College of Medicine Institutional Review Board.

We approached 518 patients to participate in the study, and 203 elected to enroll. Seventeen patients withdrew from the study because they had been discharged from the hospital or were unable to continue. Of the 186 participants who completed the study, 74 chose the painting displayed in their rooms, 69 had paintings randomly selected for them, and 43 had no paintings in their rooms, only white boards in their line of vision. The average age of participants in the trial was 56 years, 49% were male, and 89% were Caucasian. There were no significant differences between participants and decliners in terms of race (P = .13) and mean age (P = .08). However, they did differ by gender, with 49% of participants being male compared with 68% of decliners (P < .001). There were no significant differences among the 3 study groups with respect to these demographic characteristics. No harms were observed for any patients; however, several patients in the group whose artwork was randomly selected expressed distaste for the image and/or color scheme of their painting.

Psychological Outcomes: Anxiety (STAI), Mood and Depression (ETI), and Sense of Control and/or Influence (Self-Designed Instrument)

There were no differences in anxiety for the primary comparison of artwork versus no artwork or the secondary comparison of choice versus no choice. Likewise, there were no differences in mood, depression, or sense of control and/or influence across the 3 groups.

Clinical Outcomes: Self-Reported Pain, LOS, and QOL (from EMR)

There were no differences in self-reported pain, LOS, or QOL across the 3 groups. With regard to LOS, the median (quartile 1 [Q1], quartile 3 [Q3]) stay was 6 days for the choice group (4.0, 12.0), 6 days for the no-choice group (5.0, 9.5), and 9.5 days for the group with no artwork (5.0, 20.0; see supplementary Table).

Perceptions of Hospital Environment (RA Survey)

While having paintings in cancer inpatient rooms did not affect the psychological or clinical outcomes we assessed, patients who had paintings in their rooms had more positive impressions of the hospital environment. Given that healthcare administrators are under strong pressures to control costs while increasing care quality and patient satisfaction to maximize reimbursement, integrating local artwork into inpatient rooms may represent a simple and relatively inexpensive way (approximately $90 per room) to humanize clinical environments, systematically improve perceptions of the institution, and perhaps contribute to increased patient satisfaction scores. While more work must be done to establish a positive link between access to artwork and improved standardized patient satisfaction outcomes, our results suggest that there may be potential benefit in giving patients an opportunity to engage artwork as a therapeutic resource during the physical, emotional, and spiritual challenges that arise during inpatient treatment.

These findings also have implications for inpatients with illnesses other than cancer. Though we did not explicitly study noncancer patients, we know that nearly 40 million Americans are admitted annually to institutional care (ie, acute hospitalizations, rehabilitation hospitals, and skilled nursing facilities) and often find themselves in environments that can be stark and medicalized. We would anticipate that providing art in these patients’ rooms would likewise improve perceptions of the institutions where they receive their inpatient medical care.

This study had several limitations that could affect the generalizability of our findings. First, it was difficult to enroll patients, with greater than 50% of persons approached declining to participate. Second, nonparticipants were more likely to be male, and this clearly provides a biased sample. Third, we have incomplete data for some patients who were unavailable or changed rooms during the study. Fourth, while each patient room had standardized features (eg, windows, televisions, etc.), there were logistical challenges with placing paintings in the exact same location (ie, in the patient’s direct line of vision) in every hospital room because the shape, size, and idiosyncratic decorating of hospital rooms varied, so we were not able to fully control for all room décor features. Fifth, the study was conducted at a single site and only among patients with cancer; other populations could respond very differently. It is possible that other confounding factors (such as prior hospital experience, patient predilection for artwork, and usage of digital devices during hospitalization) could have affected outcomes, but these were not measured in this study.

In conclusion, as patient satisfaction continues to influence hospital reimbursement, identifying novel and effective approaches to improving patient perceptions can play a meaningful role in patient care. Future research should focus on different inpatient populations and venues; new strategies to effectively evaluate relevant clinical outcomes; comparisons with other nonpharmacological, arts-based interventions in inpatient settings (eg, music, creation of artwork, etc.); and assessment of aggregate scores on standardized patient satisfaction instruments (eg, Press Ganey, Hospital Consumer Assessment of Healthcare Providers and Systems). There may also be an additive benefit in providing “coaching” to healthcare providers on how to engage with patients regarding the artwork they have chosen. Such approaches might also examine the value of giving patients control over multiple opportunities to influence the aesthetics in their room versus a single opportunity during the course of their stay.

Acknowledgments

The authors would like to acknowledge the contributions of Lorna Davis, Lori Snyder, and Renee Stewart to this work.

Disclosure 

This work was supported by funding from the National Endowment for the Arts (grant 14-3800-7008). ClinicalTrials.gov Identifier for Penn State Milton S. Hershey Medical Center Protocol Record STUDY00000378: NCT02357160. The authors report no conflicts of interest.

With hospital reimbursement increasingly being linked to patient satisfaction,¹ about half of US hospitals have embraced arts programs as a means of humanizing clinical environments and improving the patient experience.^2,3 There is emerging evidence that integrating such programs into clinical settings is associated with less pain, stress, and anxiety^4-10 as well as improved mood,¹¹ greater levels of interaction,¹² and feeling less institutionalized.¹³ However, it has been observed that existing studies have been undertaken with variable methodological rigor,¹⁴ and few randomized controlled trials (RCTs) have linked specific design features or interventions directly to healthcare outcomes. We designed a RCT to test the hypotheses that (1) placing a painting by a local artist in the line of vision of hospitalized patients would improve psychological and clinical outcomes and patient satisfaction and (2) letting patients choose their own painting would offer even greater benefit in these areas.

Outcomes and Measures

The primary outcomes were psychological and included the following: anxiety, mood, depression, and sense of control and/or influence. These were measured using the validated State-Trait Anxiety Inventory (STAI)¹⁵ an emotional thermometer instrument (ETI)¹⁶, and a self-designed instrument measuring one’s sense of control and influence over the environment. Secondary outcomes were clinical, encompassing pain, quality of life (QOL), length of stay (LOS), and related to perceptions of the hospital environment. These were assessed using data extracted from the electronic medical record (EMR) as well as the Room Assessment (RA) survey, a validated instrument used in prior clinical studies to assess inpatient settings.¹⁷ The RA survey uses the Semantic Differential scale, a rating scale designed to measure emotional associations by using paired attributes.¹⁸ In our scale, we listed 17 paired and polar opposite attributes, with one descriptor reflecting a more positive impression than the other. Anxiety, emotional state, and control and/or influence were assessed at baseline and prior to discharge; emotional state was assessed every 1 to 2 days; and perceptions of the room and overall patient experience were measured once, prior to discharge, using the RA survey.

Data Analysis

A sample of 180 participants were chosen, with a 2:1 ratio of art group to no-art control group to provide at least 80% power to detect a difference in anxiety score of 4 units, for the comparisons of interest among the groups. The calculations assumed a 2-sided test with α = 0.05.

Comparisons were made between (1) those with paintings versus those without and (2) those with a choice of paintings versus those with no choice. For the primary psychological outcome, the average anxiety score at discharge was compared between groups of interest by using analysis of covariance, with adjustment for baseline score. Items measuring mood, depression, control, and influence that were collected more frequently were compared between groups by using repeated measures analysis of covariance, with adjustment for corresponding score at baseline. For clinical outcomes, median LOS was compared between groups by using the Wilcoxon rank sum test due to the skewed distribution of data, and QOL and pain were compared between groups by using repeated measures analysis of covariance. The model for patient-reported pain included covariates for pain medication received and level of pain tolerance. Outcomes measuring perceptions of hospital environment were collected at a single time point and compared between groups by using the 2-sample t-test. Results were reported in terms of means and 95% confidence intervals or medians and quartiles. Significance was defined by P < .05. All facets of this study were approved by the Pennsylvania State University College of Medicine Institutional Review Board.

We approached 518 patients to participate in the study, and 203 elected to enroll. Seventeen patients withdrew from the study because they had been discharged from the hospital or were unable to continue. Of the 186 participants who completed the study, 74 chose the painting displayed in their rooms, 69 had paintings randomly selected for them, and 43 had no paintings in their rooms, only white boards in their line of vision. The average age of participants in the trial was 56 years, 49% were male, and 89% were Caucasian. There were no significant differences between participants and decliners in terms of race (P = .13) and mean age (P = .08). However, they did differ by gender, with 49% of participants being male compared with 68% of decliners (P < .001). There were no significant differences among the 3 study groups with respect to these demographic characteristics. No harms were observed for any patients; however, several patients in the group whose artwork was randomly selected expressed distaste for the image and/or color scheme of their painting.

Psychological Outcomes: Anxiety (STAI), Mood and Depression (ETI), and Sense of Control and/or Influence (Self-Designed Instrument)

There were no differences in anxiety for the primary comparison of artwork versus no artwork or the secondary comparison of choice versus no choice. Likewise, there were no differences in mood, depression, or sense of control and/or influence across the 3 groups.

Clinical Outcomes: Self-Reported Pain, LOS, and QOL (from EMR)

There were no differences in self-reported pain, LOS, or QOL across the 3 groups. With regard to LOS, the median (quartile 1 [Q1], quartile 3 [Q3]) stay was 6 days for the choice group (4.0, 12.0), 6 days for the no-choice group (5.0, 9.5), and 9.5 days for the group with no artwork (5.0, 20.0; see supplementary Table).

Perceptions of Hospital Environment (RA Survey)

While having paintings in cancer inpatient rooms did not affect the psychological or clinical outcomes we assessed, patients who had paintings in their rooms had more positive impressions of the hospital environment. Given that healthcare administrators are under strong pressures to control costs while increasing care quality and patient satisfaction to maximize reimbursement, integrating local artwork into inpatient rooms may represent a simple and relatively inexpensive way (approximately $90 per room) to humanize clinical environments, systematically improve perceptions of the institution, and perhaps contribute to increased patient satisfaction scores. While more work must be done to establish a positive link between access to artwork and improved standardized patient satisfaction outcomes, our results suggest that there may be potential benefit in giving patients an opportunity to engage artwork as a therapeutic resource during the physical, emotional, and spiritual challenges that arise during inpatient treatment.

These findings also have implications for inpatients with illnesses other than cancer. Though we did not explicitly study noncancer patients, we know that nearly 40 million Americans are admitted annually to institutional care (ie, acute hospitalizations, rehabilitation hospitals, and skilled nursing facilities) and often find themselves in environments that can be stark and medicalized. We would anticipate that providing art in these patients’ rooms would likewise improve perceptions of the institutions where they receive their inpatient medical care.

This study had several limitations that could affect the generalizability of our findings. First, it was difficult to enroll patients, with greater than 50% of persons approached declining to participate. Second, nonparticipants were more likely to be male, and this clearly provides a biased sample. Third, we have incomplete data for some patients who were unavailable or changed rooms during the study. Fourth, while each patient room had standardized features (eg, windows, televisions, etc.), there were logistical challenges with placing paintings in the exact same location (ie, in the patient’s direct line of vision) in every hospital room because the shape, size, and idiosyncratic decorating of hospital rooms varied, so we were not able to fully control for all room décor features. Fifth, the study was conducted at a single site and only among patients with cancer; other populations could respond very differently. It is possible that other confounding factors (such as prior hospital experience, patient predilection for artwork, and usage of digital devices during hospitalization) could have affected outcomes, but these were not measured in this study.

In conclusion, as patient satisfaction continues to influence hospital reimbursement, identifying novel and effective approaches to improving patient perceptions can play a meaningful role in patient care. Future research should focus on different inpatient populations and venues; new strategies to effectively evaluate relevant clinical outcomes; comparisons with other nonpharmacological, arts-based interventions in inpatient settings (eg, music, creation of artwork, etc.); and assessment of aggregate scores on standardized patient satisfaction instruments (eg, Press Ganey, Hospital Consumer Assessment of Healthcare Providers and Systems). There may also be an additive benefit in providing “coaching” to healthcare providers on how to engage with patients regarding the artwork they have chosen. Such approaches might also examine the value of giving patients control over multiple opportunities to influence the aesthetics in their room versus a single opportunity during the course of their stay.

Acknowledgments

The authors would like to acknowledge the contributions of Lorna Davis, Lori Snyder, and Renee Stewart to this work.

Disclosure 

This work was supported by funding from the National Endowment for the Arts (grant 14-3800-7008). ClinicalTrials.gov Identifier for Penn State Milton S. Hershey Medical Center Protocol Record STUDY00000378: NCT02357160. The authors report no conflicts of interest.

With hospital reimbursement increasingly being linked to patient satisfaction,¹ about half of US hospitals have embraced arts programs as a means of humanizing clinical environments and improving the patient experience.^2,3 There is emerging evidence that integrating such programs into clinical settings is associated with less pain, stress, and anxiety^4-10 as well as improved mood,¹¹ greater levels of interaction,¹² and feeling less institutionalized.¹³ However, it has been observed that existing studies have been undertaken with variable methodological rigor,¹⁴ and few randomized controlled trials (RCTs) have linked specific design features or interventions directly to healthcare outcomes. We designed a RCT to test the hypotheses that (1) placing a painting by a local artist in the line of vision of hospitalized patients would improve psychological and clinical outcomes and patient satisfaction and (2) letting patients choose their own painting would offer even greater benefit in these areas.

Outcomes and Measures

The primary outcomes were psychological and included the following: anxiety, mood, depression, and sense of control and/or influence. These were measured using the validated State-Trait Anxiety Inventory (STAI)¹⁵ an emotional thermometer instrument (ETI)¹⁶, and a self-designed instrument measuring one’s sense of control and influence over the environment. Secondary outcomes were clinical, encompassing pain, quality of life (QOL), length of stay (LOS), and related to perceptions of the hospital environment. These were assessed using data extracted from the electronic medical record (EMR) as well as the Room Assessment (RA) survey, a validated instrument used in prior clinical studies to assess inpatient settings.¹⁷ The RA survey uses the Semantic Differential scale, a rating scale designed to measure emotional associations by using paired attributes.¹⁸ In our scale, we listed 17 paired and polar opposite attributes, with one descriptor reflecting a more positive impression than the other. Anxiety, emotional state, and control and/or influence were assessed at baseline and prior to discharge; emotional state was assessed every 1 to 2 days; and perceptions of the room and overall patient experience were measured once, prior to discharge, using the RA survey.

Data Analysis

A sample of 180 participants were chosen, with a 2:1 ratio of art group to no-art control group to provide at least 80% power to detect a difference in anxiety score of 4 units, for the comparisons of interest among the groups. The calculations assumed a 2-sided test with α = 0.05.

Comparisons were made between (1) those with paintings versus those without and (2) those with a choice of paintings versus those with no choice. For the primary psychological outcome, the average anxiety score at discharge was compared between groups of interest by using analysis of covariance, with adjustment for baseline score. Items measuring mood, depression, control, and influence that were collected more frequently were compared between groups by using repeated measures analysis of covariance, with adjustment for corresponding score at baseline. For clinical outcomes, median LOS was compared between groups by using the Wilcoxon rank sum test due to the skewed distribution of data, and QOL and pain were compared between groups by using repeated measures analysis of covariance. The model for patient-reported pain included covariates for pain medication received and level of pain tolerance. Outcomes measuring perceptions of hospital environment were collected at a single time point and compared between groups by using the 2-sample t-test. Results were reported in terms of means and 95% confidence intervals or medians and quartiles. Significance was defined by P < .05. All facets of this study were approved by the Pennsylvania State University College of Medicine Institutional Review Board.

We approached 518 patients to participate in the study, and 203 elected to enroll. Seventeen patients withdrew from the study because they had been discharged from the hospital or were unable to continue. Of the 186 participants who completed the study, 74 chose the painting displayed in their rooms, 69 had paintings randomly selected for them, and 43 had no paintings in their rooms, only white boards in their line of vision. The average age of participants in the trial was 56 years, 49% were male, and 89% were Caucasian. There were no significant differences between participants and decliners in terms of race (P = .13) and mean age (P = .08). However, they did differ by gender, with 49% of participants being male compared with 68% of decliners (P < .001). There were no significant differences among the 3 study groups with respect to these demographic characteristics. No harms were observed for any patients; however, several patients in the group whose artwork was randomly selected expressed distaste for the image and/or color scheme of their painting.

Psychological Outcomes: Anxiety (STAI), Mood and Depression (ETI), and Sense of Control and/or Influence (Self-Designed Instrument)

There were no differences in anxiety for the primary comparison of artwork versus no artwork or the secondary comparison of choice versus no choice. Likewise, there were no differences in mood, depression, or sense of control and/or influence across the 3 groups.

Clinical Outcomes: Self-Reported Pain, LOS, and QOL (from EMR)

There were no differences in self-reported pain, LOS, or QOL across the 3 groups. With regard to LOS, the median (quartile 1 [Q1], quartile 3 [Q3]) stay was 6 days for the choice group (4.0, 12.0), 6 days for the no-choice group (5.0, 9.5), and 9.5 days for the group with no artwork (5.0, 20.0; see supplementary Table).

Perceptions of Hospital Environment (RA Survey)

While having paintings in cancer inpatient rooms did not affect the psychological or clinical outcomes we assessed, patients who had paintings in their rooms had more positive impressions of the hospital environment. Given that healthcare administrators are under strong pressures to control costs while increasing care quality and patient satisfaction to maximize reimbursement, integrating local artwork into inpatient rooms may represent a simple and relatively inexpensive way (approximately $90 per room) to humanize clinical environments, systematically improve perceptions of the institution, and perhaps contribute to increased patient satisfaction scores. While more work must be done to establish a positive link between access to artwork and improved standardized patient satisfaction outcomes, our results suggest that there may be potential benefit in giving patients an opportunity to engage artwork as a therapeutic resource during the physical, emotional, and spiritual challenges that arise during inpatient treatment.

These findings also have implications for inpatients with illnesses other than cancer. Though we did not explicitly study noncancer patients, we know that nearly 40 million Americans are admitted annually to institutional care (ie, acute hospitalizations, rehabilitation hospitals, and skilled nursing facilities) and often find themselves in environments that can be stark and medicalized. We would anticipate that providing art in these patients’ rooms would likewise improve perceptions of the institutions where they receive their inpatient medical care.

This study had several limitations that could affect the generalizability of our findings. First, it was difficult to enroll patients, with greater than 50% of persons approached declining to participate. Second, nonparticipants were more likely to be male, and this clearly provides a biased sample. Third, we have incomplete data for some patients who were unavailable or changed rooms during the study. Fourth, while each patient room had standardized features (eg, windows, televisions, etc.), there were logistical challenges with placing paintings in the exact same location (ie, in the patient’s direct line of vision) in every hospital room because the shape, size, and idiosyncratic decorating of hospital rooms varied, so we were not able to fully control for all room décor features. Fifth, the study was conducted at a single site and only among patients with cancer; other populations could respond very differently. It is possible that other confounding factors (such as prior hospital experience, patient predilection for artwork, and usage of digital devices during hospitalization) could have affected outcomes, but these were not measured in this study.

In conclusion, as patient satisfaction continues to influence hospital reimbursement, identifying novel and effective approaches to improving patient perceptions can play a meaningful role in patient care. Future research should focus on different inpatient populations and venues; new strategies to effectively evaluate relevant clinical outcomes; comparisons with other nonpharmacological, arts-based interventions in inpatient settings (eg, music, creation of artwork, etc.); and assessment of aggregate scores on standardized patient satisfaction instruments (eg, Press Ganey, Hospital Consumer Assessment of Healthcare Providers and Systems). There may also be an additive benefit in providing “coaching” to healthcare providers on how to engage with patients regarding the artwork they have chosen. Such approaches might also examine the value of giving patients control over multiple opportunities to influence the aesthetics in their room versus a single opportunity during the course of their stay.

Acknowledgments

The authors would like to acknowledge the contributions of Lorna Davis, Lori Snyder, and Renee Stewart to this work.

Disclosure 

This work was supported by funding from the National Endowment for the Arts (grant 14-3800-7008). ClinicalTrials.gov Identifier for Penn State Milton S. Hershey Medical Center Protocol Record STUDY00000378: NCT02357160. The authors report no conflicts of interest.

Reimbursement for hospitals and physicians is increasingly tied to performance.¹ Bundled payments, for example, require hospitals to share risk for patient outcomes. Medicare bundled payments are becoming mandatory for some surgical and medical conditions, including joint replacement, acute myocardial infarction, and coronary artery bypass graft surgery.² Value-based payment is anticipated to become the norm as Medicare and private payers strive to control costs and improve outcomes. Although value-based reimbursement for hospitals targets hospital-level costs and outcomes, we know that variations at the level of individual providers explain a considerable proportion of variation in utilization and outcomes.³ However, physicians often lack awareness of their own practice patterns and relative costs, and successful participation in new payment models may require an investment by hospitals in the infrastructure needed to measure and provide feedback on performance to individual providers to affect their behavior.^4,5

Electronic health record (EHR)-based reports or “dashboards” have been proposed as one potential tool to provide individualized feedback on provider performance.⁶ Individual provider performance profiles (IPPs) offer the potential to provide peer comparisons that may adjust individual behavior by correcting misperceptions about norms.⁷ Behavioral economic theory suggests that individual performance data, if combined with information on peer behavior and normative goals, may be effective in changing behavior.⁸ Several studies have reported the effects of specific efforts to use IPPs, showing that such reports can improve care in certain clinical areas. For example, individual provider dashboards have been associated with better outcomes for hospitalized patients, such as increased compliance with recommendations for prophylaxis of venous thromboembolism, although evidence in other areas of practice is mixed.^9,10 A randomized controlled trial of peer comparison feedback reduced inappropriate antibiotic prescribing for upper respiratory infections by 11% among internists.¹¹

Despite the promise of individualized feedback to optimize behavior, however, little has been reported on trends in the use of IPPs on a population level. It is unknown whether their use is common or rare, or what hospital characteristics are associated with adoption. Such information would help guide future efforts to promote IPP use and understand its effect on practice. We used data from a nationally representative survey of US hospitals to examine the use of individual provider-level performance profiles.

We used data from the American Hospital Association (AHA) Annual Survey Information Technology (IT) Supplement, which asked respondents to indicate whether they have used electronic clinical data from the EHR or other electronic system in their hospital to create IPPs. The AHA survey is sent annually to all US operating hospitals. Survey results are supplemented by data from the AHA registration database, US Census Bureau, hospital accrediting bodies, and other organizations. The AHA IT supplement is also sent yearly to each hospital’s chief executive officer, who assigns it to the most knowledgeable person in the institution to complete.¹²

We linked data on IPP use to AHA Annual Survey responses on hospital characteristics for all general adult and pediatric hospitals. Multivariable logistic regression was used to model the odds of individual provider performance profile use as a function of hospital characteristics, including ownership (nonprofit, for profit, or government), geographic region, teaching versus nonteaching status, rural versus urban location, size, expenditures per bed, proportion of patient days covered by Medicaid, and risk-sharing models of reimbursement (participation in a health maintenance organization or bundled payments program). Variables were chosen a priori to account for important characteristics of US hospitals (eg, size, teaching status, and geographic location). These were combined with variables representing risk-sharing arrangements based on the hypothesis that hospitals whose payments are at greater risk would be more likely to invest in tracking provider performance. We eliminated any variable with an item nonresponse rate greater than 15%, which resulted in elimination of 2 variables representing hospital revenue from capitated payments and any risk-sharing arrangement, respectively. All other variables had item nonresponse rates of 0%, except for 4.7% item nonresponse for the bundled payments variable.

We also measured the trend in individual provider performance report use between 2013 and 2015 by estimating the linear probability between IPP use and year. A P value less than .05 was considered statistically significant.

Because past work has demonstrated nonresponse bias in the AHA Survey and IT Supplement, we performed additional analyses using nonresponsive weights based on hospital characteristics. Weighting methodology was based on prior work with the AHA and AHA IT surveys.^13,14 Weighting exploits the fact that a number of hospital characteristics are derived from sources outside the survey and thus are available for both respondents and nonrespondents. We created nonresponse weights based on a logistic regression model of survey response as a function of hospital characteristics (ownership, size, teaching status, systems membership, critical access hospital, and geographic region). Our findings were similar for weighted and nonweighted models and nonweighted estimates are presented throughout.

The University of Pennsylvania Institutional Review Board exempted this study from review. Analyses were performed using Stata statistical software, version 14.0 (StataCorp, College Station, TX).

The table shows the association between hospital characteristics and the odds of individual provider performance report use. Report use was associated with nonprofit status (odds ratio [OR], 2.77; 95% confidence interval [CI], 1.94-3.95; P < .01) compared to for-profit, large hospital size (OR, 2.37; 95% CI, 1.56-3.60; P < .01) compared to small size, highest (OR, 2.09; 95% CI, 1.55-2.82; P < .01) and second highest (OR, 1.43; 95% CI, 1.08-1.89; P = .01) quartiles of bed-adjusted expenditures compared to the bottom quartile, and West geographic region compared to Northeast (OR, 2.07; 95% CI, 1.45-2.95; P < .01). Individual provider performance use was also independently associated with participation in a health maintenance organization (OR, 1.50; 95% CI, 1.17-1.90; P < .01) or bundled payment program (OR, 1.61; 95% CI, 1.18-2.19; P < .01), controlling for other covariates. Adjustment for nonresponse bias did not change any coefficients by more than 10% (supplementary Table).

The Medicare Access and Children Health Insurance Program Reauthorization Act is accelerating the shift from quantity based toward value-based reimbursement. The proficient adoption of IT by healthcare providers has been cited as an important factor in adapting to new payment models.¹⁵ Physicians, and in particular hospitalists, who practice in an inpatient environment, may not directly access financial incentives aimed to adapt performance for value-based reimbursement. They may also have difficulty assessing their performance relative to peers and longitudinally over time. Individualized EHR-based provider-level performance reports offer 1 option for hospitals to measure performance and provide comparative feedback at the individual physician level. Our findings show that, in fact, a majority of US hospitals have made investments in the infrastructure necessary to create such profiles.

Nevertheless, a third of the hospitals surveyed have not adopted individualized provider performance profiles. If meeting efficiency and outcomes goals for value-based payments necessitates changes to individual provider behavior, those hospitals may be less well positioned to benefit from value-based payment models that incentivize hospitals for efficiency and outcomes. Furthermore, while we observe widespread adoption of individual performance profiles, it is unclear whether those were used to provide feedback to providers, and if so, how the feedback provided may influence its effect on behavior. Behavioral economics theory suggests, for example, that publicly reporting performance compared to peers provides stronger incentives for behavior change than “blinded” personalized reports.¹⁶

Our study has important limitations. We cannot exclude the possibility that unmeasured variables help explain individual provider performance adoption. These omitted variables may confound the association between hospital characteristics and individual provider performance adoption observed in this study. We were also unable to establish causality between bundled payments and individual provider performance profile use. For instance, hospitals may elect to make investments in IT infrastructure to enable individual provider performance profile adoption in anticipation of bundled payment reforms. Alternatively, the availability of IPPs may have led hospitals to enter bundled payments reimbursement arrangements. In addition, we are unable to describe how individual provider performance use affects physician practice or healthcare delivery more broadly. Finally, we are also unable to account for other sources of performance data. For example, some physician may receive data from their physician practice groups.

Our study suggests several avenues for future research. First, more work is needed to understand why certain types of hospitals are more likely to use IPPs. Our findings indicate that IPP use may be partly a function of hospital size and resources. However, other factors not measured here may play an important role as well, such as institutional culture. Institutions with a focus on informatics and strong IT leadership may be more likely to use their EHR to monitor performance. Second, further research should explore in greater depth how profiles are used. Future research should evaluate, for example, how hospitals are using behavioral economic principles, such as peer comparison, to motivate behavior change, and if such techniques have successfully influenced practice and patient outcomes. Ultimately, multicentered, randomized evaluations of IPP use may be necessary to understand their risks and evaluate their effect on patient outcomes. This work is necessary to inform policy and practice as hospitals transition from fee-for-service to value-based reimbursement.

In sum, we observed increasing adoption of individualized electronic provider performance profiles by US hospitals from 2013 to 2015. Hospitals that did not use IPPs were more likely to be small, for profit, and less likely to participate in bundled payment programs. Those hospitals may be less well positioned to track provider performance and implement incentives for provider behavior changes needed to meet targets for value-based reimbursement.

Disclosure

Dr. Rolnick is a consultant to Tuple Health, Inc. and was a part-time employee of Acumen, LLC outside the submitted work. Dr. Ryskina has nothing to disclose.

Reimbursement for hospitals and physicians is increasingly tied to performance.¹ Bundled payments, for example, require hospitals to share risk for patient outcomes. Medicare bundled payments are becoming mandatory for some surgical and medical conditions, including joint replacement, acute myocardial infarction, and coronary artery bypass graft surgery.² Value-based payment is anticipated to become the norm as Medicare and private payers strive to control costs and improve outcomes. Although value-based reimbursement for hospitals targets hospital-level costs and outcomes, we know that variations at the level of individual providers explain a considerable proportion of variation in utilization and outcomes.³ However, physicians often lack awareness of their own practice patterns and relative costs, and successful participation in new payment models may require an investment by hospitals in the infrastructure needed to measure and provide feedback on performance to individual providers to affect their behavior.^4,5

Electronic health record (EHR)-based reports or “dashboards” have been proposed as one potential tool to provide individualized feedback on provider performance.⁶ Individual provider performance profiles (IPPs) offer the potential to provide peer comparisons that may adjust individual behavior by correcting misperceptions about norms.⁷ Behavioral economic theory suggests that individual performance data, if combined with information on peer behavior and normative goals, may be effective in changing behavior.⁸ Several studies have reported the effects of specific efforts to use IPPs, showing that such reports can improve care in certain clinical areas. For example, individual provider dashboards have been associated with better outcomes for hospitalized patients, such as increased compliance with recommendations for prophylaxis of venous thromboembolism, although evidence in other areas of practice is mixed.^9,10 A randomized controlled trial of peer comparison feedback reduced inappropriate antibiotic prescribing for upper respiratory infections by 11% among internists.¹¹

Despite the promise of individualized feedback to optimize behavior, however, little has been reported on trends in the use of IPPs on a population level. It is unknown whether their use is common or rare, or what hospital characteristics are associated with adoption. Such information would help guide future efforts to promote IPP use and understand its effect on practice. We used data from a nationally representative survey of US hospitals to examine the use of individual provider-level performance profiles.

We used data from the American Hospital Association (AHA) Annual Survey Information Technology (IT) Supplement, which asked respondents to indicate whether they have used electronic clinical data from the EHR or other electronic system in their hospital to create IPPs. The AHA survey is sent annually to all US operating hospitals. Survey results are supplemented by data from the AHA registration database, US Census Bureau, hospital accrediting bodies, and other organizations. The AHA IT supplement is also sent yearly to each hospital’s chief executive officer, who assigns it to the most knowledgeable person in the institution to complete.¹²

We linked data on IPP use to AHA Annual Survey responses on hospital characteristics for all general adult and pediatric hospitals. Multivariable logistic regression was used to model the odds of individual provider performance profile use as a function of hospital characteristics, including ownership (nonprofit, for profit, or government), geographic region, teaching versus nonteaching status, rural versus urban location, size, expenditures per bed, proportion of patient days covered by Medicaid, and risk-sharing models of reimbursement (participation in a health maintenance organization or bundled payments program). Variables were chosen a priori to account for important characteristics of US hospitals (eg, size, teaching status, and geographic location). These were combined with variables representing risk-sharing arrangements based on the hypothesis that hospitals whose payments are at greater risk would be more likely to invest in tracking provider performance. We eliminated any variable with an item nonresponse rate greater than 15%, which resulted in elimination of 2 variables representing hospital revenue from capitated payments and any risk-sharing arrangement, respectively. All other variables had item nonresponse rates of 0%, except for 4.7% item nonresponse for the bundled payments variable.

We also measured the trend in individual provider performance report use between 2013 and 2015 by estimating the linear probability between IPP use and year. A P value less than .05 was considered statistically significant.

Because past work has demonstrated nonresponse bias in the AHA Survey and IT Supplement, we performed additional analyses using nonresponsive weights based on hospital characteristics. Weighting methodology was based on prior work with the AHA and AHA IT surveys.^13,14 Weighting exploits the fact that a number of hospital characteristics are derived from sources outside the survey and thus are available for both respondents and nonrespondents. We created nonresponse weights based on a logistic regression model of survey response as a function of hospital characteristics (ownership, size, teaching status, systems membership, critical access hospital, and geographic region). Our findings were similar for weighted and nonweighted models and nonweighted estimates are presented throughout.

The University of Pennsylvania Institutional Review Board exempted this study from review. Analyses were performed using Stata statistical software, version 14.0 (StataCorp, College Station, TX).

The table shows the association between hospital characteristics and the odds of individual provider performance report use. Report use was associated with nonprofit status (odds ratio [OR], 2.77; 95% confidence interval [CI], 1.94-3.95; P < .01) compared to for-profit, large hospital size (OR, 2.37; 95% CI, 1.56-3.60; P < .01) compared to small size, highest (OR, 2.09; 95% CI, 1.55-2.82; P < .01) and second highest (OR, 1.43; 95% CI, 1.08-1.89; P = .01) quartiles of bed-adjusted expenditures compared to the bottom quartile, and West geographic region compared to Northeast (OR, 2.07; 95% CI, 1.45-2.95; P < .01). Individual provider performance use was also independently associated with participation in a health maintenance organization (OR, 1.50; 95% CI, 1.17-1.90; P < .01) or bundled payment program (OR, 1.61; 95% CI, 1.18-2.19; P < .01), controlling for other covariates. Adjustment for nonresponse bias did not change any coefficients by more than 10% (supplementary Table).

The Medicare Access and Children Health Insurance Program Reauthorization Act is accelerating the shift from quantity based toward value-based reimbursement. The proficient adoption of IT by healthcare providers has been cited as an important factor in adapting to new payment models.¹⁵ Physicians, and in particular hospitalists, who practice in an inpatient environment, may not directly access financial incentives aimed to adapt performance for value-based reimbursement. They may also have difficulty assessing their performance relative to peers and longitudinally over time. Individualized EHR-based provider-level performance reports offer 1 option for hospitals to measure performance and provide comparative feedback at the individual physician level. Our findings show that, in fact, a majority of US hospitals have made investments in the infrastructure necessary to create such profiles.

Nevertheless, a third of the hospitals surveyed have not adopted individualized provider performance profiles. If meeting efficiency and outcomes goals for value-based payments necessitates changes to individual provider behavior, those hospitals may be less well positioned to benefit from value-based payment models that incentivize hospitals for efficiency and outcomes. Furthermore, while we observe widespread adoption of individual performance profiles, it is unclear whether those were used to provide feedback to providers, and if so, how the feedback provided may influence its effect on behavior. Behavioral economics theory suggests, for example, that publicly reporting performance compared to peers provides stronger incentives for behavior change than “blinded” personalized reports.¹⁶

Our study has important limitations. We cannot exclude the possibility that unmeasured variables help explain individual provider performance adoption. These omitted variables may confound the association between hospital characteristics and individual provider performance adoption observed in this study. We were also unable to establish causality between bundled payments and individual provider performance profile use. For instance, hospitals may elect to make investments in IT infrastructure to enable individual provider performance profile adoption in anticipation of bundled payment reforms. Alternatively, the availability of IPPs may have led hospitals to enter bundled payments reimbursement arrangements. In addition, we are unable to describe how individual provider performance use affects physician practice or healthcare delivery more broadly. Finally, we are also unable to account for other sources of performance data. For example, some physician may receive data from their physician practice groups.

Our study suggests several avenues for future research. First, more work is needed to understand why certain types of hospitals are more likely to use IPPs. Our findings indicate that IPP use may be partly a function of hospital size and resources. However, other factors not measured here may play an important role as well, such as institutional culture. Institutions with a focus on informatics and strong IT leadership may be more likely to use their EHR to monitor performance. Second, further research should explore in greater depth how profiles are used. Future research should evaluate, for example, how hospitals are using behavioral economic principles, such as peer comparison, to motivate behavior change, and if such techniques have successfully influenced practice and patient outcomes. Ultimately, multicentered, randomized evaluations of IPP use may be necessary to understand their risks and evaluate their effect on patient outcomes. This work is necessary to inform policy and practice as hospitals transition from fee-for-service to value-based reimbursement.

In sum, we observed increasing adoption of individualized electronic provider performance profiles by US hospitals from 2013 to 2015. Hospitals that did not use IPPs were more likely to be small, for profit, and less likely to participate in bundled payment programs. Those hospitals may be less well positioned to track provider performance and implement incentives for provider behavior changes needed to meet targets for value-based reimbursement.

Disclosure

Dr. Rolnick is a consultant to Tuple Health, Inc. and was a part-time employee of Acumen, LLC outside the submitted work. Dr. Ryskina has nothing to disclose.

Reimbursement for hospitals and physicians is increasingly tied to performance.¹ Bundled payments, for example, require hospitals to share risk for patient outcomes. Medicare bundled payments are becoming mandatory for some surgical and medical conditions, including joint replacement, acute myocardial infarction, and coronary artery bypass graft surgery.² Value-based payment is anticipated to become the norm as Medicare and private payers strive to control costs and improve outcomes. Although value-based reimbursement for hospitals targets hospital-level costs and outcomes, we know that variations at the level of individual providers explain a considerable proportion of variation in utilization and outcomes.³ However, physicians often lack awareness of their own practice patterns and relative costs, and successful participation in new payment models may require an investment by hospitals in the infrastructure needed to measure and provide feedback on performance to individual providers to affect their behavior.^4,5

Electronic health record (EHR)-based reports or “dashboards” have been proposed as one potential tool to provide individualized feedback on provider performance.⁶ Individual provider performance profiles (IPPs) offer the potential to provide peer comparisons that may adjust individual behavior by correcting misperceptions about norms.⁷ Behavioral economic theory suggests that individual performance data, if combined with information on peer behavior and normative goals, may be effective in changing behavior.⁸ Several studies have reported the effects of specific efforts to use IPPs, showing that such reports can improve care in certain clinical areas. For example, individual provider dashboards have been associated with better outcomes for hospitalized patients, such as increased compliance with recommendations for prophylaxis of venous thromboembolism, although evidence in other areas of practice is mixed.^9,10 A randomized controlled trial of peer comparison feedback reduced inappropriate antibiotic prescribing for upper respiratory infections by 11% among internists.¹¹

Despite the promise of individualized feedback to optimize behavior, however, little has been reported on trends in the use of IPPs on a population level. It is unknown whether their use is common or rare, or what hospital characteristics are associated with adoption. Such information would help guide future efforts to promote IPP use and understand its effect on practice. We used data from a nationally representative survey of US hospitals to examine the use of individual provider-level performance profiles.

We used data from the American Hospital Association (AHA) Annual Survey Information Technology (IT) Supplement, which asked respondents to indicate whether they have used electronic clinical data from the EHR or other electronic system in their hospital to create IPPs. The AHA survey is sent annually to all US operating hospitals. Survey results are supplemented by data from the AHA registration database, US Census Bureau, hospital accrediting bodies, and other organizations. The AHA IT supplement is also sent yearly to each hospital’s chief executive officer, who assigns it to the most knowledgeable person in the institution to complete.¹²

We linked data on IPP use to AHA Annual Survey responses on hospital characteristics for all general adult and pediatric hospitals. Multivariable logistic regression was used to model the odds of individual provider performance profile use as a function of hospital characteristics, including ownership (nonprofit, for profit, or government), geographic region, teaching versus nonteaching status, rural versus urban location, size, expenditures per bed, proportion of patient days covered by Medicaid, and risk-sharing models of reimbursement (participation in a health maintenance organization or bundled payments program). Variables were chosen a priori to account for important characteristics of US hospitals (eg, size, teaching status, and geographic location). These were combined with variables representing risk-sharing arrangements based on the hypothesis that hospitals whose payments are at greater risk would be more likely to invest in tracking provider performance. We eliminated any variable with an item nonresponse rate greater than 15%, which resulted in elimination of 2 variables representing hospital revenue from capitated payments and any risk-sharing arrangement, respectively. All other variables had item nonresponse rates of 0%, except for 4.7% item nonresponse for the bundled payments variable.

We also measured the trend in individual provider performance report use between 2013 and 2015 by estimating the linear probability between IPP use and year. A P value less than .05 was considered statistically significant.

Because past work has demonstrated nonresponse bias in the AHA Survey and IT Supplement, we performed additional analyses using nonresponsive weights based on hospital characteristics. Weighting methodology was based on prior work with the AHA and AHA IT surveys.^13,14 Weighting exploits the fact that a number of hospital characteristics are derived from sources outside the survey and thus are available for both respondents and nonrespondents. We created nonresponse weights based on a logistic regression model of survey response as a function of hospital characteristics (ownership, size, teaching status, systems membership, critical access hospital, and geographic region). Our findings were similar for weighted and nonweighted models and nonweighted estimates are presented throughout.

The University of Pennsylvania Institutional Review Board exempted this study from review. Analyses were performed using Stata statistical software, version 14.0 (StataCorp, College Station, TX).

The table shows the association between hospital characteristics and the odds of individual provider performance report use. Report use was associated with nonprofit status (odds ratio [OR], 2.77; 95% confidence interval [CI], 1.94-3.95; P < .01) compared to for-profit, large hospital size (OR, 2.37; 95% CI, 1.56-3.60; P < .01) compared to small size, highest (OR, 2.09; 95% CI, 1.55-2.82; P < .01) and second highest (OR, 1.43; 95% CI, 1.08-1.89; P = .01) quartiles of bed-adjusted expenditures compared to the bottom quartile, and West geographic region compared to Northeast (OR, 2.07; 95% CI, 1.45-2.95; P < .01). Individual provider performance use was also independently associated with participation in a health maintenance organization (OR, 1.50; 95% CI, 1.17-1.90; P < .01) or bundled payment program (OR, 1.61; 95% CI, 1.18-2.19; P < .01), controlling for other covariates. Adjustment for nonresponse bias did not change any coefficients by more than 10% (supplementary Table).

The Medicare Access and Children Health Insurance Program Reauthorization Act is accelerating the shift from quantity based toward value-based reimbursement. The proficient adoption of IT by healthcare providers has been cited as an important factor in adapting to new payment models.¹⁵ Physicians, and in particular hospitalists, who practice in an inpatient environment, may not directly access financial incentives aimed to adapt performance for value-based reimbursement. They may also have difficulty assessing their performance relative to peers and longitudinally over time. Individualized EHR-based provider-level performance reports offer 1 option for hospitals to measure performance and provide comparative feedback at the individual physician level. Our findings show that, in fact, a majority of US hospitals have made investments in the infrastructure necessary to create such profiles.

Nevertheless, a third of the hospitals surveyed have not adopted individualized provider performance profiles. If meeting efficiency and outcomes goals for value-based payments necessitates changes to individual provider behavior, those hospitals may be less well positioned to benefit from value-based payment models that incentivize hospitals for efficiency and outcomes. Furthermore, while we observe widespread adoption of individual performance profiles, it is unclear whether those were used to provide feedback to providers, and if so, how the feedback provided may influence its effect on behavior. Behavioral economics theory suggests, for example, that publicly reporting performance compared to peers provides stronger incentives for behavior change than “blinded” personalized reports.¹⁶

Our study has important limitations. We cannot exclude the possibility that unmeasured variables help explain individual provider performance adoption. These omitted variables may confound the association between hospital characteristics and individual provider performance adoption observed in this study. We were also unable to establish causality between bundled payments and individual provider performance profile use. For instance, hospitals may elect to make investments in IT infrastructure to enable individual provider performance profile adoption in anticipation of bundled payment reforms. Alternatively, the availability of IPPs may have led hospitals to enter bundled payments reimbursement arrangements. In addition, we are unable to describe how individual provider performance use affects physician practice or healthcare delivery more broadly. Finally, we are also unable to account for other sources of performance data. For example, some physician may receive data from their physician practice groups.

Our study suggests several avenues for future research. First, more work is needed to understand why certain types of hospitals are more likely to use IPPs. Our findings indicate that IPP use may be partly a function of hospital size and resources. However, other factors not measured here may play an important role as well, such as institutional culture. Institutions with a focus on informatics and strong IT leadership may be more likely to use their EHR to monitor performance. Second, further research should explore in greater depth how profiles are used. Future research should evaluate, for example, how hospitals are using behavioral economic principles, such as peer comparison, to motivate behavior change, and if such techniques have successfully influenced practice and patient outcomes. Ultimately, multicentered, randomized evaluations of IPP use may be necessary to understand their risks and evaluate their effect on patient outcomes. This work is necessary to inform policy and practice as hospitals transition from fee-for-service to value-based reimbursement.

In sum, we observed increasing adoption of individualized electronic provider performance profiles by US hospitals from 2013 to 2015. Hospitals that did not use IPPs were more likely to be small, for profit, and less likely to participate in bundled payment programs. Those hospitals may be less well positioned to track provider performance and implement incentives for provider behavior changes needed to meet targets for value-based reimbursement.

Disclosure

Dr. Rolnick is a consultant to Tuple Health, Inc. and was a part-time employee of Acumen, LLC outside the submitted work. Dr. Ryskina has nothing to disclose.

Pediatric expertise is critical in caring for children during the perioperative and postoperative periods.^1,2 Some postoperative care models involve pediatric hospitalists (PH) as collaborators for global care (comanagement),³ as consultants for specific issues, or not at all.

Single-site studies in specific pediatric surgical populations^4-7and medically fragile adults⁸ suggest improved outcomes for patients and systems by using hospitalist-surgeon collaboration. However, including PH in the care of surgical patients may also disrupt systems. No studies have broadly examined the clinical relationships between surgeons and PH.

The aims of this cross-sectional survey of US pediatric surgeons (PS) and pediatric orthopedic surgeons (OS) were to understand (1) the prevalence and characteristics of surgical care models in pediatrics, specifically those involving PH, and (2) surgeons’ perceptions of PH in caring for surgical patients.

The target US surgeon population was the estimated 850 active PS and at least 600 pediatric OS.⁹ Most US PS (n = 606) are affiliated with the American Academy of Pediatrics (AAP) Section on Surgery (SoSu), representing at least 200 programs. Nearly all pediatric OS belong to the Pediatric Orthopedic Society of North America (POSNA) (n = 706), representing 340 programs; a subset (n = 130) also belong to the AAP SoSu.

Survey Development and Distribution

Survey questions were developed to elicit surgeons’ descriptions of their program structure and their perceptions of PH involvement. For programs with PH involvement, program variables included primary assignment of clinical responsibilities by service line (surgery, hospitalist, shared) and use of a written service agreement, which defines each service’s roles and responsibilities.

The web-based survey, created by using Survey Monkey (San Mateo, CA), was pilot tested for usability and clarity among 8 surgeons and 1 PH. The survey had logic points around involvement of hospitalists and multiple hospital affiliations (supplemental Appendix A). The survey request with a web-based link was e-mailed 3 times to surgical and orthopedic distribution outlets, endorsed by organizational leadership. Respondents’ hospital ZIP codes were used as a proxy for program. If there was more than 1 complete survey response per ZIP code, 1 response with complete data was randomly selected to ensure a unique entry per program.

Classification of Care Models

Each surgical program was classified into 1 of the following 3 categories based on reported care of primary surgical patients: (1) comanagement, described as PH writing orders and/or functioning as the primary service; (2) consultation, described as PH providing clinical recommendations only; and (3) no PH involvement, described as “rarely” or “never” involving PH.

Clinical Responsibility Score

To estimate the degree of hospitalist involvement, we devised and calculated a composite score of service responsibilities for each program. This score involved the following 7 clinical domains: management of fluids or nutrition, pain, comorbidities, antibiotics, medication dosing, wound care, and discharge planning. Scores were summed for each domain: 0 for surgical team primary responsibility, 1 for shared surgical and hospitalist responsibility, and 2 for hospitalist primary responsibility. Composite scores could range from 0 to 14; lower scores represented a stronger tendency for surgeon management, and higher scores represented a stronger tendency toward PH management.

Data Analysis

For data analysis, simple exploratory tests with χ² analysis and Student t tests were performed by using Stata 14.2 (StataCorp LLC, College Station, TX) to compare differences by surgical specialty programs and individuals by role assignment and perceptions of PH involvement.

The NYU School of Medicine Institutional Review Board approved this study.

Respondents and Programs

Among the unique 185 PS programs and 212 OS programs represented, PH were often engaged in the care of primary surgical patients (Table).

Roles of PH in Collaborative Programs

Among programs that reported any hospitalist involvement (PS, n = 100; OS, n = 157), few (≤15%) programs involved hospitalists with all patients. Pediatric OS programs were significantly more likely than pediatric surgical programs to involve PH for healthy patients with any high-risk surgery (27% vs 9%; P = .001). Most PS (64%) and OS (83%) reported involving PH for all medically complex patients, regardless of surgery risk (P = .003).

In programs involving PH, few PS (11%) or OS programs (16%) reported by using a written service agreement.

Care of Surgical Patients in PH-involved programs

Composite clinical responsibility scores ranged from 0 to 8, with a median score of 2.3 (interquartile range [IQR] 0-3) for consultation programs and 5 (IQR 1-7) for comanagement programs. Composite scores were higher for OS (7.4; SD 3.4) versus PS (3.3; SD 3.4) programs (P < .001; 95% CI, 3.3-5.5; supplemental Appendix C).

Surgeons’ Perspectives on Hospitalist Involvement

Surgeons in programs without PH involvement viewed PH overall impact less positively than those with PH (27% vs 58%). Among all surgeons surveyed, few perceived positive (agree/strongly agree) PH impact on pain management (<15%) or decreasing LOS (<15%; supplemental Appendix D).

Most surgeons (n = 355) believed that PH financial support should come from separate billing (“patient fee”) (48%) or hospital budget (36%). Only 17% endorsed PH receiving part of the surgical global fee, with no significant difference by surgical specialty or current PH involvement status.

This study is the first comprehensive assessment of surgeons’ perspectives on the involvement and effectiveness of PH in the postoperative care of children undergoing inpatient general or orthopedic surgeries. The high prevalence (>70%) of PH involvement among responding surgical programs suggests that PH comanagement of hospitalized patients merits attention from providers, systems, educators, and payors.

Collaboration and Roles are Correlated with Surgical Specialty and Setting

Forty percent of inpatient pediatric surgeries occur outside of children’s hospitals.¹⁰ We found that PH involvement was higher at smaller and general hospitals where PH may provide pediatric expertise when insufficient pediatric resources, like pain teams, exist.⁷ Alternately, some quaternary centers have dedicated surgical hospitalists. The extensive involvement of PH in the bulk of certain clinical care domains, especially care coordination, in OS and in many PS programs (Figure) suggests that PH are well integrated into many programs and provide essential clinical care.

In many large freestanding children’s hospitals, though, surgical teams may have sufficient depth and breadth to manage most aspects of care. There may be an exception for care coordination of medically complex patients. Care coordination is a patient- and family-centered care best practice,¹¹ encompasses integrating and aligning medical care among clinical services, and is focused on shared decision making and communication. High-quality care coordination processes are of great value to patients and families, especially in medically complex children,¹¹ and are associated with improved transitions from hospital to home.¹² Well-planned transitions likely decrease these special populations’ postoperative readmission risk, complications, and prolonged length of stay.¹³ Reimbursement for these services could integrate these contributions needed for safe and patient-centered pediatric inpatient surgical care.

Perceptions of PH Impact

The variation in perception of PH by surgical specialty, with higher prevalence as well as higher regard for PH among OS, is intriguing. This disparity may reflect current training and clinical expectations of each surgical specialty, with larger emphasis on medical management for surgical compared with orthopedic curricula (www.acgme.org).

While PS and OS respondents perceived that PH involvement did not influence length of stay, pain management, and resource use, single-site studies suggest otherwise.^4,8,14 Objective data on the impact of PH involvement on patient and systems outcomes may help elucidate whether this is a perceived or actual lack of impact. Future metrics might include pain scores, patient centered care measures on communication and coordination, patient complaints and/or lawsuits, resource utilization and/or cost, readmission, and medical errors.

This study has several limitations. There is likely a (self) selection bias by surgeons with either strongly positive or negative views of PH involvement. Future studies may target a random sampling of programs rather than a cross-sectional survey of individual providers. Relatively few respondents represented community hospitals, possibly because these facilities are staffed by general OS and general surgeons¹⁰ who were not included in this sample.

Given the high prevalence of PH involvement in caring for surgical pediatric patients in varied settings, the field of pediatric hospital medicine should support increased PH training and standardized practice around perioperative management, particularly for medically complex patients with increased care coordination needs. Surgical comanagement, including interdisciplinary communication skills, deserves inclusion as a PH core competency and as an entrustable professional activity for pediatric hospital medicine and pediatric graduate medical education programs,¹⁵ especially orthopedic surgeries.

Further research on effective and evidence-based pediatric postoperative care and collaboration models will help PH and surgeons to most effectively and respectfully partner to improve care.

Acknowledgments

The authors thank the members of the AAP Section on Hospital Medicine Surgical Care Subcommittee, AAP SOHM leadership, and Ms. Alexandra Case.

Disclosure 

The authors have no conflicts of interest relevant to this manuscript to report. This study was supported in part by the Agency for Health Care Research and Quality (LM, R00HS022198).

Pediatric expertise is critical in caring for children during the perioperative and postoperative periods.^1,2 Some postoperative care models involve pediatric hospitalists (PH) as collaborators for global care (comanagement),³ as consultants for specific issues, or not at all.

Single-site studies in specific pediatric surgical populations^4-7and medically fragile adults⁸ suggest improved outcomes for patients and systems by using hospitalist-surgeon collaboration. However, including PH in the care of surgical patients may also disrupt systems. No studies have broadly examined the clinical relationships between surgeons and PH.

The aims of this cross-sectional survey of US pediatric surgeons (PS) and pediatric orthopedic surgeons (OS) were to understand (1) the prevalence and characteristics of surgical care models in pediatrics, specifically those involving PH, and (2) surgeons’ perceptions of PH in caring for surgical patients.

The target US surgeon population was the estimated 850 active PS and at least 600 pediatric OS.⁹ Most US PS (n = 606) are affiliated with the American Academy of Pediatrics (AAP) Section on Surgery (SoSu), representing at least 200 programs. Nearly all pediatric OS belong to the Pediatric Orthopedic Society of North America (POSNA) (n = 706), representing 340 programs; a subset (n = 130) also belong to the AAP SoSu.

Survey Development and Distribution

Survey questions were developed to elicit surgeons’ descriptions of their program structure and their perceptions of PH involvement. For programs with PH involvement, program variables included primary assignment of clinical responsibilities by service line (surgery, hospitalist, shared) and use of a written service agreement, which defines each service’s roles and responsibilities.

The web-based survey, created by using Survey Monkey (San Mateo, CA), was pilot tested for usability and clarity among 8 surgeons and 1 PH. The survey had logic points around involvement of hospitalists and multiple hospital affiliations (supplemental Appendix A). The survey request with a web-based link was e-mailed 3 times to surgical and orthopedic distribution outlets, endorsed by organizational leadership. Respondents’ hospital ZIP codes were used as a proxy for program. If there was more than 1 complete survey response per ZIP code, 1 response with complete data was randomly selected to ensure a unique entry per program.

Classification of Care Models

Each surgical program was classified into 1 of the following 3 categories based on reported care of primary surgical patients: (1) comanagement, described as PH writing orders and/or functioning as the primary service; (2) consultation, described as PH providing clinical recommendations only; and (3) no PH involvement, described as “rarely” or “never” involving PH.

Clinical Responsibility Score

To estimate the degree of hospitalist involvement, we devised and calculated a composite score of service responsibilities for each program. This score involved the following 7 clinical domains: management of fluids or nutrition, pain, comorbidities, antibiotics, medication dosing, wound care, and discharge planning. Scores were summed for each domain: 0 for surgical team primary responsibility, 1 for shared surgical and hospitalist responsibility, and 2 for hospitalist primary responsibility. Composite scores could range from 0 to 14; lower scores represented a stronger tendency for surgeon management, and higher scores represented a stronger tendency toward PH management.

Data Analysis

For data analysis, simple exploratory tests with χ² analysis and Student t tests were performed by using Stata 14.2 (StataCorp LLC, College Station, TX) to compare differences by surgical specialty programs and individuals by role assignment and perceptions of PH involvement.

The NYU School of Medicine Institutional Review Board approved this study.

Respondents and Programs

Among the unique 185 PS programs and 212 OS programs represented, PH were often engaged in the care of primary surgical patients (Table).

Roles of PH in Collaborative Programs

Among programs that reported any hospitalist involvement (PS, n = 100; OS, n = 157), few (≤15%) programs involved hospitalists with all patients. Pediatric OS programs were significantly more likely than pediatric surgical programs to involve PH for healthy patients with any high-risk surgery (27% vs 9%; P = .001). Most PS (64%) and OS (83%) reported involving PH for all medically complex patients, regardless of surgery risk (P = .003).

In programs involving PH, few PS (11%) or OS programs (16%) reported by using a written service agreement.

Care of Surgical Patients in PH-involved programs

Composite clinical responsibility scores ranged from 0 to 8, with a median score of 2.3 (interquartile range [IQR] 0-3) for consultation programs and 5 (IQR 1-7) for comanagement programs. Composite scores were higher for OS (7.4; SD 3.4) versus PS (3.3; SD 3.4) programs (P < .001; 95% CI, 3.3-5.5; supplemental Appendix C).

Surgeons’ Perspectives on Hospitalist Involvement

Surgeons in programs without PH involvement viewed PH overall impact less positively than those with PH (27% vs 58%). Among all surgeons surveyed, few perceived positive (agree/strongly agree) PH impact on pain management (<15%) or decreasing LOS (<15%; supplemental Appendix D).

Most surgeons (n = 355) believed that PH financial support should come from separate billing (“patient fee”) (48%) or hospital budget (36%). Only 17% endorsed PH receiving part of the surgical global fee, with no significant difference by surgical specialty or current PH involvement status.

This study is the first comprehensive assessment of surgeons’ perspectives on the involvement and effectiveness of PH in the postoperative care of children undergoing inpatient general or orthopedic surgeries. The high prevalence (>70%) of PH involvement among responding surgical programs suggests that PH comanagement of hospitalized patients merits attention from providers, systems, educators, and payors.

Collaboration and Roles are Correlated with Surgical Specialty and Setting

Forty percent of inpatient pediatric surgeries occur outside of children’s hospitals.¹⁰ We found that PH involvement was higher at smaller and general hospitals where PH may provide pediatric expertise when insufficient pediatric resources, like pain teams, exist.⁷ Alternately, some quaternary centers have dedicated surgical hospitalists. The extensive involvement of PH in the bulk of certain clinical care domains, especially care coordination, in OS and in many PS programs (Figure) suggests that PH are well integrated into many programs and provide essential clinical care.

In many large freestanding children’s hospitals, though, surgical teams may have sufficient depth and breadth to manage most aspects of care. There may be an exception for care coordination of medically complex patients. Care coordination is a patient- and family-centered care best practice,¹¹ encompasses integrating and aligning medical care among clinical services, and is focused on shared decision making and communication. High-quality care coordination processes are of great value to patients and families, especially in medically complex children,¹¹ and are associated with improved transitions from hospital to home.¹² Well-planned transitions likely decrease these special populations’ postoperative readmission risk, complications, and prolonged length of stay.¹³ Reimbursement for these services could integrate these contributions needed for safe and patient-centered pediatric inpatient surgical care.

Perceptions of PH Impact

The variation in perception of PH by surgical specialty, with higher prevalence as well as higher regard for PH among OS, is intriguing. This disparity may reflect current training and clinical expectations of each surgical specialty, with larger emphasis on medical management for surgical compared with orthopedic curricula (www.acgme.org).

While PS and OS respondents perceived that PH involvement did not influence length of stay, pain management, and resource use, single-site studies suggest otherwise.^4,8,14 Objective data on the impact of PH involvement on patient and systems outcomes may help elucidate whether this is a perceived or actual lack of impact. Future metrics might include pain scores, patient centered care measures on communication and coordination, patient complaints and/or lawsuits, resource utilization and/or cost, readmission, and medical errors.

This study has several limitations. There is likely a (self) selection bias by surgeons with either strongly positive or negative views of PH involvement. Future studies may target a random sampling of programs rather than a cross-sectional survey of individual providers. Relatively few respondents represented community hospitals, possibly because these facilities are staffed by general OS and general surgeons¹⁰ who were not included in this sample.

Given the high prevalence of PH involvement in caring for surgical pediatric patients in varied settings, the field of pediatric hospital medicine should support increased PH training and standardized practice around perioperative management, particularly for medically complex patients with increased care coordination needs. Surgical comanagement, including interdisciplinary communication skills, deserves inclusion as a PH core competency and as an entrustable professional activity for pediatric hospital medicine and pediatric graduate medical education programs,¹⁵ especially orthopedic surgeries.

Further research on effective and evidence-based pediatric postoperative care and collaboration models will help PH and surgeons to most effectively and respectfully partner to improve care.

Acknowledgments

The authors thank the members of the AAP Section on Hospital Medicine Surgical Care Subcommittee, AAP SOHM leadership, and Ms. Alexandra Case.

Disclosure 

The authors have no conflicts of interest relevant to this manuscript to report. This study was supported in part by the Agency for Health Care Research and Quality (LM, R00HS022198).

Pediatric expertise is critical in caring for children during the perioperative and postoperative periods.^1,2 Some postoperative care models involve pediatric hospitalists (PH) as collaborators for global care (comanagement),³ as consultants for specific issues, or not at all.

Single-site studies in specific pediatric surgical populations^4-7and medically fragile adults⁸ suggest improved outcomes for patients and systems by using hospitalist-surgeon collaboration. However, including PH in the care of surgical patients may also disrupt systems. No studies have broadly examined the clinical relationships between surgeons and PH.

The aims of this cross-sectional survey of US pediatric surgeons (PS) and pediatric orthopedic surgeons (OS) were to understand (1) the prevalence and characteristics of surgical care models in pediatrics, specifically those involving PH, and (2) surgeons’ perceptions of PH in caring for surgical patients.

The target US surgeon population was the estimated 850 active PS and at least 600 pediatric OS.⁹ Most US PS (n = 606) are affiliated with the American Academy of Pediatrics (AAP) Section on Surgery (SoSu), representing at least 200 programs. Nearly all pediatric OS belong to the Pediatric Orthopedic Society of North America (POSNA) (n = 706), representing 340 programs; a subset (n = 130) also belong to the AAP SoSu.

Survey Development and Distribution

Survey questions were developed to elicit surgeons’ descriptions of their program structure and their perceptions of PH involvement. For programs with PH involvement, program variables included primary assignment of clinical responsibilities by service line (surgery, hospitalist, shared) and use of a written service agreement, which defines each service’s roles and responsibilities.

The web-based survey, created by using Survey Monkey (San Mateo, CA), was pilot tested for usability and clarity among 8 surgeons and 1 PH. The survey had logic points around involvement of hospitalists and multiple hospital affiliations (supplemental Appendix A). The survey request with a web-based link was e-mailed 3 times to surgical and orthopedic distribution outlets, endorsed by organizational leadership. Respondents’ hospital ZIP codes were used as a proxy for program. If there was more than 1 complete survey response per ZIP code, 1 response with complete data was randomly selected to ensure a unique entry per program.

Classification of Care Models

Each surgical program was classified into 1 of the following 3 categories based on reported care of primary surgical patients: (1) comanagement, described as PH writing orders and/or functioning as the primary service; (2) consultation, described as PH providing clinical recommendations only; and (3) no PH involvement, described as “rarely” or “never” involving PH.

Clinical Responsibility Score

To estimate the degree of hospitalist involvement, we devised and calculated a composite score of service responsibilities for each program. This score involved the following 7 clinical domains: management of fluids or nutrition, pain, comorbidities, antibiotics, medication dosing, wound care, and discharge planning. Scores were summed for each domain: 0 for surgical team primary responsibility, 1 for shared surgical and hospitalist responsibility, and 2 for hospitalist primary responsibility. Composite scores could range from 0 to 14; lower scores represented a stronger tendency for surgeon management, and higher scores represented a stronger tendency toward PH management.

Data Analysis

For data analysis, simple exploratory tests with χ² analysis and Student t tests were performed by using Stata 14.2 (StataCorp LLC, College Station, TX) to compare differences by surgical specialty programs and individuals by role assignment and perceptions of PH involvement.

The NYU School of Medicine Institutional Review Board approved this study.

Respondents and Programs

Among the unique 185 PS programs and 212 OS programs represented, PH were often engaged in the care of primary surgical patients (Table).

Roles of PH in Collaborative Programs

Among programs that reported any hospitalist involvement (PS, n = 100; OS, n = 157), few (≤15%) programs involved hospitalists with all patients. Pediatric OS programs were significantly more likely than pediatric surgical programs to involve PH for healthy patients with any high-risk surgery (27% vs 9%; P = .001). Most PS (64%) and OS (83%) reported involving PH for all medically complex patients, regardless of surgery risk (P = .003).

In programs involving PH, few PS (11%) or OS programs (16%) reported by using a written service agreement.

Care of Surgical Patients in PH-involved programs

Composite clinical responsibility scores ranged from 0 to 8, with a median score of 2.3 (interquartile range [IQR] 0-3) for consultation programs and 5 (IQR 1-7) for comanagement programs. Composite scores were higher for OS (7.4; SD 3.4) versus PS (3.3; SD 3.4) programs (P < .001; 95% CI, 3.3-5.5; supplemental Appendix C).

Surgeons’ Perspectives on Hospitalist Involvement

Surgeons in programs without PH involvement viewed PH overall impact less positively than those with PH (27% vs 58%). Among all surgeons surveyed, few perceived positive (agree/strongly agree) PH impact on pain management (<15%) or decreasing LOS (<15%; supplemental Appendix D).

Most surgeons (n = 355) believed that PH financial support should come from separate billing (“patient fee”) (48%) or hospital budget (36%). Only 17% endorsed PH receiving part of the surgical global fee, with no significant difference by surgical specialty or current PH involvement status.

This study is the first comprehensive assessment of surgeons’ perspectives on the involvement and effectiveness of PH in the postoperative care of children undergoing inpatient general or orthopedic surgeries. The high prevalence (>70%) of PH involvement among responding surgical programs suggests that PH comanagement of hospitalized patients merits attention from providers, systems, educators, and payors.

Collaboration and Roles are Correlated with Surgical Specialty and Setting

Forty percent of inpatient pediatric surgeries occur outside of children’s hospitals.¹⁰ We found that PH involvement was higher at smaller and general hospitals where PH may provide pediatric expertise when insufficient pediatric resources, like pain teams, exist.⁷ Alternately, some quaternary centers have dedicated surgical hospitalists. The extensive involvement of PH in the bulk of certain clinical care domains, especially care coordination, in OS and in many PS programs (Figure) suggests that PH are well integrated into many programs and provide essential clinical care.

In many large freestanding children’s hospitals, though, surgical teams may have sufficient depth and breadth to manage most aspects of care. There may be an exception for care coordination of medically complex patients. Care coordination is a patient- and family-centered care best practice,¹¹ encompasses integrating and aligning medical care among clinical services, and is focused on shared decision making and communication. High-quality care coordination processes are of great value to patients and families, especially in medically complex children,¹¹ and are associated with improved transitions from hospital to home.¹² Well-planned transitions likely decrease these special populations’ postoperative readmission risk, complications, and prolonged length of stay.¹³ Reimbursement for these services could integrate these contributions needed for safe and patient-centered pediatric inpatient surgical care.

Perceptions of PH Impact

The variation in perception of PH by surgical specialty, with higher prevalence as well as higher regard for PH among OS, is intriguing. This disparity may reflect current training and clinical expectations of each surgical specialty, with larger emphasis on medical management for surgical compared with orthopedic curricula (www.acgme.org).

While PS and OS respondents perceived that PH involvement did not influence length of stay, pain management, and resource use, single-site studies suggest otherwise.^4,8,14 Objective data on the impact of PH involvement on patient and systems outcomes may help elucidate whether this is a perceived or actual lack of impact. Future metrics might include pain scores, patient centered care measures on communication and coordination, patient complaints and/or lawsuits, resource utilization and/or cost, readmission, and medical errors.

This study has several limitations. There is likely a (self) selection bias by surgeons with either strongly positive or negative views of PH involvement. Future studies may target a random sampling of programs rather than a cross-sectional survey of individual providers. Relatively few respondents represented community hospitals, possibly because these facilities are staffed by general OS and general surgeons¹⁰ who were not included in this sample.

Given the high prevalence of PH involvement in caring for surgical pediatric patients in varied settings, the field of pediatric hospital medicine should support increased PH training and standardized practice around perioperative management, particularly for medically complex patients with increased care coordination needs. Surgical comanagement, including interdisciplinary communication skills, deserves inclusion as a PH core competency and as an entrustable professional activity for pediatric hospital medicine and pediatric graduate medical education programs,¹⁵ especially orthopedic surgeries.

Further research on effective and evidence-based pediatric postoperative care and collaboration models will help PH and surgeons to most effectively and respectfully partner to improve care.

Acknowledgments

The authors thank the members of the AAP Section on Hospital Medicine Surgical Care Subcommittee, AAP SOHM leadership, and Ms. Alexandra Case.

Disclosure 

The authors have no conflicts of interest relevant to this manuscript to report. This study was supported in part by the Agency for Health Care Research and Quality (LM, R00HS022198).

Since the Accreditation Council for Graduate Medical Education (ACGME) posed new duty hour regulations in 2003 and again in 2011, there have been concerns that the substantial compression of resident workload may have resulted in a negative learning environment.^1-3 Residents are now expected to complete more work in a reduced amount of time and with less flexibility.⁴ In addition to time constraints, the actual work of a resident today may differ from that of a resident in the past, especially in the area of clinical documentation.⁵ Restricting resident work hours without examining the workload may result in increased work intensity and counter the potential benefits of working fewer hours.⁶ Measuring workload, as well as electronic health record (EHR)–related stress, may also help combat burnout in internal medicine.⁷ There are many components that influence resident workload, including patient census, patient comorbidities and acuity,EHR data and other available documentation, and ancillary tasks and procedures.⁷ We define resident workload intensity as the responsibilities required to provide patient care within a specified time. There is a paucity of objective data regarding the workload intensity of residents, which are essential to graduate medical education reform and optimization. Patient census, ancillary responsibilities, number of procedures, and conference length and frequency are some of the variables that can be adjusted by each residency program. As a first step to objective measurement of resident workload intensity, we endeavored to evaluate the less easily residency program–controlled workload components of patient comorbidity and EHR data the time of patient admission.

We conducted an observational, retrospective assessment of all admissions to the Louis Stokes Cleveland VA Medical Center (LSCVAMC) internal medicine service from January 1, 2000 to December 31, 2015. The inclusion criteria were admission to non-ICU internal medicine services and an admission note written by a resident physician. Otherwise, there were no exclusions. Data were accessed using VA Informatics and Computing Infrastructure. This study was approved by the LSCVAMC institutional review board.

A total of 67,346 admissions were included in the analysis. All parameters increased from 2000 to 2015. Mean CCI increased from 1.60 in 2003 (95% CI, 1.54–1.65) to 3.05 in 2011 (95% CI, 2.97–3.13) and to 3.77 in 2015 (95% CI, 3.67–3.87). Mean number of comorbidities increased from 6.21 in 2003 (95% CI, 6.05–6.36) to 16.09 in 2011 (95% CI, 15.84–16.34) and to 19.89 in 2015 (95% CI, 19.57–20.21). Mean number of notes increased from 193 in 2003 (95% CI, 186–199) to 841 in 2011 (95% CI, 815–868) and to 1289 in 2015 (95% CI, 1243–1335). Mean number of medications increased from 8.37 in 2003 (95% CI, 8.15–8.59) to 16.89 in 2011 (95% CI 16.60–17.20) and decreased to 16.49 in 2015 (95% CI, 16.18–16.80). Mean number of discharge summaries available at admission increased from 2.29 in 2003 (95% CI, 2.19–2.38) to 4.42 in 2011 (95% CI, 4.27–4.58) and to 5.48 in 2015 (95% CI, 5.27–5.69).

This retrospective, observational study shows that patient comorbidity and EHR data burden have increased over time, both of which impact resident workload at the time of admission. These findings, combined with the duty hour regulations, suggest that resident workload intensity at the time of admission may be increasing over time.

Patient comorbidity has likely increased due to a combination of factors. Elective admissions have decreased, and demographics have changed consistent with an aging population. Trainee admissions patterns also have changed over time, with less-acute admissions often admitted to nonacademic providers. Additionally, there are more stringent requirements for inpatient admissions, resulting in higher acuity and comorbidity.

As EHRs have matured and documentation requirements have expanded, the amount of electronic data has grown per patient, substantially increasing the time required to review a patient’s medical record.^5,10 In our evaluation, all EHR metrics increased between 2003 and 2011. The only metric that did not increase between 2011 and 2015 was the mean number of medications. The number of notes per patient has shown a dramatic increase. Even in an EHR that has reached maturity (in use more than 10 years), the number of notes per patient still increased by greater than 50% between 2011 and 2015. The VA EHR has been in use for more than 15 years, making it an ideal resource to study data trends. As many EHRs are in their infancy in comparison, these data may serve as a predictor of how other EHRs will mature. While all notes are not reviewed at every admission, this illustrates how increasing data burden combined with poor usability can be time consuming and promote inefficient patient care.¹¹ Moreover, many argue that poor EHR usability also affects cognitive workflow and clinical decision making, a task that is of utmost value to patient quality and safety as well as resident education.¹²Common program requirements for internal medicine as set forth by the ACGME state that residency programs should give adequate attention to scheduling, work intensity, and work compression to optimize resident well-being and prevent burnout.¹³ Resident workload intensity is multifaceted and encompasses many elements, including patient census and acuity, EHR data assessment, components of patient complexity such as comorbidity and psychosocial situation, and time.¹³ The work intensity increases with increase in the overall patient census, complexity, acuity, or data burden. Similarly, work intensity increases with time restrictions for patient care (in the form of duty hours). In addition, work intensity is affected by the time allotted for nonclinical responsibilities, such as morning reports and conferences, as these decrease the amount of time a resident can spend providing patient care.

Many programs have responded to the duty hour restrictions by decreasing patient caps.¹⁴ Our data suggest that decreasing patient census alone may not adequately mitigate the workload intensity of residents. There are other alternatives to prevent the increasing workload intensity that may have already been employed by some institutions. One such method is that programs can take into account patient complexity or acuity when allocating patients to teaching teams.¹⁴ Another method is to adjust the time spent on ancillary tasks such as obtaining outside hospital records, transporting patients, and scheduling follow-up appointments. Foregoing routine conferences such as morning reports or noon conferences would decrease work intensity, although obviously at the expense of resident education. Geographic rounding can encourage more efficient use of clinical time. One of the most difficult, but potentially impactful strategies would be to streamline EHRs to simplify and speed documentation, refocus regulations, and support and build based on the view of clinicians.¹⁵

The main limitations of this study include its retrospective design, single-center site, and focus on the internal medicine admissions to a VA hospital. Therefore, these findings may not be generalizable to other patient populations and training programs. Another potential limitation may be that changes in documentation practices have led to “upcoding” of patient comorbidy within the EHR. In addition, in this study, we looked only at the data available at the time of admission. To get a more complete picture of true workload intensity, understanding the day-to-day metrics of inpatient care would be crucial.

Our study demonstrates that components of resident workload (patient comorbidity and EHR data burden), specifically at the time of admission, have increased over time. These findings, combined with the duty hour regulations, suggest resident workload intensity at the time of admission has increased over time. This can have significant implications regarding graduate medical education, patient safety, and burnout. To optimize resident workload, innovation will be required in the areas of workflow, informatics, and curriculum. Future studies to assess the workload and intensity of the course of the entire patient hospitalization are needed.

The authors thank Paul E. Drawz, MD, MHS, MS (University of Minnesota) for contributions in designing and reviewing the study.

Ethical approval: The study was approved by the Institutional Review Board at the LSCVAMC. The contents do not represent the views of the U.S. Department of Veterans Affairs or the U.S. government. This material is the result of work supported with resources and the use of facilities of the LSCVAMC.

Disclosures

The authors declare that they have no conflicts of interest to disclose.

Since the Accreditation Council for Graduate Medical Education (ACGME) posed new duty hour regulations in 2003 and again in 2011, there have been concerns that the substantial compression of resident workload may have resulted in a negative learning environment.^1-3 Residents are now expected to complete more work in a reduced amount of time and with less flexibility.⁴ In addition to time constraints, the actual work of a resident today may differ from that of a resident in the past, especially in the area of clinical documentation.⁵ Restricting resident work hours without examining the workload may result in increased work intensity and counter the potential benefits of working fewer hours.⁶ Measuring workload, as well as electronic health record (EHR)–related stress, may also help combat burnout in internal medicine.⁷ There are many components that influence resident workload, including patient census, patient comorbidities and acuity,EHR data and other available documentation, and ancillary tasks and procedures.⁷ We define resident workload intensity as the responsibilities required to provide patient care within a specified time. There is a paucity of objective data regarding the workload intensity of residents, which are essential to graduate medical education reform and optimization. Patient census, ancillary responsibilities, number of procedures, and conference length and frequency are some of the variables that can be adjusted by each residency program. As a first step to objective measurement of resident workload intensity, we endeavored to evaluate the less easily residency program–controlled workload components of patient comorbidity and EHR data the time of patient admission.

We conducted an observational, retrospective assessment of all admissions to the Louis Stokes Cleveland VA Medical Center (LSCVAMC) internal medicine service from January 1, 2000 to December 31, 2015. The inclusion criteria were admission to non-ICU internal medicine services and an admission note written by a resident physician. Otherwise, there were no exclusions. Data were accessed using VA Informatics and Computing Infrastructure. This study was approved by the LSCVAMC institutional review board.

A total of 67,346 admissions were included in the analysis. All parameters increased from 2000 to 2015. Mean CCI increased from 1.60 in 2003 (95% CI, 1.54–1.65) to 3.05 in 2011 (95% CI, 2.97–3.13) and to 3.77 in 2015 (95% CI, 3.67–3.87). Mean number of comorbidities increased from 6.21 in 2003 (95% CI, 6.05–6.36) to 16.09 in 2011 (95% CI, 15.84–16.34) and to 19.89 in 2015 (95% CI, 19.57–20.21). Mean number of notes increased from 193 in 2003 (95% CI, 186–199) to 841 in 2011 (95% CI, 815–868) and to 1289 in 2015 (95% CI, 1243–1335). Mean number of medications increased from 8.37 in 2003 (95% CI, 8.15–8.59) to 16.89 in 2011 (95% CI 16.60–17.20) and decreased to 16.49 in 2015 (95% CI, 16.18–16.80). Mean number of discharge summaries available at admission increased from 2.29 in 2003 (95% CI, 2.19–2.38) to 4.42 in 2011 (95% CI, 4.27–4.58) and to 5.48 in 2015 (95% CI, 5.27–5.69).

This retrospective, observational study shows that patient comorbidity and EHR data burden have increased over time, both of which impact resident workload at the time of admission. These findings, combined with the duty hour regulations, suggest that resident workload intensity at the time of admission may be increasing over time.

Patient comorbidity has likely increased due to a combination of factors. Elective admissions have decreased, and demographics have changed consistent with an aging population. Trainee admissions patterns also have changed over time, with less-acute admissions often admitted to nonacademic providers. Additionally, there are more stringent requirements for inpatient admissions, resulting in higher acuity and comorbidity.

As EHRs have matured and documentation requirements have expanded, the amount of electronic data has grown per patient, substantially increasing the time required to review a patient’s medical record.^5,10 In our evaluation, all EHR metrics increased between 2003 and 2011. The only metric that did not increase between 2011 and 2015 was the mean number of medications. The number of notes per patient has shown a dramatic increase. Even in an EHR that has reached maturity (in use more than 10 years), the number of notes per patient still increased by greater than 50% between 2011 and 2015. The VA EHR has been in use for more than 15 years, making it an ideal resource to study data trends. As many EHRs are in their infancy in comparison, these data may serve as a predictor of how other EHRs will mature. While all notes are not reviewed at every admission, this illustrates how increasing data burden combined with poor usability can be time consuming and promote inefficient patient care.¹¹ Moreover, many argue that poor EHR usability also affects cognitive workflow and clinical decision making, a task that is of utmost value to patient quality and safety as well as resident education.¹²Common program requirements for internal medicine as set forth by the ACGME state that residency programs should give adequate attention to scheduling, work intensity, and work compression to optimize resident well-being and prevent burnout.¹³ Resident workload intensity is multifaceted and encompasses many elements, including patient census and acuity, EHR data assessment, components of patient complexity such as comorbidity and psychosocial situation, and time.¹³ The work intensity increases with increase in the overall patient census, complexity, acuity, or data burden. Similarly, work intensity increases with time restrictions for patient care (in the form of duty hours). In addition, work intensity is affected by the time allotted for nonclinical responsibilities, such as morning reports and conferences, as these decrease the amount of time a resident can spend providing patient care.

Many programs have responded to the duty hour restrictions by decreasing patient caps.¹⁴ Our data suggest that decreasing patient census alone may not adequately mitigate the workload intensity of residents. There are other alternatives to prevent the increasing workload intensity that may have already been employed by some institutions. One such method is that programs can take into account patient complexity or acuity when allocating patients to teaching teams.¹⁴ Another method is to adjust the time spent on ancillary tasks such as obtaining outside hospital records, transporting patients, and scheduling follow-up appointments. Foregoing routine conferences such as morning reports or noon conferences would decrease work intensity, although obviously at the expense of resident education. Geographic rounding can encourage more efficient use of clinical time. One of the most difficult, but potentially impactful strategies would be to streamline EHRs to simplify and speed documentation, refocus regulations, and support and build based on the view of clinicians.¹⁵

The main limitations of this study include its retrospective design, single-center site, and focus on the internal medicine admissions to a VA hospital. Therefore, these findings may not be generalizable to other patient populations and training programs. Another potential limitation may be that changes in documentation practices have led to “upcoding” of patient comorbidy within the EHR. In addition, in this study, we looked only at the data available at the time of admission. To get a more complete picture of true workload intensity, understanding the day-to-day metrics of inpatient care would be crucial.

Our study demonstrates that components of resident workload (patient comorbidity and EHR data burden), specifically at the time of admission, have increased over time. These findings, combined with the duty hour regulations, suggest resident workload intensity at the time of admission has increased over time. This can have significant implications regarding graduate medical education, patient safety, and burnout. To optimize resident workload, innovation will be required in the areas of workflow, informatics, and curriculum. Future studies to assess the workload and intensity of the course of the entire patient hospitalization are needed.

The authors thank Paul E. Drawz, MD, MHS, MS (University of Minnesota) for contributions in designing and reviewing the study.

Ethical approval: The study was approved by the Institutional Review Board at the LSCVAMC. The contents do not represent the views of the U.S. Department of Veterans Affairs or the U.S. government. This material is the result of work supported with resources and the use of facilities of the LSCVAMC.

Disclosures

The authors declare that they have no conflicts of interest to disclose.

Since the Accreditation Council for Graduate Medical Education (ACGME) posed new duty hour regulations in 2003 and again in 2011, there have been concerns that the substantial compression of resident workload may have resulted in a negative learning environment.^1-3 Residents are now expected to complete more work in a reduced amount of time and with less flexibility.⁴ In addition to time constraints, the actual work of a resident today may differ from that of a resident in the past, especially in the area of clinical documentation.⁵ Restricting resident work hours without examining the workload may result in increased work intensity and counter the potential benefits of working fewer hours.⁶ Measuring workload, as well as electronic health record (EHR)–related stress, may also help combat burnout in internal medicine.⁷ There are many components that influence resident workload, including patient census, patient comorbidities and acuity,EHR data and other available documentation, and ancillary tasks and procedures.⁷ We define resident workload intensity as the responsibilities required to provide patient care within a specified time. There is a paucity of objective data regarding the workload intensity of residents, which are essential to graduate medical education reform and optimization. Patient census, ancillary responsibilities, number of procedures, and conference length and frequency are some of the variables that can be adjusted by each residency program. As a first step to objective measurement of resident workload intensity, we endeavored to evaluate the less easily residency program–controlled workload components of patient comorbidity and EHR data the time of patient admission.

We conducted an observational, retrospective assessment of all admissions to the Louis Stokes Cleveland VA Medical Center (LSCVAMC) internal medicine service from January 1, 2000 to December 31, 2015. The inclusion criteria were admission to non-ICU internal medicine services and an admission note written by a resident physician. Otherwise, there were no exclusions. Data were accessed using VA Informatics and Computing Infrastructure. This study was approved by the LSCVAMC institutional review board.

A total of 67,346 admissions were included in the analysis. All parameters increased from 2000 to 2015. Mean CCI increased from 1.60 in 2003 (95% CI, 1.54–1.65) to 3.05 in 2011 (95% CI, 2.97–3.13) and to 3.77 in 2015 (95% CI, 3.67–3.87). Mean number of comorbidities increased from 6.21 in 2003 (95% CI, 6.05–6.36) to 16.09 in 2011 (95% CI, 15.84–16.34) and to 19.89 in 2015 (95% CI, 19.57–20.21). Mean number of notes increased from 193 in 2003 (95% CI, 186–199) to 841 in 2011 (95% CI, 815–868) and to 1289 in 2015 (95% CI, 1243–1335). Mean number of medications increased from 8.37 in 2003 (95% CI, 8.15–8.59) to 16.89 in 2011 (95% CI 16.60–17.20) and decreased to 16.49 in 2015 (95% CI, 16.18–16.80). Mean number of discharge summaries available at admission increased from 2.29 in 2003 (95% CI, 2.19–2.38) to 4.42 in 2011 (95% CI, 4.27–4.58) and to 5.48 in 2015 (95% CI, 5.27–5.69).

This retrospective, observational study shows that patient comorbidity and EHR data burden have increased over time, both of which impact resident workload at the time of admission. These findings, combined with the duty hour regulations, suggest that resident workload intensity at the time of admission may be increasing over time.

Patient comorbidity has likely increased due to a combination of factors. Elective admissions have decreased, and demographics have changed consistent with an aging population. Trainee admissions patterns also have changed over time, with less-acute admissions often admitted to nonacademic providers. Additionally, there are more stringent requirements for inpatient admissions, resulting in higher acuity and comorbidity.

As EHRs have matured and documentation requirements have expanded, the amount of electronic data has grown per patient, substantially increasing the time required to review a patient’s medical record.^5,10 In our evaluation, all EHR metrics increased between 2003 and 2011. The only metric that did not increase between 2011 and 2015 was the mean number of medications. The number of notes per patient has shown a dramatic increase. Even in an EHR that has reached maturity (in use more than 10 years), the number of notes per patient still increased by greater than 50% between 2011 and 2015. The VA EHR has been in use for more than 15 years, making it an ideal resource to study data trends. As many EHRs are in their infancy in comparison, these data may serve as a predictor of how other EHRs will mature. While all notes are not reviewed at every admission, this illustrates how increasing data burden combined with poor usability can be time consuming and promote inefficient patient care.¹¹ Moreover, many argue that poor EHR usability also affects cognitive workflow and clinical decision making, a task that is of utmost value to patient quality and safety as well as resident education.¹²Common program requirements for internal medicine as set forth by the ACGME state that residency programs should give adequate attention to scheduling, work intensity, and work compression to optimize resident well-being and prevent burnout.¹³ Resident workload intensity is multifaceted and encompasses many elements, including patient census and acuity, EHR data assessment, components of patient complexity such as comorbidity and psychosocial situation, and time.¹³ The work intensity increases with increase in the overall patient census, complexity, acuity, or data burden. Similarly, work intensity increases with time restrictions for patient care (in the form of duty hours). In addition, work intensity is affected by the time allotted for nonclinical responsibilities, such as morning reports and conferences, as these decrease the amount of time a resident can spend providing patient care.

Many programs have responded to the duty hour restrictions by decreasing patient caps.¹⁴ Our data suggest that decreasing patient census alone may not adequately mitigate the workload intensity of residents. There are other alternatives to prevent the increasing workload intensity that may have already been employed by some institutions. One such method is that programs can take into account patient complexity or acuity when allocating patients to teaching teams.¹⁴ Another method is to adjust the time spent on ancillary tasks such as obtaining outside hospital records, transporting patients, and scheduling follow-up appointments. Foregoing routine conferences such as morning reports or noon conferences would decrease work intensity, although obviously at the expense of resident education. Geographic rounding can encourage more efficient use of clinical time. One of the most difficult, but potentially impactful strategies would be to streamline EHRs to simplify and speed documentation, refocus regulations, and support and build based on the view of clinicians.¹⁵

The main limitations of this study include its retrospective design, single-center site, and focus on the internal medicine admissions to a VA hospital. Therefore, these findings may not be generalizable to other patient populations and training programs. Another potential limitation may be that changes in documentation practices have led to “upcoding” of patient comorbidy within the EHR. In addition, in this study, we looked only at the data available at the time of admission. To get a more complete picture of true workload intensity, understanding the day-to-day metrics of inpatient care would be crucial.

Our study demonstrates that components of resident workload (patient comorbidity and EHR data burden), specifically at the time of admission, have increased over time. These findings, combined with the duty hour regulations, suggest resident workload intensity at the time of admission has increased over time. This can have significant implications regarding graduate medical education, patient safety, and burnout. To optimize resident workload, innovation will be required in the areas of workflow, informatics, and curriculum. Future studies to assess the workload and intensity of the course of the entire patient hospitalization are needed.

The authors thank Paul E. Drawz, MD, MHS, MS (University of Minnesota) for contributions in designing and reviewing the study.

Ethical approval: The study was approved by the Institutional Review Board at the LSCVAMC. The contents do not represent the views of the U.S. Department of Veterans Affairs or the U.S. government. This material is the result of work supported with resources and the use of facilities of the LSCVAMC.

Disclosures

The authors declare that they have no conflicts of interest to disclose.

The “Things We Do for No Reason” series reviews practices which have become common parts of hospital care but which may provide little value to our patients. Practices reviewed in the TWDFR series do not represent “black and white” conclusions or clinical practice standards, but are meant as a starting place for research and active discussions among hospitalists and patients. We invite you to be part of that discussion. https://www.choosingwisely.org/

A 67-year-old man with acute myeloid leukemia who has recently completed a cycle of consolidation chemotherapy presents to the emergency room with fatigue and bruising. He is found to have pancytopenia due to chemotherapy. His absolute neutrophil count (ANC) is 380/mm³,and he has no symptoms or signs of infection. He is admitted for transfusion support and asks for a dinner tray. The provider reflexively prescribes a neutropenic diet.

Although aggressive chemotherapy regimens have significantly improved survival rates in patients with cancer, these intensive regimens put patients at risk for a number of complications, including severe, prolonged neutropenia. Patients with neutropenia, particularly those with ANC< 500/mm³, are at a significantly increased risk for infection. Common sites of infection include the blood stream, skin, lungs, urinary tract, and, particularly, the gastrointestinal tract.¹ Oncologists and dieticians first designed neutropenic diets, or low-bacteria diets, to limit the introduction of pathogenic microbes to the gastrointestinal system. Neutropenic diets typically limit the intake of fresh fruits, fresh vegetables, raw or undercooked meats and fish, and soft cheese made from unpasteurized milk. Despite the widespread recommendation of the neutropenic diet, no standardized guidelines exist, and the utilization of the diet and its contents vary widely among and within institutions.²

The neutropenic diet is a national phenomenon. A survey of 156 United States members of the Association of Community Cancer Centers revealed that 120 (78%) of the members had placed patients with neutropenia on restricted diets.² The triggers for prescription (neutropenia, or starting chemotherapy), ANC threshold for prescription, and duration of prescription (throughout chemotherapy or just when neutropenic) were not uniform. A majority of centers restricted fresh fruits, fresh vegetables, and raw eggs, while some locations also restricted tap water, herbs and spices, and alcoholic beverages.² Similarly, a study of practices in 29 countries across 6 continents found that 88% of centers have some version of a neutropenic diet guideline with significant heterogeneity in their prescription and content. For example, dried fruits were unrestricted in 23% of centers but were forbidden in 43%.³

The rationale behind the neutropenic diet is to limit the bacterial load delivered to the gut. Studies have shown that organisms such as Enterobacter, Pseudomonas, and Klebsiella have been isolated from food, particularly fruits and vegetables.^4,5 The ingestion of contaminated food products may serve as a source of pathogenic bacteria, which may cause potentially life-threatening infections. Mucositis, a common complication among cancer patients receiving therapy, predisposes patients to infection by disrupting the mucosal barrier, allowing bacteria to translocate from the gut to the bloodstream. Given that neutropenia and mucositis often occur simultaneously, these patients are at an increased risk of infections.⁶ Cooking destroys bacteria if present, rendering cooked foods safe. Thus, the avoidance of fresh fruits and vegetables and other foods considered to have high bacterial loads should theoretically decrease the risk of infections in these patients.

Researchers have investigated the ability of the neutropenic diet to reduce infection in adult and pediatric neutropenic patients. A study involving 153 patients receiving chemotherapy for acute myeloid leukemia or myelodysplastic syndrome randomized 78 patients to a diet that restricted raw fruits and vegetables and 75 patients to a diet that included those foods.⁸ The groups had similar rates of major infection (29% in the cooked group versus 35% in the raw group, P = .60) with no difference in mortality.⁷ In a randomized, multiinstitutional trial of 150 pediatric oncology patients, 77 patients received a neutropenic diet plus a diet based on the food safety guidelines approved by the Food and Drug Administration (FDA), while 73 children received a diet based on FDA-approved food safety guidelines.⁸ Infection rates between the groups were not significantly different (35% vs 33% respectively, P = .78).

Intensive conditioning regimens place hematopoietic stem-cell transplant (HSCT) recipients at an even greater risk of infectious complications than other patients and may increase gastrointestinal toxicity and prolong neutropenia. A study from a single academic US center included 726 HSCT recipients, 363 of whom received a neutropenic diet and 363 of whom received a general diet. Significantly fewer infections were observed in the general diet group than in the neutropenic diet group. Notably, this study was a retrospective trial, and approximately 75% of participants were autologous HSCT recipients, who traditionally have low risks of infection. A survey and analysis of nonpharmacologic anti-infective measures in 339 children with leukemia enrolled in the multicenter Acute Myeloid Leukemia Berlin-Frankfurt-Munster 2004 trial also did not show that the neutropenic diet has protective effects on infection rates.⁹ A metaanalysis that compiled data from the studies mentioned above found the hazard ratio for any infection (major or minor) and fever was actually higher in the neutropenic diet arm (relative risk 1.18, 95% confidence interval: 1.05-1.34, P = .007) relative to that in the unrestricted arm.¹⁰

The inefficacy of the neutropenic diet may be attributed to the fact that many of the organisms found on fresh fruits and vegetables are part of the normal flora in the gastrointestinal tract. A Dutch prospective randomized pilot study of 20 adult patients with acute myeloid leukemia undergoing chemotherapy compared the gut flora in patients on a low-bacteria diet versus that in patients on a normal hospital diet. Gut colonization by potential pathogens or infection rates were not significantly different between the 2 groups.¹¹

In addition to mucositis, the common gastrointestinal complications of chemotherapy include nausea, vomiting, diarrhea, food aversions, and changes in smells and taste, which limit oral intake.¹² Unnecessary dietary restrictions can place patients at further risk of inadequate intake and malnutrition.¹³ In the outpatient setting, compliance with the neutropenic diet is also problematic. In 1 study of 28 patients educated about the neutropenic diet, only 16 (57%) were compliant with the diet as revealed through telephone-based assessments at 6 and 12 weeks, and infection rates were not different between compliant versus noncompliant patients.¹⁴ Patients and family members reported that following the neutropenic diet requires considerably more effort than following a less restrictive diet.⁸ Maintaining nutrition in this patient population is already challenging, and the restriction of a wide variety of food items (fresh fruits, vegetables, dairy, certain meats, eggs) can cause malnutrition, low patient satisfaction, and poor quality of life.^13,14

Evidence shows no benefit of the neutropenic diet in any particular clinical scenario or patient population. However, despite the dearth of evidence to support neutropenic diets, the overall data regarding neutropenic diets are sparse. Randomized control trials to date have been limited by their small size with possible confounding by the type of malignancy and cancer therapy; use of prophylactic antibiotics, growth factors, and air-filtered rooms; variation in contents and adherence to the prescribed diet; and inpatient versus outpatient status. The study that included HSCT recipients was a retrospective trial, and a majority of patients were autologous HSCT recipients.¹⁵ Although no study has specifically investigated the neutropenic diet in preventing infection in patients with noncancer-related neutropenia, no reason exists to suspect that it is helpful. The FDA advises safe food-handling practices for other immunocompromised patients, such as transplant recipients and patients with human immunodeficiency virus/acquired immunodeficiency syndrome, and the same principles can likely be applied to patients with noncancer-related neutropenia.

WHAT WE SHOULD DO INSTEAD

Although the neutropenic diet has not been proven beneficial, the prevention of food-borne infection in this population remains important. FDA-published guidelines, which promote safe food handling to prevent food contamination in patients with cancer, should be followed in inpatient and outpatient settings.¹⁶ These guidelines allow for fresh fruits and vegetables as long as they have been adequately washed. Cleaning (eg, cleaning the lids of canned foods before opening, hand washing), separating raw meats from other foods, cooking to the right temperature (eg, cooking eggs until the yolk and white are firm), and chilling/refrigerating food appropriately are strongly emphasized. These guidelines are also recommended by the American Dietetic Association. Despite additional flexibility, patients following the FDA diet guidelines do not have increased risk of infection.⁸ At our hospitals, the neutropenic diet can no longer be ordered. Neutropenic patients are free to consume all items on the general hospital menu, including eggs, meat, soft cheeses, nuts, and washed raw fruits and vegetables. The National Comprehensive Cancer Network guidelines for the prevention and treatment of cancer-related infections do not specifically address diet.¹⁷ We call upon them to note the lack of benefit and potential harm of the neutropenic diet in the guidelines. Such an action may persuade more institutions to abandon this practice.

RECOMMENDATIONS

• Neutropenic diets, or low-bacteria diets, should not be prescribed to neutropenic patients.
• Properly handled and adequately washed fresh fruits and vegetables can safely be consumed by patients with neutropenia.
• Patients and hospitals should follow FDA-published safe food-handling guidelines to prevent food contamination.

A general diet can be safely ordered for our patient in the presented clinical scenario. Available data from individual studies and pooled data provide no evidence that neutropenic diets prevent infectious complications in patients with neutropenia.

Hospital kitchens must adhere to the food-handling guidelines issued by the FDA, and following these guidelines should provide adequate protection against food-borne infection, even in patients who are immunocompromised. Instead of restricting food groups, the FDA guidelines focus on safe food-handling practices. Less dietary restrictions provide patient’s additional opportunities for balanced nutrition and for food choices based on personal preferences or cultural practices.

Do you think this is a low-value practice? Is this truly a “Thing We Do for No Reason?” Share what you do in your practice and join in the conversation online by retweeting it on Twitter (#TWDFNR) and liking it on Facebook. We invite you to propose ideas for other “Things We Do for No Reason” topics by emailing TWDFNR@hospitalmedicine.org.Disclosures: There are no financial or other disclosures for any author.

Disclosures

There are no financial or other disclosures for any author.

The “Things We Do for No Reason” series reviews practices which have become common parts of hospital care but which may provide little value to our patients. Practices reviewed in the TWDFR series do not represent “black and white” conclusions or clinical practice standards, but are meant as a starting place for research and active discussions among hospitalists and patients. We invite you to be part of that discussion. https://www.choosingwisely.org/

A 67-year-old man with acute myeloid leukemia who has recently completed a cycle of consolidation chemotherapy presents to the emergency room with fatigue and bruising. He is found to have pancytopenia due to chemotherapy. His absolute neutrophil count (ANC) is 380/mm³,and he has no symptoms or signs of infection. He is admitted for transfusion support and asks for a dinner tray. The provider reflexively prescribes a neutropenic diet.

Although aggressive chemotherapy regimens have significantly improved survival rates in patients with cancer, these intensive regimens put patients at risk for a number of complications, including severe, prolonged neutropenia. Patients with neutropenia, particularly those with ANC< 500/mm³, are at a significantly increased risk for infection. Common sites of infection include the blood stream, skin, lungs, urinary tract, and, particularly, the gastrointestinal tract.¹ Oncologists and dieticians first designed neutropenic diets, or low-bacteria diets, to limit the introduction of pathogenic microbes to the gastrointestinal system. Neutropenic diets typically limit the intake of fresh fruits, fresh vegetables, raw or undercooked meats and fish, and soft cheese made from unpasteurized milk. Despite the widespread recommendation of the neutropenic diet, no standardized guidelines exist, and the utilization of the diet and its contents vary widely among and within institutions.²

The neutropenic diet is a national phenomenon. A survey of 156 United States members of the Association of Community Cancer Centers revealed that 120 (78%) of the members had placed patients with neutropenia on restricted diets.² The triggers for prescription (neutropenia, or starting chemotherapy), ANC threshold for prescription, and duration of prescription (throughout chemotherapy or just when neutropenic) were not uniform. A majority of centers restricted fresh fruits, fresh vegetables, and raw eggs, while some locations also restricted tap water, herbs and spices, and alcoholic beverages.² Similarly, a study of practices in 29 countries across 6 continents found that 88% of centers have some version of a neutropenic diet guideline with significant heterogeneity in their prescription and content. For example, dried fruits were unrestricted in 23% of centers but were forbidden in 43%.³

The rationale behind the neutropenic diet is to limit the bacterial load delivered to the gut. Studies have shown that organisms such as Enterobacter, Pseudomonas, and Klebsiella have been isolated from food, particularly fruits and vegetables.^4,5 The ingestion of contaminated food products may serve as a source of pathogenic bacteria, which may cause potentially life-threatening infections. Mucositis, a common complication among cancer patients receiving therapy, predisposes patients to infection by disrupting the mucosal barrier, allowing bacteria to translocate from the gut to the bloodstream. Given that neutropenia and mucositis often occur simultaneously, these patients are at an increased risk of infections.⁶ Cooking destroys bacteria if present, rendering cooked foods safe. Thus, the avoidance of fresh fruits and vegetables and other foods considered to have high bacterial loads should theoretically decrease the risk of infections in these patients.

Researchers have investigated the ability of the neutropenic diet to reduce infection in adult and pediatric neutropenic patients. A study involving 153 patients receiving chemotherapy for acute myeloid leukemia or myelodysplastic syndrome randomized 78 patients to a diet that restricted raw fruits and vegetables and 75 patients to a diet that included those foods.⁸ The groups had similar rates of major infection (29% in the cooked group versus 35% in the raw group, P = .60) with no difference in mortality.⁷ In a randomized, multiinstitutional trial of 150 pediatric oncology patients, 77 patients received a neutropenic diet plus a diet based on the food safety guidelines approved by the Food and Drug Administration (FDA), while 73 children received a diet based on FDA-approved food safety guidelines.⁸ Infection rates between the groups were not significantly different (35% vs 33% respectively, P = .78).

Intensive conditioning regimens place hematopoietic stem-cell transplant (HSCT) recipients at an even greater risk of infectious complications than other patients and may increase gastrointestinal toxicity and prolong neutropenia. A study from a single academic US center included 726 HSCT recipients, 363 of whom received a neutropenic diet and 363 of whom received a general diet. Significantly fewer infections were observed in the general diet group than in the neutropenic diet group. Notably, this study was a retrospective trial, and approximately 75% of participants were autologous HSCT recipients, who traditionally have low risks of infection. A survey and analysis of nonpharmacologic anti-infective measures in 339 children with leukemia enrolled in the multicenter Acute Myeloid Leukemia Berlin-Frankfurt-Munster 2004 trial also did not show that the neutropenic diet has protective effects on infection rates.⁹ A metaanalysis that compiled data from the studies mentioned above found the hazard ratio for any infection (major or minor) and fever was actually higher in the neutropenic diet arm (relative risk 1.18, 95% confidence interval: 1.05-1.34, P = .007) relative to that in the unrestricted arm.¹⁰

The inefficacy of the neutropenic diet may be attributed to the fact that many of the organisms found on fresh fruits and vegetables are part of the normal flora in the gastrointestinal tract. A Dutch prospective randomized pilot study of 20 adult patients with acute myeloid leukemia undergoing chemotherapy compared the gut flora in patients on a low-bacteria diet versus that in patients on a normal hospital diet. Gut colonization by potential pathogens or infection rates were not significantly different between the 2 groups.¹¹

In addition to mucositis, the common gastrointestinal complications of chemotherapy include nausea, vomiting, diarrhea, food aversions, and changes in smells and taste, which limit oral intake.¹² Unnecessary dietary restrictions can place patients at further risk of inadequate intake and malnutrition.¹³ In the outpatient setting, compliance with the neutropenic diet is also problematic. In 1 study of 28 patients educated about the neutropenic diet, only 16 (57%) were compliant with the diet as revealed through telephone-based assessments at 6 and 12 weeks, and infection rates were not different between compliant versus noncompliant patients.¹⁴ Patients and family members reported that following the neutropenic diet requires considerably more effort than following a less restrictive diet.⁸ Maintaining nutrition in this patient population is already challenging, and the restriction of a wide variety of food items (fresh fruits, vegetables, dairy, certain meats, eggs) can cause malnutrition, low patient satisfaction, and poor quality of life.^13,14

Evidence shows no benefit of the neutropenic diet in any particular clinical scenario or patient population. However, despite the dearth of evidence to support neutropenic diets, the overall data regarding neutropenic diets are sparse. Randomized control trials to date have been limited by their small size with possible confounding by the type of malignancy and cancer therapy; use of prophylactic antibiotics, growth factors, and air-filtered rooms; variation in contents and adherence to the prescribed diet; and inpatient versus outpatient status. The study that included HSCT recipients was a retrospective trial, and a majority of patients were autologous HSCT recipients.¹⁵ Although no study has specifically investigated the neutropenic diet in preventing infection in patients with noncancer-related neutropenia, no reason exists to suspect that it is helpful. The FDA advises safe food-handling practices for other immunocompromised patients, such as transplant recipients and patients with human immunodeficiency virus/acquired immunodeficiency syndrome, and the same principles can likely be applied to patients with noncancer-related neutropenia.

WHAT WE SHOULD DO INSTEAD

Although the neutropenic diet has not been proven beneficial, the prevention of food-borne infection in this population remains important. FDA-published guidelines, which promote safe food handling to prevent food contamination in patients with cancer, should be followed in inpatient and outpatient settings.¹⁶ These guidelines allow for fresh fruits and vegetables as long as they have been adequately washed. Cleaning (eg, cleaning the lids of canned foods before opening, hand washing), separating raw meats from other foods, cooking to the right temperature (eg, cooking eggs until the yolk and white are firm), and chilling/refrigerating food appropriately are strongly emphasized. These guidelines are also recommended by the American Dietetic Association. Despite additional flexibility, patients following the FDA diet guidelines do not have increased risk of infection.⁸ At our hospitals, the neutropenic diet can no longer be ordered. Neutropenic patients are free to consume all items on the general hospital menu, including eggs, meat, soft cheeses, nuts, and washed raw fruits and vegetables. The National Comprehensive Cancer Network guidelines for the prevention and treatment of cancer-related infections do not specifically address diet.¹⁷ We call upon them to note the lack of benefit and potential harm of the neutropenic diet in the guidelines. Such an action may persuade more institutions to abandon this practice.

RECOMMENDATIONS

• Neutropenic diets, or low-bacteria diets, should not be prescribed to neutropenic patients.
• Properly handled and adequately washed fresh fruits and vegetables can safely be consumed by patients with neutropenia.
• Patients and hospitals should follow FDA-published safe food-handling guidelines to prevent food contamination.

A general diet can be safely ordered for our patient in the presented clinical scenario. Available data from individual studies and pooled data provide no evidence that neutropenic diets prevent infectious complications in patients with neutropenia.

Hospital kitchens must adhere to the food-handling guidelines issued by the FDA, and following these guidelines should provide adequate protection against food-borne infection, even in patients who are immunocompromised. Instead of restricting food groups, the FDA guidelines focus on safe food-handling practices. Less dietary restrictions provide patient’s additional opportunities for balanced nutrition and for food choices based on personal preferences or cultural practices.

Do you think this is a low-value practice? Is this truly a “Thing We Do for No Reason?” Share what you do in your practice and join in the conversation online by retweeting it on Twitter (#TWDFNR) and liking it on Facebook. We invite you to propose ideas for other “Things We Do for No Reason” topics by emailing TWDFNR@hospitalmedicine.org.Disclosures: There are no financial or other disclosures for any author.

Disclosures

There are no financial or other disclosures for any author.

The “Things We Do for No Reason” series reviews practices which have become common parts of hospital care but which may provide little value to our patients. Practices reviewed in the TWDFR series do not represent “black and white” conclusions or clinical practice standards, but are meant as a starting place for research and active discussions among hospitalists and patients. We invite you to be part of that discussion. https://www.choosingwisely.org/

A 67-year-old man with acute myeloid leukemia who has recently completed a cycle of consolidation chemotherapy presents to the emergency room with fatigue and bruising. He is found to have pancytopenia due to chemotherapy. His absolute neutrophil count (ANC) is 380/mm³,and he has no symptoms or signs of infection. He is admitted for transfusion support and asks for a dinner tray. The provider reflexively prescribes a neutropenic diet.

Although aggressive chemotherapy regimens have significantly improved survival rates in patients with cancer, these intensive regimens put patients at risk for a number of complications, including severe, prolonged neutropenia. Patients with neutropenia, particularly those with ANC< 500/mm³, are at a significantly increased risk for infection. Common sites of infection include the blood stream, skin, lungs, urinary tract, and, particularly, the gastrointestinal tract.¹ Oncologists and dieticians first designed neutropenic diets, or low-bacteria diets, to limit the introduction of pathogenic microbes to the gastrointestinal system. Neutropenic diets typically limit the intake of fresh fruits, fresh vegetables, raw or undercooked meats and fish, and soft cheese made from unpasteurized milk. Despite the widespread recommendation of the neutropenic diet, no standardized guidelines exist, and the utilization of the diet and its contents vary widely among and within institutions.²

The neutropenic diet is a national phenomenon. A survey of 156 United States members of the Association of Community Cancer Centers revealed that 120 (78%) of the members had placed patients with neutropenia on restricted diets.² The triggers for prescription (neutropenia, or starting chemotherapy), ANC threshold for prescription, and duration of prescription (throughout chemotherapy or just when neutropenic) were not uniform. A majority of centers restricted fresh fruits, fresh vegetables, and raw eggs, while some locations also restricted tap water, herbs and spices, and alcoholic beverages.² Similarly, a study of practices in 29 countries across 6 continents found that 88% of centers have some version of a neutropenic diet guideline with significant heterogeneity in their prescription and content. For example, dried fruits were unrestricted in 23% of centers but were forbidden in 43%.³

The rationale behind the neutropenic diet is to limit the bacterial load delivered to the gut. Studies have shown that organisms such as Enterobacter, Pseudomonas, and Klebsiella have been isolated from food, particularly fruits and vegetables.^4,5 The ingestion of contaminated food products may serve as a source of pathogenic bacteria, which may cause potentially life-threatening infections. Mucositis, a common complication among cancer patients receiving therapy, predisposes patients to infection by disrupting the mucosal barrier, allowing bacteria to translocate from the gut to the bloodstream. Given that neutropenia and mucositis often occur simultaneously, these patients are at an increased risk of infections.⁶ Cooking destroys bacteria if present, rendering cooked foods safe. Thus, the avoidance of fresh fruits and vegetables and other foods considered to have high bacterial loads should theoretically decrease the risk of infections in these patients.

Researchers have investigated the ability of the neutropenic diet to reduce infection in adult and pediatric neutropenic patients. A study involving 153 patients receiving chemotherapy for acute myeloid leukemia or myelodysplastic syndrome randomized 78 patients to a diet that restricted raw fruits and vegetables and 75 patients to a diet that included those foods.⁸ The groups had similar rates of major infection (29% in the cooked group versus 35% in the raw group, P = .60) with no difference in mortality.⁷ In a randomized, multiinstitutional trial of 150 pediatric oncology patients, 77 patients received a neutropenic diet plus a diet based on the food safety guidelines approved by the Food and Drug Administration (FDA), while 73 children received a diet based on FDA-approved food safety guidelines.⁸ Infection rates between the groups were not significantly different (35% vs 33% respectively, P = .78).

Intensive conditioning regimens place hematopoietic stem-cell transplant (HSCT) recipients at an even greater risk of infectious complications than other patients and may increase gastrointestinal toxicity and prolong neutropenia. A study from a single academic US center included 726 HSCT recipients, 363 of whom received a neutropenic diet and 363 of whom received a general diet. Significantly fewer infections were observed in the general diet group than in the neutropenic diet group. Notably, this study was a retrospective trial, and approximately 75% of participants were autologous HSCT recipients, who traditionally have low risks of infection. A survey and analysis of nonpharmacologic anti-infective measures in 339 children with leukemia enrolled in the multicenter Acute Myeloid Leukemia Berlin-Frankfurt-Munster 2004 trial also did not show that the neutropenic diet has protective effects on infection rates.⁹ A metaanalysis that compiled data from the studies mentioned above found the hazard ratio for any infection (major or minor) and fever was actually higher in the neutropenic diet arm (relative risk 1.18, 95% confidence interval: 1.05-1.34, P = .007) relative to that in the unrestricted arm.¹⁰

The inefficacy of the neutropenic diet may be attributed to the fact that many of the organisms found on fresh fruits and vegetables are part of the normal flora in the gastrointestinal tract. A Dutch prospective randomized pilot study of 20 adult patients with acute myeloid leukemia undergoing chemotherapy compared the gut flora in patients on a low-bacteria diet versus that in patients on a normal hospital diet. Gut colonization by potential pathogens or infection rates were not significantly different between the 2 groups.¹¹

In addition to mucositis, the common gastrointestinal complications of chemotherapy include nausea, vomiting, diarrhea, food aversions, and changes in smells and taste, which limit oral intake.¹² Unnecessary dietary restrictions can place patients at further risk of inadequate intake and malnutrition.¹³ In the outpatient setting, compliance with the neutropenic diet is also problematic. In 1 study of 28 patients educated about the neutropenic diet, only 16 (57%) were compliant with the diet as revealed through telephone-based assessments at 6 and 12 weeks, and infection rates were not different between compliant versus noncompliant patients.¹⁴ Patients and family members reported that following the neutropenic diet requires considerably more effort than following a less restrictive diet.⁸ Maintaining nutrition in this patient population is already challenging, and the restriction of a wide variety of food items (fresh fruits, vegetables, dairy, certain meats, eggs) can cause malnutrition, low patient satisfaction, and poor quality of life.^13,14

Evidence shows no benefit of the neutropenic diet in any particular clinical scenario or patient population. However, despite the dearth of evidence to support neutropenic diets, the overall data regarding neutropenic diets are sparse. Randomized control trials to date have been limited by their small size with possible confounding by the type of malignancy and cancer therapy; use of prophylactic antibiotics, growth factors, and air-filtered rooms; variation in contents and adherence to the prescribed diet; and inpatient versus outpatient status. The study that included HSCT recipients was a retrospective trial, and a majority of patients were autologous HSCT recipients.¹⁵ Although no study has specifically investigated the neutropenic diet in preventing infection in patients with noncancer-related neutropenia, no reason exists to suspect that it is helpful. The FDA advises safe food-handling practices for other immunocompromised patients, such as transplant recipients and patients with human immunodeficiency virus/acquired immunodeficiency syndrome, and the same principles can likely be applied to patients with noncancer-related neutropenia.

WHAT WE SHOULD DO INSTEAD

Although the neutropenic diet has not been proven beneficial, the prevention of food-borne infection in this population remains important. FDA-published guidelines, which promote safe food handling to prevent food contamination in patients with cancer, should be followed in inpatient and outpatient settings.¹⁶ These guidelines allow for fresh fruits and vegetables as long as they have been adequately washed. Cleaning (eg, cleaning the lids of canned foods before opening, hand washing), separating raw meats from other foods, cooking to the right temperature (eg, cooking eggs until the yolk and white are firm), and chilling/refrigerating food appropriately are strongly emphasized. These guidelines are also recommended by the American Dietetic Association. Despite additional flexibility, patients following the FDA diet guidelines do not have increased risk of infection.⁸ At our hospitals, the neutropenic diet can no longer be ordered. Neutropenic patients are free to consume all items on the general hospital menu, including eggs, meat, soft cheeses, nuts, and washed raw fruits and vegetables. The National Comprehensive Cancer Network guidelines for the prevention and treatment of cancer-related infections do not specifically address diet.¹⁷ We call upon them to note the lack of benefit and potential harm of the neutropenic diet in the guidelines. Such an action may persuade more institutions to abandon this practice.

RECOMMENDATIONS

• Neutropenic diets, or low-bacteria diets, should not be prescribed to neutropenic patients.
• Properly handled and adequately washed fresh fruits and vegetables can safely be consumed by patients with neutropenia.
• Patients and hospitals should follow FDA-published safe food-handling guidelines to prevent food contamination.

A general diet can be safely ordered for our patient in the presented clinical scenario. Available data from individual studies and pooled data provide no evidence that neutropenic diets prevent infectious complications in patients with neutropenia.

Hospital kitchens must adhere to the food-handling guidelines issued by the FDA, and following these guidelines should provide adequate protection against food-borne infection, even in patients who are immunocompromised. Instead of restricting food groups, the FDA guidelines focus on safe food-handling practices. Less dietary restrictions provide patient’s additional opportunities for balanced nutrition and for food choices based on personal preferences or cultural practices.

Do you think this is a low-value practice? Is this truly a “Thing We Do for No Reason?” Share what you do in your practice and join in the conversation online by retweeting it on Twitter (#TWDFNR) and liking it on Facebook. We invite you to propose ideas for other “Things We Do for No Reason” topics by emailing TWDFNR@hospitalmedicine.org.Disclosures: There are no financial or other disclosures for any author.

Disclosures

There are no financial or other disclosures for any author.

A 70-year-old man presented to the emergency department with 5 days of decreased appetite, frequent urination, tremors, and memory difficulties. He also reported 9 months of malaise, generalized weakness, and weight loss. There was no history of fever, chills, nausea, diarrhea, constipation, pain, or focal neurologic complaints.

This patient exemplifies a common clinical challenge: an older adult with several possibly unrelated concerns. In many patients, a new presentation is usually either a different manifestation of a known condition (eg, a complication of an established malignancy) or the emergence of something they are at risk for based on health behavior or other characteristics (eg, lung cancer in a smoker). The diagnostic process in older adults can be complicated because many have, or are at risk for, multiple chronic conditions.

After reviewing the timeline of symptoms, the presence of 9 months of symptoms suggests a chronic and progressive underlying process, perhaps with subsequent superimposition of an acute problem. Although it is not certain whether chronic and acute symptoms are caused by the same process, this assumption is reasonable. The superimposition of acute symptoms on a chronic process may represent progression of the underlying condition or an acute complication of the underlying disease. However, the patient’s chronic symptoms of malaise, weakness, and weight loss are nonspecific.

Although malignancy is a consideration given the age of the patient and time course of symptoms, attributing the symptoms to a specific pattern of disease or building a cogent differential diagnosis is difficult until additional information is obtained. One strategy is to try to localize the findings to 1 or more organ systems; for example, given that tremors and memory difficulties localize to the central nervous system, neurodegenerative disorders, such as “Parkinson plus” syndromes, and cerebellar disease are possible. However, this tactic still leaves a relatively broad set of symptoms without an immediate and clear unifying cause.

The patient’s medical history included hyperlipidemia, peripheral neuropathy, prostate cancer, and papillary bladder cancer. The patient was admitted to the hospital 4 months earlier for severe sepsis presumed secondary to a urinary tract infection, although bacterial cultures were sterile. His social history was notable for a 50 pack-year smoking history. Outpatient medications included alfuzosin, gabapentin, simvastatin, hydrocodone, and cholecalciferol. He used a Bright Light Therapy lamp for 1 hour per week and occasionally used calcium carbonate for indigestion. The patient’s sister had a history of throat cancer.

On examination, the patient was detected with blood pressure of 104/56 mm Hg, pulse of 85 beats per minute, temperature of 98.2 °F, oxygen saturation of 97% on ambient air, and body mass index of 18 kg/m². The patient appeared frail with mildly decreased strength in the upper and lower extremities bilaterally. The remainder of the physical examination was normal. Reflexes were symmetric, no tremors or rigidity was noted, sensation was intact to light touch, and the response to the Romberg maneuver was normal.

Past medical history is the cornerstone of the diagnostic process. The history of 2 different malignancies is the most striking element in this case. Papillary bladder cancer is usually a local process, but additional information is needed regarding its stage and previous treatment, including whether or not the patient received Bacille Calmette Guerin (BCG) vaccine, which can rarely be associated with infectious and inflammatory complications. Metastatic prostate cancer could certainly account for his symptomatology, and bladder outlet obstruction could explain the history of urinary frequency and probable urosepsis. His medication list suggested no obvious causes to explain his presentation, except that cholecalciferol and calcium carbonate, which when taken in excess, can cause hypercalcemia. This finding is of particular importance given that many of the patient’s symptoms, including polyuria, malaise, weakness, tremor, memory difficulties, anorexia, acute kidney injury and (indirectly) hypotension and weight loss, are also seen in patients with hypercalcemia. The relatively normal result of the neurologic examination decreases the probability of a primary neurologic disorder and increases the likelihood that his neurologic symptoms are due to a global systemic process. The relative hypotension and weight loss similarly support the possibility that the patient is experiencing a chronic and progressive process.

The differential diagnosis remains broad. An underlying malignancy would explain the chronic progressive course, and superimposed hypercalcemia would explain the acute symptoms of polyuria, tremor, and memory changes. Endocrinopathies including hyperthyroidism or adrenal insufficiency are other possibilities. A chronic progressive infection, such as tuberculosis, is possible, although no epidemiologic factors that increase his risk for this disease are present.

The patient had serum calcium of 14.5 mg/dL, ionized calcium of 3.46 mEq/L, albumin of 3.6 g/dL, BUN of 62 mg/dL, and creatinine of 3.9 mg/dL (all values were normal 3 months prior). His electrolytes and liver function were otherwise normal. Moreover, he had hemoglobin level of 10.5 mg/dL, white blood cell count of 4.8 × 10⁹cells/L, and platelet count of 203 × 10⁹ cells/L.

Until this point, only nonspecific findings were identified, leading to a broad differential diagnosis with little specificity. However, laboratory examinations confirm the suspected diagnosis of hypercalcemia, provide an opportunity to explain the patient’s symptoms, and offer a “lens” to narrow the differential diagnosis and guide the diagnostic evaluation. Hypercalcemia is most commonly secondary to primary hyperparathyroidism or malignancy. Primary hyperparathyroidism is unlikely in this patient given the relatively acute onset of symptoms. The degree of hypercalcemia is also atypical for primary hyperparathyroidism because it rarely exceeds 13 mg/dL, although the use of concurrent vitamin D and calcium supplementation could explain the high calcium level. Malignancy seems more likely given the degree of hypercalcemia in the setting of weight loss, tobacco use, and history of malignancy. Malignancy may cause hypercalcemia through multiple disparate mechanisms, including development of osteolytic bone metastases, elaboration of parathyroid hormone-related Peptide (PTHrP), increased production of 1,25-dihydroxyvitamin D, or, very rarely, ectopic production of parathyroid hormone (PTH). However, none of these mechanisms are particularly common in bladder or prostate cancer, which are the known malignancies in the patient. Other less likely and less common causes of hypercalcemia are also possible given the clinical clues, including vitamin D toxicity and milk alkali syndrome (vitamin D and calcium carbonate supplementation), multiple endocrine neoplasia (a sister with “throat cancer”), and granulomatous disease (weight loss). At this point, further laboratory evaluations would be helpful, specifically determination of PTH and PTHrP levels and serum and urine protein electrophoresis.

With respect to the patient’s past medical history, his Gleason 3 + 3 prostate cancer was diagnosed 12 years prior to admission and treated with external beam radiation therapy and brachytherapy. His bladder cancer was diagnosed 3 years before admission and treated with tumor resection followed by 2 rounds of intravesical BCG (iBCG), 1 round of mitomycin C, and 2 additional rounds of iBCG over the course of treatment spanning 2 years and 6 months. The treatment was complicated by urethral strictures requiring dilation, ureteral outlet obstruction requiring left ureteral stent placement, and multiple urinary tract infections.

The patient’s last round of iBCG was delivered 6 months prior to his current presentation. The patient’s hospital admission 4 months earlier for severe sepsis was presumed secondary to a urologic source considering that significant pyuria was noted on urinalysis and he was treated with meropenem, although bacterial cultures of blood and urine were sterile. From the time of discharge until his current presentation, he experienced progressive weakness and an approximately 50 lb weight loss.

The prior cancers and associated treatments of the patient may be involved in his current presentation. The simplest explanation would be metastatic disease with resultant hypercalcemia, which is atypical of either prostate or bladder cancer. The history of genitourinary surgery could predispose the patient to a chronic infection of the urinary tract with indolent organisms, such as a fungus, especially given the prior sepsis without clear etiology. However, the history would not explain the presence of hypercalcemia. Tuberculosis must thus be considered given the weight loss, hypercalcemia, and “sterile pyuria” of the patient. A more intriguing possibility is whether or not the patient’s constellation of signs and symptoms might be a late effect of iBCG. Intravesical BCG for treatment of localized bladder cancer is occasionally associated with complications. BCG is a modified live form of Mycobacterium bovis which invokes an intense inflammatory reaction when instilled into the bladder. These complications include disseminated infection and local complications, such as genitourinary infections. BCG infection might also explain the severe sepsis of unclear etiology that the patient had experienced 4 months earlier. Most interestingly, hypercalcemia has been described in the setting of BCG infection. Diagnosis of disseminated BCG is best made via culture or polymerase chain reaction testing for M. bovis at potential sites of involvement, including the blood. Nevertheless, a common presentation of a common disorder is still most likely. If his current presentation is distilled down to a chronic presentation of weakness, weight loss, and hypercalcemia in the setting of known malignancy, then the underlying malignancy seems to offer the most unifying explanation. Given that neither of his known cancers are commonly associated with hypercalcemia, the possibility that he has developed a third malignancy must also be considered.

In the hospital, the patient received intravenous normal saline, furosemide, and pamidronate. Evaluation for hypercalcemia revealed appropriately suppressed PTH (8 mg/dL), and normal levels of PTHrP (<.74 pmol/L), prostate specific antigen (<.01 ng/mL), and morning cortisol (16.7 mcg/dL). Serum and urine electrophoresis did not show evidence for monoclonal gammopathy, and the 25-hydroxy vitamin D level (39.5 ng/mL) was within the normal limits (normal range 20.1-50.0 ng/mL). The patient had elevated levels of 1,25-dihydroxy vitamin D (122 ng/mL, normal range 19.9–79.3 pg/mL), lactate dehydrogenase (196 units/L, normal 50–150 units/L), and angiotensin-converting enzyme (153 units/L, normal 14–82 units/L).

The suppressed PTH level makes primary hyperparathyroidism unlikely, the low PTHrP level decreases the probability of a paraneoplastic process, and the normal protein electrophoresis makes multiple myeloma unlikely. The presence of a significantly elevated 1,25-dihydroxy vitamin D level with a normal 25-hydroxy vitamin D level indicates extrarenal conversion of 25-hydroxy vitamin D by 1-hydroxylase as the etiology of hypercalcemia. Increased activity of 1-hydroxylase is the most consistent with granulomatous diseases, including sarcoidosis, and, with the exception of lymphoma, would not be expected in hypercalcemia malignancy. This mechanism is also associated with tuberculosis, disseminated fungal infections, such as coccidioidomycosis and histoplasmosis, and as a late effect of BCG treatment, regardless of whether disseminated infection or granulomatous immune response. Elevated lactate dehydrogenase and angiotensin-converting enzyme levels may also be noted in many of these disorders.

Lymphoma would appear to be the most likely diagnosis as it accounts for most of the clinical findings observed in the patient and is a fairly common disorder. Sarcoidosis is also reasonably common and would explain the laboratory abnormalities but is not usually associated with weight loss and frailty. Disseminated infections, such as tuberculosis, histoplasmosis, and coccidioidomycosis, are all possible, but the patient lacks key risk factors for these infections. A complication of iBCG is the most intriguing possibility and could account for many of the patient’s clinical findings, including the septic episode, which is an event not clearly accounted for by the other diagnostic possibilities. However, disseminated BCG and hypersensitivity reactions to BCG leading to hypercalcemia are rare. When asked to choose between the most interesting possibility and the most common possibility, the most common will usually be the best (and safest) bet. Nonetheless, the effects of prior BCG treatment, including disseminated infection or diffuse immune-mediated granulomatous disease, would be near the top of the differential diagnosis in this case.

The bone survey was normal, the renal ultrasound examination showed nodular wall thickening of the bladder with areas of calcification, and the CT scan of the chest, abdomen, and pelvis showed an area of calcification in the superior portion of the bladder but no evidence of lymphadenopathy or masses to suggest lymphoma. Aerobic and anaerobic blood and urine cultures were sterile. The patient was discharged 12 days after admission with plans for further outpatient diagnostic evaluation. At this time, his serum calcium had stabilized at 10.5 mg/dL with pamidronate, diuretics, and aggressive oral hydration.

Hypercalcemia is a common finding in both hospital and ambulatory settings. The classic symptoms associated with hypercalcemia are aptly summarized with the mnemonic “bones, stones, abdominal groans, and psychiatric overtones” (to represent the associated skeletal involvement, renal disease, gastrointestinal symptoms, and effects on the nervous system). However, the severity and type of symptoms vary depending on the degree of hypercalcemia, acuity of onset, and underlying etiology. The vast majority (90%) of hypercalcemia cases are due to primary hyperparathyroidism and malignancy.³ Measuring the PTH level is a key step in the diagnostic evaluation process. An isolated elevation of PTH confirms the presence of primary or possibly tertiary hyperparathyroidism. Low PTH concentrations (<20 pg/mL) occur in the settings of PTHrP or vitamin-D-mediated hypercalcemia such as hypervitaminosis D, malignancy, or granulomatous disease.

Elevated PTHrP occurs most commonly in squamous cell, renal, bladder, and ovarian carcinomas.^3,4 Elevated levels of 25-hydroxy vitamin D can occur with excessive consumption of vitamin D-containing products and some herbal supplements. In this case, neither PTHrP nor 25-hydroxy vitamin D level was elevated, leading to an exhaustive search for other causes. Although iBCG treatment is a rare cause of hypercalcemia, 2 previous reports indicated the presence of hypercalcemia secondary to granuloma formation in treated patients.^5,6

The finding of an elevated 1,25-dihydroxy vitamin D level was unexpected. As the discussant mentioned, this finding is associated with lymphoma and with granulomatous disorders that were not initially strong diagnostic considerations in the patient. A variety of granulomatous diseases can cause hypercalcemia. Sarcoidosis and tuberculosis are the most common, but berylliosis, fungal infections, Crohn’s disease, silicone exposure, and granulomatosis with polyangiitis may also be associated with hypercalcemia.⁷ The mechanism for hypercalcemia in these situations is increased intestinal calcium absorption mediated by inappropriately increased, PTH-independent, extrarenal calcitriol (1,25-dihydroxy vitamin D) production. Activated monocytes upregulate 25(OH)D-alpha-hydroxylase, converting 25-hydroxy vitamin D to 1,25-dihydroxy vitamin D. Concurrently, the elevated levels of gamma-interferon render macrophages resistant to the normal regulatory feedback mechanisms, thereby promoting the production and inhibiting the degradation of 1,25-dihydroxy vitamin D.⁸

The tuberculosis vaccine BCG is an attenuated form of M. bovis and was originally developed by Albert Calmette and Camille Guérin at the Pasteur Institute in Paris in the early 20^th century. In addition to its use as a vaccine against tuberculosis, BCG can protect against other mycobacterial infections, help treat atopic conditions via stimulation of the Th1 cellular immune response, and has been used as an antineoplastic agent. To date, BCG remains the most effective agent available for intravesical treatment of superficial bladder cancer.^9,10 Although iBCG therapy is considered relatively safe and well-tolerated, rare complications do occur. Localized symptoms (bladder irritation, hematuria) and/or flu-like symptoms are common immediately after instillation and thought to be related to the cellular immune response and inflammatory cascade triggered by mycobacterial antigens.¹¹ Other adverse effects, such as infectious and noninfectious complications, may occur months to years after treatment with BCG, and the associated symptoms can be quite nonspecific. Infectious complications include mycobacterial prostatitis, orchiepididymitis, balantitis, pneumonia, hepatitis, nephritis, septic arthritis, osteomyelitis, infected orthopedic and vascular prostheses, endocarditis, and bacteremia. Traumatic catheterization is the most common risk factor for infection with BCG.^11-13 Noninfectious complications include reactive arthritis, hypersensitivity pneumonitis, hemophagocytic lymphohistiocytosis (HLH), and sterile granulomatous infiltration of solid organs.

The protean and nonspecific nature of the adverse effects of iBCG treatment and the fact that complications can present weeks to years after instillation can make diagnosis quite challenging.¹⁴ Even if clinical suspicion is high, it may be difficult to definitively identify BCG as the underlying etiology because acid fast staining, culture, and even PCR can lead to falsely negative results.^14,15 For this reason, biopsy and tissue culture are recommended to demonstrate granuloma formation and identify the presence of M. bovis.

Although no prospective studies have been conducted to assess the optimal therapy for BCG infection, opinion-based recommendations include cessation of BCG treatment, initiation of at least 3 tuberculostatic agents, and treatment for 3-12 months depending on the severity of the complications.^11,14 M. bovis is susceptible to isoniazid, rifampin, and ethambutol as well as to fluoroquinolones, clarithromycin, aminoglycosides, and doxycycline; however, this organism is highly resistant to pyrazinamide due to single-point mutation.^11,16Interestingly, imipenem is used to treat other nontuberculous mycobacterial diseases, such as those caused by M. abscessus, thereby raising the possibility that the patient’s exposure to meropenem during treatment for his prior sepsis may have partially treated an acute infection due to M. bovis.

Although treatment with steroids is a standard approach for management of hypercalcemia in other granulomatous disorders and leads to rapid reduction in circulating levels of 1,25-dihydroxy vitamin D and serum calcium., specific evidence has not been established to support its efficacy and effectiveness in treating hypercalcemia and other complications due to M. bovis.¹⁷ Nevertheless, some experts recommend the use of steroids in conjunction with a multidrug tuberculostatic regimen in cases of septicemia and multiorgan failure due to M. bovis.^12,14,18-20

In summary, this case illustrates the importance of making room in differential diagnosis to include iatrogenic complications. That is, when faced with an unclear diagnosis, the provider should consider common and uncommon immediate and delayed side effects of prior therapies.

Teaching Points

• Complications of intravesical BCG treatment include manifestations of granulomatous diseases, such as hypercalcemia.
• When generating a differential diagnosis, medical providers should not only consider the possibility of a new disease process or the progression of a known comorbidity but also the potential of an adverse effect related to prior treatments.
• Medical providers should be wary of accepting previously made diagnoses, particularly when key pieces of objective data are lacking.

Disclosures

 The authors have no financial or other conflicts of interest that might bias this work.

A 70-year-old man presented to the emergency department with 5 days of decreased appetite, frequent urination, tremors, and memory difficulties. He also reported 9 months of malaise, generalized weakness, and weight loss. There was no history of fever, chills, nausea, diarrhea, constipation, pain, or focal neurologic complaints.

This patient exemplifies a common clinical challenge: an older adult with several possibly unrelated concerns. In many patients, a new presentation is usually either a different manifestation of a known condition (eg, a complication of an established malignancy) or the emergence of something they are at risk for based on health behavior or other characteristics (eg, lung cancer in a smoker). The diagnostic process in older adults can be complicated because many have, or are at risk for, multiple chronic conditions.

After reviewing the timeline of symptoms, the presence of 9 months of symptoms suggests a chronic and progressive underlying process, perhaps with subsequent superimposition of an acute problem. Although it is not certain whether chronic and acute symptoms are caused by the same process, this assumption is reasonable. The superimposition of acute symptoms on a chronic process may represent progression of the underlying condition or an acute complication of the underlying disease. However, the patient’s chronic symptoms of malaise, weakness, and weight loss are nonspecific.

Although malignancy is a consideration given the age of the patient and time course of symptoms, attributing the symptoms to a specific pattern of disease or building a cogent differential diagnosis is difficult until additional information is obtained. One strategy is to try to localize the findings to 1 or more organ systems; for example, given that tremors and memory difficulties localize to the central nervous system, neurodegenerative disorders, such as “Parkinson plus” syndromes, and cerebellar disease are possible. However, this tactic still leaves a relatively broad set of symptoms without an immediate and clear unifying cause.

The patient’s medical history included hyperlipidemia, peripheral neuropathy, prostate cancer, and papillary bladder cancer. The patient was admitted to the hospital 4 months earlier for severe sepsis presumed secondary to a urinary tract infection, although bacterial cultures were sterile. His social history was notable for a 50 pack-year smoking history. Outpatient medications included alfuzosin, gabapentin, simvastatin, hydrocodone, and cholecalciferol. He used a Bright Light Therapy lamp for 1 hour per week and occasionally used calcium carbonate for indigestion. The patient’s sister had a history of throat cancer.

On examination, the patient was detected with blood pressure of 104/56 mm Hg, pulse of 85 beats per minute, temperature of 98.2 °F, oxygen saturation of 97% on ambient air, and body mass index of 18 kg/m². The patient appeared frail with mildly decreased strength in the upper and lower extremities bilaterally. The remainder of the physical examination was normal. Reflexes were symmetric, no tremors or rigidity was noted, sensation was intact to light touch, and the response to the Romberg maneuver was normal.

Past medical history is the cornerstone of the diagnostic process. The history of 2 different malignancies is the most striking element in this case. Papillary bladder cancer is usually a local process, but additional information is needed regarding its stage and previous treatment, including whether or not the patient received Bacille Calmette Guerin (BCG) vaccine, which can rarely be associated with infectious and inflammatory complications. Metastatic prostate cancer could certainly account for his symptomatology, and bladder outlet obstruction could explain the history of urinary frequency and probable urosepsis. His medication list suggested no obvious causes to explain his presentation, except that cholecalciferol and calcium carbonate, which when taken in excess, can cause hypercalcemia. This finding is of particular importance given that many of the patient’s symptoms, including polyuria, malaise, weakness, tremor, memory difficulties, anorexia, acute kidney injury and (indirectly) hypotension and weight loss, are also seen in patients with hypercalcemia. The relatively normal result of the neurologic examination decreases the probability of a primary neurologic disorder and increases the likelihood that his neurologic symptoms are due to a global systemic process. The relative hypotension and weight loss similarly support the possibility that the patient is experiencing a chronic and progressive process.

The differential diagnosis remains broad. An underlying malignancy would explain the chronic progressive course, and superimposed hypercalcemia would explain the acute symptoms of polyuria, tremor, and memory changes. Endocrinopathies including hyperthyroidism or adrenal insufficiency are other possibilities. A chronic progressive infection, such as tuberculosis, is possible, although no epidemiologic factors that increase his risk for this disease are present.

The patient had serum calcium of 14.5 mg/dL, ionized calcium of 3.46 mEq/L, albumin of 3.6 g/dL, BUN of 62 mg/dL, and creatinine of 3.9 mg/dL (all values were normal 3 months prior). His electrolytes and liver function were otherwise normal. Moreover, he had hemoglobin level of 10.5 mg/dL, white blood cell count of 4.8 × 10⁹cells/L, and platelet count of 203 × 10⁹ cells/L.

Until this point, only nonspecific findings were identified, leading to a broad differential diagnosis with little specificity. However, laboratory examinations confirm the suspected diagnosis of hypercalcemia, provide an opportunity to explain the patient’s symptoms, and offer a “lens” to narrow the differential diagnosis and guide the diagnostic evaluation. Hypercalcemia is most commonly secondary to primary hyperparathyroidism or malignancy. Primary hyperparathyroidism is unlikely in this patient given the relatively acute onset of symptoms. The degree of hypercalcemia is also atypical for primary hyperparathyroidism because it rarely exceeds 13 mg/dL, although the use of concurrent vitamin D and calcium supplementation could explain the high calcium level. Malignancy seems more likely given the degree of hypercalcemia in the setting of weight loss, tobacco use, and history of malignancy. Malignancy may cause hypercalcemia through multiple disparate mechanisms, including development of osteolytic bone metastases, elaboration of parathyroid hormone-related Peptide (PTHrP), increased production of 1,25-dihydroxyvitamin D, or, very rarely, ectopic production of parathyroid hormone (PTH). However, none of these mechanisms are particularly common in bladder or prostate cancer, which are the known malignancies in the patient. Other less likely and less common causes of hypercalcemia are also possible given the clinical clues, including vitamin D toxicity and milk alkali syndrome (vitamin D and calcium carbonate supplementation), multiple endocrine neoplasia (a sister with “throat cancer”), and granulomatous disease (weight loss). At this point, further laboratory evaluations would be helpful, specifically determination of PTH and PTHrP levels and serum and urine protein electrophoresis.

With respect to the patient’s past medical history, his Gleason 3 + 3 prostate cancer was diagnosed 12 years prior to admission and treated with external beam radiation therapy and brachytherapy. His bladder cancer was diagnosed 3 years before admission and treated with tumor resection followed by 2 rounds of intravesical BCG (iBCG), 1 round of mitomycin C, and 2 additional rounds of iBCG over the course of treatment spanning 2 years and 6 months. The treatment was complicated by urethral strictures requiring dilation, ureteral outlet obstruction requiring left ureteral stent placement, and multiple urinary tract infections.

The patient’s last round of iBCG was delivered 6 months prior to his current presentation. The patient’s hospital admission 4 months earlier for severe sepsis was presumed secondary to a urologic source considering that significant pyuria was noted on urinalysis and he was treated with meropenem, although bacterial cultures of blood and urine were sterile. From the time of discharge until his current presentation, he experienced progressive weakness and an approximately 50 lb weight loss.

The prior cancers and associated treatments of the patient may be involved in his current presentation. The simplest explanation would be metastatic disease with resultant hypercalcemia, which is atypical of either prostate or bladder cancer. The history of genitourinary surgery could predispose the patient to a chronic infection of the urinary tract with indolent organisms, such as a fungus, especially given the prior sepsis without clear etiology. However, the history would not explain the presence of hypercalcemia. Tuberculosis must thus be considered given the weight loss, hypercalcemia, and “sterile pyuria” of the patient. A more intriguing possibility is whether or not the patient’s constellation of signs and symptoms might be a late effect of iBCG. Intravesical BCG for treatment of localized bladder cancer is occasionally associated with complications. BCG is a modified live form of Mycobacterium bovis which invokes an intense inflammatory reaction when instilled into the bladder. These complications include disseminated infection and local complications, such as genitourinary infections. BCG infection might also explain the severe sepsis of unclear etiology that the patient had experienced 4 months earlier. Most interestingly, hypercalcemia has been described in the setting of BCG infection. Diagnosis of disseminated BCG is best made via culture or polymerase chain reaction testing for M. bovis at potential sites of involvement, including the blood. Nevertheless, a common presentation of a common disorder is still most likely. If his current presentation is distilled down to a chronic presentation of weakness, weight loss, and hypercalcemia in the setting of known malignancy, then the underlying malignancy seems to offer the most unifying explanation. Given that neither of his known cancers are commonly associated with hypercalcemia, the possibility that he has developed a third malignancy must also be considered.

In the hospital, the patient received intravenous normal saline, furosemide, and pamidronate. Evaluation for hypercalcemia revealed appropriately suppressed PTH (8 mg/dL), and normal levels of PTHrP (<.74 pmol/L), prostate specific antigen (<.01 ng/mL), and morning cortisol (16.7 mcg/dL). Serum and urine electrophoresis did not show evidence for monoclonal gammopathy, and the 25-hydroxy vitamin D level (39.5 ng/mL) was within the normal limits (normal range 20.1-50.0 ng/mL). The patient had elevated levels of 1,25-dihydroxy vitamin D (122 ng/mL, normal range 19.9–79.3 pg/mL), lactate dehydrogenase (196 units/L, normal 50–150 units/L), and angiotensin-converting enzyme (153 units/L, normal 14–82 units/L).

The suppressed PTH level makes primary hyperparathyroidism unlikely, the low PTHrP level decreases the probability of a paraneoplastic process, and the normal protein electrophoresis makes multiple myeloma unlikely. The presence of a significantly elevated 1,25-dihydroxy vitamin D level with a normal 25-hydroxy vitamin D level indicates extrarenal conversion of 25-hydroxy vitamin D by 1-hydroxylase as the etiology of hypercalcemia. Increased activity of 1-hydroxylase is the most consistent with granulomatous diseases, including sarcoidosis, and, with the exception of lymphoma, would not be expected in hypercalcemia malignancy. This mechanism is also associated with tuberculosis, disseminated fungal infections, such as coccidioidomycosis and histoplasmosis, and as a late effect of BCG treatment, regardless of whether disseminated infection or granulomatous immune response. Elevated lactate dehydrogenase and angiotensin-converting enzyme levels may also be noted in many of these disorders.

Lymphoma would appear to be the most likely diagnosis as it accounts for most of the clinical findings observed in the patient and is a fairly common disorder. Sarcoidosis is also reasonably common and would explain the laboratory abnormalities but is not usually associated with weight loss and frailty. Disseminated infections, such as tuberculosis, histoplasmosis, and coccidioidomycosis, are all possible, but the patient lacks key risk factors for these infections. A complication of iBCG is the most intriguing possibility and could account for many of the patient’s clinical findings, including the septic episode, which is an event not clearly accounted for by the other diagnostic possibilities. However, disseminated BCG and hypersensitivity reactions to BCG leading to hypercalcemia are rare. When asked to choose between the most interesting possibility and the most common possibility, the most common will usually be the best (and safest) bet. Nonetheless, the effects of prior BCG treatment, including disseminated infection or diffuse immune-mediated granulomatous disease, would be near the top of the differential diagnosis in this case.

The bone survey was normal, the renal ultrasound examination showed nodular wall thickening of the bladder with areas of calcification, and the CT scan of the chest, abdomen, and pelvis showed an area of calcification in the superior portion of the bladder but no evidence of lymphadenopathy or masses to suggest lymphoma. Aerobic and anaerobic blood and urine cultures were sterile. The patient was discharged 12 days after admission with plans for further outpatient diagnostic evaluation. At this time, his serum calcium had stabilized at 10.5 mg/dL with pamidronate, diuretics, and aggressive oral hydration.

Hypercalcemia is a common finding in both hospital and ambulatory settings. The classic symptoms associated with hypercalcemia are aptly summarized with the mnemonic “bones, stones, abdominal groans, and psychiatric overtones” (to represent the associated skeletal involvement, renal disease, gastrointestinal symptoms, and effects on the nervous system). However, the severity and type of symptoms vary depending on the degree of hypercalcemia, acuity of onset, and underlying etiology. The vast majority (90%) of hypercalcemia cases are due to primary hyperparathyroidism and malignancy.³ Measuring the PTH level is a key step in the diagnostic evaluation process. An isolated elevation of PTH confirms the presence of primary or possibly tertiary hyperparathyroidism. Low PTH concentrations (<20 pg/mL) occur in the settings of PTHrP or vitamin-D-mediated hypercalcemia such as hypervitaminosis D, malignancy, or granulomatous disease.

Elevated PTHrP occurs most commonly in squamous cell, renal, bladder, and ovarian carcinomas.^3,4 Elevated levels of 25-hydroxy vitamin D can occur with excessive consumption of vitamin D-containing products and some herbal supplements. In this case, neither PTHrP nor 25-hydroxy vitamin D level was elevated, leading to an exhaustive search for other causes. Although iBCG treatment is a rare cause of hypercalcemia, 2 previous reports indicated the presence of hypercalcemia secondary to granuloma formation in treated patients.^5,6

The finding of an elevated 1,25-dihydroxy vitamin D level was unexpected. As the discussant mentioned, this finding is associated with lymphoma and with granulomatous disorders that were not initially strong diagnostic considerations in the patient. A variety of granulomatous diseases can cause hypercalcemia. Sarcoidosis and tuberculosis are the most common, but berylliosis, fungal infections, Crohn’s disease, silicone exposure, and granulomatosis with polyangiitis may also be associated with hypercalcemia.⁷ The mechanism for hypercalcemia in these situations is increased intestinal calcium absorption mediated by inappropriately increased, PTH-independent, extrarenal calcitriol (1,25-dihydroxy vitamin D) production. Activated monocytes upregulate 25(OH)D-alpha-hydroxylase, converting 25-hydroxy vitamin D to 1,25-dihydroxy vitamin D. Concurrently, the elevated levels of gamma-interferon render macrophages resistant to the normal regulatory feedback mechanisms, thereby promoting the production and inhibiting the degradation of 1,25-dihydroxy vitamin D.⁸

The tuberculosis vaccine BCG is an attenuated form of M. bovis and was originally developed by Albert Calmette and Camille Guérin at the Pasteur Institute in Paris in the early 20^th century. In addition to its use as a vaccine against tuberculosis, BCG can protect against other mycobacterial infections, help treat atopic conditions via stimulation of the Th1 cellular immune response, and has been used as an antineoplastic agent. To date, BCG remains the most effective agent available for intravesical treatment of superficial bladder cancer.^9,10 Although iBCG therapy is considered relatively safe and well-tolerated, rare complications do occur. Localized symptoms (bladder irritation, hematuria) and/or flu-like symptoms are common immediately after instillation and thought to be related to the cellular immune response and inflammatory cascade triggered by mycobacterial antigens.¹¹ Other adverse effects, such as infectious and noninfectious complications, may occur months to years after treatment with BCG, and the associated symptoms can be quite nonspecific. Infectious complications include mycobacterial prostatitis, orchiepididymitis, balantitis, pneumonia, hepatitis, nephritis, septic arthritis, osteomyelitis, infected orthopedic and vascular prostheses, endocarditis, and bacteremia. Traumatic catheterization is the most common risk factor for infection with BCG.^11-13 Noninfectious complications include reactive arthritis, hypersensitivity pneumonitis, hemophagocytic lymphohistiocytosis (HLH), and sterile granulomatous infiltration of solid organs.

The protean and nonspecific nature of the adverse effects of iBCG treatment and the fact that complications can present weeks to years after instillation can make diagnosis quite challenging.¹⁴ Even if clinical suspicion is high, it may be difficult to definitively identify BCG as the underlying etiology because acid fast staining, culture, and even PCR can lead to falsely negative results.^14,15 For this reason, biopsy and tissue culture are recommended to demonstrate granuloma formation and identify the presence of M. bovis.

Although no prospective studies have been conducted to assess the optimal therapy for BCG infection, opinion-based recommendations include cessation of BCG treatment, initiation of at least 3 tuberculostatic agents, and treatment for 3-12 months depending on the severity of the complications.^11,14 M. bovis is susceptible to isoniazid, rifampin, and ethambutol as well as to fluoroquinolones, clarithromycin, aminoglycosides, and doxycycline; however, this organism is highly resistant to pyrazinamide due to single-point mutation.^11,16Interestingly, imipenem is used to treat other nontuberculous mycobacterial diseases, such as those caused by M. abscessus, thereby raising the possibility that the patient’s exposure to meropenem during treatment for his prior sepsis may have partially treated an acute infection due to M. bovis.

Although treatment with steroids is a standard approach for management of hypercalcemia in other granulomatous disorders and leads to rapid reduction in circulating levels of 1,25-dihydroxy vitamin D and serum calcium., specific evidence has not been established to support its efficacy and effectiveness in treating hypercalcemia and other complications due to M. bovis.¹⁷ Nevertheless, some experts recommend the use of steroids in conjunction with a multidrug tuberculostatic regimen in cases of septicemia and multiorgan failure due to M. bovis.^12,14,18-20

In summary, this case illustrates the importance of making room in differential diagnosis to include iatrogenic complications. That is, when faced with an unclear diagnosis, the provider should consider common and uncommon immediate and delayed side effects of prior therapies.

Teaching Points

• Complications of intravesical BCG treatment include manifestations of granulomatous diseases, such as hypercalcemia.
• When generating a differential diagnosis, medical providers should not only consider the possibility of a new disease process or the progression of a known comorbidity but also the potential of an adverse effect related to prior treatments.
• Medical providers should be wary of accepting previously made diagnoses, particularly when key pieces of objective data are lacking.

Disclosures

 The authors have no financial or other conflicts of interest that might bias this work.

A 70-year-old man presented to the emergency department with 5 days of decreased appetite, frequent urination, tremors, and memory difficulties. He also reported 9 months of malaise, generalized weakness, and weight loss. There was no history of fever, chills, nausea, diarrhea, constipation, pain, or focal neurologic complaints.

This patient exemplifies a common clinical challenge: an older adult with several possibly unrelated concerns. In many patients, a new presentation is usually either a different manifestation of a known condition (eg, a complication of an established malignancy) or the emergence of something they are at risk for based on health behavior or other characteristics (eg, lung cancer in a smoker). The diagnostic process in older adults can be complicated because many have, or are at risk for, multiple chronic conditions.

After reviewing the timeline of symptoms, the presence of 9 months of symptoms suggests a chronic and progressive underlying process, perhaps with subsequent superimposition of an acute problem. Although it is not certain whether chronic and acute symptoms are caused by the same process, this assumption is reasonable. The superimposition of acute symptoms on a chronic process may represent progression of the underlying condition or an acute complication of the underlying disease. However, the patient’s chronic symptoms of malaise, weakness, and weight loss are nonspecific.

Although malignancy is a consideration given the age of the patient and time course of symptoms, attributing the symptoms to a specific pattern of disease or building a cogent differential diagnosis is difficult until additional information is obtained. One strategy is to try to localize the findings to 1 or more organ systems; for example, given that tremors and memory difficulties localize to the central nervous system, neurodegenerative disorders, such as “Parkinson plus” syndromes, and cerebellar disease are possible. However, this tactic still leaves a relatively broad set of symptoms without an immediate and clear unifying cause.

The patient’s medical history included hyperlipidemia, peripheral neuropathy, prostate cancer, and papillary bladder cancer. The patient was admitted to the hospital 4 months earlier for severe sepsis presumed secondary to a urinary tract infection, although bacterial cultures were sterile. His social history was notable for a 50 pack-year smoking history. Outpatient medications included alfuzosin, gabapentin, simvastatin, hydrocodone, and cholecalciferol. He used a Bright Light Therapy lamp for 1 hour per week and occasionally used calcium carbonate for indigestion. The patient’s sister had a history of throat cancer.

On examination, the patient was detected with blood pressure of 104/56 mm Hg, pulse of 85 beats per minute, temperature of 98.2 °F, oxygen saturation of 97% on ambient air, and body mass index of 18 kg/m². The patient appeared frail with mildly decreased strength in the upper and lower extremities bilaterally. The remainder of the physical examination was normal. Reflexes were symmetric, no tremors or rigidity was noted, sensation was intact to light touch, and the response to the Romberg maneuver was normal.

Past medical history is the cornerstone of the diagnostic process. The history of 2 different malignancies is the most striking element in this case. Papillary bladder cancer is usually a local process, but additional information is needed regarding its stage and previous treatment, including whether or not the patient received Bacille Calmette Guerin (BCG) vaccine, which can rarely be associated with infectious and inflammatory complications. Metastatic prostate cancer could certainly account for his symptomatology, and bladder outlet obstruction could explain the history of urinary frequency and probable urosepsis. His medication list suggested no obvious causes to explain his presentation, except that cholecalciferol and calcium carbonate, which when taken in excess, can cause hypercalcemia. This finding is of particular importance given that many of the patient’s symptoms, including polyuria, malaise, weakness, tremor, memory difficulties, anorexia, acute kidney injury and (indirectly) hypotension and weight loss, are also seen in patients with hypercalcemia. The relatively normal result of the neurologic examination decreases the probability of a primary neurologic disorder and increases the likelihood that his neurologic symptoms are due to a global systemic process. The relative hypotension and weight loss similarly support the possibility that the patient is experiencing a chronic and progressive process.

The differential diagnosis remains broad. An underlying malignancy would explain the chronic progressive course, and superimposed hypercalcemia would explain the acute symptoms of polyuria, tremor, and memory changes. Endocrinopathies including hyperthyroidism or adrenal insufficiency are other possibilities. A chronic progressive infection, such as tuberculosis, is possible, although no epidemiologic factors that increase his risk for this disease are present.

The patient had serum calcium of 14.5 mg/dL, ionized calcium of 3.46 mEq/L, albumin of 3.6 g/dL, BUN of 62 mg/dL, and creatinine of 3.9 mg/dL (all values were normal 3 months prior). His electrolytes and liver function were otherwise normal. Moreover, he had hemoglobin level of 10.5 mg/dL, white blood cell count of 4.8 × 10⁹cells/L, and platelet count of 203 × 10⁹ cells/L.

Until this point, only nonspecific findings were identified, leading to a broad differential diagnosis with little specificity. However, laboratory examinations confirm the suspected diagnosis of hypercalcemia, provide an opportunity to explain the patient’s symptoms, and offer a “lens” to narrow the differential diagnosis and guide the diagnostic evaluation. Hypercalcemia is most commonly secondary to primary hyperparathyroidism or malignancy. Primary hyperparathyroidism is unlikely in this patient given the relatively acute onset of symptoms. The degree of hypercalcemia is also atypical for primary hyperparathyroidism because it rarely exceeds 13 mg/dL, although the use of concurrent vitamin D and calcium supplementation could explain the high calcium level. Malignancy seems more likely given the degree of hypercalcemia in the setting of weight loss, tobacco use, and history of malignancy. Malignancy may cause hypercalcemia through multiple disparate mechanisms, including development of osteolytic bone metastases, elaboration of parathyroid hormone-related Peptide (PTHrP), increased production of 1,25-dihydroxyvitamin D, or, very rarely, ectopic production of parathyroid hormone (PTH). However, none of these mechanisms are particularly common in bladder or prostate cancer, which are the known malignancies in the patient. Other less likely and less common causes of hypercalcemia are also possible given the clinical clues, including vitamin D toxicity and milk alkali syndrome (vitamin D and calcium carbonate supplementation), multiple endocrine neoplasia (a sister with “throat cancer”), and granulomatous disease (weight loss). At this point, further laboratory evaluations would be helpful, specifically determination of PTH and PTHrP levels and serum and urine protein electrophoresis.

With respect to the patient’s past medical history, his Gleason 3 + 3 prostate cancer was diagnosed 12 years prior to admission and treated with external beam radiation therapy and brachytherapy. His bladder cancer was diagnosed 3 years before admission and treated with tumor resection followed by 2 rounds of intravesical BCG (iBCG), 1 round of mitomycin C, and 2 additional rounds of iBCG over the course of treatment spanning 2 years and 6 months. The treatment was complicated by urethral strictures requiring dilation, ureteral outlet obstruction requiring left ureteral stent placement, and multiple urinary tract infections.

The patient’s last round of iBCG was delivered 6 months prior to his current presentation. The patient’s hospital admission 4 months earlier for severe sepsis was presumed secondary to a urologic source considering that significant pyuria was noted on urinalysis and he was treated with meropenem, although bacterial cultures of blood and urine were sterile. From the time of discharge until his current presentation, he experienced progressive weakness and an approximately 50 lb weight loss.

The prior cancers and associated treatments of the patient may be involved in his current presentation. The simplest explanation would be metastatic disease with resultant hypercalcemia, which is atypical of either prostate or bladder cancer. The history of genitourinary surgery could predispose the patient to a chronic infection of the urinary tract with indolent organisms, such as a fungus, especially given the prior sepsis without clear etiology. However, the history would not explain the presence of hypercalcemia. Tuberculosis must thus be considered given the weight loss, hypercalcemia, and “sterile pyuria” of the patient. A more intriguing possibility is whether or not the patient’s constellation of signs and symptoms might be a late effect of iBCG. Intravesical BCG for treatment of localized bladder cancer is occasionally associated with complications. BCG is a modified live form of Mycobacterium bovis which invokes an intense inflammatory reaction when instilled into the bladder. These complications include disseminated infection and local complications, such as genitourinary infections. BCG infection might also explain the severe sepsis of unclear etiology that the patient had experienced 4 months earlier. Most interestingly, hypercalcemia has been described in the setting of BCG infection. Diagnosis of disseminated BCG is best made via culture or polymerase chain reaction testing for M. bovis at potential sites of involvement, including the blood. Nevertheless, a common presentation of a common disorder is still most likely. If his current presentation is distilled down to a chronic presentation of weakness, weight loss, and hypercalcemia in the setting of known malignancy, then the underlying malignancy seems to offer the most unifying explanation. Given that neither of his known cancers are commonly associated with hypercalcemia, the possibility that he has developed a third malignancy must also be considered.

In the hospital, the patient received intravenous normal saline, furosemide, and pamidronate. Evaluation for hypercalcemia revealed appropriately suppressed PTH (8 mg/dL), and normal levels of PTHrP (<.74 pmol/L), prostate specific antigen (<.01 ng/mL), and morning cortisol (16.7 mcg/dL). Serum and urine electrophoresis did not show evidence for monoclonal gammopathy, and the 25-hydroxy vitamin D level (39.5 ng/mL) was within the normal limits (normal range 20.1-50.0 ng/mL). The patient had elevated levels of 1,25-dihydroxy vitamin D (122 ng/mL, normal range 19.9–79.3 pg/mL), lactate dehydrogenase (196 units/L, normal 50–150 units/L), and angiotensin-converting enzyme (153 units/L, normal 14–82 units/L).

The suppressed PTH level makes primary hyperparathyroidism unlikely, the low PTHrP level decreases the probability of a paraneoplastic process, and the normal protein electrophoresis makes multiple myeloma unlikely. The presence of a significantly elevated 1,25-dihydroxy vitamin D level with a normal 25-hydroxy vitamin D level indicates extrarenal conversion of 25-hydroxy vitamin D by 1-hydroxylase as the etiology of hypercalcemia. Increased activity of 1-hydroxylase is the most consistent with granulomatous diseases, including sarcoidosis, and, with the exception of lymphoma, would not be expected in hypercalcemia malignancy. This mechanism is also associated with tuberculosis, disseminated fungal infections, such as coccidioidomycosis and histoplasmosis, and as a late effect of BCG treatment, regardless of whether disseminated infection or granulomatous immune response. Elevated lactate dehydrogenase and angiotensin-converting enzyme levels may also be noted in many of these disorders.

Lymphoma would appear to be the most likely diagnosis as it accounts for most of the clinical findings observed in the patient and is a fairly common disorder. Sarcoidosis is also reasonably common and would explain the laboratory abnormalities but is not usually associated with weight loss and frailty. Disseminated infections, such as tuberculosis, histoplasmosis, and coccidioidomycosis, are all possible, but the patient lacks key risk factors for these infections. A complication of iBCG is the most intriguing possibility and could account for many of the patient’s clinical findings, including the septic episode, which is an event not clearly accounted for by the other diagnostic possibilities. However, disseminated BCG and hypersensitivity reactions to BCG leading to hypercalcemia are rare. When asked to choose between the most interesting possibility and the most common possibility, the most common will usually be the best (and safest) bet. Nonetheless, the effects of prior BCG treatment, including disseminated infection or diffuse immune-mediated granulomatous disease, would be near the top of the differential diagnosis in this case.

The bone survey was normal, the renal ultrasound examination showed nodular wall thickening of the bladder with areas of calcification, and the CT scan of the chest, abdomen, and pelvis showed an area of calcification in the superior portion of the bladder but no evidence of lymphadenopathy or masses to suggest lymphoma. Aerobic and anaerobic blood and urine cultures were sterile. The patient was discharged 12 days after admission with plans for further outpatient diagnostic evaluation. At this time, his serum calcium had stabilized at 10.5 mg/dL with pamidronate, diuretics, and aggressive oral hydration.

Hypercalcemia is a common finding in both hospital and ambulatory settings. The classic symptoms associated with hypercalcemia are aptly summarized with the mnemonic “bones, stones, abdominal groans, and psychiatric overtones” (to represent the associated skeletal involvement, renal disease, gastrointestinal symptoms, and effects on the nervous system). However, the severity and type of symptoms vary depending on the degree of hypercalcemia, acuity of onset, and underlying etiology. The vast majority (90%) of hypercalcemia cases are due to primary hyperparathyroidism and malignancy.³ Measuring the PTH level is a key step in the diagnostic evaluation process. An isolated elevation of PTH confirms the presence of primary or possibly tertiary hyperparathyroidism. Low PTH concentrations (<20 pg/mL) occur in the settings of PTHrP or vitamin-D-mediated hypercalcemia such as hypervitaminosis D, malignancy, or granulomatous disease.

Elevated PTHrP occurs most commonly in squamous cell, renal, bladder, and ovarian carcinomas.^3,4 Elevated levels of 25-hydroxy vitamin D can occur with excessive consumption of vitamin D-containing products and some herbal supplements. In this case, neither PTHrP nor 25-hydroxy vitamin D level was elevated, leading to an exhaustive search for other causes. Although iBCG treatment is a rare cause of hypercalcemia, 2 previous reports indicated the presence of hypercalcemia secondary to granuloma formation in treated patients.^5,6

The finding of an elevated 1,25-dihydroxy vitamin D level was unexpected. As the discussant mentioned, this finding is associated with lymphoma and with granulomatous disorders that were not initially strong diagnostic considerations in the patient. A variety of granulomatous diseases can cause hypercalcemia. Sarcoidosis and tuberculosis are the most common, but berylliosis, fungal infections, Crohn’s disease, silicone exposure, and granulomatosis with polyangiitis may also be associated with hypercalcemia.⁷ The mechanism for hypercalcemia in these situations is increased intestinal calcium absorption mediated by inappropriately increased, PTH-independent, extrarenal calcitriol (1,25-dihydroxy vitamin D) production. Activated monocytes upregulate 25(OH)D-alpha-hydroxylase, converting 25-hydroxy vitamin D to 1,25-dihydroxy vitamin D. Concurrently, the elevated levels of gamma-interferon render macrophages resistant to the normal regulatory feedback mechanisms, thereby promoting the production and inhibiting the degradation of 1,25-dihydroxy vitamin D.⁸

The tuberculosis vaccine BCG is an attenuated form of M. bovis and was originally developed by Albert Calmette and Camille Guérin at the Pasteur Institute in Paris in the early 20^th century. In addition to its use as a vaccine against tuberculosis, BCG can protect against other mycobacterial infections, help treat atopic conditions via stimulation of the Th1 cellular immune response, and has been used as an antineoplastic agent. To date, BCG remains the most effective agent available for intravesical treatment of superficial bladder cancer.^9,10 Although iBCG therapy is considered relatively safe and well-tolerated, rare complications do occur. Localized symptoms (bladder irritation, hematuria) and/or flu-like symptoms are common immediately after instillation and thought to be related to the cellular immune response and inflammatory cascade triggered by mycobacterial antigens.¹¹ Other adverse effects, such as infectious and noninfectious complications, may occur months to years after treatment with BCG, and the associated symptoms can be quite nonspecific. Infectious complications include mycobacterial prostatitis, orchiepididymitis, balantitis, pneumonia, hepatitis, nephritis, septic arthritis, osteomyelitis, infected orthopedic and vascular prostheses, endocarditis, and bacteremia. Traumatic catheterization is the most common risk factor for infection with BCG.^11-13 Noninfectious complications include reactive arthritis, hypersensitivity pneumonitis, hemophagocytic lymphohistiocytosis (HLH), and sterile granulomatous infiltration of solid organs.

The protean and nonspecific nature of the adverse effects of iBCG treatment and the fact that complications can present weeks to years after instillation can make diagnosis quite challenging.¹⁴ Even if clinical suspicion is high, it may be difficult to definitively identify BCG as the underlying etiology because acid fast staining, culture, and even PCR can lead to falsely negative results.^14,15 For this reason, biopsy and tissue culture are recommended to demonstrate granuloma formation and identify the presence of M. bovis.

Although no prospective studies have been conducted to assess the optimal therapy for BCG infection, opinion-based recommendations include cessation of BCG treatment, initiation of at least 3 tuberculostatic agents, and treatment for 3-12 months depending on the severity of the complications.^11,14 M. bovis is susceptible to isoniazid, rifampin, and ethambutol as well as to fluoroquinolones, clarithromycin, aminoglycosides, and doxycycline; however, this organism is highly resistant to pyrazinamide due to single-point mutation.^11,16Interestingly, imipenem is used to treat other nontuberculous mycobacterial diseases, such as those caused by M. abscessus, thereby raising the possibility that the patient’s exposure to meropenem during treatment for his prior sepsis may have partially treated an acute infection due to M. bovis.

Although treatment with steroids is a standard approach for management of hypercalcemia in other granulomatous disorders and leads to rapid reduction in circulating levels of 1,25-dihydroxy vitamin D and serum calcium., specific evidence has not been established to support its efficacy and effectiveness in treating hypercalcemia and other complications due to M. bovis.¹⁷ Nevertheless, some experts recommend the use of steroids in conjunction with a multidrug tuberculostatic regimen in cases of septicemia and multiorgan failure due to M. bovis.^12,14,18-20

In summary, this case illustrates the importance of making room in differential diagnosis to include iatrogenic complications. That is, when faced with an unclear diagnosis, the provider should consider common and uncommon immediate and delayed side effects of prior therapies.

Teaching Points

• Complications of intravesical BCG treatment include manifestations of granulomatous diseases, such as hypercalcemia.
• When generating a differential diagnosis, medical providers should not only consider the possibility of a new disease process or the progression of a known comorbidity but also the potential of an adverse effect related to prior treatments.
• Medical providers should be wary of accepting previously made diagnoses, particularly when key pieces of objective data are lacking.

Disclosures

 The authors have no financial or other conflicts of interest that might bias this work.

Royer and colleagues¹ have performed a meta-analysis comparing shorter versus longer courses of antibiotics for treating infections in hospitalized patients. They conclude that shorter courses are safe. However, the authors do not address a flaw in the analysis; they included studies in which treatment with azithromycin was considered a short antibiotic course relative to treatment with another antibiotic. Azithromycin is a macrolide antibiotic that has a relatively long terminal serum half-life, which has been reported to be 35-96 hours.^2-4 Moreover, the half-life of azithromycin in lung tissue can be as long as 132 hours,⁴ which is important because tissue concentrations are thought to be more indicative of the clinical efficacy of macrolides.⁵ In 4 of 19 studies in the meta-analysis,¹ azithromycin was used as a short course for the treatment of pneumonia and compared with longer courses of antibiotics with a much shorter half-life. This implies that in these studies, the duration of the effective antibiotic tissue concentration in the short arms was probably not shorter than in the comparator arms. It could even be longer due to azithromycin’s favorable pharmacokinetics. In our view, these studies have unfairly contributed to the clinical efficacy of short courses, thereby threatening the validity of the overall conclusions. We think that effective antibiotic blood/tissue levels determine the clinical outcome, not just shorter or longer antibiotic courses.

Disclosures

The authors declare that they have no conflicts of interest to report.

Royer and colleagues¹ have performed a meta-analysis comparing shorter versus longer courses of antibiotics for treating infections in hospitalized patients. They conclude that shorter courses are safe. However, the authors do not address a flaw in the analysis; they included studies in which treatment with azithromycin was considered a short antibiotic course relative to treatment with another antibiotic. Azithromycin is a macrolide antibiotic that has a relatively long terminal serum half-life, which has been reported to be 35-96 hours.^2-4 Moreover, the half-life of azithromycin in lung tissue can be as long as 132 hours,⁴ which is important because tissue concentrations are thought to be more indicative of the clinical efficacy of macrolides.⁵ In 4 of 19 studies in the meta-analysis,¹ azithromycin was used as a short course for the treatment of pneumonia and compared with longer courses of antibiotics with a much shorter half-life. This implies that in these studies, the duration of the effective antibiotic tissue concentration in the short arms was probably not shorter than in the comparator arms. It could even be longer due to azithromycin’s favorable pharmacokinetics. In our view, these studies have unfairly contributed to the clinical efficacy of short courses, thereby threatening the validity of the overall conclusions. We think that effective antibiotic blood/tissue levels determine the clinical outcome, not just shorter or longer antibiotic courses.

Disclosures

The authors declare that they have no conflicts of interest to report.

Royer and colleagues¹ have performed a meta-analysis comparing shorter versus longer courses of antibiotics for treating infections in hospitalized patients. They conclude that shorter courses are safe. However, the authors do not address a flaw in the analysis; they included studies in which treatment with azithromycin was considered a short antibiotic course relative to treatment with another antibiotic. Azithromycin is a macrolide antibiotic that has a relatively long terminal serum half-life, which has been reported to be 35-96 hours.^2-4 Moreover, the half-life of azithromycin in lung tissue can be as long as 132 hours,⁴ which is important because tissue concentrations are thought to be more indicative of the clinical efficacy of macrolides.⁵ In 4 of 19 studies in the meta-analysis,¹ azithromycin was used as a short course for the treatment of pneumonia and compared with longer courses of antibiotics with a much shorter half-life. This implies that in these studies, the duration of the effective antibiotic tissue concentration in the short arms was probably not shorter than in the comparator arms. It could even be longer due to azithromycin’s favorable pharmacokinetics. In our view, these studies have unfairly contributed to the clinical efficacy of short courses, thereby threatening the validity of the overall conclusions. We think that effective antibiotic blood/tissue levels determine the clinical outcome, not just shorter or longer antibiotic courses.

Disclosures

The authors declare that they have no conflicts of interest to report.