Accidental falls are among the most common incidents reported in hospitals, complicating approximately 2% of hospital stays.^1-3 Approximately 25% of falls in hospitalized patients result in injury, and 2% involve fractures.⁴ Substantial costs are associated with falls, including patient care costs associated with increased length of stay and liability.^5-7

Beginning October 1, 2008, the Centers for Medicare & Medicaid Services (CMS) stopped reimbursing hospitals for the additional care associated with eight hospital-acquired conditions (HACs), including serious fall-related injury, which were believed to be “reasonably preventable.”^8,9 Before this change, hospitals recovered the costs of these “never events” by assigning a higher level MS-DRG (Medicare Severity Diagnosis-Related Group) code for patients experiencing such an event. This is no longer allowed under the revised CMS Prospective Payment System rules.

Although the financial penalty for iatrogenic injury was modest, the payment change placed pressure on hospital staff to decrease falls, and some nurses reported changing practice to be more restrictive of patient mobility.¹⁰ Increased use of physical restraints is a potential unintended consequence of this rule change.¹¹ Restraints are known to cause agitation, delirium, decubiti, deconditioning, strangulation, and death.¹² Not surprisingly, use of restraints is discouraged in hospitals and is a CMS quality of care indicator.^13,14 Although there is no evidence that restraint use prevents patients from falling,^15,16 there is a perception among both health professionals and patients that restraints reduce the risk of falling, and they are often used as a “last resort” method of fall prevention.^17-19

The aim of this longitudinal study was to determine whether this payment change was associated with changes in short-, intermediate-, and long-term rates of falls, injurious falls, and physical restraint use in acute care hospitals. The CMS has included fall-related hip fracture in newer value-based purchasing programs by adding Patient Safety Indictor (PSI) 90 to both the HACs Reduction Program (HACRP)²⁰ and the Hospital Value-Based Purchasing (VBP)²¹ in FY2015. However, the HACs Initiative remains the only Medicare value program that directly penalizes all injurious inpatient falls.

Study Units

As previously described,²² the National Database of Nursing Quality Indicators (NDNQI) is a data collection project initiated by the American Nurses Association (ANA). The NDNQI provides national comparative data at the unit and facility levels on nursing-sensitive indicators endorsed by the National Quality Forum. More than 2,000 hospitals voluntarily participate in the NDNQI, including virtually all ANA Magnet-recognized hospitals, and more than 90% of nursing units participate in the fall measures (NDNQI, personal communication). At the start of study data collection, the project was administered by the School of Nursing at the University of Kansas Medical Center. In 2014, the ownership of the NDNQI was transferred from the ANA to Press Ganey Associates, Inc. In addition to standardized data on unit, facility, and staffing characteristics, the NDNQI member hospitals can elect to submit monthly data on falls and quarterly data on physical restraint use prevalence.

We examined the data collected from adult medical, medical-surgical, and surgical units in United States acute care hospitals that elected to participate in the fall and physical restraint use data collection within the NDNQI for the 27 months before and the 87 months after the implementation of the CMS rule change. Eligible units contributed at least one fall and physical restraint use data point during both the 27 months preceding October 1, 2008, and the 87 months immediately after. The Institutional Review Board at the University of Kansas Medical Center reviewed and approved the study before its implementation.

Endpoints

Fall Events

The NDNQI defines a patient fall as an unplanned descent to the floor, regardless of whether the fall results in injury and regardless of whether the patient was assisted to the floor by a member of the hospital staff. Events in which a patient lands on a surface where one would not expect to find a patient (eg, on a mat next to a low bed) are also counted as falls.

Using internal data sources (eg, medical records, incident reports), participating hospitals report the number of inpatient falls each month to the NDNQI. We analyzed the falls data for the period July 1, 2006, through December 31, 2015. Thus, each unit could contribute 114 months (27 months before the rule change and 87 months after the rule change) of falls data.

Hospitals classify the injury level of each fall as none, minor (resulting in bruise, pain, abrasion, wound cleaning, or limb elevation, or in the use of ice, dressing, or topical medication), moderate (resulting in suturing, splinting, muscle or joint strain, or application of steri-strips or skin glue), major (resulting in surgery, casting, traction, any type of fracture, consultation for neurological or internal injury, or receipt of blood products for patients with coagulopathy), or death (resulting from injuries sustained from falling). For this study, a fall resulting in any injury (including minor) was considered as an injurious fall. The NDNQI data have been validated for falls and fall injury.^23,24

Based on patient counts from unit censuses and/or internal data on actual patient hours on the unit, hospitals also report to the NDNQI the monthly number of patient days for each unit for which falls data are reported. The NDNQI uses these data to calculate each unit’s total and injurious fall rate per 1,000 patient days.

Physical Restraint Use

The NDNQI follows the CMS definition of restraint, which is “any manual method, physical or mechanical device, material, or equipment that immobilizes or reduces the ability of a patient to move his or her arms, legs, body, or head freely”.¹³ The NDNQI restraint use data are collected quarterly. Participating hospitals choose one day each quarter to conduct a restraint use prevalence survey on participating units. On the selected day, designated RNs within these hospitals visually assess each patient on the unit for restraint use. Based on this survey, hospitals report to the NDNQI the total count of patients surveyed and whether each was restrained. For restrained patients, hospitals also report the type of restraint as limb, vest, or other (eg, four side rails, net beds, mitts not attached to the bed).

We analyzed the restraint use data for the period October 1, 2006, through December 31, 2015. Thus, 37 quarters of data (eight pre- and 29 postrule change) were available. For this study, we computed for each unit the quarterly proportion of surveyed patients who were physically restrained by dividing the total count of restrained patients (regardless of the type of restraint) on the day of the survey by the total count of surveyed patients.

Covariates

Unit- and facility-level covariates were included in several model specifications to determine whether patient or facility characteristics affected the results. The unit-level covariates included the type of nursing unit (medical, medical and surgical, or surgical), monthly rates of total nursing hours per patient day, and nursing skill mix (percent registered nurses/total nursing personnel). The three facility-level variables included urban–rural location (defined as metropolitan [located in an area containing an urban core with a population of at least 50,000], micropolitan [located in an area containing an urban core with a population of 10,000-49,999], or neither), bed size (<300 beds or ≥300 beds), and teaching status (academic health center, major teaching hospital, or nonteaching hospital).

Because larger, academically affiliated hospitals are overrepresented in the NDNQI, we conducted stratified analyses of these variables to explore how change in the rates of falls and restraint use in the entire sample might differ between hospitals according to bed size (<300 beds, ≥300 beds) and teaching status (nonteaching versus teaching and academic health center).

Statistical Methods

We compared the mean annual rates of change in falls, injurious falls, and physical restraint use prevalence during the two years before the HACs Initiative went into effect (October 2006-September 2008) with the mean annual rates of change following the implementation of the payment rule. Short-term (one-year) change was the slope from October 2008 to September 2009, intermediate-term (four-year) change was the slope from October 2008 to September 2012, and long-term (seven-year) change was the slope from October 2008 to September 2015.

Monthly rates of falls and injurious falls over the 114-month period were modeled using negative binomial models with a random intercept to account for heterogeneity between units. Each base mean model included the preimplementation intercept and slope (over time), the postimplementation intercept, and slope (both linear and quadratic). We also fit the models that included the terms in the base model and facility-level covariates, unit-level covariates, both individually and combined. All models included terms for seasonality.

Quarterly prevalence rates of restraint use over the 37 quarters were modeled using beta-binomial models with a random intercept to account for heterogeneity between units. Each base mean model included the preimplementation intercept and slope (over time), the postimplementation intercept, and slope (both linear and quadratic). Similar to the one specified for falls, models were also fitted that included facility- and unit-level covariates as described above.

To adjust for multiple comparisons of the three postimplementation slopes, all confidence intervals were Bonferroni corrected.

Nursing Units

We included nursing units with one or more months of falls data and one or more quarters of restraint use data before and after the rule change. Of the 11,117 nursing units that submitted data to the NDNQI, 2,862 units (983 medical, 1,219 medical-surgical, and 660 surgical) with the requisite demographic, falls, and restraint use data were considered for inclusion in the study. The characteristics of the nursing units (ie, the type of unit, total nursing hours per patient day, and nursing skill mix) and hospitals (ie, location, bed size, teaching status) included in the study were similar to those of the overall NDNQI member units.

Baseline Characteristics

In the first study month (July 2006), 1,941 sample nursing units reported 5,101 falls during 1,401,652 patient-days of observation. Of these, 1,502 (29%) resulted in injury (1,281 minor, 144 moderate, 75 major, and two deaths). Across falls, the median (interquartile range [IQR]) patient age was 70 (55-80) years, with males accounting for 51% of falls. Most of the falls, 4,328 (85%), were documented as unassisted. A total of 209 (4%) falls occurred while physical restraints were in use.

In the first quarterly restraint use prevalence survey (October 2006), the 829 participating nursing units surveyed 19,979 patients (median [IQR] = 23 [20-23] patients per nursing unit). The median (IQR) age was 66 (51-78) years, and 54% of them were females. At the time of the survey, restraints were in use for 326 (1.6%) patients. Restrained patients were older than unrestrained patients (median age: 78 vs 65 years) and more likely to be male (56% vs 46%). Limb restraints were used for 139 patients, vest restraints for 66, both limb and vest restraints for 24, and other restraint types were used for 113 patients (including 11 in limb restraints and 5 in limb and vest restraints).

Change in Endpoints after Implementation of the HACs Initiative

Stratified Analysis

At baseline, fall rates and restraint use prevalence were slightly higher, whereas the rate of injurious falls was slightly lower, among teaching and academic medical centers compared to those in nonteaching hospitals. Declines in falls rate and restraint use prevalence were higher in teaching hospitals than in nonteaching hospitals (data not included).

We examined the rates of falls and fall injuries among 2,862 hospital units before and after the implementation of the HACs Initiative. Implementation of the CMS payment change was associated with a modest improvement in the rate of decrease for falls; a statistically significant effect on the rate of decrease for injurious falls was detectable only at seven years postchange. Fall reductions were the greatest among teaching and larger hospitals. These findings are consistent with our previous analysis of NDNQI data that found no short-term effect of the rule change on the rate of injurious falls.²⁵

We found no evidence indicating that restraint use prevalence increased because of this payment change. Physical -restraint use prevalence showed a rapidly decreasing trend before 2008, and although the rate of decline was attenuated seven years after the rule change, the overall physical restraint use prevalence in these units in 2015 was less than half of that in 2006. Unlike falls, the steepest declines in restraint use prevalence occurred in smaller hospitals.

The CMS decision to include falls with injury among the “reasonably preventable” HACs was controversial.¹¹ Inpatient falls are largely attributable to individual patient risk factors and are unusual among HACs in the extent to which patient behavior plays a role in their occurrence. Although hospital fall prevention guidelines have been published, only a few controlled trials have been conducted, with little evidence supporting the recommendations.^1,26 A quantitative review found no evidence of benefit in published hospital fall prevention studies using concurrent controls (internal rate of return = 0.92; 95% CI: 0.65-1.30),²⁶ and a recent, well-executed, cluster randomized trial of multifactorial fall prevention interventions found no change in fall rates compared with controls.²⁷ Current hospital fall prevention guidelines are limited to unproven and time-consuming nursing-level (eg, toileting schedules and use of alarms) or organizational-level strategies (eg, changing staff attitudes regarding the inevitability of falls or “leadership support”).^1,28

Despite the large sample size and the use of nurse-reported data that include patient falls from all age groups and not subject to bias due to the regulation itself (eg, ICD coding changes), our findings should be interpreted taking into account several limitations.

First, hospitals participating in the NDNQI self-select to participate and are larger and disproportionately urban compared with nonparticipating hospitals.²⁹ Although our findings were unchanged when hospital-level covariates were included in modeling, analyses stratified by teaching status and bed size demonstrated important differences. Larger teaching hospitals experienced greater fall reductions, whereas restraint use prevalence decreased more rapidly in smaller hospitals.

Second, the absence of a control group prevents us from conclusively attributing changes in falls rate and restraint use prevalence to the 2008 CMS payment change.³⁰ Our findings may have been influenced by other policy changes. For example, in October 2014, the CMS implemented the Hospital-Acquired Condition Reduction Program (HACRP)²⁰ and the Hospital Value-Based Purchasing (VBP)²¹ Program. Under these programs, falls with hip fractures were an indicator that could alter hospital payment.

Third, we did not ascertain the use of all available fall prevention measures such as companions, bed rails, very low beds, bed alarms, and restricted activity.³¹ Nor could the study address changes in patient functional status or discharge location. In a before- and after-study of four hospitals in a single hospital system, we found that bed alarm use increased, restraint orders decreased, and the use of room change or sitters remained stable after the implementation of the CMS payment.³²

Nevertheless, we believe that these findings are consistent with the hypothesis that the HACs Initiative increased the cost of patient falls to hospitals, and, in response, some hospitals were able to modestly reduce the rate of falls. We found no evidence that physical restraint use prevalence increased.

In summary, our findings suggest modest impact of the HACs Initiative on falls and injurious falls, but no unintended impact on restraint use. These results highlight the importance of ensuring that pay-for-performance initiatives target outcomes where there are evidence-based approaches to prevention. The creation or identification of prevention tools and guidelines does not make an outcome preventable. Despite interval improvement in these self-selected hospital units in fall rates and physical restraint use prevalence, falls remain a difficult patient safety problem for hospitals, and further research is required to develop cost-effective, generalizable strategies for their prevention.

Accidental falls are among the most common incidents reported in hospitals, complicating approximately 2% of hospital stays.^1-3 Approximately 25% of falls in hospitalized patients result in injury, and 2% involve fractures.⁴ Substantial costs are associated with falls, including patient care costs associated with increased length of stay and liability.^5-7

Beginning October 1, 2008, the Centers for Medicare & Medicaid Services (CMS) stopped reimbursing hospitals for the additional care associated with eight hospital-acquired conditions (HACs), including serious fall-related injury, which were believed to be “reasonably preventable.”^8,9 Before this change, hospitals recovered the costs of these “never events” by assigning a higher level MS-DRG (Medicare Severity Diagnosis-Related Group) code for patients experiencing such an event. This is no longer allowed under the revised CMS Prospective Payment System rules.

Although the financial penalty for iatrogenic injury was modest, the payment change placed pressure on hospital staff to decrease falls, and some nurses reported changing practice to be more restrictive of patient mobility.¹⁰ Increased use of physical restraints is a potential unintended consequence of this rule change.¹¹ Restraints are known to cause agitation, delirium, decubiti, deconditioning, strangulation, and death.¹² Not surprisingly, use of restraints is discouraged in hospitals and is a CMS quality of care indicator.^13,14 Although there is no evidence that restraint use prevents patients from falling,^15,16 there is a perception among both health professionals and patients that restraints reduce the risk of falling, and they are often used as a “last resort” method of fall prevention.^17-19

The aim of this longitudinal study was to determine whether this payment change was associated with changes in short-, intermediate-, and long-term rates of falls, injurious falls, and physical restraint use in acute care hospitals. The CMS has included fall-related hip fracture in newer value-based purchasing programs by adding Patient Safety Indictor (PSI) 90 to both the HACs Reduction Program (HACRP)²⁰ and the Hospital Value-Based Purchasing (VBP)²¹ in FY2015. However, the HACs Initiative remains the only Medicare value program that directly penalizes all injurious inpatient falls.

Study Units

As previously described,²² the National Database of Nursing Quality Indicators (NDNQI) is a data collection project initiated by the American Nurses Association (ANA). The NDNQI provides national comparative data at the unit and facility levels on nursing-sensitive indicators endorsed by the National Quality Forum. More than 2,000 hospitals voluntarily participate in the NDNQI, including virtually all ANA Magnet-recognized hospitals, and more than 90% of nursing units participate in the fall measures (NDNQI, personal communication). At the start of study data collection, the project was administered by the School of Nursing at the University of Kansas Medical Center. In 2014, the ownership of the NDNQI was transferred from the ANA to Press Ganey Associates, Inc. In addition to standardized data on unit, facility, and staffing characteristics, the NDNQI member hospitals can elect to submit monthly data on falls and quarterly data on physical restraint use prevalence.

We examined the data collected from adult medical, medical-surgical, and surgical units in United States acute care hospitals that elected to participate in the fall and physical restraint use data collection within the NDNQI for the 27 months before and the 87 months after the implementation of the CMS rule change. Eligible units contributed at least one fall and physical restraint use data point during both the 27 months preceding October 1, 2008, and the 87 months immediately after. The Institutional Review Board at the University of Kansas Medical Center reviewed and approved the study before its implementation.

Endpoints

Fall Events

The NDNQI defines a patient fall as an unplanned descent to the floor, regardless of whether the fall results in injury and regardless of whether the patient was assisted to the floor by a member of the hospital staff. Events in which a patient lands on a surface where one would not expect to find a patient (eg, on a mat next to a low bed) are also counted as falls.

Using internal data sources (eg, medical records, incident reports), participating hospitals report the number of inpatient falls each month to the NDNQI. We analyzed the falls data for the period July 1, 2006, through December 31, 2015. Thus, each unit could contribute 114 months (27 months before the rule change and 87 months after the rule change) of falls data.

Hospitals classify the injury level of each fall as none, minor (resulting in bruise, pain, abrasion, wound cleaning, or limb elevation, or in the use of ice, dressing, or topical medication), moderate (resulting in suturing, splinting, muscle or joint strain, or application of steri-strips or skin glue), major (resulting in surgery, casting, traction, any type of fracture, consultation for neurological or internal injury, or receipt of blood products for patients with coagulopathy), or death (resulting from injuries sustained from falling). For this study, a fall resulting in any injury (including minor) was considered as an injurious fall. The NDNQI data have been validated for falls and fall injury.^23,24

Based on patient counts from unit censuses and/or internal data on actual patient hours on the unit, hospitals also report to the NDNQI the monthly number of patient days for each unit for which falls data are reported. The NDNQI uses these data to calculate each unit’s total and injurious fall rate per 1,000 patient days.

Physical Restraint Use

The NDNQI follows the CMS definition of restraint, which is “any manual method, physical or mechanical device, material, or equipment that immobilizes or reduces the ability of a patient to move his or her arms, legs, body, or head freely”.¹³ The NDNQI restraint use data are collected quarterly. Participating hospitals choose one day each quarter to conduct a restraint use prevalence survey on participating units. On the selected day, designated RNs within these hospitals visually assess each patient on the unit for restraint use. Based on this survey, hospitals report to the NDNQI the total count of patients surveyed and whether each was restrained. For restrained patients, hospitals also report the type of restraint as limb, vest, or other (eg, four side rails, net beds, mitts not attached to the bed).

We analyzed the restraint use data for the period October 1, 2006, through December 31, 2015. Thus, 37 quarters of data (eight pre- and 29 postrule change) were available. For this study, we computed for each unit the quarterly proportion of surveyed patients who were physically restrained by dividing the total count of restrained patients (regardless of the type of restraint) on the day of the survey by the total count of surveyed patients.

Covariates

Unit- and facility-level covariates were included in several model specifications to determine whether patient or facility characteristics affected the results. The unit-level covariates included the type of nursing unit (medical, medical and surgical, or surgical), monthly rates of total nursing hours per patient day, and nursing skill mix (percent registered nurses/total nursing personnel). The three facility-level variables included urban–rural location (defined as metropolitan [located in an area containing an urban core with a population of at least 50,000], micropolitan [located in an area containing an urban core with a population of 10,000-49,999], or neither), bed size (<300 beds or ≥300 beds), and teaching status (academic health center, major teaching hospital, or nonteaching hospital).

Because larger, academically affiliated hospitals are overrepresented in the NDNQI, we conducted stratified analyses of these variables to explore how change in the rates of falls and restraint use in the entire sample might differ between hospitals according to bed size (<300 beds, ≥300 beds) and teaching status (nonteaching versus teaching and academic health center).

Statistical Methods

We compared the mean annual rates of change in falls, injurious falls, and physical restraint use prevalence during the two years before the HACs Initiative went into effect (October 2006-September 2008) with the mean annual rates of change following the implementation of the payment rule. Short-term (one-year) change was the slope from October 2008 to September 2009, intermediate-term (four-year) change was the slope from October 2008 to September 2012, and long-term (seven-year) change was the slope from October 2008 to September 2015.

Monthly rates of falls and injurious falls over the 114-month period were modeled using negative binomial models with a random intercept to account for heterogeneity between units. Each base mean model included the preimplementation intercept and slope (over time), the postimplementation intercept, and slope (both linear and quadratic). We also fit the models that included the terms in the base model and facility-level covariates, unit-level covariates, both individually and combined. All models included terms for seasonality.

Quarterly prevalence rates of restraint use over the 37 quarters were modeled using beta-binomial models with a random intercept to account for heterogeneity between units. Each base mean model included the preimplementation intercept and slope (over time), the postimplementation intercept, and slope (both linear and quadratic). Similar to the one specified for falls, models were also fitted that included facility- and unit-level covariates as described above.

To adjust for multiple comparisons of the three postimplementation slopes, all confidence intervals were Bonferroni corrected.

Nursing Units

We included nursing units with one or more months of falls data and one or more quarters of restraint use data before and after the rule change. Of the 11,117 nursing units that submitted data to the NDNQI, 2,862 units (983 medical, 1,219 medical-surgical, and 660 surgical) with the requisite demographic, falls, and restraint use data were considered for inclusion in the study. The characteristics of the nursing units (ie, the type of unit, total nursing hours per patient day, and nursing skill mix) and hospitals (ie, location, bed size, teaching status) included in the study were similar to those of the overall NDNQI member units.

Baseline Characteristics

In the first study month (July 2006), 1,941 sample nursing units reported 5,101 falls during 1,401,652 patient-days of observation. Of these, 1,502 (29%) resulted in injury (1,281 minor, 144 moderate, 75 major, and two deaths). Across falls, the median (interquartile range [IQR]) patient age was 70 (55-80) years, with males accounting for 51% of falls. Most of the falls, 4,328 (85%), were documented as unassisted. A total of 209 (4%) falls occurred while physical restraints were in use.

In the first quarterly restraint use prevalence survey (October 2006), the 829 participating nursing units surveyed 19,979 patients (median [IQR] = 23 [20-23] patients per nursing unit). The median (IQR) age was 66 (51-78) years, and 54% of them were females. At the time of the survey, restraints were in use for 326 (1.6%) patients. Restrained patients were older than unrestrained patients (median age: 78 vs 65 years) and more likely to be male (56% vs 46%). Limb restraints were used for 139 patients, vest restraints for 66, both limb and vest restraints for 24, and other restraint types were used for 113 patients (including 11 in limb restraints and 5 in limb and vest restraints).

Change in Endpoints after Implementation of the HACs Initiative

Stratified Analysis

At baseline, fall rates and restraint use prevalence were slightly higher, whereas the rate of injurious falls was slightly lower, among teaching and academic medical centers compared to those in nonteaching hospitals. Declines in falls rate and restraint use prevalence were higher in teaching hospitals than in nonteaching hospitals (data not included).

We examined the rates of falls and fall injuries among 2,862 hospital units before and after the implementation of the HACs Initiative. Implementation of the CMS payment change was associated with a modest improvement in the rate of decrease for falls; a statistically significant effect on the rate of decrease for injurious falls was detectable only at seven years postchange. Fall reductions were the greatest among teaching and larger hospitals. These findings are consistent with our previous analysis of NDNQI data that found no short-term effect of the rule change on the rate of injurious falls.²⁵

We found no evidence indicating that restraint use prevalence increased because of this payment change. Physical -restraint use prevalence showed a rapidly decreasing trend before 2008, and although the rate of decline was attenuated seven years after the rule change, the overall physical restraint use prevalence in these units in 2015 was less than half of that in 2006. Unlike falls, the steepest declines in restraint use prevalence occurred in smaller hospitals.

The CMS decision to include falls with injury among the “reasonably preventable” HACs was controversial.¹¹ Inpatient falls are largely attributable to individual patient risk factors and are unusual among HACs in the extent to which patient behavior plays a role in their occurrence. Although hospital fall prevention guidelines have been published, only a few controlled trials have been conducted, with little evidence supporting the recommendations.^1,26 A quantitative review found no evidence of benefit in published hospital fall prevention studies using concurrent controls (internal rate of return = 0.92; 95% CI: 0.65-1.30),²⁶ and a recent, well-executed, cluster randomized trial of multifactorial fall prevention interventions found no change in fall rates compared with controls.²⁷ Current hospital fall prevention guidelines are limited to unproven and time-consuming nursing-level (eg, toileting schedules and use of alarms) or organizational-level strategies (eg, changing staff attitudes regarding the inevitability of falls or “leadership support”).^1,28

Despite the large sample size and the use of nurse-reported data that include patient falls from all age groups and not subject to bias due to the regulation itself (eg, ICD coding changes), our findings should be interpreted taking into account several limitations.

First, hospitals participating in the NDNQI self-select to participate and are larger and disproportionately urban compared with nonparticipating hospitals.²⁹ Although our findings were unchanged when hospital-level covariates were included in modeling, analyses stratified by teaching status and bed size demonstrated important differences. Larger teaching hospitals experienced greater fall reductions, whereas restraint use prevalence decreased more rapidly in smaller hospitals.

Second, the absence of a control group prevents us from conclusively attributing changes in falls rate and restraint use prevalence to the 2008 CMS payment change.³⁰ Our findings may have been influenced by other policy changes. For example, in October 2014, the CMS implemented the Hospital-Acquired Condition Reduction Program (HACRP)²⁰ and the Hospital Value-Based Purchasing (VBP)²¹ Program. Under these programs, falls with hip fractures were an indicator that could alter hospital payment.

Third, we did not ascertain the use of all available fall prevention measures such as companions, bed rails, very low beds, bed alarms, and restricted activity.³¹ Nor could the study address changes in patient functional status or discharge location. In a before- and after-study of four hospitals in a single hospital system, we found that bed alarm use increased, restraint orders decreased, and the use of room change or sitters remained stable after the implementation of the CMS payment.³²

Nevertheless, we believe that these findings are consistent with the hypothesis that the HACs Initiative increased the cost of patient falls to hospitals, and, in response, some hospitals were able to modestly reduce the rate of falls. We found no evidence that physical restraint use prevalence increased.

In summary, our findings suggest modest impact of the HACs Initiative on falls and injurious falls, but no unintended impact on restraint use. These results highlight the importance of ensuring that pay-for-performance initiatives target outcomes where there are evidence-based approaches to prevention. The creation or identification of prevention tools and guidelines does not make an outcome preventable. Despite interval improvement in these self-selected hospital units in fall rates and physical restraint use prevalence, falls remain a difficult patient safety problem for hospitals, and further research is required to develop cost-effective, generalizable strategies for their prevention.

Accidental falls are among the most common incidents reported in hospitals, complicating approximately 2% of hospital stays.^1-3 Approximately 25% of falls in hospitalized patients result in injury, and 2% involve fractures.⁴ Substantial costs are associated with falls, including patient care costs associated with increased length of stay and liability.^5-7

Beginning October 1, 2008, the Centers for Medicare & Medicaid Services (CMS) stopped reimbursing hospitals for the additional care associated with eight hospital-acquired conditions (HACs), including serious fall-related injury, which were believed to be “reasonably preventable.”^8,9 Before this change, hospitals recovered the costs of these “never events” by assigning a higher level MS-DRG (Medicare Severity Diagnosis-Related Group) code for patients experiencing such an event. This is no longer allowed under the revised CMS Prospective Payment System rules.

Although the financial penalty for iatrogenic injury was modest, the payment change placed pressure on hospital staff to decrease falls, and some nurses reported changing practice to be more restrictive of patient mobility.¹⁰ Increased use of physical restraints is a potential unintended consequence of this rule change.¹¹ Restraints are known to cause agitation, delirium, decubiti, deconditioning, strangulation, and death.¹² Not surprisingly, use of restraints is discouraged in hospitals and is a CMS quality of care indicator.^13,14 Although there is no evidence that restraint use prevents patients from falling,^15,16 there is a perception among both health professionals and patients that restraints reduce the risk of falling, and they are often used as a “last resort” method of fall prevention.^17-19

The aim of this longitudinal study was to determine whether this payment change was associated with changes in short-, intermediate-, and long-term rates of falls, injurious falls, and physical restraint use in acute care hospitals. The CMS has included fall-related hip fracture in newer value-based purchasing programs by adding Patient Safety Indictor (PSI) 90 to both the HACs Reduction Program (HACRP)²⁰ and the Hospital Value-Based Purchasing (VBP)²¹ in FY2015. However, the HACs Initiative remains the only Medicare value program that directly penalizes all injurious inpatient falls.

Study Units

As previously described,²² the National Database of Nursing Quality Indicators (NDNQI) is a data collection project initiated by the American Nurses Association (ANA). The NDNQI provides national comparative data at the unit and facility levels on nursing-sensitive indicators endorsed by the National Quality Forum. More than 2,000 hospitals voluntarily participate in the NDNQI, including virtually all ANA Magnet-recognized hospitals, and more than 90% of nursing units participate in the fall measures (NDNQI, personal communication). At the start of study data collection, the project was administered by the School of Nursing at the University of Kansas Medical Center. In 2014, the ownership of the NDNQI was transferred from the ANA to Press Ganey Associates, Inc. In addition to standardized data on unit, facility, and staffing characteristics, the NDNQI member hospitals can elect to submit monthly data on falls and quarterly data on physical restraint use prevalence.

We examined the data collected from adult medical, medical-surgical, and surgical units in United States acute care hospitals that elected to participate in the fall and physical restraint use data collection within the NDNQI for the 27 months before and the 87 months after the implementation of the CMS rule change. Eligible units contributed at least one fall and physical restraint use data point during both the 27 months preceding October 1, 2008, and the 87 months immediately after. The Institutional Review Board at the University of Kansas Medical Center reviewed and approved the study before its implementation.

Endpoints

Fall Events

The NDNQI defines a patient fall as an unplanned descent to the floor, regardless of whether the fall results in injury and regardless of whether the patient was assisted to the floor by a member of the hospital staff. Events in which a patient lands on a surface where one would not expect to find a patient (eg, on a mat next to a low bed) are also counted as falls.

Using internal data sources (eg, medical records, incident reports), participating hospitals report the number of inpatient falls each month to the NDNQI. We analyzed the falls data for the period July 1, 2006, through December 31, 2015. Thus, each unit could contribute 114 months (27 months before the rule change and 87 months after the rule change) of falls data.

Hospitals classify the injury level of each fall as none, minor (resulting in bruise, pain, abrasion, wound cleaning, or limb elevation, or in the use of ice, dressing, or topical medication), moderate (resulting in suturing, splinting, muscle or joint strain, or application of steri-strips or skin glue), major (resulting in surgery, casting, traction, any type of fracture, consultation for neurological or internal injury, or receipt of blood products for patients with coagulopathy), or death (resulting from injuries sustained from falling). For this study, a fall resulting in any injury (including minor) was considered as an injurious fall. The NDNQI data have been validated for falls and fall injury.^23,24

Based on patient counts from unit censuses and/or internal data on actual patient hours on the unit, hospitals also report to the NDNQI the monthly number of patient days for each unit for which falls data are reported. The NDNQI uses these data to calculate each unit’s total and injurious fall rate per 1,000 patient days.

Physical Restraint Use

The NDNQI follows the CMS definition of restraint, which is “any manual method, physical or mechanical device, material, or equipment that immobilizes or reduces the ability of a patient to move his or her arms, legs, body, or head freely”.¹³ The NDNQI restraint use data are collected quarterly. Participating hospitals choose one day each quarter to conduct a restraint use prevalence survey on participating units. On the selected day, designated RNs within these hospitals visually assess each patient on the unit for restraint use. Based on this survey, hospitals report to the NDNQI the total count of patients surveyed and whether each was restrained. For restrained patients, hospitals also report the type of restraint as limb, vest, or other (eg, four side rails, net beds, mitts not attached to the bed).

We analyzed the restraint use data for the period October 1, 2006, through December 31, 2015. Thus, 37 quarters of data (eight pre- and 29 postrule change) were available. For this study, we computed for each unit the quarterly proportion of surveyed patients who were physically restrained by dividing the total count of restrained patients (regardless of the type of restraint) on the day of the survey by the total count of surveyed patients.

Covariates

Unit- and facility-level covariates were included in several model specifications to determine whether patient or facility characteristics affected the results. The unit-level covariates included the type of nursing unit (medical, medical and surgical, or surgical), monthly rates of total nursing hours per patient day, and nursing skill mix (percent registered nurses/total nursing personnel). The three facility-level variables included urban–rural location (defined as metropolitan [located in an area containing an urban core with a population of at least 50,000], micropolitan [located in an area containing an urban core with a population of 10,000-49,999], or neither), bed size (<300 beds or ≥300 beds), and teaching status (academic health center, major teaching hospital, or nonteaching hospital).

Because larger, academically affiliated hospitals are overrepresented in the NDNQI, we conducted stratified analyses of these variables to explore how change in the rates of falls and restraint use in the entire sample might differ between hospitals according to bed size (<300 beds, ≥300 beds) and teaching status (nonteaching versus teaching and academic health center).

Statistical Methods

We compared the mean annual rates of change in falls, injurious falls, and physical restraint use prevalence during the two years before the HACs Initiative went into effect (October 2006-September 2008) with the mean annual rates of change following the implementation of the payment rule. Short-term (one-year) change was the slope from October 2008 to September 2009, intermediate-term (four-year) change was the slope from October 2008 to September 2012, and long-term (seven-year) change was the slope from October 2008 to September 2015.

Monthly rates of falls and injurious falls over the 114-month period were modeled using negative binomial models with a random intercept to account for heterogeneity between units. Each base mean model included the preimplementation intercept and slope (over time), the postimplementation intercept, and slope (both linear and quadratic). We also fit the models that included the terms in the base model and facility-level covariates, unit-level covariates, both individually and combined. All models included terms for seasonality.

Quarterly prevalence rates of restraint use over the 37 quarters were modeled using beta-binomial models with a random intercept to account for heterogeneity between units. Each base mean model included the preimplementation intercept and slope (over time), the postimplementation intercept, and slope (both linear and quadratic). Similar to the one specified for falls, models were also fitted that included facility- and unit-level covariates as described above.

To adjust for multiple comparisons of the three postimplementation slopes, all confidence intervals were Bonferroni corrected.

Nursing Units

We included nursing units with one or more months of falls data and one or more quarters of restraint use data before and after the rule change. Of the 11,117 nursing units that submitted data to the NDNQI, 2,862 units (983 medical, 1,219 medical-surgical, and 660 surgical) with the requisite demographic, falls, and restraint use data were considered for inclusion in the study. The characteristics of the nursing units (ie, the type of unit, total nursing hours per patient day, and nursing skill mix) and hospitals (ie, location, bed size, teaching status) included in the study were similar to those of the overall NDNQI member units.

Baseline Characteristics

In the first study month (July 2006), 1,941 sample nursing units reported 5,101 falls during 1,401,652 patient-days of observation. Of these, 1,502 (29%) resulted in injury (1,281 minor, 144 moderate, 75 major, and two deaths). Across falls, the median (interquartile range [IQR]) patient age was 70 (55-80) years, with males accounting for 51% of falls. Most of the falls, 4,328 (85%), were documented as unassisted. A total of 209 (4%) falls occurred while physical restraints were in use.

In the first quarterly restraint use prevalence survey (October 2006), the 829 participating nursing units surveyed 19,979 patients (median [IQR] = 23 [20-23] patients per nursing unit). The median (IQR) age was 66 (51-78) years, and 54% of them were females. At the time of the survey, restraints were in use for 326 (1.6%) patients. Restrained patients were older than unrestrained patients (median age: 78 vs 65 years) and more likely to be male (56% vs 46%). Limb restraints were used for 139 patients, vest restraints for 66, both limb and vest restraints for 24, and other restraint types were used for 113 patients (including 11 in limb restraints and 5 in limb and vest restraints).

Change in Endpoints after Implementation of the HACs Initiative

Stratified Analysis

At baseline, fall rates and restraint use prevalence were slightly higher, whereas the rate of injurious falls was slightly lower, among teaching and academic medical centers compared to those in nonteaching hospitals. Declines in falls rate and restraint use prevalence were higher in teaching hospitals than in nonteaching hospitals (data not included).

We examined the rates of falls and fall injuries among 2,862 hospital units before and after the implementation of the HACs Initiative. Implementation of the CMS payment change was associated with a modest improvement in the rate of decrease for falls; a statistically significant effect on the rate of decrease for injurious falls was detectable only at seven years postchange. Fall reductions were the greatest among teaching and larger hospitals. These findings are consistent with our previous analysis of NDNQI data that found no short-term effect of the rule change on the rate of injurious falls.²⁵

We found no evidence indicating that restraint use prevalence increased because of this payment change. Physical -restraint use prevalence showed a rapidly decreasing trend before 2008, and although the rate of decline was attenuated seven years after the rule change, the overall physical restraint use prevalence in these units in 2015 was less than half of that in 2006. Unlike falls, the steepest declines in restraint use prevalence occurred in smaller hospitals.

The CMS decision to include falls with injury among the “reasonably preventable” HACs was controversial.¹¹ Inpatient falls are largely attributable to individual patient risk factors and are unusual among HACs in the extent to which patient behavior plays a role in their occurrence. Although hospital fall prevention guidelines have been published, only a few controlled trials have been conducted, with little evidence supporting the recommendations.^1,26 A quantitative review found no evidence of benefit in published hospital fall prevention studies using concurrent controls (internal rate of return = 0.92; 95% CI: 0.65-1.30),²⁶ and a recent, well-executed, cluster randomized trial of multifactorial fall prevention interventions found no change in fall rates compared with controls.²⁷ Current hospital fall prevention guidelines are limited to unproven and time-consuming nursing-level (eg, toileting schedules and use of alarms) or organizational-level strategies (eg, changing staff attitudes regarding the inevitability of falls or “leadership support”).^1,28

Despite the large sample size and the use of nurse-reported data that include patient falls from all age groups and not subject to bias due to the regulation itself (eg, ICD coding changes), our findings should be interpreted taking into account several limitations.

First, hospitals participating in the NDNQI self-select to participate and are larger and disproportionately urban compared with nonparticipating hospitals.²⁹ Although our findings were unchanged when hospital-level covariates were included in modeling, analyses stratified by teaching status and bed size demonstrated important differences. Larger teaching hospitals experienced greater fall reductions, whereas restraint use prevalence decreased more rapidly in smaller hospitals.

Second, the absence of a control group prevents us from conclusively attributing changes in falls rate and restraint use prevalence to the 2008 CMS payment change.³⁰ Our findings may have been influenced by other policy changes. For example, in October 2014, the CMS implemented the Hospital-Acquired Condition Reduction Program (HACRP)²⁰ and the Hospital Value-Based Purchasing (VBP)²¹ Program. Under these programs, falls with hip fractures were an indicator that could alter hospital payment.

Third, we did not ascertain the use of all available fall prevention measures such as companions, bed rails, very low beds, bed alarms, and restricted activity.³¹ Nor could the study address changes in patient functional status or discharge location. In a before- and after-study of four hospitals in a single hospital system, we found that bed alarm use increased, restraint orders decreased, and the use of room change or sitters remained stable after the implementation of the CMS payment.³²

Nevertheless, we believe that these findings are consistent with the hypothesis that the HACs Initiative increased the cost of patient falls to hospitals, and, in response, some hospitals were able to modestly reduce the rate of falls. We found no evidence that physical restraint use prevalence increased.

In summary, our findings suggest modest impact of the HACs Initiative on falls and injurious falls, but no unintended impact on restraint use. These results highlight the importance of ensuring that pay-for-performance initiatives target outcomes where there are evidence-based approaches to prevention. The creation or identification of prevention tools and guidelines does not make an outcome preventable. Despite interval improvement in these self-selected hospital units in fall rates and physical restraint use prevalence, falls remain a difficult patient safety problem for hospitals, and further research is required to develop cost-effective, generalizable strategies for their prevention.

Chronic obstructive pulmonary disease (COPD) is recognized as the third leading cause of death nationally. Globally, it has been estimated that 10% of the population has COPD; in the United States, approximately 15 million people are affected.^1,2 The annual estimated cost of COPD management in the United States is approximately $50 billion, one-third of which is directly related to inpatient hospitalization for COPD exacerbation.^3,4,5 The 30-day readmission rate after hospitalization for acute exacerbation of COPD (AECOPD) is approximately 21% with an approximate cost of $13 billion per year.^6,7 To reduce the cost and to improve patient outcomes, the Centers for Medicare and Medicaid Services (CMS) has designed several interventions with little effect.⁸

In October 2012, the Affordable Care Act added section 1886(q) to the Social Security Act and established the Hospital Readmission Reduction Program (HRRP), an initiative to decrease hospitalization costs by penalizing hospitals with high 30-day readmission rates. Under this program, hospitals received up to 3% penalty for excess readmissions after the index hospitalization with acute myocardial infarction (AMI), heart failure (HF), and pneumonia.^9-11 Hospitals are penalized if their annual readmission rates are significantly above the average national readmission rate. In 2014, the HRRP was extended to include AECOPD for the FY 2015.

Since the implementation of readmission penalties, data have shown a significant decrease in the 30-day readmission rates for all conditions.^12,13 On the other hand, studies have suggested that, at least for some conditions, the decrease in the 30-day readmission rate is associated with higher adverse patients outcomes, including higher mortality.^14,15 However, whether a decrease in readmission rates after an AECOPD hospitalization is associated with a concomitant increase in mortality has not been examined. Therefore, our objective was to examine the association of the 30-day risk-adjusted hospital readmission rate with the 30-day risk-adjusted hospital mortality rate for patients discharged with a diagnosis of AECOPD.

Data Sources

Publicly available data from three sources were used. The all-cause 30-day risk-standardized readmission rate (RSRR) and the 30-day risk-standardized mortality rate (RSMR) of each hospital for patients with AECOPD were obtained from the Hospital Compare database; a database maintained by the CMS.^16,17 In 2014, the CMS started reporting three-year running average of 30-day mortality and readmission rate data on hospitals for AECOPD hospitalizations; the data start date was July 2010.^18-22 We examined data from the FY 2010-2013 to 2014-2017 cycles on readmission and mortality reported by the CMS; this included data before and after the implementation of penalties.

Hospital characteristics were also obtained from the CMS website. Hospital ownership was defined as government (owned by Federal or state), for-profit (owned by physicians or another proprietary), or nonprofit (owned by a nonprofit organization such as a church). A hospital was considered as a teaching hospital if it obtained graduate medical education funding from the CMS.

Data on local population characteristics according to ZIP codes were obtained from the 2010 decennial census and the American Community Survey five-year (2009-2013) data files available at the United States Census Bureau website.²³ For each ZIP code, we obtained data on the total population, percentage of African Americans in the population, median income, poverty level, and insurance status.

We used Hospital service area (HSA) information obtained from the Dartmouth Atlas of Health Care crosswalk files to link local population characteristics to hospitals. The Dartmouth Atlas defined 3,436 HSAs by assigning the ZIP codes to the hospital area where the greatest proportion of their Medicare residents was hospitalized.^24,25

Hospital Compare data and Census Bureau population data were matched to the HSAs from the Dartmouth Atlas of Healthcare data at the ZIP code level. First, the ZIP code-level data from the Census Bureau were pooled by the HSAs obtained from the Dartmouth Atlas of Healthcare, followed by matching these data by the HSAs to the Hospital Compare data. Merging data from these three sources generated a dataset that contained information about readmission and mortality rates from a particular hospital and the population characteristics of the local healthcare market or neighborhood. Our final dataset included hospitals that had readmission and mortality information available at the Hospital Compare website and were included in the crosswalk files of the Dartmouth Atlas of Healthcare.

Statistical Analysis

Data are summarized as mean and standard deviation (SD), median with interquartile range, or frequencies as appropriate. To model the dependence of observations from the same hospital over time, we used mixed linear models with random intercept and slope. A strength of this modeling approach is that it incorporates information from all hospitals even when some hospitals are missing data for some time periods. We reached our final model through stages with increasing model complexity at each stage. In the first stage, we developed an empty model without any covariates to determine the unconditional variance components so that we can partition mortality variance into between- and within-hospital components. In the second stage, we developed an unconditional growth curve model to determine the shape of time trend in mortality over time using linear and quadratic (by including squared time in the model) growth curves. In the third stage, we added baseline readmission rates (from 2010 to 2013) to the model to determine the effect of baseline readmission rate on mortality trends and also examined its interaction with time and squared time. We generated a change in the readmission rate variable by subtracting the last readmission rate from the baseline readmission rate (readmission rate in 2010-2013 − readmission rate in 2014-2017). In the fourth stage, we included this change in readmission rate into the third-stage model to examine how changes in the readmission rate affected the time trends of mortality and also examined its interaction with time and squared time. In the final model, we included the following potential confounding variables to the fourth stage model: African American percentage in the HSA, HSA median income, percentage of people living in poverty in the HSA, median age, ownership of hospital (government, for profit), teaching status (teaching vs nonteaching), and acute care hospital beds in the HSA. Within each stage, the models were compared using the Akaike information criterion (AIC) and the Bayesian information criterion (BIC), and the model with the lowest value of each was moved to the next stage of model development. All analyses were performed in Stata 14.1 for Windows (College Station, Texas).

Of the 3,685 acute care hospitals analyzed in the 2010-2013 data cycle for COPD, the 30-day RSRR was 20.7% (1.28), which decreased to 19.6% (1.11) in 2014-2017 (Table 1). During the same period, the 30-day all-cause RSMR increased from 7.8% (1.03) in 2010-2013 to 8.4% (1.11) in 2014-2017. The partitioning of variance showed that 57% of variation in the mortality rate over the study period was due to between-hospital differences.

The unconditional growth model examining the linear time trend revealed a 0.13% per year (95% CI = 0.12 to 0.14; P < .0001) increase in mortality rate over the five data cycles. When the squared time variable was added to the model to examine a quadratic trend, both time and squared trend were statistically significant (Table 2) and the AIC and BIC were lower for the quadratic model. Thus, the unconditional growth curve model suggested that the mortality trend was nonlinear and the coefficients demonstrated that not only the mortality rate increased, but the rate of change in the mortality rate was also increasing during the study period.

When we added the baseline readmission rate to the abovementioned quadratic growth model, we found an inverse association; each 1% increase in baseline readmission rate was associated with 0.03% (95% CI = −0.05 to −0.005; P = .02) decrease in mortality rate. These findings suggest that hospitals with higher baseline readmission rates also had lower mortality rates. To examine whether the effect of baseline readmission rate on mortality varied over time, we included the interaction term with time in the model and then added the interaction term with squared time. As the AIC and BIC were the lowest for the model with interaction between time and baseline readmission (and not when interaction between squared time and baseline readmission were included), we accepted this model. In this model, although there was no difference in mortality according to readmissions at baseline, each 1% increase in baseline readmission rate was associated with a smaller increase in mortality rate by 0.015% (95% CI = −0.02 to −0.01; P < .0001; Table 2 and Figure 1). These findings suggest that hospitals with higher readmission rates at baseline had a smaller increase in mortality rate during the study period than those with lower readmission rates.

As efforts to decrease readmission rates continue as a part of the HRRP implementation by the CMS, our study shows that among hospitals that discharged patients with AECOPD during 2010-2017, the all-cause 30-day RSRR was decreased, whereas the all-cause 30-day RSMR was increased. Of particular concern is that the rate of increase in mortality also increased. We also found that hospitals with higher readmission rates in 2010-2013 had a lower rate of increase in mortality than hospitals with lower readmission rates. In addition, hospitals that had a larger decrease in readmission rates during the study period had a larger rate of increase in mortality than hospitals with a smaller decrease in readmission rates. Our findings were robust to potential confounders such as hospital characteristics and local population characteristics in which hospitals operate.

Our study findings raise the question whether the implementation of the HRRP resulted in unintentional patient harm by forcing hospitals to make changes that may affect overall patient care. This question is particularly important as other studies on hospitalized patients with HF have found similar results.^13,14 On the other hand, a similar association between readmission and mortality rates has not been observed in patients with pneumonia or AMI.¹⁴ Several possible explanations can be given for the observed discrepancy between the diseases and their effect on the relationship between readmission rate and mortality rate. Both COPD and HF are chronic diseases and characterized by exacerbations, whereas AMI and pneumonia are episodic diseases that are treatable. As the number of patients hospitalized with AECOPD and HF is much larger, hospitals may have a greater focus on reducing the 30-day readmission rates and may attempt to game the process, such as by delaying admissions through the emergency department within the 30-day period or by admitting patients for observation. In fact, a study found a 3% reduction in the within-hospital readmission rate with a concurrent 0.8% increase in observation unit use since the implementation of the HRRP.²⁶ Such approaches to patient care may lead to adverse outcomes.

It is possible that readmissions and mortality act as competing risks and hence hospitals with higher mortality rates are left with fewer patients and thus have fewer readmissions, whereas those with lower mortality rates have more patients and a higher readmission rate.²⁷ Such studies are not possible with hospital-level data, and patient-level studies will be required to examine this competing risk hypothesis. Our study results provide some support to the competing risk hypothesis (hospitals with lower baseline readmission rates had a steeper increase in mortality); however, it is not possible to draw any conclusions due to the high risk of ecological fallacy bias.

This study has important potential implications for healthcare policy, public health, and research. We found that an important national intervention aimed at decreasing readmission rates and improving the quality of care for patients with AECOPD may be associated with higher mortality rates in these patients. There may be a need to redefine measures for determining the performance of an institution. Our study supports research into the underlying mechanisms resulting in an inverse association between readmissions and mortality. In particular, health policy researchers may need to examine how incentives and penalties affect the allocation of resources within hospitals.

This study has several strengths and some potential weaknesses. We used a national dataset to examine readmission and mortality rates that include the majority of hospitals in the United States. We also included data from the local population for each hospital, thus allowing us to examine hospital performance within the context of its target population. One potential limitation is that we used hospital-level data and not patient-level data; however, the readmission penalties are designed for hospitals, which justifies our use of hospital-level data. Furthermore, data were not available for shorter time intervals; data from shorter time intervals may be associated with greater variability. Being an observational study, it is difficult to establish a causal relationship; the longitudinal nature of the study does establish temporality, an important factor in establishing causality.

In conclusion, we found that although the readmission rates decreased, there was an increase in the mortality rate within the 30 days of discharge from the hospital in patients with AECOPD. The rate of increase in mortality was higher in hospitals with lower readmission rates than in hospitals with higher readmission rates. Further research for determining the mechanism responsible for this association is needed. Future health policy interventions may need to consider the potential for adverse outcomes.

Chronic obstructive pulmonary disease (COPD) is recognized as the third leading cause of death nationally. Globally, it has been estimated that 10% of the population has COPD; in the United States, approximately 15 million people are affected.^1,2 The annual estimated cost of COPD management in the United States is approximately $50 billion, one-third of which is directly related to inpatient hospitalization for COPD exacerbation.^3,4,5 The 30-day readmission rate after hospitalization for acute exacerbation of COPD (AECOPD) is approximately 21% with an approximate cost of $13 billion per year.^6,7 To reduce the cost and to improve patient outcomes, the Centers for Medicare and Medicaid Services (CMS) has designed several interventions with little effect.⁸

In October 2012, the Affordable Care Act added section 1886(q) to the Social Security Act and established the Hospital Readmission Reduction Program (HRRP), an initiative to decrease hospitalization costs by penalizing hospitals with high 30-day readmission rates. Under this program, hospitals received up to 3% penalty for excess readmissions after the index hospitalization with acute myocardial infarction (AMI), heart failure (HF), and pneumonia.^9-11 Hospitals are penalized if their annual readmission rates are significantly above the average national readmission rate. In 2014, the HRRP was extended to include AECOPD for the FY 2015.

Since the implementation of readmission penalties, data have shown a significant decrease in the 30-day readmission rates for all conditions.^12,13 On the other hand, studies have suggested that, at least for some conditions, the decrease in the 30-day readmission rate is associated with higher adverse patients outcomes, including higher mortality.^14,15 However, whether a decrease in readmission rates after an AECOPD hospitalization is associated with a concomitant increase in mortality has not been examined. Therefore, our objective was to examine the association of the 30-day risk-adjusted hospital readmission rate with the 30-day risk-adjusted hospital mortality rate for patients discharged with a diagnosis of AECOPD.

Data Sources

Publicly available data from three sources were used. The all-cause 30-day risk-standardized readmission rate (RSRR) and the 30-day risk-standardized mortality rate (RSMR) of each hospital for patients with AECOPD were obtained from the Hospital Compare database; a database maintained by the CMS.^16,17 In 2014, the CMS started reporting three-year running average of 30-day mortality and readmission rate data on hospitals for AECOPD hospitalizations; the data start date was July 2010.^18-22 We examined data from the FY 2010-2013 to 2014-2017 cycles on readmission and mortality reported by the CMS; this included data before and after the implementation of penalties.

Hospital characteristics were also obtained from the CMS website. Hospital ownership was defined as government (owned by Federal or state), for-profit (owned by physicians or another proprietary), or nonprofit (owned by a nonprofit organization such as a church). A hospital was considered as a teaching hospital if it obtained graduate medical education funding from the CMS.

Data on local population characteristics according to ZIP codes were obtained from the 2010 decennial census and the American Community Survey five-year (2009-2013) data files available at the United States Census Bureau website.²³ For each ZIP code, we obtained data on the total population, percentage of African Americans in the population, median income, poverty level, and insurance status.

We used Hospital service area (HSA) information obtained from the Dartmouth Atlas of Health Care crosswalk files to link local population characteristics to hospitals. The Dartmouth Atlas defined 3,436 HSAs by assigning the ZIP codes to the hospital area where the greatest proportion of their Medicare residents was hospitalized.^24,25

Hospital Compare data and Census Bureau population data were matched to the HSAs from the Dartmouth Atlas of Healthcare data at the ZIP code level. First, the ZIP code-level data from the Census Bureau were pooled by the HSAs obtained from the Dartmouth Atlas of Healthcare, followed by matching these data by the HSAs to the Hospital Compare data. Merging data from these three sources generated a dataset that contained information about readmission and mortality rates from a particular hospital and the population characteristics of the local healthcare market or neighborhood. Our final dataset included hospitals that had readmission and mortality information available at the Hospital Compare website and were included in the crosswalk files of the Dartmouth Atlas of Healthcare.

Statistical Analysis

Data are summarized as mean and standard deviation (SD), median with interquartile range, or frequencies as appropriate. To model the dependence of observations from the same hospital over time, we used mixed linear models with random intercept and slope. A strength of this modeling approach is that it incorporates information from all hospitals even when some hospitals are missing data for some time periods. We reached our final model through stages with increasing model complexity at each stage. In the first stage, we developed an empty model without any covariates to determine the unconditional variance components so that we can partition mortality variance into between- and within-hospital components. In the second stage, we developed an unconditional growth curve model to determine the shape of time trend in mortality over time using linear and quadratic (by including squared time in the model) growth curves. In the third stage, we added baseline readmission rates (from 2010 to 2013) to the model to determine the effect of baseline readmission rate on mortality trends and also examined its interaction with time and squared time. We generated a change in the readmission rate variable by subtracting the last readmission rate from the baseline readmission rate (readmission rate in 2010-2013 − readmission rate in 2014-2017). In the fourth stage, we included this change in readmission rate into the third-stage model to examine how changes in the readmission rate affected the time trends of mortality and also examined its interaction with time and squared time. In the final model, we included the following potential confounding variables to the fourth stage model: African American percentage in the HSA, HSA median income, percentage of people living in poverty in the HSA, median age, ownership of hospital (government, for profit), teaching status (teaching vs nonteaching), and acute care hospital beds in the HSA. Within each stage, the models were compared using the Akaike information criterion (AIC) and the Bayesian information criterion (BIC), and the model with the lowest value of each was moved to the next stage of model development. All analyses were performed in Stata 14.1 for Windows (College Station, Texas).

Of the 3,685 acute care hospitals analyzed in the 2010-2013 data cycle for COPD, the 30-day RSRR was 20.7% (1.28), which decreased to 19.6% (1.11) in 2014-2017 (Table 1). During the same period, the 30-day all-cause RSMR increased from 7.8% (1.03) in 2010-2013 to 8.4% (1.11) in 2014-2017. The partitioning of variance showed that 57% of variation in the mortality rate over the study period was due to between-hospital differences.

The unconditional growth model examining the linear time trend revealed a 0.13% per year (95% CI = 0.12 to 0.14; P < .0001) increase in mortality rate over the five data cycles. When the squared time variable was added to the model to examine a quadratic trend, both time and squared trend were statistically significant (Table 2) and the AIC and BIC were lower for the quadratic model. Thus, the unconditional growth curve model suggested that the mortality trend was nonlinear and the coefficients demonstrated that not only the mortality rate increased, but the rate of change in the mortality rate was also increasing during the study period.

When we added the baseline readmission rate to the abovementioned quadratic growth model, we found an inverse association; each 1% increase in baseline readmission rate was associated with 0.03% (95% CI = −0.05 to −0.005; P = .02) decrease in mortality rate. These findings suggest that hospitals with higher baseline readmission rates also had lower mortality rates. To examine whether the effect of baseline readmission rate on mortality varied over time, we included the interaction term with time in the model and then added the interaction term with squared time. As the AIC and BIC were the lowest for the model with interaction between time and baseline readmission (and not when interaction between squared time and baseline readmission were included), we accepted this model. In this model, although there was no difference in mortality according to readmissions at baseline, each 1% increase in baseline readmission rate was associated with a smaller increase in mortality rate by 0.015% (95% CI = −0.02 to −0.01; P < .0001; Table 2 and Figure 1). These findings suggest that hospitals with higher readmission rates at baseline had a smaller increase in mortality rate during the study period than those with lower readmission rates.

As efforts to decrease readmission rates continue as a part of the HRRP implementation by the CMS, our study shows that among hospitals that discharged patients with AECOPD during 2010-2017, the all-cause 30-day RSRR was decreased, whereas the all-cause 30-day RSMR was increased. Of particular concern is that the rate of increase in mortality also increased. We also found that hospitals with higher readmission rates in 2010-2013 had a lower rate of increase in mortality than hospitals with lower readmission rates. In addition, hospitals that had a larger decrease in readmission rates during the study period had a larger rate of increase in mortality than hospitals with a smaller decrease in readmission rates. Our findings were robust to potential confounders such as hospital characteristics and local population characteristics in which hospitals operate.

Our study findings raise the question whether the implementation of the HRRP resulted in unintentional patient harm by forcing hospitals to make changes that may affect overall patient care. This question is particularly important as other studies on hospitalized patients with HF have found similar results.^13,14 On the other hand, a similar association between readmission and mortality rates has not been observed in patients with pneumonia or AMI.¹⁴ Several possible explanations can be given for the observed discrepancy between the diseases and their effect on the relationship between readmission rate and mortality rate. Both COPD and HF are chronic diseases and characterized by exacerbations, whereas AMI and pneumonia are episodic diseases that are treatable. As the number of patients hospitalized with AECOPD and HF is much larger, hospitals may have a greater focus on reducing the 30-day readmission rates and may attempt to game the process, such as by delaying admissions through the emergency department within the 30-day period or by admitting patients for observation. In fact, a study found a 3% reduction in the within-hospital readmission rate with a concurrent 0.8% increase in observation unit use since the implementation of the HRRP.²⁶ Such approaches to patient care may lead to adverse outcomes.

It is possible that readmissions and mortality act as competing risks and hence hospitals with higher mortality rates are left with fewer patients and thus have fewer readmissions, whereas those with lower mortality rates have more patients and a higher readmission rate.²⁷ Such studies are not possible with hospital-level data, and patient-level studies will be required to examine this competing risk hypothesis. Our study results provide some support to the competing risk hypothesis (hospitals with lower baseline readmission rates had a steeper increase in mortality); however, it is not possible to draw any conclusions due to the high risk of ecological fallacy bias.

This study has important potential implications for healthcare policy, public health, and research. We found that an important national intervention aimed at decreasing readmission rates and improving the quality of care for patients with AECOPD may be associated with higher mortality rates in these patients. There may be a need to redefine measures for determining the performance of an institution. Our study supports research into the underlying mechanisms resulting in an inverse association between readmissions and mortality. In particular, health policy researchers may need to examine how incentives and penalties affect the allocation of resources within hospitals.

This study has several strengths and some potential weaknesses. We used a national dataset to examine readmission and mortality rates that include the majority of hospitals in the United States. We also included data from the local population for each hospital, thus allowing us to examine hospital performance within the context of its target population. One potential limitation is that we used hospital-level data and not patient-level data; however, the readmission penalties are designed for hospitals, which justifies our use of hospital-level data. Furthermore, data were not available for shorter time intervals; data from shorter time intervals may be associated with greater variability. Being an observational study, it is difficult to establish a causal relationship; the longitudinal nature of the study does establish temporality, an important factor in establishing causality.

In conclusion, we found that although the readmission rates decreased, there was an increase in the mortality rate within the 30 days of discharge from the hospital in patients with AECOPD. The rate of increase in mortality was higher in hospitals with lower readmission rates than in hospitals with higher readmission rates. Further research for determining the mechanism responsible for this association is needed. Future health policy interventions may need to consider the potential for adverse outcomes.

Chronic obstructive pulmonary disease (COPD) is recognized as the third leading cause of death nationally. Globally, it has been estimated that 10% of the population has COPD; in the United States, approximately 15 million people are affected.^1,2 The annual estimated cost of COPD management in the United States is approximately $50 billion, one-third of which is directly related to inpatient hospitalization for COPD exacerbation.^3,4,5 The 30-day readmission rate after hospitalization for acute exacerbation of COPD (AECOPD) is approximately 21% with an approximate cost of $13 billion per year.^6,7 To reduce the cost and to improve patient outcomes, the Centers for Medicare and Medicaid Services (CMS) has designed several interventions with little effect.⁸

In October 2012, the Affordable Care Act added section 1886(q) to the Social Security Act and established the Hospital Readmission Reduction Program (HRRP), an initiative to decrease hospitalization costs by penalizing hospitals with high 30-day readmission rates. Under this program, hospitals received up to 3% penalty for excess readmissions after the index hospitalization with acute myocardial infarction (AMI), heart failure (HF), and pneumonia.^9-11 Hospitals are penalized if their annual readmission rates are significantly above the average national readmission rate. In 2014, the HRRP was extended to include AECOPD for the FY 2015.

Since the implementation of readmission penalties, data have shown a significant decrease in the 30-day readmission rates for all conditions.^12,13 On the other hand, studies have suggested that, at least for some conditions, the decrease in the 30-day readmission rate is associated with higher adverse patients outcomes, including higher mortality.^14,15 However, whether a decrease in readmission rates after an AECOPD hospitalization is associated with a concomitant increase in mortality has not been examined. Therefore, our objective was to examine the association of the 30-day risk-adjusted hospital readmission rate with the 30-day risk-adjusted hospital mortality rate for patients discharged with a diagnosis of AECOPD.

Data Sources

Publicly available data from three sources were used. The all-cause 30-day risk-standardized readmission rate (RSRR) and the 30-day risk-standardized mortality rate (RSMR) of each hospital for patients with AECOPD were obtained from the Hospital Compare database; a database maintained by the CMS.^16,17 In 2014, the CMS started reporting three-year running average of 30-day mortality and readmission rate data on hospitals for AECOPD hospitalizations; the data start date was July 2010.^18-22 We examined data from the FY 2010-2013 to 2014-2017 cycles on readmission and mortality reported by the CMS; this included data before and after the implementation of penalties.

Hospital characteristics were also obtained from the CMS website. Hospital ownership was defined as government (owned by Federal or state), for-profit (owned by physicians or another proprietary), or nonprofit (owned by a nonprofit organization such as a church). A hospital was considered as a teaching hospital if it obtained graduate medical education funding from the CMS.

Data on local population characteristics according to ZIP codes were obtained from the 2010 decennial census and the American Community Survey five-year (2009-2013) data files available at the United States Census Bureau website.²³ For each ZIP code, we obtained data on the total population, percentage of African Americans in the population, median income, poverty level, and insurance status.

We used Hospital service area (HSA) information obtained from the Dartmouth Atlas of Health Care crosswalk files to link local population characteristics to hospitals. The Dartmouth Atlas defined 3,436 HSAs by assigning the ZIP codes to the hospital area where the greatest proportion of their Medicare residents was hospitalized.^24,25

Hospital Compare data and Census Bureau population data were matched to the HSAs from the Dartmouth Atlas of Healthcare data at the ZIP code level. First, the ZIP code-level data from the Census Bureau were pooled by the HSAs obtained from the Dartmouth Atlas of Healthcare, followed by matching these data by the HSAs to the Hospital Compare data. Merging data from these three sources generated a dataset that contained information about readmission and mortality rates from a particular hospital and the population characteristics of the local healthcare market or neighborhood. Our final dataset included hospitals that had readmission and mortality information available at the Hospital Compare website and were included in the crosswalk files of the Dartmouth Atlas of Healthcare.

Statistical Analysis

Data are summarized as mean and standard deviation (SD), median with interquartile range, or frequencies as appropriate. To model the dependence of observations from the same hospital over time, we used mixed linear models with random intercept and slope. A strength of this modeling approach is that it incorporates information from all hospitals even when some hospitals are missing data for some time periods. We reached our final model through stages with increasing model complexity at each stage. In the first stage, we developed an empty model without any covariates to determine the unconditional variance components so that we can partition mortality variance into between- and within-hospital components. In the second stage, we developed an unconditional growth curve model to determine the shape of time trend in mortality over time using linear and quadratic (by including squared time in the model) growth curves. In the third stage, we added baseline readmission rates (from 2010 to 2013) to the model to determine the effect of baseline readmission rate on mortality trends and also examined its interaction with time and squared time. We generated a change in the readmission rate variable by subtracting the last readmission rate from the baseline readmission rate (readmission rate in 2010-2013 − readmission rate in 2014-2017). In the fourth stage, we included this change in readmission rate into the third-stage model to examine how changes in the readmission rate affected the time trends of mortality and also examined its interaction with time and squared time. In the final model, we included the following potential confounding variables to the fourth stage model: African American percentage in the HSA, HSA median income, percentage of people living in poverty in the HSA, median age, ownership of hospital (government, for profit), teaching status (teaching vs nonteaching), and acute care hospital beds in the HSA. Within each stage, the models were compared using the Akaike information criterion (AIC) and the Bayesian information criterion (BIC), and the model with the lowest value of each was moved to the next stage of model development. All analyses were performed in Stata 14.1 for Windows (College Station, Texas).

Of the 3,685 acute care hospitals analyzed in the 2010-2013 data cycle for COPD, the 30-day RSRR was 20.7% (1.28), which decreased to 19.6% (1.11) in 2014-2017 (Table 1). During the same period, the 30-day all-cause RSMR increased from 7.8% (1.03) in 2010-2013 to 8.4% (1.11) in 2014-2017. The partitioning of variance showed that 57% of variation in the mortality rate over the study period was due to between-hospital differences.

The unconditional growth model examining the linear time trend revealed a 0.13% per year (95% CI = 0.12 to 0.14; P < .0001) increase in mortality rate over the five data cycles. When the squared time variable was added to the model to examine a quadratic trend, both time and squared trend were statistically significant (Table 2) and the AIC and BIC were lower for the quadratic model. Thus, the unconditional growth curve model suggested that the mortality trend was nonlinear and the coefficients demonstrated that not only the mortality rate increased, but the rate of change in the mortality rate was also increasing during the study period.

When we added the baseline readmission rate to the abovementioned quadratic growth model, we found an inverse association; each 1% increase in baseline readmission rate was associated with 0.03% (95% CI = −0.05 to −0.005; P = .02) decrease in mortality rate. These findings suggest that hospitals with higher baseline readmission rates also had lower mortality rates. To examine whether the effect of baseline readmission rate on mortality varied over time, we included the interaction term with time in the model and then added the interaction term with squared time. As the AIC and BIC were the lowest for the model with interaction between time and baseline readmission (and not when interaction between squared time and baseline readmission were included), we accepted this model. In this model, although there was no difference in mortality according to readmissions at baseline, each 1% increase in baseline readmission rate was associated with a smaller increase in mortality rate by 0.015% (95% CI = −0.02 to −0.01; P < .0001; Table 2 and Figure 1). These findings suggest that hospitals with higher readmission rates at baseline had a smaller increase in mortality rate during the study period than those with lower readmission rates.

As efforts to decrease readmission rates continue as a part of the HRRP implementation by the CMS, our study shows that among hospitals that discharged patients with AECOPD during 2010-2017, the all-cause 30-day RSRR was decreased, whereas the all-cause 30-day RSMR was increased. Of particular concern is that the rate of increase in mortality also increased. We also found that hospitals with higher readmission rates in 2010-2013 had a lower rate of increase in mortality than hospitals with lower readmission rates. In addition, hospitals that had a larger decrease in readmission rates during the study period had a larger rate of increase in mortality than hospitals with a smaller decrease in readmission rates. Our findings were robust to potential confounders such as hospital characteristics and local population characteristics in which hospitals operate.

Our study findings raise the question whether the implementation of the HRRP resulted in unintentional patient harm by forcing hospitals to make changes that may affect overall patient care. This question is particularly important as other studies on hospitalized patients with HF have found similar results.^13,14 On the other hand, a similar association between readmission and mortality rates has not been observed in patients with pneumonia or AMI.¹⁴ Several possible explanations can be given for the observed discrepancy between the diseases and their effect on the relationship between readmission rate and mortality rate. Both COPD and HF are chronic diseases and characterized by exacerbations, whereas AMI and pneumonia are episodic diseases that are treatable. As the number of patients hospitalized with AECOPD and HF is much larger, hospitals may have a greater focus on reducing the 30-day readmission rates and may attempt to game the process, such as by delaying admissions through the emergency department within the 30-day period or by admitting patients for observation. In fact, a study found a 3% reduction in the within-hospital readmission rate with a concurrent 0.8% increase in observation unit use since the implementation of the HRRP.²⁶ Such approaches to patient care may lead to adverse outcomes.

It is possible that readmissions and mortality act as competing risks and hence hospitals with higher mortality rates are left with fewer patients and thus have fewer readmissions, whereas those with lower mortality rates have more patients and a higher readmission rate.²⁷ Such studies are not possible with hospital-level data, and patient-level studies will be required to examine this competing risk hypothesis. Our study results provide some support to the competing risk hypothesis (hospitals with lower baseline readmission rates had a steeper increase in mortality); however, it is not possible to draw any conclusions due to the high risk of ecological fallacy bias.

This study has important potential implications for healthcare policy, public health, and research. We found that an important national intervention aimed at decreasing readmission rates and improving the quality of care for patients with AECOPD may be associated with higher mortality rates in these patients. There may be a need to redefine measures for determining the performance of an institution. Our study supports research into the underlying mechanisms resulting in an inverse association between readmissions and mortality. In particular, health policy researchers may need to examine how incentives and penalties affect the allocation of resources within hospitals.

This study has several strengths and some potential weaknesses. We used a national dataset to examine readmission and mortality rates that include the majority of hospitals in the United States. We also included data from the local population for each hospital, thus allowing us to examine hospital performance within the context of its target population. One potential limitation is that we used hospital-level data and not patient-level data; however, the readmission penalties are designed for hospitals, which justifies our use of hospital-level data. Furthermore, data were not available for shorter time intervals; data from shorter time intervals may be associated with greater variability. Being an observational study, it is difficult to establish a causal relationship; the longitudinal nature of the study does establish temporality, an important factor in establishing causality.

In conclusion, we found that although the readmission rates decreased, there was an increase in the mortality rate within the 30 days of discharge from the hospital in patients with AECOPD. The rate of increase in mortality was higher in hospitals with lower readmission rates than in hospitals with higher readmission rates. Further research for determining the mechanism responsible for this association is needed. Future health policy interventions may need to consider the potential for adverse outcomes.

The Choosing Wisely^® campaign, launched in 2012 by the American Board of Internal Medicine, aims to reduce overuse of tests and treatments that do not add value for patients. The campaign has caught the attention of the medical profession and spread internationally. Over the last seven years, most specialty societies have published specific recommendations on what tests and treatments clinicians should stop doing. However, has this campaign actually had an impact on the testing and treating behaviors of clinicians?

In this issue of the Journal of Hospital Medicine, Reyes and colleagues examine changes in five overuse metrics linked with the 2013 Choosing Wisely^® Pediatric Hospital Medicine recommendations at 37 children’s hospitals from 2008 to 2017, five years before and after the recommendations were published.^1,2 The tests and treatments targeted by these recommendations are not individually costly, but given the high prevalence of the conditions, the cumulative cost is not insignificant. More importantly, reducing the potentially harmful long-term effects of unnecessary radiation and adverse effects from exposure to inappropriate systemic steroids and antacids is a laudable goal. Results from unnecessary tests may also lead to a further cascade of unnecessary testing and/or treatment.³

The authors used an administrative data source, the Pediatric Health Information System (PHIS), to measure billing charges for the tests and medications linked with the overuse measures in over 278,000 hospitalizations. The good news is that overuse declined over the 10-year study period. After adjusting for differences in patient characteristics over time, they observed a substantial absolute reduction in bronchiolitis bronchodilator use (36.6%, from 64% in 2008 to 27.4% in 2017) and chest x-ray (CXR) use (31.5%, from 58.4% to 26.9%). There were also reductions for the other metrics: acid-suppressing medications for gastroesophageal reflux (24.1%, from 63% to 48.9%), asthma CXR use (20.8%, from 52.8% to 32%), and steroids for lower respiratory tract infections (2.9%, from 15.1% to 12.2%). We would not expect the goal for these overuse metrics to be zero percent given the diagnostic uncertainties in real-world clinical decision-making.

The Choosing Wisely^® Pediatric Hospital Medicine recommendations, however, were associated with only a modest impact on the overuse decline. A before-and-after interrupted time series analysis showed that the overuse measures were on the downturn prior to the recommendations being published. Then after publication, only the rate of CXR use in asthma decreased immediately. The rate of bronchodilator use for bronchiolitis declined in the following five-year period. There were no changes in the rate of decline in overuse for the other tests and treatments associated with the recommendations.

With such a widespread national campaign, a control group of hospitals to better understand the specific influence of the Choosing Wisely^® recommendations was not possible. The decline in overuse over the 10-year period reported by Reyes et al. is likely due to a combination of efforts at multiple levels—including national society guidelines, local hospital guidelines and pathways, increased awareness by clinicians of the problem of overuse, and focused quality improvement efforts.

The use of the PHIS database provided Reyes et al. a powerful data source to evaluate overuse across a large number of patients and hospitals efficiently. However, there are limitations with administrative data that are important to consider. Detailed clinical data, such as patient disease characteristics and test and treatment indications, are not available, which limits the specificity of these measures. For example, one of the recommendations suggests that gastroesophageal reflux should not be routinely treated with acid suppression therapy. Using administrative data, it is impossible to know whether the use of antacids in hospitalized children with a primary discharge diagnosis code of gastroesophageal reflux was inappropriate or because they failed other treatments in the outpatient setting and/or had complicated disease appropriately warranting treatment. This misclassification would result in an overestimation of overuse. The authors did attempt to minimize the possibility of misclassification by excluding children with comorbidities, those who had longer hospital stays, those admitted to the intensive care unit, and those with greater severity of illness where some of these tests and treatments would be indicated.

While the report by Reyes et al. focuses on Pediatric Hospital Medicine Choosing Wisely^® recommendations, it is important to recognize that tests and treatments for conditions like asthma, bronchiolitis, and lower respiratory tract infections are initially performed in the emergency department (ED). Collaboration between the ED and the Hospital Medicine Unit is essential to tackle the issue of overuse.⁴

The study by Reyes et al. provides a nice description of the trends in the Choosing Wisely^® overuse metrics at a group of children’s hospitals and is one of few such reports. The NIH funded, Eliminating Monitor Overuse: pulse oximetry (EMO: SpO2) study is focusing on the 5th Choosing Wisely^® Pediatric Hospital Medicine recommendation that was not studied by Reyes.⁵

So then, with the decline in overuse reported in this study over 10 years, is it time to celebrate? Not yet. There is much work to do in the pursuit of Choosing Wisely^®: developing a host of valid measures of overuse in pediatric hospital care, expanding the examination of overuse to community hospitals where the majority of children are hospitalized, and using implementation science theory to de-implement the ingrained practices.

The Choosing Wisely^® campaign, launched in 2012 by the American Board of Internal Medicine, aims to reduce overuse of tests and treatments that do not add value for patients. The campaign has caught the attention of the medical profession and spread internationally. Over the last seven years, most specialty societies have published specific recommendations on what tests and treatments clinicians should stop doing. However, has this campaign actually had an impact on the testing and treating behaviors of clinicians?

In this issue of the Journal of Hospital Medicine, Reyes and colleagues examine changes in five overuse metrics linked with the 2013 Choosing Wisely^® Pediatric Hospital Medicine recommendations at 37 children’s hospitals from 2008 to 2017, five years before and after the recommendations were published.^1,2 The tests and treatments targeted by these recommendations are not individually costly, but given the high prevalence of the conditions, the cumulative cost is not insignificant. More importantly, reducing the potentially harmful long-term effects of unnecessary radiation and adverse effects from exposure to inappropriate systemic steroids and antacids is a laudable goal. Results from unnecessary tests may also lead to a further cascade of unnecessary testing and/or treatment.³

The authors used an administrative data source, the Pediatric Health Information System (PHIS), to measure billing charges for the tests and medications linked with the overuse measures in over 278,000 hospitalizations. The good news is that overuse declined over the 10-year study period. After adjusting for differences in patient characteristics over time, they observed a substantial absolute reduction in bronchiolitis bronchodilator use (36.6%, from 64% in 2008 to 27.4% in 2017) and chest x-ray (CXR) use (31.5%, from 58.4% to 26.9%). There were also reductions for the other metrics: acid-suppressing medications for gastroesophageal reflux (24.1%, from 63% to 48.9%), asthma CXR use (20.8%, from 52.8% to 32%), and steroids for lower respiratory tract infections (2.9%, from 15.1% to 12.2%). We would not expect the goal for these overuse metrics to be zero percent given the diagnostic uncertainties in real-world clinical decision-making.

The Choosing Wisely^® Pediatric Hospital Medicine recommendations, however, were associated with only a modest impact on the overuse decline. A before-and-after interrupted time series analysis showed that the overuse measures were on the downturn prior to the recommendations being published. Then after publication, only the rate of CXR use in asthma decreased immediately. The rate of bronchodilator use for bronchiolitis declined in the following five-year period. There were no changes in the rate of decline in overuse for the other tests and treatments associated with the recommendations.

With such a widespread national campaign, a control group of hospitals to better understand the specific influence of the Choosing Wisely^® recommendations was not possible. The decline in overuse over the 10-year period reported by Reyes et al. is likely due to a combination of efforts at multiple levels—including national society guidelines, local hospital guidelines and pathways, increased awareness by clinicians of the problem of overuse, and focused quality improvement efforts.

The use of the PHIS database provided Reyes et al. a powerful data source to evaluate overuse across a large number of patients and hospitals efficiently. However, there are limitations with administrative data that are important to consider. Detailed clinical data, such as patient disease characteristics and test and treatment indications, are not available, which limits the specificity of these measures. For example, one of the recommendations suggests that gastroesophageal reflux should not be routinely treated with acid suppression therapy. Using administrative data, it is impossible to know whether the use of antacids in hospitalized children with a primary discharge diagnosis code of gastroesophageal reflux was inappropriate or because they failed other treatments in the outpatient setting and/or had complicated disease appropriately warranting treatment. This misclassification would result in an overestimation of overuse. The authors did attempt to minimize the possibility of misclassification by excluding children with comorbidities, those who had longer hospital stays, those admitted to the intensive care unit, and those with greater severity of illness where some of these tests and treatments would be indicated.

While the report by Reyes et al. focuses on Pediatric Hospital Medicine Choosing Wisely^® recommendations, it is important to recognize that tests and treatments for conditions like asthma, bronchiolitis, and lower respiratory tract infections are initially performed in the emergency department (ED). Collaboration between the ED and the Hospital Medicine Unit is essential to tackle the issue of overuse.⁴

The study by Reyes et al. provides a nice description of the trends in the Choosing Wisely^® overuse metrics at a group of children’s hospitals and is one of few such reports. The NIH funded, Eliminating Monitor Overuse: pulse oximetry (EMO: SpO2) study is focusing on the 5th Choosing Wisely^® Pediatric Hospital Medicine recommendation that was not studied by Reyes.⁵

So then, with the decline in overuse reported in this study over 10 years, is it time to celebrate? Not yet. There is much work to do in the pursuit of Choosing Wisely^®: developing a host of valid measures of overuse in pediatric hospital care, expanding the examination of overuse to community hospitals where the majority of children are hospitalized, and using implementation science theory to de-implement the ingrained practices.

The Choosing Wisely^® campaign, launched in 2012 by the American Board of Internal Medicine, aims to reduce overuse of tests and treatments that do not add value for patients. The campaign has caught the attention of the medical profession and spread internationally. Over the last seven years, most specialty societies have published specific recommendations on what tests and treatments clinicians should stop doing. However, has this campaign actually had an impact on the testing and treating behaviors of clinicians?

In this issue of the Journal of Hospital Medicine, Reyes and colleagues examine changes in five overuse metrics linked with the 2013 Choosing Wisely^® Pediatric Hospital Medicine recommendations at 37 children’s hospitals from 2008 to 2017, five years before and after the recommendations were published.^1,2 The tests and treatments targeted by these recommendations are not individually costly, but given the high prevalence of the conditions, the cumulative cost is not insignificant. More importantly, reducing the potentially harmful long-term effects of unnecessary radiation and adverse effects from exposure to inappropriate systemic steroids and antacids is a laudable goal. Results from unnecessary tests may also lead to a further cascade of unnecessary testing and/or treatment.³

The authors used an administrative data source, the Pediatric Health Information System (PHIS), to measure billing charges for the tests and medications linked with the overuse measures in over 278,000 hospitalizations. The good news is that overuse declined over the 10-year study period. After adjusting for differences in patient characteristics over time, they observed a substantial absolute reduction in bronchiolitis bronchodilator use (36.6%, from 64% in 2008 to 27.4% in 2017) and chest x-ray (CXR) use (31.5%, from 58.4% to 26.9%). There were also reductions for the other metrics: acid-suppressing medications for gastroesophageal reflux (24.1%, from 63% to 48.9%), asthma CXR use (20.8%, from 52.8% to 32%), and steroids for lower respiratory tract infections (2.9%, from 15.1% to 12.2%). We would not expect the goal for these overuse metrics to be zero percent given the diagnostic uncertainties in real-world clinical decision-making.

The Choosing Wisely^® Pediatric Hospital Medicine recommendations, however, were associated with only a modest impact on the overuse decline. A before-and-after interrupted time series analysis showed that the overuse measures were on the downturn prior to the recommendations being published. Then after publication, only the rate of CXR use in asthma decreased immediately. The rate of bronchodilator use for bronchiolitis declined in the following five-year period. There were no changes in the rate of decline in overuse for the other tests and treatments associated with the recommendations.

With such a widespread national campaign, a control group of hospitals to better understand the specific influence of the Choosing Wisely^® recommendations was not possible. The decline in overuse over the 10-year period reported by Reyes et al. is likely due to a combination of efforts at multiple levels—including national society guidelines, local hospital guidelines and pathways, increased awareness by clinicians of the problem of overuse, and focused quality improvement efforts.

The use of the PHIS database provided Reyes et al. a powerful data source to evaluate overuse across a large number of patients and hospitals efficiently. However, there are limitations with administrative data that are important to consider. Detailed clinical data, such as patient disease characteristics and test and treatment indications, are not available, which limits the specificity of these measures. For example, one of the recommendations suggests that gastroesophageal reflux should not be routinely treated with acid suppression therapy. Using administrative data, it is impossible to know whether the use of antacids in hospitalized children with a primary discharge diagnosis code of gastroesophageal reflux was inappropriate or because they failed other treatments in the outpatient setting and/or had complicated disease appropriately warranting treatment. This misclassification would result in an overestimation of overuse. The authors did attempt to minimize the possibility of misclassification by excluding children with comorbidities, those who had longer hospital stays, those admitted to the intensive care unit, and those with greater severity of illness where some of these tests and treatments would be indicated.

While the report by Reyes et al. focuses on Pediatric Hospital Medicine Choosing Wisely^® recommendations, it is important to recognize that tests and treatments for conditions like asthma, bronchiolitis, and lower respiratory tract infections are initially performed in the emergency department (ED). Collaboration between the ED and the Hospital Medicine Unit is essential to tackle the issue of overuse.⁴

The study by Reyes et al. provides a nice description of the trends in the Choosing Wisely^® overuse metrics at a group of children’s hospitals and is one of few such reports. The NIH funded, Eliminating Monitor Overuse: pulse oximetry (EMO: SpO2) study is focusing on the 5th Choosing Wisely^® Pediatric Hospital Medicine recommendation that was not studied by Reyes.⁵

So then, with the decline in overuse reported in this study over 10 years, is it time to celebrate? Not yet. There is much work to do in the pursuit of Choosing Wisely^®: developing a host of valid measures of overuse in pediatric hospital care, expanding the examination of overuse to community hospitals where the majority of children are hospitalized, and using implementation science theory to de-implement the ingrained practices.

Despite efforts to provide high-quality healthcare, Americans die from medical errors each year and many patients do not receive recommended medical care. Risk is particularly acute during times of hospitalization.^1-4 In response, the Institute of Medicine (IOM, now the Academy of Medicine) has released “Crossing the Quality Chasm: A New Health System for the 21^st Century,” providing a framework to guide delivery and measurement of high-quality healthcare.⁵

Although the IOM framework has motivated the development of quality improvement (QI) and quality measurement initiatives, relatively few resources have been allocated to improving the quality of pediatric inpatient care.^6,7 The resultant gap in our knowledge of quality and safety of pediatric hospital-based care is further widened by the variability of settings in which children are hospitalized. These settings include freestanding children’s hospitals, children’s hospitals nested within larger hospitals, and community hospitals, defined as general, nonuniversity, and nonchildren’s hospitals.⁸

Although almost three-quarters of children needing hospitalization are cared for outside of freestanding children’s hospitals, we know particularly little about the quality and safety of pediatric hospital-based care outside of these settings.^6,9 Therefore, our scoping review aims to summarize literature regarding the quality and safety of pediatric inpatient care within community hospitals.

We used a scoping review approach because this methodology, by design, is utilized to synthesize evidence and map existing literature and is particularly useful when a body of literature is heterogeneous, rendering a more targeted systematic review approach infeasible.¹⁰ This methodology thereby provided an organized approach to answer our broad research question, “What evidence exists regarding the quality and safety of pediatric inpatient care in United States community hospitals?“ We followed the scoping review guidelines put forth by the Joanna Briggs Institute and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting guideline Extension for Scoping Reviews.^10,11

Data Sources and Search Strategies

We searched Medline, Medline-In-Process, Embase, the Cochrane Database of Systematic Reviews, the Cumulative Index to Nursing and Allied Health Literature (CINAHL), PsycINFO, and Scopus for studies that reported at least one outcome related to healthcare quality or patient safety and involved pediatric patients (aged <18 years) receiving inpatient care at a community hospital. Outcomes included measures from the IOM-defined aims of quality healthcare: (1) safety, (2) effectiveness, (3) efficiency, (4) timeliness, (5) patient-centeredness, and (6) equity (Appendix Table 1). Terms were searched as controlled vocabulary in applicable databases (Medline, Embase, CINAHL, PsycINFO) and as keywords in all databases. Search strategies tailored to each database were developed, tested, and refined in collaboration with a reference librarian. Date parameters for retrieval were set from 1989 to the search date, with the start date chosen to correspond with the establishment of the Agency for Health Care Policy and Research, currently known as the Agency for Healthcare Research and Quality (AHRQ), targeting literature produced in the wake of the AHRQ emphasis on quality in healthcare. Results were limited to articles published in English. Searches were conducted on October 24 and 25, 2016. Complete search strategies can be found in Appendix Methods.

Independent authors performed handsearches of Academic Pediatrics, BMJ Quality & Safety, Hospital Pediatrics, JAMA Pediatrics, and Pediatrics Quality Reports for the five years preceding the search date (July 2011-October 2016).

Study Selection and Definitions

To identify studies conducted in community hospitals, we operationalized the definition of community hospital proposed by Percelay.⁸ We used “community”, “general”, “nonuniversity”, and “nonchildren’s” as search terms and operationalized these through the addition of qualifying descriptors (Table 1).

Studies were excluded if they (1) were performed outside of the United States, as community hospital definitions may differ by country; (2) included participants aged >18 years and did not report any pediatric-specific results; (3) were performed exclusively in tertiary hospitals; (4) evaluated only outpatient or emergency department care; (5) did not report any results specific to community hospitals; (6) did not report any data-driven or parent/patient-reported measures of safety, effectiveness, efficiency, timeliness, patient-centeredness, or equity in their results; and (7) were case reports, case series, editorials, or abstracts without an associated full-text article. We read commentaries and literature reviews related to our objectives and reviewed their references to identify additional articles, but did not include these manuscripts.

Two authors independently reviewed each abstract, and full-text articles were reviewed if one or more authors determined that the abstract met the inclusion criteria. Two authors then independently reviewed each full-text article to determine whether the article met the criteria for the final review. Disagreements were resolved through consensus after discussion and review with the entire research team, and reasons for exclusion were recorded.

Charting the Results and Data Synthesis

We used a standardized charting form to collect information regarding study design, community hospital terms, population descriptors, IOM aims of quality healthcare, and outcome measures. To minimize bias in data collection, information from each full-text article was extracted independently by two authors, and differences in extraction were resolved through discussion and re-review among the same two authors. To evaluate the quality of included evidence, two authors (JCL and JKL) independently assessed the risk of bias for each study using modified Newcastle-Ottawa Quality Assessment Scales (NOS), with disagreements resolved through discussion and re-review among the same two authors. For cohort studies, five of the eight NOS domains were relevant and applied to all included studies (maximum score 6). Using the NOS adapted for cross-sectional studies, six of the seven domains were relevant and applied (maximum score 9). For cross-sectional studies, we defined the risk of bias to be low for scores ≥8, moderate for scores 5-7, and high for scores ≤4, consistent with previous work.¹² For cohort studies, we similarly defined these strata by scores ≥5, 3-4, and ≤2, given the lower maximum score for this study type.

We categorized studies as either observational, defined as cohort or cross-sectional studies of usual healthcare delivery, or interventional, defined as studies evaluating the development and/or implementation of an intervention designed to improve healthcare quality. We further categorized the studies into the following four overarching medical domains: (1) neonatal, (2) pediatric medicine, (3) surgery, or (4) radiology.

After removal of duplicates, our search identified 2,068 abstracts for screening (Figure). Of these, 1,777 did not meet the inclusion criteria, leaving 291 articles for full-text review. Of these, 43 articles met all the inclusion criteria, and one additional article was included from search of references, resulting in a total of 44 articles.

Study designs, patient populations, and quality outcome measures were heterogeneous. We identified only one randomized controlled trial (RCT). A total of 30 articles were observational studies, 27 of which used retrospective cohort or cross-sectional designs; the remaining three used prospective cohort designs (Table 2). Of these studies, 20 involved multiple hospitals, whereas 10 were conducted at a single community hospital. Sample sizes ranged from 29 (single-site) to 107,727 (multisite) patients. Twenty-two studies aimed to compare quality outcomes at community hospitals with other hospital types, of which 16 performed risk-adjusted analyses (detailed findings of observational studies are summarized in Appendix Table 2). The remaining 14 articles were interventional studies (Table 2). Of these, 12 (86%) reported improvement in quality outcomes after implementation (detailed findings of interventional studies are summarized in Appendix Table 3).

Observational Studies

Neonatal Medicine

Five multisite studies focused on neonatal care,^{13,14,16,19,20} of which four examined outcomes associated with the transfer of neonates to or from community hospitals to tertiary care hospitals, such as neonatal morbidity, readmission, completion of preventative health measures and screening, parent satisfaction, and hospital charges.^14,16,19,20 For example, in a study of extremely premature very low birth weight infants born in Hawaii, Kuo et al. demonstrated that the odds of retinopathy of prematurity was 2.9 times higher for infants born at a community hospital and transported to their tertiary center compared with those inborn at the tertiary center (P = .02).¹⁶ The fifth study examined neonatal mortality by site of birth, demonstrating that, among infants with birth weights less than 2,000 g, birth at a hospital with a community neonatal intensive care unit (NICU) was associated with 1.4 times higher odds of risk-adjusted mortality compared with birth at a regional NICU (P < .001).¹³

Three studies evaluated the quality of neonatal care at a single community hospital.^15,17,18 Quality outcomes were heterogeneous, including utility of rebound bilirubin levels for infants with jaundice, morbidity of neonates requiring mechanical ventilation, and provision of breastfeeding advice to mothers of breastfeeding infants. For instance, Meadow et al. attempted to determine the quality of care for ventilated neonates at one community NICU compared with a tertiary hospital.¹⁸ They found no difference in days on ventilation or need for home oxygen therapy between the community hospital and the tertiary center, although P values and effect sizes were not reported for these outcomes and analyses were not adjusted beyond matching on birthdate and birth weight.

Pediatric Medicine

Nine multisite studies explored the quality and safety of pediatric medical care across hospital types.^21-23,25-30 Of these, four were conducted using the Kids’ Inpatient Database (KID)^28-30 or the National Inpatient Sample (NIS),²⁷ two were conducted using other national databases,^21,25 and two were conducted using electronic medical record data.^22,26 All of the KID and NIS studies examined hospital charges, either alone or in conjunction with mortality. Two of these studies found no differences in risk-adjusted hospital charges between children’s hospitals and community hospitals (for burn injuries),^28,29 whereas two found that hospitalization at community hospitals was associated with lower risk-adjusted charges (for asthma and sepsis).^27,30 The remaining five studies evaluated diverse quality outcomes such as medication errors, therapeutic drug monitoring, practice guideline compliance, antibiotic prescribing, or hospital-to-home transition summary scores.^21-23,25,26

Three single-site studies examined quality outcomes for pediatric medical patients, including mortality, outcome ratings for dehydration, and measures of peripherally inserted central catheter (PICC) safety and effectiveness.^24,31,32 For example, Frank et al. evaluated safety of care in one community hospital pediatric intensive care unit (PICU) compared with a tertiary hospital using the Pediatric Risk of Mortality (PRISM) score.²⁴ They reported that the observed number of deaths in their community PICU did not differ significantly from the number of deaths predicted in a tertiary center (23 vs 33, respectively, P > .2).

Surgery

Three multisite studies examined quality outcomes among children with surgical conditions, including surgical complications, readmission, and hospital charges.^34,35,37 For example, Kelley-Quon et al. examined outcomes following surgery in infants with hypertrophic pyloric stenosis and found that infants who received their surgery at community hospitals had twice the odds of a surgical complication compared with those at children’s hospitals (P = .027).³⁴ They also examined how the risk of appendiceal perforation differed by hospital type, finding that black children who received their surgery at children’s hospitals had twice the odds of appendiceal perforation compared with those who received care at community hospitals.³⁵

In addition, two studies evaluated quality and safety outcomes for pediatric surgical care in a single community hospital.^33,36 For example, Beaty et al. prospectively evaluated the incidence of missed injuries in hospitalized pediatric trauma patients, reporting that the incidence of missed injury was 33% when admission evaluation was performed by a trauma surgeon alone compared to 11% when performed by a pediatric doctor or a trauma surgeon and a pediatric doctor together (P < .001).³³

Radiology

Three multisite studies examined quality and safety outcomes associated with radiographic imaging in community hospitals, including radiation dosing and frequency of preoperative imaging modalities and accuracy.^38,39,41 For instance, Marin et al. found substantial variation in radiation dose across hospital types, with children’s hospitals delivering lower median radiation doses than academic and community hospitals.³⁹ Similarly, Saito et al. demonstrated increased use of radiating modalities when evaluation was performed at community hospitals, with four times higher odds of computed tomography (CT) and five times lower odds of ultrasound use compared to a children’s hospital.⁴¹

Two single-site studies also evaluated quality and safety outcomes associated with the use of radiographic imaging for pediatric appendicitis in a community hospital.^40,42 For example, York et al. demonstrated that, compared with nonimaged patients, patients who underwent diagnostic imaging for appendicitis experienced a significant time delay from initial evaluation to surgery and incurred significantly higher hospital charges, whereas there were no significant differences in intraoperative findings, antibiotic requirements, and surgical complications between the groups.⁴²

Interventional Studies

Neonatal Medicine

We identified six studies that evaluated interventions to improve healthcare quality for neonates in community hospitals; two involved telemedicine.^43-48 Hall et al. described “Telenursery,” a program linking regional perinatal centers with a large academic neonatal practice through real-time teleconferencing, in addition to providing weekly educational conferences.⁴⁵ After its implementation, there was an increase in the state-recommended delivery of very low birth weight infants at the regional perinatal center from 24% to 33% (P < .05); clinical outcomes were not discussed. In a similar study, Sable et al. found that a videoconferencing system for cardiologists from an academic center to guide care in a community setting provided diagnostic services more quickly (28 minutes vs 12 hours) and had high diagnostic accuracy.⁴⁷ The remaining four studies evaluated heterogeneous interventions such as a maternal education program to reduce shaking injuries to infants or implementation of evidence-based order sets to reduce early onset group B streptococcal disease in neonates.^43,44,46,48 Five of the six neonatal studies demonstrated improved quality outcomes after intervention.^43-47

Pediatric Medicine

Seven studies evaluated QI interventions for children at community hospitals,^49-55 three of which described interventions to improve quality of management of respiratory diseases.^49,51,53 Dayal et al. evaluated the implementation of respiratory illness order sets and an asthma pathway, demonstrating a 41% reduction in asthma hospitalization cost per patient (P < .05), reduced bronchodilator use for all respiratory illnesses, and no change in readmission rates.⁴⁹ Similarly, Nkoy et al. evaluated an asthma “Evidence-Based Care Practice Model” implemented at seven community hospitals and demonstrated a nonsignificant reduction in readmissions (P = .12), as well as lower hospitalization costs (P = .05).⁵³ Using QI methods, Kuhlmann et al. demonstrated improved compliance from 43% to 97% with the Asthma Home Management Plan of Care measure.⁵¹ The remaining four studies evaluated heterogeneous interventions, including telemedicine critical care consultations, use of I-PASS, and consolidation of pediatric care onto one hospital unit.^50,52,54,55 Six of the seven pediatric studies (86%) demonstrated improved inpatient quality outcomes after intervention,^49,51-55 but only one study was multisite.⁵³

Surgery

We identified only one study evaluating a surgical intervention aimed at improving the quality of pediatric care.⁵⁶ Kelley-Quon evaluated the impact of a community hospital partnering with an Academic Medical Pediatric Trauma Center to become a Level II Pediatric Trauma Center (PTC). After achieving Level II PTC designation, they reported that children treated at the community hospital had reduced rates of CT use, transfers, and in-hospital mortality (from 81% to 51%, 8.5% to 2.5%, and 2% to 0.4%, respectively, P < .05 for all) compared to those treated predesignation.

Radiology

Within the domain of radiology, our review identified no interventional studies.

In this scoping review of the quality and safety of pediatric inpatient care in community hospitals, we identified 44 studies applying heterogeneous study designs and evaluating diverse patient populations and quality outcomes. We identified only one RCT in our search; all the remaining studies applied observational designs.

We found only three clinical areas that were explored in multiple studies, with consistent directionality of results: (1) perinatal regionalization, (2) telemedicine, and (3) imaging radiation. The limited evidence identified in our review suggests that delivery of early premature infants at community hospitals, rather than at tertiary hospitals, may increase risk of neonatal morbidity and/or mortality,^13,16,46 that use of telemedicine may improve the effectiveness and efficiency of intensive or specialized care in community settings,^45,47,52,55 and that CT use and radiation doses may be higher in community hospitals compared with other settings.^38,39,41 However, even within these clinical domains, the literature was limited in amount and heterogeneous; additional research is needed to systematically review the effect of individual interventions or particular community hospital quality outcomes compared with other hospital types.

Our search identified only 14 studies evaluating QI interventions within community hospitals over the almost 30-year review period. Although limited in number, 86% of these demonstrated improvements in healthcare quality and safety, providing a positive “proof of concept” that pediatric care in community hospitals can be improved by multidisciplinary efforts and be sustained over time.^{43-47,49,51-56} As pediatric departments within community hospitals may have limited resources, aligning pediatric QI efforts with adult initiatives within the same hospitals could prove advantageous. However, we did not identify any studies meeting our inclusion criteria that took this approach. Alternatively, QI collaboratives across structurally diverse hospitals may provide valuable infrastructure for QI in community hospitals. For example, the Value in Inpatient Pediatrics Network has conducted several multisite QI initiatives that have engaged both children’s and community hospitals.^57-60 However, none of these studies have reported community hospital-specific quality outcomes, resulting in their exclusion from this review. In future study, researchers may consider separating community hospital results from those of pediatric hospitals to highlight the effect of community hospital-specific QI efforts and to allow valuable direct comparisons between hospital types.

Many studies identified in our scoping review were conducted at single hospitals. Sample sizes were often very small and power calculations were rarely reported, raising questions about the validity of the “nonsignificant” differences reported by some. Although such single-center studies provide valuable information for local improvement of inpatient care, by design the findings from these studies are unlikely generalizable to other community hospital systems. Without inclusion of another hospital type and/or quality measures with clear national benchmark for comparison, the additional conclusions that can be drawn from these studies are limited. In comparison, many of the multicenter studies had large study samples, but more than half used data registries with limited data to evaluate outcomes, clinical context, or important covariates.^{13,20,21,25,27-30,34,35,37,39} The most frequently reported outcomes in these data sets—hospital charges and lengths of stay—are of limited utility in understanding healthcare quality without clear benchmarks. As a result, the evidence-base from which to draw conclusions about quality and safety in community hospitals is very limited.

Therefore, our review highlights a great need for additional research in community hospital medicine and the need for high-quality evidence generation. Risk of bias was moderate or high for the majority of included studies because of inadequate risk adjustment or statistical analysis. In the future, multicenter collaborations may help to connect research methodologists with community hospital teams to aid in the application of robust study designs and analytic techniques. Multisite collaboration may also overcome the limitation of small sample sizes that are a reality at many community hospitals.

Our study findings must be considered in the setting of several methodological limitations. The lack of a standard definition for a community hospital has led to inconsistent terms and hospital definitions used in the literature. It is possible that, while following our systematic approach to define a community hospital, we inadvertently missed relevant studies that used different terms. We also excluded unpublished articles. Given that publication bias tends to favor studies with significant associations, it is possible that some studies with insignificant changes in quality outcomes were missed. Finally, in our exclusion of all non-US studies, we may have unknowingly missed literature from countries with community hospital definitions similar to those in the United States.

Recognizing that more than half of all children admitted to hospitals in the US receive their care at community hospitals, understanding healthcare quality in community hospitals is important. This scoping review underscores the need for additional research and higher quality evidence to determine the quality of pediatric inpatient care in these settings and identifies some particularly wide gaps that could be targeted in future research. Acknowledging that further research is necessary to address all aims of quality healthcare, markedly few studies have examined timeliness, equity, or patient-centeredness. Collaborations between academic medical centers and community hospitals may be an effective means to connect researchers with community hospital clinical teams to facilitate the application of robust study designs and analytic approaches and to facilitate multisite investigations. Research in this field would benefit from a standardized definition of a community hospital that could be consistently applied in research and QI endeavors.

Disclosures

The authors have no potential conflicts of interest to disclose.

Funding

Jana Leary was supported by the National Center for Advancing Translational Sciences, National Institutes of Health, Grant Number 5TL1TR0001062-03. JoAnna Leyenaar was supported by the Agency for Healthcare Research and Quality (K08HS024133).

Disclaimer

The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or the AHRQ.

Despite efforts to provide high-quality healthcare, Americans die from medical errors each year and many patients do not receive recommended medical care. Risk is particularly acute during times of hospitalization.^1-4 In response, the Institute of Medicine (IOM, now the Academy of Medicine) has released “Crossing the Quality Chasm: A New Health System for the 21^st Century,” providing a framework to guide delivery and measurement of high-quality healthcare.⁵

Although the IOM framework has motivated the development of quality improvement (QI) and quality measurement initiatives, relatively few resources have been allocated to improving the quality of pediatric inpatient care.^6,7 The resultant gap in our knowledge of quality and safety of pediatric hospital-based care is further widened by the variability of settings in which children are hospitalized. These settings include freestanding children’s hospitals, children’s hospitals nested within larger hospitals, and community hospitals, defined as general, nonuniversity, and nonchildren’s hospitals.⁸

Although almost three-quarters of children needing hospitalization are cared for outside of freestanding children’s hospitals, we know particularly little about the quality and safety of pediatric hospital-based care outside of these settings.^6,9 Therefore, our scoping review aims to summarize literature regarding the quality and safety of pediatric inpatient care within community hospitals.

We used a scoping review approach because this methodology, by design, is utilized to synthesize evidence and map existing literature and is particularly useful when a body of literature is heterogeneous, rendering a more targeted systematic review approach infeasible.¹⁰ This methodology thereby provided an organized approach to answer our broad research question, “What evidence exists regarding the quality and safety of pediatric inpatient care in United States community hospitals?“ We followed the scoping review guidelines put forth by the Joanna Briggs Institute and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting guideline Extension for Scoping Reviews.^10,11

Data Sources and Search Strategies

We searched Medline, Medline-In-Process, Embase, the Cochrane Database of Systematic Reviews, the Cumulative Index to Nursing and Allied Health Literature (CINAHL), PsycINFO, and Scopus for studies that reported at least one outcome related to healthcare quality or patient safety and involved pediatric patients (aged <18 years) receiving inpatient care at a community hospital. Outcomes included measures from the IOM-defined aims of quality healthcare: (1) safety, (2) effectiveness, (3) efficiency, (4) timeliness, (5) patient-centeredness, and (6) equity (Appendix Table 1). Terms were searched as controlled vocabulary in applicable databases (Medline, Embase, CINAHL, PsycINFO) and as keywords in all databases. Search strategies tailored to each database were developed, tested, and refined in collaboration with a reference librarian. Date parameters for retrieval were set from 1989 to the search date, with the start date chosen to correspond with the establishment of the Agency for Health Care Policy and Research, currently known as the Agency for Healthcare Research and Quality (AHRQ), targeting literature produced in the wake of the AHRQ emphasis on quality in healthcare. Results were limited to articles published in English. Searches were conducted on October 24 and 25, 2016. Complete search strategies can be found in Appendix Methods.

Independent authors performed handsearches of Academic Pediatrics, BMJ Quality & Safety, Hospital Pediatrics, JAMA Pediatrics, and Pediatrics Quality Reports for the five years preceding the search date (July 2011-October 2016).

Study Selection and Definitions

To identify studies conducted in community hospitals, we operationalized the definition of community hospital proposed by Percelay.⁸ We used “community”, “general”, “nonuniversity”, and “nonchildren’s” as search terms and operationalized these through the addition of qualifying descriptors (Table 1).

Studies were excluded if they (1) were performed outside of the United States, as community hospital definitions may differ by country; (2) included participants aged >18 years and did not report any pediatric-specific results; (3) were performed exclusively in tertiary hospitals; (4) evaluated only outpatient or emergency department care; (5) did not report any results specific to community hospitals; (6) did not report any data-driven or parent/patient-reported measures of safety, effectiveness, efficiency, timeliness, patient-centeredness, or equity in their results; and (7) were case reports, case series, editorials, or abstracts without an associated full-text article. We read commentaries and literature reviews related to our objectives and reviewed their references to identify additional articles, but did not include these manuscripts.

Two authors independently reviewed each abstract, and full-text articles were reviewed if one or more authors determined that the abstract met the inclusion criteria. Two authors then independently reviewed each full-text article to determine whether the article met the criteria for the final review. Disagreements were resolved through consensus after discussion and review with the entire research team, and reasons for exclusion were recorded.

Charting the Results and Data Synthesis

We used a standardized charting form to collect information regarding study design, community hospital terms, population descriptors, IOM aims of quality healthcare, and outcome measures. To minimize bias in data collection, information from each full-text article was extracted independently by two authors, and differences in extraction were resolved through discussion and re-review among the same two authors. To evaluate the quality of included evidence, two authors (JCL and JKL) independently assessed the risk of bias for each study using modified Newcastle-Ottawa Quality Assessment Scales (NOS), with disagreements resolved through discussion and re-review among the same two authors. For cohort studies, five of the eight NOS domains were relevant and applied to all included studies (maximum score 6). Using the NOS adapted for cross-sectional studies, six of the seven domains were relevant and applied (maximum score 9). For cross-sectional studies, we defined the risk of bias to be low for scores ≥8, moderate for scores 5-7, and high for scores ≤4, consistent with previous work.¹² For cohort studies, we similarly defined these strata by scores ≥5, 3-4, and ≤2, given the lower maximum score for this study type.

We categorized studies as either observational, defined as cohort or cross-sectional studies of usual healthcare delivery, or interventional, defined as studies evaluating the development and/or implementation of an intervention designed to improve healthcare quality. We further categorized the studies into the following four overarching medical domains: (1) neonatal, (2) pediatric medicine, (3) surgery, or (4) radiology.

After removal of duplicates, our search identified 2,068 abstracts for screening (Figure). Of these, 1,777 did not meet the inclusion criteria, leaving 291 articles for full-text review. Of these, 43 articles met all the inclusion criteria, and one additional article was included from search of references, resulting in a total of 44 articles.

Study designs, patient populations, and quality outcome measures were heterogeneous. We identified only one randomized controlled trial (RCT). A total of 30 articles were observational studies, 27 of which used retrospective cohort or cross-sectional designs; the remaining three used prospective cohort designs (Table 2). Of these studies, 20 involved multiple hospitals, whereas 10 were conducted at a single community hospital. Sample sizes ranged from 29 (single-site) to 107,727 (multisite) patients. Twenty-two studies aimed to compare quality outcomes at community hospitals with other hospital types, of which 16 performed risk-adjusted analyses (detailed findings of observational studies are summarized in Appendix Table 2). The remaining 14 articles were interventional studies (Table 2). Of these, 12 (86%) reported improvement in quality outcomes after implementation (detailed findings of interventional studies are summarized in Appendix Table 3).

Observational Studies

Neonatal Medicine

Five multisite studies focused on neonatal care,^{13,14,16,19,20} of which four examined outcomes associated with the transfer of neonates to or from community hospitals to tertiary care hospitals, such as neonatal morbidity, readmission, completion of preventative health measures and screening, parent satisfaction, and hospital charges.^14,16,19,20 For example, in a study of extremely premature very low birth weight infants born in Hawaii, Kuo et al. demonstrated that the odds of retinopathy of prematurity was 2.9 times higher for infants born at a community hospital and transported to their tertiary center compared with those inborn at the tertiary center (P = .02).¹⁶ The fifth study examined neonatal mortality by site of birth, demonstrating that, among infants with birth weights less than 2,000 g, birth at a hospital with a community neonatal intensive care unit (NICU) was associated with 1.4 times higher odds of risk-adjusted mortality compared with birth at a regional NICU (P < .001).¹³

Three studies evaluated the quality of neonatal care at a single community hospital.^15,17,18 Quality outcomes were heterogeneous, including utility of rebound bilirubin levels for infants with jaundice, morbidity of neonates requiring mechanical ventilation, and provision of breastfeeding advice to mothers of breastfeeding infants. For instance, Meadow et al. attempted to determine the quality of care for ventilated neonates at one community NICU compared with a tertiary hospital.¹⁸ They found no difference in days on ventilation or need for home oxygen therapy between the community hospital and the tertiary center, although P values and effect sizes were not reported for these outcomes and analyses were not adjusted beyond matching on birthdate and birth weight.

Pediatric Medicine

Nine multisite studies explored the quality and safety of pediatric medical care across hospital types.^21-23,25-30 Of these, four were conducted using the Kids’ Inpatient Database (KID)^28-30 or the National Inpatient Sample (NIS),²⁷ two were conducted using other national databases,^21,25 and two were conducted using electronic medical record data.^22,26 All of the KID and NIS studies examined hospital charges, either alone or in conjunction with mortality. Two of these studies found no differences in risk-adjusted hospital charges between children’s hospitals and community hospitals (for burn injuries),^28,29 whereas two found that hospitalization at community hospitals was associated with lower risk-adjusted charges (for asthma and sepsis).^27,30 The remaining five studies evaluated diverse quality outcomes such as medication errors, therapeutic drug monitoring, practice guideline compliance, antibiotic prescribing, or hospital-to-home transition summary scores.^21-23,25,26

Three single-site studies examined quality outcomes for pediatric medical patients, including mortality, outcome ratings for dehydration, and measures of peripherally inserted central catheter (PICC) safety and effectiveness.^24,31,32 For example, Frank et al. evaluated safety of care in one community hospital pediatric intensive care unit (PICU) compared with a tertiary hospital using the Pediatric Risk of Mortality (PRISM) score.²⁴ They reported that the observed number of deaths in their community PICU did not differ significantly from the number of deaths predicted in a tertiary center (23 vs 33, respectively, P > .2).

Surgery

Three multisite studies examined quality outcomes among children with surgical conditions, including surgical complications, readmission, and hospital charges.^34,35,37 For example, Kelley-Quon et al. examined outcomes following surgery in infants with hypertrophic pyloric stenosis and found that infants who received their surgery at community hospitals had twice the odds of a surgical complication compared with those at children’s hospitals (P = .027).³⁴ They also examined how the risk of appendiceal perforation differed by hospital type, finding that black children who received their surgery at children’s hospitals had twice the odds of appendiceal perforation compared with those who received care at community hospitals.³⁵

In addition, two studies evaluated quality and safety outcomes for pediatric surgical care in a single community hospital.^33,36 For example, Beaty et al. prospectively evaluated the incidence of missed injuries in hospitalized pediatric trauma patients, reporting that the incidence of missed injury was 33% when admission evaluation was performed by a trauma surgeon alone compared to 11% when performed by a pediatric doctor or a trauma surgeon and a pediatric doctor together (P < .001).³³

Radiology

Three multisite studies examined quality and safety outcomes associated with radiographic imaging in community hospitals, including radiation dosing and frequency of preoperative imaging modalities and accuracy.^38,39,41 For instance, Marin et al. found substantial variation in radiation dose across hospital types, with children’s hospitals delivering lower median radiation doses than academic and community hospitals.³⁹ Similarly, Saito et al. demonstrated increased use of radiating modalities when evaluation was performed at community hospitals, with four times higher odds of computed tomography (CT) and five times lower odds of ultrasound use compared to a children’s hospital.⁴¹

Two single-site studies also evaluated quality and safety outcomes associated with the use of radiographic imaging for pediatric appendicitis in a community hospital.^40,42 For example, York et al. demonstrated that, compared with nonimaged patients, patients who underwent diagnostic imaging for appendicitis experienced a significant time delay from initial evaluation to surgery and incurred significantly higher hospital charges, whereas there were no significant differences in intraoperative findings, antibiotic requirements, and surgical complications between the groups.⁴²

Interventional Studies

Neonatal Medicine

We identified six studies that evaluated interventions to improve healthcare quality for neonates in community hospitals; two involved telemedicine.^43-48 Hall et al. described “Telenursery,” a program linking regional perinatal centers with a large academic neonatal practice through real-time teleconferencing, in addition to providing weekly educational conferences.⁴⁵ After its implementation, there was an increase in the state-recommended delivery of very low birth weight infants at the regional perinatal center from 24% to 33% (P < .05); clinical outcomes were not discussed. In a similar study, Sable et al. found that a videoconferencing system for cardiologists from an academic center to guide care in a community setting provided diagnostic services more quickly (28 minutes vs 12 hours) and had high diagnostic accuracy.⁴⁷ The remaining four studies evaluated heterogeneous interventions such as a maternal education program to reduce shaking injuries to infants or implementation of evidence-based order sets to reduce early onset group B streptococcal disease in neonates.^43,44,46,48 Five of the six neonatal studies demonstrated improved quality outcomes after intervention.^43-47

Pediatric Medicine

Seven studies evaluated QI interventions for children at community hospitals,^49-55 three of which described interventions to improve quality of management of respiratory diseases.^49,51,53 Dayal et al. evaluated the implementation of respiratory illness order sets and an asthma pathway, demonstrating a 41% reduction in asthma hospitalization cost per patient (P < .05), reduced bronchodilator use for all respiratory illnesses, and no change in readmission rates.⁴⁹ Similarly, Nkoy et al. evaluated an asthma “Evidence-Based Care Practice Model” implemented at seven community hospitals and demonstrated a nonsignificant reduction in readmissions (P = .12), as well as lower hospitalization costs (P = .05).⁵³ Using QI methods, Kuhlmann et al. demonstrated improved compliance from 43% to 97% with the Asthma Home Management Plan of Care measure.⁵¹ The remaining four studies evaluated heterogeneous interventions, including telemedicine critical care consultations, use of I-PASS, and consolidation of pediatric care onto one hospital unit.^50,52,54,55 Six of the seven pediatric studies (86%) demonstrated improved inpatient quality outcomes after intervention,^49,51-55 but only one study was multisite.⁵³

Surgery

We identified only one study evaluating a surgical intervention aimed at improving the quality of pediatric care.⁵⁶ Kelley-Quon evaluated the impact of a community hospital partnering with an Academic Medical Pediatric Trauma Center to become a Level II Pediatric Trauma Center (PTC). After achieving Level II PTC designation, they reported that children treated at the community hospital had reduced rates of CT use, transfers, and in-hospital mortality (from 81% to 51%, 8.5% to 2.5%, and 2% to 0.4%, respectively, P < .05 for all) compared to those treated predesignation.

Radiology

Within the domain of radiology, our review identified no interventional studies.

In this scoping review of the quality and safety of pediatric inpatient care in community hospitals, we identified 44 studies applying heterogeneous study designs and evaluating diverse patient populations and quality outcomes. We identified only one RCT in our search; all the remaining studies applied observational designs.

We found only three clinical areas that were explored in multiple studies, with consistent directionality of results: (1) perinatal regionalization, (2) telemedicine, and (3) imaging radiation. The limited evidence identified in our review suggests that delivery of early premature infants at community hospitals, rather than at tertiary hospitals, may increase risk of neonatal morbidity and/or mortality,^13,16,46 that use of telemedicine may improve the effectiveness and efficiency of intensive or specialized care in community settings,^45,47,52,55 and that CT use and radiation doses may be higher in community hospitals compared with other settings.^38,39,41 However, even within these clinical domains, the literature was limited in amount and heterogeneous; additional research is needed to systematically review the effect of individual interventions or particular community hospital quality outcomes compared with other hospital types.

Our search identified only 14 studies evaluating QI interventions within community hospitals over the almost 30-year review period. Although limited in number, 86% of these demonstrated improvements in healthcare quality and safety, providing a positive “proof of concept” that pediatric care in community hospitals can be improved by multidisciplinary efforts and be sustained over time.^{43-47,49,51-56} As pediatric departments within community hospitals may have limited resources, aligning pediatric QI efforts with adult initiatives within the same hospitals could prove advantageous. However, we did not identify any studies meeting our inclusion criteria that took this approach. Alternatively, QI collaboratives across structurally diverse hospitals may provide valuable infrastructure for QI in community hospitals. For example, the Value in Inpatient Pediatrics Network has conducted several multisite QI initiatives that have engaged both children’s and community hospitals.^57-60 However, none of these studies have reported community hospital-specific quality outcomes, resulting in their exclusion from this review. In future study, researchers may consider separating community hospital results from those of pediatric hospitals to highlight the effect of community hospital-specific QI efforts and to allow valuable direct comparisons between hospital types.

Many studies identified in our scoping review were conducted at single hospitals. Sample sizes were often very small and power calculations were rarely reported, raising questions about the validity of the “nonsignificant” differences reported by some. Although such single-center studies provide valuable information for local improvement of inpatient care, by design the findings from these studies are unlikely generalizable to other community hospital systems. Without inclusion of another hospital type and/or quality measures with clear national benchmark for comparison, the additional conclusions that can be drawn from these studies are limited. In comparison, many of the multicenter studies had large study samples, but more than half used data registries with limited data to evaluate outcomes, clinical context, or important covariates.^{13,20,21,25,27-30,34,35,37,39} The most frequently reported outcomes in these data sets—hospital charges and lengths of stay—are of limited utility in understanding healthcare quality without clear benchmarks. As a result, the evidence-base from which to draw conclusions about quality and safety in community hospitals is very limited.

Therefore, our review highlights a great need for additional research in community hospital medicine and the need for high-quality evidence generation. Risk of bias was moderate or high for the majority of included studies because of inadequate risk adjustment or statistical analysis. In the future, multicenter collaborations may help to connect research methodologists with community hospital teams to aid in the application of robust study designs and analytic techniques. Multisite collaboration may also overcome the limitation of small sample sizes that are a reality at many community hospitals.

Our study findings must be considered in the setting of several methodological limitations. The lack of a standard definition for a community hospital has led to inconsistent terms and hospital definitions used in the literature. It is possible that, while following our systematic approach to define a community hospital, we inadvertently missed relevant studies that used different terms. We also excluded unpublished articles. Given that publication bias tends to favor studies with significant associations, it is possible that some studies with insignificant changes in quality outcomes were missed. Finally, in our exclusion of all non-US studies, we may have unknowingly missed literature from countries with community hospital definitions similar to those in the United States.

Recognizing that more than half of all children admitted to hospitals in the US receive their care at community hospitals, understanding healthcare quality in community hospitals is important. This scoping review underscores the need for additional research and higher quality evidence to determine the quality of pediatric inpatient care in these settings and identifies some particularly wide gaps that could be targeted in future research. Acknowledging that further research is necessary to address all aims of quality healthcare, markedly few studies have examined timeliness, equity, or patient-centeredness. Collaborations between academic medical centers and community hospitals may be an effective means to connect researchers with community hospital clinical teams to facilitate the application of robust study designs and analytic approaches and to facilitate multisite investigations. Research in this field would benefit from a standardized definition of a community hospital that could be consistently applied in research and QI endeavors.

Disclosures

The authors have no potential conflicts of interest to disclose.

Funding

Jana Leary was supported by the National Center for Advancing Translational Sciences, National Institutes of Health, Grant Number 5TL1TR0001062-03. JoAnna Leyenaar was supported by the Agency for Healthcare Research and Quality (K08HS024133).

Disclaimer

The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or the AHRQ.

Despite efforts to provide high-quality healthcare, Americans die from medical errors each year and many patients do not receive recommended medical care. Risk is particularly acute during times of hospitalization.^1-4 In response, the Institute of Medicine (IOM, now the Academy of Medicine) has released “Crossing the Quality Chasm: A New Health System for the 21^st Century,” providing a framework to guide delivery and measurement of high-quality healthcare.⁵

Although the IOM framework has motivated the development of quality improvement (QI) and quality measurement initiatives, relatively few resources have been allocated to improving the quality of pediatric inpatient care.^6,7 The resultant gap in our knowledge of quality and safety of pediatric hospital-based care is further widened by the variability of settings in which children are hospitalized. These settings include freestanding children’s hospitals, children’s hospitals nested within larger hospitals, and community hospitals, defined as general, nonuniversity, and nonchildren’s hospitals.⁸

Although almost three-quarters of children needing hospitalization are cared for outside of freestanding children’s hospitals, we know particularly little about the quality and safety of pediatric hospital-based care outside of these settings.^6,9 Therefore, our scoping review aims to summarize literature regarding the quality and safety of pediatric inpatient care within community hospitals.

We used a scoping review approach because this methodology, by design, is utilized to synthesize evidence and map existing literature and is particularly useful when a body of literature is heterogeneous, rendering a more targeted systematic review approach infeasible.¹⁰ This methodology thereby provided an organized approach to answer our broad research question, “What evidence exists regarding the quality and safety of pediatric inpatient care in United States community hospitals?“ We followed the scoping review guidelines put forth by the Joanna Briggs Institute and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting guideline Extension for Scoping Reviews.^10,11

Data Sources and Search Strategies

We searched Medline, Medline-In-Process, Embase, the Cochrane Database of Systematic Reviews, the Cumulative Index to Nursing and Allied Health Literature (CINAHL), PsycINFO, and Scopus for studies that reported at least one outcome related to healthcare quality or patient safety and involved pediatric patients (aged <18 years) receiving inpatient care at a community hospital. Outcomes included measures from the IOM-defined aims of quality healthcare: (1) safety, (2) effectiveness, (3) efficiency, (4) timeliness, (5) patient-centeredness, and (6) equity (Appendix Table 1). Terms were searched as controlled vocabulary in applicable databases (Medline, Embase, CINAHL, PsycINFO) and as keywords in all databases. Search strategies tailored to each database were developed, tested, and refined in collaboration with a reference librarian. Date parameters for retrieval were set from 1989 to the search date, with the start date chosen to correspond with the establishment of the Agency for Health Care Policy and Research, currently known as the Agency for Healthcare Research and Quality (AHRQ), targeting literature produced in the wake of the AHRQ emphasis on quality in healthcare. Results were limited to articles published in English. Searches were conducted on October 24 and 25, 2016. Complete search strategies can be found in Appendix Methods.

Independent authors performed handsearches of Academic Pediatrics, BMJ Quality & Safety, Hospital Pediatrics, JAMA Pediatrics, and Pediatrics Quality Reports for the five years preceding the search date (July 2011-October 2016).

Study Selection and Definitions

To identify studies conducted in community hospitals, we operationalized the definition of community hospital proposed by Percelay.⁸ We used “community”, “general”, “nonuniversity”, and “nonchildren’s” as search terms and operationalized these through the addition of qualifying descriptors (Table 1).

Studies were excluded if they (1) were performed outside of the United States, as community hospital definitions may differ by country; (2) included participants aged >18 years and did not report any pediatric-specific results; (3) were performed exclusively in tertiary hospitals; (4) evaluated only outpatient or emergency department care; (5) did not report any results specific to community hospitals; (6) did not report any data-driven or parent/patient-reported measures of safety, effectiveness, efficiency, timeliness, patient-centeredness, or equity in their results; and (7) were case reports, case series, editorials, or abstracts without an associated full-text article. We read commentaries and literature reviews related to our objectives and reviewed their references to identify additional articles, but did not include these manuscripts.

Two authors independently reviewed each abstract, and full-text articles were reviewed if one or more authors determined that the abstract met the inclusion criteria. Two authors then independently reviewed each full-text article to determine whether the article met the criteria for the final review. Disagreements were resolved through consensus after discussion and review with the entire research team, and reasons for exclusion were recorded.

Charting the Results and Data Synthesis

We used a standardized charting form to collect information regarding study design, community hospital terms, population descriptors, IOM aims of quality healthcare, and outcome measures. To minimize bias in data collection, information from each full-text article was extracted independently by two authors, and differences in extraction were resolved through discussion and re-review among the same two authors. To evaluate the quality of included evidence, two authors (JCL and JKL) independently assessed the risk of bias for each study using modified Newcastle-Ottawa Quality Assessment Scales (NOS), with disagreements resolved through discussion and re-review among the same two authors. For cohort studies, five of the eight NOS domains were relevant and applied to all included studies (maximum score 6). Using the NOS adapted for cross-sectional studies, six of the seven domains were relevant and applied (maximum score 9). For cross-sectional studies, we defined the risk of bias to be low for scores ≥8, moderate for scores 5-7, and high for scores ≤4, consistent with previous work.¹² For cohort studies, we similarly defined these strata by scores ≥5, 3-4, and ≤2, given the lower maximum score for this study type.

We categorized studies as either observational, defined as cohort or cross-sectional studies of usual healthcare delivery, or interventional, defined as studies evaluating the development and/or implementation of an intervention designed to improve healthcare quality. We further categorized the studies into the following four overarching medical domains: (1) neonatal, (2) pediatric medicine, (3) surgery, or (4) radiology.

After removal of duplicates, our search identified 2,068 abstracts for screening (Figure). Of these, 1,777 did not meet the inclusion criteria, leaving 291 articles for full-text review. Of these, 43 articles met all the inclusion criteria, and one additional article was included from search of references, resulting in a total of 44 articles.

Study designs, patient populations, and quality outcome measures were heterogeneous. We identified only one randomized controlled trial (RCT). A total of 30 articles were observational studies, 27 of which used retrospective cohort or cross-sectional designs; the remaining three used prospective cohort designs (Table 2). Of these studies, 20 involved multiple hospitals, whereas 10 were conducted at a single community hospital. Sample sizes ranged from 29 (single-site) to 107,727 (multisite) patients. Twenty-two studies aimed to compare quality outcomes at community hospitals with other hospital types, of which 16 performed risk-adjusted analyses (detailed findings of observational studies are summarized in Appendix Table 2). The remaining 14 articles were interventional studies (Table 2). Of these, 12 (86%) reported improvement in quality outcomes after implementation (detailed findings of interventional studies are summarized in Appendix Table 3).

Observational Studies

Neonatal Medicine

Five multisite studies focused on neonatal care,^{13,14,16,19,20} of which four examined outcomes associated with the transfer of neonates to or from community hospitals to tertiary care hospitals, such as neonatal morbidity, readmission, completion of preventative health measures and screening, parent satisfaction, and hospital charges.^14,16,19,20 For example, in a study of extremely premature very low birth weight infants born in Hawaii, Kuo et al. demonstrated that the odds of retinopathy of prematurity was 2.9 times higher for infants born at a community hospital and transported to their tertiary center compared with those inborn at the tertiary center (P = .02).¹⁶ The fifth study examined neonatal mortality by site of birth, demonstrating that, among infants with birth weights less than 2,000 g, birth at a hospital with a community neonatal intensive care unit (NICU) was associated with 1.4 times higher odds of risk-adjusted mortality compared with birth at a regional NICU (P < .001).¹³

Three studies evaluated the quality of neonatal care at a single community hospital.^15,17,18 Quality outcomes were heterogeneous, including utility of rebound bilirubin levels for infants with jaundice, morbidity of neonates requiring mechanical ventilation, and provision of breastfeeding advice to mothers of breastfeeding infants. For instance, Meadow et al. attempted to determine the quality of care for ventilated neonates at one community NICU compared with a tertiary hospital.¹⁸ They found no difference in days on ventilation or need for home oxygen therapy between the community hospital and the tertiary center, although P values and effect sizes were not reported for these outcomes and analyses were not adjusted beyond matching on birthdate and birth weight.

Pediatric Medicine

Nine multisite studies explored the quality and safety of pediatric medical care across hospital types.^21-23,25-30 Of these, four were conducted using the Kids’ Inpatient Database (KID)^28-30 or the National Inpatient Sample (NIS),²⁷ two were conducted using other national databases,^21,25 and two were conducted using electronic medical record data.^22,26 All of the KID and NIS studies examined hospital charges, either alone or in conjunction with mortality. Two of these studies found no differences in risk-adjusted hospital charges between children’s hospitals and community hospitals (for burn injuries),^28,29 whereas two found that hospitalization at community hospitals was associated with lower risk-adjusted charges (for asthma and sepsis).^27,30 The remaining five studies evaluated diverse quality outcomes such as medication errors, therapeutic drug monitoring, practice guideline compliance, antibiotic prescribing, or hospital-to-home transition summary scores.^21-23,25,26

Three single-site studies examined quality outcomes for pediatric medical patients, including mortality, outcome ratings for dehydration, and measures of peripherally inserted central catheter (PICC) safety and effectiveness.^24,31,32 For example, Frank et al. evaluated safety of care in one community hospital pediatric intensive care unit (PICU) compared with a tertiary hospital using the Pediatric Risk of Mortality (PRISM) score.²⁴ They reported that the observed number of deaths in their community PICU did not differ significantly from the number of deaths predicted in a tertiary center (23 vs 33, respectively, P > .2).

Surgery

Three multisite studies examined quality outcomes among children with surgical conditions, including surgical complications, readmission, and hospital charges.^34,35,37 For example, Kelley-Quon et al. examined outcomes following surgery in infants with hypertrophic pyloric stenosis and found that infants who received their surgery at community hospitals had twice the odds of a surgical complication compared with those at children’s hospitals (P = .027).³⁴ They also examined how the risk of appendiceal perforation differed by hospital type, finding that black children who received their surgery at children’s hospitals had twice the odds of appendiceal perforation compared with those who received care at community hospitals.³⁵

In addition, two studies evaluated quality and safety outcomes for pediatric surgical care in a single community hospital.^33,36 For example, Beaty et al. prospectively evaluated the incidence of missed injuries in hospitalized pediatric trauma patients, reporting that the incidence of missed injury was 33% when admission evaluation was performed by a trauma surgeon alone compared to 11% when performed by a pediatric doctor or a trauma surgeon and a pediatric doctor together (P < .001).³³

Radiology

Three multisite studies examined quality and safety outcomes associated with radiographic imaging in community hospitals, including radiation dosing and frequency of preoperative imaging modalities and accuracy.^38,39,41 For instance, Marin et al. found substantial variation in radiation dose across hospital types, with children’s hospitals delivering lower median radiation doses than academic and community hospitals.³⁹ Similarly, Saito et al. demonstrated increased use of radiating modalities when evaluation was performed at community hospitals, with four times higher odds of computed tomography (CT) and five times lower odds of ultrasound use compared to a children’s hospital.⁴¹

Two single-site studies also evaluated quality and safety outcomes associated with the use of radiographic imaging for pediatric appendicitis in a community hospital.^40,42 For example, York et al. demonstrated that, compared with nonimaged patients, patients who underwent diagnostic imaging for appendicitis experienced a significant time delay from initial evaluation to surgery and incurred significantly higher hospital charges, whereas there were no significant differences in intraoperative findings, antibiotic requirements, and surgical complications between the groups.⁴²

Interventional Studies

Neonatal Medicine

We identified six studies that evaluated interventions to improve healthcare quality for neonates in community hospitals; two involved telemedicine.^43-48 Hall et al. described “Telenursery,” a program linking regional perinatal centers with a large academic neonatal practice through real-time teleconferencing, in addition to providing weekly educational conferences.⁴⁵ After its implementation, there was an increase in the state-recommended delivery of very low birth weight infants at the regional perinatal center from 24% to 33% (P < .05); clinical outcomes were not discussed. In a similar study, Sable et al. found that a videoconferencing system for cardiologists from an academic center to guide care in a community setting provided diagnostic services more quickly (28 minutes vs 12 hours) and had high diagnostic accuracy.⁴⁷ The remaining four studies evaluated heterogeneous interventions such as a maternal education program to reduce shaking injuries to infants or implementation of evidence-based order sets to reduce early onset group B streptococcal disease in neonates.^43,44,46,48 Five of the six neonatal studies demonstrated improved quality outcomes after intervention.^43-47

Pediatric Medicine

Seven studies evaluated QI interventions for children at community hospitals,^49-55 three of which described interventions to improve quality of management of respiratory diseases.^49,51,53 Dayal et al. evaluated the implementation of respiratory illness order sets and an asthma pathway, demonstrating a 41% reduction in asthma hospitalization cost per patient (P < .05), reduced bronchodilator use for all respiratory illnesses, and no change in readmission rates.⁴⁹ Similarly, Nkoy et al. evaluated an asthma “Evidence-Based Care Practice Model” implemented at seven community hospitals and demonstrated a nonsignificant reduction in readmissions (P = .12), as well as lower hospitalization costs (P = .05).⁵³ Using QI methods, Kuhlmann et al. demonstrated improved compliance from 43% to 97% with the Asthma Home Management Plan of Care measure.⁵¹ The remaining four studies evaluated heterogeneous interventions, including telemedicine critical care consultations, use of I-PASS, and consolidation of pediatric care onto one hospital unit.^50,52,54,55 Six of the seven pediatric studies (86%) demonstrated improved inpatient quality outcomes after intervention,^49,51-55 but only one study was multisite.⁵³

Surgery

We identified only one study evaluating a surgical intervention aimed at improving the quality of pediatric care.⁵⁶ Kelley-Quon evaluated the impact of a community hospital partnering with an Academic Medical Pediatric Trauma Center to become a Level II Pediatric Trauma Center (PTC). After achieving Level II PTC designation, they reported that children treated at the community hospital had reduced rates of CT use, transfers, and in-hospital mortality (from 81% to 51%, 8.5% to 2.5%, and 2% to 0.4%, respectively, P < .05 for all) compared to those treated predesignation.

Radiology

Within the domain of radiology, our review identified no interventional studies.

In this scoping review of the quality and safety of pediatric inpatient care in community hospitals, we identified 44 studies applying heterogeneous study designs and evaluating diverse patient populations and quality outcomes. We identified only one RCT in our search; all the remaining studies applied observational designs.

We found only three clinical areas that were explored in multiple studies, with consistent directionality of results: (1) perinatal regionalization, (2) telemedicine, and (3) imaging radiation. The limited evidence identified in our review suggests that delivery of early premature infants at community hospitals, rather than at tertiary hospitals, may increase risk of neonatal morbidity and/or mortality,^13,16,46 that use of telemedicine may improve the effectiveness and efficiency of intensive or specialized care in community settings,^45,47,52,55 and that CT use and radiation doses may be higher in community hospitals compared with other settings.^38,39,41 However, even within these clinical domains, the literature was limited in amount and heterogeneous; additional research is needed to systematically review the effect of individual interventions or particular community hospital quality outcomes compared with other hospital types.

Our search identified only 14 studies evaluating QI interventions within community hospitals over the almost 30-year review period. Although limited in number, 86% of these demonstrated improvements in healthcare quality and safety, providing a positive “proof of concept” that pediatric care in community hospitals can be improved by multidisciplinary efforts and be sustained over time.^{43-47,49,51-56} As pediatric departments within community hospitals may have limited resources, aligning pediatric QI efforts with adult initiatives within the same hospitals could prove advantageous. However, we did not identify any studies meeting our inclusion criteria that took this approach. Alternatively, QI collaboratives across structurally diverse hospitals may provide valuable infrastructure for QI in community hospitals. For example, the Value in Inpatient Pediatrics Network has conducted several multisite QI initiatives that have engaged both children’s and community hospitals.^57-60 However, none of these studies have reported community hospital-specific quality outcomes, resulting in their exclusion from this review. In future study, researchers may consider separating community hospital results from those of pediatric hospitals to highlight the effect of community hospital-specific QI efforts and to allow valuable direct comparisons between hospital types.

Many studies identified in our scoping review were conducted at single hospitals. Sample sizes were often very small and power calculations were rarely reported, raising questions about the validity of the “nonsignificant” differences reported by some. Although such single-center studies provide valuable information for local improvement of inpatient care, by design the findings from these studies are unlikely generalizable to other community hospital systems. Without inclusion of another hospital type and/or quality measures with clear national benchmark for comparison, the additional conclusions that can be drawn from these studies are limited. In comparison, many of the multicenter studies had large study samples, but more than half used data registries with limited data to evaluate outcomes, clinical context, or important covariates.^{13,20,21,25,27-30,34,35,37,39} The most frequently reported outcomes in these data sets—hospital charges and lengths of stay—are of limited utility in understanding healthcare quality without clear benchmarks. As a result, the evidence-base from which to draw conclusions about quality and safety in community hospitals is very limited.

Therefore, our review highlights a great need for additional research in community hospital medicine and the need for high-quality evidence generation. Risk of bias was moderate or high for the majority of included studies because of inadequate risk adjustment or statistical analysis. In the future, multicenter collaborations may help to connect research methodologists with community hospital teams to aid in the application of robust study designs and analytic techniques. Multisite collaboration may also overcome the limitation of small sample sizes that are a reality at many community hospitals.

Our study findings must be considered in the setting of several methodological limitations. The lack of a standard definition for a community hospital has led to inconsistent terms and hospital definitions used in the literature. It is possible that, while following our systematic approach to define a community hospital, we inadvertently missed relevant studies that used different terms. We also excluded unpublished articles. Given that publication bias tends to favor studies with significant associations, it is possible that some studies with insignificant changes in quality outcomes were missed. Finally, in our exclusion of all non-US studies, we may have unknowingly missed literature from countries with community hospital definitions similar to those in the United States.

Recognizing that more than half of all children admitted to hospitals in the US receive their care at community hospitals, understanding healthcare quality in community hospitals is important. This scoping review underscores the need for additional research and higher quality evidence to determine the quality of pediatric inpatient care in these settings and identifies some particularly wide gaps that could be targeted in future research. Acknowledging that further research is necessary to address all aims of quality healthcare, markedly few studies have examined timeliness, equity, or patient-centeredness. Collaborations between academic medical centers and community hospitals may be an effective means to connect researchers with community hospital clinical teams to facilitate the application of robust study designs and analytic approaches and to facilitate multisite investigations. Research in this field would benefit from a standardized definition of a community hospital that could be consistently applied in research and QI endeavors.

Disclosures

The authors have no potential conflicts of interest to disclose.

Funding

Jana Leary was supported by the National Center for Advancing Translational Sciences, National Institutes of Health, Grant Number 5TL1TR0001062-03. JoAnna Leyenaar was supported by the Agency for Healthcare Research and Quality (K08HS024133).

Disclaimer

The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or the AHRQ.

Every day in the United States, approximately 4% of patients in acute care hospitals have at least one hospital-acquired infection (HAI).^1,2 Among the top 10 causes of death in the United States, HAIs are associated with increased morbidity, mortality, and hospital length of stay (LOS).² The direct medical cost of treating HAIs is substantial for both hospitals and patients.^3,4 Urinary tract infections (UTIs) are a leading cause of HAI, and 70%-80% of these are catheter-associated urinary tract infections (CAUTIs).^5,6 In 2016, 26,983 CAUTIs occurred in acute care hospitals.⁷ The high incidence of CAUTI can substantially contribute to morbidity, length of stay, and mortality.^8-11

The recognition that a substantial proportion of HAIs may be preventable, including 55%-70% of CAUTIs,¹² has resulted in implementing multiple strategies to reduce CAUTI rates.^13-17 These include simple prevention interventions such as avoiding placement of unnecessary indwelling urinary catheters and early removal of urinary catheters when they are no longer clinically indicated. Hospitalists are responsible for the care of many, if not most, inpatients with indwelling urinary catheters and are integral in antimicrobial stewardship efforts surrounding CAUTIs.¹⁸ Diagnostic stewardship, including appropriate urine specimen ordering, collection, processing, and reporting, works synergistically with antimicrobial stewardship and allows for appropriate antibiotic prescribing in symptomatic patients.¹⁹

CAUTIs can be defined using either clinical or surveillance definitions. Clinical definitions are used at the bedside and take individual clinical characteristics into consideration, but vary among clinicians since there is no gold standard. Abnormal laboratory urinary findings in the absence of symptoms are not sufficient for the diagnosis of UTI, including CAUTI. Surveillance definitions, such as those used by the Centers for Disease Control and Prevention,²⁰ are designed to be simple, easily applicable in any healthcare setting, and standardized to all patients. Surveillance definitions generally include at least one systemic or local symptom (such as fever or dysuria) and positive urine culture in a patient with an indwelling urinary catheter (or within 48 hours after its removal).

Pyuria is leukocytes or white blood cells (WBCs) in a urine specimen, with a threshold of >10 WBCs/high-power field using urine microscopy. The predictive value of different thresholds of pyuria for UTI is unclear.

Bacteriuria denotes the presence (on microscopy or culture) of bacteria in the urine. In a patient without signs or symptoms of a UTI, this is termed asymptomatic bacteriuria (ASB). A full discussion of bacteriuria, a major reason for inappropriate antibiotic use, is beyond the scope of this article but is discussed in a recent guideline.²¹

Urinary tract infections are usually characterized by a clinical syndrome along with evidence of pyuria and/or bacteriuria. The two major clinical syndromes that are observed are lower UTI (cystitis or bladder infections) and upper UTI (pyelonephritis or kidney infections). Rarely, patients may develop asymptomatic bacteremic UTI, where blood and urine cultures grow the same pathogen in the absence of clinical symptoms. (Table 1 summarizes the key points for these definitions).

Bacterial biofilm can form on the inner and outer surfaces of an indwelling urinary catheter following its insertion and can be associated with bacteriuria and CAUTI.^22,23 The biofilm comprises bacteria from the periurethral area that migrate upwards from a colonized drainage system. Bacteria present in the biofilm tend to exhibit slow growth, are protected from antibiotic exposure, and have less susceptibility to these agents.^22-24 When a mature biofilm has formed, catheter removal may be necessary for source control and to facilitate effective antimicrobial treatment. The pathogens that most commonly cause CAUTIs are Escherichia coli (23.9%), Pseudomonas aeruginosa (10.3%), and Klebsiella pneumoniae/oxytoca (10.1%).²⁵ Although urine cultures often grow yeast, particularly Candida spp., nonbacterial pathogens rarely cause UTI.

The risk of developing a CAUTI is directly related to catheter dwell time.^26,27 For catheterized patients, the rate of development of catheter-associated bacteriuria is approximately 3% to 7% per day^14,28 and is more common in the elderly and females. The likelihood of bacteriuria approaches 100% if a patient has an indwelling urinary catheter for ≥30 days,^27,29,30 which is part of the rationale for why a urine culture alone is not sufficient to diagnose a CAUTI. While bacteriuria is a risk factor for UTI, the frequency of progression from bacteriuria to CAUTI is low and treating ASB does not decrease the risk of future CAUTI. Other risk factors for the development of CAUTI include urinary tract instrumentation, diabetes mellitus, and malnutrition.^31,32

The two most important factors that lead to the development of CAUTIs and have been the main focus of quality improvement areas are unnecessary urinary catheter placement and inappropriate delay in removing a catheter when it is no longer needed.^26,33 Unfortunately, 38% of attending physicians are unaware that their patients have a urinary catheter in place.³⁴ Furthermore, in 20% to 50% of cases, there is no clear indication for catheter placement.^2,34

A CAUTI diagnosis is typically one of exclusion, as most patients present with fever and no apparent alternative source.^14,29 Since catheterized patients may not exhibit common cystitis symptoms,²⁹ most who develop CAUTI present with fever alone. However, most fevers in patients with bacteriuria and catheters are not CAUTIs and can be attributed to other sources. If a patient with an indwelling urinary catheter develops a fever and there is a suspicion of a CAUTI, careful evaluation is warranted for alternative sources of infection. This particularly applies to patients with severe systemic illness, such as hypotension or systemic inflammatory response syndrome, since these are unusual manifestations of CAUTI. The presence of either cloudy or malodorous urine does not indicate a UTI, and should not be the sole rationale for obtaining a urine culture.

Diagnostic workup of fever should include a clinical assessment of the patient. Indeed, professional guidelines recommend against obtaining a urine culture routinely for fever, unless invasive UTI risk is elevated, such as in patients with neutropenia, history of renal transplantation, or recent genitourinary surgery.³⁵ Diagnostic stewardship, focusing on the appropriate use of urine cultures, can reduce CAUTI rates.³⁶ For catheterized patients, hospitals are increasingly adopting reflex urine culture, where urine is simultaneously collected for a urinalysis and urine culture, but a urine culture is performed only if the urinalysis is positive for a predetermined threshold for pyuria, leukocyte esterase, or both. However, the use of reflex urine cultures remains an area of debate.³⁷ In addition, the Infectious Diseases Society of America recommends against screening for or treating ASB in patients with either short-term (<30 days) or long-term indwelling urethral catheters.²¹

Ideally, a urine culture should be obtained by collecting a midstream sample. In catheterized patients, a sample should be obtained after removal of the catheter; or, in patients with a clinical indication for ongoing catheterization, a sample should be obtained after a new catheter has been placed.¹⁴ If an indwelling urinary catheter must be continued, the recommendation is to disinfect the drainage system’s aspiration port and then obtain a urine culture. Urine should never be obtained from a catheter collection bag. (Table 2 summarizes best practices for diagnosis of CAUTI).

For all CAUTIs, an indwelling urinary catheter should be removed as soon as possible. If an indwelling urinary catheter remains necessary, but the existing catheter has been in place longer than two weeks, a new catheter should be placed before initiating antibiotic therapy¹⁴ to accelerate symptom resolution and reduce the likelihood of relapse or recurrence.³²

Urinary tract agents such as fosfomycin and nitrofurantoin are recommended as first-line agents for simple cystitis in women and can be used in patients with lower UTI and sufficient renal function to achieve adequate drug concentration in urine. Upper UTIs require antibiotics with good penetration into renal parenchyma such as ceftriaxone. If empiric antimicrobial therapy is needed before culture results are available, then previous urine culture results, local antibiograms, or practice guidelines can guide selection. Definitive antimicrobial therapy should be based on urine culture results. It is important to narrow empiric therapy¹⁴ to reduce risk of Clostridioides difficile infection and emergence of other resistant bacteria. Fluoroquinolones should be avoided for lower UTIs because of these risks and multiple United States Food and Drug Administration warnings.^38-40

The optimal duration of antimicrobial therapy for a CAUTI is unclear;¹⁴ however, most patients can be treated with a relatively short duration of therapy (≤7 days) if they respond promptly to therapy. Patients with a slow response to therapy may require 10-14 days of treatment.¹⁴ (Table 2 summarizes best practices for the treatment of CAUTI).

Since CAUTI is predicated on the presence of an indwelling urinary catheter, the simplest way to reduce CAUTI is to avoid placing or retaining unnecessary catheters. Some examples of appropriate indications⁹ for placement and maintenance of an indwelling urinary catheter are listed below:

1. Accurate measurement of urinary output in severely ill patients;

2. Improved comfort for patients receiving end-of-life care;

3. Acute urinary retention or bladder outlet obstruction;

4. Need for a period of prolonged immobilization (eg, potentially unstable lumbar or thoracic spine, or has multiple traumatic injuries);

5. Selected surgical procedures, such as urologic procedures and those that are expected to have a prolonged duration, require intraoperative monitoring of urine output, require the administration of either large volumes of intravenous infusions or diuretics;

6. To promote healing of open perineal or sacral wounds in patients with incontinence;

7. Neurogenic bladder; and

8. Hematuria with clots.

To increase the timely removal of urinary catheters that are no longer indicated, daily assessment of catheter necessity must be an integral part of clinicians’ workflow.^32,33 Alternatives, such as external catheters or intermittent catheterization, should be considered before indwelling urinary catheter placement since both options are associated with a reduced CAUTI risk.^41-44 Although indwelling urinary catheters can be seen as being more convenient for both patients and healthcare providers, many patients have expressed a preference for the use of intermittent catheterization compared with indwelling urinary catheterization.⁴³

For urinary retention, bladder scanning can noninvasively assess the amount of residual urine in a patient’s bladder and can avoid unnecessary insertion of an indwelling urinary catheter. However, if indwelling urinary catheters are ultimately needed, they must be inserted and maintained appropriately. Of note, the use of antibiotic-impregnated catheters has not been shown to reduce CAUTI rates significantly.⁴⁵

CAUTI prevention requires a multidisciplinary collaborative approach. Nurse-driven protocols and checklists to remove indwelling urinary catheters that are no longer indicated can be very effective.^46,47 Automatic stop orders and catheter removal reminders are useful for reducing the duration of catheter placement.^26,48 Both of these approaches require appropriate, consistent documentation with input from bedside nurses, physicians, advanced practice providers, and information technology. (Table 3 summarizes best practices for the prevention of CAUTI).

CAUTI is one of the most common types of HAI and is associated with increased morbidity, hospital length of stay, and patient costs. Most CAUTIs are preventable by limiting the placement of unnecessary catheters to instances of true necessity and removing catheters when they are no longer clinically indicated. Proper technique for the insertion and maintenance of catheters is also important for reducing CAUTI rates. Hospitalists care for a substantial number of inpatients and can make major contributions to the appropriate diagnosis, treatment, and prevention of CAUTIs.

Every day in the United States, approximately 4% of patients in acute care hospitals have at least one hospital-acquired infection (HAI).^1,2 Among the top 10 causes of death in the United States, HAIs are associated with increased morbidity, mortality, and hospital length of stay (LOS).² The direct medical cost of treating HAIs is substantial for both hospitals and patients.^3,4 Urinary tract infections (UTIs) are a leading cause of HAI, and 70%-80% of these are catheter-associated urinary tract infections (CAUTIs).^5,6 In 2016, 26,983 CAUTIs occurred in acute care hospitals.⁷ The high incidence of CAUTI can substantially contribute to morbidity, length of stay, and mortality.^8-11

The recognition that a substantial proportion of HAIs may be preventable, including 55%-70% of CAUTIs,¹² has resulted in implementing multiple strategies to reduce CAUTI rates.^13-17 These include simple prevention interventions such as avoiding placement of unnecessary indwelling urinary catheters and early removal of urinary catheters when they are no longer clinically indicated. Hospitalists are responsible for the care of many, if not most, inpatients with indwelling urinary catheters and are integral in antimicrobial stewardship efforts surrounding CAUTIs.¹⁸ Diagnostic stewardship, including appropriate urine specimen ordering, collection, processing, and reporting, works synergistically with antimicrobial stewardship and allows for appropriate antibiotic prescribing in symptomatic patients.¹⁹

CAUTIs can be defined using either clinical or surveillance definitions. Clinical definitions are used at the bedside and take individual clinical characteristics into consideration, but vary among clinicians since there is no gold standard. Abnormal laboratory urinary findings in the absence of symptoms are not sufficient for the diagnosis of UTI, including CAUTI. Surveillance definitions, such as those used by the Centers for Disease Control and Prevention,²⁰ are designed to be simple, easily applicable in any healthcare setting, and standardized to all patients. Surveillance definitions generally include at least one systemic or local symptom (such as fever or dysuria) and positive urine culture in a patient with an indwelling urinary catheter (or within 48 hours after its removal).

Pyuria is leukocytes or white blood cells (WBCs) in a urine specimen, with a threshold of >10 WBCs/high-power field using urine microscopy. The predictive value of different thresholds of pyuria for UTI is unclear.

Bacteriuria denotes the presence (on microscopy or culture) of bacteria in the urine. In a patient without signs or symptoms of a UTI, this is termed asymptomatic bacteriuria (ASB). A full discussion of bacteriuria, a major reason for inappropriate antibiotic use, is beyond the scope of this article but is discussed in a recent guideline.²¹

Urinary tract infections are usually characterized by a clinical syndrome along with evidence of pyuria and/or bacteriuria. The two major clinical syndromes that are observed are lower UTI (cystitis or bladder infections) and upper UTI (pyelonephritis or kidney infections). Rarely, patients may develop asymptomatic bacteremic UTI, where blood and urine cultures grow the same pathogen in the absence of clinical symptoms. (Table 1 summarizes the key points for these definitions).

Bacterial biofilm can form on the inner and outer surfaces of an indwelling urinary catheter following its insertion and can be associated with bacteriuria and CAUTI.^22,23 The biofilm comprises bacteria from the periurethral area that migrate upwards from a colonized drainage system. Bacteria present in the biofilm tend to exhibit slow growth, are protected from antibiotic exposure, and have less susceptibility to these agents.^22-24 When a mature biofilm has formed, catheter removal may be necessary for source control and to facilitate effective antimicrobial treatment. The pathogens that most commonly cause CAUTIs are Escherichia coli (23.9%), Pseudomonas aeruginosa (10.3%), and Klebsiella pneumoniae/oxytoca (10.1%).²⁵ Although urine cultures often grow yeast, particularly Candida spp., nonbacterial pathogens rarely cause UTI.

The risk of developing a CAUTI is directly related to catheter dwell time.^26,27 For catheterized patients, the rate of development of catheter-associated bacteriuria is approximately 3% to 7% per day^14,28 and is more common in the elderly and females. The likelihood of bacteriuria approaches 100% if a patient has an indwelling urinary catheter for ≥30 days,^27,29,30 which is part of the rationale for why a urine culture alone is not sufficient to diagnose a CAUTI. While bacteriuria is a risk factor for UTI, the frequency of progression from bacteriuria to CAUTI is low and treating ASB does not decrease the risk of future CAUTI. Other risk factors for the development of CAUTI include urinary tract instrumentation, diabetes mellitus, and malnutrition.^31,32

The two most important factors that lead to the development of CAUTIs and have been the main focus of quality improvement areas are unnecessary urinary catheter placement and inappropriate delay in removing a catheter when it is no longer needed.^26,33 Unfortunately, 38% of attending physicians are unaware that their patients have a urinary catheter in place.³⁴ Furthermore, in 20% to 50% of cases, there is no clear indication for catheter placement.^2,34

A CAUTI diagnosis is typically one of exclusion, as most patients present with fever and no apparent alternative source.^14,29 Since catheterized patients may not exhibit common cystitis symptoms,²⁹ most who develop CAUTI present with fever alone. However, most fevers in patients with bacteriuria and catheters are not CAUTIs and can be attributed to other sources. If a patient with an indwelling urinary catheter develops a fever and there is a suspicion of a CAUTI, careful evaluation is warranted for alternative sources of infection. This particularly applies to patients with severe systemic illness, such as hypotension or systemic inflammatory response syndrome, since these are unusual manifestations of CAUTI. The presence of either cloudy or malodorous urine does not indicate a UTI, and should not be the sole rationale for obtaining a urine culture.

Diagnostic workup of fever should include a clinical assessment of the patient. Indeed, professional guidelines recommend against obtaining a urine culture routinely for fever, unless invasive UTI risk is elevated, such as in patients with neutropenia, history of renal transplantation, or recent genitourinary surgery.³⁵ Diagnostic stewardship, focusing on the appropriate use of urine cultures, can reduce CAUTI rates.³⁶ For catheterized patients, hospitals are increasingly adopting reflex urine culture, where urine is simultaneously collected for a urinalysis and urine culture, but a urine culture is performed only if the urinalysis is positive for a predetermined threshold for pyuria, leukocyte esterase, or both. However, the use of reflex urine cultures remains an area of debate.³⁷ In addition, the Infectious Diseases Society of America recommends against screening for or treating ASB in patients with either short-term (<30 days) or long-term indwelling urethral catheters.²¹

Ideally, a urine culture should be obtained by collecting a midstream sample. In catheterized patients, a sample should be obtained after removal of the catheter; or, in patients with a clinical indication for ongoing catheterization, a sample should be obtained after a new catheter has been placed.¹⁴ If an indwelling urinary catheter must be continued, the recommendation is to disinfect the drainage system’s aspiration port and then obtain a urine culture. Urine should never be obtained from a catheter collection bag. (Table 2 summarizes best practices for diagnosis of CAUTI).

For all CAUTIs, an indwelling urinary catheter should be removed as soon as possible. If an indwelling urinary catheter remains necessary, but the existing catheter has been in place longer than two weeks, a new catheter should be placed before initiating antibiotic therapy¹⁴ to accelerate symptom resolution and reduce the likelihood of relapse or recurrence.³²

Urinary tract agents such as fosfomycin and nitrofurantoin are recommended as first-line agents for simple cystitis in women and can be used in patients with lower UTI and sufficient renal function to achieve adequate drug concentration in urine. Upper UTIs require antibiotics with good penetration into renal parenchyma such as ceftriaxone. If empiric antimicrobial therapy is needed before culture results are available, then previous urine culture results, local antibiograms, or practice guidelines can guide selection. Definitive antimicrobial therapy should be based on urine culture results. It is important to narrow empiric therapy¹⁴ to reduce risk of Clostridioides difficile infection and emergence of other resistant bacteria. Fluoroquinolones should be avoided for lower UTIs because of these risks and multiple United States Food and Drug Administration warnings.^38-40

The optimal duration of antimicrobial therapy for a CAUTI is unclear;¹⁴ however, most patients can be treated with a relatively short duration of therapy (≤7 days) if they respond promptly to therapy. Patients with a slow response to therapy may require 10-14 days of treatment.¹⁴ (Table 2 summarizes best practices for the treatment of CAUTI).

Since CAUTI is predicated on the presence of an indwelling urinary catheter, the simplest way to reduce CAUTI is to avoid placing or retaining unnecessary catheters. Some examples of appropriate indications⁹ for placement and maintenance of an indwelling urinary catheter are listed below:

1. Accurate measurement of urinary output in severely ill patients;

2. Improved comfort for patients receiving end-of-life care;

3. Acute urinary retention or bladder outlet obstruction;

4. Need for a period of prolonged immobilization (eg, potentially unstable lumbar or thoracic spine, or has multiple traumatic injuries);

5. Selected surgical procedures, such as urologic procedures and those that are expected to have a prolonged duration, require intraoperative monitoring of urine output, require the administration of either large volumes of intravenous infusions or diuretics;

6. To promote healing of open perineal or sacral wounds in patients with incontinence;

7. Neurogenic bladder; and

8. Hematuria with clots.

To increase the timely removal of urinary catheters that are no longer indicated, daily assessment of catheter necessity must be an integral part of clinicians’ workflow.^32,33 Alternatives, such as external catheters or intermittent catheterization, should be considered before indwelling urinary catheter placement since both options are associated with a reduced CAUTI risk.^41-44 Although indwelling urinary catheters can be seen as being more convenient for both patients and healthcare providers, many patients have expressed a preference for the use of intermittent catheterization compared with indwelling urinary catheterization.⁴³

For urinary retention, bladder scanning can noninvasively assess the amount of residual urine in a patient’s bladder and can avoid unnecessary insertion of an indwelling urinary catheter. However, if indwelling urinary catheters are ultimately needed, they must be inserted and maintained appropriately. Of note, the use of antibiotic-impregnated catheters has not been shown to reduce CAUTI rates significantly.⁴⁵

CAUTI prevention requires a multidisciplinary collaborative approach. Nurse-driven protocols and checklists to remove indwelling urinary catheters that are no longer indicated can be very effective.^46,47 Automatic stop orders and catheter removal reminders are useful for reducing the duration of catheter placement.^26,48 Both of these approaches require appropriate, consistent documentation with input from bedside nurses, physicians, advanced practice providers, and information technology. (Table 3 summarizes best practices for the prevention of CAUTI).

CAUTI is one of the most common types of HAI and is associated with increased morbidity, hospital length of stay, and patient costs. Most CAUTIs are preventable by limiting the placement of unnecessary catheters to instances of true necessity and removing catheters when they are no longer clinically indicated. Proper technique for the insertion and maintenance of catheters is also important for reducing CAUTI rates. Hospitalists care for a substantial number of inpatients and can make major contributions to the appropriate diagnosis, treatment, and prevention of CAUTIs.

Every day in the United States, approximately 4% of patients in acute care hospitals have at least one hospital-acquired infection (HAI).^1,2 Among the top 10 causes of death in the United States, HAIs are associated with increased morbidity, mortality, and hospital length of stay (LOS).² The direct medical cost of treating HAIs is substantial for both hospitals and patients.^3,4 Urinary tract infections (UTIs) are a leading cause of HAI, and 70%-80% of these are catheter-associated urinary tract infections (CAUTIs).^5,6 In 2016, 26,983 CAUTIs occurred in acute care hospitals.⁷ The high incidence of CAUTI can substantially contribute to morbidity, length of stay, and mortality.^8-11

The recognition that a substantial proportion of HAIs may be preventable, including 55%-70% of CAUTIs,¹² has resulted in implementing multiple strategies to reduce CAUTI rates.^13-17 These include simple prevention interventions such as avoiding placement of unnecessary indwelling urinary catheters and early removal of urinary catheters when they are no longer clinically indicated. Hospitalists are responsible for the care of many, if not most, inpatients with indwelling urinary catheters and are integral in antimicrobial stewardship efforts surrounding CAUTIs.¹⁸ Diagnostic stewardship, including appropriate urine specimen ordering, collection, processing, and reporting, works synergistically with antimicrobial stewardship and allows for appropriate antibiotic prescribing in symptomatic patients.¹⁹

CAUTIs can be defined using either clinical or surveillance definitions. Clinical definitions are used at the bedside and take individual clinical characteristics into consideration, but vary among clinicians since there is no gold standard. Abnormal laboratory urinary findings in the absence of symptoms are not sufficient for the diagnosis of UTI, including CAUTI. Surveillance definitions, such as those used by the Centers for Disease Control and Prevention,²⁰ are designed to be simple, easily applicable in any healthcare setting, and standardized to all patients. Surveillance definitions generally include at least one systemic or local symptom (such as fever or dysuria) and positive urine culture in a patient with an indwelling urinary catheter (or within 48 hours after its removal).

Pyuria is leukocytes or white blood cells (WBCs) in a urine specimen, with a threshold of >10 WBCs/high-power field using urine microscopy. The predictive value of different thresholds of pyuria for UTI is unclear.

Bacteriuria denotes the presence (on microscopy or culture) of bacteria in the urine. In a patient without signs or symptoms of a UTI, this is termed asymptomatic bacteriuria (ASB). A full discussion of bacteriuria, a major reason for inappropriate antibiotic use, is beyond the scope of this article but is discussed in a recent guideline.²¹

Urinary tract infections are usually characterized by a clinical syndrome along with evidence of pyuria and/or bacteriuria. The two major clinical syndromes that are observed are lower UTI (cystitis or bladder infections) and upper UTI (pyelonephritis or kidney infections). Rarely, patients may develop asymptomatic bacteremic UTI, where blood and urine cultures grow the same pathogen in the absence of clinical symptoms. (Table 1 summarizes the key points for these definitions).

Bacterial biofilm can form on the inner and outer surfaces of an indwelling urinary catheter following its insertion and can be associated with bacteriuria and CAUTI.^22,23 The biofilm comprises bacteria from the periurethral area that migrate upwards from a colonized drainage system. Bacteria present in the biofilm tend to exhibit slow growth, are protected from antibiotic exposure, and have less susceptibility to these agents.^22-24 When a mature biofilm has formed, catheter removal may be necessary for source control and to facilitate effective antimicrobial treatment. The pathogens that most commonly cause CAUTIs are Escherichia coli (23.9%), Pseudomonas aeruginosa (10.3%), and Klebsiella pneumoniae/oxytoca (10.1%).²⁵ Although urine cultures often grow yeast, particularly Candida spp., nonbacterial pathogens rarely cause UTI.

The risk of developing a CAUTI is directly related to catheter dwell time.^26,27 For catheterized patients, the rate of development of catheter-associated bacteriuria is approximately 3% to 7% per day^14,28 and is more common in the elderly and females. The likelihood of bacteriuria approaches 100% if a patient has an indwelling urinary catheter for ≥30 days,^27,29,30 which is part of the rationale for why a urine culture alone is not sufficient to diagnose a CAUTI. While bacteriuria is a risk factor for UTI, the frequency of progression from bacteriuria to CAUTI is low and treating ASB does not decrease the risk of future CAUTI. Other risk factors for the development of CAUTI include urinary tract instrumentation, diabetes mellitus, and malnutrition.^31,32

The two most important factors that lead to the development of CAUTIs and have been the main focus of quality improvement areas are unnecessary urinary catheter placement and inappropriate delay in removing a catheter when it is no longer needed.^26,33 Unfortunately, 38% of attending physicians are unaware that their patients have a urinary catheter in place.³⁴ Furthermore, in 20% to 50% of cases, there is no clear indication for catheter placement.^2,34

A CAUTI diagnosis is typically one of exclusion, as most patients present with fever and no apparent alternative source.^14,29 Since catheterized patients may not exhibit common cystitis symptoms,²⁹ most who develop CAUTI present with fever alone. However, most fevers in patients with bacteriuria and catheters are not CAUTIs and can be attributed to other sources. If a patient with an indwelling urinary catheter develops a fever and there is a suspicion of a CAUTI, careful evaluation is warranted for alternative sources of infection. This particularly applies to patients with severe systemic illness, such as hypotension or systemic inflammatory response syndrome, since these are unusual manifestations of CAUTI. The presence of either cloudy or malodorous urine does not indicate a UTI, and should not be the sole rationale for obtaining a urine culture.

Diagnostic workup of fever should include a clinical assessment of the patient. Indeed, professional guidelines recommend against obtaining a urine culture routinely for fever, unless invasive UTI risk is elevated, such as in patients with neutropenia, history of renal transplantation, or recent genitourinary surgery.³⁵ Diagnostic stewardship, focusing on the appropriate use of urine cultures, can reduce CAUTI rates.³⁶ For catheterized patients, hospitals are increasingly adopting reflex urine culture, where urine is simultaneously collected for a urinalysis and urine culture, but a urine culture is performed only if the urinalysis is positive for a predetermined threshold for pyuria, leukocyte esterase, or both. However, the use of reflex urine cultures remains an area of debate.³⁷ In addition, the Infectious Diseases Society of America recommends against screening for or treating ASB in patients with either short-term (<30 days) or long-term indwelling urethral catheters.²¹

Ideally, a urine culture should be obtained by collecting a midstream sample. In catheterized patients, a sample should be obtained after removal of the catheter; or, in patients with a clinical indication for ongoing catheterization, a sample should be obtained after a new catheter has been placed.¹⁴ If an indwelling urinary catheter must be continued, the recommendation is to disinfect the drainage system’s aspiration port and then obtain a urine culture. Urine should never be obtained from a catheter collection bag. (Table 2 summarizes best practices for diagnosis of CAUTI).

For all CAUTIs, an indwelling urinary catheter should be removed as soon as possible. If an indwelling urinary catheter remains necessary, but the existing catheter has been in place longer than two weeks, a new catheter should be placed before initiating antibiotic therapy¹⁴ to accelerate symptom resolution and reduce the likelihood of relapse or recurrence.³²

Urinary tract agents such as fosfomycin and nitrofurantoin are recommended as first-line agents for simple cystitis in women and can be used in patients with lower UTI and sufficient renal function to achieve adequate drug concentration in urine. Upper UTIs require antibiotics with good penetration into renal parenchyma such as ceftriaxone. If empiric antimicrobial therapy is needed before culture results are available, then previous urine culture results, local antibiograms, or practice guidelines can guide selection. Definitive antimicrobial therapy should be based on urine culture results. It is important to narrow empiric therapy¹⁴ to reduce risk of Clostridioides difficile infection and emergence of other resistant bacteria. Fluoroquinolones should be avoided for lower UTIs because of these risks and multiple United States Food and Drug Administration warnings.^38-40

The optimal duration of antimicrobial therapy for a CAUTI is unclear;¹⁴ however, most patients can be treated with a relatively short duration of therapy (≤7 days) if they respond promptly to therapy. Patients with a slow response to therapy may require 10-14 days of treatment.¹⁴ (Table 2 summarizes best practices for the treatment of CAUTI).

Since CAUTI is predicated on the presence of an indwelling urinary catheter, the simplest way to reduce CAUTI is to avoid placing or retaining unnecessary catheters. Some examples of appropriate indications⁹ for placement and maintenance of an indwelling urinary catheter are listed below:

1. Accurate measurement of urinary output in severely ill patients;

2. Improved comfort for patients receiving end-of-life care;

3. Acute urinary retention or bladder outlet obstruction;

4. Need for a period of prolonged immobilization (eg, potentially unstable lumbar or thoracic spine, or has multiple traumatic injuries);

5. Selected surgical procedures, such as urologic procedures and those that are expected to have a prolonged duration, require intraoperative monitoring of urine output, require the administration of either large volumes of intravenous infusions or diuretics;

6. To promote healing of open perineal or sacral wounds in patients with incontinence;

7. Neurogenic bladder; and

8. Hematuria with clots.

To increase the timely removal of urinary catheters that are no longer indicated, daily assessment of catheter necessity must be an integral part of clinicians’ workflow.^32,33 Alternatives, such as external catheters or intermittent catheterization, should be considered before indwelling urinary catheter placement since both options are associated with a reduced CAUTI risk.^41-44 Although indwelling urinary catheters can be seen as being more convenient for both patients and healthcare providers, many patients have expressed a preference for the use of intermittent catheterization compared with indwelling urinary catheterization.⁴³

For urinary retention, bladder scanning can noninvasively assess the amount of residual urine in a patient’s bladder and can avoid unnecessary insertion of an indwelling urinary catheter. However, if indwelling urinary catheters are ultimately needed, they must be inserted and maintained appropriately. Of note, the use of antibiotic-impregnated catheters has not been shown to reduce CAUTI rates significantly.⁴⁵

CAUTI prevention requires a multidisciplinary collaborative approach. Nurse-driven protocols and checklists to remove indwelling urinary catheters that are no longer indicated can be very effective.^46,47 Automatic stop orders and catheter removal reminders are useful for reducing the duration of catheter placement.^26,48 Both of these approaches require appropriate, consistent documentation with input from bedside nurses, physicians, advanced practice providers, and information technology. (Table 3 summarizes best practices for the prevention of CAUTI).

CAUTI is one of the most common types of HAI and is associated with increased morbidity, hospital length of stay, and patient costs. Most CAUTIs are preventable by limiting the placement of unnecessary catheters to instances of true necessity and removing catheters when they are no longer clinically indicated. Proper technique for the insertion and maintenance of catheters is also important for reducing CAUTI rates. Hospitalists care for a substantial number of inpatients and can make major contributions to the appropriate diagnosis, treatment, and prevention of CAUTIs.

Procalcitonin (PCT) is a biomarker that has shown promise to identify bacterial etiology in acute infections, including bacterial lower respiratory tract infection (LRTI). In 2017, the United States Food and Drug Administration (FDA) approved the use of PCT as a diagnostic aid to guide the decisions around antibiotic therapy in acute LRTI.¹ Although most of the data supporting the use of PCT for LRTI stems from adult studies, the high disease burden, predominance of viral etiologies, and frequent diagnostic uncertainty resulting in antibiotic overuse make pediatric LRTI an ideal target for the use of PCT as a diagnostic aid. This review evaluates and summarizes the current evidence regarding the role of PCT in the clinical care of pediatric LRTI, including its use in guiding antibiotic use and prognosticating disease severity.

The commonly used PCT cut points for withholding or stopping antibiotics in adults and children are 0.1 µg/L (very low risk of bacterial etiology) or 0.25 µg/L (low risk of bacterial etiology).^2-4 Among the 532 children enrolled in the multicenter study of Etiology of Pneumonia in the Community (EPIC), a PCT threshold of 0.25 µg/L demonstrated an approximate sensitivity of 85%, specificity of 45%, positive likelihood ratio of 1.55, and negative likelihood ratio of 0.33 for community acquired pneumonia (CAP) caused by typical bacterial pathogens.⁵ Lowering the cutoff to <0.1 µg/L increased PCT sensitivity to 100%, decreased specificity, positive likelihood ratio, and negative likelihood ratio to 20%, 1.26, and 0, respectively. Although the EPIC study obtained culture and performed PCR testing on any blood sample, pleural fluid specimen, endotracheal aspirate, or bronchoalveolar–lavage specimens obtained during the study period, currently available laboratory methods show poor sensitivity for defining bacterial LRTI. Thus, bacterial etiologies may have been underestimated. The highly negative predictive value demonstrated in this study highlights the potential of PCT as a biomarker for ruling out bacterial diseases, including LRTI.

Multiple studies have evaluated the potential utility of PCT in guiding antibiotic initiation in adults with LRTI, but data on pediatric patients are sparse.⁴ In a randomized, single-center Italian study comparing a PCT-guided algorithm (withholding antibiotics when PCT < 0.25 µg/L) versus usual care among 319 hospitalized children with pneumonia, the PCT group experienced fewer antibiotic initiations (15.5% vs 100%, P < .05) without significant differences in recurrence of respiratory symptoms or new antibiotic prescriptions in the month following enrollment.²

A similar randomized trial using a PCT-guided algorithm for the initiation of antibiotics conducted among 337 Swiss children presented to the emergency department (ED) with pneumonia and other LRTIs failed to demonstrate decreases in antibiotic initiation.³ This study used an algorithm that categorized the likelihood of requiring antibiotic treatment for bacterial LRTI as “definitely” if PCT was >0.5 µg/L, “probably” if PCT was 0.26–0.5 µg/L, “probably not” if PCT was 0.1–0.25 µg/L, and “definitely not” if PCT was <0.1 µg/L. In the PCT group, 104 out of 168 (62%) patients received antibiotics within 14 days compared with 93 out of 165 (56%) patients in the control group (odds ratio [OR]: 1.26, 95% CI: 0.81, 1.95). In the subgroup analyses, the odds of administering antibiotics to those with nonpneumonia LRTI was significantly higher than those of the PCT group and control group (OR: 4.09, 95% CI: 1.8, 9.93); the odds of receiving antibiotics also showed no difference in the subgroup of children with pneumonia (OR: 0.66, 95% CI: 0.35, 1.23).

The benefit of PCT for informing decisions around the initiation of antibiotics likely varies based on perceived risk of bacterial diseases. When the pretest probability of bacterial disease is extremely high, the use of PCT is unlikely to alter treatment decisions. Similarly, PCT should not be used in situations where the pretest probability for bacterial pneumonia is very low—in these instances, an elevated PCT may lead to unnecessary antibiotic use among children presenting to the ED. However, the risk of bacterial pneumonia is often equivocal, and in these situations, PCT may provide clinicians with useful insights, primarily for ruling out bacterial disease.

In the study by Esposito et al., the PCT levels were additionally measured every two days until discharge and during two scheduled follow-up visits; the antibiotics were discontinued when PCT < 0.25 µg/L.² The PCT-guided group experienced shorter antibiotic duration (mean 5.4 vs 11.0 days, P < .05), shorter length of hospital stay (mean 4.7 vs 5.61 days for mild LRTI and 5.01 vs 5.93 for severe LRTI), and fewer antibiotic-related adverse events (3.9% vs 25.2%, P < .05). Similarly, in the study by Baer et al., the PCT-guided group had PCT levels repeated on days three and five after enrollment, and the antibiotics were discontinued when PCT was less than 0.25 µg/L. The duration of antibiotic administration was significantly lower in the PCT-guided group (mean difference: 1.8 days, 95% CI: −3.1, −0.).³ The rates of hospitalization, duration of hospital stay, and mean impairment of daily activities attributable to LRTI were similar between groups.

Considering the adult studies and the small number of pediatric LRTI research published to date, the use of PCT to safely reduce antibiotic treatment duration is encouraging.⁴ Although the studies on the kinetics of PCT are limited, the biomarker has been shown to rise two to four hours after a bacterial stimulus, peak in 24-48 hours and achieve a half-life of 24-36 hours.^6,7 As such, serial PCT measurements at 24-hour intervals for three to five days may be more beneficial than stand-alone PCT tests. Nonetheless, additional studies are needed to better define groups of patients who will most likely benefit from PCT testing and to understand how to best integrate testing into clinical practice.

PCT has also been explored as a marker of LRTI disease severity. In a 2008 multicenter cohort encompassing 1,651 adults with pneumonia, PCT < 0.1 µg/L was associated with a decreased 30-day mortality, shorter length of stay, and decreased admission to the intensive care unit (ICU) compared with those with PCT>0.1 µg/L.⁸ In a 2017 study of 317 adults hospitalized with pneumonia, the PCT level was significantly higher in those with bacteremia and in those admitted to intensive care.⁹ When used in combination with the pneumonia severity index (PSI), the addition of PCT resulted in improved prognostic performance compared with the PSI alone for both outcomes, increasing the area under the receiver operating characteristic curve from 0.67 to 0.85 for bacteremia and from 0.58 to 0.64 for intensive care. Similarly, in the adult EPIC cohort, the addition of PCT contributed significant prognostic information beyond existing severity scores for predicting the need for invasive respiratory or vasopressor support; each 1 µg/L increase in PCT was associated with a 1% to 2% absolute increase in the need for this outcome.¹⁰

A European study of 100 children with pneumonia also demonstrated higher PCT values among hospitalized children (n = 26, median PCT 17.8 µg/L) compared with outpatient children (n = 73, median PCT 0.72 µg/L, P < .01).¹¹ Among the 532 children from the EPIC study, a PCT < 0.25 µg/L was associated with the reduced odds of ICU admission (adjusted OR: 0.48; 95% CI: 0.30, 0.78) and a 2.3-day (95% CI: 1.4, 3.2) decrease in the average length of stay compared with those with higher PCT concentrations.⁵ Of the 34 children with empyema requiring drainage, 28 (82%) showed a PCT concentration ≥0.5 µg/L. Additional pediatric studies are needed, but the limited data to date suggest that PCT may play a role in predicting pediatric LRTI disease severity, including the need for mechanical ventilatory support and ICU-level care.

Although PCT shows promise as a biomarker to reliably rule out bacterial infection, several potential limitations exist in assessing its role in pediatric LRTI. Atypical bacterial infections (ie, Mycoplasma pneumoniae) and localized bacterial infection may not induce significant PCT production, as has been shown in adults and children with tonsillitis, localized skin infections, endocarditis, or empyema (Table).¹² The majority of clinical trials in LRTI have been conducted in the adult population,⁴ with the number of pediatric trials remaining small.^2,3 Given the predominance of viral LRTI in children compared with adults, the utility of PCT may differ in these populations.^13,14 Furthermore, existing studies demonstrate mixed results regarding the magnitude of benefits that PCT may provide in terms of limiting antibiotic use. Another concern is the potential of PCT to increase unnecessary antibiotic use in those with viral LRTI,³ as PCT may also be increased in populations with systemic inflammation from nonbacterial causes.^12,15

The misuse of antibiotics is a public health crisis resulting in the emergence of antibiotic-resistant pathogens and adverse outcomes, including Clostridioides difficile infection, drug toxicities, and increased healthcare costs.¹⁶ Pneumonia is responsible for more days of antibiotics than any other disease in children’s hospitals and is an important target for stewardship efforts.¹⁷ PCT is a promising biomarker for distinguishing bacterial from viral infection, and its use may help in making informed antibiotic decisions and predicting disease outcomes in pediatric LRTI. Although PCT has been cleared by the FDA for assisting with antibiotic decisions in pediatric LRTI, the majority of evidence supporting this indication is drawn from adults. Additional studies are needed prior to the widespread implementation in the pediatric population, but the results of available pediatric studies show promise. The clinical context and severity of patient presentation are important when considering whether or not to use PCT and how to best interpret PCT levels when making clinical management decisions. The utility of PCT for antibiotic initiation in the pediatric population is encouraging given the predominance of viral etiologies in pediatric LRTI. Currently available data demonstrate the value of serial PCT measurements in antibiotic de-escalation and promoting antibiotic stewardship for children and adults.^2-4 As with all new diagnostic modalities, provider education is paramount to ensure a safe and value-driven implementation.

Disclosures

Dr. Katz received investigator-initiated grant funding from Roche and bioMérieux to conduct research involving procalcitonin in the past three years. Dr. Sartori has nothing to disclose. Dr. Williams received investigator-initiated grant funding from bioMérieux to conduct research involving procalcitonin in the past three years.

Funding

This work was supported by the National Institute of Health (1T32AI095202-07).

Disclaimer

This content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health, Roche, or bioMérieux.

Procalcitonin (PCT) is a biomarker that has shown promise to identify bacterial etiology in acute infections, including bacterial lower respiratory tract infection (LRTI). In 2017, the United States Food and Drug Administration (FDA) approved the use of PCT as a diagnostic aid to guide the decisions around antibiotic therapy in acute LRTI.¹ Although most of the data supporting the use of PCT for LRTI stems from adult studies, the high disease burden, predominance of viral etiologies, and frequent diagnostic uncertainty resulting in antibiotic overuse make pediatric LRTI an ideal target for the use of PCT as a diagnostic aid. This review evaluates and summarizes the current evidence regarding the role of PCT in the clinical care of pediatric LRTI, including its use in guiding antibiotic use and prognosticating disease severity.

The commonly used PCT cut points for withholding or stopping antibiotics in adults and children are 0.1 µg/L (very low risk of bacterial etiology) or 0.25 µg/L (low risk of bacterial etiology).^2-4 Among the 532 children enrolled in the multicenter study of Etiology of Pneumonia in the Community (EPIC), a PCT threshold of 0.25 µg/L demonstrated an approximate sensitivity of 85%, specificity of 45%, positive likelihood ratio of 1.55, and negative likelihood ratio of 0.33 for community acquired pneumonia (CAP) caused by typical bacterial pathogens.⁵ Lowering the cutoff to <0.1 µg/L increased PCT sensitivity to 100%, decreased specificity, positive likelihood ratio, and negative likelihood ratio to 20%, 1.26, and 0, respectively. Although the EPIC study obtained culture and performed PCR testing on any blood sample, pleural fluid specimen, endotracheal aspirate, or bronchoalveolar–lavage specimens obtained during the study period, currently available laboratory methods show poor sensitivity for defining bacterial LRTI. Thus, bacterial etiologies may have been underestimated. The highly negative predictive value demonstrated in this study highlights the potential of PCT as a biomarker for ruling out bacterial diseases, including LRTI.

Multiple studies have evaluated the potential utility of PCT in guiding antibiotic initiation in adults with LRTI, but data on pediatric patients are sparse.⁴ In a randomized, single-center Italian study comparing a PCT-guided algorithm (withholding antibiotics when PCT < 0.25 µg/L) versus usual care among 319 hospitalized children with pneumonia, the PCT group experienced fewer antibiotic initiations (15.5% vs 100%, P < .05) without significant differences in recurrence of respiratory symptoms or new antibiotic prescriptions in the month following enrollment.²

A similar randomized trial using a PCT-guided algorithm for the initiation of antibiotics conducted among 337 Swiss children presented to the emergency department (ED) with pneumonia and other LRTIs failed to demonstrate decreases in antibiotic initiation.³ This study used an algorithm that categorized the likelihood of requiring antibiotic treatment for bacterial LRTI as “definitely” if PCT was >0.5 µg/L, “probably” if PCT was 0.26–0.5 µg/L, “probably not” if PCT was 0.1–0.25 µg/L, and “definitely not” if PCT was <0.1 µg/L. In the PCT group, 104 out of 168 (62%) patients received antibiotics within 14 days compared with 93 out of 165 (56%) patients in the control group (odds ratio [OR]: 1.26, 95% CI: 0.81, 1.95). In the subgroup analyses, the odds of administering antibiotics to those with nonpneumonia LRTI was significantly higher than those of the PCT group and control group (OR: 4.09, 95% CI: 1.8, 9.93); the odds of receiving antibiotics also showed no difference in the subgroup of children with pneumonia (OR: 0.66, 95% CI: 0.35, 1.23).

The benefit of PCT for informing decisions around the initiation of antibiotics likely varies based on perceived risk of bacterial diseases. When the pretest probability of bacterial disease is extremely high, the use of PCT is unlikely to alter treatment decisions. Similarly, PCT should not be used in situations where the pretest probability for bacterial pneumonia is very low—in these instances, an elevated PCT may lead to unnecessary antibiotic use among children presenting to the ED. However, the risk of bacterial pneumonia is often equivocal, and in these situations, PCT may provide clinicians with useful insights, primarily for ruling out bacterial disease.

In the study by Esposito et al., the PCT levels were additionally measured every two days until discharge and during two scheduled follow-up visits; the antibiotics were discontinued when PCT < 0.25 µg/L.² The PCT-guided group experienced shorter antibiotic duration (mean 5.4 vs 11.0 days, P < .05), shorter length of hospital stay (mean 4.7 vs 5.61 days for mild LRTI and 5.01 vs 5.93 for severe LRTI), and fewer antibiotic-related adverse events (3.9% vs 25.2%, P < .05). Similarly, in the study by Baer et al., the PCT-guided group had PCT levels repeated on days three and five after enrollment, and the antibiotics were discontinued when PCT was less than 0.25 µg/L. The duration of antibiotic administration was significantly lower in the PCT-guided group (mean difference: 1.8 days, 95% CI: −3.1, −0.).³ The rates of hospitalization, duration of hospital stay, and mean impairment of daily activities attributable to LRTI were similar between groups.

Considering the adult studies and the small number of pediatric LRTI research published to date, the use of PCT to safely reduce antibiotic treatment duration is encouraging.⁴ Although the studies on the kinetics of PCT are limited, the biomarker has been shown to rise two to four hours after a bacterial stimulus, peak in 24-48 hours and achieve a half-life of 24-36 hours.^6,7 As such, serial PCT measurements at 24-hour intervals for three to five days may be more beneficial than stand-alone PCT tests. Nonetheless, additional studies are needed to better define groups of patients who will most likely benefit from PCT testing and to understand how to best integrate testing into clinical practice.

PCT has also been explored as a marker of LRTI disease severity. In a 2008 multicenter cohort encompassing 1,651 adults with pneumonia, PCT < 0.1 µg/L was associated with a decreased 30-day mortality, shorter length of stay, and decreased admission to the intensive care unit (ICU) compared with those with PCT>0.1 µg/L.⁸ In a 2017 study of 317 adults hospitalized with pneumonia, the PCT level was significantly higher in those with bacteremia and in those admitted to intensive care.⁹ When used in combination with the pneumonia severity index (PSI), the addition of PCT resulted in improved prognostic performance compared with the PSI alone for both outcomes, increasing the area under the receiver operating characteristic curve from 0.67 to 0.85 for bacteremia and from 0.58 to 0.64 for intensive care. Similarly, in the adult EPIC cohort, the addition of PCT contributed significant prognostic information beyond existing severity scores for predicting the need for invasive respiratory or vasopressor support; each 1 µg/L increase in PCT was associated with a 1% to 2% absolute increase in the need for this outcome.¹⁰

A European study of 100 children with pneumonia also demonstrated higher PCT values among hospitalized children (n = 26, median PCT 17.8 µg/L) compared with outpatient children (n = 73, median PCT 0.72 µg/L, P < .01).¹¹ Among the 532 children from the EPIC study, a PCT < 0.25 µg/L was associated with the reduced odds of ICU admission (adjusted OR: 0.48; 95% CI: 0.30, 0.78) and a 2.3-day (95% CI: 1.4, 3.2) decrease in the average length of stay compared with those with higher PCT concentrations.⁵ Of the 34 children with empyema requiring drainage, 28 (82%) showed a PCT concentration ≥0.5 µg/L. Additional pediatric studies are needed, but the limited data to date suggest that PCT may play a role in predicting pediatric LRTI disease severity, including the need for mechanical ventilatory support and ICU-level care.

Although PCT shows promise as a biomarker to reliably rule out bacterial infection, several potential limitations exist in assessing its role in pediatric LRTI. Atypical bacterial infections (ie, Mycoplasma pneumoniae) and localized bacterial infection may not induce significant PCT production, as has been shown in adults and children with tonsillitis, localized skin infections, endocarditis, or empyema (Table).¹² The majority of clinical trials in LRTI have been conducted in the adult population,⁴ with the number of pediatric trials remaining small.^2,3 Given the predominance of viral LRTI in children compared with adults, the utility of PCT may differ in these populations.^13,14 Furthermore, existing studies demonstrate mixed results regarding the magnitude of benefits that PCT may provide in terms of limiting antibiotic use. Another concern is the potential of PCT to increase unnecessary antibiotic use in those with viral LRTI,³ as PCT may also be increased in populations with systemic inflammation from nonbacterial causes.^12,15

The misuse of antibiotics is a public health crisis resulting in the emergence of antibiotic-resistant pathogens and adverse outcomes, including Clostridioides difficile infection, drug toxicities, and increased healthcare costs.¹⁶ Pneumonia is responsible for more days of antibiotics than any other disease in children’s hospitals and is an important target for stewardship efforts.¹⁷ PCT is a promising biomarker for distinguishing bacterial from viral infection, and its use may help in making informed antibiotic decisions and predicting disease outcomes in pediatric LRTI. Although PCT has been cleared by the FDA for assisting with antibiotic decisions in pediatric LRTI, the majority of evidence supporting this indication is drawn from adults. Additional studies are needed prior to the widespread implementation in the pediatric population, but the results of available pediatric studies show promise. The clinical context and severity of patient presentation are important when considering whether or not to use PCT and how to best interpret PCT levels when making clinical management decisions. The utility of PCT for antibiotic initiation in the pediatric population is encouraging given the predominance of viral etiologies in pediatric LRTI. Currently available data demonstrate the value of serial PCT measurements in antibiotic de-escalation and promoting antibiotic stewardship for children and adults.^2-4 As with all new diagnostic modalities, provider education is paramount to ensure a safe and value-driven implementation.

Disclosures

Dr. Katz received investigator-initiated grant funding from Roche and bioMérieux to conduct research involving procalcitonin in the past three years. Dr. Sartori has nothing to disclose. Dr. Williams received investigator-initiated grant funding from bioMérieux to conduct research involving procalcitonin in the past three years.

Funding

This work was supported by the National Institute of Health (1T32AI095202-07).

Disclaimer

This content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health, Roche, or bioMérieux.

Procalcitonin (PCT) is a biomarker that has shown promise to identify bacterial etiology in acute infections, including bacterial lower respiratory tract infection (LRTI). In 2017, the United States Food and Drug Administration (FDA) approved the use of PCT as a diagnostic aid to guide the decisions around antibiotic therapy in acute LRTI.¹ Although most of the data supporting the use of PCT for LRTI stems from adult studies, the high disease burden, predominance of viral etiologies, and frequent diagnostic uncertainty resulting in antibiotic overuse make pediatric LRTI an ideal target for the use of PCT as a diagnostic aid. This review evaluates and summarizes the current evidence regarding the role of PCT in the clinical care of pediatric LRTI, including its use in guiding antibiotic use and prognosticating disease severity.

The commonly used PCT cut points for withholding or stopping antibiotics in adults and children are 0.1 µg/L (very low risk of bacterial etiology) or 0.25 µg/L (low risk of bacterial etiology).^2-4 Among the 532 children enrolled in the multicenter study of Etiology of Pneumonia in the Community (EPIC), a PCT threshold of 0.25 µg/L demonstrated an approximate sensitivity of 85%, specificity of 45%, positive likelihood ratio of 1.55, and negative likelihood ratio of 0.33 for community acquired pneumonia (CAP) caused by typical bacterial pathogens.⁵ Lowering the cutoff to <0.1 µg/L increased PCT sensitivity to 100%, decreased specificity, positive likelihood ratio, and negative likelihood ratio to 20%, 1.26, and 0, respectively. Although the EPIC study obtained culture and performed PCR testing on any blood sample, pleural fluid specimen, endotracheal aspirate, or bronchoalveolar–lavage specimens obtained during the study period, currently available laboratory methods show poor sensitivity for defining bacterial LRTI. Thus, bacterial etiologies may have been underestimated. The highly negative predictive value demonstrated in this study highlights the potential of PCT as a biomarker for ruling out bacterial diseases, including LRTI.

Multiple studies have evaluated the potential utility of PCT in guiding antibiotic initiation in adults with LRTI, but data on pediatric patients are sparse.⁴ In a randomized, single-center Italian study comparing a PCT-guided algorithm (withholding antibiotics when PCT < 0.25 µg/L) versus usual care among 319 hospitalized children with pneumonia, the PCT group experienced fewer antibiotic initiations (15.5% vs 100%, P < .05) without significant differences in recurrence of respiratory symptoms or new antibiotic prescriptions in the month following enrollment.²

A similar randomized trial using a PCT-guided algorithm for the initiation of antibiotics conducted among 337 Swiss children presented to the emergency department (ED) with pneumonia and other LRTIs failed to demonstrate decreases in antibiotic initiation.³ This study used an algorithm that categorized the likelihood of requiring antibiotic treatment for bacterial LRTI as “definitely” if PCT was >0.5 µg/L, “probably” if PCT was 0.26–0.5 µg/L, “probably not” if PCT was 0.1–0.25 µg/L, and “definitely not” if PCT was <0.1 µg/L. In the PCT group, 104 out of 168 (62%) patients received antibiotics within 14 days compared with 93 out of 165 (56%) patients in the control group (odds ratio [OR]: 1.26, 95% CI: 0.81, 1.95). In the subgroup analyses, the odds of administering antibiotics to those with nonpneumonia LRTI was significantly higher than those of the PCT group and control group (OR: 4.09, 95% CI: 1.8, 9.93); the odds of receiving antibiotics also showed no difference in the subgroup of children with pneumonia (OR: 0.66, 95% CI: 0.35, 1.23).

The benefit of PCT for informing decisions around the initiation of antibiotics likely varies based on perceived risk of bacterial diseases. When the pretest probability of bacterial disease is extremely high, the use of PCT is unlikely to alter treatment decisions. Similarly, PCT should not be used in situations where the pretest probability for bacterial pneumonia is very low—in these instances, an elevated PCT may lead to unnecessary antibiotic use among children presenting to the ED. However, the risk of bacterial pneumonia is often equivocal, and in these situations, PCT may provide clinicians with useful insights, primarily for ruling out bacterial disease.

In the study by Esposito et al., the PCT levels were additionally measured every two days until discharge and during two scheduled follow-up visits; the antibiotics were discontinued when PCT < 0.25 µg/L.² The PCT-guided group experienced shorter antibiotic duration (mean 5.4 vs 11.0 days, P < .05), shorter length of hospital stay (mean 4.7 vs 5.61 days for mild LRTI and 5.01 vs 5.93 for severe LRTI), and fewer antibiotic-related adverse events (3.9% vs 25.2%, P < .05). Similarly, in the study by Baer et al., the PCT-guided group had PCT levels repeated on days three and five after enrollment, and the antibiotics were discontinued when PCT was less than 0.25 µg/L. The duration of antibiotic administration was significantly lower in the PCT-guided group (mean difference: 1.8 days, 95% CI: −3.1, −0.).³ The rates of hospitalization, duration of hospital stay, and mean impairment of daily activities attributable to LRTI were similar between groups.

Considering the adult studies and the small number of pediatric LRTI research published to date, the use of PCT to safely reduce antibiotic treatment duration is encouraging.⁴ Although the studies on the kinetics of PCT are limited, the biomarker has been shown to rise two to four hours after a bacterial stimulus, peak in 24-48 hours and achieve a half-life of 24-36 hours.^6,7 As such, serial PCT measurements at 24-hour intervals for three to five days may be more beneficial than stand-alone PCT tests. Nonetheless, additional studies are needed to better define groups of patients who will most likely benefit from PCT testing and to understand how to best integrate testing into clinical practice.

PCT has also been explored as a marker of LRTI disease severity. In a 2008 multicenter cohort encompassing 1,651 adults with pneumonia, PCT < 0.1 µg/L was associated with a decreased 30-day mortality, shorter length of stay, and decreased admission to the intensive care unit (ICU) compared with those with PCT>0.1 µg/L.⁸ In a 2017 study of 317 adults hospitalized with pneumonia, the PCT level was significantly higher in those with bacteremia and in those admitted to intensive care.⁹ When used in combination with the pneumonia severity index (PSI), the addition of PCT resulted in improved prognostic performance compared with the PSI alone for both outcomes, increasing the area under the receiver operating characteristic curve from 0.67 to 0.85 for bacteremia and from 0.58 to 0.64 for intensive care. Similarly, in the adult EPIC cohort, the addition of PCT contributed significant prognostic information beyond existing severity scores for predicting the need for invasive respiratory or vasopressor support; each 1 µg/L increase in PCT was associated with a 1% to 2% absolute increase in the need for this outcome.¹⁰

A European study of 100 children with pneumonia also demonstrated higher PCT values among hospitalized children (n = 26, median PCT 17.8 µg/L) compared with outpatient children (n = 73, median PCT 0.72 µg/L, P < .01).¹¹ Among the 532 children from the EPIC study, a PCT < 0.25 µg/L was associated with the reduced odds of ICU admission (adjusted OR: 0.48; 95% CI: 0.30, 0.78) and a 2.3-day (95% CI: 1.4, 3.2) decrease in the average length of stay compared with those with higher PCT concentrations.⁵ Of the 34 children with empyema requiring drainage, 28 (82%) showed a PCT concentration ≥0.5 µg/L. Additional pediatric studies are needed, but the limited data to date suggest that PCT may play a role in predicting pediatric LRTI disease severity, including the need for mechanical ventilatory support and ICU-level care.

Although PCT shows promise as a biomarker to reliably rule out bacterial infection, several potential limitations exist in assessing its role in pediatric LRTI. Atypical bacterial infections (ie, Mycoplasma pneumoniae) and localized bacterial infection may not induce significant PCT production, as has been shown in adults and children with tonsillitis, localized skin infections, endocarditis, or empyema (Table).¹² The majority of clinical trials in LRTI have been conducted in the adult population,⁴ with the number of pediatric trials remaining small.^2,3 Given the predominance of viral LRTI in children compared with adults, the utility of PCT may differ in these populations.^13,14 Furthermore, existing studies demonstrate mixed results regarding the magnitude of benefits that PCT may provide in terms of limiting antibiotic use. Another concern is the potential of PCT to increase unnecessary antibiotic use in those with viral LRTI,³ as PCT may also be increased in populations with systemic inflammation from nonbacterial causes.^12,15

The misuse of antibiotics is a public health crisis resulting in the emergence of antibiotic-resistant pathogens and adverse outcomes, including Clostridioides difficile infection, drug toxicities, and increased healthcare costs.¹⁶ Pneumonia is responsible for more days of antibiotics than any other disease in children’s hospitals and is an important target for stewardship efforts.¹⁷ PCT is a promising biomarker for distinguishing bacterial from viral infection, and its use may help in making informed antibiotic decisions and predicting disease outcomes in pediatric LRTI. Although PCT has been cleared by the FDA for assisting with antibiotic decisions in pediatric LRTI, the majority of evidence supporting this indication is drawn from adults. Additional studies are needed prior to the widespread implementation in the pediatric population, but the results of available pediatric studies show promise. The clinical context and severity of patient presentation are important when considering whether or not to use PCT and how to best interpret PCT levels when making clinical management decisions. The utility of PCT for antibiotic initiation in the pediatric population is encouraging given the predominance of viral etiologies in pediatric LRTI. Currently available data demonstrate the value of serial PCT measurements in antibiotic de-escalation and promoting antibiotic stewardship for children and adults.^2-4 As with all new diagnostic modalities, provider education is paramount to ensure a safe and value-driven implementation.

Disclosures

Dr. Katz received investigator-initiated grant funding from Roche and bioMérieux to conduct research involving procalcitonin in the past three years. Dr. Sartori has nothing to disclose. Dr. Williams received investigator-initiated grant funding from bioMérieux to conduct research involving procalcitonin in the past three years.

Funding

This work was supported by the National Institute of Health (1T32AI095202-07).

Disclaimer

This content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health, Roche, or bioMérieux.

The Choosing Wisely^® Campaign (CWC) was launched in 2012. This ongoing national initiative encourages conversations among patients and clinicians about the need —or the lack thereof—for frequent tests, treatments, and procedures in healthcare. More than 80 professional societies have developed short lists of evidence-based recommendations aimed at avoiding unnecessary, “low-value” care. More than 550 recommendations are currently available.¹ The Society of Hospital Medicine (SHM) Pediatric Committee published a list of five recommendations for the CWC in 2013.²

After seven years, the campaign has posted several success stories highlighting the increase in clinicians’ awareness about the recommendations. Several local, regional, and national initiatives and quality improvement (QI) projects have been inspired by the CWC and its tenants.^1,3 However, limited research has been performed on the true impact of these recommendations on avoiding “low-value” services. A more comprehensive approach is required to “measure wisely” the impact of the campaign on bedside clinical practice.⁴ Stakeholders in healthcare value have been challenged to collaborate in creating high-impact lists of “low-value” interventions and designing effective tools to measure their impact on clinical practice and costs.⁵

We initially developed a report card with five metrics derived from the CWC-SHM pediatric recommendations to help individual institutions and group practices to measure their performance and benchmark their results with peers.⁶ The report card is available for hospital members of the Children’s Hospital Association (CHA).⁷

The current study analyzes the frequency of utilization and trends of five metrics included in the CHA/Pediatric Health Information System^® (PHIS) CWC report card in tertiary children’s hospitals in the United States. We analyzed data from five years before and five years after the CWC-PHM recommendations were published in 2013. We hypothesize that the publication and dissemination of the CWC-PHM recommendations—the intervention—will result in either an immediate decrease in the use of the “low-value” services studied and/or a change in the trend of utilization over time.

Study Design

We conducted an observational, longitudinal retrospective study aimed at evaluating the impact of the CWC-PHM recommendations on clinical practice in tertiary children’s hospitals in the US.

Study Population

The population included inpatient and observation stays for children aged 0-18 years admitted to the 36 children’s hospitals consistently providing data from 2008 to 2017 to the PHIS administrative database (CHA, Lenexa, Kansas). This database contains inpatient, emergency department, ambulatory, and observation encounter–level data from more than 50 not-for-profit, tertiary care pediatric hospitals and accounts for ~20% of all pediatric hospitalizations in the US every year.

A joint effort between the CHA and the participating hospitals ensures the quality of the data submitted, as previously described.⁸ These data are subjected to a routine quality check with each submission and within each report. Data were fully deidentified for this study. In total, 36 PHIS hospitals met the strict quality standards for inclusion of submitted data. The remaining hospitals were excluded because they did not have complete data or had incomplete billing information.

For external benchmarking purposes, PHIS participating hospitals provide encounter data, including demographics, diagnoses, and procedures (International Classification of Diseases versions 9 and 10).^9,10 The transition from ICD-9 to ICD-10 in the US took place during the study period. However, the CHA completed a process of translating and mapping all ICD-9 codes to every possible equivalent ICD-10 code in the PHIS database. Thus, the change from ICD-9 to ICD-10 should not have had any significant effect on population definition and data analytics, including trend analysis.

For each condition, the study population was divided into the following two cohorts for comparison of the trends: all admissions from January 1, 2008 to December 31, 2012 (before) and all admissions from January 1, 2013 to December 31, 2017 (after) the CWC-PHM recommendations were published.

This study was determined to be nonhuman subject research and was therefore exempted by Nicklaus Children’s Hospital Human Research Protection Program.

Outcomes

The outcomes for this study were the percentages of patients receiving the not-recommended “low-value” services targeted by the CWC-PHM recommendations. For this purpose, four of the five recommendations were translated into the following five metrics, operationalized in the PHIS database and displayed in the “Choosing Wisely” report card:⁶

1. Percentage of patients with uncomplicated asthma receiving chest radiograph (CXR).

2. Percentage of patients with uncomplicated bronchiolitis receiving CXR.

3. Percentage of patients with uncomplicated bronchiolitis receiving bronchodilators.

4. Percentage of patients with lower respiratory tract infection (LRTI) receiving systemic corticosteroids (relievers).

5. Percentage of patients with uncomplicated gastroesophageal reflux (GER) receiving acid suppressor therapy.

The fifth recommendation—limiting the use of continuous pulse oximetry unless the patient is receiving supplemental oxygen—could not be operationalized in the PHIS database because of inconsistent reporting of these resources.⁶

The resulting percentages represent nonadherence to the recommendations, suggesting overuse of the specific “low-value” intervention. As such, a decreasing trend over time is the desired direction of improvement.

The definition of “uncomplicated” conditions and the metrics are presented in Table 1. A complete list of the inclusion and exclusion criteria to define “uncomplicated” conditions and the complete list of the clinical translation codes used in PHIS to identify the “low-value” services are presented as an electronic supplement.

Statistical Analyses

We compared the demographic and clinical characteristics of the various cohorts before and after the release of the CWC-PHM recommendations—the intervention—using chi-square statistics. To assess the individual hospital-level trends over time for each measure, we modeled the patient-level data of each hospital using generalized linear mixed effects models with a binomial distribution. These models were adjusted for patient demographic and clinical factors that were found to be significantly different (P < .01) before and after the intervention on bivariate analyses. From these models, we generated adjusted estimates for the quarterly percentages for each hospital. We then conducted an interrupted time series (ITS) using these estimates to compare trends in the five years before (2008-2012) and five years after (2013-2017) the publication of the CWC-PHM recommendations. For the ITS analysis, we used a generalized linear mixed effects model with the quarterly adjusted hospital-level utilization rates of “low-value” services for each cohort as the unit of analysis and a random intercept for each hospital. The model used an autoregressive(1) covariance structure to account for autocorrelation. The ITS allowed us to test our hypothesis by assessing the following two important features: (a) if a significant decrease occurred right after the CWC-PHM recommendations were published (level-change) and/or (b) if the intervention altered the secular trend (slope-change). All statistical analyses were performed using SAS v. 9.4 (SAS Institute, Cary, North Carolina), and P values <.01 were considered to be statistically significant.

Table 2 presents the demographic characteristics of the cohorts before (2008-2012) and after (2013-2017) the publication of the CWC-PHM recommendations. Hospitalizations due to asthma represented the largest cohort with 142,067 cases, followed by hospitalizations due to bronchiolitis with 94,253 cases. Hospitalizations due to GER comprised the smallest cohort with 13,635 cases. Most of the children had government insurance and had “minor” severity according to the All Patient Revised Diagnosis Related Group (APR-DRG) system.

We found statistically significant differences in most of the demographic characteristics for the cohorts when comparing cases before and after the introduction of the CWC-PHM recommendations.

After adjusting for demographic characteristics, we estimated the percentages of the utilization of the “low-value” services from 2008 to 2017. We observed a steady decrease in overutilization of all services over time. The absolute percentage decrease was more evident in the reduction of the utilization of relievers by 36.6% and that of CXR by 31.5% for bronchiolitis. We also observed a 20.8% absolute reduction in the use of CXR for asthma.

The use of systemic steroids in LRTI revealed the lowest utilization among the “low-value” services studied, with 15.1% in 2008 and 12.2% in 2017, a 2.9% absolute reduction. However, the prescription of acid suppressors for GER showed the highest utilization among all the overuse metrics studied, ie, 63% in 2008 and 48.9% in 2017, with an absolute decrease of 24.1%. The yearly adjusted estimated percentages of utilization for each “low-value” service are presented in Appendix Table A.

Table 3 and the Figure (attached as supplemental online graphic) respectively present the risk-adjusted ITS parameter estimates and the graphic representation before and after the inception of the CWC-PHM recommendations for the trend analysis.

We found a steady reduction in the frequency of overutilization of five “low-value” services described in the CWC-PHM recommendations from 2008 to 2017 in 36 tertiary children’s hospitals in the US. This trend was more evident in the utilization of relievers and CXR for bronchiolitis. The ITS analysis demonstrated that immediately after the publication of the CWC-PHM recommendations, only the use of CXR for asthma decreased significantly. Then, only the use of relievers for bronchiolitis decreased significantly over time in comparison with the secular trend.

These results support our hypothesis for two of the five metrics studied, suggesting that the publication of the CWC-PHM recommendations had a modest impact in clinical practices related to those services in tertiary children’s hospitals.

These findings align with a limited number of published studies that have consistently found a modest decrease in the use of “low-value” services before 2012^11-13 and a limited impact of the CWC in clinical practices on the use of “low-value” services after the inception of the campaign.^14-17

For instance, in a cross-sectional analysis of the 1999 and 2009 samples of ambulatory care practices in the US, only two of 11 overuse quality indicators showed improvement.¹¹ The authors recognized that reducing inappropriate care will require the same attention to guideline development and performance measurement that was directed at reducing the underuse of needed therapies. However, determining whether a patient received inappropriate care generally requires a much more detailed analysis of clinical information than what is required for assessments of underuse.¹¹

Another study designed claims-based algorithms to measure the prevalence of 11 Choosing Wisely-identified “low-value” services in fee-for-service Medicare patients aged >65 years from 2006 to 2011.¹² The annual prevalence of selected CWC “low-value” services ranged from 1.2% (upper urinary tract imaging in men with benign prostatic hyperplasia) to 46.5% (preoperative cardiac testing for low-risk, noncardiac procedures). The study concluded that identifying and measuring “low-value” health services is a prerequisite for improving quality and eliminating waste.¹²

In pediatric medicine, the authors investigated a large cohort of infants aged one to 24 months hospitalized with bronchiolitis to 41 tertiary children’s hospitals reporting data to the PHIS database from 2004 to 2012.¹³ The trend analysis revealed a decrease in the utilization of diagnostics and treatment interventions before the publication of the American Academy of Pediatrics 2006 Bronchiolitis Guidelines.¹⁸ There was an additional reduction in the use of CXR, steroids, and bronchodilators after the publication of the guidelines.¹³

After the CWC was launched in 2012, several surveys have demonstrated a tangible increase in awareness of the CWC and its goals, mostly among primary care physicians and subspecialists. Clinicians who were aware of the campaign found the recommendations to be useful as a legitimate source of guidance and were more likely to reduce the indication of unnecessary care and “low-value” clinical services included in the CWC.^1,3,19,20

Few studies in adults have focused on measuring the trends in overuse metrics derived from the CWC recommendations.^14-16 The initial studies have found limited reduction on the use of “low-value” care after the inception of the CWC. They suggest that clinician education, awareness, and public promotion alone do not appear to be sufficient to achieve widespread changes in clinical practice. Additional interventions are necessary for the wider implementation and success of the CWC recommendations.^{11,14,15,19,21,22}

However, a more recent study was conducted in 91 academic centers from 2013 through 2016, before and after the publication of a CWC recommendation on the use of troponin-only testing for the diagnosis of acute myocardial infarction. Hospitals with low rates of troponin-only testing before the publication of the recommendation demonstrated a statistically significant increase over time in the rate of adherence. The authors postulated that the impact of the CWC might have been significant because of the increase in the institutional and provider attention to “high-value” care as a result of the campaign.¹⁶

In pediatrics, a cross-sectional study defined 20 “low-value” services from a list of more than 400 items from the CWC and other sources of highly regarded, evidence-based pediatrics healthcare recommendations. The list included six diagnostic tests, five imaging tests, and nine prescription drugs ordered in a robust cohort of 4.4 million children nationwide in 2014. The study concluded that approximately one in 10 children received a “low-value” service. The majority (59.4%) were related to prescription drugs, specifically the inappropriate use of antibiotics for a variety of conditions. The estimated combined cost of these unnecessary services was approximately $27 million, with one-third of the cost being paid out of pocket, arguing for significant financial harm. However, this study did not perform a trend analysis.¹⁷

Our results are comparable with these studies, reporting an initial increase in awareness and beliefs, followed by progressive changes in clinical practice among pediatric hospital-based clinicians in delivering evidence-based, high-value care after the CWC.

The attribution of the steady reduction in the absolute percentages of overuse/waste in the five metrics related to the CWC observed in this study, including the significant changes noted in two of the overuse indicators after the publication of the CWC-PHM recommendations, should be interpreted with caution. For example, the significant decrease in the use of “low-value” services in bronchiolitis could be attributed to multiple factors such as national guidelines released in 2014 after the campaign,²³ national multicenter QI collaborative projects,^24,25 and multiple local QI efforts.^26,27 The increase in the awareness and impact of the CWC recommendations among pediatric providers could also be a contributing factor, but this association cannot be established in the light of our findings.

On the other hand, despite extensive evidence for the lack of efficacy and the potential harm associated with the use of acid suppressors for uncomplicated GER in infants,^28-30 the frequency of this “low-value” therapeutic intervention remains high (~50%). The trend in utilization was not impacted by the CWC-PHM recommendations. This finding could be explained by several factors, including the possibility that several hospitalized patients may suffer from GER disease requiring acid suppressors. Another possibility is that acid suppressors are generally prescribed as an outpatient medication, and physicians treating inpatients may be reluctant to discontinue it during hospitalization. Nevertheless, this recommendation represents a target for review, update, and QI interventions in the near future.

The delivery of inappropriate “low-value” care represents the most significant dimension of waste in healthcare.³¹ The development of quality measures of “low-value” services representing overuse and waste is the most needed step toward assessing the magnitude of the problem. Overuse metrics could be incorporated into QI interventions to decrease the provision of such services. However, systematic efforts aimed at developing quality indicators of overuse based on the CWC recommendations have been limited. To our knowledge, this is the first study on the trends of metrics derived from the CWC recommendations in pediatric medicine.

Future research is needed to develop overuse metrics further to assess the specific outcomes related to the implementation of the CWC. How much has clinical practice changed as a result of the campaign? What are the outcomes and savings attributable to these efforts? These are critical questions for the immediate future that should be answered to sustain the ongoing efforts and results and to validate that the efforts are worthwhile.

This study has several limitations. First, this is a retrospective and observational study. It cannot prove a direct causal relationship between the publication of the CWC-PHM and the observed trends, as other potential factors may have contributed to the outcomes. Second, in administrative databases, the data quality is dependent on proper documentation and coding that may vary among reporting institutions. These data lack clinical information, and a fair assessment of “appropriateness” could be questioned. In addition, the study included only 36 academic, tertiary children’s hospitals. Because approximately two-thirds of all pediatric hospitalizations in the US occur in community settings,³² this study may not fully represent clinical practice in the majority of pediatric hospitalizations in the US. Finally, the validity of the ITS analysis has inherent limitations due to the variability of the data in some metrics that may affect the power of the analysis. This fact could lead to inaccurate conclusions regarding intervention effectiveness due to the data-driven model applied, as well as the lack of control for other time-varying confounders.³³

After seven years, the CWC faces important challenges. Critical to the success of the campaign is to “measure wisely” by developing quality indicators of overuse and operationalizing them into administrative and clinical data sources to assess the impact on clinical practice. Our study highlights some limited but steady reduction in the use of some “low-value” services before the campaign. It also demonstrates a modest impact of the campaign on clinical practices in tertiary care children’s hospitals in the US. Clinicians and institutions still have a long way to go in reducing the use of “low-value” interventions in pediatric medicine. These observations challenge us to step up our efforts to implement QI interventions aimed at incorporating these professional, society-endorsed recommendations into our clinical practice.

Acknowledgments

The authors thank Dr. Kristine De La Torre and Dr. Jennifer McCafferty-Fernandez and the Research Institute of Nicklaus Children’s Hospital for medical writing assistance. They also acknowledge Tatiana Consuegra, library technician, for her clerical assistance in the preparation and submission of this article.

The Choosing Wisely^® Campaign (CWC) was launched in 2012. This ongoing national initiative encourages conversations among patients and clinicians about the need —or the lack thereof—for frequent tests, treatments, and procedures in healthcare. More than 80 professional societies have developed short lists of evidence-based recommendations aimed at avoiding unnecessary, “low-value” care. More than 550 recommendations are currently available.¹ The Society of Hospital Medicine (SHM) Pediatric Committee published a list of five recommendations for the CWC in 2013.²

After seven years, the campaign has posted several success stories highlighting the increase in clinicians’ awareness about the recommendations. Several local, regional, and national initiatives and quality improvement (QI) projects have been inspired by the CWC and its tenants.^1,3 However, limited research has been performed on the true impact of these recommendations on avoiding “low-value” services. A more comprehensive approach is required to “measure wisely” the impact of the campaign on bedside clinical practice.⁴ Stakeholders in healthcare value have been challenged to collaborate in creating high-impact lists of “low-value” interventions and designing effective tools to measure their impact on clinical practice and costs.⁵

We initially developed a report card with five metrics derived from the CWC-SHM pediatric recommendations to help individual institutions and group practices to measure their performance and benchmark their results with peers.⁶ The report card is available for hospital members of the Children’s Hospital Association (CHA).⁷

The current study analyzes the frequency of utilization and trends of five metrics included in the CHA/Pediatric Health Information System^® (PHIS) CWC report card in tertiary children’s hospitals in the United States. We analyzed data from five years before and five years after the CWC-PHM recommendations were published in 2013. We hypothesize that the publication and dissemination of the CWC-PHM recommendations—the intervention—will result in either an immediate decrease in the use of the “low-value” services studied and/or a change in the trend of utilization over time.

Study Design

We conducted an observational, longitudinal retrospective study aimed at evaluating the impact of the CWC-PHM recommendations on clinical practice in tertiary children’s hospitals in the US.

Study Population

The population included inpatient and observation stays for children aged 0-18 years admitted to the 36 children’s hospitals consistently providing data from 2008 to 2017 to the PHIS administrative database (CHA, Lenexa, Kansas). This database contains inpatient, emergency department, ambulatory, and observation encounter–level data from more than 50 not-for-profit, tertiary care pediatric hospitals and accounts for ~20% of all pediatric hospitalizations in the US every year.

A joint effort between the CHA and the participating hospitals ensures the quality of the data submitted, as previously described.⁸ These data are subjected to a routine quality check with each submission and within each report. Data were fully deidentified for this study. In total, 36 PHIS hospitals met the strict quality standards for inclusion of submitted data. The remaining hospitals were excluded because they did not have complete data or had incomplete billing information.

For external benchmarking purposes, PHIS participating hospitals provide encounter data, including demographics, diagnoses, and procedures (International Classification of Diseases versions 9 and 10).^9,10 The transition from ICD-9 to ICD-10 in the US took place during the study period. However, the CHA completed a process of translating and mapping all ICD-9 codes to every possible equivalent ICD-10 code in the PHIS database. Thus, the change from ICD-9 to ICD-10 should not have had any significant effect on population definition and data analytics, including trend analysis.

For each condition, the study population was divided into the following two cohorts for comparison of the trends: all admissions from January 1, 2008 to December 31, 2012 (before) and all admissions from January 1, 2013 to December 31, 2017 (after) the CWC-PHM recommendations were published.

This study was determined to be nonhuman subject research and was therefore exempted by Nicklaus Children’s Hospital Human Research Protection Program.

Outcomes

The outcomes for this study were the percentages of patients receiving the not-recommended “low-value” services targeted by the CWC-PHM recommendations. For this purpose, four of the five recommendations were translated into the following five metrics, operationalized in the PHIS database and displayed in the “Choosing Wisely” report card:⁶

1. Percentage of patients with uncomplicated asthma receiving chest radiograph (CXR).

2. Percentage of patients with uncomplicated bronchiolitis receiving CXR.

3. Percentage of patients with uncomplicated bronchiolitis receiving bronchodilators.

4. Percentage of patients with lower respiratory tract infection (LRTI) receiving systemic corticosteroids (relievers).

5. Percentage of patients with uncomplicated gastroesophageal reflux (GER) receiving acid suppressor therapy.

The fifth recommendation—limiting the use of continuous pulse oximetry unless the patient is receiving supplemental oxygen—could not be operationalized in the PHIS database because of inconsistent reporting of these resources.⁶

The resulting percentages represent nonadherence to the recommendations, suggesting overuse of the specific “low-value” intervention. As such, a decreasing trend over time is the desired direction of improvement.

The definition of “uncomplicated” conditions and the metrics are presented in Table 1. A complete list of the inclusion and exclusion criteria to define “uncomplicated” conditions and the complete list of the clinical translation codes used in PHIS to identify the “low-value” services are presented as an electronic supplement.

Statistical Analyses

We compared the demographic and clinical characteristics of the various cohorts before and after the release of the CWC-PHM recommendations—the intervention—using chi-square statistics. To assess the individual hospital-level trends over time for each measure, we modeled the patient-level data of each hospital using generalized linear mixed effects models with a binomial distribution. These models were adjusted for patient demographic and clinical factors that were found to be significantly different (P < .01) before and after the intervention on bivariate analyses. From these models, we generated adjusted estimates for the quarterly percentages for each hospital. We then conducted an interrupted time series (ITS) using these estimates to compare trends in the five years before (2008-2012) and five years after (2013-2017) the publication of the CWC-PHM recommendations. For the ITS analysis, we used a generalized linear mixed effects model with the quarterly adjusted hospital-level utilization rates of “low-value” services for each cohort as the unit of analysis and a random intercept for each hospital. The model used an autoregressive(1) covariance structure to account for autocorrelation. The ITS allowed us to test our hypothesis by assessing the following two important features: (a) if a significant decrease occurred right after the CWC-PHM recommendations were published (level-change) and/or (b) if the intervention altered the secular trend (slope-change). All statistical analyses were performed using SAS v. 9.4 (SAS Institute, Cary, North Carolina), and P values <.01 were considered to be statistically significant.

Table 2 presents the demographic characteristics of the cohorts before (2008-2012) and after (2013-2017) the publication of the CWC-PHM recommendations. Hospitalizations due to asthma represented the largest cohort with 142,067 cases, followed by hospitalizations due to bronchiolitis with 94,253 cases. Hospitalizations due to GER comprised the smallest cohort with 13,635 cases. Most of the children had government insurance and had “minor” severity according to the All Patient Revised Diagnosis Related Group (APR-DRG) system.

We found statistically significant differences in most of the demographic characteristics for the cohorts when comparing cases before and after the introduction of the CWC-PHM recommendations.

After adjusting for demographic characteristics, we estimated the percentages of the utilization of the “low-value” services from 2008 to 2017. We observed a steady decrease in overutilization of all services over time. The absolute percentage decrease was more evident in the reduction of the utilization of relievers by 36.6% and that of CXR by 31.5% for bronchiolitis. We also observed a 20.8% absolute reduction in the use of CXR for asthma.

The use of systemic steroids in LRTI revealed the lowest utilization among the “low-value” services studied, with 15.1% in 2008 and 12.2% in 2017, a 2.9% absolute reduction. However, the prescription of acid suppressors for GER showed the highest utilization among all the overuse metrics studied, ie, 63% in 2008 and 48.9% in 2017, with an absolute decrease of 24.1%. The yearly adjusted estimated percentages of utilization for each “low-value” service are presented in Appendix Table A.

Table 3 and the Figure (attached as supplemental online graphic) respectively present the risk-adjusted ITS parameter estimates and the graphic representation before and after the inception of the CWC-PHM recommendations for the trend analysis.

We found a steady reduction in the frequency of overutilization of five “low-value” services described in the CWC-PHM recommendations from 2008 to 2017 in 36 tertiary children’s hospitals in the US. This trend was more evident in the utilization of relievers and CXR for bronchiolitis. The ITS analysis demonstrated that immediately after the publication of the CWC-PHM recommendations, only the use of CXR for asthma decreased significantly. Then, only the use of relievers for bronchiolitis decreased significantly over time in comparison with the secular trend.

These results support our hypothesis for two of the five metrics studied, suggesting that the publication of the CWC-PHM recommendations had a modest impact in clinical practices related to those services in tertiary children’s hospitals.

These findings align with a limited number of published studies that have consistently found a modest decrease in the use of “low-value” services before 2012^11-13 and a limited impact of the CWC in clinical practices on the use of “low-value” services after the inception of the campaign.^14-17

For instance, in a cross-sectional analysis of the 1999 and 2009 samples of ambulatory care practices in the US, only two of 11 overuse quality indicators showed improvement.¹¹ The authors recognized that reducing inappropriate care will require the same attention to guideline development and performance measurement that was directed at reducing the underuse of needed therapies. However, determining whether a patient received inappropriate care generally requires a much more detailed analysis of clinical information than what is required for assessments of underuse.¹¹

Another study designed claims-based algorithms to measure the prevalence of 11 Choosing Wisely-identified “low-value” services in fee-for-service Medicare patients aged >65 years from 2006 to 2011.¹² The annual prevalence of selected CWC “low-value” services ranged from 1.2% (upper urinary tract imaging in men with benign prostatic hyperplasia) to 46.5% (preoperative cardiac testing for low-risk, noncardiac procedures). The study concluded that identifying and measuring “low-value” health services is a prerequisite for improving quality and eliminating waste.¹²

In pediatric medicine, the authors investigated a large cohort of infants aged one to 24 months hospitalized with bronchiolitis to 41 tertiary children’s hospitals reporting data to the PHIS database from 2004 to 2012.¹³ The trend analysis revealed a decrease in the utilization of diagnostics and treatment interventions before the publication of the American Academy of Pediatrics 2006 Bronchiolitis Guidelines.¹⁸ There was an additional reduction in the use of CXR, steroids, and bronchodilators after the publication of the guidelines.¹³

After the CWC was launched in 2012, several surveys have demonstrated a tangible increase in awareness of the CWC and its goals, mostly among primary care physicians and subspecialists. Clinicians who were aware of the campaign found the recommendations to be useful as a legitimate source of guidance and were more likely to reduce the indication of unnecessary care and “low-value” clinical services included in the CWC.^1,3,19,20

Few studies in adults have focused on measuring the trends in overuse metrics derived from the CWC recommendations.^14-16 The initial studies have found limited reduction on the use of “low-value” care after the inception of the CWC. They suggest that clinician education, awareness, and public promotion alone do not appear to be sufficient to achieve widespread changes in clinical practice. Additional interventions are necessary for the wider implementation and success of the CWC recommendations.^{11,14,15,19,21,22}

However, a more recent study was conducted in 91 academic centers from 2013 through 2016, before and after the publication of a CWC recommendation on the use of troponin-only testing for the diagnosis of acute myocardial infarction. Hospitals with low rates of troponin-only testing before the publication of the recommendation demonstrated a statistically significant increase over time in the rate of adherence. The authors postulated that the impact of the CWC might have been significant because of the increase in the institutional and provider attention to “high-value” care as a result of the campaign.¹⁶

In pediatrics, a cross-sectional study defined 20 “low-value” services from a list of more than 400 items from the CWC and other sources of highly regarded, evidence-based pediatrics healthcare recommendations. The list included six diagnostic tests, five imaging tests, and nine prescription drugs ordered in a robust cohort of 4.4 million children nationwide in 2014. The study concluded that approximately one in 10 children received a “low-value” service. The majority (59.4%) were related to prescription drugs, specifically the inappropriate use of antibiotics for a variety of conditions. The estimated combined cost of these unnecessary services was approximately $27 million, with one-third of the cost being paid out of pocket, arguing for significant financial harm. However, this study did not perform a trend analysis.¹⁷

Our results are comparable with these studies, reporting an initial increase in awareness and beliefs, followed by progressive changes in clinical practice among pediatric hospital-based clinicians in delivering evidence-based, high-value care after the CWC.

The attribution of the steady reduction in the absolute percentages of overuse/waste in the five metrics related to the CWC observed in this study, including the significant changes noted in two of the overuse indicators after the publication of the CWC-PHM recommendations, should be interpreted with caution. For example, the significant decrease in the use of “low-value” services in bronchiolitis could be attributed to multiple factors such as national guidelines released in 2014 after the campaign,²³ national multicenter QI collaborative projects,^24,25 and multiple local QI efforts.^26,27 The increase in the awareness and impact of the CWC recommendations among pediatric providers could also be a contributing factor, but this association cannot be established in the light of our findings.

On the other hand, despite extensive evidence for the lack of efficacy and the potential harm associated with the use of acid suppressors for uncomplicated GER in infants,^28-30 the frequency of this “low-value” therapeutic intervention remains high (~50%). The trend in utilization was not impacted by the CWC-PHM recommendations. This finding could be explained by several factors, including the possibility that several hospitalized patients may suffer from GER disease requiring acid suppressors. Another possibility is that acid suppressors are generally prescribed as an outpatient medication, and physicians treating inpatients may be reluctant to discontinue it during hospitalization. Nevertheless, this recommendation represents a target for review, update, and QI interventions in the near future.

The delivery of inappropriate “low-value” care represents the most significant dimension of waste in healthcare.³¹ The development of quality measures of “low-value” services representing overuse and waste is the most needed step toward assessing the magnitude of the problem. Overuse metrics could be incorporated into QI interventions to decrease the provision of such services. However, systematic efforts aimed at developing quality indicators of overuse based on the CWC recommendations have been limited. To our knowledge, this is the first study on the trends of metrics derived from the CWC recommendations in pediatric medicine.

Future research is needed to develop overuse metrics further to assess the specific outcomes related to the implementation of the CWC. How much has clinical practice changed as a result of the campaign? What are the outcomes and savings attributable to these efforts? These are critical questions for the immediate future that should be answered to sustain the ongoing efforts and results and to validate that the efforts are worthwhile.

This study has several limitations. First, this is a retrospective and observational study. It cannot prove a direct causal relationship between the publication of the CWC-PHM and the observed trends, as other potential factors may have contributed to the outcomes. Second, in administrative databases, the data quality is dependent on proper documentation and coding that may vary among reporting institutions. These data lack clinical information, and a fair assessment of “appropriateness” could be questioned. In addition, the study included only 36 academic, tertiary children’s hospitals. Because approximately two-thirds of all pediatric hospitalizations in the US occur in community settings,³² this study may not fully represent clinical practice in the majority of pediatric hospitalizations in the US. Finally, the validity of the ITS analysis has inherent limitations due to the variability of the data in some metrics that may affect the power of the analysis. This fact could lead to inaccurate conclusions regarding intervention effectiveness due to the data-driven model applied, as well as the lack of control for other time-varying confounders.³³

After seven years, the CWC faces important challenges. Critical to the success of the campaign is to “measure wisely” by developing quality indicators of overuse and operationalizing them into administrative and clinical data sources to assess the impact on clinical practice. Our study highlights some limited but steady reduction in the use of some “low-value” services before the campaign. It also demonstrates a modest impact of the campaign on clinical practices in tertiary care children’s hospitals in the US. Clinicians and institutions still have a long way to go in reducing the use of “low-value” interventions in pediatric medicine. These observations challenge us to step up our efforts to implement QI interventions aimed at incorporating these professional, society-endorsed recommendations into our clinical practice.

Acknowledgments

The authors thank Dr. Kristine De La Torre and Dr. Jennifer McCafferty-Fernandez and the Research Institute of Nicklaus Children’s Hospital for medical writing assistance. They also acknowledge Tatiana Consuegra, library technician, for her clerical assistance in the preparation and submission of this article.

The Choosing Wisely^® Campaign (CWC) was launched in 2012. This ongoing national initiative encourages conversations among patients and clinicians about the need —or the lack thereof—for frequent tests, treatments, and procedures in healthcare. More than 80 professional societies have developed short lists of evidence-based recommendations aimed at avoiding unnecessary, “low-value” care. More than 550 recommendations are currently available.¹ The Society of Hospital Medicine (SHM) Pediatric Committee published a list of five recommendations for the CWC in 2013.²

After seven years, the campaign has posted several success stories highlighting the increase in clinicians’ awareness about the recommendations. Several local, regional, and national initiatives and quality improvement (QI) projects have been inspired by the CWC and its tenants.^1,3 However, limited research has been performed on the true impact of these recommendations on avoiding “low-value” services. A more comprehensive approach is required to “measure wisely” the impact of the campaign on bedside clinical practice.⁴ Stakeholders in healthcare value have been challenged to collaborate in creating high-impact lists of “low-value” interventions and designing effective tools to measure their impact on clinical practice and costs.⁵

We initially developed a report card with five metrics derived from the CWC-SHM pediatric recommendations to help individual institutions and group practices to measure their performance and benchmark their results with peers.⁶ The report card is available for hospital members of the Children’s Hospital Association (CHA).⁷

The current study analyzes the frequency of utilization and trends of five metrics included in the CHA/Pediatric Health Information System^® (PHIS) CWC report card in tertiary children’s hospitals in the United States. We analyzed data from five years before and five years after the CWC-PHM recommendations were published in 2013. We hypothesize that the publication and dissemination of the CWC-PHM recommendations—the intervention—will result in either an immediate decrease in the use of the “low-value” services studied and/or a change in the trend of utilization over time.

Study Design

We conducted an observational, longitudinal retrospective study aimed at evaluating the impact of the CWC-PHM recommendations on clinical practice in tertiary children’s hospitals in the US.

Study Population

The population included inpatient and observation stays for children aged 0-18 years admitted to the 36 children’s hospitals consistently providing data from 2008 to 2017 to the PHIS administrative database (CHA, Lenexa, Kansas). This database contains inpatient, emergency department, ambulatory, and observation encounter–level data from more than 50 not-for-profit, tertiary care pediatric hospitals and accounts for ~20% of all pediatric hospitalizations in the US every year.

A joint effort between the CHA and the participating hospitals ensures the quality of the data submitted, as previously described.⁸ These data are subjected to a routine quality check with each submission and within each report. Data were fully deidentified for this study. In total, 36 PHIS hospitals met the strict quality standards for inclusion of submitted data. The remaining hospitals were excluded because they did not have complete data or had incomplete billing information.

For external benchmarking purposes, PHIS participating hospitals provide encounter data, including demographics, diagnoses, and procedures (International Classification of Diseases versions 9 and 10).^9,10 The transition from ICD-9 to ICD-10 in the US took place during the study period. However, the CHA completed a process of translating and mapping all ICD-9 codes to every possible equivalent ICD-10 code in the PHIS database. Thus, the change from ICD-9 to ICD-10 should not have had any significant effect on population definition and data analytics, including trend analysis.

For each condition, the study population was divided into the following two cohorts for comparison of the trends: all admissions from January 1, 2008 to December 31, 2012 (before) and all admissions from January 1, 2013 to December 31, 2017 (after) the CWC-PHM recommendations were published.

This study was determined to be nonhuman subject research and was therefore exempted by Nicklaus Children’s Hospital Human Research Protection Program.

Outcomes

The outcomes for this study were the percentages of patients receiving the not-recommended “low-value” services targeted by the CWC-PHM recommendations. For this purpose, four of the five recommendations were translated into the following five metrics, operationalized in the PHIS database and displayed in the “Choosing Wisely” report card:⁶

1. Percentage of patients with uncomplicated asthma receiving chest radiograph (CXR).

2. Percentage of patients with uncomplicated bronchiolitis receiving CXR.

3. Percentage of patients with uncomplicated bronchiolitis receiving bronchodilators.

4. Percentage of patients with lower respiratory tract infection (LRTI) receiving systemic corticosteroids (relievers).

5. Percentage of patients with uncomplicated gastroesophageal reflux (GER) receiving acid suppressor therapy.

The fifth recommendation—limiting the use of continuous pulse oximetry unless the patient is receiving supplemental oxygen—could not be operationalized in the PHIS database because of inconsistent reporting of these resources.⁶

The resulting percentages represent nonadherence to the recommendations, suggesting overuse of the specific “low-value” intervention. As such, a decreasing trend over time is the desired direction of improvement.

The definition of “uncomplicated” conditions and the metrics are presented in Table 1. A complete list of the inclusion and exclusion criteria to define “uncomplicated” conditions and the complete list of the clinical translation codes used in PHIS to identify the “low-value” services are presented as an electronic supplement.

Statistical Analyses

We compared the demographic and clinical characteristics of the various cohorts before and after the release of the CWC-PHM recommendations—the intervention—using chi-square statistics. To assess the individual hospital-level trends over time for each measure, we modeled the patient-level data of each hospital using generalized linear mixed effects models with a binomial distribution. These models were adjusted for patient demographic and clinical factors that were found to be significantly different (P < .01) before and after the intervention on bivariate analyses. From these models, we generated adjusted estimates for the quarterly percentages for each hospital. We then conducted an interrupted time series (ITS) using these estimates to compare trends in the five years before (2008-2012) and five years after (2013-2017) the publication of the CWC-PHM recommendations. For the ITS analysis, we used a generalized linear mixed effects model with the quarterly adjusted hospital-level utilization rates of “low-value” services for each cohort as the unit of analysis and a random intercept for each hospital. The model used an autoregressive(1) covariance structure to account for autocorrelation. The ITS allowed us to test our hypothesis by assessing the following two important features: (a) if a significant decrease occurred right after the CWC-PHM recommendations were published (level-change) and/or (b) if the intervention altered the secular trend (slope-change). All statistical analyses were performed using SAS v. 9.4 (SAS Institute, Cary, North Carolina), and P values <.01 were considered to be statistically significant.

Table 2 presents the demographic characteristics of the cohorts before (2008-2012) and after (2013-2017) the publication of the CWC-PHM recommendations. Hospitalizations due to asthma represented the largest cohort with 142,067 cases, followed by hospitalizations due to bronchiolitis with 94,253 cases. Hospitalizations due to GER comprised the smallest cohort with 13,635 cases. Most of the children had government insurance and had “minor” severity according to the All Patient Revised Diagnosis Related Group (APR-DRG) system.

We found statistically significant differences in most of the demographic characteristics for the cohorts when comparing cases before and after the introduction of the CWC-PHM recommendations.

After adjusting for demographic characteristics, we estimated the percentages of the utilization of the “low-value” services from 2008 to 2017. We observed a steady decrease in overutilization of all services over time. The absolute percentage decrease was more evident in the reduction of the utilization of relievers by 36.6% and that of CXR by 31.5% for bronchiolitis. We also observed a 20.8% absolute reduction in the use of CXR for asthma.

The use of systemic steroids in LRTI revealed the lowest utilization among the “low-value” services studied, with 15.1% in 2008 and 12.2% in 2017, a 2.9% absolute reduction. However, the prescription of acid suppressors for GER showed the highest utilization among all the overuse metrics studied, ie, 63% in 2008 and 48.9% in 2017, with an absolute decrease of 24.1%. The yearly adjusted estimated percentages of utilization for each “low-value” service are presented in Appendix Table A.

Table 3 and the Figure (attached as supplemental online graphic) respectively present the risk-adjusted ITS parameter estimates and the graphic representation before and after the inception of the CWC-PHM recommendations for the trend analysis.

We found a steady reduction in the frequency of overutilization of five “low-value” services described in the CWC-PHM recommendations from 2008 to 2017 in 36 tertiary children’s hospitals in the US. This trend was more evident in the utilization of relievers and CXR for bronchiolitis. The ITS analysis demonstrated that immediately after the publication of the CWC-PHM recommendations, only the use of CXR for asthma decreased significantly. Then, only the use of relievers for bronchiolitis decreased significantly over time in comparison with the secular trend.

These results support our hypothesis for two of the five metrics studied, suggesting that the publication of the CWC-PHM recommendations had a modest impact in clinical practices related to those services in tertiary children’s hospitals.

These findings align with a limited number of published studies that have consistently found a modest decrease in the use of “low-value” services before 2012^11-13 and a limited impact of the CWC in clinical practices on the use of “low-value” services after the inception of the campaign.^14-17

For instance, in a cross-sectional analysis of the 1999 and 2009 samples of ambulatory care practices in the US, only two of 11 overuse quality indicators showed improvement.¹¹ The authors recognized that reducing inappropriate care will require the same attention to guideline development and performance measurement that was directed at reducing the underuse of needed therapies. However, determining whether a patient received inappropriate care generally requires a much more detailed analysis of clinical information than what is required for assessments of underuse.¹¹

Another study designed claims-based algorithms to measure the prevalence of 11 Choosing Wisely-identified “low-value” services in fee-for-service Medicare patients aged >65 years from 2006 to 2011.¹² The annual prevalence of selected CWC “low-value” services ranged from 1.2% (upper urinary tract imaging in men with benign prostatic hyperplasia) to 46.5% (preoperative cardiac testing for low-risk, noncardiac procedures). The study concluded that identifying and measuring “low-value” health services is a prerequisite for improving quality and eliminating waste.¹²

In pediatric medicine, the authors investigated a large cohort of infants aged one to 24 months hospitalized with bronchiolitis to 41 tertiary children’s hospitals reporting data to the PHIS database from 2004 to 2012.¹³ The trend analysis revealed a decrease in the utilization of diagnostics and treatment interventions before the publication of the American Academy of Pediatrics 2006 Bronchiolitis Guidelines.¹⁸ There was an additional reduction in the use of CXR, steroids, and bronchodilators after the publication of the guidelines.¹³

After the CWC was launched in 2012, several surveys have demonstrated a tangible increase in awareness of the CWC and its goals, mostly among primary care physicians and subspecialists. Clinicians who were aware of the campaign found the recommendations to be useful as a legitimate source of guidance and were more likely to reduce the indication of unnecessary care and “low-value” clinical services included in the CWC.^1,3,19,20

Few studies in adults have focused on measuring the trends in overuse metrics derived from the CWC recommendations.^14-16 The initial studies have found limited reduction on the use of “low-value” care after the inception of the CWC. They suggest that clinician education, awareness, and public promotion alone do not appear to be sufficient to achieve widespread changes in clinical practice. Additional interventions are necessary for the wider implementation and success of the CWC recommendations.^{11,14,15,19,21,22}

However, a more recent study was conducted in 91 academic centers from 2013 through 2016, before and after the publication of a CWC recommendation on the use of troponin-only testing for the diagnosis of acute myocardial infarction. Hospitals with low rates of troponin-only testing before the publication of the recommendation demonstrated a statistically significant increase over time in the rate of adherence. The authors postulated that the impact of the CWC might have been significant because of the increase in the institutional and provider attention to “high-value” care as a result of the campaign.¹⁶

In pediatrics, a cross-sectional study defined 20 “low-value” services from a list of more than 400 items from the CWC and other sources of highly regarded, evidence-based pediatrics healthcare recommendations. The list included six diagnostic tests, five imaging tests, and nine prescription drugs ordered in a robust cohort of 4.4 million children nationwide in 2014. The study concluded that approximately one in 10 children received a “low-value” service. The majority (59.4%) were related to prescription drugs, specifically the inappropriate use of antibiotics for a variety of conditions. The estimated combined cost of these unnecessary services was approximately $27 million, with one-third of the cost being paid out of pocket, arguing for significant financial harm. However, this study did not perform a trend analysis.¹⁷

Our results are comparable with these studies, reporting an initial increase in awareness and beliefs, followed by progressive changes in clinical practice among pediatric hospital-based clinicians in delivering evidence-based, high-value care after the CWC.

The attribution of the steady reduction in the absolute percentages of overuse/waste in the five metrics related to the CWC observed in this study, including the significant changes noted in two of the overuse indicators after the publication of the CWC-PHM recommendations, should be interpreted with caution. For example, the significant decrease in the use of “low-value” services in bronchiolitis could be attributed to multiple factors such as national guidelines released in 2014 after the campaign,²³ national multicenter QI collaborative projects,^24,25 and multiple local QI efforts.^26,27 The increase in the awareness and impact of the CWC recommendations among pediatric providers could also be a contributing factor, but this association cannot be established in the light of our findings.

On the other hand, despite extensive evidence for the lack of efficacy and the potential harm associated with the use of acid suppressors for uncomplicated GER in infants,^28-30 the frequency of this “low-value” therapeutic intervention remains high (~50%). The trend in utilization was not impacted by the CWC-PHM recommendations. This finding could be explained by several factors, including the possibility that several hospitalized patients may suffer from GER disease requiring acid suppressors. Another possibility is that acid suppressors are generally prescribed as an outpatient medication, and physicians treating inpatients may be reluctant to discontinue it during hospitalization. Nevertheless, this recommendation represents a target for review, update, and QI interventions in the near future.

The delivery of inappropriate “low-value” care represents the most significant dimension of waste in healthcare.³¹ The development of quality measures of “low-value” services representing overuse and waste is the most needed step toward assessing the magnitude of the problem. Overuse metrics could be incorporated into QI interventions to decrease the provision of such services. However, systematic efforts aimed at developing quality indicators of overuse based on the CWC recommendations have been limited. To our knowledge, this is the first study on the trends of metrics derived from the CWC recommendations in pediatric medicine.

Future research is needed to develop overuse metrics further to assess the specific outcomes related to the implementation of the CWC. How much has clinical practice changed as a result of the campaign? What are the outcomes and savings attributable to these efforts? These are critical questions for the immediate future that should be answered to sustain the ongoing efforts and results and to validate that the efforts are worthwhile.

This study has several limitations. First, this is a retrospective and observational study. It cannot prove a direct causal relationship between the publication of the CWC-PHM and the observed trends, as other potential factors may have contributed to the outcomes. Second, in administrative databases, the data quality is dependent on proper documentation and coding that may vary among reporting institutions. These data lack clinical information, and a fair assessment of “appropriateness” could be questioned. In addition, the study included only 36 academic, tertiary children’s hospitals. Because approximately two-thirds of all pediatric hospitalizations in the US occur in community settings,³² this study may not fully represent clinical practice in the majority of pediatric hospitalizations in the US. Finally, the validity of the ITS analysis has inherent limitations due to the variability of the data in some metrics that may affect the power of the analysis. This fact could lead to inaccurate conclusions regarding intervention effectiveness due to the data-driven model applied, as well as the lack of control for other time-varying confounders.³³

After seven years, the CWC faces important challenges. Critical to the success of the campaign is to “measure wisely” by developing quality indicators of overuse and operationalizing them into administrative and clinical data sources to assess the impact on clinical practice. Our study highlights some limited but steady reduction in the use of some “low-value” services before the campaign. It also demonstrates a modest impact of the campaign on clinical practices in tertiary care children’s hospitals in the US. Clinicians and institutions still have a long way to go in reducing the use of “low-value” interventions in pediatric medicine. These observations challenge us to step up our efforts to implement QI interventions aimed at incorporating these professional, society-endorsed recommendations into our clinical practice.

Acknowledgments

The authors thank Dr. Kristine De La Torre and Dr. Jennifer McCafferty-Fernandez and the Research Institute of Nicklaus Children’s Hospital for medical writing assistance. They also acknowledge Tatiana Consuegra, library technician, for her clerical assistance in the preparation and submission of this article.

Despite the widespread availability of evidence-based guidelines,¹ there is inappropriate variation in the care and outcomes for children with asthma in both the emergency department (ED) and the inpatient setting.^2-6 Operational versions of evidence-based guidelines known as “pathways” have been shown to improve adoption of evidence-based guidelines, quality of care, and health outcomes for children with asthma.^7-14 However, little is known about how to successfully implement pathways outside of free-standing children’s hospitals.^15-19

The majority of children with asthma in the United States are cared for in community hospitals, which provide services for both adults and children.²⁰ However, prior studies of pediatric asthma pathways have largely excluded community hospitals. These studies primarily focused on determining clinical effectiveness, rather than detailing the implementation process. These approaches have left critical gaps that hinder our ability to implement pathways and improve care in community hospitals, which have unique barriers and less resources.^21,22 Therefore, understanding the process of pathway implementation in community hospitals is critical to improving care for children.²² Our objective was to identify the key determinants of successful pediatric asthma pathway implementation using a national sample of community hospitals. This knowledge can guide hospital leaders and healthcare providers in efforts to improve pediatric care and outcomes in these settings.

Study Setting, Design, and Population

In Fall 2017, the Value in Inpatient Pediatrics (VIP) network launched PIPA, Pathways to Improving Pediatric Asthma care.²³ The VIP network, a part of the American Academy of Pediatrics (AAP), aims to improve the value of care delivered to any pediatric patient in a hospital bed, from rural to free-standing children’s hospitals.²⁴ PIPA used a learning collaborative model²⁵ and recruited local project leaders (physicians, nurses, respiratory therapists (RT), and pharmacists) from 89 hospitals around the country. PIPA provided these hospital teams with asthma pathways and several resources for implementation support, including educational meetings, quality improvement (QI) training, audit and feedback, and facilitation. Facilitation is a process of interactive problem-solving and support that occurs in the context of a supportive interpersonal relationship and a recognized need for improvement.²⁶ A facilitator, or a “coach”, is an external expert who provides project mentorship and assists the process of making meaningful changes to improve patient care.²⁶ Facilitation was provided by external consultants with QI expertise.

For this qualitative study, facilitators conducted semi-structured interviews with a convenience sample of project leaders from community hospitals participating in PIPA, with some interviews including multiple project leaders (eg, nursing, inpatient, and Emergency Department [ED] leaders). Verbal consent was obtained from all participants. No incentives were provided. This study was approved by the AAP institutional review board.

Data Collection

We used the constructs described in the Consolidated Framework for Implementation Research (CFIR)²⁷ and adapted those salient to pediatric asthma pathways to develop an interview guide that was used with all participants (Appendix 1). The CFIR offers an overarching typology to understand what works where and why across five major domains that influence implementation: intervention characteristics, inner setting (hospital), outer setting (economic, political, and social context of the hospital), characteristics of the individuals involved, and the process of implementation. Data were collected across these domains to inform our analysis of the key determinants of pediatric asthma pathway implementation in community hospitals.

Interviews were conducted by phone from December 2017 to April 2018 (first four months of pathway implementation). Interviews lasted 30-60 minutes and were recorded and transcribed verbatim. Transcripts were edited for accuracy using the audio recordings. As data collection occurred concurrently with analysis, the interview guide was iteratively revised to reflect new insights and patterns that emerged from our analysis. All sites were anonymized in the data analysis. New interviews were coded until thematic saturation was reached.

Analysis

We conducted an inductive thematic analysis using the CFIR as our conceptual framework.^28,29 Four investigators (CM, MJ, ES, and SK) performed the initial open coding of the data. Investigators met twice during the open coding process to develop and then finalize a codebook of standard definitions for codes. This codebook facilitated coding consistency through the remainder of the analytic process. Two investigators (CM and MJ) then independently read and coded all data to ensure intercoder reliability. During this process, CM and MJ met every two weeks to compare coding consistency, resolve discrepancies, and discuss preliminary findings. When the coding was complete, all investigators met to explore and develop themes that encompassed related common codes.

The CFIR was used at two stages of the study: (1) developing the interview guide and (2) cross-checking for any potentially important codes that were missing/needed to be explored further. Thus, the investigators maintained an inductive approach grounded in the data. To assure study rigor, we employed investigator triangulation (use of multiple investigators and participants from multiple clinical roles) and reflexivity (ongoing critique and critical reflection of the individual biases of the investigators).³⁰ Coding was performed using Dedoose (version 7.0.23; Los Angeles, California).

A total of 34 community hospitals completed the PIPA project, of which the project leaders of 25 hospitals connected with the facilitators and were approached to participate; 18 (72%) hospitals’ project leaders participated in the study. We analyzed 18 interviews conducted between facilitators and project leaders, which included a total of 32 project leaders (one to five leaders per interview). The hospitals represented were diverse in geographic location and size (range 4-50 pediatric beds per hospital), and the majority of sites (78%) supported the trainees (Table 1).

We identified four overarching themes that described the key determinants of pathway implementation in community hospitals. These themes are presented in order of their frequency of occurrence in the data. They included (1) building an implementation infrastructure, (2) engaging and motivating providers, (3) addressing organizational and resource limitations, and (4) devising implementation solutions with facilitators. Descriptions and exemplary quotations for each theme are provided in Table 2 and Appendix Figure 1.

Building an Implementation Infrastructure

“So, I’m going to sit down with the primary nursing staff and the other four physicians in the group to go over the expectations…We’re not going to have the actual EMR [changes] and we’re not going to have the nursing documentation field built right away but [we want to] make sure that people are documenting the respiratory score in their generic nursing note so that the information is easily accessible.” (Physician leader, Hospital G)

Participants also described the need to deliver education on the evidence supporting changes in practice and skills training specific to pediatric asthma care:

“Once we realized that we were going to be doing this pathway, we started training our nurses on the inpatient side on [pediatric respiratory scoring].” (Nursing leader, Hospital P)

In addition, pathway implementation required modification of clinical workflows via changes to hospital policies or guidelines, electronic medical records (EMR), and/or the physical environment (eg, placing supplies in proximity to care delivery):

“I think it can help if we could get an order set or a nursing protocol where asthmatics over a certain severity can just get steroids in triage.” (Physician leader, Hospital A)

Engaging and Motivating Providers

Another crucial step in pathway implementation was engaging and motivating providers. This included overcoming inertia to practice change, facilitating multidisciplinary collaboration, and handling conflicts regarding practice changes. Participants discussed the excitement of participating in a national collaborative as especially motivating to help drive engagement and overcome barriers to change, particularly the ability to compare local hospital performance to national peers.

“I think everyone is a little competitive. So I think that when we see how we compare to other institutions—both our group and the ER—I think it also adds a little oomph…I think for our nurses too; we’re able to say, ‘[look how we compare to] most of the other hospitals.’ I think that helps.” (Physician leader, Hospital B)

Multidisciplinary collaboration across a wide variety of frontline pediatric and nonpediatric providers was key to understanding current workflows and identifying needed modifications for pathway implementation:

“I do think clearly our biggest obstacles are the fact that we have adult ED providers. We have the opportunity on the inpatient side [with nursing and respiratory therapy], who really do awesome with pediatric changes, to take our wins where we can and make the changes with the ED. In the ED we have an RN educator. She’s very on board with doing the respiratory scoring and getting this whole thing started.” (Physician leader, Hospital L)

Intentional communication and leadership skills also played key roles in engaging hesitant providers and handling conflict:

“Just sitting and talking with our respiratory therapist about the ability to provide this type of service or support and seeing what their reservations have been, at least it’s open to conversation so that we could provide these types of therapies in the future and we’re able to see like what people’s concerns are. I think just basically increasing familiarity with not only these processes, but different types of therapy will hopefully in the future help us provide better care to our patients.” (Physician leader, Hospital Q)

Addressing Organizational and Resource Limitations

Participants recognized organizational and resource limitations, some of which may be unique to community hospitals that prioritize resources for adult care. The limitations described included EMR staff support, healthcare provider staffing/capacity, navigating IRBs, and addressing administrative processes. Competing demands for information technology staff support and lack of prioritization of pediatric-specific initiatives often hindered efforts to modify the EMR.

“Resource wise, we are hoping to implement an order set in our Epic EMR, [but] finding the availability from the Epic team may be a challenge.” (Physician Leader, Hospital A)

Participants also reported that limited staff capacity (eg, nursing, RT) hindered pathway implementation efforts. This limited capacity hindered workflow changes and limited the time available for education and training on pathways:

“[Respiratory scoring for asthma is] an added responsibility for the [nursing] staff and we don’t have patient technicians. So they’re doing everything from changing the sheets to bringing water to all of the medical patients. So, that I think may be a barrier.” (Physician leader, Hospital B)

Across sites, navigating the IRB posed various challenges. Some sites were required to obtain approval from regional IRBs, which did not have resources to devote to pediatric projects. Other sites did not have IRBs at all, but instead required separate approvals for the project from hospital leadership or other entities:

“On the IRB, I contacted the manager of the IRB and she’s said, ‘No, it’s not an IRB project,’ but she sent it to another director for review, and it took forever to be able to get a data agreement with [the local university hospital] so that we can pull the data. I just couldn’t believe it took months to get done.” (Physician Leader, Hospital K)

Finally, administrative barriers such as addressing formulary changes in the context of adult-focused settings were challenging. For example, at one hospital, metered dose inhalers (MDIs) were not used for adult patients, and the hospital administration was resistant to incorporate their use into practice for pediatric patients due to the cost of such changes.

“The [general hospital] didn’t have MDI’s anymore because of cost reasons, and when we started the pediatric work, we really made it a point to get the MDI’s for pediatric patients back in the formulary.” (Physician leader, Hospital A)

Devising Implementation Solutions with Practice Facilitators

Participants often devised pathway implementation solutions with facilitators in-the-moment during meetings. This problem-solving included figuring out work-arounds, proactive coaching by external facilitators, and just-in-time solution building. Furthermore, in meetings that included more than one project leader, leaders would often work with each other to devise solutions. Meetings provided forums that stimulated identification of implementation barriers, brainstorming, and subsequently solution building.

Physician leader: I’m wondering if we could, as an interim solution, try out an algorithm on paper, I don’t know if that’s allowed, until we get Epic approval. Do you know?

Nurse Leader: You mean having an algorithm posted in triage? Yeah, I don’t see why not. (Hospital A)

Next, problem solving was often driven by the facilitator’s experience and knowledge, drawn from their interactions with other collaborative sites or their own prior experiences with asthma, QI, or pathway implementation. The facilitators brought an outside perspective, not bound by that particular hospital’s local culture or structural intricacies. This proactive coaching spurred the identification of creative, yet practical solutions:

Project Leader: We’re still trying to get all our templates [for the EMR]…because [currently they are] all adult templates.

Facilitator: If you’re making templates right now, could you also add the three asterisks? Like smoking or exposure to second hand tobacco smoke or marijuana…then have the three asterisks there and then “Referral made?***”. That would force people to document in a certain place in the template as well.Project Leader: That’s definitely something we could add right now. (Hospital O)

Check-in meetings with facilitators offered an opportunity to trouble shoot, brainstorm work-arounds, devise in-the-moment site-specific solutions to enable successful pathway implementation, and provide ongoing support throughout implementation.

Pathways can improve the quality of care for children with asthma.³¹ However, there is little evidence-based guidance on how to implement pathways and improve pediatric care in community hospitals,^17-20 where the majority of children are cared for nationally. This is the first study to our knowledge that details the key determinants of pediatric asthma pathway implementation in community hospital settings. We identified four key determinants of implementation that can help guide others in similar settings. These include building an implementation infrastructure, engaging and motivating multidisciplinary providers, addressing organizational and resource limitations, and using external facilitators to devise implementation solutions.

Existing frameworks such as the CFIR outline the potential determinants of implementation success but do not provide population- or setting-specific guidance.²⁷ There have been prior studies detailing pathway implementation for pediatric populations, but these studies did not focus on community hospitals.^32,33 Our findings align with these prior studies, which highlight the importance of identifying implementation champions, engaging and motivating multidisciplinary providers, establishing a QI infrastructure, and addressing organizational and resource limitations, such as EMR integration.^32,33 However, our study provides unique insights into issues that are important to successful pathway implementation in community hospitals, including engagement of adult-focused healthcare providers, reprioritization of resources toward the care of children, and the potentially critical role of external facilitators.

Our findings indicate that community hospitals seeking to improve care for children may particularly benefit from using external facilitators and/or partnering with external organizations. We found that external facilitators played a significant and proactive role in community hospitals’ efforts to improve care for children. Facilitators helped devise work-arounds and engaged in just-in-time solution building with local project leaders. For instance, facilitators helped develop strategies for training healthcare providers in performing new clinical tasks, building reminders of pathway recommendations into clinical workflows, and overcoming resource barriers. Thus, community hospitals may uniquely benefit from participation in national learning collaboratives, which often provide avenues for external facilitation.^25,34,35 National networks, such as the VIP network, lead national learning collaboratives that provide external facilitation as well as other resources (eg, educational materials, data analysis support) to community hospitals seeking to improve pediatric care.²⁴ Previous work by McDaniel et al. identified that intentional partnerships between children’s and community hospitals can also potentially provide access to resources for education and training in pediatric care and support in navigating organizational and resource challenges.²²

Our results characterize the key determinants of pediatric asthma pathway implementation using a national sample of community hospitals that were diverse in geography, size, and structure. This imparts greater transferability of our findings. We also used strategies to promote the rigor of our findings, including triangulation and reflexivity. However, our study has several limitations. First, we analyzed only the meetings that occurred during the early months of pathway implementation. As such, we did not capture any key determinants that may have arisen later in implementation. However, process analyses of implementation indicate that the majority of implementation efforts occurred within these first three to four months.³⁶ Second, we did not elicit input from hospital administration or leadership. The lack of administrative/leadership input probably affected the CFIR themes we found, as no themes from the outer setting were elicited. However, the goal of our study was to characterize the experiences of those leading implementation efforts, and focusing on these leaders allows our work to better guide those doing similar work in the future. Third, we used CFIR to guide the development of our interview guide and as a reference during analysis, which may have skewed our findings to preferentially reflect CFIR constructs. However, our overall analysis was grounded in the primary data and we employed reflexivity during all stages of our analysis. In addition, having the facilitators conduct the qualitative interviews may have biased our findings toward the perspectives of the facilitators; however, the facilitators represented quite diverse clinical and QI backgrounds. Finally, our findings do not necessarily correlate with improvements in clinical outcomes. As such, they are not meant to serve as explicit recommendations for improving patient outcomes, but rather as a characterization of the context, processes, and experiences of implementing pathways in the community setting to inform others doing this important work.

CONCLUSIONS

We identified the key determinants of pediatric asthma pathway implementation in community hospitals, which may help inform QI efforts in these settings. We also identified organizational and resource limitations that are probably unique to these adult-focused hospitals. Participating in national learning collaboratives and/or working with facilitators may support pathway implementation and improved quality of care for children with asthma in community hospitals.

Future work should seek to correlate these and other determinants of pathway implementation with health outcomes for hospitalized children, as well as integrate broader and more diverse samples of community hospitals.

Despite the widespread availability of evidence-based guidelines,¹ there is inappropriate variation in the care and outcomes for children with asthma in both the emergency department (ED) and the inpatient setting.^2-6 Operational versions of evidence-based guidelines known as “pathways” have been shown to improve adoption of evidence-based guidelines, quality of care, and health outcomes for children with asthma.^7-14 However, little is known about how to successfully implement pathways outside of free-standing children’s hospitals.^15-19

The majority of children with asthma in the United States are cared for in community hospitals, which provide services for both adults and children.²⁰ However, prior studies of pediatric asthma pathways have largely excluded community hospitals. These studies primarily focused on determining clinical effectiveness, rather than detailing the implementation process. These approaches have left critical gaps that hinder our ability to implement pathways and improve care in community hospitals, which have unique barriers and less resources.^21,22 Therefore, understanding the process of pathway implementation in community hospitals is critical to improving care for children.²² Our objective was to identify the key determinants of successful pediatric asthma pathway implementation using a national sample of community hospitals. This knowledge can guide hospital leaders and healthcare providers in efforts to improve pediatric care and outcomes in these settings.

Study Setting, Design, and Population

In Fall 2017, the Value in Inpatient Pediatrics (VIP) network launched PIPA, Pathways to Improving Pediatric Asthma care.²³ The VIP network, a part of the American Academy of Pediatrics (AAP), aims to improve the value of care delivered to any pediatric patient in a hospital bed, from rural to free-standing children’s hospitals.²⁴ PIPA used a learning collaborative model²⁵ and recruited local project leaders (physicians, nurses, respiratory therapists (RT), and pharmacists) from 89 hospitals around the country. PIPA provided these hospital teams with asthma pathways and several resources for implementation support, including educational meetings, quality improvement (QI) training, audit and feedback, and facilitation. Facilitation is a process of interactive problem-solving and support that occurs in the context of a supportive interpersonal relationship and a recognized need for improvement.²⁶ A facilitator, or a “coach”, is an external expert who provides project mentorship and assists the process of making meaningful changes to improve patient care.²⁶ Facilitation was provided by external consultants with QI expertise.

For this qualitative study, facilitators conducted semi-structured interviews with a convenience sample of project leaders from community hospitals participating in PIPA, with some interviews including multiple project leaders (eg, nursing, inpatient, and Emergency Department [ED] leaders). Verbal consent was obtained from all participants. No incentives were provided. This study was approved by the AAP institutional review board.

Data Collection

We used the constructs described in the Consolidated Framework for Implementation Research (CFIR)²⁷ and adapted those salient to pediatric asthma pathways to develop an interview guide that was used with all participants (Appendix 1). The CFIR offers an overarching typology to understand what works where and why across five major domains that influence implementation: intervention characteristics, inner setting (hospital), outer setting (economic, political, and social context of the hospital), characteristics of the individuals involved, and the process of implementation. Data were collected across these domains to inform our analysis of the key determinants of pediatric asthma pathway implementation in community hospitals.

Interviews were conducted by phone from December 2017 to April 2018 (first four months of pathway implementation). Interviews lasted 30-60 minutes and were recorded and transcribed verbatim. Transcripts were edited for accuracy using the audio recordings. As data collection occurred concurrently with analysis, the interview guide was iteratively revised to reflect new insights and patterns that emerged from our analysis. All sites were anonymized in the data analysis. New interviews were coded until thematic saturation was reached.

Analysis

We conducted an inductive thematic analysis using the CFIR as our conceptual framework.^28,29 Four investigators (CM, MJ, ES, and SK) performed the initial open coding of the data. Investigators met twice during the open coding process to develop and then finalize a codebook of standard definitions for codes. This codebook facilitated coding consistency through the remainder of the analytic process. Two investigators (CM and MJ) then independently read and coded all data to ensure intercoder reliability. During this process, CM and MJ met every two weeks to compare coding consistency, resolve discrepancies, and discuss preliminary findings. When the coding was complete, all investigators met to explore and develop themes that encompassed related common codes.

The CFIR was used at two stages of the study: (1) developing the interview guide and (2) cross-checking for any potentially important codes that were missing/needed to be explored further. Thus, the investigators maintained an inductive approach grounded in the data. To assure study rigor, we employed investigator triangulation (use of multiple investigators and participants from multiple clinical roles) and reflexivity (ongoing critique and critical reflection of the individual biases of the investigators).³⁰ Coding was performed using Dedoose (version 7.0.23; Los Angeles, California).

A total of 34 community hospitals completed the PIPA project, of which the project leaders of 25 hospitals connected with the facilitators and were approached to participate; 18 (72%) hospitals’ project leaders participated in the study. We analyzed 18 interviews conducted between facilitators and project leaders, which included a total of 32 project leaders (one to five leaders per interview). The hospitals represented were diverse in geographic location and size (range 4-50 pediatric beds per hospital), and the majority of sites (78%) supported the trainees (Table 1).

We identified four overarching themes that described the key determinants of pathway implementation in community hospitals. These themes are presented in order of their frequency of occurrence in the data. They included (1) building an implementation infrastructure, (2) engaging and motivating providers, (3) addressing organizational and resource limitations, and (4) devising implementation solutions with facilitators. Descriptions and exemplary quotations for each theme are provided in Table 2 and Appendix Figure 1.

Building an Implementation Infrastructure

“So, I’m going to sit down with the primary nursing staff and the other four physicians in the group to go over the expectations…We’re not going to have the actual EMR [changes] and we’re not going to have the nursing documentation field built right away but [we want to] make sure that people are documenting the respiratory score in their generic nursing note so that the information is easily accessible.” (Physician leader, Hospital G)

Participants also described the need to deliver education on the evidence supporting changes in practice and skills training specific to pediatric asthma care:

“Once we realized that we were going to be doing this pathway, we started training our nurses on the inpatient side on [pediatric respiratory scoring].” (Nursing leader, Hospital P)

In addition, pathway implementation required modification of clinical workflows via changes to hospital policies or guidelines, electronic medical records (EMR), and/or the physical environment (eg, placing supplies in proximity to care delivery):

“I think it can help if we could get an order set or a nursing protocol where asthmatics over a certain severity can just get steroids in triage.” (Physician leader, Hospital A)

Engaging and Motivating Providers

Another crucial step in pathway implementation was engaging and motivating providers. This included overcoming inertia to practice change, facilitating multidisciplinary collaboration, and handling conflicts regarding practice changes. Participants discussed the excitement of participating in a national collaborative as especially motivating to help drive engagement and overcome barriers to change, particularly the ability to compare local hospital performance to national peers.

“I think everyone is a little competitive. So I think that when we see how we compare to other institutions—both our group and the ER—I think it also adds a little oomph…I think for our nurses too; we’re able to say, ‘[look how we compare to] most of the other hospitals.’ I think that helps.” (Physician leader, Hospital B)

Multidisciplinary collaboration across a wide variety of frontline pediatric and nonpediatric providers was key to understanding current workflows and identifying needed modifications for pathway implementation:

“I do think clearly our biggest obstacles are the fact that we have adult ED providers. We have the opportunity on the inpatient side [with nursing and respiratory therapy], who really do awesome with pediatric changes, to take our wins where we can and make the changes with the ED. In the ED we have an RN educator. She’s very on board with doing the respiratory scoring and getting this whole thing started.” (Physician leader, Hospital L)

Intentional communication and leadership skills also played key roles in engaging hesitant providers and handling conflict:

“Just sitting and talking with our respiratory therapist about the ability to provide this type of service or support and seeing what their reservations have been, at least it’s open to conversation so that we could provide these types of therapies in the future and we’re able to see like what people’s concerns are. I think just basically increasing familiarity with not only these processes, but different types of therapy will hopefully in the future help us provide better care to our patients.” (Physician leader, Hospital Q)

Addressing Organizational and Resource Limitations

Participants recognized organizational and resource limitations, some of which may be unique to community hospitals that prioritize resources for adult care. The limitations described included EMR staff support, healthcare provider staffing/capacity, navigating IRBs, and addressing administrative processes. Competing demands for information technology staff support and lack of prioritization of pediatric-specific initiatives often hindered efforts to modify the EMR.

“Resource wise, we are hoping to implement an order set in our Epic EMR, [but] finding the availability from the Epic team may be a challenge.” (Physician Leader, Hospital A)

Participants also reported that limited staff capacity (eg, nursing, RT) hindered pathway implementation efforts. This limited capacity hindered workflow changes and limited the time available for education and training on pathways:

“[Respiratory scoring for asthma is] an added responsibility for the [nursing] staff and we don’t have patient technicians. So they’re doing everything from changing the sheets to bringing water to all of the medical patients. So, that I think may be a barrier.” (Physician leader, Hospital B)

Across sites, navigating the IRB posed various challenges. Some sites were required to obtain approval from regional IRBs, which did not have resources to devote to pediatric projects. Other sites did not have IRBs at all, but instead required separate approvals for the project from hospital leadership or other entities:

“On the IRB, I contacted the manager of the IRB and she’s said, ‘No, it’s not an IRB project,’ but she sent it to another director for review, and it took forever to be able to get a data agreement with [the local university hospital] so that we can pull the data. I just couldn’t believe it took months to get done.” (Physician Leader, Hospital K)

Finally, administrative barriers such as addressing formulary changes in the context of adult-focused settings were challenging. For example, at one hospital, metered dose inhalers (MDIs) were not used for adult patients, and the hospital administration was resistant to incorporate their use into practice for pediatric patients due to the cost of such changes.

“The [general hospital] didn’t have MDI’s anymore because of cost reasons, and when we started the pediatric work, we really made it a point to get the MDI’s for pediatric patients back in the formulary.” (Physician leader, Hospital A)

Devising Implementation Solutions with Practice Facilitators

Participants often devised pathway implementation solutions with facilitators in-the-moment during meetings. This problem-solving included figuring out work-arounds, proactive coaching by external facilitators, and just-in-time solution building. Furthermore, in meetings that included more than one project leader, leaders would often work with each other to devise solutions. Meetings provided forums that stimulated identification of implementation barriers, brainstorming, and subsequently solution building.

Physician leader: I’m wondering if we could, as an interim solution, try out an algorithm on paper, I don’t know if that’s allowed, until we get Epic approval. Do you know?

Nurse Leader: You mean having an algorithm posted in triage? Yeah, I don’t see why not. (Hospital A)

Next, problem solving was often driven by the facilitator’s experience and knowledge, drawn from their interactions with other collaborative sites or their own prior experiences with asthma, QI, or pathway implementation. The facilitators brought an outside perspective, not bound by that particular hospital’s local culture or structural intricacies. This proactive coaching spurred the identification of creative, yet practical solutions:

Project Leader: We’re still trying to get all our templates [for the EMR]…because [currently they are] all adult templates.

Facilitator: If you’re making templates right now, could you also add the three asterisks? Like smoking or exposure to second hand tobacco smoke or marijuana…then have the three asterisks there and then “Referral made?***”. That would force people to document in a certain place in the template as well.Project Leader: That’s definitely something we could add right now. (Hospital O)

Check-in meetings with facilitators offered an opportunity to trouble shoot, brainstorm work-arounds, devise in-the-moment site-specific solutions to enable successful pathway implementation, and provide ongoing support throughout implementation.

Pathways can improve the quality of care for children with asthma.³¹ However, there is little evidence-based guidance on how to implement pathways and improve pediatric care in community hospitals,^17-20 where the majority of children are cared for nationally. This is the first study to our knowledge that details the key determinants of pediatric asthma pathway implementation in community hospital settings. We identified four key determinants of implementation that can help guide others in similar settings. These include building an implementation infrastructure, engaging and motivating multidisciplinary providers, addressing organizational and resource limitations, and using external facilitators to devise implementation solutions.

Existing frameworks such as the CFIR outline the potential determinants of implementation success but do not provide population- or setting-specific guidance.²⁷ There have been prior studies detailing pathway implementation for pediatric populations, but these studies did not focus on community hospitals.^32,33 Our findings align with these prior studies, which highlight the importance of identifying implementation champions, engaging and motivating multidisciplinary providers, establishing a QI infrastructure, and addressing organizational and resource limitations, such as EMR integration.^32,33 However, our study provides unique insights into issues that are important to successful pathway implementation in community hospitals, including engagement of adult-focused healthcare providers, reprioritization of resources toward the care of children, and the potentially critical role of external facilitators.

Our findings indicate that community hospitals seeking to improve care for children may particularly benefit from using external facilitators and/or partnering with external organizations. We found that external facilitators played a significant and proactive role in community hospitals’ efforts to improve care for children. Facilitators helped devise work-arounds and engaged in just-in-time solution building with local project leaders. For instance, facilitators helped develop strategies for training healthcare providers in performing new clinical tasks, building reminders of pathway recommendations into clinical workflows, and overcoming resource barriers. Thus, community hospitals may uniquely benefit from participation in national learning collaboratives, which often provide avenues for external facilitation.^25,34,35 National networks, such as the VIP network, lead national learning collaboratives that provide external facilitation as well as other resources (eg, educational materials, data analysis support) to community hospitals seeking to improve pediatric care.²⁴ Previous work by McDaniel et al. identified that intentional partnerships between children’s and community hospitals can also potentially provide access to resources for education and training in pediatric care and support in navigating organizational and resource challenges.²²

Our results characterize the key determinants of pediatric asthma pathway implementation using a national sample of community hospitals that were diverse in geography, size, and structure. This imparts greater transferability of our findings. We also used strategies to promote the rigor of our findings, including triangulation and reflexivity. However, our study has several limitations. First, we analyzed only the meetings that occurred during the early months of pathway implementation. As such, we did not capture any key determinants that may have arisen later in implementation. However, process analyses of implementation indicate that the majority of implementation efforts occurred within these first three to four months.³⁶ Second, we did not elicit input from hospital administration or leadership. The lack of administrative/leadership input probably affected the CFIR themes we found, as no themes from the outer setting were elicited. However, the goal of our study was to characterize the experiences of those leading implementation efforts, and focusing on these leaders allows our work to better guide those doing similar work in the future. Third, we used CFIR to guide the development of our interview guide and as a reference during analysis, which may have skewed our findings to preferentially reflect CFIR constructs. However, our overall analysis was grounded in the primary data and we employed reflexivity during all stages of our analysis. In addition, having the facilitators conduct the qualitative interviews may have biased our findings toward the perspectives of the facilitators; however, the facilitators represented quite diverse clinical and QI backgrounds. Finally, our findings do not necessarily correlate with improvements in clinical outcomes. As such, they are not meant to serve as explicit recommendations for improving patient outcomes, but rather as a characterization of the context, processes, and experiences of implementing pathways in the community setting to inform others doing this important work.

CONCLUSIONS

We identified the key determinants of pediatric asthma pathway implementation in community hospitals, which may help inform QI efforts in these settings. We also identified organizational and resource limitations that are probably unique to these adult-focused hospitals. Participating in national learning collaboratives and/or working with facilitators may support pathway implementation and improved quality of care for children with asthma in community hospitals.

Future work should seek to correlate these and other determinants of pathway implementation with health outcomes for hospitalized children, as well as integrate broader and more diverse samples of community hospitals.

Despite the widespread availability of evidence-based guidelines,¹ there is inappropriate variation in the care and outcomes for children with asthma in both the emergency department (ED) and the inpatient setting.^2-6 Operational versions of evidence-based guidelines known as “pathways” have been shown to improve adoption of evidence-based guidelines, quality of care, and health outcomes for children with asthma.^7-14 However, little is known about how to successfully implement pathways outside of free-standing children’s hospitals.^15-19

The majority of children with asthma in the United States are cared for in community hospitals, which provide services for both adults and children.²⁰ However, prior studies of pediatric asthma pathways have largely excluded community hospitals. These studies primarily focused on determining clinical effectiveness, rather than detailing the implementation process. These approaches have left critical gaps that hinder our ability to implement pathways and improve care in community hospitals, which have unique barriers and less resources.^21,22 Therefore, understanding the process of pathway implementation in community hospitals is critical to improving care for children.²² Our objective was to identify the key determinants of successful pediatric asthma pathway implementation using a national sample of community hospitals. This knowledge can guide hospital leaders and healthcare providers in efforts to improve pediatric care and outcomes in these settings.

Study Setting, Design, and Population

In Fall 2017, the Value in Inpatient Pediatrics (VIP) network launched PIPA, Pathways to Improving Pediatric Asthma care.²³ The VIP network, a part of the American Academy of Pediatrics (AAP), aims to improve the value of care delivered to any pediatric patient in a hospital bed, from rural to free-standing children’s hospitals.²⁴ PIPA used a learning collaborative model²⁵ and recruited local project leaders (physicians, nurses, respiratory therapists (RT), and pharmacists) from 89 hospitals around the country. PIPA provided these hospital teams with asthma pathways and several resources for implementation support, including educational meetings, quality improvement (QI) training, audit and feedback, and facilitation. Facilitation is a process of interactive problem-solving and support that occurs in the context of a supportive interpersonal relationship and a recognized need for improvement.²⁶ A facilitator, or a “coach”, is an external expert who provides project mentorship and assists the process of making meaningful changes to improve patient care.²⁶ Facilitation was provided by external consultants with QI expertise.

For this qualitative study, facilitators conducted semi-structured interviews with a convenience sample of project leaders from community hospitals participating in PIPA, with some interviews including multiple project leaders (eg, nursing, inpatient, and Emergency Department [ED] leaders). Verbal consent was obtained from all participants. No incentives were provided. This study was approved by the AAP institutional review board.

Data Collection

We used the constructs described in the Consolidated Framework for Implementation Research (CFIR)²⁷ and adapted those salient to pediatric asthma pathways to develop an interview guide that was used with all participants (Appendix 1). The CFIR offers an overarching typology to understand what works where and why across five major domains that influence implementation: intervention characteristics, inner setting (hospital), outer setting (economic, political, and social context of the hospital), characteristics of the individuals involved, and the process of implementation. Data were collected across these domains to inform our analysis of the key determinants of pediatric asthma pathway implementation in community hospitals.

Interviews were conducted by phone from December 2017 to April 2018 (first four months of pathway implementation). Interviews lasted 30-60 minutes and were recorded and transcribed verbatim. Transcripts were edited for accuracy using the audio recordings. As data collection occurred concurrently with analysis, the interview guide was iteratively revised to reflect new insights and patterns that emerged from our analysis. All sites were anonymized in the data analysis. New interviews were coded until thematic saturation was reached.

Analysis

We conducted an inductive thematic analysis using the CFIR as our conceptual framework.^28,29 Four investigators (CM, MJ, ES, and SK) performed the initial open coding of the data. Investigators met twice during the open coding process to develop and then finalize a codebook of standard definitions for codes. This codebook facilitated coding consistency through the remainder of the analytic process. Two investigators (CM and MJ) then independently read and coded all data to ensure intercoder reliability. During this process, CM and MJ met every two weeks to compare coding consistency, resolve discrepancies, and discuss preliminary findings. When the coding was complete, all investigators met to explore and develop themes that encompassed related common codes.

The CFIR was used at two stages of the study: (1) developing the interview guide and (2) cross-checking for any potentially important codes that were missing/needed to be explored further. Thus, the investigators maintained an inductive approach grounded in the data. To assure study rigor, we employed investigator triangulation (use of multiple investigators and participants from multiple clinical roles) and reflexivity (ongoing critique and critical reflection of the individual biases of the investigators).³⁰ Coding was performed using Dedoose (version 7.0.23; Los Angeles, California).

A total of 34 community hospitals completed the PIPA project, of which the project leaders of 25 hospitals connected with the facilitators and were approached to participate; 18 (72%) hospitals’ project leaders participated in the study. We analyzed 18 interviews conducted between facilitators and project leaders, which included a total of 32 project leaders (one to five leaders per interview). The hospitals represented were diverse in geographic location and size (range 4-50 pediatric beds per hospital), and the majority of sites (78%) supported the trainees (Table 1).

We identified four overarching themes that described the key determinants of pathway implementation in community hospitals. These themes are presented in order of their frequency of occurrence in the data. They included (1) building an implementation infrastructure, (2) engaging and motivating providers, (3) addressing organizational and resource limitations, and (4) devising implementation solutions with facilitators. Descriptions and exemplary quotations for each theme are provided in Table 2 and Appendix Figure 1.

Building an Implementation Infrastructure

“So, I’m going to sit down with the primary nursing staff and the other four physicians in the group to go over the expectations…We’re not going to have the actual EMR [changes] and we’re not going to have the nursing documentation field built right away but [we want to] make sure that people are documenting the respiratory score in their generic nursing note so that the information is easily accessible.” (Physician leader, Hospital G)

Participants also described the need to deliver education on the evidence supporting changes in practice and skills training specific to pediatric asthma care:

“Once we realized that we were going to be doing this pathway, we started training our nurses on the inpatient side on [pediatric respiratory scoring].” (Nursing leader, Hospital P)

In addition, pathway implementation required modification of clinical workflows via changes to hospital policies or guidelines, electronic medical records (EMR), and/or the physical environment (eg, placing supplies in proximity to care delivery):

“I think it can help if we could get an order set or a nursing protocol where asthmatics over a certain severity can just get steroids in triage.” (Physician leader, Hospital A)

Engaging and Motivating Providers

Another crucial step in pathway implementation was engaging and motivating providers. This included overcoming inertia to practice change, facilitating multidisciplinary collaboration, and handling conflicts regarding practice changes. Participants discussed the excitement of participating in a national collaborative as especially motivating to help drive engagement and overcome barriers to change, particularly the ability to compare local hospital performance to national peers.

“I think everyone is a little competitive. So I think that when we see how we compare to other institutions—both our group and the ER—I think it also adds a little oomph…I think for our nurses too; we’re able to say, ‘[look how we compare to] most of the other hospitals.’ I think that helps.” (Physician leader, Hospital B)

Multidisciplinary collaboration across a wide variety of frontline pediatric and nonpediatric providers was key to understanding current workflows and identifying needed modifications for pathway implementation:

“I do think clearly our biggest obstacles are the fact that we have adult ED providers. We have the opportunity on the inpatient side [with nursing and respiratory therapy], who really do awesome with pediatric changes, to take our wins where we can and make the changes with the ED. In the ED we have an RN educator. She’s very on board with doing the respiratory scoring and getting this whole thing started.” (Physician leader, Hospital L)

Intentional communication and leadership skills also played key roles in engaging hesitant providers and handling conflict:

“Just sitting and talking with our respiratory therapist about the ability to provide this type of service or support and seeing what their reservations have been, at least it’s open to conversation so that we could provide these types of therapies in the future and we’re able to see like what people’s concerns are. I think just basically increasing familiarity with not only these processes, but different types of therapy will hopefully in the future help us provide better care to our patients.” (Physician leader, Hospital Q)

Addressing Organizational and Resource Limitations

Participants recognized organizational and resource limitations, some of which may be unique to community hospitals that prioritize resources for adult care. The limitations described included EMR staff support, healthcare provider staffing/capacity, navigating IRBs, and addressing administrative processes. Competing demands for information technology staff support and lack of prioritization of pediatric-specific initiatives often hindered efforts to modify the EMR.

“Resource wise, we are hoping to implement an order set in our Epic EMR, [but] finding the availability from the Epic team may be a challenge.” (Physician Leader, Hospital A)

Participants also reported that limited staff capacity (eg, nursing, RT) hindered pathway implementation efforts. This limited capacity hindered workflow changes and limited the time available for education and training on pathways:

“[Respiratory scoring for asthma is] an added responsibility for the [nursing] staff and we don’t have patient technicians. So they’re doing everything from changing the sheets to bringing water to all of the medical patients. So, that I think may be a barrier.” (Physician leader, Hospital B)

Across sites, navigating the IRB posed various challenges. Some sites were required to obtain approval from regional IRBs, which did not have resources to devote to pediatric projects. Other sites did not have IRBs at all, but instead required separate approvals for the project from hospital leadership or other entities:

“On the IRB, I contacted the manager of the IRB and she’s said, ‘No, it’s not an IRB project,’ but she sent it to another director for review, and it took forever to be able to get a data agreement with [the local university hospital] so that we can pull the data. I just couldn’t believe it took months to get done.” (Physician Leader, Hospital K)

Finally, administrative barriers such as addressing formulary changes in the context of adult-focused settings were challenging. For example, at one hospital, metered dose inhalers (MDIs) were not used for adult patients, and the hospital administration was resistant to incorporate their use into practice for pediatric patients due to the cost of such changes.

“The [general hospital] didn’t have MDI’s anymore because of cost reasons, and when we started the pediatric work, we really made it a point to get the MDI’s for pediatric patients back in the formulary.” (Physician leader, Hospital A)

Devising Implementation Solutions with Practice Facilitators

Participants often devised pathway implementation solutions with facilitators in-the-moment during meetings. This problem-solving included figuring out work-arounds, proactive coaching by external facilitators, and just-in-time solution building. Furthermore, in meetings that included more than one project leader, leaders would often work with each other to devise solutions. Meetings provided forums that stimulated identification of implementation barriers, brainstorming, and subsequently solution building.

Physician leader: I’m wondering if we could, as an interim solution, try out an algorithm on paper, I don’t know if that’s allowed, until we get Epic approval. Do you know?

Nurse Leader: You mean having an algorithm posted in triage? Yeah, I don’t see why not. (Hospital A)

Next, problem solving was often driven by the facilitator’s experience and knowledge, drawn from their interactions with other collaborative sites or their own prior experiences with asthma, QI, or pathway implementation. The facilitators brought an outside perspective, not bound by that particular hospital’s local culture or structural intricacies. This proactive coaching spurred the identification of creative, yet practical solutions:

Project Leader: We’re still trying to get all our templates [for the EMR]…because [currently they are] all adult templates.

Facilitator: If you’re making templates right now, could you also add the three asterisks? Like smoking or exposure to second hand tobacco smoke or marijuana…then have the three asterisks there and then “Referral made?***”. That would force people to document in a certain place in the template as well.Project Leader: That’s definitely something we could add right now. (Hospital O)

Check-in meetings with facilitators offered an opportunity to trouble shoot, brainstorm work-arounds, devise in-the-moment site-specific solutions to enable successful pathway implementation, and provide ongoing support throughout implementation.

Pathways can improve the quality of care for children with asthma.³¹ However, there is little evidence-based guidance on how to implement pathways and improve pediatric care in community hospitals,^17-20 where the majority of children are cared for nationally. This is the first study to our knowledge that details the key determinants of pediatric asthma pathway implementation in community hospital settings. We identified four key determinants of implementation that can help guide others in similar settings. These include building an implementation infrastructure, engaging and motivating multidisciplinary providers, addressing organizational and resource limitations, and using external facilitators to devise implementation solutions.

Existing frameworks such as the CFIR outline the potential determinants of implementation success but do not provide population- or setting-specific guidance.²⁷ There have been prior studies detailing pathway implementation for pediatric populations, but these studies did not focus on community hospitals.^32,33 Our findings align with these prior studies, which highlight the importance of identifying implementation champions, engaging and motivating multidisciplinary providers, establishing a QI infrastructure, and addressing organizational and resource limitations, such as EMR integration.^32,33 However, our study provides unique insights into issues that are important to successful pathway implementation in community hospitals, including engagement of adult-focused healthcare providers, reprioritization of resources toward the care of children, and the potentially critical role of external facilitators.

Our findings indicate that community hospitals seeking to improve care for children may particularly benefit from using external facilitators and/or partnering with external organizations. We found that external facilitators played a significant and proactive role in community hospitals’ efforts to improve care for children. Facilitators helped devise work-arounds and engaged in just-in-time solution building with local project leaders. For instance, facilitators helped develop strategies for training healthcare providers in performing new clinical tasks, building reminders of pathway recommendations into clinical workflows, and overcoming resource barriers. Thus, community hospitals may uniquely benefit from participation in national learning collaboratives, which often provide avenues for external facilitation.^25,34,35 National networks, such as the VIP network, lead national learning collaboratives that provide external facilitation as well as other resources (eg, educational materials, data analysis support) to community hospitals seeking to improve pediatric care.²⁴ Previous work by McDaniel et al. identified that intentional partnerships between children’s and community hospitals can also potentially provide access to resources for education and training in pediatric care and support in navigating organizational and resource challenges.²²

Our results characterize the key determinants of pediatric asthma pathway implementation using a national sample of community hospitals that were diverse in geography, size, and structure. This imparts greater transferability of our findings. We also used strategies to promote the rigor of our findings, including triangulation and reflexivity. However, our study has several limitations. First, we analyzed only the meetings that occurred during the early months of pathway implementation. As such, we did not capture any key determinants that may have arisen later in implementation. However, process analyses of implementation indicate that the majority of implementation efforts occurred within these first three to four months.³⁶ Second, we did not elicit input from hospital administration or leadership. The lack of administrative/leadership input probably affected the CFIR themes we found, as no themes from the outer setting were elicited. However, the goal of our study was to characterize the experiences of those leading implementation efforts, and focusing on these leaders allows our work to better guide those doing similar work in the future. Third, we used CFIR to guide the development of our interview guide and as a reference during analysis, which may have skewed our findings to preferentially reflect CFIR constructs. However, our overall analysis was grounded in the primary data and we employed reflexivity during all stages of our analysis. In addition, having the facilitators conduct the qualitative interviews may have biased our findings toward the perspectives of the facilitators; however, the facilitators represented quite diverse clinical and QI backgrounds. Finally, our findings do not necessarily correlate with improvements in clinical outcomes. As such, they are not meant to serve as explicit recommendations for improving patient outcomes, but rather as a characterization of the context, processes, and experiences of implementing pathways in the community setting to inform others doing this important work.

CONCLUSIONS

We identified the key determinants of pediatric asthma pathway implementation in community hospitals, which may help inform QI efforts in these settings. We also identified organizational and resource limitations that are probably unique to these adult-focused hospitals. Participating in national learning collaboratives and/or working with facilitators may support pathway implementation and improved quality of care for children with asthma in community hospitals.

Future work should seek to correlate these and other determinants of pathway implementation with health outcomes for hospitalized children, as well as integrate broader and more diverse samples of community hospitals.

Inspired by the ABIM Foundation’s Choosing Wisely^® campaign, the “Things We Do for No Reason™” (TWDFNR) series reviews practices that have become common parts of hospital care but may provide little value to our patients. Practices reviewed in the TWDFNR series do not represent clear-cut conclusions or clinical practice standards but are meant as a starting place for research and active discussions among hospitalists and patients. We invite you to be part of that discussion.

A 4-year-old previously healthy, fully immunized boy presented to the emergency department (ED) with three days of worsening cough, fever to 103^oF, dyspnea, and decreased oral intake. In the ED, he was febrile, temperature 102.7^oF, heart rate 115 beats/min, respiratory rate 30 breaths/min, and O₂ saturation 86%. Pertinent findings identified on examination included tachypnea, dry mucous membranes, and decreased breath sounds in the posterior right lung fields. Chest radiograph revealed a right lower lobe opacification concerning for community-acquired pneumonia (CAP). He was admitted to the hospital due to hypoxemia and dehydration. A blood culture was obtained, and treatment with ampicillin was initiated. The following morning, he was afebrile, clinically improved, and no longer hypoxemic, but the blood culture grew Gram-positive cocci. Another blood culture was performed, and he was switched to vancomycin. The next day, penicillin-susceptible Streptococcus pneumoniae was confirmed from the original culture, and he was discharged home on high-dose amoxicillin.

CAP is a prominent cause of childhood morbidity and among the most common causes for acute childhood hospitalizations in the United States, with 124,900 hospital stays documented in 2012.¹ In 2011, the Infectious Diseases Society of America (IDSA) released recommendations for pediatric CAP in immunocompetent children aged >3 months without chronic medical conditions. The recommendations clearly discourage blood cultures in the outpatient setting but are less direct in the inpatient setting. The recommendations state that providers should obtain blood cultures “in children requiring hospitalization for presumed bacterial CAP that is moderate to severe, particularly those with complicated pneumonia.”² The recommendation is graded as “strong”, though the IDSA acknowledged the “low” quality of supporting evidence. Although the organization provides a classification for “complicated pneumonia,” it does not define what constitutes mild versus moderate or severe pneumonia.

Without clear recommendations, decisions to obtain blood cultures for hospitalized children with CAP vary among providers and institutions, with the reported hospital-to-hospital variation being as large as 0%-78.7%.³ Some believe that any child hospitalized with CAP meets the definition of moderate to severe pneumonia and have implemented projects to increase blood culture acquisition for this population.⁴ The decision to err on the side of routinely obtaining a blood culture may come from providers’ prevalent worry of “missing” a diagnosis, desire to target antibacterial therapy, and assumption that it will provide additional information for patients lacking improvement.

Since the publication of the 2011 IDSA guidelines, new evidence has revealed a decreasing incidence of bacteremia in pediatric populations.⁵ Moreover, viruses were the most frequently identified pathogens in children hospitalized with CAP in a large study, which were isolated in 66% of patients, whereas typical bacteria (either alone or in combination with a virus) were identified in only 7% of cases.⁶ When blood cultures are obtained for pediatric CAP, the incidence of a true bacterial bloodstream pathogen is 1.4%-7% of patients in the United States in the conjugate vaccine era.^7-11 Given that the practice of obtaining blood cultures varies widely among hospitalized patients and that cultures are often obtained based on perceived severity of presentation,^8,9,12 the true incidence of bacteremia in children with CAP would likely be lower if blood cultures were performed in all patients.

Since the introduction of the first conjugated pneumococcal vaccine, the prevalence of penicillin resistance among pneumococcal isolates dramatically declined,¹³ though with geographic variability.¹⁴ Therefore, when we isolate pneumococcus strains, resistance prevalence requires that we alter treatment much less frequently in the majority of patients with CAP receiving IDSA-recommended ampicillin/amoxicillin.² In a large six-center, geographically dispersed retrospective cohort study, Neuman et al. reported a rate of true bacteremia of 2.53%; 82% of all pathogens and 92% of pneumococcal isolates were susceptible to penicillin. Therefore, the authors estimated that 667 children hospitalized with CAP would need blood cultures to identify one child requiring an antibiotic other than an aminopenicillin.⁹ Staphylococcus aureus was identified only in 1% (23/2,138) of patients in the EPIC cohort; the pathogen was identified via blood culture in only 26% (6/23) of these patients.¹⁵ Therefore, the concern about the possibility of S. aureus may be a common reason for physicians straying from IDSA-recommended therapy, but it is an uncommon cause of CAP and infrequently identified via blood culture.

Blood culture contaminants have been reported to approach the rate of true pathogens in some studies^8,9,11 and be equal or exceed the rates in others.^7,16 While awaiting bacterial speciation, antibiotic coverage is often broadened, even for contaminants,⁸ which can result in unnecessary exposure to nephrotoxic agents such as vancomycin, cause rare adverse events such as Stevens-Johnson syndrome, contribute to antibiotic resistance and unnecessary costs, and increase the length of stay and laboratory utilization.^17-19

Given the low penicillin resistance prevalence among pneumococcal isolates in several parts of the United States, blood cultures should be used to identify patients with nonpneumococcal CAP as these patients are more likely to require antibiotics other than penicillin or aminopenicillin. Children with complicated pneumonia are more likely to have nonpneumococcal etiologies than children with uncomplicated pneumonia.² Moreover, literature published since the IDSA guidelines continues to indicate that the incidence of bacteremia in complicated pneumonia is significantly higher than that in uncomplicated pneumonia (Table). This further supports the IDSA guideline recommendation for blood culture acquisition in children with complicated pneumonia.²

One difficulty in interpreting these data is that each publication used a different definition of “complicated” pneumonia, probably due to differences in data sources. Neuman et al. incorporated the narrowest definition of severe and complicated pneumonia as patients who were either admitted to an intensive care unit (ICU) or who underwent a pleural drainage procedure.⁹ Myers’ and Shah’s definitions were similar to each other but much broader than that of Neuman et al. Shah et al. included lung abscess/necrosis, parapneumonic effusion/empyema, or bronchopleural fistula.¹¹ Myers et al. included the same indications but qualified their pleural fluid effusions as “moderate-to-large” and any effusion that required pleural drainage procedure.⁸ Myers et al. also reported bacteremia in 75% of patients with metastatic complications, including osteomyelitis.⁸ These definitions of complicated pneumonia may at least partially explain the differences noted in the rates of bacteremia in complicated pneumonia, with the patients in the study of Myers et al. potentially representing the most severe cohort and with the highest rate of bacteremia^8,9 (Table).

These studies not only support the definition of complicated pneumonia put forward by the IDSA but also provide further information, though imperfect, on how to define “moderate to severe.” All the patients with bacteremia in the report of Heine et al. had complicated pneumonia and were described on chart review as either toxic-appearing or requiring ICU care.⁷ This, in addition to the inclusion of ICU care in the definition of complicated pneumonia of Neuman et al.,⁹ indicates that children with CAP requiring ICU care may be at higher risk of bacteremia. In fact, the British Thoracic Society guidelines do not recommend microbiological investigations of children with CAP, including blood culture, unless a child requires ICU care.²⁰

Given the low rate of bacteremia in CAP, the risk of blood culture contaminants, and the small likelihood that isolation of a pathogen alters treatment for children, we recommend not using hospital admission as the determining factor for blood culture acquisition. Instead, we recommend a more targeted approach. To achieve a higher rate of true-positive bacteremia in immunocompetent children with up-to-date vaccinations, we recommend acquiring a blood culture in children with complicated pneumonia, metastatic complications, or with ICU needs. By initiating the IDSA-recommended ampicillin/amoxicillin in the remaining hospitalized patients and acquiring blood cultures for the minority of patients who do not improve, we can increase the likelihood of isolating penicillin-resistant bacteria.

Attempting to balance the importance of identifying clinically important bacteremia for children hospitalized with CAP and the inherent risks of obtaining blood cultures for all hospitalized patients, Andrews et al. created and analyzed a cost-effectiveness model. The authors compared universal acquisition of blood cultures for hospitalized children with CAP versus a targeted approach with blood cultures obtained in patients with effusion or empyema, requiring ICU care, or who are immunosuppressed. Based on this model, a targeted approach could save more than $187 million annually, reduce the number of cultures needed to result in a meaningful change in antibiotic therapy for one patient from 122 to 42, and would “miss” only approximately one case of bacteremia resulting in treatment failure per 1,400 patients.¹⁷

• Do not obtain blood culture routinely for children aged >3 months hospitalized for uncomplicated CAP.
• Obtain a blood culture for the following hospitalized patients with CAP:

a. Patients with complicated CAP as defined by the IDSA, particularly those with empyema, abscess, or fistula, or metastatic complications of pneumonia (Table); or

b. Patients with CAP requiring ICU care²⁰ for the management of shock and/or advanced respiratory support.

c. Patients with CAP judged to need antibiotic treatment with an agent other than the IDSA-recommended ampicillin/penicillin (concern for pathogens other than penicillin-sensitive S. pneumonia, immunocompromised or under-immunized status, or inadequate clinical response to empiric ampicillin therapy).

Implementing a more targeted approach to blood culture acquisition for hospitalized children with CAP will hopefully increase the yield of true bacterial pathogens that alter management decisions. A targeted approach for the child in the opening vignette would have saved him from the pain of unnecessary phlebotomy (repeat culture), exposure to vancomycin as a nephrotoxic agent, and an additional hospital day.

Do you think this is a low-value practice? Is this truly a “Thing We Do for No Reason?^™” Share what you do in your practice and join in the conversation online by retweeting it on Twitter (#TWDFNR) and liking it on Facebook. We invite you to propose ideas for other “Things We Do for No Reason^™” topics by e-mailing TWDFNR@hospitalmedicine.org.

Inspired by the ABIM Foundation’s Choosing Wisely^® campaign, the “Things We Do for No Reason™” (TWDFNR) series reviews practices that have become common parts of hospital care but may provide little value to our patients. Practices reviewed in the TWDFNR series do not represent clear-cut conclusions or clinical practice standards but are meant as a starting place for research and active discussions among hospitalists and patients. We invite you to be part of that discussion.

A 4-year-old previously healthy, fully immunized boy presented to the emergency department (ED) with three days of worsening cough, fever to 103^oF, dyspnea, and decreased oral intake. In the ED, he was febrile, temperature 102.7^oF, heart rate 115 beats/min, respiratory rate 30 breaths/min, and O₂ saturation 86%. Pertinent findings identified on examination included tachypnea, dry mucous membranes, and decreased breath sounds in the posterior right lung fields. Chest radiograph revealed a right lower lobe opacification concerning for community-acquired pneumonia (CAP). He was admitted to the hospital due to hypoxemia and dehydration. A blood culture was obtained, and treatment with ampicillin was initiated. The following morning, he was afebrile, clinically improved, and no longer hypoxemic, but the blood culture grew Gram-positive cocci. Another blood culture was performed, and he was switched to vancomycin. The next day, penicillin-susceptible Streptococcus pneumoniae was confirmed from the original culture, and he was discharged home on high-dose amoxicillin.

CAP is a prominent cause of childhood morbidity and among the most common causes for acute childhood hospitalizations in the United States, with 124,900 hospital stays documented in 2012.¹ In 2011, the Infectious Diseases Society of America (IDSA) released recommendations for pediatric CAP in immunocompetent children aged >3 months without chronic medical conditions. The recommendations clearly discourage blood cultures in the outpatient setting but are less direct in the inpatient setting. The recommendations state that providers should obtain blood cultures “in children requiring hospitalization for presumed bacterial CAP that is moderate to severe, particularly those with complicated pneumonia.”² The recommendation is graded as “strong”, though the IDSA acknowledged the “low” quality of supporting evidence. Although the organization provides a classification for “complicated pneumonia,” it does not define what constitutes mild versus moderate or severe pneumonia.

Without clear recommendations, decisions to obtain blood cultures for hospitalized children with CAP vary among providers and institutions, with the reported hospital-to-hospital variation being as large as 0%-78.7%.³ Some believe that any child hospitalized with CAP meets the definition of moderate to severe pneumonia and have implemented projects to increase blood culture acquisition for this population.⁴ The decision to err on the side of routinely obtaining a blood culture may come from providers’ prevalent worry of “missing” a diagnosis, desire to target antibacterial therapy, and assumption that it will provide additional information for patients lacking improvement.

Since the publication of the 2011 IDSA guidelines, new evidence has revealed a decreasing incidence of bacteremia in pediatric populations.⁵ Moreover, viruses were the most frequently identified pathogens in children hospitalized with CAP in a large study, which were isolated in 66% of patients, whereas typical bacteria (either alone or in combination with a virus) were identified in only 7% of cases.⁶ When blood cultures are obtained for pediatric CAP, the incidence of a true bacterial bloodstream pathogen is 1.4%-7% of patients in the United States in the conjugate vaccine era.^7-11 Given that the practice of obtaining blood cultures varies widely among hospitalized patients and that cultures are often obtained based on perceived severity of presentation,^8,9,12 the true incidence of bacteremia in children with CAP would likely be lower if blood cultures were performed in all patients.

Since the introduction of the first conjugated pneumococcal vaccine, the prevalence of penicillin resistance among pneumococcal isolates dramatically declined,¹³ though with geographic variability.¹⁴ Therefore, when we isolate pneumococcus strains, resistance prevalence requires that we alter treatment much less frequently in the majority of patients with CAP receiving IDSA-recommended ampicillin/amoxicillin.² In a large six-center, geographically dispersed retrospective cohort study, Neuman et al. reported a rate of true bacteremia of 2.53%; 82% of all pathogens and 92% of pneumococcal isolates were susceptible to penicillin. Therefore, the authors estimated that 667 children hospitalized with CAP would need blood cultures to identify one child requiring an antibiotic other than an aminopenicillin.⁹ Staphylococcus aureus was identified only in 1% (23/2,138) of patients in the EPIC cohort; the pathogen was identified via blood culture in only 26% (6/23) of these patients.¹⁵ Therefore, the concern about the possibility of S. aureus may be a common reason for physicians straying from IDSA-recommended therapy, but it is an uncommon cause of CAP and infrequently identified via blood culture.

Blood culture contaminants have been reported to approach the rate of true pathogens in some studies^8,9,11 and be equal or exceed the rates in others.^7,16 While awaiting bacterial speciation, antibiotic coverage is often broadened, even for contaminants,⁸ which can result in unnecessary exposure to nephrotoxic agents such as vancomycin, cause rare adverse events such as Stevens-Johnson syndrome, contribute to antibiotic resistance and unnecessary costs, and increase the length of stay and laboratory utilization.^17-19

Given the low penicillin resistance prevalence among pneumococcal isolates in several parts of the United States, blood cultures should be used to identify patients with nonpneumococcal CAP as these patients are more likely to require antibiotics other than penicillin or aminopenicillin. Children with complicated pneumonia are more likely to have nonpneumococcal etiologies than children with uncomplicated pneumonia.² Moreover, literature published since the IDSA guidelines continues to indicate that the incidence of bacteremia in complicated pneumonia is significantly higher than that in uncomplicated pneumonia (Table). This further supports the IDSA guideline recommendation for blood culture acquisition in children with complicated pneumonia.²

One difficulty in interpreting these data is that each publication used a different definition of “complicated” pneumonia, probably due to differences in data sources. Neuman et al. incorporated the narrowest definition of severe and complicated pneumonia as patients who were either admitted to an intensive care unit (ICU) or who underwent a pleural drainage procedure.⁹ Myers’ and Shah’s definitions were similar to each other but much broader than that of Neuman et al. Shah et al. included lung abscess/necrosis, parapneumonic effusion/empyema, or bronchopleural fistula.¹¹ Myers et al. included the same indications but qualified their pleural fluid effusions as “moderate-to-large” and any effusion that required pleural drainage procedure.⁸ Myers et al. also reported bacteremia in 75% of patients with metastatic complications, including osteomyelitis.⁸ These definitions of complicated pneumonia may at least partially explain the differences noted in the rates of bacteremia in complicated pneumonia, with the patients in the study of Myers et al. potentially representing the most severe cohort and with the highest rate of bacteremia^8,9 (Table).

These studies not only support the definition of complicated pneumonia put forward by the IDSA but also provide further information, though imperfect, on how to define “moderate to severe.” All the patients with bacteremia in the report of Heine et al. had complicated pneumonia and were described on chart review as either toxic-appearing or requiring ICU care.⁷ This, in addition to the inclusion of ICU care in the definition of complicated pneumonia of Neuman et al.,⁹ indicates that children with CAP requiring ICU care may be at higher risk of bacteremia. In fact, the British Thoracic Society guidelines do not recommend microbiological investigations of children with CAP, including blood culture, unless a child requires ICU care.²⁰

Given the low rate of bacteremia in CAP, the risk of blood culture contaminants, and the small likelihood that isolation of a pathogen alters treatment for children, we recommend not using hospital admission as the determining factor for blood culture acquisition. Instead, we recommend a more targeted approach. To achieve a higher rate of true-positive bacteremia in immunocompetent children with up-to-date vaccinations, we recommend acquiring a blood culture in children with complicated pneumonia, metastatic complications, or with ICU needs. By initiating the IDSA-recommended ampicillin/amoxicillin in the remaining hospitalized patients and acquiring blood cultures for the minority of patients who do not improve, we can increase the likelihood of isolating penicillin-resistant bacteria.

Attempting to balance the importance of identifying clinically important bacteremia for children hospitalized with CAP and the inherent risks of obtaining blood cultures for all hospitalized patients, Andrews et al. created and analyzed a cost-effectiveness model. The authors compared universal acquisition of blood cultures for hospitalized children with CAP versus a targeted approach with blood cultures obtained in patients with effusion or empyema, requiring ICU care, or who are immunosuppressed. Based on this model, a targeted approach could save more than $187 million annually, reduce the number of cultures needed to result in a meaningful change in antibiotic therapy for one patient from 122 to 42, and would “miss” only approximately one case of bacteremia resulting in treatment failure per 1,400 patients.¹⁷

• Do not obtain blood culture routinely for children aged >3 months hospitalized for uncomplicated CAP.
• Obtain a blood culture for the following hospitalized patients with CAP:

a. Patients with complicated CAP as defined by the IDSA, particularly those with empyema, abscess, or fistula, or metastatic complications of pneumonia (Table); or

b. Patients with CAP requiring ICU care²⁰ for the management of shock and/or advanced respiratory support.

c. Patients with CAP judged to need antibiotic treatment with an agent other than the IDSA-recommended ampicillin/penicillin (concern for pathogens other than penicillin-sensitive S. pneumonia, immunocompromised or under-immunized status, or inadequate clinical response to empiric ampicillin therapy).

Implementing a more targeted approach to blood culture acquisition for hospitalized children with CAP will hopefully increase the yield of true bacterial pathogens that alter management decisions. A targeted approach for the child in the opening vignette would have saved him from the pain of unnecessary phlebotomy (repeat culture), exposure to vancomycin as a nephrotoxic agent, and an additional hospital day.

Do you think this is a low-value practice? Is this truly a “Thing We Do for No Reason?^™” Share what you do in your practice and join in the conversation online by retweeting it on Twitter (#TWDFNR) and liking it on Facebook. We invite you to propose ideas for other “Things We Do for No Reason^™” topics by e-mailing TWDFNR@hospitalmedicine.org.

Inspired by the ABIM Foundation’s Choosing Wisely^® campaign, the “Things We Do for No Reason™” (TWDFNR) series reviews practices that have become common parts of hospital care but may provide little value to our patients. Practices reviewed in the TWDFNR series do not represent clear-cut conclusions or clinical practice standards but are meant as a starting place for research and active discussions among hospitalists and patients. We invite you to be part of that discussion.

A 4-year-old previously healthy, fully immunized boy presented to the emergency department (ED) with three days of worsening cough, fever to 103^oF, dyspnea, and decreased oral intake. In the ED, he was febrile, temperature 102.7^oF, heart rate 115 beats/min, respiratory rate 30 breaths/min, and O₂ saturation 86%. Pertinent findings identified on examination included tachypnea, dry mucous membranes, and decreased breath sounds in the posterior right lung fields. Chest radiograph revealed a right lower lobe opacification concerning for community-acquired pneumonia (CAP). He was admitted to the hospital due to hypoxemia and dehydration. A blood culture was obtained, and treatment with ampicillin was initiated. The following morning, he was afebrile, clinically improved, and no longer hypoxemic, but the blood culture grew Gram-positive cocci. Another blood culture was performed, and he was switched to vancomycin. The next day, penicillin-susceptible Streptococcus pneumoniae was confirmed from the original culture, and he was discharged home on high-dose amoxicillin.

CAP is a prominent cause of childhood morbidity and among the most common causes for acute childhood hospitalizations in the United States, with 124,900 hospital stays documented in 2012.¹ In 2011, the Infectious Diseases Society of America (IDSA) released recommendations for pediatric CAP in immunocompetent children aged >3 months without chronic medical conditions. The recommendations clearly discourage blood cultures in the outpatient setting but are less direct in the inpatient setting. The recommendations state that providers should obtain blood cultures “in children requiring hospitalization for presumed bacterial CAP that is moderate to severe, particularly those with complicated pneumonia.”² The recommendation is graded as “strong”, though the IDSA acknowledged the “low” quality of supporting evidence. Although the organization provides a classification for “complicated pneumonia,” it does not define what constitutes mild versus moderate or severe pneumonia.

Without clear recommendations, decisions to obtain blood cultures for hospitalized children with CAP vary among providers and institutions, with the reported hospital-to-hospital variation being as large as 0%-78.7%.³ Some believe that any child hospitalized with CAP meets the definition of moderate to severe pneumonia and have implemented projects to increase blood culture acquisition for this population.⁴ The decision to err on the side of routinely obtaining a blood culture may come from providers’ prevalent worry of “missing” a diagnosis, desire to target antibacterial therapy, and assumption that it will provide additional information for patients lacking improvement.

Since the publication of the 2011 IDSA guidelines, new evidence has revealed a decreasing incidence of bacteremia in pediatric populations.⁵ Moreover, viruses were the most frequently identified pathogens in children hospitalized with CAP in a large study, which were isolated in 66% of patients, whereas typical bacteria (either alone or in combination with a virus) were identified in only 7% of cases.⁶ When blood cultures are obtained for pediatric CAP, the incidence of a true bacterial bloodstream pathogen is 1.4%-7% of patients in the United States in the conjugate vaccine era.^7-11 Given that the practice of obtaining blood cultures varies widely among hospitalized patients and that cultures are often obtained based on perceived severity of presentation,^8,9,12 the true incidence of bacteremia in children with CAP would likely be lower if blood cultures were performed in all patients.

Since the introduction of the first conjugated pneumococcal vaccine, the prevalence of penicillin resistance among pneumococcal isolates dramatically declined,¹³ though with geographic variability.¹⁴ Therefore, when we isolate pneumococcus strains, resistance prevalence requires that we alter treatment much less frequently in the majority of patients with CAP receiving IDSA-recommended ampicillin/amoxicillin.² In a large six-center, geographically dispersed retrospective cohort study, Neuman et al. reported a rate of true bacteremia of 2.53%; 82% of all pathogens and 92% of pneumococcal isolates were susceptible to penicillin. Therefore, the authors estimated that 667 children hospitalized with CAP would need blood cultures to identify one child requiring an antibiotic other than an aminopenicillin.⁹ Staphylococcus aureus was identified only in 1% (23/2,138) of patients in the EPIC cohort; the pathogen was identified via blood culture in only 26% (6/23) of these patients.¹⁵ Therefore, the concern about the possibility of S. aureus may be a common reason for physicians straying from IDSA-recommended therapy, but it is an uncommon cause of CAP and infrequently identified via blood culture.

Blood culture contaminants have been reported to approach the rate of true pathogens in some studies^8,9,11 and be equal or exceed the rates in others.^7,16 While awaiting bacterial speciation, antibiotic coverage is often broadened, even for contaminants,⁸ which can result in unnecessary exposure to nephrotoxic agents such as vancomycin, cause rare adverse events such as Stevens-Johnson syndrome, contribute to antibiotic resistance and unnecessary costs, and increase the length of stay and laboratory utilization.^17-19

Given the low penicillin resistance prevalence among pneumococcal isolates in several parts of the United States, blood cultures should be used to identify patients with nonpneumococcal CAP as these patients are more likely to require antibiotics other than penicillin or aminopenicillin. Children with complicated pneumonia are more likely to have nonpneumococcal etiologies than children with uncomplicated pneumonia.² Moreover, literature published since the IDSA guidelines continues to indicate that the incidence of bacteremia in complicated pneumonia is significantly higher than that in uncomplicated pneumonia (Table). This further supports the IDSA guideline recommendation for blood culture acquisition in children with complicated pneumonia.²

One difficulty in interpreting these data is that each publication used a different definition of “complicated” pneumonia, probably due to differences in data sources. Neuman et al. incorporated the narrowest definition of severe and complicated pneumonia as patients who were either admitted to an intensive care unit (ICU) or who underwent a pleural drainage procedure.⁹ Myers’ and Shah’s definitions were similar to each other but much broader than that of Neuman et al. Shah et al. included lung abscess/necrosis, parapneumonic effusion/empyema, or bronchopleural fistula.¹¹ Myers et al. included the same indications but qualified their pleural fluid effusions as “moderate-to-large” and any effusion that required pleural drainage procedure.⁸ Myers et al. also reported bacteremia in 75% of patients with metastatic complications, including osteomyelitis.⁸ These definitions of complicated pneumonia may at least partially explain the differences noted in the rates of bacteremia in complicated pneumonia, with the patients in the study of Myers et al. potentially representing the most severe cohort and with the highest rate of bacteremia^8,9 (Table).

These studies not only support the definition of complicated pneumonia put forward by the IDSA but also provide further information, though imperfect, on how to define “moderate to severe.” All the patients with bacteremia in the report of Heine et al. had complicated pneumonia and were described on chart review as either toxic-appearing or requiring ICU care.⁷ This, in addition to the inclusion of ICU care in the definition of complicated pneumonia of Neuman et al.,⁹ indicates that children with CAP requiring ICU care may be at higher risk of bacteremia. In fact, the British Thoracic Society guidelines do not recommend microbiological investigations of children with CAP, including blood culture, unless a child requires ICU care.²⁰

Given the low rate of bacteremia in CAP, the risk of blood culture contaminants, and the small likelihood that isolation of a pathogen alters treatment for children, we recommend not using hospital admission as the determining factor for blood culture acquisition. Instead, we recommend a more targeted approach. To achieve a higher rate of true-positive bacteremia in immunocompetent children with up-to-date vaccinations, we recommend acquiring a blood culture in children with complicated pneumonia, metastatic complications, or with ICU needs. By initiating the IDSA-recommended ampicillin/amoxicillin in the remaining hospitalized patients and acquiring blood cultures for the minority of patients who do not improve, we can increase the likelihood of isolating penicillin-resistant bacteria.

Attempting to balance the importance of identifying clinically important bacteremia for children hospitalized with CAP and the inherent risks of obtaining blood cultures for all hospitalized patients, Andrews et al. created and analyzed a cost-effectiveness model. The authors compared universal acquisition of blood cultures for hospitalized children with CAP versus a targeted approach with blood cultures obtained in patients with effusion or empyema, requiring ICU care, or who are immunosuppressed. Based on this model, a targeted approach could save more than $187 million annually, reduce the number of cultures needed to result in a meaningful change in antibiotic therapy for one patient from 122 to 42, and would “miss” only approximately one case of bacteremia resulting in treatment failure per 1,400 patients.¹⁷

• Do not obtain blood culture routinely for children aged >3 months hospitalized for uncomplicated CAP.
• Obtain a blood culture for the following hospitalized patients with CAP:

a. Patients with complicated CAP as defined by the IDSA, particularly those with empyema, abscess, or fistula, or metastatic complications of pneumonia (Table); or

b. Patients with CAP requiring ICU care²⁰ for the management of shock and/or advanced respiratory support.

c. Patients with CAP judged to need antibiotic treatment with an agent other than the IDSA-recommended ampicillin/penicillin (concern for pathogens other than penicillin-sensitive S. pneumonia, immunocompromised or under-immunized status, or inadequate clinical response to empiric ampicillin therapy).

Implementing a more targeted approach to blood culture acquisition for hospitalized children with CAP will hopefully increase the yield of true bacterial pathogens that alter management decisions. A targeted approach for the child in the opening vignette would have saved him from the pain of unnecessary phlebotomy (repeat culture), exposure to vancomycin as a nephrotoxic agent, and an additional hospital day.

Do you think this is a low-value practice? Is this truly a “Thing We Do for No Reason?^™” Share what you do in your practice and join in the conversation online by retweeting it on Twitter (#TWDFNR) and liking it on Facebook. We invite you to propose ideas for other “Things We Do for No Reason^™” topics by e-mailing TWDFNR@hospitalmedicine.org.

Clostridium difficile, now referred to as Clostridioides difficile (C. difficile), is the most commonly identified cause of healthcare-associated infection among adults in the United States.¹ Because C. difficile infection results in significant mortality and inpatient costs, its persistence threatens to undermine patient safety and the value of healthcare delivery.¹ A standardized, evidence-based approach to diagnosis and management is crucial. However, inconsistencies remain with regard to the appropriate threshold for testing, the type of diagnostic tests used, and treatment. Knowledge of these areas has progressed since the publication of the previous C. difficile guidelines in 2010. These guidelines contain 53 recommendations across 35 sections based on a systematic weighting of the strength of recommendation and quality of evidence using the Grading of Recommendations Assessment, Development, and Evaluation system. Herein, we have chosen to highlight five of these recommendations most relevant to hospitalists.

Recommendation 1. Patients with unexplained and new-onset ≥3 unformed stools within 24 hours are the preferred target population for testing for C. difficile infection (weak recommendation, very low quality of evidence). Do not perform repeat testing (within seven days) during the same episode of diarrhea and do not test stool from asymptomatic patients (strong recommendation, moderate quality of evidence).

In the recent past, healthcare facilities employed C. difficile tests with limited sensitivity, leading to frequent and repeat testing of hospitalized patients. Excess testing puts patients at risk for false positive results and unnecessary or prolonged treatment courses. Proper testing requires consideration of pretest probability, including analysis of the alternative causes of diarrhea. Duration of hospitalization and antibiotic exposure are the most significant modifiable risk factors for C. difficile infection in adult inpatients.² Laxative use within the previous 48 hours, enteral tube feeding, and underlying medical conditions, such as inflammatory bowel disease (IBD), are common causes of improper testing.³ This decision may be difficult, as some underlying causes of diarrhea, such as IBD and enteral tube feeding, also increase the risk of C. difficile infection.³ Laboratories can help by rejecting specimens that are not liquid or soft and employing a multistep algorithm using a combination of nucleic acid testing, antigen testing, and toxin detection to maximize sensitivity and specificity. Because recurrent C. difficile infection is relatively common, repeat testing is appropriate only for recurrence of symptoms following successful treatment and should focus on detection of C. difficile toxin because the persistence of the organism itself can occur after successful treatment.⁴

Recommendation 2. Either vancomycin (125 mg orally four times per day for 10 days) or fidaxomicin (200 mg twice daily for 10 days) is recommended over metronidazole for an initial episode of nonsevere or severe C. difficile infection (strong recommendation, high quality of evidence). For fulminant C. difficile infection, the regimen of choice is a vancomycin dosage of 500 mg orally four times per day (per rectum every six hours if with ileus) in addition to intravenous metronidazole (strong recommendation, moderate quality of evidence).

For several decades now, metronidazole has been the primary antibiotic agent for initial treatment of nonsevere C. difficile infection. Two recent randomized, placebo-controlled trials, however, have found oral vancomycin to be superior to metronidazole for producing a clinical cure and resolution of diarrhea without recurrence.^5,6 Oral vancomycin remains the treatment of choice for severe C. difficile infection. Fidaxomicin, a recently FDA-approved antibiotic, can also be used as initial treatment in place of oral vancomycin. One study found fidaxomicin to be superior to oral vancomycin for producing a sustained clinical response, that is, resolution of diarrhea at the end of treatment without recurrence 25 days later.⁷ Fulminant disease, which is characterized by hypotension or shock, ileus, or megacolon, requires a higher dose of oral vancomycin (or vancomycin enema if with ileus) in addition to intravenous metronidazole.

Recommendation 3. Treat a first recurrence of C. difficile infection with oral vancomycin as a tapered and pulsed regimen rather than a second standard 10-day course of vancomycin or metronidazole (weak recommendation, low quality of evidence).

Despite the improved treatment response with oral vancomycin, one in four patients will experience recurrence. For a first recurrence of C. difficile infection after a 10-day course of oral vancomycin, an extended taper or pulsed course of vancomycin should be attempted. Various regimens have been tried and found to be effective. For a second recurrence, providers can consider addition of rifaximin following oral vancomycin. Fecal microbiota transplantation is recommended for patients with multiple recurrences of C. difficile infection who have failed these antibiotic treatments.

Recommendation 4. Minimize the frequency and duration of high-risk antibiotic therapy (based on local epidemiology) and the number of antibiotic agents prescribed to reduce C. difficile infection risk (strong recommendation, moderate quality of evidence).

Antibiotic stewardship is a necessary component of any successful effort to reduce C. difficile infections. Antibiotic stewardship programs, which are now commonplace in US hospitals, largely rely on educational initiatives or committee-based order review. Hospitalists should take a structured approach emphasizing the four critical questions of antibiotic prescribing: Does this infection require antibiotics? Have I ordered appropriate cultures and the correct empiric therapy? Can I stop, narrow, or switch to oral agents? Finally, what duration of therapy is needed at discharge?⁸ Initial efforts should focus on the restriction of fluoroquinolones, clindamycin, and cephalosporins (except for surgical antibiotic prophylaxis) given their known risk to cause C. difficile infection.

Recommendation 5. Contact precautions should be maintained for at least 48 hours after diarrhea has resolved (weak recommendation, low quality of evidence).

Although C. difficile is undetectable in stool samples from most patients by the time diarrhea has resolved, skin and environmental contaminations remain high. No studies demonstrating a benefit to further extending contact precautions beyond 48 hours after resolution of diarrhea are yet available.

Methods in Preparing Guidelines

The guideline committee consisted of an interdisciplinary team of healthcare providers with extensive experience in the diagnosis, infection control, treatment, and management of C. difficile. The literature search accessed five different databases (Medline, Embase, Cochrane, Health Technology Assessment, and Database of Abstracts of Reviews and Effects), relevant journals, conference proceedings, and regulatory websites published over the search period of 2009-2016.

A major strength of these guidelines is the extensive work that went into their preparation. The committee reviewed over 14,000 pieces of literature and performed a detailed analysis of each one to determine the quality of evidence in support of each recommendation.

Sources of Potential Conflict of Interest or Bias

To reduce bias, the committee’s work was funded by Infectious Disease Society of America and Society for Healthcare Epidemiology of America. Some authors received funding for work outside of this guideline by companies that manufacture diagnostic assays, vancomycin, and fidaxomicin. These potential conflicts were listed at the end of the article.

Generalizability of the Guideline

Not all studies included in the guideline contain exclusively hospitalized patients, but much of the guideline content is applicable to hospitalized patients. Because C. difficile infection is such a widespread public health problem and these guidelines represent a significant update in knowledge since 2010, the specific recommendations highlighted in this review will impact numerous hospitalists, regardless of the practice setting.

Areas in Need of Future Study

Based on the current literature, as well as statements in the guideline, we expect future guidance around potential screening for and isolation of asymptomatic carriers, including closer guidance on stool transplantation focusing on timing and route, as further data emerge in these areas.

Other Resources

• Grading of Recommendations Assessment, Development, and Evaluation system (http://www.gradeworkinggroup.org)
• Universal Screening for C. difficile in a Tertiary Hospital: risk factors for carriage and clinical disease (Color/Blackhttps://www.clinicalmicrobiologyandinfection.com/article/S1198-743X(19)30048-5/fulltext)
• Effectiveness of Isolating Clostridium Difficile Asymptomatic Carriers on the Incidence of Infections (Color/Blackhttps://clinicaltrials.gov/ct2/show/NCT03223415)
• Effect of Detecting and Isolating Clostridium difficile Carriers at Hospital Admission on the Incidence of C difficile Infections (Color/Blackhttps://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2516765)
• Clinical Trial Testing Fecal Microbiota Transplant for Recurrent Diarrheal Disease Begins (Color/Blackhttps://www.nih.gov/news-events/news-releases/clinical-trial-testing-fecal-microbiota-transplant-recurrent-diarrheal-disease-begins)

Clostridium difficile, now referred to as Clostridioides difficile (C. difficile), is the most commonly identified cause of healthcare-associated infection among adults in the United States.¹ Because C. difficile infection results in significant mortality and inpatient costs, its persistence threatens to undermine patient safety and the value of healthcare delivery.¹ A standardized, evidence-based approach to diagnosis and management is crucial. However, inconsistencies remain with regard to the appropriate threshold for testing, the type of diagnostic tests used, and treatment. Knowledge of these areas has progressed since the publication of the previous C. difficile guidelines in 2010. These guidelines contain 53 recommendations across 35 sections based on a systematic weighting of the strength of recommendation and quality of evidence using the Grading of Recommendations Assessment, Development, and Evaluation system. Herein, we have chosen to highlight five of these recommendations most relevant to hospitalists.

Recommendation 1. Patients with unexplained and new-onset ≥3 unformed stools within 24 hours are the preferred target population for testing for C. difficile infection (weak recommendation, very low quality of evidence). Do not perform repeat testing (within seven days) during the same episode of diarrhea and do not test stool from asymptomatic patients (strong recommendation, moderate quality of evidence).

In the recent past, healthcare facilities employed C. difficile tests with limited sensitivity, leading to frequent and repeat testing of hospitalized patients. Excess testing puts patients at risk for false positive results and unnecessary or prolonged treatment courses. Proper testing requires consideration of pretest probability, including analysis of the alternative causes of diarrhea. Duration of hospitalization and antibiotic exposure are the most significant modifiable risk factors for C. difficile infection in adult inpatients.² Laxative use within the previous 48 hours, enteral tube feeding, and underlying medical conditions, such as inflammatory bowel disease (IBD), are common causes of improper testing.³ This decision may be difficult, as some underlying causes of diarrhea, such as IBD and enteral tube feeding, also increase the risk of C. difficile infection.³ Laboratories can help by rejecting specimens that are not liquid or soft and employing a multistep algorithm using a combination of nucleic acid testing, antigen testing, and toxin detection to maximize sensitivity and specificity. Because recurrent C. difficile infection is relatively common, repeat testing is appropriate only for recurrence of symptoms following successful treatment and should focus on detection of C. difficile toxin because the persistence of the organism itself can occur after successful treatment.⁴

Recommendation 2. Either vancomycin (125 mg orally four times per day for 10 days) or fidaxomicin (200 mg twice daily for 10 days) is recommended over metronidazole for an initial episode of nonsevere or severe C. difficile infection (strong recommendation, high quality of evidence). For fulminant C. difficile infection, the regimen of choice is a vancomycin dosage of 500 mg orally four times per day (per rectum every six hours if with ileus) in addition to intravenous metronidazole (strong recommendation, moderate quality of evidence).

For several decades now, metronidazole has been the primary antibiotic agent for initial treatment of nonsevere C. difficile infection. Two recent randomized, placebo-controlled trials, however, have found oral vancomycin to be superior to metronidazole for producing a clinical cure and resolution of diarrhea without recurrence.^5,6 Oral vancomycin remains the treatment of choice for severe C. difficile infection. Fidaxomicin, a recently FDA-approved antibiotic, can also be used as initial treatment in place of oral vancomycin. One study found fidaxomicin to be superior to oral vancomycin for producing a sustained clinical response, that is, resolution of diarrhea at the end of treatment without recurrence 25 days later.⁷ Fulminant disease, which is characterized by hypotension or shock, ileus, or megacolon, requires a higher dose of oral vancomycin (or vancomycin enema if with ileus) in addition to intravenous metronidazole.

Recommendation 3. Treat a first recurrence of C. difficile infection with oral vancomycin as a tapered and pulsed regimen rather than a second standard 10-day course of vancomycin or metronidazole (weak recommendation, low quality of evidence).

Despite the improved treatment response with oral vancomycin, one in four patients will experience recurrence. For a first recurrence of C. difficile infection after a 10-day course of oral vancomycin, an extended taper or pulsed course of vancomycin should be attempted. Various regimens have been tried and found to be effective. For a second recurrence, providers can consider addition of rifaximin following oral vancomycin. Fecal microbiota transplantation is recommended for patients with multiple recurrences of C. difficile infection who have failed these antibiotic treatments.

Recommendation 4. Minimize the frequency and duration of high-risk antibiotic therapy (based on local epidemiology) and the number of antibiotic agents prescribed to reduce C. difficile infection risk (strong recommendation, moderate quality of evidence).

Antibiotic stewardship is a necessary component of any successful effort to reduce C. difficile infections. Antibiotic stewardship programs, which are now commonplace in US hospitals, largely rely on educational initiatives or committee-based order review. Hospitalists should take a structured approach emphasizing the four critical questions of antibiotic prescribing: Does this infection require antibiotics? Have I ordered appropriate cultures and the correct empiric therapy? Can I stop, narrow, or switch to oral agents? Finally, what duration of therapy is needed at discharge?⁸ Initial efforts should focus on the restriction of fluoroquinolones, clindamycin, and cephalosporins (except for surgical antibiotic prophylaxis) given their known risk to cause C. difficile infection.

Recommendation 5. Contact precautions should be maintained for at least 48 hours after diarrhea has resolved (weak recommendation, low quality of evidence).

Although C. difficile is undetectable in stool samples from most patients by the time diarrhea has resolved, skin and environmental contaminations remain high. No studies demonstrating a benefit to further extending contact precautions beyond 48 hours after resolution of diarrhea are yet available.

Methods in Preparing Guidelines

The guideline committee consisted of an interdisciplinary team of healthcare providers with extensive experience in the diagnosis, infection control, treatment, and management of C. difficile. The literature search accessed five different databases (Medline, Embase, Cochrane, Health Technology Assessment, and Database of Abstracts of Reviews and Effects), relevant journals, conference proceedings, and regulatory websites published over the search period of 2009-2016.

A major strength of these guidelines is the extensive work that went into their preparation. The committee reviewed over 14,000 pieces of literature and performed a detailed analysis of each one to determine the quality of evidence in support of each recommendation.

Sources of Potential Conflict of Interest or Bias

To reduce bias, the committee’s work was funded by Infectious Disease Society of America and Society for Healthcare Epidemiology of America. Some authors received funding for work outside of this guideline by companies that manufacture diagnostic assays, vancomycin, and fidaxomicin. These potential conflicts were listed at the end of the article.

Generalizability of the Guideline

Not all studies included in the guideline contain exclusively hospitalized patients, but much of the guideline content is applicable to hospitalized patients. Because C. difficile infection is such a widespread public health problem and these guidelines represent a significant update in knowledge since 2010, the specific recommendations highlighted in this review will impact numerous hospitalists, regardless of the practice setting.

Areas in Need of Future Study

Based on the current literature, as well as statements in the guideline, we expect future guidance around potential screening for and isolation of asymptomatic carriers, including closer guidance on stool transplantation focusing on timing and route, as further data emerge in these areas.

Other Resources

• Grading of Recommendations Assessment, Development, and Evaluation system (http://www.gradeworkinggroup.org)
• Universal Screening for C. difficile in a Tertiary Hospital: risk factors for carriage and clinical disease (Color/Blackhttps://www.clinicalmicrobiologyandinfection.com/article/S1198-743X(19)30048-5/fulltext)
• Effectiveness of Isolating Clostridium Difficile Asymptomatic Carriers on the Incidence of Infections (Color/Blackhttps://clinicaltrials.gov/ct2/show/NCT03223415)
• Effect of Detecting and Isolating Clostridium difficile Carriers at Hospital Admission on the Incidence of C difficile Infections (Color/Blackhttps://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2516765)
• Clinical Trial Testing Fecal Microbiota Transplant for Recurrent Diarrheal Disease Begins (Color/Blackhttps://www.nih.gov/news-events/news-releases/clinical-trial-testing-fecal-microbiota-transplant-recurrent-diarrheal-disease-begins)

Clostridium difficile, now referred to as Clostridioides difficile (C. difficile), is the most commonly identified cause of healthcare-associated infection among adults in the United States.¹ Because C. difficile infection results in significant mortality and inpatient costs, its persistence threatens to undermine patient safety and the value of healthcare delivery.¹ A standardized, evidence-based approach to diagnosis and management is crucial. However, inconsistencies remain with regard to the appropriate threshold for testing, the type of diagnostic tests used, and treatment. Knowledge of these areas has progressed since the publication of the previous C. difficile guidelines in 2010. These guidelines contain 53 recommendations across 35 sections based on a systematic weighting of the strength of recommendation and quality of evidence using the Grading of Recommendations Assessment, Development, and Evaluation system. Herein, we have chosen to highlight five of these recommendations most relevant to hospitalists.

Recommendation 1. Patients with unexplained and new-onset ≥3 unformed stools within 24 hours are the preferred target population for testing for C. difficile infection (weak recommendation, very low quality of evidence). Do not perform repeat testing (within seven days) during the same episode of diarrhea and do not test stool from asymptomatic patients (strong recommendation, moderate quality of evidence).

In the recent past, healthcare facilities employed C. difficile tests with limited sensitivity, leading to frequent and repeat testing of hospitalized patients. Excess testing puts patients at risk for false positive results and unnecessary or prolonged treatment courses. Proper testing requires consideration of pretest probability, including analysis of the alternative causes of diarrhea. Duration of hospitalization and antibiotic exposure are the most significant modifiable risk factors for C. difficile infection in adult inpatients.² Laxative use within the previous 48 hours, enteral tube feeding, and underlying medical conditions, such as inflammatory bowel disease (IBD), are common causes of improper testing.³ This decision may be difficult, as some underlying causes of diarrhea, such as IBD and enteral tube feeding, also increase the risk of C. difficile infection.³ Laboratories can help by rejecting specimens that are not liquid or soft and employing a multistep algorithm using a combination of nucleic acid testing, antigen testing, and toxin detection to maximize sensitivity and specificity. Because recurrent C. difficile infection is relatively common, repeat testing is appropriate only for recurrence of symptoms following successful treatment and should focus on detection of C. difficile toxin because the persistence of the organism itself can occur after successful treatment.⁴

Recommendation 2. Either vancomycin (125 mg orally four times per day for 10 days) or fidaxomicin (200 mg twice daily for 10 days) is recommended over metronidazole for an initial episode of nonsevere or severe C. difficile infection (strong recommendation, high quality of evidence). For fulminant C. difficile infection, the regimen of choice is a vancomycin dosage of 500 mg orally four times per day (per rectum every six hours if with ileus) in addition to intravenous metronidazole (strong recommendation, moderate quality of evidence).

For several decades now, metronidazole has been the primary antibiotic agent for initial treatment of nonsevere C. difficile infection. Two recent randomized, placebo-controlled trials, however, have found oral vancomycin to be superior to metronidazole for producing a clinical cure and resolution of diarrhea without recurrence.^5,6 Oral vancomycin remains the treatment of choice for severe C. difficile infection. Fidaxomicin, a recently FDA-approved antibiotic, can also be used as initial treatment in place of oral vancomycin. One study found fidaxomicin to be superior to oral vancomycin for producing a sustained clinical response, that is, resolution of diarrhea at the end of treatment without recurrence 25 days later.⁷ Fulminant disease, which is characterized by hypotension or shock, ileus, or megacolon, requires a higher dose of oral vancomycin (or vancomycin enema if with ileus) in addition to intravenous metronidazole.

Recommendation 3. Treat a first recurrence of C. difficile infection with oral vancomycin as a tapered and pulsed regimen rather than a second standard 10-day course of vancomycin or metronidazole (weak recommendation, low quality of evidence).

Despite the improved treatment response with oral vancomycin, one in four patients will experience recurrence. For a first recurrence of C. difficile infection after a 10-day course of oral vancomycin, an extended taper or pulsed course of vancomycin should be attempted. Various regimens have been tried and found to be effective. For a second recurrence, providers can consider addition of rifaximin following oral vancomycin. Fecal microbiota transplantation is recommended for patients with multiple recurrences of C. difficile infection who have failed these antibiotic treatments.

Recommendation 4. Minimize the frequency and duration of high-risk antibiotic therapy (based on local epidemiology) and the number of antibiotic agents prescribed to reduce C. difficile infection risk (strong recommendation, moderate quality of evidence).

Antibiotic stewardship is a necessary component of any successful effort to reduce C. difficile infections. Antibiotic stewardship programs, which are now commonplace in US hospitals, largely rely on educational initiatives or committee-based order review. Hospitalists should take a structured approach emphasizing the four critical questions of antibiotic prescribing: Does this infection require antibiotics? Have I ordered appropriate cultures and the correct empiric therapy? Can I stop, narrow, or switch to oral agents? Finally, what duration of therapy is needed at discharge?⁸ Initial efforts should focus on the restriction of fluoroquinolones, clindamycin, and cephalosporins (except for surgical antibiotic prophylaxis) given their known risk to cause C. difficile infection.

Recommendation 5. Contact precautions should be maintained for at least 48 hours after diarrhea has resolved (weak recommendation, low quality of evidence).

Although C. difficile is undetectable in stool samples from most patients by the time diarrhea has resolved, skin and environmental contaminations remain high. No studies demonstrating a benefit to further extending contact precautions beyond 48 hours after resolution of diarrhea are yet available.

Methods in Preparing Guidelines

The guideline committee consisted of an interdisciplinary team of healthcare providers with extensive experience in the diagnosis, infection control, treatment, and management of C. difficile. The literature search accessed five different databases (Medline, Embase, Cochrane, Health Technology Assessment, and Database of Abstracts of Reviews and Effects), relevant journals, conference proceedings, and regulatory websites published over the search period of 2009-2016.

A major strength of these guidelines is the extensive work that went into their preparation. The committee reviewed over 14,000 pieces of literature and performed a detailed analysis of each one to determine the quality of evidence in support of each recommendation.

Sources of Potential Conflict of Interest or Bias

To reduce bias, the committee’s work was funded by Infectious Disease Society of America and Society for Healthcare Epidemiology of America. Some authors received funding for work outside of this guideline by companies that manufacture diagnostic assays, vancomycin, and fidaxomicin. These potential conflicts were listed at the end of the article.

Generalizability of the Guideline

Not all studies included in the guideline contain exclusively hospitalized patients, but much of the guideline content is applicable to hospitalized patients. Because C. difficile infection is such a widespread public health problem and these guidelines represent a significant update in knowledge since 2010, the specific recommendations highlighted in this review will impact numerous hospitalists, regardless of the practice setting.

Areas in Need of Future Study

Based on the current literature, as well as statements in the guideline, we expect future guidance around potential screening for and isolation of asymptomatic carriers, including closer guidance on stool transplantation focusing on timing and route, as further data emerge in these areas.

Other Resources

• Grading of Recommendations Assessment, Development, and Evaluation system (http://www.gradeworkinggroup.org)
• Universal Screening for C. difficile in a Tertiary Hospital: risk factors for carriage and clinical disease (Color/Blackhttps://www.clinicalmicrobiologyandinfection.com/article/S1198-743X(19)30048-5/fulltext)
• Effectiveness of Isolating Clostridium Difficile Asymptomatic Carriers on the Incidence of Infections (Color/Blackhttps://clinicaltrials.gov/ct2/show/NCT03223415)
• Effect of Detecting and Isolating Clostridium difficile Carriers at Hospital Admission on the Incidence of C difficile Infections (Color/Blackhttps://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2516765)
• Clinical Trial Testing Fecal Microbiota Transplant for Recurrent Diarrheal Disease Begins (Color/Blackhttps://www.nih.gov/news-events/news-releases/clinical-trial-testing-fecal-microbiota-transplant-recurrent-diarrheal-disease-begins)