Original Research

Insulin pump technology has been available since the 1970s. Innovation in insulin pumps has had significant impact on the management of diabetes mellitus (DM). In recent years, automated insulin pump technology (AIP) has proven to be a safe and effective way to treat DM. It has been studied mostly in highly organized randomized controlled trials (RCTs) in younger populations with type 1 DM (T1DM).^1-3

One of the challenges in DM care has always been the wide variations in daily plasma glucose concentration that often cause major swings of hyperglycemia and hypoglycemia. Extreme variations in blood glucose have also been linked to adverse outcomes, including poor micro- and macrovascular outcomes.^4,5 AIP technology is a hybrid closed-loop system that attempts to solve this problem by adjusting insulin delivery in response to real-time glucose information from a continuous glucose monitor (CGM). Glucose measurements are sent to the insulin pump in real time, which uses a specialized algorithm to determine whether insulin delivery should be up-titrated, down-titrated, or suspended.⁶

Several studies have shown that AIP technology reduces glucose variability and increases the percentage of time within the optimal glucose range.^1-3,7 Its safety is especially indicated for patients with long-standing DM who often have hypoglycemia unawareness and recurrent episodes of hypoglycemia.⁷ Safety is the major advantage of the hybrid closed-loop system as long duration of DM makes patients particularly prone to emergency department (ED) visits and hospitalizations for severe hypoglycemia.⁸ Recurrent hypoglycemia also is associated with increased cardiovascular mortality in epidemiologic studies.⁹

Safety was the primary endpoint in the pivotal trial in a multicenter clinical study where 124 participants (mean age, 37.8 years; DM duration, 21.7 years; hemoglobin A_1c [HbA_1c], 7.4%) were monitored for 3 months while using a hybrid closed-loop pump, similar to the one used in our study.¹⁰ Remarkably, there were no device-related episodes of severe hypoglycemia or ketoacidosis. There was even a small but significant difference in HbA_1c (7.4% at baseline, 6.9% at 3 months) and of the time in target range measured by CGM from 66.7% at baseline to 72.2% at 3 months). However, the mean age of the population studied was young (mean age, 37.8 years). It is unclear how these results would translate for a population of older patients with T1DM. Moreover, use of AIP systems have not been systematically tested outside of carefully controlled studies, as it would be in middle-aged veterans followed in outpatient US Department of Veterans Affairs (VA) clinics. Such an approach in the context of optimal glucose monitoring combined with use of structured DM education can significantly reduce impaired awareness of hypoglycemia in patients with T1DM of long duration.¹¹

This is the first study to assess the feasibility of AIP technology in a real-world population of older veterans with T1DM in terms of safety and acceptability, because AIP has just recently become available for patient care in the Veterans Health Administration (VHA). This group of patients is of particular interest because they have been largely overlooked in earlier studies. They represent an older population with long-standing DM where hypoglycemia unawareness is often recurrent and incapacitating. In addition, long-standing DM makes optimal glycemic control mandatory to prevent microvascular complications.

Methods

In this retrospective review study,, we examined available data in patients with T1DM at the Malcom Randall VA Medical Center diabetes clinic in Gainesville, Florida, between March and December of 2018 who agreed to use AIP. In this clinic, the AIP system was offered to T1DM patients when the 4-year warranty of a previous insulin pump expired, they had frequent hypoglycemic events, or they were on multiple daily injections and were proficient with carbohydrate counting and adjusting insulin doses and willing to use an insulin pump. Veterans were trained on AIP use by a certified diabetes educator and pump trainer in sessions that lasted 2 to 4 hours depending on previous experience with AIP. Institutional review board approval was obtained at the University of Florida. 

Demographic and clinical data before and after the initiation of AIP were collected, including standard insulin pump/CGM information for the Medtronic 670G and Guardian 3 Sensor AIPs. Several variables were evaluated, including age, gender, year of DM diagnosis, time of initiation of AIP, HbA_1c, download data (percentage sensor wear, time in automated mode and manual mode, time in/above/below range, bolus information, insulin use, average sensor blood glucose, average meter blood glucose, pump settings), weight, body mass index (BMI), glucose meter information, history of hypoglycemia unawareness.

The primary outcome for this study was safety as assessed by percentage of time below target range on glucose sensor (time below target range is defined as < 70 mg/dL). We also addressed the secondary endpoint of efficacy as the percentage of time in-range defined as blood glucose per glucose sensor of 70 mg/dL to 180 mg/dL (efficacy), percentage of glucose sensor wear, and HbA_1c.

Statistics

Comparisons of changes in continuous variables between groups were performed by an analysis of covariance (ANCOVA), adjusting for baseline levels. Fisher exact test (χ²) and unpaired t test were used to compare group differences at baseline for categorical and continuous variables, respectively, while Wilcoxon rank sum test was used for nonnormally distributed values. Changes in continuous measures within the same group were tested by paired t test or Wilcoxon matched-pairs signed rank test when applicable. Analyses were performed using Stata 11.0.

Results

Thirty-seven veterans with T1DM using AIPs in 2018 were evaluated at baseline and at follow up visits (Tables 1 and 2). Time frame for follow-up was approximately 3 months, although there was some variation. Of note, the mean weight and BMI corresponded to mostly lean individuals, consistent with the diagnosis of T1DM.

Time below target range hypoglycemia (sensor glucose < 70 mg/dL) remained low at each follow-up visit (both 1.5%). Percentage of time in automated mode increased from first to second follow-up visit after initiation of AIP (41% vs 53%, P = .06). Percentage of sensor wear numerically increased from first to second follow-up visit (75% vs 85%, P = .39), same as time in range, defined as sensor glucose 70 to 180 mg/dL, from first to second follow-up visit (70% vs 73%, P = .09). Time above range, defined as sensor glucose > 180 mg/dL, demonstrated a strong trend toward decreasing between follow-up appointments (29% to 25%; P = .09). HbA_1c decreased from 7.6% to 7.3% (P = .005).

About half of the patients (18 of 37) reported hypoglycemia unawareness before the initiation of the 670G AIP. On follow-up visit 61% (11 of 18) reported significant improvement in awareness. Of the remaining 18 patients who reported normal awareness before automated mode, 17% (3 of 18) described a new onset unawareness.

Discussion

This study evaluated the safety of adopting a new DM technology in the real world of an outpatient VA clinic. To the best of our knowledge, this is the first study evaluating the use of AIP specifically in a population of middle-aged veterans with longstanding T1DM. After a mean 7 months of follow-up, participants accepted AIP use as evidenced by increased sensor wear over time and experienced improvements in DM measures that indicate successful use (ie, time in automated mode, which represents reduced glycemic variability). These results show success of an AIP approach in a demographically older group of patients.

AIP has been shown to have positive effects on glycemic control such as time in target glucose range (goal ≥ 70%). In our relatively small pilot study, there was trend for an improvement in the time in range from the first to second clinical follow-up visit, suggesting true patient involvement with the use of the device. Studies involving overall younger cohorts have proved that AIP technology is safe and efficacious for outpatient management of T1DM.^7,10,12,13 However, they were all conducted under the safety of a research setting, and trials enrolled a younger population believed to adapt with more ease to this new technology. Tauschmann and colleagues performed a multicenter, parallel randomized controlled trial that compared hybrid closed-loop AIP therapy with sensor-augmented pump therapy in patients with suboptimal T1DM control.¹² Results showed that the hybrid closed-loop system increased the time that the glucose concentration was within the target range (70-180 mg/dL) from 54% in the sensor-augmented pump group to 65% on the closed-loop system (P < .001). A small but significant improvement in HBA_1c (from 8.0 -7.4%) and low rates of hypoglycemia (2.6% of time below 70 mg/dL) were also noted.¹²

A similar benefit was observed in a 2019 landmark study by Brown and colleagues of 168 patients with T1DM at 7 university medical centers who were treated for 6 months with either a closed-loop system (closed-loop group) or a sensor-augmented pump (control group) in a parallel-group, unblinded, randomized trial study.¹³ Mean (SD) time in the target range increased in the closed-loop group from 61% (17) at baseline to 71% (12) during the 6 months. HbA_1c decreased from 7.4 to 7.1% and time ≤ 70 mg/dL was just 1.6%. However, only 13% of patients were aged ≥ 40 years in the study by Tauschmann and colleagues, and mean age was 33 years in the Brown and colleagues study.^12,13 In contrast, the mean (SD) age in our study was 59 (14) years. Our pilot study also showed comparable, or somewhat better results, as mean time in target range was 72%, HbA_1c was 7.3%, and time ≤ 70 mg/dL was just 1.5%.

Limitations

There are some limitations to our study. The small sample size and single-center nature prevent generalization. Also, the veteran population cannot be extrapolated to other populations. For instance, the majority of the patients in this study were male.

Conclusions

We report that an AIP approach for patients with long-standing T1DM is well accepted and engages patients into monitoring their blood sugars and achieving better glycemic control. This was achieved with minimal hypoglycemia in a population where often hypoglycemia unawareness makes DM care a challenge. Future studies within the VHA are needed to fully assess the long-term benefits and cost-effectiveness of this technology in veterans.

Insulin pump technology has been available since the 1970s. Innovation in insulin pumps has had significant impact on the management of diabetes mellitus (DM). In recent years, automated insulin pump technology (AIP) has proven to be a safe and effective way to treat DM. It has been studied mostly in highly organized randomized controlled trials (RCTs) in younger populations with type 1 DM (T1DM).^1-3

One of the challenges in DM care has always been the wide variations in daily plasma glucose concentration that often cause major swings of hyperglycemia and hypoglycemia. Extreme variations in blood glucose have also been linked to adverse outcomes, including poor micro- and macrovascular outcomes.^4,5 AIP technology is a hybrid closed-loop system that attempts to solve this problem by adjusting insulin delivery in response to real-time glucose information from a continuous glucose monitor (CGM). Glucose measurements are sent to the insulin pump in real time, which uses a specialized algorithm to determine whether insulin delivery should be up-titrated, down-titrated, or suspended.⁶

Several studies have shown that AIP technology reduces glucose variability and increases the percentage of time within the optimal glucose range.^1-3,7 Its safety is especially indicated for patients with long-standing DM who often have hypoglycemia unawareness and recurrent episodes of hypoglycemia.⁷ Safety is the major advantage of the hybrid closed-loop system as long duration of DM makes patients particularly prone to emergency department (ED) visits and hospitalizations for severe hypoglycemia.⁸ Recurrent hypoglycemia also is associated with increased cardiovascular mortality in epidemiologic studies.⁹

Safety was the primary endpoint in the pivotal trial in a multicenter clinical study where 124 participants (mean age, 37.8 years; DM duration, 21.7 years; hemoglobin A_1c [HbA_1c], 7.4%) were monitored for 3 months while using a hybrid closed-loop pump, similar to the one used in our study.¹⁰ Remarkably, there were no device-related episodes of severe hypoglycemia or ketoacidosis. There was even a small but significant difference in HbA_1c (7.4% at baseline, 6.9% at 3 months) and of the time in target range measured by CGM from 66.7% at baseline to 72.2% at 3 months). However, the mean age of the population studied was young (mean age, 37.8 years). It is unclear how these results would translate for a population of older patients with T1DM. Moreover, use of AIP systems have not been systematically tested outside of carefully controlled studies, as it would be in middle-aged veterans followed in outpatient US Department of Veterans Affairs (VA) clinics. Such an approach in the context of optimal glucose monitoring combined with use of structured DM education can significantly reduce impaired awareness of hypoglycemia in patients with T1DM of long duration.¹¹

This is the first study to assess the feasibility of AIP technology in a real-world population of older veterans with T1DM in terms of safety and acceptability, because AIP has just recently become available for patient care in the Veterans Health Administration (VHA). This group of patients is of particular interest because they have been largely overlooked in earlier studies. They represent an older population with long-standing DM where hypoglycemia unawareness is often recurrent and incapacitating. In addition, long-standing DM makes optimal glycemic control mandatory to prevent microvascular complications.

Methods

In this retrospective review study,, we examined available data in patients with T1DM at the Malcom Randall VA Medical Center diabetes clinic in Gainesville, Florida, between March and December of 2018 who agreed to use AIP. In this clinic, the AIP system was offered to T1DM patients when the 4-year warranty of a previous insulin pump expired, they had frequent hypoglycemic events, or they were on multiple daily injections and were proficient with carbohydrate counting and adjusting insulin doses and willing to use an insulin pump. Veterans were trained on AIP use by a certified diabetes educator and pump trainer in sessions that lasted 2 to 4 hours depending on previous experience with AIP. Institutional review board approval was obtained at the University of Florida. 

Demographic and clinical data before and after the initiation of AIP were collected, including standard insulin pump/CGM information for the Medtronic 670G and Guardian 3 Sensor AIPs. Several variables were evaluated, including age, gender, year of DM diagnosis, time of initiation of AIP, HbA_1c, download data (percentage sensor wear, time in automated mode and manual mode, time in/above/below range, bolus information, insulin use, average sensor blood glucose, average meter blood glucose, pump settings), weight, body mass index (BMI), glucose meter information, history of hypoglycemia unawareness.

The primary outcome for this study was safety as assessed by percentage of time below target range on glucose sensor (time below target range is defined as < 70 mg/dL). We also addressed the secondary endpoint of efficacy as the percentage of time in-range defined as blood glucose per glucose sensor of 70 mg/dL to 180 mg/dL (efficacy), percentage of glucose sensor wear, and HbA_1c.

Statistics

Comparisons of changes in continuous variables between groups were performed by an analysis of covariance (ANCOVA), adjusting for baseline levels. Fisher exact test (χ²) and unpaired t test were used to compare group differences at baseline for categorical and continuous variables, respectively, while Wilcoxon rank sum test was used for nonnormally distributed values. Changes in continuous measures within the same group were tested by paired t test or Wilcoxon matched-pairs signed rank test when applicable. Analyses were performed using Stata 11.0.

Results

Thirty-seven veterans with T1DM using AIPs in 2018 were evaluated at baseline and at follow up visits (Tables 1 and 2). Time frame for follow-up was approximately 3 months, although there was some variation. Of note, the mean weight and BMI corresponded to mostly lean individuals, consistent with the diagnosis of T1DM.

Time below target range hypoglycemia (sensor glucose < 70 mg/dL) remained low at each follow-up visit (both 1.5%). Percentage of time in automated mode increased from first to second follow-up visit after initiation of AIP (41% vs 53%, P = .06). Percentage of sensor wear numerically increased from first to second follow-up visit (75% vs 85%, P = .39), same as time in range, defined as sensor glucose 70 to 180 mg/dL, from first to second follow-up visit (70% vs 73%, P = .09). Time above range, defined as sensor glucose > 180 mg/dL, demonstrated a strong trend toward decreasing between follow-up appointments (29% to 25%; P = .09). HbA_1c decreased from 7.6% to 7.3% (P = .005).

About half of the patients (18 of 37) reported hypoglycemia unawareness before the initiation of the 670G AIP. On follow-up visit 61% (11 of 18) reported significant improvement in awareness. Of the remaining 18 patients who reported normal awareness before automated mode, 17% (3 of 18) described a new onset unawareness.

Discussion

This study evaluated the safety of adopting a new DM technology in the real world of an outpatient VA clinic. To the best of our knowledge, this is the first study evaluating the use of AIP specifically in a population of middle-aged veterans with longstanding T1DM. After a mean 7 months of follow-up, participants accepted AIP use as evidenced by increased sensor wear over time and experienced improvements in DM measures that indicate successful use (ie, time in automated mode, which represents reduced glycemic variability). These results show success of an AIP approach in a demographically older group of patients.

AIP has been shown to have positive effects on glycemic control such as time in target glucose range (goal ≥ 70%). In our relatively small pilot study, there was trend for an improvement in the time in range from the first to second clinical follow-up visit, suggesting true patient involvement with the use of the device. Studies involving overall younger cohorts have proved that AIP technology is safe and efficacious for outpatient management of T1DM.^7,10,12,13 However, they were all conducted under the safety of a research setting, and trials enrolled a younger population believed to adapt with more ease to this new technology. Tauschmann and colleagues performed a multicenter, parallel randomized controlled trial that compared hybrid closed-loop AIP therapy with sensor-augmented pump therapy in patients with suboptimal T1DM control.¹² Results showed that the hybrid closed-loop system increased the time that the glucose concentration was within the target range (70-180 mg/dL) from 54% in the sensor-augmented pump group to 65% on the closed-loop system (P < .001). A small but significant improvement in HBA_1c (from 8.0 -7.4%) and low rates of hypoglycemia (2.6% of time below 70 mg/dL) were also noted.¹²

A similar benefit was observed in a 2019 landmark study by Brown and colleagues of 168 patients with T1DM at 7 university medical centers who were treated for 6 months with either a closed-loop system (closed-loop group) or a sensor-augmented pump (control group) in a parallel-group, unblinded, randomized trial study.¹³ Mean (SD) time in the target range increased in the closed-loop group from 61% (17) at baseline to 71% (12) during the 6 months. HbA_1c decreased from 7.4 to 7.1% and time ≤ 70 mg/dL was just 1.6%. However, only 13% of patients were aged ≥ 40 years in the study by Tauschmann and colleagues, and mean age was 33 years in the Brown and colleagues study.^12,13 In contrast, the mean (SD) age in our study was 59 (14) years. Our pilot study also showed comparable, or somewhat better results, as mean time in target range was 72%, HbA_1c was 7.3%, and time ≤ 70 mg/dL was just 1.5%.

Limitations

There are some limitations to our study. The small sample size and single-center nature prevent generalization. Also, the veteran population cannot be extrapolated to other populations. For instance, the majority of the patients in this study were male.

Conclusions

We report that an AIP approach for patients with long-standing T1DM is well accepted and engages patients into monitoring their blood sugars and achieving better glycemic control. This was achieved with minimal hypoglycemia in a population where often hypoglycemia unawareness makes DM care a challenge. Future studies within the VHA are needed to fully assess the long-term benefits and cost-effectiveness of this technology in veterans.

Insulin pump technology has been available since the 1970s. Innovation in insulin pumps has had significant impact on the management of diabetes mellitus (DM). In recent years, automated insulin pump technology (AIP) has proven to be a safe and effective way to treat DM. It has been studied mostly in highly organized randomized controlled trials (RCTs) in younger populations with type 1 DM (T1DM).^1-3

One of the challenges in DM care has always been the wide variations in daily plasma glucose concentration that often cause major swings of hyperglycemia and hypoglycemia. Extreme variations in blood glucose have also been linked to adverse outcomes, including poor micro- and macrovascular outcomes.^4,5 AIP technology is a hybrid closed-loop system that attempts to solve this problem by adjusting insulin delivery in response to real-time glucose information from a continuous glucose monitor (CGM). Glucose measurements are sent to the insulin pump in real time, which uses a specialized algorithm to determine whether insulin delivery should be up-titrated, down-titrated, or suspended.⁶

Several studies have shown that AIP technology reduces glucose variability and increases the percentage of time within the optimal glucose range.^1-3,7 Its safety is especially indicated for patients with long-standing DM who often have hypoglycemia unawareness and recurrent episodes of hypoglycemia.⁷ Safety is the major advantage of the hybrid closed-loop system as long duration of DM makes patients particularly prone to emergency department (ED) visits and hospitalizations for severe hypoglycemia.⁸ Recurrent hypoglycemia also is associated with increased cardiovascular mortality in epidemiologic studies.⁹

Safety was the primary endpoint in the pivotal trial in a multicenter clinical study where 124 participants (mean age, 37.8 years; DM duration, 21.7 years; hemoglobin A_1c [HbA_1c], 7.4%) were monitored for 3 months while using a hybrid closed-loop pump, similar to the one used in our study.¹⁰ Remarkably, there were no device-related episodes of severe hypoglycemia or ketoacidosis. There was even a small but significant difference in HbA_1c (7.4% at baseline, 6.9% at 3 months) and of the time in target range measured by CGM from 66.7% at baseline to 72.2% at 3 months). However, the mean age of the population studied was young (mean age, 37.8 years). It is unclear how these results would translate for a population of older patients with T1DM. Moreover, use of AIP systems have not been systematically tested outside of carefully controlled studies, as it would be in middle-aged veterans followed in outpatient US Department of Veterans Affairs (VA) clinics. Such an approach in the context of optimal glucose monitoring combined with use of structured DM education can significantly reduce impaired awareness of hypoglycemia in patients with T1DM of long duration.¹¹

This is the first study to assess the feasibility of AIP technology in a real-world population of older veterans with T1DM in terms of safety and acceptability, because AIP has just recently become available for patient care in the Veterans Health Administration (VHA). This group of patients is of particular interest because they have been largely overlooked in earlier studies. They represent an older population with long-standing DM where hypoglycemia unawareness is often recurrent and incapacitating. In addition, long-standing DM makes optimal glycemic control mandatory to prevent microvascular complications.

Methods

In this retrospective review study,, we examined available data in patients with T1DM at the Malcom Randall VA Medical Center diabetes clinic in Gainesville, Florida, between March and December of 2018 who agreed to use AIP. In this clinic, the AIP system was offered to T1DM patients when the 4-year warranty of a previous insulin pump expired, they had frequent hypoglycemic events, or they were on multiple daily injections and were proficient with carbohydrate counting and adjusting insulin doses and willing to use an insulin pump. Veterans were trained on AIP use by a certified diabetes educator and pump trainer in sessions that lasted 2 to 4 hours depending on previous experience with AIP. Institutional review board approval was obtained at the University of Florida. 

Demographic and clinical data before and after the initiation of AIP were collected, including standard insulin pump/CGM information for the Medtronic 670G and Guardian 3 Sensor AIPs. Several variables were evaluated, including age, gender, year of DM diagnosis, time of initiation of AIP, HbA_1c, download data (percentage sensor wear, time in automated mode and manual mode, time in/above/below range, bolus information, insulin use, average sensor blood glucose, average meter blood glucose, pump settings), weight, body mass index (BMI), glucose meter information, history of hypoglycemia unawareness.

The primary outcome for this study was safety as assessed by percentage of time below target range on glucose sensor (time below target range is defined as < 70 mg/dL). We also addressed the secondary endpoint of efficacy as the percentage of time in-range defined as blood glucose per glucose sensor of 70 mg/dL to 180 mg/dL (efficacy), percentage of glucose sensor wear, and HbA_1c.

Statistics

Comparisons of changes in continuous variables between groups were performed by an analysis of covariance (ANCOVA), adjusting for baseline levels. Fisher exact test (χ²) and unpaired t test were used to compare group differences at baseline for categorical and continuous variables, respectively, while Wilcoxon rank sum test was used for nonnormally distributed values. Changes in continuous measures within the same group were tested by paired t test or Wilcoxon matched-pairs signed rank test when applicable. Analyses were performed using Stata 11.0.

Results

Thirty-seven veterans with T1DM using AIPs in 2018 were evaluated at baseline and at follow up visits (Tables 1 and 2). Time frame for follow-up was approximately 3 months, although there was some variation. Of note, the mean weight and BMI corresponded to mostly lean individuals, consistent with the diagnosis of T1DM.

Time below target range hypoglycemia (sensor glucose < 70 mg/dL) remained low at each follow-up visit (both 1.5%). Percentage of time in automated mode increased from first to second follow-up visit after initiation of AIP (41% vs 53%, P = .06). Percentage of sensor wear numerically increased from first to second follow-up visit (75% vs 85%, P = .39), same as time in range, defined as sensor glucose 70 to 180 mg/dL, from first to second follow-up visit (70% vs 73%, P = .09). Time above range, defined as sensor glucose > 180 mg/dL, demonstrated a strong trend toward decreasing between follow-up appointments (29% to 25%; P = .09). HbA_1c decreased from 7.6% to 7.3% (P = .005).

About half of the patients (18 of 37) reported hypoglycemia unawareness before the initiation of the 670G AIP. On follow-up visit 61% (11 of 18) reported significant improvement in awareness. Of the remaining 18 patients who reported normal awareness before automated mode, 17% (3 of 18) described a new onset unawareness.

Discussion

This study evaluated the safety of adopting a new DM technology in the real world of an outpatient VA clinic. To the best of our knowledge, this is the first study evaluating the use of AIP specifically in a population of middle-aged veterans with longstanding T1DM. After a mean 7 months of follow-up, participants accepted AIP use as evidenced by increased sensor wear over time and experienced improvements in DM measures that indicate successful use (ie, time in automated mode, which represents reduced glycemic variability). These results show success of an AIP approach in a demographically older group of patients.

AIP has been shown to have positive effects on glycemic control such as time in target glucose range (goal ≥ 70%). In our relatively small pilot study, there was trend for an improvement in the time in range from the first to second clinical follow-up visit, suggesting true patient involvement with the use of the device. Studies involving overall younger cohorts have proved that AIP technology is safe and efficacious for outpatient management of T1DM.^7,10,12,13 However, they were all conducted under the safety of a research setting, and trials enrolled a younger population believed to adapt with more ease to this new technology. Tauschmann and colleagues performed a multicenter, parallel randomized controlled trial that compared hybrid closed-loop AIP therapy with sensor-augmented pump therapy in patients with suboptimal T1DM control.¹² Results showed that the hybrid closed-loop system increased the time that the glucose concentration was within the target range (70-180 mg/dL) from 54% in the sensor-augmented pump group to 65% on the closed-loop system (P < .001). A small but significant improvement in HBA_1c (from 8.0 -7.4%) and low rates of hypoglycemia (2.6% of time below 70 mg/dL) were also noted.¹²

A similar benefit was observed in a 2019 landmark study by Brown and colleagues of 168 patients with T1DM at 7 university medical centers who were treated for 6 months with either a closed-loop system (closed-loop group) or a sensor-augmented pump (control group) in a parallel-group, unblinded, randomized trial study.¹³ Mean (SD) time in the target range increased in the closed-loop group from 61% (17) at baseline to 71% (12) during the 6 months. HbA_1c decreased from 7.4 to 7.1% and time ≤ 70 mg/dL was just 1.6%. However, only 13% of patients were aged ≥ 40 years in the study by Tauschmann and colleagues, and mean age was 33 years in the Brown and colleagues study.^12,13 In contrast, the mean (SD) age in our study was 59 (14) years. Our pilot study also showed comparable, or somewhat better results, as mean time in target range was 72%, HbA_1c was 7.3%, and time ≤ 70 mg/dL was just 1.5%.

Limitations

There are some limitations to our study. The small sample size and single-center nature prevent generalization. Also, the veteran population cannot be extrapolated to other populations. For instance, the majority of the patients in this study were male.

Conclusions

We report that an AIP approach for patients with long-standing T1DM is well accepted and engages patients into monitoring their blood sugars and achieving better glycemic control. This was achieved with minimal hypoglycemia in a population where often hypoglycemia unawareness makes DM care a challenge. Future studies within the VHA are needed to fully assess the long-term benefits and cost-effectiveness of this technology in veterans.

From the Department of Pharmaceutical and Health Economics, University of Southern California, Los Angeles, CA, (Drs. Sangha and McCombs), Department of Pediatrics, Keck School of Medicine, and Department of Clinical Pharmacy, School of Pharmacy, University of Southern California, Los Angeles, CA, (Dr. Steinberg), and Leonard Schaeffer Center for Health Policy and Economics, University of Southern California, Los Angeles, CA (Dr. McCombs).

Objective: The recommended treatment for children and adolescents under 18 years of age who have a positive test for group A Streptococcus (GAS) are antibiotics using the “test and treat” strategy to detect and treat GAS for pediatric pharyngitis. This study used paid claims data to document the extent to which real-world treatment patterns are consistent with these recommendations. We document the factors correlated with testing and treatment, then examine the effects of receiving a GAS test and being treated with an antibiotic impact the likelihood of a revisit for an acute respiratory tract infection within 28 days.

Methods: This retrospective cohort study used Optum Insight Clinformatics data for medical and pharmacy claims from 2011-2013 to identify episodes of care for children and adolescents with pharyngitis around their index visit (± 6 months). The sample population included children and adolescents under 18 years of age with a diagnosis of pharyngitis. Multivariable logistic regression analyses were used to document factors associated with receipt of GAS test and antibiotic treatment. Next, we used logistic regression models to estimate the impact of test and treat recommendation on revisit risk.

Results: There were 24 685 treatment episodes for children and adolescents diagnosed with pharyngitis. Nearly 47% of these episodes included a GAS test and 48% of tested patients were prescribed an antibiotic prescription. Failing to perform a GAS test increased the risk of a revisit within 28 days by 44%. The use of antibiotics by tested and untested patients had no impact on revisit risk.

Conclusion: While the judicious use of antibiotics is important in managing pharyngitis infections and managing complications, the use of rapid diagnostic tools was found to be the determining factor in reducing revisits for pediatric patients with pharyngitis.

Keywords: pediatrics; pharyngitis; respiratory infections; acute infections; diagnostic tests; group A Streptococcus; antibiotics; revisits.

Acute pharyngitis is a common acute respiratory tract infection (ARTI) in children. Group A β-hemolytic streptococci (GABHS) is the most common bacterial etiology for pediatric pharyngitis, accounting for 15% to 30% of cases.¹

Beyond clinical assessment, laboratory diagnostic testing generally plays a limited role in guiding appropriate antibiotic prescribing for patients with an ARTI.^2,3 Most diagnostic tests require 2 or 3 days to result, incur additional costs, and may delay treatment.⁴ While these tests do not provide clear and timely guidance on which specific antibiotic is appropriate for ARTI patients, this is not the case for patients with pharyngitis.^5,6,7 A rapid diagnostic test exists to identify pharyngitis patients with GABHS which accounts for 1 in 4 children with acute sore throat.^1,4,6 Both the American Academy of Pediatrics and the Infectious Diseases Society of America recommend antibiotic treatment for children and adolescents under 18 years of age who have a positive test for group A Streptococcus (GAS).^8,9 This “test and treat” protocol has been consistently included in the Healthcare Effectiveness Data and Information Set (HEDIS) standards over time for pediatric pharyngitis patients aged 3 to 18 years before dispensing an antibiotic.¹⁰

Sinusitis, pneumonia, and acute otitis media are considered ARTIs where antibiotic treatment is justified. Therefore, pharyngitis of unclear etiology seen with these comorbid infections may not always undergo GAS testing but move directly to the patient being prescribed antibiotics. This analysis enumerates ARTI-related comorbidities present together with the initial coded pharyngitis diagnosis to evaluate their impact on the provider’s decision to test and treat, and on revisit risk.

Antibiotic treatment for GAS patients is likely to eradicate the acute GABHS infection within 10 days. Penicillin and amoxicillin are commonly recommended because of their narrow spectrum of activity, few adverse effects, established efficacy, and modest cost. Alternative antibiotics for patients with penicillin allergy, or with polymicrobial infection seen on culture results, include a first-generation cephalosporin, clindamycin, clarithromycin (Biaxin), or azithromycin (Zithromax).^1,8,11 However, while compliance with these HEDIS guidelines has been evaluated, the outcome effects of following the HEDIS “test and treat” recommendations for children with pharyngitis have not been adequately evaluated.

These outcome evaluations have increasing importance as the latest HEDIS survey has shown testing rates in commercial Preferred Provider Organizations (PPO) falling from 86.4% in 2018 to 75.9% in 2019, the lowest rate of testing since 2009, with similar reductions under 80% for Health Maintenance Organizations (HMO).¹⁰ While health plans may execute cost-benefit analyses and algorithms to forge best practices for GAS testing in children and adolescents presenting with symptoms of pharyngitis, it is important to regard the wasteful resource utilization and additional cost of revisits that may offset any gains accrued by more focused GAS testing outside the existing clinical guidelines and HEDIS measures. This may be of particular importance in documenting infection and sparing antibiotic therapy in toddlers and younger.

The objective of this study was to investigate the correlation between testing and antibiotic use on the likelihood of a revisit for an acute respiratory tract infection within 28 days. To achieve this objective, this investigation consists of 3 sequential analyses. First, we document the factors associated with the decision to test the patient for a GABHS infection using the GAS test. Next, we document the factors associated with the decision to use an antibiotic to treat the patient as a function of having tested the patient. Finally, we investigate the impact of the testing and treatment decisions on the likelihood of a revisit within 28 days.

Methods

Study design

This was a retrospective cohort study of episodes of treatment for pediatric patients with pharyngitis. Episodes were identified using data derived from the Optum Insight Clinformatics claims database provided to the University of Southern California to facilitate the training of graduate students. These data cover commercially insured patients with both medical and pharmacy benefits. Data were retrieved from the 3-year period spanning 2011-2013. An episode of care was identified based on date of the first (index) outpatient visit for a pharyngitis diagnosis (International Classification of Diseases, Ninth Revision [ICD-9]: 462, 463, 034.0). Outpatient visits were defined by visit setting: ambulatory clinics, physician offices, emergency rooms, and urgent care facilities. Each pharyngitis treatment episode was then screened for at least a 6-month enrollment in a health insurance plan prior and subsequent to the index visit using Optum enrollment data. Finally, eligible treatment episodes were restricted to children and adolescents under 18 years of age, who had an index outpatient visit for a primary diagnosis of acute pharyngitis.

A diagnostic profile was created for each episode using the diagnoses recorded for the index visit. Up to 3 diagnoses may be recorded for any outpatient visit and the first recorded diagnosis was assumed to be the primary diagnosis for that episode. Any secondary diagnoses recorded on the index visit were used to define comorbidities present at the index visit. ARTI-related comorbidities included: acute otitis media (AOM), bronchitis, sinusitis, pneumonia, and upper respiratory infection (URI). Other comorbid medical diagnoses were documented using diagnostic data from the pre-index period. Dichotomous variables for the following categories were created: mental disorders, nervous system disorders, respiratory symptoms, fever, injury and poisoning, other, or no diseases.

Prior visits for other respiratory infections in the previous 90 days were also identified for patients based on their index visit for pharyngitis. Similarly, any subsequent visits, within 28 days of the index visit, were also recorded to measure the health outcome for analysis. Practice settings include physician offices and federally qualified health centers, state and local health clinics, outpatient hospitals facilities, emergency departments, and other outpatient settings such as walk-in retail health clinic or ambulatory centers. Providers include primary care physicians (family practice, pediatricians, internal medicine), specialty care physicians (emergency medicine, preventive medicine), nonphysician providers (nurse practitioners, physician assistants) and other providers (urgent care, acute outpatient care, ambulatory care centers). Seasons of the year were determined based on the index date of the episode to account for possible seasonality in pharyngitis treatment. Lastly, a previous visits variable was created to identify whether the child had nonpharyngitis ARTI visits in the 3 months prior to the index visit.

Demographic variables were created based on enrollment and the socioeconomic data available in the Optum socioeconomic status file. These variables include patient age, race, sex, household income, geographic location, practice setting type, provider specialty, and type of insurance. An estimate of patient household income was based on algorithms using census block groups. Income categories were informed by the federal guidelines for a family of 4. A low-income family was defined as earning less than $50 000; a middle-income family earned between $50 000 and $75 000, and a high-income family earned $75 000 and above.¹² Patient insurance type was categorized as HMO, Exclusive Provider Organization (EPO), Point of Service (POS), and PPO. Race was identified as White, Black, Hispanic, and Asian. Patient location was defined according to national census regions.

Outcomes

GAS test

The HEDIS measures for pharyngitis recommend using the GAS test to identify the bacterial etiology of the pharyngitis infection. Patients who received the test were identified based on Current Procedural Terminology (CPT) codes 87070-71, 87081, 87430, 87650-52, and 87880.¹⁰

The pharmacy administrative claims dataset was used to identify study patients who filled a prescription for an antibiotic during their pharyngitis treatment episode. Optum pharmacy data identify the medications received, specifies the date of prescription filling, National Drug Codes, and American Hospital Formulary Service (AHFS) Classification System codes for each medication. We used the AHFS Pharmacologic-Therapeutic classification of antibiotics to create dichotomous variables documenting the antibacterial used by each patient.¹³ These are categorized under antibacterial including penicillins, cephalosporins (first, second, third, fourth generation cephalosporins), macrolides (first generation and others), tetracyclines, sulfonamides, fluoroquinolones (ciprofloxacin, levofloxacin, moxifloxacin), cephamycin, carbapenems, and β-lactam antibiotics (amoxicillin, amoxicillin/clavulanate, cephalexin, cefuroxime, cefdinir).

Revisits to physician or other provider

Revisits within 28 days were used as the measure of patient outcomes related to testing and filling of an antibiotic prescription for acute pharyngitis. Revisits may also be due to a patient returning for a follow-up, alternative treatment, worsening pharyngitis, or for another ARTI. An ARTI-related revisit also increases total resources used to treat pediatric pharyngitis patients.

Statistical analysis

Logistic regression was used for all 3 analyses conducted in this study. First, we determined the patient and treating physician characteristics that impact the decision to use GAS testing for pharyngitis. Second, we identified those factors that impact the decision to use antibiotic prescriptions among children who were diagnosed with pharyngitis adding in the dichotomous variable indicating if the patient had received a GAS test. Third, we used a logit regression analysis to document if receiving a GAS test and/or an antibiotic impacted the likelihood of a revisit by comparing revisit risk. To estimate the effect of testing and/or antibiotic use, we divided patients into 4 groups based on whether the patient received a GAS test and/or an antibiotic prescription. This specification of the analysis of revisits as an outcome focuses on adherence to HEDIS “test and treat” guidelines¹⁰:

1. Patients who were not tested yet filled an antibiotic prescription. This decision was likely based on the clinician’s judgment of the patient’s signs and symptoms, and confirmational testing not performed.
2. Patients who were not tested and did not fill an antibiotic prescription. Apparently, in the clinician’s judgment the patient’s signs and symptoms were such that the infection did not warrant treatment and the clinical presentation did not necessitate the GAS test to confirm the recorded diagnosis of pharyngitis.
3. Patients who were tested and received antibiotic prescription, likely because the test was positive for GABHS.
4. Patients who were tested and did not receive antibiotic prescription.

We tested for statistically significant differences in baseline characteristics across these 4 patient groups using t tests for continuous variables and χ² tests for categorical variables. Odds ratios (OR) and CI were computed for the influential variables included the regression analyses.

We conducted a sensitivity analysis using a model specification which included the dichotomous variables for testing and for treatment, and the interaction term between these variables to assess if treatment effects varied in tested and untested patients. We also estimated this model of revisit risk using revisits within 7 days as the outcome variable.

All analyses were completed using STATA/IC 13 (StataCorp, College Station, TX).

Results

There were 24 685 treatment episodes for children diagnosed with pharyngitis. Nearly 47% of these episodes included GAS testing and 47% of the tested patients filled an antibiotic prescription. Similarly, 53% of patients were not tested and 49% of untested patients filled an antibiotic prescription. As a result, the 4 groups identified for analysis were evenly distributed: untested and no prescription (26.9%), untested and prescription (26.3%), tested and prescription (21.9%), and tested and no prescription (24.9%) (Figure).

Table 1 presents the descriptive statistics for these 4 patient groups. Note first that the rate of revisits within 28 days is under 5% across all groups. Second, the 2 tested groups have a lower revisit rate than the untested groups: the tested and treated have a revisit rate of 3.3%, and the tested and untreated have a revisit rate of 2.4%, while both the untested groups have a revisit rate of nearly 5%. These small absolute differences in revisit rates across groups were statistically significant.

Factors associated with receiving GAS test

Several factors were found to impact the decision to test (Table 2). Only 9.7% of children were reported to have any ARTI coinfection. As expected, these comorbidities resulted in a significantly lower likelihood of receiving the GAS test: AOM, bronchitis, sinusitis, pneumonia, and URI as comorbid infections had a 48%, 41%, 37%, 63%, and 13% lower likelihood of receiving the GAS test, respectively, than those with no comorbidities. Similarly, children with fever and respiratory symptoms were 35% and 45% less likely to receiving the GAS test, respectively. This is consistent with our expectation that comorbid ARTI infections will lead many providers to forgo testing.

Provider type and patient age also plays a role in receipt of the GAS test. Relative to outpatient facility providers, primary care physicians were 24% more likely and specialty physicians were 38% less likely of employing the GAS test. The child’s age played a significant role in receipt of the GAS test. Children aged 1 to 5 years and 5 to 12 years were 15% and 14% more likely to receive the test compared to children older than 12 years.

Pharyngitis patients have disproportionately higher odds of receiving a GAS test in most regions of the country compared to the Pacific region. For instance, children in the Mid-Atlantic region have 51% higher odds of receiving a GAS test while children in New England have 80% higher odds of receiving the same test.

Black children have 11% lower odds of receiving the GAS test compared to White children. Both middle-income and high-income children have 12% and 32% higher odds of receiving the test compared to low-income children. Compared to office-based visits, children visiting a clinic were twice as likely to receive a GAS test while those seen in the emergency room have 43% lower odds of receiving a GAS test. Hospital outpatient departments, which account for less than 1% of all visits, rarely used a GAS test which could be a statistical artifact due to small sample size. Lastly, insurance and season of the year had no significant impact of receipt of a GAS test.

Factors associated with receiving antibiotic prescription

Surprisingly, receiving the GAS test has a small but insignificant impact on the likelihood that the patient will receive an antibiotic prescription (Table 3) (Adjusted OR = 1.055, P = .07). After controlling for receipt of a GAS test, children with AOM and sinusitis comorbidities have an increased likelihood of being prescribed an antibiotic. Children with URI have a lower likelihood of being prescribed an antibiotic. Additionally, relative to primary care physicians, children visiting nonphysician providers for pharyngitis were more likely to be prescribed an antibiotic.

Children under 12 years of age were more likely to use an antibiotic compared to children 12 years and older. Geographically, there is some evidence of regional variation in antibiotic use as well. Children in the south Atlantic, west-south central, and southeast central regions had a significantly lower odds of being prescribed an antibiotic respectively than pharyngitis patients in the Pacific region. Black children had a 10% lower likelihood of being prescribed an antibiotic compared to White children, possibly related to their lower rate of GAS testing. Compared to office-based visits, children visiting a clinic were less likely to use an antibiotic. Household income, insurance type, and season had no significant impact on revisit risk.

Effects of GAS test and antibiotic prescriptions on likelihood of revisits

The multivariate analysis of the risk of a revisit within 28 days is presented in Table 4. Children with pharyngitis who tested and did not receive an antibiotic serve as the reference comparison group for this analysis to illustrate the impact of using the GAS test and treatment with an antibiotic. The results in Table 4 are quite clear: patients who receive the GAS test were significantly less likely to have a revisit within 28 days. Moreover, within the group of patients who were tested, those not receiving an antibiotic, presumedly because their GAS test was negative, experienced the lowest risk of a revisit. This result is consistent with the data in Table 1. Moreover, using an antibiotic had no impact on the likelihood of a revisit in patients not receiving the GAS test. This result is also consistent with Table 1.

Other results from the analysis of revisit risk may be of interest to clinicians. Pharyngitis patients with a prior episode of treatment within 90 days for an acute respiratory tract infection were more than 7 times more likely to experience a revisit within 28 days of the pharyngitis diagnosis than patients without a history of recent ARTI infections. Age is also a risk factor in likelihood of initiating a revisit. Children under 1 year and children aged 1 to 5 years were more likely to have a revisit than children aged more than 12 years. Compared to White children, Black children were 25% (P = .04) less likely to have a revisit. The care setting also has a significant impact on revisit risk. Children visiting outpatient hospital and other care settings had a significantly higher revisit risk than those visiting a physician’s office. Lastly, household income, geographic region, season, medical comorbidities, gender, and insurance type have no significant impact on revisit risk.

Sensitivity analysis

The results from the analysis of 7-day and 28-day revisit risk are summarized in Table 5. These results indicate that patients who were tested had a more significant decrease in revisit risk at 7 days (72%) than was evident at 28 days (47% reduction). Receiving an antibiotic, with or without the test, had no impact on revisit risk.

Discussion

Published data on revisits for pharyngitis are lacking with the concentration of prior research focused more on systemic complications of undertreated GABHS disease or on identifying carrier status. Our study results suggest that GAS testing is the most important factor in reducing revisit risk. Being prescribed an antibiotic, on its own, does not have a significant impact on the risk of a revisit. However, once the GAS test is used, the decision not to use an antibiotic was correlated with the lowest revisit rate, likely because the source of the pharyngitis infection was viral and more likely to resolve without a revisit. Prior studies have reported variable rates of testing among children with pharyngitis prescribed an antibiotic, ranging from 23% to 91%,^14,15 with testing important toward more appropriate antibiotic use.¹⁶ More recently, among more than 67 000 patients aged 3 to 21 years presenting with sore throat and receiving a GAS test, 32.6% were positive.¹⁷

Our analysis found that more than 46% of pediatric pharyngitis patients were given the rapid GAS test. While this testing rate is substantially lower than HEDIS recommendations and lower than testing rates achieved by several health maintenance organizations,¹⁰ it is similar to the 53% of children receiving such testing in a recent National Ambulatory Medical Care Survey.¹⁸ Furthermore, we found that when antibiotics are prescribed following a GAS test, the revisit risk is not significantly reduced, possibly because antibiotics lower revisit risk when informed by diagnostic testing tools that determine the infectious organism. This is supported by a similar population analysis in which we observed reduced revisit rates in children with AOM managed with antibiotics within 3 days of index diagnosis.¹⁹

Several other factors also affect the likelihood of a child receiving the GAS test. Children aged 1 to 12 years were significantly more likely to receive the GAS test than children over the age of 12. This included children in the 1 to 5 years old bracket who had a 15% higher likelihood of undergoing a GAS test, despite children less than 3 years of age as not recommended targets for GAS testing.²⁰ As expected, children with reported ARTI-associated comorbidities were also less likely to receive a GAS test. Additionally, specialty care physicians were less inclined to implement the GAS test, possibly because of diagnostic confidence without testing or referral after GAS was ruled out. Black and low-income children had statistically lower odds of receiving the test, even after controlling for other factors, and yet were less likely to consume a revisit. As the overall data suggested more revisits in those not tested, further study is needed to examine if race or income discrepancies are equity based. Finally, children in the Pacific region, compared to the rest of the nation, were the least likely to receive a GAS test and yet there were no significant differences in revisit rates by region. Regional differences in antibiotic use were also observed in our study, as has been seen by others.²¹

After statistically controlling for having received the diagnostic GAS test and filled a prescription for an antibiotic, there are multitude of factors that independently affect the revisit risk, the most important of which if which was a history of an ARTI infection in the prior 90 days. While prior visit history had no impact on the likelihood of being tested or filling an antibiotic, patients with prior visits were more than 7 times more likely to consume a revisit. This was not reflected in nor related to comorbid ARTIs as these patients did not have statistically higher revisits than those with pharyngitis as the sole-coded diagnosis. Moreover, speculation for bacterial etiology of primary or superinfection based on a recent history of ARTI accounting for revisits seems unlikely as it did not yield greater antibiotic use in that group. Further analysis is required to determine the clinical and behavioral factors that promote for prior ARTI history as a major factor in revisit risk after an index visit for pharyngitis.

Children aged between 1 and 5 years, though 15% more likely to be tested than those aged 12 through 17 years, were also 39% more likely to initiate a revisit compared to older children when statistically controlling for other covariates. This perhaps suggests longer illness, wrong diagnosis, delay in appropriate treatment, or more caution by parents and providers in this age group. Justification for testing children less than 3 years of age who are outside of the HEDIS suggested age group, when clinical judgement does not point to another infection source, can result in positivity rates between 22% and 30% as previously observed.^22,23 Patients visiting nonphysician providers and outpatient facility providers were less likely to have a revisit than those visiting primary and specialty care physicians, though slightly higher propensity for antibiotic prescriptions was seen for nonphysician providers. Pediatricians have been noted to be less likely to prescribe antibiotics without GAS testing than nonpediatric providers, and more guidelines-compliant in prescribing.²⁴

Recommendations to not test children under 3 years of age are based on the lack of acute rheumatic fever and other complications in this age group together with more frequent viral syndromes. Selectivity in applying clinical criteria to testing can be attempted to separate bacterial from viral illness. Postnasal drainage/rhinorrhea, hoarse voice, and cough have been used successfully to identify those with viral illness and less need for testing, with greater certainty of low risk for GABHS in those over 11 years of age without tonsillar exudates, cervical adenopathy, or fever.¹⁷ However, the marginal benefits of those who have all 3 features of viral illness versus none in identifying GAS positivity was 23.3% vs 37.6% - helpful, but certainly not diminishing the need for testing. These constitutional findings of viral URI also do not exclude the GAS carrier state that features these symptoms.²⁵ Others have reinforced the doubt of pharyngeal exudates as the premier diagnostic finding for test-positive GAS.²⁶

This study had several limitations. The Optum claims dataset only contains ICD-9 codes for diagnoses. It does not include data on infection severity and clinical findings related to symptoms, thus empiric treatment warranted based in clinical severity is not assessed. Antibiotics are commonly available as generics and very inexpensive. Patients may fill and pay for these prescriptions directly, in which case, a claim for payment may not be filed with Optum. This could result in an undercount of treated patients in our study.

There is no corresponding problem of missing medical claims for GAS testing which were obtained from the CPT codes within the Optum claims data set. However, we elected not to verify the test results due to these data being missing for 75% of the study population. Nevertheless, this study’s focus was less about justifying antibiotic treatment, but dealt with the outcomes generated by testing and treatment. Toward that end, we used CPT codes to identify a revisit, and while those can at times be affected by financial reimbursement incentives, differences related to revisits in the 4 patient groups should not be subject to bias.

Conclusion

This study used data from real world practices to document the patterns of GAS testing and antibiotic use in pediatric pharyngitis patients. Revisit rates were under 5% for all patient groups and the use of rapid diagnostic tools were found to be the determining factor in further reducing the risk of revisits. This supports the need for compliance with the HEDIS quality metric for pharyngitis to the recommended levels of rapid testing which have been falling in recent years. Use of more accurate antigen and newer molecular detection testing methods may help further delineate important factors in determining pediatric pharyngitis treatment and need for revisits.²⁷

Corresponding author: Jeffrey McCombs, MD, University of Southern California School of Pharmacy, Department of Pharmaceutical and Health Economics, Leonard D. Schaeffer Center for Health Policy & Economics, 635 Downey Way, Verna & Peter Dauterive Hall 310, Los Angeles, CA 90089-3333; jmccombs@usc.edu.

Financial disclosures: None.

From the Department of Pharmaceutical and Health Economics, University of Southern California, Los Angeles, CA, (Drs. Sangha and McCombs), Department of Pediatrics, Keck School of Medicine, and Department of Clinical Pharmacy, School of Pharmacy, University of Southern California, Los Angeles, CA, (Dr. Steinberg), and Leonard Schaeffer Center for Health Policy and Economics, University of Southern California, Los Angeles, CA (Dr. McCombs).

Objective: The recommended treatment for children and adolescents under 18 years of age who have a positive test for group A Streptococcus (GAS) are antibiotics using the “test and treat” strategy to detect and treat GAS for pediatric pharyngitis. This study used paid claims data to document the extent to which real-world treatment patterns are consistent with these recommendations. We document the factors correlated with testing and treatment, then examine the effects of receiving a GAS test and being treated with an antibiotic impact the likelihood of a revisit for an acute respiratory tract infection within 28 days.

Methods: This retrospective cohort study used Optum Insight Clinformatics data for medical and pharmacy claims from 2011-2013 to identify episodes of care for children and adolescents with pharyngitis around their index visit (± 6 months). The sample population included children and adolescents under 18 years of age with a diagnosis of pharyngitis. Multivariable logistic regression analyses were used to document factors associated with receipt of GAS test and antibiotic treatment. Next, we used logistic regression models to estimate the impact of test and treat recommendation on revisit risk.

Results: There were 24 685 treatment episodes for children and adolescents diagnosed with pharyngitis. Nearly 47% of these episodes included a GAS test and 48% of tested patients were prescribed an antibiotic prescription. Failing to perform a GAS test increased the risk of a revisit within 28 days by 44%. The use of antibiotics by tested and untested patients had no impact on revisit risk.

Conclusion: While the judicious use of antibiotics is important in managing pharyngitis infections and managing complications, the use of rapid diagnostic tools was found to be the determining factor in reducing revisits for pediatric patients with pharyngitis.

Keywords: pediatrics; pharyngitis; respiratory infections; acute infections; diagnostic tests; group A Streptococcus; antibiotics; revisits.

Acute pharyngitis is a common acute respiratory tract infection (ARTI) in children. Group A β-hemolytic streptococci (GABHS) is the most common bacterial etiology for pediatric pharyngitis, accounting for 15% to 30% of cases.¹

Beyond clinical assessment, laboratory diagnostic testing generally plays a limited role in guiding appropriate antibiotic prescribing for patients with an ARTI.^2,3 Most diagnostic tests require 2 or 3 days to result, incur additional costs, and may delay treatment.⁴ While these tests do not provide clear and timely guidance on which specific antibiotic is appropriate for ARTI patients, this is not the case for patients with pharyngitis.^5,6,7 A rapid diagnostic test exists to identify pharyngitis patients with GABHS which accounts for 1 in 4 children with acute sore throat.^1,4,6 Both the American Academy of Pediatrics and the Infectious Diseases Society of America recommend antibiotic treatment for children and adolescents under 18 years of age who have a positive test for group A Streptococcus (GAS).^8,9 This “test and treat” protocol has been consistently included in the Healthcare Effectiveness Data and Information Set (HEDIS) standards over time for pediatric pharyngitis patients aged 3 to 18 years before dispensing an antibiotic.¹⁰

Sinusitis, pneumonia, and acute otitis media are considered ARTIs where antibiotic treatment is justified. Therefore, pharyngitis of unclear etiology seen with these comorbid infections may not always undergo GAS testing but move directly to the patient being prescribed antibiotics. This analysis enumerates ARTI-related comorbidities present together with the initial coded pharyngitis diagnosis to evaluate their impact on the provider’s decision to test and treat, and on revisit risk.

Antibiotic treatment for GAS patients is likely to eradicate the acute GABHS infection within 10 days. Penicillin and amoxicillin are commonly recommended because of their narrow spectrum of activity, few adverse effects, established efficacy, and modest cost. Alternative antibiotics for patients with penicillin allergy, or with polymicrobial infection seen on culture results, include a first-generation cephalosporin, clindamycin, clarithromycin (Biaxin), or azithromycin (Zithromax).^1,8,11 However, while compliance with these HEDIS guidelines has been evaluated, the outcome effects of following the HEDIS “test and treat” recommendations for children with pharyngitis have not been adequately evaluated.

These outcome evaluations have increasing importance as the latest HEDIS survey has shown testing rates in commercial Preferred Provider Organizations (PPO) falling from 86.4% in 2018 to 75.9% in 2019, the lowest rate of testing since 2009, with similar reductions under 80% for Health Maintenance Organizations (HMO).¹⁰ While health plans may execute cost-benefit analyses and algorithms to forge best practices for GAS testing in children and adolescents presenting with symptoms of pharyngitis, it is important to regard the wasteful resource utilization and additional cost of revisits that may offset any gains accrued by more focused GAS testing outside the existing clinical guidelines and HEDIS measures. This may be of particular importance in documenting infection and sparing antibiotic therapy in toddlers and younger.

The objective of this study was to investigate the correlation between testing and antibiotic use on the likelihood of a revisit for an acute respiratory tract infection within 28 days. To achieve this objective, this investigation consists of 3 sequential analyses. First, we document the factors associated with the decision to test the patient for a GABHS infection using the GAS test. Next, we document the factors associated with the decision to use an antibiotic to treat the patient as a function of having tested the patient. Finally, we investigate the impact of the testing and treatment decisions on the likelihood of a revisit within 28 days.

Methods

Study design

This was a retrospective cohort study of episodes of treatment for pediatric patients with pharyngitis. Episodes were identified using data derived from the Optum Insight Clinformatics claims database provided to the University of Southern California to facilitate the training of graduate students. These data cover commercially insured patients with both medical and pharmacy benefits. Data were retrieved from the 3-year period spanning 2011-2013. An episode of care was identified based on date of the first (index) outpatient visit for a pharyngitis diagnosis (International Classification of Diseases, Ninth Revision [ICD-9]: 462, 463, 034.0). Outpatient visits were defined by visit setting: ambulatory clinics, physician offices, emergency rooms, and urgent care facilities. Each pharyngitis treatment episode was then screened for at least a 6-month enrollment in a health insurance plan prior and subsequent to the index visit using Optum enrollment data. Finally, eligible treatment episodes were restricted to children and adolescents under 18 years of age, who had an index outpatient visit for a primary diagnosis of acute pharyngitis.

A diagnostic profile was created for each episode using the diagnoses recorded for the index visit. Up to 3 diagnoses may be recorded for any outpatient visit and the first recorded diagnosis was assumed to be the primary diagnosis for that episode. Any secondary diagnoses recorded on the index visit were used to define comorbidities present at the index visit. ARTI-related comorbidities included: acute otitis media (AOM), bronchitis, sinusitis, pneumonia, and upper respiratory infection (URI). Other comorbid medical diagnoses were documented using diagnostic data from the pre-index period. Dichotomous variables for the following categories were created: mental disorders, nervous system disorders, respiratory symptoms, fever, injury and poisoning, other, or no diseases.

Prior visits for other respiratory infections in the previous 90 days were also identified for patients based on their index visit for pharyngitis. Similarly, any subsequent visits, within 28 days of the index visit, were also recorded to measure the health outcome for analysis. Practice settings include physician offices and federally qualified health centers, state and local health clinics, outpatient hospitals facilities, emergency departments, and other outpatient settings such as walk-in retail health clinic or ambulatory centers. Providers include primary care physicians (family practice, pediatricians, internal medicine), specialty care physicians (emergency medicine, preventive medicine), nonphysician providers (nurse practitioners, physician assistants) and other providers (urgent care, acute outpatient care, ambulatory care centers). Seasons of the year were determined based on the index date of the episode to account for possible seasonality in pharyngitis treatment. Lastly, a previous visits variable was created to identify whether the child had nonpharyngitis ARTI visits in the 3 months prior to the index visit.

Demographic variables were created based on enrollment and the socioeconomic data available in the Optum socioeconomic status file. These variables include patient age, race, sex, household income, geographic location, practice setting type, provider specialty, and type of insurance. An estimate of patient household income was based on algorithms using census block groups. Income categories were informed by the federal guidelines for a family of 4. A low-income family was defined as earning less than $50 000; a middle-income family earned between $50 000 and $75 000, and a high-income family earned $75 000 and above.¹² Patient insurance type was categorized as HMO, Exclusive Provider Organization (EPO), Point of Service (POS), and PPO. Race was identified as White, Black, Hispanic, and Asian. Patient location was defined according to national census regions.

Outcomes

GAS test

The HEDIS measures for pharyngitis recommend using the GAS test to identify the bacterial etiology of the pharyngitis infection. Patients who received the test were identified based on Current Procedural Terminology (CPT) codes 87070-71, 87081, 87430, 87650-52, and 87880.¹⁰

The pharmacy administrative claims dataset was used to identify study patients who filled a prescription for an antibiotic during their pharyngitis treatment episode. Optum pharmacy data identify the medications received, specifies the date of prescription filling, National Drug Codes, and American Hospital Formulary Service (AHFS) Classification System codes for each medication. We used the AHFS Pharmacologic-Therapeutic classification of antibiotics to create dichotomous variables documenting the antibacterial used by each patient.¹³ These are categorized under antibacterial including penicillins, cephalosporins (first, second, third, fourth generation cephalosporins), macrolides (first generation and others), tetracyclines, sulfonamides, fluoroquinolones (ciprofloxacin, levofloxacin, moxifloxacin), cephamycin, carbapenems, and β-lactam antibiotics (amoxicillin, amoxicillin/clavulanate, cephalexin, cefuroxime, cefdinir).

Revisits to physician or other provider

Revisits within 28 days were used as the measure of patient outcomes related to testing and filling of an antibiotic prescription for acute pharyngitis. Revisits may also be due to a patient returning for a follow-up, alternative treatment, worsening pharyngitis, or for another ARTI. An ARTI-related revisit also increases total resources used to treat pediatric pharyngitis patients.

Statistical analysis

Logistic regression was used for all 3 analyses conducted in this study. First, we determined the patient and treating physician characteristics that impact the decision to use GAS testing for pharyngitis. Second, we identified those factors that impact the decision to use antibiotic prescriptions among children who were diagnosed with pharyngitis adding in the dichotomous variable indicating if the patient had received a GAS test. Third, we used a logit regression analysis to document if receiving a GAS test and/or an antibiotic impacted the likelihood of a revisit by comparing revisit risk. To estimate the effect of testing and/or antibiotic use, we divided patients into 4 groups based on whether the patient received a GAS test and/or an antibiotic prescription. This specification of the analysis of revisits as an outcome focuses on adherence to HEDIS “test and treat” guidelines¹⁰:

1. Patients who were not tested yet filled an antibiotic prescription. This decision was likely based on the clinician’s judgment of the patient’s signs and symptoms, and confirmational testing not performed.
2. Patients who were not tested and did not fill an antibiotic prescription. Apparently, in the clinician’s judgment the patient’s signs and symptoms were such that the infection did not warrant treatment and the clinical presentation did not necessitate the GAS test to confirm the recorded diagnosis of pharyngitis.
3. Patients who were tested and received antibiotic prescription, likely because the test was positive for GABHS.
4. Patients who were tested and did not receive antibiotic prescription.

We tested for statistically significant differences in baseline characteristics across these 4 patient groups using t tests for continuous variables and χ² tests for categorical variables. Odds ratios (OR) and CI were computed for the influential variables included the regression analyses.

We conducted a sensitivity analysis using a model specification which included the dichotomous variables for testing and for treatment, and the interaction term between these variables to assess if treatment effects varied in tested and untested patients. We also estimated this model of revisit risk using revisits within 7 days as the outcome variable.

All analyses were completed using STATA/IC 13 (StataCorp, College Station, TX).

Results

There were 24 685 treatment episodes for children diagnosed with pharyngitis. Nearly 47% of these episodes included GAS testing and 47% of the tested patients filled an antibiotic prescription. Similarly, 53% of patients were not tested and 49% of untested patients filled an antibiotic prescription. As a result, the 4 groups identified for analysis were evenly distributed: untested and no prescription (26.9%), untested and prescription (26.3%), tested and prescription (21.9%), and tested and no prescription (24.9%) (Figure).

Table 1 presents the descriptive statistics for these 4 patient groups. Note first that the rate of revisits within 28 days is under 5% across all groups. Second, the 2 tested groups have a lower revisit rate than the untested groups: the tested and treated have a revisit rate of 3.3%, and the tested and untreated have a revisit rate of 2.4%, while both the untested groups have a revisit rate of nearly 5%. These small absolute differences in revisit rates across groups were statistically significant.

Factors associated with receiving GAS test

Several factors were found to impact the decision to test (Table 2). Only 9.7% of children were reported to have any ARTI coinfection. As expected, these comorbidities resulted in a significantly lower likelihood of receiving the GAS test: AOM, bronchitis, sinusitis, pneumonia, and URI as comorbid infections had a 48%, 41%, 37%, 63%, and 13% lower likelihood of receiving the GAS test, respectively, than those with no comorbidities. Similarly, children with fever and respiratory symptoms were 35% and 45% less likely to receiving the GAS test, respectively. This is consistent with our expectation that comorbid ARTI infections will lead many providers to forgo testing.

Provider type and patient age also plays a role in receipt of the GAS test. Relative to outpatient facility providers, primary care physicians were 24% more likely and specialty physicians were 38% less likely of employing the GAS test. The child’s age played a significant role in receipt of the GAS test. Children aged 1 to 5 years and 5 to 12 years were 15% and 14% more likely to receive the test compared to children older than 12 years.

Pharyngitis patients have disproportionately higher odds of receiving a GAS test in most regions of the country compared to the Pacific region. For instance, children in the Mid-Atlantic region have 51% higher odds of receiving a GAS test while children in New England have 80% higher odds of receiving the same test.

Black children have 11% lower odds of receiving the GAS test compared to White children. Both middle-income and high-income children have 12% and 32% higher odds of receiving the test compared to low-income children. Compared to office-based visits, children visiting a clinic were twice as likely to receive a GAS test while those seen in the emergency room have 43% lower odds of receiving a GAS test. Hospital outpatient departments, which account for less than 1% of all visits, rarely used a GAS test which could be a statistical artifact due to small sample size. Lastly, insurance and season of the year had no significant impact of receipt of a GAS test.

Factors associated with receiving antibiotic prescription

Surprisingly, receiving the GAS test has a small but insignificant impact on the likelihood that the patient will receive an antibiotic prescription (Table 3) (Adjusted OR = 1.055, P = .07). After controlling for receipt of a GAS test, children with AOM and sinusitis comorbidities have an increased likelihood of being prescribed an antibiotic. Children with URI have a lower likelihood of being prescribed an antibiotic. Additionally, relative to primary care physicians, children visiting nonphysician providers for pharyngitis were more likely to be prescribed an antibiotic.

Children under 12 years of age were more likely to use an antibiotic compared to children 12 years and older. Geographically, there is some evidence of regional variation in antibiotic use as well. Children in the south Atlantic, west-south central, and southeast central regions had a significantly lower odds of being prescribed an antibiotic respectively than pharyngitis patients in the Pacific region. Black children had a 10% lower likelihood of being prescribed an antibiotic compared to White children, possibly related to their lower rate of GAS testing. Compared to office-based visits, children visiting a clinic were less likely to use an antibiotic. Household income, insurance type, and season had no significant impact on revisit risk.

Effects of GAS test and antibiotic prescriptions on likelihood of revisits

The multivariate analysis of the risk of a revisit within 28 days is presented in Table 4. Children with pharyngitis who tested and did not receive an antibiotic serve as the reference comparison group for this analysis to illustrate the impact of using the GAS test and treatment with an antibiotic. The results in Table 4 are quite clear: patients who receive the GAS test were significantly less likely to have a revisit within 28 days. Moreover, within the group of patients who were tested, those not receiving an antibiotic, presumedly because their GAS test was negative, experienced the lowest risk of a revisit. This result is consistent with the data in Table 1. Moreover, using an antibiotic had no impact on the likelihood of a revisit in patients not receiving the GAS test. This result is also consistent with Table 1.

Other results from the analysis of revisit risk may be of interest to clinicians. Pharyngitis patients with a prior episode of treatment within 90 days for an acute respiratory tract infection were more than 7 times more likely to experience a revisit within 28 days of the pharyngitis diagnosis than patients without a history of recent ARTI infections. Age is also a risk factor in likelihood of initiating a revisit. Children under 1 year and children aged 1 to 5 years were more likely to have a revisit than children aged more than 12 years. Compared to White children, Black children were 25% (P = .04) less likely to have a revisit. The care setting also has a significant impact on revisit risk. Children visiting outpatient hospital and other care settings had a significantly higher revisit risk than those visiting a physician’s office. Lastly, household income, geographic region, season, medical comorbidities, gender, and insurance type have no significant impact on revisit risk.

Sensitivity analysis

The results from the analysis of 7-day and 28-day revisit risk are summarized in Table 5. These results indicate that patients who were tested had a more significant decrease in revisit risk at 7 days (72%) than was evident at 28 days (47% reduction). Receiving an antibiotic, with or without the test, had no impact on revisit risk.

Discussion

Published data on revisits for pharyngitis are lacking with the concentration of prior research focused more on systemic complications of undertreated GABHS disease or on identifying carrier status. Our study results suggest that GAS testing is the most important factor in reducing revisit risk. Being prescribed an antibiotic, on its own, does not have a significant impact on the risk of a revisit. However, once the GAS test is used, the decision not to use an antibiotic was correlated with the lowest revisit rate, likely because the source of the pharyngitis infection was viral and more likely to resolve without a revisit. Prior studies have reported variable rates of testing among children with pharyngitis prescribed an antibiotic, ranging from 23% to 91%,^14,15 with testing important toward more appropriate antibiotic use.¹⁶ More recently, among more than 67 000 patients aged 3 to 21 years presenting with sore throat and receiving a GAS test, 32.6% were positive.¹⁷

Our analysis found that more than 46% of pediatric pharyngitis patients were given the rapid GAS test. While this testing rate is substantially lower than HEDIS recommendations and lower than testing rates achieved by several health maintenance organizations,¹⁰ it is similar to the 53% of children receiving such testing in a recent National Ambulatory Medical Care Survey.¹⁸ Furthermore, we found that when antibiotics are prescribed following a GAS test, the revisit risk is not significantly reduced, possibly because antibiotics lower revisit risk when informed by diagnostic testing tools that determine the infectious organism. This is supported by a similar population analysis in which we observed reduced revisit rates in children with AOM managed with antibiotics within 3 days of index diagnosis.¹⁹

Several other factors also affect the likelihood of a child receiving the GAS test. Children aged 1 to 12 years were significantly more likely to receive the GAS test than children over the age of 12. This included children in the 1 to 5 years old bracket who had a 15% higher likelihood of undergoing a GAS test, despite children less than 3 years of age as not recommended targets for GAS testing.²⁰ As expected, children with reported ARTI-associated comorbidities were also less likely to receive a GAS test. Additionally, specialty care physicians were less inclined to implement the GAS test, possibly because of diagnostic confidence without testing or referral after GAS was ruled out. Black and low-income children had statistically lower odds of receiving the test, even after controlling for other factors, and yet were less likely to consume a revisit. As the overall data suggested more revisits in those not tested, further study is needed to examine if race or income discrepancies are equity based. Finally, children in the Pacific region, compared to the rest of the nation, were the least likely to receive a GAS test and yet there were no significant differences in revisit rates by region. Regional differences in antibiotic use were also observed in our study, as has been seen by others.²¹

After statistically controlling for having received the diagnostic GAS test and filled a prescription for an antibiotic, there are multitude of factors that independently affect the revisit risk, the most important of which if which was a history of an ARTI infection in the prior 90 days. While prior visit history had no impact on the likelihood of being tested or filling an antibiotic, patients with prior visits were more than 7 times more likely to consume a revisit. This was not reflected in nor related to comorbid ARTIs as these patients did not have statistically higher revisits than those with pharyngitis as the sole-coded diagnosis. Moreover, speculation for bacterial etiology of primary or superinfection based on a recent history of ARTI accounting for revisits seems unlikely as it did not yield greater antibiotic use in that group. Further analysis is required to determine the clinical and behavioral factors that promote for prior ARTI history as a major factor in revisit risk after an index visit for pharyngitis.

Children aged between 1 and 5 years, though 15% more likely to be tested than those aged 12 through 17 years, were also 39% more likely to initiate a revisit compared to older children when statistically controlling for other covariates. This perhaps suggests longer illness, wrong diagnosis, delay in appropriate treatment, or more caution by parents and providers in this age group. Justification for testing children less than 3 years of age who are outside of the HEDIS suggested age group, when clinical judgement does not point to another infection source, can result in positivity rates between 22% and 30% as previously observed.^22,23 Patients visiting nonphysician providers and outpatient facility providers were less likely to have a revisit than those visiting primary and specialty care physicians, though slightly higher propensity for antibiotic prescriptions was seen for nonphysician providers. Pediatricians have been noted to be less likely to prescribe antibiotics without GAS testing than nonpediatric providers, and more guidelines-compliant in prescribing.²⁴

Recommendations to not test children under 3 years of age are based on the lack of acute rheumatic fever and other complications in this age group together with more frequent viral syndromes. Selectivity in applying clinical criteria to testing can be attempted to separate bacterial from viral illness. Postnasal drainage/rhinorrhea, hoarse voice, and cough have been used successfully to identify those with viral illness and less need for testing, with greater certainty of low risk for GABHS in those over 11 years of age without tonsillar exudates, cervical adenopathy, or fever.¹⁷ However, the marginal benefits of those who have all 3 features of viral illness versus none in identifying GAS positivity was 23.3% vs 37.6% - helpful, but certainly not diminishing the need for testing. These constitutional findings of viral URI also do not exclude the GAS carrier state that features these symptoms.²⁵ Others have reinforced the doubt of pharyngeal exudates as the premier diagnostic finding for test-positive GAS.²⁶

This study had several limitations. The Optum claims dataset only contains ICD-9 codes for diagnoses. It does not include data on infection severity and clinical findings related to symptoms, thus empiric treatment warranted based in clinical severity is not assessed. Antibiotics are commonly available as generics and very inexpensive. Patients may fill and pay for these prescriptions directly, in which case, a claim for payment may not be filed with Optum. This could result in an undercount of treated patients in our study.

There is no corresponding problem of missing medical claims for GAS testing which were obtained from the CPT codes within the Optum claims data set. However, we elected not to verify the test results due to these data being missing for 75% of the study population. Nevertheless, this study’s focus was less about justifying antibiotic treatment, but dealt with the outcomes generated by testing and treatment. Toward that end, we used CPT codes to identify a revisit, and while those can at times be affected by financial reimbursement incentives, differences related to revisits in the 4 patient groups should not be subject to bias.

Conclusion

This study used data from real world practices to document the patterns of GAS testing and antibiotic use in pediatric pharyngitis patients. Revisit rates were under 5% for all patient groups and the use of rapid diagnostic tools were found to be the determining factor in further reducing the risk of revisits. This supports the need for compliance with the HEDIS quality metric for pharyngitis to the recommended levels of rapid testing which have been falling in recent years. Use of more accurate antigen and newer molecular detection testing methods may help further delineate important factors in determining pediatric pharyngitis treatment and need for revisits.²⁷

Corresponding author: Jeffrey McCombs, MD, University of Southern California School of Pharmacy, Department of Pharmaceutical and Health Economics, Leonard D. Schaeffer Center for Health Policy & Economics, 635 Downey Way, Verna & Peter Dauterive Hall 310, Los Angeles, CA 90089-3333; jmccombs@usc.edu.

Financial disclosures: None.

From the Department of Pharmaceutical and Health Economics, University of Southern California, Los Angeles, CA, (Drs. Sangha and McCombs), Department of Pediatrics, Keck School of Medicine, and Department of Clinical Pharmacy, School of Pharmacy, University of Southern California, Los Angeles, CA, (Dr. Steinberg), and Leonard Schaeffer Center for Health Policy and Economics, University of Southern California, Los Angeles, CA (Dr. McCombs).

Objective: The recommended treatment for children and adolescents under 18 years of age who have a positive test for group A Streptococcus (GAS) are antibiotics using the “test and treat” strategy to detect and treat GAS for pediatric pharyngitis. This study used paid claims data to document the extent to which real-world treatment patterns are consistent with these recommendations. We document the factors correlated with testing and treatment, then examine the effects of receiving a GAS test and being treated with an antibiotic impact the likelihood of a revisit for an acute respiratory tract infection within 28 days.

Methods: This retrospective cohort study used Optum Insight Clinformatics data for medical and pharmacy claims from 2011-2013 to identify episodes of care for children and adolescents with pharyngitis around their index visit (± 6 months). The sample population included children and adolescents under 18 years of age with a diagnosis of pharyngitis. Multivariable logistic regression analyses were used to document factors associated with receipt of GAS test and antibiotic treatment. Next, we used logistic regression models to estimate the impact of test and treat recommendation on revisit risk.

Results: There were 24 685 treatment episodes for children and adolescents diagnosed with pharyngitis. Nearly 47% of these episodes included a GAS test and 48% of tested patients were prescribed an antibiotic prescription. Failing to perform a GAS test increased the risk of a revisit within 28 days by 44%. The use of antibiotics by tested and untested patients had no impact on revisit risk.

Conclusion: While the judicious use of antibiotics is important in managing pharyngitis infections and managing complications, the use of rapid diagnostic tools was found to be the determining factor in reducing revisits for pediatric patients with pharyngitis.

Keywords: pediatrics; pharyngitis; respiratory infections; acute infections; diagnostic tests; group A Streptococcus; antibiotics; revisits.

Acute pharyngitis is a common acute respiratory tract infection (ARTI) in children. Group A β-hemolytic streptococci (GABHS) is the most common bacterial etiology for pediatric pharyngitis, accounting for 15% to 30% of cases.¹

Beyond clinical assessment, laboratory diagnostic testing generally plays a limited role in guiding appropriate antibiotic prescribing for patients with an ARTI.^2,3 Most diagnostic tests require 2 or 3 days to result, incur additional costs, and may delay treatment.⁴ While these tests do not provide clear and timely guidance on which specific antibiotic is appropriate for ARTI patients, this is not the case for patients with pharyngitis.^5,6,7 A rapid diagnostic test exists to identify pharyngitis patients with GABHS which accounts for 1 in 4 children with acute sore throat.^1,4,6 Both the American Academy of Pediatrics and the Infectious Diseases Society of America recommend antibiotic treatment for children and adolescents under 18 years of age who have a positive test for group A Streptococcus (GAS).^8,9 This “test and treat” protocol has been consistently included in the Healthcare Effectiveness Data and Information Set (HEDIS) standards over time for pediatric pharyngitis patients aged 3 to 18 years before dispensing an antibiotic.¹⁰

Sinusitis, pneumonia, and acute otitis media are considered ARTIs where antibiotic treatment is justified. Therefore, pharyngitis of unclear etiology seen with these comorbid infections may not always undergo GAS testing but move directly to the patient being prescribed antibiotics. This analysis enumerates ARTI-related comorbidities present together with the initial coded pharyngitis diagnosis to evaluate their impact on the provider’s decision to test and treat, and on revisit risk.

Antibiotic treatment for GAS patients is likely to eradicate the acute GABHS infection within 10 days. Penicillin and amoxicillin are commonly recommended because of their narrow spectrum of activity, few adverse effects, established efficacy, and modest cost. Alternative antibiotics for patients with penicillin allergy, or with polymicrobial infection seen on culture results, include a first-generation cephalosporin, clindamycin, clarithromycin (Biaxin), or azithromycin (Zithromax).^1,8,11 However, while compliance with these HEDIS guidelines has been evaluated, the outcome effects of following the HEDIS “test and treat” recommendations for children with pharyngitis have not been adequately evaluated.

These outcome evaluations have increasing importance as the latest HEDIS survey has shown testing rates in commercial Preferred Provider Organizations (PPO) falling from 86.4% in 2018 to 75.9% in 2019, the lowest rate of testing since 2009, with similar reductions under 80% for Health Maintenance Organizations (HMO).¹⁰ While health plans may execute cost-benefit analyses and algorithms to forge best practices for GAS testing in children and adolescents presenting with symptoms of pharyngitis, it is important to regard the wasteful resource utilization and additional cost of revisits that may offset any gains accrued by more focused GAS testing outside the existing clinical guidelines and HEDIS measures. This may be of particular importance in documenting infection and sparing antibiotic therapy in toddlers and younger.

The objective of this study was to investigate the correlation between testing and antibiotic use on the likelihood of a revisit for an acute respiratory tract infection within 28 days. To achieve this objective, this investigation consists of 3 sequential analyses. First, we document the factors associated with the decision to test the patient for a GABHS infection using the GAS test. Next, we document the factors associated with the decision to use an antibiotic to treat the patient as a function of having tested the patient. Finally, we investigate the impact of the testing and treatment decisions on the likelihood of a revisit within 28 days.

Methods

Study design

This was a retrospective cohort study of episodes of treatment for pediatric patients with pharyngitis. Episodes were identified using data derived from the Optum Insight Clinformatics claims database provided to the University of Southern California to facilitate the training of graduate students. These data cover commercially insured patients with both medical and pharmacy benefits. Data were retrieved from the 3-year period spanning 2011-2013. An episode of care was identified based on date of the first (index) outpatient visit for a pharyngitis diagnosis (International Classification of Diseases, Ninth Revision [ICD-9]: 462, 463, 034.0). Outpatient visits were defined by visit setting: ambulatory clinics, physician offices, emergency rooms, and urgent care facilities. Each pharyngitis treatment episode was then screened for at least a 6-month enrollment in a health insurance plan prior and subsequent to the index visit using Optum enrollment data. Finally, eligible treatment episodes were restricted to children and adolescents under 18 years of age, who had an index outpatient visit for a primary diagnosis of acute pharyngitis.

A diagnostic profile was created for each episode using the diagnoses recorded for the index visit. Up to 3 diagnoses may be recorded for any outpatient visit and the first recorded diagnosis was assumed to be the primary diagnosis for that episode. Any secondary diagnoses recorded on the index visit were used to define comorbidities present at the index visit. ARTI-related comorbidities included: acute otitis media (AOM), bronchitis, sinusitis, pneumonia, and upper respiratory infection (URI). Other comorbid medical diagnoses were documented using diagnostic data from the pre-index period. Dichotomous variables for the following categories were created: mental disorders, nervous system disorders, respiratory symptoms, fever, injury and poisoning, other, or no diseases.

Prior visits for other respiratory infections in the previous 90 days were also identified for patients based on their index visit for pharyngitis. Similarly, any subsequent visits, within 28 days of the index visit, were also recorded to measure the health outcome for analysis. Practice settings include physician offices and federally qualified health centers, state and local health clinics, outpatient hospitals facilities, emergency departments, and other outpatient settings such as walk-in retail health clinic or ambulatory centers. Providers include primary care physicians (family practice, pediatricians, internal medicine), specialty care physicians (emergency medicine, preventive medicine), nonphysician providers (nurse practitioners, physician assistants) and other providers (urgent care, acute outpatient care, ambulatory care centers). Seasons of the year were determined based on the index date of the episode to account for possible seasonality in pharyngitis treatment. Lastly, a previous visits variable was created to identify whether the child had nonpharyngitis ARTI visits in the 3 months prior to the index visit.

Demographic variables were created based on enrollment and the socioeconomic data available in the Optum socioeconomic status file. These variables include patient age, race, sex, household income, geographic location, practice setting type, provider specialty, and type of insurance. An estimate of patient household income was based on algorithms using census block groups. Income categories were informed by the federal guidelines for a family of 4. A low-income family was defined as earning less than $50 000; a middle-income family earned between $50 000 and $75 000, and a high-income family earned $75 000 and above.¹² Patient insurance type was categorized as HMO, Exclusive Provider Organization (EPO), Point of Service (POS), and PPO. Race was identified as White, Black, Hispanic, and Asian. Patient location was defined according to national census regions.

Outcomes

GAS test

The HEDIS measures for pharyngitis recommend using the GAS test to identify the bacterial etiology of the pharyngitis infection. Patients who received the test were identified based on Current Procedural Terminology (CPT) codes 87070-71, 87081, 87430, 87650-52, and 87880.¹⁰

The pharmacy administrative claims dataset was used to identify study patients who filled a prescription for an antibiotic during their pharyngitis treatment episode. Optum pharmacy data identify the medications received, specifies the date of prescription filling, National Drug Codes, and American Hospital Formulary Service (AHFS) Classification System codes for each medication. We used the AHFS Pharmacologic-Therapeutic classification of antibiotics to create dichotomous variables documenting the antibacterial used by each patient.¹³ These are categorized under antibacterial including penicillins, cephalosporins (first, second, third, fourth generation cephalosporins), macrolides (first generation and others), tetracyclines, sulfonamides, fluoroquinolones (ciprofloxacin, levofloxacin, moxifloxacin), cephamycin, carbapenems, and β-lactam antibiotics (amoxicillin, amoxicillin/clavulanate, cephalexin, cefuroxime, cefdinir).

Revisits to physician or other provider

Revisits within 28 days were used as the measure of patient outcomes related to testing and filling of an antibiotic prescription for acute pharyngitis. Revisits may also be due to a patient returning for a follow-up, alternative treatment, worsening pharyngitis, or for another ARTI. An ARTI-related revisit also increases total resources used to treat pediatric pharyngitis patients.

Statistical analysis

Logistic regression was used for all 3 analyses conducted in this study. First, we determined the patient and treating physician characteristics that impact the decision to use GAS testing for pharyngitis. Second, we identified those factors that impact the decision to use antibiotic prescriptions among children who were diagnosed with pharyngitis adding in the dichotomous variable indicating if the patient had received a GAS test. Third, we used a logit regression analysis to document if receiving a GAS test and/or an antibiotic impacted the likelihood of a revisit by comparing revisit risk. To estimate the effect of testing and/or antibiotic use, we divided patients into 4 groups based on whether the patient received a GAS test and/or an antibiotic prescription. This specification of the analysis of revisits as an outcome focuses on adherence to HEDIS “test and treat” guidelines¹⁰:

1. Patients who were not tested yet filled an antibiotic prescription. This decision was likely based on the clinician’s judgment of the patient’s signs and symptoms, and confirmational testing not performed.
2. Patients who were not tested and did not fill an antibiotic prescription. Apparently, in the clinician’s judgment the patient’s signs and symptoms were such that the infection did not warrant treatment and the clinical presentation did not necessitate the GAS test to confirm the recorded diagnosis of pharyngitis.
3. Patients who were tested and received antibiotic prescription, likely because the test was positive for GABHS.
4. Patients who were tested and did not receive antibiotic prescription.

We tested for statistically significant differences in baseline characteristics across these 4 patient groups using t tests for continuous variables and χ² tests for categorical variables. Odds ratios (OR) and CI were computed for the influential variables included the regression analyses.

We conducted a sensitivity analysis using a model specification which included the dichotomous variables for testing and for treatment, and the interaction term between these variables to assess if treatment effects varied in tested and untested patients. We also estimated this model of revisit risk using revisits within 7 days as the outcome variable.

All analyses were completed using STATA/IC 13 (StataCorp, College Station, TX).

Results

There were 24 685 treatment episodes for children diagnosed with pharyngitis. Nearly 47% of these episodes included GAS testing and 47% of the tested patients filled an antibiotic prescription. Similarly, 53% of patients were not tested and 49% of untested patients filled an antibiotic prescription. As a result, the 4 groups identified for analysis were evenly distributed: untested and no prescription (26.9%), untested and prescription (26.3%), tested and prescription (21.9%), and tested and no prescription (24.9%) (Figure).

Table 1 presents the descriptive statistics for these 4 patient groups. Note first that the rate of revisits within 28 days is under 5% across all groups. Second, the 2 tested groups have a lower revisit rate than the untested groups: the tested and treated have a revisit rate of 3.3%, and the tested and untreated have a revisit rate of 2.4%, while both the untested groups have a revisit rate of nearly 5%. These small absolute differences in revisit rates across groups were statistically significant.

Factors associated with receiving GAS test

Several factors were found to impact the decision to test (Table 2). Only 9.7% of children were reported to have any ARTI coinfection. As expected, these comorbidities resulted in a significantly lower likelihood of receiving the GAS test: AOM, bronchitis, sinusitis, pneumonia, and URI as comorbid infections had a 48%, 41%, 37%, 63%, and 13% lower likelihood of receiving the GAS test, respectively, than those with no comorbidities. Similarly, children with fever and respiratory symptoms were 35% and 45% less likely to receiving the GAS test, respectively. This is consistent with our expectation that comorbid ARTI infections will lead many providers to forgo testing.

Provider type and patient age also plays a role in receipt of the GAS test. Relative to outpatient facility providers, primary care physicians were 24% more likely and specialty physicians were 38% less likely of employing the GAS test. The child’s age played a significant role in receipt of the GAS test. Children aged 1 to 5 years and 5 to 12 years were 15% and 14% more likely to receive the test compared to children older than 12 years.

Pharyngitis patients have disproportionately higher odds of receiving a GAS test in most regions of the country compared to the Pacific region. For instance, children in the Mid-Atlantic region have 51% higher odds of receiving a GAS test while children in New England have 80% higher odds of receiving the same test.

Black children have 11% lower odds of receiving the GAS test compared to White children. Both middle-income and high-income children have 12% and 32% higher odds of receiving the test compared to low-income children. Compared to office-based visits, children visiting a clinic were twice as likely to receive a GAS test while those seen in the emergency room have 43% lower odds of receiving a GAS test. Hospital outpatient departments, which account for less than 1% of all visits, rarely used a GAS test which could be a statistical artifact due to small sample size. Lastly, insurance and season of the year had no significant impact of receipt of a GAS test.

Factors associated with receiving antibiotic prescription

Surprisingly, receiving the GAS test has a small but insignificant impact on the likelihood that the patient will receive an antibiotic prescription (Table 3) (Adjusted OR = 1.055, P = .07). After controlling for receipt of a GAS test, children with AOM and sinusitis comorbidities have an increased likelihood of being prescribed an antibiotic. Children with URI have a lower likelihood of being prescribed an antibiotic. Additionally, relative to primary care physicians, children visiting nonphysician providers for pharyngitis were more likely to be prescribed an antibiotic.

Children under 12 years of age were more likely to use an antibiotic compared to children 12 years and older. Geographically, there is some evidence of regional variation in antibiotic use as well. Children in the south Atlantic, west-south central, and southeast central regions had a significantly lower odds of being prescribed an antibiotic respectively than pharyngitis patients in the Pacific region. Black children had a 10% lower likelihood of being prescribed an antibiotic compared to White children, possibly related to their lower rate of GAS testing. Compared to office-based visits, children visiting a clinic were less likely to use an antibiotic. Household income, insurance type, and season had no significant impact on revisit risk.

Effects of GAS test and antibiotic prescriptions on likelihood of revisits

The multivariate analysis of the risk of a revisit within 28 days is presented in Table 4. Children with pharyngitis who tested and did not receive an antibiotic serve as the reference comparison group for this analysis to illustrate the impact of using the GAS test and treatment with an antibiotic. The results in Table 4 are quite clear: patients who receive the GAS test were significantly less likely to have a revisit within 28 days. Moreover, within the group of patients who were tested, those not receiving an antibiotic, presumedly because their GAS test was negative, experienced the lowest risk of a revisit. This result is consistent with the data in Table 1. Moreover, using an antibiotic had no impact on the likelihood of a revisit in patients not receiving the GAS test. This result is also consistent with Table 1.

Other results from the analysis of revisit risk may be of interest to clinicians. Pharyngitis patients with a prior episode of treatment within 90 days for an acute respiratory tract infection were more than 7 times more likely to experience a revisit within 28 days of the pharyngitis diagnosis than patients without a history of recent ARTI infections. Age is also a risk factor in likelihood of initiating a revisit. Children under 1 year and children aged 1 to 5 years were more likely to have a revisit than children aged more than 12 years. Compared to White children, Black children were 25% (P = .04) less likely to have a revisit. The care setting also has a significant impact on revisit risk. Children visiting outpatient hospital and other care settings had a significantly higher revisit risk than those visiting a physician’s office. Lastly, household income, geographic region, season, medical comorbidities, gender, and insurance type have no significant impact on revisit risk.

Sensitivity analysis

The results from the analysis of 7-day and 28-day revisit risk are summarized in Table 5. These results indicate that patients who were tested had a more significant decrease in revisit risk at 7 days (72%) than was evident at 28 days (47% reduction). Receiving an antibiotic, with or without the test, had no impact on revisit risk.

Discussion

Published data on revisits for pharyngitis are lacking with the concentration of prior research focused more on systemic complications of undertreated GABHS disease or on identifying carrier status. Our study results suggest that GAS testing is the most important factor in reducing revisit risk. Being prescribed an antibiotic, on its own, does not have a significant impact on the risk of a revisit. However, once the GAS test is used, the decision not to use an antibiotic was correlated with the lowest revisit rate, likely because the source of the pharyngitis infection was viral and more likely to resolve without a revisit. Prior studies have reported variable rates of testing among children with pharyngitis prescribed an antibiotic, ranging from 23% to 91%,^14,15 with testing important toward more appropriate antibiotic use.¹⁶ More recently, among more than 67 000 patients aged 3 to 21 years presenting with sore throat and receiving a GAS test, 32.6% were positive.¹⁷

Our analysis found that more than 46% of pediatric pharyngitis patients were given the rapid GAS test. While this testing rate is substantially lower than HEDIS recommendations and lower than testing rates achieved by several health maintenance organizations,¹⁰ it is similar to the 53% of children receiving such testing in a recent National Ambulatory Medical Care Survey.¹⁸ Furthermore, we found that when antibiotics are prescribed following a GAS test, the revisit risk is not significantly reduced, possibly because antibiotics lower revisit risk when informed by diagnostic testing tools that determine the infectious organism. This is supported by a similar population analysis in which we observed reduced revisit rates in children with AOM managed with antibiotics within 3 days of index diagnosis.¹⁹

Several other factors also affect the likelihood of a child receiving the GAS test. Children aged 1 to 12 years were significantly more likely to receive the GAS test than children over the age of 12. This included children in the 1 to 5 years old bracket who had a 15% higher likelihood of undergoing a GAS test, despite children less than 3 years of age as not recommended targets for GAS testing.²⁰ As expected, children with reported ARTI-associated comorbidities were also less likely to receive a GAS test. Additionally, specialty care physicians were less inclined to implement the GAS test, possibly because of diagnostic confidence without testing or referral after GAS was ruled out. Black and low-income children had statistically lower odds of receiving the test, even after controlling for other factors, and yet were less likely to consume a revisit. As the overall data suggested more revisits in those not tested, further study is needed to examine if race or income discrepancies are equity based. Finally, children in the Pacific region, compared to the rest of the nation, were the least likely to receive a GAS test and yet there were no significant differences in revisit rates by region. Regional differences in antibiotic use were also observed in our study, as has been seen by others.²¹

After statistically controlling for having received the diagnostic GAS test and filled a prescription for an antibiotic, there are multitude of factors that independently affect the revisit risk, the most important of which if which was a history of an ARTI infection in the prior 90 days. While prior visit history had no impact on the likelihood of being tested or filling an antibiotic, patients with prior visits were more than 7 times more likely to consume a revisit. This was not reflected in nor related to comorbid ARTIs as these patients did not have statistically higher revisits than those with pharyngitis as the sole-coded diagnosis. Moreover, speculation for bacterial etiology of primary or superinfection based on a recent history of ARTI accounting for revisits seems unlikely as it did not yield greater antibiotic use in that group. Further analysis is required to determine the clinical and behavioral factors that promote for prior ARTI history as a major factor in revisit risk after an index visit for pharyngitis.

Children aged between 1 and 5 years, though 15% more likely to be tested than those aged 12 through 17 years, were also 39% more likely to initiate a revisit compared to older children when statistically controlling for other covariates. This perhaps suggests longer illness, wrong diagnosis, delay in appropriate treatment, or more caution by parents and providers in this age group. Justification for testing children less than 3 years of age who are outside of the HEDIS suggested age group, when clinical judgement does not point to another infection source, can result in positivity rates between 22% and 30% as previously observed.^22,23 Patients visiting nonphysician providers and outpatient facility providers were less likely to have a revisit than those visiting primary and specialty care physicians, though slightly higher propensity for antibiotic prescriptions was seen for nonphysician providers. Pediatricians have been noted to be less likely to prescribe antibiotics without GAS testing than nonpediatric providers, and more guidelines-compliant in prescribing.²⁴

Recommendations to not test children under 3 years of age are based on the lack of acute rheumatic fever and other complications in this age group together with more frequent viral syndromes. Selectivity in applying clinical criteria to testing can be attempted to separate bacterial from viral illness. Postnasal drainage/rhinorrhea, hoarse voice, and cough have been used successfully to identify those with viral illness and less need for testing, with greater certainty of low risk for GABHS in those over 11 years of age without tonsillar exudates, cervical adenopathy, or fever.¹⁷ However, the marginal benefits of those who have all 3 features of viral illness versus none in identifying GAS positivity was 23.3% vs 37.6% - helpful, but certainly not diminishing the need for testing. These constitutional findings of viral URI also do not exclude the GAS carrier state that features these symptoms.²⁵ Others have reinforced the doubt of pharyngeal exudates as the premier diagnostic finding for test-positive GAS.²⁶

This study had several limitations. The Optum claims dataset only contains ICD-9 codes for diagnoses. It does not include data on infection severity and clinical findings related to symptoms, thus empiric treatment warranted based in clinical severity is not assessed. Antibiotics are commonly available as generics and very inexpensive. Patients may fill and pay for these prescriptions directly, in which case, a claim for payment may not be filed with Optum. This could result in an undercount of treated patients in our study.

There is no corresponding problem of missing medical claims for GAS testing which were obtained from the CPT codes within the Optum claims data set. However, we elected not to verify the test results due to these data being missing for 75% of the study population. Nevertheless, this study’s focus was less about justifying antibiotic treatment, but dealt with the outcomes generated by testing and treatment. Toward that end, we used CPT codes to identify a revisit, and while those can at times be affected by financial reimbursement incentives, differences related to revisits in the 4 patient groups should not be subject to bias.

Conclusion

This study used data from real world practices to document the patterns of GAS testing and antibiotic use in pediatric pharyngitis patients. Revisit rates were under 5% for all patient groups and the use of rapid diagnostic tools were found to be the determining factor in further reducing the risk of revisits. This supports the need for compliance with the HEDIS quality metric for pharyngitis to the recommended levels of rapid testing which have been falling in recent years. Use of more accurate antigen and newer molecular detection testing methods may help further delineate important factors in determining pediatric pharyngitis treatment and need for revisits.²⁷

Corresponding author: Jeffrey McCombs, MD, University of Southern California School of Pharmacy, Department of Pharmaceutical and Health Economics, Leonard D. Schaeffer Center for Health Policy & Economics, 635 Downey Way, Verna & Peter Dauterive Hall 310, Los Angeles, CA 90089-3333; jmccombs@usc.edu.

Financial disclosures: None.

From the Department of Anesthesiology, Thomas Jefferson University and Hospitals, Sidney Kimmel Medical College, Philadelphia, PA, and Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, PA.

Objective: Retrospective study examining hospital cost information of patients requiring endotracheal intubation with video laryngoscopy. Provide a practical cost assessment on use of the McGRATH and GlideScope video laryngoscopes (VLs).

Methods: This study examined 52 hospital locations within a single, large university hospital, with most of those locations being hospital operating rooms. A total of 34 600 endotracheal intubations performed over 24 months, of which 11 345 were video laryngoscopies. Electronic medical records containing demographic data and information related to endotracheal intubation procedures, with monthly breakdowns between GlideScope and McGRATH intubations, were reviewed. Cost information calculated for equipment, blades, batteries, repairs, and subsequent analysis performed to determine cost differences between those 2 instruments during the COVID-19 period.

Results: A total of 5501 video laryngoscopy procedures were performed using the McGRATH VL and 5305 were performed using the GlideScope VL. Costs over 24 months were $181 093 lower (55.5%) for McGRATH compared to GlideScope. The mean (SD) monthly costs for GlideScope blades were $3837 ($1050) and $3236 ($538) for years 1 and 2, respectively, vs $1652 ($663) and $2933 ($585) for McGRATH blades (P < .001). Most total cost differences were attributed to equipment and blade purchases, which were $202 595 (65.0%) higher for GlideScope. During the COVID-19 period, the use of the McGRATH increased to 61% of all video laryngoscopy cases, compared to 37% for GlideScope (P < .001). Blade cost difference for the COVID-19 period was $128 higher for the McGRATH even though 293 more intubations were performed with that device.

Conclusions: Use of the McGRATH resulted in a cost savings of 55% compared to the GlideScope, and its use was highest during the COVID-19 period, which may be explained by its more portable and practical features.

Keywords: video laryngoscope; McGRATH; GlideScope; endotracheal intubation; hospital costs; COVID-19.

Hospitals have come to rely on video laryngoscopes (VLs) for tracheal intubation as necessary tools for better visualization of airways. Modern video laryngoscopy developed in the 2000s¹ as a progression from direct laryngoscopy, which began in 1852 when Horace Green used a bent tongue spatula and sunlight to examine a child.² VLs have seen many improvements and adaptations of their own, resulting in many different styles and types circulating around hospitals. The GlideScope (Verathon Inc, Bothell, WA) and the McGRATH (Medtronic, Minneapolis, MN) are examples of such instruments, which are now widely used in the US and are the 2 VLs of choice at our institution.

A few studies have compared VLs to direct laryngoscopes. In their systematic review, Lewis et al have shown the numerous benefits of using a VL over a direct laryngoscope. Some general conclusions were that the use of video laryngoscopy reduced the number of failed intubations, decreased laryngeal trauma, and provided improved visualizations.³ Other studies have compared the different types of VLs, including the McGRATH and the GlideScope, examining factors such as intubation time and display quality of the image. Two studies found that medical students were equally successful at using both the McGRATH and the GlideScope,^4,5 while another study found that care providers using the GlideScope had quicker intubation times.⁶ Lastly, Savoldelli et al concluded that more providers preferred the McGRATH, which provided better laryngeal views,⁷ while their subsequent study showed more favorable learning curves of the Airtraq compared to the McGRATH and other VLs.⁸

Although there have been no reported differences in safety and effectiveness of the McGRATH and GlideScope devices, cost data on the use of these 2 popular laryngoscopes are lacking. Such information is important considering the increasing costs of medical technologies and the significant financial losses experienced by health care systems due to the COVID-19 crisis. The purpose of this retrospective cohort study was to compare the cost efficiency of the McGRATH MAC and GlideScope Core VLs at a large academic center.

Methods

This retrospective study was performed under exemption from the Thomas Jefferson University Institutional Review Board. The primary data sources consisted of hospital electronic patient records (EPIC) and cost information from the device manufacturers and hospital staff. The electronic patient data were provided by the EPIC Enterprise Analytics Business Intelligence group at Thomas Jefferson University Hospital (Center City Campus, Philadelphia, PA), while device costs were obtained from Verathon, Medtronic, and departmental staff responsible for purchasing equipment. Monthly data were obtained over a 24-month period (June 2018 through May 2020) when the McGRATH VL was placed into use in the department of anesthesiology. The 2 types of VLs were made available for use in a total of 52 locations, with the majority being hospital operating rooms.

The following variables were recorded: number of endotracheal intubations performed each month with breakdown between video laryngoscopy and flexible bronchoscopy airways, frequency of use for each type of laryngoscope, blades used, and equipment costs for use of each laryngoscope. Hospital cost estimates for both the McGRATH and GlideScope laryngoscopes included batteries, handles, blades, and the devices themselves. Cost data were also collected on frequency of device failure, maintenance, and replacement of parts and lost equipment.

Analysis

De-identified electronic medical records consisted of nominal and quantitative variables, with demographic data and information related to the endotracheal intubation procedure. All data were in chronological order and sorted by date after which coding was applied, to identify device type and allocate pertinent cost information. Descriptive statistics were reported as mean (SD) and sum for costs; frequency tables were generated for intubation procedures according to device type and time periods. Data were analyzed using the χ² test, the student t test, and the Wilcoxon Mann-Whitney U test, with a P value set at .05 for statistical significance. SPSS version 26 and GraphPad Prism version 6 were used for all statistical analyses.

A total of 34 600 endotracheal intubations were performed over the 24-month study period, and 11 345 (32.8%) were video laryngoscopy procedures. Out of all video laryngoscopy procedures, 5501 (48.5%) were performed using the McGRATH VL and 5305 (46.8%) were conducted using the GlideScope VL. The difference of 539 (4.8%) cases accounts for flexible bronchoscopy procedures and endotracheal intubations using other video laryngoscopy equipment. The mean (SD) monthly number of video laryngoscopy procedures for the 24 months was 221 (54) and 229 (89) for the GlideScope and McGRATH devices, respectively. Monthly endotracheal intubation distributions over 24 months trended upward for the McGRATH VL and downward for the GlideScope, but there was no statistically significant (P = .71) difference in overall use between the 2 instruments (Figure 1).

To examine the observed usage trends between the 2 VL during the first and last 12 months, a univariate ANOVA was conducted with the 2 time periods entered as predictors in the model. Video laryngoscopy intubations were performed (P = .001) more frequently with the GlideScope during the first 12 months; however, use of the McGRATH VL increased (P < .001) during the following 12 months compared to GlideScope. The GlideScope accounted for 54% of all VL intubations during the first 12 months, with the McGRATH accounting for 58% of all video laryngoscopy procedures for months 12 to 24. Additionally, the increase in video laryngoscopy procedures with the McGRATH during the last 3 months of the study period was despite an overall reduction in surgical volume due to the COVID-19 crisis, defined for this study as March 1, 2020, to May 31, 2020 (Figure 1). There was a statistically significant (P < .001) difference in the case distribution between use of the McGRATH and GlideScope VL for that period. The anesthesia personnel’s use of the McGRATH VL increased to 61% of all video laryngoscopy cases, compared to 37% for the GlideScope (Figure 2).

The total costs calculated for equipment, blades, and repairs are presented in Table 1 and yearly total costs are shown in Figure 3. Overall costs were $181 093 lower (55.5%) for the McGRATH VL compared to the GlideScope over the 24-month period. The mean (SD) monthly costs for GlideScope VL blades were $3837 ($1050) and $3236 ($538) for years 1 and 2, respectively, vs $1652 ($663) and $2933 ($585) for the McGRATH VL blades. Most of the total cost differences were attributed to equipment and blade purchases, which were $202 595 (65.0%) higher for the GlideScope compared to the McGRATH VL. The monthly blade costs alone were higher (P < .001) for the GlideScope over the 2-year period; however, the McGRATH VL required use of disposable stylets at a cost of $10 177 for all endotracheal intubations, compared to $700 for the GlideScope device.

An analysis was performed to determine whether costs differed between those 2 instruments during the COVID-19 period. There was a statistically significant (P < .001) difference in the case distribution between use of the McGRATH and GlideScope VLs during that period. The calculated blade cost difference for the COVID period was $128 higher for the McGRATH even though 293 more intubations were performed with that device (Table 2).

Discussion

We attempted to provide useful cost estimates by presenting pricing data reflecting the approximate cost that most large institutional anesthesia practices would incur for using those 2 specific devices and related peripherals. The main findings of our analysis showed that use of the McGRATH MAC VL resulted in a 55% cost savings compared to the GlideScope, with a similar number of cases performed with each device over the 24-month study period. We believe this represents a substantial savings to the department and institution, which has prompted internal review on the use of video laryngoscopy equipment. None of the McGRATH units failed; however, the GlideScope required 3 baton replacements.

Of note, use of the McGRATH MAC increased during the COVID-19 period, which may be explained by the fact that the operators found it to be a more portable device. Several physicians in the department commented that its smaller size made the McGRATH MAC more practical during the time when a plexiglass box was being used around the patient’s head to shield the intubator from aerosolized viral particles.

Although this study demonstrated the cost-saving value of the McGRATH over the GlideScope, a suggested next step would be to examine resource utilization related to video laryngoscopy use. The more dynamic tracking of the use of these devices should facilitate the assessment of existing related resources and decision making, to optimize the benefits of this initiative. We would anticipate reduced use of anesthesia personnel, such as technicians to assist with the management of this device which could be significant. As new respiratory viruses are appearing each year, video laryngoscopy will continue to gain increasing use in operating rooms and acute care locations. The adding of protective barriers between patients and providers calls for use of the most practical and effective VL devices, to protect personnel who are at high risk of contamination from airway secretions and aerosolized particles.^9,10

The COVID-19 pandemic has demonstrated the value of anesthesiology in regards to analyzing and finding solutions to effectively manage infected patients or those suspected of infection in the perioperative environment. Inexpensive products are often avoided because cheaper devices are associated with being of lower quality. However, the association with cost and quality—and the assumption that a higher price is positively correlated with higher quality—is overall inconsistent in the medical literature.¹¹ A more effective or higher quality treatment does not necessarily cost more and may actually end up costing less,¹² as was the case in this study. We have been able to directly cut departmental expenses by using a more efficient and cost-effective device for intubations, without compromising safety and efficacy. Future studies should determine whether this significant reduction in costs from video laryngoscopy intubations with the McGRATH VL will be sustained across anesthesiology departments in the Jefferson Health Enterprise Hospitals, or other health systems, as well as its impact on workflow and personnel resources.

This analysis was restricted to one of the campuses of the Jefferson Health Enterprise. However, this is the largest anesthesia practice, encompassing several locations, which should reflect the general practice patterns across other anesthesiology departments in this large institution. The costs for the devices and peripherals may vary across anesthesia practices depending on volume and contracts negotiated with the suppliers. It was not possible to estimate this variability, which could change the total costs by a few percentage points. We recognize that there may be other costs associated with securing the McGRATH VL to prevent loss from theft or misplacement, which were not included in the study. Lastly, the inability to obtain randomized samples for the 2 groups treated with each device opens up the possibility of selection bias. There were, however, multiple intubators who were free to select 1 of the devices for endotracheal intubation, which may have reduced the effect of selection bias.

Conclusion

This study demonstrated that over a 24-month period use of the McGRATH MAC VL resulted in a cost reduction of around 55% compared to using the GlideScope for endotracheal intubation procedures performed at a major academic center. Over the first 3 months of the COVID-19 crisis, which our study included, use of the McGRATH VL increased while GlideScope use decreased. This was most likely related to the portability and smaller size of the McGRATH, which better facilitated intubations of COVID-19 patients.

Acknowledgements: The authors thank Craig Smith, Senior Anesthesia Technician, for his assistance with the cost information and excellent record-keeping related to the use of video laryngoscopes.

Corresponding author: Marc C. Torjman, PhD, Professor, Department of Anesthesiology, Sidney Kimmel Medical College at Thomas Jefferson University, 111 South 11th St, Suite G-8290, Philadelphia, PA 19107; Marc.Torjman@Jefferson.edu.

Financial disclosures: Dr. Thaler has served as a consultant for Medtronic since September 2020. He has participated in 2 webinars on the routine use of video laryngoscopy.

Funding: This study was supported by the Department of Anesthesiology at Thomas Jefferson University.

From the Department of Anesthesiology, Thomas Jefferson University and Hospitals, Sidney Kimmel Medical College, Philadelphia, PA, and Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, PA.

Objective: Retrospective study examining hospital cost information of patients requiring endotracheal intubation with video laryngoscopy. Provide a practical cost assessment on use of the McGRATH and GlideScope video laryngoscopes (VLs).

Methods: This study examined 52 hospital locations within a single, large university hospital, with most of those locations being hospital operating rooms. A total of 34 600 endotracheal intubations performed over 24 months, of which 11 345 were video laryngoscopies. Electronic medical records containing demographic data and information related to endotracheal intubation procedures, with monthly breakdowns between GlideScope and McGRATH intubations, were reviewed. Cost information calculated for equipment, blades, batteries, repairs, and subsequent analysis performed to determine cost differences between those 2 instruments during the COVID-19 period.

Results: A total of 5501 video laryngoscopy procedures were performed using the McGRATH VL and 5305 were performed using the GlideScope VL. Costs over 24 months were $181 093 lower (55.5%) for McGRATH compared to GlideScope. The mean (SD) monthly costs for GlideScope blades were $3837 ($1050) and $3236 ($538) for years 1 and 2, respectively, vs $1652 ($663) and $2933 ($585) for McGRATH blades (P < .001). Most total cost differences were attributed to equipment and blade purchases, which were $202 595 (65.0%) higher for GlideScope. During the COVID-19 period, the use of the McGRATH increased to 61% of all video laryngoscopy cases, compared to 37% for GlideScope (P < .001). Blade cost difference for the COVID-19 period was $128 higher for the McGRATH even though 293 more intubations were performed with that device.

Conclusions: Use of the McGRATH resulted in a cost savings of 55% compared to the GlideScope, and its use was highest during the COVID-19 period, which may be explained by its more portable and practical features.

Keywords: video laryngoscope; McGRATH; GlideScope; endotracheal intubation; hospital costs; COVID-19.

Hospitals have come to rely on video laryngoscopes (VLs) for tracheal intubation as necessary tools for better visualization of airways. Modern video laryngoscopy developed in the 2000s¹ as a progression from direct laryngoscopy, which began in 1852 when Horace Green used a bent tongue spatula and sunlight to examine a child.² VLs have seen many improvements and adaptations of their own, resulting in many different styles and types circulating around hospitals. The GlideScope (Verathon Inc, Bothell, WA) and the McGRATH (Medtronic, Minneapolis, MN) are examples of such instruments, which are now widely used in the US and are the 2 VLs of choice at our institution.

A few studies have compared VLs to direct laryngoscopes. In their systematic review, Lewis et al have shown the numerous benefits of using a VL over a direct laryngoscope. Some general conclusions were that the use of video laryngoscopy reduced the number of failed intubations, decreased laryngeal trauma, and provided improved visualizations.³ Other studies have compared the different types of VLs, including the McGRATH and the GlideScope, examining factors such as intubation time and display quality of the image. Two studies found that medical students were equally successful at using both the McGRATH and the GlideScope,^4,5 while another study found that care providers using the GlideScope had quicker intubation times.⁶ Lastly, Savoldelli et al concluded that more providers preferred the McGRATH, which provided better laryngeal views,⁷ while their subsequent study showed more favorable learning curves of the Airtraq compared to the McGRATH and other VLs.⁸

Although there have been no reported differences in safety and effectiveness of the McGRATH and GlideScope devices, cost data on the use of these 2 popular laryngoscopes are lacking. Such information is important considering the increasing costs of medical technologies and the significant financial losses experienced by health care systems due to the COVID-19 crisis. The purpose of this retrospective cohort study was to compare the cost efficiency of the McGRATH MAC and GlideScope Core VLs at a large academic center.

Methods

This retrospective study was performed under exemption from the Thomas Jefferson University Institutional Review Board. The primary data sources consisted of hospital electronic patient records (EPIC) and cost information from the device manufacturers and hospital staff. The electronic patient data were provided by the EPIC Enterprise Analytics Business Intelligence group at Thomas Jefferson University Hospital (Center City Campus, Philadelphia, PA), while device costs were obtained from Verathon, Medtronic, and departmental staff responsible for purchasing equipment. Monthly data were obtained over a 24-month period (June 2018 through May 2020) when the McGRATH VL was placed into use in the department of anesthesiology. The 2 types of VLs were made available for use in a total of 52 locations, with the majority being hospital operating rooms.

The following variables were recorded: number of endotracheal intubations performed each month with breakdown between video laryngoscopy and flexible bronchoscopy airways, frequency of use for each type of laryngoscope, blades used, and equipment costs for use of each laryngoscope. Hospital cost estimates for both the McGRATH and GlideScope laryngoscopes included batteries, handles, blades, and the devices themselves. Cost data were also collected on frequency of device failure, maintenance, and replacement of parts and lost equipment.

Analysis

De-identified electronic medical records consisted of nominal and quantitative variables, with demographic data and information related to the endotracheal intubation procedure. All data were in chronological order and sorted by date after which coding was applied, to identify device type and allocate pertinent cost information. Descriptive statistics were reported as mean (SD) and sum for costs; frequency tables were generated for intubation procedures according to device type and time periods. Data were analyzed using the χ² test, the student t test, and the Wilcoxon Mann-Whitney U test, with a P value set at .05 for statistical significance. SPSS version 26 and GraphPad Prism version 6 were used for all statistical analyses.

A total of 34 600 endotracheal intubations were performed over the 24-month study period, and 11 345 (32.8%) were video laryngoscopy procedures. Out of all video laryngoscopy procedures, 5501 (48.5%) were performed using the McGRATH VL and 5305 (46.8%) were conducted using the GlideScope VL. The difference of 539 (4.8%) cases accounts for flexible bronchoscopy procedures and endotracheal intubations using other video laryngoscopy equipment. The mean (SD) monthly number of video laryngoscopy procedures for the 24 months was 221 (54) and 229 (89) for the GlideScope and McGRATH devices, respectively. Monthly endotracheal intubation distributions over 24 months trended upward for the McGRATH VL and downward for the GlideScope, but there was no statistically significant (P = .71) difference in overall use between the 2 instruments (Figure 1).

To examine the observed usage trends between the 2 VL during the first and last 12 months, a univariate ANOVA was conducted with the 2 time periods entered as predictors in the model. Video laryngoscopy intubations were performed (P = .001) more frequently with the GlideScope during the first 12 months; however, use of the McGRATH VL increased (P < .001) during the following 12 months compared to GlideScope. The GlideScope accounted for 54% of all VL intubations during the first 12 months, with the McGRATH accounting for 58% of all video laryngoscopy procedures for months 12 to 24. Additionally, the increase in video laryngoscopy procedures with the McGRATH during the last 3 months of the study period was despite an overall reduction in surgical volume due to the COVID-19 crisis, defined for this study as March 1, 2020, to May 31, 2020 (Figure 1). There was a statistically significant (P < .001) difference in the case distribution between use of the McGRATH and GlideScope VL for that period. The anesthesia personnel’s use of the McGRATH VL increased to 61% of all video laryngoscopy cases, compared to 37% for the GlideScope (Figure 2).

The total costs calculated for equipment, blades, and repairs are presented in Table 1 and yearly total costs are shown in Figure 3. Overall costs were $181 093 lower (55.5%) for the McGRATH VL compared to the GlideScope over the 24-month period. The mean (SD) monthly costs for GlideScope VL blades were $3837 ($1050) and $3236 ($538) for years 1 and 2, respectively, vs $1652 ($663) and $2933 ($585) for the McGRATH VL blades. Most of the total cost differences were attributed to equipment and blade purchases, which were $202 595 (65.0%) higher for the GlideScope compared to the McGRATH VL. The monthly blade costs alone were higher (P < .001) for the GlideScope over the 2-year period; however, the McGRATH VL required use of disposable stylets at a cost of $10 177 for all endotracheal intubations, compared to $700 for the GlideScope device.

An analysis was performed to determine whether costs differed between those 2 instruments during the COVID-19 period. There was a statistically significant (P < .001) difference in the case distribution between use of the McGRATH and GlideScope VLs during that period. The calculated blade cost difference for the COVID period was $128 higher for the McGRATH even though 293 more intubations were performed with that device (Table 2).

Discussion

We attempted to provide useful cost estimates by presenting pricing data reflecting the approximate cost that most large institutional anesthesia practices would incur for using those 2 specific devices and related peripherals. The main findings of our analysis showed that use of the McGRATH MAC VL resulted in a 55% cost savings compared to the GlideScope, with a similar number of cases performed with each device over the 24-month study period. We believe this represents a substantial savings to the department and institution, which has prompted internal review on the use of video laryngoscopy equipment. None of the McGRATH units failed; however, the GlideScope required 3 baton replacements.

Of note, use of the McGRATH MAC increased during the COVID-19 period, which may be explained by the fact that the operators found it to be a more portable device. Several physicians in the department commented that its smaller size made the McGRATH MAC more practical during the time when a plexiglass box was being used around the patient’s head to shield the intubator from aerosolized viral particles.

Although this study demonstrated the cost-saving value of the McGRATH over the GlideScope, a suggested next step would be to examine resource utilization related to video laryngoscopy use. The more dynamic tracking of the use of these devices should facilitate the assessment of existing related resources and decision making, to optimize the benefits of this initiative. We would anticipate reduced use of anesthesia personnel, such as technicians to assist with the management of this device which could be significant. As new respiratory viruses are appearing each year, video laryngoscopy will continue to gain increasing use in operating rooms and acute care locations. The adding of protective barriers between patients and providers calls for use of the most practical and effective VL devices, to protect personnel who are at high risk of contamination from airway secretions and aerosolized particles.^9,10

The COVID-19 pandemic has demonstrated the value of anesthesiology in regards to analyzing and finding solutions to effectively manage infected patients or those suspected of infection in the perioperative environment. Inexpensive products are often avoided because cheaper devices are associated with being of lower quality. However, the association with cost and quality—and the assumption that a higher price is positively correlated with higher quality—is overall inconsistent in the medical literature.¹¹ A more effective or higher quality treatment does not necessarily cost more and may actually end up costing less,¹² as was the case in this study. We have been able to directly cut departmental expenses by using a more efficient and cost-effective device for intubations, without compromising safety and efficacy. Future studies should determine whether this significant reduction in costs from video laryngoscopy intubations with the McGRATH VL will be sustained across anesthesiology departments in the Jefferson Health Enterprise Hospitals, or other health systems, as well as its impact on workflow and personnel resources.

This analysis was restricted to one of the campuses of the Jefferson Health Enterprise. However, this is the largest anesthesia practice, encompassing several locations, which should reflect the general practice patterns across other anesthesiology departments in this large institution. The costs for the devices and peripherals may vary across anesthesia practices depending on volume and contracts negotiated with the suppliers. It was not possible to estimate this variability, which could change the total costs by a few percentage points. We recognize that there may be other costs associated with securing the McGRATH VL to prevent loss from theft or misplacement, which were not included in the study. Lastly, the inability to obtain randomized samples for the 2 groups treated with each device opens up the possibility of selection bias. There were, however, multiple intubators who were free to select 1 of the devices for endotracheal intubation, which may have reduced the effect of selection bias.

Conclusion

This study demonstrated that over a 24-month period use of the McGRATH MAC VL resulted in a cost reduction of around 55% compared to using the GlideScope for endotracheal intubation procedures performed at a major academic center. Over the first 3 months of the COVID-19 crisis, which our study included, use of the McGRATH VL increased while GlideScope use decreased. This was most likely related to the portability and smaller size of the McGRATH, which better facilitated intubations of COVID-19 patients.

Acknowledgements: The authors thank Craig Smith, Senior Anesthesia Technician, for his assistance with the cost information and excellent record-keeping related to the use of video laryngoscopes.

Corresponding author: Marc C. Torjman, PhD, Professor, Department of Anesthesiology, Sidney Kimmel Medical College at Thomas Jefferson University, 111 South 11th St, Suite G-8290, Philadelphia, PA 19107; Marc.Torjman@Jefferson.edu.

Financial disclosures: Dr. Thaler has served as a consultant for Medtronic since September 2020. He has participated in 2 webinars on the routine use of video laryngoscopy.

Funding: This study was supported by the Department of Anesthesiology at Thomas Jefferson University.

From the Department of Anesthesiology, Thomas Jefferson University and Hospitals, Sidney Kimmel Medical College, Philadelphia, PA, and Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, PA.

Objective: Retrospective study examining hospital cost information of patients requiring endotracheal intubation with video laryngoscopy. Provide a practical cost assessment on use of the McGRATH and GlideScope video laryngoscopes (VLs).

Methods: This study examined 52 hospital locations within a single, large university hospital, with most of those locations being hospital operating rooms. A total of 34 600 endotracheal intubations performed over 24 months, of which 11 345 were video laryngoscopies. Electronic medical records containing demographic data and information related to endotracheal intubation procedures, with monthly breakdowns between GlideScope and McGRATH intubations, were reviewed. Cost information calculated for equipment, blades, batteries, repairs, and subsequent analysis performed to determine cost differences between those 2 instruments during the COVID-19 period.

Results: A total of 5501 video laryngoscopy procedures were performed using the McGRATH VL and 5305 were performed using the GlideScope VL. Costs over 24 months were $181 093 lower (55.5%) for McGRATH compared to GlideScope. The mean (SD) monthly costs for GlideScope blades were $3837 ($1050) and $3236 ($538) for years 1 and 2, respectively, vs $1652 ($663) and $2933 ($585) for McGRATH blades (P < .001). Most total cost differences were attributed to equipment and blade purchases, which were $202 595 (65.0%) higher for GlideScope. During the COVID-19 period, the use of the McGRATH increased to 61% of all video laryngoscopy cases, compared to 37% for GlideScope (P < .001). Blade cost difference for the COVID-19 period was $128 higher for the McGRATH even though 293 more intubations were performed with that device.

Conclusions: Use of the McGRATH resulted in a cost savings of 55% compared to the GlideScope, and its use was highest during the COVID-19 period, which may be explained by its more portable and practical features.

Keywords: video laryngoscope; McGRATH; GlideScope; endotracheal intubation; hospital costs; COVID-19.

Hospitals have come to rely on video laryngoscopes (VLs) for tracheal intubation as necessary tools for better visualization of airways. Modern video laryngoscopy developed in the 2000s¹ as a progression from direct laryngoscopy, which began in 1852 when Horace Green used a bent tongue spatula and sunlight to examine a child.² VLs have seen many improvements and adaptations of their own, resulting in many different styles and types circulating around hospitals. The GlideScope (Verathon Inc, Bothell, WA) and the McGRATH (Medtronic, Minneapolis, MN) are examples of such instruments, which are now widely used in the US and are the 2 VLs of choice at our institution.

A few studies have compared VLs to direct laryngoscopes. In their systematic review, Lewis et al have shown the numerous benefits of using a VL over a direct laryngoscope. Some general conclusions were that the use of video laryngoscopy reduced the number of failed intubations, decreased laryngeal trauma, and provided improved visualizations.³ Other studies have compared the different types of VLs, including the McGRATH and the GlideScope, examining factors such as intubation time and display quality of the image. Two studies found that medical students were equally successful at using both the McGRATH and the GlideScope,^4,5 while another study found that care providers using the GlideScope had quicker intubation times.⁶ Lastly, Savoldelli et al concluded that more providers preferred the McGRATH, which provided better laryngeal views,⁷ while their subsequent study showed more favorable learning curves of the Airtraq compared to the McGRATH and other VLs.⁸

Although there have been no reported differences in safety and effectiveness of the McGRATH and GlideScope devices, cost data on the use of these 2 popular laryngoscopes are lacking. Such information is important considering the increasing costs of medical technologies and the significant financial losses experienced by health care systems due to the COVID-19 crisis. The purpose of this retrospective cohort study was to compare the cost efficiency of the McGRATH MAC and GlideScope Core VLs at a large academic center.

Methods

This retrospective study was performed under exemption from the Thomas Jefferson University Institutional Review Board. The primary data sources consisted of hospital electronic patient records (EPIC) and cost information from the device manufacturers and hospital staff. The electronic patient data were provided by the EPIC Enterprise Analytics Business Intelligence group at Thomas Jefferson University Hospital (Center City Campus, Philadelphia, PA), while device costs were obtained from Verathon, Medtronic, and departmental staff responsible for purchasing equipment. Monthly data were obtained over a 24-month period (June 2018 through May 2020) when the McGRATH VL was placed into use in the department of anesthesiology. The 2 types of VLs were made available for use in a total of 52 locations, with the majority being hospital operating rooms.

The following variables were recorded: number of endotracheal intubations performed each month with breakdown between video laryngoscopy and flexible bronchoscopy airways, frequency of use for each type of laryngoscope, blades used, and equipment costs for use of each laryngoscope. Hospital cost estimates for both the McGRATH and GlideScope laryngoscopes included batteries, handles, blades, and the devices themselves. Cost data were also collected on frequency of device failure, maintenance, and replacement of parts and lost equipment.

Analysis

De-identified electronic medical records consisted of nominal and quantitative variables, with demographic data and information related to the endotracheal intubation procedure. All data were in chronological order and sorted by date after which coding was applied, to identify device type and allocate pertinent cost information. Descriptive statistics were reported as mean (SD) and sum for costs; frequency tables were generated for intubation procedures according to device type and time periods. Data were analyzed using the χ² test, the student t test, and the Wilcoxon Mann-Whitney U test, with a P value set at .05 for statistical significance. SPSS version 26 and GraphPad Prism version 6 were used for all statistical analyses.

A total of 34 600 endotracheal intubations were performed over the 24-month study period, and 11 345 (32.8%) were video laryngoscopy procedures. Out of all video laryngoscopy procedures, 5501 (48.5%) were performed using the McGRATH VL and 5305 (46.8%) were conducted using the GlideScope VL. The difference of 539 (4.8%) cases accounts for flexible bronchoscopy procedures and endotracheal intubations using other video laryngoscopy equipment. The mean (SD) monthly number of video laryngoscopy procedures for the 24 months was 221 (54) and 229 (89) for the GlideScope and McGRATH devices, respectively. Monthly endotracheal intubation distributions over 24 months trended upward for the McGRATH VL and downward for the GlideScope, but there was no statistically significant (P = .71) difference in overall use between the 2 instruments (Figure 1).

To examine the observed usage trends between the 2 VL during the first and last 12 months, a univariate ANOVA was conducted with the 2 time periods entered as predictors in the model. Video laryngoscopy intubations were performed (P = .001) more frequently with the GlideScope during the first 12 months; however, use of the McGRATH VL increased (P < .001) during the following 12 months compared to GlideScope. The GlideScope accounted for 54% of all VL intubations during the first 12 months, with the McGRATH accounting for 58% of all video laryngoscopy procedures for months 12 to 24. Additionally, the increase in video laryngoscopy procedures with the McGRATH during the last 3 months of the study period was despite an overall reduction in surgical volume due to the COVID-19 crisis, defined for this study as March 1, 2020, to May 31, 2020 (Figure 1). There was a statistically significant (P < .001) difference in the case distribution between use of the McGRATH and GlideScope VL for that period. The anesthesia personnel’s use of the McGRATH VL increased to 61% of all video laryngoscopy cases, compared to 37% for the GlideScope (Figure 2).

The total costs calculated for equipment, blades, and repairs are presented in Table 1 and yearly total costs are shown in Figure 3. Overall costs were $181 093 lower (55.5%) for the McGRATH VL compared to the GlideScope over the 24-month period. The mean (SD) monthly costs for GlideScope VL blades were $3837 ($1050) and $3236 ($538) for years 1 and 2, respectively, vs $1652 ($663) and $2933 ($585) for the McGRATH VL blades. Most of the total cost differences were attributed to equipment and blade purchases, which were $202 595 (65.0%) higher for the GlideScope compared to the McGRATH VL. The monthly blade costs alone were higher (P < .001) for the GlideScope over the 2-year period; however, the McGRATH VL required use of disposable stylets at a cost of $10 177 for all endotracheal intubations, compared to $700 for the GlideScope device.

An analysis was performed to determine whether costs differed between those 2 instruments during the COVID-19 period. There was a statistically significant (P < .001) difference in the case distribution between use of the McGRATH and GlideScope VLs during that period. The calculated blade cost difference for the COVID period was $128 higher for the McGRATH even though 293 more intubations were performed with that device (Table 2).

Discussion

We attempted to provide useful cost estimates by presenting pricing data reflecting the approximate cost that most large institutional anesthesia practices would incur for using those 2 specific devices and related peripherals. The main findings of our analysis showed that use of the McGRATH MAC VL resulted in a 55% cost savings compared to the GlideScope, with a similar number of cases performed with each device over the 24-month study period. We believe this represents a substantial savings to the department and institution, which has prompted internal review on the use of video laryngoscopy equipment. None of the McGRATH units failed; however, the GlideScope required 3 baton replacements.

Of note, use of the McGRATH MAC increased during the COVID-19 period, which may be explained by the fact that the operators found it to be a more portable device. Several physicians in the department commented that its smaller size made the McGRATH MAC more practical during the time when a plexiglass box was being used around the patient’s head to shield the intubator from aerosolized viral particles.

Although this study demonstrated the cost-saving value of the McGRATH over the GlideScope, a suggested next step would be to examine resource utilization related to video laryngoscopy use. The more dynamic tracking of the use of these devices should facilitate the assessment of existing related resources and decision making, to optimize the benefits of this initiative. We would anticipate reduced use of anesthesia personnel, such as technicians to assist with the management of this device which could be significant. As new respiratory viruses are appearing each year, video laryngoscopy will continue to gain increasing use in operating rooms and acute care locations. The adding of protective barriers between patients and providers calls for use of the most practical and effective VL devices, to protect personnel who are at high risk of contamination from airway secretions and aerosolized particles.^9,10

The COVID-19 pandemic has demonstrated the value of anesthesiology in regards to analyzing and finding solutions to effectively manage infected patients or those suspected of infection in the perioperative environment. Inexpensive products are often avoided because cheaper devices are associated with being of lower quality. However, the association with cost and quality—and the assumption that a higher price is positively correlated with higher quality—is overall inconsistent in the medical literature.¹¹ A more effective or higher quality treatment does not necessarily cost more and may actually end up costing less,¹² as was the case in this study. We have been able to directly cut departmental expenses by using a more efficient and cost-effective device for intubations, without compromising safety and efficacy. Future studies should determine whether this significant reduction in costs from video laryngoscopy intubations with the McGRATH VL will be sustained across anesthesiology departments in the Jefferson Health Enterprise Hospitals, or other health systems, as well as its impact on workflow and personnel resources.

This analysis was restricted to one of the campuses of the Jefferson Health Enterprise. However, this is the largest anesthesia practice, encompassing several locations, which should reflect the general practice patterns across other anesthesiology departments in this large institution. The costs for the devices and peripherals may vary across anesthesia practices depending on volume and contracts negotiated with the suppliers. It was not possible to estimate this variability, which could change the total costs by a few percentage points. We recognize that there may be other costs associated with securing the McGRATH VL to prevent loss from theft or misplacement, which were not included in the study. Lastly, the inability to obtain randomized samples for the 2 groups treated with each device opens up the possibility of selection bias. There were, however, multiple intubators who were free to select 1 of the devices for endotracheal intubation, which may have reduced the effect of selection bias.

Conclusion

This study demonstrated that over a 24-month period use of the McGRATH MAC VL resulted in a cost reduction of around 55% compared to using the GlideScope for endotracheal intubation procedures performed at a major academic center. Over the first 3 months of the COVID-19 crisis, which our study included, use of the McGRATH VL increased while GlideScope use decreased. This was most likely related to the portability and smaller size of the McGRATH, which better facilitated intubations of COVID-19 patients.

Acknowledgements: The authors thank Craig Smith, Senior Anesthesia Technician, for his assistance with the cost information and excellent record-keeping related to the use of video laryngoscopes.

Corresponding author: Marc C. Torjman, PhD, Professor, Department of Anesthesiology, Sidney Kimmel Medical College at Thomas Jefferson University, 111 South 11th St, Suite G-8290, Philadelphia, PA 19107; Marc.Torjman@Jefferson.edu.

Financial disclosures: Dr. Thaler has served as a consultant for Medtronic since September 2020. He has participated in 2 webinars on the routine use of video laryngoscopy.

Funding: This study was supported by the Department of Anesthesiology at Thomas Jefferson University.

Identifying safe, effective, and affordable evidence-based dermatologic treatments for older adults can be challenging because of age-related changes in the skin, comorbidities, polypharmacy, mobility issues, and cognitive changes. Phototherapy has been shown to be an effective nonpharmacologic treatment option for multiple challenging dermatologic conditions^1-8; however, few studies have specifically examined its effectiveness in older adults. The challenge for older patients with psoriasis and dermatitis is that the conditions can be difficult to control and often require multiple treatment modalities.^9,10 Patients with psoriasis also have a higher risk for diabetes, dyslipidemia, and cardiovascular disease compared to other older patients,^11,12 which poses treatment challenges and makes nonpharmacologic treatments even more appealing.

Recent studies show that phototherapy can help decrease the use of dermatologic medications. Foerster and colleagues² found that adults with psoriasis who were treated with phototherapy significantly decreased their use of topical steroids (24.5% fewer patients required steroid creams and 31.1% fewer patients required psoriasis-specific topicals)(P<.01) while their use of non–psoriasis-specific medications did not change. Click and colleagues¹³ identified a decrease in medication costs, health care utilization, and risk for immunosuppression in patients treated with phototherapy when compared to those treated with biologics and apremilast. Methotrexate is a common dermatologic medication that is highly associated with increased risks in elderly patients because of impaired immune system function and the presence of comorbidities (eg, kidney disease, obesity, diabetes, fatty liver),¹⁴ which increase in prevalence with age. Combining phototherapy with methotrexate can substantially decrease the amount of methotrexate needed to achieve disease control,¹⁵ thereby decreasing the methotrexate-associated risks. Findings from these studies suggest that a safe, effective, cost-effective, and well-tolerated nonpharmacologic alternative, such as phototherapy, is highly desirable and should be optimized. Unfortunately, most studies that report the effectiveness of phototherapy are in younger populations.

This retrospective study aimed to (1) identify the most common dermatologic conditions treated with phototherapy in older adults, (2) examine the effectiveness and safety of phototherapy in older adults, and (3) compare the outcomes with 2 similar studies in the United Kingdom¹⁶ and Turkey.¹⁷

Methods

Design, Setting, Sample, and Statistical Analysis
The institutional review boards of Kaiser Permanente Washington Health Research Institute, Seattle, and the University of Washington, Seattle, approved this study. It was conducted in a large US multispecialty health care system (Group Health, Seattle, Washington [now Kaiser Permanente Washington]) serving approximately 600,000 patients, using billing records to identify all patients treated with phototherapy between January 1, 2015, and December 31, 2015, all who received narrowband UVB (NB-UVB) phototherapy. All adults 65 years and older who received phototherapy treatment during the 12-month study period were included. Patients were included regardless of comorbidities and other dermatologic treatments to maintain as much uniformity as possible between the present study and 2 prior studies examining phototherapy in older adult populations in the United Kingdom¹⁶ and Turkey.¹⁷ Demographic and clinical factors were presented using frequencies (percentages) or means and medians as appropriate. Comparisons of dermatologic conditions and clearance levels used a Fisher exact test. The number of phototherapy treatments to clearance and total number of treatments were compared between groups of patients using independent sample t tests.

Phototherapy Protocol
All patients received treatments administered by specially trained phototherapy nurses using a Daavlin UV Series (The Daavlin Company) or an Ultralite unit (Ultralite Enterprises, Inc), both with 48 lamps. All phototherapy nurses had been previously trained to provide treatments based on standardized protocols (Table 1) and to determine the patient’s level of disease clearance using a high to low clearance scale (Table 2). Daavlin’s treatment protocols were built into the software that accompanied the units and were developed based on the American Academy of Dermatology guidelines. The starting dose for an individual patient was determined based on the estimated minimal erythema dose for each phototype. If the patient was using photosensitizing medications, then the protocol guided the nurse to start the patient at a lower dose appropriate for their phototype. Patients with vitiligo were treated with the same starting and escalation doses as patients with Fitzpatrick phototype I because of the assumption that their vitiliginous skin had an increased risk for photosensitivity. A more recent review of the evidence has indicated that this assumption was overly conservative,¹⁸ and Kaiser Permanente Washington’s vitiligo protocol has been adjusted.

Results

Patients
Billing records identified 229 total patients who received phototherapy in 2015, of whom 52 (22.7%) were at least 65 years old. The median age was 70 years (range, 65–91 years). Twenty-nine (56%) were men and 35 (67%) had previously received phototherapy treatments.

Dermatologic Conditions Treated With Phototherapy
Our primary aim was to identify the most common dermatologic conditions treated with phototherapy in older adults. Psoriasis and dermatitis were the most common conditions treated in the sample (50% [26/52] and 21% [11/52], respectively), with mycosis fungoides being the third most common (10% [5/52]) and vitiligo tied with prurigo nodularis as fourth most common (6% [3/52])(Figure 1).

Effectiveness and Safety of Phototherapy
Our secondary aim was to examine the effectiveness and safety of phototherapy in older adults. Phototherapy was effective in this population, with 50 of 52 patients (96%) achieving a high or medium level of clearance. The degree of clearance for each of the dermatologic conditions is shown in Figure 2. Psoriasis and dermatitis achieved high clearance rates in 81% (21/26) and 82% (9/11) of patients, respectively. Overall, conditions did not have significant differences in clearances rates (Fisher exact test, P=.10). On average, it took patients 33 treatments to achieve medium or high rates of clearance. Psoriasis cleared more quickly, with an average of 30.4 treatments vs 36.1 treatments for other conditions, but the difference was not significant (t test, P=.26). Patients received an average of 98 total phototherapy treatments; the median number of treatments was 81 due to many being on maintenance therapy over several months. There was no relationship between a history of treatment with phototherapy and the total number of treatments needed to achieve clearance (t test, P=.40), but interestingly, those who had a history of phototherapy took approximately 5 more treatments to achieve clearance. The present study found that a slightly larger number of men were being treated for psoriasis (15 men vs 11 women), but there was no significant difference in response rate based on gender.

Comment

Our study found that 94% of patients with psoriasis achieved clearance with an average of 30.4 treatments, which is comparable to the reported 91% response rate with an average of 30 treatments in the United Kingdom.¹⁶ The other similar study in Turkey¹⁷ reported 73.7% of psoriasis patients achieved a 75% or more improvement from baseline with an average of 42 treatments, which may reflect underlying differences in regional skin type. Of note, the scatter chart (Figure 3) shows that several patients in the present study’s analysis are listed as not clear, but many of those patients had low treatment numbers below the mean time to clearance. Thus, the present study’s response rate may have been underestimated.

In the general population, studies show that psoriasis treated with standardized phototherapy protocols typically clears with an average of 20.6 treatments.²¹ The levels of clearance were similar in our study’s older population, but more treatments were required to achieve those results, with an average of 10 more treatments needed (an additional 3.3 weeks). Similar results were found in this sample for dermatitis and mycosis fungoides, indicating comparable clearance rates and levels but a need for more treatments to achieve similar results compared to the general population.

Additionally, in the current study more patients experienced grade 1 (mild) erythema (46%) and grade 2 erythema (25%) at some point in their treatment compared with the United Kingdom¹⁶ (1.89%) and Turkey¹⁷ (35%) studies, though these side effects did not impact the clearance rate. Interestingly, the current study’s scatter chart (Figure 3) illustrates that this side effect did not seem to increase with aging in this population. If anything, the erythema response was more prevalent in the median or younger patients in the sample. Erythema may have been due to the frequent use of photosensitizing medications in older adults in the United States, some of which typically get discontinued in patients 75 years and older (eg, statins). Other potential causes might include the use of phototype vs minimal erythema dose–driven protocols, the standard utilization of protocols originally designed for psoriasis vs other condition-specific protocols, missed treatments leading to increased sensitivity, or possibly shielding mishaps (eg, not wearing a prescribed face shield). Given the number of potential causes and the possibility of overlapping factors, careful analysis is important. With NB-UVB phototherapy, near-erythemogenic doses are optimal to achieve effective treatments, but this delicate balance may be more problematic for older adults. Future studies are needed to fully determine the factors at play for this population. In the interim, it is important for phototherapy-trained nurses to consider this risk carefully in the older population. They must follow the prescribed protocols that guide them to query patients about their responses to the prior treatment (eg, erythema, tenderness, itching), photosensitizing medications, missed treatments, and placement of shielding, and then adjust the treatment dosing accordingly.

Limitations
This study had several limitations. Although clinical outcomes were recorded prospectively, the analysis was retrospective, unblinded, and not placebo controlled. It was conducted in a single organization (Group Health [now Kaiser Permanente Washington]) but did analyze data from 4 medical centers in different cities with diverse demographics and a variety of nursing staff providing the treatments. Although the vitiligo treatment protocol likely slowed the response rate for those patients with vitiligo, the numbers were small (ie, only 3 of 52 patients), so the researchers chose to include them in the current study. The sample population was relatively small, but when these data are evaluated alongside the studies in the United Kingdom¹⁶ and Turkey,¹⁷ they show a consistent picture illustrating the effectiveness and safety of phototherapy in the older population. Further epidemiologic studies could be helpful to further describe the usefulness of this modality compared with other treatments for a variety of dermatoses in this age group. Supplementary analysis specifically examining the relationship between the number and type of photosensitizing medications, frequency of erythema, and time to clearance also could be useful.

Conclusion

Older adults with a variety of dermatoses respond well to phototherapy and should have the opportunity to use it, particularly considering the potential for increased complications and costs from other treatment modalities, such as commonly used immunosuppressive pharmaceuticals. However, the current study and the comparison studies indicate that it is important to carefully consider the slower clearance rates and the potential risk for increased erythema in this population and adjust patient education and treatment dosing accordingly.

Unfortunately, many dermatology centers do not offer phototherapy because of infrastructure limitations such as space and specially trained nursing staff. Increasing accessibility of phototherapy for older adults through home treatments may be an alternative, given its effectiveness in the general population.^22,23 In addition, home phototherapy may be worth pursuing for the older population considering the challenges they may face with transportation to the clinic setting and their increased risk for serious illness if exposed to infections such as COVID-19. The COVID-19 pandemic has brought to light the need for reliable, safe, and effective treatments that can be utilized in the safety of patients’ homes and should therefore be considered as an option for older adults. Issues such as mobility and cognitive decline could pose some complicating factors, but with the help of a well-trained family member or caregiver, home phototherapy could be a viable option that improves accessibility for older patients. Future research opportunities include further examination of the slower but ultimately equivalent response to phototherapy in the older population, the influence of photosensitizing medications on phototherapy effects, and the impact of phototherapy on utilization of immunosuppressive pharmaceuticals in older adults.

Identifying safe, effective, and affordable evidence-based dermatologic treatments for older adults can be challenging because of age-related changes in the skin, comorbidities, polypharmacy, mobility issues, and cognitive changes. Phototherapy has been shown to be an effective nonpharmacologic treatment option for multiple challenging dermatologic conditions^1-8; however, few studies have specifically examined its effectiveness in older adults. The challenge for older patients with psoriasis and dermatitis is that the conditions can be difficult to control and often require multiple treatment modalities.^9,10 Patients with psoriasis also have a higher risk for diabetes, dyslipidemia, and cardiovascular disease compared to other older patients,^11,12 which poses treatment challenges and makes nonpharmacologic treatments even more appealing.

Recent studies show that phototherapy can help decrease the use of dermatologic medications. Foerster and colleagues² found that adults with psoriasis who were treated with phototherapy significantly decreased their use of topical steroids (24.5% fewer patients required steroid creams and 31.1% fewer patients required psoriasis-specific topicals)(P<.01) while their use of non–psoriasis-specific medications did not change. Click and colleagues¹³ identified a decrease in medication costs, health care utilization, and risk for immunosuppression in patients treated with phototherapy when compared to those treated with biologics and apremilast. Methotrexate is a common dermatologic medication that is highly associated with increased risks in elderly patients because of impaired immune system function and the presence of comorbidities (eg, kidney disease, obesity, diabetes, fatty liver),¹⁴ which increase in prevalence with age. Combining phototherapy with methotrexate can substantially decrease the amount of methotrexate needed to achieve disease control,¹⁵ thereby decreasing the methotrexate-associated risks. Findings from these studies suggest that a safe, effective, cost-effective, and well-tolerated nonpharmacologic alternative, such as phototherapy, is highly desirable and should be optimized. Unfortunately, most studies that report the effectiveness of phototherapy are in younger populations.

This retrospective study aimed to (1) identify the most common dermatologic conditions treated with phototherapy in older adults, (2) examine the effectiveness and safety of phototherapy in older adults, and (3) compare the outcomes with 2 similar studies in the United Kingdom¹⁶ and Turkey.¹⁷

Methods

Design, Setting, Sample, and Statistical Analysis
The institutional review boards of Kaiser Permanente Washington Health Research Institute, Seattle, and the University of Washington, Seattle, approved this study. It was conducted in a large US multispecialty health care system (Group Health, Seattle, Washington [now Kaiser Permanente Washington]) serving approximately 600,000 patients, using billing records to identify all patients treated with phototherapy between January 1, 2015, and December 31, 2015, all who received narrowband UVB (NB-UVB) phototherapy. All adults 65 years and older who received phototherapy treatment during the 12-month study period were included. Patients were included regardless of comorbidities and other dermatologic treatments to maintain as much uniformity as possible between the present study and 2 prior studies examining phototherapy in older adult populations in the United Kingdom¹⁶ and Turkey.¹⁷ Demographic and clinical factors were presented using frequencies (percentages) or means and medians as appropriate. Comparisons of dermatologic conditions and clearance levels used a Fisher exact test. The number of phototherapy treatments to clearance and total number of treatments were compared between groups of patients using independent sample t tests.

Phototherapy Protocol
All patients received treatments administered by specially trained phototherapy nurses using a Daavlin UV Series (The Daavlin Company) or an Ultralite unit (Ultralite Enterprises, Inc), both with 48 lamps. All phototherapy nurses had been previously trained to provide treatments based on standardized protocols (Table 1) and to determine the patient’s level of disease clearance using a high to low clearance scale (Table 2). Daavlin’s treatment protocols were built into the software that accompanied the units and were developed based on the American Academy of Dermatology guidelines. The starting dose for an individual patient was determined based on the estimated minimal erythema dose for each phototype. If the patient was using photosensitizing medications, then the protocol guided the nurse to start the patient at a lower dose appropriate for their phototype. Patients with vitiligo were treated with the same starting and escalation doses as patients with Fitzpatrick phototype I because of the assumption that their vitiliginous skin had an increased risk for photosensitivity. A more recent review of the evidence has indicated that this assumption was overly conservative,¹⁸ and Kaiser Permanente Washington’s vitiligo protocol has been adjusted.

Results

Patients
Billing records identified 229 total patients who received phototherapy in 2015, of whom 52 (22.7%) were at least 65 years old. The median age was 70 years (range, 65–91 years). Twenty-nine (56%) were men and 35 (67%) had previously received phototherapy treatments.

Dermatologic Conditions Treated With Phototherapy
Our primary aim was to identify the most common dermatologic conditions treated with phototherapy in older adults. Psoriasis and dermatitis were the most common conditions treated in the sample (50% [26/52] and 21% [11/52], respectively), with mycosis fungoides being the third most common (10% [5/52]) and vitiligo tied with prurigo nodularis as fourth most common (6% [3/52])(Figure 1).

Effectiveness and Safety of Phototherapy
Our secondary aim was to examine the effectiveness and safety of phototherapy in older adults. Phototherapy was effective in this population, with 50 of 52 patients (96%) achieving a high or medium level of clearance. The degree of clearance for each of the dermatologic conditions is shown in Figure 2. Psoriasis and dermatitis achieved high clearance rates in 81% (21/26) and 82% (9/11) of patients, respectively. Overall, conditions did not have significant differences in clearances rates (Fisher exact test, P=.10). On average, it took patients 33 treatments to achieve medium or high rates of clearance. Psoriasis cleared more quickly, with an average of 30.4 treatments vs 36.1 treatments for other conditions, but the difference was not significant (t test, P=.26). Patients received an average of 98 total phototherapy treatments; the median number of treatments was 81 due to many being on maintenance therapy over several months. There was no relationship between a history of treatment with phototherapy and the total number of treatments needed to achieve clearance (t test, P=.40), but interestingly, those who had a history of phototherapy took approximately 5 more treatments to achieve clearance. The present study found that a slightly larger number of men were being treated for psoriasis (15 men vs 11 women), but there was no significant difference in response rate based on gender.

Comment

Our study found that 94% of patients with psoriasis achieved clearance with an average of 30.4 treatments, which is comparable to the reported 91% response rate with an average of 30 treatments in the United Kingdom.¹⁶ The other similar study in Turkey¹⁷ reported 73.7% of psoriasis patients achieved a 75% or more improvement from baseline with an average of 42 treatments, which may reflect underlying differences in regional skin type. Of note, the scatter chart (Figure 3) shows that several patients in the present study’s analysis are listed as not clear, but many of those patients had low treatment numbers below the mean time to clearance. Thus, the present study’s response rate may have been underestimated.

In the general population, studies show that psoriasis treated with standardized phototherapy protocols typically clears with an average of 20.6 treatments.²¹ The levels of clearance were similar in our study’s older population, but more treatments were required to achieve those results, with an average of 10 more treatments needed (an additional 3.3 weeks). Similar results were found in this sample for dermatitis and mycosis fungoides, indicating comparable clearance rates and levels but a need for more treatments to achieve similar results compared to the general population.

Additionally, in the current study more patients experienced grade 1 (mild) erythema (46%) and grade 2 erythema (25%) at some point in their treatment compared with the United Kingdom¹⁶ (1.89%) and Turkey¹⁷ (35%) studies, though these side effects did not impact the clearance rate. Interestingly, the current study’s scatter chart (Figure 3) illustrates that this side effect did not seem to increase with aging in this population. If anything, the erythema response was more prevalent in the median or younger patients in the sample. Erythema may have been due to the frequent use of photosensitizing medications in older adults in the United States, some of which typically get discontinued in patients 75 years and older (eg, statins). Other potential causes might include the use of phototype vs minimal erythema dose–driven protocols, the standard utilization of protocols originally designed for psoriasis vs other condition-specific protocols, missed treatments leading to increased sensitivity, or possibly shielding mishaps (eg, not wearing a prescribed face shield). Given the number of potential causes and the possibility of overlapping factors, careful analysis is important. With NB-UVB phototherapy, near-erythemogenic doses are optimal to achieve effective treatments, but this delicate balance may be more problematic for older adults. Future studies are needed to fully determine the factors at play for this population. In the interim, it is important for phototherapy-trained nurses to consider this risk carefully in the older population. They must follow the prescribed protocols that guide them to query patients about their responses to the prior treatment (eg, erythema, tenderness, itching), photosensitizing medications, missed treatments, and placement of shielding, and then adjust the treatment dosing accordingly.

Limitations
This study had several limitations. Although clinical outcomes were recorded prospectively, the analysis was retrospective, unblinded, and not placebo controlled. It was conducted in a single organization (Group Health [now Kaiser Permanente Washington]) but did analyze data from 4 medical centers in different cities with diverse demographics and a variety of nursing staff providing the treatments. Although the vitiligo treatment protocol likely slowed the response rate for those patients with vitiligo, the numbers were small (ie, only 3 of 52 patients), so the researchers chose to include them in the current study. The sample population was relatively small, but when these data are evaluated alongside the studies in the United Kingdom¹⁶ and Turkey,¹⁷ they show a consistent picture illustrating the effectiveness and safety of phototherapy in the older population. Further epidemiologic studies could be helpful to further describe the usefulness of this modality compared with other treatments for a variety of dermatoses in this age group. Supplementary analysis specifically examining the relationship between the number and type of photosensitizing medications, frequency of erythema, and time to clearance also could be useful.

Conclusion

Older adults with a variety of dermatoses respond well to phototherapy and should have the opportunity to use it, particularly considering the potential for increased complications and costs from other treatment modalities, such as commonly used immunosuppressive pharmaceuticals. However, the current study and the comparison studies indicate that it is important to carefully consider the slower clearance rates and the potential risk for increased erythema in this population and adjust patient education and treatment dosing accordingly.

Unfortunately, many dermatology centers do not offer phototherapy because of infrastructure limitations such as space and specially trained nursing staff. Increasing accessibility of phototherapy for older adults through home treatments may be an alternative, given its effectiveness in the general population.^22,23 In addition, home phototherapy may be worth pursuing for the older population considering the challenges they may face with transportation to the clinic setting and their increased risk for serious illness if exposed to infections such as COVID-19. The COVID-19 pandemic has brought to light the need for reliable, safe, and effective treatments that can be utilized in the safety of patients’ homes and should therefore be considered as an option for older adults. Issues such as mobility and cognitive decline could pose some complicating factors, but with the help of a well-trained family member or caregiver, home phototherapy could be a viable option that improves accessibility for older patients. Future research opportunities include further examination of the slower but ultimately equivalent response to phototherapy in the older population, the influence of photosensitizing medications on phototherapy effects, and the impact of phototherapy on utilization of immunosuppressive pharmaceuticals in older adults.

Identifying safe, effective, and affordable evidence-based dermatologic treatments for older adults can be challenging because of age-related changes in the skin, comorbidities, polypharmacy, mobility issues, and cognitive changes. Phototherapy has been shown to be an effective nonpharmacologic treatment option for multiple challenging dermatologic conditions^1-8; however, few studies have specifically examined its effectiveness in older adults. The challenge for older patients with psoriasis and dermatitis is that the conditions can be difficult to control and often require multiple treatment modalities.^9,10 Patients with psoriasis also have a higher risk for diabetes, dyslipidemia, and cardiovascular disease compared to other older patients,^11,12 which poses treatment challenges and makes nonpharmacologic treatments even more appealing.

Recent studies show that phototherapy can help decrease the use of dermatologic medications. Foerster and colleagues² found that adults with psoriasis who were treated with phototherapy significantly decreased their use of topical steroids (24.5% fewer patients required steroid creams and 31.1% fewer patients required psoriasis-specific topicals)(P<.01) while their use of non–psoriasis-specific medications did not change. Click and colleagues¹³ identified a decrease in medication costs, health care utilization, and risk for immunosuppression in patients treated with phototherapy when compared to those treated with biologics and apremilast. Methotrexate is a common dermatologic medication that is highly associated with increased risks in elderly patients because of impaired immune system function and the presence of comorbidities (eg, kidney disease, obesity, diabetes, fatty liver),¹⁴ which increase in prevalence with age. Combining phototherapy with methotrexate can substantially decrease the amount of methotrexate needed to achieve disease control,¹⁵ thereby decreasing the methotrexate-associated risks. Findings from these studies suggest that a safe, effective, cost-effective, and well-tolerated nonpharmacologic alternative, such as phototherapy, is highly desirable and should be optimized. Unfortunately, most studies that report the effectiveness of phototherapy are in younger populations.

This retrospective study aimed to (1) identify the most common dermatologic conditions treated with phototherapy in older adults, (2) examine the effectiveness and safety of phototherapy in older adults, and (3) compare the outcomes with 2 similar studies in the United Kingdom¹⁶ and Turkey.¹⁷

Methods

Design, Setting, Sample, and Statistical Analysis
The institutional review boards of Kaiser Permanente Washington Health Research Institute, Seattle, and the University of Washington, Seattle, approved this study. It was conducted in a large US multispecialty health care system (Group Health, Seattle, Washington [now Kaiser Permanente Washington]) serving approximately 600,000 patients, using billing records to identify all patients treated with phototherapy between January 1, 2015, and December 31, 2015, all who received narrowband UVB (NB-UVB) phototherapy. All adults 65 years and older who received phototherapy treatment during the 12-month study period were included. Patients were included regardless of comorbidities and other dermatologic treatments to maintain as much uniformity as possible between the present study and 2 prior studies examining phototherapy in older adult populations in the United Kingdom¹⁶ and Turkey.¹⁷ Demographic and clinical factors were presented using frequencies (percentages) or means and medians as appropriate. Comparisons of dermatologic conditions and clearance levels used a Fisher exact test. The number of phototherapy treatments to clearance and total number of treatments were compared between groups of patients using independent sample t tests.

Phototherapy Protocol
All patients received treatments administered by specially trained phototherapy nurses using a Daavlin UV Series (The Daavlin Company) or an Ultralite unit (Ultralite Enterprises, Inc), both with 48 lamps. All phototherapy nurses had been previously trained to provide treatments based on standardized protocols (Table 1) and to determine the patient’s level of disease clearance using a high to low clearance scale (Table 2). Daavlin’s treatment protocols were built into the software that accompanied the units and were developed based on the American Academy of Dermatology guidelines. The starting dose for an individual patient was determined based on the estimated minimal erythema dose for each phototype. If the patient was using photosensitizing medications, then the protocol guided the nurse to start the patient at a lower dose appropriate for their phototype. Patients with vitiligo were treated with the same starting and escalation doses as patients with Fitzpatrick phototype I because of the assumption that their vitiliginous skin had an increased risk for photosensitivity. A more recent review of the evidence has indicated that this assumption was overly conservative,¹⁸ and Kaiser Permanente Washington’s vitiligo protocol has been adjusted.

Results

Patients
Billing records identified 229 total patients who received phototherapy in 2015, of whom 52 (22.7%) were at least 65 years old. The median age was 70 years (range, 65–91 years). Twenty-nine (56%) were men and 35 (67%) had previously received phototherapy treatments.

Dermatologic Conditions Treated With Phototherapy
Our primary aim was to identify the most common dermatologic conditions treated with phototherapy in older adults. Psoriasis and dermatitis were the most common conditions treated in the sample (50% [26/52] and 21% [11/52], respectively), with mycosis fungoides being the third most common (10% [5/52]) and vitiligo tied with prurigo nodularis as fourth most common (6% [3/52])(Figure 1).

Effectiveness and Safety of Phototherapy
Our secondary aim was to examine the effectiveness and safety of phototherapy in older adults. Phototherapy was effective in this population, with 50 of 52 patients (96%) achieving a high or medium level of clearance. The degree of clearance for each of the dermatologic conditions is shown in Figure 2. Psoriasis and dermatitis achieved high clearance rates in 81% (21/26) and 82% (9/11) of patients, respectively. Overall, conditions did not have significant differences in clearances rates (Fisher exact test, P=.10). On average, it took patients 33 treatments to achieve medium or high rates of clearance. Psoriasis cleared more quickly, with an average of 30.4 treatments vs 36.1 treatments for other conditions, but the difference was not significant (t test, P=.26). Patients received an average of 98 total phototherapy treatments; the median number of treatments was 81 due to many being on maintenance therapy over several months. There was no relationship between a history of treatment with phototherapy and the total number of treatments needed to achieve clearance (t test, P=.40), but interestingly, those who had a history of phototherapy took approximately 5 more treatments to achieve clearance. The present study found that a slightly larger number of men were being treated for psoriasis (15 men vs 11 women), but there was no significant difference in response rate based on gender.

Comment

Our study found that 94% of patients with psoriasis achieved clearance with an average of 30.4 treatments, which is comparable to the reported 91% response rate with an average of 30 treatments in the United Kingdom.¹⁶ The other similar study in Turkey¹⁷ reported 73.7% of psoriasis patients achieved a 75% or more improvement from baseline with an average of 42 treatments, which may reflect underlying differences in regional skin type. Of note, the scatter chart (Figure 3) shows that several patients in the present study’s analysis are listed as not clear, but many of those patients had low treatment numbers below the mean time to clearance. Thus, the present study’s response rate may have been underestimated.

In the general population, studies show that psoriasis treated with standardized phototherapy protocols typically clears with an average of 20.6 treatments.²¹ The levels of clearance were similar in our study’s older population, but more treatments were required to achieve those results, with an average of 10 more treatments needed (an additional 3.3 weeks). Similar results were found in this sample for dermatitis and mycosis fungoides, indicating comparable clearance rates and levels but a need for more treatments to achieve similar results compared to the general population.

Additionally, in the current study more patients experienced grade 1 (mild) erythema (46%) and grade 2 erythema (25%) at some point in their treatment compared with the United Kingdom¹⁶ (1.89%) and Turkey¹⁷ (35%) studies, though these side effects did not impact the clearance rate. Interestingly, the current study’s scatter chart (Figure 3) illustrates that this side effect did not seem to increase with aging in this population. If anything, the erythema response was more prevalent in the median or younger patients in the sample. Erythema may have been due to the frequent use of photosensitizing medications in older adults in the United States, some of which typically get discontinued in patients 75 years and older (eg, statins). Other potential causes might include the use of phototype vs minimal erythema dose–driven protocols, the standard utilization of protocols originally designed for psoriasis vs other condition-specific protocols, missed treatments leading to increased sensitivity, or possibly shielding mishaps (eg, not wearing a prescribed face shield). Given the number of potential causes and the possibility of overlapping factors, careful analysis is important. With NB-UVB phototherapy, near-erythemogenic doses are optimal to achieve effective treatments, but this delicate balance may be more problematic for older adults. Future studies are needed to fully determine the factors at play for this population. In the interim, it is important for phototherapy-trained nurses to consider this risk carefully in the older population. They must follow the prescribed protocols that guide them to query patients about their responses to the prior treatment (eg, erythema, tenderness, itching), photosensitizing medications, missed treatments, and placement of shielding, and then adjust the treatment dosing accordingly.

Limitations
This study had several limitations. Although clinical outcomes were recorded prospectively, the analysis was retrospective, unblinded, and not placebo controlled. It was conducted in a single organization (Group Health [now Kaiser Permanente Washington]) but did analyze data from 4 medical centers in different cities with diverse demographics and a variety of nursing staff providing the treatments. Although the vitiligo treatment protocol likely slowed the response rate for those patients with vitiligo, the numbers were small (ie, only 3 of 52 patients), so the researchers chose to include them in the current study. The sample population was relatively small, but when these data are evaluated alongside the studies in the United Kingdom¹⁶ and Turkey,¹⁷ they show a consistent picture illustrating the effectiveness and safety of phototherapy in the older population. Further epidemiologic studies could be helpful to further describe the usefulness of this modality compared with other treatments for a variety of dermatoses in this age group. Supplementary analysis specifically examining the relationship between the number and type of photosensitizing medications, frequency of erythema, and time to clearance also could be useful.

Conclusion

Older adults with a variety of dermatoses respond well to phototherapy and should have the opportunity to use it, particularly considering the potential for increased complications and costs from other treatment modalities, such as commonly used immunosuppressive pharmaceuticals. However, the current study and the comparison studies indicate that it is important to carefully consider the slower clearance rates and the potential risk for increased erythema in this population and adjust patient education and treatment dosing accordingly.

Unfortunately, many dermatology centers do not offer phototherapy because of infrastructure limitations such as space and specially trained nursing staff. Increasing accessibility of phototherapy for older adults through home treatments may be an alternative, given its effectiveness in the general population.^22,23 In addition, home phototherapy may be worth pursuing for the older population considering the challenges they may face with transportation to the clinic setting and their increased risk for serious illness if exposed to infections such as COVID-19. The COVID-19 pandemic has brought to light the need for reliable, safe, and effective treatments that can be utilized in the safety of patients’ homes and should therefore be considered as an option for older adults. Issues such as mobility and cognitive decline could pose some complicating factors, but with the help of a well-trained family member or caregiver, home phototherapy could be a viable option that improves accessibility for older patients. Future research opportunities include further examination of the slower but ultimately equivalent response to phototherapy in the older population, the influence of photosensitizing medications on phototherapy effects, and the impact of phototherapy on utilization of immunosuppressive pharmaceuticals in older adults.

Acute illnesses requiring hospitalization serve as a sentinel event, with many older adults requiring assistance with activities of daily living (ADLs) upon discharge.^1-3 Older adults who are frail experience even higher rates of hospital-associated disability, and rates of recovery to baseline functional status have varied.^4,5 Loss of independence in ADLs has been associated with nursing home (NH) utilization, caregiver burden, and mortality.⁶

To date, studies have characterized functional trajectories before and after hospitalization in older persons for broad medical conditions, noting persistence of disability and incomplete recovery to baseline functional status.⁷ Prior evaluations have also noted the long-term disabling impact of critical conditions such as acute myocardial infarction, stroke, and sepsis,^8,9 but a knowledge gap exists regarding the subsequent functional disability, recovery, and incident NH admission among older persons who are hospitalized for ambulatory care sensitive conditions (ACSCs). Often considered potentially preventable with optimal ambulatory care,^10,11 ACSCs represent acute, chronic, and vaccine-preventable conditions, including urinary tract infection, congestive heart failure, diabetes mellitus, and pneumonia. Investigating the aforementioned patient-centered measures post hospitalization could provide valuable supporting evidence for the continued recognition of ACSC-related hospitalizations in national quality payment programs set forth by the Centers for Medicare & Medicaid Services (CMS).¹² Demonstrating adverse outcomes after ACSC-related hospitalizations may help support interventions that target potentially preventable ACSC-related hospitalizations, such as home-based care or telehealth, with the goal of improving functional outcomes and reducing NH admission in older persons.

To address these gaps, we evaluated ACSC-related hospitalizations among participants of the Precipitating Events Project (PEP), a 19-year longitudinal study of community-living persons who were initially nondisabled in their basic functional activities. In the 6 months following an ACSC-related hospitalization, our objectives were to describe: (1) the 6-month course of postdischarge functional disability, (2) the cumulative monthly probability of functional recovery, and (3) the cumulative monthly probability of incident NH admission.

Study Population

Participants were drawn from the PEP study, an ongoing, prospective, longitudinal study of 754 community-dwelling persons aged 70 years or older.¹³ Potential participants were members of a large health plan in greater New Haven, Connecticut, and were enrolled from March 1998 through October 1999. As previously described,¹⁴ persons were oversampled if they were physically frail, as denoted by a timed score >10 seconds on the rapid gait test. Exclusion criteria included significant cognitive impairment with no available proxy, life expectancy less than 12 months, plans to leave the area, and inability to speak English. Participants were initially required to be nondisabled in four basic activities of daily living (bathing, dressing, walking across a room, and transferring from a chair). Eligibility was determined during a screening telephone interview and was confirmed during an in-home assessment. Of the eligible members, 75.2% agreed to participate in the project, and persons who declined to participate did not significantly differ in age or sex from those who were enrolled. The Yale Human Investigation Committee approved the study protocol, and all participants provided verbal informed consent.

Data Collection

From 1998 to 2017, comprehensive home-based assessments were completed by trained research nurses at baseline and at 18-month intervals over 234 months (except at 126 months), and telephone interviews were completed monthly through June 2018, to obtain information on disability over time. For participants who had significant cognitive impairment or who were unavailable, we interviewed a proxy informant using a rigorous protocol with demonstrated reliability and validity.¹⁴ All incident NH admissions, including both short- and long-term stays, were identified using the CMS Skilled Nursing Facility claims file and Long Term Care Minimum Data Set. Deaths were ascertained by review of obituaries and/or from a proxy informant, with a completion rate of 100%. A total of 688 participants (91.2%) had died after a median follow-up of 108 months, while 43 participants (5.7%) dropped out of the study after a median follow-up of 27 months. Among all participants, data were otherwise available for 99.2% of 85,531 monthly telephone interviews.

Assembly of Analytic Sample

PEP participants were considered for inclusion in the analytic sample if they had a hospitalization with an ACSC as the primary diagnosis on linked Medicare claims data. The complete list of ACSCs was defined using specifications from the Agency for Healthcare Research and Quality,¹⁵ and was assembled using the International Classification of Disease, Ninth Revision, Clinical Modification (ICD-9-CM) classification prior to October 1, 2015, and ICD Tenth Revision, Clinical Modification (ICD-10-CM) classification after October 1, 2015 (Appendix Table 1). Examples of ACSCs include congestive heart failure, dehydration, urinary tract infection, and angina without procedure. As performed previously,^16,17 two ACSCs (low birthweight; asthma in younger adults 18-39 years) were not included in this analysis because they were not based on full adult populations.

ACSC-related hospitalizations were included through December 2017. Participants could contribute more than one ACSC-related hospitalization over the course of the study based on the following criteria: (1) participant did not have a prior non-ACSC-related hospitalization within an 18-month interval; (2) participant did not have a prior ACSC-related hospitalization or treat-and-release emergency department (ED) visit within an 18-month interval (to ensure independence of observations if the participant was still recovering from the prior event and because some of the characteristics within Table 1 are susceptible to change in the setting of an intervening event and, hence, would not accurately reflect the status of the participant prior to ACSC-related hospitalization); (3) participant was not admitted from a NH; (4) participant did not have an in-hospital intensive care unit (ICU) stay (because persons with critical illness are a distinct population with frequent disability and prolonged recovery, as previously described¹⁸), in-hospital death, or death before first follow-up interview (because our aim was to evaluate disability and recovery after the hospitalization⁷).

Assembly of the primary analytic sample is depicted in the Appendix Figure. Of the 814 patients who were identified with ACSC-related hospitalizations, 107 had a prior non-ACSC-related hospitalization and 275 had a prior ACSC-related hospitalization or a treat-and-release ED visit within an 18-month interval. Of the remaining 432 ACSC-related hospitalizations, 181 were excluded: 114 patients were admitted from a NH, 38 had an in-hospital ICU stay, 3 died in the hospital, 11 died before their first follow-up interview, and 15 had withdrawn from the study. The primary analytic sample included the remaining 251 ACSC-related hospitalizations, contributed by 196 participants. Specifically, nine participants contributed three ACSC-related hospitalizations each, 37 participants contributed two hospitalizations each, and the remaining 150 participants contributed one hospitalization each. During the 6-month follow-up period, 40 participants contributing ACSC-related hospitalizations died after a median (interquartile range [IQR]) of 4 (2-5) months, and 1 person refused continued participation.

Comprehensive Assessments

During the comprehensive in-home assessments, data were obtained on demographic characteristics. Age was measured in years at the time of the ACSC-related hospitalization. In addition, we describe factors from the comprehensive assessment immediately prior to the ACSC-related hospitalization, grouped into two additional domains related to disability¹⁹: health-related and cognitive-psychosocial. The health-related factors included nine self-reported, physician-diagnosed chronic conditions and frailty. The cognitive-psychosocial factors included social support, cognitive impairment, and depressive symptoms.

Assessment of Disability

Complete details about the assessment of disability have been previously described.^13,14,19,20 Briefly, disability was assessed during the monthly telephone interviews, and included four basic activities (bathing, dressing, walking across a room, and transferring from a chair), five instrumental activities (shopping, housework, meal preparation, taking medications, and managing finances), and three mobility activities (walking a quarter mile, climbing a flight of stairs, and lifting or carrying 10 lb). Participants were asked, “At the present time, do you need help from another person to [complete the task]?” Disability was operationalized as the need for personal assistance or an inability to perform the task. Participants were also asked about a fourth mobility activity, “Have you driven a car during the past month?” Those who responded no were classified as being disabled in driving.¹⁹

The number of disabilities overall and for each functional domain (basic, instrumental, and mobility) was summed. Possible disability scores ranged from 0 to 13, with a score of 0 indicating complete independence in all of the items, and a score of 13 indicating complete dependence. Worse postdischarge disability was defined as a total disability score (0-13) at the first telephone interview after an ACSC-related hospitalization that was greater than the total disability score from the telephone interview immediately preceding hospitalization.

Outcome Measures

The primary outcome was the number of disabilities in all 13 basic, instrumental, and mobility activities in each of the 6 months following discharge from an ACSC-related hospitalization. To determine whether our findings were consistent across the three functional domains, we also evaluated the number of disabilities in the four basic, five instrumental, and four mobility activities separately. As secondary outcomes, we evaluated: (1) the cumulative probability of recovery within the 6-month follow-up time frame after an ACSC-related hospitalization, with “recovery” defined as return to the participant’s pre-ACSC-related hospitalization total disability score, and (2) the cumulative probability of incident NH admission within the 6 months after an ACSC-related hospitalization. Aligned with CMS and prior literature,^21,22 we defined a short-term NH stay as ≤100 days and a long-term NH stay as >100 days.

Statistical Analysis

Pre-ACSC-related hospitalization characteristics were summarized by means (SDs) and frequencies with proportions. We determined the mean number of disabilities in each of the 6 months following hospital discharge, with the prehospitalization value included as a reference point. We also determined the mean (SD) number of disabilities for the three subscales of disability (basic activities of daily living [BADLs], instrumental activities of daily living [IADLs], and mobility activities). We calculated the cumulative probability of recovery within 6 months of hospital discharge. Finally, we determined the cumulative probability of incident NH admission during the 6 months after hospital discharge.

To test the robustness of our main results, we conducted a sensitivity analysis assessing disability scores of the 150 participants that contributed only one ACSC-related hospitalization. All analyses were performed using Stata, version 16.0, statistical software (StataCorp).

Table 1 shows the characteristics of the 251 ACSC-related hospitalizations immediately prior to hospitalization. Participants’ mean (SD) age was 85.1 (6.0) years, and the mean total disability score was 5.4. The majority were female, non-Hispanic White, frail, and lived alone. As shown in Appendix Table 2, the three most common reasons for ACSC-related hospitalizations were congestive heart failure (n = 69), bacterial pneumonia (n = 53), and dehydration (n = 44).

The Figure shows the disability scores during the 6-month follow-up period for total, basic, instrumental, and mobility activities, in panels A, B, C, and D, respectively. The exact values are provided in Appendix Table 3. After hospitalization, disability scores for total, basic, instrumental, and mobility activities peaked at month 1 and tended to improve modestly over the next 5 months, but remained greater, on average, than pre-hospitalization scores. Of the 40 participants who died within the 6-month follow-up period, 36 (90%) had worse disability scores in their last month of life than in the month prior to their ACSC-related hospitalization.

Table 2 shows the cumulative probability of functional recovery after ACSC-related hospitalizations. Recovery was incomplete, with only 70% (95% CI, 64%-76%) of hospitalizations achieving a return to the pre-hospitalization total disability score within 6 months of hospitalization.

Table 3 shows the cumulative probability of incident NH admission after an ACSC-related hospitalization. Of the 251 ACSC-related hospitalizations, incident NH admission was experienced by 38% (95% CI, 32%-44%) within 1 month and 50% (95% CI, 43%-56%) within 6 months of discharge. Short-term NH stays accounted for 90 (75.6%) of the 119 incident NH admissions within the 6 months after ACSC-related hospitalizations. Sensitivity analyses yielded comparable disability scores, shown in Appendix Table 4.

In this longitudinal study of community-living older persons, we evaluated functional disability, recovery, and incident NH admission within 6 months of hospitalization for an ACSC. Our study has three major findings. First, disability scores for total, basic, instrumental, and mobility activities at months 1 to 6 of follow-up were greater on average than pre-hospitalization scores. Second, functional recovery was not achieved by 3 of 10 participants after an ACSC-related hospitalization. Third, half of them experienced an incident NH admission within 6 months of discharge from an ACSC-related hospitalization, although about three-quarters of these were short-term stays. Our findings provide evidence that older persons experience clinically meaningful adverse patient-reported outcomes after ACSC-related hospitalizations.

Prior research involving ACSCs has focused largely on rates of hospitalization as a measure of access to primary care and the associated factors predictive of ACSC-related hospitalizations,^23-26 and has not addressed subsequent patient-reported outcomes. The findings in this analysis highlight that older persons experience worsening disability immediately after an ACSC-related hospitalization, which persists for prolonged periods and often results in incomplete recovery. Prior research has assessed pre-hospitalization functional status through retrospective recall approaches,² included only older adults discharged with incident disability,³ and examined functional status after all-cause medical illness hospitalizations.⁵ Our prospective analysis extends the literature by reliably capturing pre-hospital disability scores and uniquely assessing the cohort of older persons hospitalized with ACSCs.

Our work is relevant to the continued evaluation of ACSC-related hospitalizations in national quality measurement and payment initiatives among Medicare beneficiaries. In prior evaluations of ACSC-related quality measures, stakeholders have criticized the measures for limited validity due to a lack of evidence linking each utilization outcome to other patient-centered outcomes.^10,27 Our work addresses this gap by demonstrating that ACSC-related hospitalizations are linked to persistent disability, incomplete functional recovery, and incident NH admissions. Given the large body of evidence demonstrating the priority older persons place on these patient-reported outcomes,^28,29 our work should reassure policymakers seeking to transform quality measurement programs into a more patient-oriented enterprise.

Our findings have several clinical practice, research, and policy implications. First, more-effective clinical strategies to minimize the level of care required for acute exacerbations of ACSC-related illnesses may include: (1) substituting home-based care³⁰ and telehealth interventions³¹ for traditional inpatient hospitalization, (2) making in-ED resources (ie, case management services, geriatric-focused advanced practice providers) more accessible for older persons with ACSC-related illnesses, thereby enhancing care transitions and follow-up to avoid potential current and subsequent hospitalizations, and (3) ensuring adequate ambulatory care access to all older persons, as prior work has shown variation in ACSC hospital admission rates dependent on population factors such as high-poverty neighborhoods,¹⁶ insurance status,^16,32 and race/ethnicity.³³

Clinical strategies have been narrow and not holistic for ACSCs; for example, many institutions have focused on pneumonia vaccinations to reduce hospitalizations, but our work supports the need to further evaluate the impact of preventing ACSC-related hospitalizations and their associated disabling consequences. For patients admitted to the hospital, clinical strategies, such as in-hospital or post-hospital mobility and activity programs, have been shown to be protective against hospital-associated disability.^34,35 Furthermore, hospital discharge planning could include preparing older persons for anticipated functional disabilities, associated recoveries, and NH admission after ACSC-related hospitalizations. Risk factors contributing to post-hospitalization functional disability and recovery have been identified,^19,20,36 but future work is needed to: (1) identify target populations (including those most likely to worsen) so that interventions can be offered earlier in the course of care to those who would benefit most, and (2) identify and learn from those who are resilient and have recovered, to better understand factors contributing to their success.

Our study has several strengths. First, the study is unique due to its longitudinal design, with monthly assessments of functional status. Since functional status was assessed prospectively before the ACSC-related hospitalization, we also have avoided any potential concern for recall bias that may be present if assessed after the hospitalization. Additionally, through the use of Medicare claims and the Minimum Data Set, the ascertainment of hospitalizations and NH admissions was likely complete for the studied population.

However, the study has limitations. First, functional measures were based on self-reports rather than objective measurements. Nevertheless, the self-report function is often used to guide coverage determinations in the Medicare program, as it has been shown to be associated with poor health outcomes.³⁷ Second, we are unable to comment on the rate of functional decline or NH admission when an older person was not hospitalized in relation to an ACSC. Future analyses may benefit from using a control group (eg, older adults without an ACSC hospitalization or older adults with a non-ACSC hospitalization). Third, we used strict exclusion criteria to identify a population of older adults without recent hospitalizations to determine the isolated impact of ACSC hospitalization on disability, incident NH admission, and functional recovery. Considering this potential selection bias, our findings are likely conservative estimates of the patient-centered outcomes evaluated. Fourth, participants were not asked about feeding and toileting. However, the incidence of disability in these ADLs is low among nondisabled, community-living older persons, and it is highly uncommon for disability to develop in these ADLs without concurrent disability in the ADLs within this analysis.^14,38

Finally, because our study participants were members of a single health plan in a small urban area and included nondisabled older persons living in the community, our findings may not be generalizable to geriatric patients in other settings. Nonetheless, the demographics of our cohort reflect those of older persons in New Haven County, Connecticut, which are similar to the demographics of the US population, with the exception of race and ethnicity. In addition, the generalizability of our results are strengthened by the study’s high participation rate and minimal attrition.

Within 6 months of ACSC-related hospitalizations, community-living older persons exhibited greater total disability scores than those immediately preceding hospitalization. In the same time frame, 3 of 10 older persons did not achieve functional recovery, and half experienced incident NH admission. These results provide evidence regarding the continued recognition of ACSC-related hospitalizations in federal quality measurement and payment programs and suggests the need for preventive and comprehensive interventions to meaningfully improve longitudinal outcomes.

We thank Denise Shepard, BSN, MBA, Andrea Benjamin, BSN, Barbara Foster, and Amy Shelton, MPH, for assistance with data collection; Geraldine Hawthorne, BS, for assistance with data entry and management; Peter Charpentier, MPH, for design and development of the study database and participant tracking system; and Joanne McGloin, MDiv, MBA, for leadership and advice as the Project Director. Each of these persons were paid employees of Yale School of Medicine during the conduct of this study.

Acute illnesses requiring hospitalization serve as a sentinel event, with many older adults requiring assistance with activities of daily living (ADLs) upon discharge.^1-3 Older adults who are frail experience even higher rates of hospital-associated disability, and rates of recovery to baseline functional status have varied.^4,5 Loss of independence in ADLs has been associated with nursing home (NH) utilization, caregiver burden, and mortality.⁶

To date, studies have characterized functional trajectories before and after hospitalization in older persons for broad medical conditions, noting persistence of disability and incomplete recovery to baseline functional status.⁷ Prior evaluations have also noted the long-term disabling impact of critical conditions such as acute myocardial infarction, stroke, and sepsis,^8,9 but a knowledge gap exists regarding the subsequent functional disability, recovery, and incident NH admission among older persons who are hospitalized for ambulatory care sensitive conditions (ACSCs). Often considered potentially preventable with optimal ambulatory care,^10,11 ACSCs represent acute, chronic, and vaccine-preventable conditions, including urinary tract infection, congestive heart failure, diabetes mellitus, and pneumonia. Investigating the aforementioned patient-centered measures post hospitalization could provide valuable supporting evidence for the continued recognition of ACSC-related hospitalizations in national quality payment programs set forth by the Centers for Medicare & Medicaid Services (CMS).¹² Demonstrating adverse outcomes after ACSC-related hospitalizations may help support interventions that target potentially preventable ACSC-related hospitalizations, such as home-based care or telehealth, with the goal of improving functional outcomes and reducing NH admission in older persons.

To address these gaps, we evaluated ACSC-related hospitalizations among participants of the Precipitating Events Project (PEP), a 19-year longitudinal study of community-living persons who were initially nondisabled in their basic functional activities. In the 6 months following an ACSC-related hospitalization, our objectives were to describe: (1) the 6-month course of postdischarge functional disability, (2) the cumulative monthly probability of functional recovery, and (3) the cumulative monthly probability of incident NH admission.

Study Population

Participants were drawn from the PEP study, an ongoing, prospective, longitudinal study of 754 community-dwelling persons aged 70 years or older.¹³ Potential participants were members of a large health plan in greater New Haven, Connecticut, and were enrolled from March 1998 through October 1999. As previously described,¹⁴ persons were oversampled if they were physically frail, as denoted by a timed score >10 seconds on the rapid gait test. Exclusion criteria included significant cognitive impairment with no available proxy, life expectancy less than 12 months, plans to leave the area, and inability to speak English. Participants were initially required to be nondisabled in four basic activities of daily living (bathing, dressing, walking across a room, and transferring from a chair). Eligibility was determined during a screening telephone interview and was confirmed during an in-home assessment. Of the eligible members, 75.2% agreed to participate in the project, and persons who declined to participate did not significantly differ in age or sex from those who were enrolled. The Yale Human Investigation Committee approved the study protocol, and all participants provided verbal informed consent.

Data Collection

From 1998 to 2017, comprehensive home-based assessments were completed by trained research nurses at baseline and at 18-month intervals over 234 months (except at 126 months), and telephone interviews were completed monthly through June 2018, to obtain information on disability over time. For participants who had significant cognitive impairment or who were unavailable, we interviewed a proxy informant using a rigorous protocol with demonstrated reliability and validity.¹⁴ All incident NH admissions, including both short- and long-term stays, were identified using the CMS Skilled Nursing Facility claims file and Long Term Care Minimum Data Set. Deaths were ascertained by review of obituaries and/or from a proxy informant, with a completion rate of 100%. A total of 688 participants (91.2%) had died after a median follow-up of 108 months, while 43 participants (5.7%) dropped out of the study after a median follow-up of 27 months. Among all participants, data were otherwise available for 99.2% of 85,531 monthly telephone interviews.

Assembly of Analytic Sample

PEP participants were considered for inclusion in the analytic sample if they had a hospitalization with an ACSC as the primary diagnosis on linked Medicare claims data. The complete list of ACSCs was defined using specifications from the Agency for Healthcare Research and Quality,¹⁵ and was assembled using the International Classification of Disease, Ninth Revision, Clinical Modification (ICD-9-CM) classification prior to October 1, 2015, and ICD Tenth Revision, Clinical Modification (ICD-10-CM) classification after October 1, 2015 (Appendix Table 1). Examples of ACSCs include congestive heart failure, dehydration, urinary tract infection, and angina without procedure. As performed previously,^16,17 two ACSCs (low birthweight; asthma in younger adults 18-39 years) were not included in this analysis because they were not based on full adult populations.

ACSC-related hospitalizations were included through December 2017. Participants could contribute more than one ACSC-related hospitalization over the course of the study based on the following criteria: (1) participant did not have a prior non-ACSC-related hospitalization within an 18-month interval; (2) participant did not have a prior ACSC-related hospitalization or treat-and-release emergency department (ED) visit within an 18-month interval (to ensure independence of observations if the participant was still recovering from the prior event and because some of the characteristics within Table 1 are susceptible to change in the setting of an intervening event and, hence, would not accurately reflect the status of the participant prior to ACSC-related hospitalization); (3) participant was not admitted from a NH; (4) participant did not have an in-hospital intensive care unit (ICU) stay (because persons with critical illness are a distinct population with frequent disability and prolonged recovery, as previously described¹⁸), in-hospital death, or death before first follow-up interview (because our aim was to evaluate disability and recovery after the hospitalization⁷).

Assembly of the primary analytic sample is depicted in the Appendix Figure. Of the 814 patients who were identified with ACSC-related hospitalizations, 107 had a prior non-ACSC-related hospitalization and 275 had a prior ACSC-related hospitalization or a treat-and-release ED visit within an 18-month interval. Of the remaining 432 ACSC-related hospitalizations, 181 were excluded: 114 patients were admitted from a NH, 38 had an in-hospital ICU stay, 3 died in the hospital, 11 died before their first follow-up interview, and 15 had withdrawn from the study. The primary analytic sample included the remaining 251 ACSC-related hospitalizations, contributed by 196 participants. Specifically, nine participants contributed three ACSC-related hospitalizations each, 37 participants contributed two hospitalizations each, and the remaining 150 participants contributed one hospitalization each. During the 6-month follow-up period, 40 participants contributing ACSC-related hospitalizations died after a median (interquartile range [IQR]) of 4 (2-5) months, and 1 person refused continued participation.

Comprehensive Assessments

During the comprehensive in-home assessments, data were obtained on demographic characteristics. Age was measured in years at the time of the ACSC-related hospitalization. In addition, we describe factors from the comprehensive assessment immediately prior to the ACSC-related hospitalization, grouped into two additional domains related to disability¹⁹: health-related and cognitive-psychosocial. The health-related factors included nine self-reported, physician-diagnosed chronic conditions and frailty. The cognitive-psychosocial factors included social support, cognitive impairment, and depressive symptoms.

Assessment of Disability

Complete details about the assessment of disability have been previously described.^13,14,19,20 Briefly, disability was assessed during the monthly telephone interviews, and included four basic activities (bathing, dressing, walking across a room, and transferring from a chair), five instrumental activities (shopping, housework, meal preparation, taking medications, and managing finances), and three mobility activities (walking a quarter mile, climbing a flight of stairs, and lifting or carrying 10 lb). Participants were asked, “At the present time, do you need help from another person to [complete the task]?” Disability was operationalized as the need for personal assistance or an inability to perform the task. Participants were also asked about a fourth mobility activity, “Have you driven a car during the past month?” Those who responded no were classified as being disabled in driving.¹⁹

The number of disabilities overall and for each functional domain (basic, instrumental, and mobility) was summed. Possible disability scores ranged from 0 to 13, with a score of 0 indicating complete independence in all of the items, and a score of 13 indicating complete dependence. Worse postdischarge disability was defined as a total disability score (0-13) at the first telephone interview after an ACSC-related hospitalization that was greater than the total disability score from the telephone interview immediately preceding hospitalization.

Outcome Measures

The primary outcome was the number of disabilities in all 13 basic, instrumental, and mobility activities in each of the 6 months following discharge from an ACSC-related hospitalization. To determine whether our findings were consistent across the three functional domains, we also evaluated the number of disabilities in the four basic, five instrumental, and four mobility activities separately. As secondary outcomes, we evaluated: (1) the cumulative probability of recovery within the 6-month follow-up time frame after an ACSC-related hospitalization, with “recovery” defined as return to the participant’s pre-ACSC-related hospitalization total disability score, and (2) the cumulative probability of incident NH admission within the 6 months after an ACSC-related hospitalization. Aligned with CMS and prior literature,^21,22 we defined a short-term NH stay as ≤100 days and a long-term NH stay as >100 days.

Statistical Analysis

Pre-ACSC-related hospitalization characteristics were summarized by means (SDs) and frequencies with proportions. We determined the mean number of disabilities in each of the 6 months following hospital discharge, with the prehospitalization value included as a reference point. We also determined the mean (SD) number of disabilities for the three subscales of disability (basic activities of daily living [BADLs], instrumental activities of daily living [IADLs], and mobility activities). We calculated the cumulative probability of recovery within 6 months of hospital discharge. Finally, we determined the cumulative probability of incident NH admission during the 6 months after hospital discharge.

To test the robustness of our main results, we conducted a sensitivity analysis assessing disability scores of the 150 participants that contributed only one ACSC-related hospitalization. All analyses were performed using Stata, version 16.0, statistical software (StataCorp).

Table 1 shows the characteristics of the 251 ACSC-related hospitalizations immediately prior to hospitalization. Participants’ mean (SD) age was 85.1 (6.0) years, and the mean total disability score was 5.4. The majority were female, non-Hispanic White, frail, and lived alone. As shown in Appendix Table 2, the three most common reasons for ACSC-related hospitalizations were congestive heart failure (n = 69), bacterial pneumonia (n = 53), and dehydration (n = 44).

The Figure shows the disability scores during the 6-month follow-up period for total, basic, instrumental, and mobility activities, in panels A, B, C, and D, respectively. The exact values are provided in Appendix Table 3. After hospitalization, disability scores for total, basic, instrumental, and mobility activities peaked at month 1 and tended to improve modestly over the next 5 months, but remained greater, on average, than pre-hospitalization scores. Of the 40 participants who died within the 6-month follow-up period, 36 (90%) had worse disability scores in their last month of life than in the month prior to their ACSC-related hospitalization.

Table 2 shows the cumulative probability of functional recovery after ACSC-related hospitalizations. Recovery was incomplete, with only 70% (95% CI, 64%-76%) of hospitalizations achieving a return to the pre-hospitalization total disability score within 6 months of hospitalization.

Table 3 shows the cumulative probability of incident NH admission after an ACSC-related hospitalization. Of the 251 ACSC-related hospitalizations, incident NH admission was experienced by 38% (95% CI, 32%-44%) within 1 month and 50% (95% CI, 43%-56%) within 6 months of discharge. Short-term NH stays accounted for 90 (75.6%) of the 119 incident NH admissions within the 6 months after ACSC-related hospitalizations. Sensitivity analyses yielded comparable disability scores, shown in Appendix Table 4.

In this longitudinal study of community-living older persons, we evaluated functional disability, recovery, and incident NH admission within 6 months of hospitalization for an ACSC. Our study has three major findings. First, disability scores for total, basic, instrumental, and mobility activities at months 1 to 6 of follow-up were greater on average than pre-hospitalization scores. Second, functional recovery was not achieved by 3 of 10 participants after an ACSC-related hospitalization. Third, half of them experienced an incident NH admission within 6 months of discharge from an ACSC-related hospitalization, although about three-quarters of these were short-term stays. Our findings provide evidence that older persons experience clinically meaningful adverse patient-reported outcomes after ACSC-related hospitalizations.

Prior research involving ACSCs has focused largely on rates of hospitalization as a measure of access to primary care and the associated factors predictive of ACSC-related hospitalizations,^23-26 and has not addressed subsequent patient-reported outcomes. The findings in this analysis highlight that older persons experience worsening disability immediately after an ACSC-related hospitalization, which persists for prolonged periods and often results in incomplete recovery. Prior research has assessed pre-hospitalization functional status through retrospective recall approaches,² included only older adults discharged with incident disability,³ and examined functional status after all-cause medical illness hospitalizations.⁵ Our prospective analysis extends the literature by reliably capturing pre-hospital disability scores and uniquely assessing the cohort of older persons hospitalized with ACSCs.

Our work is relevant to the continued evaluation of ACSC-related hospitalizations in national quality measurement and payment initiatives among Medicare beneficiaries. In prior evaluations of ACSC-related quality measures, stakeholders have criticized the measures for limited validity due to a lack of evidence linking each utilization outcome to other patient-centered outcomes.^10,27 Our work addresses this gap by demonstrating that ACSC-related hospitalizations are linked to persistent disability, incomplete functional recovery, and incident NH admissions. Given the large body of evidence demonstrating the priority older persons place on these patient-reported outcomes,^28,29 our work should reassure policymakers seeking to transform quality measurement programs into a more patient-oriented enterprise.

Our findings have several clinical practice, research, and policy implications. First, more-effective clinical strategies to minimize the level of care required for acute exacerbations of ACSC-related illnesses may include: (1) substituting home-based care³⁰ and telehealth interventions³¹ for traditional inpatient hospitalization, (2) making in-ED resources (ie, case management services, geriatric-focused advanced practice providers) more accessible for older persons with ACSC-related illnesses, thereby enhancing care transitions and follow-up to avoid potential current and subsequent hospitalizations, and (3) ensuring adequate ambulatory care access to all older persons, as prior work has shown variation in ACSC hospital admission rates dependent on population factors such as high-poverty neighborhoods,¹⁶ insurance status,^16,32 and race/ethnicity.³³

Clinical strategies have been narrow and not holistic for ACSCs; for example, many institutions have focused on pneumonia vaccinations to reduce hospitalizations, but our work supports the need to further evaluate the impact of preventing ACSC-related hospitalizations and their associated disabling consequences. For patients admitted to the hospital, clinical strategies, such as in-hospital or post-hospital mobility and activity programs, have been shown to be protective against hospital-associated disability.^34,35 Furthermore, hospital discharge planning could include preparing older persons for anticipated functional disabilities, associated recoveries, and NH admission after ACSC-related hospitalizations. Risk factors contributing to post-hospitalization functional disability and recovery have been identified,^19,20,36 but future work is needed to: (1) identify target populations (including those most likely to worsen) so that interventions can be offered earlier in the course of care to those who would benefit most, and (2) identify and learn from those who are resilient and have recovered, to better understand factors contributing to their success.

Our study has several strengths. First, the study is unique due to its longitudinal design, with monthly assessments of functional status. Since functional status was assessed prospectively before the ACSC-related hospitalization, we also have avoided any potential concern for recall bias that may be present if assessed after the hospitalization. Additionally, through the use of Medicare claims and the Minimum Data Set, the ascertainment of hospitalizations and NH admissions was likely complete for the studied population.

However, the study has limitations. First, functional measures were based on self-reports rather than objective measurements. Nevertheless, the self-report function is often used to guide coverage determinations in the Medicare program, as it has been shown to be associated with poor health outcomes.³⁷ Second, we are unable to comment on the rate of functional decline or NH admission when an older person was not hospitalized in relation to an ACSC. Future analyses may benefit from using a control group (eg, older adults without an ACSC hospitalization or older adults with a non-ACSC hospitalization). Third, we used strict exclusion criteria to identify a population of older adults without recent hospitalizations to determine the isolated impact of ACSC hospitalization on disability, incident NH admission, and functional recovery. Considering this potential selection bias, our findings are likely conservative estimates of the patient-centered outcomes evaluated. Fourth, participants were not asked about feeding and toileting. However, the incidence of disability in these ADLs is low among nondisabled, community-living older persons, and it is highly uncommon for disability to develop in these ADLs without concurrent disability in the ADLs within this analysis.^14,38

Finally, because our study participants were members of a single health plan in a small urban area and included nondisabled older persons living in the community, our findings may not be generalizable to geriatric patients in other settings. Nonetheless, the demographics of our cohort reflect those of older persons in New Haven County, Connecticut, which are similar to the demographics of the US population, with the exception of race and ethnicity. In addition, the generalizability of our results are strengthened by the study’s high participation rate and minimal attrition.

Within 6 months of ACSC-related hospitalizations, community-living older persons exhibited greater total disability scores than those immediately preceding hospitalization. In the same time frame, 3 of 10 older persons did not achieve functional recovery, and half experienced incident NH admission. These results provide evidence regarding the continued recognition of ACSC-related hospitalizations in federal quality measurement and payment programs and suggests the need for preventive and comprehensive interventions to meaningfully improve longitudinal outcomes.

We thank Denise Shepard, BSN, MBA, Andrea Benjamin, BSN, Barbara Foster, and Amy Shelton, MPH, for assistance with data collection; Geraldine Hawthorne, BS, for assistance with data entry and management; Peter Charpentier, MPH, for design and development of the study database and participant tracking system; and Joanne McGloin, MDiv, MBA, for leadership and advice as the Project Director. Each of these persons were paid employees of Yale School of Medicine during the conduct of this study.

Acute illnesses requiring hospitalization serve as a sentinel event, with many older adults requiring assistance with activities of daily living (ADLs) upon discharge.^1-3 Older adults who are frail experience even higher rates of hospital-associated disability, and rates of recovery to baseline functional status have varied.^4,5 Loss of independence in ADLs has been associated with nursing home (NH) utilization, caregiver burden, and mortality.⁶

To date, studies have characterized functional trajectories before and after hospitalization in older persons for broad medical conditions, noting persistence of disability and incomplete recovery to baseline functional status.⁷ Prior evaluations have also noted the long-term disabling impact of critical conditions such as acute myocardial infarction, stroke, and sepsis,^8,9 but a knowledge gap exists regarding the subsequent functional disability, recovery, and incident NH admission among older persons who are hospitalized for ambulatory care sensitive conditions (ACSCs). Often considered potentially preventable with optimal ambulatory care,^10,11 ACSCs represent acute, chronic, and vaccine-preventable conditions, including urinary tract infection, congestive heart failure, diabetes mellitus, and pneumonia. Investigating the aforementioned patient-centered measures post hospitalization could provide valuable supporting evidence for the continued recognition of ACSC-related hospitalizations in national quality payment programs set forth by the Centers for Medicare & Medicaid Services (CMS).¹² Demonstrating adverse outcomes after ACSC-related hospitalizations may help support interventions that target potentially preventable ACSC-related hospitalizations, such as home-based care or telehealth, with the goal of improving functional outcomes and reducing NH admission in older persons.

To address these gaps, we evaluated ACSC-related hospitalizations among participants of the Precipitating Events Project (PEP), a 19-year longitudinal study of community-living persons who were initially nondisabled in their basic functional activities. In the 6 months following an ACSC-related hospitalization, our objectives were to describe: (1) the 6-month course of postdischarge functional disability, (2) the cumulative monthly probability of functional recovery, and (3) the cumulative monthly probability of incident NH admission.

Study Population

Participants were drawn from the PEP study, an ongoing, prospective, longitudinal study of 754 community-dwelling persons aged 70 years or older.¹³ Potential participants were members of a large health plan in greater New Haven, Connecticut, and were enrolled from March 1998 through October 1999. As previously described,¹⁴ persons were oversampled if they were physically frail, as denoted by a timed score >10 seconds on the rapid gait test. Exclusion criteria included significant cognitive impairment with no available proxy, life expectancy less than 12 months, plans to leave the area, and inability to speak English. Participants were initially required to be nondisabled in four basic activities of daily living (bathing, dressing, walking across a room, and transferring from a chair). Eligibility was determined during a screening telephone interview and was confirmed during an in-home assessment. Of the eligible members, 75.2% agreed to participate in the project, and persons who declined to participate did not significantly differ in age or sex from those who were enrolled. The Yale Human Investigation Committee approved the study protocol, and all participants provided verbal informed consent.

Data Collection

From 1998 to 2017, comprehensive home-based assessments were completed by trained research nurses at baseline and at 18-month intervals over 234 months (except at 126 months), and telephone interviews were completed monthly through June 2018, to obtain information on disability over time. For participants who had significant cognitive impairment or who were unavailable, we interviewed a proxy informant using a rigorous protocol with demonstrated reliability and validity.¹⁴ All incident NH admissions, including both short- and long-term stays, were identified using the CMS Skilled Nursing Facility claims file and Long Term Care Minimum Data Set. Deaths were ascertained by review of obituaries and/or from a proxy informant, with a completion rate of 100%. A total of 688 participants (91.2%) had died after a median follow-up of 108 months, while 43 participants (5.7%) dropped out of the study after a median follow-up of 27 months. Among all participants, data were otherwise available for 99.2% of 85,531 monthly telephone interviews.

Assembly of Analytic Sample

PEP participants were considered for inclusion in the analytic sample if they had a hospitalization with an ACSC as the primary diagnosis on linked Medicare claims data. The complete list of ACSCs was defined using specifications from the Agency for Healthcare Research and Quality,¹⁵ and was assembled using the International Classification of Disease, Ninth Revision, Clinical Modification (ICD-9-CM) classification prior to October 1, 2015, and ICD Tenth Revision, Clinical Modification (ICD-10-CM) classification after October 1, 2015 (Appendix Table 1). Examples of ACSCs include congestive heart failure, dehydration, urinary tract infection, and angina without procedure. As performed previously,^16,17 two ACSCs (low birthweight; asthma in younger adults 18-39 years) were not included in this analysis because they were not based on full adult populations.

ACSC-related hospitalizations were included through December 2017. Participants could contribute more than one ACSC-related hospitalization over the course of the study based on the following criteria: (1) participant did not have a prior non-ACSC-related hospitalization within an 18-month interval; (2) participant did not have a prior ACSC-related hospitalization or treat-and-release emergency department (ED) visit within an 18-month interval (to ensure independence of observations if the participant was still recovering from the prior event and because some of the characteristics within Table 1 are susceptible to change in the setting of an intervening event and, hence, would not accurately reflect the status of the participant prior to ACSC-related hospitalization); (3) participant was not admitted from a NH; (4) participant did not have an in-hospital intensive care unit (ICU) stay (because persons with critical illness are a distinct population with frequent disability and prolonged recovery, as previously described¹⁸), in-hospital death, or death before first follow-up interview (because our aim was to evaluate disability and recovery after the hospitalization⁷).

Assembly of the primary analytic sample is depicted in the Appendix Figure. Of the 814 patients who were identified with ACSC-related hospitalizations, 107 had a prior non-ACSC-related hospitalization and 275 had a prior ACSC-related hospitalization or a treat-and-release ED visit within an 18-month interval. Of the remaining 432 ACSC-related hospitalizations, 181 were excluded: 114 patients were admitted from a NH, 38 had an in-hospital ICU stay, 3 died in the hospital, 11 died before their first follow-up interview, and 15 had withdrawn from the study. The primary analytic sample included the remaining 251 ACSC-related hospitalizations, contributed by 196 participants. Specifically, nine participants contributed three ACSC-related hospitalizations each, 37 participants contributed two hospitalizations each, and the remaining 150 participants contributed one hospitalization each. During the 6-month follow-up period, 40 participants contributing ACSC-related hospitalizations died after a median (interquartile range [IQR]) of 4 (2-5) months, and 1 person refused continued participation.

Comprehensive Assessments

During the comprehensive in-home assessments, data were obtained on demographic characteristics. Age was measured in years at the time of the ACSC-related hospitalization. In addition, we describe factors from the comprehensive assessment immediately prior to the ACSC-related hospitalization, grouped into two additional domains related to disability¹⁹: health-related and cognitive-psychosocial. The health-related factors included nine self-reported, physician-diagnosed chronic conditions and frailty. The cognitive-psychosocial factors included social support, cognitive impairment, and depressive symptoms.

Assessment of Disability

Complete details about the assessment of disability have been previously described.^13,14,19,20 Briefly, disability was assessed during the monthly telephone interviews, and included four basic activities (bathing, dressing, walking across a room, and transferring from a chair), five instrumental activities (shopping, housework, meal preparation, taking medications, and managing finances), and three mobility activities (walking a quarter mile, climbing a flight of stairs, and lifting or carrying 10 lb). Participants were asked, “At the present time, do you need help from another person to [complete the task]?” Disability was operationalized as the need for personal assistance or an inability to perform the task. Participants were also asked about a fourth mobility activity, “Have you driven a car during the past month?” Those who responded no were classified as being disabled in driving.¹⁹

The number of disabilities overall and for each functional domain (basic, instrumental, and mobility) was summed. Possible disability scores ranged from 0 to 13, with a score of 0 indicating complete independence in all of the items, and a score of 13 indicating complete dependence. Worse postdischarge disability was defined as a total disability score (0-13) at the first telephone interview after an ACSC-related hospitalization that was greater than the total disability score from the telephone interview immediately preceding hospitalization.

Outcome Measures

The primary outcome was the number of disabilities in all 13 basic, instrumental, and mobility activities in each of the 6 months following discharge from an ACSC-related hospitalization. To determine whether our findings were consistent across the three functional domains, we also evaluated the number of disabilities in the four basic, five instrumental, and four mobility activities separately. As secondary outcomes, we evaluated: (1) the cumulative probability of recovery within the 6-month follow-up time frame after an ACSC-related hospitalization, with “recovery” defined as return to the participant’s pre-ACSC-related hospitalization total disability score, and (2) the cumulative probability of incident NH admission within the 6 months after an ACSC-related hospitalization. Aligned with CMS and prior literature,^21,22 we defined a short-term NH stay as ≤100 days and a long-term NH stay as >100 days.

Statistical Analysis

Pre-ACSC-related hospitalization characteristics were summarized by means (SDs) and frequencies with proportions. We determined the mean number of disabilities in each of the 6 months following hospital discharge, with the prehospitalization value included as a reference point. We also determined the mean (SD) number of disabilities for the three subscales of disability (basic activities of daily living [BADLs], instrumental activities of daily living [IADLs], and mobility activities). We calculated the cumulative probability of recovery within 6 months of hospital discharge. Finally, we determined the cumulative probability of incident NH admission during the 6 months after hospital discharge.

To test the robustness of our main results, we conducted a sensitivity analysis assessing disability scores of the 150 participants that contributed only one ACSC-related hospitalization. All analyses were performed using Stata, version 16.0, statistical software (StataCorp).

Table 1 shows the characteristics of the 251 ACSC-related hospitalizations immediately prior to hospitalization. Participants’ mean (SD) age was 85.1 (6.0) years, and the mean total disability score was 5.4. The majority were female, non-Hispanic White, frail, and lived alone. As shown in Appendix Table 2, the three most common reasons for ACSC-related hospitalizations were congestive heart failure (n = 69), bacterial pneumonia (n = 53), and dehydration (n = 44).

The Figure shows the disability scores during the 6-month follow-up period for total, basic, instrumental, and mobility activities, in panels A, B, C, and D, respectively. The exact values are provided in Appendix Table 3. After hospitalization, disability scores for total, basic, instrumental, and mobility activities peaked at month 1 and tended to improve modestly over the next 5 months, but remained greater, on average, than pre-hospitalization scores. Of the 40 participants who died within the 6-month follow-up period, 36 (90%) had worse disability scores in their last month of life than in the month prior to their ACSC-related hospitalization.

Table 2 shows the cumulative probability of functional recovery after ACSC-related hospitalizations. Recovery was incomplete, with only 70% (95% CI, 64%-76%) of hospitalizations achieving a return to the pre-hospitalization total disability score within 6 months of hospitalization.

Table 3 shows the cumulative probability of incident NH admission after an ACSC-related hospitalization. Of the 251 ACSC-related hospitalizations, incident NH admission was experienced by 38% (95% CI, 32%-44%) within 1 month and 50% (95% CI, 43%-56%) within 6 months of discharge. Short-term NH stays accounted for 90 (75.6%) of the 119 incident NH admissions within the 6 months after ACSC-related hospitalizations. Sensitivity analyses yielded comparable disability scores, shown in Appendix Table 4.

In this longitudinal study of community-living older persons, we evaluated functional disability, recovery, and incident NH admission within 6 months of hospitalization for an ACSC. Our study has three major findings. First, disability scores for total, basic, instrumental, and mobility activities at months 1 to 6 of follow-up were greater on average than pre-hospitalization scores. Second, functional recovery was not achieved by 3 of 10 participants after an ACSC-related hospitalization. Third, half of them experienced an incident NH admission within 6 months of discharge from an ACSC-related hospitalization, although about three-quarters of these were short-term stays. Our findings provide evidence that older persons experience clinically meaningful adverse patient-reported outcomes after ACSC-related hospitalizations.

Prior research involving ACSCs has focused largely on rates of hospitalization as a measure of access to primary care and the associated factors predictive of ACSC-related hospitalizations,^23-26 and has not addressed subsequent patient-reported outcomes. The findings in this analysis highlight that older persons experience worsening disability immediately after an ACSC-related hospitalization, which persists for prolonged periods and often results in incomplete recovery. Prior research has assessed pre-hospitalization functional status through retrospective recall approaches,² included only older adults discharged with incident disability,³ and examined functional status after all-cause medical illness hospitalizations.⁵ Our prospective analysis extends the literature by reliably capturing pre-hospital disability scores and uniquely assessing the cohort of older persons hospitalized with ACSCs.

Our work is relevant to the continued evaluation of ACSC-related hospitalizations in national quality measurement and payment initiatives among Medicare beneficiaries. In prior evaluations of ACSC-related quality measures, stakeholders have criticized the measures for limited validity due to a lack of evidence linking each utilization outcome to other patient-centered outcomes.^10,27 Our work addresses this gap by demonstrating that ACSC-related hospitalizations are linked to persistent disability, incomplete functional recovery, and incident NH admissions. Given the large body of evidence demonstrating the priority older persons place on these patient-reported outcomes,^28,29 our work should reassure policymakers seeking to transform quality measurement programs into a more patient-oriented enterprise.

Our findings have several clinical practice, research, and policy implications. First, more-effective clinical strategies to minimize the level of care required for acute exacerbations of ACSC-related illnesses may include: (1) substituting home-based care³⁰ and telehealth interventions³¹ for traditional inpatient hospitalization, (2) making in-ED resources (ie, case management services, geriatric-focused advanced practice providers) more accessible for older persons with ACSC-related illnesses, thereby enhancing care transitions and follow-up to avoid potential current and subsequent hospitalizations, and (3) ensuring adequate ambulatory care access to all older persons, as prior work has shown variation in ACSC hospital admission rates dependent on population factors such as high-poverty neighborhoods,¹⁶ insurance status,^16,32 and race/ethnicity.³³

Clinical strategies have been narrow and not holistic for ACSCs; for example, many institutions have focused on pneumonia vaccinations to reduce hospitalizations, but our work supports the need to further evaluate the impact of preventing ACSC-related hospitalizations and their associated disabling consequences. For patients admitted to the hospital, clinical strategies, such as in-hospital or post-hospital mobility and activity programs, have been shown to be protective against hospital-associated disability.^34,35 Furthermore, hospital discharge planning could include preparing older persons for anticipated functional disabilities, associated recoveries, and NH admission after ACSC-related hospitalizations. Risk factors contributing to post-hospitalization functional disability and recovery have been identified,^19,20,36 but future work is needed to: (1) identify target populations (including those most likely to worsen) so that interventions can be offered earlier in the course of care to those who would benefit most, and (2) identify and learn from those who are resilient and have recovered, to better understand factors contributing to their success.

Our study has several strengths. First, the study is unique due to its longitudinal design, with monthly assessments of functional status. Since functional status was assessed prospectively before the ACSC-related hospitalization, we also have avoided any potential concern for recall bias that may be present if assessed after the hospitalization. Additionally, through the use of Medicare claims and the Minimum Data Set, the ascertainment of hospitalizations and NH admissions was likely complete for the studied population.

However, the study has limitations. First, functional measures were based on self-reports rather than objective measurements. Nevertheless, the self-report function is often used to guide coverage determinations in the Medicare program, as it has been shown to be associated with poor health outcomes.³⁷ Second, we are unable to comment on the rate of functional decline or NH admission when an older person was not hospitalized in relation to an ACSC. Future analyses may benefit from using a control group (eg, older adults without an ACSC hospitalization or older adults with a non-ACSC hospitalization). Third, we used strict exclusion criteria to identify a population of older adults without recent hospitalizations to determine the isolated impact of ACSC hospitalization on disability, incident NH admission, and functional recovery. Considering this potential selection bias, our findings are likely conservative estimates of the patient-centered outcomes evaluated. Fourth, participants were not asked about feeding and toileting. However, the incidence of disability in these ADLs is low among nondisabled, community-living older persons, and it is highly uncommon for disability to develop in these ADLs without concurrent disability in the ADLs within this analysis.^14,38

Finally, because our study participants were members of a single health plan in a small urban area and included nondisabled older persons living in the community, our findings may not be generalizable to geriatric patients in other settings. Nonetheless, the demographics of our cohort reflect those of older persons in New Haven County, Connecticut, which are similar to the demographics of the US population, with the exception of race and ethnicity. In addition, the generalizability of our results are strengthened by the study’s high participation rate and minimal attrition.

Within 6 months of ACSC-related hospitalizations, community-living older persons exhibited greater total disability scores than those immediately preceding hospitalization. In the same time frame, 3 of 10 older persons did not achieve functional recovery, and half experienced incident NH admission. These results provide evidence regarding the continued recognition of ACSC-related hospitalizations in federal quality measurement and payment programs and suggests the need for preventive and comprehensive interventions to meaningfully improve longitudinal outcomes.

We thank Denise Shepard, BSN, MBA, Andrea Benjamin, BSN, Barbara Foster, and Amy Shelton, MPH, for assistance with data collection; Geraldine Hawthorne, BS, for assistance with data entry and management; Peter Charpentier, MPH, for design and development of the study database and participant tracking system; and Joanne McGloin, MDiv, MBA, for leadership and advice as the Project Director. Each of these persons were paid employees of Yale School of Medicine during the conduct of this study.

One of the most striking features of the early COVID-19 pandemic was the sudden and sharp reductions in emergency department (ED) visits and hospitalizations throughout the United States.^1-4 Several studies have documented lower rates of hospitalization for many emergent, time-sensitive conditions, such as acute myocardial infarction, stroke, and hyperglycemic crises, starting shortly after community transmission of COVID-19 was recognized and social distancing guidelines were implemented.^5-8 In most cases, hospital volumes rebounded after an initial drop, stabilizing at somewhat lower levels than those expected from historic trends.⁹

The observed shifts in hospital use largely have been attributed to patients’ forgoing or delaying necessary care,¹⁰ which underscores the indirect effects of the pandemic on patients without COVID-19.¹¹ To date, the extent to which outcomes for patients without COVID-19 have been adversely affected is less well understood. Evidence suggests patients with acute and chronic illnesses have experienced increased morbidity and mortality since the onset of the pandemic. For example, in northern California, abrupt declines in ED visits for cardiac symptoms were coupled with higher rates of out-of-hospital cardiac arrest.¹² Moreover, states with higher rates of COVID-19 also reported increased deaths attributed to heart disease, diabetes, and other conditions.¹³

To better understand these potential indirect effects, this study used data from a large, multistate health care system to examine changes in hospital volume and its relationship to in-hospital mortality for patients without COVID-19 during the first 10 months of the pandemic.

Setting and Participants

We examined unplanned hospitalizations from January 2019 to December 2020 at 51 community hospitals across 6 states (Alaska, Washington, Montana, Oregon, California, and Texas) in the Providence St. Joseph Health system. Hospitals within the Providence system share a common standard dataset for each encounter with a centralized cloud data warehouse from which we extracted clinical and demographic data. No hospitals entered or left the system during the study period. Hospitalizations were considered unplanned if they had an “urgent” or “emergency” service type in the record; most originated in the ED. Hospitalizations for children younger than 18 years and those with evidence of COVID-19 (International Classification of Disease, Tenth Revision, Clinical Modification U07.1, a positive COVID-19 polymerase chain reaction test during the encounter, or an infection control-assigned label of COVID-19) were excluded. The Providence St. Joseph Health Institutional Review Board approved this study.

Measures

Trends in daily hospitalizations and their relationship to adjusted in-hospital mortality (percentage of patients who died during their hospital admission) were examined over time. In preliminary models using segmented regression, we identified three distinct pandemic periods with different trends in daily hospitalizations: (1) a 10-week period corresponding to the spring COVID-19 surge (March 4 to May 13, 2020; Period 1), (2) an intervening period extending over the summer and early fall (May 14 to October 19, 2020; Period 2), and (3) a second 10-week period corresponding to the fall COVID-19 surge (October 20 to December 31, 2020; Period 3). In-hospital mortality for these periods was compared with a baseline period (pre-COVID-19) from January 1, 2019 to March 3, 2020. To further assess differences in mortality by clinical condition, hospitalizations were first grouped by primary diagnosis using Clinical Classifications Software Refined (CCSR) categories from the Agency for Healthcare Research and Quality¹⁴ and ranked by the number of observed deaths and the percentage of patients who died while hospitalized in 2020. We selected common conditions that had >35 total deaths and an in-hospital mortality rate ≥1% for condition-specific analyses, of which 30 met these criteria.

Analysis

Multivariate logistic regression was used to evaluate changes in mortality for each of the pandemic periods compared with baseline for the overall cohort and selected diagnosis groups. Our main model adjusted for age, sex, race/ethnicity (White, Black, Latinx, Asian or Pacific Islander, and other), primary payor (commercial, Medicaid, Medicare, other, and self-pay), the presence or absence of 31 chronic comorbidities in the medical record, primary admitting diagnosis grouped by CCSR category (456 total diagnostic groups), and hospital fixed-effects to account for clustering. Results are expressed as the average marginal effects of each pandemic period on in-hospital mortality (eg, adjusted percentage point change in mortality over baseline). The number of excess deaths in each period was calculated by multiplying the estimated percentage point change in mortality for each period by the total number of hospitalizations. These excess deaths were subtracted from the number of observed deaths to derive the number of deaths that would be expected if pre-pandemic mortality rates persisted.

To further assess whether changes in adjusted mortality could be attributed to a smaller, sicker population of patients presenting to the hospital during the pandemic (meaning that less acutely ill patients stayed home), we conducted two sensitivity analyses. First, we tested whether substituting indicators for Medicare Severity Diagnosis Groups (MS-DRG) in lieu of CCSR categories had any impact on our results. MS-DRGs are designed to account for a patient’s illness severity and expected costs, whereas CCSR categories do not.¹⁵ MS-DRGs also better distinguish between surgical versus medical conditions. We re-ran our main model using indicators for CCSR to control for diagnostic mix, but further adjusted for severity using the DRG weight for the primary diagnosis and Modified Early Warning Score (MEWS) as continuous covariates. MEWS is a physiologic scoring system that incorporates abnormal vital signs and data related to mental status during the first 24 hours of a patient’s hospitalization into a risk-based score that has been shown to predict hospital mortality and need for intensive care.^16,17 These sensitivity analyses were performed on a subset of inpatient admissions because DRG data are not available for hospitalizations billed as an observation stay, and only approximately 70% of hospitals in the sample contributed vital sign data to the Providence data warehouse. All statistical analyses were conducted with R, version 3.6.3 (R Foundation for Statistical Computing) and SAS Enterprise Guide 7.1 (SAS Institute Inc).

The characteristics of our sample are described in Table 1. A total of 61,300, 159,430, and 65,923 hospitalizations occurred in each of the three pandemic periods, respectively, compared with 503,190 hospitalizations in the pre-pandemic period. The mean (SD) age of patients in the study was 63.2 (19.4) years; most were women (52.4%), White (70.6%), and had Medicare as their primary payor (53.7%). Less than half (42.7%) of hospitalizations occurred in California, and just under one-quarter were observation stays (23.2%). Patient characteristics were similar in the pre-COVID-19 and COVID-19 pandemic periods.

Figure 1 shows trends in hospital volume and mortality. Overall daily hospitalizations declined abruptly from a mean of 1176 per day in the pre-pandemic period to 617 per day (47.5% relative decrease) during the first 3 weeks of Period 1. Mean daily hospitalizations began to rise over the next 2 months (Period 1), reaching steady state at <1000 hospitalizations per day (15% relative decrease from baseline) during Period 2. During Period 3, we observed a decline in mean daily hospitalizations, with a low point of 882 per day on December 31, 2020 (25% relative decrease from baseline), corresponding to the end of our study period. Although hospital volumes declined during both COVID-19 surge periods, the percentage of patients who died during their hospitalization increased. There was an initial spike in in-hospital mortality that peaked approximately 1 month into the pandemic (middle of Period 1), a return to levels at or slightly below that before the pandemic by the beginning of Period 2, and then a rise throughout the autumn COVID-19 surge in Period 3, not yet peaking by the end of the study.

Adjusted in-hospital mortality for the three COVID-19 periods compared with the pre-pandemic period is presented in Table 2. The percentage of patients who died during their hospitalization rose from 2.9% in the pre-pandemic period to 3.4% during Period 1 (absolute difference, 0.6 percentage points; 95% CI, 0.5-0.7), corresponding to a 19.3% relative increase during the spring COVID-19 surge. Among the subset of patients hospitalized with 1 of the 30 conditions selected for individual analysis, mortality increased from 5.0% to 5.9% during the same time period (absolute difference, 0.9 percentage points; 95% CI, 0.8-1.1), corresponding to an 18.9% relative increase. In Period 2, in-hospital mortality was similar to that noted pre-pandemic for the overall cohort and the 30 selected conditions. During Period 3, in-hospital mortality increased by a magnitude similar to that observed in Period 1 for all hospitalizations combined (absolute difference, 0.5 percentage points; 95% CI, 0.0-0.6; corresponding to a 16.5% relative increase) as well as the subgroup with 1 of the 30 selected conditions (0.9 percentage points; 95% CI, 0.8-1.0; corresponding to an 18% relative increase). Further adjustment for severity by swapping CCSR categories with MS-DRG indicators or inclusion of DRG weight and MEWS score as covariates in our sensitivity analyses did not change our results.

Table 3 and the Appendix Figure describe changes in volume and adjusted in-hospital mortality for the 30 conditions selected for analysis. There was a decrease in the mean daily admissions for all conditions studied. Among the 30 conditions, 26 showed increased mortality during Period 1, although the increase was only statistically significant for 16 of these conditions. Among the 10 most commonly admitted conditions (by number of daily hospital admissions during the baseline period), there was a statistically significant relative increase in mortality for patients with sepsis (20.1%), heart failure (17.6%), ischemic stroke (12.5%), device/graft/surgical complications (14.0%), cardiac dysrhythmias (14.4%), pneumonia (24.5%), respiratory failure (16.1%), and gastrointestinal hemorrhage (23.3%). In general, mortality returned to baseline or improved during Period 2. Thereafter, all 30 conditions showed increased mortality in Period 3. This increase was significant for only 16 conditions, which were not the same ones noted during Period 1. Of note, although there was higher mortality for some cardiovascular conditions (heart failure cardiac dysrhythmias), mortality for myocardial infarction remained unchanged from baseline across all 3 periods. In contrast, several solid cancer–related conditions showed progressively worsening mortality throughout the study, with 7.7% higher mortality in Period 1, 10.3% higher mortality in Period 2, and 16.5% higher mortality in Period 3, respectively, compared with baseline. Although a similar pattern was observed for acute renal failure and some neurologic conditions (traumatic brain injury, seizure, other nervous system disorders), mortality for drug poisonings and gastrointestinal bleeds improved over time.

In this study of unplanned hospitalizations from 51 community hospitals across 6 states in the US West, we found a significant increase in mortality—at a rate of approximately 5 to 6 excess deaths per 1000 hospitalizations—among patients admitted during the pandemic with a variety of non-COVID-19 illnesses and injuries. Higher in-hospital mortality was observed in the spring (March to May) and fall (October to December) of 2020 when COVID-19 case counts surged and shelter-in-place mandates were implemented. With the initial surge, higher mortality rates were largely transient, and, for most conditions evaluated, returned to baseline approximately 3 months after the pandemic onset. For the fall surge, mortality rates had not peaked by the end of the study period. Changes in mortality were closely and inversely correlated with hospital volume for non-COVID-19 illnesses during both surge periods.

Higher morbidity and mortality for patients without COVID-19 appears to be an unfortunate spillover effect that has been reported in several studies. Recent work examining national surveillance data suggest that up to one-third of excess deaths (deaths higher than those expected for season) early in the pandemic have occurred among patients without known COVID-19.^13,18-20 Specifically, these studies estimate that mortality rates in the United States increased by 15% to 19% in the spring of 2020; of the identified excess deaths, only 38% to 77% could be attributed to COVID-19, with the remainder attributed to cardiovascular disease, diabetes, and Alzheimer’s disease, among others. In addition, reports from several European countries and China examining population death data have found similar trends,^21-25 as well as a recent study examining excess deaths in nursing homes.²⁶ Our results are largely consistent with these earlier studies in that we describe higher mortality in a sample of patients hospitalized with a variety of common conditions that otherwise are routinely treated in US hospitals. Reporting these indirect casualties of COVID-19 is important to fully understand the pandemic’s toll on patients and healthcare systems.

Our work builds on the current body of literature, highlighting the consistent relationship between rising COVID-19 case counts, hospital volume, and excess mortality over more than one surge period. Although several studies have looked at trends in hospital admissions or population mortality rates, few have examined the two outcomes together. The close correlation between daily hospital admissions and in-hospital mortality in this study suggests that the pandemic changed how patients use healthcare resources in ways that were important for their health and outcomes. The higher mortality rate that we and others have observed likely is related to patients’ delaying care because of fear of contracting COVID-19. In one survey, more than 4 in 10 adults in the United States reported that they avoided medical care during the early pandemic.¹⁰ Importantly, even a few days delay for many conditions, such as heart failure or sepsis, can result in precipitous declines in clinical status and outcomes.

It also is possible that we found increased rates of in-hospital mortality simply because patients with more moderate illness chose to stay home, resulting in a patient population enriched with those more likely to die. We found mixed evidence in our data that the observed increases in mortality could be attributable to a smaller, sicker population. Some characteristics that might be protective, such as a slightly younger mean age and lower mean DRG weight, were more common among those hospitalized during the pandemic. However, other characteristics, such as a slightly higher MEWS score and a greater percentage of total hospitalizations in the higher mortality subgroup, also were noted during the pandemic (Table 1). We do note, however, that the differences in these severity-related characteristics were small across the study periods. Further adjusting for these characteristics in our sensitivity analyses did not appreciably change our main findings, suggesting that the mortality increase could not be explained by changes in case-mix alone.

Other factors not dependent on patient behavior, such as barriers to accessing timely ambulatory care and impacts in the quality of care delivered, might have contributed. Shelter-in-place orders, reduced in-person access to clinicians in the ambulatory setting, slow implementation of telehealth services (with uncertainty about their equivalence to in-person exams), as well as delays in diagnostic tests and outpatient procedures could have played a role, especially during early months of the pandemic.²⁷ Significant changes to ambulatory health care delivery might have left many patients with chronic illnesses or complex medical needs with limited care options. Importantly, these care interruptions might have had greater implications for some patients, such as those with cancer who rely on intensive, largely outpatient-based treatment.^28,29 This, in part, could explain why we found persistently increased mortality among patients hospitalized with cancer after the spring surge. Later into the pandemic, however, most health systems had developed processes that allowed clinicians to resume timely care of ambulatory patients. Because of this, increases in mortality observed during the fall surge likely stem from other factors, such as patient behavior.

It is possible that care delays or changes in the quality of care delivered during the index hospitalization or pre-hospital setting might have contributed to the observed increase in mortality. This is particularly true for acute, time-sensitive conditions such as sepsis and stroke. Extra time spent donning personal protective equipment and/or new protocols instituted during the pandemic likely impacted the speed of emergency medical services transport, timeliness of ED evaluation, and delivery of definitive therapy. Although most hospitals in this study were not overwhelmed by the pandemic, the complexities associated with caring for known and suspected COVID-19 patients alongside those without the disease might have altered ideal care practices and strained healthcare teams.³⁰ In addition, nearly all hospitalized patients during this period were deprived of in-person advocacy by family members, who were not permitted to visit.

Important limitations with this study exist. First, the data come only from hospitals in the western United States. Second, some data elements such as triage scores or vital signs were not available for the entire population, potentially limiting some risk-adjustment. Third, we were unable to determine the root cause of excess mortality based on our study design and the coded variables available. It is unknown to what extent undiagnosed COVID-19 played a role. Early in the pandemic, many community hospitals did not have access to timely COVID-19 testing, and some cases might have not been diagnosed.³¹ However, we do not expect this to be a significant concern in the later months of the pandemic, as testing became more widespread and hospitals implemented surveillance screening for COVID-19 for inpatients.

Our study indicates that the COVID-19 pandemic was associated with increased mortality among patients hospitalized for a range of clinical conditions. Although higher observed mortality rates were limited to periods of high COVID-19 activity, future studies will need to tease out the extent to which these findings relate to patient factors (ie, delayed presentation and more severe disease) or systemic factors (reduction in access or changes in quality of care). This is of key importance, and appropriate solutions will need to be developed to mitigate adverse impacts with this and future pandemics.

One of the most striking features of the early COVID-19 pandemic was the sudden and sharp reductions in emergency department (ED) visits and hospitalizations throughout the United States.^1-4 Several studies have documented lower rates of hospitalization for many emergent, time-sensitive conditions, such as acute myocardial infarction, stroke, and hyperglycemic crises, starting shortly after community transmission of COVID-19 was recognized and social distancing guidelines were implemented.^5-8 In most cases, hospital volumes rebounded after an initial drop, stabilizing at somewhat lower levels than those expected from historic trends.⁹

The observed shifts in hospital use largely have been attributed to patients’ forgoing or delaying necessary care,¹⁰ which underscores the indirect effects of the pandemic on patients without COVID-19.¹¹ To date, the extent to which outcomes for patients without COVID-19 have been adversely affected is less well understood. Evidence suggests patients with acute and chronic illnesses have experienced increased morbidity and mortality since the onset of the pandemic. For example, in northern California, abrupt declines in ED visits for cardiac symptoms were coupled with higher rates of out-of-hospital cardiac arrest.¹² Moreover, states with higher rates of COVID-19 also reported increased deaths attributed to heart disease, diabetes, and other conditions.¹³

To better understand these potential indirect effects, this study used data from a large, multistate health care system to examine changes in hospital volume and its relationship to in-hospital mortality for patients without COVID-19 during the first 10 months of the pandemic.

Setting and Participants

We examined unplanned hospitalizations from January 2019 to December 2020 at 51 community hospitals across 6 states (Alaska, Washington, Montana, Oregon, California, and Texas) in the Providence St. Joseph Health system. Hospitals within the Providence system share a common standard dataset for each encounter with a centralized cloud data warehouse from which we extracted clinical and demographic data. No hospitals entered or left the system during the study period. Hospitalizations were considered unplanned if they had an “urgent” or “emergency” service type in the record; most originated in the ED. Hospitalizations for children younger than 18 years and those with evidence of COVID-19 (International Classification of Disease, Tenth Revision, Clinical Modification U07.1, a positive COVID-19 polymerase chain reaction test during the encounter, or an infection control-assigned label of COVID-19) were excluded. The Providence St. Joseph Health Institutional Review Board approved this study.

Measures

Trends in daily hospitalizations and their relationship to adjusted in-hospital mortality (percentage of patients who died during their hospital admission) were examined over time. In preliminary models using segmented regression, we identified three distinct pandemic periods with different trends in daily hospitalizations: (1) a 10-week period corresponding to the spring COVID-19 surge (March 4 to May 13, 2020; Period 1), (2) an intervening period extending over the summer and early fall (May 14 to October 19, 2020; Period 2), and (3) a second 10-week period corresponding to the fall COVID-19 surge (October 20 to December 31, 2020; Period 3). In-hospital mortality for these periods was compared with a baseline period (pre-COVID-19) from January 1, 2019 to March 3, 2020. To further assess differences in mortality by clinical condition, hospitalizations were first grouped by primary diagnosis using Clinical Classifications Software Refined (CCSR) categories from the Agency for Healthcare Research and Quality¹⁴ and ranked by the number of observed deaths and the percentage of patients who died while hospitalized in 2020. We selected common conditions that had >35 total deaths and an in-hospital mortality rate ≥1% for condition-specific analyses, of which 30 met these criteria.

Analysis

Multivariate logistic regression was used to evaluate changes in mortality for each of the pandemic periods compared with baseline for the overall cohort and selected diagnosis groups. Our main model adjusted for age, sex, race/ethnicity (White, Black, Latinx, Asian or Pacific Islander, and other), primary payor (commercial, Medicaid, Medicare, other, and self-pay), the presence or absence of 31 chronic comorbidities in the medical record, primary admitting diagnosis grouped by CCSR category (456 total diagnostic groups), and hospital fixed-effects to account for clustering. Results are expressed as the average marginal effects of each pandemic period on in-hospital mortality (eg, adjusted percentage point change in mortality over baseline). The number of excess deaths in each period was calculated by multiplying the estimated percentage point change in mortality for each period by the total number of hospitalizations. These excess deaths were subtracted from the number of observed deaths to derive the number of deaths that would be expected if pre-pandemic mortality rates persisted.

To further assess whether changes in adjusted mortality could be attributed to a smaller, sicker population of patients presenting to the hospital during the pandemic (meaning that less acutely ill patients stayed home), we conducted two sensitivity analyses. First, we tested whether substituting indicators for Medicare Severity Diagnosis Groups (MS-DRG) in lieu of CCSR categories had any impact on our results. MS-DRGs are designed to account for a patient’s illness severity and expected costs, whereas CCSR categories do not.¹⁵ MS-DRGs also better distinguish between surgical versus medical conditions. We re-ran our main model using indicators for CCSR to control for diagnostic mix, but further adjusted for severity using the DRG weight for the primary diagnosis and Modified Early Warning Score (MEWS) as continuous covariates. MEWS is a physiologic scoring system that incorporates abnormal vital signs and data related to mental status during the first 24 hours of a patient’s hospitalization into a risk-based score that has been shown to predict hospital mortality and need for intensive care.^16,17 These sensitivity analyses were performed on a subset of inpatient admissions because DRG data are not available for hospitalizations billed as an observation stay, and only approximately 70% of hospitals in the sample contributed vital sign data to the Providence data warehouse. All statistical analyses were conducted with R, version 3.6.3 (R Foundation for Statistical Computing) and SAS Enterprise Guide 7.1 (SAS Institute Inc).

The characteristics of our sample are described in Table 1. A total of 61,300, 159,430, and 65,923 hospitalizations occurred in each of the three pandemic periods, respectively, compared with 503,190 hospitalizations in the pre-pandemic period. The mean (SD) age of patients in the study was 63.2 (19.4) years; most were women (52.4%), White (70.6%), and had Medicare as their primary payor (53.7%). Less than half (42.7%) of hospitalizations occurred in California, and just under one-quarter were observation stays (23.2%). Patient characteristics were similar in the pre-COVID-19 and COVID-19 pandemic periods.

Figure 1 shows trends in hospital volume and mortality. Overall daily hospitalizations declined abruptly from a mean of 1176 per day in the pre-pandemic period to 617 per day (47.5% relative decrease) during the first 3 weeks of Period 1. Mean daily hospitalizations began to rise over the next 2 months (Period 1), reaching steady state at <1000 hospitalizations per day (15% relative decrease from baseline) during Period 2. During Period 3, we observed a decline in mean daily hospitalizations, with a low point of 882 per day on December 31, 2020 (25% relative decrease from baseline), corresponding to the end of our study period. Although hospital volumes declined during both COVID-19 surge periods, the percentage of patients who died during their hospitalization increased. There was an initial spike in in-hospital mortality that peaked approximately 1 month into the pandemic (middle of Period 1), a return to levels at or slightly below that before the pandemic by the beginning of Period 2, and then a rise throughout the autumn COVID-19 surge in Period 3, not yet peaking by the end of the study.

Adjusted in-hospital mortality for the three COVID-19 periods compared with the pre-pandemic period is presented in Table 2. The percentage of patients who died during their hospitalization rose from 2.9% in the pre-pandemic period to 3.4% during Period 1 (absolute difference, 0.6 percentage points; 95% CI, 0.5-0.7), corresponding to a 19.3% relative increase during the spring COVID-19 surge. Among the subset of patients hospitalized with 1 of the 30 conditions selected for individual analysis, mortality increased from 5.0% to 5.9% during the same time period (absolute difference, 0.9 percentage points; 95% CI, 0.8-1.1), corresponding to an 18.9% relative increase. In Period 2, in-hospital mortality was similar to that noted pre-pandemic for the overall cohort and the 30 selected conditions. During Period 3, in-hospital mortality increased by a magnitude similar to that observed in Period 1 for all hospitalizations combined (absolute difference, 0.5 percentage points; 95% CI, 0.0-0.6; corresponding to a 16.5% relative increase) as well as the subgroup with 1 of the 30 selected conditions (0.9 percentage points; 95% CI, 0.8-1.0; corresponding to an 18% relative increase). Further adjustment for severity by swapping CCSR categories with MS-DRG indicators or inclusion of DRG weight and MEWS score as covariates in our sensitivity analyses did not change our results.

Table 3 and the Appendix Figure describe changes in volume and adjusted in-hospital mortality for the 30 conditions selected for analysis. There was a decrease in the mean daily admissions for all conditions studied. Among the 30 conditions, 26 showed increased mortality during Period 1, although the increase was only statistically significant for 16 of these conditions. Among the 10 most commonly admitted conditions (by number of daily hospital admissions during the baseline period), there was a statistically significant relative increase in mortality for patients with sepsis (20.1%), heart failure (17.6%), ischemic stroke (12.5%), device/graft/surgical complications (14.0%), cardiac dysrhythmias (14.4%), pneumonia (24.5%), respiratory failure (16.1%), and gastrointestinal hemorrhage (23.3%). In general, mortality returned to baseline or improved during Period 2. Thereafter, all 30 conditions showed increased mortality in Period 3. This increase was significant for only 16 conditions, which were not the same ones noted during Period 1. Of note, although there was higher mortality for some cardiovascular conditions (heart failure cardiac dysrhythmias), mortality for myocardial infarction remained unchanged from baseline across all 3 periods. In contrast, several solid cancer–related conditions showed progressively worsening mortality throughout the study, with 7.7% higher mortality in Period 1, 10.3% higher mortality in Period 2, and 16.5% higher mortality in Period 3, respectively, compared with baseline. Although a similar pattern was observed for acute renal failure and some neurologic conditions (traumatic brain injury, seizure, other nervous system disorders), mortality for drug poisonings and gastrointestinal bleeds improved over time.

In this study of unplanned hospitalizations from 51 community hospitals across 6 states in the US West, we found a significant increase in mortality—at a rate of approximately 5 to 6 excess deaths per 1000 hospitalizations—among patients admitted during the pandemic with a variety of non-COVID-19 illnesses and injuries. Higher in-hospital mortality was observed in the spring (March to May) and fall (October to December) of 2020 when COVID-19 case counts surged and shelter-in-place mandates were implemented. With the initial surge, higher mortality rates were largely transient, and, for most conditions evaluated, returned to baseline approximately 3 months after the pandemic onset. For the fall surge, mortality rates had not peaked by the end of the study period. Changes in mortality were closely and inversely correlated with hospital volume for non-COVID-19 illnesses during both surge periods.

Higher morbidity and mortality for patients without COVID-19 appears to be an unfortunate spillover effect that has been reported in several studies. Recent work examining national surveillance data suggest that up to one-third of excess deaths (deaths higher than those expected for season) early in the pandemic have occurred among patients without known COVID-19.^13,18-20 Specifically, these studies estimate that mortality rates in the United States increased by 15% to 19% in the spring of 2020; of the identified excess deaths, only 38% to 77% could be attributed to COVID-19, with the remainder attributed to cardiovascular disease, diabetes, and Alzheimer’s disease, among others. In addition, reports from several European countries and China examining population death data have found similar trends,^21-25 as well as a recent study examining excess deaths in nursing homes.²⁶ Our results are largely consistent with these earlier studies in that we describe higher mortality in a sample of patients hospitalized with a variety of common conditions that otherwise are routinely treated in US hospitals. Reporting these indirect casualties of COVID-19 is important to fully understand the pandemic’s toll on patients and healthcare systems.

Our work builds on the current body of literature, highlighting the consistent relationship between rising COVID-19 case counts, hospital volume, and excess mortality over more than one surge period. Although several studies have looked at trends in hospital admissions or population mortality rates, few have examined the two outcomes together. The close correlation between daily hospital admissions and in-hospital mortality in this study suggests that the pandemic changed how patients use healthcare resources in ways that were important for their health and outcomes. The higher mortality rate that we and others have observed likely is related to patients’ delaying care because of fear of contracting COVID-19. In one survey, more than 4 in 10 adults in the United States reported that they avoided medical care during the early pandemic.¹⁰ Importantly, even a few days delay for many conditions, such as heart failure or sepsis, can result in precipitous declines in clinical status and outcomes.

It also is possible that we found increased rates of in-hospital mortality simply because patients with more moderate illness chose to stay home, resulting in a patient population enriched with those more likely to die. We found mixed evidence in our data that the observed increases in mortality could be attributable to a smaller, sicker population. Some characteristics that might be protective, such as a slightly younger mean age and lower mean DRG weight, were more common among those hospitalized during the pandemic. However, other characteristics, such as a slightly higher MEWS score and a greater percentage of total hospitalizations in the higher mortality subgroup, also were noted during the pandemic (Table 1). We do note, however, that the differences in these severity-related characteristics were small across the study periods. Further adjusting for these characteristics in our sensitivity analyses did not appreciably change our main findings, suggesting that the mortality increase could not be explained by changes in case-mix alone.

Other factors not dependent on patient behavior, such as barriers to accessing timely ambulatory care and impacts in the quality of care delivered, might have contributed. Shelter-in-place orders, reduced in-person access to clinicians in the ambulatory setting, slow implementation of telehealth services (with uncertainty about their equivalence to in-person exams), as well as delays in diagnostic tests and outpatient procedures could have played a role, especially during early months of the pandemic.²⁷ Significant changes to ambulatory health care delivery might have left many patients with chronic illnesses or complex medical needs with limited care options. Importantly, these care interruptions might have had greater implications for some patients, such as those with cancer who rely on intensive, largely outpatient-based treatment.^28,29 This, in part, could explain why we found persistently increased mortality among patients hospitalized with cancer after the spring surge. Later into the pandemic, however, most health systems had developed processes that allowed clinicians to resume timely care of ambulatory patients. Because of this, increases in mortality observed during the fall surge likely stem from other factors, such as patient behavior.

It is possible that care delays or changes in the quality of care delivered during the index hospitalization or pre-hospital setting might have contributed to the observed increase in mortality. This is particularly true for acute, time-sensitive conditions such as sepsis and stroke. Extra time spent donning personal protective equipment and/or new protocols instituted during the pandemic likely impacted the speed of emergency medical services transport, timeliness of ED evaluation, and delivery of definitive therapy. Although most hospitals in this study were not overwhelmed by the pandemic, the complexities associated with caring for known and suspected COVID-19 patients alongside those without the disease might have altered ideal care practices and strained healthcare teams.³⁰ In addition, nearly all hospitalized patients during this period were deprived of in-person advocacy by family members, who were not permitted to visit.

Important limitations with this study exist. First, the data come only from hospitals in the western United States. Second, some data elements such as triage scores or vital signs were not available for the entire population, potentially limiting some risk-adjustment. Third, we were unable to determine the root cause of excess mortality based on our study design and the coded variables available. It is unknown to what extent undiagnosed COVID-19 played a role. Early in the pandemic, many community hospitals did not have access to timely COVID-19 testing, and some cases might have not been diagnosed.³¹ However, we do not expect this to be a significant concern in the later months of the pandemic, as testing became more widespread and hospitals implemented surveillance screening for COVID-19 for inpatients.

Our study indicates that the COVID-19 pandemic was associated with increased mortality among patients hospitalized for a range of clinical conditions. Although higher observed mortality rates were limited to periods of high COVID-19 activity, future studies will need to tease out the extent to which these findings relate to patient factors (ie, delayed presentation and more severe disease) or systemic factors (reduction in access or changes in quality of care). This is of key importance, and appropriate solutions will need to be developed to mitigate adverse impacts with this and future pandemics.

One of the most striking features of the early COVID-19 pandemic was the sudden and sharp reductions in emergency department (ED) visits and hospitalizations throughout the United States.^1-4 Several studies have documented lower rates of hospitalization for many emergent, time-sensitive conditions, such as acute myocardial infarction, stroke, and hyperglycemic crises, starting shortly after community transmission of COVID-19 was recognized and social distancing guidelines were implemented.^5-8 In most cases, hospital volumes rebounded after an initial drop, stabilizing at somewhat lower levels than those expected from historic trends.⁹

The observed shifts in hospital use largely have been attributed to patients’ forgoing or delaying necessary care,¹⁰ which underscores the indirect effects of the pandemic on patients without COVID-19.¹¹ To date, the extent to which outcomes for patients without COVID-19 have been adversely affected is less well understood. Evidence suggests patients with acute and chronic illnesses have experienced increased morbidity and mortality since the onset of the pandemic. For example, in northern California, abrupt declines in ED visits for cardiac symptoms were coupled with higher rates of out-of-hospital cardiac arrest.¹² Moreover, states with higher rates of COVID-19 also reported increased deaths attributed to heart disease, diabetes, and other conditions.¹³

To better understand these potential indirect effects, this study used data from a large, multistate health care system to examine changes in hospital volume and its relationship to in-hospital mortality for patients without COVID-19 during the first 10 months of the pandemic.

Setting and Participants

We examined unplanned hospitalizations from January 2019 to December 2020 at 51 community hospitals across 6 states (Alaska, Washington, Montana, Oregon, California, and Texas) in the Providence St. Joseph Health system. Hospitals within the Providence system share a common standard dataset for each encounter with a centralized cloud data warehouse from which we extracted clinical and demographic data. No hospitals entered or left the system during the study period. Hospitalizations were considered unplanned if they had an “urgent” or “emergency” service type in the record; most originated in the ED. Hospitalizations for children younger than 18 years and those with evidence of COVID-19 (International Classification of Disease, Tenth Revision, Clinical Modification U07.1, a positive COVID-19 polymerase chain reaction test during the encounter, or an infection control-assigned label of COVID-19) were excluded. The Providence St. Joseph Health Institutional Review Board approved this study.

Measures

Trends in daily hospitalizations and their relationship to adjusted in-hospital mortality (percentage of patients who died during their hospital admission) were examined over time. In preliminary models using segmented regression, we identified three distinct pandemic periods with different trends in daily hospitalizations: (1) a 10-week period corresponding to the spring COVID-19 surge (March 4 to May 13, 2020; Period 1), (2) an intervening period extending over the summer and early fall (May 14 to October 19, 2020; Period 2), and (3) a second 10-week period corresponding to the fall COVID-19 surge (October 20 to December 31, 2020; Period 3). In-hospital mortality for these periods was compared with a baseline period (pre-COVID-19) from January 1, 2019 to March 3, 2020. To further assess differences in mortality by clinical condition, hospitalizations were first grouped by primary diagnosis using Clinical Classifications Software Refined (CCSR) categories from the Agency for Healthcare Research and Quality¹⁴ and ranked by the number of observed deaths and the percentage of patients who died while hospitalized in 2020. We selected common conditions that had >35 total deaths and an in-hospital mortality rate ≥1% for condition-specific analyses, of which 30 met these criteria.

Analysis

Multivariate logistic regression was used to evaluate changes in mortality for each of the pandemic periods compared with baseline for the overall cohort and selected diagnosis groups. Our main model adjusted for age, sex, race/ethnicity (White, Black, Latinx, Asian or Pacific Islander, and other), primary payor (commercial, Medicaid, Medicare, other, and self-pay), the presence or absence of 31 chronic comorbidities in the medical record, primary admitting diagnosis grouped by CCSR category (456 total diagnostic groups), and hospital fixed-effects to account for clustering. Results are expressed as the average marginal effects of each pandemic period on in-hospital mortality (eg, adjusted percentage point change in mortality over baseline). The number of excess deaths in each period was calculated by multiplying the estimated percentage point change in mortality for each period by the total number of hospitalizations. These excess deaths were subtracted from the number of observed deaths to derive the number of deaths that would be expected if pre-pandemic mortality rates persisted.

To further assess whether changes in adjusted mortality could be attributed to a smaller, sicker population of patients presenting to the hospital during the pandemic (meaning that less acutely ill patients stayed home), we conducted two sensitivity analyses. First, we tested whether substituting indicators for Medicare Severity Diagnosis Groups (MS-DRG) in lieu of CCSR categories had any impact on our results. MS-DRGs are designed to account for a patient’s illness severity and expected costs, whereas CCSR categories do not.¹⁵ MS-DRGs also better distinguish between surgical versus medical conditions. We re-ran our main model using indicators for CCSR to control for diagnostic mix, but further adjusted for severity using the DRG weight for the primary diagnosis and Modified Early Warning Score (MEWS) as continuous covariates. MEWS is a physiologic scoring system that incorporates abnormal vital signs and data related to mental status during the first 24 hours of a patient’s hospitalization into a risk-based score that has been shown to predict hospital mortality and need for intensive care.^16,17 These sensitivity analyses were performed on a subset of inpatient admissions because DRG data are not available for hospitalizations billed as an observation stay, and only approximately 70% of hospitals in the sample contributed vital sign data to the Providence data warehouse. All statistical analyses were conducted with R, version 3.6.3 (R Foundation for Statistical Computing) and SAS Enterprise Guide 7.1 (SAS Institute Inc).

The characteristics of our sample are described in Table 1. A total of 61,300, 159,430, and 65,923 hospitalizations occurred in each of the three pandemic periods, respectively, compared with 503,190 hospitalizations in the pre-pandemic period. The mean (SD) age of patients in the study was 63.2 (19.4) years; most were women (52.4%), White (70.6%), and had Medicare as their primary payor (53.7%). Less than half (42.7%) of hospitalizations occurred in California, and just under one-quarter were observation stays (23.2%). Patient characteristics were similar in the pre-COVID-19 and COVID-19 pandemic periods.

Figure 1 shows trends in hospital volume and mortality. Overall daily hospitalizations declined abruptly from a mean of 1176 per day in the pre-pandemic period to 617 per day (47.5% relative decrease) during the first 3 weeks of Period 1. Mean daily hospitalizations began to rise over the next 2 months (Period 1), reaching steady state at <1000 hospitalizations per day (15% relative decrease from baseline) during Period 2. During Period 3, we observed a decline in mean daily hospitalizations, with a low point of 882 per day on December 31, 2020 (25% relative decrease from baseline), corresponding to the end of our study period. Although hospital volumes declined during both COVID-19 surge periods, the percentage of patients who died during their hospitalization increased. There was an initial spike in in-hospital mortality that peaked approximately 1 month into the pandemic (middle of Period 1), a return to levels at or slightly below that before the pandemic by the beginning of Period 2, and then a rise throughout the autumn COVID-19 surge in Period 3, not yet peaking by the end of the study.

Adjusted in-hospital mortality for the three COVID-19 periods compared with the pre-pandemic period is presented in Table 2. The percentage of patients who died during their hospitalization rose from 2.9% in the pre-pandemic period to 3.4% during Period 1 (absolute difference, 0.6 percentage points; 95% CI, 0.5-0.7), corresponding to a 19.3% relative increase during the spring COVID-19 surge. Among the subset of patients hospitalized with 1 of the 30 conditions selected for individual analysis, mortality increased from 5.0% to 5.9% during the same time period (absolute difference, 0.9 percentage points; 95% CI, 0.8-1.1), corresponding to an 18.9% relative increase. In Period 2, in-hospital mortality was similar to that noted pre-pandemic for the overall cohort and the 30 selected conditions. During Period 3, in-hospital mortality increased by a magnitude similar to that observed in Period 1 for all hospitalizations combined (absolute difference, 0.5 percentage points; 95% CI, 0.0-0.6; corresponding to a 16.5% relative increase) as well as the subgroup with 1 of the 30 selected conditions (0.9 percentage points; 95% CI, 0.8-1.0; corresponding to an 18% relative increase). Further adjustment for severity by swapping CCSR categories with MS-DRG indicators or inclusion of DRG weight and MEWS score as covariates in our sensitivity analyses did not change our results.

Table 3 and the Appendix Figure describe changes in volume and adjusted in-hospital mortality for the 30 conditions selected for analysis. There was a decrease in the mean daily admissions for all conditions studied. Among the 30 conditions, 26 showed increased mortality during Period 1, although the increase was only statistically significant for 16 of these conditions. Among the 10 most commonly admitted conditions (by number of daily hospital admissions during the baseline period), there was a statistically significant relative increase in mortality for patients with sepsis (20.1%), heart failure (17.6%), ischemic stroke (12.5%), device/graft/surgical complications (14.0%), cardiac dysrhythmias (14.4%), pneumonia (24.5%), respiratory failure (16.1%), and gastrointestinal hemorrhage (23.3%). In general, mortality returned to baseline or improved during Period 2. Thereafter, all 30 conditions showed increased mortality in Period 3. This increase was significant for only 16 conditions, which were not the same ones noted during Period 1. Of note, although there was higher mortality for some cardiovascular conditions (heart failure cardiac dysrhythmias), mortality for myocardial infarction remained unchanged from baseline across all 3 periods. In contrast, several solid cancer–related conditions showed progressively worsening mortality throughout the study, with 7.7% higher mortality in Period 1, 10.3% higher mortality in Period 2, and 16.5% higher mortality in Period 3, respectively, compared with baseline. Although a similar pattern was observed for acute renal failure and some neurologic conditions (traumatic brain injury, seizure, other nervous system disorders), mortality for drug poisonings and gastrointestinal bleeds improved over time.

In this study of unplanned hospitalizations from 51 community hospitals across 6 states in the US West, we found a significant increase in mortality—at a rate of approximately 5 to 6 excess deaths per 1000 hospitalizations—among patients admitted during the pandemic with a variety of non-COVID-19 illnesses and injuries. Higher in-hospital mortality was observed in the spring (March to May) and fall (October to December) of 2020 when COVID-19 case counts surged and shelter-in-place mandates were implemented. With the initial surge, higher mortality rates were largely transient, and, for most conditions evaluated, returned to baseline approximately 3 months after the pandemic onset. For the fall surge, mortality rates had not peaked by the end of the study period. Changes in mortality were closely and inversely correlated with hospital volume for non-COVID-19 illnesses during both surge periods.

Higher morbidity and mortality for patients without COVID-19 appears to be an unfortunate spillover effect that has been reported in several studies. Recent work examining national surveillance data suggest that up to one-third of excess deaths (deaths higher than those expected for season) early in the pandemic have occurred among patients without known COVID-19.^13,18-20 Specifically, these studies estimate that mortality rates in the United States increased by 15% to 19% in the spring of 2020; of the identified excess deaths, only 38% to 77% could be attributed to COVID-19, with the remainder attributed to cardiovascular disease, diabetes, and Alzheimer’s disease, among others. In addition, reports from several European countries and China examining population death data have found similar trends,^21-25 as well as a recent study examining excess deaths in nursing homes.²⁶ Our results are largely consistent with these earlier studies in that we describe higher mortality in a sample of patients hospitalized with a variety of common conditions that otherwise are routinely treated in US hospitals. Reporting these indirect casualties of COVID-19 is important to fully understand the pandemic’s toll on patients and healthcare systems.

Our work builds on the current body of literature, highlighting the consistent relationship between rising COVID-19 case counts, hospital volume, and excess mortality over more than one surge period. Although several studies have looked at trends in hospital admissions or population mortality rates, few have examined the two outcomes together. The close correlation between daily hospital admissions and in-hospital mortality in this study suggests that the pandemic changed how patients use healthcare resources in ways that were important for their health and outcomes. The higher mortality rate that we and others have observed likely is related to patients’ delaying care because of fear of contracting COVID-19. In one survey, more than 4 in 10 adults in the United States reported that they avoided medical care during the early pandemic.¹⁰ Importantly, even a few days delay for many conditions, such as heart failure or sepsis, can result in precipitous declines in clinical status and outcomes.

It also is possible that we found increased rates of in-hospital mortality simply because patients with more moderate illness chose to stay home, resulting in a patient population enriched with those more likely to die. We found mixed evidence in our data that the observed increases in mortality could be attributable to a smaller, sicker population. Some characteristics that might be protective, such as a slightly younger mean age and lower mean DRG weight, were more common among those hospitalized during the pandemic. However, other characteristics, such as a slightly higher MEWS score and a greater percentage of total hospitalizations in the higher mortality subgroup, also were noted during the pandemic (Table 1). We do note, however, that the differences in these severity-related characteristics were small across the study periods. Further adjusting for these characteristics in our sensitivity analyses did not appreciably change our main findings, suggesting that the mortality increase could not be explained by changes in case-mix alone.

Other factors not dependent on patient behavior, such as barriers to accessing timely ambulatory care and impacts in the quality of care delivered, might have contributed. Shelter-in-place orders, reduced in-person access to clinicians in the ambulatory setting, slow implementation of telehealth services (with uncertainty about their equivalence to in-person exams), as well as delays in diagnostic tests and outpatient procedures could have played a role, especially during early months of the pandemic.²⁷ Significant changes to ambulatory health care delivery might have left many patients with chronic illnesses or complex medical needs with limited care options. Importantly, these care interruptions might have had greater implications for some patients, such as those with cancer who rely on intensive, largely outpatient-based treatment.^28,29 This, in part, could explain why we found persistently increased mortality among patients hospitalized with cancer after the spring surge. Later into the pandemic, however, most health systems had developed processes that allowed clinicians to resume timely care of ambulatory patients. Because of this, increases in mortality observed during the fall surge likely stem from other factors, such as patient behavior.

It is possible that care delays or changes in the quality of care delivered during the index hospitalization or pre-hospital setting might have contributed to the observed increase in mortality. This is particularly true for acute, time-sensitive conditions such as sepsis and stroke. Extra time spent donning personal protective equipment and/or new protocols instituted during the pandemic likely impacted the speed of emergency medical services transport, timeliness of ED evaluation, and delivery of definitive therapy. Although most hospitals in this study were not overwhelmed by the pandemic, the complexities associated with caring for known and suspected COVID-19 patients alongside those without the disease might have altered ideal care practices and strained healthcare teams.³⁰ In addition, nearly all hospitalized patients during this period were deprived of in-person advocacy by family members, who were not permitted to visit.

Important limitations with this study exist. First, the data come only from hospitals in the western United States. Second, some data elements such as triage scores or vital signs were not available for the entire population, potentially limiting some risk-adjustment. Third, we were unable to determine the root cause of excess mortality based on our study design and the coded variables available. It is unknown to what extent undiagnosed COVID-19 played a role. Early in the pandemic, many community hospitals did not have access to timely COVID-19 testing, and some cases might have not been diagnosed.³¹ However, we do not expect this to be a significant concern in the later months of the pandemic, as testing became more widespread and hospitals implemented surveillance screening for COVID-19 for inpatients.

Our study indicates that the COVID-19 pandemic was associated with increased mortality among patients hospitalized for a range of clinical conditions. Although higher observed mortality rates were limited to periods of high COVID-19 activity, future studies will need to tease out the extent to which these findings relate to patient factors (ie, delayed presentation and more severe disease) or systemic factors (reduction in access or changes in quality of care). This is of key importance, and appropriate solutions will need to be developed to mitigate adverse impacts with this and future pandemics.

Payors have been refining reimbursement policies for observation and inpatient stays over the past decade, and the effects on the healthcare payment system are significant.^1-4 Advocates claim that observation status could improve efficiency in the use of healthcare resources by reducing emergency department (ED) crowding and lowering costs for inpatient care.^5,6 Critics consider observation status to be a cost-shifting strategy that could lead to financial burdens for patients and hospitals.^7,8

Although reimbursement policies for observation stays traditionally have been set by the Centers for Medicare and Medicaid Services (CMS) in a uniform manner,^4,8 state Medicaid programs and commercial health insurers have developed a variety of policies for using observation status in broader populations and hospitals.^9-15 Coverage criteria and implementation timelines of these policies vary by states and commercial insurers.^11-15 For example, the California Department of Health Care Services did not have a specific reimbursement rate for observation stays in 2020, while some state Medicaid programs have had reimbursement policies for observation services in place since 2010.^11-15 These inconsistencies likely result in greater variation in use of observation stays across children’s hospitals than general hospitals.

Previous studies have shown rising trends in use of observation stays among adult patient populations and related implications for patients and general hospitals,^16-19 but few studies have reported the trends for pediatric populations. In this study, we sought to (1) describe recent trends of observation stays for pediatric populations at children’s hospitals from 2010 through 2019 and (2) investigate features of this shifting pattern for pediatric populations and hospital-level use of observation stays.

Study Design, Data, and Populations

We performed a retrospective analysis of the Pediatric Health Information System (PHIS), an administrative database that contains inpatient, observation, ambulatory, and ED encounter-level data from 50 not-for-profit, tertiary care children’s hospitals affiliated with the Children’s Hospital Association (CHA).²⁰ PHIS has an indicator to classify patient types (inpatient, observation, ED visits, ambulatory surgery, clinic visit, and others). The data are de-identified at the time of submission and subjected to validity and reliability checks by CHA and Truven Health Analytics (Ann Arbor, MI) before being included in PHIS. Each encounter in PHIS has only one patient type; therefore, encounters that transition to a higher level of care are assigned to their highest level of care (eg, a patient transitions from observation to inpatient status is classified as an inpatient encounter) to avoid duplicate counting.

To ensure consistent evaluations over time, we included 29 children’s hospitals that consistently reported both inpatient and observation data to PHIS across all quarters from 2010 through 2019. We identified the 20 most common clinical conditions using the All Patients Refined Diagnosis Related Groups (APR-DRGs; 3M Corporation) based upon their total frequencies of observation and inpatient stays over the study period. Regression analyses were conducted using all encounters within the 20 most common APR-DRGs.

Because all data have been de-identified in the PHIS database, the institutional review board at Ann and Robert H. Lurie Children’s Hospital of Chicago granted this study institutional review board–exempt status.

Main Outcome and Measures

We first presented longitudinal trends of observation stays for children’s hospitals using annual percentage of observation stays defined as:

To determine whether different pediatric populations have different trends of observation stays, we measured the growth rates of observation stays for each APR-DRG. Specifically, we first calculated the percentage of observation stays by APR-DRGs and years as described, and then calculated the growth rate of observation stays for each APR-DRG:

Next, we employed prolonged length of stay (LOS) and hospitalization resource-intensity scores for kids (H-RISK) to further investigate the shifting pattern of observation stays. Because most state Medicaid and commercial policies dictate that observation stays should not last longer than 48 hours, we defined prolonged LOS as >2 days.^11-15 We defined the annual percentage of observation stays with prolonged LOS for each year as:

Numerators and denominators of the three measures were obtained by pooling all children’s hospitals included in this study. H-RISK is a continuous variable developed by CHA to measure use of intensive care for children, which is comparable across various APR-DRGs.²¹ Changes in the empirical distribution of H-RISK from observation stays were presented over years using percentiles.

Other measures included sex, age, race, payor, and LOS. To investigate the use of observation stays among payors, we categorized payors into five groups: private, in-state Medicaid (managed care), in-state Medicaid (Children’s Health Insurance Program [CHIP]/others), other government, and all others, according to the data availability. The “private” group consisted of commercial preferred provider organizations, commercial health maintenance organizations, and commercial others. We combined both CHIP and in-state Medicaid (others), including Medicaid fee-for-service or unspecified Medicaid together as “in-state Medicaid (CHIP/others).” Detailed categorization information is summarized in Appendix Table 1. LOS was classified into four groups: 1 day (24 hours), 2 days (48 hours), 3 to 4 days, and >4 days.

Statistical Analysis

Descriptive statistics were stratified by inpatient and observation status and were summarized using frequency, percent, median, and interquartile range (IQR). Chi-square or Wilcoxon rank-sum tests were performed to examine differences between observation and inpatient status. Trends in annual percentage of observation stays and annual percentage of observation stays with prolonged LOS were estimated using first-order autoregressive models, in which year was considered a continuous variable. A nonparametric measure of rank correlation (Spearman’s rank correlation coefficient) was employed to evaluate the correlation between year and H-RISK from observation stays.

The risk-adjusted probability of being admitted as an observation stay was estimated using generalized linear mixed models by adjusting for year, age, sex, race, payor, LOS, H-RISK, and a random intercept for each hospital to control for patient clustering within a hospital (Appendix Model). Hospital-level use of observation stays was measured by risk-adjusted percent use of observation stays for each hospital using the predicted values from generalized linear mixed models. All analyses were performed using SAS software, version 9.4 (SAS Institute) and R (R Core Team, 2019), and P < .05 was considered statistically significant.

Increasing Trend of Observation Stays

Over the study period, there were 5,611,001 encounters, including 3,901,873 (69.5%) inpatient and 1,709,128 (30.5%) observation stays (Appendix Table 1). The number of observation stays increased from 117,246 in 2010 to 207,842 in 2019, and the number of inpatient stays slightly increased from 378,433 to 397,994 over the 10 years (Appendix Table 1). Because of different growth rates between observation and inpatient status, the annual percentage of observation stays increased from 23.7% in 2010 to 34.3% in 2019, while the annual percentage of inpatient stays decreased from 76.3% in 2010 to 65.7% in 2019 (Appendix Table 1; Figure 1, P < .001).

Different Growth Rates of Observation Stays for Various Pediatric Populations

As shown in the Table, growth rates of observation stays increased for 19 of the 20 most common APR-DRGs. The four APR-DRGs having the highest growth rates in observation stays were appendectomy, diabetes mellitus, kidney and urinary tract infections, and cellulitis and other bacterial skin infections (Appendix Figure). In particular, the annual percentage of observation stays for appendectomy increased from 19.8% in 2010 to 54.7% in 2019, with the number of observation stays growing from 2,321 to 7,876, while the number of inpatient stays decreased from 9,384 to 6,535 (Appendix Figure). The annual percentage of observation stays for diabetes mellitus increased from 8.16% in 2010 to 22.74% in 2019. Tonsil and adenoid procedures consistently held the largest numbers of observation stays across the 10 years among all the APR-DRGs, with 115,207 and 31,125 total observation and inpatient stays, respectively (Table).

Characteristics of Observation and Inpatient Stays

Patient characteristics are summarized in Appendix Table 1. There were 542,344 (32.9%) observation stays among patients with in-state Medicaid (managed care), and 241,157 (27.4%) observation stays among in-state Medicaid (CHIP/others). The percentages of observation and inpatient stays were 29.8% and 70.2% for private payor, as well as 29.6% and 70.4% for other government payor. Overall, the median (IQR) of H-RISK among observation stays was 0.79 (0.57-1.19) vs 1.23 (0.72-2.43) for inpatient stays. There were 1,410,694 (82.5%) observation stays discharged within 1 day and 243,972 (14.3%) observation stays discharged within 2 days. However, there were 47,413 (2.8%) and 7,049 (0.4%) observation stays with LOS 3 to 4 days or >4 days, respectively.

Shifting Pattern in Observation Stays

The annual percentage of observation stays with prolonged LOS (>2 days) rose from 1.1% in 2010 to 4.6% in 2019 (P < .001; Figure 2). The empirical distribution of H-RISK from observation stays by years further suggests a slightly increasing trend in intensity of care under observation stays. As shown in Appendix Table 2, although the 1st, 5th, 10th, 25th, and 99th percentiles of H-RISK were relatively stable, the 50th, 75th, 90th, and 95th percentiles of H-RISK were increasing over time. The correlation between year and intensity of care used under observation stays (H-RISK from observation stays) was found to be weak but significantly positive (Spearman correlation coefficients = 0.04; P < .001).

Interaction coefficients from our regression model demonstrate that the existing inverse association between H-RISK and odds of admission as an observation stay became less negative over the years. In 2010, the adjusted odds ratio (OR) of H-RISK was 0.57 (95% CI, 0.55-0.59). By 2017, the adjusted OR had increased to 0.65 (95% CI, 0.64-0.66). Compared with 2010, the seven adjusted ORs of H-RISK at years 2012 through 2018 were observed to be higher and statistically significant (P < .001, Appendix Table 3).

Hospitals-Level Use of Observation Stays

After adjusting for all covariates and hospital random effects, hospital-level use of observation stays increased between 2010 and 2019 for 26 out of 29 children’s hospitals. Although observation status essentially was not used at two children’s hospitals over the study period, the median hospital-level use of observation stays was 26% in 2010 (IQR, 3%-36%) and increased to 46% (IQR: 39%; 55%) in 2019. As shown in Figure 3, the number of hospitals with a low percentage of observation stays (<26%) decreased from 15 in 2010 to 4 in 2019. The number of hospitals with a high percentage of observation stays (≥51%) increased from 5 in 2010 to 10 in 2019. Nevertheless, there remained significant variation in the use of observation stays, and the hospital-level use ranged from 0% to 67% in 2019.

By 2020, observation status has become a key component of healthcare for pediatric patients, and its relevance for children’s hospitals recently has been described.^22,23 However, trends in observation stays for pediatric populations are not known. This represents the first study showing temporal trends of observation stays at children’s hospitals after 2010. Our results confirm that the increase in observation stays for pediatric populations is not attributable to decreasing patient acuity at children’s hospitals. We found a weak but significantly positive correlation between year and intensity of care used under observation stays. Although this correlation might not be clinically important, it demonstrates that patient acuity in observation stays is not decreasing. Regression results suggest that observation stays now encompass patients who need relatively higher intensity of care compared with those admitted under observation status in 2010.

This study also identifies a unique pattern in the use of observation stays among pediatric populations. Earlier studies exclusively focused on observation stays that were admitted from EDs.²⁴ Our results indicate that observation status has been used beyond a bridge from ED care to inpatient admission. In particular, observation status has expanded to include pediatric populations with more diverse clinical conditions (eg, appendicitis and diabetes mellitus), and has become a substantial component of postprocedural admissions (Appendix Figure). Looking forward, it is likely that the use of observation stays might surpass inpatient admissions for more conditions that primarily involve short-term stays.

Observation status originally was designed as a reimbursement strategy for patients who needed short stays in dedicated ED units or hospitals, but did not qualify for inpatient services.^5,25 After several changes in reimbursement policies, CMS released the “two midnight rule” for Medicare beneficiaries in 2013, which replaced condition-based criteria with time-based criteria to determine an inpatient or observation stay.¹ Some Medicaid programs and commercial payors have developed similar policies. Unlike the universal policy for Medicare populations, the regulations for pediatric populations vary by states and health insurers.^11-15,26-28 This might partially explain the wide variation observed among children’s hospital-level use of observation stays. For example, the California Medicaid program did not have a reimbursement rate for observation services as of 2020, while the Texas Medicaid program has had a policy for observation stays since 2010.^12,13 We found that two children’s hospitals in California had the lowest use of observation stays (almost zero), whereas the hospital-level use of observation stays was more than 50% for three out of four children’s hospitals in Texas. In addition to reimbursement policies, individual hospitals also might have different strategies for observation status designation. An earlier survey showed that there was lack of consistency in billing and payor-based designations of observation status at children’s hospitals.²⁹ These findings suggest that children’s hospital-level use of observation stays likely is influenced by reimbursement policy and practical strategy for observation status determination.

Earlier studies reported that observation status could be a more efficient use of healthcare resources.^5,6 However, there are still at least two concerns relevant to children’s hospitals during the last decade. The first is whether the use of observation stays can promote cost-saving or if it is just a cost-shifting strategy. An earlier study demonstrated that observation stays with prolonged LOS might increase risk of cost-sharing among adult patients.²⁹ Our study reveals an increasing trend of observation stays with prolonged LOS for pediatric patients. Similar to adult patients, LOS exceeding 24 or 48 hours could lead to uncovered healthcare costs and financial burdens on families.^30-32 Meanwhile, children’s hospitals also might take on a higher financial liability by implementing observation status. Earlier studies have indicated that resource use between observation and inpatient stays at children’s hospitals is similar, and increasing use of observation stays might lead to financial risk rather than cost effectiveness.³³ Further, administrative costs of observation determination are considerably high.³⁴ Medicaid is the major payor for pediatric patients in children’s hospitals. In this study, more than 50% of encounters were paid through Medicaid programs. It is well known that Medicaid reimbursement rates are lower than Medicare and commercial plans.³⁵ Therefore, the cost-saving conclusion drawn from Medicare patients cannot be generalized to pediatric populations at children’s hospitals without cautious reevaluation.

A second concern with increasing use of observation stays is selection bias in public reporting and comparisons of hospital performance. Presently, four main categories of quality indicators established by the Agency for Healthcare Research and Quality rely heavily on inpatient encounters.³⁶ In this study, we found that the range of hospital-level use of observation stays was large. In 2019, the risk-adjusted percent use of observation stays was less than 5% at three hospitals, while the percent use was greater than 60% in another three hospitals. Therefore, comparisons made without uniform accounting of observation stays might have significant implications for national rankings of children’s hospitals across the United States. These consequences have been investigated in several published studies.^22,23,37-39

There are several limitations to our study. First, the study sample was limited to children’s hospitals that consistently reported inpatient and observation data over the entire study period. Eighteen hospitals (86%) excluded from this study did not consistently submit inpatient and observation data to PHIS from 2010 through 2019. The primary purpose of this study was to present temporal trends of observation stays for children’s hospitals, and it was important to build the hospital cohort based on valid and consistent data during the study period. Appendix Table 4 presents differences of hospital characteristics by included and excluded groups of hospitals. Excluded hospitals might have fewer resources (eg, fewer pediatric intensive care beds). Nonetheless, the selection of hospitals was optimized based on data availability. Second, this study was a retrospective review of an administrative database of children’s hospitals and units. The sample does not represent all children’s hospitals or pediatric patients in the United States, but there are no available data sources—that we know of—that can generate national estimates for both inpatient and observation stays. Third, we did not attempt to conclusively infer any causal effects, and several factors could explain the increasing trends, such as reimbursement policies, hospital-level implementation strategies, determination guidelines for observation status designation, as well as changes in clinical care. Further studies should investigate impact of these factors on the use of observation stays for pediatric patients and children’s hospitals.

Observation status has been increasingly used for pediatric patients with more diverse clinical conditions, and there is a rising trend of prolonged LOS among observation stays since 2010. Considerable variation exists in hospital-level use of observation stays across children’s hospitals. Observation status could be an opportunity to improve efficiency of healthcare resource use or could lead to a financial risk for patients with prolonged LOS. Future studies should explore appropriateness of observation care in clinical practice through leveraging efficient care and alleviating financial risk.

Payors have been refining reimbursement policies for observation and inpatient stays over the past decade, and the effects on the healthcare payment system are significant.^1-4 Advocates claim that observation status could improve efficiency in the use of healthcare resources by reducing emergency department (ED) crowding and lowering costs for inpatient care.^5,6 Critics consider observation status to be a cost-shifting strategy that could lead to financial burdens for patients and hospitals.^7,8

Although reimbursement policies for observation stays traditionally have been set by the Centers for Medicare and Medicaid Services (CMS) in a uniform manner,^4,8 state Medicaid programs and commercial health insurers have developed a variety of policies for using observation status in broader populations and hospitals.^9-15 Coverage criteria and implementation timelines of these policies vary by states and commercial insurers.^11-15 For example, the California Department of Health Care Services did not have a specific reimbursement rate for observation stays in 2020, while some state Medicaid programs have had reimbursement policies for observation services in place since 2010.^11-15 These inconsistencies likely result in greater variation in use of observation stays across children’s hospitals than general hospitals.

Previous studies have shown rising trends in use of observation stays among adult patient populations and related implications for patients and general hospitals,^16-19 but few studies have reported the trends for pediatric populations. In this study, we sought to (1) describe recent trends of observation stays for pediatric populations at children’s hospitals from 2010 through 2019 and (2) investigate features of this shifting pattern for pediatric populations and hospital-level use of observation stays.

Study Design, Data, and Populations

We performed a retrospective analysis of the Pediatric Health Information System (PHIS), an administrative database that contains inpatient, observation, ambulatory, and ED encounter-level data from 50 not-for-profit, tertiary care children’s hospitals affiliated with the Children’s Hospital Association (CHA).²⁰ PHIS has an indicator to classify patient types (inpatient, observation, ED visits, ambulatory surgery, clinic visit, and others). The data are de-identified at the time of submission and subjected to validity and reliability checks by CHA and Truven Health Analytics (Ann Arbor, MI) before being included in PHIS. Each encounter in PHIS has only one patient type; therefore, encounters that transition to a higher level of care are assigned to their highest level of care (eg, a patient transitions from observation to inpatient status is classified as an inpatient encounter) to avoid duplicate counting.

To ensure consistent evaluations over time, we included 29 children’s hospitals that consistently reported both inpatient and observation data to PHIS across all quarters from 2010 through 2019. We identified the 20 most common clinical conditions using the All Patients Refined Diagnosis Related Groups (APR-DRGs; 3M Corporation) based upon their total frequencies of observation and inpatient stays over the study period. Regression analyses were conducted using all encounters within the 20 most common APR-DRGs.

Because all data have been de-identified in the PHIS database, the institutional review board at Ann and Robert H. Lurie Children’s Hospital of Chicago granted this study institutional review board–exempt status.

Main Outcome and Measures

We first presented longitudinal trends of observation stays for children’s hospitals using annual percentage of observation stays defined as:

To determine whether different pediatric populations have different trends of observation stays, we measured the growth rates of observation stays for each APR-DRG. Specifically, we first calculated the percentage of observation stays by APR-DRGs and years as described, and then calculated the growth rate of observation stays for each APR-DRG:

Next, we employed prolonged length of stay (LOS) and hospitalization resource-intensity scores for kids (H-RISK) to further investigate the shifting pattern of observation stays. Because most state Medicaid and commercial policies dictate that observation stays should not last longer than 48 hours, we defined prolonged LOS as >2 days.^11-15 We defined the annual percentage of observation stays with prolonged LOS for each year as:

Numerators and denominators of the three measures were obtained by pooling all children’s hospitals included in this study. H-RISK is a continuous variable developed by CHA to measure use of intensive care for children, which is comparable across various APR-DRGs.²¹ Changes in the empirical distribution of H-RISK from observation stays were presented over years using percentiles.

Other measures included sex, age, race, payor, and LOS. To investigate the use of observation stays among payors, we categorized payors into five groups: private, in-state Medicaid (managed care), in-state Medicaid (Children’s Health Insurance Program [CHIP]/others), other government, and all others, according to the data availability. The “private” group consisted of commercial preferred provider organizations, commercial health maintenance organizations, and commercial others. We combined both CHIP and in-state Medicaid (others), including Medicaid fee-for-service or unspecified Medicaid together as “in-state Medicaid (CHIP/others).” Detailed categorization information is summarized in Appendix Table 1. LOS was classified into four groups: 1 day (24 hours), 2 days (48 hours), 3 to 4 days, and >4 days.

Statistical Analysis

Descriptive statistics were stratified by inpatient and observation status and were summarized using frequency, percent, median, and interquartile range (IQR). Chi-square or Wilcoxon rank-sum tests were performed to examine differences between observation and inpatient status. Trends in annual percentage of observation stays and annual percentage of observation stays with prolonged LOS were estimated using first-order autoregressive models, in which year was considered a continuous variable. A nonparametric measure of rank correlation (Spearman’s rank correlation coefficient) was employed to evaluate the correlation between year and H-RISK from observation stays.

The risk-adjusted probability of being admitted as an observation stay was estimated using generalized linear mixed models by adjusting for year, age, sex, race, payor, LOS, H-RISK, and a random intercept for each hospital to control for patient clustering within a hospital (Appendix Model). Hospital-level use of observation stays was measured by risk-adjusted percent use of observation stays for each hospital using the predicted values from generalized linear mixed models. All analyses were performed using SAS software, version 9.4 (SAS Institute) and R (R Core Team, 2019), and P < .05 was considered statistically significant.

Increasing Trend of Observation Stays

Over the study period, there were 5,611,001 encounters, including 3,901,873 (69.5%) inpatient and 1,709,128 (30.5%) observation stays (Appendix Table 1). The number of observation stays increased from 117,246 in 2010 to 207,842 in 2019, and the number of inpatient stays slightly increased from 378,433 to 397,994 over the 10 years (Appendix Table 1). Because of different growth rates between observation and inpatient status, the annual percentage of observation stays increased from 23.7% in 2010 to 34.3% in 2019, while the annual percentage of inpatient stays decreased from 76.3% in 2010 to 65.7% in 2019 (Appendix Table 1; Figure 1, P < .001).

Different Growth Rates of Observation Stays for Various Pediatric Populations

As shown in the Table, growth rates of observation stays increased for 19 of the 20 most common APR-DRGs. The four APR-DRGs having the highest growth rates in observation stays were appendectomy, diabetes mellitus, kidney and urinary tract infections, and cellulitis and other bacterial skin infections (Appendix Figure). In particular, the annual percentage of observation stays for appendectomy increased from 19.8% in 2010 to 54.7% in 2019, with the number of observation stays growing from 2,321 to 7,876, while the number of inpatient stays decreased from 9,384 to 6,535 (Appendix Figure). The annual percentage of observation stays for diabetes mellitus increased from 8.16% in 2010 to 22.74% in 2019. Tonsil and adenoid procedures consistently held the largest numbers of observation stays across the 10 years among all the APR-DRGs, with 115,207 and 31,125 total observation and inpatient stays, respectively (Table).

Characteristics of Observation and Inpatient Stays

Patient characteristics are summarized in Appendix Table 1. There were 542,344 (32.9%) observation stays among patients with in-state Medicaid (managed care), and 241,157 (27.4%) observation stays among in-state Medicaid (CHIP/others). The percentages of observation and inpatient stays were 29.8% and 70.2% for private payor, as well as 29.6% and 70.4% for other government payor. Overall, the median (IQR) of H-RISK among observation stays was 0.79 (0.57-1.19) vs 1.23 (0.72-2.43) for inpatient stays. There were 1,410,694 (82.5%) observation stays discharged within 1 day and 243,972 (14.3%) observation stays discharged within 2 days. However, there were 47,413 (2.8%) and 7,049 (0.4%) observation stays with LOS 3 to 4 days or >4 days, respectively.

Shifting Pattern in Observation Stays

The annual percentage of observation stays with prolonged LOS (>2 days) rose from 1.1% in 2010 to 4.6% in 2019 (P < .001; Figure 2). The empirical distribution of H-RISK from observation stays by years further suggests a slightly increasing trend in intensity of care under observation stays. As shown in Appendix Table 2, although the 1st, 5th, 10th, 25th, and 99th percentiles of H-RISK were relatively stable, the 50th, 75th, 90th, and 95th percentiles of H-RISK were increasing over time. The correlation between year and intensity of care used under observation stays (H-RISK from observation stays) was found to be weak but significantly positive (Spearman correlation coefficients = 0.04; P < .001).

Interaction coefficients from our regression model demonstrate that the existing inverse association between H-RISK and odds of admission as an observation stay became less negative over the years. In 2010, the adjusted odds ratio (OR) of H-RISK was 0.57 (95% CI, 0.55-0.59). By 2017, the adjusted OR had increased to 0.65 (95% CI, 0.64-0.66). Compared with 2010, the seven adjusted ORs of H-RISK at years 2012 through 2018 were observed to be higher and statistically significant (P < .001, Appendix Table 3).

Hospitals-Level Use of Observation Stays

After adjusting for all covariates and hospital random effects, hospital-level use of observation stays increased between 2010 and 2019 for 26 out of 29 children’s hospitals. Although observation status essentially was not used at two children’s hospitals over the study period, the median hospital-level use of observation stays was 26% in 2010 (IQR, 3%-36%) and increased to 46% (IQR: 39%; 55%) in 2019. As shown in Figure 3, the number of hospitals with a low percentage of observation stays (<26%) decreased from 15 in 2010 to 4 in 2019. The number of hospitals with a high percentage of observation stays (≥51%) increased from 5 in 2010 to 10 in 2019. Nevertheless, there remained significant variation in the use of observation stays, and the hospital-level use ranged from 0% to 67% in 2019.

By 2020, observation status has become a key component of healthcare for pediatric patients, and its relevance for children’s hospitals recently has been described.^22,23 However, trends in observation stays for pediatric populations are not known. This represents the first study showing temporal trends of observation stays at children’s hospitals after 2010. Our results confirm that the increase in observation stays for pediatric populations is not attributable to decreasing patient acuity at children’s hospitals. We found a weak but significantly positive correlation between year and intensity of care used under observation stays. Although this correlation might not be clinically important, it demonstrates that patient acuity in observation stays is not decreasing. Regression results suggest that observation stays now encompass patients who need relatively higher intensity of care compared with those admitted under observation status in 2010.

This study also identifies a unique pattern in the use of observation stays among pediatric populations. Earlier studies exclusively focused on observation stays that were admitted from EDs.²⁴ Our results indicate that observation status has been used beyond a bridge from ED care to inpatient admission. In particular, observation status has expanded to include pediatric populations with more diverse clinical conditions (eg, appendicitis and diabetes mellitus), and has become a substantial component of postprocedural admissions (Appendix Figure). Looking forward, it is likely that the use of observation stays might surpass inpatient admissions for more conditions that primarily involve short-term stays.

Observation status originally was designed as a reimbursement strategy for patients who needed short stays in dedicated ED units or hospitals, but did not qualify for inpatient services.^5,25 After several changes in reimbursement policies, CMS released the “two midnight rule” for Medicare beneficiaries in 2013, which replaced condition-based criteria with time-based criteria to determine an inpatient or observation stay.¹ Some Medicaid programs and commercial payors have developed similar policies. Unlike the universal policy for Medicare populations, the regulations for pediatric populations vary by states and health insurers.^11-15,26-28 This might partially explain the wide variation observed among children’s hospital-level use of observation stays. For example, the California Medicaid program did not have a reimbursement rate for observation services as of 2020, while the Texas Medicaid program has had a policy for observation stays since 2010.^12,13 We found that two children’s hospitals in California had the lowest use of observation stays (almost zero), whereas the hospital-level use of observation stays was more than 50% for three out of four children’s hospitals in Texas. In addition to reimbursement policies, individual hospitals also might have different strategies for observation status designation. An earlier survey showed that there was lack of consistency in billing and payor-based designations of observation status at children’s hospitals.²⁹ These findings suggest that children’s hospital-level use of observation stays likely is influenced by reimbursement policy and practical strategy for observation status determination.

Earlier studies reported that observation status could be a more efficient use of healthcare resources.^5,6 However, there are still at least two concerns relevant to children’s hospitals during the last decade. The first is whether the use of observation stays can promote cost-saving or if it is just a cost-shifting strategy. An earlier study demonstrated that observation stays with prolonged LOS might increase risk of cost-sharing among adult patients.²⁹ Our study reveals an increasing trend of observation stays with prolonged LOS for pediatric patients. Similar to adult patients, LOS exceeding 24 or 48 hours could lead to uncovered healthcare costs and financial burdens on families.^30-32 Meanwhile, children’s hospitals also might take on a higher financial liability by implementing observation status. Earlier studies have indicated that resource use between observation and inpatient stays at children’s hospitals is similar, and increasing use of observation stays might lead to financial risk rather than cost effectiveness.³³ Further, administrative costs of observation determination are considerably high.³⁴ Medicaid is the major payor for pediatric patients in children’s hospitals. In this study, more than 50% of encounters were paid through Medicaid programs. It is well known that Medicaid reimbursement rates are lower than Medicare and commercial plans.³⁵ Therefore, the cost-saving conclusion drawn from Medicare patients cannot be generalized to pediatric populations at children’s hospitals without cautious reevaluation.

A second concern with increasing use of observation stays is selection bias in public reporting and comparisons of hospital performance. Presently, four main categories of quality indicators established by the Agency for Healthcare Research and Quality rely heavily on inpatient encounters.³⁶ In this study, we found that the range of hospital-level use of observation stays was large. In 2019, the risk-adjusted percent use of observation stays was less than 5% at three hospitals, while the percent use was greater than 60% in another three hospitals. Therefore, comparisons made without uniform accounting of observation stays might have significant implications for national rankings of children’s hospitals across the United States. These consequences have been investigated in several published studies.^22,23,37-39

There are several limitations to our study. First, the study sample was limited to children’s hospitals that consistently reported inpatient and observation data over the entire study period. Eighteen hospitals (86%) excluded from this study did not consistently submit inpatient and observation data to PHIS from 2010 through 2019. The primary purpose of this study was to present temporal trends of observation stays for children’s hospitals, and it was important to build the hospital cohort based on valid and consistent data during the study period. Appendix Table 4 presents differences of hospital characteristics by included and excluded groups of hospitals. Excluded hospitals might have fewer resources (eg, fewer pediatric intensive care beds). Nonetheless, the selection of hospitals was optimized based on data availability. Second, this study was a retrospective review of an administrative database of children’s hospitals and units. The sample does not represent all children’s hospitals or pediatric patients in the United States, but there are no available data sources—that we know of—that can generate national estimates for both inpatient and observation stays. Third, we did not attempt to conclusively infer any causal effects, and several factors could explain the increasing trends, such as reimbursement policies, hospital-level implementation strategies, determination guidelines for observation status designation, as well as changes in clinical care. Further studies should investigate impact of these factors on the use of observation stays for pediatric patients and children’s hospitals.

Observation status has been increasingly used for pediatric patients with more diverse clinical conditions, and there is a rising trend of prolonged LOS among observation stays since 2010. Considerable variation exists in hospital-level use of observation stays across children’s hospitals. Observation status could be an opportunity to improve efficiency of healthcare resource use or could lead to a financial risk for patients with prolonged LOS. Future studies should explore appropriateness of observation care in clinical practice through leveraging efficient care and alleviating financial risk.

Payors have been refining reimbursement policies for observation and inpatient stays over the past decade, and the effects on the healthcare payment system are significant.^1-4 Advocates claim that observation status could improve efficiency in the use of healthcare resources by reducing emergency department (ED) crowding and lowering costs for inpatient care.^5,6 Critics consider observation status to be a cost-shifting strategy that could lead to financial burdens for patients and hospitals.^7,8

Although reimbursement policies for observation stays traditionally have been set by the Centers for Medicare and Medicaid Services (CMS) in a uniform manner,^4,8 state Medicaid programs and commercial health insurers have developed a variety of policies for using observation status in broader populations and hospitals.^9-15 Coverage criteria and implementation timelines of these policies vary by states and commercial insurers.^11-15 For example, the California Department of Health Care Services did not have a specific reimbursement rate for observation stays in 2020, while some state Medicaid programs have had reimbursement policies for observation services in place since 2010.^11-15 These inconsistencies likely result in greater variation in use of observation stays across children’s hospitals than general hospitals.

Previous studies have shown rising trends in use of observation stays among adult patient populations and related implications for patients and general hospitals,^16-19 but few studies have reported the trends for pediatric populations. In this study, we sought to (1) describe recent trends of observation stays for pediatric populations at children’s hospitals from 2010 through 2019 and (2) investigate features of this shifting pattern for pediatric populations and hospital-level use of observation stays.

Study Design, Data, and Populations

We performed a retrospective analysis of the Pediatric Health Information System (PHIS), an administrative database that contains inpatient, observation, ambulatory, and ED encounter-level data from 50 not-for-profit, tertiary care children’s hospitals affiliated with the Children’s Hospital Association (CHA).²⁰ PHIS has an indicator to classify patient types (inpatient, observation, ED visits, ambulatory surgery, clinic visit, and others). The data are de-identified at the time of submission and subjected to validity and reliability checks by CHA and Truven Health Analytics (Ann Arbor, MI) before being included in PHIS. Each encounter in PHIS has only one patient type; therefore, encounters that transition to a higher level of care are assigned to their highest level of care (eg, a patient transitions from observation to inpatient status is classified as an inpatient encounter) to avoid duplicate counting.

To ensure consistent evaluations over time, we included 29 children’s hospitals that consistently reported both inpatient and observation data to PHIS across all quarters from 2010 through 2019. We identified the 20 most common clinical conditions using the All Patients Refined Diagnosis Related Groups (APR-DRGs; 3M Corporation) based upon their total frequencies of observation and inpatient stays over the study period. Regression analyses were conducted using all encounters within the 20 most common APR-DRGs.

Because all data have been de-identified in the PHIS database, the institutional review board at Ann and Robert H. Lurie Children’s Hospital of Chicago granted this study institutional review board–exempt status.

Main Outcome and Measures

We first presented longitudinal trends of observation stays for children’s hospitals using annual percentage of observation stays defined as:

To determine whether different pediatric populations have different trends of observation stays, we measured the growth rates of observation stays for each APR-DRG. Specifically, we first calculated the percentage of observation stays by APR-DRGs and years as described, and then calculated the growth rate of observation stays for each APR-DRG:

Next, we employed prolonged length of stay (LOS) and hospitalization resource-intensity scores for kids (H-RISK) to further investigate the shifting pattern of observation stays. Because most state Medicaid and commercial policies dictate that observation stays should not last longer than 48 hours, we defined prolonged LOS as >2 days.^11-15 We defined the annual percentage of observation stays with prolonged LOS for each year as:

Numerators and denominators of the three measures were obtained by pooling all children’s hospitals included in this study. H-RISK is a continuous variable developed by CHA to measure use of intensive care for children, which is comparable across various APR-DRGs.²¹ Changes in the empirical distribution of H-RISK from observation stays were presented over years using percentiles.

Other measures included sex, age, race, payor, and LOS. To investigate the use of observation stays among payors, we categorized payors into five groups: private, in-state Medicaid (managed care), in-state Medicaid (Children’s Health Insurance Program [CHIP]/others), other government, and all others, according to the data availability. The “private” group consisted of commercial preferred provider organizations, commercial health maintenance organizations, and commercial others. We combined both CHIP and in-state Medicaid (others), including Medicaid fee-for-service or unspecified Medicaid together as “in-state Medicaid (CHIP/others).” Detailed categorization information is summarized in Appendix Table 1. LOS was classified into four groups: 1 day (24 hours), 2 days (48 hours), 3 to 4 days, and >4 days.

Statistical Analysis

Descriptive statistics were stratified by inpatient and observation status and were summarized using frequency, percent, median, and interquartile range (IQR). Chi-square or Wilcoxon rank-sum tests were performed to examine differences between observation and inpatient status. Trends in annual percentage of observation stays and annual percentage of observation stays with prolonged LOS were estimated using first-order autoregressive models, in which year was considered a continuous variable. A nonparametric measure of rank correlation (Spearman’s rank correlation coefficient) was employed to evaluate the correlation between year and H-RISK from observation stays.

The risk-adjusted probability of being admitted as an observation stay was estimated using generalized linear mixed models by adjusting for year, age, sex, race, payor, LOS, H-RISK, and a random intercept for each hospital to control for patient clustering within a hospital (Appendix Model). Hospital-level use of observation stays was measured by risk-adjusted percent use of observation stays for each hospital using the predicted values from generalized linear mixed models. All analyses were performed using SAS software, version 9.4 (SAS Institute) and R (R Core Team, 2019), and P < .05 was considered statistically significant.

Increasing Trend of Observation Stays

Over the study period, there were 5,611,001 encounters, including 3,901,873 (69.5%) inpatient and 1,709,128 (30.5%) observation stays (Appendix Table 1). The number of observation stays increased from 117,246 in 2010 to 207,842 in 2019, and the number of inpatient stays slightly increased from 378,433 to 397,994 over the 10 years (Appendix Table 1). Because of different growth rates between observation and inpatient status, the annual percentage of observation stays increased from 23.7% in 2010 to 34.3% in 2019, while the annual percentage of inpatient stays decreased from 76.3% in 2010 to 65.7% in 2019 (Appendix Table 1; Figure 1, P < .001).

Different Growth Rates of Observation Stays for Various Pediatric Populations

As shown in the Table, growth rates of observation stays increased for 19 of the 20 most common APR-DRGs. The four APR-DRGs having the highest growth rates in observation stays were appendectomy, diabetes mellitus, kidney and urinary tract infections, and cellulitis and other bacterial skin infections (Appendix Figure). In particular, the annual percentage of observation stays for appendectomy increased from 19.8% in 2010 to 54.7% in 2019, with the number of observation stays growing from 2,321 to 7,876, while the number of inpatient stays decreased from 9,384 to 6,535 (Appendix Figure). The annual percentage of observation stays for diabetes mellitus increased from 8.16% in 2010 to 22.74% in 2019. Tonsil and adenoid procedures consistently held the largest numbers of observation stays across the 10 years among all the APR-DRGs, with 115,207 and 31,125 total observation and inpatient stays, respectively (Table).

Characteristics of Observation and Inpatient Stays

Patient characteristics are summarized in Appendix Table 1. There were 542,344 (32.9%) observation stays among patients with in-state Medicaid (managed care), and 241,157 (27.4%) observation stays among in-state Medicaid (CHIP/others). The percentages of observation and inpatient stays were 29.8% and 70.2% for private payor, as well as 29.6% and 70.4% for other government payor. Overall, the median (IQR) of H-RISK among observation stays was 0.79 (0.57-1.19) vs 1.23 (0.72-2.43) for inpatient stays. There were 1,410,694 (82.5%) observation stays discharged within 1 day and 243,972 (14.3%) observation stays discharged within 2 days. However, there were 47,413 (2.8%) and 7,049 (0.4%) observation stays with LOS 3 to 4 days or >4 days, respectively.

Shifting Pattern in Observation Stays

The annual percentage of observation stays with prolonged LOS (>2 days) rose from 1.1% in 2010 to 4.6% in 2019 (P < .001; Figure 2). The empirical distribution of H-RISK from observation stays by years further suggests a slightly increasing trend in intensity of care under observation stays. As shown in Appendix Table 2, although the 1st, 5th, 10th, 25th, and 99th percentiles of H-RISK were relatively stable, the 50th, 75th, 90th, and 95th percentiles of H-RISK were increasing over time. The correlation between year and intensity of care used under observation stays (H-RISK from observation stays) was found to be weak but significantly positive (Spearman correlation coefficients = 0.04; P < .001).

Interaction coefficients from our regression model demonstrate that the existing inverse association between H-RISK and odds of admission as an observation stay became less negative over the years. In 2010, the adjusted odds ratio (OR) of H-RISK was 0.57 (95% CI, 0.55-0.59). By 2017, the adjusted OR had increased to 0.65 (95% CI, 0.64-0.66). Compared with 2010, the seven adjusted ORs of H-RISK at years 2012 through 2018 were observed to be higher and statistically significant (P < .001, Appendix Table 3).

Hospitals-Level Use of Observation Stays

After adjusting for all covariates and hospital random effects, hospital-level use of observation stays increased between 2010 and 2019 for 26 out of 29 children’s hospitals. Although observation status essentially was not used at two children’s hospitals over the study period, the median hospital-level use of observation stays was 26% in 2010 (IQR, 3%-36%) and increased to 46% (IQR: 39%; 55%) in 2019. As shown in Figure 3, the number of hospitals with a low percentage of observation stays (<26%) decreased from 15 in 2010 to 4 in 2019. The number of hospitals with a high percentage of observation stays (≥51%) increased from 5 in 2010 to 10 in 2019. Nevertheless, there remained significant variation in the use of observation stays, and the hospital-level use ranged from 0% to 67% in 2019.

By 2020, observation status has become a key component of healthcare for pediatric patients, and its relevance for children’s hospitals recently has been described.^22,23 However, trends in observation stays for pediatric populations are not known. This represents the first study showing temporal trends of observation stays at children’s hospitals after 2010. Our results confirm that the increase in observation stays for pediatric populations is not attributable to decreasing patient acuity at children’s hospitals. We found a weak but significantly positive correlation between year and intensity of care used under observation stays. Although this correlation might not be clinically important, it demonstrates that patient acuity in observation stays is not decreasing. Regression results suggest that observation stays now encompass patients who need relatively higher intensity of care compared with those admitted under observation status in 2010.

This study also identifies a unique pattern in the use of observation stays among pediatric populations. Earlier studies exclusively focused on observation stays that were admitted from EDs.²⁴ Our results indicate that observation status has been used beyond a bridge from ED care to inpatient admission. In particular, observation status has expanded to include pediatric populations with more diverse clinical conditions (eg, appendicitis and diabetes mellitus), and has become a substantial component of postprocedural admissions (Appendix Figure). Looking forward, it is likely that the use of observation stays might surpass inpatient admissions for more conditions that primarily involve short-term stays.

Observation status originally was designed as a reimbursement strategy for patients who needed short stays in dedicated ED units or hospitals, but did not qualify for inpatient services.^5,25 After several changes in reimbursement policies, CMS released the “two midnight rule” for Medicare beneficiaries in 2013, which replaced condition-based criteria with time-based criteria to determine an inpatient or observation stay.¹ Some Medicaid programs and commercial payors have developed similar policies. Unlike the universal policy for Medicare populations, the regulations for pediatric populations vary by states and health insurers.^11-15,26-28 This might partially explain the wide variation observed among children’s hospital-level use of observation stays. For example, the California Medicaid program did not have a reimbursement rate for observation services as of 2020, while the Texas Medicaid program has had a policy for observation stays since 2010.^12,13 We found that two children’s hospitals in California had the lowest use of observation stays (almost zero), whereas the hospital-level use of observation stays was more than 50% for three out of four children’s hospitals in Texas. In addition to reimbursement policies, individual hospitals also might have different strategies for observation status designation. An earlier survey showed that there was lack of consistency in billing and payor-based designations of observation status at children’s hospitals.²⁹ These findings suggest that children’s hospital-level use of observation stays likely is influenced by reimbursement policy and practical strategy for observation status determination.

Earlier studies reported that observation status could be a more efficient use of healthcare resources.^5,6 However, there are still at least two concerns relevant to children’s hospitals during the last decade. The first is whether the use of observation stays can promote cost-saving or if it is just a cost-shifting strategy. An earlier study demonstrated that observation stays with prolonged LOS might increase risk of cost-sharing among adult patients.²⁹ Our study reveals an increasing trend of observation stays with prolonged LOS for pediatric patients. Similar to adult patients, LOS exceeding 24 or 48 hours could lead to uncovered healthcare costs and financial burdens on families.^30-32 Meanwhile, children’s hospitals also might take on a higher financial liability by implementing observation status. Earlier studies have indicated that resource use between observation and inpatient stays at children’s hospitals is similar, and increasing use of observation stays might lead to financial risk rather than cost effectiveness.³³ Further, administrative costs of observation determination are considerably high.³⁴ Medicaid is the major payor for pediatric patients in children’s hospitals. In this study, more than 50% of encounters were paid through Medicaid programs. It is well known that Medicaid reimbursement rates are lower than Medicare and commercial plans.³⁵ Therefore, the cost-saving conclusion drawn from Medicare patients cannot be generalized to pediatric populations at children’s hospitals without cautious reevaluation.

A second concern with increasing use of observation stays is selection bias in public reporting and comparisons of hospital performance. Presently, four main categories of quality indicators established by the Agency for Healthcare Research and Quality rely heavily on inpatient encounters.³⁶ In this study, we found that the range of hospital-level use of observation stays was large. In 2019, the risk-adjusted percent use of observation stays was less than 5% at three hospitals, while the percent use was greater than 60% in another three hospitals. Therefore, comparisons made without uniform accounting of observation stays might have significant implications for national rankings of children’s hospitals across the United States. These consequences have been investigated in several published studies.^22,23,37-39

There are several limitations to our study. First, the study sample was limited to children’s hospitals that consistently reported inpatient and observation data over the entire study period. Eighteen hospitals (86%) excluded from this study did not consistently submit inpatient and observation data to PHIS from 2010 through 2019. The primary purpose of this study was to present temporal trends of observation stays for children’s hospitals, and it was important to build the hospital cohort based on valid and consistent data during the study period. Appendix Table 4 presents differences of hospital characteristics by included and excluded groups of hospitals. Excluded hospitals might have fewer resources (eg, fewer pediatric intensive care beds). Nonetheless, the selection of hospitals was optimized based on data availability. Second, this study was a retrospective review of an administrative database of children’s hospitals and units. The sample does not represent all children’s hospitals or pediatric patients in the United States, but there are no available data sources—that we know of—that can generate national estimates for both inpatient and observation stays. Third, we did not attempt to conclusively infer any causal effects, and several factors could explain the increasing trends, such as reimbursement policies, hospital-level implementation strategies, determination guidelines for observation status designation, as well as changes in clinical care. Further studies should investigate impact of these factors on the use of observation stays for pediatric patients and children’s hospitals.

Observation status has been increasingly used for pediatric patients with more diverse clinical conditions, and there is a rising trend of prolonged LOS among observation stays since 2010. Considerable variation exists in hospital-level use of observation stays across children’s hospitals. Observation status could be an opportunity to improve efficiency of healthcare resource use or could lead to a financial risk for patients with prolonged LOS. Future studies should explore appropriateness of observation care in clinical practice through leveraging efficient care and alleviating financial risk.

Sliding scale insulin (SSI) for inpatient glycemic control was first proposed by Elliott P Joslin in 1934 when he recommended titration of insulin based on urine glucose levels.¹ As bedside glucose meters became widely available, physicians transitioned to dosing SSI based on capillary blood glucose (BG) levels,^2,3 and SSI became widely used for the management of inpatient hyperglycemia.¹ However, during the past decade, there has been strong opposition to the use of SSI in hospitals. Many authors oppose its use, highlighting the retrospective rather than prospective nature of SSI therapy and concerns about inadequate glycemic control.^4-6 In 2004, the American College of Endocrinology first released a position statement discouraging the use of SSI alone and recommended basal-bolus insulin as the preferred method of glycemic control for inpatients with type 2 diabetes (T2D).⁷ The American Diabetes Association (ADA) inpatient guidelines in 2005⁸ and the Endocrine Society guidelines in 2012⁹ also opposed SSI monotherapy and reaffirmed that a basal-bolus insulin regimen should be used for most non–critically ill patients with diabetes. Those guidelines remain in place currently.

Several randomized controlled trials (RCTs) and meta-analyses have shown that basal-bolus insulin regimens provide superior glycemic control in non–critical inpatients when compared with SSI alone.^10-14 In addition, the RABBIT 2 (Randomized Study of Basal-Bolus Insulin Therapy in the Inpatient Management of Patients With Type 2 Diabetes) trial showed a significant reduction in perioperative complications¹⁰ among surgical patients when treated with basal-bolus insulin therapy. Despite these studies and strong recommendations against its use, SSI continues to be widely used in the United States. According to a 2007 survey of 44 US hospitals, 41% of noncritical patients with hyperglycemia were treated with SSI alone.¹⁵ In addition, SSI remains one of the most commonly prescribed insulin regimens in many countries around the world.^16-19 The persistence of SSI use raises questions as to why clinicians continue to use a therapy that has been strongly criticized. Some authors point to convenience and fear of hypoglycemia with a basal-bolus insulin regimen.^20,21 Alternatively, it is possible that SSI usage remains so pervasive because it is effective in a subset of patients. In fact, a 2018 Cochrane review concluded that existing evidence is not sufficiently robust to definitively recommend basal-bolus insulin over SSI for inpatient diabetes management of non–critically ill patients despite existing guidelines.²²

Owing to the ongoing controversy and widespread use of SSI, we designed an exploratory analysis to understand the rationale for such therapy by investigating whether a certain subpopulation of hospitalized patients with T2D may achieve target glycemic control with SSI alone. We hypothesized that noncritical patients with mild hyperglycemia and admission BG <180 mg/dL would do well with SSI alone and may not require intensive treatment with basal-bolus insulin regimens. To address this question, we used electronic health records with individual-level patient data to assess inpatient glycemic control of non–critically ill patients with T2D treated with SSI alone.

Participants

Data from 25,813 adult noncritical inpatients with T2D, with an index admission between June 1, 2010, and June 30, 2018, were obtained through the Emory Healthcare Clinical Data Warehouse infrastructure program. All patients were admitted to Emory Healthcare hospitals, including Emory University Hospital, Emory University Hospital Midtown, and Emory Saint Joseph’s Hospital, in Atlanta, Georgia. Data were extracted for each patient during the index hospitalization, including demographics, anthropometrics, and admission and inpatient laboratory values. Information was collected on daily point-of-care glucose values, hemoglobin A1c (HbA1c), hypoglycemic events, insulin doses, hospital complications, comorbidities, and hospital setting (medical vs surgical admission). International Classification of Diseases, 9th and 10th Revisions (ICD-9/10) codes were used to determine diagnosis of T2D, comorbidities, and complications.

From our initial dataset, we identified 16,366 patients who were treated with SSI during hospitalization. We excluded patients who were admitted to the intensive care unit (ICU) or placed on intravenous insulin, patients with missing admission BG values, and patients with a length of stay less than 1 day. To prevent inclusion of patients presenting in diabetic ketoacidosis or hyperosmolar hyperglycemic syndrome, we excluded patients with an admission BG >500 mg/dL. We then excluded 6,739 patients who received basal insulin within the first 2 days of hospitalization, as well as 943 patients who were treated with noninsulin (oral or injectable) antidiabetic agents. Our final dataset included 8,095 patients (Appendix Figure).

Patients in the SSI cohort included all patients who were treated with short-acting insulin only (regular insulin or rapid-acting [lispro, aspart, glulisine] insulin analogs) during the first 2 days of hospitalization. Patients who remained on only short-acting insulin during the entire hospitalization were defined as continuous SSI patients. Patients who subsequently received basal insulin after day 2 of hospitalization were defined as patients who transitioned to basal. Patients were stratified according to admission BG levels (first BG available on day of admission) and HbA1c (when available during index admission). We compared the baseline characteristics and clinical outcomes of patients who remained on SSI alone throughout the entirety of hospitalization with those of patients who required transition to basal insulin. The mean hospital BG was calculated by taking the average of all BG measurements during the hospital stay. We defined hypoglycemia as a BG <70 mg/dL and severe hypoglycemia as BG <40 mg/dL. Repeated hypoglycemia values were excluded if they occurred within a period of 2 hours.

Outcome Measures

The primary outcome was the percentage of patients with T2D achieving target glycemic control with SSI therapy, defined as mean hospital BG between 70 and 180 mg/dL without hypoglycemia <70 mg/dL during hospital stay. This threshold was determined based on 2019 ADA recommendations targeting hospital BG <180 mg/dL and avoidance of hypoglycemia.²³

Statistical Analysis

Patients were stratified according to continuous SSI versus transitioned to basal treatment. Patients who remained on continuous SSI were further categorized into four categories based on admission BG: <140 mg/dL, 140 to 180 mg/dL, 180 to 250 mg/dL, and ≥250 mg/dL. Clinical characteristics were compared using Wilcoxon rank-sum tests (if continuous) and chi-square tests or Fisher exact tests (if categorical). We then compared the clinical outcomes among continuous SSI patients with different admission BG levels (<140 mg/dL, 140-180 mg/dL, 180-250 mg/dL, and ≥250 mg/dL) and with different HbA1c levels (<7%, 7%-8%, 8%-9%, ≥9%). Within each scenario, logistic regression for the outcome of poor glycemic control, defined as mean hospital BG >180 mg/dL, was performed to evaluate the HbA1c levels and admission BG levels controlling for other factors (age, gender, body mass index [BMI], race, setting [medicine versus surgery] and Charlson Comorbidity Index score). A P value < .05 was regarded as statistically significant. All analyses were performed based on available cases and conducted in SAS version 9.4 (SAS Institute Inc.).

Among 25,813 adult patients with T2D, 8,095 patients (31.4%) were treated with SSI alone during the first 2 days of hospitalization. Of those patients treated with SSI, 6,903 (85%) remained on continuous SSI alone during the entire hospitalization, and 1,192 (15%) were transitioned to basal insulin. The clinical characteristics of these patients on continuous SSI and those who transitioned to basal insulin are shown in Table 1. Patients who transitioned to basal insulin had significantly higher mean (SD) admission BG (191.8 [88.2] mg/dL vs 156.4 [65.4] mg/dL, P < .001) and higher mean (SD) HbA1c (8.1% [2.0%] vs 7.01% [1.5%], P < .001), compared with those who remained on continuous SSI. Patients who transitioned to basal insulin were also younger and more likely to have chronic kidney disease (CKD), but less likely to have congestive heart failure, coronary artery disease, or chronic obstructive pulmonary disease (COPD). The Charlson Comorbidity Index score was significantly higher for patients who transitioned to basal (4.4 [2.5]) than for those who remained on continuous SSI (4.1 [2.5], P < .001). There were no significant differences among sex, BMI, or glomerular filtration rate (GFR) on admission. Of those transitioned to basal insulin, 53% achieved a mean hospitalization BG <180 mg/dL, compared with 82% of those on continuous SSI. The overall rate of hypoglycemia in the continuous SSI group was 8% compared with 18% in those transitioned to basal insulin.

Of the patients who remained on continuous SSI throughout the hospitalization, 3,319 patients (48%) had admission BG <140 mg/dL, 1,671 patients (24%) had admission BG 140 to 180 mg/dL, and 1,913 patients (28%) had admission BG >180 mg/dL. Only 9% of patients who remained on continuous SSI had admission BG ≥250 mg/dL. Patients with admission BG <140 mg/dL were older, had lower BMI and HbA1c, had higher rates of COPD and CKD, and were more likely to be admitted to a surgical service compared with patients with admission BG >140 mg/dL (P < .05 for all; Table 2).

Hospital glycemic control for patients on continuous SSI according to admission BG is displayed in Table 3. Among patients who remained on continuous SSI, 96% of patients with admission BG <140 mg/dL had a mean hospital BG <180 mg/dL; of them, 86% achieved target control without hypoglycemia. Similar rates of target control were achieved in patients with admission BG 140 to 180 mg/dL (83%), in contrast to patients with admission BG ≥250 mg/dL, of whom only 18% achieved target control (P < .001). These findings parallel those seen in patients transitioned to basal insulin. Of patients in the transition group admitted with BG <140 mg/dL and <180 mg/dL, 88.5% and 84.6% had mean hospital BG <180 mg/dL, respectively, while 69.1% and 68.9% had mean BG between 70 and 180 mg/dL without hypoglycemia. The overall frequency of hypoglycemia <70 mg/dL among patients on continuous SSI was 8% and was more common in patients with admission BG <140 mg/dL (10%) compared with patients with higher admission glucose levels (BG 140-180 mg/dL [4%], 180-250 mg/dL [4%], or ≥250 mg/dL [6%], P < .001). There was no difference in rates of severe hypoglycemia <40 mg/dL among groups.

HbA1c data were available for 2,560 of the patients on continuous SSI (Table 3). Mean hospital BG increased significantly with increasing HbA1c values. Patients admitted with HbA1c <7% had lower mean (SD) hospital BG (132.2 [28.2] mg/dL) and were more likely to achieve target glucose control during hospitalization (85%) compared with those with HbA1c 7% to 8% (mean BG, 148.7 [30.8] mg/dL; 80% target control), HbA1c 8% to 9% (mean BG, 169.1 [37.9] mg/dL; 61% target control), or HbA1c ≥9% (mean BG, 194.9 [53.4] mg/dL; 38% target control) (P < .001).

In a logistic regression analysis adjusted for age, gender, BMI, race, setting (medicine vs surgery), and Charlson Comorbidity Index score, the odds of poor glycemic control increased with higher admission BG (admission BG 140-180 mg/dL: odds ratio [OR], 1.8; 95% CI, 1.5-2.2; admission BG 180-250 mg/dL: OR, 3.7; 95% CI, 3.1-4.4; admission BG ≥250 mg/dL: OR, 7.2; 95% CI, 5.8-9.0; reference admission BG <140 mg/dL; Figure). Similarly, the logistic regression analysis showed greater odds of poor in-hospital glycemic control with increasing HbA1c (OR, 6.1; 95% CI, 4.3-8.8 for HbA1c >9% compared with HbA1c <7%).

This large retrospective cohort study examined the effectiveness of SSI for glycemic control in noncritical inpatients with T2D. Our results indicate that SSI is still widely used in our hospital system, with 31.4% of our initial cohort managed with SSI alone. We found that 86% of patients with BG <140 mg/dL and 83% of patients with BG 140 to 180 mg/dL achieved glycemic control without hypoglycemia when managed with SSI alone, compared with 53% of those admitted with BG 180 to 250 mg/dL and only 18% of those with admission BG ≥250 mg/dL. This high success rate of achieving optimal BG control with SSI alone is comparable to that seen with transition to basal insulin and may explain the prevalent use of SSI for the management of patients with T2D and mild to moderate hyperglycemia.

Published clinical guideline recommendations promoting the use of basal-bolus insulin treatment algorithms are based on the results of a few RCTs that compared the efficacy of SSI vs a basal-bolus insulin regimen. These studies reported significantly lower mean daily BG concentration with basal or basal-bolus insulin therapy compared with SSI.^10,11,24 However, it is interesting to note that the mean admission BG of patients treated with SSI in these RCTs ranged from 184 to 225 mg/dL. Patients in these trials were excluded if admission BG was <140 mg/dL.^10,11,24 This is in contrast to our study evaluating real-world data in non–critically ill settings in which we found that 48% of patients treated with SSI had admission BG <140 mg/dL, and nearly 75% had admission BG <180 mg/dL. This suggests that by nature of study design, most RCTs excluded the population of patients who do achieve good glycemic control with SSI and may have contributed to the perception that basal insulin is preferable in all populations.

Our analysis indicates that healthcare professionals should consider admission BG when selecting the type of insulin regimen to manage patients with T2D in the hospital. Our results suggest that SSI may be appropriate for many patients with admission BG <180 mg/dL and should be avoided as monotherapy in patients with admission BG ≥180 mg/dL, as the proportion of patients achieving target control decreased with increasing admission BG. More importantly, if a patient is not controlled with SSI alone, intensification of therapy with the addition of basal insulin is indicated to achieve glycemic control. In addition, we found that the admission HbA1c is an appropriate marker to consider as well, with hospital glycemic control deteriorating with increasing HbA1c values, paralleling the admission BG. The main limitation to widespread use of HbA1c for therapeutic decision-making is access to values at time of patient admission; in our population, only 37% of patients had an HbA1c value available during the index hospitalization.

Previous publications have reported that hypoglycemia carries significant safety concerns, especially among a hospitalized population.^25-27 As such, we included hypoglycemia as an important metric in our definition of target glycemic control rather than simply using mean hospital BG or number of hyperglycemic events to define treatment effectiveness. We did find a higher rate of hypoglycemia in patients with moderate admission BG treated with SSI compared with those with higher admission BG; however, few patients overall experienced clinically significant (<54 mg/dL) or severe (<40 mg/dL) hypoglycemia.

In our population, only 15% of patients started on SSI received additional basal insulin during hospitalization. This finding is similar to data reported in the Rabbit 2 trial, in which 14% of patients failed SSI alone, with a higher failure rate among those with higher BG on admission.¹⁰ Given the observational nature of this study, we cannot definitively state why certain patients in our population required additional basal insulin, but we can hypothesize that these patients admitted with BG ≥180 mg/dL had higher treatment failure rates and greater rates of hyperglycemia, therefore receiving intensified insulin therapy as clinically indicated at the discretion of the treating physician. Patients who transitioned from SSI to basal insulin had significantly higher admission BG and HbA1c compared with patients who remained on SSI alone. We noted that the rates of hypoglycemia were higher in the group that transitioned to basal (18% vs 8%) and similar to rates reported in previous RCTs.^11,24

This observational study takes advantage of a large, diverse study population and a combination of medicine and surgery patients in a real-world setting. We acknowledge several limitations in our study. Our primary data were observational in nature, and as such, some baseline patient characteristics were notably different between groups, suggesting selection bias for treatment allocation to SSI. We do not know which patients were managed by primary teams compared with specialized diabetes consult services, which may also influence treatment regimens. We did not have access to information about patients’ at-home diabetes medication regimens or duration of diabetes, both of which have been shown in prior publications to affect an individual’s overall hospital glycemic control. Data on HbA1c values were available for only approximately one-third of patients. In addition, our study did not include patients without a history of diabetes who developed stress-induced hyperglycemia, a population that may benefit from conservative therapy such as SSI.²⁸ A diagnosis of CKD was defined based on ICD 9/10 codes and not on admission estimated GFR. More specific data regarding stage of CKD or changes in renal function over the duration of hospitalization are not available, which could influence insulin prescribing practice. In addition, we defined the basal group as patients prescribed any form of basal insulin (NPH, glargine, detemir or degludec), and we do not have information on the use of prandial versus correction doses of rapid-acting insulin in the basal insulin–treated group.

In conclusion, our observational study indicates that the use of SSI results in appropriate target glycemic control for most noncritical medicine and surgery patients with admission BG <180 mg/dL. In agreement with previous RCTs, our study confirms that SSI as monotherapy is frequently inadequate in patients with significant hyperglycemia >180 mg/dL.^10,11,24,29 We propose that an individualized approach to inpatient glycemic management is imperative, and cautious use of SSI may be a viable option for certain patients with mild hyperglycemia and admission BG <180 mg/dL. Further observational and randomized studies are needed to confirm the efficacy of SSI therapy in T2D patients with mild hyperglycemia. By identifying which subset of patients can be safely managed with SSI alone, we can better understand which patients will require escalation of therapy with intensive glucose management.

Sliding scale insulin (SSI) for inpatient glycemic control was first proposed by Elliott P Joslin in 1934 when he recommended titration of insulin based on urine glucose levels.¹ As bedside glucose meters became widely available, physicians transitioned to dosing SSI based on capillary blood glucose (BG) levels,^2,3 and SSI became widely used for the management of inpatient hyperglycemia.¹ However, during the past decade, there has been strong opposition to the use of SSI in hospitals. Many authors oppose its use, highlighting the retrospective rather than prospective nature of SSI therapy and concerns about inadequate glycemic control.^4-6 In 2004, the American College of Endocrinology first released a position statement discouraging the use of SSI alone and recommended basal-bolus insulin as the preferred method of glycemic control for inpatients with type 2 diabetes (T2D).⁷ The American Diabetes Association (ADA) inpatient guidelines in 2005⁸ and the Endocrine Society guidelines in 2012⁹ also opposed SSI monotherapy and reaffirmed that a basal-bolus insulin regimen should be used for most non–critically ill patients with diabetes. Those guidelines remain in place currently.

Several randomized controlled trials (RCTs) and meta-analyses have shown that basal-bolus insulin regimens provide superior glycemic control in non–critical inpatients when compared with SSI alone.^10-14 In addition, the RABBIT 2 (Randomized Study of Basal-Bolus Insulin Therapy in the Inpatient Management of Patients With Type 2 Diabetes) trial showed a significant reduction in perioperative complications¹⁰ among surgical patients when treated with basal-bolus insulin therapy. Despite these studies and strong recommendations against its use, SSI continues to be widely used in the United States. According to a 2007 survey of 44 US hospitals, 41% of noncritical patients with hyperglycemia were treated with SSI alone.¹⁵ In addition, SSI remains one of the most commonly prescribed insulin regimens in many countries around the world.^16-19 The persistence of SSI use raises questions as to why clinicians continue to use a therapy that has been strongly criticized. Some authors point to convenience and fear of hypoglycemia with a basal-bolus insulin regimen.^20,21 Alternatively, it is possible that SSI usage remains so pervasive because it is effective in a subset of patients. In fact, a 2018 Cochrane review concluded that existing evidence is not sufficiently robust to definitively recommend basal-bolus insulin over SSI for inpatient diabetes management of non–critically ill patients despite existing guidelines.²²

Owing to the ongoing controversy and widespread use of SSI, we designed an exploratory analysis to understand the rationale for such therapy by investigating whether a certain subpopulation of hospitalized patients with T2D may achieve target glycemic control with SSI alone. We hypothesized that noncritical patients with mild hyperglycemia and admission BG <180 mg/dL would do well with SSI alone and may not require intensive treatment with basal-bolus insulin regimens. To address this question, we used electronic health records with individual-level patient data to assess inpatient glycemic control of non–critically ill patients with T2D treated with SSI alone.

Participants

Data from 25,813 adult noncritical inpatients with T2D, with an index admission between June 1, 2010, and June 30, 2018, were obtained through the Emory Healthcare Clinical Data Warehouse infrastructure program. All patients were admitted to Emory Healthcare hospitals, including Emory University Hospital, Emory University Hospital Midtown, and Emory Saint Joseph’s Hospital, in Atlanta, Georgia. Data were extracted for each patient during the index hospitalization, including demographics, anthropometrics, and admission and inpatient laboratory values. Information was collected on daily point-of-care glucose values, hemoglobin A1c (HbA1c), hypoglycemic events, insulin doses, hospital complications, comorbidities, and hospital setting (medical vs surgical admission). International Classification of Diseases, 9th and 10th Revisions (ICD-9/10) codes were used to determine diagnosis of T2D, comorbidities, and complications.

From our initial dataset, we identified 16,366 patients who were treated with SSI during hospitalization. We excluded patients who were admitted to the intensive care unit (ICU) or placed on intravenous insulin, patients with missing admission BG values, and patients with a length of stay less than 1 day. To prevent inclusion of patients presenting in diabetic ketoacidosis or hyperosmolar hyperglycemic syndrome, we excluded patients with an admission BG >500 mg/dL. We then excluded 6,739 patients who received basal insulin within the first 2 days of hospitalization, as well as 943 patients who were treated with noninsulin (oral or injectable) antidiabetic agents. Our final dataset included 8,095 patients (Appendix Figure).

Patients in the SSI cohort included all patients who were treated with short-acting insulin only (regular insulin or rapid-acting [lispro, aspart, glulisine] insulin analogs) during the first 2 days of hospitalization. Patients who remained on only short-acting insulin during the entire hospitalization were defined as continuous SSI patients. Patients who subsequently received basal insulin after day 2 of hospitalization were defined as patients who transitioned to basal. Patients were stratified according to admission BG levels (first BG available on day of admission) and HbA1c (when available during index admission). We compared the baseline characteristics and clinical outcomes of patients who remained on SSI alone throughout the entirety of hospitalization with those of patients who required transition to basal insulin. The mean hospital BG was calculated by taking the average of all BG measurements during the hospital stay. We defined hypoglycemia as a BG <70 mg/dL and severe hypoglycemia as BG <40 mg/dL. Repeated hypoglycemia values were excluded if they occurred within a period of 2 hours.

Outcome Measures

The primary outcome was the percentage of patients with T2D achieving target glycemic control with SSI therapy, defined as mean hospital BG between 70 and 180 mg/dL without hypoglycemia <70 mg/dL during hospital stay. This threshold was determined based on 2019 ADA recommendations targeting hospital BG <180 mg/dL and avoidance of hypoglycemia.²³

Statistical Analysis

Patients were stratified according to continuous SSI versus transitioned to basal treatment. Patients who remained on continuous SSI were further categorized into four categories based on admission BG: <140 mg/dL, 140 to 180 mg/dL, 180 to 250 mg/dL, and ≥250 mg/dL. Clinical characteristics were compared using Wilcoxon rank-sum tests (if continuous) and chi-square tests or Fisher exact tests (if categorical). We then compared the clinical outcomes among continuous SSI patients with different admission BG levels (<140 mg/dL, 140-180 mg/dL, 180-250 mg/dL, and ≥250 mg/dL) and with different HbA1c levels (<7%, 7%-8%, 8%-9%, ≥9%). Within each scenario, logistic regression for the outcome of poor glycemic control, defined as mean hospital BG >180 mg/dL, was performed to evaluate the HbA1c levels and admission BG levels controlling for other factors (age, gender, body mass index [BMI], race, setting [medicine versus surgery] and Charlson Comorbidity Index score). A P value < .05 was regarded as statistically significant. All analyses were performed based on available cases and conducted in SAS version 9.4 (SAS Institute Inc.).

Among 25,813 adult patients with T2D, 8,095 patients (31.4%) were treated with SSI alone during the first 2 days of hospitalization. Of those patients treated with SSI, 6,903 (85%) remained on continuous SSI alone during the entire hospitalization, and 1,192 (15%) were transitioned to basal insulin. The clinical characteristics of these patients on continuous SSI and those who transitioned to basal insulin are shown in Table 1. Patients who transitioned to basal insulin had significantly higher mean (SD) admission BG (191.8 [88.2] mg/dL vs 156.4 [65.4] mg/dL, P < .001) and higher mean (SD) HbA1c (8.1% [2.0%] vs 7.01% [1.5%], P < .001), compared with those who remained on continuous SSI. Patients who transitioned to basal insulin were also younger and more likely to have chronic kidney disease (CKD), but less likely to have congestive heart failure, coronary artery disease, or chronic obstructive pulmonary disease (COPD). The Charlson Comorbidity Index score was significantly higher for patients who transitioned to basal (4.4 [2.5]) than for those who remained on continuous SSI (4.1 [2.5], P < .001). There were no significant differences among sex, BMI, or glomerular filtration rate (GFR) on admission. Of those transitioned to basal insulin, 53% achieved a mean hospitalization BG <180 mg/dL, compared with 82% of those on continuous SSI. The overall rate of hypoglycemia in the continuous SSI group was 8% compared with 18% in those transitioned to basal insulin.

Of the patients who remained on continuous SSI throughout the hospitalization, 3,319 patients (48%) had admission BG <140 mg/dL, 1,671 patients (24%) had admission BG 140 to 180 mg/dL, and 1,913 patients (28%) had admission BG >180 mg/dL. Only 9% of patients who remained on continuous SSI had admission BG ≥250 mg/dL. Patients with admission BG <140 mg/dL were older, had lower BMI and HbA1c, had higher rates of COPD and CKD, and were more likely to be admitted to a surgical service compared with patients with admission BG >140 mg/dL (P < .05 for all; Table 2).

Hospital glycemic control for patients on continuous SSI according to admission BG is displayed in Table 3. Among patients who remained on continuous SSI, 96% of patients with admission BG <140 mg/dL had a mean hospital BG <180 mg/dL; of them, 86% achieved target control without hypoglycemia. Similar rates of target control were achieved in patients with admission BG 140 to 180 mg/dL (83%), in contrast to patients with admission BG ≥250 mg/dL, of whom only 18% achieved target control (P < .001). These findings parallel those seen in patients transitioned to basal insulin. Of patients in the transition group admitted with BG <140 mg/dL and <180 mg/dL, 88.5% and 84.6% had mean hospital BG <180 mg/dL, respectively, while 69.1% and 68.9% had mean BG between 70 and 180 mg/dL without hypoglycemia. The overall frequency of hypoglycemia <70 mg/dL among patients on continuous SSI was 8% and was more common in patients with admission BG <140 mg/dL (10%) compared with patients with higher admission glucose levels (BG 140-180 mg/dL [4%], 180-250 mg/dL [4%], or ≥250 mg/dL [6%], P < .001). There was no difference in rates of severe hypoglycemia <40 mg/dL among groups.

HbA1c data were available for 2,560 of the patients on continuous SSI (Table 3). Mean hospital BG increased significantly with increasing HbA1c values. Patients admitted with HbA1c <7% had lower mean (SD) hospital BG (132.2 [28.2] mg/dL) and were more likely to achieve target glucose control during hospitalization (85%) compared with those with HbA1c 7% to 8% (mean BG, 148.7 [30.8] mg/dL; 80% target control), HbA1c 8% to 9% (mean BG, 169.1 [37.9] mg/dL; 61% target control), or HbA1c ≥9% (mean BG, 194.9 [53.4] mg/dL; 38% target control) (P < .001).

In a logistic regression analysis adjusted for age, gender, BMI, race, setting (medicine vs surgery), and Charlson Comorbidity Index score, the odds of poor glycemic control increased with higher admission BG (admission BG 140-180 mg/dL: odds ratio [OR], 1.8; 95% CI, 1.5-2.2; admission BG 180-250 mg/dL: OR, 3.7; 95% CI, 3.1-4.4; admission BG ≥250 mg/dL: OR, 7.2; 95% CI, 5.8-9.0; reference admission BG <140 mg/dL; Figure). Similarly, the logistic regression analysis showed greater odds of poor in-hospital glycemic control with increasing HbA1c (OR, 6.1; 95% CI, 4.3-8.8 for HbA1c >9% compared with HbA1c <7%).

This large retrospective cohort study examined the effectiveness of SSI for glycemic control in noncritical inpatients with T2D. Our results indicate that SSI is still widely used in our hospital system, with 31.4% of our initial cohort managed with SSI alone. We found that 86% of patients with BG <140 mg/dL and 83% of patients with BG 140 to 180 mg/dL achieved glycemic control without hypoglycemia when managed with SSI alone, compared with 53% of those admitted with BG 180 to 250 mg/dL and only 18% of those with admission BG ≥250 mg/dL. This high success rate of achieving optimal BG control with SSI alone is comparable to that seen with transition to basal insulin and may explain the prevalent use of SSI for the management of patients with T2D and mild to moderate hyperglycemia.

Published clinical guideline recommendations promoting the use of basal-bolus insulin treatment algorithms are based on the results of a few RCTs that compared the efficacy of SSI vs a basal-bolus insulin regimen. These studies reported significantly lower mean daily BG concentration with basal or basal-bolus insulin therapy compared with SSI.^10,11,24 However, it is interesting to note that the mean admission BG of patients treated with SSI in these RCTs ranged from 184 to 225 mg/dL. Patients in these trials were excluded if admission BG was <140 mg/dL.^10,11,24 This is in contrast to our study evaluating real-world data in non–critically ill settings in which we found that 48% of patients treated with SSI had admission BG <140 mg/dL, and nearly 75% had admission BG <180 mg/dL. This suggests that by nature of study design, most RCTs excluded the population of patients who do achieve good glycemic control with SSI and may have contributed to the perception that basal insulin is preferable in all populations.

Our analysis indicates that healthcare professionals should consider admission BG when selecting the type of insulin regimen to manage patients with T2D in the hospital. Our results suggest that SSI may be appropriate for many patients with admission BG <180 mg/dL and should be avoided as monotherapy in patients with admission BG ≥180 mg/dL, as the proportion of patients achieving target control decreased with increasing admission BG. More importantly, if a patient is not controlled with SSI alone, intensification of therapy with the addition of basal insulin is indicated to achieve glycemic control. In addition, we found that the admission HbA1c is an appropriate marker to consider as well, with hospital glycemic control deteriorating with increasing HbA1c values, paralleling the admission BG. The main limitation to widespread use of HbA1c for therapeutic decision-making is access to values at time of patient admission; in our population, only 37% of patients had an HbA1c value available during the index hospitalization.

Previous publications have reported that hypoglycemia carries significant safety concerns, especially among a hospitalized population.^25-27 As such, we included hypoglycemia as an important metric in our definition of target glycemic control rather than simply using mean hospital BG or number of hyperglycemic events to define treatment effectiveness. We did find a higher rate of hypoglycemia in patients with moderate admission BG treated with SSI compared with those with higher admission BG; however, few patients overall experienced clinically significant (<54 mg/dL) or severe (<40 mg/dL) hypoglycemia.

In our population, only 15% of patients started on SSI received additional basal insulin during hospitalization. This finding is similar to data reported in the Rabbit 2 trial, in which 14% of patients failed SSI alone, with a higher failure rate among those with higher BG on admission.¹⁰ Given the observational nature of this study, we cannot definitively state why certain patients in our population required additional basal insulin, but we can hypothesize that these patients admitted with BG ≥180 mg/dL had higher treatment failure rates and greater rates of hyperglycemia, therefore receiving intensified insulin therapy as clinically indicated at the discretion of the treating physician. Patients who transitioned from SSI to basal insulin had significantly higher admission BG and HbA1c compared with patients who remained on SSI alone. We noted that the rates of hypoglycemia were higher in the group that transitioned to basal (18% vs 8%) and similar to rates reported in previous RCTs.^11,24

This observational study takes advantage of a large, diverse study population and a combination of medicine and surgery patients in a real-world setting. We acknowledge several limitations in our study. Our primary data were observational in nature, and as such, some baseline patient characteristics were notably different between groups, suggesting selection bias for treatment allocation to SSI. We do not know which patients were managed by primary teams compared with specialized diabetes consult services, which may also influence treatment regimens. We did not have access to information about patients’ at-home diabetes medication regimens or duration of diabetes, both of which have been shown in prior publications to affect an individual’s overall hospital glycemic control. Data on HbA1c values were available for only approximately one-third of patients. In addition, our study did not include patients without a history of diabetes who developed stress-induced hyperglycemia, a population that may benefit from conservative therapy such as SSI.²⁸ A diagnosis of CKD was defined based on ICD 9/10 codes and not on admission estimated GFR. More specific data regarding stage of CKD or changes in renal function over the duration of hospitalization are not available, which could influence insulin prescribing practice. In addition, we defined the basal group as patients prescribed any form of basal insulin (NPH, glargine, detemir or degludec), and we do not have information on the use of prandial versus correction doses of rapid-acting insulin in the basal insulin–treated group.

In conclusion, our observational study indicates that the use of SSI results in appropriate target glycemic control for most noncritical medicine and surgery patients with admission BG <180 mg/dL. In agreement with previous RCTs, our study confirms that SSI as monotherapy is frequently inadequate in patients with significant hyperglycemia >180 mg/dL.^10,11,24,29 We propose that an individualized approach to inpatient glycemic management is imperative, and cautious use of SSI may be a viable option for certain patients with mild hyperglycemia and admission BG <180 mg/dL. Further observational and randomized studies are needed to confirm the efficacy of SSI therapy in T2D patients with mild hyperglycemia. By identifying which subset of patients can be safely managed with SSI alone, we can better understand which patients will require escalation of therapy with intensive glucose management.

Sliding scale insulin (SSI) for inpatient glycemic control was first proposed by Elliott P Joslin in 1934 when he recommended titration of insulin based on urine glucose levels.¹ As bedside glucose meters became widely available, physicians transitioned to dosing SSI based on capillary blood glucose (BG) levels,^2,3 and SSI became widely used for the management of inpatient hyperglycemia.¹ However, during the past decade, there has been strong opposition to the use of SSI in hospitals. Many authors oppose its use, highlighting the retrospective rather than prospective nature of SSI therapy and concerns about inadequate glycemic control.^4-6 In 2004, the American College of Endocrinology first released a position statement discouraging the use of SSI alone and recommended basal-bolus insulin as the preferred method of glycemic control for inpatients with type 2 diabetes (T2D).⁷ The American Diabetes Association (ADA) inpatient guidelines in 2005⁸ and the Endocrine Society guidelines in 2012⁹ also opposed SSI monotherapy and reaffirmed that a basal-bolus insulin regimen should be used for most non–critically ill patients with diabetes. Those guidelines remain in place currently.

Several randomized controlled trials (RCTs) and meta-analyses have shown that basal-bolus insulin regimens provide superior glycemic control in non–critical inpatients when compared with SSI alone.^10-14 In addition, the RABBIT 2 (Randomized Study of Basal-Bolus Insulin Therapy in the Inpatient Management of Patients With Type 2 Diabetes) trial showed a significant reduction in perioperative complications¹⁰ among surgical patients when treated with basal-bolus insulin therapy. Despite these studies and strong recommendations against its use, SSI continues to be widely used in the United States. According to a 2007 survey of 44 US hospitals, 41% of noncritical patients with hyperglycemia were treated with SSI alone.¹⁵ In addition, SSI remains one of the most commonly prescribed insulin regimens in many countries around the world.^16-19 The persistence of SSI use raises questions as to why clinicians continue to use a therapy that has been strongly criticized. Some authors point to convenience and fear of hypoglycemia with a basal-bolus insulin regimen.^20,21 Alternatively, it is possible that SSI usage remains so pervasive because it is effective in a subset of patients. In fact, a 2018 Cochrane review concluded that existing evidence is not sufficiently robust to definitively recommend basal-bolus insulin over SSI for inpatient diabetes management of non–critically ill patients despite existing guidelines.²²

Owing to the ongoing controversy and widespread use of SSI, we designed an exploratory analysis to understand the rationale for such therapy by investigating whether a certain subpopulation of hospitalized patients with T2D may achieve target glycemic control with SSI alone. We hypothesized that noncritical patients with mild hyperglycemia and admission BG <180 mg/dL would do well with SSI alone and may not require intensive treatment with basal-bolus insulin regimens. To address this question, we used electronic health records with individual-level patient data to assess inpatient glycemic control of non–critically ill patients with T2D treated with SSI alone.

Participants

Data from 25,813 adult noncritical inpatients with T2D, with an index admission between June 1, 2010, and June 30, 2018, were obtained through the Emory Healthcare Clinical Data Warehouse infrastructure program. All patients were admitted to Emory Healthcare hospitals, including Emory University Hospital, Emory University Hospital Midtown, and Emory Saint Joseph’s Hospital, in Atlanta, Georgia. Data were extracted for each patient during the index hospitalization, including demographics, anthropometrics, and admission and inpatient laboratory values. Information was collected on daily point-of-care glucose values, hemoglobin A1c (HbA1c), hypoglycemic events, insulin doses, hospital complications, comorbidities, and hospital setting (medical vs surgical admission). International Classification of Diseases, 9th and 10th Revisions (ICD-9/10) codes were used to determine diagnosis of T2D, comorbidities, and complications.

From our initial dataset, we identified 16,366 patients who were treated with SSI during hospitalization. We excluded patients who were admitted to the intensive care unit (ICU) or placed on intravenous insulin, patients with missing admission BG values, and patients with a length of stay less than 1 day. To prevent inclusion of patients presenting in diabetic ketoacidosis or hyperosmolar hyperglycemic syndrome, we excluded patients with an admission BG >500 mg/dL. We then excluded 6,739 patients who received basal insulin within the first 2 days of hospitalization, as well as 943 patients who were treated with noninsulin (oral or injectable) antidiabetic agents. Our final dataset included 8,095 patients (Appendix Figure).

Patients in the SSI cohort included all patients who were treated with short-acting insulin only (regular insulin or rapid-acting [lispro, aspart, glulisine] insulin analogs) during the first 2 days of hospitalization. Patients who remained on only short-acting insulin during the entire hospitalization were defined as continuous SSI patients. Patients who subsequently received basal insulin after day 2 of hospitalization were defined as patients who transitioned to basal. Patients were stratified according to admission BG levels (first BG available on day of admission) and HbA1c (when available during index admission). We compared the baseline characteristics and clinical outcomes of patients who remained on SSI alone throughout the entirety of hospitalization with those of patients who required transition to basal insulin. The mean hospital BG was calculated by taking the average of all BG measurements during the hospital stay. We defined hypoglycemia as a BG <70 mg/dL and severe hypoglycemia as BG <40 mg/dL. Repeated hypoglycemia values were excluded if they occurred within a period of 2 hours.

Outcome Measures

The primary outcome was the percentage of patients with T2D achieving target glycemic control with SSI therapy, defined as mean hospital BG between 70 and 180 mg/dL without hypoglycemia <70 mg/dL during hospital stay. This threshold was determined based on 2019 ADA recommendations targeting hospital BG <180 mg/dL and avoidance of hypoglycemia.²³

Statistical Analysis

Patients were stratified according to continuous SSI versus transitioned to basal treatment. Patients who remained on continuous SSI were further categorized into four categories based on admission BG: <140 mg/dL, 140 to 180 mg/dL, 180 to 250 mg/dL, and ≥250 mg/dL. Clinical characteristics were compared using Wilcoxon rank-sum tests (if continuous) and chi-square tests or Fisher exact tests (if categorical). We then compared the clinical outcomes among continuous SSI patients with different admission BG levels (<140 mg/dL, 140-180 mg/dL, 180-250 mg/dL, and ≥250 mg/dL) and with different HbA1c levels (<7%, 7%-8%, 8%-9%, ≥9%). Within each scenario, logistic regression for the outcome of poor glycemic control, defined as mean hospital BG >180 mg/dL, was performed to evaluate the HbA1c levels and admission BG levels controlling for other factors (age, gender, body mass index [BMI], race, setting [medicine versus surgery] and Charlson Comorbidity Index score). A P value < .05 was regarded as statistically significant. All analyses were performed based on available cases and conducted in SAS version 9.4 (SAS Institute Inc.).

Among 25,813 adult patients with T2D, 8,095 patients (31.4%) were treated with SSI alone during the first 2 days of hospitalization. Of those patients treated with SSI, 6,903 (85%) remained on continuous SSI alone during the entire hospitalization, and 1,192 (15%) were transitioned to basal insulin. The clinical characteristics of these patients on continuous SSI and those who transitioned to basal insulin are shown in Table 1. Patients who transitioned to basal insulin had significantly higher mean (SD) admission BG (191.8 [88.2] mg/dL vs 156.4 [65.4] mg/dL, P < .001) and higher mean (SD) HbA1c (8.1% [2.0%] vs 7.01% [1.5%], P < .001), compared with those who remained on continuous SSI. Patients who transitioned to basal insulin were also younger and more likely to have chronic kidney disease (CKD), but less likely to have congestive heart failure, coronary artery disease, or chronic obstructive pulmonary disease (COPD). The Charlson Comorbidity Index score was significantly higher for patients who transitioned to basal (4.4 [2.5]) than for those who remained on continuous SSI (4.1 [2.5], P < .001). There were no significant differences among sex, BMI, or glomerular filtration rate (GFR) on admission. Of those transitioned to basal insulin, 53% achieved a mean hospitalization BG <180 mg/dL, compared with 82% of those on continuous SSI. The overall rate of hypoglycemia in the continuous SSI group was 8% compared with 18% in those transitioned to basal insulin.

Of the patients who remained on continuous SSI throughout the hospitalization, 3,319 patients (48%) had admission BG <140 mg/dL, 1,671 patients (24%) had admission BG 140 to 180 mg/dL, and 1,913 patients (28%) had admission BG >180 mg/dL. Only 9% of patients who remained on continuous SSI had admission BG ≥250 mg/dL. Patients with admission BG <140 mg/dL were older, had lower BMI and HbA1c, had higher rates of COPD and CKD, and were more likely to be admitted to a surgical service compared with patients with admission BG >140 mg/dL (P < .05 for all; Table 2).

Hospital glycemic control for patients on continuous SSI according to admission BG is displayed in Table 3. Among patients who remained on continuous SSI, 96% of patients with admission BG <140 mg/dL had a mean hospital BG <180 mg/dL; of them, 86% achieved target control without hypoglycemia. Similar rates of target control were achieved in patients with admission BG 140 to 180 mg/dL (83%), in contrast to patients with admission BG ≥250 mg/dL, of whom only 18% achieved target control (P < .001). These findings parallel those seen in patients transitioned to basal insulin. Of patients in the transition group admitted with BG <140 mg/dL and <180 mg/dL, 88.5% and 84.6% had mean hospital BG <180 mg/dL, respectively, while 69.1% and 68.9% had mean BG between 70 and 180 mg/dL without hypoglycemia. The overall frequency of hypoglycemia <70 mg/dL among patients on continuous SSI was 8% and was more common in patients with admission BG <140 mg/dL (10%) compared with patients with higher admission glucose levels (BG 140-180 mg/dL [4%], 180-250 mg/dL [4%], or ≥250 mg/dL [6%], P < .001). There was no difference in rates of severe hypoglycemia <40 mg/dL among groups.

HbA1c data were available for 2,560 of the patients on continuous SSI (Table 3). Mean hospital BG increased significantly with increasing HbA1c values. Patients admitted with HbA1c <7% had lower mean (SD) hospital BG (132.2 [28.2] mg/dL) and were more likely to achieve target glucose control during hospitalization (85%) compared with those with HbA1c 7% to 8% (mean BG, 148.7 [30.8] mg/dL; 80% target control), HbA1c 8% to 9% (mean BG, 169.1 [37.9] mg/dL; 61% target control), or HbA1c ≥9% (mean BG, 194.9 [53.4] mg/dL; 38% target control) (P < .001).

In a logistic regression analysis adjusted for age, gender, BMI, race, setting (medicine vs surgery), and Charlson Comorbidity Index score, the odds of poor glycemic control increased with higher admission BG (admission BG 140-180 mg/dL: odds ratio [OR], 1.8; 95% CI, 1.5-2.2; admission BG 180-250 mg/dL: OR, 3.7; 95% CI, 3.1-4.4; admission BG ≥250 mg/dL: OR, 7.2; 95% CI, 5.8-9.0; reference admission BG <140 mg/dL; Figure). Similarly, the logistic regression analysis showed greater odds of poor in-hospital glycemic control with increasing HbA1c (OR, 6.1; 95% CI, 4.3-8.8 for HbA1c >9% compared with HbA1c <7%).

This large retrospective cohort study examined the effectiveness of SSI for glycemic control in noncritical inpatients with T2D. Our results indicate that SSI is still widely used in our hospital system, with 31.4% of our initial cohort managed with SSI alone. We found that 86% of patients with BG <140 mg/dL and 83% of patients with BG 140 to 180 mg/dL achieved glycemic control without hypoglycemia when managed with SSI alone, compared with 53% of those admitted with BG 180 to 250 mg/dL and only 18% of those with admission BG ≥250 mg/dL. This high success rate of achieving optimal BG control with SSI alone is comparable to that seen with transition to basal insulin and may explain the prevalent use of SSI for the management of patients with T2D and mild to moderate hyperglycemia.

Published clinical guideline recommendations promoting the use of basal-bolus insulin treatment algorithms are based on the results of a few RCTs that compared the efficacy of SSI vs a basal-bolus insulin regimen. These studies reported significantly lower mean daily BG concentration with basal or basal-bolus insulin therapy compared with SSI.^10,11,24 However, it is interesting to note that the mean admission BG of patients treated with SSI in these RCTs ranged from 184 to 225 mg/dL. Patients in these trials were excluded if admission BG was <140 mg/dL.^10,11,24 This is in contrast to our study evaluating real-world data in non–critically ill settings in which we found that 48% of patients treated with SSI had admission BG <140 mg/dL, and nearly 75% had admission BG <180 mg/dL. This suggests that by nature of study design, most RCTs excluded the population of patients who do achieve good glycemic control with SSI and may have contributed to the perception that basal insulin is preferable in all populations.

Our analysis indicates that healthcare professionals should consider admission BG when selecting the type of insulin regimen to manage patients with T2D in the hospital. Our results suggest that SSI may be appropriate for many patients with admission BG <180 mg/dL and should be avoided as monotherapy in patients with admission BG ≥180 mg/dL, as the proportion of patients achieving target control decreased with increasing admission BG. More importantly, if a patient is not controlled with SSI alone, intensification of therapy with the addition of basal insulin is indicated to achieve glycemic control. In addition, we found that the admission HbA1c is an appropriate marker to consider as well, with hospital glycemic control deteriorating with increasing HbA1c values, paralleling the admission BG. The main limitation to widespread use of HbA1c for therapeutic decision-making is access to values at time of patient admission; in our population, only 37% of patients had an HbA1c value available during the index hospitalization.

Previous publications have reported that hypoglycemia carries significant safety concerns, especially among a hospitalized population.^25-27 As such, we included hypoglycemia as an important metric in our definition of target glycemic control rather than simply using mean hospital BG or number of hyperglycemic events to define treatment effectiveness. We did find a higher rate of hypoglycemia in patients with moderate admission BG treated with SSI compared with those with higher admission BG; however, few patients overall experienced clinically significant (<54 mg/dL) or severe (<40 mg/dL) hypoglycemia.

In our population, only 15% of patients started on SSI received additional basal insulin during hospitalization. This finding is similar to data reported in the Rabbit 2 trial, in which 14% of patients failed SSI alone, with a higher failure rate among those with higher BG on admission.¹⁰ Given the observational nature of this study, we cannot definitively state why certain patients in our population required additional basal insulin, but we can hypothesize that these patients admitted with BG ≥180 mg/dL had higher treatment failure rates and greater rates of hyperglycemia, therefore receiving intensified insulin therapy as clinically indicated at the discretion of the treating physician. Patients who transitioned from SSI to basal insulin had significantly higher admission BG and HbA1c compared with patients who remained on SSI alone. We noted that the rates of hypoglycemia were higher in the group that transitioned to basal (18% vs 8%) and similar to rates reported in previous RCTs.^11,24

This observational study takes advantage of a large, diverse study population and a combination of medicine and surgery patients in a real-world setting. We acknowledge several limitations in our study. Our primary data were observational in nature, and as such, some baseline patient characteristics were notably different between groups, suggesting selection bias for treatment allocation to SSI. We do not know which patients were managed by primary teams compared with specialized diabetes consult services, which may also influence treatment regimens. We did not have access to information about patients’ at-home diabetes medication regimens or duration of diabetes, both of which have been shown in prior publications to affect an individual’s overall hospital glycemic control. Data on HbA1c values were available for only approximately one-third of patients. In addition, our study did not include patients without a history of diabetes who developed stress-induced hyperglycemia, a population that may benefit from conservative therapy such as SSI.²⁸ A diagnosis of CKD was defined based on ICD 9/10 codes and not on admission estimated GFR. More specific data regarding stage of CKD or changes in renal function over the duration of hospitalization are not available, which could influence insulin prescribing practice. In addition, we defined the basal group as patients prescribed any form of basal insulin (NPH, glargine, detemir or degludec), and we do not have information on the use of prandial versus correction doses of rapid-acting insulin in the basal insulin–treated group.

In conclusion, our observational study indicates that the use of SSI results in appropriate target glycemic control for most noncritical medicine and surgery patients with admission BG <180 mg/dL. In agreement with previous RCTs, our study confirms that SSI as monotherapy is frequently inadequate in patients with significant hyperglycemia >180 mg/dL.^10,11,24,29 We propose that an individualized approach to inpatient glycemic management is imperative, and cautious use of SSI may be a viable option for certain patients with mild hyperglycemia and admission BG <180 mg/dL. Further observational and randomized studies are needed to confirm the efficacy of SSI therapy in T2D patients with mild hyperglycemia. By identifying which subset of patients can be safely managed with SSI alone, we can better understand which patients will require escalation of therapy with intensive glucose management.

Sepsis is the leading cause of in-hospital mortality in the United States.¹ Sepsis is present on admission in 85% of cases, and each hour delay in antibiotic treatment is associated with 4% to 7% increased odds of mortality.^2,3 Prompt identification and treatment of sepsis is essential for reducing morbidity and mortality, but identifying sepsis during triage is challenging.²

Risk stratification scores that rely solely on data readily available at the bedside have been developed to quickly identify those at greatest risk of poor outcomes from sepsis in real time. The quick Sequential Organ Failure Assessment (qSOFA) score, the National Early Warning System (NEWS2), and the Shock Index are easy-to-calculate measures that use routinely collected clinical data that are not subject to laboratory delay. These scores can be incorporated into electronic health record (EHR)-based alerts and can be calculated longitudinally to track the risk of poor outcomes over time. qSOFA was developed to quantify patient risk at bedside in non-intensive care unit (ICU) settings, but there is no consensus about its ability to predict adverse outcomes such as mortality and ICU admission.^4-6 The United Kingdom’s National Health Service uses NEWS2 to identify patients at risk for sepsis.⁷ NEWS has been shown to have similar or better sensitivity in identifying poorer outcomes in sepsis patients compared with systemic inflammatory response syndrome (SIRS) criteria and qSOFA.^4,8-11 However, since the latest update of NEWS2 in 2017, there has been little study of its predictive ability. The Shock Index is a simple bedside score (heart rate divided by systolic blood pressure) that was developed to detect changes in cardiovascular performance before systemic shock onset. Although it was not developed for infection and has not been regularly applied in the sepsis literature, the Shock Index might be useful for identifying patients at increased risk of poor outcomes. Patients with higher and sustained Shock Index scores are more likely to experience morbidity, such as hyperlactatemia, vasopressor use, and organ failure, and also have an increased risk of mortality.^12-14

Although the predictive abilities of these bedside risk stratification scores have been assessed individually using standard binary cut-points, the comparative performance of qSOFA, the Shock Index, and NEWS2 has not been evaluated in patients presenting to an emergency department (ED) with suspected sepsis. Our objective was to provide a head-to-head comparison of the test characteristics of qSOFA, the Shock Index, and NEWS2 calculated at ED triage for predicting in-hospital mortality and ED-to-ICU admission in patients with suspected sepsis to help health systems and providers select screening measures.

Design and Setting

We conducted a retrospective cohort study of ED patients who presented with suspected sepsis to the University of California San Francisco (UCSF) Helen Diller Medical Center at Parnassus Heights between June 1, 2012, and December 31, 2018. Our institution is a 785-bed academic teaching hospital with approximately 30,000 ED encounters per year. The study was approved with a waiver of informed consent by the UCSF Human Research Protection Program.

Participants

We use an Epic-based EHR platform (Epic 2017, Epic Systems Corporation) for clinical care, which was implemented on June 1, 2012. All data elements were obtained from Clarity, the relational database that stores Epic’s inpatient data. The study included encounters for patients age ≥18 years who had blood cultures ordered within 24 hours of ED presentation and administration of intravenous antibiotics within 24 hours. Repeat encounters were treated independently in our analysis.

Outcomes and Measures

We compared the ability of qSOFA, the Shock Index, and NEWS2 to predict in-hospital mortality and admission to the ICU from the ED (ED-to-ICU admission). We used the most abnormal vital signs and clinical assessments gathered within 30 minutes of ED presentation to identify patients who were qSOFA-positive, Shock Index-positive, and NEWS2-positive based on standard cut-points of risk. Data elements used to calculate qSOFA, Shock Index, and NEWS2 included blood pressure, heart rate, respiratory rate, Glasgow Coma Scale (GCS) score, oxygen saturation, requirement for supplemental oxygen, and temperature (Table 1). Patients were considered positive if they had a qSOFAscore ≥2, Shock Index of >0.7, and NEWS2 ≥5 based on triage vital signs.^7,15,16 We considered patients to have altered mental status, a criterion used for NEWS2, if they had a GCS score <15 instead of using the “alert, verbal, confusion, pain, unresponsive” scale, which is not captured in our EHR, a method that has been used in earlier studies.^17,18 Missing assessments were considered normal. Although our primary analysis focused on the scores calculated within 30 minutes of ED presentation, we performed a sensitivity analysis examining scores calculated within 1 hour of ED presentation in the event of a delay in gathering triage vital sign data.

We compared demographic and clinical characteristics of patients who were positive for qSOFA, the Shock Index, and NEWS2. Demographic data were extracted from the EHR and included primary language, age, sex, and insurance status. All International Classification of Diseases (ICD)-9/10 diagnosis codes were pulled from Clarity billing tables. We used the Elixhauser comorbidity groupings¹⁹ of ICD-9/10 codes present on admission to identify preexisting comorbidities and underlying organ dysfunction. To estimate burden of comorbid illnesses, we calculated the validated van Walraven comorbidity index,²⁰ which provides an estimated risk of in-hospital death based on documented Elixhauser comorbidities. Admission level of care (acute, stepdown, or intensive care) was collected for inpatient admissions to assess initial illness severity.²¹ We also evaluated discharge disposition and in-hospital mortality. Index blood culture results were collected, and dates and timestamps of mechanical ventilation, fluid, vasopressor, and antibiotic administration were obtained for the duration of the encounter.

UCSF uses an automated, real-time, algorithm-based severe sepsis alert that is triggered when a patient meets ≥2 SIRS criteria and again when the patient meets severe sepsis or septic shock criteria (ie, ≥2 SIRS criteria in addition to end-organ dysfunction and/or fluid nonresponsive hypotension). This sepsis screening alert was in use for the duration of our study.²²

Statistical Analysis

We performed a subgroup analysis among those who were diagnosed with sepsis, according to the 2016 Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) criteria. Sepsis is defined as a change in Sequential Organ Failure Assessment (SOFA) score of ≥2 points within the first 48 hours.²³ Additionally, patients meeting Sepsis-3 criteria needed to (1) receive ≥4 days of sequential antibiotic therapy or experience death or discharge to hospice before 4 days of antibiotic therapy or (2) have a validated sepsis discharge billing code. These parameters were added to increase the specificity of our sample.²⁴

All statistical analyses were conducted using Stata 14 (StataCorp). We summarized differences in demographic and clinical characteristics among the populations meeting each severity score but elected not to conduct hypothesis testing because patients could be positive for one or more scores. We calculated sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for each score to predict in-hospital mortality and ED-to-ICU admission. To allow comparison with other studies, we also created a composite outcome of either in-hospital mortality or ED-to-ICU admission. To assess score discrimination to predict in-hospital mortality and ED-to-ICU admission, we calculated the area under the receiver operating characteristic curve (AUROC) along with asymptotic normal 95% CI using the “roctab” command considering a binary cut-point, as well as the full range of scores measured in the cohort. The AUROC range from 0.50 to 1.00 and a score in the 0.70 to 0.80 range can be considered fair.²⁵ We assessed significant differences between severity score AUROCs using the DeLong method²⁶ implemented through Stata 14’s “roccomp” command. As a sensitivity analysis, we explored whether the standard cut-points for qSOFA, the Shock Index, and NEWS2 provided the highest AUROC in our population by calculating test characteristics for several score cut-points.

Within our sample 23,837 ED patients had blood cultures ordered within 24 hours of ED presentation and were considered to have suspected sepsis. The mean age of the cohort was 60.8 years, and 1,612 (6.8%) had positive blood cultures. A total of 12,928 patients (54.2%) were found to have sepsis. We documented 1,427 in-hospital deaths (6.0%) and 3,149 (13.2%) ED-to-ICU admissions. At ED triage 1,921 (8.1%) were qSOFA-positive, 4,273 (17.9%) were Shock Index-positive, and 11,832 (49.6%) were NEWS2-positive. At ED triage, blood pressure, heart rate, respiratory rate, and oxygen saturated were documented in >99% of patients, 93.5% had temperature documented, and 28.5% had GCS recorded. If the window of assessment was widened to 1 hour, GCS was only documented among 44.2% of those with suspected sepsis.

Demographic Characteristics and Clinical Course

We identified significant differences when comparing demographic and clinical characteristics among patients who scored positive for the three severity measures at triage (Table 2). Although no hypothesis testing was conducted because patients could meet one or more scores, qSOFA-positive patients were older (median 70, 66, and 64 years, respectively), more likely to have Medicare as the primary payor (67.6% vs 59.7% vs 56.6%), to have chronic renal failure (26.1%, 23.1%, and 23.3%, respectively), to have a greater degree of underlying comorbidities based on the van Walraven Comorbidity Index (median 15, 12, and 11, respectively), and to be admitted to the ICU from the ED (48.1%, 36.3%, and 21.0%, respectively) compared with those positive for the Shock Index or NEWS2.

qSOFA-positive patients received antibiotics more quickly than those who were Shock Index-positive or NEWS2-positive (median 1.5, 1.8, and 2.8 hours after admission, respectively). In addition, those who were qSOFA-positive were more likely to have a positive blood culture (10.9%, 9.4%, and 8.5%, respectively) and to receive an EHR-based diagnosis of sepsis (77.0%, 69.6%, and 60.9%, respectively) than those who were Shock Index- or NEWS2-positive. Those who were qSOFA-positive also were more likely to be mechanically ventilated during their hospital stay (25.4%, 19.2%, and 10.8%, respectively) and to receive vasopressors (33.5%, 22.5%, and 12.2%, respectively). In-hospital mortality also was more common among those who were qSOFA-positive at triage (23.4%, 15.3%, and 9.2%, respectively).

Because both qSOFA and NEWS2 incorporate GCS, we explored baseline characteristics of patients with GCS documented at triage (n = 6,794). These patients were older (median age 63 and 61 years, P < .0001), more likely to be male (54.9% and 53.4%, P = .0031), more likely to have renal failure (22.8% and 20.1%, P < .0001), more likely to have liver disease (14.2% and 12.8%, P = .006), had a higher van Walraven comorbidity score on presentation (median 10 and 8, P < .0001), and were more likely to go directly to the ICU from the ED (20.2% and 10.6%, P < .0001). However, among the 6,397 GCS scores documented at triage, only 1,579 (24.7%) were abnormal.

Test Characteristics of qSOFA, Shock Index, and NEWS2 for Predicting In-hospital Mortality and ED-to-ICU Admission

Among 23,837 patients with suspected sepsis, NEWS2 had the highest sensitivity for predicting in-hospital mortality (76.0%; 95% CI, 73.7%-78.2%) and ED-to-ICU admission (78.9%; 95% CI, 77.5%-80.4%) but had the lowest specificity for in-hospital mortality (52.0%; 95% CI, 51.4%-52.7%) and for ED-to-ICU admission (54.8%; 95% CI, 54.1%-55.5%) (Table 3). qSOFA had the lowest sensitivity for in-hospital mortality (31.5%; 95% CI, 29.1%-33.9%) and ED-to-ICU admission (29.3%; 95% CI, 27.7%-30.9%) but the highest specificity for in-hospital mortality (93.4%; 95% CI, 93.1%-93.8%) and ED-to-ICU admission (95.2%; 95% CI, 94.9%-95.5%). The Shock Index had a sensitivity that fell between qSOFA and NEWS2 for in-hospital mortality (45.8%; 95% CI, 43.2%-48.5%) and ED-to-ICU admission (49.2%; 95% CI, 47.5%-51.0%). The specificity of the Shock Index also was between qSOFA and NEWS2 for in-hospital mortality (83.9%; 95% CI, 83.4%-84.3%) and ED-to-ICU admission (86.8%; 95% CI, 86.4%-87.3%). All three scores exhibited relatively low PPV, ranging from 9.2% to 23.4% for in-hospital mortality and 21.0% to 48.0% for ED-to-ICU triage. Conversely, all three scores exhibited relatively high NPV, ranging from 95.5% to 97.1% for in-hospital mortality and 89.8% to 94.5% for ED-to-ICU triage. The patterns in sensitivity and specificity for in-hospital mortality and ED-to-ICU admission were similar among the 12,928 patients who received an EHR-based sepsis diagnosis with the tests generally demonstrating lower specificities, higher PPVs, and lower NPVs (Table 3).

When considering a binary cutoff, the Shock Index exhibited the highest AUROC for in-hospital mortality (0.648; 95% CI, 0.635-0.662) and had a significantly higher AUROC than qSOFA (AUROC, 0.625; 95% CI, 0.612-0.637; P = .0005), but there was no difference compared with NEWS2 (AUROC, 0.640; 95% CI, 0.628-0.652; P = .2112). NEWS2 had a significantly higher AUROC than qSOFA for predicting in-hospital mortality (P = .0227). The Shock Index also exhibited the highest AUROC for ED-to-ICU admission (0.680; 95% CI, 0.617-0.689), which was significantly higher than the AUROC for qSOFA (P < .0001) and NEWS2 (P = 0.0151). NEWS2 had a significantly higher AUROC than qSOFA for predicting ED-to-ICU admission (P < .0001). Similar findings were seen in patients found to have sepsis. When considering the range of possible scores measured in our cohort, qSOFA and NEWS2 exhibited higher AUROCs for in-hospital mortality and ED-to-ICU admission than the Shock Index among patients with suspected infection and the subgroup with a sepsis diagnosis (Figure).

In this retrospective cohort study of 23,837 patients who presented to the ED with suspected sepsis, the standard qSOFA threshold was met least frequently, followed by the Shock Index and NEWS2. NEWS2 had the highest sensitivity but the lowest specificity for predicting in-hospital mortality and ED-to-ICU admission, making it a challenging bedside risk stratification scale for identifying patients at risk of poor clinical outcomes. When comparing predictive performance among the three scales, qSOFA had the highest specificity and the Shock Index had the highest AUROC for in-hospital mortality and ED-to-ICU admission in this cohort of patients with suspected sepsis. These trends in sensitivity, specificity, and AUROC were consistent among those who met EHR criteria for a sepsis diagnosis. In the analysis of the three scoring systems using all available cut-points, qSOFA and NEWS2 had the highest AUROCs, followed by the Shock Index.

Considering the rapid progression from organ dysfunction to death in sepsis patients, as well as the difficulty establishing a sepsis diagnosis at triage,²³ providers must quickly identify patients at increased risk of poor outcomes when they present to the ED. Sepsis alerts often are built using SIRS criteria,²⁷ including the one used for sepsis surveillance at UCSF since 2012,²² but the white blood cell count criterion is subject to a laboratory lag and could lead to a delay in identification. Implementation of a point-of-care bedside score alert that uses readily available clinical data could allow providers to identify patients at greatest risk of poor outcomes immediately at ED presentation and triage, which motivated us to explore the predictive performance of qSOFA, the Shock Index, and NEWS2.

Our study is the first to provide a head-to-head comparison of the predictive performance of qSOFA, the Shock Index, and NEWS2, three easy-to-calculate bedside risk scores that use EHR data collected among patients with suspected sepsis. The Sepsis-3 guidelines recommend qSOFA to quickly identify non-ICU patients at greatest risk of poor outcomes because the measure exhibited predictive performance similar to the more extensive SOFA score outside the ICU.^16,23 Although some studies have confirmed qSOFA’s high predictive performance,^28-31 our test characteristics and AUROC findings are in line with other published analyses.^4,6,10,17 The UK National Health Service is using NEWS2 to screen for patients at risk of poor outcomes from sepsis. Several analyses that assessed the predictive ability of NEWS have reported estimates in line with our findings.^4,10,32 The Shock Index was introduced in 1967 and provided a metric to evaluate hemodynamic stability based on heart rate and systolic blood pressure.³³ The Shock Index has been studied in several contexts, including sepsis,³⁴ and studies show that a sustained Shock Index is associated with increased odds of vasopressor administration, higher prevalence of hyperlactatemia, and increased risk of poor outcomes in the ICU.^13,14

For our study, we were particularly interested in exploring how the Shock Index would compare with more frequently used severity scores such as qSOFA and NEWS2 among patients with suspected sepsis, given the simplicity of its calculation and the easy availability of required data. In our cohort of 23,837 patients, only 159 people had missing blood pressure and only 71 had omitted heart rate. In contrast, both qSOFA and NEWS2 include an assessment of level of consciousness that can be subject to variability in assessment methods and EHR documentation across institutions.¹¹ In our cohort, GCS within 30 minutes of ED presentation was missing in 72 patients, which could have led to incomplete calculation of qSOFA and NEWS2 if a missing value was not actually within normal limits.

Several investigations relate qSOFA to NEWS but few compare qSOFA with the newer NEWS2, and even fewer evaluate the Shock Index with any of these scores.^{10,11,18,29,35-37} In general, studies have shown that NEWS exhibits a higher AUROC for predicting mortality, sepsis with organ dysfunction, and ICU admission, often as a composite outcome.^{4,11,18,37,38} A handful of studies compare the Shock Index to SIRS; however, little has been done to compare the Shock Index to qSOFA or NEWS2, scores that have been used specifically for sepsis and might be more predictive of poor outcomes than SIRS.³³ In our study, the Shock Index had a higher AUROC than either qSOFA or NEWS2 for predicting in-hospital mortality and ED-to-ICU admission measured as separate outcomes and as a composite outcome using standard cut-points for these scores.

When selecting a severity score to apply in an institution, it is important to carefully evaluate the score’s test characteristics, in addition to considering the availability of reliable data. Tests with high sensitivity and NPV for the population being studied can be useful to rule out disease or risk of poor outcome, while tests with high specificity and PPV can be useful to rule in disease or risk of poor outcome.³⁹ When considering specificity, qSOFA’s performance was superior to the Shock Index and NEWS2 in our study, but a small percentage of the population was identified using a cut-point of qSOFA ≥2. If we used qSOFA and applied this standard cut-point at our institution, we could be confident that those identified were at increased risk, but we would miss a significant number of patients who would experience a poor outcome. When considering sensitivity, performance of NEWS2 was superior to qSOFA and the Shock Index in our study, but one-half of the population was identified using a cut-point of NEWS2 ≥5. If we were to apply this standard NEWS2 cut-point at our institution, we would assume that one-half of our population was at risk, which might drive resource use towards patients who will not experience a poor outcome. Although none of the scores exhibited a robust AUROC measure, the Shock Index had the highest AUROC for in-hospital mortality and ED-to-ICU admission when using the standard binary cut-point, and its sensitivity and specificity is between that of qSOFA and NEWS2, potentially making it a score to use in settings where qSOFA and NEWS2 score components, such as altered mentation, are not reliably collected. Finally, our sensitivity analysis varying the binary cut-point of each score within our population demonstrated that the standard cut-points might not be as useful within a specific population and might need to be tailored for implementation, balancing sensitivity, specificity, PPV, and NPV to meet local priorities and ICU capacity.

Our study has limitations. It is a single-center, retrospective analysis, factors that could reduce generalizability. However, it does include a large and diverse patient population spanning several years. Missing GCS data could have affected the predictive ability of qSOFA and NEWS2 in our cohort. We could not reliably perform imputation of GCS because of the high missingness and therefore we assumed missing was normal, as was done in the Sepsis-3 derivation studies.¹⁶ Previous studies have attempted to impute GCS and have not observed improved performance of qSOFA to predict mortality.⁴⁰ Because manually collected variables such as GCS are less reliably documented in the EHR, there might be limitations in their use for triage risk scores.

Although the current analysis focused on the predictive performance of qSOFA, the Shock Index, and NEWS2 at triage, performance of these scores could affect the ED team’s treatment decisions before handoff to the hospitalist team and the expected level of care the patient will receive after in-patient admission. These tests also have the advantage of being easy to calculate at the bedside over time, which could provide an objective assessment of longitudinal predicted prognosis. Future work should assess the longitudinal performance of each of these scores among those with suspected sepsis and to determine the impact using these scores would have on clinical and resource utilization outcomes.

Local priorities should drive selection of a screening tool, balancing sensitivity, specificity, PPV, and NPV to achieve the institution’s goals. qSOFA, Shock Index, and NEWS2 are risk stratification tools that can be easily implemented at ED triage using data available at the bedside. Although none of these scores performed strongly when comparing AUROCs, qSOFA was highly specific for identifying patients with poor outcomes, and NEWS2 was the most sensitive for ruling out those at high risk among patients with suspected sepsis. The Shock Index exhibited a sensitivity and specificity that fell between qSOFA and NEWS2 and also might be considered to identify those at increased risk, given its ease of implementation, particularly in settings where altered mentation is unreliably or inconsistently documented.

The authors thank the UCSF Division of Hospital Medicine Data Core for their assistance with data acquisition.

Sepsis is the leading cause of in-hospital mortality in the United States.¹ Sepsis is present on admission in 85% of cases, and each hour delay in antibiotic treatment is associated with 4% to 7% increased odds of mortality.^2,3 Prompt identification and treatment of sepsis is essential for reducing morbidity and mortality, but identifying sepsis during triage is challenging.²

Risk stratification scores that rely solely on data readily available at the bedside have been developed to quickly identify those at greatest risk of poor outcomes from sepsis in real time. The quick Sequential Organ Failure Assessment (qSOFA) score, the National Early Warning System (NEWS2), and the Shock Index are easy-to-calculate measures that use routinely collected clinical data that are not subject to laboratory delay. These scores can be incorporated into electronic health record (EHR)-based alerts and can be calculated longitudinally to track the risk of poor outcomes over time. qSOFA was developed to quantify patient risk at bedside in non-intensive care unit (ICU) settings, but there is no consensus about its ability to predict adverse outcomes such as mortality and ICU admission.^4-6 The United Kingdom’s National Health Service uses NEWS2 to identify patients at risk for sepsis.⁷ NEWS has been shown to have similar or better sensitivity in identifying poorer outcomes in sepsis patients compared with systemic inflammatory response syndrome (SIRS) criteria and qSOFA.^4,8-11 However, since the latest update of NEWS2 in 2017, there has been little study of its predictive ability. The Shock Index is a simple bedside score (heart rate divided by systolic blood pressure) that was developed to detect changes in cardiovascular performance before systemic shock onset. Although it was not developed for infection and has not been regularly applied in the sepsis literature, the Shock Index might be useful for identifying patients at increased risk of poor outcomes. Patients with higher and sustained Shock Index scores are more likely to experience morbidity, such as hyperlactatemia, vasopressor use, and organ failure, and also have an increased risk of mortality.^12-14

Although the predictive abilities of these bedside risk stratification scores have been assessed individually using standard binary cut-points, the comparative performance of qSOFA, the Shock Index, and NEWS2 has not been evaluated in patients presenting to an emergency department (ED) with suspected sepsis. Our objective was to provide a head-to-head comparison of the test characteristics of qSOFA, the Shock Index, and NEWS2 calculated at ED triage for predicting in-hospital mortality and ED-to-ICU admission in patients with suspected sepsis to help health systems and providers select screening measures.

Design and Setting

We conducted a retrospective cohort study of ED patients who presented with suspected sepsis to the University of California San Francisco (UCSF) Helen Diller Medical Center at Parnassus Heights between June 1, 2012, and December 31, 2018. Our institution is a 785-bed academic teaching hospital with approximately 30,000 ED encounters per year. The study was approved with a waiver of informed consent by the UCSF Human Research Protection Program.

Participants

We use an Epic-based EHR platform (Epic 2017, Epic Systems Corporation) for clinical care, which was implemented on June 1, 2012. All data elements were obtained from Clarity, the relational database that stores Epic’s inpatient data. The study included encounters for patients age ≥18 years who had blood cultures ordered within 24 hours of ED presentation and administration of intravenous antibiotics within 24 hours. Repeat encounters were treated independently in our analysis.

Outcomes and Measures

We compared the ability of qSOFA, the Shock Index, and NEWS2 to predict in-hospital mortality and admission to the ICU from the ED (ED-to-ICU admission). We used the most abnormal vital signs and clinical assessments gathered within 30 minutes of ED presentation to identify patients who were qSOFA-positive, Shock Index-positive, and NEWS2-positive based on standard cut-points of risk. Data elements used to calculate qSOFA, Shock Index, and NEWS2 included blood pressure, heart rate, respiratory rate, Glasgow Coma Scale (GCS) score, oxygen saturation, requirement for supplemental oxygen, and temperature (Table 1). Patients were considered positive if they had a qSOFAscore ≥2, Shock Index of >0.7, and NEWS2 ≥5 based on triage vital signs.^7,15,16 We considered patients to have altered mental status, a criterion used for NEWS2, if they had a GCS score <15 instead of using the “alert, verbal, confusion, pain, unresponsive” scale, which is not captured in our EHR, a method that has been used in earlier studies.^17,18 Missing assessments were considered normal. Although our primary analysis focused on the scores calculated within 30 minutes of ED presentation, we performed a sensitivity analysis examining scores calculated within 1 hour of ED presentation in the event of a delay in gathering triage vital sign data.

We compared demographic and clinical characteristics of patients who were positive for qSOFA, the Shock Index, and NEWS2. Demographic data were extracted from the EHR and included primary language, age, sex, and insurance status. All International Classification of Diseases (ICD)-9/10 diagnosis codes were pulled from Clarity billing tables. We used the Elixhauser comorbidity groupings¹⁹ of ICD-9/10 codes present on admission to identify preexisting comorbidities and underlying organ dysfunction. To estimate burden of comorbid illnesses, we calculated the validated van Walraven comorbidity index,²⁰ which provides an estimated risk of in-hospital death based on documented Elixhauser comorbidities. Admission level of care (acute, stepdown, or intensive care) was collected for inpatient admissions to assess initial illness severity.²¹ We also evaluated discharge disposition and in-hospital mortality. Index blood culture results were collected, and dates and timestamps of mechanical ventilation, fluid, vasopressor, and antibiotic administration were obtained for the duration of the encounter.

UCSF uses an automated, real-time, algorithm-based severe sepsis alert that is triggered when a patient meets ≥2 SIRS criteria and again when the patient meets severe sepsis or septic shock criteria (ie, ≥2 SIRS criteria in addition to end-organ dysfunction and/or fluid nonresponsive hypotension). This sepsis screening alert was in use for the duration of our study.²²

Statistical Analysis

We performed a subgroup analysis among those who were diagnosed with sepsis, according to the 2016 Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) criteria. Sepsis is defined as a change in Sequential Organ Failure Assessment (SOFA) score of ≥2 points within the first 48 hours.²³ Additionally, patients meeting Sepsis-3 criteria needed to (1) receive ≥4 days of sequential antibiotic therapy or experience death or discharge to hospice before 4 days of antibiotic therapy or (2) have a validated sepsis discharge billing code. These parameters were added to increase the specificity of our sample.²⁴

All statistical analyses were conducted using Stata 14 (StataCorp). We summarized differences in demographic and clinical characteristics among the populations meeting each severity score but elected not to conduct hypothesis testing because patients could be positive for one or more scores. We calculated sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for each score to predict in-hospital mortality and ED-to-ICU admission. To allow comparison with other studies, we also created a composite outcome of either in-hospital mortality or ED-to-ICU admission. To assess score discrimination to predict in-hospital mortality and ED-to-ICU admission, we calculated the area under the receiver operating characteristic curve (AUROC) along with asymptotic normal 95% CI using the “roctab” command considering a binary cut-point, as well as the full range of scores measured in the cohort. The AUROC range from 0.50 to 1.00 and a score in the 0.70 to 0.80 range can be considered fair.²⁵ We assessed significant differences between severity score AUROCs using the DeLong method²⁶ implemented through Stata 14’s “roccomp” command. As a sensitivity analysis, we explored whether the standard cut-points for qSOFA, the Shock Index, and NEWS2 provided the highest AUROC in our population by calculating test characteristics for several score cut-points.

Within our sample 23,837 ED patients had blood cultures ordered within 24 hours of ED presentation and were considered to have suspected sepsis. The mean age of the cohort was 60.8 years, and 1,612 (6.8%) had positive blood cultures. A total of 12,928 patients (54.2%) were found to have sepsis. We documented 1,427 in-hospital deaths (6.0%) and 3,149 (13.2%) ED-to-ICU admissions. At ED triage 1,921 (8.1%) were qSOFA-positive, 4,273 (17.9%) were Shock Index-positive, and 11,832 (49.6%) were NEWS2-positive. At ED triage, blood pressure, heart rate, respiratory rate, and oxygen saturated were documented in >99% of patients, 93.5% had temperature documented, and 28.5% had GCS recorded. If the window of assessment was widened to 1 hour, GCS was only documented among 44.2% of those with suspected sepsis.

Demographic Characteristics and Clinical Course

We identified significant differences when comparing demographic and clinical characteristics among patients who scored positive for the three severity measures at triage (Table 2). Although no hypothesis testing was conducted because patients could meet one or more scores, qSOFA-positive patients were older (median 70, 66, and 64 years, respectively), more likely to have Medicare as the primary payor (67.6% vs 59.7% vs 56.6%), to have chronic renal failure (26.1%, 23.1%, and 23.3%, respectively), to have a greater degree of underlying comorbidities based on the van Walraven Comorbidity Index (median 15, 12, and 11, respectively), and to be admitted to the ICU from the ED (48.1%, 36.3%, and 21.0%, respectively) compared with those positive for the Shock Index or NEWS2.

qSOFA-positive patients received antibiotics more quickly than those who were Shock Index-positive or NEWS2-positive (median 1.5, 1.8, and 2.8 hours after admission, respectively). In addition, those who were qSOFA-positive were more likely to have a positive blood culture (10.9%, 9.4%, and 8.5%, respectively) and to receive an EHR-based diagnosis of sepsis (77.0%, 69.6%, and 60.9%, respectively) than those who were Shock Index- or NEWS2-positive. Those who were qSOFA-positive also were more likely to be mechanically ventilated during their hospital stay (25.4%, 19.2%, and 10.8%, respectively) and to receive vasopressors (33.5%, 22.5%, and 12.2%, respectively). In-hospital mortality also was more common among those who were qSOFA-positive at triage (23.4%, 15.3%, and 9.2%, respectively).

Because both qSOFA and NEWS2 incorporate GCS, we explored baseline characteristics of patients with GCS documented at triage (n = 6,794). These patients were older (median age 63 and 61 years, P < .0001), more likely to be male (54.9% and 53.4%, P = .0031), more likely to have renal failure (22.8% and 20.1%, P < .0001), more likely to have liver disease (14.2% and 12.8%, P = .006), had a higher van Walraven comorbidity score on presentation (median 10 and 8, P < .0001), and were more likely to go directly to the ICU from the ED (20.2% and 10.6%, P < .0001). However, among the 6,397 GCS scores documented at triage, only 1,579 (24.7%) were abnormal.

Test Characteristics of qSOFA, Shock Index, and NEWS2 for Predicting In-hospital Mortality and ED-to-ICU Admission

Among 23,837 patients with suspected sepsis, NEWS2 had the highest sensitivity for predicting in-hospital mortality (76.0%; 95% CI, 73.7%-78.2%) and ED-to-ICU admission (78.9%; 95% CI, 77.5%-80.4%) but had the lowest specificity for in-hospital mortality (52.0%; 95% CI, 51.4%-52.7%) and for ED-to-ICU admission (54.8%; 95% CI, 54.1%-55.5%) (Table 3). qSOFA had the lowest sensitivity for in-hospital mortality (31.5%; 95% CI, 29.1%-33.9%) and ED-to-ICU admission (29.3%; 95% CI, 27.7%-30.9%) but the highest specificity for in-hospital mortality (93.4%; 95% CI, 93.1%-93.8%) and ED-to-ICU admission (95.2%; 95% CI, 94.9%-95.5%). The Shock Index had a sensitivity that fell between qSOFA and NEWS2 for in-hospital mortality (45.8%; 95% CI, 43.2%-48.5%) and ED-to-ICU admission (49.2%; 95% CI, 47.5%-51.0%). The specificity of the Shock Index also was between qSOFA and NEWS2 for in-hospital mortality (83.9%; 95% CI, 83.4%-84.3%) and ED-to-ICU admission (86.8%; 95% CI, 86.4%-87.3%). All three scores exhibited relatively low PPV, ranging from 9.2% to 23.4% for in-hospital mortality and 21.0% to 48.0% for ED-to-ICU triage. Conversely, all three scores exhibited relatively high NPV, ranging from 95.5% to 97.1% for in-hospital mortality and 89.8% to 94.5% for ED-to-ICU triage. The patterns in sensitivity and specificity for in-hospital mortality and ED-to-ICU admission were similar among the 12,928 patients who received an EHR-based sepsis diagnosis with the tests generally demonstrating lower specificities, higher PPVs, and lower NPVs (Table 3).

When considering a binary cutoff, the Shock Index exhibited the highest AUROC for in-hospital mortality (0.648; 95% CI, 0.635-0.662) and had a significantly higher AUROC than qSOFA (AUROC, 0.625; 95% CI, 0.612-0.637; P = .0005), but there was no difference compared with NEWS2 (AUROC, 0.640; 95% CI, 0.628-0.652; P = .2112). NEWS2 had a significantly higher AUROC than qSOFA for predicting in-hospital mortality (P = .0227). The Shock Index also exhibited the highest AUROC for ED-to-ICU admission (0.680; 95% CI, 0.617-0.689), which was significantly higher than the AUROC for qSOFA (P < .0001) and NEWS2 (P = 0.0151). NEWS2 had a significantly higher AUROC than qSOFA for predicting ED-to-ICU admission (P < .0001). Similar findings were seen in patients found to have sepsis. When considering the range of possible scores measured in our cohort, qSOFA and NEWS2 exhibited higher AUROCs for in-hospital mortality and ED-to-ICU admission than the Shock Index among patients with suspected infection and the subgroup with a sepsis diagnosis (Figure).

In this retrospective cohort study of 23,837 patients who presented to the ED with suspected sepsis, the standard qSOFA threshold was met least frequently, followed by the Shock Index and NEWS2. NEWS2 had the highest sensitivity but the lowest specificity for predicting in-hospital mortality and ED-to-ICU admission, making it a challenging bedside risk stratification scale for identifying patients at risk of poor clinical outcomes. When comparing predictive performance among the three scales, qSOFA had the highest specificity and the Shock Index had the highest AUROC for in-hospital mortality and ED-to-ICU admission in this cohort of patients with suspected sepsis. These trends in sensitivity, specificity, and AUROC were consistent among those who met EHR criteria for a sepsis diagnosis. In the analysis of the three scoring systems using all available cut-points, qSOFA and NEWS2 had the highest AUROCs, followed by the Shock Index.

Considering the rapid progression from organ dysfunction to death in sepsis patients, as well as the difficulty establishing a sepsis diagnosis at triage,²³ providers must quickly identify patients at increased risk of poor outcomes when they present to the ED. Sepsis alerts often are built using SIRS criteria,²⁷ including the one used for sepsis surveillance at UCSF since 2012,²² but the white blood cell count criterion is subject to a laboratory lag and could lead to a delay in identification. Implementation of a point-of-care bedside score alert that uses readily available clinical data could allow providers to identify patients at greatest risk of poor outcomes immediately at ED presentation and triage, which motivated us to explore the predictive performance of qSOFA, the Shock Index, and NEWS2.

Our study is the first to provide a head-to-head comparison of the predictive performance of qSOFA, the Shock Index, and NEWS2, three easy-to-calculate bedside risk scores that use EHR data collected among patients with suspected sepsis. The Sepsis-3 guidelines recommend qSOFA to quickly identify non-ICU patients at greatest risk of poor outcomes because the measure exhibited predictive performance similar to the more extensive SOFA score outside the ICU.^16,23 Although some studies have confirmed qSOFA’s high predictive performance,^28-31 our test characteristics and AUROC findings are in line with other published analyses.^4,6,10,17 The UK National Health Service is using NEWS2 to screen for patients at risk of poor outcomes from sepsis. Several analyses that assessed the predictive ability of NEWS have reported estimates in line with our findings.^4,10,32 The Shock Index was introduced in 1967 and provided a metric to evaluate hemodynamic stability based on heart rate and systolic blood pressure.³³ The Shock Index has been studied in several contexts, including sepsis,³⁴ and studies show that a sustained Shock Index is associated with increased odds of vasopressor administration, higher prevalence of hyperlactatemia, and increased risk of poor outcomes in the ICU.^13,14

For our study, we were particularly interested in exploring how the Shock Index would compare with more frequently used severity scores such as qSOFA and NEWS2 among patients with suspected sepsis, given the simplicity of its calculation and the easy availability of required data. In our cohort of 23,837 patients, only 159 people had missing blood pressure and only 71 had omitted heart rate. In contrast, both qSOFA and NEWS2 include an assessment of level of consciousness that can be subject to variability in assessment methods and EHR documentation across institutions.¹¹ In our cohort, GCS within 30 minutes of ED presentation was missing in 72 patients, which could have led to incomplete calculation of qSOFA and NEWS2 if a missing value was not actually within normal limits.

Several investigations relate qSOFA to NEWS but few compare qSOFA with the newer NEWS2, and even fewer evaluate the Shock Index with any of these scores.^{10,11,18,29,35-37} In general, studies have shown that NEWS exhibits a higher AUROC for predicting mortality, sepsis with organ dysfunction, and ICU admission, often as a composite outcome.^{4,11,18,37,38} A handful of studies compare the Shock Index to SIRS; however, little has been done to compare the Shock Index to qSOFA or NEWS2, scores that have been used specifically for sepsis and might be more predictive of poor outcomes than SIRS.³³ In our study, the Shock Index had a higher AUROC than either qSOFA or NEWS2 for predicting in-hospital mortality and ED-to-ICU admission measured as separate outcomes and as a composite outcome using standard cut-points for these scores.

When selecting a severity score to apply in an institution, it is important to carefully evaluate the score’s test characteristics, in addition to considering the availability of reliable data. Tests with high sensitivity and NPV for the population being studied can be useful to rule out disease or risk of poor outcome, while tests with high specificity and PPV can be useful to rule in disease or risk of poor outcome.³⁹ When considering specificity, qSOFA’s performance was superior to the Shock Index and NEWS2 in our study, but a small percentage of the population was identified using a cut-point of qSOFA ≥2. If we used qSOFA and applied this standard cut-point at our institution, we could be confident that those identified were at increased risk, but we would miss a significant number of patients who would experience a poor outcome. When considering sensitivity, performance of NEWS2 was superior to qSOFA and the Shock Index in our study, but one-half of the population was identified using a cut-point of NEWS2 ≥5. If we were to apply this standard NEWS2 cut-point at our institution, we would assume that one-half of our population was at risk, which might drive resource use towards patients who will not experience a poor outcome. Although none of the scores exhibited a robust AUROC measure, the Shock Index had the highest AUROC for in-hospital mortality and ED-to-ICU admission when using the standard binary cut-point, and its sensitivity and specificity is between that of qSOFA and NEWS2, potentially making it a score to use in settings where qSOFA and NEWS2 score components, such as altered mentation, are not reliably collected. Finally, our sensitivity analysis varying the binary cut-point of each score within our population demonstrated that the standard cut-points might not be as useful within a specific population and might need to be tailored for implementation, balancing sensitivity, specificity, PPV, and NPV to meet local priorities and ICU capacity.

Our study has limitations. It is a single-center, retrospective analysis, factors that could reduce generalizability. However, it does include a large and diverse patient population spanning several years. Missing GCS data could have affected the predictive ability of qSOFA and NEWS2 in our cohort. We could not reliably perform imputation of GCS because of the high missingness and therefore we assumed missing was normal, as was done in the Sepsis-3 derivation studies.¹⁶ Previous studies have attempted to impute GCS and have not observed improved performance of qSOFA to predict mortality.⁴⁰ Because manually collected variables such as GCS are less reliably documented in the EHR, there might be limitations in their use for triage risk scores.

Although the current analysis focused on the predictive performance of qSOFA, the Shock Index, and NEWS2 at triage, performance of these scores could affect the ED team’s treatment decisions before handoff to the hospitalist team and the expected level of care the patient will receive after in-patient admission. These tests also have the advantage of being easy to calculate at the bedside over time, which could provide an objective assessment of longitudinal predicted prognosis. Future work should assess the longitudinal performance of each of these scores among those with suspected sepsis and to determine the impact using these scores would have on clinical and resource utilization outcomes.

Local priorities should drive selection of a screening tool, balancing sensitivity, specificity, PPV, and NPV to achieve the institution’s goals. qSOFA, Shock Index, and NEWS2 are risk stratification tools that can be easily implemented at ED triage using data available at the bedside. Although none of these scores performed strongly when comparing AUROCs, qSOFA was highly specific for identifying patients with poor outcomes, and NEWS2 was the most sensitive for ruling out those at high risk among patients with suspected sepsis. The Shock Index exhibited a sensitivity and specificity that fell between qSOFA and NEWS2 and also might be considered to identify those at increased risk, given its ease of implementation, particularly in settings where altered mentation is unreliably or inconsistently documented.

The authors thank the UCSF Division of Hospital Medicine Data Core for their assistance with data acquisition.

Sepsis is the leading cause of in-hospital mortality in the United States.¹ Sepsis is present on admission in 85% of cases, and each hour delay in antibiotic treatment is associated with 4% to 7% increased odds of mortality.^2,3 Prompt identification and treatment of sepsis is essential for reducing morbidity and mortality, but identifying sepsis during triage is challenging.²

Risk stratification scores that rely solely on data readily available at the bedside have been developed to quickly identify those at greatest risk of poor outcomes from sepsis in real time. The quick Sequential Organ Failure Assessment (qSOFA) score, the National Early Warning System (NEWS2), and the Shock Index are easy-to-calculate measures that use routinely collected clinical data that are not subject to laboratory delay. These scores can be incorporated into electronic health record (EHR)-based alerts and can be calculated longitudinally to track the risk of poor outcomes over time. qSOFA was developed to quantify patient risk at bedside in non-intensive care unit (ICU) settings, but there is no consensus about its ability to predict adverse outcomes such as mortality and ICU admission.^4-6 The United Kingdom’s National Health Service uses NEWS2 to identify patients at risk for sepsis.⁷ NEWS has been shown to have similar or better sensitivity in identifying poorer outcomes in sepsis patients compared with systemic inflammatory response syndrome (SIRS) criteria and qSOFA.^4,8-11 However, since the latest update of NEWS2 in 2017, there has been little study of its predictive ability. The Shock Index is a simple bedside score (heart rate divided by systolic blood pressure) that was developed to detect changes in cardiovascular performance before systemic shock onset. Although it was not developed for infection and has not been regularly applied in the sepsis literature, the Shock Index might be useful for identifying patients at increased risk of poor outcomes. Patients with higher and sustained Shock Index scores are more likely to experience morbidity, such as hyperlactatemia, vasopressor use, and organ failure, and also have an increased risk of mortality.^12-14

Although the predictive abilities of these bedside risk stratification scores have been assessed individually using standard binary cut-points, the comparative performance of qSOFA, the Shock Index, and NEWS2 has not been evaluated in patients presenting to an emergency department (ED) with suspected sepsis. Our objective was to provide a head-to-head comparison of the test characteristics of qSOFA, the Shock Index, and NEWS2 calculated at ED triage for predicting in-hospital mortality and ED-to-ICU admission in patients with suspected sepsis to help health systems and providers select screening measures.

Design and Setting

We conducted a retrospective cohort study of ED patients who presented with suspected sepsis to the University of California San Francisco (UCSF) Helen Diller Medical Center at Parnassus Heights between June 1, 2012, and December 31, 2018. Our institution is a 785-bed academic teaching hospital with approximately 30,000 ED encounters per year. The study was approved with a waiver of informed consent by the UCSF Human Research Protection Program.

Participants

We use an Epic-based EHR platform (Epic 2017, Epic Systems Corporation) for clinical care, which was implemented on June 1, 2012. All data elements were obtained from Clarity, the relational database that stores Epic’s inpatient data. The study included encounters for patients age ≥18 years who had blood cultures ordered within 24 hours of ED presentation and administration of intravenous antibiotics within 24 hours. Repeat encounters were treated independently in our analysis.

Outcomes and Measures

We compared the ability of qSOFA, the Shock Index, and NEWS2 to predict in-hospital mortality and admission to the ICU from the ED (ED-to-ICU admission). We used the most abnormal vital signs and clinical assessments gathered within 30 minutes of ED presentation to identify patients who were qSOFA-positive, Shock Index-positive, and NEWS2-positive based on standard cut-points of risk. Data elements used to calculate qSOFA, Shock Index, and NEWS2 included blood pressure, heart rate, respiratory rate, Glasgow Coma Scale (GCS) score, oxygen saturation, requirement for supplemental oxygen, and temperature (Table 1). Patients were considered positive if they had a qSOFAscore ≥2, Shock Index of >0.7, and NEWS2 ≥5 based on triage vital signs.^7,15,16 We considered patients to have altered mental status, a criterion used for NEWS2, if they had a GCS score <15 instead of using the “alert, verbal, confusion, pain, unresponsive” scale, which is not captured in our EHR, a method that has been used in earlier studies.^17,18 Missing assessments were considered normal. Although our primary analysis focused on the scores calculated within 30 minutes of ED presentation, we performed a sensitivity analysis examining scores calculated within 1 hour of ED presentation in the event of a delay in gathering triage vital sign data.

We compared demographic and clinical characteristics of patients who were positive for qSOFA, the Shock Index, and NEWS2. Demographic data were extracted from the EHR and included primary language, age, sex, and insurance status. All International Classification of Diseases (ICD)-9/10 diagnosis codes were pulled from Clarity billing tables. We used the Elixhauser comorbidity groupings¹⁹ of ICD-9/10 codes present on admission to identify preexisting comorbidities and underlying organ dysfunction. To estimate burden of comorbid illnesses, we calculated the validated van Walraven comorbidity index,²⁰ which provides an estimated risk of in-hospital death based on documented Elixhauser comorbidities. Admission level of care (acute, stepdown, or intensive care) was collected for inpatient admissions to assess initial illness severity.²¹ We also evaluated discharge disposition and in-hospital mortality. Index blood culture results were collected, and dates and timestamps of mechanical ventilation, fluid, vasopressor, and antibiotic administration were obtained for the duration of the encounter.

UCSF uses an automated, real-time, algorithm-based severe sepsis alert that is triggered when a patient meets ≥2 SIRS criteria and again when the patient meets severe sepsis or septic shock criteria (ie, ≥2 SIRS criteria in addition to end-organ dysfunction and/or fluid nonresponsive hypotension). This sepsis screening alert was in use for the duration of our study.²²

Statistical Analysis

We performed a subgroup analysis among those who were diagnosed with sepsis, according to the 2016 Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) criteria. Sepsis is defined as a change in Sequential Organ Failure Assessment (SOFA) score of ≥2 points within the first 48 hours.²³ Additionally, patients meeting Sepsis-3 criteria needed to (1) receive ≥4 days of sequential antibiotic therapy or experience death or discharge to hospice before 4 days of antibiotic therapy or (2) have a validated sepsis discharge billing code. These parameters were added to increase the specificity of our sample.²⁴

All statistical analyses were conducted using Stata 14 (StataCorp). We summarized differences in demographic and clinical characteristics among the populations meeting each severity score but elected not to conduct hypothesis testing because patients could be positive for one or more scores. We calculated sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for each score to predict in-hospital mortality and ED-to-ICU admission. To allow comparison with other studies, we also created a composite outcome of either in-hospital mortality or ED-to-ICU admission. To assess score discrimination to predict in-hospital mortality and ED-to-ICU admission, we calculated the area under the receiver operating characteristic curve (AUROC) along with asymptotic normal 95% CI using the “roctab” command considering a binary cut-point, as well as the full range of scores measured in the cohort. The AUROC range from 0.50 to 1.00 and a score in the 0.70 to 0.80 range can be considered fair.²⁵ We assessed significant differences between severity score AUROCs using the DeLong method²⁶ implemented through Stata 14’s “roccomp” command. As a sensitivity analysis, we explored whether the standard cut-points for qSOFA, the Shock Index, and NEWS2 provided the highest AUROC in our population by calculating test characteristics for several score cut-points.

Within our sample 23,837 ED patients had blood cultures ordered within 24 hours of ED presentation and were considered to have suspected sepsis. The mean age of the cohort was 60.8 years, and 1,612 (6.8%) had positive blood cultures. A total of 12,928 patients (54.2%) were found to have sepsis. We documented 1,427 in-hospital deaths (6.0%) and 3,149 (13.2%) ED-to-ICU admissions. At ED triage 1,921 (8.1%) were qSOFA-positive, 4,273 (17.9%) were Shock Index-positive, and 11,832 (49.6%) were NEWS2-positive. At ED triage, blood pressure, heart rate, respiratory rate, and oxygen saturated were documented in >99% of patients, 93.5% had temperature documented, and 28.5% had GCS recorded. If the window of assessment was widened to 1 hour, GCS was only documented among 44.2% of those with suspected sepsis.

Demographic Characteristics and Clinical Course

We identified significant differences when comparing demographic and clinical characteristics among patients who scored positive for the three severity measures at triage (Table 2). Although no hypothesis testing was conducted because patients could meet one or more scores, qSOFA-positive patients were older (median 70, 66, and 64 years, respectively), more likely to have Medicare as the primary payor (67.6% vs 59.7% vs 56.6%), to have chronic renal failure (26.1%, 23.1%, and 23.3%, respectively), to have a greater degree of underlying comorbidities based on the van Walraven Comorbidity Index (median 15, 12, and 11, respectively), and to be admitted to the ICU from the ED (48.1%, 36.3%, and 21.0%, respectively) compared with those positive for the Shock Index or NEWS2.

qSOFA-positive patients received antibiotics more quickly than those who were Shock Index-positive or NEWS2-positive (median 1.5, 1.8, and 2.8 hours after admission, respectively). In addition, those who were qSOFA-positive were more likely to have a positive blood culture (10.9%, 9.4%, and 8.5%, respectively) and to receive an EHR-based diagnosis of sepsis (77.0%, 69.6%, and 60.9%, respectively) than those who were Shock Index- or NEWS2-positive. Those who were qSOFA-positive also were more likely to be mechanically ventilated during their hospital stay (25.4%, 19.2%, and 10.8%, respectively) and to receive vasopressors (33.5%, 22.5%, and 12.2%, respectively). In-hospital mortality also was more common among those who were qSOFA-positive at triage (23.4%, 15.3%, and 9.2%, respectively).

Because both qSOFA and NEWS2 incorporate GCS, we explored baseline characteristics of patients with GCS documented at triage (n = 6,794). These patients were older (median age 63 and 61 years, P < .0001), more likely to be male (54.9% and 53.4%, P = .0031), more likely to have renal failure (22.8% and 20.1%, P < .0001), more likely to have liver disease (14.2% and 12.8%, P = .006), had a higher van Walraven comorbidity score on presentation (median 10 and 8, P < .0001), and were more likely to go directly to the ICU from the ED (20.2% and 10.6%, P < .0001). However, among the 6,397 GCS scores documented at triage, only 1,579 (24.7%) were abnormal.

Test Characteristics of qSOFA, Shock Index, and NEWS2 for Predicting In-hospital Mortality and ED-to-ICU Admission

Among 23,837 patients with suspected sepsis, NEWS2 had the highest sensitivity for predicting in-hospital mortality (76.0%; 95% CI, 73.7%-78.2%) and ED-to-ICU admission (78.9%; 95% CI, 77.5%-80.4%) but had the lowest specificity for in-hospital mortality (52.0%; 95% CI, 51.4%-52.7%) and for ED-to-ICU admission (54.8%; 95% CI, 54.1%-55.5%) (Table 3). qSOFA had the lowest sensitivity for in-hospital mortality (31.5%; 95% CI, 29.1%-33.9%) and ED-to-ICU admission (29.3%; 95% CI, 27.7%-30.9%) but the highest specificity for in-hospital mortality (93.4%; 95% CI, 93.1%-93.8%) and ED-to-ICU admission (95.2%; 95% CI, 94.9%-95.5%). The Shock Index had a sensitivity that fell between qSOFA and NEWS2 for in-hospital mortality (45.8%; 95% CI, 43.2%-48.5%) and ED-to-ICU admission (49.2%; 95% CI, 47.5%-51.0%). The specificity of the Shock Index also was between qSOFA and NEWS2 for in-hospital mortality (83.9%; 95% CI, 83.4%-84.3%) and ED-to-ICU admission (86.8%; 95% CI, 86.4%-87.3%). All three scores exhibited relatively low PPV, ranging from 9.2% to 23.4% for in-hospital mortality and 21.0% to 48.0% for ED-to-ICU triage. Conversely, all three scores exhibited relatively high NPV, ranging from 95.5% to 97.1% for in-hospital mortality and 89.8% to 94.5% for ED-to-ICU triage. The patterns in sensitivity and specificity for in-hospital mortality and ED-to-ICU admission were similar among the 12,928 patients who received an EHR-based sepsis diagnosis with the tests generally demonstrating lower specificities, higher PPVs, and lower NPVs (Table 3).

When considering a binary cutoff, the Shock Index exhibited the highest AUROC for in-hospital mortality (0.648; 95% CI, 0.635-0.662) and had a significantly higher AUROC than qSOFA (AUROC, 0.625; 95% CI, 0.612-0.637; P = .0005), but there was no difference compared with NEWS2 (AUROC, 0.640; 95% CI, 0.628-0.652; P = .2112). NEWS2 had a significantly higher AUROC than qSOFA for predicting in-hospital mortality (P = .0227). The Shock Index also exhibited the highest AUROC for ED-to-ICU admission (0.680; 95% CI, 0.617-0.689), which was significantly higher than the AUROC for qSOFA (P < .0001) and NEWS2 (P = 0.0151). NEWS2 had a significantly higher AUROC than qSOFA for predicting ED-to-ICU admission (P < .0001). Similar findings were seen in patients found to have sepsis. When considering the range of possible scores measured in our cohort, qSOFA and NEWS2 exhibited higher AUROCs for in-hospital mortality and ED-to-ICU admission than the Shock Index among patients with suspected infection and the subgroup with a sepsis diagnosis (Figure).

In this retrospective cohort study of 23,837 patients who presented to the ED with suspected sepsis, the standard qSOFA threshold was met least frequently, followed by the Shock Index and NEWS2. NEWS2 had the highest sensitivity but the lowest specificity for predicting in-hospital mortality and ED-to-ICU admission, making it a challenging bedside risk stratification scale for identifying patients at risk of poor clinical outcomes. When comparing predictive performance among the three scales, qSOFA had the highest specificity and the Shock Index had the highest AUROC for in-hospital mortality and ED-to-ICU admission in this cohort of patients with suspected sepsis. These trends in sensitivity, specificity, and AUROC were consistent among those who met EHR criteria for a sepsis diagnosis. In the analysis of the three scoring systems using all available cut-points, qSOFA and NEWS2 had the highest AUROCs, followed by the Shock Index.

Considering the rapid progression from organ dysfunction to death in sepsis patients, as well as the difficulty establishing a sepsis diagnosis at triage,²³ providers must quickly identify patients at increased risk of poor outcomes when they present to the ED. Sepsis alerts often are built using SIRS criteria,²⁷ including the one used for sepsis surveillance at UCSF since 2012,²² but the white blood cell count criterion is subject to a laboratory lag and could lead to a delay in identification. Implementation of a point-of-care bedside score alert that uses readily available clinical data could allow providers to identify patients at greatest risk of poor outcomes immediately at ED presentation and triage, which motivated us to explore the predictive performance of qSOFA, the Shock Index, and NEWS2.

Our study is the first to provide a head-to-head comparison of the predictive performance of qSOFA, the Shock Index, and NEWS2, three easy-to-calculate bedside risk scores that use EHR data collected among patients with suspected sepsis. The Sepsis-3 guidelines recommend qSOFA to quickly identify non-ICU patients at greatest risk of poor outcomes because the measure exhibited predictive performance similar to the more extensive SOFA score outside the ICU.^16,23 Although some studies have confirmed qSOFA’s high predictive performance,^28-31 our test characteristics and AUROC findings are in line with other published analyses.^4,6,10,17 The UK National Health Service is using NEWS2 to screen for patients at risk of poor outcomes from sepsis. Several analyses that assessed the predictive ability of NEWS have reported estimates in line with our findings.^4,10,32 The Shock Index was introduced in 1967 and provided a metric to evaluate hemodynamic stability based on heart rate and systolic blood pressure.³³ The Shock Index has been studied in several contexts, including sepsis,³⁴ and studies show that a sustained Shock Index is associated with increased odds of vasopressor administration, higher prevalence of hyperlactatemia, and increased risk of poor outcomes in the ICU.^13,14

For our study, we were particularly interested in exploring how the Shock Index would compare with more frequently used severity scores such as qSOFA and NEWS2 among patients with suspected sepsis, given the simplicity of its calculation and the easy availability of required data. In our cohort of 23,837 patients, only 159 people had missing blood pressure and only 71 had omitted heart rate. In contrast, both qSOFA and NEWS2 include an assessment of level of consciousness that can be subject to variability in assessment methods and EHR documentation across institutions.¹¹ In our cohort, GCS within 30 minutes of ED presentation was missing in 72 patients, which could have led to incomplete calculation of qSOFA and NEWS2 if a missing value was not actually within normal limits.

Several investigations relate qSOFA to NEWS but few compare qSOFA with the newer NEWS2, and even fewer evaluate the Shock Index with any of these scores.^{10,11,18,29,35-37} In general, studies have shown that NEWS exhibits a higher AUROC for predicting mortality, sepsis with organ dysfunction, and ICU admission, often as a composite outcome.^{4,11,18,37,38} A handful of studies compare the Shock Index to SIRS; however, little has been done to compare the Shock Index to qSOFA or NEWS2, scores that have been used specifically for sepsis and might be more predictive of poor outcomes than SIRS.³³ In our study, the Shock Index had a higher AUROC than either qSOFA or NEWS2 for predicting in-hospital mortality and ED-to-ICU admission measured as separate outcomes and as a composite outcome using standard cut-points for these scores.

When selecting a severity score to apply in an institution, it is important to carefully evaluate the score’s test characteristics, in addition to considering the availability of reliable data. Tests with high sensitivity and NPV for the population being studied can be useful to rule out disease or risk of poor outcome, while tests with high specificity and PPV can be useful to rule in disease or risk of poor outcome.³⁹ When considering specificity, qSOFA’s performance was superior to the Shock Index and NEWS2 in our study, but a small percentage of the population was identified using a cut-point of qSOFA ≥2. If we used qSOFA and applied this standard cut-point at our institution, we could be confident that those identified were at increased risk, but we would miss a significant number of patients who would experience a poor outcome. When considering sensitivity, performance of NEWS2 was superior to qSOFA and the Shock Index in our study, but one-half of the population was identified using a cut-point of NEWS2 ≥5. If we were to apply this standard NEWS2 cut-point at our institution, we would assume that one-half of our population was at risk, which might drive resource use towards patients who will not experience a poor outcome. Although none of the scores exhibited a robust AUROC measure, the Shock Index had the highest AUROC for in-hospital mortality and ED-to-ICU admission when using the standard binary cut-point, and its sensitivity and specificity is between that of qSOFA and NEWS2, potentially making it a score to use in settings where qSOFA and NEWS2 score components, such as altered mentation, are not reliably collected. Finally, our sensitivity analysis varying the binary cut-point of each score within our population demonstrated that the standard cut-points might not be as useful within a specific population and might need to be tailored for implementation, balancing sensitivity, specificity, PPV, and NPV to meet local priorities and ICU capacity.

Our study has limitations. It is a single-center, retrospective analysis, factors that could reduce generalizability. However, it does include a large and diverse patient population spanning several years. Missing GCS data could have affected the predictive ability of qSOFA and NEWS2 in our cohort. We could not reliably perform imputation of GCS because of the high missingness and therefore we assumed missing was normal, as was done in the Sepsis-3 derivation studies.¹⁶ Previous studies have attempted to impute GCS and have not observed improved performance of qSOFA to predict mortality.⁴⁰ Because manually collected variables such as GCS are less reliably documented in the EHR, there might be limitations in their use for triage risk scores.

Although the current analysis focused on the predictive performance of qSOFA, the Shock Index, and NEWS2 at triage, performance of these scores could affect the ED team’s treatment decisions before handoff to the hospitalist team and the expected level of care the patient will receive after in-patient admission. These tests also have the advantage of being easy to calculate at the bedside over time, which could provide an objective assessment of longitudinal predicted prognosis. Future work should assess the longitudinal performance of each of these scores among those with suspected sepsis and to determine the impact using these scores would have on clinical and resource utilization outcomes.

Local priorities should drive selection of a screening tool, balancing sensitivity, specificity, PPV, and NPV to achieve the institution’s goals. qSOFA, Shock Index, and NEWS2 are risk stratification tools that can be easily implemented at ED triage using data available at the bedside. Although none of these scores performed strongly when comparing AUROCs, qSOFA was highly specific for identifying patients with poor outcomes, and NEWS2 was the most sensitive for ruling out those at high risk among patients with suspected sepsis. The Shock Index exhibited a sensitivity and specificity that fell between qSOFA and NEWS2 and also might be considered to identify those at increased risk, given its ease of implementation, particularly in settings where altered mentation is unreliably or inconsistently documented.

The authors thank the UCSF Division of Hospital Medicine Data Core for their assistance with data acquisition.

The US Department of Veterans Affairs (VA) created the Home Improvements and Structural Alterations (HISA) program to help provide necessary home modifications (HMs) to veterans with disabilities (VWDs) that will facilitate the provision of medical services at home and improve home accessibility and functional independence. The Veterans Health Administration (VHA) has more than 9 million veteran enrollees; of those, 2.7 million are classified as rural or highly rural.¹ Rural veterans (RVs) possess higher rate of disability compared with that of urban veterans.^2-5 RVs have unequal access to screening of ambulatory care sensitive conditions (eg, hypertension, diabetes mellitus).⁶ Furthermore, RVs are at risk of poor medical outcomes due to distance from health care facilities and specialist care, which can be a barrier to emergency care when issues arise. These barriers, among others, are associated with compromised health quality of life and health outcomes for RVs.^3,6 The HISA program may be key to decreasing falls and other serious mishaps in the home. Therefore, understanding use of the HISA program by RVs is important. However, to date little information has been available regarding use of HISA benefits by RVs or characteristics of RVs who receive HISA benefits.

HISA Alterations Program

HISA was initially developed by VA to improve veterans’ transition from acute medical care to home.^7,8 However, to obtain HISA grants currently, there is an average 3 to 6 months application process.⁷ Through the HISA program, VWDs can be prescribed the following HMs, including (but not limited to): flooring replacement, permanent ramps, roll-in showers, installation of central air-conditioning systems, improved lighting, kitchen/bathroom modifications, and home inspections. The HMs prescribed depend on an assessment of medical need by health care providers (HCPs).⁸

As time passed and the veteran population aged, the program now primarily helps ensure the ability to enter into essential areas and safety in the home.⁵ The amount of a HISA payment is based on whether a veteran’s health condition is related to military service as defined by the VHA service connection medical evaluation process. Barriers to obtaining a HISA HM can include difficulty in navigating the evaluation process and difficulty in finding a qualified contractor or builder to do the HM.⁷

This article aims to: (1) Detail the sociodemographic and clinical characteristics of rural HISA users (RHUs); (2) report on HISA usage patterns in number, types, and cost of HMs; (3) compare use amid the diverse VA medical centers (VAMCs) and related complexity levels and Veterans Integrated Service Networks (VISNs); and (4) examine the relationship between travel time/distance and HISA utilization. The long-term goal is to provide accurate information to researchers, HM administrators, health care providers and policy makers on HISA program utilization by rural VWDs, which may help improve its use and bring awareness of its users. This study was approved by the affiliate University of Florida Institutional Review Board and VA research and development committee at the North Florida/South Georgia Veterans Health System.

Methods

Data were obtained from 3 VA sources: the National Prosthetics Patient Database (NPPD), the VHA Medical Inpatient Dataset, and the VHA Outpatient Dataset.⁷ The NPPD is a national administrative database that contains information on prosthetic-associated products ordered by HCPs for patients, such as portable ramps, handrails, home oxygen equipment, and orthotic and prosthetic apparatus. Data obtained from the NPPD included cost of HMs, clinical characteristics, VISN, and VAMC. VA facilities are categorized into complexity levels 1a, 1b, 1c, 2, and 3. Complexity level 1a to 1c VAMCs address medical cases that entail “heightening involvedness,” meaning a larger number of patients presented with medical concerns needing medical specialists. Complexity levels 2 and 3 have fewer resources, lower patient numbers, and less medically complex patients. Finally, the VHA Medical Inpatient and Outpatient Datasets administrated by VA Informatics and Computing Infrastructure, consist of in-depth health services national data on inpatient and outpatient encounters and procedures.

The study cohort was divided into those with service-connected conditions (Class 1) or those with conditions not related to military service (Class 2). If veterans were identified in both classes, they were assigned to Class 1. The cost variable is determined by using the veterans’ classification. Class 1 veterans receive a lifetime limit of $6800, and Class 2 veterans receive a lifetime limit of $2000. A Class 2 veteran with ≥ 50% disability rating is eligible for a HISA lifetime limit of $6800. Whenever a value exceeds allowed limit of $6800 or $2000, due to data entry error or other reasons, the study team reassigned the cost value to the maximum allowed value.

Travel distance and time were derived by loading patient zip codes and HISA facility locations into the geographical information system program and using the nearest facility and find-route tools. These tools used a road network that simulates real-world driving conditions to calculate distance.

Study Variables

VWDs of any age, gender, and race/ethnicity who qualified for HISA and received HMs from fiscal year ( FY) 2015 through FY 2018 were identified (N = 30,823). Most VWDs were nonrural subjects (n = 19,970), and 43 had no Federal Information Processing System data. The final study cohort consisted of 10,810 HISA recipients. The NPPD, inpatient and outpatient data were merged by scrambled social security numbers to retrieve the following data: age, gender, race, ethnicity, marital status, Class (1 or 2), mean and total number of inpatient days, and type of HMs prescribed.

We also recorded rurality using the VA Rural-Urban Commuting Areas (RUCA) system, but we combined the rural and highly rural designation.¹ Census tracts with a RUCA score of 10.0 are deemed highly rural, the remainder are considered rural except those with a RUCA score of 1.0 or 1.1. Travel time and distance from a veteran’s home to the VA facility that provided the HISA prescription were determined from zip codes. The current study focuses on VAMCs prescribing stations (affiliated sites of administrative parent medical facilities) where the HISA users obtained the HM, not the parent station (administrative parent medical facilities).

HISA Utilization

To characterize HISA utilization geographically and over time, the number of users were mapped by county. Areas where users were increasing (hot spots) or decreasing (cold spots) also were mapped. The maps were created using Environmental Systems Research Institute ArcGIS Pro 2.2.1 software. We chose to use natural breaks (Jenks) data classification method in a choropleth to symbolize the change over time map. We then used the Getis Ord GI* optimized hot spot analysis tool in the ArcGIS Pro spatial statistics tool set to generate the hot/cold spot maps. This tool identifies clusters of high values (hot spots) and low values (cold spots) creating a new output layer, RHUs by county, with a Z score, P value, and CI for each county. The Gi Bin field classifies statistically significant hot and cold spots. Counties sorted into the ± 3 category (bin) have a clustering characteristic (eg, with neighboring counties) that is statistically significant with a 99% CI; the ± 2 bin indicates a 95% CI for those county clustering sorted therein; ± 1 reflects a 90% CI; and 0 bin contains county features that have no statistical significant clustering with neighboring counties.

Data Analysis

Data were cleaned and analyzed using SAS 9.4 and R 3.5.3. Descriptive statistics are provided for sociodemographic characteristics, clinical characteristics, and class. ANOVA and t tests were used to compare continuous variables between groups, while χ² and Fisher exact tests were used for dichotomous and categorical outcome variables. The threshold for statistical significance for these tests was set at α = .001.

Results

There were 10,810 RHUs from FY 2015 through FY 2018 and HISA utilization increased each year (Figure 1). Although some years may show usage decreases relative to previous fiscal years, the cumulative trends showed an increase relative to FY 2015 for both Classes of RVs (Figure 2). There was a 45.4% increase from FY 2015 to FY 2018 with a mean 13.6% yearly increase. Class 1 increased 21.0% and Class 2 increased 39.5% from FY 2015 to FY 2016 (Figure 3).

Most RHUs were male, White, and married. Class 1 and Class 2 RHUs differed significantly by age, race, marital status, and disability conditions: Class 1 RHUs were aged 6.6 years younger with a mean age of 69.1 years compared with 75.7 years for Class 2 users. For Class 1 RHUs, a plurality (29.4%) were aged 65 to 69 years; while a plurality (41.4%) of Class 2 users were aged ≥ 80 years. Musculoskeletal was the most common identified type of condition for all RHUs (Table 1).

There were 11,166 HM prescribed to RHUs (Table 2). Bathroom HMs also were the dominant HM type for all facilities regardless of complexity levels (Table 3). The San Antonio, Texas, VAMC demonstrated the highest Class 1 vs Class 2 difference in HISA use (Class 1: 87.7% and Class 2: 12.3%). Except for the Des Moines VAMC, all other VAMCs showed HISA use > 60% by Class 1.

Cost Data

Air-conditioning installation ($5007) was the costliest HM overall (Table 4), closely followed by bathroom ($4978) and kitchen modifications ($4305). Bathroom renovations were the costliest HM type for both Class 1 and Class 2, closely followed by electrical repair and air-conditioning installation for Class 1 and driveway reconstruction and wooden ramp construction for Class 2.

The mean award received for HM was $4687 (Table 5). While the number of RHUs increased from FY 2015 to FY 2016, the average cost decreased, both overall ($280) and for Class 1 ($195) and Class 2 ($153). Except for a small decline in the number of Class 2 HISA recipients from FY 2017 to FY 2018, overall, the number of RHUs continuously grew from FY 2015 to FY 2018: 977 for the overall cohort, 678 for Class 1 and 299 for Class 2. Despite the obvious gain in the number of RHUs, the average costs did not notably change over time. VISN 21 had the highest mean cost, followed by VISNs 17, 6, 22, and 20.

Travel

Travel time and distance to the HISA prescribing facility differed significantly between Class 1 and Class 2 HISA users. RHUs had to travel about 95 minutes from their place of residence to access the HISA benefits program. There were no statistically significant differences between Class 1 and 2 users with respect to travel time and distance traveled (Table 6).

The majority of Class 1 and Class 2 veterans accessed the HISA from their nearest facility. However, nearly one-quarter of both Class 1 and 2 RHUs (24% each) did not. Among the 2598 who accessed the nonnearest facility, 97 (3.7%) accessed a facility that is ≤ 40 miles. Many (44%) users traveled 40 to 100 miles, and another 43.2% traveled 100 to 200 miles from their residence to access a HM prescription. Some 2598 users (1.1%) traveled > 500 miles to access a facility.

Discussion

Although utilization of the HISA program has steadily increased, overall participation by subpopulations such as RHUs can still be improved significantly. Veterans aged ≤ 46 years who have a disability that is common to those receiving HISA benefits have low HISA utilization. Similarly, veterans with sensory disabilities also have low use. These subpopulations are among those in great need of attention and services.

A study by Lucas and Zelaya, using the 2016 National Health Interview Survey data with an aim to measure degree of vision problems, dual sensory impairment, and hearing trouble in male veterans aged ≥ 18 years, found that veterans were more likely to report dual sensory impairment and balance difficulties when compared with nonveterans.⁹ The number of female veterans is growing but had very low representation in this study.¹⁰ This emerging VHA population requires information and education on their HM benefits.

Home Modifications

The most common HM prescribed for RHUs was for the bathroom. Further investigation is warranted as to why, given the diversity of HM types that the grant covers, low prescription rates exist across most of the HM types. There may be a lack of knowledge by providers and VWD as to the range of HMs that can be awarded under the grant. It is important that HCPs and veterans receive education on HISA HM options.

Semeah and colleagues pointed out the need for an assessment of the HISA HM ordering system to ensure that multiple HMs items (eg, kitchen, air conditioning, fees, driveway, and plumbing) are listed among the forced choices shown to clinicians to select from.⁷ Poor housing in rural America is widespread: 63% of rural dwellings need renovations and/or repairs to be accessible to individuals with disabilities, with > 6.7 million rural homes having no or faulty plumbing or kitchens; yet in this study, prescriptions for these HMs accounted for < 1%.^11,12

VISN 6 had the most HISA awards with 1364, while VISN 21 had the fewest (245). Across all VISNs, Class 1 RHUs received more prescriptions than did Class 2 RHUs. Future research may seek to examine whether prescribers are fully aware of the eligibility of HM prescription to Class 2 veterans. VISN 21 ($5354); VISN 17 ($5302); and VISN 6 ($5301) had the highest mean HM expenditures. The national mean cost for HISA HMs were $4978 for bathrooms and $4305 for kitchens; for non-HISA HMs in FY 2017, the mean costs were $6362 and $12,255, respectively. A noteworthy concern is whether the maximum grant limit awards are sufficient to perform more expensive and complex HMs, such as the kitchen or major bathroom alternations.¹³

Facilities categorized as 1a, 1b, or 1c provided prescription to more than 70.7% of all RHUs. Embedded in the infrastructures of these higher complexity level facilities are substantial research and education opportunities for staff and patients and a diversity of medical specialties. Attention must be provided to less resourced facilities located in rural areas or serving many RVs that may not have the infrastructure to monitor the HISA process or make it available to VWDs. Furthermore, the top prescribing VAMCs were almost all high-complexity facilities, and all served > 100 unique RHUs. Further research may be warranted to be better understand the variation in HM prescriptions at VAMCs.

North Florida/Sough Georgia was the highest-prescribing VAMC with 39% more HM prescriptions than the second highest prescribing facility (Durham, NC). Unfortunately, the data presented here cannot establish causality for the large variance difference between the top facilities, and the skewed distribution of total RHUs across VAMCs.