Peter K. Lindenauer, MD, MSc

Frailty is a common geriatric syndrome characterized by decreased physiological reserves leading to increased vulnerability to stressors.¹ Frail individuals are at increased risk of adverse health outcomes including falls, disability, hospitalization, and mortality.¹ Discharge to skilled nursing facilities (SNFs) is also associated with adverse outcomes,^2,3 but limited data exist on the utility of frailty in predicting discharge location in medical elders. We aimed to evaluate the association of frailty assessed by the Reported Edmonton Frailty Scale (REFS) with discharge disposition in hospitalized medical patients who were previously living in the community.

We conducted a prospective study of community dwelling elders (≥65 years) hospitalized to the medical service from January 2014 to April 2016. Trained research assistants interviewed patients and/or caregivers on hospital day 1; the REFS was used to screen for frailty and the Mini-Cog assessment for cognitive impairment (supplementary Appendixes 1 and 2). The primary outcome was discharge disposition categorized as discharge to home (with or without home health services) or discharge to a postacute care (PAC) facility (SNF or inpatient rehabilitation). Multivariable Poisson regression analysis was used to estimate the relative risk of discharge to a PAC facility. Frailty was grouped into the following 3 categories: (1) not frail, (2) apparently vulnerable/mildly frail, and (3) moderately/severely frail.

Among the 775 patients screened, 272 declined to participate, were non-English speakers, were transferred from another facility, were admitted under observation status, had advanced dementia, or died during hospitalization. Five hundred and three medical patients were included: median age was 80 years (interquartile range 75-86 years); 54.1% were female and 82.9% were white. The most common comorbidities were hypertension (51.7%), diabetes (26.0%), and renal failure (26.0%). Of the included patients, 11.1% had a known diagnosis of dementia and 52.1% screened positive for cognitive impairment (Table).

Overall, 24.9% were not frail, 49.5% were apparently vulnerable/mildly frail, and 25.6% were moderately/severely frail. About two-thirds (64.8%) returned home (40.0% with home healthcare) and 35% were discharged to a PAC facility (97.1% of them to SNF). Compared with patients who were discharged home, those discharged to a PAC facility were older (≥85 years; 26.7% vs 40.1%) and more likely to have dementia (7.7% vs 17.5%) and be frail (apparently vulnerable/mild frailty = 48.5% vs 51.4%%, moderate/severe frailty = 19.9% vs 36.2%; P < .001). Median length of hospital stay was shorter in those returning home (4 vs 5 days, P < .001).

In the multivariate analysis, which was adjusted for demographics, comorbidities, and principal diagnosis, frailty was strongly associated with discharge to PAC facility (apparently vulnerable/mild frailty vs no frailty, relative ratio [RR] = 2.00; 95% confidence interval [CI], 1.28-3.27, and moderate/severe frailty vs no frailty; RR = 2.66, 95% CI, 1.67-4.43). When the frailty score was included as a continuous variable, 1 unit increase in the score was associated with a 12% higher risk for discharge to a PAC facility (RR = 1.12; 95% CI, 1.07-1.17).

In this analysis of over 500 community-dwelling elderly medical patients hospitalized at one large tertiary center, we found that almost half of the patients were frail and over one-third had a new discharge to a PAC facility. Frailty, as assessed by REFS, was strongly associated with discharge to a PAC facility after adjusting for possible confounders.

Frailty is increasingly recognized as a useful tool to risk stratify the highly heterogeneous population of elderly people.⁴ Previous studies reported that frailty was predictive of discharge to PAC facilities in geriatric trauma and burn injury patients.^5,6 We found similar results in a population of elderly medical patients. A recent study showed that the Hospital Admission Risk Profile score comprising of age, modified Mini-Mental State Examination (MMSE), and functionality prior to admission was associated with discharge disposition in elderly patients admitted to a single geriatric unit in a rural hospital.⁷ Our study supports this finding by using a validated measure of frailty, the RFS, and does not include the lengthy MMSE.

Our study has several limitations. First, it a single-center study and results may not be generalizable; however, we included a large sample of patients with a variety of medical diagnoses. Second, the REFS is self-reported posing the risks of recall, respondent bias, and interview bias. We chose the REFS to assess frailty due to its practicality and ease of administration but also its completeness of assessing multiple important geriatric domains. Lastly, we did not collect the reason for discharge to PAC and it may have been a potential confounder.

In conclusion, our study demonstrates that frailty assessed by a practical validated scale, the REFS, is a strong predictor of a new discharge to PAC facilities in older medical patients. Accurate identification of elders at risk for discharge to PAC facilities provides the potential to counsel patients and families and plan for complex post discharge needs. Future studies should identify potential interventions targeting frail patients in which PAC is not obligatory, aiming to increase their chance of being discharged home.

Disclosure

Drs. Stefan and Ramdass had full access to all the data in the study. They take responsibility for the integrity of the data and the accuracy of the analysis. Drs. Stefan, Starr, Brennan, and Ramdass conceived the study. Ms. Liu and Dr. Pekow analyzed the data. Dr. Ramdass prepared the manuscript. Drs. Stefan, Brennan, Lindenauer, and Starr critically reviewed the manuscript for important intellectual content. A subset of the patients included in this study was part of a Health Resources and Services Administration funded Geri-Pal Transformation through Learning and Collaboration project awarded to Baystate Medical Center, grant number U1QHP28702 (PI: Maura J. Brennan). The investigators retained full independence in the conduct of this research. The authors have no conflicts of interest.

Frailty is a common geriatric syndrome characterized by decreased physiological reserves leading to increased vulnerability to stressors.¹ Frail individuals are at increased risk of adverse health outcomes including falls, disability, hospitalization, and mortality.¹ Discharge to skilled nursing facilities (SNFs) is also associated with adverse outcomes,^2,3 but limited data exist on the utility of frailty in predicting discharge location in medical elders. We aimed to evaluate the association of frailty assessed by the Reported Edmonton Frailty Scale (REFS) with discharge disposition in hospitalized medical patients who were previously living in the community.

We conducted a prospective study of community dwelling elders (≥65 years) hospitalized to the medical service from January 2014 to April 2016. Trained research assistants interviewed patients and/or caregivers on hospital day 1; the REFS was used to screen for frailty and the Mini-Cog assessment for cognitive impairment (supplementary Appendixes 1 and 2). The primary outcome was discharge disposition categorized as discharge to home (with or without home health services) or discharge to a postacute care (PAC) facility (SNF or inpatient rehabilitation). Multivariable Poisson regression analysis was used to estimate the relative risk of discharge to a PAC facility. Frailty was grouped into the following 3 categories: (1) not frail, (2) apparently vulnerable/mildly frail, and (3) moderately/severely frail.

Among the 775 patients screened, 272 declined to participate, were non-English speakers, were transferred from another facility, were admitted under observation status, had advanced dementia, or died during hospitalization. Five hundred and three medical patients were included: median age was 80 years (interquartile range 75-86 years); 54.1% were female and 82.9% were white. The most common comorbidities were hypertension (51.7%), diabetes (26.0%), and renal failure (26.0%). Of the included patients, 11.1% had a known diagnosis of dementia and 52.1% screened positive for cognitive impairment (Table).

Overall, 24.9% were not frail, 49.5% were apparently vulnerable/mildly frail, and 25.6% were moderately/severely frail. About two-thirds (64.8%) returned home (40.0% with home healthcare) and 35% were discharged to a PAC facility (97.1% of them to SNF). Compared with patients who were discharged home, those discharged to a PAC facility were older (≥85 years; 26.7% vs 40.1%) and more likely to have dementia (7.7% vs 17.5%) and be frail (apparently vulnerable/mild frailty = 48.5% vs 51.4%%, moderate/severe frailty = 19.9% vs 36.2%; P < .001). Median length of hospital stay was shorter in those returning home (4 vs 5 days, P < .001).

In the multivariate analysis, which was adjusted for demographics, comorbidities, and principal diagnosis, frailty was strongly associated with discharge to PAC facility (apparently vulnerable/mild frailty vs no frailty, relative ratio [RR] = 2.00; 95% confidence interval [CI], 1.28-3.27, and moderate/severe frailty vs no frailty; RR = 2.66, 95% CI, 1.67-4.43). When the frailty score was included as a continuous variable, 1 unit increase in the score was associated with a 12% higher risk for discharge to a PAC facility (RR = 1.12; 95% CI, 1.07-1.17).

In this analysis of over 500 community-dwelling elderly medical patients hospitalized at one large tertiary center, we found that almost half of the patients were frail and over one-third had a new discharge to a PAC facility. Frailty, as assessed by REFS, was strongly associated with discharge to a PAC facility after adjusting for possible confounders.

Frailty is increasingly recognized as a useful tool to risk stratify the highly heterogeneous population of elderly people.⁴ Previous studies reported that frailty was predictive of discharge to PAC facilities in geriatric trauma and burn injury patients.^5,6 We found similar results in a population of elderly medical patients. A recent study showed that the Hospital Admission Risk Profile score comprising of age, modified Mini-Mental State Examination (MMSE), and functionality prior to admission was associated with discharge disposition in elderly patients admitted to a single geriatric unit in a rural hospital.⁷ Our study supports this finding by using a validated measure of frailty, the RFS, and does not include the lengthy MMSE.

Our study has several limitations. First, it a single-center study and results may not be generalizable; however, we included a large sample of patients with a variety of medical diagnoses. Second, the REFS is self-reported posing the risks of recall, respondent bias, and interview bias. We chose the REFS to assess frailty due to its practicality and ease of administration but also its completeness of assessing multiple important geriatric domains. Lastly, we did not collect the reason for discharge to PAC and it may have been a potential confounder.

In conclusion, our study demonstrates that frailty assessed by a practical validated scale, the REFS, is a strong predictor of a new discharge to PAC facilities in older medical patients. Accurate identification of elders at risk for discharge to PAC facilities provides the potential to counsel patients and families and plan for complex post discharge needs. Future studies should identify potential interventions targeting frail patients in which PAC is not obligatory, aiming to increase their chance of being discharged home.

Disclosure

Drs. Stefan and Ramdass had full access to all the data in the study. They take responsibility for the integrity of the data and the accuracy of the analysis. Drs. Stefan, Starr, Brennan, and Ramdass conceived the study. Ms. Liu and Dr. Pekow analyzed the data. Dr. Ramdass prepared the manuscript. Drs. Stefan, Brennan, Lindenauer, and Starr critically reviewed the manuscript for important intellectual content. A subset of the patients included in this study was part of a Health Resources and Services Administration funded Geri-Pal Transformation through Learning and Collaboration project awarded to Baystate Medical Center, grant number U1QHP28702 (PI: Maura J. Brennan). The investigators retained full independence in the conduct of this research. The authors have no conflicts of interest.

Frailty is a common geriatric syndrome characterized by decreased physiological reserves leading to increased vulnerability to stressors.¹ Frail individuals are at increased risk of adverse health outcomes including falls, disability, hospitalization, and mortality.¹ Discharge to skilled nursing facilities (SNFs) is also associated with adverse outcomes,^2,3 but limited data exist on the utility of frailty in predicting discharge location in medical elders. We aimed to evaluate the association of frailty assessed by the Reported Edmonton Frailty Scale (REFS) with discharge disposition in hospitalized medical patients who were previously living in the community.

We conducted a prospective study of community dwelling elders (≥65 years) hospitalized to the medical service from January 2014 to April 2016. Trained research assistants interviewed patients and/or caregivers on hospital day 1; the REFS was used to screen for frailty and the Mini-Cog assessment for cognitive impairment (supplementary Appendixes 1 and 2). The primary outcome was discharge disposition categorized as discharge to home (with or without home health services) or discharge to a postacute care (PAC) facility (SNF or inpatient rehabilitation). Multivariable Poisson regression analysis was used to estimate the relative risk of discharge to a PAC facility. Frailty was grouped into the following 3 categories: (1) not frail, (2) apparently vulnerable/mildly frail, and (3) moderately/severely frail.

Among the 775 patients screened, 272 declined to participate, were non-English speakers, were transferred from another facility, were admitted under observation status, had advanced dementia, or died during hospitalization. Five hundred and three medical patients were included: median age was 80 years (interquartile range 75-86 years); 54.1% were female and 82.9% were white. The most common comorbidities were hypertension (51.7%), diabetes (26.0%), and renal failure (26.0%). Of the included patients, 11.1% had a known diagnosis of dementia and 52.1% screened positive for cognitive impairment (Table).

Overall, 24.9% were not frail, 49.5% were apparently vulnerable/mildly frail, and 25.6% were moderately/severely frail. About two-thirds (64.8%) returned home (40.0% with home healthcare) and 35% were discharged to a PAC facility (97.1% of them to SNF). Compared with patients who were discharged home, those discharged to a PAC facility were older (≥85 years; 26.7% vs 40.1%) and more likely to have dementia (7.7% vs 17.5%) and be frail (apparently vulnerable/mild frailty = 48.5% vs 51.4%%, moderate/severe frailty = 19.9% vs 36.2%; P < .001). Median length of hospital stay was shorter in those returning home (4 vs 5 days, P < .001).

In the multivariate analysis, which was adjusted for demographics, comorbidities, and principal diagnosis, frailty was strongly associated with discharge to PAC facility (apparently vulnerable/mild frailty vs no frailty, relative ratio [RR] = 2.00; 95% confidence interval [CI], 1.28-3.27, and moderate/severe frailty vs no frailty; RR = 2.66, 95% CI, 1.67-4.43). When the frailty score was included as a continuous variable, 1 unit increase in the score was associated with a 12% higher risk for discharge to a PAC facility (RR = 1.12; 95% CI, 1.07-1.17).

In this analysis of over 500 community-dwelling elderly medical patients hospitalized at one large tertiary center, we found that almost half of the patients were frail and over one-third had a new discharge to a PAC facility. Frailty, as assessed by REFS, was strongly associated with discharge to a PAC facility after adjusting for possible confounders.

Frailty is increasingly recognized as a useful tool to risk stratify the highly heterogeneous population of elderly people.⁴ Previous studies reported that frailty was predictive of discharge to PAC facilities in geriatric trauma and burn injury patients.^5,6 We found similar results in a population of elderly medical patients. A recent study showed that the Hospital Admission Risk Profile score comprising of age, modified Mini-Mental State Examination (MMSE), and functionality prior to admission was associated with discharge disposition in elderly patients admitted to a single geriatric unit in a rural hospital.⁷ Our study supports this finding by using a validated measure of frailty, the RFS, and does not include the lengthy MMSE.

Our study has several limitations. First, it a single-center study and results may not be generalizable; however, we included a large sample of patients with a variety of medical diagnoses. Second, the REFS is self-reported posing the risks of recall, respondent bias, and interview bias. We chose the REFS to assess frailty due to its practicality and ease of administration but also its completeness of assessing multiple important geriatric domains. Lastly, we did not collect the reason for discharge to PAC and it may have been a potential confounder.

In conclusion, our study demonstrates that frailty assessed by a practical validated scale, the REFS, is a strong predictor of a new discharge to PAC facilities in older medical patients. Accurate identification of elders at risk for discharge to PAC facilities provides the potential to counsel patients and families and plan for complex post discharge needs. Future studies should identify potential interventions targeting frail patients in which PAC is not obligatory, aiming to increase their chance of being discharged home.

Disclosure

Drs. Stefan and Ramdass had full access to all the data in the study. They take responsibility for the integrity of the data and the accuracy of the analysis. Drs. Stefan, Starr, Brennan, and Ramdass conceived the study. Ms. Liu and Dr. Pekow analyzed the data. Dr. Ramdass prepared the manuscript. Drs. Stefan, Brennan, Lindenauer, and Starr critically reviewed the manuscript for important intellectual content. A subset of the patients included in this study was part of a Health Resources and Services Administration funded Geri-Pal Transformation through Learning and Collaboration project awarded to Baystate Medical Center, grant number U1QHP28702 (PI: Maura J. Brennan). The investigators retained full independence in the conduct of this research. The authors have no conflicts of interest.

“I didn’t have time to write a short letter, so I wrote a long one instead.”

-Mark Twain

Sepsis is a logical target for quality measures. Specifically, sepsis represents the perfect storm of immense public health burden^1-3 combined with unexplained practice^4-6 and outcomes⁷ variation. Thus, it is not surprising that in October 2015, the Centers of Medicare and Medicaid Services (CMS) adopted a sepsis quality measure.⁸ More surprising were the complex contents of the CMS Sepsis Core Measure “SEP-1” quality measure.⁹ CMS had written a “long letter.”

The multiple processes targeted with the CMS SEP-1 quality measure can best be understood with a brief account of history. SEP-1 arose from the National Quality Forum’s (NQF) project #0500: “Severe Sepsis and Septic Shock: Management Bundle,” a measure based upon Rivers et al.’s¹⁰ single-center, randomized, controlled trial of early goal-directed therapy (EGDT) for severe sepsis. EGDT was an intervention that consisted of fluid resuscitation and hemodynamic management based upon fulfilling specific targets of central venous pressure, superior vena cava oxygen saturation (or lactic acid), and hemoglobin and mean arterial pressures.¹¹ The large mortality benefits, physiological rationale, and algorithmic responses to a variety of abnormal clinical values provided an appealing protocol to critical care and emergency physicians trained to normalize measured values, as well as policy makers looking for quality measures. Observational studies consistently showed associations between adoption of guideline-based “sepsis bundles” and improved patient outcomes,^12-14 setting the stage for the transition of NQF #0500 into SEP-1.

However, the transition from EGDT-based NQF #0500 to SEP-1 has been tumultuous. Soon after adoption of SEP-1, the consensus definitions of sepsis changed markedly. Sepsis went from being defined as the presence of infection with concomitant systemic inflammatory response syndrome (sepsis), organ dysfunction (severe sepsis), and/or shock,¹⁵ to being defined as a dysregulated response to infection resulting in life-threatening organ dysfunction (sepsis) and/or fluid-resistant hypotension requiring vasopressors and lactate greater than 2 mmol/L.¹⁶ As the study by Barbash et al.¹⁷in this issue clearly outlines, conflicting definitions of “sepsis” have left clinicians confused regarding whom the SEP-1 measure should apply. At the same time, results of 3 large, international, randomized trials investigating the efficacy of EGDT were published, providing strong evidence that EGDT did not provide improved patient outcomes over usual care.¹⁸ SEP-1 adapted with the evolving evidence base, adding putative “usual care” processes such as evaluation of skin and peripheral pulses, and use of dynamic measures of fluid responsiveness, as quality measures.¹⁹ However, as Barbash et al. also outline, the resulting process measure was incredibly complex, with potentially more than 50 data elements collected over 6 hours in the initial management of sepsis.

In addition to its unprecedented complexity, SEP-1 received criticism for the weak evidence base of its individual components. The general concepts behind SEP-1 are well-accepted tenets of sepsis management: rapid recognition, assessment and treatment of underlying infection, and institution of intravenous fluids and vasopressor support for septic shock. However, the “all or none” prescriptive nature of the SEP-1 bundle was based on a somewhat arbitrary set of measures and targets. For example, patients with septic shock must receive 30 cc/kg of intravenous fluids to be “SEP-1 compliant.” The value “30 cc/kg” was taken from the average volume of fluids reported in prior sepsis trials, essentially based on a very low level of evidence.²⁰ The strict 30 cc/kg cutoff did not take into account that “the median isn’t the message”²¹ in fluid management: optimal resuscitation targets are unclear,²² and selecting the median as a target ignores the fact that 50% of patients enrolled in international trials of EGDT received less than 30 cc/kg of initial fluid resuscitation (the interquartile range was 16-42 cc/kg).¹⁸ Thus, most participants in trials upon which the SEP-1 fluid measure was based would ironically not have met the SEP-1 measure. Mandates for physical exam and physiological measures were based on similarly low levels of evidence.

Into this context, Barbash et al. use a representative sample of US hospitals to explore the opinions of hospital quality leaders regarding the SEP-1 measure. First, the qualitative methods used by Barbash et al. warrant some explanation. Much of biomedical research is characterized by hypothesis-driven, deductive reasoning: theories are tested using observations. In contrast, the methods of Barbash et al. use inductive reasoning: observations are used to develop theories within a systematic approach called “grounded theory” that explores common themes emerging from structured interviews.²³ Inductive reasoning can later inform deductive reasoning, feeding theories into testable hypotheses. However, qualitative, inductive research is not meant to test hypotheses and is not subject to typical notions of “power and sample size” often expected of quantitative statistical analyses. Qualitative studies reach sufficient sample size when no further themes emerge, a situation called “thematic saturation”; the sample size here of 29 participants rests comfortably in the range of participants commonly needed for thematic saturation.²³

Barbash et al. identified common themes in opinions of quality leaders regarding SEP-1. Namely, the complexity of SEP-1 necessitated a major resource investment into sepsis care and data collection. The major infrastructure investments needed to comply with SEP-1 also bred frustration regarding lack of perceived fairness around the “all or none” nature of the measure and raised multiple additional challenges including lack of clinician buy-in and resistance to protocolized care. Prior qualitative studies evaluating hospital quality leaders’ opinions on performance measures identified similar concerns about lack of “fairness,”²⁴ but the implementation of SEP-1 has raised additional concern regarding the large burdens of instituting major infrastructure changes to monitor processes of care required to report on this measure. Despite the major challenges of responding to SEP-1, quality leaders expressed optimism that increased attention to sepsis would ultimately lead to better patient outcomes.

How might future sepsis quality measures achieve the adequate balance between focusing attention on improving care processes for high-impact diseases, without imposing additional burdens on the healthcare system? Lessons from Barbash et al. help us move forward. First, rather than taxing hospitals with administratively complex process measures, initial attempts at quality measures should start simply. Policy makers should consider moving forward into new areas of quality measurement in 2 ways: (1) pursue 1 or 2 processes with strong etiological links to important patient outcomes (eg, timely antibiotics in septic shock),^25-28 and/or (2) use risk-adjusted outcomes and allow individual hospitals to adopt processes that improve local patient outcomes. Evidence suggests that the introduction of a quality measure may result in improved outcomes regardless of adoption of specific target processes,²⁹ although results are mixed.^30,31 In either case, complex “all or none” measures based upon weak evidence run a high risk of inciting clinician resentment and paradoxically perpetuating poor quality by increasing healthcare costs (decreased efficiency) without gains in safety, effectiveness, timeliness, or equity.³² It has been estimated that hospitals spend on average $2 million to implement SEP-1,³³ with unclear return on the investment. The experience of SEP-1 is a reminder that, as evidence evolves, quality measures must adapt lest they become irrelevant. However, it is also a reminder that quality measures should not sit precariously on the edge of evidence. Withdrawal of process-based measures due to a changing evidence landscape breeds mistrust and impairs future attempts to improve quality.

Sepsis quality measures face additional challenges. If recent experience with interpretation of sepsis definitions can serve as a guide, variable uptake of newer sepsis definitions between/across hospitals will impair the ability to risk-adjust outcome measures and increase bias in identifying outlier hospitals.³⁴ In addition, recent studies have already raised skepticism regarding the effectiveness of individual SEP-1 bundle components, confirming suspicions that the 30 cc/kg fluid bolus is not a magic quality target. Rather, the effectiveness of prior sepsis bundles has likely been driven by improved time to antibiotics, a process unstudied in sepsis trials, but driven by increased attention to the importance of early sepsis recognition and timely management.²⁸ Timeliness of antibiotics can act as an effect modifier for more complex sepsis therapies, with quicker time to antibiotics associated with reversal of previously described effectiveness of activated protein C,³⁵ and EGDT.²⁸

Sepsis has a legacy in which improving simple processes (ie, time to antibiotics) obviates the need for more complex interventions (eg, activated protein C, EGDT). To the extent that CMS remains committed to using process-based measures of quality, those focused on sepsis are likely to be most effective when pared down to the simplest and strongest evidence base—improved recognition³⁶ and timely antibiotics (for patients with infection-induced organ dysfunction and shock). Taking the time to start simply may best serve our current patients and preserve stakeholder buy-in for quality initiatives likely to benefit our future patients.

Disclosure

Dr. Lindenauer reports that he received support from the Centers for Medicare and Medicaid Services to develop and maintain hospital outcome measures for pneumonia and COPD. Dr. Lindenauer is supported by grant K24HL132008 from the National Heart, Lung, and Blood Institute. Dr. Walkey was supported by grants K01-HL116768 and R01-HL139751 from the National Heart, Lung, and Blood Institute.

“I didn’t have time to write a short letter, so I wrote a long one instead.”

-Mark Twain

Sepsis is a logical target for quality measures. Specifically, sepsis represents the perfect storm of immense public health burden^1-3 combined with unexplained practice^4-6 and outcomes⁷ variation. Thus, it is not surprising that in October 2015, the Centers of Medicare and Medicaid Services (CMS) adopted a sepsis quality measure.⁸ More surprising were the complex contents of the CMS Sepsis Core Measure “SEP-1” quality measure.⁹ CMS had written a “long letter.”

The multiple processes targeted with the CMS SEP-1 quality measure can best be understood with a brief account of history. SEP-1 arose from the National Quality Forum’s (NQF) project #0500: “Severe Sepsis and Septic Shock: Management Bundle,” a measure based upon Rivers et al.’s¹⁰ single-center, randomized, controlled trial of early goal-directed therapy (EGDT) for severe sepsis. EGDT was an intervention that consisted of fluid resuscitation and hemodynamic management based upon fulfilling specific targets of central venous pressure, superior vena cava oxygen saturation (or lactic acid), and hemoglobin and mean arterial pressures.¹¹ The large mortality benefits, physiological rationale, and algorithmic responses to a variety of abnormal clinical values provided an appealing protocol to critical care and emergency physicians trained to normalize measured values, as well as policy makers looking for quality measures. Observational studies consistently showed associations between adoption of guideline-based “sepsis bundles” and improved patient outcomes,^12-14 setting the stage for the transition of NQF #0500 into SEP-1.

However, the transition from EGDT-based NQF #0500 to SEP-1 has been tumultuous. Soon after adoption of SEP-1, the consensus definitions of sepsis changed markedly. Sepsis went from being defined as the presence of infection with concomitant systemic inflammatory response syndrome (sepsis), organ dysfunction (severe sepsis), and/or shock,¹⁵ to being defined as a dysregulated response to infection resulting in life-threatening organ dysfunction (sepsis) and/or fluid-resistant hypotension requiring vasopressors and lactate greater than 2 mmol/L.¹⁶ As the study by Barbash et al.¹⁷in this issue clearly outlines, conflicting definitions of “sepsis” have left clinicians confused regarding whom the SEP-1 measure should apply. At the same time, results of 3 large, international, randomized trials investigating the efficacy of EGDT were published, providing strong evidence that EGDT did not provide improved patient outcomes over usual care.¹⁸ SEP-1 adapted with the evolving evidence base, adding putative “usual care” processes such as evaluation of skin and peripheral pulses, and use of dynamic measures of fluid responsiveness, as quality measures.¹⁹ However, as Barbash et al. also outline, the resulting process measure was incredibly complex, with potentially more than 50 data elements collected over 6 hours in the initial management of sepsis.

In addition to its unprecedented complexity, SEP-1 received criticism for the weak evidence base of its individual components. The general concepts behind SEP-1 are well-accepted tenets of sepsis management: rapid recognition, assessment and treatment of underlying infection, and institution of intravenous fluids and vasopressor support for septic shock. However, the “all or none” prescriptive nature of the SEP-1 bundle was based on a somewhat arbitrary set of measures and targets. For example, patients with septic shock must receive 30 cc/kg of intravenous fluids to be “SEP-1 compliant.” The value “30 cc/kg” was taken from the average volume of fluids reported in prior sepsis trials, essentially based on a very low level of evidence.²⁰ The strict 30 cc/kg cutoff did not take into account that “the median isn’t the message”²¹ in fluid management: optimal resuscitation targets are unclear,²² and selecting the median as a target ignores the fact that 50% of patients enrolled in international trials of EGDT received less than 30 cc/kg of initial fluid resuscitation (the interquartile range was 16-42 cc/kg).¹⁸ Thus, most participants in trials upon which the SEP-1 fluid measure was based would ironically not have met the SEP-1 measure. Mandates for physical exam and physiological measures were based on similarly low levels of evidence.

Into this context, Barbash et al. use a representative sample of US hospitals to explore the opinions of hospital quality leaders regarding the SEP-1 measure. First, the qualitative methods used by Barbash et al. warrant some explanation. Much of biomedical research is characterized by hypothesis-driven, deductive reasoning: theories are tested using observations. In contrast, the methods of Barbash et al. use inductive reasoning: observations are used to develop theories within a systematic approach called “grounded theory” that explores common themes emerging from structured interviews.²³ Inductive reasoning can later inform deductive reasoning, feeding theories into testable hypotheses. However, qualitative, inductive research is not meant to test hypotheses and is not subject to typical notions of “power and sample size” often expected of quantitative statistical analyses. Qualitative studies reach sufficient sample size when no further themes emerge, a situation called “thematic saturation”; the sample size here of 29 participants rests comfortably in the range of participants commonly needed for thematic saturation.²³

Barbash et al. identified common themes in opinions of quality leaders regarding SEP-1. Namely, the complexity of SEP-1 necessitated a major resource investment into sepsis care and data collection. The major infrastructure investments needed to comply with SEP-1 also bred frustration regarding lack of perceived fairness around the “all or none” nature of the measure and raised multiple additional challenges including lack of clinician buy-in and resistance to protocolized care. Prior qualitative studies evaluating hospital quality leaders’ opinions on performance measures identified similar concerns about lack of “fairness,”²⁴ but the implementation of SEP-1 has raised additional concern regarding the large burdens of instituting major infrastructure changes to monitor processes of care required to report on this measure. Despite the major challenges of responding to SEP-1, quality leaders expressed optimism that increased attention to sepsis would ultimately lead to better patient outcomes.

How might future sepsis quality measures achieve the adequate balance between focusing attention on improving care processes for high-impact diseases, without imposing additional burdens on the healthcare system? Lessons from Barbash et al. help us move forward. First, rather than taxing hospitals with administratively complex process measures, initial attempts at quality measures should start simply. Policy makers should consider moving forward into new areas of quality measurement in 2 ways: (1) pursue 1 or 2 processes with strong etiological links to important patient outcomes (eg, timely antibiotics in septic shock),^25-28 and/or (2) use risk-adjusted outcomes and allow individual hospitals to adopt processes that improve local patient outcomes. Evidence suggests that the introduction of a quality measure may result in improved outcomes regardless of adoption of specific target processes,²⁹ although results are mixed.^30,31 In either case, complex “all or none” measures based upon weak evidence run a high risk of inciting clinician resentment and paradoxically perpetuating poor quality by increasing healthcare costs (decreased efficiency) without gains in safety, effectiveness, timeliness, or equity.³² It has been estimated that hospitals spend on average $2 million to implement SEP-1,³³ with unclear return on the investment. The experience of SEP-1 is a reminder that, as evidence evolves, quality measures must adapt lest they become irrelevant. However, it is also a reminder that quality measures should not sit precariously on the edge of evidence. Withdrawal of process-based measures due to a changing evidence landscape breeds mistrust and impairs future attempts to improve quality.

Sepsis quality measures face additional challenges. If recent experience with interpretation of sepsis definitions can serve as a guide, variable uptake of newer sepsis definitions between/across hospitals will impair the ability to risk-adjust outcome measures and increase bias in identifying outlier hospitals.³⁴ In addition, recent studies have already raised skepticism regarding the effectiveness of individual SEP-1 bundle components, confirming suspicions that the 30 cc/kg fluid bolus is not a magic quality target. Rather, the effectiveness of prior sepsis bundles has likely been driven by improved time to antibiotics, a process unstudied in sepsis trials, but driven by increased attention to the importance of early sepsis recognition and timely management.²⁸ Timeliness of antibiotics can act as an effect modifier for more complex sepsis therapies, with quicker time to antibiotics associated with reversal of previously described effectiveness of activated protein C,³⁵ and EGDT.²⁸

Sepsis has a legacy in which improving simple processes (ie, time to antibiotics) obviates the need for more complex interventions (eg, activated protein C, EGDT). To the extent that CMS remains committed to using process-based measures of quality, those focused on sepsis are likely to be most effective when pared down to the simplest and strongest evidence base—improved recognition³⁶ and timely antibiotics (for patients with infection-induced organ dysfunction and shock). Taking the time to start simply may best serve our current patients and preserve stakeholder buy-in for quality initiatives likely to benefit our future patients.

Disclosure

Dr. Lindenauer reports that he received support from the Centers for Medicare and Medicaid Services to develop and maintain hospital outcome measures for pneumonia and COPD. Dr. Lindenauer is supported by grant K24HL132008 from the National Heart, Lung, and Blood Institute. Dr. Walkey was supported by grants K01-HL116768 and R01-HL139751 from the National Heart, Lung, and Blood Institute.

“I didn’t have time to write a short letter, so I wrote a long one instead.”

-Mark Twain

Sepsis is a logical target for quality measures. Specifically, sepsis represents the perfect storm of immense public health burden^1-3 combined with unexplained practice^4-6 and outcomes⁷ variation. Thus, it is not surprising that in October 2015, the Centers of Medicare and Medicaid Services (CMS) adopted a sepsis quality measure.⁸ More surprising were the complex contents of the CMS Sepsis Core Measure “SEP-1” quality measure.⁹ CMS had written a “long letter.”

The multiple processes targeted with the CMS SEP-1 quality measure can best be understood with a brief account of history. SEP-1 arose from the National Quality Forum’s (NQF) project #0500: “Severe Sepsis and Septic Shock: Management Bundle,” a measure based upon Rivers et al.’s¹⁰ single-center, randomized, controlled trial of early goal-directed therapy (EGDT) for severe sepsis. EGDT was an intervention that consisted of fluid resuscitation and hemodynamic management based upon fulfilling specific targets of central venous pressure, superior vena cava oxygen saturation (or lactic acid), and hemoglobin and mean arterial pressures.¹¹ The large mortality benefits, physiological rationale, and algorithmic responses to a variety of abnormal clinical values provided an appealing protocol to critical care and emergency physicians trained to normalize measured values, as well as policy makers looking for quality measures. Observational studies consistently showed associations between adoption of guideline-based “sepsis bundles” and improved patient outcomes,^12-14 setting the stage for the transition of NQF #0500 into SEP-1.

However, the transition from EGDT-based NQF #0500 to SEP-1 has been tumultuous. Soon after adoption of SEP-1, the consensus definitions of sepsis changed markedly. Sepsis went from being defined as the presence of infection with concomitant systemic inflammatory response syndrome (sepsis), organ dysfunction (severe sepsis), and/or shock,¹⁵ to being defined as a dysregulated response to infection resulting in life-threatening organ dysfunction (sepsis) and/or fluid-resistant hypotension requiring vasopressors and lactate greater than 2 mmol/L.¹⁶ As the study by Barbash et al.¹⁷in this issue clearly outlines, conflicting definitions of “sepsis” have left clinicians confused regarding whom the SEP-1 measure should apply. At the same time, results of 3 large, international, randomized trials investigating the efficacy of EGDT were published, providing strong evidence that EGDT did not provide improved patient outcomes over usual care.¹⁸ SEP-1 adapted with the evolving evidence base, adding putative “usual care” processes such as evaluation of skin and peripheral pulses, and use of dynamic measures of fluid responsiveness, as quality measures.¹⁹ However, as Barbash et al. also outline, the resulting process measure was incredibly complex, with potentially more than 50 data elements collected over 6 hours in the initial management of sepsis.

In addition to its unprecedented complexity, SEP-1 received criticism for the weak evidence base of its individual components. The general concepts behind SEP-1 are well-accepted tenets of sepsis management: rapid recognition, assessment and treatment of underlying infection, and institution of intravenous fluids and vasopressor support for septic shock. However, the “all or none” prescriptive nature of the SEP-1 bundle was based on a somewhat arbitrary set of measures and targets. For example, patients with septic shock must receive 30 cc/kg of intravenous fluids to be “SEP-1 compliant.” The value “30 cc/kg” was taken from the average volume of fluids reported in prior sepsis trials, essentially based on a very low level of evidence.²⁰ The strict 30 cc/kg cutoff did not take into account that “the median isn’t the message”²¹ in fluid management: optimal resuscitation targets are unclear,²² and selecting the median as a target ignores the fact that 50% of patients enrolled in international trials of EGDT received less than 30 cc/kg of initial fluid resuscitation (the interquartile range was 16-42 cc/kg).¹⁸ Thus, most participants in trials upon which the SEP-1 fluid measure was based would ironically not have met the SEP-1 measure. Mandates for physical exam and physiological measures were based on similarly low levels of evidence.

Into this context, Barbash et al. use a representative sample of US hospitals to explore the opinions of hospital quality leaders regarding the SEP-1 measure. First, the qualitative methods used by Barbash et al. warrant some explanation. Much of biomedical research is characterized by hypothesis-driven, deductive reasoning: theories are tested using observations. In contrast, the methods of Barbash et al. use inductive reasoning: observations are used to develop theories within a systematic approach called “grounded theory” that explores common themes emerging from structured interviews.²³ Inductive reasoning can later inform deductive reasoning, feeding theories into testable hypotheses. However, qualitative, inductive research is not meant to test hypotheses and is not subject to typical notions of “power and sample size” often expected of quantitative statistical analyses. Qualitative studies reach sufficient sample size when no further themes emerge, a situation called “thematic saturation”; the sample size here of 29 participants rests comfortably in the range of participants commonly needed for thematic saturation.²³

Barbash et al. identified common themes in opinions of quality leaders regarding SEP-1. Namely, the complexity of SEP-1 necessitated a major resource investment into sepsis care and data collection. The major infrastructure investments needed to comply with SEP-1 also bred frustration regarding lack of perceived fairness around the “all or none” nature of the measure and raised multiple additional challenges including lack of clinician buy-in and resistance to protocolized care. Prior qualitative studies evaluating hospital quality leaders’ opinions on performance measures identified similar concerns about lack of “fairness,”²⁴ but the implementation of SEP-1 has raised additional concern regarding the large burdens of instituting major infrastructure changes to monitor processes of care required to report on this measure. Despite the major challenges of responding to SEP-1, quality leaders expressed optimism that increased attention to sepsis would ultimately lead to better patient outcomes.

How might future sepsis quality measures achieve the adequate balance between focusing attention on improving care processes for high-impact diseases, without imposing additional burdens on the healthcare system? Lessons from Barbash et al. help us move forward. First, rather than taxing hospitals with administratively complex process measures, initial attempts at quality measures should start simply. Policy makers should consider moving forward into new areas of quality measurement in 2 ways: (1) pursue 1 or 2 processes with strong etiological links to important patient outcomes (eg, timely antibiotics in septic shock),^25-28 and/or (2) use risk-adjusted outcomes and allow individual hospitals to adopt processes that improve local patient outcomes. Evidence suggests that the introduction of a quality measure may result in improved outcomes regardless of adoption of specific target processes,²⁹ although results are mixed.^30,31 In either case, complex “all or none” measures based upon weak evidence run a high risk of inciting clinician resentment and paradoxically perpetuating poor quality by increasing healthcare costs (decreased efficiency) without gains in safety, effectiveness, timeliness, or equity.³² It has been estimated that hospitals spend on average $2 million to implement SEP-1,³³ with unclear return on the investment. The experience of SEP-1 is a reminder that, as evidence evolves, quality measures must adapt lest they become irrelevant. However, it is also a reminder that quality measures should not sit precariously on the edge of evidence. Withdrawal of process-based measures due to a changing evidence landscape breeds mistrust and impairs future attempts to improve quality.

Sepsis quality measures face additional challenges. If recent experience with interpretation of sepsis definitions can serve as a guide, variable uptake of newer sepsis definitions between/across hospitals will impair the ability to risk-adjust outcome measures and increase bias in identifying outlier hospitals.³⁴ In addition, recent studies have already raised skepticism regarding the effectiveness of individual SEP-1 bundle components, confirming suspicions that the 30 cc/kg fluid bolus is not a magic quality target. Rather, the effectiveness of prior sepsis bundles has likely been driven by improved time to antibiotics, a process unstudied in sepsis trials, but driven by increased attention to the importance of early sepsis recognition and timely management.²⁸ Timeliness of antibiotics can act as an effect modifier for more complex sepsis therapies, with quicker time to antibiotics associated with reversal of previously described effectiveness of activated protein C,³⁵ and EGDT.²⁸

Sepsis has a legacy in which improving simple processes (ie, time to antibiotics) obviates the need for more complex interventions (eg, activated protein C, EGDT). To the extent that CMS remains committed to using process-based measures of quality, those focused on sepsis are likely to be most effective when pared down to the simplest and strongest evidence base—improved recognition³⁶ and timely antibiotics (for patients with infection-induced organ dysfunction and shock). Taking the time to start simply may best serve our current patients and preserve stakeholder buy-in for quality initiatives likely to benefit our future patients.

Disclosure

Dr. Lindenauer reports that he received support from the Centers for Medicare and Medicaid Services to develop and maintain hospital outcome measures for pneumonia and COPD. Dr. Lindenauer is supported by grant K24HL132008 from the National Heart, Lung, and Blood Institute. Dr. Walkey was supported by grants K01-HL116768 and R01-HL139751 from the National Heart, Lung, and Blood Institute.

Bacterial pneumonia remains an important cause of morbidity and mortality in the United States, and is the 8^th leading cause of death with 55,227 deaths among adults annually.¹ In 2005, the American Thoracic Society (ATS) and the Infectious Diseases Society of America (IDSA) collaborated to update guidelines for hospital-acquired pneumonia (HAP), ventilator-associated pneumonia, and healthcare-associated pneumonia (HCAP).² This broad document outlines an evidence-based approach to diagnostic testing and antibiotic management based on the epidemiology and risk factors for these conditions. The guideline specifies the following criteria for HCAP: hospitalization in the past 90 days, residence in a skilled nursing facility (SNF), home infusion therapy, hemodialysis, home wound care, family members with multidrug resistant organisms (MDRO), and immunosuppressive diseases or medications, with the presumption that these patients are more likely to be harboring MDRO and should thus be treated empirically with broad-spectrum antibiotic therapy. Prior studies have shown that patients with HCAP have a more severe illness, are more likely to have MDRO, are more likely to be inadequately treated, and are at a higher risk for mortality than patients with community-acquired pneumonia (CAP).^3,4

These guidelines are controversial, especially in regard to the recommendations to empirically treat broadly with 2 antibiotics targeting Pseudomonas species, whether patients with HCAP merit broader spectrum coverage than patients with CAP, and whether the criteria for defining HCAP are adequate to predict which patients are harboring MDRO. It has subsequently been proposed that HCAP is more related to CAP than to HAP, and a recent update to the guideline removed recommendations for treatment of HCAP and will be placing HCAP into the guidelines for CAP instead.⁵ We sought to investigate the degree of uptake of the ATS and IDSA guideline recommendations by physicians over time, and whether this led to a change in outcomes among patients who met the criteria for HCAP.

Setting and Patients

We identified patients discharged between July 1, 2007, and November 30, 2011, from 488 US hospitals that participated in the Premier database (Premier Inc., Charlotte, North Carolina), an inpatient database developed for measuring quality and healthcare utilization. The database is frequently used for healthcare research and has been described previously.⁶ Member hospitals are in all regions of the US and are generally reflective of US hospitals. This database contains multiple data elements, including sociodemographic information, International Classification of Diseases, 9th Revision-Clinical Modification (ICD-9-CM) diagnosis and procedure codes, hospital and physician information, source of admission, and discharge status. It also includes a date-stamped log of all billed items and services, including diagnostic tests, medications, and other treatments. Because the data do not contain identifiable information, the institutional review board at our medical center determined that this study did not constitute human subjects research.

We included all patients aged ≥18 years with a principal diagnosis of pneumonia or with a secondary diagnosis of pneumonia paired with a principal diagnosis of respiratory failure, acute respiratory distress syndrome, respiratory arrest, sepsis, or influenza. Patients were excluded if they were transferred to or from another acute care institution, had a length of stay of 1 day or less, had cystic fibrosis, did not have a chest radiograph, or did not receive antibiotics within 48 hours of admission.

For each patient, we extracted age, gender, principal diagnosis, comorbidities, and the specialty of the attending physician. Comorbidities were identified from ICD-9-CM secondary diagnosis codes and Diagnosis Related Groups by using Healthcare Cost and Utilization Project Comorbidity Software, version 3.1, based on the work of Elixhauser (Agency for Healthcare Research and Quality, Rockville, Maryland).⁷ In order to ensure that patients had HCAP, we required the presence of ≥1 HCAP criteria, including hospitalization in the past 90 days, hemodialysis, admission from an SNF, or immune suppression (which was derived from either a secondary diagnosis for neutropenia, hematological malignancy, organ transplant, acquired immunodeficiency virus, or receiving immunosuppressant drugs or corticosteroids [equivalent to ≥20 mg/day of prednisone]).

Definitions of Guideline-Concordant and Discordant Antibiotic Therapy

The ATS and IDSA guidelines recommended the following antibiotic combinations for HCAP: an antipseudomonal cephalosporin or carbapenem or a beta-lactam/lactamase inhibitor, plus an antipseudomonal quinolone or aminoglycoside, plus an antibiotic with activity versus methicillin resistant Staphylococcus aureus (MRSA), such as vancomycin or linezolid. Based on these guidelines, we defined the receipt of fully guideline-concordant antibiotics as 2 recommended antibiotics for Pseudomonas species plus 1 for MRSA administered by the second day of admission. Partially guideline-concordant antibiotics were defined as 1 recommended antibiotic for Pseudomonas species plus 1 for MRSA by the second day of hospitalization. Guideline-discordant antibiotics were defined as all other combinations.

Statistical Analysis

Descriptive statistics on patient characteristics are presented as frequency, proportions for categorical factors, and median with interquartile range (IQR) for continuous variables for the full cohort and by treatment group, defined as fully or partially guideline-concordant antibiotic therapy or discordant therapy. Hospital rates of fully guideline-concordant treatment are presented overall and by hospital characteristics. The association of hospital characteristics with rates of fully guideline-concordant therapy were assessed by using 1-way analysis of variance tests.

To assess trends across hospitals for the association between the use of guideline-concordant therapy and mortality, progression to respiratory failure as measured by the late initiation of invasive mechanical ventilation (day 3 or later), and the length of stay among survivors, we divided the 4.5-year study period into 9 intervals of 6 months each; 292 hospitals that submitted data for all 9 time points were examined in this analysis. Based on the distribution of length of stay in the first time period, we created an indicator variable for extended length of stay with length of stay at or above the 75th percentile, defined as extended. For each hospital at each 6-month interval, we then computed risk-standardized guideline-concordant treatment (RS-treatment) rates and risk-standardized in-hospital outcome rates similar to methods used by the Centers for Medicare and Medicaid Services for public reporting.⁸ For each hospital at each time interval, we estimated a predicted rate of guideline-concordant treatment as the sum of predicted probabilities of guideline-concordant treatment from patient factors and the random intercept for the hospital in which they were admitted. We then calculated the expected rate of guideline-concordant treatment as the sum of expected probabilities of treatment received from patient factors only. RS-treatment was then calculated as the ratio of predicted to expected rates multiplied by the overall unadjusted mean treatment rate from all patients.⁹ We repeated the same modeling strategy to calculate risk-standardized outcome (RS-outcome) rates for each hospital across all time points. All models were adjusted for patient demographics and comorbidities. Similar models using administrative data have moderate discrimination for mortality.¹⁰

We then fit mixed-effects linear models with random hospital intercept and slope across time for the RS-treatment and outcome rates, respectively. From these models, we estimated the mean slope for RS-treatment and for RS-outcome over time. In addition, we estimated a slope or trend over time for each hospital for treatment and for outcome and evaluated the correlation between the treatment and outcome trends.

All analyses were performed using the Statistical Analysis System version 9.4 (SAS Institute Inc., Cary, NC) and STATA release 13 (StataCorp, LLC, College Station, Texas).

In this large, retrospective cohort study, we found that there was a substantial gap between the empiric antibiotics recommended by the ATS and IDSA guidelines and the empiric antibiotics that patients actually received. Over the study period, we saw an increased adherence to guidelines, in spite of growing evidence that HCAP risk factors do not adequately predict which patients are at risk for infection with an MDRO.¹¹ We used this change in antibiotic prescribing behavior over time to determine if there was a clinical impact on patient outcomes and found that at the hospital level, there were no improvements in mortality, excess length of stay, or progression to respiratory failure despite a doubling in guideline-concordant antibiotic use.

At least 2 other large studies have assessed the association between guideline-concordant therapy and outcomes in HCAP.^12,13 Both found that guideline-concordant therapy was associated with increased mortality, despite propensity matching. Both were conducted at the individual patient level by using administrative data, and results were likely affected by unmeasured clinical confounders, with sicker patients being more likely to receive guideline-concordant therapy. Our focus on the outcomes at the hospital level avoids this selection bias because the overall severity of illness of patients at any given hospital would not be expected to change over the study period, while physician uptake of antibiotic prescribing guidelines would be expected to increase over time. Determining the correlation between increases in guideline adherence and changes in patient outcome may offer a better assessment of the impact of guideline adherence. In this regard, our results are similar to those achieved by 1 quality improvement collaborative that was aimed at increasing guideline concordant therapy in ICUs. Despite an increase in guideline concordance from 33% to 47% of patients, they found no change in overall mortality.¹⁴

There were several limitations to our study. We did not have access to microbiologic data, so we were unable to determine which patients had MDRO infection or determine antibiotic-pathogen matching. However, the treating physicians in our study population presumably did not have access to this data at the time of treatment either because the time period we examined was within the first 48 hours of hospitalization, the interval during which cultures are incubating and the patients are being treated empirically. In addition, there may have been HCAP patients that we failed to identify, such as patients who were admitted in the past 90 days to a hospital that does not submit data to Premier. However, it is unlikely that prescribing for such patients should differ systematically from what we observed. While the database draws from 488 hospitals nationwide, it is possible that practices may be different at facilities that are not contained within the Premier database, such as Veterans Administration Hospitals. Similarly, we did not have readings for chest x-rays; hence, there could be some patients in the dataset who did not have pneumonia. However, we tried to overcome this by including only those patients with a principal diagnosis of pneumonia or sepsis with a secondary pneumonia diagnosis, a chest x-ray, and antibiotics administered within the first 48 hours of admission.

There are likely several reasons why so few HCAP patients in our study received guideline-concordant antibiotics. A lack of knowledge about the ATS and IDSA guidelines may have impacted the physicians in our study population. El-Solh et al.¹⁵ surveyed physicians about the ATS-IDSA guidelines 4 years after publication and found that only 45% were familiar with the document. We found that the rate of prescribing at least partially guideline-concordant antibiotics rose steadily over time, supporting the idea that the newness of the guidelines was 1 barrier. Additionally, prior studies have shown that many physicians may not agree with or choose to follow guidelines, with only 20% of physicians indicating that guidelines have a major impact on their clinical decision making,¹⁶ and the majority do not choose HCAP guideline-concordant antibiotics when tested.¹⁷ Alternatively, clinicians may not follow the guidelines because of a belief that the HCAP criteria do not adequately indicate patients who are at risk for MDRO. Previous studies have demonstrated the relative inability of HCAP risk factors to predict patients who harbor MDRO¹⁸ and suggest that better tools such as clinical scoring systems, which include not only the traditional HCAP risk factors but also prior exposure to antibiotics, prior culture data, and a cumulative assessment of both intrinsic and extrinsic factors, could more accurately predict MDRO and lead to a more judicious use of broad-spectrum antimicrobial agents.^19-25 Indeed, these collective findings have led the authors of the recently updated guidelines to remove HCAP as a clinical entity from the hospital-acquired or ventilator-associated pneumonia guidelines and place them instead in the upcoming updated guidelines on the management of CAP.⁵ Of these 3 explanations, the lack of familiarity fits best with our observation that guideline-concordant therapy increased steadily over time with no evidence of reaching a plateau. Ironically, as consensus was building that HCAP is a poor marker for MDROs, routine empiric treatment with vancomycin and piperacillin-tazobactam (“vanco and zosyn”) have become routine in many hospitals. Additional studies are needed to know if this trend has stabilized or reversed.

In conclusion, clinicians in our large, nationally representative sample treated the majority of HCAP patients as though they had CAP. Although there was an increase in the administration of guideline-concordant therapy over time, this increase was not associated with improved outcomes. This study supports the growing consensus that HCAP criteria do not accurately predict which patients benefit from broad-spectrum antibiotics for pneumonia, and most patients fare well with antibiotics targeting common community-acquired organisms.

Disclosure

 This work was supported by grant # R01HS018723 from the Agency for Healthcare Research and Quality. Dr. Lagu is also supported by the National Heart, Lung, and Blood Institute of the National Institutes of Health under award number K01HL114745. Dr. Lindenauer is supported by grant K24HL132008 from the National Heart, Lung, and Blood Institute. The funding agency had no role in the data acquisition, analysis, or manuscript preparation for this study. Drs. Haessler and Rothberg had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs. Haessler, Lagu, Lindenauer, Skiest, Zilberberg, Higgins, and Rothberg conceived of the study and analyzed and interpreted the data. Dr. Lindenauer acquired the data. Dr. Pekow and Ms. Priya carried out the statistical analyses. Dr. Haessler drafted the manuscript. All authors critically reviewed the manuscript for accuracy and integrity. All authors certify no potential conflicts of interest. Preliminary results from this study were presented in oral and poster format at IDWeek in 2012 and 2013.

Bacterial pneumonia remains an important cause of morbidity and mortality in the United States, and is the 8^th leading cause of death with 55,227 deaths among adults annually.¹ In 2005, the American Thoracic Society (ATS) and the Infectious Diseases Society of America (IDSA) collaborated to update guidelines for hospital-acquired pneumonia (HAP), ventilator-associated pneumonia, and healthcare-associated pneumonia (HCAP).² This broad document outlines an evidence-based approach to diagnostic testing and antibiotic management based on the epidemiology and risk factors for these conditions. The guideline specifies the following criteria for HCAP: hospitalization in the past 90 days, residence in a skilled nursing facility (SNF), home infusion therapy, hemodialysis, home wound care, family members with multidrug resistant organisms (MDRO), and immunosuppressive diseases or medications, with the presumption that these patients are more likely to be harboring MDRO and should thus be treated empirically with broad-spectrum antibiotic therapy. Prior studies have shown that patients with HCAP have a more severe illness, are more likely to have MDRO, are more likely to be inadequately treated, and are at a higher risk for mortality than patients with community-acquired pneumonia (CAP).^3,4

These guidelines are controversial, especially in regard to the recommendations to empirically treat broadly with 2 antibiotics targeting Pseudomonas species, whether patients with HCAP merit broader spectrum coverage than patients with CAP, and whether the criteria for defining HCAP are adequate to predict which patients are harboring MDRO. It has subsequently been proposed that HCAP is more related to CAP than to HAP, and a recent update to the guideline removed recommendations for treatment of HCAP and will be placing HCAP into the guidelines for CAP instead.⁵ We sought to investigate the degree of uptake of the ATS and IDSA guideline recommendations by physicians over time, and whether this led to a change in outcomes among patients who met the criteria for HCAP.

Setting and Patients

We identified patients discharged between July 1, 2007, and November 30, 2011, from 488 US hospitals that participated in the Premier database (Premier Inc., Charlotte, North Carolina), an inpatient database developed for measuring quality and healthcare utilization. The database is frequently used for healthcare research and has been described previously.⁶ Member hospitals are in all regions of the US and are generally reflective of US hospitals. This database contains multiple data elements, including sociodemographic information, International Classification of Diseases, 9th Revision-Clinical Modification (ICD-9-CM) diagnosis and procedure codes, hospital and physician information, source of admission, and discharge status. It also includes a date-stamped log of all billed items and services, including diagnostic tests, medications, and other treatments. Because the data do not contain identifiable information, the institutional review board at our medical center determined that this study did not constitute human subjects research.

We included all patients aged ≥18 years with a principal diagnosis of pneumonia or with a secondary diagnosis of pneumonia paired with a principal diagnosis of respiratory failure, acute respiratory distress syndrome, respiratory arrest, sepsis, or influenza. Patients were excluded if they were transferred to or from another acute care institution, had a length of stay of 1 day or less, had cystic fibrosis, did not have a chest radiograph, or did not receive antibiotics within 48 hours of admission.

For each patient, we extracted age, gender, principal diagnosis, comorbidities, and the specialty of the attending physician. Comorbidities were identified from ICD-9-CM secondary diagnosis codes and Diagnosis Related Groups by using Healthcare Cost and Utilization Project Comorbidity Software, version 3.1, based on the work of Elixhauser (Agency for Healthcare Research and Quality, Rockville, Maryland).⁷ In order to ensure that patients had HCAP, we required the presence of ≥1 HCAP criteria, including hospitalization in the past 90 days, hemodialysis, admission from an SNF, or immune suppression (which was derived from either a secondary diagnosis for neutropenia, hematological malignancy, organ transplant, acquired immunodeficiency virus, or receiving immunosuppressant drugs or corticosteroids [equivalent to ≥20 mg/day of prednisone]).

Definitions of Guideline-Concordant and Discordant Antibiotic Therapy

The ATS and IDSA guidelines recommended the following antibiotic combinations for HCAP: an antipseudomonal cephalosporin or carbapenem or a beta-lactam/lactamase inhibitor, plus an antipseudomonal quinolone or aminoglycoside, plus an antibiotic with activity versus methicillin resistant Staphylococcus aureus (MRSA), such as vancomycin or linezolid. Based on these guidelines, we defined the receipt of fully guideline-concordant antibiotics as 2 recommended antibiotics for Pseudomonas species plus 1 for MRSA administered by the second day of admission. Partially guideline-concordant antibiotics were defined as 1 recommended antibiotic for Pseudomonas species plus 1 for MRSA by the second day of hospitalization. Guideline-discordant antibiotics were defined as all other combinations.

Statistical Analysis

Descriptive statistics on patient characteristics are presented as frequency, proportions for categorical factors, and median with interquartile range (IQR) for continuous variables for the full cohort and by treatment group, defined as fully or partially guideline-concordant antibiotic therapy or discordant therapy. Hospital rates of fully guideline-concordant treatment are presented overall and by hospital characteristics. The association of hospital characteristics with rates of fully guideline-concordant therapy were assessed by using 1-way analysis of variance tests.

To assess trends across hospitals for the association between the use of guideline-concordant therapy and mortality, progression to respiratory failure as measured by the late initiation of invasive mechanical ventilation (day 3 or later), and the length of stay among survivors, we divided the 4.5-year study period into 9 intervals of 6 months each; 292 hospitals that submitted data for all 9 time points were examined in this analysis. Based on the distribution of length of stay in the first time period, we created an indicator variable for extended length of stay with length of stay at or above the 75th percentile, defined as extended. For each hospital at each 6-month interval, we then computed risk-standardized guideline-concordant treatment (RS-treatment) rates and risk-standardized in-hospital outcome rates similar to methods used by the Centers for Medicare and Medicaid Services for public reporting.⁸ For each hospital at each time interval, we estimated a predicted rate of guideline-concordant treatment as the sum of predicted probabilities of guideline-concordant treatment from patient factors and the random intercept for the hospital in which they were admitted. We then calculated the expected rate of guideline-concordant treatment as the sum of expected probabilities of treatment received from patient factors only. RS-treatment was then calculated as the ratio of predicted to expected rates multiplied by the overall unadjusted mean treatment rate from all patients.⁹ We repeated the same modeling strategy to calculate risk-standardized outcome (RS-outcome) rates for each hospital across all time points. All models were adjusted for patient demographics and comorbidities. Similar models using administrative data have moderate discrimination for mortality.¹⁰

We then fit mixed-effects linear models with random hospital intercept and slope across time for the RS-treatment and outcome rates, respectively. From these models, we estimated the mean slope for RS-treatment and for RS-outcome over time. In addition, we estimated a slope or trend over time for each hospital for treatment and for outcome and evaluated the correlation between the treatment and outcome trends.

All analyses were performed using the Statistical Analysis System version 9.4 (SAS Institute Inc., Cary, NC) and STATA release 13 (StataCorp, LLC, College Station, Texas).

In this large, retrospective cohort study, we found that there was a substantial gap between the empiric antibiotics recommended by the ATS and IDSA guidelines and the empiric antibiotics that patients actually received. Over the study period, we saw an increased adherence to guidelines, in spite of growing evidence that HCAP risk factors do not adequately predict which patients are at risk for infection with an MDRO.¹¹ We used this change in antibiotic prescribing behavior over time to determine if there was a clinical impact on patient outcomes and found that at the hospital level, there were no improvements in mortality, excess length of stay, or progression to respiratory failure despite a doubling in guideline-concordant antibiotic use.

At least 2 other large studies have assessed the association between guideline-concordant therapy and outcomes in HCAP.^12,13 Both found that guideline-concordant therapy was associated with increased mortality, despite propensity matching. Both were conducted at the individual patient level by using administrative data, and results were likely affected by unmeasured clinical confounders, with sicker patients being more likely to receive guideline-concordant therapy. Our focus on the outcomes at the hospital level avoids this selection bias because the overall severity of illness of patients at any given hospital would not be expected to change over the study period, while physician uptake of antibiotic prescribing guidelines would be expected to increase over time. Determining the correlation between increases in guideline adherence and changes in patient outcome may offer a better assessment of the impact of guideline adherence. In this regard, our results are similar to those achieved by 1 quality improvement collaborative that was aimed at increasing guideline concordant therapy in ICUs. Despite an increase in guideline concordance from 33% to 47% of patients, they found no change in overall mortality.¹⁴

There were several limitations to our study. We did not have access to microbiologic data, so we were unable to determine which patients had MDRO infection or determine antibiotic-pathogen matching. However, the treating physicians in our study population presumably did not have access to this data at the time of treatment either because the time period we examined was within the first 48 hours of hospitalization, the interval during which cultures are incubating and the patients are being treated empirically. In addition, there may have been HCAP patients that we failed to identify, such as patients who were admitted in the past 90 days to a hospital that does not submit data to Premier. However, it is unlikely that prescribing for such patients should differ systematically from what we observed. While the database draws from 488 hospitals nationwide, it is possible that practices may be different at facilities that are not contained within the Premier database, such as Veterans Administration Hospitals. Similarly, we did not have readings for chest x-rays; hence, there could be some patients in the dataset who did not have pneumonia. However, we tried to overcome this by including only those patients with a principal diagnosis of pneumonia or sepsis with a secondary pneumonia diagnosis, a chest x-ray, and antibiotics administered within the first 48 hours of admission.

There are likely several reasons why so few HCAP patients in our study received guideline-concordant antibiotics. A lack of knowledge about the ATS and IDSA guidelines may have impacted the physicians in our study population. El-Solh et al.¹⁵ surveyed physicians about the ATS-IDSA guidelines 4 years after publication and found that only 45% were familiar with the document. We found that the rate of prescribing at least partially guideline-concordant antibiotics rose steadily over time, supporting the idea that the newness of the guidelines was 1 barrier. Additionally, prior studies have shown that many physicians may not agree with or choose to follow guidelines, with only 20% of physicians indicating that guidelines have a major impact on their clinical decision making,¹⁶ and the majority do not choose HCAP guideline-concordant antibiotics when tested.¹⁷ Alternatively, clinicians may not follow the guidelines because of a belief that the HCAP criteria do not adequately indicate patients who are at risk for MDRO. Previous studies have demonstrated the relative inability of HCAP risk factors to predict patients who harbor MDRO¹⁸ and suggest that better tools such as clinical scoring systems, which include not only the traditional HCAP risk factors but also prior exposure to antibiotics, prior culture data, and a cumulative assessment of both intrinsic and extrinsic factors, could more accurately predict MDRO and lead to a more judicious use of broad-spectrum antimicrobial agents.^19-25 Indeed, these collective findings have led the authors of the recently updated guidelines to remove HCAP as a clinical entity from the hospital-acquired or ventilator-associated pneumonia guidelines and place them instead in the upcoming updated guidelines on the management of CAP.⁵ Of these 3 explanations, the lack of familiarity fits best with our observation that guideline-concordant therapy increased steadily over time with no evidence of reaching a plateau. Ironically, as consensus was building that HCAP is a poor marker for MDROs, routine empiric treatment with vancomycin and piperacillin-tazobactam (“vanco and zosyn”) have become routine in many hospitals. Additional studies are needed to know if this trend has stabilized or reversed.

In conclusion, clinicians in our large, nationally representative sample treated the majority of HCAP patients as though they had CAP. Although there was an increase in the administration of guideline-concordant therapy over time, this increase was not associated with improved outcomes. This study supports the growing consensus that HCAP criteria do not accurately predict which patients benefit from broad-spectrum antibiotics for pneumonia, and most patients fare well with antibiotics targeting common community-acquired organisms.

Disclosure

 This work was supported by grant # R01HS018723 from the Agency for Healthcare Research and Quality. Dr. Lagu is also supported by the National Heart, Lung, and Blood Institute of the National Institutes of Health under award number K01HL114745. Dr. Lindenauer is supported by grant K24HL132008 from the National Heart, Lung, and Blood Institute. The funding agency had no role in the data acquisition, analysis, or manuscript preparation for this study. Drs. Haessler and Rothberg had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs. Haessler, Lagu, Lindenauer, Skiest, Zilberberg, Higgins, and Rothberg conceived of the study and analyzed and interpreted the data. Dr. Lindenauer acquired the data. Dr. Pekow and Ms. Priya carried out the statistical analyses. Dr. Haessler drafted the manuscript. All authors critically reviewed the manuscript for accuracy and integrity. All authors certify no potential conflicts of interest. Preliminary results from this study were presented in oral and poster format at IDWeek in 2012 and 2013.

Bacterial pneumonia remains an important cause of morbidity and mortality in the United States, and is the 8^th leading cause of death with 55,227 deaths among adults annually.¹ In 2005, the American Thoracic Society (ATS) and the Infectious Diseases Society of America (IDSA) collaborated to update guidelines for hospital-acquired pneumonia (HAP), ventilator-associated pneumonia, and healthcare-associated pneumonia (HCAP).² This broad document outlines an evidence-based approach to diagnostic testing and antibiotic management based on the epidemiology and risk factors for these conditions. The guideline specifies the following criteria for HCAP: hospitalization in the past 90 days, residence in a skilled nursing facility (SNF), home infusion therapy, hemodialysis, home wound care, family members with multidrug resistant organisms (MDRO), and immunosuppressive diseases or medications, with the presumption that these patients are more likely to be harboring MDRO and should thus be treated empirically with broad-spectrum antibiotic therapy. Prior studies have shown that patients with HCAP have a more severe illness, are more likely to have MDRO, are more likely to be inadequately treated, and are at a higher risk for mortality than patients with community-acquired pneumonia (CAP).^3,4

These guidelines are controversial, especially in regard to the recommendations to empirically treat broadly with 2 antibiotics targeting Pseudomonas species, whether patients with HCAP merit broader spectrum coverage than patients with CAP, and whether the criteria for defining HCAP are adequate to predict which patients are harboring MDRO. It has subsequently been proposed that HCAP is more related to CAP than to HAP, and a recent update to the guideline removed recommendations for treatment of HCAP and will be placing HCAP into the guidelines for CAP instead.⁵ We sought to investigate the degree of uptake of the ATS and IDSA guideline recommendations by physicians over time, and whether this led to a change in outcomes among patients who met the criteria for HCAP.

Setting and Patients

We identified patients discharged between July 1, 2007, and November 30, 2011, from 488 US hospitals that participated in the Premier database (Premier Inc., Charlotte, North Carolina), an inpatient database developed for measuring quality and healthcare utilization. The database is frequently used for healthcare research and has been described previously.⁶ Member hospitals are in all regions of the US and are generally reflective of US hospitals. This database contains multiple data elements, including sociodemographic information, International Classification of Diseases, 9th Revision-Clinical Modification (ICD-9-CM) diagnosis and procedure codes, hospital and physician information, source of admission, and discharge status. It also includes a date-stamped log of all billed items and services, including diagnostic tests, medications, and other treatments. Because the data do not contain identifiable information, the institutional review board at our medical center determined that this study did not constitute human subjects research.

We included all patients aged ≥18 years with a principal diagnosis of pneumonia or with a secondary diagnosis of pneumonia paired with a principal diagnosis of respiratory failure, acute respiratory distress syndrome, respiratory arrest, sepsis, or influenza. Patients were excluded if they were transferred to or from another acute care institution, had a length of stay of 1 day or less, had cystic fibrosis, did not have a chest radiograph, or did not receive antibiotics within 48 hours of admission.

For each patient, we extracted age, gender, principal diagnosis, comorbidities, and the specialty of the attending physician. Comorbidities were identified from ICD-9-CM secondary diagnosis codes and Diagnosis Related Groups by using Healthcare Cost and Utilization Project Comorbidity Software, version 3.1, based on the work of Elixhauser (Agency for Healthcare Research and Quality, Rockville, Maryland).⁷ In order to ensure that patients had HCAP, we required the presence of ≥1 HCAP criteria, including hospitalization in the past 90 days, hemodialysis, admission from an SNF, or immune suppression (which was derived from either a secondary diagnosis for neutropenia, hematological malignancy, organ transplant, acquired immunodeficiency virus, or receiving immunosuppressant drugs or corticosteroids [equivalent to ≥20 mg/day of prednisone]).

Definitions of Guideline-Concordant and Discordant Antibiotic Therapy

The ATS and IDSA guidelines recommended the following antibiotic combinations for HCAP: an antipseudomonal cephalosporin or carbapenem or a beta-lactam/lactamase inhibitor, plus an antipseudomonal quinolone or aminoglycoside, plus an antibiotic with activity versus methicillin resistant Staphylococcus aureus (MRSA), such as vancomycin or linezolid. Based on these guidelines, we defined the receipt of fully guideline-concordant antibiotics as 2 recommended antibiotics for Pseudomonas species plus 1 for MRSA administered by the second day of admission. Partially guideline-concordant antibiotics were defined as 1 recommended antibiotic for Pseudomonas species plus 1 for MRSA by the second day of hospitalization. Guideline-discordant antibiotics were defined as all other combinations.

Statistical Analysis

Descriptive statistics on patient characteristics are presented as frequency, proportions for categorical factors, and median with interquartile range (IQR) for continuous variables for the full cohort and by treatment group, defined as fully or partially guideline-concordant antibiotic therapy or discordant therapy. Hospital rates of fully guideline-concordant treatment are presented overall and by hospital characteristics. The association of hospital characteristics with rates of fully guideline-concordant therapy were assessed by using 1-way analysis of variance tests.

To assess trends across hospitals for the association between the use of guideline-concordant therapy and mortality, progression to respiratory failure as measured by the late initiation of invasive mechanical ventilation (day 3 or later), and the length of stay among survivors, we divided the 4.5-year study period into 9 intervals of 6 months each; 292 hospitals that submitted data for all 9 time points were examined in this analysis. Based on the distribution of length of stay in the first time period, we created an indicator variable for extended length of stay with length of stay at or above the 75th percentile, defined as extended. For each hospital at each 6-month interval, we then computed risk-standardized guideline-concordant treatment (RS-treatment) rates and risk-standardized in-hospital outcome rates similar to methods used by the Centers for Medicare and Medicaid Services for public reporting.⁸ For each hospital at each time interval, we estimated a predicted rate of guideline-concordant treatment as the sum of predicted probabilities of guideline-concordant treatment from patient factors and the random intercept for the hospital in which they were admitted. We then calculated the expected rate of guideline-concordant treatment as the sum of expected probabilities of treatment received from patient factors only. RS-treatment was then calculated as the ratio of predicted to expected rates multiplied by the overall unadjusted mean treatment rate from all patients.⁹ We repeated the same modeling strategy to calculate risk-standardized outcome (RS-outcome) rates for each hospital across all time points. All models were adjusted for patient demographics and comorbidities. Similar models using administrative data have moderate discrimination for mortality.¹⁰

We then fit mixed-effects linear models with random hospital intercept and slope across time for the RS-treatment and outcome rates, respectively. From these models, we estimated the mean slope for RS-treatment and for RS-outcome over time. In addition, we estimated a slope or trend over time for each hospital for treatment and for outcome and evaluated the correlation between the treatment and outcome trends.

All analyses were performed using the Statistical Analysis System version 9.4 (SAS Institute Inc., Cary, NC) and STATA release 13 (StataCorp, LLC, College Station, Texas).

In this large, retrospective cohort study, we found that there was a substantial gap between the empiric antibiotics recommended by the ATS and IDSA guidelines and the empiric antibiotics that patients actually received. Over the study period, we saw an increased adherence to guidelines, in spite of growing evidence that HCAP risk factors do not adequately predict which patients are at risk for infection with an MDRO.¹¹ We used this change in antibiotic prescribing behavior over time to determine if there was a clinical impact on patient outcomes and found that at the hospital level, there were no improvements in mortality, excess length of stay, or progression to respiratory failure despite a doubling in guideline-concordant antibiotic use.

At least 2 other large studies have assessed the association between guideline-concordant therapy and outcomes in HCAP.^12,13 Both found that guideline-concordant therapy was associated with increased mortality, despite propensity matching. Both were conducted at the individual patient level by using administrative data, and results were likely affected by unmeasured clinical confounders, with sicker patients being more likely to receive guideline-concordant therapy. Our focus on the outcomes at the hospital level avoids this selection bias because the overall severity of illness of patients at any given hospital would not be expected to change over the study period, while physician uptake of antibiotic prescribing guidelines would be expected to increase over time. Determining the correlation between increases in guideline adherence and changes in patient outcome may offer a better assessment of the impact of guideline adherence. In this regard, our results are similar to those achieved by 1 quality improvement collaborative that was aimed at increasing guideline concordant therapy in ICUs. Despite an increase in guideline concordance from 33% to 47% of patients, they found no change in overall mortality.¹⁴

There were several limitations to our study. We did not have access to microbiologic data, so we were unable to determine which patients had MDRO infection or determine antibiotic-pathogen matching. However, the treating physicians in our study population presumably did not have access to this data at the time of treatment either because the time period we examined was within the first 48 hours of hospitalization, the interval during which cultures are incubating and the patients are being treated empirically. In addition, there may have been HCAP patients that we failed to identify, such as patients who were admitted in the past 90 days to a hospital that does not submit data to Premier. However, it is unlikely that prescribing for such patients should differ systematically from what we observed. While the database draws from 488 hospitals nationwide, it is possible that practices may be different at facilities that are not contained within the Premier database, such as Veterans Administration Hospitals. Similarly, we did not have readings for chest x-rays; hence, there could be some patients in the dataset who did not have pneumonia. However, we tried to overcome this by including only those patients with a principal diagnosis of pneumonia or sepsis with a secondary pneumonia diagnosis, a chest x-ray, and antibiotics administered within the first 48 hours of admission.

There are likely several reasons why so few HCAP patients in our study received guideline-concordant antibiotics. A lack of knowledge about the ATS and IDSA guidelines may have impacted the physicians in our study population. El-Solh et al.¹⁵ surveyed physicians about the ATS-IDSA guidelines 4 years after publication and found that only 45% were familiar with the document. We found that the rate of prescribing at least partially guideline-concordant antibiotics rose steadily over time, supporting the idea that the newness of the guidelines was 1 barrier. Additionally, prior studies have shown that many physicians may not agree with or choose to follow guidelines, with only 20% of physicians indicating that guidelines have a major impact on their clinical decision making,¹⁶ and the majority do not choose HCAP guideline-concordant antibiotics when tested.¹⁷ Alternatively, clinicians may not follow the guidelines because of a belief that the HCAP criteria do not adequately indicate patients who are at risk for MDRO. Previous studies have demonstrated the relative inability of HCAP risk factors to predict patients who harbor MDRO¹⁸ and suggest that better tools such as clinical scoring systems, which include not only the traditional HCAP risk factors but also prior exposure to antibiotics, prior culture data, and a cumulative assessment of both intrinsic and extrinsic factors, could more accurately predict MDRO and lead to a more judicious use of broad-spectrum antimicrobial agents.^19-25 Indeed, these collective findings have led the authors of the recently updated guidelines to remove HCAP as a clinical entity from the hospital-acquired or ventilator-associated pneumonia guidelines and place them instead in the upcoming updated guidelines on the management of CAP.⁵ Of these 3 explanations, the lack of familiarity fits best with our observation that guideline-concordant therapy increased steadily over time with no evidence of reaching a plateau. Ironically, as consensus was building that HCAP is a poor marker for MDROs, routine empiric treatment with vancomycin and piperacillin-tazobactam (“vanco and zosyn”) have become routine in many hospitals. Additional studies are needed to know if this trend has stabilized or reversed.

In conclusion, clinicians in our large, nationally representative sample treated the majority of HCAP patients as though they had CAP. Although there was an increase in the administration of guideline-concordant therapy over time, this increase was not associated with improved outcomes. This study supports the growing consensus that HCAP criteria do not accurately predict which patients benefit from broad-spectrum antibiotics for pneumonia, and most patients fare well with antibiotics targeting common community-acquired organisms.

Disclosure

 This work was supported by grant # R01HS018723 from the Agency for Healthcare Research and Quality. Dr. Lagu is also supported by the National Heart, Lung, and Blood Institute of the National Institutes of Health under award number K01HL114745. Dr. Lindenauer is supported by grant K24HL132008 from the National Heart, Lung, and Blood Institute. The funding agency had no role in the data acquisition, analysis, or manuscript preparation for this study. Drs. Haessler and Rothberg had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs. Haessler, Lagu, Lindenauer, Skiest, Zilberberg, Higgins, and Rothberg conceived of the study and analyzed and interpreted the data. Dr. Lindenauer acquired the data. Dr. Pekow and Ms. Priya carried out the statistical analyses. Dr. Haessler drafted the manuscript. All authors critically reviewed the manuscript for accuracy and integrity. All authors certify no potential conflicts of interest. Preliminary results from this study were presented in oral and poster format at IDWeek in 2012 and 2013.