User login
Keeping It Simple in Sepsis Measures
“I didn’t have time to write a short letter, so I wrote a long one instead.”
-Mark Twain
Sepsis is a logical target for quality measures. Specifically, sepsis represents the perfect storm of immense public health burden1-3 combined with unexplained practice4-6 and outcomes7 variation. Thus, it is not surprising that in October 2015, the Centers of Medicare and Medicaid Services (CMS) adopted a sepsis quality measure.8 More surprising were the complex contents of the CMS Sepsis Core Measure “SEP-1” quality measure.9 CMS had written a “long letter.”
The multiple processes targeted with the CMS SEP-1 quality measure can best be understood with a brief account of history. SEP-1 arose from the National Quality Forum’s (NQF) project #0500: “Severe Sepsis and Septic Shock: Management Bundle,” a measure based upon Rivers et al.’s10 single-center, randomized, controlled trial of early goal-directed therapy (EGDT) for severe sepsis. EGDT was an intervention that consisted of fluid resuscitation and hemodynamic management based upon fulfilling specific targets of central venous pressure, superior vena cava oxygen saturation (or lactic acid), and hemoglobin and mean arterial pressures.11 The large mortality benefits, physiological rationale, and algorithmic responses to a variety of abnormal clinical values provided an appealing protocol to critical care and emergency physicians trained to normalize measured values, as well as policy makers looking for quality measures. Observational studies consistently showed associations between adoption of guideline-based “sepsis bundles” and improved patient outcomes,12-14 setting the stage for the transition of NQF #0500 into SEP-1.
However, the transition from EGDT-based NQF #0500 to SEP-1 has been tumultuous. Soon after adoption of SEP-1, the consensus definitions of sepsis changed markedly. Sepsis went from being defined as the presence of infection with concomitant systemic inflammatory response syndrome (sepsis), organ dysfunction (severe sepsis), and/or shock,15 to being defined as a dysregulated response to infection resulting in life-threatening organ dysfunction (sepsis) and/or fluid-resistant hypotension requiring vasopressors and lactate greater than 2 mmol/L.16 As the study by Barbash et al.17
In addition to its unprecedented complexity, SEP-1 received criticism for the weak evidence base of its individual components. The general concepts behind SEP-1 are well-accepted tenets of sepsis management: rapid recognition, assessment and treatment of underlying infection, and institution of intravenous fluids and vasopressor support for septic shock. However, the “all or none” prescriptive nature of the SEP-1 bundle was based on a somewhat arbitrary set of measures and targets. For example, patients with septic shock must receive 30 cc/kg of intravenous fluids to be “SEP-1 compliant.” The value “30 cc/kg” was taken from the average volume of fluids reported in prior sepsis trials, essentially based on a very low level of evidence.20 The strict 30 cc/kg cutoff did not take into account that “the median isn’t the message”21 in fluid management: optimal resuscitation targets are unclear,22 and selecting the median as a target ignores the fact that 50% of patients enrolled in international trials of EGDT received less than 30 cc/kg of initial fluid resuscitation (the interquartile range was 16-42 cc/kg).18 Thus, most participants in trials upon which the SEP-1 fluid measure was based would ironically not have met the SEP-1 measure. Mandates for physical exam and physiological measures were based on similarly low levels of evidence.
Into this context, Barbash et al. use a representative sample of US hospitals to explore the opinions of hospital quality leaders regarding the SEP-1 measure. First, the qualitative methods used by Barbash et al. warrant some explanation. Much of biomedical research is characterized by hypothesis-driven, deductive reasoning: theories are tested using observations. In contrast, the methods of Barbash et al. use inductive reasoning: observations are used to develop theories within a systematic approach called “grounded theory” that explores common themes emerging from structured interviews.23 Inductive reasoning can later inform deductive reasoning, feeding theories into testable hypotheses. However, qualitative, inductive research is not meant to test hypotheses and is not subject to typical notions of “power and sample size” often expected of quantitative statistical analyses. Qualitative studies reach sufficient sample size when no further themes emerge, a situation called “thematic saturation”; the sample size here of 29 participants rests comfortably in the range of participants commonly needed for thematic saturation.23
Barbash et al. identified common themes in opinions of quality leaders regarding SEP-1. Namely, the complexity of SEP-1 necessitated a major resource investment into sepsis care and data collection. The major infrastructure investments needed to comply with SEP-1 also bred frustration regarding lack of perceived fairness around the “all or none” nature of the measure and raised multiple additional challenges including lack of clinician buy-in and resistance to protocolized care. Prior qualitative studies evaluating hospital quality leaders’ opinions on performance measures identified similar concerns about lack of “fairness,”24 but the implementation of SEP-1 has raised additional concern regarding the large burdens of instituting major infrastructure changes to monitor processes of care required to report on this measure. Despite the major challenges of responding to SEP-1, quality leaders expressed optimism that increased attention to sepsis would ultimately lead to better patient outcomes.
How might future sepsis quality measures achieve the adequate balance between focusing attention on improving care processes for high-impact diseases, without imposing additional burdens on the healthcare system? Lessons from Barbash et al. help us move forward. First, rather than taxing hospitals with administratively complex process measures, initial attempts at quality measures should start simply. Policy makers should consider moving forward into new areas of quality measurement in 2 ways: (1) pursue 1 or 2 processes with strong etiological links to important patient outcomes (eg, timely antibiotics in septic shock),25-28 and/or (2) use risk-adjusted outcomes and allow individual hospitals to adopt processes that improve local patient outcomes. Evidence suggests that the introduction of a quality measure may result in improved outcomes regardless of adoption of specific target processes,29 although results are mixed.30,31 In either case, complex “all or none” measures based upon weak evidence run a high risk of inciting clinician resentment and paradoxically perpetuating poor quality by increasing healthcare costs (decreased efficiency) without gains in safety, effectiveness, timeliness, or equity.32 It has been estimated that hospitals spend on average $2 million to implement SEP-1,33 with unclear return on the investment. The experience of SEP-1 is a reminder that, as evidence evolves, quality measures must adapt lest they become irrelevant. However, it is also a reminder that quality measures should not sit precariously on the edge of evidence. Withdrawal of process-based measures due to a changing evidence landscape breeds mistrust and impairs future attempts to improve quality.
Sepsis quality measures face additional challenges. If recent experience with interpretation of sepsis definitions can serve as a guide, variable uptake of newer sepsis definitions between/across hospitals will impair the ability to risk-adjust outcome measures and increase bias in identifying outlier hospitals.34 In addition, recent studies have already raised skepticism regarding the effectiveness of individual SEP-1 bundle components, confirming suspicions that the 30 cc/kg fluid bolus is not a magic quality target. Rather, the effectiveness of prior sepsis bundles has likely been driven by improved time to antibiotics, a process unstudied in sepsis trials, but driven by increased attention to the importance of early sepsis recognition and timely management.28 Timeliness of antibiotics can act as an effect modifier for more complex sepsis therapies, with quicker time to antibiotics associated with reversal of previously described effectiveness of activated protein C,35 and EGDT.28
Sepsis has a legacy in which improving simple processes (ie, time to antibiotics) obviates the need for more complex interventions (eg, activated protein C, EGDT). To the extent that CMS remains committed to using process-based measures of quality, those focused on sepsis are likely to be most effective when pared down to the simplest and strongest evidence base—improved recognition36 and timely antibiotics (for patients with infection-induced organ dysfunction and shock). Taking the time to start simply may best serve our current patients and preserve stakeholder buy-in for quality initiatives likely to benefit our future patients.
Disclosure
Dr. Lindenauer reports that he received support from the Centers for Medicare and Medicaid Services to develop and maintain hospital outcome measures for pneumonia and COPD. Dr. Lindenauer is supported by grant K24HL132008 from the National Heart, Lung, and Blood Institute. Dr. Walkey was supported by grants K01-HL116768 and R01-HL139751 from the National Heart, Lung, and Blood Institute.
1. Elixhauser A, Friedman B, Stranges E. Septicemia in U.S. Hospitals, 2009. HCUP. Statistical Brief #122. Rockville MD: Agency for Healthcare Research and Quality; 2011; p 1-13. PubMed
2. Liu V, Lei X, Prescott HC, Kipnis P, Iwashyna TJ, Escobar GJ. Hospital readmission and healthcare utilization following sepsis in community settings. J Hosp Med. 2014;9(8):502-507. PubMed
3. Liu V, Escobar GJ, Greene JD, et al. Hospital deaths in patients with sepsis from 2 independent cohorts. JAMA. 2014;312(1):90-92. PubMed
4. Peltan ID, Mitchell KH, Rudd KE, et al. Physician Variation in Time to Antimicrobial Treatment for Septic Patients Presenting to the Emergency Department. Crit Care Med. 2017;45(6):1011-1018. PubMed
5. Marik PE, Linde-Zwirble WT, Bittner EA, Sahatjian J, Hansell D. Fluid administration in severe sepsis and septic shock, patterns and outcomes: an analysis of a large national database. Intensive Care Med. 2017;43(5):625-632. PubMed
6. Lagu T, Rothberg MB, Nathanson BH, Pekow PS, Steingrub JS, Lindenauer PK. Variation in the care of septic shock: the impact of patient and hospital characteristics. J Crit Care. 2012;27(4):329-336. PubMed
7. Wang HE, Donnelly JP, Shapiro NI, Hohmann SF, Levitan EB. Hospital variations in severe sepsis mortality. Am J Med Qual. 2015;30(4):328-336. PubMed
8. Centers for Medicare & Medicaid Services. CMS Measures Inventory. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityMeasures/CMS-Measures-Inventory.html. Accessed June 8, 2017.
9. QualityNet. Specifications Manual, Version 5.0b, Section 2.2. Severe Sepsis and Septic Shock. https://www.qualitynet.org/dcs/ContentServer?c=Page&pagename=QnetPublic%2FPage%2FQnetTier4&cid=1228774725171. Accessed June 8, 2017.
10. National Quality Forum. 0500 Severe sepsis and septic shock management bundle. http://www.qualityforum.org. Accessed June 8, 2017.
11. Rivers E, Nguyen B, Havstad S, et al. Early Goal-Directed Therapy in the Treatment of Severe Sepsis and Septic Shock. N Engl J Med. 2001;345:1368-1377. PubMed
12. Levy MM, Dellinger RP, Townsend SR, et al. The Surviving Sepsis Campaign: results of an international guideline-based performance improvement program targeting severe sepsis. Crit Care Med. 2010;38(2):367-374. PubMed
13. Levy MM, Artigas A, Phillips GS, et al. Outcomes of the Surviving Sepsis Campaign in intensive care units in the USA and Europe: a prospective cohort study. Lancet Infect Dis. 2012;12(12):919-924. PubMed
14. Ferrer R, Artigas A, Levy MM, et al. Improvement in process of care and outcome after a multicenter severe sepsis educational program in Spain. JAMA. 2008;299(19):2294-2303. PubMed
15. Bone RC, Balk RA, Cerra FB, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest. 1992;101(6):1644-1655. PubMed
16. Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):801-810. PubMed
17. Barbash IJ, Rak KJ, Kuza CC, Kahn JM. Hospital Perceptions of Medicare’s Sepsis Quality Reporting Initiative. J Hosp Med. 2017;12(12):963-967.
18. The PRISM Investigators. Early, Goal-Directed Therapy for Septic Shock — A Patient-Level Meta-Analysis. N Engl J Med. 2017;376:2223-2234. PubMed
19. National Quality Forum. NQF Revises Sepsis Measure. http://www.qualityforum.org/NQF_Revises_Sepsis_Measure.aspx. Accessed June 8, 2017.
20. Rhodes A, Evans LE, Alhazzani W, et al. Surviving Sepsis Campaign: International Guidelines for Management of Sepsis and Septic Shock: 2016. Intensive Care Med. 2017;43(3):304-377. PubMed
21. Gould SJ. The median isn’t the message. Discover. 1985;6:40-42. PubMed
22. Hernandez G, Teboul JL. Fourth Surviving Sepsis Campaign’s hemodynamic recommendations: a step forward or a return to chaos? Crit Care. 2017;21(1):133. PubMed
23. Fugard AJ, Potts HW. Supporting thinking on sample sizes for thematic analyses. Int J Soc Res Methodol. 2015;18(6):669-684.
24. Goff SL, Lagu T, Pekow PS, et al. A qualitative analysis of hospital leaders’ opinions about publicly reported measures of health care quality. Jt Comm J Qual Patient Saf. 2015;41(4):169-176. PubMed
25. Kumar A, Haery C, Paladugu B, et al. The duration of hypotension before the initiation of antibiotic treatment is a critical determinant of survival in a murine model of Escherichia coli septic shock: association with serum lactate and inflammatory cytokine levels. J Infect Dis. 2006;193(2):251-258.
PubMed
26. Liu VX, Fielding-Singh V, Greene JD, et al. The Timing of Early Antibiotics and Hospital Mortality in Sepsis. Am J Respir Crit Care Med. 2017. [Epub ahead of print]. PubMed
27. Seymour CW, Gesten F, Prescott HC, et al. Time to Treatment and Mortality during Mandated Emergency Care for Sepsis. N Engl J Med. 2017;376:2235-2244. PubMed
28. Kalil AC, Johnson DW, Lisco SJ, Sun J. Early Goal-Directed Therapy for Sepsis: A Novel Solution for Discordant Survival Outcomes in Clinical Trials. Crit Care Med. 2017;45(4):607-614. PubMed
29. Tu JV, Donovan LR, Lee DS, et al. Effectiveness of public report cards for improving the quality of cardiac care: the EFFECT study: a randomized trial. JAMA. 2009;302(21):2330-2337. PubMed
30. Joynt KE, Blumenthal DM, Orav EJ, Resnic FS, Jha AK. Association of public reporting for percutaneous coronary intervention with utilization and outcomes among Medicare beneficiaries with acute myocardial infarction. JAMA. 2012;308(14):1460-1468. PubMed
31. Osborne NH, Nicholas LH, Ryan AM, Thumma JR, Dimick JB. Association of hospital participation in a quality reporting program with surgical outcomes and expenditures for Medicare beneficiaries. JAMA. 2015;313(5):496-504. PubMed
32. Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington (DC): National Academies Press (US); 2001. PubMed
33. 2015;12(11):1676-1684.Ann Am Thorac Soc36. Kramer RD, Cooke CR, Liu V, Miller RR 3rd, Iwashyna TJ. Variation in the Contents of Sepsis Bundles and Quality Measures. A Systematic Review. PubMed
34. 2012;40(11):2974-2981.Crit Care Med35. Rimmer E, Kumar A, Doucette S, et al. Activated protein C and septic shock: a propensity-matched cohort study*. PubMed
35. 2014;160(6):380-388.Ann Intern Med34. Rothberg MB, Pekow PS, Priya A, Lindenauer PK. Variation in diagnostic coding of patients with pneumonia and its association with hospital risk-standardized mortality rates: a cross-sectional analysis. PubMed
36. 2015;12(11):1597-1599. Ann Am Thorac Soc33. Wall MJ, Howell MD. Variation and Cost-effectiveness of Quality Measurement Programs. The Case of Sepsis Bundles. PubMed
“I didn’t have time to write a short letter, so I wrote a long one instead.”
-Mark Twain
Sepsis is a logical target for quality measures. Specifically, sepsis represents the perfect storm of immense public health burden1-3 combined with unexplained practice4-6 and outcomes7 variation. Thus, it is not surprising that in October 2015, the Centers of Medicare and Medicaid Services (CMS) adopted a sepsis quality measure.8 More surprising were the complex contents of the CMS Sepsis Core Measure “SEP-1” quality measure.9 CMS had written a “long letter.”
The multiple processes targeted with the CMS SEP-1 quality measure can best be understood with a brief account of history. SEP-1 arose from the National Quality Forum’s (NQF) project #0500: “Severe Sepsis and Septic Shock: Management Bundle,” a measure based upon Rivers et al.’s10 single-center, randomized, controlled trial of early goal-directed therapy (EGDT) for severe sepsis. EGDT was an intervention that consisted of fluid resuscitation and hemodynamic management based upon fulfilling specific targets of central venous pressure, superior vena cava oxygen saturation (or lactic acid), and hemoglobin and mean arterial pressures.11 The large mortality benefits, physiological rationale, and algorithmic responses to a variety of abnormal clinical values provided an appealing protocol to critical care and emergency physicians trained to normalize measured values, as well as policy makers looking for quality measures. Observational studies consistently showed associations between adoption of guideline-based “sepsis bundles” and improved patient outcomes,12-14 setting the stage for the transition of NQF #0500 into SEP-1.
However, the transition from EGDT-based NQF #0500 to SEP-1 has been tumultuous. Soon after adoption of SEP-1, the consensus definitions of sepsis changed markedly. Sepsis went from being defined as the presence of infection with concomitant systemic inflammatory response syndrome (sepsis), organ dysfunction (severe sepsis), and/or shock,15 to being defined as a dysregulated response to infection resulting in life-threatening organ dysfunction (sepsis) and/or fluid-resistant hypotension requiring vasopressors and lactate greater than 2 mmol/L.16 As the study by Barbash et al.17
In addition to its unprecedented complexity, SEP-1 received criticism for the weak evidence base of its individual components. The general concepts behind SEP-1 are well-accepted tenets of sepsis management: rapid recognition, assessment and treatment of underlying infection, and institution of intravenous fluids and vasopressor support for septic shock. However, the “all or none” prescriptive nature of the SEP-1 bundle was based on a somewhat arbitrary set of measures and targets. For example, patients with septic shock must receive 30 cc/kg of intravenous fluids to be “SEP-1 compliant.” The value “30 cc/kg” was taken from the average volume of fluids reported in prior sepsis trials, essentially based on a very low level of evidence.20 The strict 30 cc/kg cutoff did not take into account that “the median isn’t the message”21 in fluid management: optimal resuscitation targets are unclear,22 and selecting the median as a target ignores the fact that 50% of patients enrolled in international trials of EGDT received less than 30 cc/kg of initial fluid resuscitation (the interquartile range was 16-42 cc/kg).18 Thus, most participants in trials upon which the SEP-1 fluid measure was based would ironically not have met the SEP-1 measure. Mandates for physical exam and physiological measures were based on similarly low levels of evidence.
Into this context, Barbash et al. use a representative sample of US hospitals to explore the opinions of hospital quality leaders regarding the SEP-1 measure. First, the qualitative methods used by Barbash et al. warrant some explanation. Much of biomedical research is characterized by hypothesis-driven, deductive reasoning: theories are tested using observations. In contrast, the methods of Barbash et al. use inductive reasoning: observations are used to develop theories within a systematic approach called “grounded theory” that explores common themes emerging from structured interviews.23 Inductive reasoning can later inform deductive reasoning, feeding theories into testable hypotheses. However, qualitative, inductive research is not meant to test hypotheses and is not subject to typical notions of “power and sample size” often expected of quantitative statistical analyses. Qualitative studies reach sufficient sample size when no further themes emerge, a situation called “thematic saturation”; the sample size here of 29 participants rests comfortably in the range of participants commonly needed for thematic saturation.23
Barbash et al. identified common themes in opinions of quality leaders regarding SEP-1. Namely, the complexity of SEP-1 necessitated a major resource investment into sepsis care and data collection. The major infrastructure investments needed to comply with SEP-1 also bred frustration regarding lack of perceived fairness around the “all or none” nature of the measure and raised multiple additional challenges including lack of clinician buy-in and resistance to protocolized care. Prior qualitative studies evaluating hospital quality leaders’ opinions on performance measures identified similar concerns about lack of “fairness,”24 but the implementation of SEP-1 has raised additional concern regarding the large burdens of instituting major infrastructure changes to monitor processes of care required to report on this measure. Despite the major challenges of responding to SEP-1, quality leaders expressed optimism that increased attention to sepsis would ultimately lead to better patient outcomes.
How might future sepsis quality measures achieve the adequate balance between focusing attention on improving care processes for high-impact diseases, without imposing additional burdens on the healthcare system? Lessons from Barbash et al. help us move forward. First, rather than taxing hospitals with administratively complex process measures, initial attempts at quality measures should start simply. Policy makers should consider moving forward into new areas of quality measurement in 2 ways: (1) pursue 1 or 2 processes with strong etiological links to important patient outcomes (eg, timely antibiotics in septic shock),25-28 and/or (2) use risk-adjusted outcomes and allow individual hospitals to adopt processes that improve local patient outcomes. Evidence suggests that the introduction of a quality measure may result in improved outcomes regardless of adoption of specific target processes,29 although results are mixed.30,31 In either case, complex “all or none” measures based upon weak evidence run a high risk of inciting clinician resentment and paradoxically perpetuating poor quality by increasing healthcare costs (decreased efficiency) without gains in safety, effectiveness, timeliness, or equity.32 It has been estimated that hospitals spend on average $2 million to implement SEP-1,33 with unclear return on the investment. The experience of SEP-1 is a reminder that, as evidence evolves, quality measures must adapt lest they become irrelevant. However, it is also a reminder that quality measures should not sit precariously on the edge of evidence. Withdrawal of process-based measures due to a changing evidence landscape breeds mistrust and impairs future attempts to improve quality.
Sepsis quality measures face additional challenges. If recent experience with interpretation of sepsis definitions can serve as a guide, variable uptake of newer sepsis definitions between/across hospitals will impair the ability to risk-adjust outcome measures and increase bias in identifying outlier hospitals.34 In addition, recent studies have already raised skepticism regarding the effectiveness of individual SEP-1 bundle components, confirming suspicions that the 30 cc/kg fluid bolus is not a magic quality target. Rather, the effectiveness of prior sepsis bundles has likely been driven by improved time to antibiotics, a process unstudied in sepsis trials, but driven by increased attention to the importance of early sepsis recognition and timely management.28 Timeliness of antibiotics can act as an effect modifier for more complex sepsis therapies, with quicker time to antibiotics associated with reversal of previously described effectiveness of activated protein C,35 and EGDT.28
Sepsis has a legacy in which improving simple processes (ie, time to antibiotics) obviates the need for more complex interventions (eg, activated protein C, EGDT). To the extent that CMS remains committed to using process-based measures of quality, those focused on sepsis are likely to be most effective when pared down to the simplest and strongest evidence base—improved recognition36 and timely antibiotics (for patients with infection-induced organ dysfunction and shock). Taking the time to start simply may best serve our current patients and preserve stakeholder buy-in for quality initiatives likely to benefit our future patients.
Disclosure
Dr. Lindenauer reports that he received support from the Centers for Medicare and Medicaid Services to develop and maintain hospital outcome measures for pneumonia and COPD. Dr. Lindenauer is supported by grant K24HL132008 from the National Heart, Lung, and Blood Institute. Dr. Walkey was supported by grants K01-HL116768 and R01-HL139751 from the National Heart, Lung, and Blood Institute.
“I didn’t have time to write a short letter, so I wrote a long one instead.”
-Mark Twain
Sepsis is a logical target for quality measures. Specifically, sepsis represents the perfect storm of immense public health burden1-3 combined with unexplained practice4-6 and outcomes7 variation. Thus, it is not surprising that in October 2015, the Centers of Medicare and Medicaid Services (CMS) adopted a sepsis quality measure.8 More surprising were the complex contents of the CMS Sepsis Core Measure “SEP-1” quality measure.9 CMS had written a “long letter.”
The multiple processes targeted with the CMS SEP-1 quality measure can best be understood with a brief account of history. SEP-1 arose from the National Quality Forum’s (NQF) project #0500: “Severe Sepsis and Septic Shock: Management Bundle,” a measure based upon Rivers et al.’s10 single-center, randomized, controlled trial of early goal-directed therapy (EGDT) for severe sepsis. EGDT was an intervention that consisted of fluid resuscitation and hemodynamic management based upon fulfilling specific targets of central venous pressure, superior vena cava oxygen saturation (or lactic acid), and hemoglobin and mean arterial pressures.11 The large mortality benefits, physiological rationale, and algorithmic responses to a variety of abnormal clinical values provided an appealing protocol to critical care and emergency physicians trained to normalize measured values, as well as policy makers looking for quality measures. Observational studies consistently showed associations between adoption of guideline-based “sepsis bundles” and improved patient outcomes,12-14 setting the stage for the transition of NQF #0500 into SEP-1.
However, the transition from EGDT-based NQF #0500 to SEP-1 has been tumultuous. Soon after adoption of SEP-1, the consensus definitions of sepsis changed markedly. Sepsis went from being defined as the presence of infection with concomitant systemic inflammatory response syndrome (sepsis), organ dysfunction (severe sepsis), and/or shock,15 to being defined as a dysregulated response to infection resulting in life-threatening organ dysfunction (sepsis) and/or fluid-resistant hypotension requiring vasopressors and lactate greater than 2 mmol/L.16 As the study by Barbash et al.17
In addition to its unprecedented complexity, SEP-1 received criticism for the weak evidence base of its individual components. The general concepts behind SEP-1 are well-accepted tenets of sepsis management: rapid recognition, assessment and treatment of underlying infection, and institution of intravenous fluids and vasopressor support for septic shock. However, the “all or none” prescriptive nature of the SEP-1 bundle was based on a somewhat arbitrary set of measures and targets. For example, patients with septic shock must receive 30 cc/kg of intravenous fluids to be “SEP-1 compliant.” The value “30 cc/kg” was taken from the average volume of fluids reported in prior sepsis trials, essentially based on a very low level of evidence.20 The strict 30 cc/kg cutoff did not take into account that “the median isn’t the message”21 in fluid management: optimal resuscitation targets are unclear,22 and selecting the median as a target ignores the fact that 50% of patients enrolled in international trials of EGDT received less than 30 cc/kg of initial fluid resuscitation (the interquartile range was 16-42 cc/kg).18 Thus, most participants in trials upon which the SEP-1 fluid measure was based would ironically not have met the SEP-1 measure. Mandates for physical exam and physiological measures were based on similarly low levels of evidence.
Into this context, Barbash et al. use a representative sample of US hospitals to explore the opinions of hospital quality leaders regarding the SEP-1 measure. First, the qualitative methods used by Barbash et al. warrant some explanation. Much of biomedical research is characterized by hypothesis-driven, deductive reasoning: theories are tested using observations. In contrast, the methods of Barbash et al. use inductive reasoning: observations are used to develop theories within a systematic approach called “grounded theory” that explores common themes emerging from structured interviews.23 Inductive reasoning can later inform deductive reasoning, feeding theories into testable hypotheses. However, qualitative, inductive research is not meant to test hypotheses and is not subject to typical notions of “power and sample size” often expected of quantitative statistical analyses. Qualitative studies reach sufficient sample size when no further themes emerge, a situation called “thematic saturation”; the sample size here of 29 participants rests comfortably in the range of participants commonly needed for thematic saturation.23
Barbash et al. identified common themes in opinions of quality leaders regarding SEP-1. Namely, the complexity of SEP-1 necessitated a major resource investment into sepsis care and data collection. The major infrastructure investments needed to comply with SEP-1 also bred frustration regarding lack of perceived fairness around the “all or none” nature of the measure and raised multiple additional challenges including lack of clinician buy-in and resistance to protocolized care. Prior qualitative studies evaluating hospital quality leaders’ opinions on performance measures identified similar concerns about lack of “fairness,”24 but the implementation of SEP-1 has raised additional concern regarding the large burdens of instituting major infrastructure changes to monitor processes of care required to report on this measure. Despite the major challenges of responding to SEP-1, quality leaders expressed optimism that increased attention to sepsis would ultimately lead to better patient outcomes.
How might future sepsis quality measures achieve the adequate balance between focusing attention on improving care processes for high-impact diseases, without imposing additional burdens on the healthcare system? Lessons from Barbash et al. help us move forward. First, rather than taxing hospitals with administratively complex process measures, initial attempts at quality measures should start simply. Policy makers should consider moving forward into new areas of quality measurement in 2 ways: (1) pursue 1 or 2 processes with strong etiological links to important patient outcomes (eg, timely antibiotics in septic shock),25-28 and/or (2) use risk-adjusted outcomes and allow individual hospitals to adopt processes that improve local patient outcomes. Evidence suggests that the introduction of a quality measure may result in improved outcomes regardless of adoption of specific target processes,29 although results are mixed.30,31 In either case, complex “all or none” measures based upon weak evidence run a high risk of inciting clinician resentment and paradoxically perpetuating poor quality by increasing healthcare costs (decreased efficiency) without gains in safety, effectiveness, timeliness, or equity.32 It has been estimated that hospitals spend on average $2 million to implement SEP-1,33 with unclear return on the investment. The experience of SEP-1 is a reminder that, as evidence evolves, quality measures must adapt lest they become irrelevant. However, it is also a reminder that quality measures should not sit precariously on the edge of evidence. Withdrawal of process-based measures due to a changing evidence landscape breeds mistrust and impairs future attempts to improve quality.
Sepsis quality measures face additional challenges. If recent experience with interpretation of sepsis definitions can serve as a guide, variable uptake of newer sepsis definitions between/across hospitals will impair the ability to risk-adjust outcome measures and increase bias in identifying outlier hospitals.34 In addition, recent studies have already raised skepticism regarding the effectiveness of individual SEP-1 bundle components, confirming suspicions that the 30 cc/kg fluid bolus is not a magic quality target. Rather, the effectiveness of prior sepsis bundles has likely been driven by improved time to antibiotics, a process unstudied in sepsis trials, but driven by increased attention to the importance of early sepsis recognition and timely management.28 Timeliness of antibiotics can act as an effect modifier for more complex sepsis therapies, with quicker time to antibiotics associated with reversal of previously described effectiveness of activated protein C,35 and EGDT.28
Sepsis has a legacy in which improving simple processes (ie, time to antibiotics) obviates the need for more complex interventions (eg, activated protein C, EGDT). To the extent that CMS remains committed to using process-based measures of quality, those focused on sepsis are likely to be most effective when pared down to the simplest and strongest evidence base—improved recognition36 and timely antibiotics (for patients with infection-induced organ dysfunction and shock). Taking the time to start simply may best serve our current patients and preserve stakeholder buy-in for quality initiatives likely to benefit our future patients.
Disclosure
Dr. Lindenauer reports that he received support from the Centers for Medicare and Medicaid Services to develop and maintain hospital outcome measures for pneumonia and COPD. Dr. Lindenauer is supported by grant K24HL132008 from the National Heart, Lung, and Blood Institute. Dr. Walkey was supported by grants K01-HL116768 and R01-HL139751 from the National Heart, Lung, and Blood Institute.
1. Elixhauser A, Friedman B, Stranges E. Septicemia in U.S. Hospitals, 2009. HCUP. Statistical Brief #122. Rockville MD: Agency for Healthcare Research and Quality; 2011; p 1-13. PubMed
2. Liu V, Lei X, Prescott HC, Kipnis P, Iwashyna TJ, Escobar GJ. Hospital readmission and healthcare utilization following sepsis in community settings. J Hosp Med. 2014;9(8):502-507. PubMed
3. Liu V, Escobar GJ, Greene JD, et al. Hospital deaths in patients with sepsis from 2 independent cohorts. JAMA. 2014;312(1):90-92. PubMed
4. Peltan ID, Mitchell KH, Rudd KE, et al. Physician Variation in Time to Antimicrobial Treatment for Septic Patients Presenting to the Emergency Department. Crit Care Med. 2017;45(6):1011-1018. PubMed
5. Marik PE, Linde-Zwirble WT, Bittner EA, Sahatjian J, Hansell D. Fluid administration in severe sepsis and septic shock, patterns and outcomes: an analysis of a large national database. Intensive Care Med. 2017;43(5):625-632. PubMed
6. Lagu T, Rothberg MB, Nathanson BH, Pekow PS, Steingrub JS, Lindenauer PK. Variation in the care of septic shock: the impact of patient and hospital characteristics. J Crit Care. 2012;27(4):329-336. PubMed
7. Wang HE, Donnelly JP, Shapiro NI, Hohmann SF, Levitan EB. Hospital variations in severe sepsis mortality. Am J Med Qual. 2015;30(4):328-336. PubMed
8. Centers for Medicare & Medicaid Services. CMS Measures Inventory. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityMeasures/CMS-Measures-Inventory.html. Accessed June 8, 2017.
9. QualityNet. Specifications Manual, Version 5.0b, Section 2.2. Severe Sepsis and Septic Shock. https://www.qualitynet.org/dcs/ContentServer?c=Page&pagename=QnetPublic%2FPage%2FQnetTier4&cid=1228774725171. Accessed June 8, 2017.
10. National Quality Forum. 0500 Severe sepsis and septic shock management bundle. http://www.qualityforum.org. Accessed June 8, 2017.
11. Rivers E, Nguyen B, Havstad S, et al. Early Goal-Directed Therapy in the Treatment of Severe Sepsis and Septic Shock. N Engl J Med. 2001;345:1368-1377. PubMed
12. Levy MM, Dellinger RP, Townsend SR, et al. The Surviving Sepsis Campaign: results of an international guideline-based performance improvement program targeting severe sepsis. Crit Care Med. 2010;38(2):367-374. PubMed
13. Levy MM, Artigas A, Phillips GS, et al. Outcomes of the Surviving Sepsis Campaign in intensive care units in the USA and Europe: a prospective cohort study. Lancet Infect Dis. 2012;12(12):919-924. PubMed
14. Ferrer R, Artigas A, Levy MM, et al. Improvement in process of care and outcome after a multicenter severe sepsis educational program in Spain. JAMA. 2008;299(19):2294-2303. PubMed
15. Bone RC, Balk RA, Cerra FB, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest. 1992;101(6):1644-1655. PubMed
16. Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):801-810. PubMed
17. Barbash IJ, Rak KJ, Kuza CC, Kahn JM. Hospital Perceptions of Medicare’s Sepsis Quality Reporting Initiative. J Hosp Med. 2017;12(12):963-967.
18. The PRISM Investigators. Early, Goal-Directed Therapy for Septic Shock — A Patient-Level Meta-Analysis. N Engl J Med. 2017;376:2223-2234. PubMed
19. National Quality Forum. NQF Revises Sepsis Measure. http://www.qualityforum.org/NQF_Revises_Sepsis_Measure.aspx. Accessed June 8, 2017.
20. Rhodes A, Evans LE, Alhazzani W, et al. Surviving Sepsis Campaign: International Guidelines for Management of Sepsis and Septic Shock: 2016. Intensive Care Med. 2017;43(3):304-377. PubMed
21. Gould SJ. The median isn’t the message. Discover. 1985;6:40-42. PubMed
22. Hernandez G, Teboul JL. Fourth Surviving Sepsis Campaign’s hemodynamic recommendations: a step forward or a return to chaos? Crit Care. 2017;21(1):133. PubMed
23. Fugard AJ, Potts HW. Supporting thinking on sample sizes for thematic analyses. Int J Soc Res Methodol. 2015;18(6):669-684.
24. Goff SL, Lagu T, Pekow PS, et al. A qualitative analysis of hospital leaders’ opinions about publicly reported measures of health care quality. Jt Comm J Qual Patient Saf. 2015;41(4):169-176. PubMed
25. Kumar A, Haery C, Paladugu B, et al. The duration of hypotension before the initiation of antibiotic treatment is a critical determinant of survival in a murine model of Escherichia coli septic shock: association with serum lactate and inflammatory cytokine levels. J Infect Dis. 2006;193(2):251-258.
PubMed
26. Liu VX, Fielding-Singh V, Greene JD, et al. The Timing of Early Antibiotics and Hospital Mortality in Sepsis. Am J Respir Crit Care Med. 2017. [Epub ahead of print]. PubMed
27. Seymour CW, Gesten F, Prescott HC, et al. Time to Treatment and Mortality during Mandated Emergency Care for Sepsis. N Engl J Med. 2017;376:2235-2244. PubMed
28. Kalil AC, Johnson DW, Lisco SJ, Sun J. Early Goal-Directed Therapy for Sepsis: A Novel Solution for Discordant Survival Outcomes in Clinical Trials. Crit Care Med. 2017;45(4):607-614. PubMed
29. Tu JV, Donovan LR, Lee DS, et al. Effectiveness of public report cards for improving the quality of cardiac care: the EFFECT study: a randomized trial. JAMA. 2009;302(21):2330-2337. PubMed
30. Joynt KE, Blumenthal DM, Orav EJ, Resnic FS, Jha AK. Association of public reporting for percutaneous coronary intervention with utilization and outcomes among Medicare beneficiaries with acute myocardial infarction. JAMA. 2012;308(14):1460-1468. PubMed
31. Osborne NH, Nicholas LH, Ryan AM, Thumma JR, Dimick JB. Association of hospital participation in a quality reporting program with surgical outcomes and expenditures for Medicare beneficiaries. JAMA. 2015;313(5):496-504. PubMed
32. Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington (DC): National Academies Press (US); 2001. PubMed
33. 2015;12(11):1676-1684.Ann Am Thorac Soc36. Kramer RD, Cooke CR, Liu V, Miller RR 3rd, Iwashyna TJ. Variation in the Contents of Sepsis Bundles and Quality Measures. A Systematic Review. PubMed
34. 2012;40(11):2974-2981.Crit Care Med35. Rimmer E, Kumar A, Doucette S, et al. Activated protein C and septic shock: a propensity-matched cohort study*. PubMed
35. 2014;160(6):380-388.Ann Intern Med34. Rothberg MB, Pekow PS, Priya A, Lindenauer PK. Variation in diagnostic coding of patients with pneumonia and its association with hospital risk-standardized mortality rates: a cross-sectional analysis. PubMed
36. 2015;12(11):1597-1599. Ann Am Thorac Soc33. Wall MJ, Howell MD. Variation and Cost-effectiveness of Quality Measurement Programs. The Case of Sepsis Bundles. PubMed
1. Elixhauser A, Friedman B, Stranges E. Septicemia in U.S. Hospitals, 2009. HCUP. Statistical Brief #122. Rockville MD: Agency for Healthcare Research and Quality; 2011; p 1-13. PubMed
2. Liu V, Lei X, Prescott HC, Kipnis P, Iwashyna TJ, Escobar GJ. Hospital readmission and healthcare utilization following sepsis in community settings. J Hosp Med. 2014;9(8):502-507. PubMed
3. Liu V, Escobar GJ, Greene JD, et al. Hospital deaths in patients with sepsis from 2 independent cohorts. JAMA. 2014;312(1):90-92. PubMed
4. Peltan ID, Mitchell KH, Rudd KE, et al. Physician Variation in Time to Antimicrobial Treatment for Septic Patients Presenting to the Emergency Department. Crit Care Med. 2017;45(6):1011-1018. PubMed
5. Marik PE, Linde-Zwirble WT, Bittner EA, Sahatjian J, Hansell D. Fluid administration in severe sepsis and septic shock, patterns and outcomes: an analysis of a large national database. Intensive Care Med. 2017;43(5):625-632. PubMed
6. Lagu T, Rothberg MB, Nathanson BH, Pekow PS, Steingrub JS, Lindenauer PK. Variation in the care of septic shock: the impact of patient and hospital characteristics. J Crit Care. 2012;27(4):329-336. PubMed
7. Wang HE, Donnelly JP, Shapiro NI, Hohmann SF, Levitan EB. Hospital variations in severe sepsis mortality. Am J Med Qual. 2015;30(4):328-336. PubMed
8. Centers for Medicare & Medicaid Services. CMS Measures Inventory. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/QualityMeasures/CMS-Measures-Inventory.html. Accessed June 8, 2017.
9. QualityNet. Specifications Manual, Version 5.0b, Section 2.2. Severe Sepsis and Septic Shock. https://www.qualitynet.org/dcs/ContentServer?c=Page&pagename=QnetPublic%2FPage%2FQnetTier4&cid=1228774725171. Accessed June 8, 2017.
10. National Quality Forum. 0500 Severe sepsis and septic shock management bundle. http://www.qualityforum.org. Accessed June 8, 2017.
11. Rivers E, Nguyen B, Havstad S, et al. Early Goal-Directed Therapy in the Treatment of Severe Sepsis and Septic Shock. N Engl J Med. 2001;345:1368-1377. PubMed
12. Levy MM, Dellinger RP, Townsend SR, et al. The Surviving Sepsis Campaign: results of an international guideline-based performance improvement program targeting severe sepsis. Crit Care Med. 2010;38(2):367-374. PubMed
13. Levy MM, Artigas A, Phillips GS, et al. Outcomes of the Surviving Sepsis Campaign in intensive care units in the USA and Europe: a prospective cohort study. Lancet Infect Dis. 2012;12(12):919-924. PubMed
14. Ferrer R, Artigas A, Levy MM, et al. Improvement in process of care and outcome after a multicenter severe sepsis educational program in Spain. JAMA. 2008;299(19):2294-2303. PubMed
15. Bone RC, Balk RA, Cerra FB, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest. 1992;101(6):1644-1655. PubMed
16. Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):801-810. PubMed
17. Barbash IJ, Rak KJ, Kuza CC, Kahn JM. Hospital Perceptions of Medicare’s Sepsis Quality Reporting Initiative. J Hosp Med. 2017;12(12):963-967.
18. The PRISM Investigators. Early, Goal-Directed Therapy for Septic Shock — A Patient-Level Meta-Analysis. N Engl J Med. 2017;376:2223-2234. PubMed
19. National Quality Forum. NQF Revises Sepsis Measure. http://www.qualityforum.org/NQF_Revises_Sepsis_Measure.aspx. Accessed June 8, 2017.
20. Rhodes A, Evans LE, Alhazzani W, et al. Surviving Sepsis Campaign: International Guidelines for Management of Sepsis and Septic Shock: 2016. Intensive Care Med. 2017;43(3):304-377. PubMed
21. Gould SJ. The median isn’t the message. Discover. 1985;6:40-42. PubMed
22. Hernandez G, Teboul JL. Fourth Surviving Sepsis Campaign’s hemodynamic recommendations: a step forward or a return to chaos? Crit Care. 2017;21(1):133. PubMed
23. Fugard AJ, Potts HW. Supporting thinking on sample sizes for thematic analyses. Int J Soc Res Methodol. 2015;18(6):669-684.
24. Goff SL, Lagu T, Pekow PS, et al. A qualitative analysis of hospital leaders’ opinions about publicly reported measures of health care quality. Jt Comm J Qual Patient Saf. 2015;41(4):169-176. PubMed
25. Kumar A, Haery C, Paladugu B, et al. The duration of hypotension before the initiation of antibiotic treatment is a critical determinant of survival in a murine model of Escherichia coli septic shock: association with serum lactate and inflammatory cytokine levels. J Infect Dis. 2006;193(2):251-258.
PubMed
26. Liu VX, Fielding-Singh V, Greene JD, et al. The Timing of Early Antibiotics and Hospital Mortality in Sepsis. Am J Respir Crit Care Med. 2017. [Epub ahead of print]. PubMed
27. Seymour CW, Gesten F, Prescott HC, et al. Time to Treatment and Mortality during Mandated Emergency Care for Sepsis. N Engl J Med. 2017;376:2235-2244. PubMed
28. Kalil AC, Johnson DW, Lisco SJ, Sun J. Early Goal-Directed Therapy for Sepsis: A Novel Solution for Discordant Survival Outcomes in Clinical Trials. Crit Care Med. 2017;45(4):607-614. PubMed
29. Tu JV, Donovan LR, Lee DS, et al. Effectiveness of public report cards for improving the quality of cardiac care: the EFFECT study: a randomized trial. JAMA. 2009;302(21):2330-2337. PubMed
30. Joynt KE, Blumenthal DM, Orav EJ, Resnic FS, Jha AK. Association of public reporting for percutaneous coronary intervention with utilization and outcomes among Medicare beneficiaries with acute myocardial infarction. JAMA. 2012;308(14):1460-1468. PubMed
31. Osborne NH, Nicholas LH, Ryan AM, Thumma JR, Dimick JB. Association of hospital participation in a quality reporting program with surgical outcomes and expenditures for Medicare beneficiaries. JAMA. 2015;313(5):496-504. PubMed
32. Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington (DC): National Academies Press (US); 2001. PubMed
33. 2015;12(11):1676-1684.Ann Am Thorac Soc36. Kramer RD, Cooke CR, Liu V, Miller RR 3rd, Iwashyna TJ. Variation in the Contents of Sepsis Bundles and Quality Measures. A Systematic Review. PubMed
34. 2012;40(11):2974-2981.Crit Care Med35. Rimmer E, Kumar A, Doucette S, et al. Activated protein C and septic shock: a propensity-matched cohort study*. PubMed
35. 2014;160(6):380-388.Ann Intern Med34. Rothberg MB, Pekow PS, Priya A, Lindenauer PK. Variation in diagnostic coding of patients with pneumonia and its association with hospital risk-standardized mortality rates: a cross-sectional analysis. PubMed
36. 2015;12(11):1597-1599. Ann Am Thorac Soc33. Wall MJ, Howell MD. Variation and Cost-effectiveness of Quality Measurement Programs. The Case of Sepsis Bundles. PubMed
©2017 Society of Hospital Medicine