Hospital Medicine

From the T1D Exchange, Boston, MA (Ann Mungmode, Nicole Rioles, Jesse Cases, Dr. Ebekozien); The Leona M. and Harry B. Hemsley Charitable Trust, New York, NY (Laurel Koester); and the University of Mississippi School of Population Health, Jackson, MS (Dr. Ebekozien).

Abstract

There have been remarkable innovations in diabetes management since the start of the COVID-19 pandemic, but these groundbreaking innovations are drawing limited focus as the field focuses on the adverse impact of the pandemic on patients with diabetes. This article reviews select population health innovations in diabetes management that have become available over the past 2 years of the COVID-19 pandemic from the perspective of the T1D Exchange Quality Improvement Collaborative, a learning health network that focuses on improving care and outcomes for individuals with type 1 diabetes (T1D). Such innovations include expanded telemedicine access, collection of real-world data, machine learning and artificial intelligence, and new diabetes medications and devices. In addition, multiple innovative studies have been undertaken to explore contributors to health inequities in diabetes, and advocacy efforts for specific populations have been successful. Looking to the future, work is required to explore additional health equity successes that do not further exacerbate inequities and to look for additional innovative ways to engage people with T1D in their health care through conversations on social determinants of health and societal structures.

Keywords: type 1 diabetes, learning health network, continuous glucose monitoring, health equity

One in 10 people in the United States has diabetes.¹ Diabetes is the nation’s second leading cause of death, costing the US health system more than $300 billion annually.² The COVID-19 pandemic presented additional health burdens for people living with diabetes. For example, preexisting diabetes was identified as a risk factor for COVID-19–associated morbidity and mortality.^3,4 Over the past 2 years, there have been remarkable innovations in diabetes management, including stem cell therapy and new medication options. Additionally, improved technology solutions have aided in diabetes management through continuous glucose monitors (CGM), smart insulin pens, advanced hybrid closed-loop systems, and continuous subcutaneous insulin injections.^5,6 Unfortunately, these groundbreaking innovations are drawing limited focus, as the field is rightfully focused on the adverse impact of the pandemic on patients with diabetes.

Learning health networks like the T1D Exchange Quality Improvement Collaborative (T1DX-QI) have implemented some of these innovative solutions to improve care for people with diabetes.⁷ T1DX-QI has more than 50 data-sharing endocrinology centers that care for over 75,000 people with diabetes across the United States (Figure 1). Centers participating in the T1DX-QI use quality improvement (QI) and implementation science methods to quickly translate research into evidence-based clinical practice. T1DX-QI leads diabetes population health and health system research and supports widespread transferability across health care organizations through regular collaborative calls, conferences, and case study documentation.⁸

In this review, we summarize impactful population health innovations in diabetes management that have become available over the past 2 years of the COVID-19 pandemic from the perspective of T1DX-QI (see Figure 2 for relevant definitions). This review is limited in scope and is not meant to be an exhaustive list of innovations. The review also reflects significant changes from the perspective of academic diabetes centers, which may not apply to rural or primary care diabetes practices.

The first (A.M.), second (H.H.), and senior (O.E.) authors conducted a scoping review of published literature using terms related to diabetes, population health, and innovation on PubMed Central and Google Scholar for the period March 2020 to June 2022. To complement the review, A.M. and O.E. also reviewed abstracts from presentations at major international diabetes conferences, including the American Diabetes Association (ADA), the International Society for Pediatric and Adolescent Diabetes (ISPAD), the T1DX-QI Learning Session Conference, and the Advanced Technologies & Treatments for Diabetes (ATTD) 2020 to 2022 conferences.^9-14 The authors also searched FDA.gov and ClinicalTrials.gov for relevant insights. A.M. and O.E. sorted the reviewed literature into major themes (Figure 3) from the population health improvement perspective of the T1DX-QI.

Population Health Innovations in Diabetes Management

Expansion of Telemedicine Access

Telemedicine is cost-effective for patients with diabetes,¹⁵ including those with complex cases.¹⁶ Before the COVID-19 pandemic, telemedicine and virtual care were rare in diabetes management. However, the pandemic offered a new opportunity to expand the practice of telemedicine in diabetes management. A study from the T1DX-QI showed that telemedicine visits grew from comprising <1% of visits pre-pandemic (December 2019) to 95.2% during the pandemic (August 2020).¹⁷ Additional studies, like those conducted by Phillip et al,¹⁸ confirmed the noninferiority of telemedicine practice for patients with diabetes.Telemedicine was also found to be an effective strategy to educate patients on the use of diabetes technologies.¹⁹

Real-World Data and Disease Surveillance

As the COVID-19 pandemic exacerbated outcomes for people with type 1 diabetes (T1D), a need arose to understand the immediate effects of the pandemic on people with T1D through real-world data and disease surveillance. In April 2020, the T1DX-QI initiated a multicenter surveillance study to collect data and analyze the impact of COVID-19 on people with T1D. The existing health collaborative served as a springboard for robust surveillance study, documenting numerous works on the effects of COVID-19.^3,4,20-28 Other investigators also embraced the power of real-world surveillance and real-world data.^29,30

Big Data, Machine Learning, and Artificial Intelligence

The past 2 years have seen a shift toward embracing the incredible opportunity to tap the large volume of data generated from routine care for practical insights.³¹ In particular, researchers have demonstrated the widespread application of machine learning and artificial intelligence to improve diabetes management.³² The T1DX-QI also harnessed the growing power of big data by expanding the functionality of innovative benchmarking software. The T1DX QI Portal uses electronic medical record data of diabetes patients for clinic-to-clinic benchmarking and data analysis, using business intelligence solutions.³³

Health Equity

While inequities across various health outcomes have been well documented for years,³⁴ the COVID-19 pandemic further exaggerated racial/ethnic health inequities in T1D.^23,35 In response, several organizations have outlined specific strategies to address these health inequities. Emboldened by the pandemic, the T1DX-QI announced a multipronged approach to address health inequities among patients with T1D through the Health Equity Advancement Lab (HEAL).³⁶ One of HEAL’s main components is using real-world data to champion population-level insights and demonstrate progress in QI efforts.

Multiple innovative studies have been undertaken to explore contributors to health inequities in diabetes, and these studies are expanding our understanding of the chasm.³⁷ There have also been innovative solutions to addressing these inequities, with multiple studies published over the past 2 years.³⁸ A source of inequity among patients with T1D is the lack of representation of racial/ethnic minorities with T1D in clinical trials.³⁹ The T1DX-QI suggests that the equity-adapted framework for QI can be applied by research leaders to support trial diversity and representation, ensuring future device innovations are meaningful for all people with T1D.⁴⁰

Diabetes Devices

Glucose monitoring and insulin therapy are vital tools to support individuals living with T1D, and devices such as CGM and insulin pumps have become the standard of care for diabetes management (Table).⁴¹ Innovations in diabetes technology and device access are imperative for a chronic disease with no cure.

The COVID-19 pandemic created an opportunity to increase access to diabetes devices in inpatient settings. In 2020, the US Food and Drug Administration expanded the use of CGM to support remote monitoring of patients in inpatient hospital settings, simultaneously supporting the glucose monitoring needs of patients with T1D and reducing COVID-19 transmission through reduced patient-clinician contact.⁴² This effort has been expanded and will continue in 2022 and beyond,⁴³ and aligns with the growing consensus that supports patients wearing both CGMs and insulin pumps in ambulatory settings to improve patient health outcomes.⁴⁴

Since 2020, innovations in diabetes technology have improved and increased the variety of options available to people with T1D and made them easier to use (Table). New, advanced hybrid closed-loop systems have progressed to offer Bluetooth features, including automatic software upgrades, tubeless systems, and the ability to allow parents to use their smartphones to bolus for children.^45-47 The next big step in insulin delivery innovation is the release of functioning, fully closed loop systems, of which several are currently in clinical trials.⁴⁸ These systems support reduced hypoglycemia and improved time in range.⁴⁹

Additional innovations in insulin delivery have improved the user experience and expanded therapeutic options, including a variety of smart insulin pens complete with dosing logs^50,51 and even a patch to deliver insulin without the burden of injections.⁵² As barriers to diabetes technology persist,⁵³ innovations in alternate insulin delivery provide people with T1D more options to align with their personal access and technology preferences.

Innovations in CGM address cited barriers to their use, including size or overall wear.^53-55 CGMs released in the past few years are smaller in physical size, have longer durations of time between changings, are more accurate, and do not require calibrations for accuracy.

New Diabetes Medications

Many new medications and therapeutic advances have become available in the past 2 years.⁵⁶ Additionally, more medications are being tested as adjunct therapies to support glycemic management in patients with T1D, including metformin, sodium-glucose cotransporter 1 and 2 inhibitors, pramlintide, glucagon-like polypeptide-1 analogs, and glucagon receptor agonists.⁵⁷ Other recent advances include stem cell replacement therapy for patients with T1D.⁵⁸ The ultra-long-acting biosimilar insulins are one medical innovation that has been stalled, rather than propelled, during the COVID-19 pandemic.⁵⁹

Diabetes Policy Advocacy

People with T1D require insulin to survive. The cost of insulin has increased in recent years, with some studies citing a 64% to 100% increase in the past decade.^60,61 In fact, 1 in 4 insulin users report that cost has impacted their insulin use, including rationing their insulin.⁶² Lockdowns during the COVID-19 pandemic stressed US families financially, increasing the urgency for insulin cost caps.

Although the COVID-19 pandemic halted national conversations on drug financing,⁶³ advocacy efforts have succeeded for specific populations. The new Medicare Part D Senior Savings Model will cap the cost of insulin at $35 for a 30-day supply,⁶⁴ and 20 states passed legislation capping insulin pricing.⁶² Efforts to codify national cost caps are under debate, including the passage of the Affordable Insulin Now Act, which passed the House in March 2022 and is currently under review in the Senate.⁶⁵

Funders and industry partners play a crucial role in leading and supporting innovations that improve the lives of people with T1D and reduce society’s costs of living with the disease. Data infrastructure is critical to supporting population health. While building the data infrastructure to support population health is both time- and resource-intensive, private foundations such as Helmsley are uniquely positioned—and have a responsibility—to take large, informed risks to help reach all communities with T1D.

The T1DX-QI is the largest source of population health data on T1D in the United States and is becoming the premiere data authority on its incidence, prevalence, and outcomes. The T1DX-QI enables a robust understanding of T1D-related health trends at the population level, as well as trends among clinics and providers. Pilot centers in the T1DX-QI have reported reductions in patients’ A1c and acute diabetes-related events, as well as improvements in device usage and depression screening. The ability to capture changes speaks to the promise and power of these data to demonstrate the clinical impact of QI interventions and to support the spread of best practices and learnings across health systems.

Additional philanthropic efforts have supported innovation in the last 2 years. For example, the JDRF, a nonprofit philanthropic equity firm, has supported efforts in developing artificial pancreas systems and cell therapies currently in clinical trials like teplizumab, a drug that has demonstrated delayed onset of T1D through JDRF’s T1D Fund.⁶⁶ Industry partners also have an opportunity for significant influence in this area, as they continue to fund meaningful projects to advance care for people with T1D.⁶⁷

Conclusion

We are optimistic that the innovations summarized here describe a shift in the tide of equitable T1D outcomes; however, future work is required to explore additional health equity successes that do not further exacerbate inequities. We also see further opportunities for innovative ways to engage people with T1D in their health care through conversations on social determinants of health and societal structures.

Corresponding author: Ann Mungmode, MPH, T1D Exchange, 11 Avenue de Lafayette, Boston, MA 02111; Email: amungmode@t1dexchange.org

Disclosures: Dr. Ebekozien serve(d) as a director, officer, partner, employee, advisor, consultant, or trustee for the Medtronic Advisory Board and received research grants from Medtronic Diabetes, Eli Lilly, and Dexcom.

Funding: The T1DX-QI is funded by The Leona M. and Harry B. Hemsley Charitable Trust.

From the T1D Exchange, Boston, MA (Ann Mungmode, Nicole Rioles, Jesse Cases, Dr. Ebekozien); The Leona M. and Harry B. Hemsley Charitable Trust, New York, NY (Laurel Koester); and the University of Mississippi School of Population Health, Jackson, MS (Dr. Ebekozien).

Abstract

There have been remarkable innovations in diabetes management since the start of the COVID-19 pandemic, but these groundbreaking innovations are drawing limited focus as the field focuses on the adverse impact of the pandemic on patients with diabetes. This article reviews select population health innovations in diabetes management that have become available over the past 2 years of the COVID-19 pandemic from the perspective of the T1D Exchange Quality Improvement Collaborative, a learning health network that focuses on improving care and outcomes for individuals with type 1 diabetes (T1D). Such innovations include expanded telemedicine access, collection of real-world data, machine learning and artificial intelligence, and new diabetes medications and devices. In addition, multiple innovative studies have been undertaken to explore contributors to health inequities in diabetes, and advocacy efforts for specific populations have been successful. Looking to the future, work is required to explore additional health equity successes that do not further exacerbate inequities and to look for additional innovative ways to engage people with T1D in their health care through conversations on social determinants of health and societal structures.

Keywords: type 1 diabetes, learning health network, continuous glucose monitoring, health equity

One in 10 people in the United States has diabetes.¹ Diabetes is the nation’s second leading cause of death, costing the US health system more than $300 billion annually.² The COVID-19 pandemic presented additional health burdens for people living with diabetes. For example, preexisting diabetes was identified as a risk factor for COVID-19–associated morbidity and mortality.^3,4 Over the past 2 years, there have been remarkable innovations in diabetes management, including stem cell therapy and new medication options. Additionally, improved technology solutions have aided in diabetes management through continuous glucose monitors (CGM), smart insulin pens, advanced hybrid closed-loop systems, and continuous subcutaneous insulin injections.^5,6 Unfortunately, these groundbreaking innovations are drawing limited focus, as the field is rightfully focused on the adverse impact of the pandemic on patients with diabetes.

Learning health networks like the T1D Exchange Quality Improvement Collaborative (T1DX-QI) have implemented some of these innovative solutions to improve care for people with diabetes.⁷ T1DX-QI has more than 50 data-sharing endocrinology centers that care for over 75,000 people with diabetes across the United States (Figure 1). Centers participating in the T1DX-QI use quality improvement (QI) and implementation science methods to quickly translate research into evidence-based clinical practice. T1DX-QI leads diabetes population health and health system research and supports widespread transferability across health care organizations through regular collaborative calls, conferences, and case study documentation.⁸

In this review, we summarize impactful population health innovations in diabetes management that have become available over the past 2 years of the COVID-19 pandemic from the perspective of T1DX-QI (see Figure 2 for relevant definitions). This review is limited in scope and is not meant to be an exhaustive list of innovations. The review also reflects significant changes from the perspective of academic diabetes centers, which may not apply to rural or primary care diabetes practices.

The first (A.M.), second (H.H.), and senior (O.E.) authors conducted a scoping review of published literature using terms related to diabetes, population health, and innovation on PubMed Central and Google Scholar for the period March 2020 to June 2022. To complement the review, A.M. and O.E. also reviewed abstracts from presentations at major international diabetes conferences, including the American Diabetes Association (ADA), the International Society for Pediatric and Adolescent Diabetes (ISPAD), the T1DX-QI Learning Session Conference, and the Advanced Technologies & Treatments for Diabetes (ATTD) 2020 to 2022 conferences.^9-14 The authors also searched FDA.gov and ClinicalTrials.gov for relevant insights. A.M. and O.E. sorted the reviewed literature into major themes (Figure 3) from the population health improvement perspective of the T1DX-QI.

Population Health Innovations in Diabetes Management

Expansion of Telemedicine Access

Telemedicine is cost-effective for patients with diabetes,¹⁵ including those with complex cases.¹⁶ Before the COVID-19 pandemic, telemedicine and virtual care were rare in diabetes management. However, the pandemic offered a new opportunity to expand the practice of telemedicine in diabetes management. A study from the T1DX-QI showed that telemedicine visits grew from comprising <1% of visits pre-pandemic (December 2019) to 95.2% during the pandemic (August 2020).¹⁷ Additional studies, like those conducted by Phillip et al,¹⁸ confirmed the noninferiority of telemedicine practice for patients with diabetes.Telemedicine was also found to be an effective strategy to educate patients on the use of diabetes technologies.¹⁹

Real-World Data and Disease Surveillance

As the COVID-19 pandemic exacerbated outcomes for people with type 1 diabetes (T1D), a need arose to understand the immediate effects of the pandemic on people with T1D through real-world data and disease surveillance. In April 2020, the T1DX-QI initiated a multicenter surveillance study to collect data and analyze the impact of COVID-19 on people with T1D. The existing health collaborative served as a springboard for robust surveillance study, documenting numerous works on the effects of COVID-19.^3,4,20-28 Other investigators also embraced the power of real-world surveillance and real-world data.^29,30

Big Data, Machine Learning, and Artificial Intelligence

The past 2 years have seen a shift toward embracing the incredible opportunity to tap the large volume of data generated from routine care for practical insights.³¹ In particular, researchers have demonstrated the widespread application of machine learning and artificial intelligence to improve diabetes management.³² The T1DX-QI also harnessed the growing power of big data by expanding the functionality of innovative benchmarking software. The T1DX QI Portal uses electronic medical record data of diabetes patients for clinic-to-clinic benchmarking and data analysis, using business intelligence solutions.³³

Health Equity

While inequities across various health outcomes have been well documented for years,³⁴ the COVID-19 pandemic further exaggerated racial/ethnic health inequities in T1D.^23,35 In response, several organizations have outlined specific strategies to address these health inequities. Emboldened by the pandemic, the T1DX-QI announced a multipronged approach to address health inequities among patients with T1D through the Health Equity Advancement Lab (HEAL).³⁶ One of HEAL’s main components is using real-world data to champion population-level insights and demonstrate progress in QI efforts.

Multiple innovative studies have been undertaken to explore contributors to health inequities in diabetes, and these studies are expanding our understanding of the chasm.³⁷ There have also been innovative solutions to addressing these inequities, with multiple studies published over the past 2 years.³⁸ A source of inequity among patients with T1D is the lack of representation of racial/ethnic minorities with T1D in clinical trials.³⁹ The T1DX-QI suggests that the equity-adapted framework for QI can be applied by research leaders to support trial diversity and representation, ensuring future device innovations are meaningful for all people with T1D.⁴⁰

Diabetes Devices

Glucose monitoring and insulin therapy are vital tools to support individuals living with T1D, and devices such as CGM and insulin pumps have become the standard of care for diabetes management (Table).⁴¹ Innovations in diabetes technology and device access are imperative for a chronic disease with no cure.

The COVID-19 pandemic created an opportunity to increase access to diabetes devices in inpatient settings. In 2020, the US Food and Drug Administration expanded the use of CGM to support remote monitoring of patients in inpatient hospital settings, simultaneously supporting the glucose monitoring needs of patients with T1D and reducing COVID-19 transmission through reduced patient-clinician contact.⁴² This effort has been expanded and will continue in 2022 and beyond,⁴³ and aligns with the growing consensus that supports patients wearing both CGMs and insulin pumps in ambulatory settings to improve patient health outcomes.⁴⁴

Since 2020, innovations in diabetes technology have improved and increased the variety of options available to people with T1D and made them easier to use (Table). New, advanced hybrid closed-loop systems have progressed to offer Bluetooth features, including automatic software upgrades, tubeless systems, and the ability to allow parents to use their smartphones to bolus for children.^45-47 The next big step in insulin delivery innovation is the release of functioning, fully closed loop systems, of which several are currently in clinical trials.⁴⁸ These systems support reduced hypoglycemia and improved time in range.⁴⁹

Additional innovations in insulin delivery have improved the user experience and expanded therapeutic options, including a variety of smart insulin pens complete with dosing logs^50,51 and even a patch to deliver insulin without the burden of injections.⁵² As barriers to diabetes technology persist,⁵³ innovations in alternate insulin delivery provide people with T1D more options to align with their personal access and technology preferences.

Innovations in CGM address cited barriers to their use, including size or overall wear.^53-55 CGMs released in the past few years are smaller in physical size, have longer durations of time between changings, are more accurate, and do not require calibrations for accuracy.

New Diabetes Medications

Many new medications and therapeutic advances have become available in the past 2 years.⁵⁶ Additionally, more medications are being tested as adjunct therapies to support glycemic management in patients with T1D, including metformin, sodium-glucose cotransporter 1 and 2 inhibitors, pramlintide, glucagon-like polypeptide-1 analogs, and glucagon receptor agonists.⁵⁷ Other recent advances include stem cell replacement therapy for patients with T1D.⁵⁸ The ultra-long-acting biosimilar insulins are one medical innovation that has been stalled, rather than propelled, during the COVID-19 pandemic.⁵⁹

Diabetes Policy Advocacy

People with T1D require insulin to survive. The cost of insulin has increased in recent years, with some studies citing a 64% to 100% increase in the past decade.^60,61 In fact, 1 in 4 insulin users report that cost has impacted their insulin use, including rationing their insulin.⁶² Lockdowns during the COVID-19 pandemic stressed US families financially, increasing the urgency for insulin cost caps.

Although the COVID-19 pandemic halted national conversations on drug financing,⁶³ advocacy efforts have succeeded for specific populations. The new Medicare Part D Senior Savings Model will cap the cost of insulin at $35 for a 30-day supply,⁶⁴ and 20 states passed legislation capping insulin pricing.⁶² Efforts to codify national cost caps are under debate, including the passage of the Affordable Insulin Now Act, which passed the House in March 2022 and is currently under review in the Senate.⁶⁵

Funders and industry partners play a crucial role in leading and supporting innovations that improve the lives of people with T1D and reduce society’s costs of living with the disease. Data infrastructure is critical to supporting population health. While building the data infrastructure to support population health is both time- and resource-intensive, private foundations such as Helmsley are uniquely positioned—and have a responsibility—to take large, informed risks to help reach all communities with T1D.

The T1DX-QI is the largest source of population health data on T1D in the United States and is becoming the premiere data authority on its incidence, prevalence, and outcomes. The T1DX-QI enables a robust understanding of T1D-related health trends at the population level, as well as trends among clinics and providers. Pilot centers in the T1DX-QI have reported reductions in patients’ A1c and acute diabetes-related events, as well as improvements in device usage and depression screening. The ability to capture changes speaks to the promise and power of these data to demonstrate the clinical impact of QI interventions and to support the spread of best practices and learnings across health systems.

Additional philanthropic efforts have supported innovation in the last 2 years. For example, the JDRF, a nonprofit philanthropic equity firm, has supported efforts in developing artificial pancreas systems and cell therapies currently in clinical trials like teplizumab, a drug that has demonstrated delayed onset of T1D through JDRF’s T1D Fund.⁶⁶ Industry partners also have an opportunity for significant influence in this area, as they continue to fund meaningful projects to advance care for people with T1D.⁶⁷

Conclusion

We are optimistic that the innovations summarized here describe a shift in the tide of equitable T1D outcomes; however, future work is required to explore additional health equity successes that do not further exacerbate inequities. We also see further opportunities for innovative ways to engage people with T1D in their health care through conversations on social determinants of health and societal structures.

Corresponding author: Ann Mungmode, MPH, T1D Exchange, 11 Avenue de Lafayette, Boston, MA 02111; Email: amungmode@t1dexchange.org

Disclosures: Dr. Ebekozien serve(d) as a director, officer, partner, employee, advisor, consultant, or trustee for the Medtronic Advisory Board and received research grants from Medtronic Diabetes, Eli Lilly, and Dexcom.

Funding: The T1DX-QI is funded by The Leona M. and Harry B. Hemsley Charitable Trust.

From the T1D Exchange, Boston, MA (Ann Mungmode, Nicole Rioles, Jesse Cases, Dr. Ebekozien); The Leona M. and Harry B. Hemsley Charitable Trust, New York, NY (Laurel Koester); and the University of Mississippi School of Population Health, Jackson, MS (Dr. Ebekozien).

Abstract

There have been remarkable innovations in diabetes management since the start of the COVID-19 pandemic, but these groundbreaking innovations are drawing limited focus as the field focuses on the adverse impact of the pandemic on patients with diabetes. This article reviews select population health innovations in diabetes management that have become available over the past 2 years of the COVID-19 pandemic from the perspective of the T1D Exchange Quality Improvement Collaborative, a learning health network that focuses on improving care and outcomes for individuals with type 1 diabetes (T1D). Such innovations include expanded telemedicine access, collection of real-world data, machine learning and artificial intelligence, and new diabetes medications and devices. In addition, multiple innovative studies have been undertaken to explore contributors to health inequities in diabetes, and advocacy efforts for specific populations have been successful. Looking to the future, work is required to explore additional health equity successes that do not further exacerbate inequities and to look for additional innovative ways to engage people with T1D in their health care through conversations on social determinants of health and societal structures.

Keywords: type 1 diabetes, learning health network, continuous glucose monitoring, health equity

One in 10 people in the United States has diabetes.¹ Diabetes is the nation’s second leading cause of death, costing the US health system more than $300 billion annually.² The COVID-19 pandemic presented additional health burdens for people living with diabetes. For example, preexisting diabetes was identified as a risk factor for COVID-19–associated morbidity and mortality.^3,4 Over the past 2 years, there have been remarkable innovations in diabetes management, including stem cell therapy and new medication options. Additionally, improved technology solutions have aided in diabetes management through continuous glucose monitors (CGM), smart insulin pens, advanced hybrid closed-loop systems, and continuous subcutaneous insulin injections.^5,6 Unfortunately, these groundbreaking innovations are drawing limited focus, as the field is rightfully focused on the adverse impact of the pandemic on patients with diabetes.

Learning health networks like the T1D Exchange Quality Improvement Collaborative (T1DX-QI) have implemented some of these innovative solutions to improve care for people with diabetes.⁷ T1DX-QI has more than 50 data-sharing endocrinology centers that care for over 75,000 people with diabetes across the United States (Figure 1). Centers participating in the T1DX-QI use quality improvement (QI) and implementation science methods to quickly translate research into evidence-based clinical practice. T1DX-QI leads diabetes population health and health system research and supports widespread transferability across health care organizations through regular collaborative calls, conferences, and case study documentation.⁸

In this review, we summarize impactful population health innovations in diabetes management that have become available over the past 2 years of the COVID-19 pandemic from the perspective of T1DX-QI (see Figure 2 for relevant definitions). This review is limited in scope and is not meant to be an exhaustive list of innovations. The review also reflects significant changes from the perspective of academic diabetes centers, which may not apply to rural or primary care diabetes practices.

The first (A.M.), second (H.H.), and senior (O.E.) authors conducted a scoping review of published literature using terms related to diabetes, population health, and innovation on PubMed Central and Google Scholar for the period March 2020 to June 2022. To complement the review, A.M. and O.E. also reviewed abstracts from presentations at major international diabetes conferences, including the American Diabetes Association (ADA), the International Society for Pediatric and Adolescent Diabetes (ISPAD), the T1DX-QI Learning Session Conference, and the Advanced Technologies & Treatments for Diabetes (ATTD) 2020 to 2022 conferences.^9-14 The authors also searched FDA.gov and ClinicalTrials.gov for relevant insights. A.M. and O.E. sorted the reviewed literature into major themes (Figure 3) from the population health improvement perspective of the T1DX-QI.

Population Health Innovations in Diabetes Management

Expansion of Telemedicine Access

Telemedicine is cost-effective for patients with diabetes,¹⁵ including those with complex cases.¹⁶ Before the COVID-19 pandemic, telemedicine and virtual care were rare in diabetes management. However, the pandemic offered a new opportunity to expand the practice of telemedicine in diabetes management. A study from the T1DX-QI showed that telemedicine visits grew from comprising <1% of visits pre-pandemic (December 2019) to 95.2% during the pandemic (August 2020).¹⁷ Additional studies, like those conducted by Phillip et al,¹⁸ confirmed the noninferiority of telemedicine practice for patients with diabetes.Telemedicine was also found to be an effective strategy to educate patients on the use of diabetes technologies.¹⁹

Real-World Data and Disease Surveillance

As the COVID-19 pandemic exacerbated outcomes for people with type 1 diabetes (T1D), a need arose to understand the immediate effects of the pandemic on people with T1D through real-world data and disease surveillance. In April 2020, the T1DX-QI initiated a multicenter surveillance study to collect data and analyze the impact of COVID-19 on people with T1D. The existing health collaborative served as a springboard for robust surveillance study, documenting numerous works on the effects of COVID-19.^3,4,20-28 Other investigators also embraced the power of real-world surveillance and real-world data.^29,30

Big Data, Machine Learning, and Artificial Intelligence

The past 2 years have seen a shift toward embracing the incredible opportunity to tap the large volume of data generated from routine care for practical insights.³¹ In particular, researchers have demonstrated the widespread application of machine learning and artificial intelligence to improve diabetes management.³² The T1DX-QI also harnessed the growing power of big data by expanding the functionality of innovative benchmarking software. The T1DX QI Portal uses electronic medical record data of diabetes patients for clinic-to-clinic benchmarking and data analysis, using business intelligence solutions.³³

Health Equity

While inequities across various health outcomes have been well documented for years,³⁴ the COVID-19 pandemic further exaggerated racial/ethnic health inequities in T1D.^23,35 In response, several organizations have outlined specific strategies to address these health inequities. Emboldened by the pandemic, the T1DX-QI announced a multipronged approach to address health inequities among patients with T1D through the Health Equity Advancement Lab (HEAL).³⁶ One of HEAL’s main components is using real-world data to champion population-level insights and demonstrate progress in QI efforts.

Multiple innovative studies have been undertaken to explore contributors to health inequities in diabetes, and these studies are expanding our understanding of the chasm.³⁷ There have also been innovative solutions to addressing these inequities, with multiple studies published over the past 2 years.³⁸ A source of inequity among patients with T1D is the lack of representation of racial/ethnic minorities with T1D in clinical trials.³⁹ The T1DX-QI suggests that the equity-adapted framework for QI can be applied by research leaders to support trial diversity and representation, ensuring future device innovations are meaningful for all people with T1D.⁴⁰

Diabetes Devices

Glucose monitoring and insulin therapy are vital tools to support individuals living with T1D, and devices such as CGM and insulin pumps have become the standard of care for diabetes management (Table).⁴¹ Innovations in diabetes technology and device access are imperative for a chronic disease with no cure.

The COVID-19 pandemic created an opportunity to increase access to diabetes devices in inpatient settings. In 2020, the US Food and Drug Administration expanded the use of CGM to support remote monitoring of patients in inpatient hospital settings, simultaneously supporting the glucose monitoring needs of patients with T1D and reducing COVID-19 transmission through reduced patient-clinician contact.⁴² This effort has been expanded and will continue in 2022 and beyond,⁴³ and aligns with the growing consensus that supports patients wearing both CGMs and insulin pumps in ambulatory settings to improve patient health outcomes.⁴⁴

Since 2020, innovations in diabetes technology have improved and increased the variety of options available to people with T1D and made them easier to use (Table). New, advanced hybrid closed-loop systems have progressed to offer Bluetooth features, including automatic software upgrades, tubeless systems, and the ability to allow parents to use their smartphones to bolus for children.^45-47 The next big step in insulin delivery innovation is the release of functioning, fully closed loop systems, of which several are currently in clinical trials.⁴⁸ These systems support reduced hypoglycemia and improved time in range.⁴⁹

Additional innovations in insulin delivery have improved the user experience and expanded therapeutic options, including a variety of smart insulin pens complete with dosing logs^50,51 and even a patch to deliver insulin without the burden of injections.⁵² As barriers to diabetes technology persist,⁵³ innovations in alternate insulin delivery provide people with T1D more options to align with their personal access and technology preferences.

Innovations in CGM address cited barriers to their use, including size or overall wear.^53-55 CGMs released in the past few years are smaller in physical size, have longer durations of time between changings, are more accurate, and do not require calibrations for accuracy.

New Diabetes Medications

Many new medications and therapeutic advances have become available in the past 2 years.⁵⁶ Additionally, more medications are being tested as adjunct therapies to support glycemic management in patients with T1D, including metformin, sodium-glucose cotransporter 1 and 2 inhibitors, pramlintide, glucagon-like polypeptide-1 analogs, and glucagon receptor agonists.⁵⁷ Other recent advances include stem cell replacement therapy for patients with T1D.⁵⁸ The ultra-long-acting biosimilar insulins are one medical innovation that has been stalled, rather than propelled, during the COVID-19 pandemic.⁵⁹

Diabetes Policy Advocacy

People with T1D require insulin to survive. The cost of insulin has increased in recent years, with some studies citing a 64% to 100% increase in the past decade.^60,61 In fact, 1 in 4 insulin users report that cost has impacted their insulin use, including rationing their insulin.⁶² Lockdowns during the COVID-19 pandemic stressed US families financially, increasing the urgency for insulin cost caps.

Although the COVID-19 pandemic halted national conversations on drug financing,⁶³ advocacy efforts have succeeded for specific populations. The new Medicare Part D Senior Savings Model will cap the cost of insulin at $35 for a 30-day supply,⁶⁴ and 20 states passed legislation capping insulin pricing.⁶² Efforts to codify national cost caps are under debate, including the passage of the Affordable Insulin Now Act, which passed the House in March 2022 and is currently under review in the Senate.⁶⁵

Funders and industry partners play a crucial role in leading and supporting innovations that improve the lives of people with T1D and reduce society’s costs of living with the disease. Data infrastructure is critical to supporting population health. While building the data infrastructure to support population health is both time- and resource-intensive, private foundations such as Helmsley are uniquely positioned—and have a responsibility—to take large, informed risks to help reach all communities with T1D.

The T1DX-QI is the largest source of population health data on T1D in the United States and is becoming the premiere data authority on its incidence, prevalence, and outcomes. The T1DX-QI enables a robust understanding of T1D-related health trends at the population level, as well as trends among clinics and providers. Pilot centers in the T1DX-QI have reported reductions in patients’ A1c and acute diabetes-related events, as well as improvements in device usage and depression screening. The ability to capture changes speaks to the promise and power of these data to demonstrate the clinical impact of QI interventions and to support the spread of best practices and learnings across health systems.

Additional philanthropic efforts have supported innovation in the last 2 years. For example, the JDRF, a nonprofit philanthropic equity firm, has supported efforts in developing artificial pancreas systems and cell therapies currently in clinical trials like teplizumab, a drug that has demonstrated delayed onset of T1D through JDRF’s T1D Fund.⁶⁶ Industry partners also have an opportunity for significant influence in this area, as they continue to fund meaningful projects to advance care for people with T1D.⁶⁷

Conclusion

We are optimistic that the innovations summarized here describe a shift in the tide of equitable T1D outcomes; however, future work is required to explore additional health equity successes that do not further exacerbate inequities. We also see further opportunities for innovative ways to engage people with T1D in their health care through conversations on social determinants of health and societal structures.

Corresponding author: Ann Mungmode, MPH, T1D Exchange, 11 Avenue de Lafayette, Boston, MA 02111; Email: amungmode@t1dexchange.org

Disclosures: Dr. Ebekozien serve(d) as a director, officer, partner, employee, advisor, consultant, or trustee for the Medtronic Advisory Board and received research grants from Medtronic Diabetes, Eli Lilly, and Dexcom.

Funding: The T1DX-QI is funded by The Leona M. and Harry B. Hemsley Charitable Trust.

Study 1 Overview (Bayliss et al)

Objective: To examine the effect of a deprescribing educational intervention on medication use in older adults with cognitive impairment.

Design: This was a pragmatic, cluster randomized trial conducted in 8 primary care clinics that are part of a nonprofit health care system.

Setting and participants: The primary care clinic populations ranged from 170 to 1125 patients per clinic. The primary care clinics were randomly assigned to intervention or control using a uniform distribution in blocks by clinic size. Eligibility criteria for participants at those practices included age 65 years or older; health plan enrollment at least 1 year prior to intervention; diagnosis of Alzheimer disease and related dementia (ADRD) or mild cognitive impairment (MCI) by International Statistical Classification of Diseases and Related Health Problems, Tenth Revision code or from problem list; 1 or more chronic conditions from those in the Chronic Conditions Warehouse; and 5 or more long-term medications. Those who scheduled a visit at their primary care clinic in advance were eligible for the intervention. Primary care clinicians in intervention clinics were eligible to receive the clinician portion of the intervention. A total of 1433 participants were enrolled in the intervention group, and 1579 participants were enrolled in the control group.

Intervention: The intervention included 2 components: a patient and family component with materials mailed in advance of their primary care visits and a clinician component comprising monthly educational materials on deprescribing and notification in the electronic health record about visits with patient participants. The patient and family component consisted of a brochure titled “Managing Medication” and a questionnaire on attitudes toward deprescribing intended to educate patients and family about deprescribing. Clinicians at intervention clinics received an educational presentation at a monthly clinician meeting as well as tip sheets and a poster on deprescribing topics, and they also were notified of upcoming appointments with patients who received the patient component of the intervention. For the control group, patients and family did not receive any materials, and clinicians did not receive intervention materials or notification of participants enrolled in the trial. Usual care in both intervention and control groups included medication reconciliation and electronic health record alerts for potentially high-risk medications.

Main outcome measures: The primary outcomes of the study were the number of long-term medications per individual and the proportion of patients prescribed 1 or more potentially inappropriate medications. Outcome measurements were extracted from the electronic clinical data, and outcomes were assessed at 6 months, which involved comparing counts of medications at baseline with medications at 6 months. Long-term medications were defined as medications that are prescribed for 28 days or more based on pharmacy dispensing data. Potentially inappropriate medications (PIMs) were defined using the Beers list of medications to avoid in those with cognitive impairment and opioid medications. Analyses were conducted as intention to treat.

Main results: In the intervention group and control group, 56.2% and 54.4% of participants were women, and the mean age was 80.1 years (SD, 7.2) and 79.9 years (SD, 7.5), respectively. At baseline, the mean number of long-term medications was 7.0 (SD, 2.1) in the intervention group and 7.0 (SD, 2.2) in the control group. The proportion of patients taking any PIMs was 30.5% in the intervention group and 29.6% in the control group. At 6 months, the mean number of long-term medications was 6.4 in the intervention group and 6.5 in the control group, with an adjusted difference of –0.1 (95% CI, –0.2 to 0.04; P = .14); the proportion of patients with any PIMs was 17.8% in the intervention group and 20.9% in the control group, with an adjusted difference of –3.2% (95% CI, –6.2 to 0.4; P = .08). Preplanned analyses to examine subgroup differences for those with a higher number of medications (7+ vs 5 or 6 medications) did not find different effects of the intervention.

Conclusion: This educational intervention on deprescribing did not result in reductions in the number of medications or the use of PIMs in patients with cognitive impairment.

Study 2 Overview (Gedde et al)

Objective: To examine the effect of a deprescribing intervention (COSMOS) on medication use for nursing home residents.

Design: This was a randomized clinical trial.

Setting and participants: This trial was conducted in 67 units in 33 nursing homes in Norway. Participants were nursing home residents recruited from August 2014 to March 2015. Inclusion criteria included adults aged 65 years and older with at least 2 years of residency in nursing homes. Exclusion criteria included diagnosis of schizophrenia and a life expectancy of 6 months or less. Participants were followed for 4 months; participants were considered lost to follow-up if they died or moved from the nursing home unit. The analyses were per protocol and did not include those lost to follow-up or those who did not undergo a medication review in the intervention group. A total of 217 and 211 residents were included in the intervention and control groups, respectively.

Intervention: The intervention contained 5 components: communication and advance care planning, systematic pain management, medication reviews with collegial mentoring, organization of activities adjusted to needs and preferences, and safety. For medication review, the nursing home physician reviewed medications together with a nurse and study physicians who provided mentoring. The medication review involved a structured process that used assessment tools for behavioral and psychological symptoms of dementia (BPSD), activities of daily living (ADL), pain, cognitive status, well-being and quality of life, and clinical metrics of blood pressure, pulse, and body mass index. The study utilized the START/STOPP criteria¹ for medication use in addition to a list of medications with anticholinergic properties for the medication review. In addition, drug interactions were documented through a drug interaction database; the team also incorporated patient wishes and concerns in the medication reviews. The nursing home physician made final decisions on medications. For the control group, nursing home residents received usual care without this intervention.

Main outcome measures: The primary outcome of the study was the mean change in the number of prescribed psychotropic medications, both regularly scheduled and total medications (which also included on-demand drugs) received at 4 months when compared to baseline. Psychotropic medications included antipsychotics, anxiolytics, hypnotics or sedatives, antidepressants, and antidementia drugs. Secondary outcomes included mean changes in BPSD using the Neuropsychiatric Inventory-Nursing home version (NPI-NH) and the Cornell Scale for Depression for Dementia (CSDD) and ADL using the Physical Self Maintenance Scale (PSMS).

Main results: In both the intervention and control groups, 76% of participants were women, and mean age was 86.3 years (SD, 7.95) in the intervention group and 86.6 years (SD, 7.21) in the control group. At baseline, the mean number of total medications was 10.9 (SD, 4.6) in the intervention group and 10.9 (SD, 4.7) in the control group, and the mean number of psychotropic medications was 2.2 (SD, 1.6) and 2.2 (SD, 1.7) in the intervention and control groups, respectively. At 4 months, the mean change from baseline of total psychotropic medications was –0.34 in the intervention group and 0.01 in the control group (P < .001), and the mean change of regularly scheduled psychotropic medications was –0.21 in the intervention group and 0.02 in the control group (P < .001). Measures of BPSD and depression did not differ between intervention and control groups, and ADL showed a small improvement in the intervention group.

Conclusion: This intervention reduced the use of psychotropic medications in nursing home residents without worsening BPSD or depression and may have yielded improvements in ADL.

Commentary

Polypharmacy is common among older adults, as many of them have multiple chronic conditions and often take multiple medications for managing them. Polypharmacy increases the risk of drug interactions and adverse effects from medications; older adults who are frail and/or who have cognitive impairment are especially at risk. Reducing medication use, especially medications likely to cause adverse effects such as those with anticholinergic properties, has the potential to yield beneficial effects while reducing the burden of taking medications. A large randomized trial found that a pharmacist-led education intervention can be effective in reducing PIM use in community-dwelling older adults,² and that targeting patient motivation and capacity to deprescribe could be effective.³ This study by Bayliss and colleagues (Study 1), however, fell short of the effects seen in the earlier D-PRESCRIBE trial. One of the reasons for these findings may be that the clinician portion of the intervention was less intensive than that used in the earlier trial; specifically, in the present study, clinicians were not provided with or expected to utilize tools for structured medication review or deprescribing. Although the intervention primes the patient and family for discussions around deprescribing through the use of a brochure and questionnaire, the clinician portion of the intervention was less structured. Another example of an effective intervention that provided a more structured deprescribing intervention beyond education of clinicians utilized electronic decision-support to assist with deprescribing.⁴

The findings from the Gedde et al study (Study 2) are comparable to those of prior studies in the nursing home population,⁵ where participants are likely to take a large number of medications, including psychotropic medications, and are more likely to be frail. However, Gedde and colleagues employed a bundled intervention⁶ that included other components besides medication review, and thus it is unclear whether the effect on ADL can be attributed to the deprescribing of medications alone. Gedde et al’s finding that deprescribing can reduce the use of psychotropic medications while not leading to differences in behavioral and psychologic symptoms or depression is an important contribution to our knowledge about polypharmacy and deprescribing in older patients. Thus, nursing home residents, their families, and clinicians could expect that the deprescribing of psychotropic medications does not lead to worsening symptoms. Of note, the clinician portion of the intervention in the Gedde et al study was quite structured, and this structure may have contributed to the observed effects.

Applications for Clinical Practice and System Implementation

Both studies add to the literature on deprescribing and may offer options for researchers and clinicians who are considering potential components of an effective deprescribing intervention. Patient activation for deprescribing via the methods used in these 2 studies may help to prime patients for conversations about deprescribing; however, as shown by the Bayliss et al study, a more structured approach to clinical encounters may be needed when deprescribing, such as the use of tools in the electronic health record, in order to reduce the use of medication deemed unnecessary or potentially harmful. Further studies should examine the effect of deprescribing on medication use, but perhaps even more importantly, how deprescribing impacts patient outcomes both in terms of risks and benefits.

Practice Points

• A more structured approach to clinical encounters (eg, the use of tools in the electronic health record) may be needed when deprescribing unnecessary or potentially harmful medications in older patients in community settings.
• In the nursing home setting, structured deprescribing intervention can reduce the use of psychotropic medications while not leading to differences in behavioral and psychologic symptoms or depression.

–William W. Hung, MD, MPH

Study 1 Overview (Bayliss et al)

Objective: To examine the effect of a deprescribing educational intervention on medication use in older adults with cognitive impairment.

Design: This was a pragmatic, cluster randomized trial conducted in 8 primary care clinics that are part of a nonprofit health care system.

Setting and participants: The primary care clinic populations ranged from 170 to 1125 patients per clinic. The primary care clinics were randomly assigned to intervention or control using a uniform distribution in blocks by clinic size. Eligibility criteria for participants at those practices included age 65 years or older; health plan enrollment at least 1 year prior to intervention; diagnosis of Alzheimer disease and related dementia (ADRD) or mild cognitive impairment (MCI) by International Statistical Classification of Diseases and Related Health Problems, Tenth Revision code or from problem list; 1 or more chronic conditions from those in the Chronic Conditions Warehouse; and 5 or more long-term medications. Those who scheduled a visit at their primary care clinic in advance were eligible for the intervention. Primary care clinicians in intervention clinics were eligible to receive the clinician portion of the intervention. A total of 1433 participants were enrolled in the intervention group, and 1579 participants were enrolled in the control group.

Intervention: The intervention included 2 components: a patient and family component with materials mailed in advance of their primary care visits and a clinician component comprising monthly educational materials on deprescribing and notification in the electronic health record about visits with patient participants. The patient and family component consisted of a brochure titled “Managing Medication” and a questionnaire on attitudes toward deprescribing intended to educate patients and family about deprescribing. Clinicians at intervention clinics received an educational presentation at a monthly clinician meeting as well as tip sheets and a poster on deprescribing topics, and they also were notified of upcoming appointments with patients who received the patient component of the intervention. For the control group, patients and family did not receive any materials, and clinicians did not receive intervention materials or notification of participants enrolled in the trial. Usual care in both intervention and control groups included medication reconciliation and electronic health record alerts for potentially high-risk medications.

Main outcome measures: The primary outcomes of the study were the number of long-term medications per individual and the proportion of patients prescribed 1 or more potentially inappropriate medications. Outcome measurements were extracted from the electronic clinical data, and outcomes were assessed at 6 months, which involved comparing counts of medications at baseline with medications at 6 months. Long-term medications were defined as medications that are prescribed for 28 days or more based on pharmacy dispensing data. Potentially inappropriate medications (PIMs) were defined using the Beers list of medications to avoid in those with cognitive impairment and opioid medications. Analyses were conducted as intention to treat.

Main results: In the intervention group and control group, 56.2% and 54.4% of participants were women, and the mean age was 80.1 years (SD, 7.2) and 79.9 years (SD, 7.5), respectively. At baseline, the mean number of long-term medications was 7.0 (SD, 2.1) in the intervention group and 7.0 (SD, 2.2) in the control group. The proportion of patients taking any PIMs was 30.5% in the intervention group and 29.6% in the control group. At 6 months, the mean number of long-term medications was 6.4 in the intervention group and 6.5 in the control group, with an adjusted difference of –0.1 (95% CI, –0.2 to 0.04; P = .14); the proportion of patients with any PIMs was 17.8% in the intervention group and 20.9% in the control group, with an adjusted difference of –3.2% (95% CI, –6.2 to 0.4; P = .08). Preplanned analyses to examine subgroup differences for those with a higher number of medications (7+ vs 5 or 6 medications) did not find different effects of the intervention.

Conclusion: This educational intervention on deprescribing did not result in reductions in the number of medications or the use of PIMs in patients with cognitive impairment.

Study 2 Overview (Gedde et al)

Objective: To examine the effect of a deprescribing intervention (COSMOS) on medication use for nursing home residents.

Design: This was a randomized clinical trial.

Setting and participants: This trial was conducted in 67 units in 33 nursing homes in Norway. Participants were nursing home residents recruited from August 2014 to March 2015. Inclusion criteria included adults aged 65 years and older with at least 2 years of residency in nursing homes. Exclusion criteria included diagnosis of schizophrenia and a life expectancy of 6 months or less. Participants were followed for 4 months; participants were considered lost to follow-up if they died or moved from the nursing home unit. The analyses were per protocol and did not include those lost to follow-up or those who did not undergo a medication review in the intervention group. A total of 217 and 211 residents were included in the intervention and control groups, respectively.

Intervention: The intervention contained 5 components: communication and advance care planning, systematic pain management, medication reviews with collegial mentoring, organization of activities adjusted to needs and preferences, and safety. For medication review, the nursing home physician reviewed medications together with a nurse and study physicians who provided mentoring. The medication review involved a structured process that used assessment tools for behavioral and psychological symptoms of dementia (BPSD), activities of daily living (ADL), pain, cognitive status, well-being and quality of life, and clinical metrics of blood pressure, pulse, and body mass index. The study utilized the START/STOPP criteria¹ for medication use in addition to a list of medications with anticholinergic properties for the medication review. In addition, drug interactions were documented through a drug interaction database; the team also incorporated patient wishes and concerns in the medication reviews. The nursing home physician made final decisions on medications. For the control group, nursing home residents received usual care without this intervention.

Main outcome measures: The primary outcome of the study was the mean change in the number of prescribed psychotropic medications, both regularly scheduled and total medications (which also included on-demand drugs) received at 4 months when compared to baseline. Psychotropic medications included antipsychotics, anxiolytics, hypnotics or sedatives, antidepressants, and antidementia drugs. Secondary outcomes included mean changes in BPSD using the Neuropsychiatric Inventory-Nursing home version (NPI-NH) and the Cornell Scale for Depression for Dementia (CSDD) and ADL using the Physical Self Maintenance Scale (PSMS).

Main results: In both the intervention and control groups, 76% of participants were women, and mean age was 86.3 years (SD, 7.95) in the intervention group and 86.6 years (SD, 7.21) in the control group. At baseline, the mean number of total medications was 10.9 (SD, 4.6) in the intervention group and 10.9 (SD, 4.7) in the control group, and the mean number of psychotropic medications was 2.2 (SD, 1.6) and 2.2 (SD, 1.7) in the intervention and control groups, respectively. At 4 months, the mean change from baseline of total psychotropic medications was –0.34 in the intervention group and 0.01 in the control group (P < .001), and the mean change of regularly scheduled psychotropic medications was –0.21 in the intervention group and 0.02 in the control group (P < .001). Measures of BPSD and depression did not differ between intervention and control groups, and ADL showed a small improvement in the intervention group.

Conclusion: This intervention reduced the use of psychotropic medications in nursing home residents without worsening BPSD or depression and may have yielded improvements in ADL.

Commentary

Polypharmacy is common among older adults, as many of them have multiple chronic conditions and often take multiple medications for managing them. Polypharmacy increases the risk of drug interactions and adverse effects from medications; older adults who are frail and/or who have cognitive impairment are especially at risk. Reducing medication use, especially medications likely to cause adverse effects such as those with anticholinergic properties, has the potential to yield beneficial effects while reducing the burden of taking medications. A large randomized trial found that a pharmacist-led education intervention can be effective in reducing PIM use in community-dwelling older adults,² and that targeting patient motivation and capacity to deprescribe could be effective.³ This study by Bayliss and colleagues (Study 1), however, fell short of the effects seen in the earlier D-PRESCRIBE trial. One of the reasons for these findings may be that the clinician portion of the intervention was less intensive than that used in the earlier trial; specifically, in the present study, clinicians were not provided with or expected to utilize tools for structured medication review or deprescribing. Although the intervention primes the patient and family for discussions around deprescribing through the use of a brochure and questionnaire, the clinician portion of the intervention was less structured. Another example of an effective intervention that provided a more structured deprescribing intervention beyond education of clinicians utilized electronic decision-support to assist with deprescribing.⁴

The findings from the Gedde et al study (Study 2) are comparable to those of prior studies in the nursing home population,⁵ where participants are likely to take a large number of medications, including psychotropic medications, and are more likely to be frail. However, Gedde and colleagues employed a bundled intervention⁶ that included other components besides medication review, and thus it is unclear whether the effect on ADL can be attributed to the deprescribing of medications alone. Gedde et al’s finding that deprescribing can reduce the use of psychotropic medications while not leading to differences in behavioral and psychologic symptoms or depression is an important contribution to our knowledge about polypharmacy and deprescribing in older patients. Thus, nursing home residents, their families, and clinicians could expect that the deprescribing of psychotropic medications does not lead to worsening symptoms. Of note, the clinician portion of the intervention in the Gedde et al study was quite structured, and this structure may have contributed to the observed effects.

Applications for Clinical Practice and System Implementation

Both studies add to the literature on deprescribing and may offer options for researchers and clinicians who are considering potential components of an effective deprescribing intervention. Patient activation for deprescribing via the methods used in these 2 studies may help to prime patients for conversations about deprescribing; however, as shown by the Bayliss et al study, a more structured approach to clinical encounters may be needed when deprescribing, such as the use of tools in the electronic health record, in order to reduce the use of medication deemed unnecessary or potentially harmful. Further studies should examine the effect of deprescribing on medication use, but perhaps even more importantly, how deprescribing impacts patient outcomes both in terms of risks and benefits.

Practice Points

• A more structured approach to clinical encounters (eg, the use of tools in the electronic health record) may be needed when deprescribing unnecessary or potentially harmful medications in older patients in community settings.
• In the nursing home setting, structured deprescribing intervention can reduce the use of psychotropic medications while not leading to differences in behavioral and psychologic symptoms or depression.

–William W. Hung, MD, MPH

Study 1 Overview (Bayliss et al)

Objective: To examine the effect of a deprescribing educational intervention on medication use in older adults with cognitive impairment.

Design: This was a pragmatic, cluster randomized trial conducted in 8 primary care clinics that are part of a nonprofit health care system.

Setting and participants: The primary care clinic populations ranged from 170 to 1125 patients per clinic. The primary care clinics were randomly assigned to intervention or control using a uniform distribution in blocks by clinic size. Eligibility criteria for participants at those practices included age 65 years or older; health plan enrollment at least 1 year prior to intervention; diagnosis of Alzheimer disease and related dementia (ADRD) or mild cognitive impairment (MCI) by International Statistical Classification of Diseases and Related Health Problems, Tenth Revision code or from problem list; 1 or more chronic conditions from those in the Chronic Conditions Warehouse; and 5 or more long-term medications. Those who scheduled a visit at their primary care clinic in advance were eligible for the intervention. Primary care clinicians in intervention clinics were eligible to receive the clinician portion of the intervention. A total of 1433 participants were enrolled in the intervention group, and 1579 participants were enrolled in the control group.

Intervention: The intervention included 2 components: a patient and family component with materials mailed in advance of their primary care visits and a clinician component comprising monthly educational materials on deprescribing and notification in the electronic health record about visits with patient participants. The patient and family component consisted of a brochure titled “Managing Medication” and a questionnaire on attitudes toward deprescribing intended to educate patients and family about deprescribing. Clinicians at intervention clinics received an educational presentation at a monthly clinician meeting as well as tip sheets and a poster on deprescribing topics, and they also were notified of upcoming appointments with patients who received the patient component of the intervention. For the control group, patients and family did not receive any materials, and clinicians did not receive intervention materials or notification of participants enrolled in the trial. Usual care in both intervention and control groups included medication reconciliation and electronic health record alerts for potentially high-risk medications.

Main outcome measures: The primary outcomes of the study were the number of long-term medications per individual and the proportion of patients prescribed 1 or more potentially inappropriate medications. Outcome measurements were extracted from the electronic clinical data, and outcomes were assessed at 6 months, which involved comparing counts of medications at baseline with medications at 6 months. Long-term medications were defined as medications that are prescribed for 28 days or more based on pharmacy dispensing data. Potentially inappropriate medications (PIMs) were defined using the Beers list of medications to avoid in those with cognitive impairment and opioid medications. Analyses were conducted as intention to treat.

Main results: In the intervention group and control group, 56.2% and 54.4% of participants were women, and the mean age was 80.1 years (SD, 7.2) and 79.9 years (SD, 7.5), respectively. At baseline, the mean number of long-term medications was 7.0 (SD, 2.1) in the intervention group and 7.0 (SD, 2.2) in the control group. The proportion of patients taking any PIMs was 30.5% in the intervention group and 29.6% in the control group. At 6 months, the mean number of long-term medications was 6.4 in the intervention group and 6.5 in the control group, with an adjusted difference of –0.1 (95% CI, –0.2 to 0.04; P = .14); the proportion of patients with any PIMs was 17.8% in the intervention group and 20.9% in the control group, with an adjusted difference of –3.2% (95% CI, –6.2 to 0.4; P = .08). Preplanned analyses to examine subgroup differences for those with a higher number of medications (7+ vs 5 or 6 medications) did not find different effects of the intervention.

Conclusion: This educational intervention on deprescribing did not result in reductions in the number of medications or the use of PIMs in patients with cognitive impairment.

Study 2 Overview (Gedde et al)

Objective: To examine the effect of a deprescribing intervention (COSMOS) on medication use for nursing home residents.

Design: This was a randomized clinical trial.

Setting and participants: This trial was conducted in 67 units in 33 nursing homes in Norway. Participants were nursing home residents recruited from August 2014 to March 2015. Inclusion criteria included adults aged 65 years and older with at least 2 years of residency in nursing homes. Exclusion criteria included diagnosis of schizophrenia and a life expectancy of 6 months or less. Participants were followed for 4 months; participants were considered lost to follow-up if they died or moved from the nursing home unit. The analyses were per protocol and did not include those lost to follow-up or those who did not undergo a medication review in the intervention group. A total of 217 and 211 residents were included in the intervention and control groups, respectively.

Intervention: The intervention contained 5 components: communication and advance care planning, systematic pain management, medication reviews with collegial mentoring, organization of activities adjusted to needs and preferences, and safety. For medication review, the nursing home physician reviewed medications together with a nurse and study physicians who provided mentoring. The medication review involved a structured process that used assessment tools for behavioral and psychological symptoms of dementia (BPSD), activities of daily living (ADL), pain, cognitive status, well-being and quality of life, and clinical metrics of blood pressure, pulse, and body mass index. The study utilized the START/STOPP criteria¹ for medication use in addition to a list of medications with anticholinergic properties for the medication review. In addition, drug interactions were documented through a drug interaction database; the team also incorporated patient wishes and concerns in the medication reviews. The nursing home physician made final decisions on medications. For the control group, nursing home residents received usual care without this intervention.

Main outcome measures: The primary outcome of the study was the mean change in the number of prescribed psychotropic medications, both regularly scheduled and total medications (which also included on-demand drugs) received at 4 months when compared to baseline. Psychotropic medications included antipsychotics, anxiolytics, hypnotics or sedatives, antidepressants, and antidementia drugs. Secondary outcomes included mean changes in BPSD using the Neuropsychiatric Inventory-Nursing home version (NPI-NH) and the Cornell Scale for Depression for Dementia (CSDD) and ADL using the Physical Self Maintenance Scale (PSMS).

Main results: In both the intervention and control groups, 76% of participants were women, and mean age was 86.3 years (SD, 7.95) in the intervention group and 86.6 years (SD, 7.21) in the control group. At baseline, the mean number of total medications was 10.9 (SD, 4.6) in the intervention group and 10.9 (SD, 4.7) in the control group, and the mean number of psychotropic medications was 2.2 (SD, 1.6) and 2.2 (SD, 1.7) in the intervention and control groups, respectively. At 4 months, the mean change from baseline of total psychotropic medications was –0.34 in the intervention group and 0.01 in the control group (P < .001), and the mean change of regularly scheduled psychotropic medications was –0.21 in the intervention group and 0.02 in the control group (P < .001). Measures of BPSD and depression did not differ between intervention and control groups, and ADL showed a small improvement in the intervention group.

Conclusion: This intervention reduced the use of psychotropic medications in nursing home residents without worsening BPSD or depression and may have yielded improvements in ADL.

Commentary

Polypharmacy is common among older adults, as many of them have multiple chronic conditions and often take multiple medications for managing them. Polypharmacy increases the risk of drug interactions and adverse effects from medications; older adults who are frail and/or who have cognitive impairment are especially at risk. Reducing medication use, especially medications likely to cause adverse effects such as those with anticholinergic properties, has the potential to yield beneficial effects while reducing the burden of taking medications. A large randomized trial found that a pharmacist-led education intervention can be effective in reducing PIM use in community-dwelling older adults,² and that targeting patient motivation and capacity to deprescribe could be effective.³ This study by Bayliss and colleagues (Study 1), however, fell short of the effects seen in the earlier D-PRESCRIBE trial. One of the reasons for these findings may be that the clinician portion of the intervention was less intensive than that used in the earlier trial; specifically, in the present study, clinicians were not provided with or expected to utilize tools for structured medication review or deprescribing. Although the intervention primes the patient and family for discussions around deprescribing through the use of a brochure and questionnaire, the clinician portion of the intervention was less structured. Another example of an effective intervention that provided a more structured deprescribing intervention beyond education of clinicians utilized electronic decision-support to assist with deprescribing.⁴

The findings from the Gedde et al study (Study 2) are comparable to those of prior studies in the nursing home population,⁵ where participants are likely to take a large number of medications, including psychotropic medications, and are more likely to be frail. However, Gedde and colleagues employed a bundled intervention⁶ that included other components besides medication review, and thus it is unclear whether the effect on ADL can be attributed to the deprescribing of medications alone. Gedde et al’s finding that deprescribing can reduce the use of psychotropic medications while not leading to differences in behavioral and psychologic symptoms or depression is an important contribution to our knowledge about polypharmacy and deprescribing in older patients. Thus, nursing home residents, their families, and clinicians could expect that the deprescribing of psychotropic medications does not lead to worsening symptoms. Of note, the clinician portion of the intervention in the Gedde et al study was quite structured, and this structure may have contributed to the observed effects.

Applications for Clinical Practice and System Implementation

Both studies add to the literature on deprescribing and may offer options for researchers and clinicians who are considering potential components of an effective deprescribing intervention. Patient activation for deprescribing via the methods used in these 2 studies may help to prime patients for conversations about deprescribing; however, as shown by the Bayliss et al study, a more structured approach to clinical encounters may be needed when deprescribing, such as the use of tools in the electronic health record, in order to reduce the use of medication deemed unnecessary or potentially harmful. Further studies should examine the effect of deprescribing on medication use, but perhaps even more importantly, how deprescribing impacts patient outcomes both in terms of risks and benefits.

Practice Points

• A more structured approach to clinical encounters (eg, the use of tools in the electronic health record) may be needed when deprescribing unnecessary or potentially harmful medications in older patients in community settings.
• In the nursing home setting, structured deprescribing intervention can reduce the use of psychotropic medications while not leading to differences in behavioral and psychologic symptoms or depression.

–William W. Hung, MD, MPH

Study 1 Overview (Oberhaus et al)

Objective: To compare the 3-Minute Diagnostic Confusion Assessment Method (3D-CAM) to the long-form Confusion Assessment Method (CAM) in detecting postoperative delirium.

Design: Prospective concurrent comparison of 3D-CAM and CAM evaluations in a cohort of postoperative geriatric patients.

Setting and participants: Eligible participants were patients aged 60 years or older undergoing major elective surgery at Barnes Jewish Hospital (St. Louis, Missouri) who were enrolled in ongoing clinical trials (PODCAST, ENGAGES, SATISFY-SOS) between 2015 and 2018. Surgeries were at least 2 hours in length and required general anesthesia, planned extubation, and a minimum 2-day hospital stay. Investigators were extensively trained in administering 3D-CAM and CAM instruments. Participants were evaluated 2 hours after the end of anesthesia care on the day of surgery, then daily until follow-up was completed per clinical trial protocol or until the participant was determined by CAM to be nondelirious for 3 consecutive days. For each evaluation, both 3D-CAM and CAM assessors approached the participant together, but the evaluation was conducted such that the 3D-CAM assessor was masked to the additional questions ascertained by the long-form CAM assessment. The 3D-CAM or CAM assessor independently scored their respective assessments blinded to the results of the other assessor.

Main outcome measures: Participants were concurrently evaluated for postoperative delirium by both 3D-CAM and long-form CAM assessments. Comparisons between 3D-CAM and CAM scores were made using Cohen κ with repeated measures, generalized linear mixed-effects model, and Bland-Altman analysis.

Main results: Sixteen raters performed 471 concurrent 3D-CAM and CAM assessments in 299 participants (mean [SD] age, 69 [6.5] years). Of these participants, 152 (50.8%) were men, 263 (88.0%) were White, and 211 (70.6%) underwent noncardiac surgery. Both instruments showed good intraclass correlation (0.98 for 3D-CAM, 0.84 for CAM) with good overall agreement (Cohen κ = 0.71; 95% CI, 0.58-0.83). The mixed-effects model indicated a significant disagreement between the 3D-CAM and CAM assessments (estimated difference in fixed effect, –0.68; 95% CI, –1.32 to –0.05; P = .04). The Bland-Altman analysis showed that the probability of a delirium diagnosis with the 3D-CAM was more than twice that with the CAM (probability ratio, 2.78; 95% CI, 2.44-3.23).

Conclusion: The high degree of agreement between 3D-CAM and long-form CAM assessments suggests that the former may be a pragmatic and easy-to-administer clinical tool to screen for postoperative delirium in vulnerable older surgical patients.

Study 2 Overview (Shenkin et al)

Objective: To assess the accuracy of the 4 ‘A’s Test (4AT) for delirium detection in the medical inpatient setting and to compare the 4AT to the CAM.

Design: Prospective randomized diagnostic test accuracy study.

Setting and participants: This study was conducted in emergency departments and acute medical wards at 3 UK sites (Edinburgh, Bradford, and Sheffield) and enrolled acute medical patients aged 70 years or older without acute life-threatening illnesses and/or coma. Assessors administering the delirium evaluation were nurses or graduate clinical research associates who underwent systematic training in delirium and delirium assessment. Additional training was provided to those administering the CAM but not to those administering the 4AT as the latter is designed to be administered without special training. First, all participants underwent a reference standard delirium assessment using Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition) (DSM-IV) criteria to derive a final definitive diagnosis of delirium via expert consensus (1 psychiatrist and 2 geriatricians). Then, the participants were randomized to either the 4AT or the comparator CAM group using computer-generated pseudo-random numbers, stratified by study site, with block allocation. All assessments were performed by pairs of independent assessors blinded to the results of the other assessment.

Main outcome measures: All participants were evaluated by the reference standard (DSM-IV criteria for delirium) and by either 4AT or CAM instruments for delirium. The accuracy of the 4AT instrument was evaluated by comparing its positive and negative predictive values, sensitivity, and specificity to the reference standard and analyzed via the area under the receiver operating characteristic curve. The diagnostic accuracy of 4AT, compared to the CAM, was evaluated by comparing positive and negative predictive values, sensitivity, and specificity using Fisher’s exact test. The overall performance of 4AT and CAM was summarized using Youden’s Index and the diagnostic odds ratio of sensitivity to specificity.

Results: All 843 individuals enrolled in the study were randomized and 785 were included in the analysis (23 withdrew, 3 lost contact, 32 indeterminate diagnosis, 2 missing outcome). Of the participants analyzed, the mean age was 81.4 [6.4] years, and 12.1% (95/785) had delirium by reference standard assessment, 14.3% (56/392) by 4AT, and 4.7% (18/384) by CAM. The 4AT group had an area under the receiver operating characteristic curve of 0.90 (95% CI, 0.84-0.96), a sensitivity of 76% (95% CI, 61%-87%), and a specificity of 94% (95% CI, 92%-97%). In comparison, the CAM group had a sensitivity of 40% (95% CI, 26%-57%) and a specificity of 100% (95% CI, 98%-100%).

Conclusions: The 4AT is a pragmatic screening test for delirium in a medical space that does not require special training to administer. The use of this instrument may help to improve delirium detection as a part of routine clinical care in hospitalized older adults.

Delirium is an acute confusional state marked by fluctuating mental status, inattention, disorganized thinking, and altered level of consciousness. It is exceedingly common in older patients in both surgical and medical settings and is associated with increased morbidity, mortality, hospital length of stay, institutionalization, and health care costs. Delirium is frequently underdiagnosed in the hospitalized setting, perhaps due to a combination of its waxing and waning nature and a lack of pragmatic and easily implementable screening tools that can be readily administered by clinicians and nonclinicians alike.¹ While the CAM is a well-validated instrument to diagnose delirium, it requires specific training in the rating of each of the cardinal features ascertained through a brief cognitive assessment and takes 5 to 10 minutes to complete. Taken together, given the high patient load for clinicians in the hospital setting, the validation and application of brief delirium screening instruments that can be reliably administered by nonphysicians and nonclinicians may enhance delirium detection in vulnerable patients and consequently improve their outcomes.

In Study 1, Oberhaus et al approach the challenge of underdiagnosing delirium in the postoperative setting by investigating whether the widely accepted long-form CAM and an abbreviated 3-minute version, the 3D-CAM, provide similar delirium detection in older surgical patients. The authors found that both instruments were reliable tests individually (high interrater reliability) and had good overall agreement. However, the 3D-CAM was more likely to yield a positive diagnosis of delirium compared to the long-form CAM, consistent with its purpose as a screening tool with a high sensitivity. It is important to emphasize that the 3D-CAM takes less time to administer, but also requires less extensive training and clinical knowledge than the long-form CAM. Therefore, this instrument meets the prerequisite of a brief screening test that can be rapidly administered by nonclinicians, and if affirmative, followed by a more extensive confirmatory test performed by a clinician. Limitations of this study include a lack of a reference standard structured interview conducted by a physician-rater to better determine the true diagnostic accuracy of both 3D-CAM and CAM assessments, and the use of convenience sampling at a single center, which reduces the generalizability of its findings.

In a similar vein, Shenkin et al in Study 2 attempt to evaluate the utility of the 4AT instrument in diagnosing delirium in older medical inpatients by testing the diagnostic accuracy of the 4AT against a reference standard (ie, DSM-IV–based evaluation by physicians) as well as comparing it to CAM. The 4AT takes less time (~2 minutes) and requires less knowledge and training to administer as compared to the CAM. The study showed that the abbreviated 4AT, compared to CAM, had a higher sensitivity (76% vs 40%) and lower specificity (94% vs 100%) in delirium detection. Thus, akin to the application of 3D-CAM in the postoperative setting, 4AT possesses key characteristics of a brief delirium screening test for older patients in the acute medical setting. In contrast to the Oberhaus et al study, a major strength of this study was the utilization of a reference standard that was validated by expert consensus. This allowed the 4AT and CAM assessments to be compared to a more objective standard, thereby directly testing their diagnostic performance in detecting delirium.

Application for Clinical Practice and System Implementation

The findings from both Study 1 and 2 suggest that using an abbreviated delirium instrument in both surgical and acute medical settings may provide a pragmatic and sensitive method to detect delirium in older patients. The brevity of administration of 3D-CAM (~3 minutes) and 4AT (~2 minutes), combined with their higher sensitivity for detecting delirium compared to CAM, make these instruments potentially effective rapid screening tests for delirium in hospitalized older patients. Importantly, the utilization of such instruments might be a feasible way to mitigate the issue of underdiagnosing delirium in the hospital.

Several additional aspects of these abbreviated delirium instruments increase their suitability for clinical application. Specifically, the 3D-CAM and 4AT require less extensive training and clinical knowledge to both administer and interpret the results than the CAM.² For instance, a multistage, multiday training for CAM is a key factor in maintaining its diagnostic accuracy.^3,4 In contrast, the 3D-CAM requires only a 1- to 2-hour training session, and the 4AT can be administered by a nonclinician without the need for instrument-specific training. Thus, implementation of these instruments can be particularly pragmatic in clinical settings in which the staff involved in delirium screening cannot undergo the substantial training required to administer CAM. Moreover, these abbreviated tests enable nonphysician care team members to assume the role of delirium screener in the hospital. Taken together, the adoption of these abbreviated instruments may facilitate brief screenings of delirium in older patients by caregivers who see them most often—nurses and certified nursing assistants—thereby improving early detection and prevention of delirium-related complications in the hospital.

The feasibility of using abbreviated delirium screening instruments in the hospital setting raises a system implementation question—if these instruments are designed to be administered by those with limited to no training, could nonclinicians, such as hospital volunteers, effectively take on delirium screening roles in the hospital? If volunteers are able to take on this role, the integration of hospital volunteers into the clinical team can greatly expand the capacity for delirium screening in the hospital setting. Further research is warranted to validate the diagnostic accuracy of 3D-CAM and 4AT by nonclinician administrators in order to more broadly adopt this approach to delirium screening.

Practice Points

• Abbreviated delirium screening tools such as 3D-CAM and 4AT may be pragmatic instruments to improve delirium detection in surgical and hospitalized older patients, respectively.
• Further studies are warranted to validate the diagnostic accuracy of 3D-CAM and 4AT by nonclinician administrators in order to more broadly adopt this approach to delirium screening.

Jared Doan, BS, and Fred Ko, MD
Geriatrics and Palliative Medicine, Icahn School of Medicine at Mount Sinai

Study 1 Overview (Oberhaus et al)

Objective: To compare the 3-Minute Diagnostic Confusion Assessment Method (3D-CAM) to the long-form Confusion Assessment Method (CAM) in detecting postoperative delirium.

Design: Prospective concurrent comparison of 3D-CAM and CAM evaluations in a cohort of postoperative geriatric patients.

Setting and participants: Eligible participants were patients aged 60 years or older undergoing major elective surgery at Barnes Jewish Hospital (St. Louis, Missouri) who were enrolled in ongoing clinical trials (PODCAST, ENGAGES, SATISFY-SOS) between 2015 and 2018. Surgeries were at least 2 hours in length and required general anesthesia, planned extubation, and a minimum 2-day hospital stay. Investigators were extensively trained in administering 3D-CAM and CAM instruments. Participants were evaluated 2 hours after the end of anesthesia care on the day of surgery, then daily until follow-up was completed per clinical trial protocol or until the participant was determined by CAM to be nondelirious for 3 consecutive days. For each evaluation, both 3D-CAM and CAM assessors approached the participant together, but the evaluation was conducted such that the 3D-CAM assessor was masked to the additional questions ascertained by the long-form CAM assessment. The 3D-CAM or CAM assessor independently scored their respective assessments blinded to the results of the other assessor.

Main outcome measures: Participants were concurrently evaluated for postoperative delirium by both 3D-CAM and long-form CAM assessments. Comparisons between 3D-CAM and CAM scores were made using Cohen κ with repeated measures, generalized linear mixed-effects model, and Bland-Altman analysis.

Main results: Sixteen raters performed 471 concurrent 3D-CAM and CAM assessments in 299 participants (mean [SD] age, 69 [6.5] years). Of these participants, 152 (50.8%) were men, 263 (88.0%) were White, and 211 (70.6%) underwent noncardiac surgery. Both instruments showed good intraclass correlation (0.98 for 3D-CAM, 0.84 for CAM) with good overall agreement (Cohen κ = 0.71; 95% CI, 0.58-0.83). The mixed-effects model indicated a significant disagreement between the 3D-CAM and CAM assessments (estimated difference in fixed effect, –0.68; 95% CI, –1.32 to –0.05; P = .04). The Bland-Altman analysis showed that the probability of a delirium diagnosis with the 3D-CAM was more than twice that with the CAM (probability ratio, 2.78; 95% CI, 2.44-3.23).

Conclusion: The high degree of agreement between 3D-CAM and long-form CAM assessments suggests that the former may be a pragmatic and easy-to-administer clinical tool to screen for postoperative delirium in vulnerable older surgical patients.

Study 2 Overview (Shenkin et al)

Objective: To assess the accuracy of the 4 ‘A’s Test (4AT) for delirium detection in the medical inpatient setting and to compare the 4AT to the CAM.

Design: Prospective randomized diagnostic test accuracy study.

Setting and participants: This study was conducted in emergency departments and acute medical wards at 3 UK sites (Edinburgh, Bradford, and Sheffield) and enrolled acute medical patients aged 70 years or older without acute life-threatening illnesses and/or coma. Assessors administering the delirium evaluation were nurses or graduate clinical research associates who underwent systematic training in delirium and delirium assessment. Additional training was provided to those administering the CAM but not to those administering the 4AT as the latter is designed to be administered without special training. First, all participants underwent a reference standard delirium assessment using Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition) (DSM-IV) criteria to derive a final definitive diagnosis of delirium via expert consensus (1 psychiatrist and 2 geriatricians). Then, the participants were randomized to either the 4AT or the comparator CAM group using computer-generated pseudo-random numbers, stratified by study site, with block allocation. All assessments were performed by pairs of independent assessors blinded to the results of the other assessment.

Main outcome measures: All participants were evaluated by the reference standard (DSM-IV criteria for delirium) and by either 4AT or CAM instruments for delirium. The accuracy of the 4AT instrument was evaluated by comparing its positive and negative predictive values, sensitivity, and specificity to the reference standard and analyzed via the area under the receiver operating characteristic curve. The diagnostic accuracy of 4AT, compared to the CAM, was evaluated by comparing positive and negative predictive values, sensitivity, and specificity using Fisher’s exact test. The overall performance of 4AT and CAM was summarized using Youden’s Index and the diagnostic odds ratio of sensitivity to specificity.

Results: All 843 individuals enrolled in the study were randomized and 785 were included in the analysis (23 withdrew, 3 lost contact, 32 indeterminate diagnosis, 2 missing outcome). Of the participants analyzed, the mean age was 81.4 [6.4] years, and 12.1% (95/785) had delirium by reference standard assessment, 14.3% (56/392) by 4AT, and 4.7% (18/384) by CAM. The 4AT group had an area under the receiver operating characteristic curve of 0.90 (95% CI, 0.84-0.96), a sensitivity of 76% (95% CI, 61%-87%), and a specificity of 94% (95% CI, 92%-97%). In comparison, the CAM group had a sensitivity of 40% (95% CI, 26%-57%) and a specificity of 100% (95% CI, 98%-100%).

Conclusions: The 4AT is a pragmatic screening test for delirium in a medical space that does not require special training to administer. The use of this instrument may help to improve delirium detection as a part of routine clinical care in hospitalized older adults.

Delirium is an acute confusional state marked by fluctuating mental status, inattention, disorganized thinking, and altered level of consciousness. It is exceedingly common in older patients in both surgical and medical settings and is associated with increased morbidity, mortality, hospital length of stay, institutionalization, and health care costs. Delirium is frequently underdiagnosed in the hospitalized setting, perhaps due to a combination of its waxing and waning nature and a lack of pragmatic and easily implementable screening tools that can be readily administered by clinicians and nonclinicians alike.¹ While the CAM is a well-validated instrument to diagnose delirium, it requires specific training in the rating of each of the cardinal features ascertained through a brief cognitive assessment and takes 5 to 10 minutes to complete. Taken together, given the high patient load for clinicians in the hospital setting, the validation and application of brief delirium screening instruments that can be reliably administered by nonphysicians and nonclinicians may enhance delirium detection in vulnerable patients and consequently improve their outcomes.

In Study 1, Oberhaus et al approach the challenge of underdiagnosing delirium in the postoperative setting by investigating whether the widely accepted long-form CAM and an abbreviated 3-minute version, the 3D-CAM, provide similar delirium detection in older surgical patients. The authors found that both instruments were reliable tests individually (high interrater reliability) and had good overall agreement. However, the 3D-CAM was more likely to yield a positive diagnosis of delirium compared to the long-form CAM, consistent with its purpose as a screening tool with a high sensitivity. It is important to emphasize that the 3D-CAM takes less time to administer, but also requires less extensive training and clinical knowledge than the long-form CAM. Therefore, this instrument meets the prerequisite of a brief screening test that can be rapidly administered by nonclinicians, and if affirmative, followed by a more extensive confirmatory test performed by a clinician. Limitations of this study include a lack of a reference standard structured interview conducted by a physician-rater to better determine the true diagnostic accuracy of both 3D-CAM and CAM assessments, and the use of convenience sampling at a single center, which reduces the generalizability of its findings.

In a similar vein, Shenkin et al in Study 2 attempt to evaluate the utility of the 4AT instrument in diagnosing delirium in older medical inpatients by testing the diagnostic accuracy of the 4AT against a reference standard (ie, DSM-IV–based evaluation by physicians) as well as comparing it to CAM. The 4AT takes less time (~2 minutes) and requires less knowledge and training to administer as compared to the CAM. The study showed that the abbreviated 4AT, compared to CAM, had a higher sensitivity (76% vs 40%) and lower specificity (94% vs 100%) in delirium detection. Thus, akin to the application of 3D-CAM in the postoperative setting, 4AT possesses key characteristics of a brief delirium screening test for older patients in the acute medical setting. In contrast to the Oberhaus et al study, a major strength of this study was the utilization of a reference standard that was validated by expert consensus. This allowed the 4AT and CAM assessments to be compared to a more objective standard, thereby directly testing their diagnostic performance in detecting delirium.

Application for Clinical Practice and System Implementation

The findings from both Study 1 and 2 suggest that using an abbreviated delirium instrument in both surgical and acute medical settings may provide a pragmatic and sensitive method to detect delirium in older patients. The brevity of administration of 3D-CAM (~3 minutes) and 4AT (~2 minutes), combined with their higher sensitivity for detecting delirium compared to CAM, make these instruments potentially effective rapid screening tests for delirium in hospitalized older patients. Importantly, the utilization of such instruments might be a feasible way to mitigate the issue of underdiagnosing delirium in the hospital.

Several additional aspects of these abbreviated delirium instruments increase their suitability for clinical application. Specifically, the 3D-CAM and 4AT require less extensive training and clinical knowledge to both administer and interpret the results than the CAM.² For instance, a multistage, multiday training for CAM is a key factor in maintaining its diagnostic accuracy.^3,4 In contrast, the 3D-CAM requires only a 1- to 2-hour training session, and the 4AT can be administered by a nonclinician without the need for instrument-specific training. Thus, implementation of these instruments can be particularly pragmatic in clinical settings in which the staff involved in delirium screening cannot undergo the substantial training required to administer CAM. Moreover, these abbreviated tests enable nonphysician care team members to assume the role of delirium screener in the hospital. Taken together, the adoption of these abbreviated instruments may facilitate brief screenings of delirium in older patients by caregivers who see them most often—nurses and certified nursing assistants—thereby improving early detection and prevention of delirium-related complications in the hospital.

The feasibility of using abbreviated delirium screening instruments in the hospital setting raises a system implementation question—if these instruments are designed to be administered by those with limited to no training, could nonclinicians, such as hospital volunteers, effectively take on delirium screening roles in the hospital? If volunteers are able to take on this role, the integration of hospital volunteers into the clinical team can greatly expand the capacity for delirium screening in the hospital setting. Further research is warranted to validate the diagnostic accuracy of 3D-CAM and 4AT by nonclinician administrators in order to more broadly adopt this approach to delirium screening.

Practice Points

• Abbreviated delirium screening tools such as 3D-CAM and 4AT may be pragmatic instruments to improve delirium detection in surgical and hospitalized older patients, respectively.
• Further studies are warranted to validate the diagnostic accuracy of 3D-CAM and 4AT by nonclinician administrators in order to more broadly adopt this approach to delirium screening.

Jared Doan, BS, and Fred Ko, MD
Geriatrics and Palliative Medicine, Icahn School of Medicine at Mount Sinai

Study 1 Overview (Oberhaus et al)

Objective: To compare the 3-Minute Diagnostic Confusion Assessment Method (3D-CAM) to the long-form Confusion Assessment Method (CAM) in detecting postoperative delirium.

Design: Prospective concurrent comparison of 3D-CAM and CAM evaluations in a cohort of postoperative geriatric patients.

Setting and participants: Eligible participants were patients aged 60 years or older undergoing major elective surgery at Barnes Jewish Hospital (St. Louis, Missouri) who were enrolled in ongoing clinical trials (PODCAST, ENGAGES, SATISFY-SOS) between 2015 and 2018. Surgeries were at least 2 hours in length and required general anesthesia, planned extubation, and a minimum 2-day hospital stay. Investigators were extensively trained in administering 3D-CAM and CAM instruments. Participants were evaluated 2 hours after the end of anesthesia care on the day of surgery, then daily until follow-up was completed per clinical trial protocol or until the participant was determined by CAM to be nondelirious for 3 consecutive days. For each evaluation, both 3D-CAM and CAM assessors approached the participant together, but the evaluation was conducted such that the 3D-CAM assessor was masked to the additional questions ascertained by the long-form CAM assessment. The 3D-CAM or CAM assessor independently scored their respective assessments blinded to the results of the other assessor.

Main outcome measures: Participants were concurrently evaluated for postoperative delirium by both 3D-CAM and long-form CAM assessments. Comparisons between 3D-CAM and CAM scores were made using Cohen κ with repeated measures, generalized linear mixed-effects model, and Bland-Altman analysis.

Main results: Sixteen raters performed 471 concurrent 3D-CAM and CAM assessments in 299 participants (mean [SD] age, 69 [6.5] years). Of these participants, 152 (50.8%) were men, 263 (88.0%) were White, and 211 (70.6%) underwent noncardiac surgery. Both instruments showed good intraclass correlation (0.98 for 3D-CAM, 0.84 for CAM) with good overall agreement (Cohen κ = 0.71; 95% CI, 0.58-0.83). The mixed-effects model indicated a significant disagreement between the 3D-CAM and CAM assessments (estimated difference in fixed effect, –0.68; 95% CI, –1.32 to –0.05; P = .04). The Bland-Altman analysis showed that the probability of a delirium diagnosis with the 3D-CAM was more than twice that with the CAM (probability ratio, 2.78; 95% CI, 2.44-3.23).

Conclusion: The high degree of agreement between 3D-CAM and long-form CAM assessments suggests that the former may be a pragmatic and easy-to-administer clinical tool to screen for postoperative delirium in vulnerable older surgical patients.

Study 2 Overview (Shenkin et al)

Objective: To assess the accuracy of the 4 ‘A’s Test (4AT) for delirium detection in the medical inpatient setting and to compare the 4AT to the CAM.

Design: Prospective randomized diagnostic test accuracy study.

Setting and participants: This study was conducted in emergency departments and acute medical wards at 3 UK sites (Edinburgh, Bradford, and Sheffield) and enrolled acute medical patients aged 70 years or older without acute life-threatening illnesses and/or coma. Assessors administering the delirium evaluation were nurses or graduate clinical research associates who underwent systematic training in delirium and delirium assessment. Additional training was provided to those administering the CAM but not to those administering the 4AT as the latter is designed to be administered without special training. First, all participants underwent a reference standard delirium assessment using Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition) (DSM-IV) criteria to derive a final definitive diagnosis of delirium via expert consensus (1 psychiatrist and 2 geriatricians). Then, the participants were randomized to either the 4AT or the comparator CAM group using computer-generated pseudo-random numbers, stratified by study site, with block allocation. All assessments were performed by pairs of independent assessors blinded to the results of the other assessment.

Main outcome measures: All participants were evaluated by the reference standard (DSM-IV criteria for delirium) and by either 4AT or CAM instruments for delirium. The accuracy of the 4AT instrument was evaluated by comparing its positive and negative predictive values, sensitivity, and specificity to the reference standard and analyzed via the area under the receiver operating characteristic curve. The diagnostic accuracy of 4AT, compared to the CAM, was evaluated by comparing positive and negative predictive values, sensitivity, and specificity using Fisher’s exact test. The overall performance of 4AT and CAM was summarized using Youden’s Index and the diagnostic odds ratio of sensitivity to specificity.

Results: All 843 individuals enrolled in the study were randomized and 785 were included in the analysis (23 withdrew, 3 lost contact, 32 indeterminate diagnosis, 2 missing outcome). Of the participants analyzed, the mean age was 81.4 [6.4] years, and 12.1% (95/785) had delirium by reference standard assessment, 14.3% (56/392) by 4AT, and 4.7% (18/384) by CAM. The 4AT group had an area under the receiver operating characteristic curve of 0.90 (95% CI, 0.84-0.96), a sensitivity of 76% (95% CI, 61%-87%), and a specificity of 94% (95% CI, 92%-97%). In comparison, the CAM group had a sensitivity of 40% (95% CI, 26%-57%) and a specificity of 100% (95% CI, 98%-100%).

Conclusions: The 4AT is a pragmatic screening test for delirium in a medical space that does not require special training to administer. The use of this instrument may help to improve delirium detection as a part of routine clinical care in hospitalized older adults.

Delirium is an acute confusional state marked by fluctuating mental status, inattention, disorganized thinking, and altered level of consciousness. It is exceedingly common in older patients in both surgical and medical settings and is associated with increased morbidity, mortality, hospital length of stay, institutionalization, and health care costs. Delirium is frequently underdiagnosed in the hospitalized setting, perhaps due to a combination of its waxing and waning nature and a lack of pragmatic and easily implementable screening tools that can be readily administered by clinicians and nonclinicians alike.¹ While the CAM is a well-validated instrument to diagnose delirium, it requires specific training in the rating of each of the cardinal features ascertained through a brief cognitive assessment and takes 5 to 10 minutes to complete. Taken together, given the high patient load for clinicians in the hospital setting, the validation and application of brief delirium screening instruments that can be reliably administered by nonphysicians and nonclinicians may enhance delirium detection in vulnerable patients and consequently improve their outcomes.

In Study 1, Oberhaus et al approach the challenge of underdiagnosing delirium in the postoperative setting by investigating whether the widely accepted long-form CAM and an abbreviated 3-minute version, the 3D-CAM, provide similar delirium detection in older surgical patients. The authors found that both instruments were reliable tests individually (high interrater reliability) and had good overall agreement. However, the 3D-CAM was more likely to yield a positive diagnosis of delirium compared to the long-form CAM, consistent with its purpose as a screening tool with a high sensitivity. It is important to emphasize that the 3D-CAM takes less time to administer, but also requires less extensive training and clinical knowledge than the long-form CAM. Therefore, this instrument meets the prerequisite of a brief screening test that can be rapidly administered by nonclinicians, and if affirmative, followed by a more extensive confirmatory test performed by a clinician. Limitations of this study include a lack of a reference standard structured interview conducted by a physician-rater to better determine the true diagnostic accuracy of both 3D-CAM and CAM assessments, and the use of convenience sampling at a single center, which reduces the generalizability of its findings.

In a similar vein, Shenkin et al in Study 2 attempt to evaluate the utility of the 4AT instrument in diagnosing delirium in older medical inpatients by testing the diagnostic accuracy of the 4AT against a reference standard (ie, DSM-IV–based evaluation by physicians) as well as comparing it to CAM. The 4AT takes less time (~2 minutes) and requires less knowledge and training to administer as compared to the CAM. The study showed that the abbreviated 4AT, compared to CAM, had a higher sensitivity (76% vs 40%) and lower specificity (94% vs 100%) in delirium detection. Thus, akin to the application of 3D-CAM in the postoperative setting, 4AT possesses key characteristics of a brief delirium screening test for older patients in the acute medical setting. In contrast to the Oberhaus et al study, a major strength of this study was the utilization of a reference standard that was validated by expert consensus. This allowed the 4AT and CAM assessments to be compared to a more objective standard, thereby directly testing their diagnostic performance in detecting delirium.

Application for Clinical Practice and System Implementation

The findings from both Study 1 and 2 suggest that using an abbreviated delirium instrument in both surgical and acute medical settings may provide a pragmatic and sensitive method to detect delirium in older patients. The brevity of administration of 3D-CAM (~3 minutes) and 4AT (~2 minutes), combined with their higher sensitivity for detecting delirium compared to CAM, make these instruments potentially effective rapid screening tests for delirium in hospitalized older patients. Importantly, the utilization of such instruments might be a feasible way to mitigate the issue of underdiagnosing delirium in the hospital.

Several additional aspects of these abbreviated delirium instruments increase their suitability for clinical application. Specifically, the 3D-CAM and 4AT require less extensive training and clinical knowledge to both administer and interpret the results than the CAM.² For instance, a multistage, multiday training for CAM is a key factor in maintaining its diagnostic accuracy.^3,4 In contrast, the 3D-CAM requires only a 1- to 2-hour training session, and the 4AT can be administered by a nonclinician without the need for instrument-specific training. Thus, implementation of these instruments can be particularly pragmatic in clinical settings in which the staff involved in delirium screening cannot undergo the substantial training required to administer CAM. Moreover, these abbreviated tests enable nonphysician care team members to assume the role of delirium screener in the hospital. Taken together, the adoption of these abbreviated instruments may facilitate brief screenings of delirium in older patients by caregivers who see them most often—nurses and certified nursing assistants—thereby improving early detection and prevention of delirium-related complications in the hospital.

The feasibility of using abbreviated delirium screening instruments in the hospital setting raises a system implementation question—if these instruments are designed to be administered by those with limited to no training, could nonclinicians, such as hospital volunteers, effectively take on delirium screening roles in the hospital? If volunteers are able to take on this role, the integration of hospital volunteers into the clinical team can greatly expand the capacity for delirium screening in the hospital setting. Further research is warranted to validate the diagnostic accuracy of 3D-CAM and 4AT by nonclinician administrators in order to more broadly adopt this approach to delirium screening.

Practice Points

• Abbreviated delirium screening tools such as 3D-CAM and 4AT may be pragmatic instruments to improve delirium detection in surgical and hospitalized older patients, respectively.
• Further studies are warranted to validate the diagnostic accuracy of 3D-CAM and 4AT by nonclinician administrators in order to more broadly adopt this approach to delirium screening.

Jared Doan, BS, and Fred Ko, MD
Geriatrics and Palliative Medicine, Icahn School of Medicine at Mount Sinai

Corresponding author: Ebrahim Barkoudah, MD, MPH, Department of Medicine, Brigham and Women’s Hospital, Boston, MA; ebarkoudah@bwh.harvard.edu

Process improvement in any industry sector aims to increase the efficiency of resource utilization and delivery methods (cost) and the quality of the product (outcomes), with the goal of ultimately achieving continuous development.¹ In the health care industry, variation in processes and outcomes along with inefficiency in resource use that result in changes in value (the product of outcomes/costs) are the general targets of quality improvement (QI) efforts employing various implementation methodologies.² When the ultimate aim is to serve the patient (customer), best clinical practice includes both maintaining high quality (individual care delivery) and controlling costs (efficient care system delivery), leading to optimal delivery (value-based care). High-quality individual care and efficient care delivery are not competing concepts, but when working to improve both health care outcomes and cost, traditional and nontraditional barriers to system QI often arise.³

The possible scenarios after a QI intervention include backsliding (regression to the mean over time), steady-state (minimal fixed improvement that could sustain), and continuous improvement (tangible enhancement after completing the intervention with legacy effect).⁴ The scalability of results can be considered during the process measurement and the intervention design phases of all QI projects; however, the complex nature of barriers in the health care environment during each level of implementation should be accounted for to prevent failure in the scalability phase.⁵

The barriers to optimal QI outcomes leading to continuous improvement are multifactorial and are related to intrinsic and extrinsic factors.⁶ These factors include 3 fundamental levels: (1) individual level inertia/beliefs, prior personal knowledge, and team-related factors^7,8; (2) intervention-related and process-specific barriers and clinical practice obstacles; and (3) organizational level challenges and macro-level and population-level barriers (Figure). The obstacles faced during the implementation phase will likely include 2 of these levels simultaneously, which could add complexity and hinder or prevent the implementation of a tangible successful QI process and eventually lead to backsliding or minimal fixed improvement rather than continuous improvement. Furthermore, a patient-centered approach to QI would contribute to further complexity in design and execution, given the importance of reaching sustainable, meaningful improvement by adding elements of patient’s preferences, caregiver engagement, and the shared decision-making processes.⁹

Overcoming these multidomain barriers and reaching resilience and sustainability requires thoughtful planning and execution through a multifaceted approach.¹⁰ A meaningful start could include addressing the clinical inertia for the individual and the team by promoting open innovation and allowing outside institutional collaborations and ideas through networks.¹¹ On the individual level, encouraging participation and motivating health care workers in QI to reach a multidisciplinary operation approach will lead to harmony in collaboration. Concurrently, the organization should support the QI capability and scalability by removing competing priorities and establishing effective leadership that ensures resource allocation, communicates clear value-based principles, and engenders a psychological safety environment.

A continuous improvement state is the optimal QI target, a target that can be attained by removing obstacles and paving a clear pathway to implementation. Focusing on the 3 levels of barriers will position the organization for meaningful and successful QI phases to achieve continuous improvement.

Corresponding author: Ebrahim Barkoudah, MD, MPH, Department of Medicine, Brigham and Women’s Hospital, Boston, MA; ebarkoudah@bwh.harvard.edu

Process improvement in any industry sector aims to increase the efficiency of resource utilization and delivery methods (cost) and the quality of the product (outcomes), with the goal of ultimately achieving continuous development.¹ In the health care industry, variation in processes and outcomes along with inefficiency in resource use that result in changes in value (the product of outcomes/costs) are the general targets of quality improvement (QI) efforts employing various implementation methodologies.² When the ultimate aim is to serve the patient (customer), best clinical practice includes both maintaining high quality (individual care delivery) and controlling costs (efficient care system delivery), leading to optimal delivery (value-based care). High-quality individual care and efficient care delivery are not competing concepts, but when working to improve both health care outcomes and cost, traditional and nontraditional barriers to system QI often arise.³

The possible scenarios after a QI intervention include backsliding (regression to the mean over time), steady-state (minimal fixed improvement that could sustain), and continuous improvement (tangible enhancement after completing the intervention with legacy effect).⁴ The scalability of results can be considered during the process measurement and the intervention design phases of all QI projects; however, the complex nature of barriers in the health care environment during each level of implementation should be accounted for to prevent failure in the scalability phase.⁵

The barriers to optimal QI outcomes leading to continuous improvement are multifactorial and are related to intrinsic and extrinsic factors.⁶ These factors include 3 fundamental levels: (1) individual level inertia/beliefs, prior personal knowledge, and team-related factors^7,8; (2) intervention-related and process-specific barriers and clinical practice obstacles; and (3) organizational level challenges and macro-level and population-level barriers (Figure). The obstacles faced during the implementation phase will likely include 2 of these levels simultaneously, which could add complexity and hinder or prevent the implementation of a tangible successful QI process and eventually lead to backsliding or minimal fixed improvement rather than continuous improvement. Furthermore, a patient-centered approach to QI would contribute to further complexity in design and execution, given the importance of reaching sustainable, meaningful improvement by adding elements of patient’s preferences, caregiver engagement, and the shared decision-making processes.⁹

Overcoming these multidomain barriers and reaching resilience and sustainability requires thoughtful planning and execution through a multifaceted approach.¹⁰ A meaningful start could include addressing the clinical inertia for the individual and the team by promoting open innovation and allowing outside institutional collaborations and ideas through networks.¹¹ On the individual level, encouraging participation and motivating health care workers in QI to reach a multidisciplinary operation approach will lead to harmony in collaboration. Concurrently, the organization should support the QI capability and scalability by removing competing priorities and establishing effective leadership that ensures resource allocation, communicates clear value-based principles, and engenders a psychological safety environment.

A continuous improvement state is the optimal QI target, a target that can be attained by removing obstacles and paving a clear pathway to implementation. Focusing on the 3 levels of barriers will position the organization for meaningful and successful QI phases to achieve continuous improvement.

Corresponding author: Ebrahim Barkoudah, MD, MPH, Department of Medicine, Brigham and Women’s Hospital, Boston, MA; ebarkoudah@bwh.harvard.edu

Process improvement in any industry sector aims to increase the efficiency of resource utilization and delivery methods (cost) and the quality of the product (outcomes), with the goal of ultimately achieving continuous development.¹ In the health care industry, variation in processes and outcomes along with inefficiency in resource use that result in changes in value (the product of outcomes/costs) are the general targets of quality improvement (QI) efforts employing various implementation methodologies.² When the ultimate aim is to serve the patient (customer), best clinical practice includes both maintaining high quality (individual care delivery) and controlling costs (efficient care system delivery), leading to optimal delivery (value-based care). High-quality individual care and efficient care delivery are not competing concepts, but when working to improve both health care outcomes and cost, traditional and nontraditional barriers to system QI often arise.³

The possible scenarios after a QI intervention include backsliding (regression to the mean over time), steady-state (minimal fixed improvement that could sustain), and continuous improvement (tangible enhancement after completing the intervention with legacy effect).⁴ The scalability of results can be considered during the process measurement and the intervention design phases of all QI projects; however, the complex nature of barriers in the health care environment during each level of implementation should be accounted for to prevent failure in the scalability phase.⁵

The barriers to optimal QI outcomes leading to continuous improvement are multifactorial and are related to intrinsic and extrinsic factors.⁶ These factors include 3 fundamental levels: (1) individual level inertia/beliefs, prior personal knowledge, and team-related factors^7,8; (2) intervention-related and process-specific barriers and clinical practice obstacles; and (3) organizational level challenges and macro-level and population-level barriers (Figure). The obstacles faced during the implementation phase will likely include 2 of these levels simultaneously, which could add complexity and hinder or prevent the implementation of a tangible successful QI process and eventually lead to backsliding or minimal fixed improvement rather than continuous improvement. Furthermore, a patient-centered approach to QI would contribute to further complexity in design and execution, given the importance of reaching sustainable, meaningful improvement by adding elements of patient’s preferences, caregiver engagement, and the shared decision-making processes.⁹

Overcoming these multidomain barriers and reaching resilience and sustainability requires thoughtful planning and execution through a multifaceted approach.¹⁰ A meaningful start could include addressing the clinical inertia for the individual and the team by promoting open innovation and allowing outside institutional collaborations and ideas through networks.¹¹ On the individual level, encouraging participation and motivating health care workers in QI to reach a multidisciplinary operation approach will lead to harmony in collaboration. Concurrently, the organization should support the QI capability and scalability by removing competing priorities and establishing effective leadership that ensures resource allocation, communicates clear value-based principles, and engenders a psychological safety environment.

A continuous improvement state is the optimal QI target, a target that can be attained by removing obstacles and paving a clear pathway to implementation. Focusing on the 3 levels of barriers will position the organization for meaningful and successful QI phases to achieve continuous improvement.

Purpose

Ensuring that patients/families are engaged as partners in their health care is an effective way to measure the quality of patient care. Self-reported patient data, such as symptom burden, provides an accurate and effective way to measure patient-reported outcomes. Our team reviewed 20 patient charts, randomly, to assess for documentation of at least 3 or more domains of toxicities of immune checkpoint inhibitors. The baseline comprehensive documentation rate was 50%. Our goal is to improve the documentation rate to 80% for our first process improvement cycle.

Aim Statement

Increase documentation of 3 or more toxicities immune checkpoint inhibitors to a goal rate of 80%.

Methods

A free online patient monitoring checklist tool was printed and provided to patients receiving immune checkpoint inhibitors (on their infusion day) during the check-in process. The patients were instructed to complete the tool prior to the provider clinic visit, while in the waiting area. The completed tool was given to the provider on the day of their visit. Prior to the start of this Plan-Do-Study-Act (PDSA) cycle, all providers were “reminded”/ instructed to ensure documentation of 3 or more toxicities immune checkpoint inhibitors. The cycle lasted for 3 weeks. At the end of the 3 weeks, our team reviewed the charts of those patients.

Results

Documentation rate of 3 or more toxicities increased from 50% to 90%.

Conclusions

When completed patient monitoring tools were provided to the providers during the clinic visit, the providers increased their documentation rate of the toxicities. There is literature supporting improving patient satisfaction using self-reported symptoms monitoring tools. Also, given the burden of documentation and shorter visit times, providers found this to be an easy way to address patient symptoms. While electronic patient-reported outcome (e-PRO) tools are ideal for ongoing symptom monitoring, this is a simple way to address the same in low-resourced communities. For our next cycle, we plan on using patient feedback to improve the documentation form incorporating larger fonts for patients with low vision.

Purpose

Ensuring that patients/families are engaged as partners in their health care is an effective way to measure the quality of patient care. Self-reported patient data, such as symptom burden, provides an accurate and effective way to measure patient-reported outcomes. Our team reviewed 20 patient charts, randomly, to assess for documentation of at least 3 or more domains of toxicities of immune checkpoint inhibitors. The baseline comprehensive documentation rate was 50%. Our goal is to improve the documentation rate to 80% for our first process improvement cycle.

Aim Statement

Increase documentation of 3 or more toxicities immune checkpoint inhibitors to a goal rate of 80%.

Methods

A free online patient monitoring checklist tool was printed and provided to patients receiving immune checkpoint inhibitors (on their infusion day) during the check-in process. The patients were instructed to complete the tool prior to the provider clinic visit, while in the waiting area. The completed tool was given to the provider on the day of their visit. Prior to the start of this Plan-Do-Study-Act (PDSA) cycle, all providers were “reminded”/ instructed to ensure documentation of 3 or more toxicities immune checkpoint inhibitors. The cycle lasted for 3 weeks. At the end of the 3 weeks, our team reviewed the charts of those patients.

Results

Documentation rate of 3 or more toxicities increased from 50% to 90%.

Conclusions

When completed patient monitoring tools were provided to the providers during the clinic visit, the providers increased their documentation rate of the toxicities. There is literature supporting improving patient satisfaction using self-reported symptoms monitoring tools. Also, given the burden of documentation and shorter visit times, providers found this to be an easy way to address patient symptoms. While electronic patient-reported outcome (e-PRO) tools are ideal for ongoing symptom monitoring, this is a simple way to address the same in low-resourced communities. For our next cycle, we plan on using patient feedback to improve the documentation form incorporating larger fonts for patients with low vision.

Purpose

Ensuring that patients/families are engaged as partners in their health care is an effective way to measure the quality of patient care. Self-reported patient data, such as symptom burden, provides an accurate and effective way to measure patient-reported outcomes. Our team reviewed 20 patient charts, randomly, to assess for documentation of at least 3 or more domains of toxicities of immune checkpoint inhibitors. The baseline comprehensive documentation rate was 50%. Our goal is to improve the documentation rate to 80% for our first process improvement cycle.

Aim Statement

Increase documentation of 3 or more toxicities immune checkpoint inhibitors to a goal rate of 80%.

Methods

A free online patient monitoring checklist tool was printed and provided to patients receiving immune checkpoint inhibitors (on their infusion day) during the check-in process. The patients were instructed to complete the tool prior to the provider clinic visit, while in the waiting area. The completed tool was given to the provider on the day of their visit. Prior to the start of this Plan-Do-Study-Act (PDSA) cycle, all providers were “reminded”/ instructed to ensure documentation of 3 or more toxicities immune checkpoint inhibitors. The cycle lasted for 3 weeks. At the end of the 3 weeks, our team reviewed the charts of those patients.

Results

Documentation rate of 3 or more toxicities increased from 50% to 90%.

Conclusions

When completed patient monitoring tools were provided to the providers during the clinic visit, the providers increased their documentation rate of the toxicities. There is literature supporting improving patient satisfaction using self-reported symptoms monitoring tools. Also, given the burden of documentation and shorter visit times, providers found this to be an easy way to address patient symptoms. While electronic patient-reported outcome (e-PRO) tools are ideal for ongoing symptom monitoring, this is a simple way to address the same in low-resourced communities. For our next cycle, we plan on using patient feedback to improve the documentation form incorporating larger fonts for patients with low vision.

Veterans’ access to medical care was expanded outside of US Department of Veterans Affairs (VA) facilities with the inception of the 2014 Veterans Access, Choice, and Accountability Act (Choice Act).¹ This legislation aimed to remove barriers some veterans were experiencing, specifically access to health care. In subsequent years, approximately 17% of veterans receiving care from the VA did so under the Choice Act.² The Choice Act positively impacted medical care access for veterans but presented new challenges for VA pharmacies processing community care (CC) prescriptions, including limited access to outside health records, lack of interface between CC prescribers and the VA order entry system, and limited awareness of the VA national formulary.^3,4 These factors made it difficult for VA pharmacies to assess prescriptions for clinical appropriateness, evaluate patient safety parameters, and manage expenditures.

In 2019, the Maintaining Internal Systems and Strengthening Integrated Outside Networks (MISSION) Act, which expanded CC support and better defined which veterans are able to receive care outside the VA, updated the Choice Act.^4,5 However, VA pharmacies faced challenges in managing pharmacy drug costs and ensuring clinical appropriateness of prescription drug therapy. As a result, VA pharmacy departments have adjusted how they allocate workload, time, and funds.⁵

Pharmacists improve clinical outcomes and reduce health care costs by decreasing medication errors, unnecessary prescribing, and adverse drug events.^6-12 Pharmacist-driven formulary management through evaluation of prior authorization drug requests (PADRs) has shown economic value.^13,14 VA pharmacy review of community care PADRs is important because outside health care professionals (HCPs) might not be familiar with the VA formulary. This could lead to high volume of PADRs that do not meet criteria and could result in increased potential for medication misuse, adverse drug events, medication errors, and cost to the health system. It is imperative that CC orders are evaluated as critically as traditional orders.

The value of a centralized CC pharmacy team has not been assessed in the literature. The primary objective of this study was to assess the direct cost savings achieved through a centralized CC PADR process. Secondary objectives were to characterize the CC PADRs submitted to the site, including approval rate, reason for nonapproval, which medications were requested and by whom, and to compare CC prescriptions with other high-complexity (1a) VA facilities.

Community Care Pharmacy

VA health systems are stratified according to complexity, which reflects size, patient population, and services offered. This study was conducted at the Durham Veterans Affairs Health Care System (DVAHCS), North Carolina, a high-complexity, 251-bed, tertiary care referral, teaching, and research system. DVAHCS provides general and specialty medical, surgical, inpatient psychiatric, and ambulatory services, and serves as a major referral center.

DVAHCS created a centralized pharmacy team for processing CC prescriptions and managing customer service. This team’s goal is to increase CC prescription processing efficiency and transparency, ensure accountability of the health care team, and promote veteran-centric customer service. The pharmacy team includes a pharmacist program manager and a dedicated CC pharmacist with administrative support from a health benefits assistant and 4 pharmacy technicians. The CC pharmacy team assesses every new prescription to ensure the veteran is authorized to receive care in the community. Once eligibility is verified, a pharmacy technician or pharmacist evaluates the prescription to ensure it contains all required information, then contacts the prescriber for any missing data. If clinically appropriate, the pharmacist processes the prescription.

In 2020, the CC pharmacy team implemented a new process for reviewing and documenting CC prescriptions that require a PADR. The closed national VA formulary is set up so that all nonformulary medications and some formulary medications, including those that are restricted because of safety and/or cost, require a PADR.¹⁵ After a CC pharmacy technician confirms a veteran’s eligibility, the technician assesses whether the requested medication requires submitting a PADR to the VA internal electronic health record. The PADR is then adjudicated by a formulary management pharmacist, CC program manager, or CC pharmacist who reviews health records to determine whether the CC prescription meets VA medication use policy requirements.

If additional information is needed or an alternate medication is suggested, the pharmacist comments back on the PADR and a CC pharmacy technician contacts the prescriber. The PADR is canceled administratively then resubmitted once all information is obtained. While waiting for a response from the prescriber, the CC pharmacy technician contacts that veteran to give an update on the prescription status, as appropriate. Once there is sufficient information to adjudicate the PADR, the outcome is documented, and if approved, the order is processed.

Methods

The DVAHCS Institutional Review Board approved this retrospective review of CC PADRs submitted from June 1, 2020, through November 30, 2020. CC PADRs were excluded if they were duplicates or were reactivated administratively but had an initial submission date before the study period. Local data were collected for nonapproved CC PADRs including drug requested, dosage and directions, medication specialty, alternative drug recommended, drug acquisition cost, PADR submission date, PADR completion date, PADR nonapproval rationale, and documented time spent per PADR. Additional data was obtained for CC prescriptions at all 42 high-complexity VA facilities from the VA national CC prescription database for the study time interval and included total PADRs, PADR approval status, total CC prescription cost, and total CC fills.

Direct cost savings were calculated by assessing the cost of requested therapy that was not approved minus the cost of recommended therapy and cost to review all PADRs, as described by Britt and colleagues.¹³ The cost of the requested and recommended therapy was calculated based on VA drug acquisition cost at time of data collection and multiplied by the expected duration of therapy up to 1 year. For each CC prescription, duration of therapy was based on the duration limit in the prescription or annualized if no duration limit was documented. Cost of PADR review was calculated based on the total time pharmacists and pharmacy technicians documented for each step of the review process for a representative sample of 100 nonapproved PADRs and then multiplied by the salary plus benefits of an entry-level pharmacist and pharmacy technician.¹⁶ The eAppendix  describes specific equations used for determining direct cost savings. Descriptive statistics were used to evaluate study results.

Results

During the 6-month study period, 611 CC PADRs were submitted to the pharmacy and 526 met inclusion criteria (Figure 1). Of those, 243 (46.2%) were approved and 283 (53.8%) were not approved. The cost of requested therapies for nonapproved CC PADRs totaled $584,565.48 and the cost of all recommended therapies was $57,473.59. The mean time per CC PADR was 24 minutes; 16 minutes for pharmacists and 8 minutes for pharmacy technicians. Given an hourly wage (plus benefits) of $67.25 for a pharmacist and $25.53 for a pharmacy technician, the total cost of review per CC PADR was $21.33. After subtracting the costs of all recommended therapies and review of all included CC PADRs, the process generated $515,872.31 in direct cost savings. After factoring in administrative lag time, such as HCP communication, an average of 8 calendar days was needed to complete a nonapproved PADR.

The most common rationale for PADR nonapproval was that the formulary alternative was not exhausted. Ondansetron orally disintegrating tablets was the most commonly nonapproved medication and azelastine was the most commonly approved medication. Dulaglutide was the most expensive nonapproved and tafamidis was the most expensive approved PADR (Table 1). Gastroenterology, endocrinology, and neurology were the top specialties for nonapproved PADRs while neurology, pulmonology, and endocrinology were the top specialties for approved PADRs (Table 2).

Several high-complexity VA facilities had no reported data; we used the median for the analysis to account for these outliers (Figure 2). The median (IQR) adjudicated CC PADRs for all facilities was 97 (20-175), median (IQR) CC PADR approval rate was 80.9% (63.7%-96.8%), median (IQR) total CC prescriptions was 8440 (2464-14,466), and median (IQR) cost per fill was $136.05 ($76.27-$221.28).

Discussion

This study demonstrated direct cost savings of $515,872.31 over 6 months with theadjudication of CC PADRs by a centralized CC pharmacy team. This could result in > $1,000,000 of cost savings per fiscal year.

The CC PADRs observed at DVAHCS had a 46.2% approval rate; almost one-half the approval rate of 84.1% of all PADRs submitted to the study site by VA HCPs captured by Britt and colleagues.¹³ Results from this study showed that coordination of care for nonapproved CC PADRs between the VA pharmacy and non-VA prescriber took an average of 8 calendar days. The noted CC PADR approval rate and administrative burden might be because of lack of familiarity of non-VA providers regarding the VA national formulary. The National VA Pharmacy Benefits Management determines the formulary using cost-effectiveness criteria that considers the medical literature and VA-specific contract pricing and prepares extensive guidance for restricted medications via relevant criteria for use.¹⁵ HCPs outside the VA might not know this information is available online. Because gastroenterology, endocrinology, and neurology specialty medications were among the most frequently nonapproved PADRs, VA formulary education could begin with CC HCPs in these practice areas.

This study showed that the CC PADR process was not solely driven by cost, but also included patient safety. Nonapproval rationale for some requests included submission without an indication, submission by a prescriber that did not have the authority to prescribe a type of medication, or contraindication based on patient-specific factors.

Limitations

CC PADRs were assessed during the COVID-19 pandemic, which might have resulted in lower-than-normal CC prescription and PADR volumes, therefore underestimating the potential for direct cost savings. Entry-level salary was used to demonstrate cost savings potential from the perspective of a newly hired CC team; however, the cost savings might have been less if the actual salaries of site personnel were higher. National contract pricing data were gathered at the time of data collection and might have been different than at the time of PADR submission. Chronic medication prescriptions were annualized, which could overestimate cost savings if the medication was discontinued or changed to an alternative therapy within that time period.

The study’s exclusion criteria could only be applied locally and did not include data received from the VA CC prescription database. This can be seen by the discrepancy in CC PADR approval rates from the local and national data (46.2% vs 44.2%, respectively) and CC PADR volume. High-complexity VA facility data were captured without assessing the CC prescription process at each site. As a result, definitive conclusions cannot be made regarding the impact of a centralized CC pharmacy team compared with other facilities.

Conclusions

Adjudication of CC PADRs by a centralized CC pharmacy team over a 6-month period provided > $500,000 in direct cost savings to a VA health care system. Considering the CC PADR approval rate seen in this study, the VA could allocate resources to educate CC providers about the VA formulary to increase the PADR approval rate and reduce administrative burden for VA pharmacies and prescribers. Future research should evaluate CC prescription handling practices at other VA facilities to compare the effectiveness among varying approaches and develop recommendations for a nationally standardized process.

Acknowledgments

Concept and design (AJJ, JNB, RBB, LAM, MD, MGH); acquisition of data (AJJ, MGH); analysis and interpretation of data (AJJ, JNB, RBB, LAM, MD, MGH); drafting of the manuscript (AJJ); critical revision of the manuscript for important intellectual content (AJJ, JNB, RBB, LAM, MD, MGH); statistical analysis (AJJ); administrative, technical, or logistic support (LAM, MGH); and supervision (MGH).

Veterans’ access to medical care was expanded outside of US Department of Veterans Affairs (VA) facilities with the inception of the 2014 Veterans Access, Choice, and Accountability Act (Choice Act).¹ This legislation aimed to remove barriers some veterans were experiencing, specifically access to health care. In subsequent years, approximately 17% of veterans receiving care from the VA did so under the Choice Act.² The Choice Act positively impacted medical care access for veterans but presented new challenges for VA pharmacies processing community care (CC) prescriptions, including limited access to outside health records, lack of interface between CC prescribers and the VA order entry system, and limited awareness of the VA national formulary.^3,4 These factors made it difficult for VA pharmacies to assess prescriptions for clinical appropriateness, evaluate patient safety parameters, and manage expenditures.

In 2019, the Maintaining Internal Systems and Strengthening Integrated Outside Networks (MISSION) Act, which expanded CC support and better defined which veterans are able to receive care outside the VA, updated the Choice Act.^4,5 However, VA pharmacies faced challenges in managing pharmacy drug costs and ensuring clinical appropriateness of prescription drug therapy. As a result, VA pharmacy departments have adjusted how they allocate workload, time, and funds.⁵

Pharmacists improve clinical outcomes and reduce health care costs by decreasing medication errors, unnecessary prescribing, and adverse drug events.^6-12 Pharmacist-driven formulary management through evaluation of prior authorization drug requests (PADRs) has shown economic value.^13,14 VA pharmacy review of community care PADRs is important because outside health care professionals (HCPs) might not be familiar with the VA formulary. This could lead to high volume of PADRs that do not meet criteria and could result in increased potential for medication misuse, adverse drug events, medication errors, and cost to the health system. It is imperative that CC orders are evaluated as critically as traditional orders.

The value of a centralized CC pharmacy team has not been assessed in the literature. The primary objective of this study was to assess the direct cost savings achieved through a centralized CC PADR process. Secondary objectives were to characterize the CC PADRs submitted to the site, including approval rate, reason for nonapproval, which medications were requested and by whom, and to compare CC prescriptions with other high-complexity (1a) VA facilities.

Community Care Pharmacy

VA health systems are stratified according to complexity, which reflects size, patient population, and services offered. This study was conducted at the Durham Veterans Affairs Health Care System (DVAHCS), North Carolina, a high-complexity, 251-bed, tertiary care referral, teaching, and research system. DVAHCS provides general and specialty medical, surgical, inpatient psychiatric, and ambulatory services, and serves as a major referral center.

DVAHCS created a centralized pharmacy team for processing CC prescriptions and managing customer service. This team’s goal is to increase CC prescription processing efficiency and transparency, ensure accountability of the health care team, and promote veteran-centric customer service. The pharmacy team includes a pharmacist program manager and a dedicated CC pharmacist with administrative support from a health benefits assistant and 4 pharmacy technicians. The CC pharmacy team assesses every new prescription to ensure the veteran is authorized to receive care in the community. Once eligibility is verified, a pharmacy technician or pharmacist evaluates the prescription to ensure it contains all required information, then contacts the prescriber for any missing data. If clinically appropriate, the pharmacist processes the prescription.

In 2020, the CC pharmacy team implemented a new process for reviewing and documenting CC prescriptions that require a PADR. The closed national VA formulary is set up so that all nonformulary medications and some formulary medications, including those that are restricted because of safety and/or cost, require a PADR.¹⁵ After a CC pharmacy technician confirms a veteran’s eligibility, the technician assesses whether the requested medication requires submitting a PADR to the VA internal electronic health record. The PADR is then adjudicated by a formulary management pharmacist, CC program manager, or CC pharmacist who reviews health records to determine whether the CC prescription meets VA medication use policy requirements.

If additional information is needed or an alternate medication is suggested, the pharmacist comments back on the PADR and a CC pharmacy technician contacts the prescriber. The PADR is canceled administratively then resubmitted once all information is obtained. While waiting for a response from the prescriber, the CC pharmacy technician contacts that veteran to give an update on the prescription status, as appropriate. Once there is sufficient information to adjudicate the PADR, the outcome is documented, and if approved, the order is processed.

Methods

The DVAHCS Institutional Review Board approved this retrospective review of CC PADRs submitted from June 1, 2020, through November 30, 2020. CC PADRs were excluded if they were duplicates or were reactivated administratively but had an initial submission date before the study period. Local data were collected for nonapproved CC PADRs including drug requested, dosage and directions, medication specialty, alternative drug recommended, drug acquisition cost, PADR submission date, PADR completion date, PADR nonapproval rationale, and documented time spent per PADR. Additional data was obtained for CC prescriptions at all 42 high-complexity VA facilities from the VA national CC prescription database for the study time interval and included total PADRs, PADR approval status, total CC prescription cost, and total CC fills.

Direct cost savings were calculated by assessing the cost of requested therapy that was not approved minus the cost of recommended therapy and cost to review all PADRs, as described by Britt and colleagues.¹³ The cost of the requested and recommended therapy was calculated based on VA drug acquisition cost at time of data collection and multiplied by the expected duration of therapy up to 1 year. For each CC prescription, duration of therapy was based on the duration limit in the prescription or annualized if no duration limit was documented. Cost of PADR review was calculated based on the total time pharmacists and pharmacy technicians documented for each step of the review process for a representative sample of 100 nonapproved PADRs and then multiplied by the salary plus benefits of an entry-level pharmacist and pharmacy technician.¹⁶ The eAppendix  describes specific equations used for determining direct cost savings. Descriptive statistics were used to evaluate study results.

Results

During the 6-month study period, 611 CC PADRs were submitted to the pharmacy and 526 met inclusion criteria (Figure 1). Of those, 243 (46.2%) were approved and 283 (53.8%) were not approved. The cost of requested therapies for nonapproved CC PADRs totaled $584,565.48 and the cost of all recommended therapies was $57,473.59. The mean time per CC PADR was 24 minutes; 16 minutes for pharmacists and 8 minutes for pharmacy technicians. Given an hourly wage (plus benefits) of $67.25 for a pharmacist and $25.53 for a pharmacy technician, the total cost of review per CC PADR was $21.33. After subtracting the costs of all recommended therapies and review of all included CC PADRs, the process generated $515,872.31 in direct cost savings. After factoring in administrative lag time, such as HCP communication, an average of 8 calendar days was needed to complete a nonapproved PADR.

The most common rationale for PADR nonapproval was that the formulary alternative was not exhausted. Ondansetron orally disintegrating tablets was the most commonly nonapproved medication and azelastine was the most commonly approved medication. Dulaglutide was the most expensive nonapproved and tafamidis was the most expensive approved PADR (Table 1). Gastroenterology, endocrinology, and neurology were the top specialties for nonapproved PADRs while neurology, pulmonology, and endocrinology were the top specialties for approved PADRs (Table 2).

Several high-complexity VA facilities had no reported data; we used the median for the analysis to account for these outliers (Figure 2). The median (IQR) adjudicated CC PADRs for all facilities was 97 (20-175), median (IQR) CC PADR approval rate was 80.9% (63.7%-96.8%), median (IQR) total CC prescriptions was 8440 (2464-14,466), and median (IQR) cost per fill was $136.05 ($76.27-$221.28).

Discussion

This study demonstrated direct cost savings of $515,872.31 over 6 months with theadjudication of CC PADRs by a centralized CC pharmacy team. This could result in > $1,000,000 of cost savings per fiscal year.

The CC PADRs observed at DVAHCS had a 46.2% approval rate; almost one-half the approval rate of 84.1% of all PADRs submitted to the study site by VA HCPs captured by Britt and colleagues.¹³ Results from this study showed that coordination of care for nonapproved CC PADRs between the VA pharmacy and non-VA prescriber took an average of 8 calendar days. The noted CC PADR approval rate and administrative burden might be because of lack of familiarity of non-VA providers regarding the VA national formulary. The National VA Pharmacy Benefits Management determines the formulary using cost-effectiveness criteria that considers the medical literature and VA-specific contract pricing and prepares extensive guidance for restricted medications via relevant criteria for use.¹⁵ HCPs outside the VA might not know this information is available online. Because gastroenterology, endocrinology, and neurology specialty medications were among the most frequently nonapproved PADRs, VA formulary education could begin with CC HCPs in these practice areas.

This study showed that the CC PADR process was not solely driven by cost, but also included patient safety. Nonapproval rationale for some requests included submission without an indication, submission by a prescriber that did not have the authority to prescribe a type of medication, or contraindication based on patient-specific factors.

Limitations

CC PADRs were assessed during the COVID-19 pandemic, which might have resulted in lower-than-normal CC prescription and PADR volumes, therefore underestimating the potential for direct cost savings. Entry-level salary was used to demonstrate cost savings potential from the perspective of a newly hired CC team; however, the cost savings might have been less if the actual salaries of site personnel were higher. National contract pricing data were gathered at the time of data collection and might have been different than at the time of PADR submission. Chronic medication prescriptions were annualized, which could overestimate cost savings if the medication was discontinued or changed to an alternative therapy within that time period.

The study’s exclusion criteria could only be applied locally and did not include data received from the VA CC prescription database. This can be seen by the discrepancy in CC PADR approval rates from the local and national data (46.2% vs 44.2%, respectively) and CC PADR volume. High-complexity VA facility data were captured without assessing the CC prescription process at each site. As a result, definitive conclusions cannot be made regarding the impact of a centralized CC pharmacy team compared with other facilities.

Conclusions

Adjudication of CC PADRs by a centralized CC pharmacy team over a 6-month period provided > $500,000 in direct cost savings to a VA health care system. Considering the CC PADR approval rate seen in this study, the VA could allocate resources to educate CC providers about the VA formulary to increase the PADR approval rate and reduce administrative burden for VA pharmacies and prescribers. Future research should evaluate CC prescription handling practices at other VA facilities to compare the effectiveness among varying approaches and develop recommendations for a nationally standardized process.

Acknowledgments

Concept and design (AJJ, JNB, RBB, LAM, MD, MGH); acquisition of data (AJJ, MGH); analysis and interpretation of data (AJJ, JNB, RBB, LAM, MD, MGH); drafting of the manuscript (AJJ); critical revision of the manuscript for important intellectual content (AJJ, JNB, RBB, LAM, MD, MGH); statistical analysis (AJJ); administrative, technical, or logistic support (LAM, MGH); and supervision (MGH).

Veterans’ access to medical care was expanded outside of US Department of Veterans Affairs (VA) facilities with the inception of the 2014 Veterans Access, Choice, and Accountability Act (Choice Act).¹ This legislation aimed to remove barriers some veterans were experiencing, specifically access to health care. In subsequent years, approximately 17% of veterans receiving care from the VA did so under the Choice Act.² The Choice Act positively impacted medical care access for veterans but presented new challenges for VA pharmacies processing community care (CC) prescriptions, including limited access to outside health records, lack of interface between CC prescribers and the VA order entry system, and limited awareness of the VA national formulary.^3,4 These factors made it difficult for VA pharmacies to assess prescriptions for clinical appropriateness, evaluate patient safety parameters, and manage expenditures.

In 2019, the Maintaining Internal Systems and Strengthening Integrated Outside Networks (MISSION) Act, which expanded CC support and better defined which veterans are able to receive care outside the VA, updated the Choice Act.^4,5 However, VA pharmacies faced challenges in managing pharmacy drug costs and ensuring clinical appropriateness of prescription drug therapy. As a result, VA pharmacy departments have adjusted how they allocate workload, time, and funds.⁵

Pharmacists improve clinical outcomes and reduce health care costs by decreasing medication errors, unnecessary prescribing, and adverse drug events.^6-12 Pharmacist-driven formulary management through evaluation of prior authorization drug requests (PADRs) has shown economic value.^13,14 VA pharmacy review of community care PADRs is important because outside health care professionals (HCPs) might not be familiar with the VA formulary. This could lead to high volume of PADRs that do not meet criteria and could result in increased potential for medication misuse, adverse drug events, medication errors, and cost to the health system. It is imperative that CC orders are evaluated as critically as traditional orders.

The value of a centralized CC pharmacy team has not been assessed in the literature. The primary objective of this study was to assess the direct cost savings achieved through a centralized CC PADR process. Secondary objectives were to characterize the CC PADRs submitted to the site, including approval rate, reason for nonapproval, which medications were requested and by whom, and to compare CC prescriptions with other high-complexity (1a) VA facilities.

Community Care Pharmacy

VA health systems are stratified according to complexity, which reflects size, patient population, and services offered. This study was conducted at the Durham Veterans Affairs Health Care System (DVAHCS), North Carolina, a high-complexity, 251-bed, tertiary care referral, teaching, and research system. DVAHCS provides general and specialty medical, surgical, inpatient psychiatric, and ambulatory services, and serves as a major referral center.

DVAHCS created a centralized pharmacy team for processing CC prescriptions and managing customer service. This team’s goal is to increase CC prescription processing efficiency and transparency, ensure accountability of the health care team, and promote veteran-centric customer service. The pharmacy team includes a pharmacist program manager and a dedicated CC pharmacist with administrative support from a health benefits assistant and 4 pharmacy technicians. The CC pharmacy team assesses every new prescription to ensure the veteran is authorized to receive care in the community. Once eligibility is verified, a pharmacy technician or pharmacist evaluates the prescription to ensure it contains all required information, then contacts the prescriber for any missing data. If clinically appropriate, the pharmacist processes the prescription.

In 2020, the CC pharmacy team implemented a new process for reviewing and documenting CC prescriptions that require a PADR. The closed national VA formulary is set up so that all nonformulary medications and some formulary medications, including those that are restricted because of safety and/or cost, require a PADR.¹⁵ After a CC pharmacy technician confirms a veteran’s eligibility, the technician assesses whether the requested medication requires submitting a PADR to the VA internal electronic health record. The PADR is then adjudicated by a formulary management pharmacist, CC program manager, or CC pharmacist who reviews health records to determine whether the CC prescription meets VA medication use policy requirements.

If additional information is needed or an alternate medication is suggested, the pharmacist comments back on the PADR and a CC pharmacy technician contacts the prescriber. The PADR is canceled administratively then resubmitted once all information is obtained. While waiting for a response from the prescriber, the CC pharmacy technician contacts that veteran to give an update on the prescription status, as appropriate. Once there is sufficient information to adjudicate the PADR, the outcome is documented, and if approved, the order is processed.

Methods

The DVAHCS Institutional Review Board approved this retrospective review of CC PADRs submitted from June 1, 2020, through November 30, 2020. CC PADRs were excluded if they were duplicates or were reactivated administratively but had an initial submission date before the study period. Local data were collected for nonapproved CC PADRs including drug requested, dosage and directions, medication specialty, alternative drug recommended, drug acquisition cost, PADR submission date, PADR completion date, PADR nonapproval rationale, and documented time spent per PADR. Additional data was obtained for CC prescriptions at all 42 high-complexity VA facilities from the VA national CC prescription database for the study time interval and included total PADRs, PADR approval status, total CC prescription cost, and total CC fills.

Direct cost savings were calculated by assessing the cost of requested therapy that was not approved minus the cost of recommended therapy and cost to review all PADRs, as described by Britt and colleagues.¹³ The cost of the requested and recommended therapy was calculated based on VA drug acquisition cost at time of data collection and multiplied by the expected duration of therapy up to 1 year. For each CC prescription, duration of therapy was based on the duration limit in the prescription or annualized if no duration limit was documented. Cost of PADR review was calculated based on the total time pharmacists and pharmacy technicians documented for each step of the review process for a representative sample of 100 nonapproved PADRs and then multiplied by the salary plus benefits of an entry-level pharmacist and pharmacy technician.¹⁶ The eAppendix  describes specific equations used for determining direct cost savings. Descriptive statistics were used to evaluate study results.

Results

During the 6-month study period, 611 CC PADRs were submitted to the pharmacy and 526 met inclusion criteria (Figure 1). Of those, 243 (46.2%) were approved and 283 (53.8%) were not approved. The cost of requested therapies for nonapproved CC PADRs totaled $584,565.48 and the cost of all recommended therapies was $57,473.59. The mean time per CC PADR was 24 minutes; 16 minutes for pharmacists and 8 minutes for pharmacy technicians. Given an hourly wage (plus benefits) of $67.25 for a pharmacist and $25.53 for a pharmacy technician, the total cost of review per CC PADR was $21.33. After subtracting the costs of all recommended therapies and review of all included CC PADRs, the process generated $515,872.31 in direct cost savings. After factoring in administrative lag time, such as HCP communication, an average of 8 calendar days was needed to complete a nonapproved PADR.

The most common rationale for PADR nonapproval was that the formulary alternative was not exhausted. Ondansetron orally disintegrating tablets was the most commonly nonapproved medication and azelastine was the most commonly approved medication. Dulaglutide was the most expensive nonapproved and tafamidis was the most expensive approved PADR (Table 1). Gastroenterology, endocrinology, and neurology were the top specialties for nonapproved PADRs while neurology, pulmonology, and endocrinology were the top specialties for approved PADRs (Table 2).

Several high-complexity VA facilities had no reported data; we used the median for the analysis to account for these outliers (Figure 2). The median (IQR) adjudicated CC PADRs for all facilities was 97 (20-175), median (IQR) CC PADR approval rate was 80.9% (63.7%-96.8%), median (IQR) total CC prescriptions was 8440 (2464-14,466), and median (IQR) cost per fill was $136.05 ($76.27-$221.28).

Discussion

This study demonstrated direct cost savings of $515,872.31 over 6 months with theadjudication of CC PADRs by a centralized CC pharmacy team. This could result in > $1,000,000 of cost savings per fiscal year.

The CC PADRs observed at DVAHCS had a 46.2% approval rate; almost one-half the approval rate of 84.1% of all PADRs submitted to the study site by VA HCPs captured by Britt and colleagues.¹³ Results from this study showed that coordination of care for nonapproved CC PADRs between the VA pharmacy and non-VA prescriber took an average of 8 calendar days. The noted CC PADR approval rate and administrative burden might be because of lack of familiarity of non-VA providers regarding the VA national formulary. The National VA Pharmacy Benefits Management determines the formulary using cost-effectiveness criteria that considers the medical literature and VA-specific contract pricing and prepares extensive guidance for restricted medications via relevant criteria for use.¹⁵ HCPs outside the VA might not know this information is available online. Because gastroenterology, endocrinology, and neurology specialty medications were among the most frequently nonapproved PADRs, VA formulary education could begin with CC HCPs in these practice areas.

This study showed that the CC PADR process was not solely driven by cost, but also included patient safety. Nonapproval rationale for some requests included submission without an indication, submission by a prescriber that did not have the authority to prescribe a type of medication, or contraindication based on patient-specific factors.

Limitations

CC PADRs were assessed during the COVID-19 pandemic, which might have resulted in lower-than-normal CC prescription and PADR volumes, therefore underestimating the potential for direct cost savings. Entry-level salary was used to demonstrate cost savings potential from the perspective of a newly hired CC team; however, the cost savings might have been less if the actual salaries of site personnel were higher. National contract pricing data were gathered at the time of data collection and might have been different than at the time of PADR submission. Chronic medication prescriptions were annualized, which could overestimate cost savings if the medication was discontinued or changed to an alternative therapy within that time period.

The study’s exclusion criteria could only be applied locally and did not include data received from the VA CC prescription database. This can be seen by the discrepancy in CC PADR approval rates from the local and national data (46.2% vs 44.2%, respectively) and CC PADR volume. High-complexity VA facility data were captured without assessing the CC prescription process at each site. As a result, definitive conclusions cannot be made regarding the impact of a centralized CC pharmacy team compared with other facilities.

Conclusions

Adjudication of CC PADRs by a centralized CC pharmacy team over a 6-month period provided > $500,000 in direct cost savings to a VA health care system. Considering the CC PADR approval rate seen in this study, the VA could allocate resources to educate CC providers about the VA formulary to increase the PADR approval rate and reduce administrative burden for VA pharmacies and prescribers. Future research should evaluate CC prescription handling practices at other VA facilities to compare the effectiveness among varying approaches and develop recommendations for a nationally standardized process.

Acknowledgments

Concept and design (AJJ, JNB, RBB, LAM, MD, MGH); acquisition of data (AJJ, MGH); analysis and interpretation of data (AJJ, JNB, RBB, LAM, MD, MGH); drafting of the manuscript (AJJ); critical revision of the manuscript for important intellectual content (AJJ, JNB, RBB, LAM, MD, MGH); statistical analysis (AJJ); administrative, technical, or logistic support (LAM, MGH); and supervision (MGH).

The negative impact of the unnecessary prescribing of antibiotic is well known. Consequences include exposing patients to antibiotic adverse effects, risk of overgrowth of pathogenetic organisms such as clostridial species, unnecessary cost of drugs, and development of selection of antibiotic-resistant organisms in the populace at large. Acute viral respiratory infections are among the leading causes of inappropriate antibiotic usage.¹ In a study of 1000 adults with respiratory tract infections in an outpatient setting, 77% of patients were prescribed antibiotics, and the treatment was inappropriate in 64% of those who received prescriptions.² Patient expectations and clinician perceptions of these expectations play a role. One study showed that 54% of clinicians felt their patients expected to receive antibiotics for a visit due to an acute respiratory infection (ARI), such as a cough or cold; 26% of patients did in fact have this expectation.³

The US Department of Veterans Affairs (VA) Central Ohio Health Care System is a large ambulatory care facility, with 4 associated community-based outpatient clinics, serving more than 43,000 central Ohio veterans and completing more than 500,000 medical appointments annually. An antimicrobial stewardship program has been in place since 2013. In May 2018, the clinical pharmacist assigned to the program alerted medical leadership that, of 67 patients seen in primary care for ARIs between April 16, 2018, and May 15, 2018, 42 (63%) had been prescribed an antibiotic. Based on this finding, clinical leadership designed a process improvement program aimed at reducing inappropriate antibiotic usage for the treatment of uncomplicated ARls likely due to viral pathogens. Key components were clinician and patient education and the substitution of a symptomatic treatment kit in place of an antibiotic prescription.

Methods

Facility clinical leadership, assisted by Volunteer Services, developed a Viral Illness Support Pack to be dispensed by primary care practitioners (PCPs) to patients presenting with symptoms of viral ARIs. The contents of this support pack are shown in the Figure. Patients were provided with tissues, throat lozenges, lip balm, acetaminophen, hand sanitizer, a surgical mask, patient instructions, and the Antibiotics Aren’t Always the Answer pamphlet.⁴ The contents of the viral support pack were purchased through Volunteer Services using donated funds. In total, 460 packs were distributed to the primary care patient aligned care teams (PACTs), including the community-based outpatient clinics.

Clinicians and care teams received academic detailing prior to distribution of the viral support packs, stressing the importance of avoiding antibiotics to treat viral illnesses. Viral illness support packs were available for distribution from December 1, 2018, through March 31, 2019. The frequency of antibiotic dispensing to patients coded for ARI during this period was compared with that of the same time period in the previous year. All charts were reviewed for coding accuracy. Patients with illnesses requiring antibiotic treatment, such as pneumonia, exacerbations of chronic obstructive pulmonary disease and chronic bronchitis, and streptococcal pharyngitis, were excluded from the study. Statistical significance was determined using the unpaired t test.

Results

From December 1, 2018, to March 31, 2019, 357 viral support packs were distributed to patients (Table). For the historical control period from December 1, 2017, through March 31, 2018, 508 patients were treated for ARIs. Of these, 295 (58%) received clinically inappropriate antibiotics. In contrast, of the 627 patients treated for ARIs during the study period from December 1, 2018, through March 31, 2019, 310 (49%) received clinically inappropriate antibiotics. The 9% decrease during the period when viral support packs were distributed, compared with the prior year, was statistically significant (P = .02).

Discussion

The decrease in antibiotic prescriptions for ARIs was statistically significant. The success of this project can be attributed to 3 factors: clinician education, patient education, and the option for PCPs to provide symptomatic treatment for these patients rather than prescribe an antibiotic.

The importance of antibiotic stewardship has been emphasized to all PCPs at the VA Central Ohio Health Care System. Antibiotic stewardship has been the subject of grand rounds. Prior to distribution of the viral support pack, the chief of specialty medicine, the project’s champion, spoke to all PCPs. Adequate numbers of viral support packs were distributed to all primary care teams.

In addition to direct clinician-to-patient education at the time of the patients’ visits, educational materials were included in the viral support pack. The Antibiotics Aren’t Always the Answer pamphlet is available from the Centers for Disease Control and Prevention. It covers the importance of antibiotic awareness, discusses what antibiotics do and do not treat, how to stay healthy, and causes of antibiotic resistance. The pamphlet contains the clear message that antibiotics are not only ineffective against viral illness, but also can cause significant undesirable outcomes.

The pamphlet Viral Illness Support Pack Traffic Light Card (eAppendix available online at doi:10.12788/fp.0302) provides important clinical information to the patients about their illness. Patients are instructed to contact their primary care team if they are worse after 3 days of illness; symptoms are not improving after 10 days; or they experience blood in respiratory secretions, chills or generalized aching, and localized pain that is one-sided or significantly worsening. Patients are clearly informed to seek further care if not improving with symptomatic treatment.

The ability to provide patients with symptomatic relief, including throat lozenges, lip balm, and acetaminophen, was felt to be important in the success of the project. Furthermore, this eliminated an extra step of the patient needing to visit the pharmacy.

Limitations

Limitations of the study included starting distribution of the support packs somewhat after the onset of the viral illness season, failure to reach all prescribers for academic detailing at the start of the program, and several instances of temporary unavailability of the support packs in some areas.

Conclusions

Patients with ARIs are often significantly symptomatic and frequently believe that they require an antibiotic for treatment. Clinicians may adjust their behavior in response to their patients’ expectations, stated or unstated. The results of this project demonstrate that the combination of patient education and the ready availability of a nonantibiotic symptomatic treatment option can significantly decrease the unnecessary prescribing of antibiotics for viral illnesses.

Acknowledgments

The authors are grateful to Ms. Traci Washington for assistance in sourcing materials; to Karen Corr, PhD, and Anthony Restuccio, MD, for advice on methods; to Mr. Daniel Pignatelli for assistance with data interpretation; and to Mr. Keith Skidmore, Ms. Crystal Conley, and Ms. Megan Harris for assistance with assembling the Viral Illness Support Packs.

The negative impact of the unnecessary prescribing of antibiotic is well known. Consequences include exposing patients to antibiotic adverse effects, risk of overgrowth of pathogenetic organisms such as clostridial species, unnecessary cost of drugs, and development of selection of antibiotic-resistant organisms in the populace at large. Acute viral respiratory infections are among the leading causes of inappropriate antibiotic usage.¹ In a study of 1000 adults with respiratory tract infections in an outpatient setting, 77% of patients were prescribed antibiotics, and the treatment was inappropriate in 64% of those who received prescriptions.² Patient expectations and clinician perceptions of these expectations play a role. One study showed that 54% of clinicians felt their patients expected to receive antibiotics for a visit due to an acute respiratory infection (ARI), such as a cough or cold; 26% of patients did in fact have this expectation.³

The US Department of Veterans Affairs (VA) Central Ohio Health Care System is a large ambulatory care facility, with 4 associated community-based outpatient clinics, serving more than 43,000 central Ohio veterans and completing more than 500,000 medical appointments annually. An antimicrobial stewardship program has been in place since 2013. In May 2018, the clinical pharmacist assigned to the program alerted medical leadership that, of 67 patients seen in primary care for ARIs between April 16, 2018, and May 15, 2018, 42 (63%) had been prescribed an antibiotic. Based on this finding, clinical leadership designed a process improvement program aimed at reducing inappropriate antibiotic usage for the treatment of uncomplicated ARls likely due to viral pathogens. Key components were clinician and patient education and the substitution of a symptomatic treatment kit in place of an antibiotic prescription.

Methods

Facility clinical leadership, assisted by Volunteer Services, developed a Viral Illness Support Pack to be dispensed by primary care practitioners (PCPs) to patients presenting with symptoms of viral ARIs. The contents of this support pack are shown in the Figure. Patients were provided with tissues, throat lozenges, lip balm, acetaminophen, hand sanitizer, a surgical mask, patient instructions, and the Antibiotics Aren’t Always the Answer pamphlet.⁴ The contents of the viral support pack were purchased through Volunteer Services using donated funds. In total, 460 packs were distributed to the primary care patient aligned care teams (PACTs), including the community-based outpatient clinics.

Clinicians and care teams received academic detailing prior to distribution of the viral support packs, stressing the importance of avoiding antibiotics to treat viral illnesses. Viral illness support packs were available for distribution from December 1, 2018, through March 31, 2019. The frequency of antibiotic dispensing to patients coded for ARI during this period was compared with that of the same time period in the previous year. All charts were reviewed for coding accuracy. Patients with illnesses requiring antibiotic treatment, such as pneumonia, exacerbations of chronic obstructive pulmonary disease and chronic bronchitis, and streptococcal pharyngitis, were excluded from the study. Statistical significance was determined using the unpaired t test.

Results

From December 1, 2018, to March 31, 2019, 357 viral support packs were distributed to patients (Table). For the historical control period from December 1, 2017, through March 31, 2018, 508 patients were treated for ARIs. Of these, 295 (58%) received clinically inappropriate antibiotics. In contrast, of the 627 patients treated for ARIs during the study period from December 1, 2018, through March 31, 2019, 310 (49%) received clinically inappropriate antibiotics. The 9% decrease during the period when viral support packs were distributed, compared with the prior year, was statistically significant (P = .02).

Discussion

The decrease in antibiotic prescriptions for ARIs was statistically significant. The success of this project can be attributed to 3 factors: clinician education, patient education, and the option for PCPs to provide symptomatic treatment for these patients rather than prescribe an antibiotic.

The importance of antibiotic stewardship has been emphasized to all PCPs at the VA Central Ohio Health Care System. Antibiotic stewardship has been the subject of grand rounds. Prior to distribution of the viral support pack, the chief of specialty medicine, the project’s champion, spoke to all PCPs. Adequate numbers of viral support packs were distributed to all primary care teams.

In addition to direct clinician-to-patient education at the time of the patients’ visits, educational materials were included in the viral support pack. The Antibiotics Aren’t Always the Answer pamphlet is available from the Centers for Disease Control and Prevention. It covers the importance of antibiotic awareness, discusses what antibiotics do and do not treat, how to stay healthy, and causes of antibiotic resistance. The pamphlet contains the clear message that antibiotics are not only ineffective against viral illness, but also can cause significant undesirable outcomes.

The pamphlet Viral Illness Support Pack Traffic Light Card (eAppendix available online at doi:10.12788/fp.0302) provides important clinical information to the patients about their illness. Patients are instructed to contact their primary care team if they are worse after 3 days of illness; symptoms are not improving after 10 days; or they experience blood in respiratory secretions, chills or generalized aching, and localized pain that is one-sided or significantly worsening. Patients are clearly informed to seek further care if not improving with symptomatic treatment.

The ability to provide patients with symptomatic relief, including throat lozenges, lip balm, and acetaminophen, was felt to be important in the success of the project. Furthermore, this eliminated an extra step of the patient needing to visit the pharmacy.

Limitations

Limitations of the study included starting distribution of the support packs somewhat after the onset of the viral illness season, failure to reach all prescribers for academic detailing at the start of the program, and several instances of temporary unavailability of the support packs in some areas.

Conclusions

Patients with ARIs are often significantly symptomatic and frequently believe that they require an antibiotic for treatment. Clinicians may adjust their behavior in response to their patients’ expectations, stated or unstated. The results of this project demonstrate that the combination of patient education and the ready availability of a nonantibiotic symptomatic treatment option can significantly decrease the unnecessary prescribing of antibiotics for viral illnesses.

Acknowledgments

The authors are grateful to Ms. Traci Washington for assistance in sourcing materials; to Karen Corr, PhD, and Anthony Restuccio, MD, for advice on methods; to Mr. Daniel Pignatelli for assistance with data interpretation; and to Mr. Keith Skidmore, Ms. Crystal Conley, and Ms. Megan Harris for assistance with assembling the Viral Illness Support Packs.

The negative impact of the unnecessary prescribing of antibiotic is well known. Consequences include exposing patients to antibiotic adverse effects, risk of overgrowth of pathogenetic organisms such as clostridial species, unnecessary cost of drugs, and development of selection of antibiotic-resistant organisms in the populace at large. Acute viral respiratory infections are among the leading causes of inappropriate antibiotic usage.¹ In a study of 1000 adults with respiratory tract infections in an outpatient setting, 77% of patients were prescribed antibiotics, and the treatment was inappropriate in 64% of those who received prescriptions.² Patient expectations and clinician perceptions of these expectations play a role. One study showed that 54% of clinicians felt their patients expected to receive antibiotics for a visit due to an acute respiratory infection (ARI), such as a cough or cold; 26% of patients did in fact have this expectation.³

The US Department of Veterans Affairs (VA) Central Ohio Health Care System is a large ambulatory care facility, with 4 associated community-based outpatient clinics, serving more than 43,000 central Ohio veterans and completing more than 500,000 medical appointments annually. An antimicrobial stewardship program has been in place since 2013. In May 2018, the clinical pharmacist assigned to the program alerted medical leadership that, of 67 patients seen in primary care for ARIs between April 16, 2018, and May 15, 2018, 42 (63%) had been prescribed an antibiotic. Based on this finding, clinical leadership designed a process improvement program aimed at reducing inappropriate antibiotic usage for the treatment of uncomplicated ARls likely due to viral pathogens. Key components were clinician and patient education and the substitution of a symptomatic treatment kit in place of an antibiotic prescription.

Methods

Facility clinical leadership, assisted by Volunteer Services, developed a Viral Illness Support Pack to be dispensed by primary care practitioners (PCPs) to patients presenting with symptoms of viral ARIs. The contents of this support pack are shown in the Figure. Patients were provided with tissues, throat lozenges, lip balm, acetaminophen, hand sanitizer, a surgical mask, patient instructions, and the Antibiotics Aren’t Always the Answer pamphlet.⁴ The contents of the viral support pack were purchased through Volunteer Services using donated funds. In total, 460 packs were distributed to the primary care patient aligned care teams (PACTs), including the community-based outpatient clinics.

Clinicians and care teams received academic detailing prior to distribution of the viral support packs, stressing the importance of avoiding antibiotics to treat viral illnesses. Viral illness support packs were available for distribution from December 1, 2018, through March 31, 2019. The frequency of antibiotic dispensing to patients coded for ARI during this period was compared with that of the same time period in the previous year. All charts were reviewed for coding accuracy. Patients with illnesses requiring antibiotic treatment, such as pneumonia, exacerbations of chronic obstructive pulmonary disease and chronic bronchitis, and streptococcal pharyngitis, were excluded from the study. Statistical significance was determined using the unpaired t test.

Results

From December 1, 2018, to March 31, 2019, 357 viral support packs were distributed to patients (Table). For the historical control period from December 1, 2017, through March 31, 2018, 508 patients were treated for ARIs. Of these, 295 (58%) received clinically inappropriate antibiotics. In contrast, of the 627 patients treated for ARIs during the study period from December 1, 2018, through March 31, 2019, 310 (49%) received clinically inappropriate antibiotics. The 9% decrease during the period when viral support packs were distributed, compared with the prior year, was statistically significant (P = .02).

Discussion

The decrease in antibiotic prescriptions for ARIs was statistically significant. The success of this project can be attributed to 3 factors: clinician education, patient education, and the option for PCPs to provide symptomatic treatment for these patients rather than prescribe an antibiotic.

The importance of antibiotic stewardship has been emphasized to all PCPs at the VA Central Ohio Health Care System. Antibiotic stewardship has been the subject of grand rounds. Prior to distribution of the viral support pack, the chief of specialty medicine, the project’s champion, spoke to all PCPs. Adequate numbers of viral support packs were distributed to all primary care teams.

In addition to direct clinician-to-patient education at the time of the patients’ visits, educational materials were included in the viral support pack. The Antibiotics Aren’t Always the Answer pamphlet is available from the Centers for Disease Control and Prevention. It covers the importance of antibiotic awareness, discusses what antibiotics do and do not treat, how to stay healthy, and causes of antibiotic resistance. The pamphlet contains the clear message that antibiotics are not only ineffective against viral illness, but also can cause significant undesirable outcomes.

The pamphlet Viral Illness Support Pack Traffic Light Card (eAppendix available online at doi:10.12788/fp.0302) provides important clinical information to the patients about their illness. Patients are instructed to contact their primary care team if they are worse after 3 days of illness; symptoms are not improving after 10 days; or they experience blood in respiratory secretions, chills or generalized aching, and localized pain that is one-sided or significantly worsening. Patients are clearly informed to seek further care if not improving with symptomatic treatment.

The ability to provide patients with symptomatic relief, including throat lozenges, lip balm, and acetaminophen, was felt to be important in the success of the project. Furthermore, this eliminated an extra step of the patient needing to visit the pharmacy.

Limitations

Limitations of the study included starting distribution of the support packs somewhat after the onset of the viral illness season, failure to reach all prescribers for academic detailing at the start of the program, and several instances of temporary unavailability of the support packs in some areas.

Conclusions

Patients with ARIs are often significantly symptomatic and frequently believe that they require an antibiotic for treatment. Clinicians may adjust their behavior in response to their patients’ expectations, stated or unstated. The results of this project demonstrate that the combination of patient education and the ready availability of a nonantibiotic symptomatic treatment option can significantly decrease the unnecessary prescribing of antibiotics for viral illnesses.

Acknowledgments

The authors are grateful to Ms. Traci Washington for assistance in sourcing materials; to Karen Corr, PhD, and Anthony Restuccio, MD, for advice on methods; to Mr. Daniel Pignatelli for assistance with data interpretation; and to Mr. Keith Skidmore, Ms. Crystal Conley, and Ms. Megan Harris for assistance with assembling the Viral Illness Support Packs.

In the past 10 years, artificial intelligence (AI) applications have exploded in numerous fields, including medicine. Myriad publications report that the use of AI in health care is increasing, and AI has shown utility in many medical specialties, eg, pathology, radiology, and oncology.^1,2

In cancer pathology, AI was able not only to detect various cancers, but also to subtype and grade them. In addition, AI could predict survival, the success of therapeutic response, and underlying mutations from histopathologic images.³ In other medical fields, AI applications are as notable. For example, in imaging specialties like radiology, ophthalmology, dermatology, and gastroenterology, AI is being used for image recognition, enhancement, and segmentation. In addition, AI is beneficial for predicting disease progression, survival, and response to therapy in other medical specialties. Finally, AI may help with administrative tasks like scheduling.

However, many obstacles to successfully implementing AI programs in the clinical setting exist, including clinical data limitations and ethical use of data, trust in the AI models, regulatory barriers, and lack of clinical buy-in due to insufficient basic AI understanding.² To address these barriers to successful clinical AI implementation, we decided to create a formal governing body at James A. Haley Veterans’ Hospital in Tampa, Florida. Accordingly, the hospital AI committee charter was officially approved on July 22, 2021. Our model could be used by both US Department of Veterans Affairs (VA) and non-VA hospitals throughout the country.

AI Committee

The vision of the AI committee is to improve outcomes and experiences for our veterans by developing trustworthy AI capabilities to support the VA mission. The mission is to build robust capacity in AI to create and apply innovative AI solutions and transform the VA by facilitating a learning environment that supports the delivery of world-class benefits and services to our veterans. Our vision and mission are aligned with the VA National AI Institute. ⁴

The AI Committee comprises 7 subcommittees: ethics, AI clinical product evaluation, education, data sharing and acquisition, research, 3D printing, and improvement and innovation. The role of the ethics subcommittee is to ensure the ethical and equitable implementation of clinical AI. We created the ethics subcommittee guidelines based on the World Health Organization ethics and governance of AI for health documents.⁵ They include 6 basic principles: protecting human autonomy; promoting human well-being and safety and the public interest; ensuring transparency, explainability, and intelligibility; fostering responsibility and accountability; ensuring inclusiveness and equity; and promoting AI that is responsive and sustainable (Table 1).

Implementations

After a comprehensive evaluation, we implemented 2 ClearRead (Riverain Technologies) AI radiology solutions. ClearRead CT Vessel Suppress produces a secondary series of computed tomography (CT) images, suppressing vessels and other normal structures within the lungs to improve nodule detectability, and ClearRead Xray Bone Suppress, which increases the visibility of soft tissue in standard chest X-rays by suppressing the bone on the digital image without the need for 2 exposures.

The role of the education subcommittee is to educate the staff about AI and how it can improve patient care. Every Friday, we email an AI article of the week to our practitioners. In addition, we publish a newsletter, and we organize an annual AI conference. The first conference in 2022 included speakers from the National AI Institute, Moffitt Cancer Center, the University of South Florida, and our facility.

As the name indicates, the data sharing and acquisition subcommittee oversees preparing data for our clinical and research projects. The role of the research subcommittee is to coordinate and promote AI research with the ultimate goal of improving patient care.

Other Technologies

Although 3D printing does not fall under the umbrella of AI, we have decided to include it in our future-oriented AI committee. We created an online 3D printing course to promote the technology throughout the VA. We 3D print organ models to help surgeons prepare for complicated operations. In addition, together with our colleagues from the University of Florida, we used 3D printing to address the shortage of swabs for COVID-19 testing. The VA Sunshine Healthcare Network (Veterans Integrated Services Network 8) has an active Innovation and Improvement Committee. ⁹ Our improvement and innovation subcommittee serves as a coordinating body with the network committee .

Conclusions

Through the hospital AI committee, we believe that we may overcome many obstacles to successfully implementing AI applications in the clinical setting, including the ethical use of data, trust in the AI models, regulatory barriers, and lack of clinical buy-in due to insufficient basic AI knowledge.

Acknowledgments

This material is the result of work supported with resources and the use of facilities at the James A. Haley Veterans’ Hospital.

In the past 10 years, artificial intelligence (AI) applications have exploded in numerous fields, including medicine. Myriad publications report that the use of AI in health care is increasing, and AI has shown utility in many medical specialties, eg, pathology, radiology, and oncology.^1,2

In cancer pathology, AI was able not only to detect various cancers, but also to subtype and grade them. In addition, AI could predict survival, the success of therapeutic response, and underlying mutations from histopathologic images.³ In other medical fields, AI applications are as notable. For example, in imaging specialties like radiology, ophthalmology, dermatology, and gastroenterology, AI is being used for image recognition, enhancement, and segmentation. In addition, AI is beneficial for predicting disease progression, survival, and response to therapy in other medical specialties. Finally, AI may help with administrative tasks like scheduling.

However, many obstacles to successfully implementing AI programs in the clinical setting exist, including clinical data limitations and ethical use of data, trust in the AI models, regulatory barriers, and lack of clinical buy-in due to insufficient basic AI understanding.² To address these barriers to successful clinical AI implementation, we decided to create a formal governing body at James A. Haley Veterans’ Hospital in Tampa, Florida. Accordingly, the hospital AI committee charter was officially approved on July 22, 2021. Our model could be used by both US Department of Veterans Affairs (VA) and non-VA hospitals throughout the country.

AI Committee

The vision of the AI committee is to improve outcomes and experiences for our veterans by developing trustworthy AI capabilities to support the VA mission. The mission is to build robust capacity in AI to create and apply innovative AI solutions and transform the VA by facilitating a learning environment that supports the delivery of world-class benefits and services to our veterans. Our vision and mission are aligned with the VA National AI Institute. ⁴

The AI Committee comprises 7 subcommittees: ethics, AI clinical product evaluation, education, data sharing and acquisition, research, 3D printing, and improvement and innovation. The role of the ethics subcommittee is to ensure the ethical and equitable implementation of clinical AI. We created the ethics subcommittee guidelines based on the World Health Organization ethics and governance of AI for health documents.⁵ They include 6 basic principles: protecting human autonomy; promoting human well-being and safety and the public interest; ensuring transparency, explainability, and intelligibility; fostering responsibility and accountability; ensuring inclusiveness and equity; and promoting AI that is responsive and sustainable (Table 1).

Implementations

After a comprehensive evaluation, we implemented 2 ClearRead (Riverain Technologies) AI radiology solutions. ClearRead CT Vessel Suppress produces a secondary series of computed tomography (CT) images, suppressing vessels and other normal structures within the lungs to improve nodule detectability, and ClearRead Xray Bone Suppress, which increases the visibility of soft tissue in standard chest X-rays by suppressing the bone on the digital image without the need for 2 exposures.

The role of the education subcommittee is to educate the staff about AI and how it can improve patient care. Every Friday, we email an AI article of the week to our practitioners. In addition, we publish a newsletter, and we organize an annual AI conference. The first conference in 2022 included speakers from the National AI Institute, Moffitt Cancer Center, the University of South Florida, and our facility.

As the name indicates, the data sharing and acquisition subcommittee oversees preparing data for our clinical and research projects. The role of the research subcommittee is to coordinate and promote AI research with the ultimate goal of improving patient care.

Other Technologies

Although 3D printing does not fall under the umbrella of AI, we have decided to include it in our future-oriented AI committee. We created an online 3D printing course to promote the technology throughout the VA. We 3D print organ models to help surgeons prepare for complicated operations. In addition, together with our colleagues from the University of Florida, we used 3D printing to address the shortage of swabs for COVID-19 testing. The VA Sunshine Healthcare Network (Veterans Integrated Services Network 8) has an active Innovation and Improvement Committee. ⁹ Our improvement and innovation subcommittee serves as a coordinating body with the network committee .

Conclusions

Through the hospital AI committee, we believe that we may overcome many obstacles to successfully implementing AI applications in the clinical setting, including the ethical use of data, trust in the AI models, regulatory barriers, and lack of clinical buy-in due to insufficient basic AI knowledge.

Acknowledgments

This material is the result of work supported with resources and the use of facilities at the James A. Haley Veterans’ Hospital.

In the past 10 years, artificial intelligence (AI) applications have exploded in numerous fields, including medicine. Myriad publications report that the use of AI in health care is increasing, and AI has shown utility in many medical specialties, eg, pathology, radiology, and oncology.^1,2

In cancer pathology, AI was able not only to detect various cancers, but also to subtype and grade them. In addition, AI could predict survival, the success of therapeutic response, and underlying mutations from histopathologic images.³ In other medical fields, AI applications are as notable. For example, in imaging specialties like radiology, ophthalmology, dermatology, and gastroenterology, AI is being used for image recognition, enhancement, and segmentation. In addition, AI is beneficial for predicting disease progression, survival, and response to therapy in other medical specialties. Finally, AI may help with administrative tasks like scheduling.

However, many obstacles to successfully implementing AI programs in the clinical setting exist, including clinical data limitations and ethical use of data, trust in the AI models, regulatory barriers, and lack of clinical buy-in due to insufficient basic AI understanding.² To address these barriers to successful clinical AI implementation, we decided to create a formal governing body at James A. Haley Veterans’ Hospital in Tampa, Florida. Accordingly, the hospital AI committee charter was officially approved on July 22, 2021. Our model could be used by both US Department of Veterans Affairs (VA) and non-VA hospitals throughout the country.

AI Committee

The vision of the AI committee is to improve outcomes and experiences for our veterans by developing trustworthy AI capabilities to support the VA mission. The mission is to build robust capacity in AI to create and apply innovative AI solutions and transform the VA by facilitating a learning environment that supports the delivery of world-class benefits and services to our veterans. Our vision and mission are aligned with the VA National AI Institute. ⁴

The AI Committee comprises 7 subcommittees: ethics, AI clinical product evaluation, education, data sharing and acquisition, research, 3D printing, and improvement and innovation. The role of the ethics subcommittee is to ensure the ethical and equitable implementation of clinical AI. We created the ethics subcommittee guidelines based on the World Health Organization ethics and governance of AI for health documents.⁵ They include 6 basic principles: protecting human autonomy; promoting human well-being and safety and the public interest; ensuring transparency, explainability, and intelligibility; fostering responsibility and accountability; ensuring inclusiveness and equity; and promoting AI that is responsive and sustainable (Table 1).

Implementations

After a comprehensive evaluation, we implemented 2 ClearRead (Riverain Technologies) AI radiology solutions. ClearRead CT Vessel Suppress produces a secondary series of computed tomography (CT) images, suppressing vessels and other normal structures within the lungs to improve nodule detectability, and ClearRead Xray Bone Suppress, which increases the visibility of soft tissue in standard chest X-rays by suppressing the bone on the digital image without the need for 2 exposures.

The role of the education subcommittee is to educate the staff about AI and how it can improve patient care. Every Friday, we email an AI article of the week to our practitioners. In addition, we publish a newsletter, and we organize an annual AI conference. The first conference in 2022 included speakers from the National AI Institute, Moffitt Cancer Center, the University of South Florida, and our facility.

As the name indicates, the data sharing and acquisition subcommittee oversees preparing data for our clinical and research projects. The role of the research subcommittee is to coordinate and promote AI research with the ultimate goal of improving patient care.

Other Technologies

Although 3D printing does not fall under the umbrella of AI, we have decided to include it in our future-oriented AI committee. We created an online 3D printing course to promote the technology throughout the VA. We 3D print organ models to help surgeons prepare for complicated operations. In addition, together with our colleagues from the University of Florida, we used 3D printing to address the shortage of swabs for COVID-19 testing. The VA Sunshine Healthcare Network (Veterans Integrated Services Network 8) has an active Innovation and Improvement Committee. ⁹ Our improvement and innovation subcommittee serves as a coordinating body with the network committee .

Conclusions

Through the hospital AI committee, we believe that we may overcome many obstacles to successfully implementing AI applications in the clinical setting, including the ethical use of data, trust in the AI models, regulatory barriers, and lack of clinical buy-in due to insufficient basic AI knowledge.

Acknowledgments

This material is the result of work supported with resources and the use of facilities at the James A. Haley Veterans’ Hospital.