Chaim M. Bell, MD, PhD

Healthcare systems are targeting effective strategies to improve patient safety and reduce hospital readmissions. Hospital readmissions can be detrimental to patients’ health, a source of avoidable healthcare costs, and are frequently a reflection of the quality of patient care during transitions of care. Medication reconciliation (Med Rec) was identified as 1 of 12 interventions that may reduce 30-day readmissions; however, rigorously designed studies are scarce.^1,2 Published systematic reviews and meta-analyses have produced mixed conclusions regarding the impact of Med Rec on unplanned 30-day readmissions.^2-4

In several studies, researchers have established the positive impact of Med Rec on reducing patient medication discrepancies and potential adverse drug events.^4-8 Pharmacy-led Med Rec interventions have been shown to easily identify more clinically relevant and higher impact medication discrepancies when compared to usual care.⁸ In a systematic review, Mueller et al.² suggest that there are several interrelated elements that determine if a Med Rec intervention will influence hospital readmissions. These elements form a multicomponent “bundle” of interventions, including a systematic medication history process, admission reconciliation, patient education on discharge, discharge reconciliation, and communication to outpatient providers.⁹ Several prospective randomized controlled studies have demonstrated lower readmission rates and fewer visits to the emergency department (ED) after implementing a comprehensive, interprofessional, bundled intervention (including Med Rec) from admission to discharge.^10-13 A 2016 systematic review and meta-analysis specifically evaluated pharmacy-led Med Rec programs (the majority of which included interventions involving multicomponent bundles) and demonstrated a significant reduction in posthospital healthcare utilization.¹⁴

Although comprehensive, interprofessional, bundled interventions have been shown to reduce readmission rates and ED visits in randomized controlled trials (RCTs), limited resources often prevent hospitals from consistently implementing all aspects of these multicomponent interventions. In practice, clinicians may provide varying components of the bundle, such as the combination of admission medication history by the pharmacist and discharge Med Rec completed by the physician alone. The unique impact of combined pharmacist and prescriber Med Rec interventions from admission to discharge on readmissions remains inconclusive. Further, it is unclear which high-risk patient groups will benefit the most from these interventions. We set out to evaluate the impact of an enhanced, interprofessional Med Rec process from admission to discharge (characterized within the context of a novel taxonomy continuum that specifies clinician involvement and intensity of services) on readmissions to hospital and ED visits within 30 days of discharge.

We conducted a retrospective, observational, analytical cohort study using QuadraMed’s Computerized Patient Record and the EMITT (Electronic Medication Information Transfer Tool)¹⁵ to collect data from 2007 to 2011.

Setting

The study was conducted at a 417-bed tertiary care teaching hospital in Toronto, Ontario, Canada.

Med Rec Process and Description of Exposure (Intervention)

The targeted clinical areas had sustained interprofessional models of patient care in place from admission to discharge. They also were actively using an in-house EMITT to facilitate the documentation and tracking of Med Rec efforts throughout patient admission, transfer, and discharge.¹⁵ On admission, the pharmacist conducted a best possible medication history (BPMH). A BPMH provides the cornerstone for Med Rec. It differs from a routine medication history in that it involves (1) a systematic process for interviewing the patient (or family) and (2) a review of at least one other reliable source of information (eg, a provincial medication database, an inspection of medication vials, or contact with the community pharmacy) to obtain and verify patient medications (prescribed and nonprescribed). The pharmacist recorded the BPMH in the electronic patient record. The application supported admission and discharge Med Rec. On discharge, there were 2 options: (1) the prescriber alone would review and complete the discharge Med Rec and generate electronic prescriptions (Table 1, Silver level care) or (2) the pharmacist would collaborate with the prescriber to complete the discharge reconciliation and the prescriber would electronically generate prescriptions (Table 1, Gold level care). All clinical areas had a combined pharmacist and prescriber Med Rec model in place at admission, but the proportion of patients receiving discharge reconciliation completed by pharmacist and prescriber versus the prescriber-alone varied based on the individual clinician’s practices.

Patient Selection

All consecutive hospitalized patients admitted and discharged by the general internal medicine [GIM] service from March 2007 to December 2011 were included. The GIM service was chosen for the main analysis because they had been performing the intervention for the longest period of time and had the largest population of patients. Patients were identified via their hospital-specific medical record identification number and specific hospital-visit number. Patients were excluded if any of the following occurred: (1) the length of stay of their index admission was less than 24 hours; (2) they died during the visit; (3) they were transferred to a separate acute care inpatient facility; or (4) they left hospital against medical advice. Patient visits were excluded as index cases from the analysis if they were returning within 90 days of a previous discharge.

Outcomes

The primary study outcome was the occurrence of an inpatient readmission or ED visit within 30 days of discharge. In our secondary analyses, we examined the impact of the intervention on high-risk patient populations, such as those ≥65 years of age, with a length of stay, acuity of admission, Charlson comorbidity index, and emergency department visits in past 6 months (LACE) index score ≥10 (see supplementary Appendix 1 for LACE score description), on high-alert medications (1 or more of warfarin, insulin, digoxin, and opioids), and on ≥10 medications.

Data Collection

Identification of Exposure of Interest

We used the electronic database to capture all patients who received pharmacist and prescriber supported admission-to-discharge reconciliation. We explicitly defined increasing intensity of Med Rec care in categories of Bronze, Silver, and Gold care levels (Table 1). The exposed (intervention) group received an enhanced Med Rec bundle (patients receiving Gold level care). The control group was made of patients receiving a partial Med Rec Bundle (patients receiving Silver or Bronze level of care or below).

Determination of Hospital Visits

A search of administrative databases was used to determine if patients admitted to the targeted services had an ED visit or urgent inpatient admission to the study hospital within 30 days.

Statistical Analysis

A logistic regression for outcomes was performed. This yielded an adjusted odds ratio with a 95% confidence interval (CI) between the intervention and control groups. Statistical significance was determined with a 2-sided α level of 0.05. In the analysis, we used Statistical Analysis Software version 9.2.

In our multivariate logistic regression model, we adjusted for confounding factors that might influence the patients’ risk of readmission or the type of Med Rec they received upon discharge. By using administrative databases, patient level demographics, and the Charlson comorbidity index, the most responsible diagnosis and disease burden were collected. Medication-related factors collected included the number of medications on discharge and the presence of predefined high-alert medications. The number of medications on the medication discharge list was determined by using the electronic database. The final adjustment model included age, gender, the number of medications on discharge, and the LACE index score (supplementary Appendix 1). The LACE index score has been validated in Ontario, Canada, populations to quantify the risk of death or unplanned readmission within 30 days of discharge.²⁴

Propensity Score Adjustment

Propensity scoring (probability of treatment assignment conditional on observed baseline characteristics) was planned a priori to account for possible factors that would impact whether a patient received the intervention or control care levels. The propensity score for receiving Med Rec was computed from a logistic model using Med Rec as the outcome. A structured iterative approach was used to refine this model to achieve covariate balance within the matched pairs. Covariate balance was measured by the standardized difference, in which an absolute standardized difference >10% represents meaningful imbalance.²⁵ From the original cohort, we attempted to match patients who had the intervention to patients from the control by means of a matching algorithm using the logit of the propensity score for receiving the intervention.²⁶

Subgroup Analysis

We also examined the impact of the intervention on high-risk patient populations such as those ≥65 years of age, with a LACE index score ≥10, on high-alert medications, and on ≥10 medications. A univariate analysis was conducted to identify patient-related risk predictors that may be independently correlated with a higher risk of hospital visits.

Baseline Characteristics

A total of 8678 patients representing 9931 unique visits met the inclusion criteria for analysis. There were 2541 unique visits (approximately 26% of visits) in the intervention group that received Gold level care and 7390 unique visits in the control group. The patients in the control group were largely patients who received the original standard of care at the institution, Silver level care (67% of the control group). Patients who received Bronze level care or less comprised 33% of the control group.

Patients in the intervention group were significantly older (average of 68 years old versus 64 years old) and on more medications. They also notably had a longer duration of stay in hospital, an increased percentage of visits with a LACE index score ≥10, and were more likely to be discharged home on a high-alert medication and with supports (Table 2).

Main Analysis

The main unadjusted analysis of GIM patients (n = 9931 visits) did not detect a difference in 30-day ED visits and readmissions between the intervention group (540 out of 2541; 21.2%) and control (1423 out of 7390; 19.3%; Table 3). By using a multivariate logistic regression model to account for age, sex, LACE index, and number of medications on discharge, the adjusted odds ratio was 1.06 (95% CI, 0.95-1.19; P = 0.33). After propensity score adjustment, the relative risk of readmission was 0.88 (16.7% vs 18.9%; 95% CI, 0.59-1.32; P = 0.54).

Secondary Analyses

In each predefined high-risk patient subgroup (age ≥65, LACE index score ≥10, number of discharge medications ≥10, and the presence of high-alert medications), analyses of our primary endpoint did not detect significant adjusted odds ratios (Table 4). In our univariate analysis, increasing number of medications, LACE index score, and male gender were independently correlated with a higher risk of hospital visits (supplementary Appendix 2).

Med Rec is widely recommended as a patient safety strategy to prevent clinically significant medication discrepancies at transitions in care.^4-9 However, Med Rec varies widely in terms of what it entails and who delivers it, with the preponderance of evidence suggesting an impact on clinically significant medication discrepancies only when interprofessional care delivered includes a central role for pharmacists.²⁷ Furthermore, Med Rec appears to impact short term readmissions only when embedded in a broader, multifaceted, bundled intervention in which pharmacists or other team members educate patients about their medications and deliver postdischarge follow-up phone calls.^10-13

As very few hospitals have the resources to sustainably deliver intensive care bundles that are represented in RCTs (characterized by Platinum and Diamond levels of care in Table 1), in our observational study, we sought to explore whether a resource-attainable, enhanced Med Rec care bundle (Gold level) had an impact on hospital utilization compared to partial Med Rec care bundles (Bronze and Silver levels). In our findings, we did not detect a significant difference on ED visits and readmissions within 30 days between enhanced and partial care bundles. In a secondary analysis of the influence of the intervention on prespecified high-risk patient subgroups, we also did not detect a difference.

As far as we are aware, our long-term, observational study is the largest to date to explore a real-life, enhanced Med Rec intervention and examine its impact on meaningful patient outcomes. We extrapolated that our intervention group received several critical attributes of a successful bundle as discussed by Mueller in a systematic review.² Our intervention included the following: (1) a systematic BPMH process on admission; (2) integrated admission-to-discharge reconciliation processes; (3) discharge delineation of medication changes since admission; (4) pharmacist involvement in reconciliation from admission to discharge; (5) an electronic platform; and (6) formal discharge reconciliation with interprofessional collaboration. Additional components in the bundle described by Mueller included the following: patient education at discharge, postdischarge communication with the patient, and communication with outpatient providers and medication management.

In our results, we did not find a difference in outcomes between the intervention and control groups. Therefore, it is possible that the enhanced bundle’s focus on interprofessional involvement in discharge reconciliation (Gold care level) has no impact on hospital utilization compared to partial care bundles (Silver and Bronze levels). Kwan et al.³ describe similar findings in their systematic review, in which they evaluated the effects of hospital-based Med Rec on unintentional discrepancies with nontrivial risks for harm to patients on 30-day postdischarge hospital visits. Kwan et al.³ concluded that larger well-designed studies are required to further evaluate this outcome, but authors of current published studies suggest that Med Rec alone probably does not reduce postdischarge hospital utilization within 30 days. Med Rec may have a more significant impact on utilization when bundled with other interventions that improve discharge coordination.³

There may be several reasons why we were unable to detect a significant difference between the intervention and control groups. One limitation is that our nonrandomized, retrospective design may have led to unmeasured confounders that impacted allocation into the intervention group versus the control group. It was notable that patients in the intervention group had an increased age, longer duration of hospital stay, more medications, and high-alert medications on discharge compared to the control group and that may have biased our results towards the null hypothesis. Although the propensity score analysis attempted to adjust for this, it also did not detect a significant difference between groups.

In addition, the existing standard of care during the study period allowed for patients in the control group to receive varying levels of Med Rec. Ideally, we would have compared the intervention to a placebo group that did not receive any Med Rec-related care elements. However, as this was a real-life observational study, the majority of patients received some Med Rec services as a part of the standard of care. As a result, 67% of patients in the control group received Silver level Med Rec with a BPMH, admission reconciliation, and prescriber-only discharge reconciliation. This may have made it more difficult to show an incremental benefit on readmissions between the intervention and control.

Also, our primary outcome of all-cause ED or hospital readmissions within 30 days may not have been sensitive enough to detect the effect of Med Rec interventions alone. Only a small proportion of readmissions within 30 days of discharge are preventable and many patient and community level factors responsible for readmissions cannot be controlled by the hospital’s actions.²⁸ Comprehensive pharmacy interventions have demonstrated decreased hospitalizations and emergency visits at 12 months; however, the largest impact was seen on the more specific outcome of medication-related hospitalizations (80% reduction).²⁹ Lastly, another limitation was that we were unable to capture hospital visits to other centres. However, in our region, almost 75% of readmissions are to the same site as the initial hospitalization.³⁰

Overall, our findings in this study and novel characterization of Med Rec services are relevant to many hospital sites that are striving to implement integrated Med Rec with limited healthcare resources. Although interprofessional Med Rec likely reduces clinically significant medication discrepancies, enhanced interprofessional Med Rec on discharge (Gold Med Rec) alone may not be enough to impact hospital utilization compared to partial Med Rec services (Silver and Bronze Med Rec). Further research into practical, targeted Med Rec bundles on more specific outcomes (such as preventable postdischarge adverse events, “avoidable” hospital readmissions, and medication-related readmissions) may detect a significant benefit.

A long-term observational evaluation of interprofessional Med Rec did not detect a difference in 30-day postdischarge patient hospital visits between patients who received enhanced versus partial Med Rec patient care bundles. Researchers of future prospective studies could focus on evaluating high-risk populations or specific elements of Med Rec services on avoidable medication-related hospital admissions and postdischarge adverse drug events.

Acknowledgments

The authors thank Nita Dhir, MBA.

Presented as a poster and oral presentation at the 2012 American College of Clinical Pharmacy Annual Meeting, Hollywood, Florida, October 21-24, 2012, and as an encore poster presentation at the Canadian Society of Hospital Pharmacists Professional Practice Conference, Toronto, Canada, Feb 3, 2013.

Disclosure

The authors declare no conflicts of interest related to the manuscript submitted. All monies used for the research came from the University Health Network Department of Pharmacy Budget, including the pharmacy residency program.

Healthcare systems are targeting effective strategies to improve patient safety and reduce hospital readmissions. Hospital readmissions can be detrimental to patients’ health, a source of avoidable healthcare costs, and are frequently a reflection of the quality of patient care during transitions of care. Medication reconciliation (Med Rec) was identified as 1 of 12 interventions that may reduce 30-day readmissions; however, rigorously designed studies are scarce.^1,2 Published systematic reviews and meta-analyses have produced mixed conclusions regarding the impact of Med Rec on unplanned 30-day readmissions.^2-4

In several studies, researchers have established the positive impact of Med Rec on reducing patient medication discrepancies and potential adverse drug events.^4-8 Pharmacy-led Med Rec interventions have been shown to easily identify more clinically relevant and higher impact medication discrepancies when compared to usual care.⁸ In a systematic review, Mueller et al.² suggest that there are several interrelated elements that determine if a Med Rec intervention will influence hospital readmissions. These elements form a multicomponent “bundle” of interventions, including a systematic medication history process, admission reconciliation, patient education on discharge, discharge reconciliation, and communication to outpatient providers.⁹ Several prospective randomized controlled studies have demonstrated lower readmission rates and fewer visits to the emergency department (ED) after implementing a comprehensive, interprofessional, bundled intervention (including Med Rec) from admission to discharge.^10-13 A 2016 systematic review and meta-analysis specifically evaluated pharmacy-led Med Rec programs (the majority of which included interventions involving multicomponent bundles) and demonstrated a significant reduction in posthospital healthcare utilization.¹⁴

Although comprehensive, interprofessional, bundled interventions have been shown to reduce readmission rates and ED visits in randomized controlled trials (RCTs), limited resources often prevent hospitals from consistently implementing all aspects of these multicomponent interventions. In practice, clinicians may provide varying components of the bundle, such as the combination of admission medication history by the pharmacist and discharge Med Rec completed by the physician alone. The unique impact of combined pharmacist and prescriber Med Rec interventions from admission to discharge on readmissions remains inconclusive. Further, it is unclear which high-risk patient groups will benefit the most from these interventions. We set out to evaluate the impact of an enhanced, interprofessional Med Rec process from admission to discharge (characterized within the context of a novel taxonomy continuum that specifies clinician involvement and intensity of services) on readmissions to hospital and ED visits within 30 days of discharge.

We conducted a retrospective, observational, analytical cohort study using QuadraMed’s Computerized Patient Record and the EMITT (Electronic Medication Information Transfer Tool)¹⁵ to collect data from 2007 to 2011.

Setting

The study was conducted at a 417-bed tertiary care teaching hospital in Toronto, Ontario, Canada.

Med Rec Process and Description of Exposure (Intervention)

The targeted clinical areas had sustained interprofessional models of patient care in place from admission to discharge. They also were actively using an in-house EMITT to facilitate the documentation and tracking of Med Rec efforts throughout patient admission, transfer, and discharge.¹⁵ On admission, the pharmacist conducted a best possible medication history (BPMH). A BPMH provides the cornerstone for Med Rec. It differs from a routine medication history in that it involves (1) a systematic process for interviewing the patient (or family) and (2) a review of at least one other reliable source of information (eg, a provincial medication database, an inspection of medication vials, or contact with the community pharmacy) to obtain and verify patient medications (prescribed and nonprescribed). The pharmacist recorded the BPMH in the electronic patient record. The application supported admission and discharge Med Rec. On discharge, there were 2 options: (1) the prescriber alone would review and complete the discharge Med Rec and generate electronic prescriptions (Table 1, Silver level care) or (2) the pharmacist would collaborate with the prescriber to complete the discharge reconciliation and the prescriber would electronically generate prescriptions (Table 1, Gold level care). All clinical areas had a combined pharmacist and prescriber Med Rec model in place at admission, but the proportion of patients receiving discharge reconciliation completed by pharmacist and prescriber versus the prescriber-alone varied based on the individual clinician’s practices.

Patient Selection

All consecutive hospitalized patients admitted and discharged by the general internal medicine [GIM] service from March 2007 to December 2011 were included. The GIM service was chosen for the main analysis because they had been performing the intervention for the longest period of time and had the largest population of patients. Patients were identified via their hospital-specific medical record identification number and specific hospital-visit number. Patients were excluded if any of the following occurred: (1) the length of stay of their index admission was less than 24 hours; (2) they died during the visit; (3) they were transferred to a separate acute care inpatient facility; or (4) they left hospital against medical advice. Patient visits were excluded as index cases from the analysis if they were returning within 90 days of a previous discharge.

Outcomes

The primary study outcome was the occurrence of an inpatient readmission or ED visit within 30 days of discharge. In our secondary analyses, we examined the impact of the intervention on high-risk patient populations, such as those ≥65 years of age, with a length of stay, acuity of admission, Charlson comorbidity index, and emergency department visits in past 6 months (LACE) index score ≥10 (see supplementary Appendix 1 for LACE score description), on high-alert medications (1 or more of warfarin, insulin, digoxin, and opioids), and on ≥10 medications.

Data Collection

Identification of Exposure of Interest

We used the electronic database to capture all patients who received pharmacist and prescriber supported admission-to-discharge reconciliation. We explicitly defined increasing intensity of Med Rec care in categories of Bronze, Silver, and Gold care levels (Table 1). The exposed (intervention) group received an enhanced Med Rec bundle (patients receiving Gold level care). The control group was made of patients receiving a partial Med Rec Bundle (patients receiving Silver or Bronze level of care or below).

Determination of Hospital Visits

A search of administrative databases was used to determine if patients admitted to the targeted services had an ED visit or urgent inpatient admission to the study hospital within 30 days.

Statistical Analysis

A logistic regression for outcomes was performed. This yielded an adjusted odds ratio with a 95% confidence interval (CI) between the intervention and control groups. Statistical significance was determined with a 2-sided α level of 0.05. In the analysis, we used Statistical Analysis Software version 9.2.

In our multivariate logistic regression model, we adjusted for confounding factors that might influence the patients’ risk of readmission or the type of Med Rec they received upon discharge. By using administrative databases, patient level demographics, and the Charlson comorbidity index, the most responsible diagnosis and disease burden were collected. Medication-related factors collected included the number of medications on discharge and the presence of predefined high-alert medications. The number of medications on the medication discharge list was determined by using the electronic database. The final adjustment model included age, gender, the number of medications on discharge, and the LACE index score (supplementary Appendix 1). The LACE index score has been validated in Ontario, Canada, populations to quantify the risk of death or unplanned readmission within 30 days of discharge.²⁴

Propensity Score Adjustment

Propensity scoring (probability of treatment assignment conditional on observed baseline characteristics) was planned a priori to account for possible factors that would impact whether a patient received the intervention or control care levels. The propensity score for receiving Med Rec was computed from a logistic model using Med Rec as the outcome. A structured iterative approach was used to refine this model to achieve covariate balance within the matched pairs. Covariate balance was measured by the standardized difference, in which an absolute standardized difference >10% represents meaningful imbalance.²⁵ From the original cohort, we attempted to match patients who had the intervention to patients from the control by means of a matching algorithm using the logit of the propensity score for receiving the intervention.²⁶

Subgroup Analysis

We also examined the impact of the intervention on high-risk patient populations such as those ≥65 years of age, with a LACE index score ≥10, on high-alert medications, and on ≥10 medications. A univariate analysis was conducted to identify patient-related risk predictors that may be independently correlated with a higher risk of hospital visits.

Baseline Characteristics

A total of 8678 patients representing 9931 unique visits met the inclusion criteria for analysis. There were 2541 unique visits (approximately 26% of visits) in the intervention group that received Gold level care and 7390 unique visits in the control group. The patients in the control group were largely patients who received the original standard of care at the institution, Silver level care (67% of the control group). Patients who received Bronze level care or less comprised 33% of the control group.

Patients in the intervention group were significantly older (average of 68 years old versus 64 years old) and on more medications. They also notably had a longer duration of stay in hospital, an increased percentage of visits with a LACE index score ≥10, and were more likely to be discharged home on a high-alert medication and with supports (Table 2).

Main Analysis

The main unadjusted analysis of GIM patients (n = 9931 visits) did not detect a difference in 30-day ED visits and readmissions between the intervention group (540 out of 2541; 21.2%) and control (1423 out of 7390; 19.3%; Table 3). By using a multivariate logistic regression model to account for age, sex, LACE index, and number of medications on discharge, the adjusted odds ratio was 1.06 (95% CI, 0.95-1.19; P = 0.33). After propensity score adjustment, the relative risk of readmission was 0.88 (16.7% vs 18.9%; 95% CI, 0.59-1.32; P = 0.54).

Secondary Analyses

In each predefined high-risk patient subgroup (age ≥65, LACE index score ≥10, number of discharge medications ≥10, and the presence of high-alert medications), analyses of our primary endpoint did not detect significant adjusted odds ratios (Table 4). In our univariate analysis, increasing number of medications, LACE index score, and male gender were independently correlated with a higher risk of hospital visits (supplementary Appendix 2).

Med Rec is widely recommended as a patient safety strategy to prevent clinically significant medication discrepancies at transitions in care.^4-9 However, Med Rec varies widely in terms of what it entails and who delivers it, with the preponderance of evidence suggesting an impact on clinically significant medication discrepancies only when interprofessional care delivered includes a central role for pharmacists.²⁷ Furthermore, Med Rec appears to impact short term readmissions only when embedded in a broader, multifaceted, bundled intervention in which pharmacists or other team members educate patients about their medications and deliver postdischarge follow-up phone calls.^10-13

As very few hospitals have the resources to sustainably deliver intensive care bundles that are represented in RCTs (characterized by Platinum and Diamond levels of care in Table 1), in our observational study, we sought to explore whether a resource-attainable, enhanced Med Rec care bundle (Gold level) had an impact on hospital utilization compared to partial Med Rec care bundles (Bronze and Silver levels). In our findings, we did not detect a significant difference on ED visits and readmissions within 30 days between enhanced and partial care bundles. In a secondary analysis of the influence of the intervention on prespecified high-risk patient subgroups, we also did not detect a difference.

As far as we are aware, our long-term, observational study is the largest to date to explore a real-life, enhanced Med Rec intervention and examine its impact on meaningful patient outcomes. We extrapolated that our intervention group received several critical attributes of a successful bundle as discussed by Mueller in a systematic review.² Our intervention included the following: (1) a systematic BPMH process on admission; (2) integrated admission-to-discharge reconciliation processes; (3) discharge delineation of medication changes since admission; (4) pharmacist involvement in reconciliation from admission to discharge; (5) an electronic platform; and (6) formal discharge reconciliation with interprofessional collaboration. Additional components in the bundle described by Mueller included the following: patient education at discharge, postdischarge communication with the patient, and communication with outpatient providers and medication management.

In our results, we did not find a difference in outcomes between the intervention and control groups. Therefore, it is possible that the enhanced bundle’s focus on interprofessional involvement in discharge reconciliation (Gold care level) has no impact on hospital utilization compared to partial care bundles (Silver and Bronze levels). Kwan et al.³ describe similar findings in their systematic review, in which they evaluated the effects of hospital-based Med Rec on unintentional discrepancies with nontrivial risks for harm to patients on 30-day postdischarge hospital visits. Kwan et al.³ concluded that larger well-designed studies are required to further evaluate this outcome, but authors of current published studies suggest that Med Rec alone probably does not reduce postdischarge hospital utilization within 30 days. Med Rec may have a more significant impact on utilization when bundled with other interventions that improve discharge coordination.³

There may be several reasons why we were unable to detect a significant difference between the intervention and control groups. One limitation is that our nonrandomized, retrospective design may have led to unmeasured confounders that impacted allocation into the intervention group versus the control group. It was notable that patients in the intervention group had an increased age, longer duration of hospital stay, more medications, and high-alert medications on discharge compared to the control group and that may have biased our results towards the null hypothesis. Although the propensity score analysis attempted to adjust for this, it also did not detect a significant difference between groups.

In addition, the existing standard of care during the study period allowed for patients in the control group to receive varying levels of Med Rec. Ideally, we would have compared the intervention to a placebo group that did not receive any Med Rec-related care elements. However, as this was a real-life observational study, the majority of patients received some Med Rec services as a part of the standard of care. As a result, 67% of patients in the control group received Silver level Med Rec with a BPMH, admission reconciliation, and prescriber-only discharge reconciliation. This may have made it more difficult to show an incremental benefit on readmissions between the intervention and control.

Also, our primary outcome of all-cause ED or hospital readmissions within 30 days may not have been sensitive enough to detect the effect of Med Rec interventions alone. Only a small proportion of readmissions within 30 days of discharge are preventable and many patient and community level factors responsible for readmissions cannot be controlled by the hospital’s actions.²⁸ Comprehensive pharmacy interventions have demonstrated decreased hospitalizations and emergency visits at 12 months; however, the largest impact was seen on the more specific outcome of medication-related hospitalizations (80% reduction).²⁹ Lastly, another limitation was that we were unable to capture hospital visits to other centres. However, in our region, almost 75% of readmissions are to the same site as the initial hospitalization.³⁰

Overall, our findings in this study and novel characterization of Med Rec services are relevant to many hospital sites that are striving to implement integrated Med Rec with limited healthcare resources. Although interprofessional Med Rec likely reduces clinically significant medication discrepancies, enhanced interprofessional Med Rec on discharge (Gold Med Rec) alone may not be enough to impact hospital utilization compared to partial Med Rec services (Silver and Bronze Med Rec). Further research into practical, targeted Med Rec bundles on more specific outcomes (such as preventable postdischarge adverse events, “avoidable” hospital readmissions, and medication-related readmissions) may detect a significant benefit.

A long-term observational evaluation of interprofessional Med Rec did not detect a difference in 30-day postdischarge patient hospital visits between patients who received enhanced versus partial Med Rec patient care bundles. Researchers of future prospective studies could focus on evaluating high-risk populations or specific elements of Med Rec services on avoidable medication-related hospital admissions and postdischarge adverse drug events.

Acknowledgments

The authors thank Nita Dhir, MBA.

Presented as a poster and oral presentation at the 2012 American College of Clinical Pharmacy Annual Meeting, Hollywood, Florida, October 21-24, 2012, and as an encore poster presentation at the Canadian Society of Hospital Pharmacists Professional Practice Conference, Toronto, Canada, Feb 3, 2013.

Disclosure

The authors declare no conflicts of interest related to the manuscript submitted. All monies used for the research came from the University Health Network Department of Pharmacy Budget, including the pharmacy residency program.

Healthcare systems are targeting effective strategies to improve patient safety and reduce hospital readmissions. Hospital readmissions can be detrimental to patients’ health, a source of avoidable healthcare costs, and are frequently a reflection of the quality of patient care during transitions of care. Medication reconciliation (Med Rec) was identified as 1 of 12 interventions that may reduce 30-day readmissions; however, rigorously designed studies are scarce.^1,2 Published systematic reviews and meta-analyses have produced mixed conclusions regarding the impact of Med Rec on unplanned 30-day readmissions.^2-4

In several studies, researchers have established the positive impact of Med Rec on reducing patient medication discrepancies and potential adverse drug events.^4-8 Pharmacy-led Med Rec interventions have been shown to easily identify more clinically relevant and higher impact medication discrepancies when compared to usual care.⁸ In a systematic review, Mueller et al.² suggest that there are several interrelated elements that determine if a Med Rec intervention will influence hospital readmissions. These elements form a multicomponent “bundle” of interventions, including a systematic medication history process, admission reconciliation, patient education on discharge, discharge reconciliation, and communication to outpatient providers.⁹ Several prospective randomized controlled studies have demonstrated lower readmission rates and fewer visits to the emergency department (ED) after implementing a comprehensive, interprofessional, bundled intervention (including Med Rec) from admission to discharge.^10-13 A 2016 systematic review and meta-analysis specifically evaluated pharmacy-led Med Rec programs (the majority of which included interventions involving multicomponent bundles) and demonstrated a significant reduction in posthospital healthcare utilization.¹⁴

Although comprehensive, interprofessional, bundled interventions have been shown to reduce readmission rates and ED visits in randomized controlled trials (RCTs), limited resources often prevent hospitals from consistently implementing all aspects of these multicomponent interventions. In practice, clinicians may provide varying components of the bundle, such as the combination of admission medication history by the pharmacist and discharge Med Rec completed by the physician alone. The unique impact of combined pharmacist and prescriber Med Rec interventions from admission to discharge on readmissions remains inconclusive. Further, it is unclear which high-risk patient groups will benefit the most from these interventions. We set out to evaluate the impact of an enhanced, interprofessional Med Rec process from admission to discharge (characterized within the context of a novel taxonomy continuum that specifies clinician involvement and intensity of services) on readmissions to hospital and ED visits within 30 days of discharge.

We conducted a retrospective, observational, analytical cohort study using QuadraMed’s Computerized Patient Record and the EMITT (Electronic Medication Information Transfer Tool)¹⁵ to collect data from 2007 to 2011.

Setting

The study was conducted at a 417-bed tertiary care teaching hospital in Toronto, Ontario, Canada.

Med Rec Process and Description of Exposure (Intervention)

The targeted clinical areas had sustained interprofessional models of patient care in place from admission to discharge. They also were actively using an in-house EMITT to facilitate the documentation and tracking of Med Rec efforts throughout patient admission, transfer, and discharge.¹⁵ On admission, the pharmacist conducted a best possible medication history (BPMH). A BPMH provides the cornerstone for Med Rec. It differs from a routine medication history in that it involves (1) a systematic process for interviewing the patient (or family) and (2) a review of at least one other reliable source of information (eg, a provincial medication database, an inspection of medication vials, or contact with the community pharmacy) to obtain and verify patient medications (prescribed and nonprescribed). The pharmacist recorded the BPMH in the electronic patient record. The application supported admission and discharge Med Rec. On discharge, there were 2 options: (1) the prescriber alone would review and complete the discharge Med Rec and generate electronic prescriptions (Table 1, Silver level care) or (2) the pharmacist would collaborate with the prescriber to complete the discharge reconciliation and the prescriber would electronically generate prescriptions (Table 1, Gold level care). All clinical areas had a combined pharmacist and prescriber Med Rec model in place at admission, but the proportion of patients receiving discharge reconciliation completed by pharmacist and prescriber versus the prescriber-alone varied based on the individual clinician’s practices.

Patient Selection

All consecutive hospitalized patients admitted and discharged by the general internal medicine [GIM] service from March 2007 to December 2011 were included. The GIM service was chosen for the main analysis because they had been performing the intervention for the longest period of time and had the largest population of patients. Patients were identified via their hospital-specific medical record identification number and specific hospital-visit number. Patients were excluded if any of the following occurred: (1) the length of stay of their index admission was less than 24 hours; (2) they died during the visit; (3) they were transferred to a separate acute care inpatient facility; or (4) they left hospital against medical advice. Patient visits were excluded as index cases from the analysis if they were returning within 90 days of a previous discharge.

Outcomes

The primary study outcome was the occurrence of an inpatient readmission or ED visit within 30 days of discharge. In our secondary analyses, we examined the impact of the intervention on high-risk patient populations, such as those ≥65 years of age, with a length of stay, acuity of admission, Charlson comorbidity index, and emergency department visits in past 6 months (LACE) index score ≥10 (see supplementary Appendix 1 for LACE score description), on high-alert medications (1 or more of warfarin, insulin, digoxin, and opioids), and on ≥10 medications.

Data Collection

Identification of Exposure of Interest

We used the electronic database to capture all patients who received pharmacist and prescriber supported admission-to-discharge reconciliation. We explicitly defined increasing intensity of Med Rec care in categories of Bronze, Silver, and Gold care levels (Table 1). The exposed (intervention) group received an enhanced Med Rec bundle (patients receiving Gold level care). The control group was made of patients receiving a partial Med Rec Bundle (patients receiving Silver or Bronze level of care or below).

Determination of Hospital Visits

A search of administrative databases was used to determine if patients admitted to the targeted services had an ED visit or urgent inpatient admission to the study hospital within 30 days.

Statistical Analysis

A logistic regression for outcomes was performed. This yielded an adjusted odds ratio with a 95% confidence interval (CI) between the intervention and control groups. Statistical significance was determined with a 2-sided α level of 0.05. In the analysis, we used Statistical Analysis Software version 9.2.

In our multivariate logistic regression model, we adjusted for confounding factors that might influence the patients’ risk of readmission or the type of Med Rec they received upon discharge. By using administrative databases, patient level demographics, and the Charlson comorbidity index, the most responsible diagnosis and disease burden were collected. Medication-related factors collected included the number of medications on discharge and the presence of predefined high-alert medications. The number of medications on the medication discharge list was determined by using the electronic database. The final adjustment model included age, gender, the number of medications on discharge, and the LACE index score (supplementary Appendix 1). The LACE index score has been validated in Ontario, Canada, populations to quantify the risk of death or unplanned readmission within 30 days of discharge.²⁴

Propensity Score Adjustment

Propensity scoring (probability of treatment assignment conditional on observed baseline characteristics) was planned a priori to account for possible factors that would impact whether a patient received the intervention or control care levels. The propensity score for receiving Med Rec was computed from a logistic model using Med Rec as the outcome. A structured iterative approach was used to refine this model to achieve covariate balance within the matched pairs. Covariate balance was measured by the standardized difference, in which an absolute standardized difference >10% represents meaningful imbalance.²⁵ From the original cohort, we attempted to match patients who had the intervention to patients from the control by means of a matching algorithm using the logit of the propensity score for receiving the intervention.²⁶

Subgroup Analysis

We also examined the impact of the intervention on high-risk patient populations such as those ≥65 years of age, with a LACE index score ≥10, on high-alert medications, and on ≥10 medications. A univariate analysis was conducted to identify patient-related risk predictors that may be independently correlated with a higher risk of hospital visits.

Baseline Characteristics

A total of 8678 patients representing 9931 unique visits met the inclusion criteria for analysis. There were 2541 unique visits (approximately 26% of visits) in the intervention group that received Gold level care and 7390 unique visits in the control group. The patients in the control group were largely patients who received the original standard of care at the institution, Silver level care (67% of the control group). Patients who received Bronze level care or less comprised 33% of the control group.

Patients in the intervention group were significantly older (average of 68 years old versus 64 years old) and on more medications. They also notably had a longer duration of stay in hospital, an increased percentage of visits with a LACE index score ≥10, and were more likely to be discharged home on a high-alert medication and with supports (Table 2).

Main Analysis

The main unadjusted analysis of GIM patients (n = 9931 visits) did not detect a difference in 30-day ED visits and readmissions between the intervention group (540 out of 2541; 21.2%) and control (1423 out of 7390; 19.3%; Table 3). By using a multivariate logistic regression model to account for age, sex, LACE index, and number of medications on discharge, the adjusted odds ratio was 1.06 (95% CI, 0.95-1.19; P = 0.33). After propensity score adjustment, the relative risk of readmission was 0.88 (16.7% vs 18.9%; 95% CI, 0.59-1.32; P = 0.54).

Secondary Analyses

In each predefined high-risk patient subgroup (age ≥65, LACE index score ≥10, number of discharge medications ≥10, and the presence of high-alert medications), analyses of our primary endpoint did not detect significant adjusted odds ratios (Table 4). In our univariate analysis, increasing number of medications, LACE index score, and male gender were independently correlated with a higher risk of hospital visits (supplementary Appendix 2).

Med Rec is widely recommended as a patient safety strategy to prevent clinically significant medication discrepancies at transitions in care.^4-9 However, Med Rec varies widely in terms of what it entails and who delivers it, with the preponderance of evidence suggesting an impact on clinically significant medication discrepancies only when interprofessional care delivered includes a central role for pharmacists.²⁷ Furthermore, Med Rec appears to impact short term readmissions only when embedded in a broader, multifaceted, bundled intervention in which pharmacists or other team members educate patients about their medications and deliver postdischarge follow-up phone calls.^10-13

As very few hospitals have the resources to sustainably deliver intensive care bundles that are represented in RCTs (characterized by Platinum and Diamond levels of care in Table 1), in our observational study, we sought to explore whether a resource-attainable, enhanced Med Rec care bundle (Gold level) had an impact on hospital utilization compared to partial Med Rec care bundles (Bronze and Silver levels). In our findings, we did not detect a significant difference on ED visits and readmissions within 30 days between enhanced and partial care bundles. In a secondary analysis of the influence of the intervention on prespecified high-risk patient subgroups, we also did not detect a difference.

As far as we are aware, our long-term, observational study is the largest to date to explore a real-life, enhanced Med Rec intervention and examine its impact on meaningful patient outcomes. We extrapolated that our intervention group received several critical attributes of a successful bundle as discussed by Mueller in a systematic review.² Our intervention included the following: (1) a systematic BPMH process on admission; (2) integrated admission-to-discharge reconciliation processes; (3) discharge delineation of medication changes since admission; (4) pharmacist involvement in reconciliation from admission to discharge; (5) an electronic platform; and (6) formal discharge reconciliation with interprofessional collaboration. Additional components in the bundle described by Mueller included the following: patient education at discharge, postdischarge communication with the patient, and communication with outpatient providers and medication management.

In our results, we did not find a difference in outcomes between the intervention and control groups. Therefore, it is possible that the enhanced bundle’s focus on interprofessional involvement in discharge reconciliation (Gold care level) has no impact on hospital utilization compared to partial care bundles (Silver and Bronze levels). Kwan et al.³ describe similar findings in their systematic review, in which they evaluated the effects of hospital-based Med Rec on unintentional discrepancies with nontrivial risks for harm to patients on 30-day postdischarge hospital visits. Kwan et al.³ concluded that larger well-designed studies are required to further evaluate this outcome, but authors of current published studies suggest that Med Rec alone probably does not reduce postdischarge hospital utilization within 30 days. Med Rec may have a more significant impact on utilization when bundled with other interventions that improve discharge coordination.³

There may be several reasons why we were unable to detect a significant difference between the intervention and control groups. One limitation is that our nonrandomized, retrospective design may have led to unmeasured confounders that impacted allocation into the intervention group versus the control group. It was notable that patients in the intervention group had an increased age, longer duration of hospital stay, more medications, and high-alert medications on discharge compared to the control group and that may have biased our results towards the null hypothesis. Although the propensity score analysis attempted to adjust for this, it also did not detect a significant difference between groups.

In addition, the existing standard of care during the study period allowed for patients in the control group to receive varying levels of Med Rec. Ideally, we would have compared the intervention to a placebo group that did not receive any Med Rec-related care elements. However, as this was a real-life observational study, the majority of patients received some Med Rec services as a part of the standard of care. As a result, 67% of patients in the control group received Silver level Med Rec with a BPMH, admission reconciliation, and prescriber-only discharge reconciliation. This may have made it more difficult to show an incremental benefit on readmissions between the intervention and control.

Also, our primary outcome of all-cause ED or hospital readmissions within 30 days may not have been sensitive enough to detect the effect of Med Rec interventions alone. Only a small proportion of readmissions within 30 days of discharge are preventable and many patient and community level factors responsible for readmissions cannot be controlled by the hospital’s actions.²⁸ Comprehensive pharmacy interventions have demonstrated decreased hospitalizations and emergency visits at 12 months; however, the largest impact was seen on the more specific outcome of medication-related hospitalizations (80% reduction).²⁹ Lastly, another limitation was that we were unable to capture hospital visits to other centres. However, in our region, almost 75% of readmissions are to the same site as the initial hospitalization.³⁰

Overall, our findings in this study and novel characterization of Med Rec services are relevant to many hospital sites that are striving to implement integrated Med Rec with limited healthcare resources. Although interprofessional Med Rec likely reduces clinically significant medication discrepancies, enhanced interprofessional Med Rec on discharge (Gold Med Rec) alone may not be enough to impact hospital utilization compared to partial Med Rec services (Silver and Bronze Med Rec). Further research into practical, targeted Med Rec bundles on more specific outcomes (such as preventable postdischarge adverse events, “avoidable” hospital readmissions, and medication-related readmissions) may detect a significant benefit.

A long-term observational evaluation of interprofessional Med Rec did not detect a difference in 30-day postdischarge patient hospital visits between patients who received enhanced versus partial Med Rec patient care bundles. Researchers of future prospective studies could focus on evaluating high-risk populations or specific elements of Med Rec services on avoidable medication-related hospital admissions and postdischarge adverse drug events.

Acknowledgments

The authors thank Nita Dhir, MBA.

Presented as a poster and oral presentation at the 2012 American College of Clinical Pharmacy Annual Meeting, Hollywood, Florida, October 21-24, 2012, and as an encore poster presentation at the Canadian Society of Hospital Pharmacists Professional Practice Conference, Toronto, Canada, Feb 3, 2013.

Disclosure

The authors declare no conflicts of interest related to the manuscript submitted. All monies used for the research came from the University Health Network Department of Pharmacy Budget, including the pharmacy residency program.

Let’s face it—rates of hospital admission are on the rise, but there are still just 7 days in a week. That means that patients are increasingly admitted on weekdays and on the weekend, requiring more nurses and doctors to look after them. Why then are there no lines for coffee on a Saturday? Does this reduced intensity of staffing translate into worse care for our patients?

Since one of its earliest descriptions in hospitalized patients, the “weekend effect” has been extensively studied in various patient populations and hospital settings.^1-5 The results have been varied, depending on the place of care,⁶ reason for care, type of admission,^5,7 or admitting diagnosis.^1,8,9 Many researchers have posited the drivers behind the weekend effect, including understaffed wards, intensity of specialist care, delays in procedural treatments, or severity of illness, but the truth is that we still don’t know.

Pauls et al. performed a robust systematic review and meta-analysis examining the rates of in-hospital mortality in patients admitted on the weekend compared with those admitted on weekdays.¹⁰ They analyzed predetermined subgroups to identify system- and patient-level factors associated with a difference in weekend mortality.

A total of 97 studies—comprising an astounding 51 million patients—was included in the study. They found that individuals admitted on the weekend carried an almost 20% increase in the risk of death compared with those who landed in hospital on a weekday. The effect was present for both in-hospital deaths and when looking specifically at 30-day mortality. Translating these findings into practice, an additional 14 deaths per 1000 admissions occur when patients are admitted on the weekend. Brain surgery can be less risky.¹¹

Despite this concerning finding, no individual factor was identified that could account for the effect. There was a 16% and 11% increase in mortality in weekend patients associated with decreased hospital staffing and delays to procedural therapies, respectively. No differences were found when examining reduced rates of procedures or illness severity on weekends compared with weekdays. But one must always interpret subgroup analyses, even prespecified ones, with caution because they often lack the statistical power to make concrete conclusions.

To this end, an important finding of the study by Pauls et al. highlights the variation in mortality risk as it relates to the weekend effect.¹⁰ Even for individuals with cancer, a disease with a relatively predictable rate of decline, there are weekend differences in mortality risk that depend upon the type of cancer.^8,12 This heterogeneity persists when examining for the possible factors that contribute to the effect, introducing a significant amount of noise into the analysis, and may explain why research to date has been unable to find the proverbial black cat in the coal cellar.

One thing Pauls et al. makes clear is that the weekend effect appears to be a real phenomenon, despite significant heterogeneity in the literature.¹⁰ Only a high-quality, systematic review has the capability to draw such conclusions. Prior work demonstrates that this effect is substantial in some individuals,and this study confirms that it perseveres beyond an immediate time period following admission.^1,9 The elements contributing to the weekend effect remain undefined and are likely as complex as our healthcare system itself.

Society and policy makers should resist the tantalizing urge to invoke interventions aimed at fixing this issue before fully understanding the drivers of a system problem. The government of the United Kingdom has decreed a manifesto to create a “7-day National Health Service,” in which weekend services and physician staffing will match that of the weekdays. Considering recent labor tensions between junior doctors in the United Kingdom over pay and working hours, the stakes are at an all-time high.

But such drastic measures violate a primary directive of quality improvement science to study and understand the problem before reflexively jumping to solutions. This will require new research endeavors aimed at determining the underlying factor(s) responsible for the weekend effect. Once we are confident in its cause, only then can careful evaluation of targeted interventions aimed at the highest-risk admissions be instituted. As global hospital and healthcare budgets bend under increasing strain, a critical component of any proposed intervention must be to examine the cost-effectiveness in doing so. Because the weekend effect is one of increased mortality, it will be hard to justify an acceptable price for an individual’s life. And it is not as straightforward as a randomized trial examining the efficacy of parachutes. Any formal evaluation must account for the unintended consequences and opportunity costs of implementing a potential fix aimed at minimizing the weekend effect.

The weekend effect has now been studied for over 15 years. Pauls et al. add to our knowledge of this phenomenon, confirming that the overall risk of mortality for patients admitted on the weekend is real, variable, and substantial.¹⁰ As more individuals are admitted to hospitals, resulting in increasing numbers of admissions on the weekend, a desperate search for the underlying cause must be carried out before we can fix it. Whatever the means to the end, our elation will continue to be tempered by a feeling of uneasiness every time our coworkers joyously exclaim, “TGIF!”

Disclosure

The authors have nothing to disclose.

Let’s face it—rates of hospital admission are on the rise, but there are still just 7 days in a week. That means that patients are increasingly admitted on weekdays and on the weekend, requiring more nurses and doctors to look after them. Why then are there no lines for coffee on a Saturday? Does this reduced intensity of staffing translate into worse care for our patients?

Since one of its earliest descriptions in hospitalized patients, the “weekend effect” has been extensively studied in various patient populations and hospital settings.^1-5 The results have been varied, depending on the place of care,⁶ reason for care, type of admission,^5,7 or admitting diagnosis.^1,8,9 Many researchers have posited the drivers behind the weekend effect, including understaffed wards, intensity of specialist care, delays in procedural treatments, or severity of illness, but the truth is that we still don’t know.

Pauls et al. performed a robust systematic review and meta-analysis examining the rates of in-hospital mortality in patients admitted on the weekend compared with those admitted on weekdays.¹⁰ They analyzed predetermined subgroups to identify system- and patient-level factors associated with a difference in weekend mortality.

A total of 97 studies—comprising an astounding 51 million patients—was included in the study. They found that individuals admitted on the weekend carried an almost 20% increase in the risk of death compared with those who landed in hospital on a weekday. The effect was present for both in-hospital deaths and when looking specifically at 30-day mortality. Translating these findings into practice, an additional 14 deaths per 1000 admissions occur when patients are admitted on the weekend. Brain surgery can be less risky.¹¹

Despite this concerning finding, no individual factor was identified that could account for the effect. There was a 16% and 11% increase in mortality in weekend patients associated with decreased hospital staffing and delays to procedural therapies, respectively. No differences were found when examining reduced rates of procedures or illness severity on weekends compared with weekdays. But one must always interpret subgroup analyses, even prespecified ones, with caution because they often lack the statistical power to make concrete conclusions.

To this end, an important finding of the study by Pauls et al. highlights the variation in mortality risk as it relates to the weekend effect.¹⁰ Even for individuals with cancer, a disease with a relatively predictable rate of decline, there are weekend differences in mortality risk that depend upon the type of cancer.^8,12 This heterogeneity persists when examining for the possible factors that contribute to the effect, introducing a significant amount of noise into the analysis, and may explain why research to date has been unable to find the proverbial black cat in the coal cellar.

One thing Pauls et al. makes clear is that the weekend effect appears to be a real phenomenon, despite significant heterogeneity in the literature.¹⁰ Only a high-quality, systematic review has the capability to draw such conclusions. Prior work demonstrates that this effect is substantial in some individuals,and this study confirms that it perseveres beyond an immediate time period following admission.^1,9 The elements contributing to the weekend effect remain undefined and are likely as complex as our healthcare system itself.

Society and policy makers should resist the tantalizing urge to invoke interventions aimed at fixing this issue before fully understanding the drivers of a system problem. The government of the United Kingdom has decreed a manifesto to create a “7-day National Health Service,” in which weekend services and physician staffing will match that of the weekdays. Considering recent labor tensions between junior doctors in the United Kingdom over pay and working hours, the stakes are at an all-time high.

But such drastic measures violate a primary directive of quality improvement science to study and understand the problem before reflexively jumping to solutions. This will require new research endeavors aimed at determining the underlying factor(s) responsible for the weekend effect. Once we are confident in its cause, only then can careful evaluation of targeted interventions aimed at the highest-risk admissions be instituted. As global hospital and healthcare budgets bend under increasing strain, a critical component of any proposed intervention must be to examine the cost-effectiveness in doing so. Because the weekend effect is one of increased mortality, it will be hard to justify an acceptable price for an individual’s life. And it is not as straightforward as a randomized trial examining the efficacy of parachutes. Any formal evaluation must account for the unintended consequences and opportunity costs of implementing a potential fix aimed at minimizing the weekend effect.

The weekend effect has now been studied for over 15 years. Pauls et al. add to our knowledge of this phenomenon, confirming that the overall risk of mortality for patients admitted on the weekend is real, variable, and substantial.¹⁰ As more individuals are admitted to hospitals, resulting in increasing numbers of admissions on the weekend, a desperate search for the underlying cause must be carried out before we can fix it. Whatever the means to the end, our elation will continue to be tempered by a feeling of uneasiness every time our coworkers joyously exclaim, “TGIF!”

Disclosure

The authors have nothing to disclose.

Let’s face it—rates of hospital admission are on the rise, but there are still just 7 days in a week. That means that patients are increasingly admitted on weekdays and on the weekend, requiring more nurses and doctors to look after them. Why then are there no lines for coffee on a Saturday? Does this reduced intensity of staffing translate into worse care for our patients?

Since one of its earliest descriptions in hospitalized patients, the “weekend effect” has been extensively studied in various patient populations and hospital settings.^1-5 The results have been varied, depending on the place of care,⁶ reason for care, type of admission,^5,7 or admitting diagnosis.^1,8,9 Many researchers have posited the drivers behind the weekend effect, including understaffed wards, intensity of specialist care, delays in procedural treatments, or severity of illness, but the truth is that we still don’t know.

Pauls et al. performed a robust systematic review and meta-analysis examining the rates of in-hospital mortality in patients admitted on the weekend compared with those admitted on weekdays.¹⁰ They analyzed predetermined subgroups to identify system- and patient-level factors associated with a difference in weekend mortality.

A total of 97 studies—comprising an astounding 51 million patients—was included in the study. They found that individuals admitted on the weekend carried an almost 20% increase in the risk of death compared with those who landed in hospital on a weekday. The effect was present for both in-hospital deaths and when looking specifically at 30-day mortality. Translating these findings into practice, an additional 14 deaths per 1000 admissions occur when patients are admitted on the weekend. Brain surgery can be less risky.¹¹

Despite this concerning finding, no individual factor was identified that could account for the effect. There was a 16% and 11% increase in mortality in weekend patients associated with decreased hospital staffing and delays to procedural therapies, respectively. No differences were found when examining reduced rates of procedures or illness severity on weekends compared with weekdays. But one must always interpret subgroup analyses, even prespecified ones, with caution because they often lack the statistical power to make concrete conclusions.

To this end, an important finding of the study by Pauls et al. highlights the variation in mortality risk as it relates to the weekend effect.¹⁰ Even for individuals with cancer, a disease with a relatively predictable rate of decline, there are weekend differences in mortality risk that depend upon the type of cancer.^8,12 This heterogeneity persists when examining for the possible factors that contribute to the effect, introducing a significant amount of noise into the analysis, and may explain why research to date has been unable to find the proverbial black cat in the coal cellar.

One thing Pauls et al. makes clear is that the weekend effect appears to be a real phenomenon, despite significant heterogeneity in the literature.¹⁰ Only a high-quality, systematic review has the capability to draw such conclusions. Prior work demonstrates that this effect is substantial in some individuals,and this study confirms that it perseveres beyond an immediate time period following admission.^1,9 The elements contributing to the weekend effect remain undefined and are likely as complex as our healthcare system itself.

Society and policy makers should resist the tantalizing urge to invoke interventions aimed at fixing this issue before fully understanding the drivers of a system problem. The government of the United Kingdom has decreed a manifesto to create a “7-day National Health Service,” in which weekend services and physician staffing will match that of the weekdays. Considering recent labor tensions between junior doctors in the United Kingdom over pay and working hours, the stakes are at an all-time high.

But such drastic measures violate a primary directive of quality improvement science to study and understand the problem before reflexively jumping to solutions. This will require new research endeavors aimed at determining the underlying factor(s) responsible for the weekend effect. Once we are confident in its cause, only then can careful evaluation of targeted interventions aimed at the highest-risk admissions be instituted. As global hospital and healthcare budgets bend under increasing strain, a critical component of any proposed intervention must be to examine the cost-effectiveness in doing so. Because the weekend effect is one of increased mortality, it will be hard to justify an acceptable price for an individual’s life. And it is not as straightforward as a randomized trial examining the efficacy of parachutes. Any formal evaluation must account for the unintended consequences and opportunity costs of implementing a potential fix aimed at minimizing the weekend effect.

The weekend effect has now been studied for over 15 years. Pauls et al. add to our knowledge of this phenomenon, confirming that the overall risk of mortality for patients admitted on the weekend is real, variable, and substantial.¹⁰ As more individuals are admitted to hospitals, resulting in increasing numbers of admissions on the weekend, a desperate search for the underlying cause must be carried out before we can fix it. Whatever the means to the end, our elation will continue to be tempered by a feeling of uneasiness every time our coworkers joyously exclaim, “TGIF!”

Disclosure

The authors have nothing to disclose.

Patient-centered care, defined by the Institute of Medicine as “health care that establishes a partnership among practitioners, patients, and their families to ensure that decisions respect patients’ wants, needs and preferences and that patients have the education and support they need to make decisions and participate in their own care,” has been recognized as an important factor in improving care transitions after discharge from the hospital.¹ Previous efforts to improve the discharge process for hospitalized patients and reduce avoidable readmissions have focused on improving systems surrounding the patient, such as by increasing the availability of outpatient follow-up or standardizing communication between the inpatient and outpatient care teams.^1,2 In fact, successful programs such as Project BOOST and the Care Transitions Interventions™ provide healthcare institutions with a “bundle” of evidence-based transitional care guidelines for discharge: they provide postdischarge transition coaches, assistance with medication self-management, timely follow-up tips, and improved patient records in order to improve postdischarge outcomes.^3,4 Successful interventions, however, may not provide more services, but also engage the patient in their own care.^5,6 The impact of engaging the patient in his or her own care by providing patient-friendly discharge instructions alone, however, is unknown.

A patient-centered discharge may use tools that were designed with patients, or may involve engaging patients in an interactive process of reviewing discharge instructions and empowering them to manage aspects of their own care after leaving the hospital. This endeavour may lead to more effective use of discharge instructions and reduce the need for additional or more intensive (and costly) interventions. For example, a patient-centered discharge tool could include an educational intervention that uses the “teach-back” method, in which patients are asked to restate in their own words what they thought they heard, or in which staff use additional media or a visual design tool meant to enhance comprehension of discharge instructions.^6,7 Visual aids and the use of larger fonts are particularly useful design elements for improving comprehension among non-English speakers and patients with low health literacy, who tend to have poorer recall of instructions.^8-10 What may constitute essential design elements to include in a discharge instruction tool, however, is not clear.

Moreover, whether the use of discharge tools with a specific focus on patient engagement may improve postdischarge outcomes is not known. Particularly, the ability of patient-centered discharge tools to improve outcomes beyond comprehension such as self-management, adherence to discharge instructions, a reduction in unplanned visits, and a reduction in mortality has not been studied systematically. The objective of this systematic review was to review the literature on discharge instruction tools with a focus on patient engagement and their impact among hospitalized patients.

The Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) Statement was followed as a guideline for reporting throughout this review.¹¹

Data Sources

A literature search was undertaken using the following databases from January 1994 or their inception date to May 2014: Medline, Embase, SIGLE, HTA, Bioethics, ASSIA, Psych Lit, CINAHL, Cochrane Library, EconLit, ERIC, and BioMed Central. We also searched relevant design-focused journals such as Design Issues, Journal of Design Research, Information Design Journal, Innovation, Design Studies, and International Journal of Design, as well as reference lists from studies obtained by electronic searching. The following key words and combination of key words were used with the assistance of a medical librarian: patient discharge, patient-centered discharge, patient-centered design, design thinking, user based design, patient education, discharge summary, education. Additional search terms were added when identified from relevant articles (Appendix).

Inclusion Criteria

We included all English-language studies with patients admitted to the hospital irrespective of age, sex, or medical condition, which included a control group or time period and which measured patient outcomes within 3 months of discharge. The 3-month period after discharge is often cited as a time when outcomes could reasonably be associated with an intervention at discharge.²

Exclusion Criteria

Studies that did not have clear implementation of a patient-centered tool, a control group, or those whose tool was used in the emergency department or as an outpatient were excluded. Studies that included postdischarge tools such as home visits or telephone calls were excluded unless independent effects of the predischarge interventions were measured. Studies with outcomes reported after 3 months were excluded unless outcomes before 3 months were also clearly noted.

All searches were entered into Endnote and duplicates were removed. A 2-stage inclusion process was used. Titles and abstracts of articles were first screened for meeting inclusion and exclusion criteria by 1 reviewer. A second reviewer independently checked a 10% random sample of all the abstracts that met the initial screening criteria. If the agreement to exclude studies was less than 95%, criteria were reviewed before checking the rest of the 90% sample. In the second stage, 2 independent reviewers examined paper copies of the full articles selected in the first stage. Disagreement between reviewers was resolved by discussion or a third reviewer if no agreement could be reached.

Data Analysis and Synthesis

The following information was extracted from the full reference: type of study, population studied, control group or time period, tool used, and outcomes measured. Based on the National Health Care Quality report’s priorities and goals on patient and/or family engagement during transitions of care, educational tools were further described based on method of teaching, involvement of the care team, involvement of the patient in the design or delivery of the tool, and/or the use of visual aids.¹² All primary outcomes were classified according to 3 categories: improved knowledge/comprehension, patient experience (patient satisfaction, self-management/efficacy such as functional status, both physical and mental), and health outcomes (unscheduled visits or readmissions, adherence with medications, diet, exercise, or follow-up, and mortality).

No quantitative pooling of results or meta-analysis was done given the variability and heterogeneity of studies reviewed. However, following guidelines for Effect Practice and Organisation of Care (EPOC) Risk of Bias criteria,¹³ studies that had a higher risk of bias such as uncontrolled before-after studies or studies with only 1 intervention or control site (historical controls, eg) were excluded from the final review because of the difficulties in attributing causation. Only primary outcomes were reported in order to minimize type II errors.

Our search revealed a total of 3699 studies after duplicates had been removed (Figure). A total of 714 references were included after initial review by title and abstract and 30 studies after full-text review. Agreement on a 10% random sample of all abstracts and full text was 79% (k=0.58) and 86% (k=0.72), respectively. Discussion was needed for fewer than 100 references, and agreement was subsequently reached for 100%.

There were 22 randomized controlled trials and 8 nonrandomized studies (5 nonrandomized controlled trials and 3 controlled before-after studies). Most of these studies were conducted in the United States (13/30 studies), followed by other European countries (5 studies), and the United Kingdom (4 studies). A large number of studies were conducted among patients with cardiovascular disease or risk factors (10 studies), followed by postsurgical patients such as coronary artery bypass graft surgery or orthopaedic surgery (5 studies). Five of 30 studies were conducted among individuals older than 65 years. Most studies excluded patients who did not speak English or the country’s official language; only 3 studies included patients with limited literacy, patients who spoke other languages, or caregivers if the patients could not communicate.

Most studies tested the impact of educational discharge interventions (28 of 30 studies) (Table 1). Quite often, it was a member of the research team who carried out the patient education. Only 3 studies involved multiple members of the care team in designing or reviewing the discharge tool with the patient. Almost half (12 studies) targeted multiple aspects of postdischarge care, including medications and side effects, signs and symptoms to consider, plans for follow-up, dietary restrictions, and/or exercise modifications. Many (19 studies) provided education using one-on-one teaching in association with a discharge tool, accompanied by a written handout (13 studies), audiotape (2 studies), or video (3 studies). While 13 studies had patients involved in creating what content was discussed and 14 studies had patients involved in the delivery of the tool, only 6 studies had patients involved in both design and delivery of the tool. Nine studies also used visual aids such as pictures, larger font, or use of a tool enhanced for patients with language barriers or limited health literacy.

Our systematic review found 30 studies that engaged patients during the design or the delivery of a discharge instruction tool and that tested the effect of the tool on postdischarge outcomes.^6-10,14–38 Our review suggests that there is sufficient evidence that patient-centered discharge tools improve comprehension. However, evidence is currently insufficient to determine if patient-centered tools improve adherence with discharge instructions. Moreover, though limited studies show promising results, more studies are needed to determine if patient engagement improves self-efficacy and healthcare utilization after discharge.

A major limitation of current studies is the variability in the level of patient engagement in tool design or delivery. Patients were involved in the design mostly through targeted development of a discharge management plan and the delivery by encouraging them to ask questions. Few studies involved patients in the design of the tool such that patients were responsible for coming up with content that was of interest to them. The few that did, often with the additional use of video media, demonstrated significant outcomes. Only a minority of studies used an interactive process to assess understanding such as “teach-back” or maximize patient comprehension such as visual aids. Even fewer studies engaged patients in both developing the discharge tool and providing discharge instructions.

Several previous studies have demonstrated that most complications after discharge are the result of ineffective communication, which can be exacerbated by lack of fluency in English or by limited health literacy.^2,39-43 As a result, poor understanding of discharge instructions by patients and their caregivers can create an important care gap.⁴⁴ Therefore, the use of patient-centered tools to engage patients at discharge in their own care is needed. How to engage patients consistently and effectively is perhaps less evident, as demonstrated in this review of the literature in which different levels of patient engagement were found. Many of the tools tested placed attention on patient education, sometimes in the context of bundled care along with home visits or follow-up, all of which can require extensive resources and time. Providing patients with information that the patients themselves state is of value may be the easiest refinement to a discharge educational tool, although this was surprisingly uncommon.^{6,9,10,17,23,33,37} Only 2 studies were found that engaged patients in the initial stage of design of the discharge tool, by incorporating information of interest to them.^23,32 For example, a study testing the impact of a computer-generated written education package on poststroke outcomes designed the information by asking patients to identify which topics they would like to receive information about (along with the amount of information and font size).²³ Secondly, although most of the discharge tools reviewed included the use of one-on-one teaching and the use of media such as patient handouts, these tools were often used in such a way that patients were passive recipients. In fact, studies that used additional video media that incorporated personalized content were the most likely to demonstrate positive outcomes.^17,34 The next level of patient engagement may therefore be to involve the patient as an interactive partner when delivering the tool in order to empower patients to self-care. For example, 1 study designed a structured education program by first assessing lifestyle risk factors related to hypertension that were modifiable along with preconceived notions through open-ended questions during a one-on-one interview.³⁷ Patients were subsequently educated on any knowledge deficits regarding the management of their lifestyle. Another level of patient engagement may be to use visual aids during discussions, as a well-known complement to verbal instructions.^45,46 For example, in a controlled study that randomized a ward of elderly patients with 4 or more prescriptions to predischarge counseling, the counseling session aimed to review reasons for their prescriptions along with corresponding side effects, doses, and dosage times with the help of a medicine reminder card. Other uses of visual aid tools identified in our review included the use of pictograms or illustrations or, at minimum, attention to font size.^{7,8,16,29,33,35} In the absence of a visual aid, asking the patient to repeat or demonstrate what was just communicated can be used to assess the amount of information retained.^18,33

An important result discovered in our review of the literature was also the lack of studies that tested the impact of discharge tools on usability of discharge information once at home. Conducting an evaluation of the benefits to patients after discharge can help objectify vague outcomes like health gains or qualify benefits in patient’s views. This might also explain why many studies with documented patient engagement at the time of discharge were able to demonstrate improvements in comprehension but not adherence to instructions. Although patients and caregivers may understand the information, this comprehension does not necessarily mean they will find the information useful or adhere to it once at home. For example, in 1 study, patients discharged with at least 1 medication were randomized to a structured discharge interview during which the treatment plan was reviewed verbally and questions clarified along with a visually enhanced treatment card.²⁶ Although knowledge of medications increased, no effect was found on adherence at 1 week postdischarge. However, use of the treatment card at home was not assessed. Similarly, another study tested the effect of an individualized video of exercises and failed to find a difference in patient adherence at 4 weeks.²⁸ The authors suggested that the lack of benefit may have been because patients were not using the video once at home. This is in contrast to 2 studies that involved patients in their own care by requiring them to request their medication as part of a self-medication tool predischarge.^16,30 Patients were engaged in the process such that increasing independence was given to patients based on their demonstration of understanding and adherence to their treatment while still in the hospital, a learning tool that can be applied once at home. Feeling knowledgeable and involved, as others have suggested, may be the intermediary outcomes that led to improved adherence.⁴⁷ It is also possible that adherence to discharge instructions may vary based on complexity of the information provided, such that instructions focusing solely on medication use may require less patient engagement than discharge instructions that include information on medications, diet, exercise modifications, and follow-up.⁴⁸

Our review has a few limitations. Previous systematic reviews have demonstrated that bundled discharge interventions that include patient-centered education have a positive effect on outcomes postdischarge.^2,5 However, we sought to describe and study the individual and distinct impact of patient engagement in the creation and delivery of discharge tools on outcomes postdischarge. We hoped that this may provide others with key information regarding elements of patient engagement that were particularly useful when designing a new discharge tool. The variability of the studies we identified, however, made it difficult to ascertain what level of patient engagement is required to observe improvements in health outcomes. It is also possible that a higher level of patient engagement may have been used but not described in the studies we reviewed. As only primary outcomes were included, we may have underestimated the effect of patient-centered discharge tools on outcomes that were reported as secondary outcomes. As we were interested in reviewing as many studies of patient-centered discharge tools as possible, we did not assess the quality of the studies and cannot comment on the role of bias in these studies. However, we excluded studies with study designs known to have the highest risk of bias. Lastly, we also cannot comment on whether patient-centered tools may have an effect on outcomes more than 3 months after a hospital discharge. However, several studies included in this review suggest a sustained effect beyond this time period.^8,25,32,37

Patient-centered discharge tools in which patients were engaged in the design or the delivery were found to improve comprehension of but not adherence with discharge instructions. The perceived lack of improved adherence may be due to a lack of studies that measured the usefulness and utilization of information for patients once at home. There was also substantial variability in the extent of patient involvement in designing the style and content of information provided to patients at discharge, as well as the extent of patient engagement when receiving discharge instructions. Future studies would benefit from detailing the level of patient engagement needed in designing and delivery of discharge tools. This information may lead to the discovery of barriers and facilitators to utilization of discharge information once at home and lead to a better understanding of the patient’s journey from hospital to home and onwards.

C.M.B. and this work were funded by a CIHR Canadian Patient Safety Institute Chair in Patient Safety and Continuity of Care. Funding was provided to cover fees to obtain articles from the Donald J. Matthews Complex Care Fund of the University Health Network in Toronto, Canada. The Toronto Central Local Health Integration Network provided funding for the design and implementation of a patient-oriented discharge summary. None of the funding or supportive agencies were involved in the design or conduct of the present study, analysis, or interpretation of the data, or approval of the manuscript.

Disclosures

The authors report no conflicts of interest.

Patient-centered care, defined by the Institute of Medicine as “health care that establishes a partnership among practitioners, patients, and their families to ensure that decisions respect patients’ wants, needs and preferences and that patients have the education and support they need to make decisions and participate in their own care,” has been recognized as an important factor in improving care transitions after discharge from the hospital.¹ Previous efforts to improve the discharge process for hospitalized patients and reduce avoidable readmissions have focused on improving systems surrounding the patient, such as by increasing the availability of outpatient follow-up or standardizing communication between the inpatient and outpatient care teams.^1,2 In fact, successful programs such as Project BOOST and the Care Transitions Interventions™ provide healthcare institutions with a “bundle” of evidence-based transitional care guidelines for discharge: they provide postdischarge transition coaches, assistance with medication self-management, timely follow-up tips, and improved patient records in order to improve postdischarge outcomes.^3,4 Successful interventions, however, may not provide more services, but also engage the patient in their own care.^5,6 The impact of engaging the patient in his or her own care by providing patient-friendly discharge instructions alone, however, is unknown.

A patient-centered discharge may use tools that were designed with patients, or may involve engaging patients in an interactive process of reviewing discharge instructions and empowering them to manage aspects of their own care after leaving the hospital. This endeavour may lead to more effective use of discharge instructions and reduce the need for additional or more intensive (and costly) interventions. For example, a patient-centered discharge tool could include an educational intervention that uses the “teach-back” method, in which patients are asked to restate in their own words what they thought they heard, or in which staff use additional media or a visual design tool meant to enhance comprehension of discharge instructions.^6,7 Visual aids and the use of larger fonts are particularly useful design elements for improving comprehension among non-English speakers and patients with low health literacy, who tend to have poorer recall of instructions.^8-10 What may constitute essential design elements to include in a discharge instruction tool, however, is not clear.

Moreover, whether the use of discharge tools with a specific focus on patient engagement may improve postdischarge outcomes is not known. Particularly, the ability of patient-centered discharge tools to improve outcomes beyond comprehension such as self-management, adherence to discharge instructions, a reduction in unplanned visits, and a reduction in mortality has not been studied systematically. The objective of this systematic review was to review the literature on discharge instruction tools with a focus on patient engagement and their impact among hospitalized patients.

The Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) Statement was followed as a guideline for reporting throughout this review.¹¹

Data Sources

A literature search was undertaken using the following databases from January 1994 or their inception date to May 2014: Medline, Embase, SIGLE, HTA, Bioethics, ASSIA, Psych Lit, CINAHL, Cochrane Library, EconLit, ERIC, and BioMed Central. We also searched relevant design-focused journals such as Design Issues, Journal of Design Research, Information Design Journal, Innovation, Design Studies, and International Journal of Design, as well as reference lists from studies obtained by electronic searching. The following key words and combination of key words were used with the assistance of a medical librarian: patient discharge, patient-centered discharge, patient-centered design, design thinking, user based design, patient education, discharge summary, education. Additional search terms were added when identified from relevant articles (Appendix).

Inclusion Criteria

We included all English-language studies with patients admitted to the hospital irrespective of age, sex, or medical condition, which included a control group or time period and which measured patient outcomes within 3 months of discharge. The 3-month period after discharge is often cited as a time when outcomes could reasonably be associated with an intervention at discharge.²

Exclusion Criteria

Studies that did not have clear implementation of a patient-centered tool, a control group, or those whose tool was used in the emergency department or as an outpatient were excluded. Studies that included postdischarge tools such as home visits or telephone calls were excluded unless independent effects of the predischarge interventions were measured. Studies with outcomes reported after 3 months were excluded unless outcomes before 3 months were also clearly noted.

All searches were entered into Endnote and duplicates were removed. A 2-stage inclusion process was used. Titles and abstracts of articles were first screened for meeting inclusion and exclusion criteria by 1 reviewer. A second reviewer independently checked a 10% random sample of all the abstracts that met the initial screening criteria. If the agreement to exclude studies was less than 95%, criteria were reviewed before checking the rest of the 90% sample. In the second stage, 2 independent reviewers examined paper copies of the full articles selected in the first stage. Disagreement between reviewers was resolved by discussion or a third reviewer if no agreement could be reached.

Data Analysis and Synthesis

The following information was extracted from the full reference: type of study, population studied, control group or time period, tool used, and outcomes measured. Based on the National Health Care Quality report’s priorities and goals on patient and/or family engagement during transitions of care, educational tools were further described based on method of teaching, involvement of the care team, involvement of the patient in the design or delivery of the tool, and/or the use of visual aids.¹² All primary outcomes were classified according to 3 categories: improved knowledge/comprehension, patient experience (patient satisfaction, self-management/efficacy such as functional status, both physical and mental), and health outcomes (unscheduled visits or readmissions, adherence with medications, diet, exercise, or follow-up, and mortality).

No quantitative pooling of results or meta-analysis was done given the variability and heterogeneity of studies reviewed. However, following guidelines for Effect Practice and Organisation of Care (EPOC) Risk of Bias criteria,¹³ studies that had a higher risk of bias such as uncontrolled before-after studies or studies with only 1 intervention or control site (historical controls, eg) were excluded from the final review because of the difficulties in attributing causation. Only primary outcomes were reported in order to minimize type II errors.

Our search revealed a total of 3699 studies after duplicates had been removed (Figure). A total of 714 references were included after initial review by title and abstract and 30 studies after full-text review. Agreement on a 10% random sample of all abstracts and full text was 79% (k=0.58) and 86% (k=0.72), respectively. Discussion was needed for fewer than 100 references, and agreement was subsequently reached for 100%.

There were 22 randomized controlled trials and 8 nonrandomized studies (5 nonrandomized controlled trials and 3 controlled before-after studies). Most of these studies were conducted in the United States (13/30 studies), followed by other European countries (5 studies), and the United Kingdom (4 studies). A large number of studies were conducted among patients with cardiovascular disease or risk factors (10 studies), followed by postsurgical patients such as coronary artery bypass graft surgery or orthopaedic surgery (5 studies). Five of 30 studies were conducted among individuals older than 65 years. Most studies excluded patients who did not speak English or the country’s official language; only 3 studies included patients with limited literacy, patients who spoke other languages, or caregivers if the patients could not communicate.

Most studies tested the impact of educational discharge interventions (28 of 30 studies) (Table 1). Quite often, it was a member of the research team who carried out the patient education. Only 3 studies involved multiple members of the care team in designing or reviewing the discharge tool with the patient. Almost half (12 studies) targeted multiple aspects of postdischarge care, including medications and side effects, signs and symptoms to consider, plans for follow-up, dietary restrictions, and/or exercise modifications. Many (19 studies) provided education using one-on-one teaching in association with a discharge tool, accompanied by a written handout (13 studies), audiotape (2 studies), or video (3 studies). While 13 studies had patients involved in creating what content was discussed and 14 studies had patients involved in the delivery of the tool, only 6 studies had patients involved in both design and delivery of the tool. Nine studies also used visual aids such as pictures, larger font, or use of a tool enhanced for patients with language barriers or limited health literacy.

Our systematic review found 30 studies that engaged patients during the design or the delivery of a discharge instruction tool and that tested the effect of the tool on postdischarge outcomes.^6-10,14–38 Our review suggests that there is sufficient evidence that patient-centered discharge tools improve comprehension. However, evidence is currently insufficient to determine if patient-centered tools improve adherence with discharge instructions. Moreover, though limited studies show promising results, more studies are needed to determine if patient engagement improves self-efficacy and healthcare utilization after discharge.

A major limitation of current studies is the variability in the level of patient engagement in tool design or delivery. Patients were involved in the design mostly through targeted development of a discharge management plan and the delivery by encouraging them to ask questions. Few studies involved patients in the design of the tool such that patients were responsible for coming up with content that was of interest to them. The few that did, often with the additional use of video media, demonstrated significant outcomes. Only a minority of studies used an interactive process to assess understanding such as “teach-back” or maximize patient comprehension such as visual aids. Even fewer studies engaged patients in both developing the discharge tool and providing discharge instructions.

Several previous studies have demonstrated that most complications after discharge are the result of ineffective communication, which can be exacerbated by lack of fluency in English or by limited health literacy.^2,39-43 As a result, poor understanding of discharge instructions by patients and their caregivers can create an important care gap.⁴⁴ Therefore, the use of patient-centered tools to engage patients at discharge in their own care is needed. How to engage patients consistently and effectively is perhaps less evident, as demonstrated in this review of the literature in which different levels of patient engagement were found. Many of the tools tested placed attention on patient education, sometimes in the context of bundled care along with home visits or follow-up, all of which can require extensive resources and time. Providing patients with information that the patients themselves state is of value may be the easiest refinement to a discharge educational tool, although this was surprisingly uncommon.^{6,9,10,17,23,33,37} Only 2 studies were found that engaged patients in the initial stage of design of the discharge tool, by incorporating information of interest to them.^23,32 For example, a study testing the impact of a computer-generated written education package on poststroke outcomes designed the information by asking patients to identify which topics they would like to receive information about (along with the amount of information and font size).²³ Secondly, although most of the discharge tools reviewed included the use of one-on-one teaching and the use of media such as patient handouts, these tools were often used in such a way that patients were passive recipients. In fact, studies that used additional video media that incorporated personalized content were the most likely to demonstrate positive outcomes.^17,34 The next level of patient engagement may therefore be to involve the patient as an interactive partner when delivering the tool in order to empower patients to self-care. For example, 1 study designed a structured education program by first assessing lifestyle risk factors related to hypertension that were modifiable along with preconceived notions through open-ended questions during a one-on-one interview.³⁷ Patients were subsequently educated on any knowledge deficits regarding the management of their lifestyle. Another level of patient engagement may be to use visual aids during discussions, as a well-known complement to verbal instructions.^45,46 For example, in a controlled study that randomized a ward of elderly patients with 4 or more prescriptions to predischarge counseling, the counseling session aimed to review reasons for their prescriptions along with corresponding side effects, doses, and dosage times with the help of a medicine reminder card. Other uses of visual aid tools identified in our review included the use of pictograms or illustrations or, at minimum, attention to font size.^{7,8,16,29,33,35} In the absence of a visual aid, asking the patient to repeat or demonstrate what was just communicated can be used to assess the amount of information retained.^18,33

An important result discovered in our review of the literature was also the lack of studies that tested the impact of discharge tools on usability of discharge information once at home. Conducting an evaluation of the benefits to patients after discharge can help objectify vague outcomes like health gains or qualify benefits in patient’s views. This might also explain why many studies with documented patient engagement at the time of discharge were able to demonstrate improvements in comprehension but not adherence to instructions. Although patients and caregivers may understand the information, this comprehension does not necessarily mean they will find the information useful or adhere to it once at home. For example, in 1 study, patients discharged with at least 1 medication were randomized to a structured discharge interview during which the treatment plan was reviewed verbally and questions clarified along with a visually enhanced treatment card.²⁶ Although knowledge of medications increased, no effect was found on adherence at 1 week postdischarge. However, use of the treatment card at home was not assessed. Similarly, another study tested the effect of an individualized video of exercises and failed to find a difference in patient adherence at 4 weeks.²⁸ The authors suggested that the lack of benefit may have been because patients were not using the video once at home. This is in contrast to 2 studies that involved patients in their own care by requiring them to request their medication as part of a self-medication tool predischarge.^16,30 Patients were engaged in the process such that increasing independence was given to patients based on their demonstration of understanding and adherence to their treatment while still in the hospital, a learning tool that can be applied once at home. Feeling knowledgeable and involved, as others have suggested, may be the intermediary outcomes that led to improved adherence.⁴⁷ It is also possible that adherence to discharge instructions may vary based on complexity of the information provided, such that instructions focusing solely on medication use may require less patient engagement than discharge instructions that include information on medications, diet, exercise modifications, and follow-up.⁴⁸

Our review has a few limitations. Previous systematic reviews have demonstrated that bundled discharge interventions that include patient-centered education have a positive effect on outcomes postdischarge.^2,5 However, we sought to describe and study the individual and distinct impact of patient engagement in the creation and delivery of discharge tools on outcomes postdischarge. We hoped that this may provide others with key information regarding elements of patient engagement that were particularly useful when designing a new discharge tool. The variability of the studies we identified, however, made it difficult to ascertain what level of patient engagement is required to observe improvements in health outcomes. It is also possible that a higher level of patient engagement may have been used but not described in the studies we reviewed. As only primary outcomes were included, we may have underestimated the effect of patient-centered discharge tools on outcomes that were reported as secondary outcomes. As we were interested in reviewing as many studies of patient-centered discharge tools as possible, we did not assess the quality of the studies and cannot comment on the role of bias in these studies. However, we excluded studies with study designs known to have the highest risk of bias. Lastly, we also cannot comment on whether patient-centered tools may have an effect on outcomes more than 3 months after a hospital discharge. However, several studies included in this review suggest a sustained effect beyond this time period.^8,25,32,37

Patient-centered discharge tools in which patients were engaged in the design or the delivery were found to improve comprehension of but not adherence with discharge instructions. The perceived lack of improved adherence may be due to a lack of studies that measured the usefulness and utilization of information for patients once at home. There was also substantial variability in the extent of patient involvement in designing the style and content of information provided to patients at discharge, as well as the extent of patient engagement when receiving discharge instructions. Future studies would benefit from detailing the level of patient engagement needed in designing and delivery of discharge tools. This information may lead to the discovery of barriers and facilitators to utilization of discharge information once at home and lead to a better understanding of the patient’s journey from hospital to home and onwards.

C.M.B. and this work were funded by a CIHR Canadian Patient Safety Institute Chair in Patient Safety and Continuity of Care. Funding was provided to cover fees to obtain articles from the Donald J. Matthews Complex Care Fund of the University Health Network in Toronto, Canada. The Toronto Central Local Health Integration Network provided funding for the design and implementation of a patient-oriented discharge summary. None of the funding or supportive agencies were involved in the design or conduct of the present study, analysis, or interpretation of the data, or approval of the manuscript.

Disclosures

The authors report no conflicts of interest.

Patient-centered care, defined by the Institute of Medicine as “health care that establishes a partnership among practitioners, patients, and their families to ensure that decisions respect patients’ wants, needs and preferences and that patients have the education and support they need to make decisions and participate in their own care,” has been recognized as an important factor in improving care transitions after discharge from the hospital.¹ Previous efforts to improve the discharge process for hospitalized patients and reduce avoidable readmissions have focused on improving systems surrounding the patient, such as by increasing the availability of outpatient follow-up or standardizing communication between the inpatient and outpatient care teams.^1,2 In fact, successful programs such as Project BOOST and the Care Transitions Interventions™ provide healthcare institutions with a “bundle” of evidence-based transitional care guidelines for discharge: they provide postdischarge transition coaches, assistance with medication self-management, timely follow-up tips, and improved patient records in order to improve postdischarge outcomes.^3,4 Successful interventions, however, may not provide more services, but also engage the patient in their own care.^5,6 The impact of engaging the patient in his or her own care by providing patient-friendly discharge instructions alone, however, is unknown.

A patient-centered discharge may use tools that were designed with patients, or may involve engaging patients in an interactive process of reviewing discharge instructions and empowering them to manage aspects of their own care after leaving the hospital. This endeavour may lead to more effective use of discharge instructions and reduce the need for additional or more intensive (and costly) interventions. For example, a patient-centered discharge tool could include an educational intervention that uses the “teach-back” method, in which patients are asked to restate in their own words what they thought they heard, or in which staff use additional media or a visual design tool meant to enhance comprehension of discharge instructions.^6,7 Visual aids and the use of larger fonts are particularly useful design elements for improving comprehension among non-English speakers and patients with low health literacy, who tend to have poorer recall of instructions.^8-10 What may constitute essential design elements to include in a discharge instruction tool, however, is not clear.

Moreover, whether the use of discharge tools with a specific focus on patient engagement may improve postdischarge outcomes is not known. Particularly, the ability of patient-centered discharge tools to improve outcomes beyond comprehension such as self-management, adherence to discharge instructions, a reduction in unplanned visits, and a reduction in mortality has not been studied systematically. The objective of this systematic review was to review the literature on discharge instruction tools with a focus on patient engagement and their impact among hospitalized patients.

The Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) Statement was followed as a guideline for reporting throughout this review.¹¹

Data Sources

A literature search was undertaken using the following databases from January 1994 or their inception date to May 2014: Medline, Embase, SIGLE, HTA, Bioethics, ASSIA, Psych Lit, CINAHL, Cochrane Library, EconLit, ERIC, and BioMed Central. We also searched relevant design-focused journals such as Design Issues, Journal of Design Research, Information Design Journal, Innovation, Design Studies, and International Journal of Design, as well as reference lists from studies obtained by electronic searching. The following key words and combination of key words were used with the assistance of a medical librarian: patient discharge, patient-centered discharge, patient-centered design, design thinking, user based design, patient education, discharge summary, education. Additional search terms were added when identified from relevant articles (Appendix).

Inclusion Criteria

We included all English-language studies with patients admitted to the hospital irrespective of age, sex, or medical condition, which included a control group or time period and which measured patient outcomes within 3 months of discharge. The 3-month period after discharge is often cited as a time when outcomes could reasonably be associated with an intervention at discharge.²

Exclusion Criteria

Studies that did not have clear implementation of a patient-centered tool, a control group, or those whose tool was used in the emergency department or as an outpatient were excluded. Studies that included postdischarge tools such as home visits or telephone calls were excluded unless independent effects of the predischarge interventions were measured. Studies with outcomes reported after 3 months were excluded unless outcomes before 3 months were also clearly noted.

All searches were entered into Endnote and duplicates were removed. A 2-stage inclusion process was used. Titles and abstracts of articles were first screened for meeting inclusion and exclusion criteria by 1 reviewer. A second reviewer independently checked a 10% random sample of all the abstracts that met the initial screening criteria. If the agreement to exclude studies was less than 95%, criteria were reviewed before checking the rest of the 90% sample. In the second stage, 2 independent reviewers examined paper copies of the full articles selected in the first stage. Disagreement between reviewers was resolved by discussion or a third reviewer if no agreement could be reached.

Data Analysis and Synthesis

The following information was extracted from the full reference: type of study, population studied, control group or time period, tool used, and outcomes measured. Based on the National Health Care Quality report’s priorities and goals on patient and/or family engagement during transitions of care, educational tools were further described based on method of teaching, involvement of the care team, involvement of the patient in the design or delivery of the tool, and/or the use of visual aids.¹² All primary outcomes were classified according to 3 categories: improved knowledge/comprehension, patient experience (patient satisfaction, self-management/efficacy such as functional status, both physical and mental), and health outcomes (unscheduled visits or readmissions, adherence with medications, diet, exercise, or follow-up, and mortality).

No quantitative pooling of results or meta-analysis was done given the variability and heterogeneity of studies reviewed. However, following guidelines for Effect Practice and Organisation of Care (EPOC) Risk of Bias criteria,¹³ studies that had a higher risk of bias such as uncontrolled before-after studies or studies with only 1 intervention or control site (historical controls, eg) were excluded from the final review because of the difficulties in attributing causation. Only primary outcomes were reported in order to minimize type II errors.

Our search revealed a total of 3699 studies after duplicates had been removed (Figure). A total of 714 references were included after initial review by title and abstract and 30 studies after full-text review. Agreement on a 10% random sample of all abstracts and full text was 79% (k=0.58) and 86% (k=0.72), respectively. Discussion was needed for fewer than 100 references, and agreement was subsequently reached for 100%.

There were 22 randomized controlled trials and 8 nonrandomized studies (5 nonrandomized controlled trials and 3 controlled before-after studies). Most of these studies were conducted in the United States (13/30 studies), followed by other European countries (5 studies), and the United Kingdom (4 studies). A large number of studies were conducted among patients with cardiovascular disease or risk factors (10 studies), followed by postsurgical patients such as coronary artery bypass graft surgery or orthopaedic surgery (5 studies). Five of 30 studies were conducted among individuals older than 65 years. Most studies excluded patients who did not speak English or the country’s official language; only 3 studies included patients with limited literacy, patients who spoke other languages, or caregivers if the patients could not communicate.

Most studies tested the impact of educational discharge interventions (28 of 30 studies) (Table 1). Quite often, it was a member of the research team who carried out the patient education. Only 3 studies involved multiple members of the care team in designing or reviewing the discharge tool with the patient. Almost half (12 studies) targeted multiple aspects of postdischarge care, including medications and side effects, signs and symptoms to consider, plans for follow-up, dietary restrictions, and/or exercise modifications. Many (19 studies) provided education using one-on-one teaching in association with a discharge tool, accompanied by a written handout (13 studies), audiotape (2 studies), or video (3 studies). While 13 studies had patients involved in creating what content was discussed and 14 studies had patients involved in the delivery of the tool, only 6 studies had patients involved in both design and delivery of the tool. Nine studies also used visual aids such as pictures, larger font, or use of a tool enhanced for patients with language barriers or limited health literacy.

Our systematic review found 30 studies that engaged patients during the design or the delivery of a discharge instruction tool and that tested the effect of the tool on postdischarge outcomes.^6-10,14–38 Our review suggests that there is sufficient evidence that patient-centered discharge tools improve comprehension. However, evidence is currently insufficient to determine if patient-centered tools improve adherence with discharge instructions. Moreover, though limited studies show promising results, more studies are needed to determine if patient engagement improves self-efficacy and healthcare utilization after discharge.

A major limitation of current studies is the variability in the level of patient engagement in tool design or delivery. Patients were involved in the design mostly through targeted development of a discharge management plan and the delivery by encouraging them to ask questions. Few studies involved patients in the design of the tool such that patients were responsible for coming up with content that was of interest to them. The few that did, often with the additional use of video media, demonstrated significant outcomes. Only a minority of studies used an interactive process to assess understanding such as “teach-back” or maximize patient comprehension such as visual aids. Even fewer studies engaged patients in both developing the discharge tool and providing discharge instructions.

Several previous studies have demonstrated that most complications after discharge are the result of ineffective communication, which can be exacerbated by lack of fluency in English or by limited health literacy.^2,39-43 As a result, poor understanding of discharge instructions by patients and their caregivers can create an important care gap.⁴⁴ Therefore, the use of patient-centered tools to engage patients at discharge in their own care is needed. How to engage patients consistently and effectively is perhaps less evident, as demonstrated in this review of the literature in which different levels of patient engagement were found. Many of the tools tested placed attention on patient education, sometimes in the context of bundled care along with home visits or follow-up, all of which can require extensive resources and time. Providing patients with information that the patients themselves state is of value may be the easiest refinement to a discharge educational tool, although this was surprisingly uncommon.^{6,9,10,17,23,33,37} Only 2 studies were found that engaged patients in the initial stage of design of the discharge tool, by incorporating information of interest to them.^23,32 For example, a study testing the impact of a computer-generated written education package on poststroke outcomes designed the information by asking patients to identify which topics they would like to receive information about (along with the amount of information and font size).²³ Secondly, although most of the discharge tools reviewed included the use of one-on-one teaching and the use of media such as patient handouts, these tools were often used in such a way that patients were passive recipients. In fact, studies that used additional video media that incorporated personalized content were the most likely to demonstrate positive outcomes.^17,34 The next level of patient engagement may therefore be to involve the patient as an interactive partner when delivering the tool in order to empower patients to self-care. For example, 1 study designed a structured education program by first assessing lifestyle risk factors related to hypertension that were modifiable along with preconceived notions through open-ended questions during a one-on-one interview.³⁷ Patients were subsequently educated on any knowledge deficits regarding the management of their lifestyle. Another level of patient engagement may be to use visual aids during discussions, as a well-known complement to verbal instructions.^45,46 For example, in a controlled study that randomized a ward of elderly patients with 4 or more prescriptions to predischarge counseling, the counseling session aimed to review reasons for their prescriptions along with corresponding side effects, doses, and dosage times with the help of a medicine reminder card. Other uses of visual aid tools identified in our review included the use of pictograms or illustrations or, at minimum, attention to font size.^{7,8,16,29,33,35} In the absence of a visual aid, asking the patient to repeat or demonstrate what was just communicated can be used to assess the amount of information retained.^18,33

An important result discovered in our review of the literature was also the lack of studies that tested the impact of discharge tools on usability of discharge information once at home. Conducting an evaluation of the benefits to patients after discharge can help objectify vague outcomes like health gains or qualify benefits in patient’s views. This might also explain why many studies with documented patient engagement at the time of discharge were able to demonstrate improvements in comprehension but not adherence to instructions. Although patients and caregivers may understand the information, this comprehension does not necessarily mean they will find the information useful or adhere to it once at home. For example, in 1 study, patients discharged with at least 1 medication were randomized to a structured discharge interview during which the treatment plan was reviewed verbally and questions clarified along with a visually enhanced treatment card.²⁶ Although knowledge of medications increased, no effect was found on adherence at 1 week postdischarge. However, use of the treatment card at home was not assessed. Similarly, another study tested the effect of an individualized video of exercises and failed to find a difference in patient adherence at 4 weeks.²⁸ The authors suggested that the lack of benefit may have been because patients were not using the video once at home. This is in contrast to 2 studies that involved patients in their own care by requiring them to request their medication as part of a self-medication tool predischarge.^16,30 Patients were engaged in the process such that increasing independence was given to patients based on their demonstration of understanding and adherence to their treatment while still in the hospital, a learning tool that can be applied once at home. Feeling knowledgeable and involved, as others have suggested, may be the intermediary outcomes that led to improved adherence.⁴⁷ It is also possible that adherence to discharge instructions may vary based on complexity of the information provided, such that instructions focusing solely on medication use may require less patient engagement than discharge instructions that include information on medications, diet, exercise modifications, and follow-up.⁴⁸

Our review has a few limitations. Previous systematic reviews have demonstrated that bundled discharge interventions that include patient-centered education have a positive effect on outcomes postdischarge.^2,5 However, we sought to describe and study the individual and distinct impact of patient engagement in the creation and delivery of discharge tools on outcomes postdischarge. We hoped that this may provide others with key information regarding elements of patient engagement that were particularly useful when designing a new discharge tool. The variability of the studies we identified, however, made it difficult to ascertain what level of patient engagement is required to observe improvements in health outcomes. It is also possible that a higher level of patient engagement may have been used but not described in the studies we reviewed. As only primary outcomes were included, we may have underestimated the effect of patient-centered discharge tools on outcomes that were reported as secondary outcomes. As we were interested in reviewing as many studies of patient-centered discharge tools as possible, we did not assess the quality of the studies and cannot comment on the role of bias in these studies. However, we excluded studies with study designs known to have the highest risk of bias. Lastly, we also cannot comment on whether patient-centered tools may have an effect on outcomes more than 3 months after a hospital discharge. However, several studies included in this review suggest a sustained effect beyond this time period.^8,25,32,37

Patient-centered discharge tools in which patients were engaged in the design or the delivery were found to improve comprehension of but not adherence with discharge instructions. The perceived lack of improved adherence may be due to a lack of studies that measured the usefulness and utilization of information for patients once at home. There was also substantial variability in the extent of patient involvement in designing the style and content of information provided to patients at discharge, as well as the extent of patient engagement when receiving discharge instructions. Future studies would benefit from detailing the level of patient engagement needed in designing and delivery of discharge tools. This information may lead to the discovery of barriers and facilitators to utilization of discharge information once at home and lead to a better understanding of the patient’s journey from hospital to home and onwards.

C.M.B. and this work were funded by a CIHR Canadian Patient Safety Institute Chair in Patient Safety and Continuity of Care. Funding was provided to cover fees to obtain articles from the Donald J. Matthews Complex Care Fund of the University Health Network in Toronto, Canada. The Toronto Central Local Health Integration Network provided funding for the design and implementation of a patient-oriented discharge summary. None of the funding or supportive agencies were involved in the design or conduct of the present study, analysis, or interpretation of the data, or approval of the manuscript.

Disclosures

The authors report no conflicts of interest.

This month’s column is the second in a series of three articles written by a group from Toronto and Houston. The series imagined that a community of gastroenterologists set out to improve the adenoma detection rates of physicians in their practice. The first article described the design and launch of the project. This month, Dr. Bollegala and her colleagues explain the plan-do-study-act (PDSA) cycle of improvement within a small practice. The PDSA cycle is a fundamental component of successful quality improvement initiatives; it allows a group to systematically analyze what works and what doesn’t. The focus of this article is squarely on small community practices (still the majority of gastrointestinal practices nationally), so its relevance is high. PDSA cycles are small, narrowly focused projects that can be accomplished by all as we strive to improve our care of the patients we serve. Next month, we will learn how to embed a quality initiative within our practices so sustained improvement can be seen.

John I. Allen, MD, MBA, AGAF

Editor in Chief
 

Article 1 of our series focused on the emergence of the adenoma detection rate (ADR) as a quality indicator for colonoscopy-based colorectal cancer screening programs.¹ A target ADR of 25% has been established by several national gastroenterology societies and serves as a focus area for those seeking to develop quality improvement (QI) initiatives aimed at reducing the interval incidence of colorectal cancer.² In this series, you are a community-based urban general gastroenterologist interested in improving your current group ADR of 19% to the established target of 25% for each individual endoscopist within the group over a 12-month period.

This article focuses on a clinician-friendly description of the plan-do-study-act (PDSA) cycle, a key construct within the Model for Improvement framework for QI initiatives. It also describes the importance and key elements of QI data reporting, including the run chart. All core concepts will be framed within the series example of the development of an institutional QI initiative for ADR improvement.
 

Plan-Do-Study-Act cycle

Conventional scientific research in health care generally is based on large-scale projects, performed over long periods of time and producing aggregate data analyzed through summary statistics. QI-related research, as it relates to PDSA, in contrast, is characterized by smaller-scale projects performed over shorter periods of time, with iterative protocols to accommodate local context and therefore optimize intervention success. As such, the framework for their development, implementation, and continual modification requires a conceptual and methodologic shift.

The PDSA cycle is characterized by four key steps. The first step is to plan. This step involves addressing the following questions: 1) what are we trying to accomplish? (aim); 2) how will we know that a change is an improvement? (measure); and 3) what changes can we make that will lead to improvement? (change). Additional considerations include ensuring that the stated goal is attainable, relevant, and that the timeline is feasible. An important aspect of the plan stage is gaining an understanding for the current local context, key participants and their roles, and areas in which performance is excelling or is challenged. This understanding is critical to conceptually linking the identified problem with its proposed solution. Formulating an impact prediction allows subsequent learning and adaptation.

The second step is to do. This step involves execution of the identified plan over a specified period of time. It also involves rigorous qualitative and quantitative data collection, allowing the research team to assess change and document unexpected events. The identification of an implementation leader or champion to ensure protocol adherence, effective communication among team members, and coordinate accurate data collection can be critical for overall success.

The third step is to study. This step requires evaluating whether a change in the outcome measure has occurred, which intervention was successful, and whether an identified change is sustained over time. It also requires interpretation of change within the local context, specifically with respect to unintended consequences, unanticipated events, and the sustainability of any gains. To interpret study findings appropriately, feedback with involved process members, endoscopists, and/or other stakeholder groups may be necessary. This can be important for explaining the results of each cycle, identifying protocol modifications for future cycles, and optimizing the opportunity for success. Studying the data generated by a QI initiative requires clear and accurate data display and rules for interpretation.

The fourth step is to act. This final step allows team members to reflect on the results generated and decide whether the same intervention should be continued, modified, or changed, thereby incorporating lessons learned from previous PDSA cycles (Figure 1).³

AGA Institute

Displaying data

The documentation, analysis, and interpretation of data generated by multiple PDSA cycles must be displayed accurately and succinctly. The run chart has been developed as a simple technique for identifying nonrandom patterns (that is, signals), which allows QI researchers to determine the impact of each cycle of change and the stability of that change over a given time period.⁹ This often is contrasted with conventional statistical approaches that aggregate data and perform summary statistical comparisons at static time points. Instead, the run chart allows for an appreciation of the dynamic nature of PDSA-driven process manipulation and resulting outcome changes.

Correct interpretation of the presented data requires an understanding of common cause variation (CCV) and special cause variation (SCV). CCV occurs randomly and is present in all health care processes. It can never be eliminated completely. SCV, in contrast, is the result of external factors that are imposed on normal processes. For example, the introduction of audible timers within endoscopy rooms to ensure adequate withdrawal time may result in an increase in the ADR. The relatively stable ADR measured in both the pre-intervention and postintervention periods are subject to CCV. However, the postintervention increase in ADR is the result of SCV.¹⁰

As shown in Figure 2, the horizontal axis shows the time scale and spans the entire duration of the intervention period. The y-axis shows the outcome measure of interest. A horizontal line representing the median is shown.⁹ A goal line also may be depicted. Annotations to indicate the implementation of change or other important events (such as unintended consequences or unexpected events) also may be added to facilitate data interpretation.

AGA Institute

Summary and next steps

This article selectively reviews the process of change framed by the PDSA cycle. We also discuss the role of data display and interpretation using a run chart. The final article in this series will cover how to sustain change and support a culture of continuous improvement.

References

1. Corley, D.A., Jensen, C.D., Marks, A.R., et al. Adenoma detection rate and risk of colorectal cancer and death. N Engl J Med. 2014;370:1298-306.

2. Cohen, J., Schoenfeld, P., Park, W., et al. Quality indicators for colonoscopy. Gastrointest Endosc. 2015;81:31-53.

3. Module 5: Improvement Cycle. (2013). Available at: http://implementation.fpg.unc.edu/book/export/html/326. Accessed Feb. 1, 2016.

4. Taylor, M.J., McNicholas, C., Nicolay, C., et al. Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Qual Saf. 2014;23(4):290-8.

5. Davidoff, F., Batalden, P., Stevens, D. et al. Publication guidelines for quality improvement in health care: evolution of the SQUIRE project. Qual Saf Health Care. 2008;17:i3-9.

6. Ogrinc, G., Mooney, S., Estrada, C., et al. The SQUIRE (standards for Quality Improvement Reporting Excellence) guidelines for quality improvement reporting: explanation and elaboration. Qual Saf Health Care. 2008;17:i13-32.

7. Nelson, E.C., Batalden, B.P., Godfrey, M.M. Quality by design: a clinical microsystems approach. Jossey-Bass, San Francisco; 2007.

8. Coe, S.G.C.J., Diehl, N.N., Wallace, M.B. An endoscopic quality improvement program improves detection of colorectal adenomas. Am J Gastroenterol. 2013;108(2):219-26.

9. Perla, R.J., Provost, L.P., Murray, S.K. The run chart: a simple analytical tool for learning from variation in healthcare processes. BMJ Qual Saf. 2011;20:46-51.

10. Neuhauser, D., Provost, L., Bergman, B. The meaning of variation to healthcare managers, clinical and health-services researchers, and individual patients. BMJ Qual Saf. 2011;20:i36-40.

11. Swed, F.S. Eisenhart, C. Tables for testing randomness of grouping in a sequence of alternatives. Ann Math Statist. 1943;14:66-87

Dr. Bollegala is in the division of gastroenterology, department of medicine, Women’s College Hospital; Dr. Mosko is in the division of gastroenterology, department of medicine, St. Michael’s Hospital, and the Institute of Health Policy, Management, and Evaluation; Dr. Bernstein is in the division of gastroenterology, department of medicine, Sunnybrook Health Sciences Centre; Dr. Brahmania is in the Toronto Center for Liver Diseases, division of gastroenterology, department of medicine, University Health Network; Dr. Liu is in the division of gastroenterology, department of medicine, University Health Network; Dr. Steinhart is at Mount Sinai Hospital Centre for Inflammatory Bowel Disease, department of medicine and Institute of Health Policy, Management, and Evaluation; Dr. Silver is in the division of nephrology, St. Michael’s Hospital; Dr. Bell is in the division of internal medicine, department of medicine, Mount Sinai Hospital; Dr. Nguyen is at Mount Sinai Hospital Centre for Inflammatory Bowel Disease, department of medicine; Dr. Weizman is at the Mount Sinai Hospital Centre for Inflammatory Bowel Disease, department of medicine, and Institute of Health Policy, Management and Evaluation. All are at the University of Toronto. Dr. Patel is in the division of gastroenterology and hepatology, department of medicine, Baylor College of Medicine, Houston. The authors disclose no conflicts.

This month’s column is the second in a series of three articles written by a group from Toronto and Houston. The series imagined that a community of gastroenterologists set out to improve the adenoma detection rates of physicians in their practice. The first article described the design and launch of the project. This month, Dr. Bollegala and her colleagues explain the plan-do-study-act (PDSA) cycle of improvement within a small practice. The PDSA cycle is a fundamental component of successful quality improvement initiatives; it allows a group to systematically analyze what works and what doesn’t. The focus of this article is squarely on small community practices (still the majority of gastrointestinal practices nationally), so its relevance is high. PDSA cycles are small, narrowly focused projects that can be accomplished by all as we strive to improve our care of the patients we serve. Next month, we will learn how to embed a quality initiative within our practices so sustained improvement can be seen.

John I. Allen, MD, MBA, AGAF

Editor in Chief
 

Article 1 of our series focused on the emergence of the adenoma detection rate (ADR) as a quality indicator for colonoscopy-based colorectal cancer screening programs.¹ A target ADR of 25% has been established by several national gastroenterology societies and serves as a focus area for those seeking to develop quality improvement (QI) initiatives aimed at reducing the interval incidence of colorectal cancer.² In this series, you are a community-based urban general gastroenterologist interested in improving your current group ADR of 19% to the established target of 25% for each individual endoscopist within the group over a 12-month period.

This article focuses on a clinician-friendly description of the plan-do-study-act (PDSA) cycle, a key construct within the Model for Improvement framework for QI initiatives. It also describes the importance and key elements of QI data reporting, including the run chart. All core concepts will be framed within the series example of the development of an institutional QI initiative for ADR improvement.
 

Plan-Do-Study-Act cycle

Conventional scientific research in health care generally is based on large-scale projects, performed over long periods of time and producing aggregate data analyzed through summary statistics. QI-related research, as it relates to PDSA, in contrast, is characterized by smaller-scale projects performed over shorter periods of time, with iterative protocols to accommodate local context and therefore optimize intervention success. As such, the framework for their development, implementation, and continual modification requires a conceptual and methodologic shift.

The PDSA cycle is characterized by four key steps. The first step is to plan. This step involves addressing the following questions: 1) what are we trying to accomplish? (aim); 2) how will we know that a change is an improvement? (measure); and 3) what changes can we make that will lead to improvement? (change). Additional considerations include ensuring that the stated goal is attainable, relevant, and that the timeline is feasible. An important aspect of the plan stage is gaining an understanding for the current local context, key participants and their roles, and areas in which performance is excelling or is challenged. This understanding is critical to conceptually linking the identified problem with its proposed solution. Formulating an impact prediction allows subsequent learning and adaptation.

The second step is to do. This step involves execution of the identified plan over a specified period of time. It also involves rigorous qualitative and quantitative data collection, allowing the research team to assess change and document unexpected events. The identification of an implementation leader or champion to ensure protocol adherence, effective communication among team members, and coordinate accurate data collection can be critical for overall success.

The third step is to study. This step requires evaluating whether a change in the outcome measure has occurred, which intervention was successful, and whether an identified change is sustained over time. It also requires interpretation of change within the local context, specifically with respect to unintended consequences, unanticipated events, and the sustainability of any gains. To interpret study findings appropriately, feedback with involved process members, endoscopists, and/or other stakeholder groups may be necessary. This can be important for explaining the results of each cycle, identifying protocol modifications for future cycles, and optimizing the opportunity for success. Studying the data generated by a QI initiative requires clear and accurate data display and rules for interpretation.

The fourth step is to act. This final step allows team members to reflect on the results generated and decide whether the same intervention should be continued, modified, or changed, thereby incorporating lessons learned from previous PDSA cycles (Figure 1).³

AGA Institute

Displaying data

The documentation, analysis, and interpretation of data generated by multiple PDSA cycles must be displayed accurately and succinctly. The run chart has been developed as a simple technique for identifying nonrandom patterns (that is, signals), which allows QI researchers to determine the impact of each cycle of change and the stability of that change over a given time period.⁹ This often is contrasted with conventional statistical approaches that aggregate data and perform summary statistical comparisons at static time points. Instead, the run chart allows for an appreciation of the dynamic nature of PDSA-driven process manipulation and resulting outcome changes.

Correct interpretation of the presented data requires an understanding of common cause variation (CCV) and special cause variation (SCV). CCV occurs randomly and is present in all health care processes. It can never be eliminated completely. SCV, in contrast, is the result of external factors that are imposed on normal processes. For example, the introduction of audible timers within endoscopy rooms to ensure adequate withdrawal time may result in an increase in the ADR. The relatively stable ADR measured in both the pre-intervention and postintervention periods are subject to CCV. However, the postintervention increase in ADR is the result of SCV.¹⁰

As shown in Figure 2, the horizontal axis shows the time scale and spans the entire duration of the intervention period. The y-axis shows the outcome measure of interest. A horizontal line representing the median is shown.⁹ A goal line also may be depicted. Annotations to indicate the implementation of change or other important events (such as unintended consequences or unexpected events) also may be added to facilitate data interpretation.

AGA Institute

Summary and next steps

This article selectively reviews the process of change framed by the PDSA cycle. We also discuss the role of data display and interpretation using a run chart. The final article in this series will cover how to sustain change and support a culture of continuous improvement.

References

1. Corley, D.A., Jensen, C.D., Marks, A.R., et al. Adenoma detection rate and risk of colorectal cancer and death. N Engl J Med. 2014;370:1298-306.

2. Cohen, J., Schoenfeld, P., Park, W., et al. Quality indicators for colonoscopy. Gastrointest Endosc. 2015;81:31-53.

3. Module 5: Improvement Cycle. (2013). Available at: http://implementation.fpg.unc.edu/book/export/html/326. Accessed Feb. 1, 2016.

4. Taylor, M.J., McNicholas, C., Nicolay, C., et al. Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Qual Saf. 2014;23(4):290-8.

5. Davidoff, F., Batalden, P., Stevens, D. et al. Publication guidelines for quality improvement in health care: evolution of the SQUIRE project. Qual Saf Health Care. 2008;17:i3-9.

6. Ogrinc, G., Mooney, S., Estrada, C., et al. The SQUIRE (standards for Quality Improvement Reporting Excellence) guidelines for quality improvement reporting: explanation and elaboration. Qual Saf Health Care. 2008;17:i13-32.

7. Nelson, E.C., Batalden, B.P., Godfrey, M.M. Quality by design: a clinical microsystems approach. Jossey-Bass, San Francisco; 2007.

8. Coe, S.G.C.J., Diehl, N.N., Wallace, M.B. An endoscopic quality improvement program improves detection of colorectal adenomas. Am J Gastroenterol. 2013;108(2):219-26.

9. Perla, R.J., Provost, L.P., Murray, S.K. The run chart: a simple analytical tool for learning from variation in healthcare processes. BMJ Qual Saf. 2011;20:46-51.

10. Neuhauser, D., Provost, L., Bergman, B. The meaning of variation to healthcare managers, clinical and health-services researchers, and individual patients. BMJ Qual Saf. 2011;20:i36-40.

11. Swed, F.S. Eisenhart, C. Tables for testing randomness of grouping in a sequence of alternatives. Ann Math Statist. 1943;14:66-87

Dr. Bollegala is in the division of gastroenterology, department of medicine, Women’s College Hospital; Dr. Mosko is in the division of gastroenterology, department of medicine, St. Michael’s Hospital, and the Institute of Health Policy, Management, and Evaluation; Dr. Bernstein is in the division of gastroenterology, department of medicine, Sunnybrook Health Sciences Centre; Dr. Brahmania is in the Toronto Center for Liver Diseases, division of gastroenterology, department of medicine, University Health Network; Dr. Liu is in the division of gastroenterology, department of medicine, University Health Network; Dr. Steinhart is at Mount Sinai Hospital Centre for Inflammatory Bowel Disease, department of medicine and Institute of Health Policy, Management, and Evaluation; Dr. Silver is in the division of nephrology, St. Michael’s Hospital; Dr. Bell is in the division of internal medicine, department of medicine, Mount Sinai Hospital; Dr. Nguyen is at Mount Sinai Hospital Centre for Inflammatory Bowel Disease, department of medicine; Dr. Weizman is at the Mount Sinai Hospital Centre for Inflammatory Bowel Disease, department of medicine, and Institute of Health Policy, Management and Evaluation. All are at the University of Toronto. Dr. Patel is in the division of gastroenterology and hepatology, department of medicine, Baylor College of Medicine, Houston. The authors disclose no conflicts.

This month’s column is the second in a series of three articles written by a group from Toronto and Houston. The series imagined that a community of gastroenterologists set out to improve the adenoma detection rates of physicians in their practice. The first article described the design and launch of the project. This month, Dr. Bollegala and her colleagues explain the plan-do-study-act (PDSA) cycle of improvement within a small practice. The PDSA cycle is a fundamental component of successful quality improvement initiatives; it allows a group to systematically analyze what works and what doesn’t. The focus of this article is squarely on small community practices (still the majority of gastrointestinal practices nationally), so its relevance is high. PDSA cycles are small, narrowly focused projects that can be accomplished by all as we strive to improve our care of the patients we serve. Next month, we will learn how to embed a quality initiative within our practices so sustained improvement can be seen.

John I. Allen, MD, MBA, AGAF

Editor in Chief
 

Article 1 of our series focused on the emergence of the adenoma detection rate (ADR) as a quality indicator for colonoscopy-based colorectal cancer screening programs.¹ A target ADR of 25% has been established by several national gastroenterology societies and serves as a focus area for those seeking to develop quality improvement (QI) initiatives aimed at reducing the interval incidence of colorectal cancer.² In this series, you are a community-based urban general gastroenterologist interested in improving your current group ADR of 19% to the established target of 25% for each individual endoscopist within the group over a 12-month period.

This article focuses on a clinician-friendly description of the plan-do-study-act (PDSA) cycle, a key construct within the Model for Improvement framework for QI initiatives. It also describes the importance and key elements of QI data reporting, including the run chart. All core concepts will be framed within the series example of the development of an institutional QI initiative for ADR improvement.
 

Plan-Do-Study-Act cycle

Conventional scientific research in health care generally is based on large-scale projects, performed over long periods of time and producing aggregate data analyzed through summary statistics. QI-related research, as it relates to PDSA, in contrast, is characterized by smaller-scale projects performed over shorter periods of time, with iterative protocols to accommodate local context and therefore optimize intervention success. As such, the framework for their development, implementation, and continual modification requires a conceptual and methodologic shift.

The PDSA cycle is characterized by four key steps. The first step is to plan. This step involves addressing the following questions: 1) what are we trying to accomplish? (aim); 2) how will we know that a change is an improvement? (measure); and 3) what changes can we make that will lead to improvement? (change). Additional considerations include ensuring that the stated goal is attainable, relevant, and that the timeline is feasible. An important aspect of the plan stage is gaining an understanding for the current local context, key participants and their roles, and areas in which performance is excelling or is challenged. This understanding is critical to conceptually linking the identified problem with its proposed solution. Formulating an impact prediction allows subsequent learning and adaptation.

The second step is to do. This step involves execution of the identified plan over a specified period of time. It also involves rigorous qualitative and quantitative data collection, allowing the research team to assess change and document unexpected events. The identification of an implementation leader or champion to ensure protocol adherence, effective communication among team members, and coordinate accurate data collection can be critical for overall success.

The third step is to study. This step requires evaluating whether a change in the outcome measure has occurred, which intervention was successful, and whether an identified change is sustained over time. It also requires interpretation of change within the local context, specifically with respect to unintended consequences, unanticipated events, and the sustainability of any gains. To interpret study findings appropriately, feedback with involved process members, endoscopists, and/or other stakeholder groups may be necessary. This can be important for explaining the results of each cycle, identifying protocol modifications for future cycles, and optimizing the opportunity for success. Studying the data generated by a QI initiative requires clear and accurate data display and rules for interpretation.

The fourth step is to act. This final step allows team members to reflect on the results generated and decide whether the same intervention should be continued, modified, or changed, thereby incorporating lessons learned from previous PDSA cycles (Figure 1).³

AGA Institute

Displaying data

The documentation, analysis, and interpretation of data generated by multiple PDSA cycles must be displayed accurately and succinctly. The run chart has been developed as a simple technique for identifying nonrandom patterns (that is, signals), which allows QI researchers to determine the impact of each cycle of change and the stability of that change over a given time period.⁹ This often is contrasted with conventional statistical approaches that aggregate data and perform summary statistical comparisons at static time points. Instead, the run chart allows for an appreciation of the dynamic nature of PDSA-driven process manipulation and resulting outcome changes.

Correct interpretation of the presented data requires an understanding of common cause variation (CCV) and special cause variation (SCV). CCV occurs randomly and is present in all health care processes. It can never be eliminated completely. SCV, in contrast, is the result of external factors that are imposed on normal processes. For example, the introduction of audible timers within endoscopy rooms to ensure adequate withdrawal time may result in an increase in the ADR. The relatively stable ADR measured in both the pre-intervention and postintervention periods are subject to CCV. However, the postintervention increase in ADR is the result of SCV.¹⁰

As shown in Figure 2, the horizontal axis shows the time scale and spans the entire duration of the intervention period. The y-axis shows the outcome measure of interest. A horizontal line representing the median is shown.⁹ A goal line also may be depicted. Annotations to indicate the implementation of change or other important events (such as unintended consequences or unexpected events) also may be added to facilitate data interpretation.

AGA Institute

Summary and next steps

This article selectively reviews the process of change framed by the PDSA cycle. We also discuss the role of data display and interpretation using a run chart. The final article in this series will cover how to sustain change and support a culture of continuous improvement.

References

1. Corley, D.A., Jensen, C.D., Marks, A.R., et al. Adenoma detection rate and risk of colorectal cancer and death. N Engl J Med. 2014;370:1298-306.

2. Cohen, J., Schoenfeld, P., Park, W., et al. Quality indicators for colonoscopy. Gastrointest Endosc. 2015;81:31-53.

3. Module 5: Improvement Cycle. (2013). Available at: http://implementation.fpg.unc.edu/book/export/html/326. Accessed Feb. 1, 2016.

4. Taylor, M.J., McNicholas, C., Nicolay, C., et al. Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Qual Saf. 2014;23(4):290-8.

5. Davidoff, F., Batalden, P., Stevens, D. et al. Publication guidelines for quality improvement in health care: evolution of the SQUIRE project. Qual Saf Health Care. 2008;17:i3-9.

6. Ogrinc, G., Mooney, S., Estrada, C., et al. The SQUIRE (standards for Quality Improvement Reporting Excellence) guidelines for quality improvement reporting: explanation and elaboration. Qual Saf Health Care. 2008;17:i13-32.

7. Nelson, E.C., Batalden, B.P., Godfrey, M.M. Quality by design: a clinical microsystems approach. Jossey-Bass, San Francisco; 2007.

8. Coe, S.G.C.J., Diehl, N.N., Wallace, M.B. An endoscopic quality improvement program improves detection of colorectal adenomas. Am J Gastroenterol. 2013;108(2):219-26.

9. Perla, R.J., Provost, L.P., Murray, S.K. The run chart: a simple analytical tool for learning from variation in healthcare processes. BMJ Qual Saf. 2011;20:46-51.

10. Neuhauser, D., Provost, L., Bergman, B. The meaning of variation to healthcare managers, clinical and health-services researchers, and individual patients. BMJ Qual Saf. 2011;20:i36-40.

11. Swed, F.S. Eisenhart, C. Tables for testing randomness of grouping in a sequence of alternatives. Ann Math Statist. 1943;14:66-87

Dr. Bollegala is in the division of gastroenterology, department of medicine, Women’s College Hospital; Dr. Mosko is in the division of gastroenterology, department of medicine, St. Michael’s Hospital, and the Institute of Health Policy, Management, and Evaluation; Dr. Bernstein is in the division of gastroenterology, department of medicine, Sunnybrook Health Sciences Centre; Dr. Brahmania is in the Toronto Center for Liver Diseases, division of gastroenterology, department of medicine, University Health Network; Dr. Liu is in the division of gastroenterology, department of medicine, University Health Network; Dr. Steinhart is at Mount Sinai Hospital Centre for Inflammatory Bowel Disease, department of medicine and Institute of Health Policy, Management, and Evaluation; Dr. Silver is in the division of nephrology, St. Michael’s Hospital; Dr. Bell is in the division of internal medicine, department of medicine, Mount Sinai Hospital; Dr. Nguyen is at Mount Sinai Hospital Centre for Inflammatory Bowel Disease, department of medicine; Dr. Weizman is at the Mount Sinai Hospital Centre for Inflammatory Bowel Disease, department of medicine, and Institute of Health Policy, Management and Evaluation. All are at the University of Toronto. Dr. Patel is in the division of gastroenterology and hepatology, department of medicine, Baylor College of Medicine, Houston. The authors disclose no conflicts.

This article by Adam Weizman and colleagues is the first of a three-part series that will provide practical advice for practices that wish to develop a quality initiative. The first article, “Launching a quality improvement initiative” describes the infrastructure, personnel, and structure needed to approach an identified problem within a practice (variability in adenoma detection rates). This case-based approach helps us understand the step-by-step approach needed to reduce variability and improve quality. The authors present a plan (road map) in a straightforward and practical way that seems simple, but if followed carefully, leads to success. These articles are rich in resources and link to state-of-the-art advice.

John I. Allen, MD, MBA, AGAF, Special Section Editor

There has been increasing focus on measuring quality indicators in gastroenterology over the past few years. The adenoma detection rate (ADR) has emerged as one of the most important quality indicators because it is supported by robust clinical evidence.^1-3 With every 1% increase in ADR, a 3% reduction in interval colorectal cancer has been noted.³ As such, an ADR of 25% has been designated as an important quality target for all endoscopists who perform colorectal cancer screening.¹

You work at a community hospital in a large, metropolitan area. Your colleagues in a number of other departments across your hospital have been increasingly interested in quality improvement (QI) and have launched QI interventions, although none in your department. Moreover, there have been reforms in how hospital endoscopy units are funded in your jurisdiction, with a move toward volume-based funding with a quality overlay. In an effort to improve efficiency and better characterize performance, the hospital has been auditing the performance of all endoscopists at your institution over the past year. Among the eight endoscopists who work at your hospital, the overall ADR has been found to be 19%, decreasing to less than the generally accepted benchmark.¹

In response to the results of the audit in your unit, you decide that you would like to develop an initiative to improve your group’s ADR.

Forming a quality improvement team

The first step in any QI project is to establish an improvement team. This working group consists of individuals with specific roles who perform interdependent tasks and share a common goal.⁴ Usually, frontline health care workers who are impacted most by the quality-of-care problem form the foundation of the team. A team lead is identified who will oversee the project. Content experts are also helpful members of the team who may have particular expertise in the clinical domain that will be the focus of the project. In addition, an improvement adviser, an individual with some expertise in QI, is needed on the team. This adviser may be from within your department or from outside. Although they may not possess expertise in the clinical problem you are trying to tackle, they should have skills in QI methodology and process to aid the team. An executive sponsor also needs to be identified. This should be an influential and well-respected individual who holds a senior administrative position at your institution who can help the team overcome barriers and secure resources. Physician engagement is a critical, often-overlooked step in any improvement effort. Regardless of the initiative, physicians continue to have tremendous influence over hospital-based outcomes.⁵ Identifying a physician champion, a prominent and respected physician at your organization to help spread the importance of your efforts and create a burning platform for change, is helpful. It also is valuable to have a patient on the improvement team to provide unique perspectives that only the end user of health care can convey and to ensure that the project is patient centered, as all improvement efforts should be.⁶

Improvement framework

Before starting any improvement effort, there are several important considerations that need to be addressed when choosing a quality improvement target.7 It is important to have a good understanding of the burden and severity of the problem. This often requires audit and measuring. For example, although we may think there is a problem with ADR in our endoscopy unit based on a general impression, it is critical to have data to support this suspicion. This is part of a current state analysis (discussed later). It also is important to select a quality-of-care problem that is under you or your group’s direct control. For example, it would be difficult to initiate a quality improvement project aimed at changing the practice of radiology reporting as a gastroenterologist. It is important to pick a problem that is focused and within a narrow scope that is feasible to address and then improve. Consideration of the unintended consequences of an improvement initiative often is overlooked, but needs to be considered because not all that comes out of quality improvement efforts is good. Finally, the likelihood of success of a quality initiative is increased significantly if it can generate momentum and lead to other interventions both within your department and beyond.

There are several specific improvement frameworks that can be used by a team to address a quality-of-care problem and perform a quality improvement project. The framework chosen depends on the type of problem that is being targeted and the training of the individuals on the improvement team. Three of the most commonly used improvement frameworks include the following: 1) Six Sigma; 2) Lean; and 3) Model for Improvement.

AGA Institute

Six Sigma

Six Sigma is focused on improvement by reducing variability.⁸ It is a highly analytic framework relying on statistical analysis and mathematical modeling. It is best suited for projects in which the root cause and contributors to the target problem remain unclear and the aim of the intervention is to reduce variation.

Lean

Lean emphasizes improvement through elimination of waste and classifies all parts of any process as value added and nonvalue added.⁹ It is estimated that 95% of activities in any health care process are nonvalue added and the objective of Lean is to identify opportunities to simplify and create efficiencies. It is best suited for target problems that directly can be observed and mapped out, for example, process of care, flow, and efficiency of an endoscopy unit.

Model for Improvement

The Model for Improvement has been popularized by the Institute for Healthcare Improvement.^10,11 It is well suited for health care teams, and its advantages are its adaptability to many improvement targets and lack of extensive training, consultant support, or statistical training as required by the previous frameworks mentioned earlier. As a result, it is the most commonly used improvement framework.

Using the Model for Improvement

The Model for Improvement is organized around three main questions: 1) What are we trying to accomplish? 2) How will we know that a change is an improvement? and 3) What changes can result in improvement?

Question 1: What are we trying to accomplish?

The first stage using the Model for Improvement is developing a clear project aim. A good aim statement should be specific in defining what measures one is hoping to improve and setting a concrete deadline by which to achieve it.^10,11 It should answer the questions of what the team is trying to improve, by how much, and by what date. It is more effective for the target to be an ambitious, stretch goal to ensure the effort is worth the resources and time that will be invested by the team. Not only does a good aim statement serve as the foundation for the project, but it can redirect the team if the improvement effort is getting off track. In the earlier example of improving ADR, an aim statement could be “to increase the ADR of all endoscopists who perform colonoscopy at your hospital to 25% over a 12-month period.”

Question 2: How will we know that a change is an improvement?

This step involves defining measures that will allow you to understand if changes implemented are impacting the system within which your target problem resides and if this represents an improvement. This usually involves continuous, real-time measurement. Outcome measures are clinically relevant outcomes and are the ultimate goal of what the project team is trying to accomplish. In the example of ADR, this could be the proportion of endoscopists at your institution with an ADR greater than 25%. Process measures are relevant to the system within which you are working and your target problem resides. Typically, the intervention that you implement will have impact that is measurable much earlier by process outcomes than outcomes measures, which are usually a downstream effect. As such, an improvement project still may be a success if it shows improvements in process measures only. For example, the proportion of endoscopists measuring withdrawal time would be a process measure in an intervention aimed at improving ADR. In time, improvement in process measures may translate to improvements in the outcome measure. Balancing measures are indicators of unintended consequences of the project. Not all that comes from an improvement effort is necessarily positive. If improvements in certain process measures come at the cost of harms shown by the balancing measures, such as deterioration in staff satisfaction or increase in time per procedure, the improvement project may not be worth continuing.

Importance of understanding the target problem: Current-state analysis

In contrast to classic enumerative research in which the clinical environment can be well controlled, quality improvement work focuses on sampling and intervening upon a less controlled and dynamic process or system with the intent of improving it.¹⁰ Just as treatment strategies in clinical medicine are based on diagnostic testing, so too in quality improvement work, the strategy of diagnosing the current state allows for linking the root cause of quality problems with solutions that can induce positive change.

Several common diagnostic tools are used to identify root causes of quality and safety issues. These include the following: 1) process mapping, 2) cause-and-effect diagrams, and 3) Pareto charts.

Process mapping

Process maps are tools used to understand the system that is being studied. A process map is a graphic depiction of the flow through a process, which creates a collaborative awareness of the current state and identifies opportunities for improvement. It is important that multiple individuals who have knowledge of the process in question are involved in its creation. Process maps are created by first establishing the start and end of the process. Second, the high-level steps are included. Third, a more detailed set of steps can be included within each of the high-level steps.

Cause-and-effect diagrams

Cause-and-effect diagrams, also known as Ishikawa or fishbone diagrams, are helpful brainstorming tools used to graphically display and explore potential causes of a target problem. They illustrate that there often are many contributing factors to one underlying problem and the relationship between contributing factors. Classic examples of categories include equipment, environment, materials, methods and process, people, and measurement.¹⁰ Figure 1 provides an example of these tools in an effort to improve ADR.
 

To identify the most important contributors to the target problem and thus where to focus improvement efforts, a Pareto chart, a bar graph that places all defects/causes in the order of the frequency in which they occur, is constructed. The x-axis is a list of possible defects (Figure 1). The y-axis is the frequency with which any one defect is occurring, and the third (x-2) axis is the cumulative frequency. In theory, it is expected that there will be a vital few defects that account for 80% of all occurrences (referred to by some as the 80:20 rule).10, 11 Populating this graph requires measurement, which, as discussed earlier, is the key to understanding any problem. Measurement can be accomplished through direct observation/audit, chart review, and/or multivoting.

Question 3: What changes can result in improvement?

Once the improvement team has defined an aim and established its family of measures, it is time to develop and implement an intervention. Rather than investing time and resources into one intervention that may or may not be successful, it is preferable to perform small change cycles in which the intervention is conducted on a small scale, refined, and either repeated or changed. As a result, most quality improvement projects consist of an iterative process. The Model for Improvement defines four steps that allow the improvement team to perform this: Plan, do, study, act (PDSA).^4,10,11 The first two questions listed earlier allowed the improvement team to plan the intervention. The next step, do, involves implementing your project on a small scale, thereby testing your change while collecting continuous measurements. Study involves interpreting your data using both conventional methods and several improvement-specific methods (discussed later) that help answer the question of how will we know that a change is improvement? Finally, act involves making a conclusion about your first PDSA cycle, helping to inform subsequent cycles. This results in a series of small, rapid cycle changes, one building on the next, that lead to implementation of change(s) that ultimately serve to address your improvement problem and your project aim.

A change concept is an approach known to be useful in developing specific changes that result in improvement. Change concepts are used as a starting point to generate change ideas. A number of change concepts spanning nine main categories have been defined by the Associates for Process Improvement,¹⁰ including eliminating waste, improving work flow, managing variation, and designing systems to prevent error. For the purpose of improving ADR, your team may choose a few change concepts and ideas based on the diagnostic work-up. For example, the change concept of designing the system to prevent errors through standardizing withdrawal time for all physicians may lead to an improvement in ADR. This then is linked to the change idea of audible timers placed in endoscopy suites to ensure longer withdrawal times.¹² The impact of this change would be measured and the next cycle would build on these results.
 

Summary and next steps

In this first article of the series, the QI team moved forward with their aim to increase ADR. A root cause analysis was undertaken using multiple diagnostic tools including a fishbone diagram and a Pareto chart. Finally, change ideas were generated based on the earlier-described root causes and established change concepts. The next steps involve undertaking PDSA cycles to test change ideas and monitor for improvement.

References

1. Rex, D.K., Schoenfeld, P.S., Cohen, J. et al. Quality indicators for colonoscopy. Gastrointest Endosc. 2015;81:31-53.

2. Rex, D.K., Bond, J.H., Winawer, S. et al. Quality in the technical performance of colonoscopy and the continuous quality improvement process for colonoscopy: recommendations of the U.S. Multi-Society Task Force on Colorectal Cancer. Am J Gastroenterol. 2002;97:1296-308.

3. Corley, D., Jensen, C.D., Marks, A.R. et al. Adenoma detection rate and risk of colorectal cancer and death. N Engl J Med. 2014;370:1298-306.
4. Kotter, J.P. Leading change. Harvard Business Review Press, Boston; 2012

5. Taitz, J.M., Lee, T.H., and Sequist, T.D. A framework for engaging physicians in quality and safety. BMJ Qual Saf. 2012;21:722-8.

6. Carman, K.L., Dardess, P., Maurer, M. et al. Patient and family engagement: a framework for understanding the elements and developing interventions and policies. Health Aff (Millwood). 2013;33:223-31.

7.Ranji, S.R. and Shojania, S.G. Implementing patient safety interventions in your hospital: what to try and what to avoid. Med Clin North Am. 2008;92:275-93.

8. Antony, J. Six Sigma vs Lean: some perspectives from leading academics and practitioners. Int J Product Perform Manage. 2011;60:185-90.

9. Bercaw, R. Taking improvement from the assembly line to healthcare: the application of lean within the healthcare industry. Taylor and Francis, Boca Raton, FL; 2012

10. Langley, G.J., Nolan, K.M., Nolan, T.W. et al. The improvement guide: a practical approach to enhancing organizational performance. Jossey-Bass, San Francisco; 2009

11. Berwick, D.M. A primer on leading the improvement of systems. BMJ. 1996;312:619-22.

12. Corley, D.A., Jensen, C.D., and Marks, A.R. Can we improve adenoma detection rates? A systematic review of intervention studies. Gastrointest Endosc. 2011;74:656-65.

Dr. Weizman is at the Mount Sinai Hospital Centre for Inflammatory Bowel Disease, and Institute of Health Policy, Management and Evaluation, department of medicine; Dr. Mosko is in the division of gastroenterology, St. Michael’s Hospital, department of medicine, and Institute of Health Policy, Management and Evaluation; Dr. Bollegala is in the division of gastroenterology, Women’s College Hospital, department of medicine; Dr. Bernstein is in the division of gastroenterology, Sunnybrook Health Sciences Centre, department of medicine; Dr. Brahmania is at the Toronto Center for Liver Diseases, division of gastroenterology, University Health Network, department of medicine; Dr. Liu is in the division of gastroenterology, University Health Network, department of medicine; Dr. Steinhart is at Mount Sinai Hospital Centre for Inflammatory Bowel Disease, department of medicine, and Institute of Health Policy, Management and Evaluation; Dr. Silver is in the division of nephrology, St. Michael’s Hospital; Dr. Bell is in the division of internal medicine, Mount Sinai Hospital, department of medicine, and Institute of Health Policy, Management and Evaluation; and Dr. Nguyen is at Mount Sinai Hospital Centre for Inflammatory Bowel Disease, department of medicine and Institute of Health Policy, Management and Evaluation; all are at the University of Toronto. Dr. Steinhart is an advisory board member for Abbvie, Janssen, Takeda, Shire, Allergan, Pfizer, Merck, Ferring, and Pharmascience; has received research grants from Abbvie, Amgen, Genentech, Cellgene, Arena Pharmaceuticals, Red Hill Biopharma, Millenium, Roche, and Centocor; and has received speaking honoraria from Abbvie, Janssen, Takeda, Shire, Pfizer, Merck, and Ferring.

The remaining authors declare no conflicts for this article.

This article by Adam Weizman and colleagues is the first of a three-part series that will provide practical advice for practices that wish to develop a quality initiative. The first article, “Launching a quality improvement initiative” describes the infrastructure, personnel, and structure needed to approach an identified problem within a practice (variability in adenoma detection rates). This case-based approach helps us understand the step-by-step approach needed to reduce variability and improve quality. The authors present a plan (road map) in a straightforward and practical way that seems simple, but if followed carefully, leads to success. These articles are rich in resources and link to state-of-the-art advice.

John I. Allen, MD, MBA, AGAF, Special Section Editor

There has been increasing focus on measuring quality indicators in gastroenterology over the past few years. The adenoma detection rate (ADR) has emerged as one of the most important quality indicators because it is supported by robust clinical evidence.^1-3 With every 1% increase in ADR, a 3% reduction in interval colorectal cancer has been noted.³ As such, an ADR of 25% has been designated as an important quality target for all endoscopists who perform colorectal cancer screening.¹

You work at a community hospital in a large, metropolitan area. Your colleagues in a number of other departments across your hospital have been increasingly interested in quality improvement (QI) and have launched QI interventions, although none in your department. Moreover, there have been reforms in how hospital endoscopy units are funded in your jurisdiction, with a move toward volume-based funding with a quality overlay. In an effort to improve efficiency and better characterize performance, the hospital has been auditing the performance of all endoscopists at your institution over the past year. Among the eight endoscopists who work at your hospital, the overall ADR has been found to be 19%, decreasing to less than the generally accepted benchmark.¹

In response to the results of the audit in your unit, you decide that you would like to develop an initiative to improve your group’s ADR.

Forming a quality improvement team

The first step in any QI project is to establish an improvement team. This working group consists of individuals with specific roles who perform interdependent tasks and share a common goal.⁴ Usually, frontline health care workers who are impacted most by the quality-of-care problem form the foundation of the team. A team lead is identified who will oversee the project. Content experts are also helpful members of the team who may have particular expertise in the clinical domain that will be the focus of the project. In addition, an improvement adviser, an individual with some expertise in QI, is needed on the team. This adviser may be from within your department or from outside. Although they may not possess expertise in the clinical problem you are trying to tackle, they should have skills in QI methodology and process to aid the team. An executive sponsor also needs to be identified. This should be an influential and well-respected individual who holds a senior administrative position at your institution who can help the team overcome barriers and secure resources. Physician engagement is a critical, often-overlooked step in any improvement effort. Regardless of the initiative, physicians continue to have tremendous influence over hospital-based outcomes.⁵ Identifying a physician champion, a prominent and respected physician at your organization to help spread the importance of your efforts and create a burning platform for change, is helpful. It also is valuable to have a patient on the improvement team to provide unique perspectives that only the end user of health care can convey and to ensure that the project is patient centered, as all improvement efforts should be.⁶

Improvement framework

Before starting any improvement effort, there are several important considerations that need to be addressed when choosing a quality improvement target.7 It is important to have a good understanding of the burden and severity of the problem. This often requires audit and measuring. For example, although we may think there is a problem with ADR in our endoscopy unit based on a general impression, it is critical to have data to support this suspicion. This is part of a current state analysis (discussed later). It also is important to select a quality-of-care problem that is under you or your group’s direct control. For example, it would be difficult to initiate a quality improvement project aimed at changing the practice of radiology reporting as a gastroenterologist. It is important to pick a problem that is focused and within a narrow scope that is feasible to address and then improve. Consideration of the unintended consequences of an improvement initiative often is overlooked, but needs to be considered because not all that comes out of quality improvement efforts is good. Finally, the likelihood of success of a quality initiative is increased significantly if it can generate momentum and lead to other interventions both within your department and beyond.

There are several specific improvement frameworks that can be used by a team to address a quality-of-care problem and perform a quality improvement project. The framework chosen depends on the type of problem that is being targeted and the training of the individuals on the improvement team. Three of the most commonly used improvement frameworks include the following: 1) Six Sigma; 2) Lean; and 3) Model for Improvement.

AGA Institute

Six Sigma

Six Sigma is focused on improvement by reducing variability.⁸ It is a highly analytic framework relying on statistical analysis and mathematical modeling. It is best suited for projects in which the root cause and contributors to the target problem remain unclear and the aim of the intervention is to reduce variation.

Lean

Lean emphasizes improvement through elimination of waste and classifies all parts of any process as value added and nonvalue added.⁹ It is estimated that 95% of activities in any health care process are nonvalue added and the objective of Lean is to identify opportunities to simplify and create efficiencies. It is best suited for target problems that directly can be observed and mapped out, for example, process of care, flow, and efficiency of an endoscopy unit.

Model for Improvement

The Model for Improvement has been popularized by the Institute for Healthcare Improvement.^10,11 It is well suited for health care teams, and its advantages are its adaptability to many improvement targets and lack of extensive training, consultant support, or statistical training as required by the previous frameworks mentioned earlier. As a result, it is the most commonly used improvement framework.

Using the Model for Improvement

The Model for Improvement is organized around three main questions: 1) What are we trying to accomplish? 2) How will we know that a change is an improvement? and 3) What changes can result in improvement?

Question 1: What are we trying to accomplish?

The first stage using the Model for Improvement is developing a clear project aim. A good aim statement should be specific in defining what measures one is hoping to improve and setting a concrete deadline by which to achieve it.^10,11 It should answer the questions of what the team is trying to improve, by how much, and by what date. It is more effective for the target to be an ambitious, stretch goal to ensure the effort is worth the resources and time that will be invested by the team. Not only does a good aim statement serve as the foundation for the project, but it can redirect the team if the improvement effort is getting off track. In the earlier example of improving ADR, an aim statement could be “to increase the ADR of all endoscopists who perform colonoscopy at your hospital to 25% over a 12-month period.”

Question 2: How will we know that a change is an improvement?

This step involves defining measures that will allow you to understand if changes implemented are impacting the system within which your target problem resides and if this represents an improvement. This usually involves continuous, real-time measurement. Outcome measures are clinically relevant outcomes and are the ultimate goal of what the project team is trying to accomplish. In the example of ADR, this could be the proportion of endoscopists at your institution with an ADR greater than 25%. Process measures are relevant to the system within which you are working and your target problem resides. Typically, the intervention that you implement will have impact that is measurable much earlier by process outcomes than outcomes measures, which are usually a downstream effect. As such, an improvement project still may be a success if it shows improvements in process measures only. For example, the proportion of endoscopists measuring withdrawal time would be a process measure in an intervention aimed at improving ADR. In time, improvement in process measures may translate to improvements in the outcome measure. Balancing measures are indicators of unintended consequences of the project. Not all that comes from an improvement effort is necessarily positive. If improvements in certain process measures come at the cost of harms shown by the balancing measures, such as deterioration in staff satisfaction or increase in time per procedure, the improvement project may not be worth continuing.

Importance of understanding the target problem: Current-state analysis

In contrast to classic enumerative research in which the clinical environment can be well controlled, quality improvement work focuses on sampling and intervening upon a less controlled and dynamic process or system with the intent of improving it.¹⁰ Just as treatment strategies in clinical medicine are based on diagnostic testing, so too in quality improvement work, the strategy of diagnosing the current state allows for linking the root cause of quality problems with solutions that can induce positive change.

Several common diagnostic tools are used to identify root causes of quality and safety issues. These include the following: 1) process mapping, 2) cause-and-effect diagrams, and 3) Pareto charts.

Process mapping

Process maps are tools used to understand the system that is being studied. A process map is a graphic depiction of the flow through a process, which creates a collaborative awareness of the current state and identifies opportunities for improvement. It is important that multiple individuals who have knowledge of the process in question are involved in its creation. Process maps are created by first establishing the start and end of the process. Second, the high-level steps are included. Third, a more detailed set of steps can be included within each of the high-level steps.

Cause-and-effect diagrams

Cause-and-effect diagrams, also known as Ishikawa or fishbone diagrams, are helpful brainstorming tools used to graphically display and explore potential causes of a target problem. They illustrate that there often are many contributing factors to one underlying problem and the relationship between contributing factors. Classic examples of categories include equipment, environment, materials, methods and process, people, and measurement.¹⁰ Figure 1 provides an example of these tools in an effort to improve ADR.
 

To identify the most important contributors to the target problem and thus where to focus improvement efforts, a Pareto chart, a bar graph that places all defects/causes in the order of the frequency in which they occur, is constructed. The x-axis is a list of possible defects (Figure 1). The y-axis is the frequency with which any one defect is occurring, and the third (x-2) axis is the cumulative frequency. In theory, it is expected that there will be a vital few defects that account for 80% of all occurrences (referred to by some as the 80:20 rule).10, 11 Populating this graph requires measurement, which, as discussed earlier, is the key to understanding any problem. Measurement can be accomplished through direct observation/audit, chart review, and/or multivoting.

Question 3: What changes can result in improvement?

Once the improvement team has defined an aim and established its family of measures, it is time to develop and implement an intervention. Rather than investing time and resources into one intervention that may or may not be successful, it is preferable to perform small change cycles in which the intervention is conducted on a small scale, refined, and either repeated or changed. As a result, most quality improvement projects consist of an iterative process. The Model for Improvement defines four steps that allow the improvement team to perform this: Plan, do, study, act (PDSA).^4,10,11 The first two questions listed earlier allowed the improvement team to plan the intervention. The next step, do, involves implementing your project on a small scale, thereby testing your change while collecting continuous measurements. Study involves interpreting your data using both conventional methods and several improvement-specific methods (discussed later) that help answer the question of how will we know that a change is improvement? Finally, act involves making a conclusion about your first PDSA cycle, helping to inform subsequent cycles. This results in a series of small, rapid cycle changes, one building on the next, that lead to implementation of change(s) that ultimately serve to address your improvement problem and your project aim.

A change concept is an approach known to be useful in developing specific changes that result in improvement. Change concepts are used as a starting point to generate change ideas. A number of change concepts spanning nine main categories have been defined by the Associates for Process Improvement,¹⁰ including eliminating waste, improving work flow, managing variation, and designing systems to prevent error. For the purpose of improving ADR, your team may choose a few change concepts and ideas based on the diagnostic work-up. For example, the change concept of designing the system to prevent errors through standardizing withdrawal time for all physicians may lead to an improvement in ADR. This then is linked to the change idea of audible timers placed in endoscopy suites to ensure longer withdrawal times.¹² The impact of this change would be measured and the next cycle would build on these results.
 

Summary and next steps

In this first article of the series, the QI team moved forward with their aim to increase ADR. A root cause analysis was undertaken using multiple diagnostic tools including a fishbone diagram and a Pareto chart. Finally, change ideas were generated based on the earlier-described root causes and established change concepts. The next steps involve undertaking PDSA cycles to test change ideas and monitor for improvement.

References

1. Rex, D.K., Schoenfeld, P.S., Cohen, J. et al. Quality indicators for colonoscopy. Gastrointest Endosc. 2015;81:31-53.

2. Rex, D.K., Bond, J.H., Winawer, S. et al. Quality in the technical performance of colonoscopy and the continuous quality improvement process for colonoscopy: recommendations of the U.S. Multi-Society Task Force on Colorectal Cancer. Am J Gastroenterol. 2002;97:1296-308.

3. Corley, D., Jensen, C.D., Marks, A.R. et al. Adenoma detection rate and risk of colorectal cancer and death. N Engl J Med. 2014;370:1298-306.
4. Kotter, J.P. Leading change. Harvard Business Review Press, Boston; 2012

5. Taitz, J.M., Lee, T.H., and Sequist, T.D. A framework for engaging physicians in quality and safety. BMJ Qual Saf. 2012;21:722-8.

6. Carman, K.L., Dardess, P., Maurer, M. et al. Patient and family engagement: a framework for understanding the elements and developing interventions and policies. Health Aff (Millwood). 2013;33:223-31.

7.Ranji, S.R. and Shojania, S.G. Implementing patient safety interventions in your hospital: what to try and what to avoid. Med Clin North Am. 2008;92:275-93.

8. Antony, J. Six Sigma vs Lean: some perspectives from leading academics and practitioners. Int J Product Perform Manage. 2011;60:185-90.

9. Bercaw, R. Taking improvement from the assembly line to healthcare: the application of lean within the healthcare industry. Taylor and Francis, Boca Raton, FL; 2012

10. Langley, G.J., Nolan, K.M., Nolan, T.W. et al. The improvement guide: a practical approach to enhancing organizational performance. Jossey-Bass, San Francisco; 2009

11. Berwick, D.M. A primer on leading the improvement of systems. BMJ. 1996;312:619-22.

12. Corley, D.A., Jensen, C.D., and Marks, A.R. Can we improve adenoma detection rates? A systematic review of intervention studies. Gastrointest Endosc. 2011;74:656-65.

Dr. Weizman is at the Mount Sinai Hospital Centre for Inflammatory Bowel Disease, and Institute of Health Policy, Management and Evaluation, department of medicine; Dr. Mosko is in the division of gastroenterology, St. Michael’s Hospital, department of medicine, and Institute of Health Policy, Management and Evaluation; Dr. Bollegala is in the division of gastroenterology, Women’s College Hospital, department of medicine; Dr. Bernstein is in the division of gastroenterology, Sunnybrook Health Sciences Centre, department of medicine; Dr. Brahmania is at the Toronto Center for Liver Diseases, division of gastroenterology, University Health Network, department of medicine; Dr. Liu is in the division of gastroenterology, University Health Network, department of medicine; Dr. Steinhart is at Mount Sinai Hospital Centre for Inflammatory Bowel Disease, department of medicine, and Institute of Health Policy, Management and Evaluation; Dr. Silver is in the division of nephrology, St. Michael’s Hospital; Dr. Bell is in the division of internal medicine, Mount Sinai Hospital, department of medicine, and Institute of Health Policy, Management and Evaluation; and Dr. Nguyen is at Mount Sinai Hospital Centre for Inflammatory Bowel Disease, department of medicine and Institute of Health Policy, Management and Evaluation; all are at the University of Toronto. Dr. Steinhart is an advisory board member for Abbvie, Janssen, Takeda, Shire, Allergan, Pfizer, Merck, Ferring, and Pharmascience; has received research grants from Abbvie, Amgen, Genentech, Cellgene, Arena Pharmaceuticals, Red Hill Biopharma, Millenium, Roche, and Centocor; and has received speaking honoraria from Abbvie, Janssen, Takeda, Shire, Pfizer, Merck, and Ferring.

The remaining authors declare no conflicts for this article.

This article by Adam Weizman and colleagues is the first of a three-part series that will provide practical advice for practices that wish to develop a quality initiative. The first article, “Launching a quality improvement initiative” describes the infrastructure, personnel, and structure needed to approach an identified problem within a practice (variability in adenoma detection rates). This case-based approach helps us understand the step-by-step approach needed to reduce variability and improve quality. The authors present a plan (road map) in a straightforward and practical way that seems simple, but if followed carefully, leads to success. These articles are rich in resources and link to state-of-the-art advice.

John I. Allen, MD, MBA, AGAF, Special Section Editor

There has been increasing focus on measuring quality indicators in gastroenterology over the past few years. The adenoma detection rate (ADR) has emerged as one of the most important quality indicators because it is supported by robust clinical evidence.^1-3 With every 1% increase in ADR, a 3% reduction in interval colorectal cancer has been noted.³ As such, an ADR of 25% has been designated as an important quality target for all endoscopists who perform colorectal cancer screening.¹

You work at a community hospital in a large, metropolitan area. Your colleagues in a number of other departments across your hospital have been increasingly interested in quality improvement (QI) and have launched QI interventions, although none in your department. Moreover, there have been reforms in how hospital endoscopy units are funded in your jurisdiction, with a move toward volume-based funding with a quality overlay. In an effort to improve efficiency and better characterize performance, the hospital has been auditing the performance of all endoscopists at your institution over the past year. Among the eight endoscopists who work at your hospital, the overall ADR has been found to be 19%, decreasing to less than the generally accepted benchmark.¹

In response to the results of the audit in your unit, you decide that you would like to develop an initiative to improve your group’s ADR.

Forming a quality improvement team

The first step in any QI project is to establish an improvement team. This working group consists of individuals with specific roles who perform interdependent tasks and share a common goal.⁴ Usually, frontline health care workers who are impacted most by the quality-of-care problem form the foundation of the team. A team lead is identified who will oversee the project. Content experts are also helpful members of the team who may have particular expertise in the clinical domain that will be the focus of the project. In addition, an improvement adviser, an individual with some expertise in QI, is needed on the team. This adviser may be from within your department or from outside. Although they may not possess expertise in the clinical problem you are trying to tackle, they should have skills in QI methodology and process to aid the team. An executive sponsor also needs to be identified. This should be an influential and well-respected individual who holds a senior administrative position at your institution who can help the team overcome barriers and secure resources. Physician engagement is a critical, often-overlooked step in any improvement effort. Regardless of the initiative, physicians continue to have tremendous influence over hospital-based outcomes.⁵ Identifying a physician champion, a prominent and respected physician at your organization to help spread the importance of your efforts and create a burning platform for change, is helpful. It also is valuable to have a patient on the improvement team to provide unique perspectives that only the end user of health care can convey and to ensure that the project is patient centered, as all improvement efforts should be.⁶

Improvement framework

Before starting any improvement effort, there are several important considerations that need to be addressed when choosing a quality improvement target.7 It is important to have a good understanding of the burden and severity of the problem. This often requires audit and measuring. For example, although we may think there is a problem with ADR in our endoscopy unit based on a general impression, it is critical to have data to support this suspicion. This is part of a current state analysis (discussed later). It also is important to select a quality-of-care problem that is under you or your group’s direct control. For example, it would be difficult to initiate a quality improvement project aimed at changing the practice of radiology reporting as a gastroenterologist. It is important to pick a problem that is focused and within a narrow scope that is feasible to address and then improve. Consideration of the unintended consequences of an improvement initiative often is overlooked, but needs to be considered because not all that comes out of quality improvement efforts is good. Finally, the likelihood of success of a quality initiative is increased significantly if it can generate momentum and lead to other interventions both within your department and beyond.

There are several specific improvement frameworks that can be used by a team to address a quality-of-care problem and perform a quality improvement project. The framework chosen depends on the type of problem that is being targeted and the training of the individuals on the improvement team. Three of the most commonly used improvement frameworks include the following: 1) Six Sigma; 2) Lean; and 3) Model for Improvement.

AGA Institute

Six Sigma

Six Sigma is focused on improvement by reducing variability.⁸ It is a highly analytic framework relying on statistical analysis and mathematical modeling. It is best suited for projects in which the root cause and contributors to the target problem remain unclear and the aim of the intervention is to reduce variation.

Lean

Lean emphasizes improvement through elimination of waste and classifies all parts of any process as value added and nonvalue added.⁹ It is estimated that 95% of activities in any health care process are nonvalue added and the objective of Lean is to identify opportunities to simplify and create efficiencies. It is best suited for target problems that directly can be observed and mapped out, for example, process of care, flow, and efficiency of an endoscopy unit.

Model for Improvement

The Model for Improvement has been popularized by the Institute for Healthcare Improvement.^10,11 It is well suited for health care teams, and its advantages are its adaptability to many improvement targets and lack of extensive training, consultant support, or statistical training as required by the previous frameworks mentioned earlier. As a result, it is the most commonly used improvement framework.

Using the Model for Improvement

The Model for Improvement is organized around three main questions: 1) What are we trying to accomplish? 2) How will we know that a change is an improvement? and 3) What changes can result in improvement?

Question 1: What are we trying to accomplish?

The first stage using the Model for Improvement is developing a clear project aim. A good aim statement should be specific in defining what measures one is hoping to improve and setting a concrete deadline by which to achieve it.^10,11 It should answer the questions of what the team is trying to improve, by how much, and by what date. It is more effective for the target to be an ambitious, stretch goal to ensure the effort is worth the resources and time that will be invested by the team. Not only does a good aim statement serve as the foundation for the project, but it can redirect the team if the improvement effort is getting off track. In the earlier example of improving ADR, an aim statement could be “to increase the ADR of all endoscopists who perform colonoscopy at your hospital to 25% over a 12-month period.”

Question 2: How will we know that a change is an improvement?

This step involves defining measures that will allow you to understand if changes implemented are impacting the system within which your target problem resides and if this represents an improvement. This usually involves continuous, real-time measurement. Outcome measures are clinically relevant outcomes and are the ultimate goal of what the project team is trying to accomplish. In the example of ADR, this could be the proportion of endoscopists at your institution with an ADR greater than 25%. Process measures are relevant to the system within which you are working and your target problem resides. Typically, the intervention that you implement will have impact that is measurable much earlier by process outcomes than outcomes measures, which are usually a downstream effect. As such, an improvement project still may be a success if it shows improvements in process measures only. For example, the proportion of endoscopists measuring withdrawal time would be a process measure in an intervention aimed at improving ADR. In time, improvement in process measures may translate to improvements in the outcome measure. Balancing measures are indicators of unintended consequences of the project. Not all that comes from an improvement effort is necessarily positive. If improvements in certain process measures come at the cost of harms shown by the balancing measures, such as deterioration in staff satisfaction or increase in time per procedure, the improvement project may not be worth continuing.

Importance of understanding the target problem: Current-state analysis

In contrast to classic enumerative research in which the clinical environment can be well controlled, quality improvement work focuses on sampling and intervening upon a less controlled and dynamic process or system with the intent of improving it.¹⁰ Just as treatment strategies in clinical medicine are based on diagnostic testing, so too in quality improvement work, the strategy of diagnosing the current state allows for linking the root cause of quality problems with solutions that can induce positive change.

Several common diagnostic tools are used to identify root causes of quality and safety issues. These include the following: 1) process mapping, 2) cause-and-effect diagrams, and 3) Pareto charts.

Process mapping

Process maps are tools used to understand the system that is being studied. A process map is a graphic depiction of the flow through a process, which creates a collaborative awareness of the current state and identifies opportunities for improvement. It is important that multiple individuals who have knowledge of the process in question are involved in its creation. Process maps are created by first establishing the start and end of the process. Second, the high-level steps are included. Third, a more detailed set of steps can be included within each of the high-level steps.

Cause-and-effect diagrams

Cause-and-effect diagrams, also known as Ishikawa or fishbone diagrams, are helpful brainstorming tools used to graphically display and explore potential causes of a target problem. They illustrate that there often are many contributing factors to one underlying problem and the relationship between contributing factors. Classic examples of categories include equipment, environment, materials, methods and process, people, and measurement.¹⁰ Figure 1 provides an example of these tools in an effort to improve ADR.
 

To identify the most important contributors to the target problem and thus where to focus improvement efforts, a Pareto chart, a bar graph that places all defects/causes in the order of the frequency in which they occur, is constructed. The x-axis is a list of possible defects (Figure 1). The y-axis is the frequency with which any one defect is occurring, and the third (x-2) axis is the cumulative frequency. In theory, it is expected that there will be a vital few defects that account for 80% of all occurrences (referred to by some as the 80:20 rule).10, 11 Populating this graph requires measurement, which, as discussed earlier, is the key to understanding any problem. Measurement can be accomplished through direct observation/audit, chart review, and/or multivoting.

Question 3: What changes can result in improvement?

Once the improvement team has defined an aim and established its family of measures, it is time to develop and implement an intervention. Rather than investing time and resources into one intervention that may or may not be successful, it is preferable to perform small change cycles in which the intervention is conducted on a small scale, refined, and either repeated or changed. As a result, most quality improvement projects consist of an iterative process. The Model for Improvement defines four steps that allow the improvement team to perform this: Plan, do, study, act (PDSA).^4,10,11 The first two questions listed earlier allowed the improvement team to plan the intervention. The next step, do, involves implementing your project on a small scale, thereby testing your change while collecting continuous measurements. Study involves interpreting your data using both conventional methods and several improvement-specific methods (discussed later) that help answer the question of how will we know that a change is improvement? Finally, act involves making a conclusion about your first PDSA cycle, helping to inform subsequent cycles. This results in a series of small, rapid cycle changes, one building on the next, that lead to implementation of change(s) that ultimately serve to address your improvement problem and your project aim.

A change concept is an approach known to be useful in developing specific changes that result in improvement. Change concepts are used as a starting point to generate change ideas. A number of change concepts spanning nine main categories have been defined by the Associates for Process Improvement,¹⁰ including eliminating waste, improving work flow, managing variation, and designing systems to prevent error. For the purpose of improving ADR, your team may choose a few change concepts and ideas based on the diagnostic work-up. For example, the change concept of designing the system to prevent errors through standardizing withdrawal time for all physicians may lead to an improvement in ADR. This then is linked to the change idea of audible timers placed in endoscopy suites to ensure longer withdrawal times.¹² The impact of this change would be measured and the next cycle would build on these results.
 

Summary and next steps

In this first article of the series, the QI team moved forward with their aim to increase ADR. A root cause analysis was undertaken using multiple diagnostic tools including a fishbone diagram and a Pareto chart. Finally, change ideas were generated based on the earlier-described root causes and established change concepts. The next steps involve undertaking PDSA cycles to test change ideas and monitor for improvement.

References

1. Rex, D.K., Schoenfeld, P.S., Cohen, J. et al. Quality indicators for colonoscopy. Gastrointest Endosc. 2015;81:31-53.

2. Rex, D.K., Bond, J.H., Winawer, S. et al. Quality in the technical performance of colonoscopy and the continuous quality improvement process for colonoscopy: recommendations of the U.S. Multi-Society Task Force on Colorectal Cancer. Am J Gastroenterol. 2002;97:1296-308.

3. Corley, D., Jensen, C.D., Marks, A.R. et al. Adenoma detection rate and risk of colorectal cancer and death. N Engl J Med. 2014;370:1298-306.
4. Kotter, J.P. Leading change. Harvard Business Review Press, Boston; 2012

5. Taitz, J.M., Lee, T.H., and Sequist, T.D. A framework for engaging physicians in quality and safety. BMJ Qual Saf. 2012;21:722-8.

6. Carman, K.L., Dardess, P., Maurer, M. et al. Patient and family engagement: a framework for understanding the elements and developing interventions and policies. Health Aff (Millwood). 2013;33:223-31.

7.Ranji, S.R. and Shojania, S.G. Implementing patient safety interventions in your hospital: what to try and what to avoid. Med Clin North Am. 2008;92:275-93.

8. Antony, J. Six Sigma vs Lean: some perspectives from leading academics and practitioners. Int J Product Perform Manage. 2011;60:185-90.

9. Bercaw, R. Taking improvement from the assembly line to healthcare: the application of lean within the healthcare industry. Taylor and Francis, Boca Raton, FL; 2012

10. Langley, G.J., Nolan, K.M., Nolan, T.W. et al. The improvement guide: a practical approach to enhancing organizational performance. Jossey-Bass, San Francisco; 2009

11. Berwick, D.M. A primer on leading the improvement of systems. BMJ. 1996;312:619-22.

12. Corley, D.A., Jensen, C.D., and Marks, A.R. Can we improve adenoma detection rates? A systematic review of intervention studies. Gastrointest Endosc. 2011;74:656-65.

Dr. Weizman is at the Mount Sinai Hospital Centre for Inflammatory Bowel Disease, and Institute of Health Policy, Management and Evaluation, department of medicine; Dr. Mosko is in the division of gastroenterology, St. Michael’s Hospital, department of medicine, and Institute of Health Policy, Management and Evaluation; Dr. Bollegala is in the division of gastroenterology, Women’s College Hospital, department of medicine; Dr. Bernstein is in the division of gastroenterology, Sunnybrook Health Sciences Centre, department of medicine; Dr. Brahmania is at the Toronto Center for Liver Diseases, division of gastroenterology, University Health Network, department of medicine; Dr. Liu is in the division of gastroenterology, University Health Network, department of medicine; Dr. Steinhart is at Mount Sinai Hospital Centre for Inflammatory Bowel Disease, department of medicine, and Institute of Health Policy, Management and Evaluation; Dr. Silver is in the division of nephrology, St. Michael’s Hospital; Dr. Bell is in the division of internal medicine, Mount Sinai Hospital, department of medicine, and Institute of Health Policy, Management and Evaluation; and Dr. Nguyen is at Mount Sinai Hospital Centre for Inflammatory Bowel Disease, department of medicine and Institute of Health Policy, Management and Evaluation; all are at the University of Toronto. Dr. Steinhart is an advisory board member for Abbvie, Janssen, Takeda, Shire, Allergan, Pfizer, Merck, Ferring, and Pharmascience; has received research grants from Abbvie, Amgen, Genentech, Cellgene, Arena Pharmaceuticals, Red Hill Biopharma, Millenium, Roche, and Centocor; and has received speaking honoraria from Abbvie, Janssen, Takeda, Shire, Pfizer, Merck, and Ferring.

The remaining authors declare no conflicts for this article.