The Accreditation Council for Graduate Medical Education mandates that internal medicine residents develop technical proficiency in performing central venous catheterization (CVC).1 Likewise in Canada, technical expertise in performing CVC is a specific competency requirement in the training objectives in internal medicine by the Royal College of Physicians and Surgeons of Canada.2 For certification in Internal Medicine, the American Board of Internal Medicine expects its candidates to be competent with respect to knowledge and understanding of the indications, contraindications, recognition, and management of complications of CVC.3 Despite the importance of procedural teaching in medical education, most internal medicine residency programs do not have formal procedural teaching.4 A number of potential barriers to teaching CVC exist. A recent study by the American College of Physicians found that fewer general internists perform procedures, compared with internists in the 1980s.5 With fewer internists performing procedures, there may be fewer educators with adequate confidence in teaching these procedures.6 Second, with increasing recognition of patient safety, there is a growing reluctance of patients to be used as teaching subjects for those who are learning CVC.7 Not surprisingly, residents report low comfort in performing CVC.8

Teaching procedures on simulators offers the benefits of allowing both teaching and evaluation of procedures without jeopardizing patient safety9 and is a welcomed variation from the traditional see 1, do 1, teach 1 approach.10 Despite the theoretical benefits of the use of simulators in medical education, the efficacy of their use in teaching internal medicine residents remains understudied.

The purpose of our study was to evaluate the benefits of using simulator models in teaching CVC to internal medicine residents. We hypothesized that participation in a simulator‐based teaching session on CVC would result in improved knowledge and performance of the procedure and confidence.

Methods

Results

Of the 33 residents in the first‐year internal medicine program, all consented to participate. Thirty participants (91%) completed the study protocol between January and June 2007. The remaining three residents completed the presession assessments and the simulation session, but did not complete the postsession assessments. These were excluded from our analysis. At baseline, 20 participants had rotated through a previous 1‐month intensive care unit (ICU) block, 8 residents had no prior ICU training, and 2 residents had 2 months of ICU training. Table 1 summarizes the residents' baseline CVC experience prior to the simulation training. In general, participants had limited experience with CVC at baseline, having placed very few internal jugular CVCs (median = 4).

Simulation training was associated with an increase in knowledge. Mean scores on multiple‐choice tests increased from 65.7 11.9% to 81.2 10.7 (P < 0.001). Furthermore, simulation training was associated with improvement in CVC performance. Median global average rating score increased from 3.5 (IQR = 34) to 4.5 (IQR = 44.5) (P < 0.001).

The procedural checklist median score increased from 9 (IQR = 69.5) to 9.5 (IQR = 99.5) (P <.001), median time required for completion of CVC decreased from 8 minutes 22 seconds (IQR = 7 minutes 37 seconds to 12 minutes 1 second) to 6 minutes 43 seconds (IQR = 6 minutes to 7 minutes 25 seconds) (P = 0.002). Other performance measures are summarized in Table 2. Last, simulation training was associated with an increase in self‐rated confidence. The median confidence rating (0 lowest, 5 highest) increased from 3 (moderate, IQR = 23) to 4 (good, IQR = 34) (P < 0.001). The overall satisfaction with the CVC simulation course was good (Table 3) and all 30 participants felt this course should continue to be offered.

Discussion

CVC is an important procedural competency for internists to master. Indications for CVC range from measuring of central venous pressure to administration of specific medications or solutions such as inotropic agents and total parental nutrition. Its use is important in the care of those who are acutely or critically ill and those who are chronically ill. CVC is associated with a number of potentially serious mechanical complications, including arterial puncture, pneumothorax, and death.11, 17 Proficiency in CVC insertion is found to be closely related to operator experience: catheter insertion by physicians who have performed 50 or more CVC are one‐half as likely to sustain complications as catheter insertion by less experienced physicians.18 Furthermore, training has been found to be associated with a decrease in risk for catheter associated infection19 and pneumothorax.20 Therefore, cumulative experience from repeated clinical encounters and education on CVC play an important role in optimizing patient safety and reducing potential errors associated with CVC. It is well‐documented that patients are generally reluctant to allow medical trainees to learn or practice procedures on them,7 especially in the early phases of training. Indeed, close to 50% of internal medicine residents in 1 study reported poor comfort in performing CVCs.8

Deficiencies in procedural teaching in internal medicine residency curriculum have long been recognized.4 Over the past 20 years, there has been a decrease in the percentage of internists who place central lines in practice.5 In the United States, more than 5 million central venous catheters are inserted every year.21 Presumably, concurrent with this trend is an increasing reliance on other specialists such as radiologists or surgical colleagues in the placement of CVCs. Without a concerted effort for medical educators in improving the quality of procedural teaching, this alarming trend is unlikely to be halted.

With the advance of modern technology and medical education, patient simulation offers a new platform for procedural teaching to take place: a way for trainees to practice without jeopardizing patient safety, as well as an opportunity for formal teaching and supervision to occur,9, 22 serving as a welcomed departure from the traditional see 1, do 1, teach 1 model to procedural teaching. While there is data to support the role of simulation teaching in the surgical training programs,16, 23 less objective evidence exists to support its use in internal medicine residency training programs.

Our study supports the use of simulation in training first‐year internal medicine residents in CVC. Our formal curriculum on CVC involving simulation was associated with an increase in knowledge of the procedure, observed procedural performance, and subjective confidence. Furthermore, this improvement in knowledge and confidence was maintained at 18 months. Our study has several limitations, such as those inherent to all studies using the prestudy and poststudy design. Improvement in performance under study may be a result of the Hawthorne effect. We are not able to isolate the effect of self‐directed learning from the benefits from training using simulators. Furthermore, improvement in knowledge and confidence may be a function of time rather than training. However, by not comparing our results to that from a historical cohort, we hope to minimize confounders by having participants serve as their own control. Results from our study would be strengthened by the introduction of a control group and a randomized design. Future study should include a randomized, controlled crossover educational trial. The second limitation of our study is the low interobserver agreement between video raters, indicating only fair to moderate agreement. Although our video raters were trained senior medical residents and faculty members who frequently supervise and evaluate procedures, we did not specifically train them for the purposes of scoring videotaped performances. Further faculty development is needed. Overall, the magnitudes of the interobserver differences were small (mean global rating scale between raters differed by 1 0.9), and may not translate into clinically important differences. Third, competence on performing CVC on simulators has not been directly correlated with clinical competence on real‐life patients, although a previous study on surgical interns suggested that simulator training was associated with clinical competence on real patients.16 Future studies are warranted to investigate the association between educational benefits from simulation and actual error reductions from CVC during patient encounters. Fourth, our study did not examine residents in the use of ultrasound during simulator training. Use of ultrasound devices has been shown to decrease complication rates of CVC insertion24 and increase rates of success,25 and is recommended as the preferred method for insertion of elective internal jugular CVCs.26 However, because our course was introduced during first‐year residency, when majority of the residents have performed fewer than 5 internal jugular CVCs prior to their simulation training session, the objective of our introductory course was to first ensure technical competency by landmark techniques. Once learners have mastered the technical aspect of line insertion, ultrasound‐guided techniques were introduced later in the residency training curriculum. Fifth, only 48% of participants completed the knowledge assessment at 18 months. Selection bias may be present in our long‐term data. However, baseline comparisons between those who completed the 18 month assessment vs. those who did not demonstrated no obvious differences between the 2 groups. Performance was not examined in our long‐term data. Demonstration of retention of psychomotor skills would be reassuring. However, by 18 months, many of our learners had completed a number of procedurally‐intensive rotations. Therefore, because retention of skills could not be reliably attributed to our initial training in the simulator laboratory, the participants were not retested in their performance of the CVC. Future study should include a control group that undergoes delayed training at 18 months to best assess the effect of simulation training on performance. Sixth, only overall confidence in performing CVC was captured. Confidence in each venous access site pretraining and posttraining was not assessed and warrants further study. Last, our study did not document overall costs of delivering the program, including cost of the simulator, facility, supplies, and stipends for teaching faculty. One full‐time equivalent (FTE) staff has previously been estimated to be a requirement for the success of the program.9 Our program required 3 medical educators, all of whom are clinical faculty members, as well as the help of 4 chief medical residents.

Despite our study's limitations, our study has a number of strengths. To our knowledge, our study is the first of its kind examining the use of simulators on CVC procedural performance, with the use of video‐recording, in an independent and blinded manner. Blinding should minimize bias associated with assessments of resident competency by evaluators who may be aware of residents' previous experience. Second, we were able to recruit all of our first‐year residents for our pretests and posttests, although 3 did not complete the study in its entirety due to on‐call or post‐call schedules. We were able to obtain detailed assessment of knowledge, performance, and subjective confidence before and after the simulation training on our cohort of residents and were able to demonstrate an improvement in knowledge, skills, and confidence. Furthermore, we were able to demonstrate that benefits in knowledge and confidence were retained at 18 months. Last, we readily implemented our formal CVC curriculum into the existing residency training schedule. It requires a single, 2‐hour session in addition to brief pretest and posttest surveys. It requires minimal equipment, using only 2 CVC mannequins, 4 to 6 central line kits, and an ultrasound machine. The simulation training adheres to and incorporates key elements that have been previously identified as effective teaching tools for simulators: feedback, repetitive practice, and curriculum integration.27 With increasing interest in the use of medical simulation for training and competency assessment in medical education,10 our study provides evidence in support of its use in improving knowledge, skills, and confidence. One of the most important goals of using simulators as an educational tool is to improve patient safety. While assessing for whether or not training on simulators lead to improved patient outcomes is beyond the scope of our study, this question merits further study and should be the focus of future studies.

Conclusions

Training on CVC simulators was associated with an improvement in performance, and increase in knowledge assessment scores and self‐reported confidence in internal medicine residents. Improvement in knowledge and confidence was maintained at 18 months.

The Accreditation Council for Graduate Medical Education mandates that internal medicine residents develop technical proficiency in performing central venous catheterization (CVC).1 Likewise in Canada, technical expertise in performing CVC is a specific competency requirement in the training objectives in internal medicine by the Royal College of Physicians and Surgeons of Canada.2 For certification in Internal Medicine, the American Board of Internal Medicine expects its candidates to be competent with respect to knowledge and understanding of the indications, contraindications, recognition, and management of complications of CVC.3 Despite the importance of procedural teaching in medical education, most internal medicine residency programs do not have formal procedural teaching.4 A number of potential barriers to teaching CVC exist. A recent study by the American College of Physicians found that fewer general internists perform procedures, compared with internists in the 1980s.5 With fewer internists performing procedures, there may be fewer educators with adequate confidence in teaching these procedures.6 Second, with increasing recognition of patient safety, there is a growing reluctance of patients to be used as teaching subjects for those who are learning CVC.7 Not surprisingly, residents report low comfort in performing CVC.8

Teaching procedures on simulators offers the benefits of allowing both teaching and evaluation of procedures without jeopardizing patient safety9 and is a welcomed variation from the traditional see 1, do 1, teach 1 approach.10 Despite the theoretical benefits of the use of simulators in medical education, the efficacy of their use in teaching internal medicine residents remains understudied.

The purpose of our study was to evaluate the benefits of using simulator models in teaching CVC to internal medicine residents. We hypothesized that participation in a simulator‐based teaching session on CVC would result in improved knowledge and performance of the procedure and confidence.

Methods

Results

Of the 33 residents in the first‐year internal medicine program, all consented to participate. Thirty participants (91%) completed the study protocol between January and June 2007. The remaining three residents completed the presession assessments and the simulation session, but did not complete the postsession assessments. These were excluded from our analysis. At baseline, 20 participants had rotated through a previous 1‐month intensive care unit (ICU) block, 8 residents had no prior ICU training, and 2 residents had 2 months of ICU training. Table 1 summarizes the residents' baseline CVC experience prior to the simulation training. In general, participants had limited experience with CVC at baseline, having placed very few internal jugular CVCs (median = 4).

Simulation training was associated with an increase in knowledge. Mean scores on multiple‐choice tests increased from 65.7 11.9% to 81.2 10.7 (P < 0.001). Furthermore, simulation training was associated with improvement in CVC performance. Median global average rating score increased from 3.5 (IQR = 34) to 4.5 (IQR = 44.5) (P < 0.001).

The procedural checklist median score increased from 9 (IQR = 69.5) to 9.5 (IQR = 99.5) (P <.001), median time required for completion of CVC decreased from 8 minutes 22 seconds (IQR = 7 minutes 37 seconds to 12 minutes 1 second) to 6 minutes 43 seconds (IQR = 6 minutes to 7 minutes 25 seconds) (P = 0.002). Other performance measures are summarized in Table 2. Last, simulation training was associated with an increase in self‐rated confidence. The median confidence rating (0 lowest, 5 highest) increased from 3 (moderate, IQR = 23) to 4 (good, IQR = 34) (P < 0.001). The overall satisfaction with the CVC simulation course was good (Table 3) and all 30 participants felt this course should continue to be offered.

Discussion

CVC is an important procedural competency for internists to master. Indications for CVC range from measuring of central venous pressure to administration of specific medications or solutions such as inotropic agents and total parental nutrition. Its use is important in the care of those who are acutely or critically ill and those who are chronically ill. CVC is associated with a number of potentially serious mechanical complications, including arterial puncture, pneumothorax, and death.11, 17 Proficiency in CVC insertion is found to be closely related to operator experience: catheter insertion by physicians who have performed 50 or more CVC are one‐half as likely to sustain complications as catheter insertion by less experienced physicians.18 Furthermore, training has been found to be associated with a decrease in risk for catheter associated infection19 and pneumothorax.20 Therefore, cumulative experience from repeated clinical encounters and education on CVC play an important role in optimizing patient safety and reducing potential errors associated with CVC. It is well‐documented that patients are generally reluctant to allow medical trainees to learn or practice procedures on them,7 especially in the early phases of training. Indeed, close to 50% of internal medicine residents in 1 study reported poor comfort in performing CVCs.8

Deficiencies in procedural teaching in internal medicine residency curriculum have long been recognized.4 Over the past 20 years, there has been a decrease in the percentage of internists who place central lines in practice.5 In the United States, more than 5 million central venous catheters are inserted every year.21 Presumably, concurrent with this trend is an increasing reliance on other specialists such as radiologists or surgical colleagues in the placement of CVCs. Without a concerted effort for medical educators in improving the quality of procedural teaching, this alarming trend is unlikely to be halted.

With the advance of modern technology and medical education, patient simulation offers a new platform for procedural teaching to take place: a way for trainees to practice without jeopardizing patient safety, as well as an opportunity for formal teaching and supervision to occur,9, 22 serving as a welcomed departure from the traditional see 1, do 1, teach 1 model to procedural teaching. While there is data to support the role of simulation teaching in the surgical training programs,16, 23 less objective evidence exists to support its use in internal medicine residency training programs.

Our study supports the use of simulation in training first‐year internal medicine residents in CVC. Our formal curriculum on CVC involving simulation was associated with an increase in knowledge of the procedure, observed procedural performance, and subjective confidence. Furthermore, this improvement in knowledge and confidence was maintained at 18 months. Our study has several limitations, such as those inherent to all studies using the prestudy and poststudy design. Improvement in performance under study may be a result of the Hawthorne effect. We are not able to isolate the effect of self‐directed learning from the benefits from training using simulators. Furthermore, improvement in knowledge and confidence may be a function of time rather than training. However, by not comparing our results to that from a historical cohort, we hope to minimize confounders by having participants serve as their own control. Results from our study would be strengthened by the introduction of a control group and a randomized design. Future study should include a randomized, controlled crossover educational trial. The second limitation of our study is the low interobserver agreement between video raters, indicating only fair to moderate agreement. Although our video raters were trained senior medical residents and faculty members who frequently supervise and evaluate procedures, we did not specifically train them for the purposes of scoring videotaped performances. Further faculty development is needed. Overall, the magnitudes of the interobserver differences were small (mean global rating scale between raters differed by 1 0.9), and may not translate into clinically important differences. Third, competence on performing CVC on simulators has not been directly correlated with clinical competence on real‐life patients, although a previous study on surgical interns suggested that simulator training was associated with clinical competence on real patients.16 Future studies are warranted to investigate the association between educational benefits from simulation and actual error reductions from CVC during patient encounters. Fourth, our study did not examine residents in the use of ultrasound during simulator training. Use of ultrasound devices has been shown to decrease complication rates of CVC insertion24 and increase rates of success,25 and is recommended as the preferred method for insertion of elective internal jugular CVCs.26 However, because our course was introduced during first‐year residency, when majority of the residents have performed fewer than 5 internal jugular CVCs prior to their simulation training session, the objective of our introductory course was to first ensure technical competency by landmark techniques. Once learners have mastered the technical aspect of line insertion, ultrasound‐guided techniques were introduced later in the residency training curriculum. Fifth, only 48% of participants completed the knowledge assessment at 18 months. Selection bias may be present in our long‐term data. However, baseline comparisons between those who completed the 18 month assessment vs. those who did not demonstrated no obvious differences between the 2 groups. Performance was not examined in our long‐term data. Demonstration of retention of psychomotor skills would be reassuring. However, by 18 months, many of our learners had completed a number of procedurally‐intensive rotations. Therefore, because retention of skills could not be reliably attributed to our initial training in the simulator laboratory, the participants were not retested in their performance of the CVC. Future study should include a control group that undergoes delayed training at 18 months to best assess the effect of simulation training on performance. Sixth, only overall confidence in performing CVC was captured. Confidence in each venous access site pretraining and posttraining was not assessed and warrants further study. Last, our study did not document overall costs of delivering the program, including cost of the simulator, facility, supplies, and stipends for teaching faculty. One full‐time equivalent (FTE) staff has previously been estimated to be a requirement for the success of the program.9 Our program required 3 medical educators, all of whom are clinical faculty members, as well as the help of 4 chief medical residents.

Despite our study's limitations, our study has a number of strengths. To our knowledge, our study is the first of its kind examining the use of simulators on CVC procedural performance, with the use of video‐recording, in an independent and blinded manner. Blinding should minimize bias associated with assessments of resident competency by evaluators who may be aware of residents' previous experience. Second, we were able to recruit all of our first‐year residents for our pretests and posttests, although 3 did not complete the study in its entirety due to on‐call or post‐call schedules. We were able to obtain detailed assessment of knowledge, performance, and subjective confidence before and after the simulation training on our cohort of residents and were able to demonstrate an improvement in knowledge, skills, and confidence. Furthermore, we were able to demonstrate that benefits in knowledge and confidence were retained at 18 months. Last, we readily implemented our formal CVC curriculum into the existing residency training schedule. It requires a single, 2‐hour session in addition to brief pretest and posttest surveys. It requires minimal equipment, using only 2 CVC mannequins, 4 to 6 central line kits, and an ultrasound machine. The simulation training adheres to and incorporates key elements that have been previously identified as effective teaching tools for simulators: feedback, repetitive practice, and curriculum integration.27 With increasing interest in the use of medical simulation for training and competency assessment in medical education,10 our study provides evidence in support of its use in improving knowledge, skills, and confidence. One of the most important goals of using simulators as an educational tool is to improve patient safety. While assessing for whether or not training on simulators lead to improved patient outcomes is beyond the scope of our study, this question merits further study and should be the focus of future studies.

Conclusions

Training on CVC simulators was associated with an improvement in performance, and increase in knowledge assessment scores and self‐reported confidence in internal medicine residents. Improvement in knowledge and confidence was maintained at 18 months.

The Accreditation Council for Graduate Medical Education mandates that internal medicine residents develop technical proficiency in performing central venous catheterization (CVC).1 Likewise in Canada, technical expertise in performing CVC is a specific competency requirement in the training objectives in internal medicine by the Royal College of Physicians and Surgeons of Canada.2 For certification in Internal Medicine, the American Board of Internal Medicine expects its candidates to be competent with respect to knowledge and understanding of the indications, contraindications, recognition, and management of complications of CVC.3 Despite the importance of procedural teaching in medical education, most internal medicine residency programs do not have formal procedural teaching.4 A number of potential barriers to teaching CVC exist. A recent study by the American College of Physicians found that fewer general internists perform procedures, compared with internists in the 1980s.5 With fewer internists performing procedures, there may be fewer educators with adequate confidence in teaching these procedures.6 Second, with increasing recognition of patient safety, there is a growing reluctance of patients to be used as teaching subjects for those who are learning CVC.7 Not surprisingly, residents report low comfort in performing CVC.8

Teaching procedures on simulators offers the benefits of allowing both teaching and evaluation of procedures without jeopardizing patient safety9 and is a welcomed variation from the traditional see 1, do 1, teach 1 approach.10 Despite the theoretical benefits of the use of simulators in medical education, the efficacy of their use in teaching internal medicine residents remains understudied.

The purpose of our study was to evaluate the benefits of using simulator models in teaching CVC to internal medicine residents. We hypothesized that participation in a simulator‐based teaching session on CVC would result in improved knowledge and performance of the procedure and confidence.

Methods

Results

Of the 33 residents in the first‐year internal medicine program, all consented to participate. Thirty participants (91%) completed the study protocol between January and June 2007. The remaining three residents completed the presession assessments and the simulation session, but did not complete the postsession assessments. These were excluded from our analysis. At baseline, 20 participants had rotated through a previous 1‐month intensive care unit (ICU) block, 8 residents had no prior ICU training, and 2 residents had 2 months of ICU training. Table 1 summarizes the residents' baseline CVC experience prior to the simulation training. In general, participants had limited experience with CVC at baseline, having placed very few internal jugular CVCs (median = 4).

Simulation training was associated with an increase in knowledge. Mean scores on multiple‐choice tests increased from 65.7 11.9% to 81.2 10.7 (P < 0.001). Furthermore, simulation training was associated with improvement in CVC performance. Median global average rating score increased from 3.5 (IQR = 34) to 4.5 (IQR = 44.5) (P < 0.001).

The procedural checklist median score increased from 9 (IQR = 69.5) to 9.5 (IQR = 99.5) (P <.001), median time required for completion of CVC decreased from 8 minutes 22 seconds (IQR = 7 minutes 37 seconds to 12 minutes 1 second) to 6 minutes 43 seconds (IQR = 6 minutes to 7 minutes 25 seconds) (P = 0.002). Other performance measures are summarized in Table 2. Last, simulation training was associated with an increase in self‐rated confidence. The median confidence rating (0 lowest, 5 highest) increased from 3 (moderate, IQR = 23) to 4 (good, IQR = 34) (P < 0.001). The overall satisfaction with the CVC simulation course was good (Table 3) and all 30 participants felt this course should continue to be offered.

Discussion

CVC is an important procedural competency for internists to master. Indications for CVC range from measuring of central venous pressure to administration of specific medications or solutions such as inotropic agents and total parental nutrition. Its use is important in the care of those who are acutely or critically ill and those who are chronically ill. CVC is associated with a number of potentially serious mechanical complications, including arterial puncture, pneumothorax, and death.11, 17 Proficiency in CVC insertion is found to be closely related to operator experience: catheter insertion by physicians who have performed 50 or more CVC are one‐half as likely to sustain complications as catheter insertion by less experienced physicians.18 Furthermore, training has been found to be associated with a decrease in risk for catheter associated infection19 and pneumothorax.20 Therefore, cumulative experience from repeated clinical encounters and education on CVC play an important role in optimizing patient safety and reducing potential errors associated with CVC. It is well‐documented that patients are generally reluctant to allow medical trainees to learn or practice procedures on them,7 especially in the early phases of training. Indeed, close to 50% of internal medicine residents in 1 study reported poor comfort in performing CVCs.8

Deficiencies in procedural teaching in internal medicine residency curriculum have long been recognized.4 Over the past 20 years, there has been a decrease in the percentage of internists who place central lines in practice.5 In the United States, more than 5 million central venous catheters are inserted every year.21 Presumably, concurrent with this trend is an increasing reliance on other specialists such as radiologists or surgical colleagues in the placement of CVCs. Without a concerted effort for medical educators in improving the quality of procedural teaching, this alarming trend is unlikely to be halted.

With the advance of modern technology and medical education, patient simulation offers a new platform for procedural teaching to take place: a way for trainees to practice without jeopardizing patient safety, as well as an opportunity for formal teaching and supervision to occur,9, 22 serving as a welcomed departure from the traditional see 1, do 1, teach 1 model to procedural teaching. While there is data to support the role of simulation teaching in the surgical training programs,16, 23 less objective evidence exists to support its use in internal medicine residency training programs.

Our study supports the use of simulation in training first‐year internal medicine residents in CVC. Our formal curriculum on CVC involving simulation was associated with an increase in knowledge of the procedure, observed procedural performance, and subjective confidence. Furthermore, this improvement in knowledge and confidence was maintained at 18 months. Our study has several limitations, such as those inherent to all studies using the prestudy and poststudy design. Improvement in performance under study may be a result of the Hawthorne effect. We are not able to isolate the effect of self‐directed learning from the benefits from training using simulators. Furthermore, improvement in knowledge and confidence may be a function of time rather than training. However, by not comparing our results to that from a historical cohort, we hope to minimize confounders by having participants serve as their own control. Results from our study would be strengthened by the introduction of a control group and a randomized design. Future study should include a randomized, controlled crossover educational trial. The second limitation of our study is the low interobserver agreement between video raters, indicating only fair to moderate agreement. Although our video raters were trained senior medical residents and faculty members who frequently supervise and evaluate procedures, we did not specifically train them for the purposes of scoring videotaped performances. Further faculty development is needed. Overall, the magnitudes of the interobserver differences were small (mean global rating scale between raters differed by 1 0.9), and may not translate into clinically important differences. Third, competence on performing CVC on simulators has not been directly correlated with clinical competence on real‐life patients, although a previous study on surgical interns suggested that simulator training was associated with clinical competence on real patients.16 Future studies are warranted to investigate the association between educational benefits from simulation and actual error reductions from CVC during patient encounters. Fourth, our study did not examine residents in the use of ultrasound during simulator training. Use of ultrasound devices has been shown to decrease complication rates of CVC insertion24 and increase rates of success,25 and is recommended as the preferred method for insertion of elective internal jugular CVCs.26 However, because our course was introduced during first‐year residency, when majority of the residents have performed fewer than 5 internal jugular CVCs prior to their simulation training session, the objective of our introductory course was to first ensure technical competency by landmark techniques. Once learners have mastered the technical aspect of line insertion, ultrasound‐guided techniques were introduced later in the residency training curriculum. Fifth, only 48% of participants completed the knowledge assessment at 18 months. Selection bias may be present in our long‐term data. However, baseline comparisons between those who completed the 18 month assessment vs. those who did not demonstrated no obvious differences between the 2 groups. Performance was not examined in our long‐term data. Demonstration of retention of psychomotor skills would be reassuring. However, by 18 months, many of our learners had completed a number of procedurally‐intensive rotations. Therefore, because retention of skills could not be reliably attributed to our initial training in the simulator laboratory, the participants were not retested in their performance of the CVC. Future study should include a control group that undergoes delayed training at 18 months to best assess the effect of simulation training on performance. Sixth, only overall confidence in performing CVC was captured. Confidence in each venous access site pretraining and posttraining was not assessed and warrants further study. Last, our study did not document overall costs of delivering the program, including cost of the simulator, facility, supplies, and stipends for teaching faculty. One full‐time equivalent (FTE) staff has previously been estimated to be a requirement for the success of the program.9 Our program required 3 medical educators, all of whom are clinical faculty members, as well as the help of 4 chief medical residents.

Despite our study's limitations, our study has a number of strengths. To our knowledge, our study is the first of its kind examining the use of simulators on CVC procedural performance, with the use of video‐recording, in an independent and blinded manner. Blinding should minimize bias associated with assessments of resident competency by evaluators who may be aware of residents' previous experience. Second, we were able to recruit all of our first‐year residents for our pretests and posttests, although 3 did not complete the study in its entirety due to on‐call or post‐call schedules. We were able to obtain detailed assessment of knowledge, performance, and subjective confidence before and after the simulation training on our cohort of residents and were able to demonstrate an improvement in knowledge, skills, and confidence. Furthermore, we were able to demonstrate that benefits in knowledge and confidence were retained at 18 months. Last, we readily implemented our formal CVC curriculum into the existing residency training schedule. It requires a single, 2‐hour session in addition to brief pretest and posttest surveys. It requires minimal equipment, using only 2 CVC mannequins, 4 to 6 central line kits, and an ultrasound machine. The simulation training adheres to and incorporates key elements that have been previously identified as effective teaching tools for simulators: feedback, repetitive practice, and curriculum integration.27 With increasing interest in the use of medical simulation for training and competency assessment in medical education,10 our study provides evidence in support of its use in improving knowledge, skills, and confidence. One of the most important goals of using simulators as an educational tool is to improve patient safety. While assessing for whether or not training on simulators lead to improved patient outcomes is beyond the scope of our study, this question merits further study and should be the focus of future studies.

Conclusions

Training on CVC simulators was associated with an improvement in performance, and increase in knowledge assessment scores and self‐reported confidence in internal medicine residents. Improvement in knowledge and confidence was maintained at 18 months.

The benefits of glycemic control include decreased patient morbidity, mortality, length of stay, and reduced hospital costs. In 2004, the American College of Endocrinology (ACE) issued glycemic guidelines for non‐critical‐care units (fasting glucose <110 mg/dL, nonfasting glucose <180 mg/dL).1 A comprehensive review of inpatient glycemic management called for development and evaluation of inpatient programs and tools.2 The 2006 ACE/American Diabetes Association (ADA) Statement on Inpatient Diabetes and Glycemic Control identified key components of an inpatient glycemic control program as: (1) solid administrative support; (2) a multidisciplinary committee; (3) assessment of current processes, care, and barriers; (4) development and implementation of order sets, protocols, policies, and educational efforts; and (5) metrics for evaluation.3

In 2003, Harborview Medical Center (HMC) formed a multidisciplinary committee to institute a Glycemic Control Program. The early goals were to decrease the use of sliding‐scale insulin, increase the appropriate use of basal and prandial insulin, and to avoid hypoglycemia. Here we report our program design and trends in physician insulin ordering from 2003 through 2006.

Patients and Methods

Program Description

Since 2003, the multidisciplinary committeephysicians, nurses, pharmacy representatives, and dietary and administrative representativeshas directed the development of the Glycemic Control Program with support from hospital administration and the Department of Quality Improvement. Funding for this program has been provided by the hospital based on the prominence of glycemic control among quality and safety measures, a projected decrease in costs, and the high incidence of diabetes in our patient population. Figure 1 outlines the program's key interventions.

Figure 1

First, a Subcutaneous Insulin Order Form was released for elective use in May 2004 (Figure 2). This form incorporated the 3 components of quality insulin ordering (basal, scheduled prandial, and prandial correction dosing) and provides prompts and education. A Diabetes Nurse Specialist trained nursing staff on the use of the form.

Figure 2

Second, we developed an automated daily data report identifying patients with out‐of‐range glucose levels defined as having any single glucose readings <60 mg/dL or any 2 readings 180 mg/dL within the prior 24 hours. In February 2006, this daily report became available to the clinicians on the committee.

Third, the Glycemic Control Program recruited a full‐time clinical Advanced Registered Nurse Practitioner (ARNP) and part‐time supervising physician to provide directed intervention and education for patients and medical personnel. Since August 2006, the ARNP has reviewed the out‐of‐range report daily, performs assessments, refines insulin orders, and educates clinicians. The assessments include chart review (of history and glycemic control), discussion with primary physician and nurse (and often the dietician), and interview of the patient and/or family. This leads to development and implementation of a glycemic control plan. Clinician education is performed both as direct education of the primary physician at the time of intervention and as didactic sessions.

Results

Physician Insulin Ordering

Ordering of both short‐acting and basal insulin increased (Figure 3). The ratio of short‐acting to basal orders decreased from 3.36 (1668/496) in 2003 to 1.97 (2226/1128) in 2006.

Figure 3

Chart review of the 100 randomly selected dysglycemic patients revealed increased ordering of prandial correction dosing from 8% of patients in 2003 to 32% in 2006. Yet, only 1 patient in 2003 and only 2 in 2006 had scheduled prandial. Ordering of sliding scale insulin fell from 16% in 2003 to 4% in 2006.

Glycemic Control Outcomes

The percentage of dysglycemic patients with hyperglycemia ranged from 19 to 24 without significant decline over the 4 years (Figure 4A). The percentage of hypoglycemic dysglycemic patients was increasing from 2003 to 2004, but in the years following the interventions (2005 through 2006) this declined significantly (P = 0.003; Figure 4B). On average, the observed LOS was higher for dysglycemic vs. euglycemic patients (mean SD days: 9.4 12.2 and 5.8 8.5, respectively). The mean observed to expected mortality ratio was 0.45 0.08 and 0.44 0.17 for the dysglycemic and euglycemic patients, respectively. Over the 4 years no statistically significant change in observed LOS or adjusted mortality was found (data not shown).

Figure 4

Conclusions

HMC, a safety net hospital with the highest UHC expected mortality of 131 hospitals nationwide, has demonstrated early successes in building its Glycemic Control Program, including: (1) decreased prescription of sliding scale; (2) a marked increase in prescription of basal insulin; and (3) significantly decreasing hypoglycemic events subsequent to the interventions. The decreased sliding scale and increased overall ordering of insulin could reflect increased awareness brought internationally through the literature and locally through our program. Two distinctive aspects of HMC's Glycemic Control Program, when compared to others,68 include: (1) the daily use of real‐time data to identify and target patients with out‐of‐range glucose levels; and (2) the coverage of all non‐critical‐care floors with a single clinician.

In 2003 and 2004, the increasing hypoglycemia we observed paralleled the international focus on aggressively treating hyperglycemia in the acute care setting. We observed a significant decrease in hypoglycemia in 2005 and 2006 that could be attributed to the education provided by the Glycemic Control Program and 2 features on the subcutaneous insulin order set: the prominent hypoglycemia protocol and the order hold prandial insulin if the patient cannot eat. These are similar features identified in a report on preventing hospital hypoglycemia.9 Additionally, hypoglycemia may have decreased secondary to the emphasis on not using short‐acting insulin at bedtime.

Despite increased and improved insulin ordering, we did not observe a significant change in the percent of dysglycemic patients with 2 glucose levels 180 mg/dL. In our program patients are identified for intervention after their glucose levels are out‐of‐range. To better evaluate the impact of our interventions on the glycemic control of each patient, we plan to analyze the glucose levels in the days following identification of patients. Alternatively, we could provide intervention to all patients with dysglycemia rather than waiting for glucoses to be out‐of‐range. Though this approach would require greater resources than the single clinician we currently employ.

Our early experience highlights areas for future evaluation and intervention. First, the lack of scheduled prandial insulin and that less than one‐third of dysglycemic patients have basal insulin ordered underscore a continued need to target quality insulin ordering to include all componentsbasal, scheduled prandial, and prandial correction. Second, while the daily report is a good rudimentary identification tool for at‐risk patients, it offers limited information as to the impact of our clinical intervention. Thus, refined evaluative metrics need be developed to prospectively assess the course of glycemic control for patients.

We acknowledge the limitations of this study. First, our most involved interventionthe addition of the clinical intervention teamcame only 6 months before the end of the study period. Second, this is an observational retrospective analysis and cannot distinguish confounders, such as physician preferences and decisions, that not easily quantified or controlled for. Third, our definition of dysglycemic incorporated 41% of non‐critical‐care patients, possibly reflecting too broad a definition.

In summary, we have described an inpatient Glycemic Control Program that relies on real‐time data to identify patients in need of intervention. Early in our program we observed improved insulin ordering quality and decreased rates of hypoglycemia. Future steps include evaluating the impact of our clinical intervention team and further refining glycemic control metrics to prospectively identify patients at risk for hyper‐ and hypoglycemia.

The benefits of glycemic control include decreased patient morbidity, mortality, length of stay, and reduced hospital costs. In 2004, the American College of Endocrinology (ACE) issued glycemic guidelines for non‐critical‐care units (fasting glucose <110 mg/dL, nonfasting glucose <180 mg/dL).1 A comprehensive review of inpatient glycemic management called for development and evaluation of inpatient programs and tools.2 The 2006 ACE/American Diabetes Association (ADA) Statement on Inpatient Diabetes and Glycemic Control identified key components of an inpatient glycemic control program as: (1) solid administrative support; (2) a multidisciplinary committee; (3) assessment of current processes, care, and barriers; (4) development and implementation of order sets, protocols, policies, and educational efforts; and (5) metrics for evaluation.3

In 2003, Harborview Medical Center (HMC) formed a multidisciplinary committee to institute a Glycemic Control Program. The early goals were to decrease the use of sliding‐scale insulin, increase the appropriate use of basal and prandial insulin, and to avoid hypoglycemia. Here we report our program design and trends in physician insulin ordering from 2003 through 2006.

Patients and Methods

Program Description

Since 2003, the multidisciplinary committeephysicians, nurses, pharmacy representatives, and dietary and administrative representativeshas directed the development of the Glycemic Control Program with support from hospital administration and the Department of Quality Improvement. Funding for this program has been provided by the hospital based on the prominence of glycemic control among quality and safety measures, a projected decrease in costs, and the high incidence of diabetes in our patient population. Figure 1 outlines the program's key interventions.

Figure 1

First, a Subcutaneous Insulin Order Form was released for elective use in May 2004 (Figure 2). This form incorporated the 3 components of quality insulin ordering (basal, scheduled prandial, and prandial correction dosing) and provides prompts and education. A Diabetes Nurse Specialist trained nursing staff on the use of the form.

Figure 2

Second, we developed an automated daily data report identifying patients with out‐of‐range glucose levels defined as having any single glucose readings <60 mg/dL or any 2 readings 180 mg/dL within the prior 24 hours. In February 2006, this daily report became available to the clinicians on the committee.

Third, the Glycemic Control Program recruited a full‐time clinical Advanced Registered Nurse Practitioner (ARNP) and part‐time supervising physician to provide directed intervention and education for patients and medical personnel. Since August 2006, the ARNP has reviewed the out‐of‐range report daily, performs assessments, refines insulin orders, and educates clinicians. The assessments include chart review (of history and glycemic control), discussion with primary physician and nurse (and often the dietician), and interview of the patient and/or family. This leads to development and implementation of a glycemic control plan. Clinician education is performed both as direct education of the primary physician at the time of intervention and as didactic sessions.

Results

Physician Insulin Ordering

Ordering of both short‐acting and basal insulin increased (Figure 3). The ratio of short‐acting to basal orders decreased from 3.36 (1668/496) in 2003 to 1.97 (2226/1128) in 2006.

Figure 3

Chart review of the 100 randomly selected dysglycemic patients revealed increased ordering of prandial correction dosing from 8% of patients in 2003 to 32% in 2006. Yet, only 1 patient in 2003 and only 2 in 2006 had scheduled prandial. Ordering of sliding scale insulin fell from 16% in 2003 to 4% in 2006.

Glycemic Control Outcomes

The percentage of dysglycemic patients with hyperglycemia ranged from 19 to 24 without significant decline over the 4 years (Figure 4A). The percentage of hypoglycemic dysglycemic patients was increasing from 2003 to 2004, but in the years following the interventions (2005 through 2006) this declined significantly (P = 0.003; Figure 4B). On average, the observed LOS was higher for dysglycemic vs. euglycemic patients (mean SD days: 9.4 12.2 and 5.8 8.5, respectively). The mean observed to expected mortality ratio was 0.45 0.08 and 0.44 0.17 for the dysglycemic and euglycemic patients, respectively. Over the 4 years no statistically significant change in observed LOS or adjusted mortality was found (data not shown).

Figure 4

Conclusions

HMC, a safety net hospital with the highest UHC expected mortality of 131 hospitals nationwide, has demonstrated early successes in building its Glycemic Control Program, including: (1) decreased prescription of sliding scale; (2) a marked increase in prescription of basal insulin; and (3) significantly decreasing hypoglycemic events subsequent to the interventions. The decreased sliding scale and increased overall ordering of insulin could reflect increased awareness brought internationally through the literature and locally through our program. Two distinctive aspects of HMC's Glycemic Control Program, when compared to others,68 include: (1) the daily use of real‐time data to identify and target patients with out‐of‐range glucose levels; and (2) the coverage of all non‐critical‐care floors with a single clinician.

In 2003 and 2004, the increasing hypoglycemia we observed paralleled the international focus on aggressively treating hyperglycemia in the acute care setting. We observed a significant decrease in hypoglycemia in 2005 and 2006 that could be attributed to the education provided by the Glycemic Control Program and 2 features on the subcutaneous insulin order set: the prominent hypoglycemia protocol and the order hold prandial insulin if the patient cannot eat. These are similar features identified in a report on preventing hospital hypoglycemia.9 Additionally, hypoglycemia may have decreased secondary to the emphasis on not using short‐acting insulin at bedtime.

Despite increased and improved insulin ordering, we did not observe a significant change in the percent of dysglycemic patients with 2 glucose levels 180 mg/dL. In our program patients are identified for intervention after their glucose levels are out‐of‐range. To better evaluate the impact of our interventions on the glycemic control of each patient, we plan to analyze the glucose levels in the days following identification of patients. Alternatively, we could provide intervention to all patients with dysglycemia rather than waiting for glucoses to be out‐of‐range. Though this approach would require greater resources than the single clinician we currently employ.

Our early experience highlights areas for future evaluation and intervention. First, the lack of scheduled prandial insulin and that less than one‐third of dysglycemic patients have basal insulin ordered underscore a continued need to target quality insulin ordering to include all componentsbasal, scheduled prandial, and prandial correction. Second, while the daily report is a good rudimentary identification tool for at‐risk patients, it offers limited information as to the impact of our clinical intervention. Thus, refined evaluative metrics need be developed to prospectively assess the course of glycemic control for patients.

We acknowledge the limitations of this study. First, our most involved interventionthe addition of the clinical intervention teamcame only 6 months before the end of the study period. Second, this is an observational retrospective analysis and cannot distinguish confounders, such as physician preferences and decisions, that not easily quantified or controlled for. Third, our definition of dysglycemic incorporated 41% of non‐critical‐care patients, possibly reflecting too broad a definition.

In summary, we have described an inpatient Glycemic Control Program that relies on real‐time data to identify patients in need of intervention. Early in our program we observed improved insulin ordering quality and decreased rates of hypoglycemia. Future steps include evaluating the impact of our clinical intervention team and further refining glycemic control metrics to prospectively identify patients at risk for hyper‐ and hypoglycemia.

The benefits of glycemic control include decreased patient morbidity, mortality, length of stay, and reduced hospital costs. In 2004, the American College of Endocrinology (ACE) issued glycemic guidelines for non‐critical‐care units (fasting glucose <110 mg/dL, nonfasting glucose <180 mg/dL).1 A comprehensive review of inpatient glycemic management called for development and evaluation of inpatient programs and tools.2 The 2006 ACE/American Diabetes Association (ADA) Statement on Inpatient Diabetes and Glycemic Control identified key components of an inpatient glycemic control program as: (1) solid administrative support; (2) a multidisciplinary committee; (3) assessment of current processes, care, and barriers; (4) development and implementation of order sets, protocols, policies, and educational efforts; and (5) metrics for evaluation.3

In 2003, Harborview Medical Center (HMC) formed a multidisciplinary committee to institute a Glycemic Control Program. The early goals were to decrease the use of sliding‐scale insulin, increase the appropriate use of basal and prandial insulin, and to avoid hypoglycemia. Here we report our program design and trends in physician insulin ordering from 2003 through 2006.

Patients and Methods

Program Description

Since 2003, the multidisciplinary committeephysicians, nurses, pharmacy representatives, and dietary and administrative representativeshas directed the development of the Glycemic Control Program with support from hospital administration and the Department of Quality Improvement. Funding for this program has been provided by the hospital based on the prominence of glycemic control among quality and safety measures, a projected decrease in costs, and the high incidence of diabetes in our patient population. Figure 1 outlines the program's key interventions.

Figure 1

First, a Subcutaneous Insulin Order Form was released for elective use in May 2004 (Figure 2). This form incorporated the 3 components of quality insulin ordering (basal, scheduled prandial, and prandial correction dosing) and provides prompts and education. A Diabetes Nurse Specialist trained nursing staff on the use of the form.

Figure 2

Second, we developed an automated daily data report identifying patients with out‐of‐range glucose levels defined as having any single glucose readings <60 mg/dL or any 2 readings 180 mg/dL within the prior 24 hours. In February 2006, this daily report became available to the clinicians on the committee.

Third, the Glycemic Control Program recruited a full‐time clinical Advanced Registered Nurse Practitioner (ARNP) and part‐time supervising physician to provide directed intervention and education for patients and medical personnel. Since August 2006, the ARNP has reviewed the out‐of‐range report daily, performs assessments, refines insulin orders, and educates clinicians. The assessments include chart review (of history and glycemic control), discussion with primary physician and nurse (and often the dietician), and interview of the patient and/or family. This leads to development and implementation of a glycemic control plan. Clinician education is performed both as direct education of the primary physician at the time of intervention and as didactic sessions.

Results

Physician Insulin Ordering

Ordering of both short‐acting and basal insulin increased (Figure 3). The ratio of short‐acting to basal orders decreased from 3.36 (1668/496) in 2003 to 1.97 (2226/1128) in 2006.

Figure 3

Chart review of the 100 randomly selected dysglycemic patients revealed increased ordering of prandial correction dosing from 8% of patients in 2003 to 32% in 2006. Yet, only 1 patient in 2003 and only 2 in 2006 had scheduled prandial. Ordering of sliding scale insulin fell from 16% in 2003 to 4% in 2006.

Glycemic Control Outcomes

The percentage of dysglycemic patients with hyperglycemia ranged from 19 to 24 without significant decline over the 4 years (Figure 4A). The percentage of hypoglycemic dysglycemic patients was increasing from 2003 to 2004, but in the years following the interventions (2005 through 2006) this declined significantly (P = 0.003; Figure 4B). On average, the observed LOS was higher for dysglycemic vs. euglycemic patients (mean SD days: 9.4 12.2 and 5.8 8.5, respectively). The mean observed to expected mortality ratio was 0.45 0.08 and 0.44 0.17 for the dysglycemic and euglycemic patients, respectively. Over the 4 years no statistically significant change in observed LOS or adjusted mortality was found (data not shown).

Figure 4

Conclusions

HMC, a safety net hospital with the highest UHC expected mortality of 131 hospitals nationwide, has demonstrated early successes in building its Glycemic Control Program, including: (1) decreased prescription of sliding scale; (2) a marked increase in prescription of basal insulin; and (3) significantly decreasing hypoglycemic events subsequent to the interventions. The decreased sliding scale and increased overall ordering of insulin could reflect increased awareness brought internationally through the literature and locally through our program. Two distinctive aspects of HMC's Glycemic Control Program, when compared to others,68 include: (1) the daily use of real‐time data to identify and target patients with out‐of‐range glucose levels; and (2) the coverage of all non‐critical‐care floors with a single clinician.

In 2003 and 2004, the increasing hypoglycemia we observed paralleled the international focus on aggressively treating hyperglycemia in the acute care setting. We observed a significant decrease in hypoglycemia in 2005 and 2006 that could be attributed to the education provided by the Glycemic Control Program and 2 features on the subcutaneous insulin order set: the prominent hypoglycemia protocol and the order hold prandial insulin if the patient cannot eat. These are similar features identified in a report on preventing hospital hypoglycemia.9 Additionally, hypoglycemia may have decreased secondary to the emphasis on not using short‐acting insulin at bedtime.

Despite increased and improved insulin ordering, we did not observe a significant change in the percent of dysglycemic patients with 2 glucose levels 180 mg/dL. In our program patients are identified for intervention after their glucose levels are out‐of‐range. To better evaluate the impact of our interventions on the glycemic control of each patient, we plan to analyze the glucose levels in the days following identification of patients. Alternatively, we could provide intervention to all patients with dysglycemia rather than waiting for glucoses to be out‐of‐range. Though this approach would require greater resources than the single clinician we currently employ.

Our early experience highlights areas for future evaluation and intervention. First, the lack of scheduled prandial insulin and that less than one‐third of dysglycemic patients have basal insulin ordered underscore a continued need to target quality insulin ordering to include all componentsbasal, scheduled prandial, and prandial correction. Second, while the daily report is a good rudimentary identification tool for at‐risk patients, it offers limited information as to the impact of our clinical intervention. Thus, refined evaluative metrics need be developed to prospectively assess the course of glycemic control for patients.

We acknowledge the limitations of this study. First, our most involved interventionthe addition of the clinical intervention teamcame only 6 months before the end of the study period. Second, this is an observational retrospective analysis and cannot distinguish confounders, such as physician preferences and decisions, that not easily quantified or controlled for. Third, our definition of dysglycemic incorporated 41% of non‐critical‐care patients, possibly reflecting too broad a definition.

In summary, we have described an inpatient Glycemic Control Program that relies on real‐time data to identify patients in need of intervention. Early in our program we observed improved insulin ordering quality and decreased rates of hypoglycemia. Future steps include evaluating the impact of our clinical intervention team and further refining glycemic control metrics to prospectively identify patients at risk for hyper‐ and hypoglycemia.

Medical procedures, an essential and highly valued part of medical education, are often undertaught and inconsistently evaluated. Hospitalists play an increasingly important role in developing the skills of resident‐learners. Alumni rate procedure skills as some of the most important skills learned during residency training,1, 2 but frequently identify training in procedural skills as having been insufficient.3, 4 For certification in internal medicine, the American Board of Internal Medicine (ABIM) has identified a limited set of procedures in which it expects all candidates to be cognitively competent with regard to their knowledge of these procedures. Although active participation in procedures is recommended for certification in internal medicine, the demonstration of procedural proficiency is not required.5

Resident competence in performing procedures remains highly variable and procedural complications can be a source of morbidity and mortality.2, 6, 7 A validated tool for the assessment of procedure related knowledge is currently lacking. In existing standardized tests, including the in‐training examination (ITE) and ABIM certification examination, only a fraction of questions pertain to medical procedures. The necessity for a specifically designed, standardized instrument that can objectively measure procedure related knowledge has been highlighted by studies that have demonstrated that there is little correlation between the rate of procedure‐related complications and ABIM/ITE scores.8 A validated tool to assess the knowledge of residents in selected medical procedures could serve to assess the readiness of residents to begin supervised practice and form part of a proficiency assessment.

In this study we aimed to develop a valid and reliable test of procedural knowledge in 3 procedures associated with potentially serious complications.

Methods

Placement of an arterial line, central venous catheter and thoracentesis were selected as the focus for test development. Using the National Board of Medical Examiners question development guidelines, multiple‐choice questions were developed to test residents on specific points of a prepared curriculum. Questions were designed to test the essential cognitive aspects of medical procedures, including indications, contraindications, and the management of complications, with an emphasis on the elements that were considered by a panel of experts to be frequently misunderstood. Questions were written by faculty trained in question writing (G.M.) and assessed for clarity by other members of faculty. Content evidence of the 36‐item examination (12 questions per procedure) was established by a panel of 4 critical care specialists with expertise in medical education. The study was approved by the Institutional Review Board at all sites.

Item performance characteristics were evaluated by administering the test online to a series of 30 trainees and specialty clinicians. Postadministration interviews with the critical care experts were performed to determine whether test questions were clear and appropriate for residents. Following initial testing, 4 test items with the lowest discrimination according to a point‐biserial correlation (Integrity; Castle Rock Research, Canada) were deleted from the test. The resulting 32‐item test contained items of varying difficulty to allow for effective discrimination between examinees (Appendix 1).

The test was then administered to residents beginning rotations in either the medical intensive care unit or in the coronary care unit at 4 medical centers in Massachusetts (Brigham and Women's Hospital; Massachusetts General Hospital; Faulkner Hospital; and North Shore Medical Center). In addition to completing the on‐line, self‐administered examination, participants provided baseline data including year of residency training, anticipated career path, and the number of prior procedures performed. On a 5‐point Likert scale participants estimated their self‐perceived confidence at performing the procedure (with and without supervision) and supervising each of the procedures. Residents were invited to complete a second test before the end of their rotation (2‐4 weeks after the initial test) in order to assess test‐retest reliability. Answers were made available only after the conclusion of the study.

Reliability of the 32‐item instrument was measured by Cronbach's analysis; a value of 0.6 is considered adequate and values of 0.7 or higher indicate good reliability. Pearson's correlation (Pearson's r) was used to compute test‐retest reliability. Univariate analyses were used to assess the association of the demographic variables with the test scores. Comparison of test scores between groups was made using a t test/Wilcoxon rank sum (2 groups) and analysis of variance (ANOVA)/Kruskal‐Wallis (3 or more groups). The associations of number of prior procedures attempted and self‐reported confidence with test scores was explored using Spearman's correlation. Inferences were made at the 0.05 level of significance, using 2‐tailed tests. Statistical analyses were performed using SPSS 15.0 (SPSS, Inc., Chicago, IL).

Results

Of the 192 internal medicine residents who consented to participate in the study between February and June 2006, 188 completed the initial and repeat test. Subject characteristics are detailed in Table 1.

Reliability of the 32‐item instrument measured by Cronbach's was 0.79 and its test‐retest reliability was 0.82. The items difficulty mean was 0.52 with a corrected point biserial correlation mean of 0.26. The test was of high difficulty, with a mean overall score of 50% (median 53%, interquartile range 44‐59%). Baseline scores differed significantly by residency program (P = 0.03). Residents with anticipated careers in critical care had significantly higher scores than those with anticipated careers in primary care (median scores critical care 56%, primary care and other nonprocedural medical subspecialties 50%, P = 0.01).

Residents in their final year reported performing a median of 13 arterial lines, 14 central venous lines, and 3 thoracenteses over the course of their residency training (Table 2). Increase in the number of performed procedures (central lines, arterial lines, and thoracenteses) was associated with an increase in test score (Spearman's correlation coefficient 0.35, P < 0.001). Residents in the highest and lowest decile of procedures performed had median scores of 56% and 43%, respectively (P < 0.001). Increasing seniority in residency was associated with an increase in overall test scores (median score by program year 49%, 54%, 50%, and 64%, P = 0.02).

Increase in self‐reported confidence was significantly associated with an increase in the number of performed procedures (Spearman's correlation coefficients for central line 0.83, arterial lines 0.76, and thoracentesis 0.78, all P < 0.001) and increasing seniority (0.66, 0.59, and 0.52, respectively, all P < 0.001).

Discussion

The determination of procedural competence has long been a challenge for trainers and internal medicine programs; methods for measuring procedural skills have not been rigorously studied. Procedural competence requires a combination of theoretical knowledge and practical skill. However, given the declining number of procedures performed by internists,4 the new ABIM guidelines mandate cognitive competence in contrast to the demonstration of hands‐on procedural proficiency.

We therefore sought to develop and validate the results of an examination of the theoretical knowledge necessary to perform 3 procedures associated with potentially serious complications. Following establishment of content evidence, item performance characteristics and postadministration interviews were used to develop a 32‐item test. We confirmed the test's internal structure by assessment of reliability and assessed the association of test scores with other variables for which correlation would be expected.

We found that residents performed poorly on test content considered to be important by procedure specialists. These findings highlight the limitations in current procedure training that is frequently sporadic and often variable. The numbers of procedures reported over the duration of residency by residents at these centers were low. It is unclear if the low number of procedures performed was due to limitations in resident content knowledge or if it reflects the increasing use of interventional services with fewer opportunities for experiential learning. Nevertheless, an increasing number of prior procedures was associated with higher self‐reported confidence for all procedures and translated to higher test scores.

This study was limited to 4 teaching hospitals and further studies may be needed to investigate the wider generalizability of the study instrument. However, participants were from 3 distinct internal medicine residency programs that included both community and university hospitals. We relied on resident self‐reports and did not independently verify the number of prior procedures performed. However, similar assumptions have been made in prior studies that physicians who rarely perform procedures are able to provide reasonable estimates of the total number performed.3

The reliability of the 32‐item test (Cronbach's = 0.79) is in the expected range for this length of test and indicates good reliability.9, 10 Given the potential complications associated with advanced medical procedures, there is increasing need to establish criteria for competence. Although we have not established a score threshold, the development of this validated tool to assess procedural knowledge is an important step toward establishing such a goal.

This test may facilitate efforts by hospitalists and others to evaluate the efficacy and refine existing methods of procedure training. Feedback to educators using this assessment tool may assist in the improvement of teaching strategies. In addition, the assessment of cognitive competence in procedure‐related knowledge using a rigorous and reliable means of assessment such as outlined in this study may help identify residents who need further training. Recognition for the necessity for additional training and oversight are likely to be especially important if residents are expected to perform procedures safely yet have fewer opportunities for practice.

Medical procedures, an essential and highly valued part of medical education, are often undertaught and inconsistently evaluated. Hospitalists play an increasingly important role in developing the skills of resident‐learners. Alumni rate procedure skills as some of the most important skills learned during residency training,1, 2 but frequently identify training in procedural skills as having been insufficient.3, 4 For certification in internal medicine, the American Board of Internal Medicine (ABIM) has identified a limited set of procedures in which it expects all candidates to be cognitively competent with regard to their knowledge of these procedures. Although active participation in procedures is recommended for certification in internal medicine, the demonstration of procedural proficiency is not required.5

Resident competence in performing procedures remains highly variable and procedural complications can be a source of morbidity and mortality.2, 6, 7 A validated tool for the assessment of procedure related knowledge is currently lacking. In existing standardized tests, including the in‐training examination (ITE) and ABIM certification examination, only a fraction of questions pertain to medical procedures. The necessity for a specifically designed, standardized instrument that can objectively measure procedure related knowledge has been highlighted by studies that have demonstrated that there is little correlation between the rate of procedure‐related complications and ABIM/ITE scores.8 A validated tool to assess the knowledge of residents in selected medical procedures could serve to assess the readiness of residents to begin supervised practice and form part of a proficiency assessment.

In this study we aimed to develop a valid and reliable test of procedural knowledge in 3 procedures associated with potentially serious complications.

Methods

Placement of an arterial line, central venous catheter and thoracentesis were selected as the focus for test development. Using the National Board of Medical Examiners question development guidelines, multiple‐choice questions were developed to test residents on specific points of a prepared curriculum. Questions were designed to test the essential cognitive aspects of medical procedures, including indications, contraindications, and the management of complications, with an emphasis on the elements that were considered by a panel of experts to be frequently misunderstood. Questions were written by faculty trained in question writing (G.M.) and assessed for clarity by other members of faculty. Content evidence of the 36‐item examination (12 questions per procedure) was established by a panel of 4 critical care specialists with expertise in medical education. The study was approved by the Institutional Review Board at all sites.

Item performance characteristics were evaluated by administering the test online to a series of 30 trainees and specialty clinicians. Postadministration interviews with the critical care experts were performed to determine whether test questions were clear and appropriate for residents. Following initial testing, 4 test items with the lowest discrimination according to a point‐biserial correlation (Integrity; Castle Rock Research, Canada) were deleted from the test. The resulting 32‐item test contained items of varying difficulty to allow for effective discrimination between examinees (Appendix 1).

The test was then administered to residents beginning rotations in either the medical intensive care unit or in the coronary care unit at 4 medical centers in Massachusetts (Brigham and Women's Hospital; Massachusetts General Hospital; Faulkner Hospital; and North Shore Medical Center). In addition to completing the on‐line, self‐administered examination, participants provided baseline data including year of residency training, anticipated career path, and the number of prior procedures performed. On a 5‐point Likert scale participants estimated their self‐perceived confidence at performing the procedure (with and without supervision) and supervising each of the procedures. Residents were invited to complete a second test before the end of their rotation (2‐4 weeks after the initial test) in order to assess test‐retest reliability. Answers were made available only after the conclusion of the study.

Reliability of the 32‐item instrument was measured by Cronbach's analysis; a value of 0.6 is considered adequate and values of 0.7 or higher indicate good reliability. Pearson's correlation (Pearson's r) was used to compute test‐retest reliability. Univariate analyses were used to assess the association of the demographic variables with the test scores. Comparison of test scores between groups was made using a t test/Wilcoxon rank sum (2 groups) and analysis of variance (ANOVA)/Kruskal‐Wallis (3 or more groups). The associations of number of prior procedures attempted and self‐reported confidence with test scores was explored using Spearman's correlation. Inferences were made at the 0.05 level of significance, using 2‐tailed tests. Statistical analyses were performed using SPSS 15.0 (SPSS, Inc., Chicago, IL).

Results

Of the 192 internal medicine residents who consented to participate in the study between February and June 2006, 188 completed the initial and repeat test. Subject characteristics are detailed in Table 1.

Reliability of the 32‐item instrument measured by Cronbach's was 0.79 and its test‐retest reliability was 0.82. The items difficulty mean was 0.52 with a corrected point biserial correlation mean of 0.26. The test was of high difficulty, with a mean overall score of 50% (median 53%, interquartile range 44‐59%). Baseline scores differed significantly by residency program (P = 0.03). Residents with anticipated careers in critical care had significantly higher scores than those with anticipated careers in primary care (median scores critical care 56%, primary care and other nonprocedural medical subspecialties 50%, P = 0.01).

Residents in their final year reported performing a median of 13 arterial lines, 14 central venous lines, and 3 thoracenteses over the course of their residency training (Table 2). Increase in the number of performed procedures (central lines, arterial lines, and thoracenteses) was associated with an increase in test score (Spearman's correlation coefficient 0.35, P < 0.001). Residents in the highest and lowest decile of procedures performed had median scores of 56% and 43%, respectively (P < 0.001). Increasing seniority in residency was associated with an increase in overall test scores (median score by program year 49%, 54%, 50%, and 64%, P = 0.02).

Increase in self‐reported confidence was significantly associated with an increase in the number of performed procedures (Spearman's correlation coefficients for central line 0.83, arterial lines 0.76, and thoracentesis 0.78, all P < 0.001) and increasing seniority (0.66, 0.59, and 0.52, respectively, all P < 0.001).

Discussion

The determination of procedural competence has long been a challenge for trainers and internal medicine programs; methods for measuring procedural skills have not been rigorously studied. Procedural competence requires a combination of theoretical knowledge and practical skill. However, given the declining number of procedures performed by internists,4 the new ABIM guidelines mandate cognitive competence in contrast to the demonstration of hands‐on procedural proficiency.

We therefore sought to develop and validate the results of an examination of the theoretical knowledge necessary to perform 3 procedures associated with potentially serious complications. Following establishment of content evidence, item performance characteristics and postadministration interviews were used to develop a 32‐item test. We confirmed the test's internal structure by assessment of reliability and assessed the association of test scores with other variables for which correlation would be expected.

We found that residents performed poorly on test content considered to be important by procedure specialists. These findings highlight the limitations in current procedure training that is frequently sporadic and often variable. The numbers of procedures reported over the duration of residency by residents at these centers were low. It is unclear if the low number of procedures performed was due to limitations in resident content knowledge or if it reflects the increasing use of interventional services with fewer opportunities for experiential learning. Nevertheless, an increasing number of prior procedures was associated with higher self‐reported confidence for all procedures and translated to higher test scores.

This study was limited to 4 teaching hospitals and further studies may be needed to investigate the wider generalizability of the study instrument. However, participants were from 3 distinct internal medicine residency programs that included both community and university hospitals. We relied on resident self‐reports and did not independently verify the number of prior procedures performed. However, similar assumptions have been made in prior studies that physicians who rarely perform procedures are able to provide reasonable estimates of the total number performed.3

The reliability of the 32‐item test (Cronbach's = 0.79) is in the expected range for this length of test and indicates good reliability.9, 10 Given the potential complications associated with advanced medical procedures, there is increasing need to establish criteria for competence. Although we have not established a score threshold, the development of this validated tool to assess procedural knowledge is an important step toward establishing such a goal.

This test may facilitate efforts by hospitalists and others to evaluate the efficacy and refine existing methods of procedure training. Feedback to educators using this assessment tool may assist in the improvement of teaching strategies. In addition, the assessment of cognitive competence in procedure‐related knowledge using a rigorous and reliable means of assessment such as outlined in this study may help identify residents who need further training. Recognition for the necessity for additional training and oversight are likely to be especially important if residents are expected to perform procedures safely yet have fewer opportunities for practice.

Medical procedures, an essential and highly valued part of medical education, are often undertaught and inconsistently evaluated. Hospitalists play an increasingly important role in developing the skills of resident‐learners. Alumni rate procedure skills as some of the most important skills learned during residency training,1, 2 but frequently identify training in procedural skills as having been insufficient.3, 4 For certification in internal medicine, the American Board of Internal Medicine (ABIM) has identified a limited set of procedures in which it expects all candidates to be cognitively competent with regard to their knowledge of these procedures. Although active participation in procedures is recommended for certification in internal medicine, the demonstration of procedural proficiency is not required.5

Resident competence in performing procedures remains highly variable and procedural complications can be a source of morbidity and mortality.2, 6, 7 A validated tool for the assessment of procedure related knowledge is currently lacking. In existing standardized tests, including the in‐training examination (ITE) and ABIM certification examination, only a fraction of questions pertain to medical procedures. The necessity for a specifically designed, standardized instrument that can objectively measure procedure related knowledge has been highlighted by studies that have demonstrated that there is little correlation between the rate of procedure‐related complications and ABIM/ITE scores.8 A validated tool to assess the knowledge of residents in selected medical procedures could serve to assess the readiness of residents to begin supervised practice and form part of a proficiency assessment.

In this study we aimed to develop a valid and reliable test of procedural knowledge in 3 procedures associated with potentially serious complications.

Methods

Placement of an arterial line, central venous catheter and thoracentesis were selected as the focus for test development. Using the National Board of Medical Examiners question development guidelines, multiple‐choice questions were developed to test residents on specific points of a prepared curriculum. Questions were designed to test the essential cognitive aspects of medical procedures, including indications, contraindications, and the management of complications, with an emphasis on the elements that were considered by a panel of experts to be frequently misunderstood. Questions were written by faculty trained in question writing (G.M.) and assessed for clarity by other members of faculty. Content evidence of the 36‐item examination (12 questions per procedure) was established by a panel of 4 critical care specialists with expertise in medical education. The study was approved by the Institutional Review Board at all sites.

Item performance characteristics were evaluated by administering the test online to a series of 30 trainees and specialty clinicians. Postadministration interviews with the critical care experts were performed to determine whether test questions were clear and appropriate for residents. Following initial testing, 4 test items with the lowest discrimination according to a point‐biserial correlation (Integrity; Castle Rock Research, Canada) were deleted from the test. The resulting 32‐item test contained items of varying difficulty to allow for effective discrimination between examinees (Appendix 1).

The test was then administered to residents beginning rotations in either the medical intensive care unit or in the coronary care unit at 4 medical centers in Massachusetts (Brigham and Women's Hospital; Massachusetts General Hospital; Faulkner Hospital; and North Shore Medical Center). In addition to completing the on‐line, self‐administered examination, participants provided baseline data including year of residency training, anticipated career path, and the number of prior procedures performed. On a 5‐point Likert scale participants estimated their self‐perceived confidence at performing the procedure (with and without supervision) and supervising each of the procedures. Residents were invited to complete a second test before the end of their rotation (2‐4 weeks after the initial test) in order to assess test‐retest reliability. Answers were made available only after the conclusion of the study.

Reliability of the 32‐item instrument was measured by Cronbach's analysis; a value of 0.6 is considered adequate and values of 0.7 or higher indicate good reliability. Pearson's correlation (Pearson's r) was used to compute test‐retest reliability. Univariate analyses were used to assess the association of the demographic variables with the test scores. Comparison of test scores between groups was made using a t test/Wilcoxon rank sum (2 groups) and analysis of variance (ANOVA)/Kruskal‐Wallis (3 or more groups). The associations of number of prior procedures attempted and self‐reported confidence with test scores was explored using Spearman's correlation. Inferences were made at the 0.05 level of significance, using 2‐tailed tests. Statistical analyses were performed using SPSS 15.0 (SPSS, Inc., Chicago, IL).

Results

Of the 192 internal medicine residents who consented to participate in the study between February and June 2006, 188 completed the initial and repeat test. Subject characteristics are detailed in Table 1.

Reliability of the 32‐item instrument measured by Cronbach's was 0.79 and its test‐retest reliability was 0.82. The items difficulty mean was 0.52 with a corrected point biserial correlation mean of 0.26. The test was of high difficulty, with a mean overall score of 50% (median 53%, interquartile range 44‐59%). Baseline scores differed significantly by residency program (P = 0.03). Residents with anticipated careers in critical care had significantly higher scores than those with anticipated careers in primary care (median scores critical care 56%, primary care and other nonprocedural medical subspecialties 50%, P = 0.01).

Residents in their final year reported performing a median of 13 arterial lines, 14 central venous lines, and 3 thoracenteses over the course of their residency training (Table 2). Increase in the number of performed procedures (central lines, arterial lines, and thoracenteses) was associated with an increase in test score (Spearman's correlation coefficient 0.35, P < 0.001). Residents in the highest and lowest decile of procedures performed had median scores of 56% and 43%, respectively (P < 0.001). Increasing seniority in residency was associated with an increase in overall test scores (median score by program year 49%, 54%, 50%, and 64%, P = 0.02).

Increase in self‐reported confidence was significantly associated with an increase in the number of performed procedures (Spearman's correlation coefficients for central line 0.83, arterial lines 0.76, and thoracentesis 0.78, all P < 0.001) and increasing seniority (0.66, 0.59, and 0.52, respectively, all P < 0.001).

Discussion

The determination of procedural competence has long been a challenge for trainers and internal medicine programs; methods for measuring procedural skills have not been rigorously studied. Procedural competence requires a combination of theoretical knowledge and practical skill. However, given the declining number of procedures performed by internists,4 the new ABIM guidelines mandate cognitive competence in contrast to the demonstration of hands‐on procedural proficiency.

We therefore sought to develop and validate the results of an examination of the theoretical knowledge necessary to perform 3 procedures associated with potentially serious complications. Following establishment of content evidence, item performance characteristics and postadministration interviews were used to develop a 32‐item test. We confirmed the test's internal structure by assessment of reliability and assessed the association of test scores with other variables for which correlation would be expected.

We found that residents performed poorly on test content considered to be important by procedure specialists. These findings highlight the limitations in current procedure training that is frequently sporadic and often variable. The numbers of procedures reported over the duration of residency by residents at these centers were low. It is unclear if the low number of procedures performed was due to limitations in resident content knowledge or if it reflects the increasing use of interventional services with fewer opportunities for experiential learning. Nevertheless, an increasing number of prior procedures was associated with higher self‐reported confidence for all procedures and translated to higher test scores.

This study was limited to 4 teaching hospitals and further studies may be needed to investigate the wider generalizability of the study instrument. However, participants were from 3 distinct internal medicine residency programs that included both community and university hospitals. We relied on resident self‐reports and did not independently verify the number of prior procedures performed. However, similar assumptions have been made in prior studies that physicians who rarely perform procedures are able to provide reasonable estimates of the total number performed.3

The reliability of the 32‐item test (Cronbach's = 0.79) is in the expected range for this length of test and indicates good reliability.9, 10 Given the potential complications associated with advanced medical procedures, there is increasing need to establish criteria for competence. Although we have not established a score threshold, the development of this validated tool to assess procedural knowledge is an important step toward establishing such a goal.

This test may facilitate efforts by hospitalists and others to evaluate the efficacy and refine existing methods of procedure training. Feedback to educators using this assessment tool may assist in the improvement of teaching strategies. In addition, the assessment of cognitive competence in procedure‐related knowledge using a rigorous and reliable means of assessment such as outlined in this study may help identify residents who need further training. Recognition for the necessity for additional training and oversight are likely to be especially important if residents are expected to perform procedures safely yet have fewer opportunities for practice.

Handoffs during hospitalization from one provider to another represent critical transition points in patient care.1 In‐hospital handoffs are a frequent occurrence, with 1 teaching hospital reporting 4000 handoffs daily for a total of 1.6 million per year.2

Incomplete or poor‐quality handoffs have been implicated as a source of adverse events and near misses in hospitalized patients.35 Standardizing the handoff process may improve patient safety during care transitions.6 In 2006, the Joint Commission issued a National Patient Safety Goal that requires care providers to adopt a standardized approach for handoff communications, including an opportunity to ask and respond to questions about a patient's care.7 The reductions in resident work hours by the Accreditation Council for Graduate Medical Education (ACGME) has also resulted in a greater number and greater scrutiny of handoffs in teaching hospitals.8, 9

In response to these issues, and because handoffs are a core competency for hospitalists, the Society of Hospital Medicine (SHM)convened a task force.10 Our goal was to develop a set of recommendations for handoffs that would be applicable in both community and academic settings; among physicians (hospitalists, internists, subspecialists, residents), nurse practitioners, and physicians assistants; and across roles including serving as the primary provider of hospital care, comanager, or consultant. This work focuses on handoffs that occur at shift change and service change.11 Shift changes are transitions of care between an outgoing provider and an incoming provider that occur at the end of the outgoing provider's continuous on‐duty period. Service changesa special type of shift changeare transitions of care between an outgoing provider and an incoming provider that occur when an outgoing provider is leaving a rotation or period of consecutive daily care for patients on the same service.

For this initiative, transfers of care in which the patient is moving from one patient area to another (eg, Emergency Department to inpatient floor, or floor to intensive care unit [ICU]) were excluded since they likely require unique consideration given their cross‐disciplinary and multispecialty nature. Likewise, transitions of care at hospital admission and discharge were also excluded because recommendations for discharge are already summarized in 2 complementary reports.12, 13

To develop recommendations for handoffs at routine shift change and service changes, the Handoff Task Force performed a systematic review of the literature to develop initial recommendations, obtained feedback from hospital‐based clinicians in addition to a panel of handoff experts, and finalized handoff recommendations, as well as a proposed research agenda, for the SHM.

Methods

The SHM Healthcare Quality and Patient Safety (HQPS) Committee convened the Handoff Task Force, which was comprised of 6 geographically diverse, predominantly academic hospitalists with backgrounds in education, patient safety, health communication, evidence‐based medicine, and handoffs. The Task Force then engaged a panel of 4 content experts selected for their work on handoffs in the fields of nursing, information technology, human factors engineering, and hospital medicine. Similar to clinical guideline development by professional societies, the Task Force used a combination of evidence‐based review and expert opinions to propose recommendations.

Results

Literature Review

Of the 374 articles identified by the electronic search of PubMed and the AHRQ Patient Safety Network, 109 were retrieved for detailed review, and 10 of these met the criteria for inclusion (Figure 1). Of these studies, 3 were derived from nursing literature and the remaining were tests of technology solutions or structured templates (Table 1).1618, 20, 22, 3842 No studies examined hospitalist handoffs. All eligible studies concerned shift change. There were no studies of service change. Only 1 study was a randomized controlled trial; the rest were pre‐post studies with historical controls or a controlled simulation. All reports were single‐site studies. Most outcomes were staff‐related or system‐related, with only 2 studies using patient outcomes.

Table 1.Characteristics of Studies Included in Review

Author (Year)	Study Design	Intervention	Setting and Study Population	Target	Outcomes
Abbreviations: IM, internal medicine; IS, ; UW, University of Washington.
Nursing
Kelly22 (2005)	Pre‐post	Change to walk‐round handover (at bedside) from baseline (control)	12‐bed rehab unit with 18 nurses and 10 patients	Staff, patient	11/18 nurses felt more or much more informed and involved; 8/10 patients felt more involved
Pothier et al.20 (2005)	Controlled simulation	Compared pure verbal to verbal with note‐taking to verbal plus typed content	Handover of 12 simulated patients over 5 cycles	System (data loss)	Minimal data loss with typed content, compared to 31% data retained with note‐taking, and no data retained with verbal only
Wallum38 (1995)	Pre‐post	Change from oral handover (baseline) to written template read with exchange	20 nurses in a geriatric dementia ward	Staff	83% of nurses felt care plans followed better; 88% knew care plans better
Technology or structured template
Cheah et al.39 (2005)	Pre‐post	Electronic template with free‐text entry compared to baseline	14 UK Surgery residents	Staff	100% (14) of residents rated electronic system as desirable, but 7 (50%) reported that information was not updated
Lee et al.40 (1996)	Pre‐post	Standardized signout card for interns to transmit information during handoffs compared to handwritten (baseline)	Inpatient cardiology service at IM residency program in Minnesota with 19 new interns over a 3‐month period	Staff	Intervention interns (n = 10) reported poor sign‐out less often than controls (n = 9) [intervention 8 nights (5.8%) vs. control 17 nights (14.9%); P = 0.016]
Kannry and Moore18 (1999)	Pre‐post	Compared web‐based signout program to usual system (baseline)	An academic teaching hospital in New York (34 patients admitted in 1997; 40 patients admitted in 1998)	System	Improved provider identification (86% web signout vs. 57% hospital census)
Petersen et al.17 (1998)	Pre‐post	4 months of computerized signouts compared to baseline period (control)	3747 patients admitted to the medical service at an academic teaching hospital	Patient	Preventable adverse events (ADE) decreased (1.7% to 1.2%, P < 0.10); risk of cross‐cover physician for ADE eliminated
Ram and Block41 (1993)	Pre‐post	Compared handwritten (baseline) to computer‐generated	Family medicine residents at 2 academic teaching hospitals [Buffalo (n = 16) and Pittsburgh (n = 16)]	Staff	Higher satisfaction after electronic signout, but complaints with burden of data entry and need to keep information updated
Van Eaton et al.42 (2004)	Pre‐post	Use of UW Cores links sign‐out to list for rounds and IS data	28 surgical and medical residents at 2 teaching hospitals	System	At 6 months, 66% of patients entered in system (adoption)
Van Eaton et al.16 (2005)	Prospective, randomized, crossover study.	Compared UW Cores* integrated system compared to usual system	14 inpatient resident teams (6 surgery, 8 IM) at 2 teaching hospitals for 5 months	Staff, system	50% reduction in the perceived time spent copying data [from 24% to 12% (P < 0.0001)] and number of patients missed on rounds (2.5 vs. 5 patients/team/month, P = 0.0001); improved signout quality (69.6% agree or strongly agree); and improved continuity of care (66.1% agree or strongly agree)

Figure 1

Overall, the literature presented supports the use of a verbal handoff supplemented with written documentation in a structured format or technology solution. The 2 most rigorous studies were led by Van Eaton et al.16 and Petersen et al.17 and focused on evaluating technology solutions. Van Eaton et al.16 performed a randomized controlled trial of a locally created rounding template with 161 surgical residents. This template downloads certain information (lab values and recent vital signs) from the hospital system into a sign‐out sheet and allows residents to enter notes about diagnoses, allergies, medications and to‐do items. When implemented, the investigators found the number of patients missed on rounds decreased by 50%. Residents reported an increase of 40% in the amount of time available to pre‐round, due largely to not having to copy data such as vital signs. They reported a decrease in rounding time by 3 hours per week, and this was perceived as helping them meet the ACGME 80 hours work rules. Lastly, the residents reported a higher quality of sign‐outs from their peers and perceived an overall improvement in continuity of care. Petersen and colleagues implemented a computerized sign‐out (auto‐imported medications, name, room number) in an internal medicine residency to improve continuity of care during cross‐coverage and decrease adverse events.17 Prior to the intervention, the frequency of preventable adverse events was 1.7% and it was significantly associated with cross‐coverage. Preventable adverse events were identified using a confidential self‐report system that was also validated by clinician review. After the intervention, the frequency of preventable adverse events dropped to 1.2% (P < 0.1), and cross‐coverage was no longer associated with preventable adverse events. In other studies, technological solutions also improved provider identification and staff communication.18, 19 Together, these technology‐based intervention studies suggest that a computerized sign‐out with auto‐imported fields has the ability to improve physician efficiency and also improve inpatient care (reduction in number of patients missed on rounds, decrease in preventable adverse events).

Studies from nursing demonstrated that supplementing a verbal exchange with written information improved transfer of information, compared to verbal exchange alone.20 One of these studies rated the transfer of information using videotaped simulated handoff cases.21 Last, 1 nursing study that more directly involved patients in the handoff process resulted in improved nursing knowledge and greater patient empowerment (Table 1).22

White papers or consensus statements originated from international and national consortia in patient safety including the Australian Council for Safety and Quality in Healthcare,23 the Junior Doctors Committee of the British Medical Association,24 University Health Consortium,25 the Department of Defense Patient Safety Program,26 and The Joint Commission.27 Several common themes were prevalent in all white papers. First, there exists a need to train new personnel on how to perform an effective handoff. Second, efforts should be undertaken to ensure adequate time for handoffs and reduce interruptions during handoffs. Third, several of the papers supported verbal exchange that facilitates interactive questioning, focuses on ill patients, and delineates actions to be taken. Lastly, content should be updated to ensure transfer of the latest clinical information.

Final Handoff Recommendations

The final handoff recommendations are shown in Figure 2. The recommendations were designed to be consistent with the overall finding of the literature review, which supports the use of a verbal handoff supplemented with written documentation or a technological solution in a structured format. With the exception of 1 recommendation that is specific to service changes, all recommendations are designed to refer to shift changes and service changes. One overarching recommendation refers to the need for a formally recognized handoff plan at a shift change or change of service. The remaining 12 recommendations are divided into 4 that refer to hospitalist groups or programs, 3 that refer to verbal exchange, and 5 that refer to content exchange. The distinction is an important one because program‐level recommendations require organizational support and buy‐in to promote clinician participation and adherence. The 4 program recommendations also form the necessary framework for the remaining recommendations. For example, the second program recommendation describes the need for a standardized template or technology solution for accessing and recording patient information during the handoff. After a program adopts such a mechanism for exchanging patient information, the specific details for use and maintenance are outlined in greater detail in content exchange recommendations.

Figure 2

Because of the limited trials of handoff strategies, none of the recommendations are supported with level of evidence A (multiple numerous randomized controlled trials). In fact, with the exception of using a template or technology solution which was supported with level of evidence B, all handoff recommendations were supported with C level of evidence. The recommendations, however, were rated as Class I (effective) because there were no conflicting expert opinions or studies (Figure 2).

Discussion

In summary, our review of the literature supports the use of face‐to‐face verbal handoffs that are aided by the use of structured template to guide exchange of information. Furthermore, the development of these recommendations is the first effort of its kind for hospitalist handoffs and a movement towards standardizing the handoff process. While these recommendations are meant to provide structure to the hospitalist handoff process, the use and implementation by individual hospitalist programs may require more specific detail than these recommendations provide. Local modifications can allow for improved acceptance and adoption by practicing hospitalists. These recommendations can also help guide teaching efforts for academic hospitalists who are responsible for supervising residents.

The limitations of these recommendations related to lack of evidence in this field. Studies suffered from small size, poor description of methods, and a paucity of controlled interventions. The described technology solutions are not standardized or commercially available. Only 1 study included patient outcomes.28 There are no multicenter studies, studies of hospitalist handoffs, or studies to guide inclusion of specific content. Randomized controlled trials, interrupted time series analyses, and other rigorous study designs are needed in both teaching and non‐teaching settings to evaluate these recommendations and other approaches to improving handoffs. Ideally, these studies would occur through multicenter collaboratives and with human factors researchers familiar with mixed methods approaches to evaluate how and why interventions work.29 Efforts should focus on developing surrogate measures that are sensitive to handoff quality and related to important patient outcomes. The results of future studies should be used to refine the present recommendations. Locating new literature could be facilitated through the introduction of Medical Subject Heading for the term handoff by the National Library of Medicine. After completing this systematic review and developing the handoff recommendations described here, a few other noteworthy articles have been published on this topic, to which we refer interested readers. Several of these studies demonstrate that standardizing content and process during medical or surgical intern sign‐out improves resident confidence with handoffs,30 resident perceptions of accuracy and completeness of signout,31 and perceptions of patient safety.32 Another prospective audiotape study of 12 days of resident signout of clinical information demonstrated that poor quality oral sign‐outs was associated with an increased risk of post‐call resident reported signout‐related problems.5 Lastly, 1 nursing study demonstrated improved staff reports of safety, efficiency, and teamwork after a change from verbal reporting in an isolated room to bedside handover.33 Overall, these additional studies continue to support the current recommendations presented in this paper and do not significantly impact the conclusions of our literature review.

While lacking specific content domain recommendations, this report can be used as a starting point to guide development of self and peer assessment of hospitalist handoff quality. Development and validation of such assessments is especially important and can be incorporated into efforts to certify hospitalists through the recently approved certificate of focused practice in hospital medicine by the American Board of Internal Medicine (ABIM). Initiatives by several related organizations may help guide these effortsThe Joint Commission, the ABIM's Stepping Up to the Plate (SUTTP) Alliance, the Institute for Healthcare Improvement, the Information Transfer and Communication Practices (ITCP) Project for surgical care transitions, and the Hospital at Night (H@N) Program sponsored by the United Kingdom's National Health Service.3437 Professional medical organizations can also serve as powerful mediators of change in this area, not only by raising the visibility of handoffs, but also by mobilizing research funding. Patients and their caregivers may also play an important role in increasing awareness and education in this area. Future efforts should target handoffs not addressed in this initiative, such as transfers from emergency departments to inpatient care units, or between ICUs and the medical floor.

Conclusion

With the growth of hospital medicine and the increased acuity of inpatients, improving handoffs becomes an important part of ensuring patient safety. The goal of the SHM Handoffs Task Force was to begin to standardize handoffs at change of shift and change of servicea fundamental activity of hospitalists. These recommendations build on the limited literature in surgery, nursing, and medical informatics and provide a starting point for promoting safe and seamless in‐hospital handoffs for practitioners of Hospital Medicine.

Handoffs during hospitalization from one provider to another represent critical transition points in patient care.1 In‐hospital handoffs are a frequent occurrence, with 1 teaching hospital reporting 4000 handoffs daily for a total of 1.6 million per year.2

Incomplete or poor‐quality handoffs have been implicated as a source of adverse events and near misses in hospitalized patients.35 Standardizing the handoff process may improve patient safety during care transitions.6 In 2006, the Joint Commission issued a National Patient Safety Goal that requires care providers to adopt a standardized approach for handoff communications, including an opportunity to ask and respond to questions about a patient's care.7 The reductions in resident work hours by the Accreditation Council for Graduate Medical Education (ACGME) has also resulted in a greater number and greater scrutiny of handoffs in teaching hospitals.8, 9

In response to these issues, and because handoffs are a core competency for hospitalists, the Society of Hospital Medicine (SHM)convened a task force.10 Our goal was to develop a set of recommendations for handoffs that would be applicable in both community and academic settings; among physicians (hospitalists, internists, subspecialists, residents), nurse practitioners, and physicians assistants; and across roles including serving as the primary provider of hospital care, comanager, or consultant. This work focuses on handoffs that occur at shift change and service change.11 Shift changes are transitions of care between an outgoing provider and an incoming provider that occur at the end of the outgoing provider's continuous on‐duty period. Service changesa special type of shift changeare transitions of care between an outgoing provider and an incoming provider that occur when an outgoing provider is leaving a rotation or period of consecutive daily care for patients on the same service.

For this initiative, transfers of care in which the patient is moving from one patient area to another (eg, Emergency Department to inpatient floor, or floor to intensive care unit [ICU]) were excluded since they likely require unique consideration given their cross‐disciplinary and multispecialty nature. Likewise, transitions of care at hospital admission and discharge were also excluded because recommendations for discharge are already summarized in 2 complementary reports.12, 13

To develop recommendations for handoffs at routine shift change and service changes, the Handoff Task Force performed a systematic review of the literature to develop initial recommendations, obtained feedback from hospital‐based clinicians in addition to a panel of handoff experts, and finalized handoff recommendations, as well as a proposed research agenda, for the SHM.

Methods

The SHM Healthcare Quality and Patient Safety (HQPS) Committee convened the Handoff Task Force, which was comprised of 6 geographically diverse, predominantly academic hospitalists with backgrounds in education, patient safety, health communication, evidence‐based medicine, and handoffs. The Task Force then engaged a panel of 4 content experts selected for their work on handoffs in the fields of nursing, information technology, human factors engineering, and hospital medicine. Similar to clinical guideline development by professional societies, the Task Force used a combination of evidence‐based review and expert opinions to propose recommendations.

Results

Literature Review

Of the 374 articles identified by the electronic search of PubMed and the AHRQ Patient Safety Network, 109 were retrieved for detailed review, and 10 of these met the criteria for inclusion (Figure 1). Of these studies, 3 were derived from nursing literature and the remaining were tests of technology solutions or structured templates (Table 1).1618, 20, 22, 3842 No studies examined hospitalist handoffs. All eligible studies concerned shift change. There were no studies of service change. Only 1 study was a randomized controlled trial; the rest were pre‐post studies with historical controls or a controlled simulation. All reports were single‐site studies. Most outcomes were staff‐related or system‐related, with only 2 studies using patient outcomes.

Table 1.Characteristics of Studies Included in Review

Author (Year)	Study Design	Intervention	Setting and Study Population	Target	Outcomes
Abbreviations: IM, internal medicine; IS, ; UW, University of Washington.
Nursing
Kelly22 (2005)	Pre‐post	Change to walk‐round handover (at bedside) from baseline (control)	12‐bed rehab unit with 18 nurses and 10 patients	Staff, patient	11/18 nurses felt more or much more informed and involved; 8/10 patients felt more involved
Pothier et al.20 (2005)	Controlled simulation	Compared pure verbal to verbal with note‐taking to verbal plus typed content	Handover of 12 simulated patients over 5 cycles	System (data loss)	Minimal data loss with typed content, compared to 31% data retained with note‐taking, and no data retained with verbal only
Wallum38 (1995)	Pre‐post	Change from oral handover (baseline) to written template read with exchange	20 nurses in a geriatric dementia ward	Staff	83% of nurses felt care plans followed better; 88% knew care plans better
Technology or structured template
Cheah et al.39 (2005)	Pre‐post	Electronic template with free‐text entry compared to baseline	14 UK Surgery residents	Staff	100% (14) of residents rated electronic system as desirable, but 7 (50%) reported that information was not updated
Lee et al.40 (1996)	Pre‐post	Standardized signout card for interns to transmit information during handoffs compared to handwritten (baseline)	Inpatient cardiology service at IM residency program in Minnesota with 19 new interns over a 3‐month period	Staff	Intervention interns (n = 10) reported poor sign‐out less often than controls (n = 9) [intervention 8 nights (5.8%) vs. control 17 nights (14.9%); P = 0.016]
Kannry and Moore18 (1999)	Pre‐post	Compared web‐based signout program to usual system (baseline)	An academic teaching hospital in New York (34 patients admitted in 1997; 40 patients admitted in 1998)	System	Improved provider identification (86% web signout vs. 57% hospital census)
Petersen et al.17 (1998)	Pre‐post	4 months of computerized signouts compared to baseline period (control)	3747 patients admitted to the medical service at an academic teaching hospital	Patient	Preventable adverse events (ADE) decreased (1.7% to 1.2%, P < 0.10); risk of cross‐cover physician for ADE eliminated
Ram and Block41 (1993)	Pre‐post	Compared handwritten (baseline) to computer‐generated	Family medicine residents at 2 academic teaching hospitals [Buffalo (n = 16) and Pittsburgh (n = 16)]	Staff	Higher satisfaction after electronic signout, but complaints with burden of data entry and need to keep information updated
Van Eaton et al.42 (2004)	Pre‐post	Use of UW Cores links sign‐out to list for rounds and IS data	28 surgical and medical residents at 2 teaching hospitals	System	At 6 months, 66% of patients entered in system (adoption)
Van Eaton et al.16 (2005)	Prospective, randomized, crossover study.	Compared UW Cores* integrated system compared to usual system	14 inpatient resident teams (6 surgery, 8 IM) at 2 teaching hospitals for 5 months	Staff, system	50% reduction in the perceived time spent copying data [from 24% to 12% (P < 0.0001)] and number of patients missed on rounds (2.5 vs. 5 patients/team/month, P = 0.0001); improved signout quality (69.6% agree or strongly agree); and improved continuity of care (66.1% agree or strongly agree)

Figure 1

Overall, the literature presented supports the use of a verbal handoff supplemented with written documentation in a structured format or technology solution. The 2 most rigorous studies were led by Van Eaton et al.16 and Petersen et al.17 and focused on evaluating technology solutions. Van Eaton et al.16 performed a randomized controlled trial of a locally created rounding template with 161 surgical residents. This template downloads certain information (lab values and recent vital signs) from the hospital system into a sign‐out sheet and allows residents to enter notes about diagnoses, allergies, medications and to‐do items. When implemented, the investigators found the number of patients missed on rounds decreased by 50%. Residents reported an increase of 40% in the amount of time available to pre‐round, due largely to not having to copy data such as vital signs. They reported a decrease in rounding time by 3 hours per week, and this was perceived as helping them meet the ACGME 80 hours work rules. Lastly, the residents reported a higher quality of sign‐outs from their peers and perceived an overall improvement in continuity of care. Petersen and colleagues implemented a computerized sign‐out (auto‐imported medications, name, room number) in an internal medicine residency to improve continuity of care during cross‐coverage and decrease adverse events.17 Prior to the intervention, the frequency of preventable adverse events was 1.7% and it was significantly associated with cross‐coverage. Preventable adverse events were identified using a confidential self‐report system that was also validated by clinician review. After the intervention, the frequency of preventable adverse events dropped to 1.2% (P < 0.1), and cross‐coverage was no longer associated with preventable adverse events. In other studies, technological solutions also improved provider identification and staff communication.18, 19 Together, these technology‐based intervention studies suggest that a computerized sign‐out with auto‐imported fields has the ability to improve physician efficiency and also improve inpatient care (reduction in number of patients missed on rounds, decrease in preventable adverse events).

Studies from nursing demonstrated that supplementing a verbal exchange with written information improved transfer of information, compared to verbal exchange alone.20 One of these studies rated the transfer of information using videotaped simulated handoff cases.21 Last, 1 nursing study that more directly involved patients in the handoff process resulted in improved nursing knowledge and greater patient empowerment (Table 1).22

White papers or consensus statements originated from international and national consortia in patient safety including the Australian Council for Safety and Quality in Healthcare,23 the Junior Doctors Committee of the British Medical Association,24 University Health Consortium,25 the Department of Defense Patient Safety Program,26 and The Joint Commission.27 Several common themes were prevalent in all white papers. First, there exists a need to train new personnel on how to perform an effective handoff. Second, efforts should be undertaken to ensure adequate time for handoffs and reduce interruptions during handoffs. Third, several of the papers supported verbal exchange that facilitates interactive questioning, focuses on ill patients, and delineates actions to be taken. Lastly, content should be updated to ensure transfer of the latest clinical information.

Final Handoff Recommendations

The final handoff recommendations are shown in Figure 2. The recommendations were designed to be consistent with the overall finding of the literature review, which supports the use of a verbal handoff supplemented with written documentation or a technological solution in a structured format. With the exception of 1 recommendation that is specific to service changes, all recommendations are designed to refer to shift changes and service changes. One overarching recommendation refers to the need for a formally recognized handoff plan at a shift change or change of service. The remaining 12 recommendations are divided into 4 that refer to hospitalist groups or programs, 3 that refer to verbal exchange, and 5 that refer to content exchange. The distinction is an important one because program‐level recommendations require organizational support and buy‐in to promote clinician participation and adherence. The 4 program recommendations also form the necessary framework for the remaining recommendations. For example, the second program recommendation describes the need for a standardized template or technology solution for accessing and recording patient information during the handoff. After a program adopts such a mechanism for exchanging patient information, the specific details for use and maintenance are outlined in greater detail in content exchange recommendations.

Figure 2

Because of the limited trials of handoff strategies, none of the recommendations are supported with level of evidence A (multiple numerous randomized controlled trials). In fact, with the exception of using a template or technology solution which was supported with level of evidence B, all handoff recommendations were supported with C level of evidence. The recommendations, however, were rated as Class I (effective) because there were no conflicting expert opinions or studies (Figure 2).

Discussion

In summary, our review of the literature supports the use of face‐to‐face verbal handoffs that are aided by the use of structured template to guide exchange of information. Furthermore, the development of these recommendations is the first effort of its kind for hospitalist handoffs and a movement towards standardizing the handoff process. While these recommendations are meant to provide structure to the hospitalist handoff process, the use and implementation by individual hospitalist programs may require more specific detail than these recommendations provide. Local modifications can allow for improved acceptance and adoption by practicing hospitalists. These recommendations can also help guide teaching efforts for academic hospitalists who are responsible for supervising residents.

The limitations of these recommendations related to lack of evidence in this field. Studies suffered from small size, poor description of methods, and a paucity of controlled interventions. The described technology solutions are not standardized or commercially available. Only 1 study included patient outcomes.28 There are no multicenter studies, studies of hospitalist handoffs, or studies to guide inclusion of specific content. Randomized controlled trials, interrupted time series analyses, and other rigorous study designs are needed in both teaching and non‐teaching settings to evaluate these recommendations and other approaches to improving handoffs. Ideally, these studies would occur through multicenter collaboratives and with human factors researchers familiar with mixed methods approaches to evaluate how and why interventions work.29 Efforts should focus on developing surrogate measures that are sensitive to handoff quality and related to important patient outcomes. The results of future studies should be used to refine the present recommendations. Locating new literature could be facilitated through the introduction of Medical Subject Heading for the term handoff by the National Library of Medicine. After completing this systematic review and developing the handoff recommendations described here, a few other noteworthy articles have been published on this topic, to which we refer interested readers. Several of these studies demonstrate that standardizing content and process during medical or surgical intern sign‐out improves resident confidence with handoffs,30 resident perceptions of accuracy and completeness of signout,31 and perceptions of patient safety.32 Another prospective audiotape study of 12 days of resident signout of clinical information demonstrated that poor quality oral sign‐outs was associated with an increased risk of post‐call resident reported signout‐related problems.5 Lastly, 1 nursing study demonstrated improved staff reports of safety, efficiency, and teamwork after a change from verbal reporting in an isolated room to bedside handover.33 Overall, these additional studies continue to support the current recommendations presented in this paper and do not significantly impact the conclusions of our literature review.

While lacking specific content domain recommendations, this report can be used as a starting point to guide development of self and peer assessment of hospitalist handoff quality. Development and validation of such assessments is especially important and can be incorporated into efforts to certify hospitalists through the recently approved certificate of focused practice in hospital medicine by the American Board of Internal Medicine (ABIM). Initiatives by several related organizations may help guide these effortsThe Joint Commission, the ABIM's Stepping Up to the Plate (SUTTP) Alliance, the Institute for Healthcare Improvement, the Information Transfer and Communication Practices (ITCP) Project for surgical care transitions, and the Hospital at Night (H@N) Program sponsored by the United Kingdom's National Health Service.3437 Professional medical organizations can also serve as powerful mediators of change in this area, not only by raising the visibility of handoffs, but also by mobilizing research funding. Patients and their caregivers may also play an important role in increasing awareness and education in this area. Future efforts should target handoffs not addressed in this initiative, such as transfers from emergency departments to inpatient care units, or between ICUs and the medical floor.

Conclusion

With the growth of hospital medicine and the increased acuity of inpatients, improving handoffs becomes an important part of ensuring patient safety. The goal of the SHM Handoffs Task Force was to begin to standardize handoffs at change of shift and change of servicea fundamental activity of hospitalists. These recommendations build on the limited literature in surgery, nursing, and medical informatics and provide a starting point for promoting safe and seamless in‐hospital handoffs for practitioners of Hospital Medicine.

Handoffs during hospitalization from one provider to another represent critical transition points in patient care.1 In‐hospital handoffs are a frequent occurrence, with 1 teaching hospital reporting 4000 handoffs daily for a total of 1.6 million per year.2

Incomplete or poor‐quality handoffs have been implicated as a source of adverse events and near misses in hospitalized patients.35 Standardizing the handoff process may improve patient safety during care transitions.6 In 2006, the Joint Commission issued a National Patient Safety Goal that requires care providers to adopt a standardized approach for handoff communications, including an opportunity to ask and respond to questions about a patient's care.7 The reductions in resident work hours by the Accreditation Council for Graduate Medical Education (ACGME) has also resulted in a greater number and greater scrutiny of handoffs in teaching hospitals.8, 9

In response to these issues, and because handoffs are a core competency for hospitalists, the Society of Hospital Medicine (SHM)convened a task force.10 Our goal was to develop a set of recommendations for handoffs that would be applicable in both community and academic settings; among physicians (hospitalists, internists, subspecialists, residents), nurse practitioners, and physicians assistants; and across roles including serving as the primary provider of hospital care, comanager, or consultant. This work focuses on handoffs that occur at shift change and service change.11 Shift changes are transitions of care between an outgoing provider and an incoming provider that occur at the end of the outgoing provider's continuous on‐duty period. Service changesa special type of shift changeare transitions of care between an outgoing provider and an incoming provider that occur when an outgoing provider is leaving a rotation or period of consecutive daily care for patients on the same service.

For this initiative, transfers of care in which the patient is moving from one patient area to another (eg, Emergency Department to inpatient floor, or floor to intensive care unit [ICU]) were excluded since they likely require unique consideration given their cross‐disciplinary and multispecialty nature. Likewise, transitions of care at hospital admission and discharge were also excluded because recommendations for discharge are already summarized in 2 complementary reports.12, 13

To develop recommendations for handoffs at routine shift change and service changes, the Handoff Task Force performed a systematic review of the literature to develop initial recommendations, obtained feedback from hospital‐based clinicians in addition to a panel of handoff experts, and finalized handoff recommendations, as well as a proposed research agenda, for the SHM.

Methods

The SHM Healthcare Quality and Patient Safety (HQPS) Committee convened the Handoff Task Force, which was comprised of 6 geographically diverse, predominantly academic hospitalists with backgrounds in education, patient safety, health communication, evidence‐based medicine, and handoffs. The Task Force then engaged a panel of 4 content experts selected for their work on handoffs in the fields of nursing, information technology, human factors engineering, and hospital medicine. Similar to clinical guideline development by professional societies, the Task Force used a combination of evidence‐based review and expert opinions to propose recommendations.

Results

Literature Review

Of the 374 articles identified by the electronic search of PubMed and the AHRQ Patient Safety Network, 109 were retrieved for detailed review, and 10 of these met the criteria for inclusion (Figure 1). Of these studies, 3 were derived from nursing literature and the remaining were tests of technology solutions or structured templates (Table 1).1618, 20, 22, 3842 No studies examined hospitalist handoffs. All eligible studies concerned shift change. There were no studies of service change. Only 1 study was a randomized controlled trial; the rest were pre‐post studies with historical controls or a controlled simulation. All reports were single‐site studies. Most outcomes were staff‐related or system‐related, with only 2 studies using patient outcomes.

Table 1.Characteristics of Studies Included in Review

Author (Year)	Study Design	Intervention	Setting and Study Population	Target	Outcomes
Abbreviations: IM, internal medicine; IS, ; UW, University of Washington.
Nursing
Kelly22 (2005)	Pre‐post	Change to walk‐round handover (at bedside) from baseline (control)	12‐bed rehab unit with 18 nurses and 10 patients	Staff, patient	11/18 nurses felt more or much more informed and involved; 8/10 patients felt more involved
Pothier et al.20 (2005)	Controlled simulation	Compared pure verbal to verbal with note‐taking to verbal plus typed content	Handover of 12 simulated patients over 5 cycles	System (data loss)	Minimal data loss with typed content, compared to 31% data retained with note‐taking, and no data retained with verbal only
Wallum38 (1995)	Pre‐post	Change from oral handover (baseline) to written template read with exchange	20 nurses in a geriatric dementia ward	Staff	83% of nurses felt care plans followed better; 88% knew care plans better
Technology or structured template
Cheah et al.39 (2005)	Pre‐post	Electronic template with free‐text entry compared to baseline	14 UK Surgery residents	Staff	100% (14) of residents rated electronic system as desirable, but 7 (50%) reported that information was not updated
Lee et al.40 (1996)	Pre‐post	Standardized signout card for interns to transmit information during handoffs compared to handwritten (baseline)	Inpatient cardiology service at IM residency program in Minnesota with 19 new interns over a 3‐month period	Staff	Intervention interns (n = 10) reported poor sign‐out less often than controls (n = 9) [intervention 8 nights (5.8%) vs. control 17 nights (14.9%); P = 0.016]
Kannry and Moore18 (1999)	Pre‐post	Compared web‐based signout program to usual system (baseline)	An academic teaching hospital in New York (34 patients admitted in 1997; 40 patients admitted in 1998)	System	Improved provider identification (86% web signout vs. 57% hospital census)
Petersen et al.17 (1998)	Pre‐post	4 months of computerized signouts compared to baseline period (control)	3747 patients admitted to the medical service at an academic teaching hospital	Patient	Preventable adverse events (ADE) decreased (1.7% to 1.2%, P < 0.10); risk of cross‐cover physician for ADE eliminated
Ram and Block41 (1993)	Pre‐post	Compared handwritten (baseline) to computer‐generated	Family medicine residents at 2 academic teaching hospitals [Buffalo (n = 16) and Pittsburgh (n = 16)]	Staff	Higher satisfaction after electronic signout, but complaints with burden of data entry and need to keep information updated
Van Eaton et al.42 (2004)	Pre‐post	Use of UW Cores links sign‐out to list for rounds and IS data	28 surgical and medical residents at 2 teaching hospitals	System	At 6 months, 66% of patients entered in system (adoption)
Van Eaton et al.16 (2005)	Prospective, randomized, crossover study.	Compared UW Cores* integrated system compared to usual system	14 inpatient resident teams (6 surgery, 8 IM) at 2 teaching hospitals for 5 months	Staff, system	50% reduction in the perceived time spent copying data [from 24% to 12% (P < 0.0001)] and number of patients missed on rounds (2.5 vs. 5 patients/team/month, P = 0.0001); improved signout quality (69.6% agree or strongly agree); and improved continuity of care (66.1% agree or strongly agree)

Figure 1

Overall, the literature presented supports the use of a verbal handoff supplemented with written documentation in a structured format or technology solution. The 2 most rigorous studies were led by Van Eaton et al.16 and Petersen et al.17 and focused on evaluating technology solutions. Van Eaton et al.16 performed a randomized controlled trial of a locally created rounding template with 161 surgical residents. This template downloads certain information (lab values and recent vital signs) from the hospital system into a sign‐out sheet and allows residents to enter notes about diagnoses, allergies, medications and to‐do items. When implemented, the investigators found the number of patients missed on rounds decreased by 50%. Residents reported an increase of 40% in the amount of time available to pre‐round, due largely to not having to copy data such as vital signs. They reported a decrease in rounding time by 3 hours per week, and this was perceived as helping them meet the ACGME 80 hours work rules. Lastly, the residents reported a higher quality of sign‐outs from their peers and perceived an overall improvement in continuity of care. Petersen and colleagues implemented a computerized sign‐out (auto‐imported medications, name, room number) in an internal medicine residency to improve continuity of care during cross‐coverage and decrease adverse events.17 Prior to the intervention, the frequency of preventable adverse events was 1.7% and it was significantly associated with cross‐coverage. Preventable adverse events were identified using a confidential self‐report system that was also validated by clinician review. After the intervention, the frequency of preventable adverse events dropped to 1.2% (P < 0.1), and cross‐coverage was no longer associated with preventable adverse events. In other studies, technological solutions also improved provider identification and staff communication.18, 19 Together, these technology‐based intervention studies suggest that a computerized sign‐out with auto‐imported fields has the ability to improve physician efficiency and also improve inpatient care (reduction in number of patients missed on rounds, decrease in preventable adverse events).

Studies from nursing demonstrated that supplementing a verbal exchange with written information improved transfer of information, compared to verbal exchange alone.20 One of these studies rated the transfer of information using videotaped simulated handoff cases.21 Last, 1 nursing study that more directly involved patients in the handoff process resulted in improved nursing knowledge and greater patient empowerment (Table 1).22

White papers or consensus statements originated from international and national consortia in patient safety including the Australian Council for Safety and Quality in Healthcare,23 the Junior Doctors Committee of the British Medical Association,24 University Health Consortium,25 the Department of Defense Patient Safety Program,26 and The Joint Commission.27 Several common themes were prevalent in all white papers. First, there exists a need to train new personnel on how to perform an effective handoff. Second, efforts should be undertaken to ensure adequate time for handoffs and reduce interruptions during handoffs. Third, several of the papers supported verbal exchange that facilitates interactive questioning, focuses on ill patients, and delineates actions to be taken. Lastly, content should be updated to ensure transfer of the latest clinical information.

Final Handoff Recommendations

The final handoff recommendations are shown in Figure 2. The recommendations were designed to be consistent with the overall finding of the literature review, which supports the use of a verbal handoff supplemented with written documentation or a technological solution in a structured format. With the exception of 1 recommendation that is specific to service changes, all recommendations are designed to refer to shift changes and service changes. One overarching recommendation refers to the need for a formally recognized handoff plan at a shift change or change of service. The remaining 12 recommendations are divided into 4 that refer to hospitalist groups or programs, 3 that refer to verbal exchange, and 5 that refer to content exchange. The distinction is an important one because program‐level recommendations require organizational support and buy‐in to promote clinician participation and adherence. The 4 program recommendations also form the necessary framework for the remaining recommendations. For example, the second program recommendation describes the need for a standardized template or technology solution for accessing and recording patient information during the handoff. After a program adopts such a mechanism for exchanging patient information, the specific details for use and maintenance are outlined in greater detail in content exchange recommendations.

Figure 2

Because of the limited trials of handoff strategies, none of the recommendations are supported with level of evidence A (multiple numerous randomized controlled trials). In fact, with the exception of using a template or technology solution which was supported with level of evidence B, all handoff recommendations were supported with C level of evidence. The recommendations, however, were rated as Class I (effective) because there were no conflicting expert opinions or studies (Figure 2).

Discussion

In summary, our review of the literature supports the use of face‐to‐face verbal handoffs that are aided by the use of structured template to guide exchange of information. Furthermore, the development of these recommendations is the first effort of its kind for hospitalist handoffs and a movement towards standardizing the handoff process. While these recommendations are meant to provide structure to the hospitalist handoff process, the use and implementation by individual hospitalist programs may require more specific detail than these recommendations provide. Local modifications can allow for improved acceptance and adoption by practicing hospitalists. These recommendations can also help guide teaching efforts for academic hospitalists who are responsible for supervising residents.

The limitations of these recommendations related to lack of evidence in this field. Studies suffered from small size, poor description of methods, and a paucity of controlled interventions. The described technology solutions are not standardized or commercially available. Only 1 study included patient outcomes.28 There are no multicenter studies, studies of hospitalist handoffs, or studies to guide inclusion of specific content. Randomized controlled trials, interrupted time series analyses, and other rigorous study designs are needed in both teaching and non‐teaching settings to evaluate these recommendations and other approaches to improving handoffs. Ideally, these studies would occur through multicenter collaboratives and with human factors researchers familiar with mixed methods approaches to evaluate how and why interventions work.29 Efforts should focus on developing surrogate measures that are sensitive to handoff quality and related to important patient outcomes. The results of future studies should be used to refine the present recommendations. Locating new literature could be facilitated through the introduction of Medical Subject Heading for the term handoff by the National Library of Medicine. After completing this systematic review and developing the handoff recommendations described here, a few other noteworthy articles have been published on this topic, to which we refer interested readers. Several of these studies demonstrate that standardizing content and process during medical or surgical intern sign‐out improves resident confidence with handoffs,30 resident perceptions of accuracy and completeness of signout,31 and perceptions of patient safety.32 Another prospective audiotape study of 12 days of resident signout of clinical information demonstrated that poor quality oral sign‐outs was associated with an increased risk of post‐call resident reported signout‐related problems.5 Lastly, 1 nursing study demonstrated improved staff reports of safety, efficiency, and teamwork after a change from verbal reporting in an isolated room to bedside handover.33 Overall, these additional studies continue to support the current recommendations presented in this paper and do not significantly impact the conclusions of our literature review.

While lacking specific content domain recommendations, this report can be used as a starting point to guide development of self and peer assessment of hospitalist handoff quality. Development and validation of such assessments is especially important and can be incorporated into efforts to certify hospitalists through the recently approved certificate of focused practice in hospital medicine by the American Board of Internal Medicine (ABIM). Initiatives by several related organizations may help guide these effortsThe Joint Commission, the ABIM's Stepping Up to the Plate (SUTTP) Alliance, the Institute for Healthcare Improvement, the Information Transfer and Communication Practices (ITCP) Project for surgical care transitions, and the Hospital at Night (H@N) Program sponsored by the United Kingdom's National Health Service.3437 Professional medical organizations can also serve as powerful mediators of change in this area, not only by raising the visibility of handoffs, but also by mobilizing research funding. Patients and their caregivers may also play an important role in increasing awareness and education in this area. Future efforts should target handoffs not addressed in this initiative, such as transfers from emergency departments to inpatient care units, or between ICUs and the medical floor.

Conclusion

With the growth of hospital medicine and the increased acuity of inpatients, improving handoffs becomes an important part of ensuring patient safety. The goal of the SHM Handoffs Task Force was to begin to standardize handoffs at change of shift and change of servicea fundamental activity of hospitalists. These recommendations build on the limited literature in surgery, nursing, and medical informatics and provide a starting point for promoting safe and seamless in‐hospital handoffs for practitioners of Hospital Medicine.

In 2007, the Joint Commission for Accreditation of Healthcare Organizations (JCAHO) recommended deployment of rapid response teams (RRTs) in U.S. hospitals to hasten identification and treatment of physiologically unstable hospitalized patients.1 Clinical studies that have focused on whether RRTs improve restorative care outcomes, frequency of cardiac arrest, and critical care utilization have yielded mixed results.2‐11 One study suggested that RRTs might provide an opportunity to enhance palliative care of hospitalized patients.11 In this study, RRT personnel felt that prior do‐not‐resuscitate orders would have been appropriate in nearly a quarter of cases. However, no previous study has examined whether the RRT might be deployed to identify acutely decompensating patients who either do not want or would not benefit from a trial of aggressive restorative treatments. We hypothesized that actuation of an RRT in our hospital would expedite identification of patients not likely to benefit from restorative care and would promote more timely commencement of end‐of‐life comfort care, thereby improving their quality of death (QOD).12‐16

Materials and Methods

Results

A total of 394 patients died on the hospital wards and were not admitted with palliative, end‐of‐life medical therapies. The combined (pre‐RRT and post‐RRT epochs) cohort had a mean age of 77.2 13.2 years. A total of 48% were male, 79% White, 12% Black, and 8% Hispanic. A total of 128 patients (33%) were admitted to the hospital from a skilled nursing facility and 135 (35%) had written advance directives.

A total of 197 patients met the inclusion criteria during the pre‐RRT (October 1, 2005 to May 31, 2006) and 197 during the post‐RRT epochs (October 1, 2006 to May 31, 2007). There were no differences in age, sex, advance directives, ethnicity, or religion between the groups (Table 1). Primary admission diagnoses were significantly different; pre‐RRT patients were 9% more likely to die with malignancy compared to post‐RRT patients and less likely to come from nursing homes (38% vs. 27%; P = 0.02).

Discussion

This pilot study hypothesizes that our RRT impacted patients' QOD. Deployment of the RRT in our hospital was associated with improvement in both symptom and psychospiritual domains of care. Theoretically, RRTs should improve quality‐of‐care via early identification/reversal of physiologic decompensation. By either reversing acute diatheses with an expeditious trial of therapy or failing to reverse with early actuation of palliative therapies, the duration and magnitude of human suffering should be reduced. Attenuation of both duration and magnitude of suffering is the ultimate goal of both restorative and palliative care and is as important an outcome as mortality or length of stay. Previous studies of RRTs have focused on efficacy in reversing the decompensation: preventing cardiopulmonary arrest, avoiding the need for invasive, expensive, labor‐intensive interventions. Our RRT, like others, had no demonstrable impact on restorative outcomes. However, deployment of the RRT was highly associated with improved QOD of our patients. The impact was significant across WHO‐specified domains: pain scores decreased by 19%; (documentation of) patients' distress decreased by 50%; and chaplains' visits were more often documented in the 24 hours prior to death. These relationships held across common disease diagnoses, so the association is unlikely to be spurious.

Outcomes were similarly improved in patients who did not receive RRT care in the post‐RRT epoch. Our hospital did not have a palliative care service in either time period. No new educational efforts among physicians or nurses accounted for this observation. While it is possible that temporal effects accounted for our observation, an equally plausible explanation is that staff observed RRT interventions and applied them to dying patients not seen by the RRT. Our hospital educated caregivers regarding the RRT triggers, and simply making hospital personnel more vigilant for signs of suffering and/or observing the RRT approach may have contributed to enhanced end‐of‐life care for non‐RRT patients.

There are a number of limitations in this study. First, the sample size was relatively small compared to other published studies,2‐11 promoting the possibility that either epoch was not representative of pre‐RRT and post‐RRT parent populations. Another weakness is that QOD was measured using surrogate endpoints. The dead cannot be interviewed to definitively examine QOD; indices of cardiopulmonary distress and psychosocial measures (eg, religious preparations, family involvement) are endpoints suggested by palliative care investigators12, 13 and the World Health Organization.14 While some validated tools17 and consensus measures18 exist for critically ill patients, they do not readily apply to RRT patients. Retrospective records reviews raise the possibility of bias in extracting objective and subjective data. While we attempted to control for this by creating uniform a priori rules for data acquisition (ie, at what intervals and in which parts of the record they could be extracted), we cannot discount the possibility that bias affected the observed results. Finally, improvements in end‐of‐life care could have resulted from temporal trends. This retrospective study cannot prove a causeeffect relationship; a prospective randomized trial would be required to answer the question definitively. Based on the available data suggesting some benefit in restorative outcomes2‐8 and pressure from federal regulators to deploy RRTs regardless,1 a retrospective cohort design may provide the only realistic means of addressing this question.

In conclusion, this is the first (pilot) study to examine end‐of‐life outcomes associated with deployment of an RRT. While the limitations of these observations preclude firm conclusions, the plausibility of the hypothesis, coupled with our observations, suggests that this is a fertile area for future research. While RRTs may enhance restorative outcomes, to the extent that they hasten identification of candidates for palliative end‐of‐life‐care, before administration of invasive modalities that some patients do not want, these teams may simultaneously serve patients and reduce hospital resource utilization.

Addendum

Prior to publication, a contemporaneous study was published that concluded: These findings suggest that rapid response teams may not be decreasing code rates as much as catalyzing a compassionate dialogue of end‐of‐life care among terminally ill patients. This ability to improve end‐of‐life care may be an important benefit of rapid response teams, particularly given the difficulties in prior trials to increase rates of DNR status among seriously ill inpatients and potential decreases in resource use. Chan PS, Khalid A, Longmore LS, Berg RA, Midhail Kosiborod M, Spertus JA. Hospital‐wide code rates and mortality before and after implementation of a rapid response team. JAMA 2008;300: 25062513.

In 2007, the Joint Commission for Accreditation of Healthcare Organizations (JCAHO) recommended deployment of rapid response teams (RRTs) in U.S. hospitals to hasten identification and treatment of physiologically unstable hospitalized patients.1 Clinical studies that have focused on whether RRTs improve restorative care outcomes, frequency of cardiac arrest, and critical care utilization have yielded mixed results.2‐11 One study suggested that RRTs might provide an opportunity to enhance palliative care of hospitalized patients.11 In this study, RRT personnel felt that prior do‐not‐resuscitate orders would have been appropriate in nearly a quarter of cases. However, no previous study has examined whether the RRT might be deployed to identify acutely decompensating patients who either do not want or would not benefit from a trial of aggressive restorative treatments. We hypothesized that actuation of an RRT in our hospital would expedite identification of patients not likely to benefit from restorative care and would promote more timely commencement of end‐of‐life comfort care, thereby improving their quality of death (QOD).12‐16

Materials and Methods

Results

A total of 394 patients died on the hospital wards and were not admitted with palliative, end‐of‐life medical therapies. The combined (pre‐RRT and post‐RRT epochs) cohort had a mean age of 77.2 13.2 years. A total of 48% were male, 79% White, 12% Black, and 8% Hispanic. A total of 128 patients (33%) were admitted to the hospital from a skilled nursing facility and 135 (35%) had written advance directives.

A total of 197 patients met the inclusion criteria during the pre‐RRT (October 1, 2005 to May 31, 2006) and 197 during the post‐RRT epochs (October 1, 2006 to May 31, 2007). There were no differences in age, sex, advance directives, ethnicity, or religion between the groups (Table 1). Primary admission diagnoses were significantly different; pre‐RRT patients were 9% more likely to die with malignancy compared to post‐RRT patients and less likely to come from nursing homes (38% vs. 27%; P = 0.02).

Discussion

This pilot study hypothesizes that our RRT impacted patients' QOD. Deployment of the RRT in our hospital was associated with improvement in both symptom and psychospiritual domains of care. Theoretically, RRTs should improve quality‐of‐care via early identification/reversal of physiologic decompensation. By either reversing acute diatheses with an expeditious trial of therapy or failing to reverse with early actuation of palliative therapies, the duration and magnitude of human suffering should be reduced. Attenuation of both duration and magnitude of suffering is the ultimate goal of both restorative and palliative care and is as important an outcome as mortality or length of stay. Previous studies of RRTs have focused on efficacy in reversing the decompensation: preventing cardiopulmonary arrest, avoiding the need for invasive, expensive, labor‐intensive interventions. Our RRT, like others, had no demonstrable impact on restorative outcomes. However, deployment of the RRT was highly associated with improved QOD of our patients. The impact was significant across WHO‐specified domains: pain scores decreased by 19%; (documentation of) patients' distress decreased by 50%; and chaplains' visits were more often documented in the 24 hours prior to death. These relationships held across common disease diagnoses, so the association is unlikely to be spurious.

Outcomes were similarly improved in patients who did not receive RRT care in the post‐RRT epoch. Our hospital did not have a palliative care service in either time period. No new educational efforts among physicians or nurses accounted for this observation. While it is possible that temporal effects accounted for our observation, an equally plausible explanation is that staff observed RRT interventions and applied them to dying patients not seen by the RRT. Our hospital educated caregivers regarding the RRT triggers, and simply making hospital personnel more vigilant for signs of suffering and/or observing the RRT approach may have contributed to enhanced end‐of‐life care for non‐RRT patients.

There are a number of limitations in this study. First, the sample size was relatively small compared to other published studies,2‐11 promoting the possibility that either epoch was not representative of pre‐RRT and post‐RRT parent populations. Another weakness is that QOD was measured using surrogate endpoints. The dead cannot be interviewed to definitively examine QOD; indices of cardiopulmonary distress and psychosocial measures (eg, religious preparations, family involvement) are endpoints suggested by palliative care investigators12, 13 and the World Health Organization.14 While some validated tools17 and consensus measures18 exist for critically ill patients, they do not readily apply to RRT patients. Retrospective records reviews raise the possibility of bias in extracting objective and subjective data. While we attempted to control for this by creating uniform a priori rules for data acquisition (ie, at what intervals and in which parts of the record they could be extracted), we cannot discount the possibility that bias affected the observed results. Finally, improvements in end‐of‐life care could have resulted from temporal trends. This retrospective study cannot prove a causeeffect relationship; a prospective randomized trial would be required to answer the question definitively. Based on the available data suggesting some benefit in restorative outcomes2‐8 and pressure from federal regulators to deploy RRTs regardless,1 a retrospective cohort design may provide the only realistic means of addressing this question.

In conclusion, this is the first (pilot) study to examine end‐of‐life outcomes associated with deployment of an RRT. While the limitations of these observations preclude firm conclusions, the plausibility of the hypothesis, coupled with our observations, suggests that this is a fertile area for future research. While RRTs may enhance restorative outcomes, to the extent that they hasten identification of candidates for palliative end‐of‐life‐care, before administration of invasive modalities that some patients do not want, these teams may simultaneously serve patients and reduce hospital resource utilization.

Addendum

Prior to publication, a contemporaneous study was published that concluded: These findings suggest that rapid response teams may not be decreasing code rates as much as catalyzing a compassionate dialogue of end‐of‐life care among terminally ill patients. This ability to improve end‐of‐life care may be an important benefit of rapid response teams, particularly given the difficulties in prior trials to increase rates of DNR status among seriously ill inpatients and potential decreases in resource use. Chan PS, Khalid A, Longmore LS, Berg RA, Midhail Kosiborod M, Spertus JA. Hospital‐wide code rates and mortality before and after implementation of a rapid response team. JAMA 2008;300: 25062513.

In 2007, the Joint Commission for Accreditation of Healthcare Organizations (JCAHO) recommended deployment of rapid response teams (RRTs) in U.S. hospitals to hasten identification and treatment of physiologically unstable hospitalized patients.1 Clinical studies that have focused on whether RRTs improve restorative care outcomes, frequency of cardiac arrest, and critical care utilization have yielded mixed results.2‐11 One study suggested that RRTs might provide an opportunity to enhance palliative care of hospitalized patients.11 In this study, RRT personnel felt that prior do‐not‐resuscitate orders would have been appropriate in nearly a quarter of cases. However, no previous study has examined whether the RRT might be deployed to identify acutely decompensating patients who either do not want or would not benefit from a trial of aggressive restorative treatments. We hypothesized that actuation of an RRT in our hospital would expedite identification of patients not likely to benefit from restorative care and would promote more timely commencement of end‐of‐life comfort care, thereby improving their quality of death (QOD).12‐16

Materials and Methods

Results

A total of 394 patients died on the hospital wards and were not admitted with palliative, end‐of‐life medical therapies. The combined (pre‐RRT and post‐RRT epochs) cohort had a mean age of 77.2 13.2 years. A total of 48% were male, 79% White, 12% Black, and 8% Hispanic. A total of 128 patients (33%) were admitted to the hospital from a skilled nursing facility and 135 (35%) had written advance directives.

A total of 197 patients met the inclusion criteria during the pre‐RRT (October 1, 2005 to May 31, 2006) and 197 during the post‐RRT epochs (October 1, 2006 to May 31, 2007). There were no differences in age, sex, advance directives, ethnicity, or religion between the groups (Table 1). Primary admission diagnoses were significantly different; pre‐RRT patients were 9% more likely to die with malignancy compared to post‐RRT patients and less likely to come from nursing homes (38% vs. 27%; P = 0.02).

Discussion

This pilot study hypothesizes that our RRT impacted patients' QOD. Deployment of the RRT in our hospital was associated with improvement in both symptom and psychospiritual domains of care. Theoretically, RRTs should improve quality‐of‐care via early identification/reversal of physiologic decompensation. By either reversing acute diatheses with an expeditious trial of therapy or failing to reverse with early actuation of palliative therapies, the duration and magnitude of human suffering should be reduced. Attenuation of both duration and magnitude of suffering is the ultimate goal of both restorative and palliative care and is as important an outcome as mortality or length of stay. Previous studies of RRTs have focused on efficacy in reversing the decompensation: preventing cardiopulmonary arrest, avoiding the need for invasive, expensive, labor‐intensive interventions. Our RRT, like others, had no demonstrable impact on restorative outcomes. However, deployment of the RRT was highly associated with improved QOD of our patients. The impact was significant across WHO‐specified domains: pain scores decreased by 19%; (documentation of) patients' distress decreased by 50%; and chaplains' visits were more often documented in the 24 hours prior to death. These relationships held across common disease diagnoses, so the association is unlikely to be spurious.

Outcomes were similarly improved in patients who did not receive RRT care in the post‐RRT epoch. Our hospital did not have a palliative care service in either time period. No new educational efforts among physicians or nurses accounted for this observation. While it is possible that temporal effects accounted for our observation, an equally plausible explanation is that staff observed RRT interventions and applied them to dying patients not seen by the RRT. Our hospital educated caregivers regarding the RRT triggers, and simply making hospital personnel more vigilant for signs of suffering and/or observing the RRT approach may have contributed to enhanced end‐of‐life care for non‐RRT patients.

There are a number of limitations in this study. First, the sample size was relatively small compared to other published studies,2‐11 promoting the possibility that either epoch was not representative of pre‐RRT and post‐RRT parent populations. Another weakness is that QOD was measured using surrogate endpoints. The dead cannot be interviewed to definitively examine QOD; indices of cardiopulmonary distress and psychosocial measures (eg, religious preparations, family involvement) are endpoints suggested by palliative care investigators12, 13 and the World Health Organization.14 While some validated tools17 and consensus measures18 exist for critically ill patients, they do not readily apply to RRT patients. Retrospective records reviews raise the possibility of bias in extracting objective and subjective data. While we attempted to control for this by creating uniform a priori rules for data acquisition (ie, at what intervals and in which parts of the record they could be extracted), we cannot discount the possibility that bias affected the observed results. Finally, improvements in end‐of‐life care could have resulted from temporal trends. This retrospective study cannot prove a causeeffect relationship; a prospective randomized trial would be required to answer the question definitively. Based on the available data suggesting some benefit in restorative outcomes2‐8 and pressure from federal regulators to deploy RRTs regardless,1 a retrospective cohort design may provide the only realistic means of addressing this question.

In conclusion, this is the first (pilot) study to examine end‐of‐life outcomes associated with deployment of an RRT. While the limitations of these observations preclude firm conclusions, the plausibility of the hypothesis, coupled with our observations, suggests that this is a fertile area for future research. While RRTs may enhance restorative outcomes, to the extent that they hasten identification of candidates for palliative end‐of‐life‐care, before administration of invasive modalities that some patients do not want, these teams may simultaneously serve patients and reduce hospital resource utilization.

Addendum

Prior to publication, a contemporaneous study was published that concluded: These findings suggest that rapid response teams may not be decreasing code rates as much as catalyzing a compassionate dialogue of end‐of‐life care among terminally ill patients. This ability to improve end‐of‐life care may be an important benefit of rapid response teams, particularly given the difficulties in prior trials to increase rates of DNR status among seriously ill inpatients and potential decreases in resource use. Chan PS, Khalid A, Longmore LS, Berg RA, Midhail Kosiborod M, Spertus JA. Hospital‐wide code rates and mortality before and after implementation of a rapid response team. JAMA 2008;300: 25062513.