Affiliations
The Feinstein Institute for Medical Research, North Shore–Long Island Jewish Health System, Great Neck, New York
Given name(s)
Renee
Family name
Pekmezaris
Degrees
PhD

Implementing ACOVE quality indicators as an intervention checklist to improve care for hospitalized older adults

Article Type
Changed
Wed, 07/19/2017 - 13:49
Display Headline
Implementing ACOVE quality indicators as an intervention checklist to improve care for hospitalized older adults

In 2014, the United States spent $3 trillion on healthcare; hospitalization consumed 32% of these expenditures.1 Today, Medicare patients account for over 50% of hospital days and over 30% of all hospital discharges in the United States.2 Despite this staggering financial burden, hospitalization of older adults often results in poor patient outcomes.3-6 The exponential growth of the hospitalist movement, from 350 hospitalists nationwide in 1995 to over 44,000 in 2014, has become the key strategy for providing care to hospitalized geriatric patients.7-10 Most of these hospitalists have not received geriatric training.11-15

There is growing evidence that a geriatric approach, emphasizing multidisciplinary management of the complex needs of older patients, leads to improved outcomes. Geriatric Evaluation and Management Units (GEMUs), such as Acute Care for Elderly (ACE) models, have demonstrated significant decreases in functional decline, institutionalization, and death in randomized controlled trials.16,17 Multidisciplinary, nonunit based efforts, such as the mobile acute care of elderly (MACE), proactive consultation models (Sennour/Counsell), and the Hospital Elder Life Program (HELP), have demonstrated success in preventing adverse events and decreasing length of stay (LOS).17-20

However, these models have not been systematically implemented due to challenges in generalizability and replicability in diverse settings. To address this concern, an alternative approach must be developed to widely “generalize” geriatric expertise throughout hospitals, regardless of their location, size, and resources. This initiative will require systematic integration of evidence-based decision support tools for the standardization of clinical management in hospitalized older adults.21

The 1998 Assessing Care of Vulnerable Elders (ACOVE) project developed a standardized tool to measure and evaluate the quality of care by using a comprehensive set of quality indicators (QIs) to improve the care of “vulnerable elders” (VEs) at a high risk for functional and cognitive decline and death.22-24 The latest systematic review concludes that, although many studies have used ACOVE as an assessment tool of quality, there has been a dearth of studies investigating the ACOVE QIs as an intervention to improve patient care.25

Our study investigated the role of ACOVE as an intervention by using the QIs as a standardized checklist in the acute care setting. We selected the 4 most commonly encountered QIs in the hospital setting, namely venous thrombosis prophylaxis (VTE), indwelling bladder catheter, mobilization, and delirium evaluation, in order to test the feasibility and impact of systematically implementing these ACOVE QIs as a therapeutic intervention for all hospitalized older adults.

METHODS

This study (IRB #13-644B) was conducted using a prospective intervention with a nonequivalent control group design comprised of retrospective chart data from May 1, 2014, to June 30, 2015. Process and outcome variables were extracted from electronic medical records ([EMR], Sunrise Clinical Manager [SCM]) of 2,396 patients, with 530 patients in the intervention unit and 1,866 on the control units, at a large academic tertiary center operating in the greater New York metropolitan area. Our study investigated the role of ACOVE as an intervention to improve patient care by using selected QIs as a standardized checklist tool in the acute care setting. Of the original 30 hospital-specific QIs, our study focused on the care of older adults admitted to the medicine service.26 We selected commonly encountered QIs, with the objective of testing the feasibility and impact of implementing the ACOVE QIs as an intervention to improve care of hospitalized older adults. This intervention consisted of applying the checklist tool, constructed with 4 selected ACOVE QIs and administered daily during interdisciplinary rounds, namely: 2 general “medical” indicators, VTE prophylaxis and indwelling bladder catheters, and 2 “geriatric”-focused indicators, mobilization and delirium evaluation.

Integration of Selected ACOVE QIs Into a Checklist-Based Tool for Decision Support
Table 1

 

 

Subject matter experts (hospitalists, geriatricians, researchers, administrators, and nurses) reviewed the ACOVE QIs and agreed upon the adaptation of the QIs from a quality measure assessment into a feasible and acceptable intervention checklist tool (Table 1). The checklist was reviewed during daily interdisciplinary rounds for all patients 75 years and older. While ACOVE defined vulnerable elders by using the Vulnerable Elder Screen (VES), we wanted to apply this intervention more broadly to all hospitalized older adults who are most at risk for poor outcomes.27 Patients admitted to the intensive care unit, inpatient psychiatry, inpatient leukemia/lymphoma, and surgical services were excluded.

Daily interdisciplinary rounds are held on every one of the five 40-bed medical units; they last approximately 1 hour, and consist of a lead hospitalist, nurse manager, nurse practitioners, case managers, and the nursing staff. During interdisciplinary rounds, nurses present the case to the team members who then discuss the care plan. These 5 medical units did not differ in terms of patient characteristics or staffing patterns; the intervention unit was chosen simply for logistical reasons, in that the principal investigator (PI) had been assigned to this unit prior to study start-up.

Prior to the intervention, LS held an education session for staff on the intervention unit staff (who participated on interdisciplinary rounds) to explain the concept of the ACOVE QI initiative and describe the four QIs selected for the study. Three subsequent educational sessions were held during the first week of the intervention, with new incoming staff receiving a brief individual educational session. The staff demonstrated significant knowledge improvement after session completion (pre/post mean score 70.6% vs 90.0%; P < .0001).

The Clinical Information System for the Health System EMR, The Eclipsys SCM, has alerts with different levels of severity from “soft” (user must acknowledge a recommendation) to “hard” (requires an action in order to proceed).

To measure compliance of the quality indicators, we collected the following variables:

QI 1: VTE prophylaxis

Through SCM, we collected type of VTE prophylaxis ordered (pharmacologic and/or mechanical) as well as start and stop dates for all agents. International normalized ratio levels were checked for patients receiving warfarin. Days of compliance were calculated.

QI 2: Indwelling Bladder Catheters

SCM data were collected on catheter entry and discontinuation dates, the presence of an indication, and order renewal for bladder catheter at least every 3 days.

QI 3: Mobilization

Ambulation status prior to admission was extracted from nursing documentation completed on admission to the medical ward. Patients documented as bedfast were categorized as nonambulatory prior to admission. Nursing documentation of activity level and amount of feet ambulated per nursing shift were collected. In addition, hospital day of physical therapy (PT) order and hospital days with PT performed were charted. Compliance with QI 3 in patients documented as ambulatory prior to hospital admission was recorded as present if there was a PT order within 48 hours of admission.

QI 4: Delirium Evaluation

During daily rounds, the hospitalist (PI) questioned nurses about delirium evaluation, using the first feature of the Confusion Assessment Method (CAM) as well as the “single question in delirium,” namely, “Is there evidence of an acute change in mental status from the patient’s baseline?” and “Do you think [name of patient] has been more confused lately?”28,29 Because EMR does not contain a specified field for delirium screening and documentation, and patients are not routinely included in rounds, documentation with QI 4 was recorded using the “key words” method as described in the work by Puelle et al.30 To extract SCM key words, nursing documentation of the “cognitive/perceptual/neurological exam” section of the EMR on admission and on all subsequent documentation (once per shift) was retrieved to identify acute changes in mental status (eg, “altered mental status, delirium/delirious, alert and oriented X 3, confused/confusion, disoriented, lethargy/lethargic”).30 In addition, nurses were asked to activate an SCM parameter, “Acute Confusion” SCM parameter, in the nursing documentation section, which includes potential risk factors for confusion.

In addition to QI compliance, we collected LOS, discharge disposition, and 30-day readmission data.

Generalized linear mixed models (GLMM) for binary clustered (ie, hierarchical) data were used to estimate compliance rates (ie, nurse adherence) for each group (intervention group or control group) in the postintervention period, along with their corresponding 95% confidence intervals. GLMM was used to account for the hierarchical structure of the data: nursing units within a hospital. In order to calculate the Charlson Comorbidity Index, we extracted past medical history from the EMR.31

Subjects (N = 2,396) were included in the comparison of the intervention group vs control group for each of the following 4 ACOVE QI compliance measures: DVT, mobilization, bladder catheter, and delirium.

Patient Characteristics
Table 2

 

 

RESULTS

Of the 2,396 patient admissions, 530 were in the intervention unit and 1,866 were in the control unit. In the intervention group, the average age was 84.65 years, 75.58% were white and 47.21% were married. There was no difference in patient demographics between groups (Table 2).

 

QI 1: VTE Prophylaxis

Compliance with VTE prophylaxis was met in 78.3% of the intervention subjects and 76.5% of the controls (P < .4371) (Table 3). Of note, the rate of VTE prophylaxis was 57% in the intervention vs 39% in the control group (P < .0056), in the 554 patients for whom compliance was not met. Mechanical prophylaxis was used in 35.6% of intervention subjects vs 30.6 in the control (P = .048). Patients who received no form of prophylaxis were 0.5% in the intervention and 3% in the control (P = .027).

Quality Indicator Outcomes/Measurements
Table 3

QI 2: Indwelling Bladder Catheters

Out of 2,396 subjects, 406 had an indwelling bladder catheter (16.9%). Compliance with the catheter was met in 72.2% of the intervention group vs 54.4% in the control group (P = .1061). An indication for indwelling bladder catheters was documented in 100% of the subjects. The average number of catheter days was 5.16 in the intervention vs 5.88 in the control (P < .2284). There was statistical significance in catheter compliance in the longer stay (>15 days) subjects, decreasing to 23.32% in the control group while staying constant in the intervention group 71.5% (P = .0006).

QI 3: Mobilization

Of the 2,396 patients, 1,991 (83.1%) were reported as ambulatory prior to admission. In the intervention vs control group, 74 (14%) vs 297 (15.7%), respectively, were nonambulatory. Overall compliance with Q3 was 62.9% in the intervention vs 48.2% in the control (P < .0001). More specifically, the average time to PT order in the intervention group was 1.83 days vs 2.22 days in the control group (P < .0051) and the time to PT evaluation was 2.14 days vs 2.42 days, respectively (P < .0108). In the intervention group, 84 patients (15.8%) did not have a PT consult vs 511 (27%) in the control group (P < .0001). The average times per subject in which the nurses documented the approximate number of feet ambulated was 6.48 in the intervention group vs 0.11 in the control group.

QI 4: Delirium Evaluation

In terms of nursing documentation indicating the presence of an acute confusional state, the intervention group had 148 out of 530 nursing notes (27.9%) vs 405 out of 1,866 in the control group (21.7%; P = .0027). However, utilization of the “acute confusion” parameter with documentation of a risk factor did not differ between the groups (5.8% in the intervention group vs 5.6% in the control group, P < .94).

LOS, Discharge Disposition, and 30-Day Readmissions

LOS did not differ between intervention and control groups (6.37 days vs 6.27 days, respectively), with a median of 5 days (P = .877). Discharge disposition in the 2 groups included the following: home/home with services (71.32% vs 68.7%), skilled nursing facility/assisted living/long-term care (24.34 versus 25.83), inpatient hospice/home hospice (2.64 vs 2.25), and expired (1.13 vs 1.77; P < .3282). In addition, 30-day readmissions did not differ (21% vs 20%, respectively, P = .41).

DISCUSSION

Our goal was to explore an evidence-based, standardized approach to improve the care of hospitalized older adults. This approach leverages existing automated EMR alert functions with an additional level of decision support for VEs, integrated into daily multidisciplinary rounds. The use of a daily checklist-based tool offers a cost-effective and practical pathway to distribute the burden of compliance responsibility amongst team members.

As we anticipated and similar to study findings in hospitalized medicine, geriatric trauma, and primary care, compliance with general care QIs was better than geriatric-focused QIs.27,32 Wenger et al33 demonstrated significant improvements with screening for falls and incontinence; however, screening for cognitive impairment did not improve in the outpatient setting by imbedding ACOVE QIs into routine physician practice.

Increased compliance with VTE prophylaxis and indwelling bladder catheters may be explained by national financial incentives for widespread implementation of EMR alert systems. Conversely, mobilization, delirium assessment, and management in hospitalized older adults don’t benefit from similar incentives.

VTE Prophylaxis

The American College of Chest Physicians (ACCP) supports the use of VTE prophylaxis, especially in hospitalized older adults with decreased mobility.34 While greater adoption of EMR has already increased adherence, our intervention resulted in an even higher rate of compliance with the use of pharmacologic VTE prophylaxis.35 In the future, validated scores for risk of thrombosis and bleeding may be integrated into our QI-based checklist.

 

 

Indwelling Bladder Catheters

The potential harms of catheters have been described for over 50 years, yet remain frequently used.36,37 Previous studies have shown success in decreasing catheter days with computer-based and multidisciplinary protocols.36-39

Our health system’s EMR has built-in “soft” and “hard” alerts for indwelling bladder catheters, so we did not expect intervention-associated changes in compliance.

Mobilization

Hospitalization in older adults frequently results in functional decline.4,5,40 In response, the mobilization QI recommends an ambulation plan within 48 hours for those patients who were ambulatory prior to admission; it does not specifically define the components of the plan.26 There are several multicomponent interventions that have demonstrated improvement in functional decline, yet they require skilled providers.41,42 Our intervention implemented specific ambulation plan components: daily ambulation and documentation reminders and early PT evaluation.

While functional status measures have existed for decades, most are primarily geared to assess community-residing individuals and not designed to measure changes in function during hospitalization.43,44 Furthermore, performance-based hospital measures are difficult to integrate into the daily nursing workflow as they are time consuming.45,46 In practice, nurses routinely use free text to document functional status in the hospital setting, rendering comparative analysis problematic. Yet, we demonstrated that nurses were more engaged in reporting mobilization (increased documentation of ambulation distance and a decrease in time to PT). Future research should focus on the development of a standardized tool, integrated into the EMR, to accurately measure function in the acute care setting.

Delirium Evaluation

Delirium evaluation remains one of the most difficult clinical challenges for healthcare providers in hospitalized individuals, and our study reiterated these concerns. Previous research has consistently demonstrated that the diagnosis of delirium is missed by up to 75% of clinicians.47,48 Indeed, our study, which exclusively examined nursing documentation of the delirium evaluation QI, found that both groups showed strikingly low compliance rates. This may have been due to the fact that we only evaluated nursing documentation of suspected or definite diagnosis of delirium and a documented attempt to attribute the altered mental state to a potential etiology.31 By utilizing the concept of “key words,” as developed by Puelle et al.30, we were able to demonstrate a statistically significant improvement in nursing delirium documentation in the intervention group. This result should be interpreted with caution, as this approach is not validated. Furthermore, our operational definition of delirium compliance (ie, nurse documentation of delirium, requiring the launching of a separate parameter) may have been simply too cumbersome to readily integrate into the daily workflow. Future research should study the efficacy of a sensitive EMR-integrated screening tool that facilitates recognition, by all team members, of acute changes in cognition.

Although a number of QI improved for the intervention group, acute care utilization measures such as LOS, discharge disposition, and 30-day readmissions did not differ between groups. It may well be that improving quality for this very frail, vulnerable population may simply not result in decreased utilization. Our ability to further decrease LOS and readmission rates may be limited due to restriction of range in this complex patient population (eg, median LOS value of 5 days).

Limitations

Although our study had a large sample size, data were only collected from a single-center and thus require further exploration in different settings to ensure generalizability. In addition, QI observance was based on the medical record, which was problematic for some indicators, notably delirium identification. While prior literature highlights the difficulty in identifying delirium, especially during clinical practice without specialized training, our compliance was strikingly low.47 While validated measures such as CAM may have been included as part of the assessment, there is currently no EMR documentation of such measures and therefore, these data could not be obtained.

CONCLUSION

In summary, our study demonstrates the successful integration of the established ACOVE QIs as an intervention, rather than as an assessment method, for improving care of hospitalized older patients. By utilizing a checklist-based tool at the bedside allows the multidisciplinary team to implement evidence-based practices with the ultimate goal of standardizing care, not only for VEs, but potentially for other high-risk populations with multimorbidity.49 This innovative approach provides a much-needed direction to healthcare providers in the ever increasing stressful conditions of today’s acute care environment and for the ultimate benefit and safety of our older patients.

Disclosure

The authors declare no conflicts of interest. This study was supported by New York State Empire Clinical Research Investigators Program (ECRIP). The sponsor had no role in the conception, study design, data collection, data analysis, interpretation of data, manuscript preparation, or the decision to submit the manuscript for publication.

 

 

 

References

1. National Center for Health Statistics (US). Health, United States, 2015: With Special Feature on Racial and Ethnic Health Disparities. Hyattsville, MD: National Center for Health Statistics (US); 2016. http://www.ncbi.nlm.nih.gov/books/NBK367640/. Accessed November 2, 2016.
2. Weiss AJ, Elixhauser A. Overview of Hospital Stays in the United States, 2012: Statistical Brief #180. In: Healthcare Cost and Utilization Project (HCUP) Statistical Briefs. Rockville (MD): Agency for Healthcare Research and Quality (US); 2006. http://www.ncbi.nlm.nih.gov/books/NBK259100/. Accessed November 2, 2016.
3. Jencks SF, Cuerdon T, Burwen DR, et al. Quality of medical care delivered to medicare beneficiaries: A profile at state and national levels. JAMA. 2000;284(13):1670-1676. PubMed
4. Covinsky KE, Pierluissi E, Johnston C. Hospitalization-associated disability: “She was probably able to ambulate, but I’m not sure.” JAMA. 2011;306(16):1782-1793. PubMed
5. Creditor MC. Hazards of Hospitalization of the Elderly. Ann Intern Med. 1993;118(3):219-223. PubMed
6. Graf C. Functional decline in hospitalized older adults. Am J Nurs. 2006;106(1):58-67, NaN-68. PubMed
7. Wachter RM, Goldman L. The emerging role of “hospitalists” in the American health care system. N Engl J Med. 1996;335(7):514-517. PubMed
8. Lindenauer PK, Pantilat SZ, Katz PP, Wachter RM. Hospitalists and the practice of inpatient medicine: results of a survey of the National Association of Inpatient Physicians. Ann Intern Med. 1999;130(4 Pt 2):343-349. PubMed
9. Wachter RM. The hospitalist movement 5 years later. JAMA. 2002;287(4):487. PubMed
10. Shank B. 2016: Celebrating 20 years of hospital medicine and looking toward a bright future. Hosp Natl Assoc Inpatient Physicians. 2016. http://www.the-hospitalist.org/hospitalist/article/121925/2016-celebrating-20-years-hospital-medicine-and-looking-toward-bright. Accessed June 2, 2017.
11. Retooling for an Aging America: Building the Health Care Workforce. Washington, DC.: National Academies Press; 2008. http://www.nap.edu/catalog/12089. Accessed November 2, 2016.
12. Boult C, Counsell SR, Leipzig RM, Berenson RA. The urgency of preparing primary care physicians to care for older people with chronic illnesses. Health Aff Proj Hope. 2010;29(5):811-818. PubMed
13. Warshaw GA, Bragg EJ, Thomas DC, Ho ML, Brewer DE, Association of Directors of Geriatric Academic Programs. Are internal medicine residency programs adequately preparing physicians to care for the baby boomers? A national survey from the Association of Directors of Geriatric Academic Programs Status of Geriatrics Workforce Study. J Am Geriatr Soc. 2006;54(10):1603-1609. PubMed
14. Tanner CE, Eckstrom E, Desai SS, Joseph CL, Ririe MR, Bowen JL. Uncovering frustrations: A qualitative needs assessment of academic general internists as geriatric care providers and teachers. J Gen Intern Med. 2006;21(1):51-55. PubMed
15. Warshaw GA, Bragg EJ, Brewer DE, Meganathan K, Ho M. The development of academic geriatric medicine: progress toward preparing the nation’s physicians to care for an aging population. J Am Geriatr Soc. 2007;55(12):2075-2082. PubMed
16. Fox MT, Sidani S, Persaud M, et al. Acute care for elders components of acute geriatric unit care: Systematic descriptive review. J Am Geriatr Soc. 2013;61(6):939-946. PubMed
17. Palmer RM, Landefeld CS, Kresevic D, Kowal J. A medical unit for the acute care of the elderly. J Am Geriatr Soc. 1994;42(5):545-552.
18. Hung WW, Ross JS, Farber J, Siu AL. Evaluation of the Mobile Acute Care of the Elderly (MACE) service. JAMA Intern Med. 2013;173(11):990-996. PubMed
19. Sennour Y, Counsell SR, Jones J, Weiner M. Development and implementation of a proactive geriatrics consultation model in collaboration with hospitalists. J Am Geriatr Soc. 2009;57(11):2139-2145. PubMed
20. Ellis G, Whitehead MA, O’Neill D, Langhorne P, Robinson D. Comprehensive geriatric assessment for older adults admitted to hospital. Cochrane Database Syst Rev. 2011;(7):CD006211. PubMed
21. Mattison MLP, Catic A, Davis RB, et al. A standardized, bundled approach to providing geriatric-focused acute care. J Am Geriatr Soc. 2014;62(5):936-942. doi:10.1111/jgs.12780. PubMed
22. Wenger NS, Shekelle PG. Assessing care of vulnerable elders: ACOVE project overview. Ann Intern Med. 2001;135(8 Pt 2):642-646. PubMed
23. Wenger NS, Roth CP, Shekelle P, ACOVE Investigators. Introduction to the assessing care of vulnerable elders-3 quality indicator measurement set. J Am Geriatr Soc. 2007;55 Suppl 2:S247-S252. PubMed
24. Reuben DB, Roth C, Kamberg C, Wenger NS. Restructuring primary care practices to manage geriatric syndromes: the ACOVE-2 intervention. J Am Geriatr Soc. 2003;51(12):1787-1793. PubMed
25. Askari M, Wierenga PC, Eslami S, Medlock S, De Rooij SE, Abu-Hanna A. Studies pertaining to the ACOVE quality criteria: a systematic review. Int J Qual Health Care. 2012;24(1):80-87. PubMed
26. Arora VM, McGory ML, Fung CH. Quality indicators for hospitalization and surgery in vulnerable elders. J Am Geriatr Soc. 2007;55 Suppl 2:S347-S358. PubMed
27. Arora VM, Johnson M, Olson J, et al. Using assessing care of vulnerable elders quality indicators to measure quality of hospital care for vulnerable elders. J Am Geriatr Soc. 2007;55(11):1705-1711. PubMed
28. Sands M, Dantoc B, Hartshorn A, Ryan C, Lujic S. Single Question in Delirium (SQiD): testing its efficacy against psychiatrist interview, the Confusion Assessment Method and the Memorial Delirium Assessment Scale. Palliat Med. 2010;24(6):561-565. PubMed
29. Inouye SK, van Dyck CH, Alessi CA, Balkin S, Siegal AP, Horwitz RI. Clarifying confusion: the confusion assessment method. A new method for detection of delirium. Ann Intern Med. 1990;113(12):941-948. PubMed
30. Puelle MR, Kosar CM, Xu G, et al. The language of delirium: Keywords for identifying delirium from medical records. J Gerontol Nurs. 2015;41(8):34-42. PubMed
31. Quan H, Sundararajan V, Halfon P, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43(11):1130-1139. PubMed
32. Boult C, Boult L, Morishita L, Smith SL, Kane RL. Outpatient geriatric evaluation and management. J Am Geriatr Soc. 1998;46(3):296-302.33. Wenger NS, Roth CP, Shekelle PG, et al. A practice-based intervention to improve primary care for falls, urinary incontinence, and dementia. J Am Geriatr Soc. 2009;57(3):547-555. PubMed
34. Geerts WH. Prevention of Venous Thromboembolism: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines (8th Edition). Chest J. 2008;133(6_suppl):381S. 
35. Rosenman M, Liu X, Phatak H, et al. Pharmacological prophylaxis for venous thromboembolism among hospitalized patients with acute medical illness: An electronic medical records study. Am J Ther. 2016;23(2):e328-e335. PubMed
36. Ghanem A, Artime C, Moser M, Caceres L, Basconcillo A. Holy moley! Take out that foley! Measuring compliance with a nurse driven protocol for foley catheter removal to decrease utilization. Am J Infect Control. 2015;43(6):S51.
37. Cornia PB, Amory JK, Fraser S, Saint S, Lipsky BA. Computer-based order entry decreases duration of indwelling urinary catheterization in hospitalized patients. Am J Med. 2003;114(5):404-407. PubMed
38. Huang W-C, Wann S-R, Lin S-L, et al. Catheter-associated urinary tract infections in intensive care units can be reduced by prompting physicians to remove unnecessary catheters. Infect Control Hosp Epidemiol. 2004;25(11):974-978. PubMed
39. Topal J, Conklin S, Camp K, Morris V, Balcezak T, Herbert P. Prevention of nosocomial catheter-associated urinary tract infections through computerized feedback to physicians and a nurse-directed protocol. Am J Med Qual. 2005;20(3):121-126. PubMed
40. Zisberg A, Shadmi E, Gur-Yaish N, Tonkikh O, Sinoff G. Hospital-associated functional decline: the role of hospitalization processes beyond individual risk factors. J Am Geriatr Soc. 2015;63(1):55-62. PubMed
41. Inouye SK, Bogardus ST, Baker DI, Leo-Summers L, Cooney LM. The Hospital Elder Life Program: a model of care to prevent cognitive and functional decline in older hospitalized patients. Hospital Elder Life Program. J Am Geriatr Soc. 2000;48(12):1697-1706. PubMed
42. Hoyer EH, Friedman M, Lavezza A, et al. Promoting mobility and reducing length of stay in hospitalized general medicine patients: A quality-improvement project. J Hosp Med. 2016;11(5):341-347. PubMed
43. Mahoney FI, Barthel DW. Functional evaluation: the barthel index. Md State Med J. 1965;14:61-65. PubMed
44. Katz S, Ford AB, Moskowitz RW, Jackson BA, Jaffe MW. Studies of illness in the aged. the index of adl: a standardized measure of biological and psychosocial function. JAMA. 1963;185:914-919. PubMed
45. Tinetti ME. Performance-oriented assessment of mobility problems in elderly patients. J Am Geriatr Soc. 1986;34(2):119-126. PubMed
46. Smith R. Validation and Reliability of the Elderly Mobility Scale. Physiotherapy. 1994;80(11):744-747. 
47. Inouye SK, Foreman MD, Mion LC, Katz KH, Cooney LM. Nurses’ recognition of delirium and its symptoms: comparison of nurse and researcher ratings. Arch Intern Med. 2001;161(20):2467-2473. PubMed
48. Gustafson Y, Brännström B, Norberg A, Bucht G, Winblad B. Underdiagnosis and poor documentation of acute confusional states in elderly hip fracture patients. J Am Geriatr Soc. 1991;39(8):760-765. PubMed
49. Brenner SK, Kaushal R, Grinspan Z, et al. Effects of health information technology on patient outcomes: a systematic review. J Am Med Inform Assoc. 2016;23(5):1016-1036. PubMed

 

 

Article PDF
Issue
Journal of Hospital Medicine 12(7)
Publications
Topics
Page Number
517-522
Sections
Article PDF
Article PDF

In 2014, the United States spent $3 trillion on healthcare; hospitalization consumed 32% of these expenditures.1 Today, Medicare patients account for over 50% of hospital days and over 30% of all hospital discharges in the United States.2 Despite this staggering financial burden, hospitalization of older adults often results in poor patient outcomes.3-6 The exponential growth of the hospitalist movement, from 350 hospitalists nationwide in 1995 to over 44,000 in 2014, has become the key strategy for providing care to hospitalized geriatric patients.7-10 Most of these hospitalists have not received geriatric training.11-15

There is growing evidence that a geriatric approach, emphasizing multidisciplinary management of the complex needs of older patients, leads to improved outcomes. Geriatric Evaluation and Management Units (GEMUs), such as Acute Care for Elderly (ACE) models, have demonstrated significant decreases in functional decline, institutionalization, and death in randomized controlled trials.16,17 Multidisciplinary, nonunit based efforts, such as the mobile acute care of elderly (MACE), proactive consultation models (Sennour/Counsell), and the Hospital Elder Life Program (HELP), have demonstrated success in preventing adverse events and decreasing length of stay (LOS).17-20

However, these models have not been systematically implemented due to challenges in generalizability and replicability in diverse settings. To address this concern, an alternative approach must be developed to widely “generalize” geriatric expertise throughout hospitals, regardless of their location, size, and resources. This initiative will require systematic integration of evidence-based decision support tools for the standardization of clinical management in hospitalized older adults.21

The 1998 Assessing Care of Vulnerable Elders (ACOVE) project developed a standardized tool to measure and evaluate the quality of care by using a comprehensive set of quality indicators (QIs) to improve the care of “vulnerable elders” (VEs) at a high risk for functional and cognitive decline and death.22-24 The latest systematic review concludes that, although many studies have used ACOVE as an assessment tool of quality, there has been a dearth of studies investigating the ACOVE QIs as an intervention to improve patient care.25

Our study investigated the role of ACOVE as an intervention by using the QIs as a standardized checklist in the acute care setting. We selected the 4 most commonly encountered QIs in the hospital setting, namely venous thrombosis prophylaxis (VTE), indwelling bladder catheter, mobilization, and delirium evaluation, in order to test the feasibility and impact of systematically implementing these ACOVE QIs as a therapeutic intervention for all hospitalized older adults.

METHODS

This study (IRB #13-644B) was conducted using a prospective intervention with a nonequivalent control group design comprised of retrospective chart data from May 1, 2014, to June 30, 2015. Process and outcome variables were extracted from electronic medical records ([EMR], Sunrise Clinical Manager [SCM]) of 2,396 patients, with 530 patients in the intervention unit and 1,866 on the control units, at a large academic tertiary center operating in the greater New York metropolitan area. Our study investigated the role of ACOVE as an intervention to improve patient care by using selected QIs as a standardized checklist tool in the acute care setting. Of the original 30 hospital-specific QIs, our study focused on the care of older adults admitted to the medicine service.26 We selected commonly encountered QIs, with the objective of testing the feasibility and impact of implementing the ACOVE QIs as an intervention to improve care of hospitalized older adults. This intervention consisted of applying the checklist tool, constructed with 4 selected ACOVE QIs and administered daily during interdisciplinary rounds, namely: 2 general “medical” indicators, VTE prophylaxis and indwelling bladder catheters, and 2 “geriatric”-focused indicators, mobilization and delirium evaluation.

Integration of Selected ACOVE QIs Into a Checklist-Based Tool for Decision Support
Table 1

 

 

Subject matter experts (hospitalists, geriatricians, researchers, administrators, and nurses) reviewed the ACOVE QIs and agreed upon the adaptation of the QIs from a quality measure assessment into a feasible and acceptable intervention checklist tool (Table 1). The checklist was reviewed during daily interdisciplinary rounds for all patients 75 years and older. While ACOVE defined vulnerable elders by using the Vulnerable Elder Screen (VES), we wanted to apply this intervention more broadly to all hospitalized older adults who are most at risk for poor outcomes.27 Patients admitted to the intensive care unit, inpatient psychiatry, inpatient leukemia/lymphoma, and surgical services were excluded.

Daily interdisciplinary rounds are held on every one of the five 40-bed medical units; they last approximately 1 hour, and consist of a lead hospitalist, nurse manager, nurse practitioners, case managers, and the nursing staff. During interdisciplinary rounds, nurses present the case to the team members who then discuss the care plan. These 5 medical units did not differ in terms of patient characteristics or staffing patterns; the intervention unit was chosen simply for logistical reasons, in that the principal investigator (PI) had been assigned to this unit prior to study start-up.

Prior to the intervention, LS held an education session for staff on the intervention unit staff (who participated on interdisciplinary rounds) to explain the concept of the ACOVE QI initiative and describe the four QIs selected for the study. Three subsequent educational sessions were held during the first week of the intervention, with new incoming staff receiving a brief individual educational session. The staff demonstrated significant knowledge improvement after session completion (pre/post mean score 70.6% vs 90.0%; P < .0001).

The Clinical Information System for the Health System EMR, The Eclipsys SCM, has alerts with different levels of severity from “soft” (user must acknowledge a recommendation) to “hard” (requires an action in order to proceed).

To measure compliance of the quality indicators, we collected the following variables:

QI 1: VTE prophylaxis

Through SCM, we collected type of VTE prophylaxis ordered (pharmacologic and/or mechanical) as well as start and stop dates for all agents. International normalized ratio levels were checked for patients receiving warfarin. Days of compliance were calculated.

QI 2: Indwelling Bladder Catheters

SCM data were collected on catheter entry and discontinuation dates, the presence of an indication, and order renewal for bladder catheter at least every 3 days.

QI 3: Mobilization

Ambulation status prior to admission was extracted from nursing documentation completed on admission to the medical ward. Patients documented as bedfast were categorized as nonambulatory prior to admission. Nursing documentation of activity level and amount of feet ambulated per nursing shift were collected. In addition, hospital day of physical therapy (PT) order and hospital days with PT performed were charted. Compliance with QI 3 in patients documented as ambulatory prior to hospital admission was recorded as present if there was a PT order within 48 hours of admission.

QI 4: Delirium Evaluation

During daily rounds, the hospitalist (PI) questioned nurses about delirium evaluation, using the first feature of the Confusion Assessment Method (CAM) as well as the “single question in delirium,” namely, “Is there evidence of an acute change in mental status from the patient’s baseline?” and “Do you think [name of patient] has been more confused lately?”28,29 Because EMR does not contain a specified field for delirium screening and documentation, and patients are not routinely included in rounds, documentation with QI 4 was recorded using the “key words” method as described in the work by Puelle et al.30 To extract SCM key words, nursing documentation of the “cognitive/perceptual/neurological exam” section of the EMR on admission and on all subsequent documentation (once per shift) was retrieved to identify acute changes in mental status (eg, “altered mental status, delirium/delirious, alert and oriented X 3, confused/confusion, disoriented, lethargy/lethargic”).30 In addition, nurses were asked to activate an SCM parameter, “Acute Confusion” SCM parameter, in the nursing documentation section, which includes potential risk factors for confusion.

In addition to QI compliance, we collected LOS, discharge disposition, and 30-day readmission data.

Generalized linear mixed models (GLMM) for binary clustered (ie, hierarchical) data were used to estimate compliance rates (ie, nurse adherence) for each group (intervention group or control group) in the postintervention period, along with their corresponding 95% confidence intervals. GLMM was used to account for the hierarchical structure of the data: nursing units within a hospital. In order to calculate the Charlson Comorbidity Index, we extracted past medical history from the EMR.31

Subjects (N = 2,396) were included in the comparison of the intervention group vs control group for each of the following 4 ACOVE QI compliance measures: DVT, mobilization, bladder catheter, and delirium.

Patient Characteristics
Table 2

 

 

RESULTS

Of the 2,396 patient admissions, 530 were in the intervention unit and 1,866 were in the control unit. In the intervention group, the average age was 84.65 years, 75.58% were white and 47.21% were married. There was no difference in patient demographics between groups (Table 2).

 

QI 1: VTE Prophylaxis

Compliance with VTE prophylaxis was met in 78.3% of the intervention subjects and 76.5% of the controls (P < .4371) (Table 3). Of note, the rate of VTE prophylaxis was 57% in the intervention vs 39% in the control group (P < .0056), in the 554 patients for whom compliance was not met. Mechanical prophylaxis was used in 35.6% of intervention subjects vs 30.6 in the control (P = .048). Patients who received no form of prophylaxis were 0.5% in the intervention and 3% in the control (P = .027).

Quality Indicator Outcomes/Measurements
Table 3

QI 2: Indwelling Bladder Catheters

Out of 2,396 subjects, 406 had an indwelling bladder catheter (16.9%). Compliance with the catheter was met in 72.2% of the intervention group vs 54.4% in the control group (P = .1061). An indication for indwelling bladder catheters was documented in 100% of the subjects. The average number of catheter days was 5.16 in the intervention vs 5.88 in the control (P < .2284). There was statistical significance in catheter compliance in the longer stay (>15 days) subjects, decreasing to 23.32% in the control group while staying constant in the intervention group 71.5% (P = .0006).

QI 3: Mobilization

Of the 2,396 patients, 1,991 (83.1%) were reported as ambulatory prior to admission. In the intervention vs control group, 74 (14%) vs 297 (15.7%), respectively, were nonambulatory. Overall compliance with Q3 was 62.9% in the intervention vs 48.2% in the control (P < .0001). More specifically, the average time to PT order in the intervention group was 1.83 days vs 2.22 days in the control group (P < .0051) and the time to PT evaluation was 2.14 days vs 2.42 days, respectively (P < .0108). In the intervention group, 84 patients (15.8%) did not have a PT consult vs 511 (27%) in the control group (P < .0001). The average times per subject in which the nurses documented the approximate number of feet ambulated was 6.48 in the intervention group vs 0.11 in the control group.

QI 4: Delirium Evaluation

In terms of nursing documentation indicating the presence of an acute confusional state, the intervention group had 148 out of 530 nursing notes (27.9%) vs 405 out of 1,866 in the control group (21.7%; P = .0027). However, utilization of the “acute confusion” parameter with documentation of a risk factor did not differ between the groups (5.8% in the intervention group vs 5.6% in the control group, P < .94).

LOS, Discharge Disposition, and 30-Day Readmissions

LOS did not differ between intervention and control groups (6.37 days vs 6.27 days, respectively), with a median of 5 days (P = .877). Discharge disposition in the 2 groups included the following: home/home with services (71.32% vs 68.7%), skilled nursing facility/assisted living/long-term care (24.34 versus 25.83), inpatient hospice/home hospice (2.64 vs 2.25), and expired (1.13 vs 1.77; P < .3282). In addition, 30-day readmissions did not differ (21% vs 20%, respectively, P = .41).

DISCUSSION

Our goal was to explore an evidence-based, standardized approach to improve the care of hospitalized older adults. This approach leverages existing automated EMR alert functions with an additional level of decision support for VEs, integrated into daily multidisciplinary rounds. The use of a daily checklist-based tool offers a cost-effective and practical pathway to distribute the burden of compliance responsibility amongst team members.

As we anticipated and similar to study findings in hospitalized medicine, geriatric trauma, and primary care, compliance with general care QIs was better than geriatric-focused QIs.27,32 Wenger et al33 demonstrated significant improvements with screening for falls and incontinence; however, screening for cognitive impairment did not improve in the outpatient setting by imbedding ACOVE QIs into routine physician practice.

Increased compliance with VTE prophylaxis and indwelling bladder catheters may be explained by national financial incentives for widespread implementation of EMR alert systems. Conversely, mobilization, delirium assessment, and management in hospitalized older adults don’t benefit from similar incentives.

VTE Prophylaxis

The American College of Chest Physicians (ACCP) supports the use of VTE prophylaxis, especially in hospitalized older adults with decreased mobility.34 While greater adoption of EMR has already increased adherence, our intervention resulted in an even higher rate of compliance with the use of pharmacologic VTE prophylaxis.35 In the future, validated scores for risk of thrombosis and bleeding may be integrated into our QI-based checklist.

 

 

Indwelling Bladder Catheters

The potential harms of catheters have been described for over 50 years, yet remain frequently used.36,37 Previous studies have shown success in decreasing catheter days with computer-based and multidisciplinary protocols.36-39

Our health system’s EMR has built-in “soft” and “hard” alerts for indwelling bladder catheters, so we did not expect intervention-associated changes in compliance.

Mobilization

Hospitalization in older adults frequently results in functional decline.4,5,40 In response, the mobilization QI recommends an ambulation plan within 48 hours for those patients who were ambulatory prior to admission; it does not specifically define the components of the plan.26 There are several multicomponent interventions that have demonstrated improvement in functional decline, yet they require skilled providers.41,42 Our intervention implemented specific ambulation plan components: daily ambulation and documentation reminders and early PT evaluation.

While functional status measures have existed for decades, most are primarily geared to assess community-residing individuals and not designed to measure changes in function during hospitalization.43,44 Furthermore, performance-based hospital measures are difficult to integrate into the daily nursing workflow as they are time consuming.45,46 In practice, nurses routinely use free text to document functional status in the hospital setting, rendering comparative analysis problematic. Yet, we demonstrated that nurses were more engaged in reporting mobilization (increased documentation of ambulation distance and a decrease in time to PT). Future research should focus on the development of a standardized tool, integrated into the EMR, to accurately measure function in the acute care setting.

Delirium Evaluation

Delirium evaluation remains one of the most difficult clinical challenges for healthcare providers in hospitalized individuals, and our study reiterated these concerns. Previous research has consistently demonstrated that the diagnosis of delirium is missed by up to 75% of clinicians.47,48 Indeed, our study, which exclusively examined nursing documentation of the delirium evaluation QI, found that both groups showed strikingly low compliance rates. This may have been due to the fact that we only evaluated nursing documentation of suspected or definite diagnosis of delirium and a documented attempt to attribute the altered mental state to a potential etiology.31 By utilizing the concept of “key words,” as developed by Puelle et al.30, we were able to demonstrate a statistically significant improvement in nursing delirium documentation in the intervention group. This result should be interpreted with caution, as this approach is not validated. Furthermore, our operational definition of delirium compliance (ie, nurse documentation of delirium, requiring the launching of a separate parameter) may have been simply too cumbersome to readily integrate into the daily workflow. Future research should study the efficacy of a sensitive EMR-integrated screening tool that facilitates recognition, by all team members, of acute changes in cognition.

Although a number of QI improved for the intervention group, acute care utilization measures such as LOS, discharge disposition, and 30-day readmissions did not differ between groups. It may well be that improving quality for this very frail, vulnerable population may simply not result in decreased utilization. Our ability to further decrease LOS and readmission rates may be limited due to restriction of range in this complex patient population (eg, median LOS value of 5 days).

Limitations

Although our study had a large sample size, data were only collected from a single-center and thus require further exploration in different settings to ensure generalizability. In addition, QI observance was based on the medical record, which was problematic for some indicators, notably delirium identification. While prior literature highlights the difficulty in identifying delirium, especially during clinical practice without specialized training, our compliance was strikingly low.47 While validated measures such as CAM may have been included as part of the assessment, there is currently no EMR documentation of such measures and therefore, these data could not be obtained.

CONCLUSION

In summary, our study demonstrates the successful integration of the established ACOVE QIs as an intervention, rather than as an assessment method, for improving care of hospitalized older patients. By utilizing a checklist-based tool at the bedside allows the multidisciplinary team to implement evidence-based practices with the ultimate goal of standardizing care, not only for VEs, but potentially for other high-risk populations with multimorbidity.49 This innovative approach provides a much-needed direction to healthcare providers in the ever increasing stressful conditions of today’s acute care environment and for the ultimate benefit and safety of our older patients.

Disclosure

The authors declare no conflicts of interest. This study was supported by New York State Empire Clinical Research Investigators Program (ECRIP). The sponsor had no role in the conception, study design, data collection, data analysis, interpretation of data, manuscript preparation, or the decision to submit the manuscript for publication.

 

 

 

In 2014, the United States spent $3 trillion on healthcare; hospitalization consumed 32% of these expenditures.1 Today, Medicare patients account for over 50% of hospital days and over 30% of all hospital discharges in the United States.2 Despite this staggering financial burden, hospitalization of older adults often results in poor patient outcomes.3-6 The exponential growth of the hospitalist movement, from 350 hospitalists nationwide in 1995 to over 44,000 in 2014, has become the key strategy for providing care to hospitalized geriatric patients.7-10 Most of these hospitalists have not received geriatric training.11-15

There is growing evidence that a geriatric approach, emphasizing multidisciplinary management of the complex needs of older patients, leads to improved outcomes. Geriatric Evaluation and Management Units (GEMUs), such as Acute Care for Elderly (ACE) models, have demonstrated significant decreases in functional decline, institutionalization, and death in randomized controlled trials.16,17 Multidisciplinary, nonunit based efforts, such as the mobile acute care of elderly (MACE), proactive consultation models (Sennour/Counsell), and the Hospital Elder Life Program (HELP), have demonstrated success in preventing adverse events and decreasing length of stay (LOS).17-20

However, these models have not been systematically implemented due to challenges in generalizability and replicability in diverse settings. To address this concern, an alternative approach must be developed to widely “generalize” geriatric expertise throughout hospitals, regardless of their location, size, and resources. This initiative will require systematic integration of evidence-based decision support tools for the standardization of clinical management in hospitalized older adults.21

The 1998 Assessing Care of Vulnerable Elders (ACOVE) project developed a standardized tool to measure and evaluate the quality of care by using a comprehensive set of quality indicators (QIs) to improve the care of “vulnerable elders” (VEs) at a high risk for functional and cognitive decline and death.22-24 The latest systematic review concludes that, although many studies have used ACOVE as an assessment tool of quality, there has been a dearth of studies investigating the ACOVE QIs as an intervention to improve patient care.25

Our study investigated the role of ACOVE as an intervention by using the QIs as a standardized checklist in the acute care setting. We selected the 4 most commonly encountered QIs in the hospital setting, namely venous thrombosis prophylaxis (VTE), indwelling bladder catheter, mobilization, and delirium evaluation, in order to test the feasibility and impact of systematically implementing these ACOVE QIs as a therapeutic intervention for all hospitalized older adults.

METHODS

This study (IRB #13-644B) was conducted using a prospective intervention with a nonequivalent control group design comprised of retrospective chart data from May 1, 2014, to June 30, 2015. Process and outcome variables were extracted from electronic medical records ([EMR], Sunrise Clinical Manager [SCM]) of 2,396 patients, with 530 patients in the intervention unit and 1,866 on the control units, at a large academic tertiary center operating in the greater New York metropolitan area. Our study investigated the role of ACOVE as an intervention to improve patient care by using selected QIs as a standardized checklist tool in the acute care setting. Of the original 30 hospital-specific QIs, our study focused on the care of older adults admitted to the medicine service.26 We selected commonly encountered QIs, with the objective of testing the feasibility and impact of implementing the ACOVE QIs as an intervention to improve care of hospitalized older adults. This intervention consisted of applying the checklist tool, constructed with 4 selected ACOVE QIs and administered daily during interdisciplinary rounds, namely: 2 general “medical” indicators, VTE prophylaxis and indwelling bladder catheters, and 2 “geriatric”-focused indicators, mobilization and delirium evaluation.

Integration of Selected ACOVE QIs Into a Checklist-Based Tool for Decision Support
Table 1

 

 

Subject matter experts (hospitalists, geriatricians, researchers, administrators, and nurses) reviewed the ACOVE QIs and agreed upon the adaptation of the QIs from a quality measure assessment into a feasible and acceptable intervention checklist tool (Table 1). The checklist was reviewed during daily interdisciplinary rounds for all patients 75 years and older. While ACOVE defined vulnerable elders by using the Vulnerable Elder Screen (VES), we wanted to apply this intervention more broadly to all hospitalized older adults who are most at risk for poor outcomes.27 Patients admitted to the intensive care unit, inpatient psychiatry, inpatient leukemia/lymphoma, and surgical services were excluded.

Daily interdisciplinary rounds are held on every one of the five 40-bed medical units; they last approximately 1 hour, and consist of a lead hospitalist, nurse manager, nurse practitioners, case managers, and the nursing staff. During interdisciplinary rounds, nurses present the case to the team members who then discuss the care plan. These 5 medical units did not differ in terms of patient characteristics or staffing patterns; the intervention unit was chosen simply for logistical reasons, in that the principal investigator (PI) had been assigned to this unit prior to study start-up.

Prior to the intervention, LS held an education session for staff on the intervention unit staff (who participated on interdisciplinary rounds) to explain the concept of the ACOVE QI initiative and describe the four QIs selected for the study. Three subsequent educational sessions were held during the first week of the intervention, with new incoming staff receiving a brief individual educational session. The staff demonstrated significant knowledge improvement after session completion (pre/post mean score 70.6% vs 90.0%; P < .0001).

The Clinical Information System for the Health System EMR, The Eclipsys SCM, has alerts with different levels of severity from “soft” (user must acknowledge a recommendation) to “hard” (requires an action in order to proceed).

To measure compliance of the quality indicators, we collected the following variables:

QI 1: VTE prophylaxis

Through SCM, we collected type of VTE prophylaxis ordered (pharmacologic and/or mechanical) as well as start and stop dates for all agents. International normalized ratio levels were checked for patients receiving warfarin. Days of compliance were calculated.

QI 2: Indwelling Bladder Catheters

SCM data were collected on catheter entry and discontinuation dates, the presence of an indication, and order renewal for bladder catheter at least every 3 days.

QI 3: Mobilization

Ambulation status prior to admission was extracted from nursing documentation completed on admission to the medical ward. Patients documented as bedfast were categorized as nonambulatory prior to admission. Nursing documentation of activity level and amount of feet ambulated per nursing shift were collected. In addition, hospital day of physical therapy (PT) order and hospital days with PT performed were charted. Compliance with QI 3 in patients documented as ambulatory prior to hospital admission was recorded as present if there was a PT order within 48 hours of admission.

QI 4: Delirium Evaluation

During daily rounds, the hospitalist (PI) questioned nurses about delirium evaluation, using the first feature of the Confusion Assessment Method (CAM) as well as the “single question in delirium,” namely, “Is there evidence of an acute change in mental status from the patient’s baseline?” and “Do you think [name of patient] has been more confused lately?”28,29 Because EMR does not contain a specified field for delirium screening and documentation, and patients are not routinely included in rounds, documentation with QI 4 was recorded using the “key words” method as described in the work by Puelle et al.30 To extract SCM key words, nursing documentation of the “cognitive/perceptual/neurological exam” section of the EMR on admission and on all subsequent documentation (once per shift) was retrieved to identify acute changes in mental status (eg, “altered mental status, delirium/delirious, alert and oriented X 3, confused/confusion, disoriented, lethargy/lethargic”).30 In addition, nurses were asked to activate an SCM parameter, “Acute Confusion” SCM parameter, in the nursing documentation section, which includes potential risk factors for confusion.

In addition to QI compliance, we collected LOS, discharge disposition, and 30-day readmission data.

Generalized linear mixed models (GLMM) for binary clustered (ie, hierarchical) data were used to estimate compliance rates (ie, nurse adherence) for each group (intervention group or control group) in the postintervention period, along with their corresponding 95% confidence intervals. GLMM was used to account for the hierarchical structure of the data: nursing units within a hospital. In order to calculate the Charlson Comorbidity Index, we extracted past medical history from the EMR.31

Subjects (N = 2,396) were included in the comparison of the intervention group vs control group for each of the following 4 ACOVE QI compliance measures: DVT, mobilization, bladder catheter, and delirium.

Patient Characteristics
Table 2

 

 

RESULTS

Of the 2,396 patient admissions, 530 were in the intervention unit and 1,866 were in the control unit. In the intervention group, the average age was 84.65 years, 75.58% were white and 47.21% were married. There was no difference in patient demographics between groups (Table 2).

 

QI 1: VTE Prophylaxis

Compliance with VTE prophylaxis was met in 78.3% of the intervention subjects and 76.5% of the controls (P < .4371) (Table 3). Of note, the rate of VTE prophylaxis was 57% in the intervention vs 39% in the control group (P < .0056), in the 554 patients for whom compliance was not met. Mechanical prophylaxis was used in 35.6% of intervention subjects vs 30.6 in the control (P = .048). Patients who received no form of prophylaxis were 0.5% in the intervention and 3% in the control (P = .027).

Quality Indicator Outcomes/Measurements
Table 3

QI 2: Indwelling Bladder Catheters

Out of 2,396 subjects, 406 had an indwelling bladder catheter (16.9%). Compliance with the catheter was met in 72.2% of the intervention group vs 54.4% in the control group (P = .1061). An indication for indwelling bladder catheters was documented in 100% of the subjects. The average number of catheter days was 5.16 in the intervention vs 5.88 in the control (P < .2284). There was statistical significance in catheter compliance in the longer stay (>15 days) subjects, decreasing to 23.32% in the control group while staying constant in the intervention group 71.5% (P = .0006).

QI 3: Mobilization

Of the 2,396 patients, 1,991 (83.1%) were reported as ambulatory prior to admission. In the intervention vs control group, 74 (14%) vs 297 (15.7%), respectively, were nonambulatory. Overall compliance with Q3 was 62.9% in the intervention vs 48.2% in the control (P < .0001). More specifically, the average time to PT order in the intervention group was 1.83 days vs 2.22 days in the control group (P < .0051) and the time to PT evaluation was 2.14 days vs 2.42 days, respectively (P < .0108). In the intervention group, 84 patients (15.8%) did not have a PT consult vs 511 (27%) in the control group (P < .0001). The average times per subject in which the nurses documented the approximate number of feet ambulated was 6.48 in the intervention group vs 0.11 in the control group.

QI 4: Delirium Evaluation

In terms of nursing documentation indicating the presence of an acute confusional state, the intervention group had 148 out of 530 nursing notes (27.9%) vs 405 out of 1,866 in the control group (21.7%; P = .0027). However, utilization of the “acute confusion” parameter with documentation of a risk factor did not differ between the groups (5.8% in the intervention group vs 5.6% in the control group, P < .94).

LOS, Discharge Disposition, and 30-Day Readmissions

LOS did not differ between intervention and control groups (6.37 days vs 6.27 days, respectively), with a median of 5 days (P = .877). Discharge disposition in the 2 groups included the following: home/home with services (71.32% vs 68.7%), skilled nursing facility/assisted living/long-term care (24.34 versus 25.83), inpatient hospice/home hospice (2.64 vs 2.25), and expired (1.13 vs 1.77; P < .3282). In addition, 30-day readmissions did not differ (21% vs 20%, respectively, P = .41).

DISCUSSION

Our goal was to explore an evidence-based, standardized approach to improve the care of hospitalized older adults. This approach leverages existing automated EMR alert functions with an additional level of decision support for VEs, integrated into daily multidisciplinary rounds. The use of a daily checklist-based tool offers a cost-effective and practical pathway to distribute the burden of compliance responsibility amongst team members.

As we anticipated and similar to study findings in hospitalized medicine, geriatric trauma, and primary care, compliance with general care QIs was better than geriatric-focused QIs.27,32 Wenger et al33 demonstrated significant improvements with screening for falls and incontinence; however, screening for cognitive impairment did not improve in the outpatient setting by imbedding ACOVE QIs into routine physician practice.

Increased compliance with VTE prophylaxis and indwelling bladder catheters may be explained by national financial incentives for widespread implementation of EMR alert systems. Conversely, mobilization, delirium assessment, and management in hospitalized older adults don’t benefit from similar incentives.

VTE Prophylaxis

The American College of Chest Physicians (ACCP) supports the use of VTE prophylaxis, especially in hospitalized older adults with decreased mobility.34 While greater adoption of EMR has already increased adherence, our intervention resulted in an even higher rate of compliance with the use of pharmacologic VTE prophylaxis.35 In the future, validated scores for risk of thrombosis and bleeding may be integrated into our QI-based checklist.

 

 

Indwelling Bladder Catheters

The potential harms of catheters have been described for over 50 years, yet remain frequently used.36,37 Previous studies have shown success in decreasing catheter days with computer-based and multidisciplinary protocols.36-39

Our health system’s EMR has built-in “soft” and “hard” alerts for indwelling bladder catheters, so we did not expect intervention-associated changes in compliance.

Mobilization

Hospitalization in older adults frequently results in functional decline.4,5,40 In response, the mobilization QI recommends an ambulation plan within 48 hours for those patients who were ambulatory prior to admission; it does not specifically define the components of the plan.26 There are several multicomponent interventions that have demonstrated improvement in functional decline, yet they require skilled providers.41,42 Our intervention implemented specific ambulation plan components: daily ambulation and documentation reminders and early PT evaluation.

While functional status measures have existed for decades, most are primarily geared to assess community-residing individuals and not designed to measure changes in function during hospitalization.43,44 Furthermore, performance-based hospital measures are difficult to integrate into the daily nursing workflow as they are time consuming.45,46 In practice, nurses routinely use free text to document functional status in the hospital setting, rendering comparative analysis problematic. Yet, we demonstrated that nurses were more engaged in reporting mobilization (increased documentation of ambulation distance and a decrease in time to PT). Future research should focus on the development of a standardized tool, integrated into the EMR, to accurately measure function in the acute care setting.

Delirium Evaluation

Delirium evaluation remains one of the most difficult clinical challenges for healthcare providers in hospitalized individuals, and our study reiterated these concerns. Previous research has consistently demonstrated that the diagnosis of delirium is missed by up to 75% of clinicians.47,48 Indeed, our study, which exclusively examined nursing documentation of the delirium evaluation QI, found that both groups showed strikingly low compliance rates. This may have been due to the fact that we only evaluated nursing documentation of suspected or definite diagnosis of delirium and a documented attempt to attribute the altered mental state to a potential etiology.31 By utilizing the concept of “key words,” as developed by Puelle et al.30, we were able to demonstrate a statistically significant improvement in nursing delirium documentation in the intervention group. This result should be interpreted with caution, as this approach is not validated. Furthermore, our operational definition of delirium compliance (ie, nurse documentation of delirium, requiring the launching of a separate parameter) may have been simply too cumbersome to readily integrate into the daily workflow. Future research should study the efficacy of a sensitive EMR-integrated screening tool that facilitates recognition, by all team members, of acute changes in cognition.

Although a number of QI improved for the intervention group, acute care utilization measures such as LOS, discharge disposition, and 30-day readmissions did not differ between groups. It may well be that improving quality for this very frail, vulnerable population may simply not result in decreased utilization. Our ability to further decrease LOS and readmission rates may be limited due to restriction of range in this complex patient population (eg, median LOS value of 5 days).

Limitations

Although our study had a large sample size, data were only collected from a single-center and thus require further exploration in different settings to ensure generalizability. In addition, QI observance was based on the medical record, which was problematic for some indicators, notably delirium identification. While prior literature highlights the difficulty in identifying delirium, especially during clinical practice without specialized training, our compliance was strikingly low.47 While validated measures such as CAM may have been included as part of the assessment, there is currently no EMR documentation of such measures and therefore, these data could not be obtained.

CONCLUSION

In summary, our study demonstrates the successful integration of the established ACOVE QIs as an intervention, rather than as an assessment method, for improving care of hospitalized older patients. By utilizing a checklist-based tool at the bedside allows the multidisciplinary team to implement evidence-based practices with the ultimate goal of standardizing care, not only for VEs, but potentially for other high-risk populations with multimorbidity.49 This innovative approach provides a much-needed direction to healthcare providers in the ever increasing stressful conditions of today’s acute care environment and for the ultimate benefit and safety of our older patients.

Disclosure

The authors declare no conflicts of interest. This study was supported by New York State Empire Clinical Research Investigators Program (ECRIP). The sponsor had no role in the conception, study design, data collection, data analysis, interpretation of data, manuscript preparation, or the decision to submit the manuscript for publication.

 

 

 

References

1. National Center for Health Statistics (US). Health, United States, 2015: With Special Feature on Racial and Ethnic Health Disparities. Hyattsville, MD: National Center for Health Statistics (US); 2016. http://www.ncbi.nlm.nih.gov/books/NBK367640/. Accessed November 2, 2016.
2. Weiss AJ, Elixhauser A. Overview of Hospital Stays in the United States, 2012: Statistical Brief #180. In: Healthcare Cost and Utilization Project (HCUP) Statistical Briefs. Rockville (MD): Agency for Healthcare Research and Quality (US); 2006. http://www.ncbi.nlm.nih.gov/books/NBK259100/. Accessed November 2, 2016.
3. Jencks SF, Cuerdon T, Burwen DR, et al. Quality of medical care delivered to medicare beneficiaries: A profile at state and national levels. JAMA. 2000;284(13):1670-1676. PubMed
4. Covinsky KE, Pierluissi E, Johnston C. Hospitalization-associated disability: “She was probably able to ambulate, but I’m not sure.” JAMA. 2011;306(16):1782-1793. PubMed
5. Creditor MC. Hazards of Hospitalization of the Elderly. Ann Intern Med. 1993;118(3):219-223. PubMed
6. Graf C. Functional decline in hospitalized older adults. Am J Nurs. 2006;106(1):58-67, NaN-68. PubMed
7. Wachter RM, Goldman L. The emerging role of “hospitalists” in the American health care system. N Engl J Med. 1996;335(7):514-517. PubMed
8. Lindenauer PK, Pantilat SZ, Katz PP, Wachter RM. Hospitalists and the practice of inpatient medicine: results of a survey of the National Association of Inpatient Physicians. Ann Intern Med. 1999;130(4 Pt 2):343-349. PubMed
9. Wachter RM. The hospitalist movement 5 years later. JAMA. 2002;287(4):487. PubMed
10. Shank B. 2016: Celebrating 20 years of hospital medicine and looking toward a bright future. Hosp Natl Assoc Inpatient Physicians. 2016. http://www.the-hospitalist.org/hospitalist/article/121925/2016-celebrating-20-years-hospital-medicine-and-looking-toward-bright. Accessed June 2, 2017.
11. Retooling for an Aging America: Building the Health Care Workforce. Washington, DC.: National Academies Press; 2008. http://www.nap.edu/catalog/12089. Accessed November 2, 2016.
12. Boult C, Counsell SR, Leipzig RM, Berenson RA. The urgency of preparing primary care physicians to care for older people with chronic illnesses. Health Aff Proj Hope. 2010;29(5):811-818. PubMed
13. Warshaw GA, Bragg EJ, Thomas DC, Ho ML, Brewer DE, Association of Directors of Geriatric Academic Programs. Are internal medicine residency programs adequately preparing physicians to care for the baby boomers? A national survey from the Association of Directors of Geriatric Academic Programs Status of Geriatrics Workforce Study. J Am Geriatr Soc. 2006;54(10):1603-1609. PubMed
14. Tanner CE, Eckstrom E, Desai SS, Joseph CL, Ririe MR, Bowen JL. Uncovering frustrations: A qualitative needs assessment of academic general internists as geriatric care providers and teachers. J Gen Intern Med. 2006;21(1):51-55. PubMed
15. Warshaw GA, Bragg EJ, Brewer DE, Meganathan K, Ho M. The development of academic geriatric medicine: progress toward preparing the nation’s physicians to care for an aging population. J Am Geriatr Soc. 2007;55(12):2075-2082. PubMed
16. Fox MT, Sidani S, Persaud M, et al. Acute care for elders components of acute geriatric unit care: Systematic descriptive review. J Am Geriatr Soc. 2013;61(6):939-946. PubMed
17. Palmer RM, Landefeld CS, Kresevic D, Kowal J. A medical unit for the acute care of the elderly. J Am Geriatr Soc. 1994;42(5):545-552.
18. Hung WW, Ross JS, Farber J, Siu AL. Evaluation of the Mobile Acute Care of the Elderly (MACE) service. JAMA Intern Med. 2013;173(11):990-996. PubMed
19. Sennour Y, Counsell SR, Jones J, Weiner M. Development and implementation of a proactive geriatrics consultation model in collaboration with hospitalists. J Am Geriatr Soc. 2009;57(11):2139-2145. PubMed
20. Ellis G, Whitehead MA, O’Neill D, Langhorne P, Robinson D. Comprehensive geriatric assessment for older adults admitted to hospital. Cochrane Database Syst Rev. 2011;(7):CD006211. PubMed
21. Mattison MLP, Catic A, Davis RB, et al. A standardized, bundled approach to providing geriatric-focused acute care. J Am Geriatr Soc. 2014;62(5):936-942. doi:10.1111/jgs.12780. PubMed
22. Wenger NS, Shekelle PG. Assessing care of vulnerable elders: ACOVE project overview. Ann Intern Med. 2001;135(8 Pt 2):642-646. PubMed
23. Wenger NS, Roth CP, Shekelle P, ACOVE Investigators. Introduction to the assessing care of vulnerable elders-3 quality indicator measurement set. J Am Geriatr Soc. 2007;55 Suppl 2:S247-S252. PubMed
24. Reuben DB, Roth C, Kamberg C, Wenger NS. Restructuring primary care practices to manage geriatric syndromes: the ACOVE-2 intervention. J Am Geriatr Soc. 2003;51(12):1787-1793. PubMed
25. Askari M, Wierenga PC, Eslami S, Medlock S, De Rooij SE, Abu-Hanna A. Studies pertaining to the ACOVE quality criteria: a systematic review. Int J Qual Health Care. 2012;24(1):80-87. PubMed
26. Arora VM, McGory ML, Fung CH. Quality indicators for hospitalization and surgery in vulnerable elders. J Am Geriatr Soc. 2007;55 Suppl 2:S347-S358. PubMed
27. Arora VM, Johnson M, Olson J, et al. Using assessing care of vulnerable elders quality indicators to measure quality of hospital care for vulnerable elders. J Am Geriatr Soc. 2007;55(11):1705-1711. PubMed
28. Sands M, Dantoc B, Hartshorn A, Ryan C, Lujic S. Single Question in Delirium (SQiD): testing its efficacy against psychiatrist interview, the Confusion Assessment Method and the Memorial Delirium Assessment Scale. Palliat Med. 2010;24(6):561-565. PubMed
29. Inouye SK, van Dyck CH, Alessi CA, Balkin S, Siegal AP, Horwitz RI. Clarifying confusion: the confusion assessment method. A new method for detection of delirium. Ann Intern Med. 1990;113(12):941-948. PubMed
30. Puelle MR, Kosar CM, Xu G, et al. The language of delirium: Keywords for identifying delirium from medical records. J Gerontol Nurs. 2015;41(8):34-42. PubMed
31. Quan H, Sundararajan V, Halfon P, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43(11):1130-1139. PubMed
32. Boult C, Boult L, Morishita L, Smith SL, Kane RL. Outpatient geriatric evaluation and management. J Am Geriatr Soc. 1998;46(3):296-302.33. Wenger NS, Roth CP, Shekelle PG, et al. A practice-based intervention to improve primary care for falls, urinary incontinence, and dementia. J Am Geriatr Soc. 2009;57(3):547-555. PubMed
34. Geerts WH. Prevention of Venous Thromboembolism: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines (8th Edition). Chest J. 2008;133(6_suppl):381S. 
35. Rosenman M, Liu X, Phatak H, et al. Pharmacological prophylaxis for venous thromboembolism among hospitalized patients with acute medical illness: An electronic medical records study. Am J Ther. 2016;23(2):e328-e335. PubMed
36. Ghanem A, Artime C, Moser M, Caceres L, Basconcillo A. Holy moley! Take out that foley! Measuring compliance with a nurse driven protocol for foley catheter removal to decrease utilization. Am J Infect Control. 2015;43(6):S51.
37. Cornia PB, Amory JK, Fraser S, Saint S, Lipsky BA. Computer-based order entry decreases duration of indwelling urinary catheterization in hospitalized patients. Am J Med. 2003;114(5):404-407. PubMed
38. Huang W-C, Wann S-R, Lin S-L, et al. Catheter-associated urinary tract infections in intensive care units can be reduced by prompting physicians to remove unnecessary catheters. Infect Control Hosp Epidemiol. 2004;25(11):974-978. PubMed
39. Topal J, Conklin S, Camp K, Morris V, Balcezak T, Herbert P. Prevention of nosocomial catheter-associated urinary tract infections through computerized feedback to physicians and a nurse-directed protocol. Am J Med Qual. 2005;20(3):121-126. PubMed
40. Zisberg A, Shadmi E, Gur-Yaish N, Tonkikh O, Sinoff G. Hospital-associated functional decline: the role of hospitalization processes beyond individual risk factors. J Am Geriatr Soc. 2015;63(1):55-62. PubMed
41. Inouye SK, Bogardus ST, Baker DI, Leo-Summers L, Cooney LM. The Hospital Elder Life Program: a model of care to prevent cognitive and functional decline in older hospitalized patients. Hospital Elder Life Program. J Am Geriatr Soc. 2000;48(12):1697-1706. PubMed
42. Hoyer EH, Friedman M, Lavezza A, et al. Promoting mobility and reducing length of stay in hospitalized general medicine patients: A quality-improvement project. J Hosp Med. 2016;11(5):341-347. PubMed
43. Mahoney FI, Barthel DW. Functional evaluation: the barthel index. Md State Med J. 1965;14:61-65. PubMed
44. Katz S, Ford AB, Moskowitz RW, Jackson BA, Jaffe MW. Studies of illness in the aged. the index of adl: a standardized measure of biological and psychosocial function. JAMA. 1963;185:914-919. PubMed
45. Tinetti ME. Performance-oriented assessment of mobility problems in elderly patients. J Am Geriatr Soc. 1986;34(2):119-126. PubMed
46. Smith R. Validation and Reliability of the Elderly Mobility Scale. Physiotherapy. 1994;80(11):744-747. 
47. Inouye SK, Foreman MD, Mion LC, Katz KH, Cooney LM. Nurses’ recognition of delirium and its symptoms: comparison of nurse and researcher ratings. Arch Intern Med. 2001;161(20):2467-2473. PubMed
48. Gustafson Y, Brännström B, Norberg A, Bucht G, Winblad B. Underdiagnosis and poor documentation of acute confusional states in elderly hip fracture patients. J Am Geriatr Soc. 1991;39(8):760-765. PubMed
49. Brenner SK, Kaushal R, Grinspan Z, et al. Effects of health information technology on patient outcomes: a systematic review. J Am Med Inform Assoc. 2016;23(5):1016-1036. PubMed

 

 

References

1. National Center for Health Statistics (US). Health, United States, 2015: With Special Feature on Racial and Ethnic Health Disparities. Hyattsville, MD: National Center for Health Statistics (US); 2016. http://www.ncbi.nlm.nih.gov/books/NBK367640/. Accessed November 2, 2016.
2. Weiss AJ, Elixhauser A. Overview of Hospital Stays in the United States, 2012: Statistical Brief #180. In: Healthcare Cost and Utilization Project (HCUP) Statistical Briefs. Rockville (MD): Agency for Healthcare Research and Quality (US); 2006. http://www.ncbi.nlm.nih.gov/books/NBK259100/. Accessed November 2, 2016.
3. Jencks SF, Cuerdon T, Burwen DR, et al. Quality of medical care delivered to medicare beneficiaries: A profile at state and national levels. JAMA. 2000;284(13):1670-1676. PubMed
4. Covinsky KE, Pierluissi E, Johnston C. Hospitalization-associated disability: “She was probably able to ambulate, but I’m not sure.” JAMA. 2011;306(16):1782-1793. PubMed
5. Creditor MC. Hazards of Hospitalization of the Elderly. Ann Intern Med. 1993;118(3):219-223. PubMed
6. Graf C. Functional decline in hospitalized older adults. Am J Nurs. 2006;106(1):58-67, NaN-68. PubMed
7. Wachter RM, Goldman L. The emerging role of “hospitalists” in the American health care system. N Engl J Med. 1996;335(7):514-517. PubMed
8. Lindenauer PK, Pantilat SZ, Katz PP, Wachter RM. Hospitalists and the practice of inpatient medicine: results of a survey of the National Association of Inpatient Physicians. Ann Intern Med. 1999;130(4 Pt 2):343-349. PubMed
9. Wachter RM. The hospitalist movement 5 years later. JAMA. 2002;287(4):487. PubMed
10. Shank B. 2016: Celebrating 20 years of hospital medicine and looking toward a bright future. Hosp Natl Assoc Inpatient Physicians. 2016. http://www.the-hospitalist.org/hospitalist/article/121925/2016-celebrating-20-years-hospital-medicine-and-looking-toward-bright. Accessed June 2, 2017.
11. Retooling for an Aging America: Building the Health Care Workforce. Washington, DC.: National Academies Press; 2008. http://www.nap.edu/catalog/12089. Accessed November 2, 2016.
12. Boult C, Counsell SR, Leipzig RM, Berenson RA. The urgency of preparing primary care physicians to care for older people with chronic illnesses. Health Aff Proj Hope. 2010;29(5):811-818. PubMed
13. Warshaw GA, Bragg EJ, Thomas DC, Ho ML, Brewer DE, Association of Directors of Geriatric Academic Programs. Are internal medicine residency programs adequately preparing physicians to care for the baby boomers? A national survey from the Association of Directors of Geriatric Academic Programs Status of Geriatrics Workforce Study. J Am Geriatr Soc. 2006;54(10):1603-1609. PubMed
14. Tanner CE, Eckstrom E, Desai SS, Joseph CL, Ririe MR, Bowen JL. Uncovering frustrations: A qualitative needs assessment of academic general internists as geriatric care providers and teachers. J Gen Intern Med. 2006;21(1):51-55. PubMed
15. Warshaw GA, Bragg EJ, Brewer DE, Meganathan K, Ho M. The development of academic geriatric medicine: progress toward preparing the nation’s physicians to care for an aging population. J Am Geriatr Soc. 2007;55(12):2075-2082. PubMed
16. Fox MT, Sidani S, Persaud M, et al. Acute care for elders components of acute geriatric unit care: Systematic descriptive review. J Am Geriatr Soc. 2013;61(6):939-946. PubMed
17. Palmer RM, Landefeld CS, Kresevic D, Kowal J. A medical unit for the acute care of the elderly. J Am Geriatr Soc. 1994;42(5):545-552.
18. Hung WW, Ross JS, Farber J, Siu AL. Evaluation of the Mobile Acute Care of the Elderly (MACE) service. JAMA Intern Med. 2013;173(11):990-996. PubMed
19. Sennour Y, Counsell SR, Jones J, Weiner M. Development and implementation of a proactive geriatrics consultation model in collaboration with hospitalists. J Am Geriatr Soc. 2009;57(11):2139-2145. PubMed
20. Ellis G, Whitehead MA, O’Neill D, Langhorne P, Robinson D. Comprehensive geriatric assessment for older adults admitted to hospital. Cochrane Database Syst Rev. 2011;(7):CD006211. PubMed
21. Mattison MLP, Catic A, Davis RB, et al. A standardized, bundled approach to providing geriatric-focused acute care. J Am Geriatr Soc. 2014;62(5):936-942. doi:10.1111/jgs.12780. PubMed
22. Wenger NS, Shekelle PG. Assessing care of vulnerable elders: ACOVE project overview. Ann Intern Med. 2001;135(8 Pt 2):642-646. PubMed
23. Wenger NS, Roth CP, Shekelle P, ACOVE Investigators. Introduction to the assessing care of vulnerable elders-3 quality indicator measurement set. J Am Geriatr Soc. 2007;55 Suppl 2:S247-S252. PubMed
24. Reuben DB, Roth C, Kamberg C, Wenger NS. Restructuring primary care practices to manage geriatric syndromes: the ACOVE-2 intervention. J Am Geriatr Soc. 2003;51(12):1787-1793. PubMed
25. Askari M, Wierenga PC, Eslami S, Medlock S, De Rooij SE, Abu-Hanna A. Studies pertaining to the ACOVE quality criteria: a systematic review. Int J Qual Health Care. 2012;24(1):80-87. PubMed
26. Arora VM, McGory ML, Fung CH. Quality indicators for hospitalization and surgery in vulnerable elders. J Am Geriatr Soc. 2007;55 Suppl 2:S347-S358. PubMed
27. Arora VM, Johnson M, Olson J, et al. Using assessing care of vulnerable elders quality indicators to measure quality of hospital care for vulnerable elders. J Am Geriatr Soc. 2007;55(11):1705-1711. PubMed
28. Sands M, Dantoc B, Hartshorn A, Ryan C, Lujic S. Single Question in Delirium (SQiD): testing its efficacy against psychiatrist interview, the Confusion Assessment Method and the Memorial Delirium Assessment Scale. Palliat Med. 2010;24(6):561-565. PubMed
29. Inouye SK, van Dyck CH, Alessi CA, Balkin S, Siegal AP, Horwitz RI. Clarifying confusion: the confusion assessment method. A new method for detection of delirium. Ann Intern Med. 1990;113(12):941-948. PubMed
30. Puelle MR, Kosar CM, Xu G, et al. The language of delirium: Keywords for identifying delirium from medical records. J Gerontol Nurs. 2015;41(8):34-42. PubMed
31. Quan H, Sundararajan V, Halfon P, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43(11):1130-1139. PubMed
32. Boult C, Boult L, Morishita L, Smith SL, Kane RL. Outpatient geriatric evaluation and management. J Am Geriatr Soc. 1998;46(3):296-302.33. Wenger NS, Roth CP, Shekelle PG, et al. A practice-based intervention to improve primary care for falls, urinary incontinence, and dementia. J Am Geriatr Soc. 2009;57(3):547-555. PubMed
34. Geerts WH. Prevention of Venous Thromboembolism: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines (8th Edition). Chest J. 2008;133(6_suppl):381S. 
35. Rosenman M, Liu X, Phatak H, et al. Pharmacological prophylaxis for venous thromboembolism among hospitalized patients with acute medical illness: An electronic medical records study. Am J Ther. 2016;23(2):e328-e335. PubMed
36. Ghanem A, Artime C, Moser M, Caceres L, Basconcillo A. Holy moley! Take out that foley! Measuring compliance with a nurse driven protocol for foley catheter removal to decrease utilization. Am J Infect Control. 2015;43(6):S51.
37. Cornia PB, Amory JK, Fraser S, Saint S, Lipsky BA. Computer-based order entry decreases duration of indwelling urinary catheterization in hospitalized patients. Am J Med. 2003;114(5):404-407. PubMed
38. Huang W-C, Wann S-R, Lin S-L, et al. Catheter-associated urinary tract infections in intensive care units can be reduced by prompting physicians to remove unnecessary catheters. Infect Control Hosp Epidemiol. 2004;25(11):974-978. PubMed
39. Topal J, Conklin S, Camp K, Morris V, Balcezak T, Herbert P. Prevention of nosocomial catheter-associated urinary tract infections through computerized feedback to physicians and a nurse-directed protocol. Am J Med Qual. 2005;20(3):121-126. PubMed
40. Zisberg A, Shadmi E, Gur-Yaish N, Tonkikh O, Sinoff G. Hospital-associated functional decline: the role of hospitalization processes beyond individual risk factors. J Am Geriatr Soc. 2015;63(1):55-62. PubMed
41. Inouye SK, Bogardus ST, Baker DI, Leo-Summers L, Cooney LM. The Hospital Elder Life Program: a model of care to prevent cognitive and functional decline in older hospitalized patients. Hospital Elder Life Program. J Am Geriatr Soc. 2000;48(12):1697-1706. PubMed
42. Hoyer EH, Friedman M, Lavezza A, et al. Promoting mobility and reducing length of stay in hospitalized general medicine patients: A quality-improvement project. J Hosp Med. 2016;11(5):341-347. PubMed
43. Mahoney FI, Barthel DW. Functional evaluation: the barthel index. Md State Med J. 1965;14:61-65. PubMed
44. Katz S, Ford AB, Moskowitz RW, Jackson BA, Jaffe MW. Studies of illness in the aged. the index of adl: a standardized measure of biological and psychosocial function. JAMA. 1963;185:914-919. PubMed
45. Tinetti ME. Performance-oriented assessment of mobility problems in elderly patients. J Am Geriatr Soc. 1986;34(2):119-126. PubMed
46. Smith R. Validation and Reliability of the Elderly Mobility Scale. Physiotherapy. 1994;80(11):744-747. 
47. Inouye SK, Foreman MD, Mion LC, Katz KH, Cooney LM. Nurses’ recognition of delirium and its symptoms: comparison of nurse and researcher ratings. Arch Intern Med. 2001;161(20):2467-2473. PubMed
48. Gustafson Y, Brännström B, Norberg A, Bucht G, Winblad B. Underdiagnosis and poor documentation of acute confusional states in elderly hip fracture patients. J Am Geriatr Soc. 1991;39(8):760-765. PubMed
49. Brenner SK, Kaushal R, Grinspan Z, et al. Effects of health information technology on patient outcomes: a systematic review. J Am Med Inform Assoc. 2016;23(5):1016-1036. PubMed

 

 

Issue
Journal of Hospital Medicine 12(7)
Issue
Journal of Hospital Medicine 12(7)
Page Number
517-522
Page Number
517-522
Publications
Publications
Topics
Article Type
Display Headline
Implementing ACOVE quality indicators as an intervention checklist to improve care for hospitalized older adults
Display Headline
Implementing ACOVE quality indicators as an intervention checklist to improve care for hospitalized older adults
Sections
Article Source

© 2017 Society of Hospital Medicine

Disallow All Ads
Correspondence Location
*Address for correspondence and reprint requests: Andrzej Kozikowski, Northwell Health, 175 Community Drive, Second Floor, Great Neck, NY 11021; Telephone: (516) 465-3211; Fax: (516) 465-2699; E-mail: akozikowski@northwell.edu
Content Gating
Gated (full article locked unless allowed per User)
Alternative CME
Disqus Comments
Default
Gating Strategy
First Peek Free
Article PDF Media

Sneak Peek: Journal of Hospital Medicine – July 2017

Article Type
Changed
Fri, 09/14/2018 - 11:58
Implementing ACOVE quality indicators as an intervention checklist to improve care for hospitalized older adults

 

BACKGROUND: Medicare patients account for approximately 50% of hospital days. Hospitalization in older adults often results in poor outcomes.

OBJECTIVE: To test the feasibility and impact of using Assessing Care of Vulnerable Elders (ACOVE) quality indicators (QIs) as a therapeutic intervention to improve care of hospitalized older adults.

DESIGN: Post-test only prospective intervention with a nonequivalent retrospective control group.

SETTING: Large tertiary hospital in the greater New York Metropolitan area.

PATIENTS: Hospitalized patients, 75 and over, admitted to medical units.

INTERVENTION: A checklist, comprised of four ACOVE QIs, administered during daily interdisciplinary rounds: venous thrombosis prophylaxis (VTE) (QI 1), indwelling bladder catheters (QI 2), mobilization (QI 3), and delirium evaluation (QI 4).

MEASUREMENTS: Variables were extracted from electronic medical records with QI compliance as the primary outcome, and length of stay (LOS), discharge disposition, and readmissions as secondary outcomes. Generalized linear mixed models for binary clustered data were used to estimate compliance rates for each group (intervention group or control group) in the postintervention period, along with their corresponding 95% confidence intervals.

RESULTS: Of the 2,396 patients, 530 were on an intervention unit. In those patients not already compliant with VTE, the compliance rate was 57% in intervention vs. 39% in control (P less than .0056). For indwelling catheters, mobilization, and delirium evaluation, overall compliance was significantly higher in the intervention group 72.2% vs. 54.4% (P = .1061), 62.9% vs. 48.2% (P less than .0001), and 27.9% vs. 21.7% (P = .0027), respectively.

CONCLUSIONS: The study demonstrates the feasibility and effectiveness of integrating ACOVE QIs to improve the quality of care in hospitalized older adults.

Also in JHM

Use of simulation to assess incoming interns’ recognition of opportunities to choose wisely
AUTHORS:
Kathleen M. Wiest, Jeanne M. Farnan, MD, MHPE, Ellen Byrne, Lukas Matern, Melissa Cappaert, MA, Kristen Hirsch, Vineet M. Arora, MD, MAPP

Clinician attitudes regarding ICD deactivation in DNR/DNI patients
AUTHORS: Andrew J. Bradley, MD, Adam D. Marks, MD, MPH

Using standardized patients to assess hospitalist communication skills
AUTHORS: Dennis T. Chang, MD, Micah Mann, MD, Terry Sommer, BFA, Robert Fallar, PhD, Alan Weinberg, MS, Erica Friedman, MD

Techniques and behaviors associated with exemplary inpatient general medicine teaching: An exploratory qualitative study
AUTHORS: Nathan Houchens, MD, Molly Harrod, PhD, Stephanie Moody, PhD, Karen E. Fowler, MPH, Sanjay Saint, MD, MPH

A simple algorithm for predicting bacteremia using food consumption and shaking chills: A prospective observational study
AUTHORS: Takayuki Komatsu, MD, PhD, Erika Takahashi, MD, Kentaro Mishima, MD, Takeo Toyoda, MD, Fumihiro Saitoh, MD, Akari Yasuda, RN, Joe Matsuoka, PhD, Manabu Sugita, MD, PhD, Joel Branch, MD, Makoto Aoki, MD, Lawrence M. Tierney Jr., MD, Kenji Inoue, MD, PhD

For more articles and subscription information, visit www.journalofhospitalmedicine.com.

Publications
Topics
Sections
Implementing ACOVE quality indicators as an intervention checklist to improve care for hospitalized older adults
Implementing ACOVE quality indicators as an intervention checklist to improve care for hospitalized older adults

 

BACKGROUND: Medicare patients account for approximately 50% of hospital days. Hospitalization in older adults often results in poor outcomes.

OBJECTIVE: To test the feasibility and impact of using Assessing Care of Vulnerable Elders (ACOVE) quality indicators (QIs) as a therapeutic intervention to improve care of hospitalized older adults.

DESIGN: Post-test only prospective intervention with a nonequivalent retrospective control group.

SETTING: Large tertiary hospital in the greater New York Metropolitan area.

PATIENTS: Hospitalized patients, 75 and over, admitted to medical units.

INTERVENTION: A checklist, comprised of four ACOVE QIs, administered during daily interdisciplinary rounds: venous thrombosis prophylaxis (VTE) (QI 1), indwelling bladder catheters (QI 2), mobilization (QI 3), and delirium evaluation (QI 4).

MEASUREMENTS: Variables were extracted from electronic medical records with QI compliance as the primary outcome, and length of stay (LOS), discharge disposition, and readmissions as secondary outcomes. Generalized linear mixed models for binary clustered data were used to estimate compliance rates for each group (intervention group or control group) in the postintervention period, along with their corresponding 95% confidence intervals.

RESULTS: Of the 2,396 patients, 530 were on an intervention unit. In those patients not already compliant with VTE, the compliance rate was 57% in intervention vs. 39% in control (P less than .0056). For indwelling catheters, mobilization, and delirium evaluation, overall compliance was significantly higher in the intervention group 72.2% vs. 54.4% (P = .1061), 62.9% vs. 48.2% (P less than .0001), and 27.9% vs. 21.7% (P = .0027), respectively.

CONCLUSIONS: The study demonstrates the feasibility and effectiveness of integrating ACOVE QIs to improve the quality of care in hospitalized older adults.

Also in JHM

Use of simulation to assess incoming interns’ recognition of opportunities to choose wisely
AUTHORS:
Kathleen M. Wiest, Jeanne M. Farnan, MD, MHPE, Ellen Byrne, Lukas Matern, Melissa Cappaert, MA, Kristen Hirsch, Vineet M. Arora, MD, MAPP

Clinician attitudes regarding ICD deactivation in DNR/DNI patients
AUTHORS: Andrew J. Bradley, MD, Adam D. Marks, MD, MPH

Using standardized patients to assess hospitalist communication skills
AUTHORS: Dennis T. Chang, MD, Micah Mann, MD, Terry Sommer, BFA, Robert Fallar, PhD, Alan Weinberg, MS, Erica Friedman, MD

Techniques and behaviors associated with exemplary inpatient general medicine teaching: An exploratory qualitative study
AUTHORS: Nathan Houchens, MD, Molly Harrod, PhD, Stephanie Moody, PhD, Karen E. Fowler, MPH, Sanjay Saint, MD, MPH

A simple algorithm for predicting bacteremia using food consumption and shaking chills: A prospective observational study
AUTHORS: Takayuki Komatsu, MD, PhD, Erika Takahashi, MD, Kentaro Mishima, MD, Takeo Toyoda, MD, Fumihiro Saitoh, MD, Akari Yasuda, RN, Joe Matsuoka, PhD, Manabu Sugita, MD, PhD, Joel Branch, MD, Makoto Aoki, MD, Lawrence M. Tierney Jr., MD, Kenji Inoue, MD, PhD

For more articles and subscription information, visit www.journalofhospitalmedicine.com.

 

BACKGROUND: Medicare patients account for approximately 50% of hospital days. Hospitalization in older adults often results in poor outcomes.

OBJECTIVE: To test the feasibility and impact of using Assessing Care of Vulnerable Elders (ACOVE) quality indicators (QIs) as a therapeutic intervention to improve care of hospitalized older adults.

DESIGN: Post-test only prospective intervention with a nonequivalent retrospective control group.

SETTING: Large tertiary hospital in the greater New York Metropolitan area.

PATIENTS: Hospitalized patients, 75 and over, admitted to medical units.

INTERVENTION: A checklist, comprised of four ACOVE QIs, administered during daily interdisciplinary rounds: venous thrombosis prophylaxis (VTE) (QI 1), indwelling bladder catheters (QI 2), mobilization (QI 3), and delirium evaluation (QI 4).

MEASUREMENTS: Variables were extracted from electronic medical records with QI compliance as the primary outcome, and length of stay (LOS), discharge disposition, and readmissions as secondary outcomes. Generalized linear mixed models for binary clustered data were used to estimate compliance rates for each group (intervention group or control group) in the postintervention period, along with their corresponding 95% confidence intervals.

RESULTS: Of the 2,396 patients, 530 were on an intervention unit. In those patients not already compliant with VTE, the compliance rate was 57% in intervention vs. 39% in control (P less than .0056). For indwelling catheters, mobilization, and delirium evaluation, overall compliance was significantly higher in the intervention group 72.2% vs. 54.4% (P = .1061), 62.9% vs. 48.2% (P less than .0001), and 27.9% vs. 21.7% (P = .0027), respectively.

CONCLUSIONS: The study demonstrates the feasibility and effectiveness of integrating ACOVE QIs to improve the quality of care in hospitalized older adults.

Also in JHM

Use of simulation to assess incoming interns’ recognition of opportunities to choose wisely
AUTHORS:
Kathleen M. Wiest, Jeanne M. Farnan, MD, MHPE, Ellen Byrne, Lukas Matern, Melissa Cappaert, MA, Kristen Hirsch, Vineet M. Arora, MD, MAPP

Clinician attitudes regarding ICD deactivation in DNR/DNI patients
AUTHORS: Andrew J. Bradley, MD, Adam D. Marks, MD, MPH

Using standardized patients to assess hospitalist communication skills
AUTHORS: Dennis T. Chang, MD, Micah Mann, MD, Terry Sommer, BFA, Robert Fallar, PhD, Alan Weinberg, MS, Erica Friedman, MD

Techniques and behaviors associated with exemplary inpatient general medicine teaching: An exploratory qualitative study
AUTHORS: Nathan Houchens, MD, Molly Harrod, PhD, Stephanie Moody, PhD, Karen E. Fowler, MPH, Sanjay Saint, MD, MPH

A simple algorithm for predicting bacteremia using food consumption and shaking chills: A prospective observational study
AUTHORS: Takayuki Komatsu, MD, PhD, Erika Takahashi, MD, Kentaro Mishima, MD, Takeo Toyoda, MD, Fumihiro Saitoh, MD, Akari Yasuda, RN, Joe Matsuoka, PhD, Manabu Sugita, MD, PhD, Joel Branch, MD, Makoto Aoki, MD, Lawrence M. Tierney Jr., MD, Kenji Inoue, MD, PhD

For more articles and subscription information, visit www.journalofhospitalmedicine.com.

Publications
Publications
Topics
Article Type
Sections
Disallow All Ads
Content Gating
No Gating (article Unlocked/Free)
Alternative CME

Quantifying Resident Clinical Experience

Article Type
Changed
Mon, 05/22/2017 - 21:34
Display Headline
Quantifying internal medicine resident clinical experience using resident‐selected primary diagnosis codes

Internal medicine residency training continues to evolve as competency‐based and with education organized around patient care.13 Making the patient the center of resident education provides an opportunity for experiential learning in which learning can be organized around the clinical conditions that residents encounter. Despite the renewed emphasis on using patient experience as the basis for residency education, little is known regarding what specific diagnostic conditions are seen by internal medicine residents throughout their training. Attempts have been made to quantify resident clinical experience in various fields, using approaches such as review of medical records, case logs, and prescription profiles, but to date, we lack systematic methods to obtain clinical experience data for internal medicine residents.47

While residency curricula in internal medicine typically outlines specific rotations in various clinical areas such as general medical wards, cardiology services, and intensive care units, time spent on such rotations does not necessarily provide quantitative data on the actual clinical conditions that residents encounter, nor does it ensure consistent clinical experience between residents. It is plausible that there may be substantial variability in clinical experience between residents within the same program, and that the overall spectrum of clinical disorders seen by residents in a program may or may not be consistent with a desired optimum, though this is yet to be defined.

If residency education in internal medicine is to progressively incorporate more experiential learning, detailed knowledge of the clinical conditions seen by residents should be useful, not only for overall curriculum design, but this might also allow for various educational interventions to be made when there are variations in clinical experience between residents. Our program has been interested in the application of electronic resources for the improvement of patient care, such as through the handoff process and the use of personal digital assistants.8 We previously did a small analysis of clinical conditions seen by residents through non‐International Classification of Diseases, Ninth Revision (ICD‐9)‐based data they entered onto personal digital assistants. This suggested to us that electronic resources used by residents might serve as a venue by which they could enter diagnostic information which we could use to generate a more detailed analysis of the clinical conditions that they see. Here we describe a method by which we have attempted to quantify resident clinical experience in internal medicine using a modification of an electronic handoff system.

METHODS

The study was conducted within the Internal Medicine Residency Program at the Long Island Jewish Medical Center in New Hyde Park, New York, part of the North ShoreLong Island Jewish Health System, and was approved by the Institutional Review Board. This work was carried out as part of our participation in the Educational Innovation Project of the Residency Review Committee for Internal Medicine. A central objective of our proposal was to develop a method to assess residents' clinical experience on an individual and an aggregate basis. A group of faculty and residents in our residency program developed an electronic handoff tool which residents use for rapid access to key clinical data for their patients and for the handoff of clinical information for on call coverage. This handoff tool was developed with the technical assistance of MedTech Notes LLC which owns Patient Data Transfer System (PDTS) HandOff Note. We modified the handoff tool to include a section in which residents were required to enter a primary diagnosis for each of their patients (a hard stop design). We chose to use the ICD‐9 system for standardization and created two methods to select the code: 1) an organ system‐based dropdown list containing frequently used codes and 2) a search box allowing for searching of the complete ICD‐9 database. For the organ‐based dropdown list, selection of that organ system would reveal a brief list of frequently used codes to make it easier for residents to find them. Prior to using the handoff tool with the ICD‐9based primary diagnosis coding system, training sessions with the residents were conducted by 3 of the investigators along with 3 chief medical residents. These sessions included training not only in technical aspects of how to find diagnosis codes, but also how to make decisions regarding what the primary diagnosis should be. We also instructed our postgraduate year (PGY)‐1s to update their diagnostic selections during the course of the hospital stay.

Each data point represents a resident caring for a patient with a specific diagnostic entity, and is counted once for that resident's period of taking care of that patient. Thirty‐three PGY‐1s were studied and, on the internal medicine service, they were supervised by either hospitalist faculty or voluntary faculty in comparable proportions. If the patient's care is taken over by another resident, that second resident was also recorded as having had a diagnostic encounter with that patient, hence 1 patient could provide experience with the same diagnostic entity for 1 or more residents. Using this method, the denominator is not patients seen, but residentpatient diagnostic encounters that have taken place. The ICD‐9 diagnostic conditions entered by the residents were grouped using the ICD‐9 system. Individual diagnostic profiles for each resident, as well as an aggregate profile for all residents to reflect the residency program as a whole, were generated. We also carried out an analysis of the ICD‐9 codes entered by 6 consecutive PGY‐1s to assess how the diagnostic spectrum might vary among a small sampling of PGY‐1s. In order to evaluate the accuracy of the residents' diagnostic selections, we carried out a validation assessment using a tool used by the residents' supervising hospitalists (who were the attendings of record for those patients). This was carried out on a subset of patients and could be done at any time during the hospital stay. The hospitalists were asked to review their residents' ICD‐9 codes and indicate whether they agreed or disagreed.

RESULTS

A total of 7562 residentpatient diagnostic encounters were studied from July 1, 2007 through June 1, 2008. Mean patient age was 66 19.4 years. The age distribution is given in Table 1 and reveals that 65% of diagnostic encounters were with patients age 60 years or greater. Twelve housestaff teams were studied, each consisting of 2 PGY‐1s and a supervising PGY‐2 or PGY‐3 resident. All ICD‐9 codes were selected by categorical and preliminary internal medicine PGY‐1s on medical ward and intensive care unit rotations. Residents from other departments doing rotations on the medical service were excluded. A validation assessment of 341 patients indicated 83.3% agreement by the supervising hospitalist with the primary ICD‐9 code selected. ICD‐9 codes were then grouped and categorized using ICD‐9 nomenclature with the distribution provided in Table 2. A wide spectrum of clinical conditions is apparent including symptoms and ill‐defined conditions, circulatory disorders, respiratory disorders, neoplasms, genitourinary disorders, digestive disorders, diseases of the blood/blood forming organs, endocrinologic/nutritional/metabolic/emmmune disorders, and disorders of the skin and subcutaneous tissue, overall accounting for about 86% of resident clinical experience.

Patient Age Categories (n = 7,562)
Age CategoryNo.Percent of Total
18294415.83
30394556.02
40497059.32
50591,01013.36
60691,21816.11
70791,46519.37
80891,67322.12
901105957.87
Frequency of the Most Commonly Encountered Diagnoses by ICD‐9 Category Among Patients Evaluated by Internal Medicine Residents
ICD‐9 Category DescriptionFrequencyPercent
  • Abbreviations: ICD‐9, International Classification of Diseases, Ninth Revision.

Symptoms/Ill‐Defined Conditions1,47519.51
Circulatory System1,38118.26
Respiratory System93912.42
Neoplasms5727.56
Genitourinary System5026.64
Digestive System4646.14
Blood/Blood‐Forming Organs4445.87
Endo/Nutritional/Metabolic/Immunity3935.20
Skin and Subcutaneous Tissue3805.03
Injury and Poisoning2222.94
Musculoskeletal/Connective Tissue1992.63
Infectious/Parasitic1942.57
Mental Disorders1662.20
Nervous System/Sense Organs1251.65
Health Status/Contact with Health Services811.07
Pregnancy/Childbirth/Puerperium140.19

We also examined the most common diagnostic conditions within each of these categories. The 3 most common ICD‐9 codes entered by residents within each category are provided in Table 3. Symptoms and ill‐defined conditions represent a sizable portion of resident clinical experience (19.51%). Within this category, the most common conditions were fever; abdominal pain (unspecified site); and chest pain, unspecified. Disorders of the circulatory and respiratory systems were the next most common categories of conditions seen by residents, comprising 18.26% and 12.42%, respectively, of resident clinical experience. Within the category of circulatory disorders, congestive heart failure and acute myocardial infarction were the most common conditions seen; for respiratory disorders, pneumonia, chronic airway obstruction, and asthma were most commonly encountered. In aggregate, symptoms and ill‐defined conditions, and disorders of the circulatory and respiratory systems accounted for 50% of resident clinical experience.

Top 3 ICD‐9 Diagnosis Codes Within Each ICD‐9 Category
ICD‐9 Category DescriptionICD‐9 CodeCode DescriptionFrequencyPercent
  • Abbreviations: Hb‐C, hemoglobin C; ICD‐9, International Classification of Diseases, Ninth Revision.

Symptoms/Ill‐Defined Conditions780.6Fever1902.51
789Abdominal pain; unspecified site1491.97
786.5Chest pain, unspecified1401.85
Circulatory System428Congestive heart failure, unspecified3464.58
410.9Acute myocardial infarction; unspecified site; unspecified episode of care1351.79
410.1Acute myocardial infarction; other anterior wall; unspecified episode of care1061.40
Respiratory System486Pneumonia, organism unspecified3634.80
496Chronic airway obstruction, not elsewhere classified1622.14
493.9Asthma, unspecified; unspecified961.27
Neoplasms199.1Malignant neoplasm without specification of site; other861.14
162.9Malignant neoplasm; bronchus lung; unspecified730.97
202.8Other lymphomas; unspecified site, extranodal and solid organ sites710.94
Genitourinary System599Urinary tract infection, site not specified2473.27
584.9Acute renal failure, unspecified911.20
585.6End stage renal disease400.53
Digestive System578.9Hemorrhage of gastrointestinal tract, unspecified1191.57
558.9Other and unspecified noninfectious gastroenteritis and colitis690.91
577Acute pancreatitis360.48
Blood/Blood‐Forming Organs285.9Anemia, unspecified1271.68
282.64Sickle‐cell/Hb‐C disease with crisis801.06
282.6Sickle‐cell disease, unspecified730.97
Endo/Nutritional/Metabolic/Immunity276.1Hypoosmolality and/or hyponatremia570.75
251.2Hypoglycemia, unspecified560.74
250.1Diabetes with ketoacidosis; type II, not stated as uncontrolled500.66
Skin and Subcutaneous Tissue682.9Other cellulitis and abscess; unspecified site2563.39
682.5Other cellulitis and abscess; buttock370.49
686.9Unspecified local infection of skin and subcutaneous tissue230.30
Injury and Poisoning848.9Unspecified site of sprain and strain320.42
977.9Poisoning by unspecified drug or medicinal substance320.42
829Fracture; unspecified bone, closed220.29
Musculoskeletal/Connective Tissue730.2Unspecified osteomyelitis; site unspecified330.44
710Systemic lupus erythematosus250.33
728.87Muscle weakness (generalized)190.25
Infectious/Parasitic38.9Unspecified septicemia580.77
8.45Intestinal infection/clostridium difficile540.71
9.1Colitis, enteritis, and gastroenteritis of presumed infectious organ150.20
Mental Disorders291.81Alcohol withdrawal430.57
307.9Other and unspecified special symptoms or syndromes, not elsewhere classified350.46
294.8Other persistent mental disorders due to conditions classified elsewhere200.26
Nervous System/Sense Organs322.9Meningitis, unspecified300.40
331Alzheimer's disease140.19
340Multiple sclerosis60.08
Health Status/Contact with Health Services885.9Accidental fall from other slipping tripping or stumbling180.24
884.4Accidental fall from bed70.09
V13.02Personal history of urinary (tract) infection40.05
Pregnancy/Childbirth/Puerperium673.8Other pulmonary embolism; unspecified episode of care90.12
665Rupture of uterus before onset of labor; unspecified episode of care10.01
665.7Pelvic hematoma, unspecified episode of care10.01

Individual resident clinical experience varied as well. As shown in Table 4, for a group of 6 PGY‐1s, there was substantial variability in the ICD‐9 diagnostic categories. For example, the percentages of codes falling into the cardiovascular disease category ranged from 15.27% to 27.91%, and for respiratory disease ranged from 8.22% to 18.55%. These data suggest that there may be sizable differences in the proportions of various clinical conditions seen by residents over a year of training.

ICD‐9 Category Variability Among PGY‐1s
ICD‐9 Category DescriptionMeanSDMinMax
  • NOTE: To evaluate the extent of variability in diagnostic conditions seen by PGY‐1s based on their entry of ICD‐9 codes, we examined ICD‐9 data for 6 PGY‐1s over the time period of the study, calculated percentages in each ICD‐9 category, and evaluated the mean, standard deviation (SD), minimum (Min), and maximum (Max) values in each category.

  • Abbreviations: ICD‐9, International Classification of Diseases, Ninth Revision; PGY‐1s, postgraduate year‐1s.

Symptoms/Ill‐Defined Conditions21.435.0715.5029.90
Circulatory System21.844.3815.2727.91
Respiratory System12.433.838.2218.55
Neoplasms8.472.644.1211.80
Genitourinary System5.261.094.036.98
Digestive System4.530.963.095.65
Blood/Blood‐Forming Organs4.642.733.0510.05
Endo/Nutritional/Metabolic/Immunity5.641.683.117.22
Skin and Subcutaneous Tissue4.281.632.426.19
Injury and Poisoning3.901.013.095.43
Musculoskeletal/Connective Tissue2.861.361.554.58
Infectious/Parasitic3.862.622.428.53
Mental Disorders1.470.620.812.28
Nervous System/Sense Organs1.490.870.623.09

DISCUSSION

Years ago, residency training transitioned from a predominantly bedside experience to a curriculum with a large didactic, non‐bedside component, following parameters defined by organizations such as the Accreditation Council for Graduate Medical Education. Residency training is undergoing substantial change to become competency‐based and to organize learning around patient care experiences.2, 3, 9 The Educational Innovation Project of the Residency Review Committee for Internal Medicine is one such endeavor to help develop new methods by which to accomplish this.1 Effective incorporation of innovative experiential learning methods, based on the core competencies, will require a detailed knowledge of resident clinical experience during the course of their training, yet such data have been sparse in internal medicine. Sequist et al. analyzed data from an electronic medical record to assess resident clinical experience in the outpatient setting.4 Bachur and Nagler have used an electronic patient tracking system to assess the clinical experience of pediatric emergency medicine fellows.5, 6 Most attempts to describe resident clinical experience have relied upon extracting diagnostic information from medical records, case logs, etc, though in another approach, Rohrbaugh et al. reviewed psychiatric resident prescription profiles,7 which might provide some indirect data on clinical experience if applied to internal medicine.

In this study, we attempted to quantify resident clinical experience using resident‐selected ICD‐9 codes, in contrast to other methods that have relied upon medical record review and other resident‐independent approaches. There are various strengths and limitations to this approach. Using the ICD‐9 system provides a number of strengths, a major one being standardization, allowing comparisons between different programs and perhaps even facilitating the development of guidelines for resident clinical experience. In addition, this approach using the ICD‐9 system could be readily implemented at any institution and does not require any specific technology. While we chose to do this through our handoff system, an institution could use any of a variety of other systems to accomplish this. For example, resident‐entered ICD‐9 coding systems could be incorporated into electronic discharge summaries, history and physicals, or progress notes. There may also be some practical benefits to having residents learn how to use the ICD‐9 system at this stage of their careers.

There are limitations to this approach as well. The ICD‐9 system was not intended to be used for medical education purposes. There are features of it that can make finding the best diagnosis difficult, and routes to it may at times seem counterintuitive. While we did not carry out resident surveys, a number of residents anecdotally mentioned that it took time to become comfortable using the system, and it could be challenging at times to find a diagnosis description that best fit what they were looking for. To make diagnosis selection easier, we created an organ system‐based dropdown list in the handoff tool so that when residents select an organ system, another list opens up containing commonly used ICD‐9 codes. This grouping is based on organ system alone and does not necessarily follow the ICD‐9 grouping (in contrast, our reported data in this article are all based on ICD‐9 grouping). A search tool to allow searching the entire ICD‐9 database was also made available on the handoff tool. Other factors that could limit diagnosis code accuracy could be lack of clinical knowledge, and error as a result of pressure to come up with a diagnosis because of the hard stop design of our system, in which residents were required to enter a primary diagnosis, potentially causing alert fatigue. A validation assessment that we carried out revealed fairly good agreement with the specific ICD‐9 codes chosen by the resident, but greater accuracy would be desirable. Further education on diagnosis selection and refinements to the handoff tool should help facilitate this. We are currently addressing this by ongoing education on diagnosis selection and by having the hospitalists share the handoff tool with the residents, allowing them to provide direct feedback on diagnostic selections.

More than 19% of the diagnoses selected by residents fell into the category of symptoms and ill‐defined conditions. This raises a number of potential educational issues. One of those is that if residents do, in fact, encounter such entities at such a high frequency, then the internal medicine curriculum must be structured in such a way as to complement this clinical experience with a comprehensive learning program. However, we must also consider the possibility that, in many such instances, a more definitive diagnosis became evident by the time of discharge and this may not have been reflected in the ICD‐9 code that the resident chose. Hence, the category of symptoms and ill‐defined conditions may actually be somewhat smaller than our findings would suggest.

Many issues will need to be addressed as programs obtain more data on their residents' clinical experience. While there may be many reasons to use the ICD‐9 system for selecting diagnoses including those listed above, the system by which ICD‐9 groups diagnoses might not provide ideal educational information, again as the ICD‐9 system was not designed for this purpose. While in this article we have reported the residents' diagnostic encounters grouped according to the ICD‐9 grouping system to provide an initial standardized description, grouping according to another diagnostic system that is felt to be more educationally meaningful may be preferred.

While one might assume that a higher frequency of exposure to certain clinical conditions should enhance competency, that relationship may not be straightforward in internal medicine. For surgical procedures, there are, in fact, data to show improved outcomes for surgeons with higher operative volumes for those procedures,10 but in internal medicine, we do not have data to demonstrate that competence of a resident caring for a particular condition is enhanced by experience alone. Therefore, as programs obtain more data on clinical experience, it will be important that the focus be kept on quality as opposed to quantity.

Obtaining data on resident clinical experience might greatly facilitate experiential learning approaches. For example, as residents go through training and encounter specific diagnostic conditions, those experiences could be supplemented by various learning innovations to make those experiences more meaningful and, hopefully, more likely to result in the development of competence, though that will require measurement. In our program, for example, we have incorporated an approach using illness scenarios, in that when residents have had a certain level of clinical experience with a given clinical condition, they are assembled in small groups and competency‐based case discussions are carried out with a preceptor. In addition, for those instances in which an individual resident may lack direct clinical experience in a certain area, this might be addressed by interventions to increase their contact with those conditions and/or targeted learning interventions to help develop competence. A resident found to be lacking in clinical experience in a certain area could be assigned to the care of more patients with that condition, or to spending more time in a venue in which that condition is more likely to be encountered. Various learning activities including didactics, case discussions, simulation, self‐directed learning, and others could also be used to compensate for such variability. Furthermore, if a residency program's aggregate clinical experience is divergent from some desirable standard yet to be determined, a detailed knowledge of this could help guide that program's curriculum revision. For example, for residents in a program in which there is relatively low exposure to patients with oncological issues, this could be compensated for by external rotations to achieve more clinical experience in oncology, as well as supplementation of the curriculum with additional learning activities in oncology, which could include small group discussions, self‐directed learning activities, case discussions, and others. While at present there are no defined standards for clinical experience and it remains to be seen if there would be a correlation with development of competence, no such standard would serve a purpose if programs did not have reliable and practical means of clinical experience assessment.

In summary, resident‐selected ICD‐9 codes may be a useful means to obtain data regarding resident clinical experience in internal medicine. Such data may be useful to residency training programs in developing new curricula based on experiential learning.

Files
References
  1. Mladenovic J,Bush R,Frohna J.Internal medicine's Educational Innovations Project: improving health care and learning.Am J Med.2009;122:398404.
  2. Fitzgibbons JP,Bordley DR,Berkowitz LR,Miller BW,Henderson MC.Redesigning residency education in internal medicine: a position paper from the Association of Program Directors in Internal Medicine.Ann Intern Med.2006;144:920926.
  3. Weinberger SE,Smith LG,Collier VUfor the Education Committee of the American College of Physicians.Redesigning training for internal medicine.Ann Intern Med.2006;144:927932.
  4. Sequist TD,Singh S,Pereira AG,Rusinak D,Pearson SD.Use of an electronic medical record to profile the continuity clinic experiences of primary care residents.Acad Med.2005;80:390394.
  5. Nagler J,Harper MB,Bachur RG.An automated electronic case log: using electronic information systems to assess training in emergency medicine.Acad Emerg Med.2006;13:733739.
  6. Bachur RG,Nagler J.Use of an automated electronic case log to assess fellowship training: tracking the pediatric emergency medicine experience.Pediatr Emerg Care.2008;24:7582.
  7. Rohrbaugh R,Federman DG,Borysiuk L,Sernyak M.Utilizing VA information technology to develop psychiatric resident prescription profiles.Acad Psychiatry.2009;33:2730.
  8. Mattana J,Charitou M,Mills L, et al.Personal digital assistants (PDAs): a review of their application in graduate medical education.Am J Med Qual.2005;20:262267.
  9. Meyers FJ,Weinberger SE,Fitzgibbons JP, et al.Redesigning residency training in internal medicine: the consensus report of the Alliance for Academic Internal Medicine Education Redesign Task Force.Acad Med.2007;82:12111219.
  10. Birkmeyer JD,Stukel TA,Siewers AE,Goodney PP,Wennberg DE,Lucas FL.Surgeon volume and operative mortality in the United States.N Engl J Med.2003;349:21172127.
Article PDF
Issue
Journal of Hospital Medicine - 6(7)
Publications
Page Number
395-400
Legacy Keywords
clinical experience, experiential learning, residency training
Sections
Files
Files
Article PDF
Article PDF

Internal medicine residency training continues to evolve as competency‐based and with education organized around patient care.13 Making the patient the center of resident education provides an opportunity for experiential learning in which learning can be organized around the clinical conditions that residents encounter. Despite the renewed emphasis on using patient experience as the basis for residency education, little is known regarding what specific diagnostic conditions are seen by internal medicine residents throughout their training. Attempts have been made to quantify resident clinical experience in various fields, using approaches such as review of medical records, case logs, and prescription profiles, but to date, we lack systematic methods to obtain clinical experience data for internal medicine residents.47

While residency curricula in internal medicine typically outlines specific rotations in various clinical areas such as general medical wards, cardiology services, and intensive care units, time spent on such rotations does not necessarily provide quantitative data on the actual clinical conditions that residents encounter, nor does it ensure consistent clinical experience between residents. It is plausible that there may be substantial variability in clinical experience between residents within the same program, and that the overall spectrum of clinical disorders seen by residents in a program may or may not be consistent with a desired optimum, though this is yet to be defined.

If residency education in internal medicine is to progressively incorporate more experiential learning, detailed knowledge of the clinical conditions seen by residents should be useful, not only for overall curriculum design, but this might also allow for various educational interventions to be made when there are variations in clinical experience between residents. Our program has been interested in the application of electronic resources for the improvement of patient care, such as through the handoff process and the use of personal digital assistants.8 We previously did a small analysis of clinical conditions seen by residents through non‐International Classification of Diseases, Ninth Revision (ICD‐9)‐based data they entered onto personal digital assistants. This suggested to us that electronic resources used by residents might serve as a venue by which they could enter diagnostic information which we could use to generate a more detailed analysis of the clinical conditions that they see. Here we describe a method by which we have attempted to quantify resident clinical experience in internal medicine using a modification of an electronic handoff system.

METHODS

The study was conducted within the Internal Medicine Residency Program at the Long Island Jewish Medical Center in New Hyde Park, New York, part of the North ShoreLong Island Jewish Health System, and was approved by the Institutional Review Board. This work was carried out as part of our participation in the Educational Innovation Project of the Residency Review Committee for Internal Medicine. A central objective of our proposal was to develop a method to assess residents' clinical experience on an individual and an aggregate basis. A group of faculty and residents in our residency program developed an electronic handoff tool which residents use for rapid access to key clinical data for their patients and for the handoff of clinical information for on call coverage. This handoff tool was developed with the technical assistance of MedTech Notes LLC which owns Patient Data Transfer System (PDTS) HandOff Note. We modified the handoff tool to include a section in which residents were required to enter a primary diagnosis for each of their patients (a hard stop design). We chose to use the ICD‐9 system for standardization and created two methods to select the code: 1) an organ system‐based dropdown list containing frequently used codes and 2) a search box allowing for searching of the complete ICD‐9 database. For the organ‐based dropdown list, selection of that organ system would reveal a brief list of frequently used codes to make it easier for residents to find them. Prior to using the handoff tool with the ICD‐9based primary diagnosis coding system, training sessions with the residents were conducted by 3 of the investigators along with 3 chief medical residents. These sessions included training not only in technical aspects of how to find diagnosis codes, but also how to make decisions regarding what the primary diagnosis should be. We also instructed our postgraduate year (PGY)‐1s to update their diagnostic selections during the course of the hospital stay.

Each data point represents a resident caring for a patient with a specific diagnostic entity, and is counted once for that resident's period of taking care of that patient. Thirty‐three PGY‐1s were studied and, on the internal medicine service, they were supervised by either hospitalist faculty or voluntary faculty in comparable proportions. If the patient's care is taken over by another resident, that second resident was also recorded as having had a diagnostic encounter with that patient, hence 1 patient could provide experience with the same diagnostic entity for 1 or more residents. Using this method, the denominator is not patients seen, but residentpatient diagnostic encounters that have taken place. The ICD‐9 diagnostic conditions entered by the residents were grouped using the ICD‐9 system. Individual diagnostic profiles for each resident, as well as an aggregate profile for all residents to reflect the residency program as a whole, were generated. We also carried out an analysis of the ICD‐9 codes entered by 6 consecutive PGY‐1s to assess how the diagnostic spectrum might vary among a small sampling of PGY‐1s. In order to evaluate the accuracy of the residents' diagnostic selections, we carried out a validation assessment using a tool used by the residents' supervising hospitalists (who were the attendings of record for those patients). This was carried out on a subset of patients and could be done at any time during the hospital stay. The hospitalists were asked to review their residents' ICD‐9 codes and indicate whether they agreed or disagreed.

RESULTS

A total of 7562 residentpatient diagnostic encounters were studied from July 1, 2007 through June 1, 2008. Mean patient age was 66 19.4 years. The age distribution is given in Table 1 and reveals that 65% of diagnostic encounters were with patients age 60 years or greater. Twelve housestaff teams were studied, each consisting of 2 PGY‐1s and a supervising PGY‐2 or PGY‐3 resident. All ICD‐9 codes were selected by categorical and preliminary internal medicine PGY‐1s on medical ward and intensive care unit rotations. Residents from other departments doing rotations on the medical service were excluded. A validation assessment of 341 patients indicated 83.3% agreement by the supervising hospitalist with the primary ICD‐9 code selected. ICD‐9 codes were then grouped and categorized using ICD‐9 nomenclature with the distribution provided in Table 2. A wide spectrum of clinical conditions is apparent including symptoms and ill‐defined conditions, circulatory disorders, respiratory disorders, neoplasms, genitourinary disorders, digestive disorders, diseases of the blood/blood forming organs, endocrinologic/nutritional/metabolic/emmmune disorders, and disorders of the skin and subcutaneous tissue, overall accounting for about 86% of resident clinical experience.

Patient Age Categories (n = 7,562)
Age CategoryNo.Percent of Total
18294415.83
30394556.02
40497059.32
50591,01013.36
60691,21816.11
70791,46519.37
80891,67322.12
901105957.87
Frequency of the Most Commonly Encountered Diagnoses by ICD‐9 Category Among Patients Evaluated by Internal Medicine Residents
ICD‐9 Category DescriptionFrequencyPercent
  • Abbreviations: ICD‐9, International Classification of Diseases, Ninth Revision.

Symptoms/Ill‐Defined Conditions1,47519.51
Circulatory System1,38118.26
Respiratory System93912.42
Neoplasms5727.56
Genitourinary System5026.64
Digestive System4646.14
Blood/Blood‐Forming Organs4445.87
Endo/Nutritional/Metabolic/Immunity3935.20
Skin and Subcutaneous Tissue3805.03
Injury and Poisoning2222.94
Musculoskeletal/Connective Tissue1992.63
Infectious/Parasitic1942.57
Mental Disorders1662.20
Nervous System/Sense Organs1251.65
Health Status/Contact with Health Services811.07
Pregnancy/Childbirth/Puerperium140.19

We also examined the most common diagnostic conditions within each of these categories. The 3 most common ICD‐9 codes entered by residents within each category are provided in Table 3. Symptoms and ill‐defined conditions represent a sizable portion of resident clinical experience (19.51%). Within this category, the most common conditions were fever; abdominal pain (unspecified site); and chest pain, unspecified. Disorders of the circulatory and respiratory systems were the next most common categories of conditions seen by residents, comprising 18.26% and 12.42%, respectively, of resident clinical experience. Within the category of circulatory disorders, congestive heart failure and acute myocardial infarction were the most common conditions seen; for respiratory disorders, pneumonia, chronic airway obstruction, and asthma were most commonly encountered. In aggregate, symptoms and ill‐defined conditions, and disorders of the circulatory and respiratory systems accounted for 50% of resident clinical experience.

Top 3 ICD‐9 Diagnosis Codes Within Each ICD‐9 Category
ICD‐9 Category DescriptionICD‐9 CodeCode DescriptionFrequencyPercent
  • Abbreviations: Hb‐C, hemoglobin C; ICD‐9, International Classification of Diseases, Ninth Revision.

Symptoms/Ill‐Defined Conditions780.6Fever1902.51
789Abdominal pain; unspecified site1491.97
786.5Chest pain, unspecified1401.85
Circulatory System428Congestive heart failure, unspecified3464.58
410.9Acute myocardial infarction; unspecified site; unspecified episode of care1351.79
410.1Acute myocardial infarction; other anterior wall; unspecified episode of care1061.40
Respiratory System486Pneumonia, organism unspecified3634.80
496Chronic airway obstruction, not elsewhere classified1622.14
493.9Asthma, unspecified; unspecified961.27
Neoplasms199.1Malignant neoplasm without specification of site; other861.14
162.9Malignant neoplasm; bronchus lung; unspecified730.97
202.8Other lymphomas; unspecified site, extranodal and solid organ sites710.94
Genitourinary System599Urinary tract infection, site not specified2473.27
584.9Acute renal failure, unspecified911.20
585.6End stage renal disease400.53
Digestive System578.9Hemorrhage of gastrointestinal tract, unspecified1191.57
558.9Other and unspecified noninfectious gastroenteritis and colitis690.91
577Acute pancreatitis360.48
Blood/Blood‐Forming Organs285.9Anemia, unspecified1271.68
282.64Sickle‐cell/Hb‐C disease with crisis801.06
282.6Sickle‐cell disease, unspecified730.97
Endo/Nutritional/Metabolic/Immunity276.1Hypoosmolality and/or hyponatremia570.75
251.2Hypoglycemia, unspecified560.74
250.1Diabetes with ketoacidosis; type II, not stated as uncontrolled500.66
Skin and Subcutaneous Tissue682.9Other cellulitis and abscess; unspecified site2563.39
682.5Other cellulitis and abscess; buttock370.49
686.9Unspecified local infection of skin and subcutaneous tissue230.30
Injury and Poisoning848.9Unspecified site of sprain and strain320.42
977.9Poisoning by unspecified drug or medicinal substance320.42
829Fracture; unspecified bone, closed220.29
Musculoskeletal/Connective Tissue730.2Unspecified osteomyelitis; site unspecified330.44
710Systemic lupus erythematosus250.33
728.87Muscle weakness (generalized)190.25
Infectious/Parasitic38.9Unspecified septicemia580.77
8.45Intestinal infection/clostridium difficile540.71
9.1Colitis, enteritis, and gastroenteritis of presumed infectious organ150.20
Mental Disorders291.81Alcohol withdrawal430.57
307.9Other and unspecified special symptoms or syndromes, not elsewhere classified350.46
294.8Other persistent mental disorders due to conditions classified elsewhere200.26
Nervous System/Sense Organs322.9Meningitis, unspecified300.40
331Alzheimer's disease140.19
340Multiple sclerosis60.08
Health Status/Contact with Health Services885.9Accidental fall from other slipping tripping or stumbling180.24
884.4Accidental fall from bed70.09
V13.02Personal history of urinary (tract) infection40.05
Pregnancy/Childbirth/Puerperium673.8Other pulmonary embolism; unspecified episode of care90.12
665Rupture of uterus before onset of labor; unspecified episode of care10.01
665.7Pelvic hematoma, unspecified episode of care10.01

Individual resident clinical experience varied as well. As shown in Table 4, for a group of 6 PGY‐1s, there was substantial variability in the ICD‐9 diagnostic categories. For example, the percentages of codes falling into the cardiovascular disease category ranged from 15.27% to 27.91%, and for respiratory disease ranged from 8.22% to 18.55%. These data suggest that there may be sizable differences in the proportions of various clinical conditions seen by residents over a year of training.

ICD‐9 Category Variability Among PGY‐1s
ICD‐9 Category DescriptionMeanSDMinMax
  • NOTE: To evaluate the extent of variability in diagnostic conditions seen by PGY‐1s based on their entry of ICD‐9 codes, we examined ICD‐9 data for 6 PGY‐1s over the time period of the study, calculated percentages in each ICD‐9 category, and evaluated the mean, standard deviation (SD), minimum (Min), and maximum (Max) values in each category.

  • Abbreviations: ICD‐9, International Classification of Diseases, Ninth Revision; PGY‐1s, postgraduate year‐1s.

Symptoms/Ill‐Defined Conditions21.435.0715.5029.90
Circulatory System21.844.3815.2727.91
Respiratory System12.433.838.2218.55
Neoplasms8.472.644.1211.80
Genitourinary System5.261.094.036.98
Digestive System4.530.963.095.65
Blood/Blood‐Forming Organs4.642.733.0510.05
Endo/Nutritional/Metabolic/Immunity5.641.683.117.22
Skin and Subcutaneous Tissue4.281.632.426.19
Injury and Poisoning3.901.013.095.43
Musculoskeletal/Connective Tissue2.861.361.554.58
Infectious/Parasitic3.862.622.428.53
Mental Disorders1.470.620.812.28
Nervous System/Sense Organs1.490.870.623.09

DISCUSSION

Years ago, residency training transitioned from a predominantly bedside experience to a curriculum with a large didactic, non‐bedside component, following parameters defined by organizations such as the Accreditation Council for Graduate Medical Education. Residency training is undergoing substantial change to become competency‐based and to organize learning around patient care experiences.2, 3, 9 The Educational Innovation Project of the Residency Review Committee for Internal Medicine is one such endeavor to help develop new methods by which to accomplish this.1 Effective incorporation of innovative experiential learning methods, based on the core competencies, will require a detailed knowledge of resident clinical experience during the course of their training, yet such data have been sparse in internal medicine. Sequist et al. analyzed data from an electronic medical record to assess resident clinical experience in the outpatient setting.4 Bachur and Nagler have used an electronic patient tracking system to assess the clinical experience of pediatric emergency medicine fellows.5, 6 Most attempts to describe resident clinical experience have relied upon extracting diagnostic information from medical records, case logs, etc, though in another approach, Rohrbaugh et al. reviewed psychiatric resident prescription profiles,7 which might provide some indirect data on clinical experience if applied to internal medicine.

In this study, we attempted to quantify resident clinical experience using resident‐selected ICD‐9 codes, in contrast to other methods that have relied upon medical record review and other resident‐independent approaches. There are various strengths and limitations to this approach. Using the ICD‐9 system provides a number of strengths, a major one being standardization, allowing comparisons between different programs and perhaps even facilitating the development of guidelines for resident clinical experience. In addition, this approach using the ICD‐9 system could be readily implemented at any institution and does not require any specific technology. While we chose to do this through our handoff system, an institution could use any of a variety of other systems to accomplish this. For example, resident‐entered ICD‐9 coding systems could be incorporated into electronic discharge summaries, history and physicals, or progress notes. There may also be some practical benefits to having residents learn how to use the ICD‐9 system at this stage of their careers.

There are limitations to this approach as well. The ICD‐9 system was not intended to be used for medical education purposes. There are features of it that can make finding the best diagnosis difficult, and routes to it may at times seem counterintuitive. While we did not carry out resident surveys, a number of residents anecdotally mentioned that it took time to become comfortable using the system, and it could be challenging at times to find a diagnosis description that best fit what they were looking for. To make diagnosis selection easier, we created an organ system‐based dropdown list in the handoff tool so that when residents select an organ system, another list opens up containing commonly used ICD‐9 codes. This grouping is based on organ system alone and does not necessarily follow the ICD‐9 grouping (in contrast, our reported data in this article are all based on ICD‐9 grouping). A search tool to allow searching the entire ICD‐9 database was also made available on the handoff tool. Other factors that could limit diagnosis code accuracy could be lack of clinical knowledge, and error as a result of pressure to come up with a diagnosis because of the hard stop design of our system, in which residents were required to enter a primary diagnosis, potentially causing alert fatigue. A validation assessment that we carried out revealed fairly good agreement with the specific ICD‐9 codes chosen by the resident, but greater accuracy would be desirable. Further education on diagnosis selection and refinements to the handoff tool should help facilitate this. We are currently addressing this by ongoing education on diagnosis selection and by having the hospitalists share the handoff tool with the residents, allowing them to provide direct feedback on diagnostic selections.

More than 19% of the diagnoses selected by residents fell into the category of symptoms and ill‐defined conditions. This raises a number of potential educational issues. One of those is that if residents do, in fact, encounter such entities at such a high frequency, then the internal medicine curriculum must be structured in such a way as to complement this clinical experience with a comprehensive learning program. However, we must also consider the possibility that, in many such instances, a more definitive diagnosis became evident by the time of discharge and this may not have been reflected in the ICD‐9 code that the resident chose. Hence, the category of symptoms and ill‐defined conditions may actually be somewhat smaller than our findings would suggest.

Many issues will need to be addressed as programs obtain more data on their residents' clinical experience. While there may be many reasons to use the ICD‐9 system for selecting diagnoses including those listed above, the system by which ICD‐9 groups diagnoses might not provide ideal educational information, again as the ICD‐9 system was not designed for this purpose. While in this article we have reported the residents' diagnostic encounters grouped according to the ICD‐9 grouping system to provide an initial standardized description, grouping according to another diagnostic system that is felt to be more educationally meaningful may be preferred.

While one might assume that a higher frequency of exposure to certain clinical conditions should enhance competency, that relationship may not be straightforward in internal medicine. For surgical procedures, there are, in fact, data to show improved outcomes for surgeons with higher operative volumes for those procedures,10 but in internal medicine, we do not have data to demonstrate that competence of a resident caring for a particular condition is enhanced by experience alone. Therefore, as programs obtain more data on clinical experience, it will be important that the focus be kept on quality as opposed to quantity.

Obtaining data on resident clinical experience might greatly facilitate experiential learning approaches. For example, as residents go through training and encounter specific diagnostic conditions, those experiences could be supplemented by various learning innovations to make those experiences more meaningful and, hopefully, more likely to result in the development of competence, though that will require measurement. In our program, for example, we have incorporated an approach using illness scenarios, in that when residents have had a certain level of clinical experience with a given clinical condition, they are assembled in small groups and competency‐based case discussions are carried out with a preceptor. In addition, for those instances in which an individual resident may lack direct clinical experience in a certain area, this might be addressed by interventions to increase their contact with those conditions and/or targeted learning interventions to help develop competence. A resident found to be lacking in clinical experience in a certain area could be assigned to the care of more patients with that condition, or to spending more time in a venue in which that condition is more likely to be encountered. Various learning activities including didactics, case discussions, simulation, self‐directed learning, and others could also be used to compensate for such variability. Furthermore, if a residency program's aggregate clinical experience is divergent from some desirable standard yet to be determined, a detailed knowledge of this could help guide that program's curriculum revision. For example, for residents in a program in which there is relatively low exposure to patients with oncological issues, this could be compensated for by external rotations to achieve more clinical experience in oncology, as well as supplementation of the curriculum with additional learning activities in oncology, which could include small group discussions, self‐directed learning activities, case discussions, and others. While at present there are no defined standards for clinical experience and it remains to be seen if there would be a correlation with development of competence, no such standard would serve a purpose if programs did not have reliable and practical means of clinical experience assessment.

In summary, resident‐selected ICD‐9 codes may be a useful means to obtain data regarding resident clinical experience in internal medicine. Such data may be useful to residency training programs in developing new curricula based on experiential learning.

Internal medicine residency training continues to evolve as competency‐based and with education organized around patient care.13 Making the patient the center of resident education provides an opportunity for experiential learning in which learning can be organized around the clinical conditions that residents encounter. Despite the renewed emphasis on using patient experience as the basis for residency education, little is known regarding what specific diagnostic conditions are seen by internal medicine residents throughout their training. Attempts have been made to quantify resident clinical experience in various fields, using approaches such as review of medical records, case logs, and prescription profiles, but to date, we lack systematic methods to obtain clinical experience data for internal medicine residents.47

While residency curricula in internal medicine typically outlines specific rotations in various clinical areas such as general medical wards, cardiology services, and intensive care units, time spent on such rotations does not necessarily provide quantitative data on the actual clinical conditions that residents encounter, nor does it ensure consistent clinical experience between residents. It is plausible that there may be substantial variability in clinical experience between residents within the same program, and that the overall spectrum of clinical disorders seen by residents in a program may or may not be consistent with a desired optimum, though this is yet to be defined.

If residency education in internal medicine is to progressively incorporate more experiential learning, detailed knowledge of the clinical conditions seen by residents should be useful, not only for overall curriculum design, but this might also allow for various educational interventions to be made when there are variations in clinical experience between residents. Our program has been interested in the application of electronic resources for the improvement of patient care, such as through the handoff process and the use of personal digital assistants.8 We previously did a small analysis of clinical conditions seen by residents through non‐International Classification of Diseases, Ninth Revision (ICD‐9)‐based data they entered onto personal digital assistants. This suggested to us that electronic resources used by residents might serve as a venue by which they could enter diagnostic information which we could use to generate a more detailed analysis of the clinical conditions that they see. Here we describe a method by which we have attempted to quantify resident clinical experience in internal medicine using a modification of an electronic handoff system.

METHODS

The study was conducted within the Internal Medicine Residency Program at the Long Island Jewish Medical Center in New Hyde Park, New York, part of the North ShoreLong Island Jewish Health System, and was approved by the Institutional Review Board. This work was carried out as part of our participation in the Educational Innovation Project of the Residency Review Committee for Internal Medicine. A central objective of our proposal was to develop a method to assess residents' clinical experience on an individual and an aggregate basis. A group of faculty and residents in our residency program developed an electronic handoff tool which residents use for rapid access to key clinical data for their patients and for the handoff of clinical information for on call coverage. This handoff tool was developed with the technical assistance of MedTech Notes LLC which owns Patient Data Transfer System (PDTS) HandOff Note. We modified the handoff tool to include a section in which residents were required to enter a primary diagnosis for each of their patients (a hard stop design). We chose to use the ICD‐9 system for standardization and created two methods to select the code: 1) an organ system‐based dropdown list containing frequently used codes and 2) a search box allowing for searching of the complete ICD‐9 database. For the organ‐based dropdown list, selection of that organ system would reveal a brief list of frequently used codes to make it easier for residents to find them. Prior to using the handoff tool with the ICD‐9based primary diagnosis coding system, training sessions with the residents were conducted by 3 of the investigators along with 3 chief medical residents. These sessions included training not only in technical aspects of how to find diagnosis codes, but also how to make decisions regarding what the primary diagnosis should be. We also instructed our postgraduate year (PGY)‐1s to update their diagnostic selections during the course of the hospital stay.

Each data point represents a resident caring for a patient with a specific diagnostic entity, and is counted once for that resident's period of taking care of that patient. Thirty‐three PGY‐1s were studied and, on the internal medicine service, they were supervised by either hospitalist faculty or voluntary faculty in comparable proportions. If the patient's care is taken over by another resident, that second resident was also recorded as having had a diagnostic encounter with that patient, hence 1 patient could provide experience with the same diagnostic entity for 1 or more residents. Using this method, the denominator is not patients seen, but residentpatient diagnostic encounters that have taken place. The ICD‐9 diagnostic conditions entered by the residents were grouped using the ICD‐9 system. Individual diagnostic profiles for each resident, as well as an aggregate profile for all residents to reflect the residency program as a whole, were generated. We also carried out an analysis of the ICD‐9 codes entered by 6 consecutive PGY‐1s to assess how the diagnostic spectrum might vary among a small sampling of PGY‐1s. In order to evaluate the accuracy of the residents' diagnostic selections, we carried out a validation assessment using a tool used by the residents' supervising hospitalists (who were the attendings of record for those patients). This was carried out on a subset of patients and could be done at any time during the hospital stay. The hospitalists were asked to review their residents' ICD‐9 codes and indicate whether they agreed or disagreed.

RESULTS

A total of 7562 residentpatient diagnostic encounters were studied from July 1, 2007 through June 1, 2008. Mean patient age was 66 19.4 years. The age distribution is given in Table 1 and reveals that 65% of diagnostic encounters were with patients age 60 years or greater. Twelve housestaff teams were studied, each consisting of 2 PGY‐1s and a supervising PGY‐2 or PGY‐3 resident. All ICD‐9 codes were selected by categorical and preliminary internal medicine PGY‐1s on medical ward and intensive care unit rotations. Residents from other departments doing rotations on the medical service were excluded. A validation assessment of 341 patients indicated 83.3% agreement by the supervising hospitalist with the primary ICD‐9 code selected. ICD‐9 codes were then grouped and categorized using ICD‐9 nomenclature with the distribution provided in Table 2. A wide spectrum of clinical conditions is apparent including symptoms and ill‐defined conditions, circulatory disorders, respiratory disorders, neoplasms, genitourinary disorders, digestive disorders, diseases of the blood/blood forming organs, endocrinologic/nutritional/metabolic/emmmune disorders, and disorders of the skin and subcutaneous tissue, overall accounting for about 86% of resident clinical experience.

Patient Age Categories (n = 7,562)
Age CategoryNo.Percent of Total
18294415.83
30394556.02
40497059.32
50591,01013.36
60691,21816.11
70791,46519.37
80891,67322.12
901105957.87
Frequency of the Most Commonly Encountered Diagnoses by ICD‐9 Category Among Patients Evaluated by Internal Medicine Residents
ICD‐9 Category DescriptionFrequencyPercent
  • Abbreviations: ICD‐9, International Classification of Diseases, Ninth Revision.

Symptoms/Ill‐Defined Conditions1,47519.51
Circulatory System1,38118.26
Respiratory System93912.42
Neoplasms5727.56
Genitourinary System5026.64
Digestive System4646.14
Blood/Blood‐Forming Organs4445.87
Endo/Nutritional/Metabolic/Immunity3935.20
Skin and Subcutaneous Tissue3805.03
Injury and Poisoning2222.94
Musculoskeletal/Connective Tissue1992.63
Infectious/Parasitic1942.57
Mental Disorders1662.20
Nervous System/Sense Organs1251.65
Health Status/Contact with Health Services811.07
Pregnancy/Childbirth/Puerperium140.19

We also examined the most common diagnostic conditions within each of these categories. The 3 most common ICD‐9 codes entered by residents within each category are provided in Table 3. Symptoms and ill‐defined conditions represent a sizable portion of resident clinical experience (19.51%). Within this category, the most common conditions were fever; abdominal pain (unspecified site); and chest pain, unspecified. Disorders of the circulatory and respiratory systems were the next most common categories of conditions seen by residents, comprising 18.26% and 12.42%, respectively, of resident clinical experience. Within the category of circulatory disorders, congestive heart failure and acute myocardial infarction were the most common conditions seen; for respiratory disorders, pneumonia, chronic airway obstruction, and asthma were most commonly encountered. In aggregate, symptoms and ill‐defined conditions, and disorders of the circulatory and respiratory systems accounted for 50% of resident clinical experience.

Top 3 ICD‐9 Diagnosis Codes Within Each ICD‐9 Category
ICD‐9 Category DescriptionICD‐9 CodeCode DescriptionFrequencyPercent
  • Abbreviations: Hb‐C, hemoglobin C; ICD‐9, International Classification of Diseases, Ninth Revision.

Symptoms/Ill‐Defined Conditions780.6Fever1902.51
789Abdominal pain; unspecified site1491.97
786.5Chest pain, unspecified1401.85
Circulatory System428Congestive heart failure, unspecified3464.58
410.9Acute myocardial infarction; unspecified site; unspecified episode of care1351.79
410.1Acute myocardial infarction; other anterior wall; unspecified episode of care1061.40
Respiratory System486Pneumonia, organism unspecified3634.80
496Chronic airway obstruction, not elsewhere classified1622.14
493.9Asthma, unspecified; unspecified961.27
Neoplasms199.1Malignant neoplasm without specification of site; other861.14
162.9Malignant neoplasm; bronchus lung; unspecified730.97
202.8Other lymphomas; unspecified site, extranodal and solid organ sites710.94
Genitourinary System599Urinary tract infection, site not specified2473.27
584.9Acute renal failure, unspecified911.20
585.6End stage renal disease400.53
Digestive System578.9Hemorrhage of gastrointestinal tract, unspecified1191.57
558.9Other and unspecified noninfectious gastroenteritis and colitis690.91
577Acute pancreatitis360.48
Blood/Blood‐Forming Organs285.9Anemia, unspecified1271.68
282.64Sickle‐cell/Hb‐C disease with crisis801.06
282.6Sickle‐cell disease, unspecified730.97
Endo/Nutritional/Metabolic/Immunity276.1Hypoosmolality and/or hyponatremia570.75
251.2Hypoglycemia, unspecified560.74
250.1Diabetes with ketoacidosis; type II, not stated as uncontrolled500.66
Skin and Subcutaneous Tissue682.9Other cellulitis and abscess; unspecified site2563.39
682.5Other cellulitis and abscess; buttock370.49
686.9Unspecified local infection of skin and subcutaneous tissue230.30
Injury and Poisoning848.9Unspecified site of sprain and strain320.42
977.9Poisoning by unspecified drug or medicinal substance320.42
829Fracture; unspecified bone, closed220.29
Musculoskeletal/Connective Tissue730.2Unspecified osteomyelitis; site unspecified330.44
710Systemic lupus erythematosus250.33
728.87Muscle weakness (generalized)190.25
Infectious/Parasitic38.9Unspecified septicemia580.77
8.45Intestinal infection/clostridium difficile540.71
9.1Colitis, enteritis, and gastroenteritis of presumed infectious organ150.20
Mental Disorders291.81Alcohol withdrawal430.57
307.9Other and unspecified special symptoms or syndromes, not elsewhere classified350.46
294.8Other persistent mental disorders due to conditions classified elsewhere200.26
Nervous System/Sense Organs322.9Meningitis, unspecified300.40
331Alzheimer's disease140.19
340Multiple sclerosis60.08
Health Status/Contact with Health Services885.9Accidental fall from other slipping tripping or stumbling180.24
884.4Accidental fall from bed70.09
V13.02Personal history of urinary (tract) infection40.05
Pregnancy/Childbirth/Puerperium673.8Other pulmonary embolism; unspecified episode of care90.12
665Rupture of uterus before onset of labor; unspecified episode of care10.01
665.7Pelvic hematoma, unspecified episode of care10.01

Individual resident clinical experience varied as well. As shown in Table 4, for a group of 6 PGY‐1s, there was substantial variability in the ICD‐9 diagnostic categories. For example, the percentages of codes falling into the cardiovascular disease category ranged from 15.27% to 27.91%, and for respiratory disease ranged from 8.22% to 18.55%. These data suggest that there may be sizable differences in the proportions of various clinical conditions seen by residents over a year of training.

ICD‐9 Category Variability Among PGY‐1s
ICD‐9 Category DescriptionMeanSDMinMax
  • NOTE: To evaluate the extent of variability in diagnostic conditions seen by PGY‐1s based on their entry of ICD‐9 codes, we examined ICD‐9 data for 6 PGY‐1s over the time period of the study, calculated percentages in each ICD‐9 category, and evaluated the mean, standard deviation (SD), minimum (Min), and maximum (Max) values in each category.

  • Abbreviations: ICD‐9, International Classification of Diseases, Ninth Revision; PGY‐1s, postgraduate year‐1s.

Symptoms/Ill‐Defined Conditions21.435.0715.5029.90
Circulatory System21.844.3815.2727.91
Respiratory System12.433.838.2218.55
Neoplasms8.472.644.1211.80
Genitourinary System5.261.094.036.98
Digestive System4.530.963.095.65
Blood/Blood‐Forming Organs4.642.733.0510.05
Endo/Nutritional/Metabolic/Immunity5.641.683.117.22
Skin and Subcutaneous Tissue4.281.632.426.19
Injury and Poisoning3.901.013.095.43
Musculoskeletal/Connective Tissue2.861.361.554.58
Infectious/Parasitic3.862.622.428.53
Mental Disorders1.470.620.812.28
Nervous System/Sense Organs1.490.870.623.09

DISCUSSION

Years ago, residency training transitioned from a predominantly bedside experience to a curriculum with a large didactic, non‐bedside component, following parameters defined by organizations such as the Accreditation Council for Graduate Medical Education. Residency training is undergoing substantial change to become competency‐based and to organize learning around patient care experiences.2, 3, 9 The Educational Innovation Project of the Residency Review Committee for Internal Medicine is one such endeavor to help develop new methods by which to accomplish this.1 Effective incorporation of innovative experiential learning methods, based on the core competencies, will require a detailed knowledge of resident clinical experience during the course of their training, yet such data have been sparse in internal medicine. Sequist et al. analyzed data from an electronic medical record to assess resident clinical experience in the outpatient setting.4 Bachur and Nagler have used an electronic patient tracking system to assess the clinical experience of pediatric emergency medicine fellows.5, 6 Most attempts to describe resident clinical experience have relied upon extracting diagnostic information from medical records, case logs, etc, though in another approach, Rohrbaugh et al. reviewed psychiatric resident prescription profiles,7 which might provide some indirect data on clinical experience if applied to internal medicine.

In this study, we attempted to quantify resident clinical experience using resident‐selected ICD‐9 codes, in contrast to other methods that have relied upon medical record review and other resident‐independent approaches. There are various strengths and limitations to this approach. Using the ICD‐9 system provides a number of strengths, a major one being standardization, allowing comparisons between different programs and perhaps even facilitating the development of guidelines for resident clinical experience. In addition, this approach using the ICD‐9 system could be readily implemented at any institution and does not require any specific technology. While we chose to do this through our handoff system, an institution could use any of a variety of other systems to accomplish this. For example, resident‐entered ICD‐9 coding systems could be incorporated into electronic discharge summaries, history and physicals, or progress notes. There may also be some practical benefits to having residents learn how to use the ICD‐9 system at this stage of their careers.

There are limitations to this approach as well. The ICD‐9 system was not intended to be used for medical education purposes. There are features of it that can make finding the best diagnosis difficult, and routes to it may at times seem counterintuitive. While we did not carry out resident surveys, a number of residents anecdotally mentioned that it took time to become comfortable using the system, and it could be challenging at times to find a diagnosis description that best fit what they were looking for. To make diagnosis selection easier, we created an organ system‐based dropdown list in the handoff tool so that when residents select an organ system, another list opens up containing commonly used ICD‐9 codes. This grouping is based on organ system alone and does not necessarily follow the ICD‐9 grouping (in contrast, our reported data in this article are all based on ICD‐9 grouping). A search tool to allow searching the entire ICD‐9 database was also made available on the handoff tool. Other factors that could limit diagnosis code accuracy could be lack of clinical knowledge, and error as a result of pressure to come up with a diagnosis because of the hard stop design of our system, in which residents were required to enter a primary diagnosis, potentially causing alert fatigue. A validation assessment that we carried out revealed fairly good agreement with the specific ICD‐9 codes chosen by the resident, but greater accuracy would be desirable. Further education on diagnosis selection and refinements to the handoff tool should help facilitate this. We are currently addressing this by ongoing education on diagnosis selection and by having the hospitalists share the handoff tool with the residents, allowing them to provide direct feedback on diagnostic selections.

More than 19% of the diagnoses selected by residents fell into the category of symptoms and ill‐defined conditions. This raises a number of potential educational issues. One of those is that if residents do, in fact, encounter such entities at such a high frequency, then the internal medicine curriculum must be structured in such a way as to complement this clinical experience with a comprehensive learning program. However, we must also consider the possibility that, in many such instances, a more definitive diagnosis became evident by the time of discharge and this may not have been reflected in the ICD‐9 code that the resident chose. Hence, the category of symptoms and ill‐defined conditions may actually be somewhat smaller than our findings would suggest.

Many issues will need to be addressed as programs obtain more data on their residents' clinical experience. While there may be many reasons to use the ICD‐9 system for selecting diagnoses including those listed above, the system by which ICD‐9 groups diagnoses might not provide ideal educational information, again as the ICD‐9 system was not designed for this purpose. While in this article we have reported the residents' diagnostic encounters grouped according to the ICD‐9 grouping system to provide an initial standardized description, grouping according to another diagnostic system that is felt to be more educationally meaningful may be preferred.

While one might assume that a higher frequency of exposure to certain clinical conditions should enhance competency, that relationship may not be straightforward in internal medicine. For surgical procedures, there are, in fact, data to show improved outcomes for surgeons with higher operative volumes for those procedures,10 but in internal medicine, we do not have data to demonstrate that competence of a resident caring for a particular condition is enhanced by experience alone. Therefore, as programs obtain more data on clinical experience, it will be important that the focus be kept on quality as opposed to quantity.

Obtaining data on resident clinical experience might greatly facilitate experiential learning approaches. For example, as residents go through training and encounter specific diagnostic conditions, those experiences could be supplemented by various learning innovations to make those experiences more meaningful and, hopefully, more likely to result in the development of competence, though that will require measurement. In our program, for example, we have incorporated an approach using illness scenarios, in that when residents have had a certain level of clinical experience with a given clinical condition, they are assembled in small groups and competency‐based case discussions are carried out with a preceptor. In addition, for those instances in which an individual resident may lack direct clinical experience in a certain area, this might be addressed by interventions to increase their contact with those conditions and/or targeted learning interventions to help develop competence. A resident found to be lacking in clinical experience in a certain area could be assigned to the care of more patients with that condition, or to spending more time in a venue in which that condition is more likely to be encountered. Various learning activities including didactics, case discussions, simulation, self‐directed learning, and others could also be used to compensate for such variability. Furthermore, if a residency program's aggregate clinical experience is divergent from some desirable standard yet to be determined, a detailed knowledge of this could help guide that program's curriculum revision. For example, for residents in a program in which there is relatively low exposure to patients with oncological issues, this could be compensated for by external rotations to achieve more clinical experience in oncology, as well as supplementation of the curriculum with additional learning activities in oncology, which could include small group discussions, self‐directed learning activities, case discussions, and others. While at present there are no defined standards for clinical experience and it remains to be seen if there would be a correlation with development of competence, no such standard would serve a purpose if programs did not have reliable and practical means of clinical experience assessment.

In summary, resident‐selected ICD‐9 codes may be a useful means to obtain data regarding resident clinical experience in internal medicine. Such data may be useful to residency training programs in developing new curricula based on experiential learning.

References
  1. Mladenovic J,Bush R,Frohna J.Internal medicine's Educational Innovations Project: improving health care and learning.Am J Med.2009;122:398404.
  2. Fitzgibbons JP,Bordley DR,Berkowitz LR,Miller BW,Henderson MC.Redesigning residency education in internal medicine: a position paper from the Association of Program Directors in Internal Medicine.Ann Intern Med.2006;144:920926.
  3. Weinberger SE,Smith LG,Collier VUfor the Education Committee of the American College of Physicians.Redesigning training for internal medicine.Ann Intern Med.2006;144:927932.
  4. Sequist TD,Singh S,Pereira AG,Rusinak D,Pearson SD.Use of an electronic medical record to profile the continuity clinic experiences of primary care residents.Acad Med.2005;80:390394.
  5. Nagler J,Harper MB,Bachur RG.An automated electronic case log: using electronic information systems to assess training in emergency medicine.Acad Emerg Med.2006;13:733739.
  6. Bachur RG,Nagler J.Use of an automated electronic case log to assess fellowship training: tracking the pediatric emergency medicine experience.Pediatr Emerg Care.2008;24:7582.
  7. Rohrbaugh R,Federman DG,Borysiuk L,Sernyak M.Utilizing VA information technology to develop psychiatric resident prescription profiles.Acad Psychiatry.2009;33:2730.
  8. Mattana J,Charitou M,Mills L, et al.Personal digital assistants (PDAs): a review of their application in graduate medical education.Am J Med Qual.2005;20:262267.
  9. Meyers FJ,Weinberger SE,Fitzgibbons JP, et al.Redesigning residency training in internal medicine: the consensus report of the Alliance for Academic Internal Medicine Education Redesign Task Force.Acad Med.2007;82:12111219.
  10. Birkmeyer JD,Stukel TA,Siewers AE,Goodney PP,Wennberg DE,Lucas FL.Surgeon volume and operative mortality in the United States.N Engl J Med.2003;349:21172127.
References
  1. Mladenovic J,Bush R,Frohna J.Internal medicine's Educational Innovations Project: improving health care and learning.Am J Med.2009;122:398404.
  2. Fitzgibbons JP,Bordley DR,Berkowitz LR,Miller BW,Henderson MC.Redesigning residency education in internal medicine: a position paper from the Association of Program Directors in Internal Medicine.Ann Intern Med.2006;144:920926.
  3. Weinberger SE,Smith LG,Collier VUfor the Education Committee of the American College of Physicians.Redesigning training for internal medicine.Ann Intern Med.2006;144:927932.
  4. Sequist TD,Singh S,Pereira AG,Rusinak D,Pearson SD.Use of an electronic medical record to profile the continuity clinic experiences of primary care residents.Acad Med.2005;80:390394.
  5. Nagler J,Harper MB,Bachur RG.An automated electronic case log: using electronic information systems to assess training in emergency medicine.Acad Emerg Med.2006;13:733739.
  6. Bachur RG,Nagler J.Use of an automated electronic case log to assess fellowship training: tracking the pediatric emergency medicine experience.Pediatr Emerg Care.2008;24:7582.
  7. Rohrbaugh R,Federman DG,Borysiuk L,Sernyak M.Utilizing VA information technology to develop psychiatric resident prescription profiles.Acad Psychiatry.2009;33:2730.
  8. Mattana J,Charitou M,Mills L, et al.Personal digital assistants (PDAs): a review of their application in graduate medical education.Am J Med Qual.2005;20:262267.
  9. Meyers FJ,Weinberger SE,Fitzgibbons JP, et al.Redesigning residency training in internal medicine: the consensus report of the Alliance for Academic Internal Medicine Education Redesign Task Force.Acad Med.2007;82:12111219.
  10. Birkmeyer JD,Stukel TA,Siewers AE,Goodney PP,Wennberg DE,Lucas FL.Surgeon volume and operative mortality in the United States.N Engl J Med.2003;349:21172127.
Issue
Journal of Hospital Medicine - 6(7)
Issue
Journal of Hospital Medicine - 6(7)
Page Number
395-400
Page Number
395-400
Publications
Publications
Article Type
Display Headline
Quantifying internal medicine resident clinical experience using resident‐selected primary diagnosis codes
Display Headline
Quantifying internal medicine resident clinical experience using resident‐selected primary diagnosis codes
Legacy Keywords
clinical experience, experiential learning, residency training
Legacy Keywords
clinical experience, experiential learning, residency training
Sections
Article Source

Copyright © 2011 Society of Hospital Medicine

Disallow All Ads
Correspondence Location
Department of Medicine, Long Island Jewish Medical Center, 270‐05 76th Avenue, New Hyde Park, NY 11040
Content Gating
No Gating (article Unlocked/Free)
Alternative CME
Article PDF Media
Media Files