Introducing the Hospitalist Morale Index

Article Type
Changed
Thu, 06/08/2017 - 12:16
Display Headline
Introducing the Hospitalist Morale Index: A new tool that may be relevant for improving provider retention

Explosive growth in hospital medicine has led to hospitalists having the option to change jobs easily. Annual turnover for all physicians is 6.8%, whereas that of hospitalists exceeds 14.8%.[1] Losing a single physician has significant financial and operational implications, with estimates of $20,000 to $120,000 in recruiting costs, and up to $500,000 in lost revenue that may take years to recoup due to the time required for new physician assimilation.[2, 3] In 2006, the Society of Hospital Medicine (SHM) appointed a career task force to develop retention recommendations, 1 of which includes monitoring hospitalists' job satisfaction.[4]

Studies examining physician satisfaction have demonstrated that high physician job satisfaction is associated with lower physician turnover.[5] However, surveys of hospitalists, including SHM's Hospital Medicine Physician Worklife Survey (HMPWS), have reported high job satisfaction among hospitalists,[6, 7, 8, 9, 10] suggesting that high job satisfaction may not be enough to overcome forces that pull hospitalists toward other opportunities.

Morale, a more complex construct related to an individual's contentment and happiness, might provide insight into reducing hospitalist turnover. Morale has been defined as the emotional or mental condition with respect to cheerfulness, confidence, or zeal and is especially relevant in the face of opposition or hardship.[11] Job satisfaction is 1 element that contributes to morale, but alone does not equate morale.[12] Morale, more than satisfaction, relates to how people see themselves within the group and may be closely tied to the concept of esprit de corps. To illustrate, workers may feel satisfied with the content of their job, but frustration with the organization may result in low morale.[13] Efforts focused on assessing provider morale may provide deeper understanding of hospitalists' professional needs and garner insight for retention strategies.

The construct of hospitalist morale and its underlying drivers has not been explored in the literature. Using literature within and outside of healthcare,[1, 12, 14, 15, 16, 17, 18, 19, 20, 21, 22] and our own prior work,[23] we sought to characterize elements that contribute to hospitalist morale and develop a metric to measure it. The HMPWS found that job satisfaction factors vary across hospitalist groups.[9] We suspected that the same would hold true for factors important to morale at the individual level. This study describes the development and validation of the Hospitalist Morale Index (HMI), and explores the relationship between morale and intent to leave due to unhappiness.

METHODS

2009 Pilot Survey

To establish content validity, after reviewing employee morale literature, and examining qualitative comments from our 2007 and 2008 morale surveys, our expert panel, consisting of practicing hospitalists, hospitalist leaders, and administrative staff, identified 46 potential drivers of hospitalist morale. In May 2009, all hospitalists, including physicians, nurse practitioners (NPs), and physician assistants (PAs) from a single hospitalist group received invitations to complete the pilot survey. We asked hospitalists to assess on 5‐point Likert scales the importance of (not at all to tremendously) and contentment with (extremely discontent to extremely content) each of the 46 items as it relates to their work morale. Also included were demographic questions and general morale questions (including rating participants' own morale), investment, long‐term career plans, and intent to leave due to unhappiness.

Data Collection

To maintain anonymity and limit social desirability bias, a database manager, working outside the Division of Hospital Medicine and otherwise not associated with the research team, used Survey Monkey to coordinate survey distribution and data collection. Each respondent had a unique identifier code that was unrelated to the respondent's name and email address. Personal identifiers were maintained in a secure database accessible only to the database manager.

Establishing Internal Structure Validity Evidence

Response frequency to each question was examined for irregularities in distribution. For continuous variables, descriptive statistics were examined for evidence of skewness, outliers, and non‐normality to ensure appropriate use of parametric statistical tests. Upon ranking importance ratings by mode, 15 of 46 items were judged to be of low importance by almost all participants and removed from further consideration.

Stata 13.1 (StataCorp, College Station, TX) was used for exploratory factor analysis (EFA) of the importance responses for all 31 remaining items by principal components factoring. Eigenvalues >1 were designated as a cutoff point for inclusion in varimax rotation. Factor loading of 0.50 was the threshold for inclusion in a factor.

The 31 items loaded across 10 factors; however, 3 factors included 1 item each. After reviewing the scree plot and considering their face value, these items/factors were omitted. Repeating the factor analysis resulted in a 28‐item, 7‐factor solution that accounted for 75% variance. All items were considered informative as demonstrated by low uniqueness scores (0.050.38). Using standard validation procedures, all 7 factors were found to have acceptable factor loadings (0.460.98) and face validity. Cronbach's quantified internal reliability of the 7 factors with scores ranging from 0.68 to 0.92. We named the resultant solution the Hospitalist Morale Index (HMI).

Establishing Response Process Validity Evidence

In developing the HMI, we asked respondents to rate the importance of and their contentment with each variable as related to their work morale. From pilot testing, which included discussions with respondents immediately after completing the survey, we learned that the 2‐part consideration of each variable resulted in thoughtful reflection about their morale. Further, by multiplying the contentment score for each item (scaled from 15) by the corresponding importance score (scaled 01), we quantified the relative contribution and contentment of each item for each hospitalist. Scaling importance scores from 0 to 1 insured that items that were not considered important to the respondent did not affect the respondent's personal morale score. Averaging resultant item scores that were greater than 0 resulted in a personal morale score for each hospitalist. Averaging item scores >0 that constituted each factor resulted in factor scores.

May 2011 Survey

The refined survey was distributed in May 2011 to a convenience sample of 5 hospitalist programs at separate hospitals (3 community hospitals, 2 academic hospitals) encompassing 108 hospitalists in 3 different states. Responses to the 2011 survey were used to complete confirmatory factor analyses (CFA) and establish further validity and reliability evidence.

Based on the 28‐item, 7‐factor solution developed from the pilot study, we developed the theoretical model of factors constituting hospitalist morale. We used the structural equation modeling command in Stata 13 to perform CFA. Factor loading of 0.50 was the threshold for inclusion of an item in a factor. To measure internal consistency, we considered Cronbach's score of 0.60 acceptable. Iterative models were reviewed to find the optimal solution for the data. Four items did not fit into any of the 5 resulting factors and were evaluated in terms of mean importance score and face value. Three items were considered important enough to warrant being stand‐alone items, whereas 1 was omitted. Two additional items had borderline factor loadings (0.48, 0.49) and were included in the model as stand‐alone items due to their overall relevance. The resultant solution was a 5‐factor model with 5 additional stand‐alone items (Table 1).

Confirmatory Factor Analysis Using Standardized Structured Equation Modeling of Importance Scores Retained in the Final Model Based on Survey Responses Gathered From Hospitalist Providers in 2011
 FactorCronbach's
ClinicalWorkloadLeadershipAppreciation and AcknowledgementMaterial Rewards
How much does the following item contribute to your morale?
Paperwork0.72    0.89
Relationship with patients0.69    0.90
Electronic medical system0.60    0.90
Intellectual stimulation0.59    0.90
Variety of cases0.58    0.90
Relationship with consultants0.51    0.89
No. of night shifts 0.74   0.89
Patient census 0.61   0.90
No. of shifts 0.52   0.90
Fairness of leadership  0.82  0.89
Effectiveness of leadership  0.82  0.89
Leadership's receptiveness to my thoughts and suggestions  0.78  0.89
Leadership as advocate for my needs  0.77  0.89
Approachability of leadership  0.77  0.89
Accessibility of leadership  0.69  0.89
Alignment of the group's goals with my goals  0.50  0.89
Recognition within the group   0.82 0.89
Feeling valued within the institution   0.73 0.89
Feeling valued within the group   0.73 0.89
Feedback   0.52 0.89
Pay    0.990.90
Benefits    0.560.89
Cronbach's 0.780.650.890.780.71 
How much does the following item contribute to your morale?Single item indicators 
Family time 0.90
Job security 0.90
Institutional climate 0.89
Opportunities for professional growth 0.90
Autonomy 0.89
Cronbach's  0.90

Establishing Convergent, Concurrent, and Discriminant Validity Evidence

To establish convergent, concurrent, and discriminant validity, linear and logistic regression models were examined for continuous and categorical data accordingly.

Self‐perceived overall work morale and perceived group morale, as assessed by 6‐point Likert questions with response options from terrible to excellent, were modeled as predictors for personal morale as calculated by the HMI.

Personal morale scores were modeled as predictors of professional growth, stress, investment in the group, and intent to leave due to unhappiness. While completing the HMI, hospitalists simultaneously completed a validated professional growth scale[24] and Cohen stress scale.[25] We hypothesized that those with higher morale would have more professional growth. Stress, although an important issue in the workplace, is a distinct construct from morale, and we did not expect a significant relationship between personal morale and stress. We used Pearson's r to assess the strength of association between the HMI and these scales. Participants' level of investment in their group was assessed on a 5‐point Likert scale. To simplify presentation, highly invested represents those claiming to be very or tremendously invested in the success of their current hospitalist group. Intent to leave due to unhappiness was assessed on a 5‐point Likert scale, I have had serious thoughts about leaving my current hospitalist group because I am unhappy, with responses from strongly disagree (1) to strongly agree (5). To simplify presentation, responses higher than 2 are considered to be consistent with intending to leave due to unhappiness.

Our institutional review board approved the study.

RESULTS

Respondents

In May 2009, 30 of the 33 (91%) invited hospitalists completed the original pilot morale survey; 19 (63%) were women. Eleven hospitalists (37%) had been part of the group 1 year or less, whereas 4 (13%) had been with the group for more than 5 years.

In May 2011, 93 of the 108 (86%) hospitalists from 5 hospitals completed the demographic and global parts of the survey. Fifty (53%) were from community hospitals; 47 (51%) were women. Thirty‐seven (40%) physicians and 6 (60%) NPs/PAs were from academic hospitals. Thirty‐nine hospitalists (42%) had been with their current group 1 year or less. Ten hospitalists (11%) had been with their current group over 5 years. Sixty‐three respondents (68%) considered themselves career hospitalists, whereas 5 (5%) did not; the rest were undecided.

Internal Structure Validity Evidence

The final CFA from the 2011 survey resulted in a 5‐factor plus 5stand‐alone‐items HMI. The solution with item‐level and factor‐level Cronbach's scores (range, 0.890.90 and range, 0.650.89, respectively) are shown in Table 1.

Personal Morale Scores and Factor Scores

Personal morale scores were normally distributed (mean = 2.79; standard deviation [SD] = 0.58), ranging from 1.23 to 4.22, with a theoretical low of 0 and high of 5 (Figure 1). Mean personal morale scores across hospitalist groups ranged from 2.70 to 2.99 (P > 0.05). Personal morale scores, factor sores and item scores for NPs and PAs did not significantly differ from those of physicians (P > 0.05 for all analyses). Personal morale scores were lower for those in their first 3 years with their current group, compared to those with greater institutional longevity. For every categorical increase in a participant's response to seeing oneself as a career hospitalist, the personal morale score rose 0.23 points (P < 0.001).

Figure 1
2011 personal moral scores for all hospitalists.

Factor scores for material reward and mean item scores for professional growth were significantly different across the 5 hospitalist groups (P = 0.03 and P < 0.001, respectively). Community hospitalists had significantly higher factor scores, despite having similar importance scores, for material rewards than academic hospitalists (diff. = 0.44, P = 0.02). Academic hospitalists had significantly higher scores for professional growth (diff. = 0.94, P < 0.001) (Table 2). Professional growth had the highest importance score for academic hospitalists (mean = 0.87, SD = 0.18) and the lowest importance score for community hospitalists (mean = 0.65, SD = 0.24, P < 0.001).

Personal Morale Scores, Factor Scores,* and Five Item Scores* by Hospitalist Groups
 Personal Morale ScoreFactor 1Factor 2Factor 3Factor 4Factor 5Item 1Item 2Item 3Item 4Item 5
ClinicalWorkloadLeadershipAppreciation and AcknowledgementMaterial RewardsFamily TimeInstitutional ClimateJob SecurityAutonomyProfessional Growth
  • NOTE: Abbreviations: SD, standard deviation.*Factor scores and item scores represent the combined product of importance and contentment.

All participantsMean2.792.542.783.182.582.483.052.672.923.002.76
SD0.580.630.700.950.860.851.150.971.111.101.21
Academic AMean2.772.432.923.102.542.283.162.703.063.203.08
SD0.570.620.640.920.840.771.190.951.081.121.24
Academic BMean2.992.582.993.882.692.002.582.131.653.294.33
SD0.360.700.800.290.800.350.920.880.781.010.82
Community AMean2.862.612.513.232.733.032.882.842.953.232.66
SD0.750.790.681.211.111.141.371.170.981.241.15
Community BMean2.862.742.973.372.672.443.282.352.702.502.25
SD0.670.550.861.040.940.871.001.151.400.721.26
Community CMean2.702.562.642.992.472.533.032.793.072.682.15
SD0.490.530.670.850.730.641.080.761.051.070.71
Academic combinedMean2.802.452.933.222.562.243.072.622.883.213.28
SD0.540.630.660.890.820.721.160.951.141.101.26
Community combinedMean2.792.612.663.142.602.683.032.722.952.822.34
SD0.620.620.721.010.900.901.150.991.091.091.00
P value>0.05>0.05>0.05>0.05>0.050.02>0.05>0.05>0.05>0.05<0.001

Convergent, Concurrent, and Discriminant Validity Evidence

For every categorical increase on the question assessing overall morale, the personal morale score was 0.23 points higher (P < 0.001). For every categorical increase in a participant's perception of the group's morale, the personal morale score was 0.29 points higher (P < 0.001).

For every 1‐point increase in personal morale score, the odds of being highly invested in the group increased by 5 times (odds ratio [OR]: 5.23, 95% confidence interval [CI]: 1.91‐14.35, P = 0.001). The mean personal morale score for highly invested hospitalists was 2.92, whereas that of those less invested was 2.43 (diff. = 0.49, P < 0.001) (Table 3). Highly invested hospitalists had significantly higher importance factor scores for leadership (diff. = 0.08, P = 0.03) as well as appreciation and acknowledgement (diff. = 0.08, P = 0.02).

Personal Morale Scores, Factor Scores,* and Five Item Scores* by Investment and Intent to Leave
 Personal Morale ScoreFactor 1Factor 2Factor 3Factor 4Factor 5Item 1Item 2Item 3Item 4Item 5
ClinicalWorkloadLeadershipAppreciation and AcknowledgementMaterial RewardsFamily TimeInstitutional ClimateJob SecurityAutonomyProfessional Growth
  • NOTE: Abbreviations: SD, standard deviation. *Factor scores and item scores represent the combined product of importance and contentment.

Highly invested in success of current hospitalist group
Mean2.922.612.893.382.782.453.212.782.863.102.95
SD0.550.590.680.920.880.771.111.001.091.061.25
Less invested in success of current hospitalist group
Mean2.432.342.482.602.022.572.602.383.082.692.24
SD0.520.690.690.810.491.041.170.831.181.190.94
P value<0.001>0.050.020.001<0.001>0.050.03>0.05>0.05>0.050.02
Not intending to leave because unhappy
Mean2.972.672.893.482.772.523.242.853.053.063.01
SD0.510.540.610.910.890.781.030.991.101.071.25
Intending to leave current group because unhappy
Mean2.452.302.592.592.212.402.682.332.672.882.28
SD0.560.720.820.740.680.971.290.831.111.170.97
P value<0.0010.01>0.05<0.0010.003>0.050.030.01>0.05>0.050.01

Every 1‐point increase in personal morale was associated with a rise of 2.27 on the professional growth scale (P = 0.01). The correlation between these 2 scales was 0.26 (P = 0.01). Every 1‐point increase in personal morale was associated with a 2.21 point decrease on the Cohen stress scale (P > 0.05). The correlation between these 2 scales was 0.21 (P > 0.05).

Morale and Intent to Leave Due to Unhappiness

Sixteen (37%) academic and 18 (36%) community hospitalists reported having thoughts of leaving their current hospitalist program due to unhappiness. The mean personal morale score for hospitalists with no intent to leave their current group was 2.97, whereas that of those with intent to leave was 2.45 (diff. = 0.53, P < 0.001). Each 1‐point increase in the personal morale score was associated with an 85% decrease (OR: 0.15, 95% CI: 0.05‐0.41, P < 0.001) in the odds of leaving because of unhappiness. Holding self‐perception of being a career hospitalist constant, each 1‐point increase in the personal morale score was associated with an 83% decrease (OR: 0.17, 95% CI: 0.05‐0.51, P = 0.002) in the odds of leaving because of unhappiness. Hospitalists who reported intent to leave had significantly lower factor scores for all factors and items except workload, material reward, and autonomy than those who did not report intent to leave (Table 3). Within the academic groups, those who reported intent to leave had significantly lower scores for professional growth (diff. = 1.08, P = 0.01). For community groups, those who reported intent to leave had significantly lower scores for clinical work (diff. = 0.54, P = 0.003), workload (diff. = 0.50, P = 0.02), leadership (diff. = 1.19, P < 0.001), feeling appreciated and acknowledged (diff. = 0.68, P = 0.01), job security (diff. = 0.70, P = 0.03), and institutional climate (diff. = 0.67, P = 0.02) than those who did not report intent to leave.

DISCUSSION

The HMI is a validated tool that objectively measures and quantifies hospitalist morale. The HMI's capacity to comprehensively assess morale comes from its breadth and depth in uncovering work‐related areas that may be sources of contentment or displeasure. Furthermore, the fact that HMI scores varied among groups of individuals, including those who are thinking about leaving their hospitalist group because they are unhappy and those who are highly invested in their hospitalist group, speaks to its ability to highlight and account for what is most important to hospitalist providers.

Low employee morale has been associated with decreased productivity, increased absenteeism, increased turnover, and decreased patient satisfaction.[2, 26, 27, 28] A few frustrated workers can breed group discontentment and lower the entire group's morale.[28] In addition to its financial impact, departures due to low morale can be sudden and devastating, leading to loss of team cohesiveness, increased work burden on the remaining workforce, burnout, and cascades of more turnover.[2] In contrast, when morale is high, workers more commonly go the extra mile, are more committed to the organization's mission, and are more supportive of their coworkers.[28]

While we asked the informants about plans to leave their job, there are many factors that drive an individual's intent and ultimate decision to make changes in his or her employment. Some factors are outside the control of the employer or practice leaders, such as change in an individual's family life or desire and opportunity to pursue fellowship training. Others variables, however, are more directly tied to the job or practice environment. In a specialty where providers are relatively mobile and turnover is high, it is important for hospitalist practices to cultivate a climate in which the sacrifices associated with leaving outweigh the promised benefits.[29]

Results from the HMPWS suggested the need to address climate and fairness issues in hospitalist programs to improve satisfaction and retention.[9] Two large healthcare systems achieved success by investing in multipronged physician retention strategies including recruiting advisors, sign‐on bonuses, extensive onboarding, family support, and the promotion of ongoing effective communication.[3, 30]

Our findings suggest that morale for hospitalists is a complex amalgam of contentment and importance, and that there may not be a one size fits all solution to improving morale for all. While we did not find a difference in personal morale scores across individual hospitalist groups, or even between academic and community groups, each group had a unique profile with variability in the dynamics between importance and contentment of different factors. If practice group leaders review HMI data for their providers and use the information to facilitate meaningful dialogue with them about the factors influencing their morale, such leaders will have great insight into allocating resources for the best return on investment.

While we believe that the HMI is providing unique perspective compared to other commonly used metrics, it may be best to employ HMI data as complementary measures alongside that of some of the benchmarked scales that explore job satisfaction, job fit, and burnout among hospitalists.[6, 9, 10, 31, 32, 33, 34, 35] Aggregate HMI data at the group level may allow for the identification of factors that are highly important to morale but scored low in contentment. Such factors deserve priority and attention such that the subgroups within a practice can collaborate to come to consensus on strategies for amelioration. Because the HMI generates a score and profile for each provider, we can imagine effective leaders using the HMI with individuals as part of an annual review to facilitate discussion about maximizing contentment at work. Being fully transparent and sharing an honest nonanonymous version of the HMI with a superior would require a special relationship founded on trust and mutual respect.

Several limitations of this study should be considered. First, the initial item reduction and EFA were based on a single‐site survey, and our overall sample size was relatively small. We plan on expanding our sample size in the future for further validation of our exploratory findings. Second, the data were collected at 2 specific times several years ago. In continuing to analyze the data from subsequent years, validity and reliability results remain stable, thereby minimizing the likelihood of significant historical bias. Third, there may have been some recall bias, in that respondents may have overlooked the good and perseverated over variables that disappointed them. Fourth, although intention to leave does not necessarily equate actual employee turnover, intention has been found to be a strong predictor of quitting a job.[36, 37] Finally, while we had high response rates, response bias may have existed wherein those with lower morale may have elected not to complete the survey or became apathetic in their responses.

The HMI is a validated instrument that evaluates hospitalist morale by incorporating each provider's characterization of the importance of and contentment with 27 variables. By accounting for the multidimensional and dynamic nature of morale, the HMI may help program leaders tailor retention and engagement strategies specific to their own group. Future studies may explore trends in contributors to morale and examine whether interventions to augment low morale can result in improved morale and hospitalist retention.

Acknowledgements

The authors are indebted to the hospitalists who were willing to share their perspectives about their work, and grateful to Ms. Lisa Roberts, Ms. Barbara Brigade, and Ms. Regina Landis for insuring confidentiality in managing the survey database.

Disclosures: Dr. Chandra had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Dr. Wright is a Miller‐Coulson Family Scholar through the Johns Hopkins Center for Innovative Medicine. Ethical approval has been granted for studies involving human subjects by a Johns Hopkins University School of Medicine institutional review board. The authors report no conflicts of interest.

Files
References
  1. 2014 State of Hospital Medicine Report. Philadelphia, PA: Society of Hospital Medicine; 2014.
  2. Misra‐Hebert AD, Kay R, Stoller JK. A review of physician turnover: rates, causes, and consequences. Am J Med Qual. 2004;19(2):5666.
  3. Scott K. Physician retention plans help reduce costs and optimize revenues. Healthc Financ Manage. 1998;52(1):7577.
  4. SHM Career Satisfaction Task Force. A Challenge for a New Specialty: A White Paper on Hospitalist Career Satisfaction.; 2006. Available at: www.hospitalmedicine.org. Accessed February 28, 2009.
  5. Williams ES, Skinner AC. Outcomes of physician job satisfaction: a narrative review, implications, and directions for future research. Health Care Manage Rev. 2003;28(2):119139.
  6. Hoff TH, Whitcomb WF, Williams K, Nelson JR, Cheesman RA. Characteristics and work experiences of hospitalists in the United States. Arch Intern Med. 2001;161(6):851858.
  7. Hoff TJ. Doing the same and earning less: male and female physicians in a new medical specialty. Inquiry. 2004;41(3):301315.
  8. Clark‐Cox K. Physician satisfaction and communication. National findings and best practices. Available at: http://www.pressganey.com/files/clark_cox_acpe_apr06.pdf. Accessed October 10, 2010.
  9. Hinami K, Whelan CT, Wolosin RJ, Miller JA, Wetterneck TB. Worklife and satisfaction of hospitalists: toward flourishing careers. J Gen Intern Med. 2012;27(1):2836.
  10. Hinami K, Whelan CT, Miller JA, Wolosin RJ, Wetterneck TB; Society of Hospital Medicine Career Satisfaction Task Force. Job characteristics, satisfaction, and burnout across hospitalist practice models. J Hosp Med. 2012;7(5):402410.
  11. Morale | Define Morale at Dictionary.com. Morale | Define Morale at Dictionary.com. Morale | Define Morale at Dictionary.com. Available at: http://dictionary.reference.com/browse/morale. Accessed June 5, 2014.
  12. Guba EG. Morale and satisfaction: a study in past‐future time perspective. Adm Sci Q. 1958:195209.
  13. Kanter RM. Men and Women of the Corporation. 2nd ed. New York, NY: Basic Books; 1993.
  14. Charters WW. The relation of morale to turnover among teachers. Am Educ Res J. 1965:163173.
  15. Zeitz G. Structural and individual determinants of organization morale and satisfaction. Soc Forces. 1982;61:1088.
  16. Johnsrud LK, Heck RH, Rosser VJ. Morale matters: midlevel administrators and their intent to leave. J Higher Educ. 2000:3459.
  17. Worthy JC. Factors influencing employee morale. Harv Bus Rev. 1950;28(1):6173.
  18. Coughlan RJ. Dimensions of teacher morale. Am Educ Res J. 1970;7(2):221.
  19. Baehr ME, Renck R. The definition and measurement of employee morale. Adm Sci Q. 1958:157184.
  20. Konrad TR, Williams ES, Linzer M, et al. Measuring physician job satisfaction in a changing workplace and a challenging environment. SGIM Career Satisfaction Study Group. Society of General Internal Medicine. Med Care. 1999;37(11):11741182.
  21. Zeitz G. Structural and individual determinants of organization morale and satisfaction. Soc Forces. 1983;61(4):10881108.
  22. Durant H. Morale and its measurement. Am J Sociol. 1941;47(3):406414.
  23. Chandra S, Wright SM, Kargul G, Howell EE. Following morale over time within an academic hospitalist division. J Clin Outcomes Manag. 2011;18(1):2126.
  24. Wright SM, Levine RB, Beasley B, et al. Personal growth and its correlates during residency training. Med Educ. 2006;40(8):737745.
  25. Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav. 1983:385396.
  26. Johnsrud LK, Heck RH, Rosser VJ. Morale matters: midlevel administrators and their intent to leave. J Higher Educ. 2000;71(1):3459.
  27. Johnsrud LK, Rosser VJ. Faculty members' morale and their intention to leave: a multilevel explanation. J Higher Educ. 2002;73(4):518542.
  28. Bowles D, Cooper C. Employee Morale. New York, NY: Palgrave Macmillan; 2009.
  29. Maxfield D, Grenny J, McMillan R, Patterson K, Switzler A. Silence Kills. Silence Kills: The Seven Crucial Conversations® for Healthcare. VitalSmarts™ in association with the American Association of Critical Care Nurses, USA. 2005. Accessed October 10, 2014.
  30. Cohn KH, Bethancourt B, Simington M. The lifelong iterative process of physician retention. J Healthc Manag. 2009;54(4):220226.
  31. Chabot JM. Physicians' burnout. Rev Prat. 2004;54(7):753754.
  32. Virtanen P, Oksanen T, Kivimaki M, Virtanen M, Pentti J, Vahtera J. Work stress and health in primary health care physicians and hospital physicians. Occup Environ Med. 2008;65(5):364366.
  33. Williams ES, Konrad TR, Scheckler WE, et al. Understanding physicians' intentions to withdraw from practice: the role of job satisfaction, job stress, mental and physical health. 2001. Health Care Manage Rev. 2010;35(2):105115.
  34. Dyrbye LN, Varkey P, Boone SL, Satele DV, Sloan JA, Shanafelt TD. Physician satisfaction and burnout at different career stages. Mayo Clin Proc. 2013;88(12):13581367.
  35. Wetterneck TB, Williams MA. Burnout and Hospitalists: Etiology and Prevention. In: What Exactly Does A Hospitalist Do? Best of the Best Hospital Medicine 2005: Strategies for Success. Society of Hospital Medicine; 2005:5.
  36. Blau G, Boal K. Using job involvement and organizational commitment interactively to predict turnover. J Manage. 1989;15(1):115127.
  37. Hayes LJ, O'Brien‐Pallas L, Duffield C, et al. Nurse turnover: a literature review. Int J Nurs Stud. 2006;43(2):237263.
Article PDF
Issue
Journal of Hospital Medicine - 11(6)
Page Number
425-431
Sections
Files
Files
Article PDF
Article PDF

Explosive growth in hospital medicine has led to hospitalists having the option to change jobs easily. Annual turnover for all physicians is 6.8%, whereas that of hospitalists exceeds 14.8%.[1] Losing a single physician has significant financial and operational implications, with estimates of $20,000 to $120,000 in recruiting costs, and up to $500,000 in lost revenue that may take years to recoup due to the time required for new physician assimilation.[2, 3] In 2006, the Society of Hospital Medicine (SHM) appointed a career task force to develop retention recommendations, 1 of which includes monitoring hospitalists' job satisfaction.[4]

Studies examining physician satisfaction have demonstrated that high physician job satisfaction is associated with lower physician turnover.[5] However, surveys of hospitalists, including SHM's Hospital Medicine Physician Worklife Survey (HMPWS), have reported high job satisfaction among hospitalists,[6, 7, 8, 9, 10] suggesting that high job satisfaction may not be enough to overcome forces that pull hospitalists toward other opportunities.

Morale, a more complex construct related to an individual's contentment and happiness, might provide insight into reducing hospitalist turnover. Morale has been defined as the emotional or mental condition with respect to cheerfulness, confidence, or zeal and is especially relevant in the face of opposition or hardship.[11] Job satisfaction is 1 element that contributes to morale, but alone does not equate morale.[12] Morale, more than satisfaction, relates to how people see themselves within the group and may be closely tied to the concept of esprit de corps. To illustrate, workers may feel satisfied with the content of their job, but frustration with the organization may result in low morale.[13] Efforts focused on assessing provider morale may provide deeper understanding of hospitalists' professional needs and garner insight for retention strategies.

The construct of hospitalist morale and its underlying drivers has not been explored in the literature. Using literature within and outside of healthcare,[1, 12, 14, 15, 16, 17, 18, 19, 20, 21, 22] and our own prior work,[23] we sought to characterize elements that contribute to hospitalist morale and develop a metric to measure it. The HMPWS found that job satisfaction factors vary across hospitalist groups.[9] We suspected that the same would hold true for factors important to morale at the individual level. This study describes the development and validation of the Hospitalist Morale Index (HMI), and explores the relationship between morale and intent to leave due to unhappiness.

METHODS

2009 Pilot Survey

To establish content validity, after reviewing employee morale literature, and examining qualitative comments from our 2007 and 2008 morale surveys, our expert panel, consisting of practicing hospitalists, hospitalist leaders, and administrative staff, identified 46 potential drivers of hospitalist morale. In May 2009, all hospitalists, including physicians, nurse practitioners (NPs), and physician assistants (PAs) from a single hospitalist group received invitations to complete the pilot survey. We asked hospitalists to assess on 5‐point Likert scales the importance of (not at all to tremendously) and contentment with (extremely discontent to extremely content) each of the 46 items as it relates to their work morale. Also included were demographic questions and general morale questions (including rating participants' own morale), investment, long‐term career plans, and intent to leave due to unhappiness.

Data Collection

To maintain anonymity and limit social desirability bias, a database manager, working outside the Division of Hospital Medicine and otherwise not associated with the research team, used Survey Monkey to coordinate survey distribution and data collection. Each respondent had a unique identifier code that was unrelated to the respondent's name and email address. Personal identifiers were maintained in a secure database accessible only to the database manager.

Establishing Internal Structure Validity Evidence

Response frequency to each question was examined for irregularities in distribution. For continuous variables, descriptive statistics were examined for evidence of skewness, outliers, and non‐normality to ensure appropriate use of parametric statistical tests. Upon ranking importance ratings by mode, 15 of 46 items were judged to be of low importance by almost all participants and removed from further consideration.

Stata 13.1 (StataCorp, College Station, TX) was used for exploratory factor analysis (EFA) of the importance responses for all 31 remaining items by principal components factoring. Eigenvalues >1 were designated as a cutoff point for inclusion in varimax rotation. Factor loading of 0.50 was the threshold for inclusion in a factor.

The 31 items loaded across 10 factors; however, 3 factors included 1 item each. After reviewing the scree plot and considering their face value, these items/factors were omitted. Repeating the factor analysis resulted in a 28‐item, 7‐factor solution that accounted for 75% variance. All items were considered informative as demonstrated by low uniqueness scores (0.050.38). Using standard validation procedures, all 7 factors were found to have acceptable factor loadings (0.460.98) and face validity. Cronbach's quantified internal reliability of the 7 factors with scores ranging from 0.68 to 0.92. We named the resultant solution the Hospitalist Morale Index (HMI).

Establishing Response Process Validity Evidence

In developing the HMI, we asked respondents to rate the importance of and their contentment with each variable as related to their work morale. From pilot testing, which included discussions with respondents immediately after completing the survey, we learned that the 2‐part consideration of each variable resulted in thoughtful reflection about their morale. Further, by multiplying the contentment score for each item (scaled from 15) by the corresponding importance score (scaled 01), we quantified the relative contribution and contentment of each item for each hospitalist. Scaling importance scores from 0 to 1 insured that items that were not considered important to the respondent did not affect the respondent's personal morale score. Averaging resultant item scores that were greater than 0 resulted in a personal morale score for each hospitalist. Averaging item scores >0 that constituted each factor resulted in factor scores.

May 2011 Survey

The refined survey was distributed in May 2011 to a convenience sample of 5 hospitalist programs at separate hospitals (3 community hospitals, 2 academic hospitals) encompassing 108 hospitalists in 3 different states. Responses to the 2011 survey were used to complete confirmatory factor analyses (CFA) and establish further validity and reliability evidence.

Based on the 28‐item, 7‐factor solution developed from the pilot study, we developed the theoretical model of factors constituting hospitalist morale. We used the structural equation modeling command in Stata 13 to perform CFA. Factor loading of 0.50 was the threshold for inclusion of an item in a factor. To measure internal consistency, we considered Cronbach's score of 0.60 acceptable. Iterative models were reviewed to find the optimal solution for the data. Four items did not fit into any of the 5 resulting factors and were evaluated in terms of mean importance score and face value. Three items were considered important enough to warrant being stand‐alone items, whereas 1 was omitted. Two additional items had borderline factor loadings (0.48, 0.49) and were included in the model as stand‐alone items due to their overall relevance. The resultant solution was a 5‐factor model with 5 additional stand‐alone items (Table 1).

Confirmatory Factor Analysis Using Standardized Structured Equation Modeling of Importance Scores Retained in the Final Model Based on Survey Responses Gathered From Hospitalist Providers in 2011
 FactorCronbach's
ClinicalWorkloadLeadershipAppreciation and AcknowledgementMaterial Rewards
How much does the following item contribute to your morale?
Paperwork0.72    0.89
Relationship with patients0.69    0.90
Electronic medical system0.60    0.90
Intellectual stimulation0.59    0.90
Variety of cases0.58    0.90
Relationship with consultants0.51    0.89
No. of night shifts 0.74   0.89
Patient census 0.61   0.90
No. of shifts 0.52   0.90
Fairness of leadership  0.82  0.89
Effectiveness of leadership  0.82  0.89
Leadership's receptiveness to my thoughts and suggestions  0.78  0.89
Leadership as advocate for my needs  0.77  0.89
Approachability of leadership  0.77  0.89
Accessibility of leadership  0.69  0.89
Alignment of the group's goals with my goals  0.50  0.89
Recognition within the group   0.82 0.89
Feeling valued within the institution   0.73 0.89
Feeling valued within the group   0.73 0.89
Feedback   0.52 0.89
Pay    0.990.90
Benefits    0.560.89
Cronbach's 0.780.650.890.780.71 
How much does the following item contribute to your morale?Single item indicators 
Family time 0.90
Job security 0.90
Institutional climate 0.89
Opportunities for professional growth 0.90
Autonomy 0.89
Cronbach's  0.90

Establishing Convergent, Concurrent, and Discriminant Validity Evidence

To establish convergent, concurrent, and discriminant validity, linear and logistic regression models were examined for continuous and categorical data accordingly.

Self‐perceived overall work morale and perceived group morale, as assessed by 6‐point Likert questions with response options from terrible to excellent, were modeled as predictors for personal morale as calculated by the HMI.

Personal morale scores were modeled as predictors of professional growth, stress, investment in the group, and intent to leave due to unhappiness. While completing the HMI, hospitalists simultaneously completed a validated professional growth scale[24] and Cohen stress scale.[25] We hypothesized that those with higher morale would have more professional growth. Stress, although an important issue in the workplace, is a distinct construct from morale, and we did not expect a significant relationship between personal morale and stress. We used Pearson's r to assess the strength of association between the HMI and these scales. Participants' level of investment in their group was assessed on a 5‐point Likert scale. To simplify presentation, highly invested represents those claiming to be very or tremendously invested in the success of their current hospitalist group. Intent to leave due to unhappiness was assessed on a 5‐point Likert scale, I have had serious thoughts about leaving my current hospitalist group because I am unhappy, with responses from strongly disagree (1) to strongly agree (5). To simplify presentation, responses higher than 2 are considered to be consistent with intending to leave due to unhappiness.

Our institutional review board approved the study.

RESULTS

Respondents

In May 2009, 30 of the 33 (91%) invited hospitalists completed the original pilot morale survey; 19 (63%) were women. Eleven hospitalists (37%) had been part of the group 1 year or less, whereas 4 (13%) had been with the group for more than 5 years.

In May 2011, 93 of the 108 (86%) hospitalists from 5 hospitals completed the demographic and global parts of the survey. Fifty (53%) were from community hospitals; 47 (51%) were women. Thirty‐seven (40%) physicians and 6 (60%) NPs/PAs were from academic hospitals. Thirty‐nine hospitalists (42%) had been with their current group 1 year or less. Ten hospitalists (11%) had been with their current group over 5 years. Sixty‐three respondents (68%) considered themselves career hospitalists, whereas 5 (5%) did not; the rest were undecided.

Internal Structure Validity Evidence

The final CFA from the 2011 survey resulted in a 5‐factor plus 5stand‐alone‐items HMI. The solution with item‐level and factor‐level Cronbach's scores (range, 0.890.90 and range, 0.650.89, respectively) are shown in Table 1.

Personal Morale Scores and Factor Scores

Personal morale scores were normally distributed (mean = 2.79; standard deviation [SD] = 0.58), ranging from 1.23 to 4.22, with a theoretical low of 0 and high of 5 (Figure 1). Mean personal morale scores across hospitalist groups ranged from 2.70 to 2.99 (P > 0.05). Personal morale scores, factor sores and item scores for NPs and PAs did not significantly differ from those of physicians (P > 0.05 for all analyses). Personal morale scores were lower for those in their first 3 years with their current group, compared to those with greater institutional longevity. For every categorical increase in a participant's response to seeing oneself as a career hospitalist, the personal morale score rose 0.23 points (P < 0.001).

Figure 1
2011 personal moral scores for all hospitalists.

Factor scores for material reward and mean item scores for professional growth were significantly different across the 5 hospitalist groups (P = 0.03 and P < 0.001, respectively). Community hospitalists had significantly higher factor scores, despite having similar importance scores, for material rewards than academic hospitalists (diff. = 0.44, P = 0.02). Academic hospitalists had significantly higher scores for professional growth (diff. = 0.94, P < 0.001) (Table 2). Professional growth had the highest importance score for academic hospitalists (mean = 0.87, SD = 0.18) and the lowest importance score for community hospitalists (mean = 0.65, SD = 0.24, P < 0.001).

Personal Morale Scores, Factor Scores,* and Five Item Scores* by Hospitalist Groups
 Personal Morale ScoreFactor 1Factor 2Factor 3Factor 4Factor 5Item 1Item 2Item 3Item 4Item 5
ClinicalWorkloadLeadershipAppreciation and AcknowledgementMaterial RewardsFamily TimeInstitutional ClimateJob SecurityAutonomyProfessional Growth
  • NOTE: Abbreviations: SD, standard deviation.*Factor scores and item scores represent the combined product of importance and contentment.

All participantsMean2.792.542.783.182.582.483.052.672.923.002.76
SD0.580.630.700.950.860.851.150.971.111.101.21
Academic AMean2.772.432.923.102.542.283.162.703.063.203.08
SD0.570.620.640.920.840.771.190.951.081.121.24
Academic BMean2.992.582.993.882.692.002.582.131.653.294.33
SD0.360.700.800.290.800.350.920.880.781.010.82
Community AMean2.862.612.513.232.733.032.882.842.953.232.66
SD0.750.790.681.211.111.141.371.170.981.241.15
Community BMean2.862.742.973.372.672.443.282.352.702.502.25
SD0.670.550.861.040.940.871.001.151.400.721.26
Community CMean2.702.562.642.992.472.533.032.793.072.682.15
SD0.490.530.670.850.730.641.080.761.051.070.71
Academic combinedMean2.802.452.933.222.562.243.072.622.883.213.28
SD0.540.630.660.890.820.721.160.951.141.101.26
Community combinedMean2.792.612.663.142.602.683.032.722.952.822.34
SD0.620.620.721.010.900.901.150.991.091.091.00
P value>0.05>0.05>0.05>0.05>0.050.02>0.05>0.05>0.05>0.05<0.001

Convergent, Concurrent, and Discriminant Validity Evidence

For every categorical increase on the question assessing overall morale, the personal morale score was 0.23 points higher (P < 0.001). For every categorical increase in a participant's perception of the group's morale, the personal morale score was 0.29 points higher (P < 0.001).

For every 1‐point increase in personal morale score, the odds of being highly invested in the group increased by 5 times (odds ratio [OR]: 5.23, 95% confidence interval [CI]: 1.91‐14.35, P = 0.001). The mean personal morale score for highly invested hospitalists was 2.92, whereas that of those less invested was 2.43 (diff. = 0.49, P < 0.001) (Table 3). Highly invested hospitalists had significantly higher importance factor scores for leadership (diff. = 0.08, P = 0.03) as well as appreciation and acknowledgement (diff. = 0.08, P = 0.02).

Personal Morale Scores, Factor Scores,* and Five Item Scores* by Investment and Intent to Leave
 Personal Morale ScoreFactor 1Factor 2Factor 3Factor 4Factor 5Item 1Item 2Item 3Item 4Item 5
ClinicalWorkloadLeadershipAppreciation and AcknowledgementMaterial RewardsFamily TimeInstitutional ClimateJob SecurityAutonomyProfessional Growth
  • NOTE: Abbreviations: SD, standard deviation. *Factor scores and item scores represent the combined product of importance and contentment.

Highly invested in success of current hospitalist group
Mean2.922.612.893.382.782.453.212.782.863.102.95
SD0.550.590.680.920.880.771.111.001.091.061.25
Less invested in success of current hospitalist group
Mean2.432.342.482.602.022.572.602.383.082.692.24
SD0.520.690.690.810.491.041.170.831.181.190.94
P value<0.001>0.050.020.001<0.001>0.050.03>0.05>0.05>0.050.02
Not intending to leave because unhappy
Mean2.972.672.893.482.772.523.242.853.053.063.01
SD0.510.540.610.910.890.781.030.991.101.071.25
Intending to leave current group because unhappy
Mean2.452.302.592.592.212.402.682.332.672.882.28
SD0.560.720.820.740.680.971.290.831.111.170.97
P value<0.0010.01>0.05<0.0010.003>0.050.030.01>0.05>0.050.01

Every 1‐point increase in personal morale was associated with a rise of 2.27 on the professional growth scale (P = 0.01). The correlation between these 2 scales was 0.26 (P = 0.01). Every 1‐point increase in personal morale was associated with a 2.21 point decrease on the Cohen stress scale (P > 0.05). The correlation between these 2 scales was 0.21 (P > 0.05).

Morale and Intent to Leave Due to Unhappiness

Sixteen (37%) academic and 18 (36%) community hospitalists reported having thoughts of leaving their current hospitalist program due to unhappiness. The mean personal morale score for hospitalists with no intent to leave their current group was 2.97, whereas that of those with intent to leave was 2.45 (diff. = 0.53, P < 0.001). Each 1‐point increase in the personal morale score was associated with an 85% decrease (OR: 0.15, 95% CI: 0.05‐0.41, P < 0.001) in the odds of leaving because of unhappiness. Holding self‐perception of being a career hospitalist constant, each 1‐point increase in the personal morale score was associated with an 83% decrease (OR: 0.17, 95% CI: 0.05‐0.51, P = 0.002) in the odds of leaving because of unhappiness. Hospitalists who reported intent to leave had significantly lower factor scores for all factors and items except workload, material reward, and autonomy than those who did not report intent to leave (Table 3). Within the academic groups, those who reported intent to leave had significantly lower scores for professional growth (diff. = 1.08, P = 0.01). For community groups, those who reported intent to leave had significantly lower scores for clinical work (diff. = 0.54, P = 0.003), workload (diff. = 0.50, P = 0.02), leadership (diff. = 1.19, P < 0.001), feeling appreciated and acknowledged (diff. = 0.68, P = 0.01), job security (diff. = 0.70, P = 0.03), and institutional climate (diff. = 0.67, P = 0.02) than those who did not report intent to leave.

DISCUSSION

The HMI is a validated tool that objectively measures and quantifies hospitalist morale. The HMI's capacity to comprehensively assess morale comes from its breadth and depth in uncovering work‐related areas that may be sources of contentment or displeasure. Furthermore, the fact that HMI scores varied among groups of individuals, including those who are thinking about leaving their hospitalist group because they are unhappy and those who are highly invested in their hospitalist group, speaks to its ability to highlight and account for what is most important to hospitalist providers.

Low employee morale has been associated with decreased productivity, increased absenteeism, increased turnover, and decreased patient satisfaction.[2, 26, 27, 28] A few frustrated workers can breed group discontentment and lower the entire group's morale.[28] In addition to its financial impact, departures due to low morale can be sudden and devastating, leading to loss of team cohesiveness, increased work burden on the remaining workforce, burnout, and cascades of more turnover.[2] In contrast, when morale is high, workers more commonly go the extra mile, are more committed to the organization's mission, and are more supportive of their coworkers.[28]

While we asked the informants about plans to leave their job, there are many factors that drive an individual's intent and ultimate decision to make changes in his or her employment. Some factors are outside the control of the employer or practice leaders, such as change in an individual's family life or desire and opportunity to pursue fellowship training. Others variables, however, are more directly tied to the job or practice environment. In a specialty where providers are relatively mobile and turnover is high, it is important for hospitalist practices to cultivate a climate in which the sacrifices associated with leaving outweigh the promised benefits.[29]

Results from the HMPWS suggested the need to address climate and fairness issues in hospitalist programs to improve satisfaction and retention.[9] Two large healthcare systems achieved success by investing in multipronged physician retention strategies including recruiting advisors, sign‐on bonuses, extensive onboarding, family support, and the promotion of ongoing effective communication.[3, 30]

Our findings suggest that morale for hospitalists is a complex amalgam of contentment and importance, and that there may not be a one size fits all solution to improving morale for all. While we did not find a difference in personal morale scores across individual hospitalist groups, or even between academic and community groups, each group had a unique profile with variability in the dynamics between importance and contentment of different factors. If practice group leaders review HMI data for their providers and use the information to facilitate meaningful dialogue with them about the factors influencing their morale, such leaders will have great insight into allocating resources for the best return on investment.

While we believe that the HMI is providing unique perspective compared to other commonly used metrics, it may be best to employ HMI data as complementary measures alongside that of some of the benchmarked scales that explore job satisfaction, job fit, and burnout among hospitalists.[6, 9, 10, 31, 32, 33, 34, 35] Aggregate HMI data at the group level may allow for the identification of factors that are highly important to morale but scored low in contentment. Such factors deserve priority and attention such that the subgroups within a practice can collaborate to come to consensus on strategies for amelioration. Because the HMI generates a score and profile for each provider, we can imagine effective leaders using the HMI with individuals as part of an annual review to facilitate discussion about maximizing contentment at work. Being fully transparent and sharing an honest nonanonymous version of the HMI with a superior would require a special relationship founded on trust and mutual respect.

Several limitations of this study should be considered. First, the initial item reduction and EFA were based on a single‐site survey, and our overall sample size was relatively small. We plan on expanding our sample size in the future for further validation of our exploratory findings. Second, the data were collected at 2 specific times several years ago. In continuing to analyze the data from subsequent years, validity and reliability results remain stable, thereby minimizing the likelihood of significant historical bias. Third, there may have been some recall bias, in that respondents may have overlooked the good and perseverated over variables that disappointed them. Fourth, although intention to leave does not necessarily equate actual employee turnover, intention has been found to be a strong predictor of quitting a job.[36, 37] Finally, while we had high response rates, response bias may have existed wherein those with lower morale may have elected not to complete the survey or became apathetic in their responses.

The HMI is a validated instrument that evaluates hospitalist morale by incorporating each provider's characterization of the importance of and contentment with 27 variables. By accounting for the multidimensional and dynamic nature of morale, the HMI may help program leaders tailor retention and engagement strategies specific to their own group. Future studies may explore trends in contributors to morale and examine whether interventions to augment low morale can result in improved morale and hospitalist retention.

Acknowledgements

The authors are indebted to the hospitalists who were willing to share their perspectives about their work, and grateful to Ms. Lisa Roberts, Ms. Barbara Brigade, and Ms. Regina Landis for insuring confidentiality in managing the survey database.

Disclosures: Dr. Chandra had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Dr. Wright is a Miller‐Coulson Family Scholar through the Johns Hopkins Center for Innovative Medicine. Ethical approval has been granted for studies involving human subjects by a Johns Hopkins University School of Medicine institutional review board. The authors report no conflicts of interest.

Explosive growth in hospital medicine has led to hospitalists having the option to change jobs easily. Annual turnover for all physicians is 6.8%, whereas that of hospitalists exceeds 14.8%.[1] Losing a single physician has significant financial and operational implications, with estimates of $20,000 to $120,000 in recruiting costs, and up to $500,000 in lost revenue that may take years to recoup due to the time required for new physician assimilation.[2, 3] In 2006, the Society of Hospital Medicine (SHM) appointed a career task force to develop retention recommendations, 1 of which includes monitoring hospitalists' job satisfaction.[4]

Studies examining physician satisfaction have demonstrated that high physician job satisfaction is associated with lower physician turnover.[5] However, surveys of hospitalists, including SHM's Hospital Medicine Physician Worklife Survey (HMPWS), have reported high job satisfaction among hospitalists,[6, 7, 8, 9, 10] suggesting that high job satisfaction may not be enough to overcome forces that pull hospitalists toward other opportunities.

Morale, a more complex construct related to an individual's contentment and happiness, might provide insight into reducing hospitalist turnover. Morale has been defined as the emotional or mental condition with respect to cheerfulness, confidence, or zeal and is especially relevant in the face of opposition or hardship.[11] Job satisfaction is 1 element that contributes to morale, but alone does not equate morale.[12] Morale, more than satisfaction, relates to how people see themselves within the group and may be closely tied to the concept of esprit de corps. To illustrate, workers may feel satisfied with the content of their job, but frustration with the organization may result in low morale.[13] Efforts focused on assessing provider morale may provide deeper understanding of hospitalists' professional needs and garner insight for retention strategies.

The construct of hospitalist morale and its underlying drivers has not been explored in the literature. Using literature within and outside of healthcare,[1, 12, 14, 15, 16, 17, 18, 19, 20, 21, 22] and our own prior work,[23] we sought to characterize elements that contribute to hospitalist morale and develop a metric to measure it. The HMPWS found that job satisfaction factors vary across hospitalist groups.[9] We suspected that the same would hold true for factors important to morale at the individual level. This study describes the development and validation of the Hospitalist Morale Index (HMI), and explores the relationship between morale and intent to leave due to unhappiness.

METHODS

2009 Pilot Survey

To establish content validity, after reviewing employee morale literature, and examining qualitative comments from our 2007 and 2008 morale surveys, our expert panel, consisting of practicing hospitalists, hospitalist leaders, and administrative staff, identified 46 potential drivers of hospitalist morale. In May 2009, all hospitalists, including physicians, nurse practitioners (NPs), and physician assistants (PAs) from a single hospitalist group received invitations to complete the pilot survey. We asked hospitalists to assess on 5‐point Likert scales the importance of (not at all to tremendously) and contentment with (extremely discontent to extremely content) each of the 46 items as it relates to their work morale. Also included were demographic questions and general morale questions (including rating participants' own morale), investment, long‐term career plans, and intent to leave due to unhappiness.

Data Collection

To maintain anonymity and limit social desirability bias, a database manager, working outside the Division of Hospital Medicine and otherwise not associated with the research team, used Survey Monkey to coordinate survey distribution and data collection. Each respondent had a unique identifier code that was unrelated to the respondent's name and email address. Personal identifiers were maintained in a secure database accessible only to the database manager.

Establishing Internal Structure Validity Evidence

Response frequency to each question was examined for irregularities in distribution. For continuous variables, descriptive statistics were examined for evidence of skewness, outliers, and non‐normality to ensure appropriate use of parametric statistical tests. Upon ranking importance ratings by mode, 15 of 46 items were judged to be of low importance by almost all participants and removed from further consideration.

Stata 13.1 (StataCorp, College Station, TX) was used for exploratory factor analysis (EFA) of the importance responses for all 31 remaining items by principal components factoring. Eigenvalues >1 were designated as a cutoff point for inclusion in varimax rotation. Factor loading of 0.50 was the threshold for inclusion in a factor.

The 31 items loaded across 10 factors; however, 3 factors included 1 item each. After reviewing the scree plot and considering their face value, these items/factors were omitted. Repeating the factor analysis resulted in a 28‐item, 7‐factor solution that accounted for 75% variance. All items were considered informative as demonstrated by low uniqueness scores (0.050.38). Using standard validation procedures, all 7 factors were found to have acceptable factor loadings (0.460.98) and face validity. Cronbach's quantified internal reliability of the 7 factors with scores ranging from 0.68 to 0.92. We named the resultant solution the Hospitalist Morale Index (HMI).

Establishing Response Process Validity Evidence

In developing the HMI, we asked respondents to rate the importance of and their contentment with each variable as related to their work morale. From pilot testing, which included discussions with respondents immediately after completing the survey, we learned that the 2‐part consideration of each variable resulted in thoughtful reflection about their morale. Further, by multiplying the contentment score for each item (scaled from 15) by the corresponding importance score (scaled 01), we quantified the relative contribution and contentment of each item for each hospitalist. Scaling importance scores from 0 to 1 insured that items that were not considered important to the respondent did not affect the respondent's personal morale score. Averaging resultant item scores that were greater than 0 resulted in a personal morale score for each hospitalist. Averaging item scores >0 that constituted each factor resulted in factor scores.

May 2011 Survey

The refined survey was distributed in May 2011 to a convenience sample of 5 hospitalist programs at separate hospitals (3 community hospitals, 2 academic hospitals) encompassing 108 hospitalists in 3 different states. Responses to the 2011 survey were used to complete confirmatory factor analyses (CFA) and establish further validity and reliability evidence.

Based on the 28‐item, 7‐factor solution developed from the pilot study, we developed the theoretical model of factors constituting hospitalist morale. We used the structural equation modeling command in Stata 13 to perform CFA. Factor loading of 0.50 was the threshold for inclusion of an item in a factor. To measure internal consistency, we considered Cronbach's score of 0.60 acceptable. Iterative models were reviewed to find the optimal solution for the data. Four items did not fit into any of the 5 resulting factors and were evaluated in terms of mean importance score and face value. Three items were considered important enough to warrant being stand‐alone items, whereas 1 was omitted. Two additional items had borderline factor loadings (0.48, 0.49) and were included in the model as stand‐alone items due to their overall relevance. The resultant solution was a 5‐factor model with 5 additional stand‐alone items (Table 1).

Confirmatory Factor Analysis Using Standardized Structured Equation Modeling of Importance Scores Retained in the Final Model Based on Survey Responses Gathered From Hospitalist Providers in 2011
 FactorCronbach's
ClinicalWorkloadLeadershipAppreciation and AcknowledgementMaterial Rewards
How much does the following item contribute to your morale?
Paperwork0.72    0.89
Relationship with patients0.69    0.90
Electronic medical system0.60    0.90
Intellectual stimulation0.59    0.90
Variety of cases0.58    0.90
Relationship with consultants0.51    0.89
No. of night shifts 0.74   0.89
Patient census 0.61   0.90
No. of shifts 0.52   0.90
Fairness of leadership  0.82  0.89
Effectiveness of leadership  0.82  0.89
Leadership's receptiveness to my thoughts and suggestions  0.78  0.89
Leadership as advocate for my needs  0.77  0.89
Approachability of leadership  0.77  0.89
Accessibility of leadership  0.69  0.89
Alignment of the group's goals with my goals  0.50  0.89
Recognition within the group   0.82 0.89
Feeling valued within the institution   0.73 0.89
Feeling valued within the group   0.73 0.89
Feedback   0.52 0.89
Pay    0.990.90
Benefits    0.560.89
Cronbach's 0.780.650.890.780.71 
How much does the following item contribute to your morale?Single item indicators 
Family time 0.90
Job security 0.90
Institutional climate 0.89
Opportunities for professional growth 0.90
Autonomy 0.89
Cronbach's  0.90

Establishing Convergent, Concurrent, and Discriminant Validity Evidence

To establish convergent, concurrent, and discriminant validity, linear and logistic regression models were examined for continuous and categorical data accordingly.

Self‐perceived overall work morale and perceived group morale, as assessed by 6‐point Likert questions with response options from terrible to excellent, were modeled as predictors for personal morale as calculated by the HMI.

Personal morale scores were modeled as predictors of professional growth, stress, investment in the group, and intent to leave due to unhappiness. While completing the HMI, hospitalists simultaneously completed a validated professional growth scale[24] and Cohen stress scale.[25] We hypothesized that those with higher morale would have more professional growth. Stress, although an important issue in the workplace, is a distinct construct from morale, and we did not expect a significant relationship between personal morale and stress. We used Pearson's r to assess the strength of association between the HMI and these scales. Participants' level of investment in their group was assessed on a 5‐point Likert scale. To simplify presentation, highly invested represents those claiming to be very or tremendously invested in the success of their current hospitalist group. Intent to leave due to unhappiness was assessed on a 5‐point Likert scale, I have had serious thoughts about leaving my current hospitalist group because I am unhappy, with responses from strongly disagree (1) to strongly agree (5). To simplify presentation, responses higher than 2 are considered to be consistent with intending to leave due to unhappiness.

Our institutional review board approved the study.

RESULTS

Respondents

In May 2009, 30 of the 33 (91%) invited hospitalists completed the original pilot morale survey; 19 (63%) were women. Eleven hospitalists (37%) had been part of the group 1 year or less, whereas 4 (13%) had been with the group for more than 5 years.

In May 2011, 93 of the 108 (86%) hospitalists from 5 hospitals completed the demographic and global parts of the survey. Fifty (53%) were from community hospitals; 47 (51%) were women. Thirty‐seven (40%) physicians and 6 (60%) NPs/PAs were from academic hospitals. Thirty‐nine hospitalists (42%) had been with their current group 1 year or less. Ten hospitalists (11%) had been with their current group over 5 years. Sixty‐three respondents (68%) considered themselves career hospitalists, whereas 5 (5%) did not; the rest were undecided.

Internal Structure Validity Evidence

The final CFA from the 2011 survey resulted in a 5‐factor plus 5stand‐alone‐items HMI. The solution with item‐level and factor‐level Cronbach's scores (range, 0.890.90 and range, 0.650.89, respectively) are shown in Table 1.

Personal Morale Scores and Factor Scores

Personal morale scores were normally distributed (mean = 2.79; standard deviation [SD] = 0.58), ranging from 1.23 to 4.22, with a theoretical low of 0 and high of 5 (Figure 1). Mean personal morale scores across hospitalist groups ranged from 2.70 to 2.99 (P > 0.05). Personal morale scores, factor sores and item scores for NPs and PAs did not significantly differ from those of physicians (P > 0.05 for all analyses). Personal morale scores were lower for those in their first 3 years with their current group, compared to those with greater institutional longevity. For every categorical increase in a participant's response to seeing oneself as a career hospitalist, the personal morale score rose 0.23 points (P < 0.001).

Figure 1
2011 personal moral scores for all hospitalists.

Factor scores for material reward and mean item scores for professional growth were significantly different across the 5 hospitalist groups (P = 0.03 and P < 0.001, respectively). Community hospitalists had significantly higher factor scores, despite having similar importance scores, for material rewards than academic hospitalists (diff. = 0.44, P = 0.02). Academic hospitalists had significantly higher scores for professional growth (diff. = 0.94, P < 0.001) (Table 2). Professional growth had the highest importance score for academic hospitalists (mean = 0.87, SD = 0.18) and the lowest importance score for community hospitalists (mean = 0.65, SD = 0.24, P < 0.001).

Personal Morale Scores, Factor Scores,* and Five Item Scores* by Hospitalist Groups
 Personal Morale ScoreFactor 1Factor 2Factor 3Factor 4Factor 5Item 1Item 2Item 3Item 4Item 5
ClinicalWorkloadLeadershipAppreciation and AcknowledgementMaterial RewardsFamily TimeInstitutional ClimateJob SecurityAutonomyProfessional Growth
  • NOTE: Abbreviations: SD, standard deviation.*Factor scores and item scores represent the combined product of importance and contentment.

All participantsMean2.792.542.783.182.582.483.052.672.923.002.76
SD0.580.630.700.950.860.851.150.971.111.101.21
Academic AMean2.772.432.923.102.542.283.162.703.063.203.08
SD0.570.620.640.920.840.771.190.951.081.121.24
Academic BMean2.992.582.993.882.692.002.582.131.653.294.33
SD0.360.700.800.290.800.350.920.880.781.010.82
Community AMean2.862.612.513.232.733.032.882.842.953.232.66
SD0.750.790.681.211.111.141.371.170.981.241.15
Community BMean2.862.742.973.372.672.443.282.352.702.502.25
SD0.670.550.861.040.940.871.001.151.400.721.26
Community CMean2.702.562.642.992.472.533.032.793.072.682.15
SD0.490.530.670.850.730.641.080.761.051.070.71
Academic combinedMean2.802.452.933.222.562.243.072.622.883.213.28
SD0.540.630.660.890.820.721.160.951.141.101.26
Community combinedMean2.792.612.663.142.602.683.032.722.952.822.34
SD0.620.620.721.010.900.901.150.991.091.091.00
P value>0.05>0.05>0.05>0.05>0.050.02>0.05>0.05>0.05>0.05<0.001

Convergent, Concurrent, and Discriminant Validity Evidence

For every categorical increase on the question assessing overall morale, the personal morale score was 0.23 points higher (P < 0.001). For every categorical increase in a participant's perception of the group's morale, the personal morale score was 0.29 points higher (P < 0.001).

For every 1‐point increase in personal morale score, the odds of being highly invested in the group increased by 5 times (odds ratio [OR]: 5.23, 95% confidence interval [CI]: 1.91‐14.35, P = 0.001). The mean personal morale score for highly invested hospitalists was 2.92, whereas that of those less invested was 2.43 (diff. = 0.49, P < 0.001) (Table 3). Highly invested hospitalists had significantly higher importance factor scores for leadership (diff. = 0.08, P = 0.03) as well as appreciation and acknowledgement (diff. = 0.08, P = 0.02).

Personal Morale Scores, Factor Scores,* and Five Item Scores* by Investment and Intent to Leave
 Personal Morale ScoreFactor 1Factor 2Factor 3Factor 4Factor 5Item 1Item 2Item 3Item 4Item 5
ClinicalWorkloadLeadershipAppreciation and AcknowledgementMaterial RewardsFamily TimeInstitutional ClimateJob SecurityAutonomyProfessional Growth
  • NOTE: Abbreviations: SD, standard deviation. *Factor scores and item scores represent the combined product of importance and contentment.

Highly invested in success of current hospitalist group
Mean2.922.612.893.382.782.453.212.782.863.102.95
SD0.550.590.680.920.880.771.111.001.091.061.25
Less invested in success of current hospitalist group
Mean2.432.342.482.602.022.572.602.383.082.692.24
SD0.520.690.690.810.491.041.170.831.181.190.94
P value<0.001>0.050.020.001<0.001>0.050.03>0.05>0.05>0.050.02
Not intending to leave because unhappy
Mean2.972.672.893.482.772.523.242.853.053.063.01
SD0.510.540.610.910.890.781.030.991.101.071.25
Intending to leave current group because unhappy
Mean2.452.302.592.592.212.402.682.332.672.882.28
SD0.560.720.820.740.680.971.290.831.111.170.97
P value<0.0010.01>0.05<0.0010.003>0.050.030.01>0.05>0.050.01

Every 1‐point increase in personal morale was associated with a rise of 2.27 on the professional growth scale (P = 0.01). The correlation between these 2 scales was 0.26 (P = 0.01). Every 1‐point increase in personal morale was associated with a 2.21 point decrease on the Cohen stress scale (P > 0.05). The correlation between these 2 scales was 0.21 (P > 0.05).

Morale and Intent to Leave Due to Unhappiness

Sixteen (37%) academic and 18 (36%) community hospitalists reported having thoughts of leaving their current hospitalist program due to unhappiness. The mean personal morale score for hospitalists with no intent to leave their current group was 2.97, whereas that of those with intent to leave was 2.45 (diff. = 0.53, P < 0.001). Each 1‐point increase in the personal morale score was associated with an 85% decrease (OR: 0.15, 95% CI: 0.05‐0.41, P < 0.001) in the odds of leaving because of unhappiness. Holding self‐perception of being a career hospitalist constant, each 1‐point increase in the personal morale score was associated with an 83% decrease (OR: 0.17, 95% CI: 0.05‐0.51, P = 0.002) in the odds of leaving because of unhappiness. Hospitalists who reported intent to leave had significantly lower factor scores for all factors and items except workload, material reward, and autonomy than those who did not report intent to leave (Table 3). Within the academic groups, those who reported intent to leave had significantly lower scores for professional growth (diff. = 1.08, P = 0.01). For community groups, those who reported intent to leave had significantly lower scores for clinical work (diff. = 0.54, P = 0.003), workload (diff. = 0.50, P = 0.02), leadership (diff. = 1.19, P < 0.001), feeling appreciated and acknowledged (diff. = 0.68, P = 0.01), job security (diff. = 0.70, P = 0.03), and institutional climate (diff. = 0.67, P = 0.02) than those who did not report intent to leave.

DISCUSSION

The HMI is a validated tool that objectively measures and quantifies hospitalist morale. The HMI's capacity to comprehensively assess morale comes from its breadth and depth in uncovering work‐related areas that may be sources of contentment or displeasure. Furthermore, the fact that HMI scores varied among groups of individuals, including those who are thinking about leaving their hospitalist group because they are unhappy and those who are highly invested in their hospitalist group, speaks to its ability to highlight and account for what is most important to hospitalist providers.

Low employee morale has been associated with decreased productivity, increased absenteeism, increased turnover, and decreased patient satisfaction.[2, 26, 27, 28] A few frustrated workers can breed group discontentment and lower the entire group's morale.[28] In addition to its financial impact, departures due to low morale can be sudden and devastating, leading to loss of team cohesiveness, increased work burden on the remaining workforce, burnout, and cascades of more turnover.[2] In contrast, when morale is high, workers more commonly go the extra mile, are more committed to the organization's mission, and are more supportive of their coworkers.[28]

While we asked the informants about plans to leave their job, there are many factors that drive an individual's intent and ultimate decision to make changes in his or her employment. Some factors are outside the control of the employer or practice leaders, such as change in an individual's family life or desire and opportunity to pursue fellowship training. Others variables, however, are more directly tied to the job or practice environment. In a specialty where providers are relatively mobile and turnover is high, it is important for hospitalist practices to cultivate a climate in which the sacrifices associated with leaving outweigh the promised benefits.[29]

Results from the HMPWS suggested the need to address climate and fairness issues in hospitalist programs to improve satisfaction and retention.[9] Two large healthcare systems achieved success by investing in multipronged physician retention strategies including recruiting advisors, sign‐on bonuses, extensive onboarding, family support, and the promotion of ongoing effective communication.[3, 30]

Our findings suggest that morale for hospitalists is a complex amalgam of contentment and importance, and that there may not be a one size fits all solution to improving morale for all. While we did not find a difference in personal morale scores across individual hospitalist groups, or even between academic and community groups, each group had a unique profile with variability in the dynamics between importance and contentment of different factors. If practice group leaders review HMI data for their providers and use the information to facilitate meaningful dialogue with them about the factors influencing their morale, such leaders will have great insight into allocating resources for the best return on investment.

While we believe that the HMI is providing unique perspective compared to other commonly used metrics, it may be best to employ HMI data as complementary measures alongside that of some of the benchmarked scales that explore job satisfaction, job fit, and burnout among hospitalists.[6, 9, 10, 31, 32, 33, 34, 35] Aggregate HMI data at the group level may allow for the identification of factors that are highly important to morale but scored low in contentment. Such factors deserve priority and attention such that the subgroups within a practice can collaborate to come to consensus on strategies for amelioration. Because the HMI generates a score and profile for each provider, we can imagine effective leaders using the HMI with individuals as part of an annual review to facilitate discussion about maximizing contentment at work. Being fully transparent and sharing an honest nonanonymous version of the HMI with a superior would require a special relationship founded on trust and mutual respect.

Several limitations of this study should be considered. First, the initial item reduction and EFA were based on a single‐site survey, and our overall sample size was relatively small. We plan on expanding our sample size in the future for further validation of our exploratory findings. Second, the data were collected at 2 specific times several years ago. In continuing to analyze the data from subsequent years, validity and reliability results remain stable, thereby minimizing the likelihood of significant historical bias. Third, there may have been some recall bias, in that respondents may have overlooked the good and perseverated over variables that disappointed them. Fourth, although intention to leave does not necessarily equate actual employee turnover, intention has been found to be a strong predictor of quitting a job.[36, 37] Finally, while we had high response rates, response bias may have existed wherein those with lower morale may have elected not to complete the survey or became apathetic in their responses.

The HMI is a validated instrument that evaluates hospitalist morale by incorporating each provider's characterization of the importance of and contentment with 27 variables. By accounting for the multidimensional and dynamic nature of morale, the HMI may help program leaders tailor retention and engagement strategies specific to their own group. Future studies may explore trends in contributors to morale and examine whether interventions to augment low morale can result in improved morale and hospitalist retention.

Acknowledgements

The authors are indebted to the hospitalists who were willing to share their perspectives about their work, and grateful to Ms. Lisa Roberts, Ms. Barbara Brigade, and Ms. Regina Landis for insuring confidentiality in managing the survey database.

Disclosures: Dr. Chandra had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Dr. Wright is a Miller‐Coulson Family Scholar through the Johns Hopkins Center for Innovative Medicine. Ethical approval has been granted for studies involving human subjects by a Johns Hopkins University School of Medicine institutional review board. The authors report no conflicts of interest.

References
  1. 2014 State of Hospital Medicine Report. Philadelphia, PA: Society of Hospital Medicine; 2014.
  2. Misra‐Hebert AD, Kay R, Stoller JK. A review of physician turnover: rates, causes, and consequences. Am J Med Qual. 2004;19(2):5666.
  3. Scott K. Physician retention plans help reduce costs and optimize revenues. Healthc Financ Manage. 1998;52(1):7577.
  4. SHM Career Satisfaction Task Force. A Challenge for a New Specialty: A White Paper on Hospitalist Career Satisfaction.; 2006. Available at: www.hospitalmedicine.org. Accessed February 28, 2009.
  5. Williams ES, Skinner AC. Outcomes of physician job satisfaction: a narrative review, implications, and directions for future research. Health Care Manage Rev. 2003;28(2):119139.
  6. Hoff TH, Whitcomb WF, Williams K, Nelson JR, Cheesman RA. Characteristics and work experiences of hospitalists in the United States. Arch Intern Med. 2001;161(6):851858.
  7. Hoff TJ. Doing the same and earning less: male and female physicians in a new medical specialty. Inquiry. 2004;41(3):301315.
  8. Clark‐Cox K. Physician satisfaction and communication. National findings and best practices. Available at: http://www.pressganey.com/files/clark_cox_acpe_apr06.pdf. Accessed October 10, 2010.
  9. Hinami K, Whelan CT, Wolosin RJ, Miller JA, Wetterneck TB. Worklife and satisfaction of hospitalists: toward flourishing careers. J Gen Intern Med. 2012;27(1):2836.
  10. Hinami K, Whelan CT, Miller JA, Wolosin RJ, Wetterneck TB; Society of Hospital Medicine Career Satisfaction Task Force. Job characteristics, satisfaction, and burnout across hospitalist practice models. J Hosp Med. 2012;7(5):402410.
  11. Morale | Define Morale at Dictionary.com. Morale | Define Morale at Dictionary.com. Morale | Define Morale at Dictionary.com. Available at: http://dictionary.reference.com/browse/morale. Accessed June 5, 2014.
  12. Guba EG. Morale and satisfaction: a study in past‐future time perspective. Adm Sci Q. 1958:195209.
  13. Kanter RM. Men and Women of the Corporation. 2nd ed. New York, NY: Basic Books; 1993.
  14. Charters WW. The relation of morale to turnover among teachers. Am Educ Res J. 1965:163173.
  15. Zeitz G. Structural and individual determinants of organization morale and satisfaction. Soc Forces. 1982;61:1088.
  16. Johnsrud LK, Heck RH, Rosser VJ. Morale matters: midlevel administrators and their intent to leave. J Higher Educ. 2000:3459.
  17. Worthy JC. Factors influencing employee morale. Harv Bus Rev. 1950;28(1):6173.
  18. Coughlan RJ. Dimensions of teacher morale. Am Educ Res J. 1970;7(2):221.
  19. Baehr ME, Renck R. The definition and measurement of employee morale. Adm Sci Q. 1958:157184.
  20. Konrad TR, Williams ES, Linzer M, et al. Measuring physician job satisfaction in a changing workplace and a challenging environment. SGIM Career Satisfaction Study Group. Society of General Internal Medicine. Med Care. 1999;37(11):11741182.
  21. Zeitz G. Structural and individual determinants of organization morale and satisfaction. Soc Forces. 1983;61(4):10881108.
  22. Durant H. Morale and its measurement. Am J Sociol. 1941;47(3):406414.
  23. Chandra S, Wright SM, Kargul G, Howell EE. Following morale over time within an academic hospitalist division. J Clin Outcomes Manag. 2011;18(1):2126.
  24. Wright SM, Levine RB, Beasley B, et al. Personal growth and its correlates during residency training. Med Educ. 2006;40(8):737745.
  25. Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav. 1983:385396.
  26. Johnsrud LK, Heck RH, Rosser VJ. Morale matters: midlevel administrators and their intent to leave. J Higher Educ. 2000;71(1):3459.
  27. Johnsrud LK, Rosser VJ. Faculty members' morale and their intention to leave: a multilevel explanation. J Higher Educ. 2002;73(4):518542.
  28. Bowles D, Cooper C. Employee Morale. New York, NY: Palgrave Macmillan; 2009.
  29. Maxfield D, Grenny J, McMillan R, Patterson K, Switzler A. Silence Kills. Silence Kills: The Seven Crucial Conversations® for Healthcare. VitalSmarts™ in association with the American Association of Critical Care Nurses, USA. 2005. Accessed October 10, 2014.
  30. Cohn KH, Bethancourt B, Simington M. The lifelong iterative process of physician retention. J Healthc Manag. 2009;54(4):220226.
  31. Chabot JM. Physicians' burnout. Rev Prat. 2004;54(7):753754.
  32. Virtanen P, Oksanen T, Kivimaki M, Virtanen M, Pentti J, Vahtera J. Work stress and health in primary health care physicians and hospital physicians. Occup Environ Med. 2008;65(5):364366.
  33. Williams ES, Konrad TR, Scheckler WE, et al. Understanding physicians' intentions to withdraw from practice: the role of job satisfaction, job stress, mental and physical health. 2001. Health Care Manage Rev. 2010;35(2):105115.
  34. Dyrbye LN, Varkey P, Boone SL, Satele DV, Sloan JA, Shanafelt TD. Physician satisfaction and burnout at different career stages. Mayo Clin Proc. 2013;88(12):13581367.
  35. Wetterneck TB, Williams MA. Burnout and Hospitalists: Etiology and Prevention. In: What Exactly Does A Hospitalist Do? Best of the Best Hospital Medicine 2005: Strategies for Success. Society of Hospital Medicine; 2005:5.
  36. Blau G, Boal K. Using job involvement and organizational commitment interactively to predict turnover. J Manage. 1989;15(1):115127.
  37. Hayes LJ, O'Brien‐Pallas L, Duffield C, et al. Nurse turnover: a literature review. Int J Nurs Stud. 2006;43(2):237263.
References
  1. 2014 State of Hospital Medicine Report. Philadelphia, PA: Society of Hospital Medicine; 2014.
  2. Misra‐Hebert AD, Kay R, Stoller JK. A review of physician turnover: rates, causes, and consequences. Am J Med Qual. 2004;19(2):5666.
  3. Scott K. Physician retention plans help reduce costs and optimize revenues. Healthc Financ Manage. 1998;52(1):7577.
  4. SHM Career Satisfaction Task Force. A Challenge for a New Specialty: A White Paper on Hospitalist Career Satisfaction.; 2006. Available at: www.hospitalmedicine.org. Accessed February 28, 2009.
  5. Williams ES, Skinner AC. Outcomes of physician job satisfaction: a narrative review, implications, and directions for future research. Health Care Manage Rev. 2003;28(2):119139.
  6. Hoff TH, Whitcomb WF, Williams K, Nelson JR, Cheesman RA. Characteristics and work experiences of hospitalists in the United States. Arch Intern Med. 2001;161(6):851858.
  7. Hoff TJ. Doing the same and earning less: male and female physicians in a new medical specialty. Inquiry. 2004;41(3):301315.
  8. Clark‐Cox K. Physician satisfaction and communication. National findings and best practices. Available at: http://www.pressganey.com/files/clark_cox_acpe_apr06.pdf. Accessed October 10, 2010.
  9. Hinami K, Whelan CT, Wolosin RJ, Miller JA, Wetterneck TB. Worklife and satisfaction of hospitalists: toward flourishing careers. J Gen Intern Med. 2012;27(1):2836.
  10. Hinami K, Whelan CT, Miller JA, Wolosin RJ, Wetterneck TB; Society of Hospital Medicine Career Satisfaction Task Force. Job characteristics, satisfaction, and burnout across hospitalist practice models. J Hosp Med. 2012;7(5):402410.
  11. Morale | Define Morale at Dictionary.com. Morale | Define Morale at Dictionary.com. Morale | Define Morale at Dictionary.com. Available at: http://dictionary.reference.com/browse/morale. Accessed June 5, 2014.
  12. Guba EG. Morale and satisfaction: a study in past‐future time perspective. Adm Sci Q. 1958:195209.
  13. Kanter RM. Men and Women of the Corporation. 2nd ed. New York, NY: Basic Books; 1993.
  14. Charters WW. The relation of morale to turnover among teachers. Am Educ Res J. 1965:163173.
  15. Zeitz G. Structural and individual determinants of organization morale and satisfaction. Soc Forces. 1982;61:1088.
  16. Johnsrud LK, Heck RH, Rosser VJ. Morale matters: midlevel administrators and their intent to leave. J Higher Educ. 2000:3459.
  17. Worthy JC. Factors influencing employee morale. Harv Bus Rev. 1950;28(1):6173.
  18. Coughlan RJ. Dimensions of teacher morale. Am Educ Res J. 1970;7(2):221.
  19. Baehr ME, Renck R. The definition and measurement of employee morale. Adm Sci Q. 1958:157184.
  20. Konrad TR, Williams ES, Linzer M, et al. Measuring physician job satisfaction in a changing workplace and a challenging environment. SGIM Career Satisfaction Study Group. Society of General Internal Medicine. Med Care. 1999;37(11):11741182.
  21. Zeitz G. Structural and individual determinants of organization morale and satisfaction. Soc Forces. 1983;61(4):10881108.
  22. Durant H. Morale and its measurement. Am J Sociol. 1941;47(3):406414.
  23. Chandra S, Wright SM, Kargul G, Howell EE. Following morale over time within an academic hospitalist division. J Clin Outcomes Manag. 2011;18(1):2126.
  24. Wright SM, Levine RB, Beasley B, et al. Personal growth and its correlates during residency training. Med Educ. 2006;40(8):737745.
  25. Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav. 1983:385396.
  26. Johnsrud LK, Heck RH, Rosser VJ. Morale matters: midlevel administrators and their intent to leave. J Higher Educ. 2000;71(1):3459.
  27. Johnsrud LK, Rosser VJ. Faculty members' morale and their intention to leave: a multilevel explanation. J Higher Educ. 2002;73(4):518542.
  28. Bowles D, Cooper C. Employee Morale. New York, NY: Palgrave Macmillan; 2009.
  29. Maxfield D, Grenny J, McMillan R, Patterson K, Switzler A. Silence Kills. Silence Kills: The Seven Crucial Conversations® for Healthcare. VitalSmarts™ in association with the American Association of Critical Care Nurses, USA. 2005. Accessed October 10, 2014.
  30. Cohn KH, Bethancourt B, Simington M. The lifelong iterative process of physician retention. J Healthc Manag. 2009;54(4):220226.
  31. Chabot JM. Physicians' burnout. Rev Prat. 2004;54(7):753754.
  32. Virtanen P, Oksanen T, Kivimaki M, Virtanen M, Pentti J, Vahtera J. Work stress and health in primary health care physicians and hospital physicians. Occup Environ Med. 2008;65(5):364366.
  33. Williams ES, Konrad TR, Scheckler WE, et al. Understanding physicians' intentions to withdraw from practice: the role of job satisfaction, job stress, mental and physical health. 2001. Health Care Manage Rev. 2010;35(2):105115.
  34. Dyrbye LN, Varkey P, Boone SL, Satele DV, Sloan JA, Shanafelt TD. Physician satisfaction and burnout at different career stages. Mayo Clin Proc. 2013;88(12):13581367.
  35. Wetterneck TB, Williams MA. Burnout and Hospitalists: Etiology and Prevention. In: What Exactly Does A Hospitalist Do? Best of the Best Hospital Medicine 2005: Strategies for Success. Society of Hospital Medicine; 2005:5.
  36. Blau G, Boal K. Using job involvement and organizational commitment interactively to predict turnover. J Manage. 1989;15(1):115127.
  37. Hayes LJ, O'Brien‐Pallas L, Duffield C, et al. Nurse turnover: a literature review. Int J Nurs Stud. 2006;43(2):237263.
Issue
Journal of Hospital Medicine - 11(6)
Issue
Journal of Hospital Medicine - 11(6)
Page Number
425-431
Page Number
425-431
Article Type
Display Headline
Introducing the Hospitalist Morale Index: A new tool that may be relevant for improving provider retention
Display Headline
Introducing the Hospitalist Morale Index: A new tool that may be relevant for improving provider retention
Sections
Article Source

© 2016 Society of Hospital Medicine

Disallow All Ads
Correspondence Location
Address for correspondence and reprint requests: Shalini Chandra, MD, MS, Johns Hopkins Bayview Medical Center, Johns Hopkins University School of Medicine, 5200 Eastern Avenue, MFL West, 6th Floor, Baltimore, MD 21224; Telephone: 410‐550‐0817; Fax: 410‐550‐340; E‐mail: schand12@jhmi.edu
Content Gating
No Gating (article Unlocked/Free)
Alternative CME
Article PDF Media
Media Files

On Track to Professorship? A Bibliometric Analysis of Early Scholarly Output

Article Type
Changed
Thu, 09/19/2019 - 13:27
Display Headline
On Track to Professorship? A Bibliometric Analysis of Early Scholarly Output

Professors of orthopedic surgery, by dint of their elevation to the highest academic rank, are men and women of achievement. Some of these surgeons have made their professional contribution primarily as clinicians; some have excelled as teachers. The common attribute of all medical school professors, though, is academic productivity, manifest in the form of scholarly publications.

The question of how much scholarly productivity is enough is of practical concern to junior faculty members contemplating their own chances for being promoted to the rank of professor. Specifically, a junior faculty member may wonder if his or her current performance augurs well for promotion. For these young faculty members (and the mentors advising them), there are not much objective data to offer guidance.

Research within other surgical subspecialties has revealed that the Hirsch index (h-index) is correlated with promotion to full professorship status.1,2 (An author earns an h-index of h if h of his or her papers has at least h citations.3 For example, an author of 10 papers each cited once and an author of 1 paper cited 10 times both have an h-index of 1, whereas an author of 5 papers each cited 5 times has an h-index of 5, as does an author of 10 papers, 5 of which were cited 5 times or more, and 5 of which were cited 4 or fewer times.) To our knowledge, within orthopedic surgery there has been only 1 study of the relationship between early-career academic output and ultimate academic rank—a single-institution study of 130 residents showing that those pursuing academic careers published more articles during residency.4

To help address the relationship between early-career academic output and the attainment of professorship, we performed a bibliometric benchmarking analysis of current orthopedic surgery professors’ productivity at a point likely before they were promoted to that rank. In measuring the early scholarly output of these now senior surgeons, we aim to give younger faculty members a basis of comparison for their own output and thus a sense of where they stand. Although a purely bibliometric analysis must be understood as a crude measure—one that fails to capture any of a professor’s attributes in a domain other than scholarly output—it may nevertheless serve as a basis for meaningful advice.

Therefore, we performed a bibliometric analysis to determine the number of scholarly papers published by current professors of orthopedic surgery within 5 years after their having acquired American Board of Orthopaedic Surgery (ABOS) certification (termed early scholarly output). We tried to determine not only quantity (how many papers were published) but quality (how often papers were cited). Last, by comparing professors across periods, we tried to address the relevant question of whether professor-worthy early output is increasing over time.

Methods

A cohort of orthopedic surgery professors at nominally elite medical schools was constructed as follows. The U.S. News & World Report ranking list was consulted to identify the top 10 US medical schools, and in February 2014 the website of each school was accessed to identify the orthopedic surgery faculty. Names of orthopedic surgery professors were noted. The website for Duke University did not list academic ranks, so data for this school were obtained by personal communication. Whether a professor’s title included the clinical descriptor was documented.

The ABOS website was then consulted to determine which of the faculty members were board-certified. Only certified faculty members were retained.

The Web of Science research platform (wokinfo.com) was used to identify each faculty member’s early scholarly output in the field of orthopedics. After limiting the period under consideration to 5 years after the author was ABOS-certified, we performed an author search using all combinations of first and middle initials. Results were then refined by category orthopedics and document type article. To reinforce the search specificity, we manually reviewed the generated bibliography and retained only correctly identified papers.

A Web of Science citation report was then generated for the author. All bibliometric data were recorded. The quantity of early output was logged as number of papers in 1 of 3 bins: first author, last author, and middle author (any author except first or last). Quality was approximated by total number of times the author was cited across total output. In addition, number of publications in Clinical Orthopaedics and Related Research (CORR) and Journal of Bone and Joint Surgery (JBJS) was recorded.

To further make an inference about the importance of papers published in this early career window, we calculated an h-index for this “5 years post ABOS certification” bibliography. As noted, an author earns an h-index of h if h of his or her papers has at least h citations.

 

 

The faculty member was assessed for publication of any “blockbuster” research, defined as a paper that had been cited at least 50 times between publication date and present day.

Last, to assess trends, we compared our output metrics for nonclinical professors ABOS-certified before 1990 versus after 1995. Significance was set at P < .006 using a conservative Bonferroni correction. Scatter plots were generated for total publications, citations, and h-index versus time since ABOS certification. Stata Statistical Software Release 11 (StataCorp) was used to analyze the data.

Results

Of the 108 professors identified, 88 did not have a clinical designation. Within this nonclinical group, median number of total publications and total citations 5 years after ABOS certification were 11.5 (mean, 15.4; SD, 12.3) and 33.5 (mean, 87.5; SD, 130.4), respectively. This group had a median h-index of 3 (mean, 3.9; SD, 3.1). Median number of papers published in CORR and JBJS was 4 (mean, 6.2; SD, 6.2). Median number of papers cited at least 50 times was 2 (mean, 3.2; SD, 4.0). A complete bibliometric summary is detailed in Tables 1 and 2.

 

Mean certification year was 1989 (range, 1968-2005; SD, 9.1 years). T tests revealed that total publications, first-author publications, last-author publications, middle-author publications, total citations, and h­-indexes were higher (Ps < .001-.004) for those certified after 1995 (n = 30) than for those certified before 1990 (n = 39) (Table 3). Scatter plots suggested that early total publications, citations, and h-indexes were increasing over time (Figure).

 

Discussion

Publication in the medical literature is an indication of academic productivity. However, there are no data establishing early-career productivity milestones. These data would interest young faculty members aspiring to attain professor status. We conducted the present study to describe the early academic productivity of current professors of orthopedic surgery at elite medical schools.

This study had several limitations. First, using bibliometric analysis to measure merit is admittedly crude, as it fails to capture contributions in nonacademic domains. For some faculty members, achievement in nonclinical areas may be substantial, and indeed the reason for their promotion. Second, the method used here tends to emphasize quantity over quality. Although we attempted to compensate for this bias—by reporting total citations, h-indexes, and numbers of CORR, JBJS, and blockbuster publications—we could not remove it completely. Third, choice of schools was arbitrary. Fourth, the sample included only those who attained professor rank; no data are available for orthopedic surgeons who were once assistant or associate professors and were not promoted further. Thus, even if number of publications was the sole criterion for promotion, no statement can be made about the likelihood of promotion given a certain number. Meaningful inferences about a candidate’s chance for promotion (assuming that the standards have not changed) can be made only with complete data, including “failures.”

Despite its limitations, this study provided novel information that can be useful to junior faculty members. Our cohort of orthopedic surgery professors at a select group of schools published 11 papers by year 5 after ABOS certification. A faculty member was the first or last author of 7 of these papers, and 3 papers were published in CORR or JBJS. Each of the 11 papers was cited almost 30 times, and 2 of the 11 eventually received at least 50 citations each. Faculty members had an h-index of about 3 at the 5-year mark. As expected, those who were clinical professors were less academically productive (nevertheless, some had formidable achievements). As schools may have different criteria for various academic titles, it is not possible to generalize across all schools. Of particular importance is the wide range for all data categories, particularly at the low end—buttressing the idea that, at some schools, clinical or teaching work may be sufficient for promotion.

Younger professors demonstrated higher early output than their senior counterparts did, as evidenced by increases in publications of any authorship, citations, and h-indexes. However, number of publications in CORR and JBJS was stagnant, as was number of publications cited more than 50 times. These findings may parallel the proliferation of journals, publications, and citations since the digitization of scientific media. For example, number of orthopedic Medline articles nearly doubled over the period 2000–2010, from 29,471 to 55,074 per year; in addition, number of authors per JBJS article increased from 1.6 in 1949 to 5.1 in 2009.5 This inflationary landscape may impose higher expectations on young faculty members, and, though this report suggests that professor-worthy output is increasing, it makes no effort to predict future milestones.To be sure, the information presented here does not represent a complete assessment of a faculty member’s contribution. In addition, standards for promotion will be different in the future than they were in the past. Nevertheless, our study results provide the best available (though imperfect) benchmarks for professor-worthy early productivity.

References

1.    Tomei KL, Nahass MM, Husain Q, et al. A gender-based comparison of academic rank and scholarly productivity in academic neurological surgery. J Clin Neurosci. 2014;21(7):1102-1105.

2.    Svider PF, Choudhry ZA, Choudhry OJ, Baredes S, Liu JK, Eloy JA. The use of the h-index in academic otolaryngology. Laryngoscope. 2013;123(1):103-106.

3.    Sharma B, Boet S, Grantcharov T, Shin E, Barrowman NJ, Bould MD. The h-index outperforms other bibliometrics in the assessment of research performance in general surgery: a province-wide study. Surgery. 2013;153(4):493-501.

4.    Namdari S, Jani S, Baldwin K, Mehta S. What is the relationship between number of publications during orthopaedic residency and selection of an academic career? J Bone Joint Surg Am. 2013;95(7):e45.

5.    Camp M, Escott BG. Authorship proliferation in the orthopaedic literature. J Bone Joint Surg Am. 2013;95(7):e44.

Article PDF
Author and Disclosure Information

Michael T. Milone, MD, and Joseph Bernstein, MD

Authors’ Disclosure Statement: The authors report no actual or potential conflict of interest in relation to this article.

Issue
The American Journal of Orthopedics - 45(3)
Publications
Topics
Page Number
E119-E123
Legacy Keywords
academic, career, data, publication, statistics, analysis, online exclusive, original study, study, milone, bernstein
Sections
Author and Disclosure Information

Michael T. Milone, MD, and Joseph Bernstein, MD

Authors’ Disclosure Statement: The authors report no actual or potential conflict of interest in relation to this article.

Author and Disclosure Information

Michael T. Milone, MD, and Joseph Bernstein, MD

Authors’ Disclosure Statement: The authors report no actual or potential conflict of interest in relation to this article.

Article PDF
Article PDF

Professors of orthopedic surgery, by dint of their elevation to the highest academic rank, are men and women of achievement. Some of these surgeons have made their professional contribution primarily as clinicians; some have excelled as teachers. The common attribute of all medical school professors, though, is academic productivity, manifest in the form of scholarly publications.

The question of how much scholarly productivity is enough is of practical concern to junior faculty members contemplating their own chances for being promoted to the rank of professor. Specifically, a junior faculty member may wonder if his or her current performance augurs well for promotion. For these young faculty members (and the mentors advising them), there are not much objective data to offer guidance.

Research within other surgical subspecialties has revealed that the Hirsch index (h-index) is correlated with promotion to full professorship status.1,2 (An author earns an h-index of h if h of his or her papers has at least h citations.3 For example, an author of 10 papers each cited once and an author of 1 paper cited 10 times both have an h-index of 1, whereas an author of 5 papers each cited 5 times has an h-index of 5, as does an author of 10 papers, 5 of which were cited 5 times or more, and 5 of which were cited 4 or fewer times.) To our knowledge, within orthopedic surgery there has been only 1 study of the relationship between early-career academic output and ultimate academic rank—a single-institution study of 130 residents showing that those pursuing academic careers published more articles during residency.4

To help address the relationship between early-career academic output and the attainment of professorship, we performed a bibliometric benchmarking analysis of current orthopedic surgery professors’ productivity at a point likely before they were promoted to that rank. In measuring the early scholarly output of these now senior surgeons, we aim to give younger faculty members a basis of comparison for their own output and thus a sense of where they stand. Although a purely bibliometric analysis must be understood as a crude measure—one that fails to capture any of a professor’s attributes in a domain other than scholarly output—it may nevertheless serve as a basis for meaningful advice.

Therefore, we performed a bibliometric analysis to determine the number of scholarly papers published by current professors of orthopedic surgery within 5 years after their having acquired American Board of Orthopaedic Surgery (ABOS) certification (termed early scholarly output). We tried to determine not only quantity (how many papers were published) but quality (how often papers were cited). Last, by comparing professors across periods, we tried to address the relevant question of whether professor-worthy early output is increasing over time.

Methods

A cohort of orthopedic surgery professors at nominally elite medical schools was constructed as follows. The U.S. News & World Report ranking list was consulted to identify the top 10 US medical schools, and in February 2014 the website of each school was accessed to identify the orthopedic surgery faculty. Names of orthopedic surgery professors were noted. The website for Duke University did not list academic ranks, so data for this school were obtained by personal communication. Whether a professor’s title included the clinical descriptor was documented.

The ABOS website was then consulted to determine which of the faculty members were board-certified. Only certified faculty members were retained.

The Web of Science research platform (wokinfo.com) was used to identify each faculty member’s early scholarly output in the field of orthopedics. After limiting the period under consideration to 5 years after the author was ABOS-certified, we performed an author search using all combinations of first and middle initials. Results were then refined by category orthopedics and document type article. To reinforce the search specificity, we manually reviewed the generated bibliography and retained only correctly identified papers.

A Web of Science citation report was then generated for the author. All bibliometric data were recorded. The quantity of early output was logged as number of papers in 1 of 3 bins: first author, last author, and middle author (any author except first or last). Quality was approximated by total number of times the author was cited across total output. In addition, number of publications in Clinical Orthopaedics and Related Research (CORR) and Journal of Bone and Joint Surgery (JBJS) was recorded.

To further make an inference about the importance of papers published in this early career window, we calculated an h-index for this “5 years post ABOS certification” bibliography. As noted, an author earns an h-index of h if h of his or her papers has at least h citations.

 

 

The faculty member was assessed for publication of any “blockbuster” research, defined as a paper that had been cited at least 50 times between publication date and present day.

Last, to assess trends, we compared our output metrics for nonclinical professors ABOS-certified before 1990 versus after 1995. Significance was set at P < .006 using a conservative Bonferroni correction. Scatter plots were generated for total publications, citations, and h-index versus time since ABOS certification. Stata Statistical Software Release 11 (StataCorp) was used to analyze the data.

Results

Of the 108 professors identified, 88 did not have a clinical designation. Within this nonclinical group, median number of total publications and total citations 5 years after ABOS certification were 11.5 (mean, 15.4; SD, 12.3) and 33.5 (mean, 87.5; SD, 130.4), respectively. This group had a median h-index of 3 (mean, 3.9; SD, 3.1). Median number of papers published in CORR and JBJS was 4 (mean, 6.2; SD, 6.2). Median number of papers cited at least 50 times was 2 (mean, 3.2; SD, 4.0). A complete bibliometric summary is detailed in Tables 1 and 2.

 

Mean certification year was 1989 (range, 1968-2005; SD, 9.1 years). T tests revealed that total publications, first-author publications, last-author publications, middle-author publications, total citations, and h­-indexes were higher (Ps < .001-.004) for those certified after 1995 (n = 30) than for those certified before 1990 (n = 39) (Table 3). Scatter plots suggested that early total publications, citations, and h-indexes were increasing over time (Figure).

 

Discussion

Publication in the medical literature is an indication of academic productivity. However, there are no data establishing early-career productivity milestones. These data would interest young faculty members aspiring to attain professor status. We conducted the present study to describe the early academic productivity of current professors of orthopedic surgery at elite medical schools.

This study had several limitations. First, using bibliometric analysis to measure merit is admittedly crude, as it fails to capture contributions in nonacademic domains. For some faculty members, achievement in nonclinical areas may be substantial, and indeed the reason for their promotion. Second, the method used here tends to emphasize quantity over quality. Although we attempted to compensate for this bias—by reporting total citations, h-indexes, and numbers of CORR, JBJS, and blockbuster publications—we could not remove it completely. Third, choice of schools was arbitrary. Fourth, the sample included only those who attained professor rank; no data are available for orthopedic surgeons who were once assistant or associate professors and were not promoted further. Thus, even if number of publications was the sole criterion for promotion, no statement can be made about the likelihood of promotion given a certain number. Meaningful inferences about a candidate’s chance for promotion (assuming that the standards have not changed) can be made only with complete data, including “failures.”

Despite its limitations, this study provided novel information that can be useful to junior faculty members. Our cohort of orthopedic surgery professors at a select group of schools published 11 papers by year 5 after ABOS certification. A faculty member was the first or last author of 7 of these papers, and 3 papers were published in CORR or JBJS. Each of the 11 papers was cited almost 30 times, and 2 of the 11 eventually received at least 50 citations each. Faculty members had an h-index of about 3 at the 5-year mark. As expected, those who were clinical professors were less academically productive (nevertheless, some had formidable achievements). As schools may have different criteria for various academic titles, it is not possible to generalize across all schools. Of particular importance is the wide range for all data categories, particularly at the low end—buttressing the idea that, at some schools, clinical or teaching work may be sufficient for promotion.

Younger professors demonstrated higher early output than their senior counterparts did, as evidenced by increases in publications of any authorship, citations, and h-indexes. However, number of publications in CORR and JBJS was stagnant, as was number of publications cited more than 50 times. These findings may parallel the proliferation of journals, publications, and citations since the digitization of scientific media. For example, number of orthopedic Medline articles nearly doubled over the period 2000–2010, from 29,471 to 55,074 per year; in addition, number of authors per JBJS article increased from 1.6 in 1949 to 5.1 in 2009.5 This inflationary landscape may impose higher expectations on young faculty members, and, though this report suggests that professor-worthy output is increasing, it makes no effort to predict future milestones.To be sure, the information presented here does not represent a complete assessment of a faculty member’s contribution. In addition, standards for promotion will be different in the future than they were in the past. Nevertheless, our study results provide the best available (though imperfect) benchmarks for professor-worthy early productivity.

Professors of orthopedic surgery, by dint of their elevation to the highest academic rank, are men and women of achievement. Some of these surgeons have made their professional contribution primarily as clinicians; some have excelled as teachers. The common attribute of all medical school professors, though, is academic productivity, manifest in the form of scholarly publications.

The question of how much scholarly productivity is enough is of practical concern to junior faculty members contemplating their own chances for being promoted to the rank of professor. Specifically, a junior faculty member may wonder if his or her current performance augurs well for promotion. For these young faculty members (and the mentors advising them), there are not much objective data to offer guidance.

Research within other surgical subspecialties has revealed that the Hirsch index (h-index) is correlated with promotion to full professorship status.1,2 (An author earns an h-index of h if h of his or her papers has at least h citations.3 For example, an author of 10 papers each cited once and an author of 1 paper cited 10 times both have an h-index of 1, whereas an author of 5 papers each cited 5 times has an h-index of 5, as does an author of 10 papers, 5 of which were cited 5 times or more, and 5 of which were cited 4 or fewer times.) To our knowledge, within orthopedic surgery there has been only 1 study of the relationship between early-career academic output and ultimate academic rank—a single-institution study of 130 residents showing that those pursuing academic careers published more articles during residency.4

To help address the relationship between early-career academic output and the attainment of professorship, we performed a bibliometric benchmarking analysis of current orthopedic surgery professors’ productivity at a point likely before they were promoted to that rank. In measuring the early scholarly output of these now senior surgeons, we aim to give younger faculty members a basis of comparison for their own output and thus a sense of where they stand. Although a purely bibliometric analysis must be understood as a crude measure—one that fails to capture any of a professor’s attributes in a domain other than scholarly output—it may nevertheless serve as a basis for meaningful advice.

Therefore, we performed a bibliometric analysis to determine the number of scholarly papers published by current professors of orthopedic surgery within 5 years after their having acquired American Board of Orthopaedic Surgery (ABOS) certification (termed early scholarly output). We tried to determine not only quantity (how many papers were published) but quality (how often papers were cited). Last, by comparing professors across periods, we tried to address the relevant question of whether professor-worthy early output is increasing over time.

Methods

A cohort of orthopedic surgery professors at nominally elite medical schools was constructed as follows. The U.S. News & World Report ranking list was consulted to identify the top 10 US medical schools, and in February 2014 the website of each school was accessed to identify the orthopedic surgery faculty. Names of orthopedic surgery professors were noted. The website for Duke University did not list academic ranks, so data for this school were obtained by personal communication. Whether a professor’s title included the clinical descriptor was documented.

The ABOS website was then consulted to determine which of the faculty members were board-certified. Only certified faculty members were retained.

The Web of Science research platform (wokinfo.com) was used to identify each faculty member’s early scholarly output in the field of orthopedics. After limiting the period under consideration to 5 years after the author was ABOS-certified, we performed an author search using all combinations of first and middle initials. Results were then refined by category orthopedics and document type article. To reinforce the search specificity, we manually reviewed the generated bibliography and retained only correctly identified papers.

A Web of Science citation report was then generated for the author. All bibliometric data were recorded. The quantity of early output was logged as number of papers in 1 of 3 bins: first author, last author, and middle author (any author except first or last). Quality was approximated by total number of times the author was cited across total output. In addition, number of publications in Clinical Orthopaedics and Related Research (CORR) and Journal of Bone and Joint Surgery (JBJS) was recorded.

To further make an inference about the importance of papers published in this early career window, we calculated an h-index for this “5 years post ABOS certification” bibliography. As noted, an author earns an h-index of h if h of his or her papers has at least h citations.

 

 

The faculty member was assessed for publication of any “blockbuster” research, defined as a paper that had been cited at least 50 times between publication date and present day.

Last, to assess trends, we compared our output metrics for nonclinical professors ABOS-certified before 1990 versus after 1995. Significance was set at P < .006 using a conservative Bonferroni correction. Scatter plots were generated for total publications, citations, and h-index versus time since ABOS certification. Stata Statistical Software Release 11 (StataCorp) was used to analyze the data.

Results

Of the 108 professors identified, 88 did not have a clinical designation. Within this nonclinical group, median number of total publications and total citations 5 years after ABOS certification were 11.5 (mean, 15.4; SD, 12.3) and 33.5 (mean, 87.5; SD, 130.4), respectively. This group had a median h-index of 3 (mean, 3.9; SD, 3.1). Median number of papers published in CORR and JBJS was 4 (mean, 6.2; SD, 6.2). Median number of papers cited at least 50 times was 2 (mean, 3.2; SD, 4.0). A complete bibliometric summary is detailed in Tables 1 and 2.

 

Mean certification year was 1989 (range, 1968-2005; SD, 9.1 years). T tests revealed that total publications, first-author publications, last-author publications, middle-author publications, total citations, and h­-indexes were higher (Ps < .001-.004) for those certified after 1995 (n = 30) than for those certified before 1990 (n = 39) (Table 3). Scatter plots suggested that early total publications, citations, and h-indexes were increasing over time (Figure).

 

Discussion

Publication in the medical literature is an indication of academic productivity. However, there are no data establishing early-career productivity milestones. These data would interest young faculty members aspiring to attain professor status. We conducted the present study to describe the early academic productivity of current professors of orthopedic surgery at elite medical schools.

This study had several limitations. First, using bibliometric analysis to measure merit is admittedly crude, as it fails to capture contributions in nonacademic domains. For some faculty members, achievement in nonclinical areas may be substantial, and indeed the reason for their promotion. Second, the method used here tends to emphasize quantity over quality. Although we attempted to compensate for this bias—by reporting total citations, h-indexes, and numbers of CORR, JBJS, and blockbuster publications—we could not remove it completely. Third, choice of schools was arbitrary. Fourth, the sample included only those who attained professor rank; no data are available for orthopedic surgeons who were once assistant or associate professors and were not promoted further. Thus, even if number of publications was the sole criterion for promotion, no statement can be made about the likelihood of promotion given a certain number. Meaningful inferences about a candidate’s chance for promotion (assuming that the standards have not changed) can be made only with complete data, including “failures.”

Despite its limitations, this study provided novel information that can be useful to junior faculty members. Our cohort of orthopedic surgery professors at a select group of schools published 11 papers by year 5 after ABOS certification. A faculty member was the first or last author of 7 of these papers, and 3 papers were published in CORR or JBJS. Each of the 11 papers was cited almost 30 times, and 2 of the 11 eventually received at least 50 citations each. Faculty members had an h-index of about 3 at the 5-year mark. As expected, those who were clinical professors were less academically productive (nevertheless, some had formidable achievements). As schools may have different criteria for various academic titles, it is not possible to generalize across all schools. Of particular importance is the wide range for all data categories, particularly at the low end—buttressing the idea that, at some schools, clinical or teaching work may be sufficient for promotion.

Younger professors demonstrated higher early output than their senior counterparts did, as evidenced by increases in publications of any authorship, citations, and h-indexes. However, number of publications in CORR and JBJS was stagnant, as was number of publications cited more than 50 times. These findings may parallel the proliferation of journals, publications, and citations since the digitization of scientific media. For example, number of orthopedic Medline articles nearly doubled over the period 2000–2010, from 29,471 to 55,074 per year; in addition, number of authors per JBJS article increased from 1.6 in 1949 to 5.1 in 2009.5 This inflationary landscape may impose higher expectations on young faculty members, and, though this report suggests that professor-worthy output is increasing, it makes no effort to predict future milestones.To be sure, the information presented here does not represent a complete assessment of a faculty member’s contribution. In addition, standards for promotion will be different in the future than they were in the past. Nevertheless, our study results provide the best available (though imperfect) benchmarks for professor-worthy early productivity.

References

1.    Tomei KL, Nahass MM, Husain Q, et al. A gender-based comparison of academic rank and scholarly productivity in academic neurological surgery. J Clin Neurosci. 2014;21(7):1102-1105.

2.    Svider PF, Choudhry ZA, Choudhry OJ, Baredes S, Liu JK, Eloy JA. The use of the h-index in academic otolaryngology. Laryngoscope. 2013;123(1):103-106.

3.    Sharma B, Boet S, Grantcharov T, Shin E, Barrowman NJ, Bould MD. The h-index outperforms other bibliometrics in the assessment of research performance in general surgery: a province-wide study. Surgery. 2013;153(4):493-501.

4.    Namdari S, Jani S, Baldwin K, Mehta S. What is the relationship between number of publications during orthopaedic residency and selection of an academic career? J Bone Joint Surg Am. 2013;95(7):e45.

5.    Camp M, Escott BG. Authorship proliferation in the orthopaedic literature. J Bone Joint Surg Am. 2013;95(7):e44.

References

1.    Tomei KL, Nahass MM, Husain Q, et al. A gender-based comparison of academic rank and scholarly productivity in academic neurological surgery. J Clin Neurosci. 2014;21(7):1102-1105.

2.    Svider PF, Choudhry ZA, Choudhry OJ, Baredes S, Liu JK, Eloy JA. The use of the h-index in academic otolaryngology. Laryngoscope. 2013;123(1):103-106.

3.    Sharma B, Boet S, Grantcharov T, Shin E, Barrowman NJ, Bould MD. The h-index outperforms other bibliometrics in the assessment of research performance in general surgery: a province-wide study. Surgery. 2013;153(4):493-501.

4.    Namdari S, Jani S, Baldwin K, Mehta S. What is the relationship between number of publications during orthopaedic residency and selection of an academic career? J Bone Joint Surg Am. 2013;95(7):e45.

5.    Camp M, Escott BG. Authorship proliferation in the orthopaedic literature. J Bone Joint Surg Am. 2013;95(7):e44.

Issue
The American Journal of Orthopedics - 45(3)
Issue
The American Journal of Orthopedics - 45(3)
Page Number
E119-E123
Page Number
E119-E123
Publications
Publications
Topics
Article Type
Display Headline
On Track to Professorship? A Bibliometric Analysis of Early Scholarly Output
Display Headline
On Track to Professorship? A Bibliometric Analysis of Early Scholarly Output
Legacy Keywords
academic, career, data, publication, statistics, analysis, online exclusive, original study, study, milone, bernstein
Legacy Keywords
academic, career, data, publication, statistics, analysis, online exclusive, original study, study, milone, bernstein
Sections
Article Source

PURLs Copyright

Inside the Article

Article PDF Media

Effects of Tumor Necrosis Factor α Inhibitors Extend Beyond Psoriasis: Insulin Sensitivity in Psoriasis Patients With Type 2 Diabetes Mellitus

Article Type
Changed
Thu, 12/15/2022 - 14:58
Display Headline
Effects of Tumor Necrosis Factor α Inhibitors Extend Beyond Psoriasis: Insulin Sensitivity in Psoriasis Patients With Type 2 Diabetes Mellitus

Psoriasis is a chronic inflammatory disorder associated with increased expression of proinflammatory mediators such as tumor necrosis factor (TNF) α.1 Anti-TNF drugs (eg, etanercept, adalimumab, infliximab) were proven to be highly effective for the treatment of psoriasis over the last 2 decades.2 Interestingly, TNF inhibitors have been thought to be effective in improving insulin resistance in patients with type 2 diabetes mellitus (DM) by blocking TNF, which is involved in the inflammatory condition in DM.

Type 2 DM is a common chronic condition characterized by hyperglycemia resulting from a combination of peripheral and hepatic insulin resistance and impaired insulin secretion.3 It is characterized by defects in both insulin secretion and insulin sensitivity.4,5 Type 2 DM has been linked with a marked increase in cardiovascular disease, morbidity, and mortality.6 Evidence-based literature regarding the role of chronic inflammation as an important pathogenetic factor in type 2 DM has been growing.7-9 It also has been suggested that pharmacological strategies to reduce this underlying associated silent inflammation are useful in treating DM, which also is true for other conditions such as obesity, metabolic syndrome, and cardiovascular diseases.10

Psoriasis predisposes patients to insulin resistance and may put them at risk for developing DM.11,12 The association between psoriasis and DM suggests that systemic immunosuppression also may diminish the risk for developing DM. Several longitudinal studies have found that TNF inhibitors improve insulin resistance.13,14 Dandona et al15 reported a considerable decrease of TNF-α levels with the concurrent restoration of insulin sensitivity during weight loss.

Pereira et al16 found a notable connection between psoriasis, DM, and insulin resistance with an odds ratio of 2.63 of abnormal glucose homeostasis in patients with psoriasis compared to controls. Yazdani-Biuki et al17 proved that extended administration of anti–TNF-α antibody was able to improve insulin sensitivity in insulin-resistant patients. The same finding was established by Kiortsis et al.14

In this prospective controlled study, we evaluated the effects of anti-TNF agents on insulin resistance and sensitivity in psoriasis patients with type 2 DM treated with anti-TNF agents.

Methods

A total of 70 patients attending the dermatological outpatient clinics at Farwaniya Hospital (Kuwait City, Kuwait) between January 2012 and September 2014 were enrolled in the study and were randomly distributed into 2 equal groups (n=35 each). The study was approved by the hospital ethics committee. Patients were included in the study if they had moderate to severe psoriasis (ie, psoriasis area severity index score ≥10) with documented type 2 DM and high fasting plasma glucose (FPG) levels (ie, >10 mmol/L). Patients who were currently being treated with oral hypoglycemic agents but not insulin therapy were included in the study. Patients were excluded if they had phototherapy within the last 4 weeks, prior biologic therapy, current and prior insulin therapy, a change in oral hypoglycemic drug dosage in the last 2 months, other serious systemic illness (eg, malignancy, hepatitis B or C virus, metabolic or endocrine disease), and/or abnormal laboratory investigations (eg, liver/kidney profile, chest radiograph abnormality, positive Mantoux test). All of the patients enrolled in the study provided informed consent and underwent routine baseline investigations including complete blood cell counts, general health profile, chest radiograph, antinuclear antibody test, Mantoux test, FPG and insulin levels, glycated hemoglobin (HbA1C), homeostasis model assessment (HOMA), routine urine examination, and enzyme-linked immunosorbent assay for tuberculosis. The study group was treated with anti-TNF agents, and the control group received conventional antipsoriatic medications.

All the patients included in the study had high FPG levels (ie, >10 mmol/L) at the time of enrollment and were currently being treated with oral hypoglycemic agents. The dose of oral hypoglycemic agents was unchanged for at least 2 months before entry into the run-in period and throughout the 24-week study period. Demographic details including sex, age, medical history (eg, type of psoriasis, prior and concomitant treatments) were collected from the participants’ clinical histories. Participants from both groups were appropriately matched in terms of age, sex, body weight, body mass index, and duration of type 2 DM (Table 1). The primary end point of the study was to analyze and compare clinical and serum data collected at baseline and after 24 weeks of therapy.

A complete biochemical profile was repeated in both groups after 24 weeks of treatment. Each participant underwent a baseline short insulin sensitivity test immediately before treatment and at 4 and 24 weeks of treatment. We assessed insulin resistance via HOMA, calculated as follows: FPG [mmol/L] × fasting serum insulin [pmol/L] / 22.5.18 Oral glucose tolerance tests were performed to calculate the HOMA of insulin resistance. Serum insulin concentration was determined via enzyme-linked immunosorbent assay.

 

 

Statistical analysis was performed using SPSS software (version 12.0). Continuous patient characteristics were analyzed using mean and SD as well as discrete data as counts and proportions. Association was examined using χ2 tests for categorical variables and 2-sided t test/Wilcoxon rank sum test for continuous variables. Analysis of variance was used to compare the results in 3 different anti-TNF agents used in the study group.

Results

Of the 35 participants enrolled in the study group, 34 (97.1%) completed the study and were evaluated. The study group included 16 men and 18 women aged 19 to 63 years (mean age [SD], 43.7 [21.6] years) who were treated with TNF-α inhibitors—8 participants with etanercept, 14 with adalimumab, and 12 with infliximab—according to the standard dosage schedule for 24 weeks.

Of the 35 participants enrolled in the control group, 29 (82.9%) completed the study and were evaluated. Six patients did not follow up for the complete duration of the 24-week study period and were not evaluated. The control group included 14 men and 15 women aged 18 to 65 years (mean age [SD], 47.7 [14.2] years) who were treated with other systemic therapies—8 participants with topical corticosteroids or calcipotriol only, 7 with cyclosporine A, and 14 with methotrexate. The dose of the drug was kept stable throughout the 24-week study period.

Demographic and baseline characteristics for all participants are shown in Table 1. There were no significant differences in demographic or baseline characteristics among the study group versus the control group, and all participants were similar in age; body mass index; as well as FPG, fasting insulin, and HbA1C levels.

At baseline, both study and control participants had elevated mean (SD) FPG levels (10 [25] mmol/L and 11 [0.4] mmol/L, respectively), fasting insulin levels (2.79 [0.17] pmol/L and 2.82 [0.13] pmol/L, respectively), and HbA1C levels (8.4% [0.38%] and 8.1% [0.21%], respectively)(Table 1).

The study group showed significant improvements in glycemic control at the end of the study (Table 2). At week 24, study group participants had a mean (SD) decrease in FPG levels of 2.74 (0.34) mmol/L versus 0.02 (0.16) mmol/L in the control group. This difference between the 2 groups after 24 weeks was found to be statistically significant (P<.01). On further analysis of the study group, no statistically significant difference (P>.01) was noted in the 3 anti-TNF agents used. Compared to the control group, the study group showed a significant decrease from baseline values of FPG and HbA1C (P<.01). Fasting insulin levels decreased significantly for study group participants as compared with control (–1.91 pmol/L vs 0.04 pmol/L)(P<.001)(Table 2). However, on analysis of the 3 anti-TNF agents, no statistically significant difference was found (P>.05). Participants in the control group showed no significant change in fasting insulin and FPG levels.

To confirm that there was a change in insulin sensitivity in response to TNF-α inhibitors, we analyzed FPG and fasting insulin values using the HOMA method. There was no change in mean relative insulin resistance in the control group in response to therapy (mean [SD], 5.4 [0.31] vs 5.6 [0.15], before vs after therapy), while there was mild improvement in relative insulin resistance in the study group (5.9 [0.52] vs 4.8 [0.34], before vs after therapy). There also was a significant difference in the change in relative insulin resistance in response to treatment between the study and control groups (1.2 [0.40] vs –0.3 [0.12]; P<.01)(Table 2).

Comment

There has been an unprecedented rise in the rate of obesity and associated metabolic diseases such as type 2 DM. Following the current trend, it is estimated that the world will have approximately 592 million cases of type 2 DM by the year 2035.19 Almost two-thirds of these patients are estimated to die of cardiovascular diseases.

Although the pathophysiology of type 2 DM is not known, insulin resistance in the muscles and liver as well as failure of pancreatic β cells represent the core of the complex pathophysiology. The associated underlying silent inflammation was thought to have a key role in both insulin resistance and insulin secretory defects seen in type 2 DM. Furthermore, recent data suggest the central role of TNF-α, IL-1, and IL-6 pathways in this inflammation.10 Tumor necrosis factor α has been shown to have a dual effect on insulin resistance as well as pancreatic β cell function. It blocks the function of insulin at the receptor level and has been implicated as a causative factor in obesity-associated insulin resistance and also in the pathogenesis of type 2 DM.20,21 Furthermore, cytokines that activate nuclear factor κβ (a nuclear transcription factor closely involved in the regulation of cellular inflammatory response), such as TNF-α, are thought to be a common denominator for β-cell apoptosis in types 1 and 2 DM.22 Additionally, it has been suggested that TNF-α is a powerful regulator of adipose tissue.23 Neutralizing TNF-α in obese Zucker rats has shown increased insulin sensitivity.3 Tumor necrosis factor α and IL-6 as well as C-reactive protein and plasminogen activator inhibitor 1 are negatively associated with insulin sensitivity.24-27 These findings have led researchers to investigate the role of anti-TNF agents for the management of type 2 DM.28

 

 

Psoriasis has now come to be known as a systemic inflammatory disorder and is associated with increased expression of TNF-α. It predisposes patients to insulin resistance and places them at higher risk for developing DM.11,12 Systematic reports recommend that there is a link between psoriasis and DM featured by helper T cell (TH1) cytokines.29 This link can stimulate insulin resistance and metabolic syndrome as well as inflammatory cytokines identified to motivate psoriasis.29,30 The association between psoriasis and type 2 DM proposes a possible pathophysiologic connection between the 2 diseases. Patients with psoriasis have altered T-cell subtype 1 pathways and dysregulated oxidative and angiogenic mechanisms.31,32 Many of these immune pathways may similarly predispose psoriasis patients to impaired glucose tolerance and DM. Inflammation may cause insulin resistance and DM through numerous mechanisms. Systemic inflammation linked with psoriasis may lead to high levels of circulating IL-1, IL-6, and TNF-α that predispose patients to impaired glucose tolerance and type 2 DM.33

Several longitudinal investigations have found that TNF inhibitors improve insulin resistance.13,14,34-38 Gonzalez-Gay et al13 confirmed a rapid beneficial effect of infliximab on insulin resistance and insulin sensitivity in rheumatoid arthritis (RA) patients, which might support the long-term use of drugs that act by blocking TNF-α to diminish the mechanisms implicated in the development of atherosclerosis in patients with RA. Kiortsis et al14 performed a complete biochemical profile before and after 6 months of treatment with infliximab in 17 patients with ankylosing spondylitis and 28 patients with RA. The researchers found a significant decrease of the HOMA index in the percentile of their patients with the highest insulin resistance (P<.01).14

Stagakis et al34 found that 12 weeks of treatment with anti-TNF agents may improve insulin resistance in patients with active RA and high insulin resistance. Treatment with anti-TNF agents was shown to restore the phosphorylation status of serine phosphorylation of insulin receptor substrate 1 (Ser312-IRS-1) and AKT (protein kinase B), which are important mediators in the insulin signaling cascade. The investigators concluded that treatment with anti-TNF agents may improve insulin resistance and sensitivity in RA patients with active disease and high insulin resistance.34

Solomon et al35 studied the link between disease-modifying antirheumatic drugs and DM risk in patients with RA and psoriasis. The authors proposed that initiation of treatment with TNF inhibitors in psoriasis patients was associated with a reduced incidence of DM. The results showed a lower risk for developing DM in patients with psoriasis who were treated with a TNF inhibitor compared with numerous other drugs.35

Marra et al36 studied the effects of etanercept on insulin sensitivity in 9 patients with psoriasis. They reported a decrease in insulin resistance evaluated by HOMA after 24 weeks of etanercept treatment.36 Wambier et al37 reported severe hypoglycemia after initiation of anti-TNF therapy with etanercept in a patient with generalized pustular psoriasis and type 2 DM.

Yazdani-Biuki et al38 reported the case of a patient who demonstrated a relapse of type 2 DM after an interruption of prolonged treatment with infliximab, an anti–TNF-α antibody for psoriatic arthritis. The improvement in insulin sensitivity of this patient has been reported along with post hoc evidence that chronic administration of infliximab improves insulin resistance in a small sample of patients with inflammatory joint diseases.17

Other studies on the effects of TNF inhibitors on insulin resistance and sensitivity have yielded conflicting results. Martínez-Abundis et al39 studied the effects of etanercept on insulin resistance and sensitivity in a randomized trial of psoriatic patients at risk for developing type 2 DM. Results indicated that anti-TNF therapy had no significant influence on insulin sensitivity measured using a hyperinsulinemic clamp during 2 weeks of etanercept treatment in psoriatic patients with risk factors for type 2 DM. The explanation of this discrepancy may be due to the short duration of the study period.

It is still unknown if psoriasis treatment affects a patient’s risk for developing DM. However, Solomon et al35 evaluated the association of incidental DM among patients with prescribed TNF inhibitors or methotrexate and proposed that initiation of treatment with TNF inhibitors was associated with a diminished incidence of DM.

Our study supports and confirms that psoriasis patients treated with TNF-α inhibitors showed improved glycemic indices and insulin resistance compared with control patients treated with other common systemic drugs for psoriasis. We did not take into consideration other conventional risk factors such as hypertension and coronary artery disease. The number of participants included in the current study was not large enough to evaluate each of the anti-TNF agents in a separate group, and participants were not followed up long enough to see the impact of the biochemical changes noted in the results on long-term morbidity or mortality.

 

 

Conclusion

Our study confirms a beneficial effect of TNF-α inhibitors on insulin resistance and insulin sensitivity in psoriasis patients with type 2 DM. Treatment with TNF-α inhibitors may have beneficial effects on insulin sensitivity in even the most insulin-resistant patients with psoriasis. The study results may support the hypothesis that long-term use of TNF inhibitors may reduce the mechanisms involved in the development of DM in patients with psoriasis. The improvement in insulin sensitivity may in turn decrease the coronary artery disease risk in these patients. Additional large, prospective, multicenter studies are required to further analyze the effects of anti–TNF-α antibodies on insulin sensitivity and β cell function in insulin-resistant or diabetic psoriasis patients.

References
  1. Lowes MA, Bowcock AM, Krueger JG. Pathogenesis and therapy of psoriasis. Nature. 2007;445:866-873.
  2. Boehncke WH, Prinz J, Gottlieb AB. Biologic therapies for psoriasis. a systematic review. J Rheumatol. 2006;33:1447-1451.
  3. DeFronzo RA, Bonadonna RC, Ferrannini E. Pathogenesis of NIDDM. a balanced overview. Diabetes Care. 1992;15:318-368.
  4. DeFronzo RA. Pathogenesis of type 2 diabetes: metabolic and molecular implications for identifying diabetes genes. Diabetes Rev. 1997;4:177-269.
  5. Reaven GM. Banting Lecture 1988. role of insulin resistance in human disease. Diabetes. 1988;37:1595-1607.
  6. Erkelens DW. Insulin resistance syndrome and type 2 diabetes mellitus. Am J Cardiol. 2001;88(7B):38J-42J.
  7. Hotamisligil GS, Shargill NS, Spiegelman BM. Adipose expression of tumor necrosis factor alpha: direct role in obesity-linked insulin resistance. Science. 1993;259:87-91.
  8. Hotamisligil GS, Spiegelman BM. Tumor necrosis factor alpha: a key component of the obesity-diabetes link. Diabetes. 1994;43:1271-1278.
  9. Van der Poll T, Romijn JA, Endert E, et al. Tumor necrosis factor mimics the metabolic response to acute infection in healthy humans. Am J Physiol. 1991;261:457-465.
  10. Tabas I, Glass CK. Anti-inflammatory therapy in chronic disease: challenges and opportunities. Science. 2013;339:166-172.
  11. Qureshi AA, Choi HK, Setty AR, et al. Psoriasis and the risk of diabetes and hypertension: a prospective study of US female nurses. Arch Dermatol. 2009;145:379-382.
  12. Solomon DH, Love TJ, Canning C, et al. Risk of diabetes among patients with rheumatoid arthritis, psoriatic arthritis and psoriasis. Ann Rheum Dis. 2010;69:2114-2117.
  13. Gonzalez-Gay MA, De Matias JM, Gonzalez-Juanatey C, et al. Anti-tumor necrosis factor-alpha blockade improves insulin resistance in patients with rheumatoid arthritis. Clin Exp Rheumatol. 2006;24:83-86.
  14. Kiortsis DN, Mavridis AK, Vasakos S, et al. Effects of infliximab treatment on insulin resistance in patients with rheumatoid arthritis and ankylosing spondylitis. Ann Rheum Dis. 2005;64:765-766.
  15. Dandona P, Weinstock R, Thusu K, et al. Tumor necrosis factor-α in sera of obese patients: fall with weight loss. J Clin Endocrinol Metabol. 1998;83:2907-2910.
  16. Pereira RR, Amladi ST, Varthakavi PK. A study of the prevalence of diabetes, insulin resistance, lipid abnormalities, and cardiovascular risk factors in patients with chronic plaque psoriasis. Ind J Dermatol. 2011;56:520-526.
  17. Yazdani-Biuki B, Stelzl H, Brezinschek HP, et al. Improvement of insulin sensitivity in insulin resistant subjects during prolonged treatment with the anti-TNF-α antibody infliximab. Eur J Clin Invest. 2004;34:641-642.
  18. Bonora E, Kiechl S, Willeit J, et al. Prevalence of insulin resistance in metabolic disorders. The Bruneck Study. Diabetes. 1998;47:1643-1649.
  19. Ryden L, Grant PJ, Anker SD, et al. ESC Guidelines on diabetes, prediabetes, and cardiovascular diseases developed in collaboration with the EASD: the Task Force on diabetes, prediabetes, and cardiovascular diseases of the European Society of Cardiology (ESC) and developed in collaboration with the European Association for the Study of Diabetes (EASD). Eur Heart J. 2013;34:3035-3087.
  20. Peraldi P, Spiegelman B. TNF-alpha and insulin resistance: summary and future prospects. Mol Cell Biochem. 1998;182:169-175.
  21. Moller DE. Potential role of TNF-alpha in the pathogenesis of insulin resistance and type 2 diabetes. Trends Endocrinol Metab. 2000;11:212-217.
  22. Mandrup-Poulsen T. Apoptotic signal transduction pathways in diabetes. Biochem Pharmacol. 2003;66:1433-1440.
  23. Coppack SW. Pro-inflammatory cytokines and adipose tissue. Proc Nutr Soc. 2001;60:349-356.
  24. Pradhan AD, Manson JE, Rifai N, et al. C-reactive protein, interleukin 6, and risk of developing type 2 diabetes mellitus. JAMA. 2001;286:327-334.
  25. Festa A, D’Agostino R Jr, Tracy RP, et al. Insulin Resistance Atherosclerosis Study. elevated levels of acute-phase proteins and plasminogen activator inhibitor-1 predict the development of type 2 diabetes: the insulin resistance atherosclerosis study. Diabetes. 2002;51:1131-1137.
  26. Meigs JB, O’Donnell CJ, Tofler GH, et al. Hemostatic markers of endothelial dysfunction and risk of incident type 2 diabetes: the Framingham Offspring Study. Diabetes. 2006;55:530-537.
  27. Liu S, Tinker L, Song Y, et al. A prospective study of inflammatory cytokines and diabetes mellitus in a multiethnic cohort of postmenopausal women. Arch Intern Med. 2007;167:1676-1685.
  28. Esser N, Paquot N, Scheen AJ. Anti-inflammatory agents to treat or prevent type 2 diabetes, metabolic syndrome and cardiovascular disease. Exp Opin Invest Drugs. 2014;24:1-25.
  29. Wellen KE, Hotamisligil GS. Inflammation, stress, and diabetes. J Clin Invest. 2005;115:1111-1119.
  30. Karadag AS, Yavuz B, Ertugrul DT, et al. Is psoriasis a pre-atherosclerotic disease? increased insulin resistance and impaired endothelial function in patients with psoriasis. Int J Dermatol. 2010;49:642-646.
  31. Armstrong AW, Voyles SV, Armstrong EJ, et al. A tale of two plaques: convergent mechanisms of T-cell–mediated inflammation in psoriasis and atherosclerosis. Exp Dermatol. 2011;20:544-549.
  32. Armstrong AW, Voyles SV, Armstrong EJ, et al. Angiogenesis and oxidative stress: common mechanisms linking psoriasis with atherosclerosis. J Dermatol Sci. 2011;63:1-9.
  33. Boehncke S, Thaci D, Beschmann H, et al. Psoriasis patients show signs of insulin resistance. Br J Dermatol. 2007;157:1249-1251.
  34. Stagakis I, Bertsias G, Karvounaris S, et al. Anti-tumor necrosis factor therapy improves insulin resistance, beta cell function and insulin signaling in active rheumatoid arthritis patients with high insulin resistance. Arthritis Res Ther. 2012;14:R141.
  35. Solomon DH, Massarotti E, Garg R, et al. Association between disease-modifying antirheumatic drugs and diabetes risk in patients with rheumatoid arthritis and psoriasis. JAMA. 2011;305:2525-2531.
  36. Marra M, Campanati A, Testa R, et al. Effect of etanercept on insulin sensitivity in nine patients with psoriasis. Int J Immunopathol Pharmacol. 2007;20:731-736.
  37. Wambier CG, Foss-Freitas MC, Paschoal RS, et al. Severe hypoglycemia after initiation of anti-tumor necrosis factor therapy with etanercept in a patient with generalized pustular psoriasis and type 2 diabetes mellitus. J Am Acad Dermatol. 2009;60:883-885.
  38. Yazdani-Biuki B, Mueller T, Brezinschek HP, et al. Relapse of diabetes after interruption of chronic administration of anti-tumor necrosis factor-alpha antibody infliximab: a case observation. Diabetes Care. 2006;29:1712.
  39. Martínez-Abundis E, Reynoso-von Drateln C, Hernández-Salazar E, et al. Effect of etanercept on insulin secretion and insulin sensitivity in a randomized trial with psoriatic patients at risk for developing type 2 diabetes mellitus. Arch Dermatol Res. 2007;299:461-465.
Article PDF
Author and Disclosure Information

Dr. Al-Mutairi is from the Department of Medicine, Kuwait University, Kuwait City. Dr. Shaaban is from the Department of Dermatology, Farwaniya Hospital, Kuwait City.

The authors report no conflict of interest.

Correspondence: Nawaf Al-Mutairi, MD, FRCPC, Post Box No. 280, Farwaniya 80000, Kuwait (nalmut@usa.net).

Issue
Cutis - 97(3)
Publications
Topics
Page Number
235-241
Legacy Keywords
diabetes mellitus; psoriasis; biologics
Sections
Author and Disclosure Information

Dr. Al-Mutairi is from the Department of Medicine, Kuwait University, Kuwait City. Dr. Shaaban is from the Department of Dermatology, Farwaniya Hospital, Kuwait City.

The authors report no conflict of interest.

Correspondence: Nawaf Al-Mutairi, MD, FRCPC, Post Box No. 280, Farwaniya 80000, Kuwait (nalmut@usa.net).

Author and Disclosure Information

Dr. Al-Mutairi is from the Department of Medicine, Kuwait University, Kuwait City. Dr. Shaaban is from the Department of Dermatology, Farwaniya Hospital, Kuwait City.

The authors report no conflict of interest.

Correspondence: Nawaf Al-Mutairi, MD, FRCPC, Post Box No. 280, Farwaniya 80000, Kuwait (nalmut@usa.net).

Article PDF
Article PDF
Related Articles

Psoriasis is a chronic inflammatory disorder associated with increased expression of proinflammatory mediators such as tumor necrosis factor (TNF) α.1 Anti-TNF drugs (eg, etanercept, adalimumab, infliximab) were proven to be highly effective for the treatment of psoriasis over the last 2 decades.2 Interestingly, TNF inhibitors have been thought to be effective in improving insulin resistance in patients with type 2 diabetes mellitus (DM) by blocking TNF, which is involved in the inflammatory condition in DM.

Type 2 DM is a common chronic condition characterized by hyperglycemia resulting from a combination of peripheral and hepatic insulin resistance and impaired insulin secretion.3 It is characterized by defects in both insulin secretion and insulin sensitivity.4,5 Type 2 DM has been linked with a marked increase in cardiovascular disease, morbidity, and mortality.6 Evidence-based literature regarding the role of chronic inflammation as an important pathogenetic factor in type 2 DM has been growing.7-9 It also has been suggested that pharmacological strategies to reduce this underlying associated silent inflammation are useful in treating DM, which also is true for other conditions such as obesity, metabolic syndrome, and cardiovascular diseases.10

Psoriasis predisposes patients to insulin resistance and may put them at risk for developing DM.11,12 The association between psoriasis and DM suggests that systemic immunosuppression also may diminish the risk for developing DM. Several longitudinal studies have found that TNF inhibitors improve insulin resistance.13,14 Dandona et al15 reported a considerable decrease of TNF-α levels with the concurrent restoration of insulin sensitivity during weight loss.

Pereira et al16 found a notable connection between psoriasis, DM, and insulin resistance with an odds ratio of 2.63 of abnormal glucose homeostasis in patients with psoriasis compared to controls. Yazdani-Biuki et al17 proved that extended administration of anti–TNF-α antibody was able to improve insulin sensitivity in insulin-resistant patients. The same finding was established by Kiortsis et al.14

In this prospective controlled study, we evaluated the effects of anti-TNF agents on insulin resistance and sensitivity in psoriasis patients with type 2 DM treated with anti-TNF agents.

Methods

A total of 70 patients attending the dermatological outpatient clinics at Farwaniya Hospital (Kuwait City, Kuwait) between January 2012 and September 2014 were enrolled in the study and were randomly distributed into 2 equal groups (n=35 each). The study was approved by the hospital ethics committee. Patients were included in the study if they had moderate to severe psoriasis (ie, psoriasis area severity index score ≥10) with documented type 2 DM and high fasting plasma glucose (FPG) levels (ie, >10 mmol/L). Patients who were currently being treated with oral hypoglycemic agents but not insulin therapy were included in the study. Patients were excluded if they had phototherapy within the last 4 weeks, prior biologic therapy, current and prior insulin therapy, a change in oral hypoglycemic drug dosage in the last 2 months, other serious systemic illness (eg, malignancy, hepatitis B or C virus, metabolic or endocrine disease), and/or abnormal laboratory investigations (eg, liver/kidney profile, chest radiograph abnormality, positive Mantoux test). All of the patients enrolled in the study provided informed consent and underwent routine baseline investigations including complete blood cell counts, general health profile, chest radiograph, antinuclear antibody test, Mantoux test, FPG and insulin levels, glycated hemoglobin (HbA1C), homeostasis model assessment (HOMA), routine urine examination, and enzyme-linked immunosorbent assay for tuberculosis. The study group was treated with anti-TNF agents, and the control group received conventional antipsoriatic medications.

All the patients included in the study had high FPG levels (ie, >10 mmol/L) at the time of enrollment and were currently being treated with oral hypoglycemic agents. The dose of oral hypoglycemic agents was unchanged for at least 2 months before entry into the run-in period and throughout the 24-week study period. Demographic details including sex, age, medical history (eg, type of psoriasis, prior and concomitant treatments) were collected from the participants’ clinical histories. Participants from both groups were appropriately matched in terms of age, sex, body weight, body mass index, and duration of type 2 DM (Table 1). The primary end point of the study was to analyze and compare clinical and serum data collected at baseline and after 24 weeks of therapy.

A complete biochemical profile was repeated in both groups after 24 weeks of treatment. Each participant underwent a baseline short insulin sensitivity test immediately before treatment and at 4 and 24 weeks of treatment. We assessed insulin resistance via HOMA, calculated as follows: FPG [mmol/L] × fasting serum insulin [pmol/L] / 22.5.18 Oral glucose tolerance tests were performed to calculate the HOMA of insulin resistance. Serum insulin concentration was determined via enzyme-linked immunosorbent assay.

 

 

Statistical analysis was performed using SPSS software (version 12.0). Continuous patient characteristics were analyzed using mean and SD as well as discrete data as counts and proportions. Association was examined using χ2 tests for categorical variables and 2-sided t test/Wilcoxon rank sum test for continuous variables. Analysis of variance was used to compare the results in 3 different anti-TNF agents used in the study group.

Results

Of the 35 participants enrolled in the study group, 34 (97.1%) completed the study and were evaluated. The study group included 16 men and 18 women aged 19 to 63 years (mean age [SD], 43.7 [21.6] years) who were treated with TNF-α inhibitors—8 participants with etanercept, 14 with adalimumab, and 12 with infliximab—according to the standard dosage schedule for 24 weeks.

Of the 35 participants enrolled in the control group, 29 (82.9%) completed the study and were evaluated. Six patients did not follow up for the complete duration of the 24-week study period and were not evaluated. The control group included 14 men and 15 women aged 18 to 65 years (mean age [SD], 47.7 [14.2] years) who were treated with other systemic therapies—8 participants with topical corticosteroids or calcipotriol only, 7 with cyclosporine A, and 14 with methotrexate. The dose of the drug was kept stable throughout the 24-week study period.

Demographic and baseline characteristics for all participants are shown in Table 1. There were no significant differences in demographic or baseline characteristics among the study group versus the control group, and all participants were similar in age; body mass index; as well as FPG, fasting insulin, and HbA1C levels.

At baseline, both study and control participants had elevated mean (SD) FPG levels (10 [25] mmol/L and 11 [0.4] mmol/L, respectively), fasting insulin levels (2.79 [0.17] pmol/L and 2.82 [0.13] pmol/L, respectively), and HbA1C levels (8.4% [0.38%] and 8.1% [0.21%], respectively)(Table 1).

The study group showed significant improvements in glycemic control at the end of the study (Table 2). At week 24, study group participants had a mean (SD) decrease in FPG levels of 2.74 (0.34) mmol/L versus 0.02 (0.16) mmol/L in the control group. This difference between the 2 groups after 24 weeks was found to be statistically significant (P<.01). On further analysis of the study group, no statistically significant difference (P>.01) was noted in the 3 anti-TNF agents used. Compared to the control group, the study group showed a significant decrease from baseline values of FPG and HbA1C (P<.01). Fasting insulin levels decreased significantly for study group participants as compared with control (–1.91 pmol/L vs 0.04 pmol/L)(P<.001)(Table 2). However, on analysis of the 3 anti-TNF agents, no statistically significant difference was found (P>.05). Participants in the control group showed no significant change in fasting insulin and FPG levels.

To confirm that there was a change in insulin sensitivity in response to TNF-α inhibitors, we analyzed FPG and fasting insulin values using the HOMA method. There was no change in mean relative insulin resistance in the control group in response to therapy (mean [SD], 5.4 [0.31] vs 5.6 [0.15], before vs after therapy), while there was mild improvement in relative insulin resistance in the study group (5.9 [0.52] vs 4.8 [0.34], before vs after therapy). There also was a significant difference in the change in relative insulin resistance in response to treatment between the study and control groups (1.2 [0.40] vs –0.3 [0.12]; P<.01)(Table 2).

Comment

There has been an unprecedented rise in the rate of obesity and associated metabolic diseases such as type 2 DM. Following the current trend, it is estimated that the world will have approximately 592 million cases of type 2 DM by the year 2035.19 Almost two-thirds of these patients are estimated to die of cardiovascular diseases.

Although the pathophysiology of type 2 DM is not known, insulin resistance in the muscles and liver as well as failure of pancreatic β cells represent the core of the complex pathophysiology. The associated underlying silent inflammation was thought to have a key role in both insulin resistance and insulin secretory defects seen in type 2 DM. Furthermore, recent data suggest the central role of TNF-α, IL-1, and IL-6 pathways in this inflammation.10 Tumor necrosis factor α has been shown to have a dual effect on insulin resistance as well as pancreatic β cell function. It blocks the function of insulin at the receptor level and has been implicated as a causative factor in obesity-associated insulin resistance and also in the pathogenesis of type 2 DM.20,21 Furthermore, cytokines that activate nuclear factor κβ (a nuclear transcription factor closely involved in the regulation of cellular inflammatory response), such as TNF-α, are thought to be a common denominator for β-cell apoptosis in types 1 and 2 DM.22 Additionally, it has been suggested that TNF-α is a powerful regulator of adipose tissue.23 Neutralizing TNF-α in obese Zucker rats has shown increased insulin sensitivity.3 Tumor necrosis factor α and IL-6 as well as C-reactive protein and plasminogen activator inhibitor 1 are negatively associated with insulin sensitivity.24-27 These findings have led researchers to investigate the role of anti-TNF agents for the management of type 2 DM.28

 

 

Psoriasis has now come to be known as a systemic inflammatory disorder and is associated with increased expression of TNF-α. It predisposes patients to insulin resistance and places them at higher risk for developing DM.11,12 Systematic reports recommend that there is a link between psoriasis and DM featured by helper T cell (TH1) cytokines.29 This link can stimulate insulin resistance and metabolic syndrome as well as inflammatory cytokines identified to motivate psoriasis.29,30 The association between psoriasis and type 2 DM proposes a possible pathophysiologic connection between the 2 diseases. Patients with psoriasis have altered T-cell subtype 1 pathways and dysregulated oxidative and angiogenic mechanisms.31,32 Many of these immune pathways may similarly predispose psoriasis patients to impaired glucose tolerance and DM. Inflammation may cause insulin resistance and DM through numerous mechanisms. Systemic inflammation linked with psoriasis may lead to high levels of circulating IL-1, IL-6, and TNF-α that predispose patients to impaired glucose tolerance and type 2 DM.33

Several longitudinal investigations have found that TNF inhibitors improve insulin resistance.13,14,34-38 Gonzalez-Gay et al13 confirmed a rapid beneficial effect of infliximab on insulin resistance and insulin sensitivity in rheumatoid arthritis (RA) patients, which might support the long-term use of drugs that act by blocking TNF-α to diminish the mechanisms implicated in the development of atherosclerosis in patients with RA. Kiortsis et al14 performed a complete biochemical profile before and after 6 months of treatment with infliximab in 17 patients with ankylosing spondylitis and 28 patients with RA. The researchers found a significant decrease of the HOMA index in the percentile of their patients with the highest insulin resistance (P<.01).14

Stagakis et al34 found that 12 weeks of treatment with anti-TNF agents may improve insulin resistance in patients with active RA and high insulin resistance. Treatment with anti-TNF agents was shown to restore the phosphorylation status of serine phosphorylation of insulin receptor substrate 1 (Ser312-IRS-1) and AKT (protein kinase B), which are important mediators in the insulin signaling cascade. The investigators concluded that treatment with anti-TNF agents may improve insulin resistance and sensitivity in RA patients with active disease and high insulin resistance.34

Solomon et al35 studied the link between disease-modifying antirheumatic drugs and DM risk in patients with RA and psoriasis. The authors proposed that initiation of treatment with TNF inhibitors in psoriasis patients was associated with a reduced incidence of DM. The results showed a lower risk for developing DM in patients with psoriasis who were treated with a TNF inhibitor compared with numerous other drugs.35

Marra et al36 studied the effects of etanercept on insulin sensitivity in 9 patients with psoriasis. They reported a decrease in insulin resistance evaluated by HOMA after 24 weeks of etanercept treatment.36 Wambier et al37 reported severe hypoglycemia after initiation of anti-TNF therapy with etanercept in a patient with generalized pustular psoriasis and type 2 DM.

Yazdani-Biuki et al38 reported the case of a patient who demonstrated a relapse of type 2 DM after an interruption of prolonged treatment with infliximab, an anti–TNF-α antibody for psoriatic arthritis. The improvement in insulin sensitivity of this patient has been reported along with post hoc evidence that chronic administration of infliximab improves insulin resistance in a small sample of patients with inflammatory joint diseases.17

Other studies on the effects of TNF inhibitors on insulin resistance and sensitivity have yielded conflicting results. Martínez-Abundis et al39 studied the effects of etanercept on insulin resistance and sensitivity in a randomized trial of psoriatic patients at risk for developing type 2 DM. Results indicated that anti-TNF therapy had no significant influence on insulin sensitivity measured using a hyperinsulinemic clamp during 2 weeks of etanercept treatment in psoriatic patients with risk factors for type 2 DM. The explanation of this discrepancy may be due to the short duration of the study period.

It is still unknown if psoriasis treatment affects a patient’s risk for developing DM. However, Solomon et al35 evaluated the association of incidental DM among patients with prescribed TNF inhibitors or methotrexate and proposed that initiation of treatment with TNF inhibitors was associated with a diminished incidence of DM.

Our study supports and confirms that psoriasis patients treated with TNF-α inhibitors showed improved glycemic indices and insulin resistance compared with control patients treated with other common systemic drugs for psoriasis. We did not take into consideration other conventional risk factors such as hypertension and coronary artery disease. The number of participants included in the current study was not large enough to evaluate each of the anti-TNF agents in a separate group, and participants were not followed up long enough to see the impact of the biochemical changes noted in the results on long-term morbidity or mortality.

 

 

Conclusion

Our study confirms a beneficial effect of TNF-α inhibitors on insulin resistance and insulin sensitivity in psoriasis patients with type 2 DM. Treatment with TNF-α inhibitors may have beneficial effects on insulin sensitivity in even the most insulin-resistant patients with psoriasis. The study results may support the hypothesis that long-term use of TNF inhibitors may reduce the mechanisms involved in the development of DM in patients with psoriasis. The improvement in insulin sensitivity may in turn decrease the coronary artery disease risk in these patients. Additional large, prospective, multicenter studies are required to further analyze the effects of anti–TNF-α antibodies on insulin sensitivity and β cell function in insulin-resistant or diabetic psoriasis patients.

Psoriasis is a chronic inflammatory disorder associated with increased expression of proinflammatory mediators such as tumor necrosis factor (TNF) α.1 Anti-TNF drugs (eg, etanercept, adalimumab, infliximab) were proven to be highly effective for the treatment of psoriasis over the last 2 decades.2 Interestingly, TNF inhibitors have been thought to be effective in improving insulin resistance in patients with type 2 diabetes mellitus (DM) by blocking TNF, which is involved in the inflammatory condition in DM.

Type 2 DM is a common chronic condition characterized by hyperglycemia resulting from a combination of peripheral and hepatic insulin resistance and impaired insulin secretion.3 It is characterized by defects in both insulin secretion and insulin sensitivity.4,5 Type 2 DM has been linked with a marked increase in cardiovascular disease, morbidity, and mortality.6 Evidence-based literature regarding the role of chronic inflammation as an important pathogenetic factor in type 2 DM has been growing.7-9 It also has been suggested that pharmacological strategies to reduce this underlying associated silent inflammation are useful in treating DM, which also is true for other conditions such as obesity, metabolic syndrome, and cardiovascular diseases.10

Psoriasis predisposes patients to insulin resistance and may put them at risk for developing DM.11,12 The association between psoriasis and DM suggests that systemic immunosuppression also may diminish the risk for developing DM. Several longitudinal studies have found that TNF inhibitors improve insulin resistance.13,14 Dandona et al15 reported a considerable decrease of TNF-α levels with the concurrent restoration of insulin sensitivity during weight loss.

Pereira et al16 found a notable connection between psoriasis, DM, and insulin resistance with an odds ratio of 2.63 of abnormal glucose homeostasis in patients with psoriasis compared to controls. Yazdani-Biuki et al17 proved that extended administration of anti–TNF-α antibody was able to improve insulin sensitivity in insulin-resistant patients. The same finding was established by Kiortsis et al.14

In this prospective controlled study, we evaluated the effects of anti-TNF agents on insulin resistance and sensitivity in psoriasis patients with type 2 DM treated with anti-TNF agents.

Methods

A total of 70 patients attending the dermatological outpatient clinics at Farwaniya Hospital (Kuwait City, Kuwait) between January 2012 and September 2014 were enrolled in the study and were randomly distributed into 2 equal groups (n=35 each). The study was approved by the hospital ethics committee. Patients were included in the study if they had moderate to severe psoriasis (ie, psoriasis area severity index score ≥10) with documented type 2 DM and high fasting plasma glucose (FPG) levels (ie, >10 mmol/L). Patients who were currently being treated with oral hypoglycemic agents but not insulin therapy were included in the study. Patients were excluded if they had phototherapy within the last 4 weeks, prior biologic therapy, current and prior insulin therapy, a change in oral hypoglycemic drug dosage in the last 2 months, other serious systemic illness (eg, malignancy, hepatitis B or C virus, metabolic or endocrine disease), and/or abnormal laboratory investigations (eg, liver/kidney profile, chest radiograph abnormality, positive Mantoux test). All of the patients enrolled in the study provided informed consent and underwent routine baseline investigations including complete blood cell counts, general health profile, chest radiograph, antinuclear antibody test, Mantoux test, FPG and insulin levels, glycated hemoglobin (HbA1C), homeostasis model assessment (HOMA), routine urine examination, and enzyme-linked immunosorbent assay for tuberculosis. The study group was treated with anti-TNF agents, and the control group received conventional antipsoriatic medications.

All the patients included in the study had high FPG levels (ie, >10 mmol/L) at the time of enrollment and were currently being treated with oral hypoglycemic agents. The dose of oral hypoglycemic agents was unchanged for at least 2 months before entry into the run-in period and throughout the 24-week study period. Demographic details including sex, age, medical history (eg, type of psoriasis, prior and concomitant treatments) were collected from the participants’ clinical histories. Participants from both groups were appropriately matched in terms of age, sex, body weight, body mass index, and duration of type 2 DM (Table 1). The primary end point of the study was to analyze and compare clinical and serum data collected at baseline and after 24 weeks of therapy.

A complete biochemical profile was repeated in both groups after 24 weeks of treatment. Each participant underwent a baseline short insulin sensitivity test immediately before treatment and at 4 and 24 weeks of treatment. We assessed insulin resistance via HOMA, calculated as follows: FPG [mmol/L] × fasting serum insulin [pmol/L] / 22.5.18 Oral glucose tolerance tests were performed to calculate the HOMA of insulin resistance. Serum insulin concentration was determined via enzyme-linked immunosorbent assay.

 

 

Statistical analysis was performed using SPSS software (version 12.0). Continuous patient characteristics were analyzed using mean and SD as well as discrete data as counts and proportions. Association was examined using χ2 tests for categorical variables and 2-sided t test/Wilcoxon rank sum test for continuous variables. Analysis of variance was used to compare the results in 3 different anti-TNF agents used in the study group.

Results

Of the 35 participants enrolled in the study group, 34 (97.1%) completed the study and were evaluated. The study group included 16 men and 18 women aged 19 to 63 years (mean age [SD], 43.7 [21.6] years) who were treated with TNF-α inhibitors—8 participants with etanercept, 14 with adalimumab, and 12 with infliximab—according to the standard dosage schedule for 24 weeks.

Of the 35 participants enrolled in the control group, 29 (82.9%) completed the study and were evaluated. Six patients did not follow up for the complete duration of the 24-week study period and were not evaluated. The control group included 14 men and 15 women aged 18 to 65 years (mean age [SD], 47.7 [14.2] years) who were treated with other systemic therapies—8 participants with topical corticosteroids or calcipotriol only, 7 with cyclosporine A, and 14 with methotrexate. The dose of the drug was kept stable throughout the 24-week study period.

Demographic and baseline characteristics for all participants are shown in Table 1. There were no significant differences in demographic or baseline characteristics among the study group versus the control group, and all participants were similar in age; body mass index; as well as FPG, fasting insulin, and HbA1C levels.

At baseline, both study and control participants had elevated mean (SD) FPG levels (10 [25] mmol/L and 11 [0.4] mmol/L, respectively), fasting insulin levels (2.79 [0.17] pmol/L and 2.82 [0.13] pmol/L, respectively), and HbA1C levels (8.4% [0.38%] and 8.1% [0.21%], respectively)(Table 1).

The study group showed significant improvements in glycemic control at the end of the study (Table 2). At week 24, study group participants had a mean (SD) decrease in FPG levels of 2.74 (0.34) mmol/L versus 0.02 (0.16) mmol/L in the control group. This difference between the 2 groups after 24 weeks was found to be statistically significant (P<.01). On further analysis of the study group, no statistically significant difference (P>.01) was noted in the 3 anti-TNF agents used. Compared to the control group, the study group showed a significant decrease from baseline values of FPG and HbA1C (P<.01). Fasting insulin levels decreased significantly for study group participants as compared with control (–1.91 pmol/L vs 0.04 pmol/L)(P<.001)(Table 2). However, on analysis of the 3 anti-TNF agents, no statistically significant difference was found (P>.05). Participants in the control group showed no significant change in fasting insulin and FPG levels.

To confirm that there was a change in insulin sensitivity in response to TNF-α inhibitors, we analyzed FPG and fasting insulin values using the HOMA method. There was no change in mean relative insulin resistance in the control group in response to therapy (mean [SD], 5.4 [0.31] vs 5.6 [0.15], before vs after therapy), while there was mild improvement in relative insulin resistance in the study group (5.9 [0.52] vs 4.8 [0.34], before vs after therapy). There also was a significant difference in the change in relative insulin resistance in response to treatment between the study and control groups (1.2 [0.40] vs –0.3 [0.12]; P<.01)(Table 2).

Comment

There has been an unprecedented rise in the rate of obesity and associated metabolic diseases such as type 2 DM. Following the current trend, it is estimated that the world will have approximately 592 million cases of type 2 DM by the year 2035.19 Almost two-thirds of these patients are estimated to die of cardiovascular diseases.

Although the pathophysiology of type 2 DM is not known, insulin resistance in the muscles and liver as well as failure of pancreatic β cells represent the core of the complex pathophysiology. The associated underlying silent inflammation was thought to have a key role in both insulin resistance and insulin secretory defects seen in type 2 DM. Furthermore, recent data suggest the central role of TNF-α, IL-1, and IL-6 pathways in this inflammation.10 Tumor necrosis factor α has been shown to have a dual effect on insulin resistance as well as pancreatic β cell function. It blocks the function of insulin at the receptor level and has been implicated as a causative factor in obesity-associated insulin resistance and also in the pathogenesis of type 2 DM.20,21 Furthermore, cytokines that activate nuclear factor κβ (a nuclear transcription factor closely involved in the regulation of cellular inflammatory response), such as TNF-α, are thought to be a common denominator for β-cell apoptosis in types 1 and 2 DM.22 Additionally, it has been suggested that TNF-α is a powerful regulator of adipose tissue.23 Neutralizing TNF-α in obese Zucker rats has shown increased insulin sensitivity.3 Tumor necrosis factor α and IL-6 as well as C-reactive protein and plasminogen activator inhibitor 1 are negatively associated with insulin sensitivity.24-27 These findings have led researchers to investigate the role of anti-TNF agents for the management of type 2 DM.28

 

 

Psoriasis has now come to be known as a systemic inflammatory disorder and is associated with increased expression of TNF-α. It predisposes patients to insulin resistance and places them at higher risk for developing DM.11,12 Systematic reports recommend that there is a link between psoriasis and DM featured by helper T cell (TH1) cytokines.29 This link can stimulate insulin resistance and metabolic syndrome as well as inflammatory cytokines identified to motivate psoriasis.29,30 The association between psoriasis and type 2 DM proposes a possible pathophysiologic connection between the 2 diseases. Patients with psoriasis have altered T-cell subtype 1 pathways and dysregulated oxidative and angiogenic mechanisms.31,32 Many of these immune pathways may similarly predispose psoriasis patients to impaired glucose tolerance and DM. Inflammation may cause insulin resistance and DM through numerous mechanisms. Systemic inflammation linked with psoriasis may lead to high levels of circulating IL-1, IL-6, and TNF-α that predispose patients to impaired glucose tolerance and type 2 DM.33

Several longitudinal investigations have found that TNF inhibitors improve insulin resistance.13,14,34-38 Gonzalez-Gay et al13 confirmed a rapid beneficial effect of infliximab on insulin resistance and insulin sensitivity in rheumatoid arthritis (RA) patients, which might support the long-term use of drugs that act by blocking TNF-α to diminish the mechanisms implicated in the development of atherosclerosis in patients with RA. Kiortsis et al14 performed a complete biochemical profile before and after 6 months of treatment with infliximab in 17 patients with ankylosing spondylitis and 28 patients with RA. The researchers found a significant decrease of the HOMA index in the percentile of their patients with the highest insulin resistance (P<.01).14

Stagakis et al34 found that 12 weeks of treatment with anti-TNF agents may improve insulin resistance in patients with active RA and high insulin resistance. Treatment with anti-TNF agents was shown to restore the phosphorylation status of serine phosphorylation of insulin receptor substrate 1 (Ser312-IRS-1) and AKT (protein kinase B), which are important mediators in the insulin signaling cascade. The investigators concluded that treatment with anti-TNF agents may improve insulin resistance and sensitivity in RA patients with active disease and high insulin resistance.34

Solomon et al35 studied the link between disease-modifying antirheumatic drugs and DM risk in patients with RA and psoriasis. The authors proposed that initiation of treatment with TNF inhibitors in psoriasis patients was associated with a reduced incidence of DM. The results showed a lower risk for developing DM in patients with psoriasis who were treated with a TNF inhibitor compared with numerous other drugs.35

Marra et al36 studied the effects of etanercept on insulin sensitivity in 9 patients with psoriasis. They reported a decrease in insulin resistance evaluated by HOMA after 24 weeks of etanercept treatment.36 Wambier et al37 reported severe hypoglycemia after initiation of anti-TNF therapy with etanercept in a patient with generalized pustular psoriasis and type 2 DM.

Yazdani-Biuki et al38 reported the case of a patient who demonstrated a relapse of type 2 DM after an interruption of prolonged treatment with infliximab, an anti–TNF-α antibody for psoriatic arthritis. The improvement in insulin sensitivity of this patient has been reported along with post hoc evidence that chronic administration of infliximab improves insulin resistance in a small sample of patients with inflammatory joint diseases.17

Other studies on the effects of TNF inhibitors on insulin resistance and sensitivity have yielded conflicting results. Martínez-Abundis et al39 studied the effects of etanercept on insulin resistance and sensitivity in a randomized trial of psoriatic patients at risk for developing type 2 DM. Results indicated that anti-TNF therapy had no significant influence on insulin sensitivity measured using a hyperinsulinemic clamp during 2 weeks of etanercept treatment in psoriatic patients with risk factors for type 2 DM. The explanation of this discrepancy may be due to the short duration of the study period.

It is still unknown if psoriasis treatment affects a patient’s risk for developing DM. However, Solomon et al35 evaluated the association of incidental DM among patients with prescribed TNF inhibitors or methotrexate and proposed that initiation of treatment with TNF inhibitors was associated with a diminished incidence of DM.

Our study supports and confirms that psoriasis patients treated with TNF-α inhibitors showed improved glycemic indices and insulin resistance compared with control patients treated with other common systemic drugs for psoriasis. We did not take into consideration other conventional risk factors such as hypertension and coronary artery disease. The number of participants included in the current study was not large enough to evaluate each of the anti-TNF agents in a separate group, and participants were not followed up long enough to see the impact of the biochemical changes noted in the results on long-term morbidity or mortality.

 

 

Conclusion

Our study confirms a beneficial effect of TNF-α inhibitors on insulin resistance and insulin sensitivity in psoriasis patients with type 2 DM. Treatment with TNF-α inhibitors may have beneficial effects on insulin sensitivity in even the most insulin-resistant patients with psoriasis. The study results may support the hypothesis that long-term use of TNF inhibitors may reduce the mechanisms involved in the development of DM in patients with psoriasis. The improvement in insulin sensitivity may in turn decrease the coronary artery disease risk in these patients. Additional large, prospective, multicenter studies are required to further analyze the effects of anti–TNF-α antibodies on insulin sensitivity and β cell function in insulin-resistant or diabetic psoriasis patients.

References
  1. Lowes MA, Bowcock AM, Krueger JG. Pathogenesis and therapy of psoriasis. Nature. 2007;445:866-873.
  2. Boehncke WH, Prinz J, Gottlieb AB. Biologic therapies for psoriasis. a systematic review. J Rheumatol. 2006;33:1447-1451.
  3. DeFronzo RA, Bonadonna RC, Ferrannini E. Pathogenesis of NIDDM. a balanced overview. Diabetes Care. 1992;15:318-368.
  4. DeFronzo RA. Pathogenesis of type 2 diabetes: metabolic and molecular implications for identifying diabetes genes. Diabetes Rev. 1997;4:177-269.
  5. Reaven GM. Banting Lecture 1988. role of insulin resistance in human disease. Diabetes. 1988;37:1595-1607.
  6. Erkelens DW. Insulin resistance syndrome and type 2 diabetes mellitus. Am J Cardiol. 2001;88(7B):38J-42J.
  7. Hotamisligil GS, Shargill NS, Spiegelman BM. Adipose expression of tumor necrosis factor alpha: direct role in obesity-linked insulin resistance. Science. 1993;259:87-91.
  8. Hotamisligil GS, Spiegelman BM. Tumor necrosis factor alpha: a key component of the obesity-diabetes link. Diabetes. 1994;43:1271-1278.
  9. Van der Poll T, Romijn JA, Endert E, et al. Tumor necrosis factor mimics the metabolic response to acute infection in healthy humans. Am J Physiol. 1991;261:457-465.
  10. Tabas I, Glass CK. Anti-inflammatory therapy in chronic disease: challenges and opportunities. Science. 2013;339:166-172.
  11. Qureshi AA, Choi HK, Setty AR, et al. Psoriasis and the risk of diabetes and hypertension: a prospective study of US female nurses. Arch Dermatol. 2009;145:379-382.
  12. Solomon DH, Love TJ, Canning C, et al. Risk of diabetes among patients with rheumatoid arthritis, psoriatic arthritis and psoriasis. Ann Rheum Dis. 2010;69:2114-2117.
  13. Gonzalez-Gay MA, De Matias JM, Gonzalez-Juanatey C, et al. Anti-tumor necrosis factor-alpha blockade improves insulin resistance in patients with rheumatoid arthritis. Clin Exp Rheumatol. 2006;24:83-86.
  14. Kiortsis DN, Mavridis AK, Vasakos S, et al. Effects of infliximab treatment on insulin resistance in patients with rheumatoid arthritis and ankylosing spondylitis. Ann Rheum Dis. 2005;64:765-766.
  15. Dandona P, Weinstock R, Thusu K, et al. Tumor necrosis factor-α in sera of obese patients: fall with weight loss. J Clin Endocrinol Metabol. 1998;83:2907-2910.
  16. Pereira RR, Amladi ST, Varthakavi PK. A study of the prevalence of diabetes, insulin resistance, lipid abnormalities, and cardiovascular risk factors in patients with chronic plaque psoriasis. Ind J Dermatol. 2011;56:520-526.
  17. Yazdani-Biuki B, Stelzl H, Brezinschek HP, et al. Improvement of insulin sensitivity in insulin resistant subjects during prolonged treatment with the anti-TNF-α antibody infliximab. Eur J Clin Invest. 2004;34:641-642.
  18. Bonora E, Kiechl S, Willeit J, et al. Prevalence of insulin resistance in metabolic disorders. The Bruneck Study. Diabetes. 1998;47:1643-1649.
  19. Ryden L, Grant PJ, Anker SD, et al. ESC Guidelines on diabetes, prediabetes, and cardiovascular diseases developed in collaboration with the EASD: the Task Force on diabetes, prediabetes, and cardiovascular diseases of the European Society of Cardiology (ESC) and developed in collaboration with the European Association for the Study of Diabetes (EASD). Eur Heart J. 2013;34:3035-3087.
  20. Peraldi P, Spiegelman B. TNF-alpha and insulin resistance: summary and future prospects. Mol Cell Biochem. 1998;182:169-175.
  21. Moller DE. Potential role of TNF-alpha in the pathogenesis of insulin resistance and type 2 diabetes. Trends Endocrinol Metab. 2000;11:212-217.
  22. Mandrup-Poulsen T. Apoptotic signal transduction pathways in diabetes. Biochem Pharmacol. 2003;66:1433-1440.
  23. Coppack SW. Pro-inflammatory cytokines and adipose tissue. Proc Nutr Soc. 2001;60:349-356.
  24. Pradhan AD, Manson JE, Rifai N, et al. C-reactive protein, interleukin 6, and risk of developing type 2 diabetes mellitus. JAMA. 2001;286:327-334.
  25. Festa A, D’Agostino R Jr, Tracy RP, et al. Insulin Resistance Atherosclerosis Study. elevated levels of acute-phase proteins and plasminogen activator inhibitor-1 predict the development of type 2 diabetes: the insulin resistance atherosclerosis study. Diabetes. 2002;51:1131-1137.
  26. Meigs JB, O’Donnell CJ, Tofler GH, et al. Hemostatic markers of endothelial dysfunction and risk of incident type 2 diabetes: the Framingham Offspring Study. Diabetes. 2006;55:530-537.
  27. Liu S, Tinker L, Song Y, et al. A prospective study of inflammatory cytokines and diabetes mellitus in a multiethnic cohort of postmenopausal women. Arch Intern Med. 2007;167:1676-1685.
  28. Esser N, Paquot N, Scheen AJ. Anti-inflammatory agents to treat or prevent type 2 diabetes, metabolic syndrome and cardiovascular disease. Exp Opin Invest Drugs. 2014;24:1-25.
  29. Wellen KE, Hotamisligil GS. Inflammation, stress, and diabetes. J Clin Invest. 2005;115:1111-1119.
  30. Karadag AS, Yavuz B, Ertugrul DT, et al. Is psoriasis a pre-atherosclerotic disease? increased insulin resistance and impaired endothelial function in patients with psoriasis. Int J Dermatol. 2010;49:642-646.
  31. Armstrong AW, Voyles SV, Armstrong EJ, et al. A tale of two plaques: convergent mechanisms of T-cell–mediated inflammation in psoriasis and atherosclerosis. Exp Dermatol. 2011;20:544-549.
  32. Armstrong AW, Voyles SV, Armstrong EJ, et al. Angiogenesis and oxidative stress: common mechanisms linking psoriasis with atherosclerosis. J Dermatol Sci. 2011;63:1-9.
  33. Boehncke S, Thaci D, Beschmann H, et al. Psoriasis patients show signs of insulin resistance. Br J Dermatol. 2007;157:1249-1251.
  34. Stagakis I, Bertsias G, Karvounaris S, et al. Anti-tumor necrosis factor therapy improves insulin resistance, beta cell function and insulin signaling in active rheumatoid arthritis patients with high insulin resistance. Arthritis Res Ther. 2012;14:R141.
  35. Solomon DH, Massarotti E, Garg R, et al. Association between disease-modifying antirheumatic drugs and diabetes risk in patients with rheumatoid arthritis and psoriasis. JAMA. 2011;305:2525-2531.
  36. Marra M, Campanati A, Testa R, et al. Effect of etanercept on insulin sensitivity in nine patients with psoriasis. Int J Immunopathol Pharmacol. 2007;20:731-736.
  37. Wambier CG, Foss-Freitas MC, Paschoal RS, et al. Severe hypoglycemia after initiation of anti-tumor necrosis factor therapy with etanercept in a patient with generalized pustular psoriasis and type 2 diabetes mellitus. J Am Acad Dermatol. 2009;60:883-885.
  38. Yazdani-Biuki B, Mueller T, Brezinschek HP, et al. Relapse of diabetes after interruption of chronic administration of anti-tumor necrosis factor-alpha antibody infliximab: a case observation. Diabetes Care. 2006;29:1712.
  39. Martínez-Abundis E, Reynoso-von Drateln C, Hernández-Salazar E, et al. Effect of etanercept on insulin secretion and insulin sensitivity in a randomized trial with psoriatic patients at risk for developing type 2 diabetes mellitus. Arch Dermatol Res. 2007;299:461-465.
References
  1. Lowes MA, Bowcock AM, Krueger JG. Pathogenesis and therapy of psoriasis. Nature. 2007;445:866-873.
  2. Boehncke WH, Prinz J, Gottlieb AB. Biologic therapies for psoriasis. a systematic review. J Rheumatol. 2006;33:1447-1451.
  3. DeFronzo RA, Bonadonna RC, Ferrannini E. Pathogenesis of NIDDM. a balanced overview. Diabetes Care. 1992;15:318-368.
  4. DeFronzo RA. Pathogenesis of type 2 diabetes: metabolic and molecular implications for identifying diabetes genes. Diabetes Rev. 1997;4:177-269.
  5. Reaven GM. Banting Lecture 1988. role of insulin resistance in human disease. Diabetes. 1988;37:1595-1607.
  6. Erkelens DW. Insulin resistance syndrome and type 2 diabetes mellitus. Am J Cardiol. 2001;88(7B):38J-42J.
  7. Hotamisligil GS, Shargill NS, Spiegelman BM. Adipose expression of tumor necrosis factor alpha: direct role in obesity-linked insulin resistance. Science. 1993;259:87-91.
  8. Hotamisligil GS, Spiegelman BM. Tumor necrosis factor alpha: a key component of the obesity-diabetes link. Diabetes. 1994;43:1271-1278.
  9. Van der Poll T, Romijn JA, Endert E, et al. Tumor necrosis factor mimics the metabolic response to acute infection in healthy humans. Am J Physiol. 1991;261:457-465.
  10. Tabas I, Glass CK. Anti-inflammatory therapy in chronic disease: challenges and opportunities. Science. 2013;339:166-172.
  11. Qureshi AA, Choi HK, Setty AR, et al. Psoriasis and the risk of diabetes and hypertension: a prospective study of US female nurses. Arch Dermatol. 2009;145:379-382.
  12. Solomon DH, Love TJ, Canning C, et al. Risk of diabetes among patients with rheumatoid arthritis, psoriatic arthritis and psoriasis. Ann Rheum Dis. 2010;69:2114-2117.
  13. Gonzalez-Gay MA, De Matias JM, Gonzalez-Juanatey C, et al. Anti-tumor necrosis factor-alpha blockade improves insulin resistance in patients with rheumatoid arthritis. Clin Exp Rheumatol. 2006;24:83-86.
  14. Kiortsis DN, Mavridis AK, Vasakos S, et al. Effects of infliximab treatment on insulin resistance in patients with rheumatoid arthritis and ankylosing spondylitis. Ann Rheum Dis. 2005;64:765-766.
  15. Dandona P, Weinstock R, Thusu K, et al. Tumor necrosis factor-α in sera of obese patients: fall with weight loss. J Clin Endocrinol Metabol. 1998;83:2907-2910.
  16. Pereira RR, Amladi ST, Varthakavi PK. A study of the prevalence of diabetes, insulin resistance, lipid abnormalities, and cardiovascular risk factors in patients with chronic plaque psoriasis. Ind J Dermatol. 2011;56:520-526.
  17. Yazdani-Biuki B, Stelzl H, Brezinschek HP, et al. Improvement of insulin sensitivity in insulin resistant subjects during prolonged treatment with the anti-TNF-α antibody infliximab. Eur J Clin Invest. 2004;34:641-642.
  18. Bonora E, Kiechl S, Willeit J, et al. Prevalence of insulin resistance in metabolic disorders. The Bruneck Study. Diabetes. 1998;47:1643-1649.
  19. Ryden L, Grant PJ, Anker SD, et al. ESC Guidelines on diabetes, prediabetes, and cardiovascular diseases developed in collaboration with the EASD: the Task Force on diabetes, prediabetes, and cardiovascular diseases of the European Society of Cardiology (ESC) and developed in collaboration with the European Association for the Study of Diabetes (EASD). Eur Heart J. 2013;34:3035-3087.
  20. Peraldi P, Spiegelman B. TNF-alpha and insulin resistance: summary and future prospects. Mol Cell Biochem. 1998;182:169-175.
  21. Moller DE. Potential role of TNF-alpha in the pathogenesis of insulin resistance and type 2 diabetes. Trends Endocrinol Metab. 2000;11:212-217.
  22. Mandrup-Poulsen T. Apoptotic signal transduction pathways in diabetes. Biochem Pharmacol. 2003;66:1433-1440.
  23. Coppack SW. Pro-inflammatory cytokines and adipose tissue. Proc Nutr Soc. 2001;60:349-356.
  24. Pradhan AD, Manson JE, Rifai N, et al. C-reactive protein, interleukin 6, and risk of developing type 2 diabetes mellitus. JAMA. 2001;286:327-334.
  25. Festa A, D’Agostino R Jr, Tracy RP, et al. Insulin Resistance Atherosclerosis Study. elevated levels of acute-phase proteins and plasminogen activator inhibitor-1 predict the development of type 2 diabetes: the insulin resistance atherosclerosis study. Diabetes. 2002;51:1131-1137.
  26. Meigs JB, O’Donnell CJ, Tofler GH, et al. Hemostatic markers of endothelial dysfunction and risk of incident type 2 diabetes: the Framingham Offspring Study. Diabetes. 2006;55:530-537.
  27. Liu S, Tinker L, Song Y, et al. A prospective study of inflammatory cytokines and diabetes mellitus in a multiethnic cohort of postmenopausal women. Arch Intern Med. 2007;167:1676-1685.
  28. Esser N, Paquot N, Scheen AJ. Anti-inflammatory agents to treat or prevent type 2 diabetes, metabolic syndrome and cardiovascular disease. Exp Opin Invest Drugs. 2014;24:1-25.
  29. Wellen KE, Hotamisligil GS. Inflammation, stress, and diabetes. J Clin Invest. 2005;115:1111-1119.
  30. Karadag AS, Yavuz B, Ertugrul DT, et al. Is psoriasis a pre-atherosclerotic disease? increased insulin resistance and impaired endothelial function in patients with psoriasis. Int J Dermatol. 2010;49:642-646.
  31. Armstrong AW, Voyles SV, Armstrong EJ, et al. A tale of two plaques: convergent mechanisms of T-cell–mediated inflammation in psoriasis and atherosclerosis. Exp Dermatol. 2011;20:544-549.
  32. Armstrong AW, Voyles SV, Armstrong EJ, et al. Angiogenesis and oxidative stress: common mechanisms linking psoriasis with atherosclerosis. J Dermatol Sci. 2011;63:1-9.
  33. Boehncke S, Thaci D, Beschmann H, et al. Psoriasis patients show signs of insulin resistance. Br J Dermatol. 2007;157:1249-1251.
  34. Stagakis I, Bertsias G, Karvounaris S, et al. Anti-tumor necrosis factor therapy improves insulin resistance, beta cell function and insulin signaling in active rheumatoid arthritis patients with high insulin resistance. Arthritis Res Ther. 2012;14:R141.
  35. Solomon DH, Massarotti E, Garg R, et al. Association between disease-modifying antirheumatic drugs and diabetes risk in patients with rheumatoid arthritis and psoriasis. JAMA. 2011;305:2525-2531.
  36. Marra M, Campanati A, Testa R, et al. Effect of etanercept on insulin sensitivity in nine patients with psoriasis. Int J Immunopathol Pharmacol. 2007;20:731-736.
  37. Wambier CG, Foss-Freitas MC, Paschoal RS, et al. Severe hypoglycemia after initiation of anti-tumor necrosis factor therapy with etanercept in a patient with generalized pustular psoriasis and type 2 diabetes mellitus. J Am Acad Dermatol. 2009;60:883-885.
  38. Yazdani-Biuki B, Mueller T, Brezinschek HP, et al. Relapse of diabetes after interruption of chronic administration of anti-tumor necrosis factor-alpha antibody infliximab: a case observation. Diabetes Care. 2006;29:1712.
  39. Martínez-Abundis E, Reynoso-von Drateln C, Hernández-Salazar E, et al. Effect of etanercept on insulin secretion and insulin sensitivity in a randomized trial with psoriatic patients at risk for developing type 2 diabetes mellitus. Arch Dermatol Res. 2007;299:461-465.
Issue
Cutis - 97(3)
Issue
Cutis - 97(3)
Page Number
235-241
Page Number
235-241
Publications
Publications
Topics
Article Type
Display Headline
Effects of Tumor Necrosis Factor α Inhibitors Extend Beyond Psoriasis: Insulin Sensitivity in Psoriasis Patients With Type 2 Diabetes Mellitus
Display Headline
Effects of Tumor Necrosis Factor α Inhibitors Extend Beyond Psoriasis: Insulin Sensitivity in Psoriasis Patients With Type 2 Diabetes Mellitus
Legacy Keywords
diabetes mellitus; psoriasis; biologics
Legacy Keywords
diabetes mellitus; psoriasis; biologics
Sections
Inside the Article

Practice Points

  • Psoriasis is associated with an increased incidence of insulin resistance and type 2 diabetes mellitus (DM).
  • Anti–tumor necrosis factor drugs, which are effective for the treatment of psoriasis, were found to improve insulin resistance in psoriasis patients with type 2 DM.
Disallow All Ads
Article PDF Media

Predicting Readmissions from EHR Data

Article Type
Changed
Mon, 01/02/2017 - 19:34
Display Headline
Predicting all‐cause readmissions using electronic health record data from the entire hospitalization: Model development and comparison

Unplanned hospital readmissions are frequent, costly, and potentially avoidable.[1, 2] Due to major federal financial readmissions penalties targeting excessive 30‐day readmissions, there is increasing attention to implementing hospital‐initiated interventions to reduce readmissions.[3, 4] However, universal enrollment of all hospitalized patients into such programs may be too resource intensive for many hospitals.[5] To optimize efficiency and effectiveness, interventions should be targeted to individuals most likely to benefit.[6, 7] However, existing readmission risk‐prediction models have achieved only modest discrimination, have largely used administrative claims data not available until months after discharge, or are limited to only a subset of patients with Medicare or a specific clinical condition.[8, 9, 10, 11, 12, 13, 14] These limitations have precluded accurate identification of high‐risk individuals in an all‐payer general medical inpatient population to provide actionable information for intervention prior to discharge.

Approaches using electronic health record (EHR) data could allow early identification of high‐risk patients during the index hospitalization to enable initiation of interventions prior to discharge. To date, such strategies have relied largely on EHR data from the day of admission.[15, 16] However, given that variation in 30‐day readmission rates are thought to reflect the quality of in‐hospital care, incorporating EHR data from the entire hospital stay to reflect hospital care processes and clinical trajectory may more accurately identify at‐risk patients.[17, 18, 19, 20] Improved accuracy in risk prediction would help better target intervention efforts in the immediate postdischarge period, an interval characterized by heightened vulnerability for adverse events.[21]

To help hospitals target transitional care interventions more effectively to high‐risk individuals prior to discharge, we derived and validated a readmissions risk‐prediction model incorporating EHR data from the entire course of the index hospitalization, which we termed the full‐stay EHR model. We also compared the full‐stay EHR model performance to our group's previously derived prediction model based on EHR data on the day of admission, termed the first‐day EHR model, as well as to 2 other validated readmission models similarly intended to yield near real‐time risk predictions prior to or shortly after hospital discharge.[9, 10, 15]

METHODS

Study Design, Population, and Data Sources

We conducted an observational cohort study using EHR data from 6 hospitals in the DallasFort Worth metroplex between November 1, 2009 and October 30, 2010 using the same EHR system (Epic Systems Corp., Verona, WI). One site was a university‐affiliated safety net hospital; the remaining 5 sites were teaching and nonteaching community sites.

We included consecutive hospitalizations among adults 18 years old discharged alive from any medicine inpatient service. For individuals with multiple hospitalizations during the study period, we included only the first hospitalization. We excluded individuals who died during the index hospitalization, were transferred to another acute care facility, left against medical advice, or who died outside of the hospital within 30 days of discharge. For model derivation, we randomly split the sample into separate derivation (50%) and validation cohorts (50%).

Outcomes

The primary outcome was 30‐day hospital readmission, defined as a nonelective hospitalization within 30 days of discharge to any of 75 acute care hospitals within a 100‐mile radius of Dallas, ascertained from an all‐payer regional hospitalization database. Nonelective hospitalizations included all hospitalizations classified as a emergency, urgent, or trauma, and excluded those classified as elective as per the Centers for Medicare and Medicaid Services Claim Inpatient Admission Type Code definitions.

Predictor Variables for the Full‐Stay EHR Model

The full‐stay EHR model was iteratively developed from our group's previously derived and validated risk‐prediction model using EHR data available on admission (first‐day EHR model).[15] For the full‐stay EHR model, we included all predictor variables included in our published first‐day EHR model as candidate risk factors. Based on prior literature, we additionally expanded candidate predictors available on admission to include marital status (proxy for social isolation) and socioeconomic disadvantage (percent poverty, unemployment, median income, and educational attainment by zip code of residence as proxy measures of the social and built environment).[22, 23, 24, 25, 26, 27] We also expanded the ascertainment of prior hospitalization to include admissions at both the index hospital and any of 75 acute care hospitals from the same, separate all‐payer regional hospitalization database used to ascertain 30‐day readmissions.

Candidate predictors from the remainder of the hospital stay (ie, following the first 24 hours of admission) were included if they were: (1) available in the EHR of all participating hospitals, (2) routinely collected or available at the time of hospital discharge, and (3) plausible predictors of adverse outcomes based on prior literature and clinical expertise. These included length of stay, in‐hospital complications, transfer to an intensive or coronary care unit, blood transfusions, vital sign instabilities within 24 hours of discharge, select laboratory values at time of discharge, and disposition status. We also assessed trajectories of vital signs and selected laboratory values (defined as changes in these measures from admission to discharge).

Statistical Analysis

Model Derivation

Univariate relationships between readmission and each of the candidate predictors were assessed in the derivation cohort using a prespecified significance threshold of P 0.05. We included all factors from our previously derived and validated first‐day EHR model as candidate predictors.[15] Continuous laboratory and vital sign values at the time of discharge were categorized based on clinically meaningful cutoffs; predictors with missing values were assumed to be normal (<1% missing for each variable). Significant univariate candidate variables were entered in a multivariate logistic regression model using stepwise backward selection with a prespecified significance threshold of P 0.05. We performed several sensitivity analyses to confirm the robustness of our model. First, we alternately derived the full‐stay model using stepwise forward selection. Second, we forced in all significant variables from our first‐day EHR model, and entered the candidate variables from the remainder of the hospital stay using both stepwise backward and forward selection separately. Third, prespecified interactions between variables were evaluated for inclusion. Though final predictors varied slightly between the different approaches, discrimination of each model was similar to the model derived using our primary analytic approach (C statistics 0.01, data not shown).

Model Validation

We assessed model discrimination and calibration of the derived full‐stay EHR model using the validation cohort. Model discrimination was estimated by the C statistic. The C statistic represents the probability that, given 2 hospitalized individuals (1 who was readmitted and the other who was not), the model will predict a higher risk for the readmitted patient than for the nonreadmitted patient. Model calibration was assessed by comparing predicted to observed probabilities of readmission by quintiles of risk, and with the Hosmer‐Lemeshow goodness‐of‐fit test.

Comparison to Existing Models

We compared the full‐stay EHR model performance to 3 previously published models: our group's first‐day EHR model, and the LACE (includes Length of stay, Acute (nonelective) admission status, Charlson Comorbidity Index, and Emergency department visits in the past year) and HOSPITAL (includes Hemoglobin at discharge, discharge from Oncology service, Sodium level at discharge, Procedure during index hospitalization, Index hospitalization Type (nonelective), number of Admissions in the past year, and Length of stay) models, which were both derived to predict 30‐day readmissions among general medical inpatients and were intended to help clinicians identify high‐risk patients to target for discharge interventions.[9, 10, 15] We assessed each model's performance in our validation cohort, calculating the C statistic, integrated discrimination index (IDI), and net reclassification index (NRI) compared to the full‐stay model. IDI is a summary measure of both discrimination and reclassification, where more positive values suggest improvement in model performance in both these domains compared to a reference model.[28] The NRI is defined as the sum of the net proportions of correctly reclassified persons with and without the event of interest.[29] The theoretical range of values is 2 to 2, with more positive values indicating improved net reclassification compared to a reference model. Here, we calculated a category‐based NRI to evaluate the performance of models in correctly classifying individuals with and without readmissions into the highest readmission risk quintile versus the lowest 4 risk quintiles compared to the full‐stay EHR model.[29] This prespecified cutoff is relevant for hospitals interested in identifying the highest‐risk individuals for targeted intervention.[6] Because some hospitals may be able to target a greater number of individuals for intervention, we performed a sensitivity analysis by assessing category‐based NRI for reclassification into the top 2 risk quintiles versus the lowest 3 risk quintiles and found no meaningful difference in our results (data not shown). Finally, we qualitatively assessed calibration of comparator models in our validation cohort by comparing predicted probability to observed probability of readmission by quintiles of risk for each model. We conducted all analyses using Stata 12.1 (StataCorp, College Station, TX). This study was approved by the UT Southwestern Medical Center institutional review board.

RESULTS

Overall, 32,922 index hospitalizations were included in our study cohort; 12.7% resulted in a 30‐day readmission (see Supporting Figure 1 in the online version of this article). Individuals had a mean age of 62 years and had diverse race/ethnicity and primary insurance status; half were female (Table 1). The study sample was randomly split into a derivation cohort (50%, n = 16,492) and validation cohort (50%, n = 16,430). Individuals in the derivation cohort with a 30‐day readmission had markedly different socioeconomic and clinical characteristics compared to those not readmitted (Table 1).

Baseline Characteristics and Candidate Variables for Risk‐Prediction Model
Entire Cohort, N = 32,922 Derivation Cohort, N = 16,492

No Readmission, N = 14,312

Readmission, N = 2,180

P Value
  • NOTE: Abbreviations: ED, emergency department; ICU, intensive care unit; IQR, interquartile range; SD, standard deviation. *20% poverty in zip code as per high poverty area US Census designation. Prior ED visit at site of index hospitalization within the past year. Prior hospitalization at any of 75 acute care hospitals in the North Texas region within the past year. Nonelective admission defined as hospitalization categorized as medical emergency, urgent, or trauma. ∥Calculated from diagnoses available within 1 year prior to index hospitalization. Conditions were considered complications if they were not listed as a principle diagnosis for hospitalization or as a previous diagnosis in the prior year. #On day of discharge or last known observation before discharge. Instabilities were defined as temperature 37.8C, heart rate >100 beats/minute, respiratory rate >24 breaths/minute, systolic blood pressure 90 mm Hg, or oxygen saturation <90%. **Discharges to nursing home, skilled nursing facility, or long‐term acute care hospital.

Demographic characteristics
Age, y, mean (SD) 62 (17.3) 61 (17.4) 64 (17.0) 0.001
Female, n (%) 17,715 (53.8) 7,694 (53.8) 1,163 (53.3) 0.72
Race/ethnicity 0.001
White 21,359 (64.9) 9,329 (65.2) 1,361 (62.4)
Black 5,964 (18.1) 2,520 (17.6) 434 (19.9)
Hispanic 4,452 (13.5) 1,931 (13.5) 338 (15.5)
Other 1,147 (3.5) 532 (3.7) 47 (2.2)
Marital status, n (%) 0.001
Single 8,076 (24.5) 3,516 (24.6) 514 (23.6)
Married 13,394 (40.7) 5,950 (41.6) 812 (37.3)
Separated/divorced 3,468 (10.5) 1,460 (10.2) 251 (11.5)
Widowed 4,487 (13.7) 1,868 (13.1) 388 (17.8)
Other 3,497 (10.6) 1,518 (10.6) 215 (9.9)
Primary payer, n (%) 0.001
Private 13,090 (39.8) 5,855 (40.9) 726 (33.3)
Medicare 13,015 (39.5) 5,597 (39.1) 987 (45.3)
Medicaid 2,204 (6.7) 852 (5.9) 242 (11.1)
Charity, self‐pay, or other 4,613 (14.0) 2,008 (14.0) 225 (10.3)
High‐poverty neighborhood, n (%)* 7,468 (22.7) 3,208 (22.4) 548 (25.1) 0.001
Utilization history
1 ED visits in past year, n (%) 9,299 (28.2) 3,793 (26.5) 823 (37.8) 0.001
1 hospitalizations in past year, n (%) 10,189 (30.9) 4,074 (28.5) 1,012 (46.4) 0.001
Clinical factors from first day of hospitalization
Nonelective admission, n (%) 27,818 (84.5) 11,960 (83.6) 1,960 (89.9) 0.001
Charlson Comorbidity Index, median (IQR)∥ 0 (01) 0 (00) 0 (03) 0.001
Laboratory abnormalities within 24 hours of admission
Albumin <2 g/dL 355 (1.1) 119 (0.8) 46 (2.1) 0.001
Albumin 23 g/dL 4,732 (14.4) 1,956 (13.7) 458 (21.0) 0.001
Aspartate aminotransferase >40 U/L 4,610 (14.0) 1,922 (13.4) 383 (17.6) 0.001
Creatine phosphokinase <60 g/L 3,728 (11.3) 1,536 (10.7) 330 (15.1) 0.001
Mean corpuscular volume >100 fL/red cell 1,346 (4.1) 537 (3.8) 134 (6.2) 0.001
Platelets <90 103/L 912 (2.8) 357 (2.5) 116 (5.3) 0.001
Platelets >350 103/L 3,332 (10.1) 1,433 (10.0) 283 (13.0) 0.001
Prothrombin time >35 seconds 248 (0.8) 90 (0.6) 35 (1.6) 0.001
Clinical factors from remainder of hospital stay
Length of stay, d, median (IQR) 4 (26) 4 (26) 5 (38) 0.001
ICU transfer after first 24 hours, n (%) 988 (3.0) 408 (2.9) 94 (4.3) 0.001
Hospital complications, n (%)
Clostridium difficile infection 119 (0.4) 44 (0.3) 24 (1.1) 0.001
Pressure ulcer 358 (1.1) 126 (0.9) 46 (2.1) 0.001
Venous thromboembolism 301 (0.9) 112 (0.8) 34 (1.6) 0.001
Respiratory failure 1,048 (3.2) 463 (3.2) 112 (5.1) 0.001
Central line‐associated bloodstream infection 22 (0.07) 6 (0.04) 5 (0.23) 0.005
Catheter‐associated urinary tract infection 47 (0.14) 20 (0.14) 6 (0.28) 0.15
Acute myocardial infarction 293 (0.9) 110 (0.8) 32 (1.5) 0.001
Pneumonia 1,754 (5.3) 719 (5.0) 154 (7.1) 0.001
Sepsis 853 (2.6) 368 (2.6) 73 (3.4) 0.04
Blood transfusion during hospitalization, n (%) 4,511 (13.7) 1,837 (12.8) 425 (19.5) 0.001
Laboratory abnormalities at discharge#
Blood urea nitrogen >20 mg/dL, n (%) 10,014 (30.4) 4,077 (28.5) 929 (42.6) 0.001
Sodium <135 mEq/L, n (%) 4,583 (13.9) 1,850 (12.9) 440 (20.2) 0.001
Hematocrit 27 3,104 (9.4) 1,231 (8.6) 287 (13.2) 0.001
1 vital sign instability at discharge, n (%)# 6,192 (18.8) 2,624 (18.3) 525 (24.1) 0.001
Discharge location, n (%) 0.001
Home 23,339 (70.9) 10,282 (71.8) 1,383 (63.4)
Home health 3,185 (9.7) 1,356 (9.5) 234 (10.7)
Postacute care** 5,990 (18.2) 2,496 (17.4) 549 (25.2)
Hospice 408 (1.2) 178 (1.2) 14 (0.6)

Derivation and Validation of the Full‐Stay EHR Model for 30‐Day Readmission

Our final model included 24 independent variables, including demographic characteristics, utilization history, clinical factors from the first day of admission, and clinical factors from the remainder of the hospital stay (Table 2). The strongest independent predictor of readmission was hospital‐acquired Clostridium difficile infection (adjusted odds ratio [AOR]: 2.03, 95% confidence interval [CI] 1.18‐3.48); other hospital‐acquired complications including pressure ulcers and venous thromboembolism were also significant predictors. Though having Medicaid was associated with increased odds of readmission (AOR: 1.55, 95% CI: 1.31‐1.83), other zip codelevel measures of socioeconomic disadvantage were not predictive and were not included in the final model. Being discharged to hospice was associated with markedly lower odds of readmission (AOR: 0.23, 95% CI: 0.13‐0.40).

Final Full‐Stay EHR Model Predicting 30‐Day Readmissions (Derivation Cohort, N = 16,492)
Odds Ratio (95% CI)
Univariate Multivariate*
  • NOTE: Abbreviations: CI, confidence interval; ED, emergency department. *Values shown reflect adjusted odds ratios and 95% CI for each factor after adjustment for all other factors listed in the table.

Demographic characteristics
Age, per 10 years 1.08 (1.051.11) 1.07 (1.041.10)
Medicaid 1.97 (1.702.29) 1.55 (1.311.83)
Widow 1.44 (1.281.63) 1.27 (1.111.45)
Utilization history
Prior ED visit, per visit 1.08 (1.061.10) 1.04 (1.021.06)
Prior hospitalization, per hospitalization 1.30 (1.271.34) 1.16 (1.121.20)
Hospital and clinical factors from first day of hospitalization
Nonelective admission 1.75 (1.512.03) 1.42 (1.221.65)
Charlson Comorbidity Index, per point 1.19 (1.171.21) 1.06 (1.041.09)
Laboratory abnormalities within 24 hours of admission
Albumin <2 g/dL 2.57 (1.823.62) 1.52 (1.052.21)
Albumin 23 g/dL 1.68 (1.501.88) 1.20 (1.061.36)
Aspartate aminotransferase >40 U/L 1.37 (1.221.55) 1.21 (1.061.38)
Creatine phosphokinase <60 g/L 1.48 (1.301.69) 1.28 (1.111.46)
Mean corpuscular volume >100 fL/red cell 1.68 (1.382.04) 1.32 (1.071.62)
Platelets <90 103/L 2.20 (1.772.72) 1.56 (1.231.97)
Platelets >350 103/L 1.34 (1.171.54) 1.24 (1.081.44)
Prothrombin time >35 seconds 2.58 (1.743.82) 1.92 (1.272.90)
Hospital and clinical factors from remainder of hospital stay
Length of stay, per day 1.08 (1.071.09) 1.06 (1.041.07)
Hospital complications
Clostridium difficile infection 3.61 (2.195.95) 2.03 (1.183.48)
Pressure ulcer 2.43 (1.733.41) 1.64 (1.152.34)
Venous thromboembolism 2.01 (1.362.96) 1.55 (1.032.32)
Laboratory abnormalities at discharge
Blood urea nitrogen >20 mg/dL 1.86 (1.702.04) 1.37 (1.241.52)
Sodium <135 mEq/L 1.70 (1.521.91) 1.34 (1.181.51)
Hematocrit 27 1.61 (1.401.85) 1.22 (1.051.41)
Vital sign instability at discharge, per instability 1.29 (1.201.40) 1.25 (1.151.36)
Discharged to hospice 0.51 (0.300.89) 0.23 (0.130.40)

In our validation cohort, the full‐stay EHR model had fair discrimination, with a C statistic of 0.69 (95% CI: 0.68‐0.70) (Table 3). The full‐stay EHR model was well calibrated across all quintiles of risk, with slight overestimation of predicted risk in the lowest and highest quintiles (Figure 1a) (see Supporting Table 5 in the online version of this article). It also effectively stratified individuals across a broad range of predicted readmission risk from 4.1% in the lowest decile to 36.5% in the highest decile (Table 3).

Comparison of the Discrimination and Reclassification of Different Readmission Models*
Model Name C‐Statistic (95% CI) IDI, % (95% CI) NRI (95% CI) Average Predicted Risk, %
Lowest Decile Highest Decile
  • NOTE: Abbreviations; CI, confidence interval; EHR, electronic health record; IDI, Integrated Discrimination Improvement; NRI, Net Reclassification Index. *All measures were assessed using the validation cohort (N = 16,430), except for estimating the C‐statistic for the derivation cohort. P value <0.001 for all pairwise comparisons of C‐statistic between full‐stay model and first‐day, LACE, and HOSPITAL models, respectively. The LACE model includes Length of stay, Acute (nonelective) admission status, Charlson Comorbidity Index, and Emergency department visits in the past year. The HOSPITAL model includes Hemoglobin at discharge, discharge from Oncology service, Sodium level at discharge, Procedure during index hospitalization, Index hospitalization Type (nonelective), number of Admissions in the past year, and Length of stay.

Full‐stay EHR model
Derivation cohort 0.72 (0.70 to 0.73) 4.1 36.5
Validation cohort 0.69 (0.68 to 0.70) [Reference] [Reference] 4.1 36.5
First‐day EHR model 0.67 (0.66 to 0.68) 1.2 (1.4 to 1.0) 0.020 (0.038 to 0.002) 5.8 31.9
LACE model 0.65 (0.64 to 0.66) 2.6 (2.9 to 2.3) 0.046 (0.067 to 0.024) 6.1 27.5
HOSPITAL model 0.64 (0.62 to 0.65) 3.2 (3.5 to 2.9) 0.058 (0.080 to 0.035) 6.7 26.6
Figure 1
Comparison of the calibration of different readmission models. Calibration graphs for full‐stay (a), first‐day (b), LACE (c), and HOSPITAL (d) models in the validation cohort. Each graph shows predicted probability compared to observed probability of readmission by quintiles of risk for each model. The LACE model includes Length of stay, Acute (nonelective) admission status, Charlson Comorbidity Index, and Emergency department visits in the past year. The HOSPITAL model includes Hemoglobin at discharge, discharge from Oncology service, Sodium level at discharge, Procedure during index hospitalization, Index hospitalization Type (nonelective), number of Admissions in the past year, and Length of stay.

Comparing the Performance of the Full‐Stay EHR Model to Other Models

The full‐stay EHR model had better discrimination compared to the first‐day EHR model and the LACE and HOSPITAL models, though the magnitude of improvement was modest (Table 3). The full‐stay EHR model also stratified individuals across a broader range of readmission risk, and was better able to discriminate and classify those in the highest quintile of risk from those in the lowest 4 quintiles of risk compared to other models as assessed by the IDI and NRI (Table 3) (see Supporting Tables 14 and Supporting Figure 2 in the online version of this article). In terms of model calibration, both the first‐day EHR and LACE models were also well calibrated, whereas the HOSPITAL model was less robust (Figure 1).

The diagnostic accuracy of the full‐stay EHR model in correctly predicting those in the highest quintile of risk was better than that of the first‐day, LACE, and HOSPITAL models, though overall improvements in the sensitivity, specificity, positive and negative predictive values, and positive and negative likelihood ratios were also modest (see Supporting Table 6 in the online version of this article).

DISCUSSION

In this study, we used clinically detailed EHR data from the entire hospitalization on 32,922 individuals treated in 6 diverse hospitals to develop an all‐payer, multicondition readmission risk‐prediction model. To our knowledge, this is the first 30‐day hospital readmission risk‐prediction model to use a comprehensive set of factors from EHR data from the entire hospital stay. Prior EHR‐based models have focused exclusively on data available on or prior to the first day of admission, which account for clinical severity on admission but do not account for factors uncovered during the inpatient stay that influence the chance of a postdischarge adverse outcome.[15, 30] We specifically assessed the prognostic impact of a comprehensive set of factors from the entire index hospitalization, including hospital‐acquired complications, clinical trajectory, and stability on discharge in predicting hospital readmissions. Our full‐stay EHR model had statistically better discrimination, calibration, and diagnostic accuracy than our existing all‐cause first‐day EHR model[15] and 2 previously published readmissions models that included more limited information from hospitalization (such as length of stay).[9, 10] However, although the more complicated full‐stay EHR model was statistically better than previously published models, we were surprised that the predictive performance was only modestly improved despite the inclusion of many additional clinically relevant prognostic factors.

Taken together, our study has several important implications. First, the added complexity and resource intensity of implementing a full‐stay EHR model yields only modestly improved readmission risk prediction. Thus, hospitals and healthcare systems interested in targeting their highest‐risk individuals for interventions to reduce 30‐day readmission should consider doing so within the first day of hospital admission. Our group's previously derived and validated first‐day EHR model, which used data only from the first day of admission, qualitatively performed nearly as well as the full‐stay EHR model.[15] Additionally, a recent study using only preadmission EHR data to predict 30‐day readmissions also achieved similar discrimination and diagnostic accuracy as our full‐stay model.[30]

Second, the field of readmissions risk‐prediction modeling may be reaching the maximum achievable model performance using data that are currently available in the EHR. Our limited ability to accurately predict all‐cause 30‐day readmission risk may reflect the influence of currently unmeasured patient, system, and community factors on readmissions.[31, 32, 33] Due to the constraints of data collected in the EHR, we were unable to include several patient‐level clinical characteristics associated with hospital readmission, including self‐perceived health status, functional impairment, and cognition.[33, 34, 35, 36] However, given their modest effect sizes (ORs ranging from 1.062.10), adequately measuring and including these risk factors in our model may not meaningfully improve model performance and diagnostic accuracy. Further, many social and behavioral patient‐level factors are also not consistently available in EHR data. Though we explored the role of several neighborhood‐level socioeconomic measuresincluding prevalence of poverty, median income, education, and unemploymentwe found that none were significantly associated with 30‐day readmissions. These particular measures may have been inadequate to characterize individual‐level social and behavioral factors, as several previous studies have demonstrated that patient‐level factors such as social support, substance abuse, and medication and visit adherence can influence readmission risk in heart failure and pneumonia.[11, 16, 22, 25] This underscores the need for more standardized routine collection of data across functional, social, and behavioral domains in clinical settings, as recently championed by the Institute of Medicine.[11, 37] Integrating data from outside the EHR on postdischarge health behaviors, self‐management, follow‐up care, recovery, and home environment may be another important but untapped strategy for further improving prediction of readmissions.[25, 38]

Third, a multicondition readmission risk‐prediction model may be a less effective strategy than more customized disease‐specific models for selected conditions associated with high 30‐day readmission rates. Our group's previously derived and internally validated models for heart failure and human immunodeficiency virus had superior discrimination compared to our full‐stay EHR model (C statistic of 0.72 for each).[11, 13] However, given differences in the included population and time periods studied, a head‐to‐head comparison of these different strategies is needed to assess differences in model performance and utility.

Our study had several strengths. To our knowledge, this is the first study to rigorously measure the additive influence of in‐hospital complications, clinical trajectory, and stability on discharge on the risk of 30‐day hospital readmission. Additionally, our study included a large, diverse study population that included all payers, all ages of adults, a mix of community, academic, and safety net hospitals, and individuals from a broad array of racial/ethnic and socioeconomic backgrounds.

Our results should be interpreted in light of several limitations. First, though we sought to represent a diverse group of hospitals, all study sites were located within north Texas and generalizability to other regions is uncertain. Second, our ascertainment of prior hospitalizations and readmissions was more inclusive than what could be typically accomplished in real time using only EHR data from a single clinical site. We performed a sensitivity analysis using only prior utilization data available within the EHR from the index hospital with no meaningful difference in our findings (data not shown). Additionally, a recent study found that 30‐day readmissions occur at the index hospital for over 75% of events, suggesting that 30‐day readmissions are fairly comprehensively captured even with only single‐site data.[39] Third, we were not able to include data on outpatient visits before or after the index hospitalization, which may influence the risk of readmission.[1, 40]

In conclusion, incorporating clinically granular EHR data from the entire course of hospitalization modestly improves prediction of 30‐day readmissions compared to models that only include information from the first 24 hours of hospital admission or models that use far fewer variables. However, given the limited improvement in prediction, our findings suggest that from the practical perspective of implementing real‐time models to identify those at highest risk for readmission, it may not be worth the added complexity of waiting until the end of a hospitalization to leverage additional data on hospital complications, and the trajectory of laboratory and vital sign values currently available in the EHR. Further improvement in prediction of readmissions will likely require accounting for psychosocial, functional, behavioral, and postdischarge factors not currently present in the inpatient EHR.

Disclosures: This study was presented at the Society of Hospital Medicine 2015 Annual Meeting in National Harbor, Maryland, and the Society of General Internal Medicine 2015 Annual Meeting in Toronto, Canada. This work was supported by the Agency for Healthcare Research and Qualityfunded UT Southwestern Center for Patient‐Centered Outcomes Research (1R24HS022418‐01) and the Commonwealth Foundation (#20100323). Drs. Nguyen and Makam received funding from the UT Southwestern KL2 Scholars Program (NIH/NCATS KL2 TR001103). Dr. Halm was also supported in part by NIH/NCATS U54 RFA‐TR‐12‐006. The study sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. The authors have no conflicts of interest to disclose.

Files
References
  1. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee‐for‐service program. N Engl J Med. 2009;360(14):14181428.
  2. Walraven C, Bennett C, Jennings A, Austin PC, Forster AJ. Proportion of hospital readmissions deemed avoidable: a systematic review. CMAJ. 2011;183(7):E391E402.
  3. Rennke S, Nguyen OK, Shoeb MH, Magan Y, Wachter RM, Ranji SR. Hospital‐initiated transitional care interventions as a patient safety strategy: a systematic review. Ann Intern Med. 2013;158(5 pt 2):433440.
  4. Hansen LO, Young RS, Hinami K, Leung A, Williams MV. Interventions to reduce 30‐day rehospitalization: a systematic review. Ann Intern Med. 2011;155(8):520528.
  5. Rennke S, Shoeb MH, Nguyen OK, Magan Y, Wachter RM, Ranji SR. Interventions to Improve Care Transitions at Hospital Discharge. Rockville, MD: Agency for Healthcare Research and Quality; 2013.
  6. Amarasingham R, Patel PC, Toto K, et al. Allocating scarce resources in real‐time to reduce heart failure readmissions: a prospective, controlled study. BMJ Qual Saf. 2013;22(12):9981005.
  7. Amarasingham R, Patzer RE, Huesch M, Nguyen NQ, Xie B. Implementing electronic health care predictive analytics: considerations and challenges. Health Aff (Millwood). 2014;33(7):11481154.
  8. Kansagara D, Englander H, Salanitro A, et al. Risk prediction models for hospital readmission: a systematic review. JAMA. 2011;306(15):16881698.
  9. Walraven C, Dhalla IA, Bell C, et al. Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community. CMAJ. 2010;182(6):551557.
  10. Donze J, Aujesky D, Williams D, Schnipper JL. Potentially avoidable 30‐day hospital readmissions in medical patients: derivation and validation of a prediction model. JAMA Intern Med. 2013;173(8):632638.
  11. Amarasingham R, Moore BJ, Tabak YP, et al. An automated model to identify heart failure patients at risk for 30‐day readmission or death using electronic medical record data. Med Care. 2010;48(11):981988.
  12. Singal AG, Rahimi RS, Clark C, et al. An automated model using electronic medical record data identifies patients with cirrhosis at high risk for readmission. Clin Gastroenterol Hepatol. 2013;11(10):13351341.e1331.
  13. Nijhawan AE, Clark C, Kaplan R, Moore B, Halm EA, Amarasingham R. An electronic medical record‐based model to predict 30‐day risk of readmission and death among HIV‐infected inpatients. J Acquir Immune Defic Syndr. 2012;61(3):349358.
  14. Horwitz LI, Partovian C, Lin Z, et al. Development and use of an administrative claims measure for profiling hospital‐wide performance on 30‐day unplanned readmission. Ann Intern Med. 2014;161(10 suppl):S66S75.
  15. Amarasingham R, Velasco F, Xie B, et al. Electronic medical record‐based multicondition models to predict the risk of 30 day readmission or death among adult medicine patients: validation and comparison to existing models. BMC Med Inform Decis Mak. 2015;15(1):39.
  16. Watson AJ, O'Rourke J, Jethwani K, et al. Linking electronic health record‐extracted psychosocial data in real‐time to risk of readmission for heart failure. Psychosomatics. 2011;52(4):319327.
  17. Ashton CM, Wray NP. A conceptual framework for the study of early readmission as an indicator of quality of care. Soc Sci Med. 1996;43(11):15331541.
  18. Dharmarajan K, Hsieh AF, Lin Z, et al. Hospital readmission performance and patterns of readmission: retrospective cohort study of Medicare admissions. BMJ. 2013;347:f6571.
  19. Cassel CK, Conway PH, Delbanco SF, Jha AK, Saunders RS, Lee TH. Getting more performance from performance measurement. N Engl J Med. 2014;371(23):21452147.
  20. Bradley EH, Sipsma H, Horwitz LI, et al. Hospital strategy uptake and reductions in unplanned readmission rates for patients with heart failure: a prospective study. J Gen Intern Med. 2015;30(5):605611.
  21. Krumholz HM. Post‐hospital syndrome—an acquired, transient condition of generalized risk. N Engl J Med. 2013;368(2):100102.
  22. Calvillo‐King L, Arnold D, Eubank KJ, et al. Impact of social factors on risk of readmission or mortality in pneumonia and heart failure: systematic review. J Gen Intern Med. 2013;28(2):269282.
  23. Keyhani S, Myers LJ, Cheng E, Hebert P, Williams LS, Bravata DM. Effect of clinical and social risk factors on hospital profiling for stroke readmission: a cohort study. Ann Intern Med. 2014;161(11):775784.
  24. Kind AJ, Jencks S, Brock J, et al. Neighborhood socioeconomic disadvantage and 30‐day rehospitalization: a retrospective cohort study. Ann Intern Med. 2014;161(11):765774.
  25. Arbaje AI, Wolff JL, Yu Q, Powe NR, Anderson GF, Boult C. Postdischarge environmental and socioeconomic factors and the likelihood of early hospital readmission among community‐dwelling Medicare beneficiaries. Gerontologist. 2008;48(4):495504.
  26. Hu J, Gonsahn MD, Nerenz DR. Socioeconomic status and readmissions: evidence from an urban teaching hospital. Health Aff (Millwood). 2014;33(5):778785.
  27. Nagasako EM, Reidhead M, Waterman B, Dunagan WC. Adding socioeconomic data to hospital readmissions calculations may produce more useful results. Health Aff (Millwood). 2014;33(5):786791.
  28. Pencina MJ, D'Agostino RB, D'Agostino RB, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157172; discussion 207–212.
  29. Leening MJ, Vedder MM, Witteman JC, Pencina MJ, Steyerberg EW. Net reclassification improvement: computation, interpretation, and controversies: a literature review and clinician's guide. Ann Intern Med. 2014;160(2):122131.
  30. Shadmi E, Flaks‐Manov N, Hoshen M, Goldman O, Bitterman H, Balicer RD. Predicting 30‐day readmissions with preadmission electronic health record data. Med Care. 2015;53(3):283289.
  31. Kangovi S, Grande D. Hospital readmissions—not just a measure of quality. JAMA. 2011;306(16):17961797.
  32. Joynt KE, Jha AK. Thirty‐day readmissions—truth and consequences. N Engl J Med. 2012;366(15):13661369.
  33. Greysen SR, Stijacic Cenzer I, Auerbach AD, Covinsky KE. Functional impairment and hospital readmission in medicare seniors. JAMA Intern Med. 2015;175(4):559565.
  34. Holloway JJ, Thomas JW, Shapiro L. Clinical and sociodemographic risk factors for readmission of Medicare beneficiaries. Health Care Financ Rev. 1988;10(1):2736.
  35. Patel A, Parikh R, Howell EH, Hsich E, Landers SH, Gorodeski EZ. Mini‐cog performance: novel marker of post discharge risk among patients hospitalized for heart failure. Circ Heart Fail. 2015;8(1):816.
  36. Hoyer EH, Needham DM, Atanelov L, Knox B, Friedman M, Brotman DJ. Association of impaired functional status at hospital discharge and subsequent rehospitalization. J Hosp Med. 2014;9(5):277282.
  37. Adler NE, Stead WW. Patients in context—EHR capture of social and behavioral determinants of health. N Engl J Med. 2015;372(8):698701.
  38. Nguyen OK, Chan CV, Makam A, Stieglitz H, Amarasingham R. Envisioning a social‐health information exchange as a platform to support a patient‐centered medical neighborhood: a feasibility study. J Gen Intern Med. 2015;30(1):6067.
  39. Henke RM, Karaca Z, Lin H, Wier LM, Marder W, Wong HS. Patient factors contributing to variation in same‐hospital readmission rate. Med Care Res Review. 2015;72(3):338358.
  40. Weinberger M, Oddone EZ, Henderson WG. Does increased access to primary care reduce hospital readmissions? Veterans Affairs Cooperative Study Group on Primary Care and Hospital Readmission. N Engl J Med. 1996;334(22):14411447.
Article PDF
Issue
Journal of Hospital Medicine - 11(7)
Page Number
473-480
Sections
Files
Files
Article PDF
Article PDF

Unplanned hospital readmissions are frequent, costly, and potentially avoidable.[1, 2] Due to major federal financial readmissions penalties targeting excessive 30‐day readmissions, there is increasing attention to implementing hospital‐initiated interventions to reduce readmissions.[3, 4] However, universal enrollment of all hospitalized patients into such programs may be too resource intensive for many hospitals.[5] To optimize efficiency and effectiveness, interventions should be targeted to individuals most likely to benefit.[6, 7] However, existing readmission risk‐prediction models have achieved only modest discrimination, have largely used administrative claims data not available until months after discharge, or are limited to only a subset of patients with Medicare or a specific clinical condition.[8, 9, 10, 11, 12, 13, 14] These limitations have precluded accurate identification of high‐risk individuals in an all‐payer general medical inpatient population to provide actionable information for intervention prior to discharge.

Approaches using electronic health record (EHR) data could allow early identification of high‐risk patients during the index hospitalization to enable initiation of interventions prior to discharge. To date, such strategies have relied largely on EHR data from the day of admission.[15, 16] However, given that variation in 30‐day readmission rates are thought to reflect the quality of in‐hospital care, incorporating EHR data from the entire hospital stay to reflect hospital care processes and clinical trajectory may more accurately identify at‐risk patients.[17, 18, 19, 20] Improved accuracy in risk prediction would help better target intervention efforts in the immediate postdischarge period, an interval characterized by heightened vulnerability for adverse events.[21]

To help hospitals target transitional care interventions more effectively to high‐risk individuals prior to discharge, we derived and validated a readmissions risk‐prediction model incorporating EHR data from the entire course of the index hospitalization, which we termed the full‐stay EHR model. We also compared the full‐stay EHR model performance to our group's previously derived prediction model based on EHR data on the day of admission, termed the first‐day EHR model, as well as to 2 other validated readmission models similarly intended to yield near real‐time risk predictions prior to or shortly after hospital discharge.[9, 10, 15]

METHODS

Study Design, Population, and Data Sources

We conducted an observational cohort study using EHR data from 6 hospitals in the DallasFort Worth metroplex between November 1, 2009 and October 30, 2010 using the same EHR system (Epic Systems Corp., Verona, WI). One site was a university‐affiliated safety net hospital; the remaining 5 sites were teaching and nonteaching community sites.

We included consecutive hospitalizations among adults 18 years old discharged alive from any medicine inpatient service. For individuals with multiple hospitalizations during the study period, we included only the first hospitalization. We excluded individuals who died during the index hospitalization, were transferred to another acute care facility, left against medical advice, or who died outside of the hospital within 30 days of discharge. For model derivation, we randomly split the sample into separate derivation (50%) and validation cohorts (50%).

Outcomes

The primary outcome was 30‐day hospital readmission, defined as a nonelective hospitalization within 30 days of discharge to any of 75 acute care hospitals within a 100‐mile radius of Dallas, ascertained from an all‐payer regional hospitalization database. Nonelective hospitalizations included all hospitalizations classified as a emergency, urgent, or trauma, and excluded those classified as elective as per the Centers for Medicare and Medicaid Services Claim Inpatient Admission Type Code definitions.

Predictor Variables for the Full‐Stay EHR Model

The full‐stay EHR model was iteratively developed from our group's previously derived and validated risk‐prediction model using EHR data available on admission (first‐day EHR model).[15] For the full‐stay EHR model, we included all predictor variables included in our published first‐day EHR model as candidate risk factors. Based on prior literature, we additionally expanded candidate predictors available on admission to include marital status (proxy for social isolation) and socioeconomic disadvantage (percent poverty, unemployment, median income, and educational attainment by zip code of residence as proxy measures of the social and built environment).[22, 23, 24, 25, 26, 27] We also expanded the ascertainment of prior hospitalization to include admissions at both the index hospital and any of 75 acute care hospitals from the same, separate all‐payer regional hospitalization database used to ascertain 30‐day readmissions.

Candidate predictors from the remainder of the hospital stay (ie, following the first 24 hours of admission) were included if they were: (1) available in the EHR of all participating hospitals, (2) routinely collected or available at the time of hospital discharge, and (3) plausible predictors of adverse outcomes based on prior literature and clinical expertise. These included length of stay, in‐hospital complications, transfer to an intensive or coronary care unit, blood transfusions, vital sign instabilities within 24 hours of discharge, select laboratory values at time of discharge, and disposition status. We also assessed trajectories of vital signs and selected laboratory values (defined as changes in these measures from admission to discharge).

Statistical Analysis

Model Derivation

Univariate relationships between readmission and each of the candidate predictors were assessed in the derivation cohort using a prespecified significance threshold of P 0.05. We included all factors from our previously derived and validated first‐day EHR model as candidate predictors.[15] Continuous laboratory and vital sign values at the time of discharge were categorized based on clinically meaningful cutoffs; predictors with missing values were assumed to be normal (<1% missing for each variable). Significant univariate candidate variables were entered in a multivariate logistic regression model using stepwise backward selection with a prespecified significance threshold of P 0.05. We performed several sensitivity analyses to confirm the robustness of our model. First, we alternately derived the full‐stay model using stepwise forward selection. Second, we forced in all significant variables from our first‐day EHR model, and entered the candidate variables from the remainder of the hospital stay using both stepwise backward and forward selection separately. Third, prespecified interactions between variables were evaluated for inclusion. Though final predictors varied slightly between the different approaches, discrimination of each model was similar to the model derived using our primary analytic approach (C statistics 0.01, data not shown).

Model Validation

We assessed model discrimination and calibration of the derived full‐stay EHR model using the validation cohort. Model discrimination was estimated by the C statistic. The C statistic represents the probability that, given 2 hospitalized individuals (1 who was readmitted and the other who was not), the model will predict a higher risk for the readmitted patient than for the nonreadmitted patient. Model calibration was assessed by comparing predicted to observed probabilities of readmission by quintiles of risk, and with the Hosmer‐Lemeshow goodness‐of‐fit test.

Comparison to Existing Models

We compared the full‐stay EHR model performance to 3 previously published models: our group's first‐day EHR model, and the LACE (includes Length of stay, Acute (nonelective) admission status, Charlson Comorbidity Index, and Emergency department visits in the past year) and HOSPITAL (includes Hemoglobin at discharge, discharge from Oncology service, Sodium level at discharge, Procedure during index hospitalization, Index hospitalization Type (nonelective), number of Admissions in the past year, and Length of stay) models, which were both derived to predict 30‐day readmissions among general medical inpatients and were intended to help clinicians identify high‐risk patients to target for discharge interventions.[9, 10, 15] We assessed each model's performance in our validation cohort, calculating the C statistic, integrated discrimination index (IDI), and net reclassification index (NRI) compared to the full‐stay model. IDI is a summary measure of both discrimination and reclassification, where more positive values suggest improvement in model performance in both these domains compared to a reference model.[28] The NRI is defined as the sum of the net proportions of correctly reclassified persons with and without the event of interest.[29] The theoretical range of values is 2 to 2, with more positive values indicating improved net reclassification compared to a reference model. Here, we calculated a category‐based NRI to evaluate the performance of models in correctly classifying individuals with and without readmissions into the highest readmission risk quintile versus the lowest 4 risk quintiles compared to the full‐stay EHR model.[29] This prespecified cutoff is relevant for hospitals interested in identifying the highest‐risk individuals for targeted intervention.[6] Because some hospitals may be able to target a greater number of individuals for intervention, we performed a sensitivity analysis by assessing category‐based NRI for reclassification into the top 2 risk quintiles versus the lowest 3 risk quintiles and found no meaningful difference in our results (data not shown). Finally, we qualitatively assessed calibration of comparator models in our validation cohort by comparing predicted probability to observed probability of readmission by quintiles of risk for each model. We conducted all analyses using Stata 12.1 (StataCorp, College Station, TX). This study was approved by the UT Southwestern Medical Center institutional review board.

RESULTS

Overall, 32,922 index hospitalizations were included in our study cohort; 12.7% resulted in a 30‐day readmission (see Supporting Figure 1 in the online version of this article). Individuals had a mean age of 62 years and had diverse race/ethnicity and primary insurance status; half were female (Table 1). The study sample was randomly split into a derivation cohort (50%, n = 16,492) and validation cohort (50%, n = 16,430). Individuals in the derivation cohort with a 30‐day readmission had markedly different socioeconomic and clinical characteristics compared to those not readmitted (Table 1).

Baseline Characteristics and Candidate Variables for Risk‐Prediction Model
Entire Cohort, N = 32,922 Derivation Cohort, N = 16,492

No Readmission, N = 14,312

Readmission, N = 2,180

P Value
  • NOTE: Abbreviations: ED, emergency department; ICU, intensive care unit; IQR, interquartile range; SD, standard deviation. *20% poverty in zip code as per high poverty area US Census designation. Prior ED visit at site of index hospitalization within the past year. Prior hospitalization at any of 75 acute care hospitals in the North Texas region within the past year. Nonelective admission defined as hospitalization categorized as medical emergency, urgent, or trauma. ∥Calculated from diagnoses available within 1 year prior to index hospitalization. Conditions were considered complications if they were not listed as a principle diagnosis for hospitalization or as a previous diagnosis in the prior year. #On day of discharge or last known observation before discharge. Instabilities were defined as temperature 37.8C, heart rate >100 beats/minute, respiratory rate >24 breaths/minute, systolic blood pressure 90 mm Hg, or oxygen saturation <90%. **Discharges to nursing home, skilled nursing facility, or long‐term acute care hospital.

Demographic characteristics
Age, y, mean (SD) 62 (17.3) 61 (17.4) 64 (17.0) 0.001
Female, n (%) 17,715 (53.8) 7,694 (53.8) 1,163 (53.3) 0.72
Race/ethnicity 0.001
White 21,359 (64.9) 9,329 (65.2) 1,361 (62.4)
Black 5,964 (18.1) 2,520 (17.6) 434 (19.9)
Hispanic 4,452 (13.5) 1,931 (13.5) 338 (15.5)
Other 1,147 (3.5) 532 (3.7) 47 (2.2)
Marital status, n (%) 0.001
Single 8,076 (24.5) 3,516 (24.6) 514 (23.6)
Married 13,394 (40.7) 5,950 (41.6) 812 (37.3)
Separated/divorced 3,468 (10.5) 1,460 (10.2) 251 (11.5)
Widowed 4,487 (13.7) 1,868 (13.1) 388 (17.8)
Other 3,497 (10.6) 1,518 (10.6) 215 (9.9)
Primary payer, n (%) 0.001
Private 13,090 (39.8) 5,855 (40.9) 726 (33.3)
Medicare 13,015 (39.5) 5,597 (39.1) 987 (45.3)
Medicaid 2,204 (6.7) 852 (5.9) 242 (11.1)
Charity, self‐pay, or other 4,613 (14.0) 2,008 (14.0) 225 (10.3)
High‐poverty neighborhood, n (%)* 7,468 (22.7) 3,208 (22.4) 548 (25.1) 0.001
Utilization history
1 ED visits in past year, n (%) 9,299 (28.2) 3,793 (26.5) 823 (37.8) 0.001
1 hospitalizations in past year, n (%) 10,189 (30.9) 4,074 (28.5) 1,012 (46.4) 0.001
Clinical factors from first day of hospitalization
Nonelective admission, n (%) 27,818 (84.5) 11,960 (83.6) 1,960 (89.9) 0.001
Charlson Comorbidity Index, median (IQR)∥ 0 (01) 0 (00) 0 (03) 0.001
Laboratory abnormalities within 24 hours of admission
Albumin <2 g/dL 355 (1.1) 119 (0.8) 46 (2.1) 0.001
Albumin 23 g/dL 4,732 (14.4) 1,956 (13.7) 458 (21.0) 0.001
Aspartate aminotransferase >40 U/L 4,610 (14.0) 1,922 (13.4) 383 (17.6) 0.001
Creatine phosphokinase <60 g/L 3,728 (11.3) 1,536 (10.7) 330 (15.1) 0.001
Mean corpuscular volume >100 fL/red cell 1,346 (4.1) 537 (3.8) 134 (6.2) 0.001
Platelets <90 103/L 912 (2.8) 357 (2.5) 116 (5.3) 0.001
Platelets >350 103/L 3,332 (10.1) 1,433 (10.0) 283 (13.0) 0.001
Prothrombin time >35 seconds 248 (0.8) 90 (0.6) 35 (1.6) 0.001
Clinical factors from remainder of hospital stay
Length of stay, d, median (IQR) 4 (26) 4 (26) 5 (38) 0.001
ICU transfer after first 24 hours, n (%) 988 (3.0) 408 (2.9) 94 (4.3) 0.001
Hospital complications, n (%)
Clostridium difficile infection 119 (0.4) 44 (0.3) 24 (1.1) 0.001
Pressure ulcer 358 (1.1) 126 (0.9) 46 (2.1) 0.001
Venous thromboembolism 301 (0.9) 112 (0.8) 34 (1.6) 0.001
Respiratory failure 1,048 (3.2) 463 (3.2) 112 (5.1) 0.001
Central line‐associated bloodstream infection 22 (0.07) 6 (0.04) 5 (0.23) 0.005
Catheter‐associated urinary tract infection 47 (0.14) 20 (0.14) 6 (0.28) 0.15
Acute myocardial infarction 293 (0.9) 110 (0.8) 32 (1.5) 0.001
Pneumonia 1,754 (5.3) 719 (5.0) 154 (7.1) 0.001
Sepsis 853 (2.6) 368 (2.6) 73 (3.4) 0.04
Blood transfusion during hospitalization, n (%) 4,511 (13.7) 1,837 (12.8) 425 (19.5) 0.001
Laboratory abnormalities at discharge#
Blood urea nitrogen >20 mg/dL, n (%) 10,014 (30.4) 4,077 (28.5) 929 (42.6) 0.001
Sodium <135 mEq/L, n (%) 4,583 (13.9) 1,850 (12.9) 440 (20.2) 0.001
Hematocrit 27 3,104 (9.4) 1,231 (8.6) 287 (13.2) 0.001
1 vital sign instability at discharge, n (%)# 6,192 (18.8) 2,624 (18.3) 525 (24.1) 0.001
Discharge location, n (%) 0.001
Home 23,339 (70.9) 10,282 (71.8) 1,383 (63.4)
Home health 3,185 (9.7) 1,356 (9.5) 234 (10.7)
Postacute care** 5,990 (18.2) 2,496 (17.4) 549 (25.2)
Hospice 408 (1.2) 178 (1.2) 14 (0.6)

Derivation and Validation of the Full‐Stay EHR Model for 30‐Day Readmission

Our final model included 24 independent variables, including demographic characteristics, utilization history, clinical factors from the first day of admission, and clinical factors from the remainder of the hospital stay (Table 2). The strongest independent predictor of readmission was hospital‐acquired Clostridium difficile infection (adjusted odds ratio [AOR]: 2.03, 95% confidence interval [CI] 1.18‐3.48); other hospital‐acquired complications including pressure ulcers and venous thromboembolism were also significant predictors. Though having Medicaid was associated with increased odds of readmission (AOR: 1.55, 95% CI: 1.31‐1.83), other zip codelevel measures of socioeconomic disadvantage were not predictive and were not included in the final model. Being discharged to hospice was associated with markedly lower odds of readmission (AOR: 0.23, 95% CI: 0.13‐0.40).

Final Full‐Stay EHR Model Predicting 30‐Day Readmissions (Derivation Cohort, N = 16,492)
Odds Ratio (95% CI)
Univariate Multivariate*
  • NOTE: Abbreviations: CI, confidence interval; ED, emergency department. *Values shown reflect adjusted odds ratios and 95% CI for each factor after adjustment for all other factors listed in the table.

Demographic characteristics
Age, per 10 years 1.08 (1.051.11) 1.07 (1.041.10)
Medicaid 1.97 (1.702.29) 1.55 (1.311.83)
Widow 1.44 (1.281.63) 1.27 (1.111.45)
Utilization history
Prior ED visit, per visit 1.08 (1.061.10) 1.04 (1.021.06)
Prior hospitalization, per hospitalization 1.30 (1.271.34) 1.16 (1.121.20)
Hospital and clinical factors from first day of hospitalization
Nonelective admission 1.75 (1.512.03) 1.42 (1.221.65)
Charlson Comorbidity Index, per point 1.19 (1.171.21) 1.06 (1.041.09)
Laboratory abnormalities within 24 hours of admission
Albumin <2 g/dL 2.57 (1.823.62) 1.52 (1.052.21)
Albumin 23 g/dL 1.68 (1.501.88) 1.20 (1.061.36)
Aspartate aminotransferase >40 U/L 1.37 (1.221.55) 1.21 (1.061.38)
Creatine phosphokinase <60 g/L 1.48 (1.301.69) 1.28 (1.111.46)
Mean corpuscular volume >100 fL/red cell 1.68 (1.382.04) 1.32 (1.071.62)
Platelets <90 103/L 2.20 (1.772.72) 1.56 (1.231.97)
Platelets >350 103/L 1.34 (1.171.54) 1.24 (1.081.44)
Prothrombin time >35 seconds 2.58 (1.743.82) 1.92 (1.272.90)
Hospital and clinical factors from remainder of hospital stay
Length of stay, per day 1.08 (1.071.09) 1.06 (1.041.07)
Hospital complications
Clostridium difficile infection 3.61 (2.195.95) 2.03 (1.183.48)
Pressure ulcer 2.43 (1.733.41) 1.64 (1.152.34)
Venous thromboembolism 2.01 (1.362.96) 1.55 (1.032.32)
Laboratory abnormalities at discharge
Blood urea nitrogen >20 mg/dL 1.86 (1.702.04) 1.37 (1.241.52)
Sodium <135 mEq/L 1.70 (1.521.91) 1.34 (1.181.51)
Hematocrit 27 1.61 (1.401.85) 1.22 (1.051.41)
Vital sign instability at discharge, per instability 1.29 (1.201.40) 1.25 (1.151.36)
Discharged to hospice 0.51 (0.300.89) 0.23 (0.130.40)

In our validation cohort, the full‐stay EHR model had fair discrimination, with a C statistic of 0.69 (95% CI: 0.68‐0.70) (Table 3). The full‐stay EHR model was well calibrated across all quintiles of risk, with slight overestimation of predicted risk in the lowest and highest quintiles (Figure 1a) (see Supporting Table 5 in the online version of this article). It also effectively stratified individuals across a broad range of predicted readmission risk from 4.1% in the lowest decile to 36.5% in the highest decile (Table 3).

Comparison of the Discrimination and Reclassification of Different Readmission Models*
Model Name C‐Statistic (95% CI) IDI, % (95% CI) NRI (95% CI) Average Predicted Risk, %
Lowest Decile Highest Decile
  • NOTE: Abbreviations; CI, confidence interval; EHR, electronic health record; IDI, Integrated Discrimination Improvement; NRI, Net Reclassification Index. *All measures were assessed using the validation cohort (N = 16,430), except for estimating the C‐statistic for the derivation cohort. P value <0.001 for all pairwise comparisons of C‐statistic between full‐stay model and first‐day, LACE, and HOSPITAL models, respectively. The LACE model includes Length of stay, Acute (nonelective) admission status, Charlson Comorbidity Index, and Emergency department visits in the past year. The HOSPITAL model includes Hemoglobin at discharge, discharge from Oncology service, Sodium level at discharge, Procedure during index hospitalization, Index hospitalization Type (nonelective), number of Admissions in the past year, and Length of stay.

Full‐stay EHR model
Derivation cohort 0.72 (0.70 to 0.73) 4.1 36.5
Validation cohort 0.69 (0.68 to 0.70) [Reference] [Reference] 4.1 36.5
First‐day EHR model 0.67 (0.66 to 0.68) 1.2 (1.4 to 1.0) 0.020 (0.038 to 0.002) 5.8 31.9
LACE model 0.65 (0.64 to 0.66) 2.6 (2.9 to 2.3) 0.046 (0.067 to 0.024) 6.1 27.5
HOSPITAL model 0.64 (0.62 to 0.65) 3.2 (3.5 to 2.9) 0.058 (0.080 to 0.035) 6.7 26.6
Figure 1
Comparison of the calibration of different readmission models. Calibration graphs for full‐stay (a), first‐day (b), LACE (c), and HOSPITAL (d) models in the validation cohort. Each graph shows predicted probability compared to observed probability of readmission by quintiles of risk for each model. The LACE model includes Length of stay, Acute (nonelective) admission status, Charlson Comorbidity Index, and Emergency department visits in the past year. The HOSPITAL model includes Hemoglobin at discharge, discharge from Oncology service, Sodium level at discharge, Procedure during index hospitalization, Index hospitalization Type (nonelective), number of Admissions in the past year, and Length of stay.

Comparing the Performance of the Full‐Stay EHR Model to Other Models

The full‐stay EHR model had better discrimination compared to the first‐day EHR model and the LACE and HOSPITAL models, though the magnitude of improvement was modest (Table 3). The full‐stay EHR model also stratified individuals across a broader range of readmission risk, and was better able to discriminate and classify those in the highest quintile of risk from those in the lowest 4 quintiles of risk compared to other models as assessed by the IDI and NRI (Table 3) (see Supporting Tables 14 and Supporting Figure 2 in the online version of this article). In terms of model calibration, both the first‐day EHR and LACE models were also well calibrated, whereas the HOSPITAL model was less robust (Figure 1).

The diagnostic accuracy of the full‐stay EHR model in correctly predicting those in the highest quintile of risk was better than that of the first‐day, LACE, and HOSPITAL models, though overall improvements in the sensitivity, specificity, positive and negative predictive values, and positive and negative likelihood ratios were also modest (see Supporting Table 6 in the online version of this article).

DISCUSSION

In this study, we used clinically detailed EHR data from the entire hospitalization on 32,922 individuals treated in 6 diverse hospitals to develop an all‐payer, multicondition readmission risk‐prediction model. To our knowledge, this is the first 30‐day hospital readmission risk‐prediction model to use a comprehensive set of factors from EHR data from the entire hospital stay. Prior EHR‐based models have focused exclusively on data available on or prior to the first day of admission, which account for clinical severity on admission but do not account for factors uncovered during the inpatient stay that influence the chance of a postdischarge adverse outcome.[15, 30] We specifically assessed the prognostic impact of a comprehensive set of factors from the entire index hospitalization, including hospital‐acquired complications, clinical trajectory, and stability on discharge in predicting hospital readmissions. Our full‐stay EHR model had statistically better discrimination, calibration, and diagnostic accuracy than our existing all‐cause first‐day EHR model[15] and 2 previously published readmissions models that included more limited information from hospitalization (such as length of stay).[9, 10] However, although the more complicated full‐stay EHR model was statistically better than previously published models, we were surprised that the predictive performance was only modestly improved despite the inclusion of many additional clinically relevant prognostic factors.

Taken together, our study has several important implications. First, the added complexity and resource intensity of implementing a full‐stay EHR model yields only modestly improved readmission risk prediction. Thus, hospitals and healthcare systems interested in targeting their highest‐risk individuals for interventions to reduce 30‐day readmission should consider doing so within the first day of hospital admission. Our group's previously derived and validated first‐day EHR model, which used data only from the first day of admission, qualitatively performed nearly as well as the full‐stay EHR model.[15] Additionally, a recent study using only preadmission EHR data to predict 30‐day readmissions also achieved similar discrimination and diagnostic accuracy as our full‐stay model.[30]

Second, the field of readmissions risk‐prediction modeling may be reaching the maximum achievable model performance using data that are currently available in the EHR. Our limited ability to accurately predict all‐cause 30‐day readmission risk may reflect the influence of currently unmeasured patient, system, and community factors on readmissions.[31, 32, 33] Due to the constraints of data collected in the EHR, we were unable to include several patient‐level clinical characteristics associated with hospital readmission, including self‐perceived health status, functional impairment, and cognition.[33, 34, 35, 36] However, given their modest effect sizes (ORs ranging from 1.062.10), adequately measuring and including these risk factors in our model may not meaningfully improve model performance and diagnostic accuracy. Further, many social and behavioral patient‐level factors are also not consistently available in EHR data. Though we explored the role of several neighborhood‐level socioeconomic measuresincluding prevalence of poverty, median income, education, and unemploymentwe found that none were significantly associated with 30‐day readmissions. These particular measures may have been inadequate to characterize individual‐level social and behavioral factors, as several previous studies have demonstrated that patient‐level factors such as social support, substance abuse, and medication and visit adherence can influence readmission risk in heart failure and pneumonia.[11, 16, 22, 25] This underscores the need for more standardized routine collection of data across functional, social, and behavioral domains in clinical settings, as recently championed by the Institute of Medicine.[11, 37] Integrating data from outside the EHR on postdischarge health behaviors, self‐management, follow‐up care, recovery, and home environment may be another important but untapped strategy for further improving prediction of readmissions.[25, 38]

Third, a multicondition readmission risk‐prediction model may be a less effective strategy than more customized disease‐specific models for selected conditions associated with high 30‐day readmission rates. Our group's previously derived and internally validated models for heart failure and human immunodeficiency virus had superior discrimination compared to our full‐stay EHR model (C statistic of 0.72 for each).[11, 13] However, given differences in the included population and time periods studied, a head‐to‐head comparison of these different strategies is needed to assess differences in model performance and utility.

Our study had several strengths. To our knowledge, this is the first study to rigorously measure the additive influence of in‐hospital complications, clinical trajectory, and stability on discharge on the risk of 30‐day hospital readmission. Additionally, our study included a large, diverse study population that included all payers, all ages of adults, a mix of community, academic, and safety net hospitals, and individuals from a broad array of racial/ethnic and socioeconomic backgrounds.

Our results should be interpreted in light of several limitations. First, though we sought to represent a diverse group of hospitals, all study sites were located within north Texas and generalizability to other regions is uncertain. Second, our ascertainment of prior hospitalizations and readmissions was more inclusive than what could be typically accomplished in real time using only EHR data from a single clinical site. We performed a sensitivity analysis using only prior utilization data available within the EHR from the index hospital with no meaningful difference in our findings (data not shown). Additionally, a recent study found that 30‐day readmissions occur at the index hospital for over 75% of events, suggesting that 30‐day readmissions are fairly comprehensively captured even with only single‐site data.[39] Third, we were not able to include data on outpatient visits before or after the index hospitalization, which may influence the risk of readmission.[1, 40]

In conclusion, incorporating clinically granular EHR data from the entire course of hospitalization modestly improves prediction of 30‐day readmissions compared to models that only include information from the first 24 hours of hospital admission or models that use far fewer variables. However, given the limited improvement in prediction, our findings suggest that from the practical perspective of implementing real‐time models to identify those at highest risk for readmission, it may not be worth the added complexity of waiting until the end of a hospitalization to leverage additional data on hospital complications, and the trajectory of laboratory and vital sign values currently available in the EHR. Further improvement in prediction of readmissions will likely require accounting for psychosocial, functional, behavioral, and postdischarge factors not currently present in the inpatient EHR.

Disclosures: This study was presented at the Society of Hospital Medicine 2015 Annual Meeting in National Harbor, Maryland, and the Society of General Internal Medicine 2015 Annual Meeting in Toronto, Canada. This work was supported by the Agency for Healthcare Research and Qualityfunded UT Southwestern Center for Patient‐Centered Outcomes Research (1R24HS022418‐01) and the Commonwealth Foundation (#20100323). Drs. Nguyen and Makam received funding from the UT Southwestern KL2 Scholars Program (NIH/NCATS KL2 TR001103). Dr. Halm was also supported in part by NIH/NCATS U54 RFA‐TR‐12‐006. The study sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. The authors have no conflicts of interest to disclose.

Unplanned hospital readmissions are frequent, costly, and potentially avoidable.[1, 2] Due to major federal financial readmissions penalties targeting excessive 30‐day readmissions, there is increasing attention to implementing hospital‐initiated interventions to reduce readmissions.[3, 4] However, universal enrollment of all hospitalized patients into such programs may be too resource intensive for many hospitals.[5] To optimize efficiency and effectiveness, interventions should be targeted to individuals most likely to benefit.[6, 7] However, existing readmission risk‐prediction models have achieved only modest discrimination, have largely used administrative claims data not available until months after discharge, or are limited to only a subset of patients with Medicare or a specific clinical condition.[8, 9, 10, 11, 12, 13, 14] These limitations have precluded accurate identification of high‐risk individuals in an all‐payer general medical inpatient population to provide actionable information for intervention prior to discharge.

Approaches using electronic health record (EHR) data could allow early identification of high‐risk patients during the index hospitalization to enable initiation of interventions prior to discharge. To date, such strategies have relied largely on EHR data from the day of admission.[15, 16] However, given that variation in 30‐day readmission rates are thought to reflect the quality of in‐hospital care, incorporating EHR data from the entire hospital stay to reflect hospital care processes and clinical trajectory may more accurately identify at‐risk patients.[17, 18, 19, 20] Improved accuracy in risk prediction would help better target intervention efforts in the immediate postdischarge period, an interval characterized by heightened vulnerability for adverse events.[21]

To help hospitals target transitional care interventions more effectively to high‐risk individuals prior to discharge, we derived and validated a readmissions risk‐prediction model incorporating EHR data from the entire course of the index hospitalization, which we termed the full‐stay EHR model. We also compared the full‐stay EHR model performance to our group's previously derived prediction model based on EHR data on the day of admission, termed the first‐day EHR model, as well as to 2 other validated readmission models similarly intended to yield near real‐time risk predictions prior to or shortly after hospital discharge.[9, 10, 15]

METHODS

Study Design, Population, and Data Sources

We conducted an observational cohort study using EHR data from 6 hospitals in the DallasFort Worth metroplex between November 1, 2009 and October 30, 2010 using the same EHR system (Epic Systems Corp., Verona, WI). One site was a university‐affiliated safety net hospital; the remaining 5 sites were teaching and nonteaching community sites.

We included consecutive hospitalizations among adults 18 years old discharged alive from any medicine inpatient service. For individuals with multiple hospitalizations during the study period, we included only the first hospitalization. We excluded individuals who died during the index hospitalization, were transferred to another acute care facility, left against medical advice, or who died outside of the hospital within 30 days of discharge. For model derivation, we randomly split the sample into separate derivation (50%) and validation cohorts (50%).

Outcomes

The primary outcome was 30‐day hospital readmission, defined as a nonelective hospitalization within 30 days of discharge to any of 75 acute care hospitals within a 100‐mile radius of Dallas, ascertained from an all‐payer regional hospitalization database. Nonelective hospitalizations included all hospitalizations classified as a emergency, urgent, or trauma, and excluded those classified as elective as per the Centers for Medicare and Medicaid Services Claim Inpatient Admission Type Code definitions.

Predictor Variables for the Full‐Stay EHR Model

The full‐stay EHR model was iteratively developed from our group's previously derived and validated risk‐prediction model using EHR data available on admission (first‐day EHR model).[15] For the full‐stay EHR model, we included all predictor variables included in our published first‐day EHR model as candidate risk factors. Based on prior literature, we additionally expanded candidate predictors available on admission to include marital status (proxy for social isolation) and socioeconomic disadvantage (percent poverty, unemployment, median income, and educational attainment by zip code of residence as proxy measures of the social and built environment).[22, 23, 24, 25, 26, 27] We also expanded the ascertainment of prior hospitalization to include admissions at both the index hospital and any of 75 acute care hospitals from the same, separate all‐payer regional hospitalization database used to ascertain 30‐day readmissions.

Candidate predictors from the remainder of the hospital stay (ie, following the first 24 hours of admission) were included if they were: (1) available in the EHR of all participating hospitals, (2) routinely collected or available at the time of hospital discharge, and (3) plausible predictors of adverse outcomes based on prior literature and clinical expertise. These included length of stay, in‐hospital complications, transfer to an intensive or coronary care unit, blood transfusions, vital sign instabilities within 24 hours of discharge, select laboratory values at time of discharge, and disposition status. We also assessed trajectories of vital signs and selected laboratory values (defined as changes in these measures from admission to discharge).

Statistical Analysis

Model Derivation

Univariate relationships between readmission and each of the candidate predictors were assessed in the derivation cohort using a prespecified significance threshold of P 0.05. We included all factors from our previously derived and validated first‐day EHR model as candidate predictors.[15] Continuous laboratory and vital sign values at the time of discharge were categorized based on clinically meaningful cutoffs; predictors with missing values were assumed to be normal (<1% missing for each variable). Significant univariate candidate variables were entered in a multivariate logistic regression model using stepwise backward selection with a prespecified significance threshold of P 0.05. We performed several sensitivity analyses to confirm the robustness of our model. First, we alternately derived the full‐stay model using stepwise forward selection. Second, we forced in all significant variables from our first‐day EHR model, and entered the candidate variables from the remainder of the hospital stay using both stepwise backward and forward selection separately. Third, prespecified interactions between variables were evaluated for inclusion. Though final predictors varied slightly between the different approaches, discrimination of each model was similar to the model derived using our primary analytic approach (C statistics 0.01, data not shown).

Model Validation

We assessed model discrimination and calibration of the derived full‐stay EHR model using the validation cohort. Model discrimination was estimated by the C statistic. The C statistic represents the probability that, given 2 hospitalized individuals (1 who was readmitted and the other who was not), the model will predict a higher risk for the readmitted patient than for the nonreadmitted patient. Model calibration was assessed by comparing predicted to observed probabilities of readmission by quintiles of risk, and with the Hosmer‐Lemeshow goodness‐of‐fit test.

Comparison to Existing Models

We compared the full‐stay EHR model performance to 3 previously published models: our group's first‐day EHR model, and the LACE (includes Length of stay, Acute (nonelective) admission status, Charlson Comorbidity Index, and Emergency department visits in the past year) and HOSPITAL (includes Hemoglobin at discharge, discharge from Oncology service, Sodium level at discharge, Procedure during index hospitalization, Index hospitalization Type (nonelective), number of Admissions in the past year, and Length of stay) models, which were both derived to predict 30‐day readmissions among general medical inpatients and were intended to help clinicians identify high‐risk patients to target for discharge interventions.[9, 10, 15] We assessed each model's performance in our validation cohort, calculating the C statistic, integrated discrimination index (IDI), and net reclassification index (NRI) compared to the full‐stay model. IDI is a summary measure of both discrimination and reclassification, where more positive values suggest improvement in model performance in both these domains compared to a reference model.[28] The NRI is defined as the sum of the net proportions of correctly reclassified persons with and without the event of interest.[29] The theoretical range of values is 2 to 2, with more positive values indicating improved net reclassification compared to a reference model. Here, we calculated a category‐based NRI to evaluate the performance of models in correctly classifying individuals with and without readmissions into the highest readmission risk quintile versus the lowest 4 risk quintiles compared to the full‐stay EHR model.[29] This prespecified cutoff is relevant for hospitals interested in identifying the highest‐risk individuals for targeted intervention.[6] Because some hospitals may be able to target a greater number of individuals for intervention, we performed a sensitivity analysis by assessing category‐based NRI for reclassification into the top 2 risk quintiles versus the lowest 3 risk quintiles and found no meaningful difference in our results (data not shown). Finally, we qualitatively assessed calibration of comparator models in our validation cohort by comparing predicted probability to observed probability of readmission by quintiles of risk for each model. We conducted all analyses using Stata 12.1 (StataCorp, College Station, TX). This study was approved by the UT Southwestern Medical Center institutional review board.

RESULTS

Overall, 32,922 index hospitalizations were included in our study cohort; 12.7% resulted in a 30‐day readmission (see Supporting Figure 1 in the online version of this article). Individuals had a mean age of 62 years and had diverse race/ethnicity and primary insurance status; half were female (Table 1). The study sample was randomly split into a derivation cohort (50%, n = 16,492) and validation cohort (50%, n = 16,430). Individuals in the derivation cohort with a 30‐day readmission had markedly different socioeconomic and clinical characteristics compared to those not readmitted (Table 1).

Baseline Characteristics and Candidate Variables for Risk‐Prediction Model
Entire Cohort, N = 32,922 Derivation Cohort, N = 16,492

No Readmission, N = 14,312

Readmission, N = 2,180

P Value
  • NOTE: Abbreviations: ED, emergency department; ICU, intensive care unit; IQR, interquartile range; SD, standard deviation. *20% poverty in zip code as per high poverty area US Census designation. Prior ED visit at site of index hospitalization within the past year. Prior hospitalization at any of 75 acute care hospitals in the North Texas region within the past year. Nonelective admission defined as hospitalization categorized as medical emergency, urgent, or trauma. ∥Calculated from diagnoses available within 1 year prior to index hospitalization. Conditions were considered complications if they were not listed as a principle diagnosis for hospitalization or as a previous diagnosis in the prior year. #On day of discharge or last known observation before discharge. Instabilities were defined as temperature 37.8C, heart rate >100 beats/minute, respiratory rate >24 breaths/minute, systolic blood pressure 90 mm Hg, or oxygen saturation <90%. **Discharges to nursing home, skilled nursing facility, or long‐term acute care hospital.

Demographic characteristics
Age, y, mean (SD) 62 (17.3) 61 (17.4) 64 (17.0) 0.001
Female, n (%) 17,715 (53.8) 7,694 (53.8) 1,163 (53.3) 0.72
Race/ethnicity 0.001
White 21,359 (64.9) 9,329 (65.2) 1,361 (62.4)
Black 5,964 (18.1) 2,520 (17.6) 434 (19.9)
Hispanic 4,452 (13.5) 1,931 (13.5) 338 (15.5)
Other 1,147 (3.5) 532 (3.7) 47 (2.2)
Marital status, n (%) 0.001
Single 8,076 (24.5) 3,516 (24.6) 514 (23.6)
Married 13,394 (40.7) 5,950 (41.6) 812 (37.3)
Separated/divorced 3,468 (10.5) 1,460 (10.2) 251 (11.5)
Widowed 4,487 (13.7) 1,868 (13.1) 388 (17.8)
Other 3,497 (10.6) 1,518 (10.6) 215 (9.9)
Primary payer, n (%) 0.001
Private 13,090 (39.8) 5,855 (40.9) 726 (33.3)
Medicare 13,015 (39.5) 5,597 (39.1) 987 (45.3)
Medicaid 2,204 (6.7) 852 (5.9) 242 (11.1)
Charity, self‐pay, or other 4,613 (14.0) 2,008 (14.0) 225 (10.3)
High‐poverty neighborhood, n (%)* 7,468 (22.7) 3,208 (22.4) 548 (25.1) 0.001
Utilization history
1 ED visits in past year, n (%) 9,299 (28.2) 3,793 (26.5) 823 (37.8) 0.001
1 hospitalizations in past year, n (%) 10,189 (30.9) 4,074 (28.5) 1,012 (46.4) 0.001
Clinical factors from first day of hospitalization
Nonelective admission, n (%) 27,818 (84.5) 11,960 (83.6) 1,960 (89.9) 0.001
Charlson Comorbidity Index, median (IQR)∥ 0 (01) 0 (00) 0 (03) 0.001
Laboratory abnormalities within 24 hours of admission
Albumin <2 g/dL 355 (1.1) 119 (0.8) 46 (2.1) 0.001
Albumin 23 g/dL 4,732 (14.4) 1,956 (13.7) 458 (21.0) 0.001
Aspartate aminotransferase >40 U/L 4,610 (14.0) 1,922 (13.4) 383 (17.6) 0.001
Creatine phosphokinase <60 g/L 3,728 (11.3) 1,536 (10.7) 330 (15.1) 0.001
Mean corpuscular volume >100 fL/red cell 1,346 (4.1) 537 (3.8) 134 (6.2) 0.001
Platelets <90 103/L 912 (2.8) 357 (2.5) 116 (5.3) 0.001
Platelets >350 103/L 3,332 (10.1) 1,433 (10.0) 283 (13.0) 0.001
Prothrombin time >35 seconds 248 (0.8) 90 (0.6) 35 (1.6) 0.001
Clinical factors from remainder of hospital stay
Length of stay, d, median (IQR) 4 (26) 4 (26) 5 (38) 0.001
ICU transfer after first 24 hours, n (%) 988 (3.0) 408 (2.9) 94 (4.3) 0.001
Hospital complications, n (%)
Clostridium difficile infection 119 (0.4) 44 (0.3) 24 (1.1) 0.001
Pressure ulcer 358 (1.1) 126 (0.9) 46 (2.1) 0.001
Venous thromboembolism 301 (0.9) 112 (0.8) 34 (1.6) 0.001
Respiratory failure 1,048 (3.2) 463 (3.2) 112 (5.1) 0.001
Central line‐associated bloodstream infection 22 (0.07) 6 (0.04) 5 (0.23) 0.005
Catheter‐associated urinary tract infection 47 (0.14) 20 (0.14) 6 (0.28) 0.15
Acute myocardial infarction 293 (0.9) 110 (0.8) 32 (1.5) 0.001
Pneumonia 1,754 (5.3) 719 (5.0) 154 (7.1) 0.001
Sepsis 853 (2.6) 368 (2.6) 73 (3.4) 0.04
Blood transfusion during hospitalization, n (%) 4,511 (13.7) 1,837 (12.8) 425 (19.5) 0.001
Laboratory abnormalities at discharge#
Blood urea nitrogen >20 mg/dL, n (%) 10,014 (30.4) 4,077 (28.5) 929 (42.6) 0.001
Sodium <135 mEq/L, n (%) 4,583 (13.9) 1,850 (12.9) 440 (20.2) 0.001
Hematocrit 27 3,104 (9.4) 1,231 (8.6) 287 (13.2) 0.001
1 vital sign instability at discharge, n (%)# 6,192 (18.8) 2,624 (18.3) 525 (24.1) 0.001
Discharge location, n (%) 0.001
Home 23,339 (70.9) 10,282 (71.8) 1,383 (63.4)
Home health 3,185 (9.7) 1,356 (9.5) 234 (10.7)
Postacute care** 5,990 (18.2) 2,496 (17.4) 549 (25.2)
Hospice 408 (1.2) 178 (1.2) 14 (0.6)

Derivation and Validation of the Full‐Stay EHR Model for 30‐Day Readmission

Our final model included 24 independent variables, including demographic characteristics, utilization history, clinical factors from the first day of admission, and clinical factors from the remainder of the hospital stay (Table 2). The strongest independent predictor of readmission was hospital‐acquired Clostridium difficile infection (adjusted odds ratio [AOR]: 2.03, 95% confidence interval [CI] 1.18‐3.48); other hospital‐acquired complications including pressure ulcers and venous thromboembolism were also significant predictors. Though having Medicaid was associated with increased odds of readmission (AOR: 1.55, 95% CI: 1.31‐1.83), other zip codelevel measures of socioeconomic disadvantage were not predictive and were not included in the final model. Being discharged to hospice was associated with markedly lower odds of readmission (AOR: 0.23, 95% CI: 0.13‐0.40).

Final Full‐Stay EHR Model Predicting 30‐Day Readmissions (Derivation Cohort, N = 16,492)
Odds Ratio (95% CI)
Univariate Multivariate*
  • NOTE: Abbreviations: CI, confidence interval; ED, emergency department. *Values shown reflect adjusted odds ratios and 95% CI for each factor after adjustment for all other factors listed in the table.

Demographic characteristics
Age, per 10 years 1.08 (1.051.11) 1.07 (1.041.10)
Medicaid 1.97 (1.702.29) 1.55 (1.311.83)
Widow 1.44 (1.281.63) 1.27 (1.111.45)
Utilization history
Prior ED visit, per visit 1.08 (1.061.10) 1.04 (1.021.06)
Prior hospitalization, per hospitalization 1.30 (1.271.34) 1.16 (1.121.20)
Hospital and clinical factors from first day of hospitalization
Nonelective admission 1.75 (1.512.03) 1.42 (1.221.65)
Charlson Comorbidity Index, per point 1.19 (1.171.21) 1.06 (1.041.09)
Laboratory abnormalities within 24 hours of admission
Albumin <2 g/dL 2.57 (1.823.62) 1.52 (1.052.21)
Albumin 23 g/dL 1.68 (1.501.88) 1.20 (1.061.36)
Aspartate aminotransferase >40 U/L 1.37 (1.221.55) 1.21 (1.061.38)
Creatine phosphokinase <60 g/L 1.48 (1.301.69) 1.28 (1.111.46)
Mean corpuscular volume >100 fL/red cell 1.68 (1.382.04) 1.32 (1.071.62)
Platelets <90 103/L 2.20 (1.772.72) 1.56 (1.231.97)
Platelets >350 103/L 1.34 (1.171.54) 1.24 (1.081.44)
Prothrombin time >35 seconds 2.58 (1.743.82) 1.92 (1.272.90)
Hospital and clinical factors from remainder of hospital stay
Length of stay, per day 1.08 (1.071.09) 1.06 (1.041.07)
Hospital complications
Clostridium difficile infection 3.61 (2.195.95) 2.03 (1.183.48)
Pressure ulcer 2.43 (1.733.41) 1.64 (1.152.34)
Venous thromboembolism 2.01 (1.362.96) 1.55 (1.032.32)
Laboratory abnormalities at discharge
Blood urea nitrogen >20 mg/dL 1.86 (1.702.04) 1.37 (1.241.52)
Sodium <135 mEq/L 1.70 (1.521.91) 1.34 (1.181.51)
Hematocrit 27 1.61 (1.401.85) 1.22 (1.051.41)
Vital sign instability at discharge, per instability 1.29 (1.201.40) 1.25 (1.151.36)
Discharged to hospice 0.51 (0.300.89) 0.23 (0.130.40)

In our validation cohort, the full‐stay EHR model had fair discrimination, with a C statistic of 0.69 (95% CI: 0.68‐0.70) (Table 3). The full‐stay EHR model was well calibrated across all quintiles of risk, with slight overestimation of predicted risk in the lowest and highest quintiles (Figure 1a) (see Supporting Table 5 in the online version of this article). It also effectively stratified individuals across a broad range of predicted readmission risk from 4.1% in the lowest decile to 36.5% in the highest decile (Table 3).

Comparison of the Discrimination and Reclassification of Different Readmission Models*
Model Name C‐Statistic (95% CI) IDI, % (95% CI) NRI (95% CI) Average Predicted Risk, %
Lowest Decile Highest Decile
  • NOTE: Abbreviations; CI, confidence interval; EHR, electronic health record; IDI, Integrated Discrimination Improvement; NRI, Net Reclassification Index. *All measures were assessed using the validation cohort (N = 16,430), except for estimating the C‐statistic for the derivation cohort. P value <0.001 for all pairwise comparisons of C‐statistic between full‐stay model and first‐day, LACE, and HOSPITAL models, respectively. The LACE model includes Length of stay, Acute (nonelective) admission status, Charlson Comorbidity Index, and Emergency department visits in the past year. The HOSPITAL model includes Hemoglobin at discharge, discharge from Oncology service, Sodium level at discharge, Procedure during index hospitalization, Index hospitalization Type (nonelective), number of Admissions in the past year, and Length of stay.

Full‐stay EHR model
Derivation cohort 0.72 (0.70 to 0.73) 4.1 36.5
Validation cohort 0.69 (0.68 to 0.70) [Reference] [Reference] 4.1 36.5
First‐day EHR model 0.67 (0.66 to 0.68) 1.2 (1.4 to 1.0) 0.020 (0.038 to 0.002) 5.8 31.9
LACE model 0.65 (0.64 to 0.66) 2.6 (2.9 to 2.3) 0.046 (0.067 to 0.024) 6.1 27.5
HOSPITAL model 0.64 (0.62 to 0.65) 3.2 (3.5 to 2.9) 0.058 (0.080 to 0.035) 6.7 26.6
Figure 1
Comparison of the calibration of different readmission models. Calibration graphs for full‐stay (a), first‐day (b), LACE (c), and HOSPITAL (d) models in the validation cohort. Each graph shows predicted probability compared to observed probability of readmission by quintiles of risk for each model. The LACE model includes Length of stay, Acute (nonelective) admission status, Charlson Comorbidity Index, and Emergency department visits in the past year. The HOSPITAL model includes Hemoglobin at discharge, discharge from Oncology service, Sodium level at discharge, Procedure during index hospitalization, Index hospitalization Type (nonelective), number of Admissions in the past year, and Length of stay.

Comparing the Performance of the Full‐Stay EHR Model to Other Models

The full‐stay EHR model had better discrimination compared to the first‐day EHR model and the LACE and HOSPITAL models, though the magnitude of improvement was modest (Table 3). The full‐stay EHR model also stratified individuals across a broader range of readmission risk, and was better able to discriminate and classify those in the highest quintile of risk from those in the lowest 4 quintiles of risk compared to other models as assessed by the IDI and NRI (Table 3) (see Supporting Tables 14 and Supporting Figure 2 in the online version of this article). In terms of model calibration, both the first‐day EHR and LACE models were also well calibrated, whereas the HOSPITAL model was less robust (Figure 1).

The diagnostic accuracy of the full‐stay EHR model in correctly predicting those in the highest quintile of risk was better than that of the first‐day, LACE, and HOSPITAL models, though overall improvements in the sensitivity, specificity, positive and negative predictive values, and positive and negative likelihood ratios were also modest (see Supporting Table 6 in the online version of this article).

DISCUSSION

In this study, we used clinically detailed EHR data from the entire hospitalization on 32,922 individuals treated in 6 diverse hospitals to develop an all‐payer, multicondition readmission risk‐prediction model. To our knowledge, this is the first 30‐day hospital readmission risk‐prediction model to use a comprehensive set of factors from EHR data from the entire hospital stay. Prior EHR‐based models have focused exclusively on data available on or prior to the first day of admission, which account for clinical severity on admission but do not account for factors uncovered during the inpatient stay that influence the chance of a postdischarge adverse outcome.[15, 30] We specifically assessed the prognostic impact of a comprehensive set of factors from the entire index hospitalization, including hospital‐acquired complications, clinical trajectory, and stability on discharge in predicting hospital readmissions. Our full‐stay EHR model had statistically better discrimination, calibration, and diagnostic accuracy than our existing all‐cause first‐day EHR model[15] and 2 previously published readmissions models that included more limited information from hospitalization (such as length of stay).[9, 10] However, although the more complicated full‐stay EHR model was statistically better than previously published models, we were surprised that the predictive performance was only modestly improved despite the inclusion of many additional clinically relevant prognostic factors.

Taken together, our study has several important implications. First, the added complexity and resource intensity of implementing a full‐stay EHR model yields only modestly improved readmission risk prediction. Thus, hospitals and healthcare systems interested in targeting their highest‐risk individuals for interventions to reduce 30‐day readmission should consider doing so within the first day of hospital admission. Our group's previously derived and validated first‐day EHR model, which used data only from the first day of admission, qualitatively performed nearly as well as the full‐stay EHR model.[15] Additionally, a recent study using only preadmission EHR data to predict 30‐day readmissions also achieved similar discrimination and diagnostic accuracy as our full‐stay model.[30]

Second, the field of readmissions risk‐prediction modeling may be reaching the maximum achievable model performance using data that are currently available in the EHR. Our limited ability to accurately predict all‐cause 30‐day readmission risk may reflect the influence of currently unmeasured patient, system, and community factors on readmissions.[31, 32, 33] Due to the constraints of data collected in the EHR, we were unable to include several patient‐level clinical characteristics associated with hospital readmission, including self‐perceived health status, functional impairment, and cognition.[33, 34, 35, 36] However, given their modest effect sizes (ORs ranging from 1.062.10), adequately measuring and including these risk factors in our model may not meaningfully improve model performance and diagnostic accuracy. Further, many social and behavioral patient‐level factors are also not consistently available in EHR data. Though we explored the role of several neighborhood‐level socioeconomic measuresincluding prevalence of poverty, median income, education, and unemploymentwe found that none were significantly associated with 30‐day readmissions. These particular measures may have been inadequate to characterize individual‐level social and behavioral factors, as several previous studies have demonstrated that patient‐level factors such as social support, substance abuse, and medication and visit adherence can influence readmission risk in heart failure and pneumonia.[11, 16, 22, 25] This underscores the need for more standardized routine collection of data across functional, social, and behavioral domains in clinical settings, as recently championed by the Institute of Medicine.[11, 37] Integrating data from outside the EHR on postdischarge health behaviors, self‐management, follow‐up care, recovery, and home environment may be another important but untapped strategy for further improving prediction of readmissions.[25, 38]

Third, a multicondition readmission risk‐prediction model may be a less effective strategy than more customized disease‐specific models for selected conditions associated with high 30‐day readmission rates. Our group's previously derived and internally validated models for heart failure and human immunodeficiency virus had superior discrimination compared to our full‐stay EHR model (C statistic of 0.72 for each).[11, 13] However, given differences in the included population and time periods studied, a head‐to‐head comparison of these different strategies is needed to assess differences in model performance and utility.

Our study had several strengths. To our knowledge, this is the first study to rigorously measure the additive influence of in‐hospital complications, clinical trajectory, and stability on discharge on the risk of 30‐day hospital readmission. Additionally, our study included a large, diverse study population that included all payers, all ages of adults, a mix of community, academic, and safety net hospitals, and individuals from a broad array of racial/ethnic and socioeconomic backgrounds.

Our results should be interpreted in light of several limitations. First, though we sought to represent a diverse group of hospitals, all study sites were located within north Texas and generalizability to other regions is uncertain. Second, our ascertainment of prior hospitalizations and readmissions was more inclusive than what could be typically accomplished in real time using only EHR data from a single clinical site. We performed a sensitivity analysis using only prior utilization data available within the EHR from the index hospital with no meaningful difference in our findings (data not shown). Additionally, a recent study found that 30‐day readmissions occur at the index hospital for over 75% of events, suggesting that 30‐day readmissions are fairly comprehensively captured even with only single‐site data.[39] Third, we were not able to include data on outpatient visits before or after the index hospitalization, which may influence the risk of readmission.[1, 40]

In conclusion, incorporating clinically granular EHR data from the entire course of hospitalization modestly improves prediction of 30‐day readmissions compared to models that only include information from the first 24 hours of hospital admission or models that use far fewer variables. However, given the limited improvement in prediction, our findings suggest that from the practical perspective of implementing real‐time models to identify those at highest risk for readmission, it may not be worth the added complexity of waiting until the end of a hospitalization to leverage additional data on hospital complications, and the trajectory of laboratory and vital sign values currently available in the EHR. Further improvement in prediction of readmissions will likely require accounting for psychosocial, functional, behavioral, and postdischarge factors not currently present in the inpatient EHR.

Disclosures: This study was presented at the Society of Hospital Medicine 2015 Annual Meeting in National Harbor, Maryland, and the Society of General Internal Medicine 2015 Annual Meeting in Toronto, Canada. This work was supported by the Agency for Healthcare Research and Qualityfunded UT Southwestern Center for Patient‐Centered Outcomes Research (1R24HS022418‐01) and the Commonwealth Foundation (#20100323). Drs. Nguyen and Makam received funding from the UT Southwestern KL2 Scholars Program (NIH/NCATS KL2 TR001103). Dr. Halm was also supported in part by NIH/NCATS U54 RFA‐TR‐12‐006. The study sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. The authors have no conflicts of interest to disclose.

References
  1. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee‐for‐service program. N Engl J Med. 2009;360(14):14181428.
  2. Walraven C, Bennett C, Jennings A, Austin PC, Forster AJ. Proportion of hospital readmissions deemed avoidable: a systematic review. CMAJ. 2011;183(7):E391E402.
  3. Rennke S, Nguyen OK, Shoeb MH, Magan Y, Wachter RM, Ranji SR. Hospital‐initiated transitional care interventions as a patient safety strategy: a systematic review. Ann Intern Med. 2013;158(5 pt 2):433440.
  4. Hansen LO, Young RS, Hinami K, Leung A, Williams MV. Interventions to reduce 30‐day rehospitalization: a systematic review. Ann Intern Med. 2011;155(8):520528.
  5. Rennke S, Shoeb MH, Nguyen OK, Magan Y, Wachter RM, Ranji SR. Interventions to Improve Care Transitions at Hospital Discharge. Rockville, MD: Agency for Healthcare Research and Quality; 2013.
  6. Amarasingham R, Patel PC, Toto K, et al. Allocating scarce resources in real‐time to reduce heart failure readmissions: a prospective, controlled study. BMJ Qual Saf. 2013;22(12):9981005.
  7. Amarasingham R, Patzer RE, Huesch M, Nguyen NQ, Xie B. Implementing electronic health care predictive analytics: considerations and challenges. Health Aff (Millwood). 2014;33(7):11481154.
  8. Kansagara D, Englander H, Salanitro A, et al. Risk prediction models for hospital readmission: a systematic review. JAMA. 2011;306(15):16881698.
  9. Walraven C, Dhalla IA, Bell C, et al. Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community. CMAJ. 2010;182(6):551557.
  10. Donze J, Aujesky D, Williams D, Schnipper JL. Potentially avoidable 30‐day hospital readmissions in medical patients: derivation and validation of a prediction model. JAMA Intern Med. 2013;173(8):632638.
  11. Amarasingham R, Moore BJ, Tabak YP, et al. An automated model to identify heart failure patients at risk for 30‐day readmission or death using electronic medical record data. Med Care. 2010;48(11):981988.
  12. Singal AG, Rahimi RS, Clark C, et al. An automated model using electronic medical record data identifies patients with cirrhosis at high risk for readmission. Clin Gastroenterol Hepatol. 2013;11(10):13351341.e1331.
  13. Nijhawan AE, Clark C, Kaplan R, Moore B, Halm EA, Amarasingham R. An electronic medical record‐based model to predict 30‐day risk of readmission and death among HIV‐infected inpatients. J Acquir Immune Defic Syndr. 2012;61(3):349358.
  14. Horwitz LI, Partovian C, Lin Z, et al. Development and use of an administrative claims measure for profiling hospital‐wide performance on 30‐day unplanned readmission. Ann Intern Med. 2014;161(10 suppl):S66S75.
  15. Amarasingham R, Velasco F, Xie B, et al. Electronic medical record‐based multicondition models to predict the risk of 30 day readmission or death among adult medicine patients: validation and comparison to existing models. BMC Med Inform Decis Mak. 2015;15(1):39.
  16. Watson AJ, O'Rourke J, Jethwani K, et al. Linking electronic health record‐extracted psychosocial data in real‐time to risk of readmission for heart failure. Psychosomatics. 2011;52(4):319327.
  17. Ashton CM, Wray NP. A conceptual framework for the study of early readmission as an indicator of quality of care. Soc Sci Med. 1996;43(11):15331541.
  18. Dharmarajan K, Hsieh AF, Lin Z, et al. Hospital readmission performance and patterns of readmission: retrospective cohort study of Medicare admissions. BMJ. 2013;347:f6571.
  19. Cassel CK, Conway PH, Delbanco SF, Jha AK, Saunders RS, Lee TH. Getting more performance from performance measurement. N Engl J Med. 2014;371(23):21452147.
  20. Bradley EH, Sipsma H, Horwitz LI, et al. Hospital strategy uptake and reductions in unplanned readmission rates for patients with heart failure: a prospective study. J Gen Intern Med. 2015;30(5):605611.
  21. Krumholz HM. Post‐hospital syndrome—an acquired, transient condition of generalized risk. N Engl J Med. 2013;368(2):100102.
  22. Calvillo‐King L, Arnold D, Eubank KJ, et al. Impact of social factors on risk of readmission or mortality in pneumonia and heart failure: systematic review. J Gen Intern Med. 2013;28(2):269282.
  23. Keyhani S, Myers LJ, Cheng E, Hebert P, Williams LS, Bravata DM. Effect of clinical and social risk factors on hospital profiling for stroke readmission: a cohort study. Ann Intern Med. 2014;161(11):775784.
  24. Kind AJ, Jencks S, Brock J, et al. Neighborhood socioeconomic disadvantage and 30‐day rehospitalization: a retrospective cohort study. Ann Intern Med. 2014;161(11):765774.
  25. Arbaje AI, Wolff JL, Yu Q, Powe NR, Anderson GF, Boult C. Postdischarge environmental and socioeconomic factors and the likelihood of early hospital readmission among community‐dwelling Medicare beneficiaries. Gerontologist. 2008;48(4):495504.
  26. Hu J, Gonsahn MD, Nerenz DR. Socioeconomic status and readmissions: evidence from an urban teaching hospital. Health Aff (Millwood). 2014;33(5):778785.
  27. Nagasako EM, Reidhead M, Waterman B, Dunagan WC. Adding socioeconomic data to hospital readmissions calculations may produce more useful results. Health Aff (Millwood). 2014;33(5):786791.
  28. Pencina MJ, D'Agostino RB, D'Agostino RB, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157172; discussion 207–212.
  29. Leening MJ, Vedder MM, Witteman JC, Pencina MJ, Steyerberg EW. Net reclassification improvement: computation, interpretation, and controversies: a literature review and clinician's guide. Ann Intern Med. 2014;160(2):122131.
  30. Shadmi E, Flaks‐Manov N, Hoshen M, Goldman O, Bitterman H, Balicer RD. Predicting 30‐day readmissions with preadmission electronic health record data. Med Care. 2015;53(3):283289.
  31. Kangovi S, Grande D. Hospital readmissions—not just a measure of quality. JAMA. 2011;306(16):17961797.
  32. Joynt KE, Jha AK. Thirty‐day readmissions—truth and consequences. N Engl J Med. 2012;366(15):13661369.
  33. Greysen SR, Stijacic Cenzer I, Auerbach AD, Covinsky KE. Functional impairment and hospital readmission in medicare seniors. JAMA Intern Med. 2015;175(4):559565.
  34. Holloway JJ, Thomas JW, Shapiro L. Clinical and sociodemographic risk factors for readmission of Medicare beneficiaries. Health Care Financ Rev. 1988;10(1):2736.
  35. Patel A, Parikh R, Howell EH, Hsich E, Landers SH, Gorodeski EZ. Mini‐cog performance: novel marker of post discharge risk among patients hospitalized for heart failure. Circ Heart Fail. 2015;8(1):816.
  36. Hoyer EH, Needham DM, Atanelov L, Knox B, Friedman M, Brotman DJ. Association of impaired functional status at hospital discharge and subsequent rehospitalization. J Hosp Med. 2014;9(5):277282.
  37. Adler NE, Stead WW. Patients in context—EHR capture of social and behavioral determinants of health. N Engl J Med. 2015;372(8):698701.
  38. Nguyen OK, Chan CV, Makam A, Stieglitz H, Amarasingham R. Envisioning a social‐health information exchange as a platform to support a patient‐centered medical neighborhood: a feasibility study. J Gen Intern Med. 2015;30(1):6067.
  39. Henke RM, Karaca Z, Lin H, Wier LM, Marder W, Wong HS. Patient factors contributing to variation in same‐hospital readmission rate. Med Care Res Review. 2015;72(3):338358.
  40. Weinberger M, Oddone EZ, Henderson WG. Does increased access to primary care reduce hospital readmissions? Veterans Affairs Cooperative Study Group on Primary Care and Hospital Readmission. N Engl J Med. 1996;334(22):14411447.
References
  1. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee‐for‐service program. N Engl J Med. 2009;360(14):14181428.
  2. Walraven C, Bennett C, Jennings A, Austin PC, Forster AJ. Proportion of hospital readmissions deemed avoidable: a systematic review. CMAJ. 2011;183(7):E391E402.
  3. Rennke S, Nguyen OK, Shoeb MH, Magan Y, Wachter RM, Ranji SR. Hospital‐initiated transitional care interventions as a patient safety strategy: a systematic review. Ann Intern Med. 2013;158(5 pt 2):433440.
  4. Hansen LO, Young RS, Hinami K, Leung A, Williams MV. Interventions to reduce 30‐day rehospitalization: a systematic review. Ann Intern Med. 2011;155(8):520528.
  5. Rennke S, Shoeb MH, Nguyen OK, Magan Y, Wachter RM, Ranji SR. Interventions to Improve Care Transitions at Hospital Discharge. Rockville, MD: Agency for Healthcare Research and Quality; 2013.
  6. Amarasingham R, Patel PC, Toto K, et al. Allocating scarce resources in real‐time to reduce heart failure readmissions: a prospective, controlled study. BMJ Qual Saf. 2013;22(12):9981005.
  7. Amarasingham R, Patzer RE, Huesch M, Nguyen NQ, Xie B. Implementing electronic health care predictive analytics: considerations and challenges. Health Aff (Millwood). 2014;33(7):11481154.
  8. Kansagara D, Englander H, Salanitro A, et al. Risk prediction models for hospital readmission: a systematic review. JAMA. 2011;306(15):16881698.
  9. Walraven C, Dhalla IA, Bell C, et al. Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community. CMAJ. 2010;182(6):551557.
  10. Donze J, Aujesky D, Williams D, Schnipper JL. Potentially avoidable 30‐day hospital readmissions in medical patients: derivation and validation of a prediction model. JAMA Intern Med. 2013;173(8):632638.
  11. Amarasingham R, Moore BJ, Tabak YP, et al. An automated model to identify heart failure patients at risk for 30‐day readmission or death using electronic medical record data. Med Care. 2010;48(11):981988.
  12. Singal AG, Rahimi RS, Clark C, et al. An automated model using electronic medical record data identifies patients with cirrhosis at high risk for readmission. Clin Gastroenterol Hepatol. 2013;11(10):13351341.e1331.
  13. Nijhawan AE, Clark C, Kaplan R, Moore B, Halm EA, Amarasingham R. An electronic medical record‐based model to predict 30‐day risk of readmission and death among HIV‐infected inpatients. J Acquir Immune Defic Syndr. 2012;61(3):349358.
  14. Horwitz LI, Partovian C, Lin Z, et al. Development and use of an administrative claims measure for profiling hospital‐wide performance on 30‐day unplanned readmission. Ann Intern Med. 2014;161(10 suppl):S66S75.
  15. Amarasingham R, Velasco F, Xie B, et al. Electronic medical record‐based multicondition models to predict the risk of 30 day readmission or death among adult medicine patients: validation and comparison to existing models. BMC Med Inform Decis Mak. 2015;15(1):39.
  16. Watson AJ, O'Rourke J, Jethwani K, et al. Linking electronic health record‐extracted psychosocial data in real‐time to risk of readmission for heart failure. Psychosomatics. 2011;52(4):319327.
  17. Ashton CM, Wray NP. A conceptual framework for the study of early readmission as an indicator of quality of care. Soc Sci Med. 1996;43(11):15331541.
  18. Dharmarajan K, Hsieh AF, Lin Z, et al. Hospital readmission performance and patterns of readmission: retrospective cohort study of Medicare admissions. BMJ. 2013;347:f6571.
  19. Cassel CK, Conway PH, Delbanco SF, Jha AK, Saunders RS, Lee TH. Getting more performance from performance measurement. N Engl J Med. 2014;371(23):21452147.
  20. Bradley EH, Sipsma H, Horwitz LI, et al. Hospital strategy uptake and reductions in unplanned readmission rates for patients with heart failure: a prospective study. J Gen Intern Med. 2015;30(5):605611.
  21. Krumholz HM. Post‐hospital syndrome—an acquired, transient condition of generalized risk. N Engl J Med. 2013;368(2):100102.
  22. Calvillo‐King L, Arnold D, Eubank KJ, et al. Impact of social factors on risk of readmission or mortality in pneumonia and heart failure: systematic review. J Gen Intern Med. 2013;28(2):269282.
  23. Keyhani S, Myers LJ, Cheng E, Hebert P, Williams LS, Bravata DM. Effect of clinical and social risk factors on hospital profiling for stroke readmission: a cohort study. Ann Intern Med. 2014;161(11):775784.
  24. Kind AJ, Jencks S, Brock J, et al. Neighborhood socioeconomic disadvantage and 30‐day rehospitalization: a retrospective cohort study. Ann Intern Med. 2014;161(11):765774.
  25. Arbaje AI, Wolff JL, Yu Q, Powe NR, Anderson GF, Boult C. Postdischarge environmental and socioeconomic factors and the likelihood of early hospital readmission among community‐dwelling Medicare beneficiaries. Gerontologist. 2008;48(4):495504.
  26. Hu J, Gonsahn MD, Nerenz DR. Socioeconomic status and readmissions: evidence from an urban teaching hospital. Health Aff (Millwood). 2014;33(5):778785.
  27. Nagasako EM, Reidhead M, Waterman B, Dunagan WC. Adding socioeconomic data to hospital readmissions calculations may produce more useful results. Health Aff (Millwood). 2014;33(5):786791.
  28. Pencina MJ, D'Agostino RB, D'Agostino RB, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157172; discussion 207–212.
  29. Leening MJ, Vedder MM, Witteman JC, Pencina MJ, Steyerberg EW. Net reclassification improvement: computation, interpretation, and controversies: a literature review and clinician's guide. Ann Intern Med. 2014;160(2):122131.
  30. Shadmi E, Flaks‐Manov N, Hoshen M, Goldman O, Bitterman H, Balicer RD. Predicting 30‐day readmissions with preadmission electronic health record data. Med Care. 2015;53(3):283289.
  31. Kangovi S, Grande D. Hospital readmissions—not just a measure of quality. JAMA. 2011;306(16):17961797.
  32. Joynt KE, Jha AK. Thirty‐day readmissions—truth and consequences. N Engl J Med. 2012;366(15):13661369.
  33. Greysen SR, Stijacic Cenzer I, Auerbach AD, Covinsky KE. Functional impairment and hospital readmission in medicare seniors. JAMA Intern Med. 2015;175(4):559565.
  34. Holloway JJ, Thomas JW, Shapiro L. Clinical and sociodemographic risk factors for readmission of Medicare beneficiaries. Health Care Financ Rev. 1988;10(1):2736.
  35. Patel A, Parikh R, Howell EH, Hsich E, Landers SH, Gorodeski EZ. Mini‐cog performance: novel marker of post discharge risk among patients hospitalized for heart failure. Circ Heart Fail. 2015;8(1):816.
  36. Hoyer EH, Needham DM, Atanelov L, Knox B, Friedman M, Brotman DJ. Association of impaired functional status at hospital discharge and subsequent rehospitalization. J Hosp Med. 2014;9(5):277282.
  37. Adler NE, Stead WW. Patients in context—EHR capture of social and behavioral determinants of health. N Engl J Med. 2015;372(8):698701.
  38. Nguyen OK, Chan CV, Makam A, Stieglitz H, Amarasingham R. Envisioning a social‐health information exchange as a platform to support a patient‐centered medical neighborhood: a feasibility study. J Gen Intern Med. 2015;30(1):6067.
  39. Henke RM, Karaca Z, Lin H, Wier LM, Marder W, Wong HS. Patient factors contributing to variation in same‐hospital readmission rate. Med Care Res Review. 2015;72(3):338358.
  40. Weinberger M, Oddone EZ, Henderson WG. Does increased access to primary care reduce hospital readmissions? Veterans Affairs Cooperative Study Group on Primary Care and Hospital Readmission. N Engl J Med. 1996;334(22):14411447.
Issue
Journal of Hospital Medicine - 11(7)
Issue
Journal of Hospital Medicine - 11(7)
Page Number
473-480
Page Number
473-480
Article Type
Display Headline
Predicting all‐cause readmissions using electronic health record data from the entire hospitalization: Model development and comparison
Display Headline
Predicting all‐cause readmissions using electronic health record data from the entire hospitalization: Model development and comparison
Sections
Article Source
© 2016 Society of Hospital Medicine
Disallow All Ads
Correspondence Location
Address for correspondence and reprint requests: Oanh Kieu Nguyen, MD, 5323 Harry Hines Blvd., Dallas, Texas 75390‐9169; Telephone: 214‐648‐3135; Fax: 214‐648‐3232; E‐mail: oanhK.nguyen@UTSouthwestern.edu
Content Gating
Gated (full article locked unless allowed per User)
Gating Strategy
First Peek Free
Article PDF Media
Media Files

Financial Performance and Outcomes

Article Type
Changed
Mon, 01/02/2017 - 19:34
Display Headline
Relationship between hospital financial performance and publicly reported outcomes

Hospital care accounts for the single largest category of national healthcare expenditures, totaling $936.9 billion in 2013.[1] With ongoing scrutiny of US healthcare spending, hospitals are under increasing pressure to justify high costs and robust profits.[2] However, the dominant fee‐for‐service reimbursement model creates incentives for hospitals to prioritize high volume over high‐quality care to maximize profits.[3] Because hospitals may be reluctant to implement improvements if better quality is not accompanied by better payment or improved financial margins, an approach to stimulate quality improvement among hospitals has been to leverage consumer pressure through required public reporting of selected outcome metrics.[4, 5] Public reporting of outcomes is thought to influence hospital reputation; in turn, reputation affects patient perceptions and influences demand for hospital services, potentially enabling reputable hospitals to command higher prices for services to enhance hospital revenue.[6, 7]

Though improving outcomes is thought to reduce overall healthcare costs, it is unclear whether improving outcomes results in a hospital's financial return on investment.[4, 5, 8] Quality improvement can require substantial upfront investment, requiring that hospitals already have robust financial health to engage in such initiatives.[9, 10] Consequently, instead of stimulating broad efforts in quality improvement, public reporting may exacerbate existing disparities in hospital quality and finances, by rewarding already financially healthy hospitals, and by inadvertently penalizing hospitals without the means to invest in quality improvement.[11, 12, 13, 14, 15] Alternately, because fee‐for‐service remains the dominant reimbursement model for hospitals, loss of revenue through reducing readmissions may outweigh any financial gains from improved public reputation and result in worse overall financial performance, though robust evidence for this concern is lacking.[16, 17]

A small number of existing studies suggest a limited correlation between improved hospital financial performance and improved quality, patient safety, and lower readmission rates.[18, 19, 20] However, these studies had several limitations. They were conducted prior to public reporting of selected outcome metrics by the Centers for Medicare and Medicaid Services (CMS)[18, 19, 20]; used data from the Medicare Cost Report, which is not uniformly audited and thus prone to measurement error[19, 20]; used only relative measures of hospital financial performance (eg, operating margin), which do not capture the absolute amount of revenue potentially available for investment in quality improvement[18, 19]; or compared only hospitals at the extremes of financial performance, potentially exaggerating the magnitude of the relationship between hospital financial performance and quality outcomes.[19]

To address this gap in the literature, we sought to assess whether hospitals with robust financial performance have lower 30‐day risk‐standardized mortality and hospital readmission rates for acute myocardial infarction (AMI), congestive heart failure (CHF), and pneumonia (PNA). Given the concern that hospitals with the lowest mortality and readmission rates may experience a decrease in financial performance due to the lower volume of hospitalizations, we also assessed whether hospitals with the lowest readmission and mortality rates had a differential change in financial performance over time compared to hospitals with the highest rates.

METHODS

Data Sources and Study Population

This was an observational study using audited financial data from the 2008 and 2012 Hospital Annual Financial Data Files from the Office of Statewide Health Planning and Development (OSHPD) in the state of California, merged with data on outcome measures publicly reported by CMS via the Hospital Compare website for July 1, 2008 to June 30, 2011.[21, 22] We included all general acute care hospitals with available OSHPD data in 2008 and at least 1 publicly reported outcome from 2008 to 2011. We excluded hospitals without 1 year of audited financial data for 2008 and hospitals that closed during 2008 to 2011.

Measures of Financial Performance

Because we hypothesized that the absolute amount of revenue generated from clinical operations would influence investment in quality improvement programs more so than relative changes in revenue,[20] we used net revenue from operations (total operating revenue minus total operating expense) as our primary measure of hospital financial performance. We also performed 2 companion analyses using 2 commonly reported relative measures of financial performanceoperating margin (net revenue from operations divided by total operating revenue) and total margin (net total revenue divided by total revenue from all sources). Net revenue from operations for 2008 was adjusted to 2012 US dollars using the chained Consumer Price Index for all urban consumers.

Outcomes

For our primary analysis, the primary outcomes were publicly reported all‐cause 30‐day risk‐standardized mortality rates (RSMR) and readmission rates (RSRR) for AMI, CHF, and PNA aggregated over a 3‐year period. These measures were adjusted for key demographic and clinical characteristics available in Medicare data. CMS began publicly reporting 30‐day RSMR for AMI and CHF in June 2007, RSMR for PNA in June 2008, and RSRR for all 3 conditions in July 2009.[23, 24]

To assess whether public reporting had an effect on subsequent hospital financial performance, we conducted a companion analysis where the primary outcome of interest was change in hospital financial performance over time, using the same definitions of financial performance outlined above. For this companion analysis, publicly reported 30‐day RSMR and RSRR for AMI, CHF, and PNA were assessed as predictors of subsequent financial performance.

Hospital Characteristics

Hospital characteristics were ascertained from the OSHPD data. Safety‐net status was defined as hospitals with an annual Medicaid caseload (number of Medicaid discharges divided by the total number of discharges) 1 standard deviation above the mean Medicaid caseload, as defined in previous studies.[25]

Statistical Analyses

Effect of Baseline Financial Performance on Subsequent Publicly Reported Outcomes

To estimate the relationship between baseline hospital financial performance in 2008 and subsequent RSMR and RSRR for AMI, CHF, and PNA from 2008 to 2011, we used linear regression adjusted for the following hospital characteristics: teaching status, rural location, bed size, safety‐net status, ownership, Medicare caseload, and volume of cases reported for the respective outcome. We accounted for clustering of hospitals by ownership. We adjusted for hospital volume of reported cases for each condition given that the risk‐standardization models used by CMS shrink outcomes for small hospitals to the mean, and therefore do not account for a potential volume‐outcome relationship.[26] We conducted a sensitivity analysis excluding hospitals at the extremes of financial performance, defined as hospitals with extreme outlier values for each financial performance measure (eg, values more than 3 times the interquartile range above the first quartile or below the third quartile).[27] Nonlinearity of financial performance measures was assessed using restricted cubic splines. For ease of interpretation, we scaled the estimated change in RSMR and RSRR per $50 million increase in net revenue from operations, and graphed nonparametric relationships using restricted cubic splines.

Effect of Public Reporting on Subsequent Hospital Financial Performance

To assess whether public reporting had an effect on subsequent hospital financial performance, we conducted a companion hospital‐level difference‐in‐differences analysis to assess for differential changes in hospital financial performance between 2008 and 2012, stratified by tertiles of RSMR and RSRR rates from 2008 to 2011. This approach compares differences in an outcome of interest (hospital financial performance) within each group (where each group is a tertile of publicly reported rates of RSMR or RSRR), and then compares the difference in these differences between groups. Therefore, these analyses use each group as their own historical control and the opposite group as a concurrent control to account for potential secular trends. To conduct our difference‐in‐differences analysis, we compared the change in financial performance over time in the top tertile of hospitals to the change in financial performance over time in the bottom tertile of hospitals with respect to AMI, CHF, and PNA RSMR and RSRR. Our models therefore included year (2008 vs 2012), tertile of publicly reported rates for RSMR or RSRR, and the interaction between them as predictors, where the interaction was the difference‐in‐differences term and the primary predictor of interest. In addition to adjusting for hospital characteristics and accounting for clustering as mentioned above, we also included 3 separate interaction terms for year with bed size, safety‐net status, and Medicare caseload, to account for potential changes in the hospitals over time that may have independently influenced financial performance and publicly reported 30‐day measures. For sensitivity analyses, we repeated our difference‐in‐differences analyses excluding hospitals with a change in ownership and extreme outliers with respect to financial performance in 2008. We performed model diagnostics including assessment of functional form, linearity, normality, constant variance, and model misspecification. All analyses were conducted using Stata version 12.1 (StataCorp, College Station, TX). This study was deemed exempt from review by the UT Southwestern Medical Center institutional review board.

RESULTS

Among the 279 included hospitals (see Supporting Figure 1 in the online version of this article), 278 also had financial data available for 2012. In 2008, the median net revenue from operations was $1.6 million (interquartile range [IQR], $2.4 to $10.3 million), the median operating margin was 1.5% (IQR, 4.6% to 6%), and the median total margin was 2.5% (IQR, 2.2% to 7.5% (Table 1). The number of hospitals reporting each outcome, and median outcome rates, are shown in Table 2.

Hospital Characteristics and Financial Performance in 2008 and 2012
2008, n = 279 2012, n = 278
  • NOTE: Abbreviations: IQR, interquartile range; SD, standard deviation. *Medicaid caseload equivalent to 1 standard deviation above the mean (41.8% for 2008 and 42.1% for 2012). Operated by an investor‐individual, investor‐partnership, or investor‐corporation.

Hospital characteristics
Teaching, n (%) 28 (10.0) 28 (10.0)
Rural, n (%) 55 (19.7) 55 (19.7)
Bed size, n (%)
099 (small) 57 (20.4) 55 (19.8)
100299 (medium) 130 (46.6) 132 (47.5)
300 (large) 92 (33.0) 91 (32.7)
Safety‐net hospital, n (%)* 46 (16.5) 48 (17.3)
Hospital ownership, n (%)
City or county 15 (5.4) 16 (5.8)
District 42 (15.1) 39 (14.0)
Investor 66 (23.7) 66 (23.7)
Private nonprofit 156 (55.9) 157 (56.5)
Medicare caseload, mean % (SD) 41.6 (14.7) 43.6 (14.7)
Financial performance measures
Net revenue from operations, median $ in millions (IQR; range) 1.6 (2.4 to 10.3; 495.9 to 144.1) 3.2 (2.9 to 15.4; 396.2 to 276.8)
Operating margin, median % (IQR; range) 1.5 (4.6 to 6.8; 77.8 to 26.4) 2.3 (3.9 to 8.2; 134.8 to 21.1)
Total margin, median % (IQR; range) 2.5 (2.2 to 7.5; 101.0 to 26.3) 4.5 (0.7 to 9.8; 132.2 to 31.1)
Relationship Between Hospital Financial Performance and 30‐Day Mortality and Readmission Rates*
No. Median % (IQR) Adjusted % Change (95% CI) per $50 Million Increase in Net Revenue From Operations
Overall Extreme Outliers Excluded
  • NOTE: Abbreviations: CI, confidence interval; IQR, interquartile range. *Thirty‐day outcomes are risk standardized for age, sex, comorbidity count, and indicators of patient frailty.[3] Each outcome was modeled separately and adjusted for teaching status, metropolitan status (urban vs rural), bed size, safety‐net hospital status, hospital ownership type, Medicare caseload, and volume of cases reported for the respective outcome, accounting for clustering of hospitals by owner. Twenty‐three hospitals were identified as extreme outliers with respect to net revenue from operations (10 underperformers with net revenue <$49.4 million and 13 overperformers with net revenue >$52.1 million). There was a nonlinear and statistically significant relationship between net revenue from operations and readmission rate for myocardial infarction. Net revenue from operations was modeled as a cubic spline function. See Figure 1. The overall adjusted F statistic was 4.8 (P < 0.001). There was a nonlinear and statistically significant relationship between net revenue from operations and mortality rate for heart failure after exclusion of extreme outliers. Net revenue from operations was modeled as a cubic spline function. See Figure 1.The overall adjusted F statistic was 3.6 (P = 0.008).

Myocardial infarction
Mortality rate 211 15.2 (14.216.2) 0.07 (0.10 to 0.24) 0.63 (0.21 to 1.48)
Readmission rate 184 19.4 (18.520.2) Nonlinear 0.34 (1.17 to 0.50)
Congestive heart failure
Mortality rate 259 11.1 (10.112.1) 0.17 (0.01 to 0.35) Nonlinear
Readmission rate 264 24.5 (23.525.6) 0.07 (0.27 to 0.14) 0.45 (1.36 to 0.47)
Pneumonia
Mortality rate 268 11.6 (10.413.2) 0.17 (0.42 to 0.07) 0.35 (1.19 to 0.49)
Readmission rate 268 18.2 (17.319.1) 0.04 (0.20 to 0.11) 0.56 (1.27 to 0.16)

Relationship Between Financial Performance and Publicly Reported Outcomes

Acute Myocardial Infarction

We did not observe a consistent relationship between hospital financial performance and AMI mortality and readmission rates. In our overall adjusted analyses, net revenue from operations was not associated with mortality, but was significantly associated with a decrease in AMI readmissions among hospitals with net revenue from operations between approximately $5 million to $145 million (nonlinear relationship, F statistic = 4.8, P < 0.001 (Table 2, Figure 1A). However, after excluding 23 extreme outlying hospitals by net revenue from operations (10 underperformers with net revenue <$49.4 million and 13 overperformers with net revenue >$52.1 million), this relationship was no longer observed. Using operating margin instead of net revenue from operations as the measure of hospital financial performance, we observed a 0.2% increase in AMI mortality (95% confidence interval [CI]: 0.06%‐0.35%) (see Supporting Table 1 and Supporting Figure 2 in the online version of this article) for each 10% increase in operating margin, which persisted with the exclusion of 5 outlying hospitals by operating margin (all 5 were underperformers, with operating margins <38.6%). However, using total margin as the measure of financial performance, there was no significant relationship with either mortality or readmissions (see Supporting Table 2 and Supporting Figure 3 in the online version of this article).

Figure 1
Relationship between financial performance and 30‐day readmission and mortality. The open circles represent individual hospitals. The bold dashed line and the bold solid line are the unadjusted and adjusted cubic spline curves, respectively, representing the nonlinear relationship between net revenue from operations and each outcome. The shaded grey area represents the 95% confidence interval for the adjusted cubic spline curve. Thin vertical dashed lines represent median values for net revenue from operations. Multivariate models were adjusted for teaching status, metropolitan status (urban vs rural), bed size, safety‐net hospital status, hospital ownership, Medicare caseload, and volume of cases reported for the respective outcome, accounting for clustering of hospitals by owner. *Twenty‐three hospitals were identified as outliers with respect to net revenue from clinical operations (10 “underperformers” with net revenue <−$49.4 million and 13 “overperformers” with net revenue >$52.1 million.

Congestive Heart Failure

In our primary analyses, we did not observe a significant relationship between net revenue from operations and CHF mortality and readmission rates. However, after excluding 23 extreme outliers, increasing net revenue from operations was associated with a modest increase in CHF mortality among hospitals, with net revenue between approximately $35 million and $20 million (nonlinear relationship, F statistic = 3.6, P = 0.008 (Table 2, Figure 1B). Using alternate measures of financial performance, we observed a consistent relationship between increasing hospital financial performance and higher 30‐day CHF mortality rate. Using operating margin, we observed a slight increase in the mortality rate for CHF (0.26% increase in CHF RSMR for every 10% increase in operating margin) (95% CI: 0.07%‐0.45%) (see Supporting Table 1 and Supporting Figure 2 in the online version of this article), which persisted after the exclusion of 5 extreme outliers. Using total margin, we observed a significant but modest association between improved hospital financial performance and increased mortality rate for CHF (nonlinear relationship, F statistic = 2.9, P = 0.03) (see Supporting Table 2 and Supporting Figure 3 in the online version of this article), which persisted after the exclusion of 3 extreme outliers (0.32% increase in CHF RSMR for every 10% increase in total margin) (95% CI: 0.03%‐0.62%).

Pneumonia

Hospital financial performance (using net revenue, operating margin, or total margin) was not associated with 30‐day PNA mortality or readmission rates.

Relationship of Readmission and Mortality Rates on Subsequent Hospital Financial Performance

Compared to hospitals in the highest tertile of readmission and mortality rates (ie, those with the worst rates), hospitals in the lowest tertile of readmission and mortality rates (ie, those with the best rates) had a similar magnitude of increase in net revenue from operations from 2008 to 2012 (Table 3). The difference‐in‐differences analyses showed no relationship between readmission or mortality rates for AMI, CHF, and PNA and changes in net revenue from operations from 2008 to 2012 (difference‐in‐differences estimates ranged from $8.61 to $6.77 million, P > 0.3 for all). These results were robust to the exclusion of hospitals with a change in ownership and extreme outliers by net revenue from operations (data not reported).

Difference in the Differences in Financial Performance Between the Worst‐ and the Best‐Performing Hospitals
Outcome Tertile With Highest Outcome Rates (Worst Hospitals) Tertile With Lowest Outcome Rates (Best Hospitals) Difference in Net From Operations Differences Between Highest and Lowest Outcome Rate Tertiles, $ Million (95% CI) P
Outcome, Median % (IQR) Gain/Loss in Net Revenue From Operations From 2008 to 2012, $ Million* Outcome, Median % (IQR) Gain/Loss in Net Revenue from Operations From 2008 to 2012, $ Million*
  • NOTE: Abbreviations: AMI, acute myocardial infarction; CHF, congestive heart failure; CI, confidence interval; IQR, interquartile range; PNA, pneumonia. *Differences were calculated as net revenue from clinical operations in 2012 minus net revenue from clinical operations in 2008. Net revenue in 2008 was adjusted to 2012 US dollars using the chained Consumer Price Index for all urban consumers. Each outcome was modeled separately and adjusted for year, tertile of performance for the respective outcome, the interaction between year and tertile (difference‐in‐differences term), teaching status, metropolitan status (urban vs rural), bed size, safety‐net hospital status, hospital ownership type, Medicare caseload, volume of cases reported for the respective outcome, and interactions for year with bed size, safety‐net hospital status, and Medicare caseload, accounting for clustering of hospitals by owner.

AMI mortality 16.7 (16.217.4) +65.62 13.8 (13.314.2) +74.23 8.61 (27.95 to 10.73) 0.38
AMI readmit 20.7 (20.321.5) +38.62 18.3 (17.718.6) +31.85 +6.77 (13.24 to 26.77) 0.50
CHF mortality 13.0 (12.313.9) +45.66 9.6 (8.910.1) +48.60 2.94 (11.61 to 5.73) 0.50
CHF readmit 26.2 (25.726.9) +47.08 23.0 (22.323.5) +46.08 +0.99 (10.51 to 12.50) 0.87
PNA mortality 13.9 (13.314.7) +43.46 9.9 (9.310.4) +38.28 +5.18 (7.01 to 17.37) 0.40
PNA readmit 19.4 (19.120.1) +47.21 17.0 (16.517.3) +45.45 +1.76 (8.34 to 11.86) 0.73

DISCUSSION

Using audited financial data from California hospitals in 2008 and 2012, and CMS data on publicly reported outcomes from 2008 to 2011, we found no consistent relationship between hospital financial performance and publicly reported outcomes for AMI and PNA. However, better hospital financial performance was associated with a modest increase in 30‐day risk‐standardized CHF mortality rates, which was consistent across all 3 measures of hospital financial performance. Reassuringly, there was no difference in the change in net revenue from operations between 2008 and 2012 between hospitals in the highest and lowest tertiles of readmission and mortality rates for AMI, CHF, and PNA. In other words, hospitals with the lowest rates of 30‐day readmissions and mortality for AMI, CHF, and PNA did not experience a loss in net revenue from operations over time, compared to hospitals with the highest readmission and mortality rates.

Our study differs in several important ways from Ly et al., the only other study to our knowledge that investigated the relationship between hospital financial performance and outcomes for patients with AMI, CHF, and PNA.[19] First, outcomes in the Ly et al. study were ascertained in 2007, which preceded public reporting of outcomes. Second, the primary comparison was between hospitals in the bottom versus top decile of operating margin. Although Ly and colleagues also found no association between hospital financial performance and mortality rates for these 3 conditions, they found a significant absolute decrease of approximately 3% in readmission rates among hospitals in the top decile of operating margin versus those in bottom decile. However, readmission rates were comparable among the remaining 80% of hospitals, suggesting that these findings primarily reflected the influence of a few outlier hospitals. Third, the use of nonuniformly audited hospital financial data may have resulted in misclassification of financial performance. Our findings also differ from 2 previous studies that identified a modest association between improved hospital financial performance and decreased adverse patient safety events.[18, 20] However, publicly reported outcomes may not be fully representative of hospital quality and patient safety.[28, 29]

The limited association between hospital financial performance and publicly reported outcomes for AMI and PNA is noteworthy for several reasons. First, publicly reporting outcomes alone without concomitant changes to reimbursement may be inadequate to create strong financial incentives for hospital investment in quality improvement initiatives. Hospitals participating in both public reporting of outcomes and pay‐for‐performance have been shown to achieve greater improvements in outcomes than hospitals engaged only in public reporting.[30] Our time interval for ascertainment of outcomes preceded CMS implementation of the Hospital Readmissions Reduction Program (HRRP) in October 2012, which withholds up to 3% of Medicare hospital reimbursements for higher than expected mortality and readmission rates for AMI, CHF, and PNA. Once outcomes data become available for a 3‐year post‐HRRP implementation period, the impact of this combined approach can be assessed. Second, because adherence to many evidence‐based process measures for these conditions (ie, aspirin use in AMI) is already high, there may be a ceiling effect present that obviates the need for further hospital financial investment to optimize delivery of best practices.[31, 32] Third, hospitals themselves may contribute little to variation in mortality and readmission risk. Of the total variation in mortality and readmission rates among Texas Medicare beneficiaries, only about 1% is attributable to hospitals, whereas 42% to 56% of the variation is explained by differences in patient characteristics.[33, 34] Fourth, there is either low‐quality or insufficient evidence that transitional care interventions specifically targeted to patients with AMI or PNA result in better outcomes.[35] Thus, greater financial investment in hospital‐initiated and postdischarge transitional care interventions for these specific conditions may result in less than the desired effect. Lastly, many hospitalizations for these conditions are emergency hospitalizations that occur after patients present to the emergency department with unexpected and potentially life‐threatening symptoms. Thus, patients may not be able to incorporate the reputation or performance metrics of a hospital in their decisions for where they are hospitalized for AMI, CHF, or PNA despite the public reporting of outcomes.

Given the strong evidence that transitional care interventions reduce readmissions and mortality among patients hospitalized with CHF, we were surprised to find that improved hospital financial performance was associated with an increased risk‐adjusted CHF mortality rate.[36] This association held true for all 3 different measures of hospital financial performance, suggesting that this unexpected finding is unlikely to be the result of statistical chance, though potential reasons for this association remain unclear. One possibility is that the CMS model for CHF mortality may not adequately risk adjust for severity of illness.[37, 38] Thus, robust financial performance may be a marker for hospitals with more advanced heart failure services that care for more patients with severe illness.

Our findings should be interpreted in the context of certain limitations. Our study only included an analysis of outcomes for AMI, CHF, and PNA among older fee‐for‐service Medicare beneficiaries aggregated at the hospital level in California between 2008 and 2012, so generalizability to other populations, conditions, states, and time periods is uncertain. The observational design precludes a robust causal inference between financial performance and outcomes. For readmissions, rates were publicly reported for only the last 2 years of the 3‐year reporting period; thus, our findings may underestimate the association between hospital financial performance and publicly reported readmission rates.

CONCLUSION

There is no consistent relationship between hospital financial performance and subsequent publicly reported outcomes for AMI and PNA. However, for unclear reasons, hospitals with better financial performance had modestly higher CHF mortality rates. Given this limited association, public reporting of outcomes may have had less than the intended impact in motivating hospitals to invest in quality improvement. Additional financial incentives in addition to public reporting, such as readmissions penalties, may help motivate hospitals with robust financial performance to further improve outcomes. This would be a key area for future investigation once outcomes data are available for the 3‐year period following CMS implementation of readmissions penalties in 2012. Reassuringly, there was no association between low 30‐day mortality and readmissions rates and subsequent poor financial performance, suggesting that improved outcomes do not necessarily lead to loss of revenue.

Disclosures

Drs. Nguyen, Halm, and Makam were supported in part by the Agency for Healthcare Research and Quality University of Texas Southwestern Center for Patient‐Centered Outcomes Research (1R24HS022418‐01). Drs. Nguyen and Makam received funding from the University of Texas Southwestern KL2 Scholars Program (NIH/NCATS KL2 TR001103). The study sponsors had no role in design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. The authors have no conflicts of interest to disclose.

Files
References
  1. Centers for Medicare and Medicaid Services. Office of the Actuary. National Health Statistics Group. National Healthcare Expenditures Data. Baltimore, MD; 2013. https://www.cms.gov/Research-Statistics-Data-and-Systems/Statistics-Trends-and-Reports/NationalHealthExpendData/NationalHealthAccountsHistorical.html. Accessed February 16, 2016.
  2. Brill S. Bitter pill: why medical bills are killing us. Time Magazine. February 20, 2013:1655.
  3. Ginsburg PB. Fee‐for‐service will remain a feature of major payment reforms, requiring more changes in Medicare physician payment. Health Aff (Millwood). 2012;31(9):19771983.
  4. Leatherman S, Berwick D, Iles D, et al. The business case for quality: case studies and an analysis. Health Aff (Millwood). 2003;22(2):1730.
  5. Marshall MN, Shekelle PG, Davies HT, Smith PC. Public reporting on quality in the United States and the United Kingdom. Health Aff (Millwood). 2003;22(3):134148.
  6. Swensen SJ, Dilling JA, Mc Carty PM, Bolton JW, Harper CM. The business case for health‐care quality improvement. J Patient Saf. 2013;9(1):4452.
  7. Hibbard JH, Stockard J, Tusler M. Hospital performance reports: impact on quality, market share, and reputation. Health Aff (Millwood). 2005;24(4):11501160.
  8. Rauh SS, Wadsworth EB, Weeks WB, Weinstein JN. The savings illusion—why clinical quality improvement fails to deliver bottom‐line results. N Engl J Med. 2011;365(26):e48.
  9. Meyer JA, Silow‐Carroll S, Kutyla T, Stepnick LS, Rybowski LS. Hospital Quality: Ingredients for Success—Overview and Lessons Learned. New York, NY: Commonwealth Fund; 2004.
  10. Silow‐Carroll S, Alteras T, Meyer JA. Hospital Quality Improvement: Strategies and Lessons from U.S. Hospitals. New York, NY: Commonwealth Fund; 2007.
  11. Bazzoli GJ, Clement JP, Lindrooth RC, et al. Hospital financial condition and operational decisions related to the quality of hospital care. Med Care Res Rev. 2007;64(2):148168.
  12. Casalino LP, Elster A, Eisenberg A, Lewis E, Montgomery J, Ramos D. Will pay‐for‐performance and quality reporting affect health care disparities? Health Aff (Millwood). 2007;26(3):w405w414.
  13. Werner RM, Goldman LE, Dudley RA. Comparison of change in quality of care between safety‐net and non‐safety‐net hospitals. JAMA. 2008;299(18):21802187.
  14. Bhalla R, Kalkut G. Could Medicare readmission policy exacerbate health care system inequity? Ann Intern Med. 2010;152(2):114117.
  15. Hernandez AF, Curtis LH. Minding the gap between efforts to reduce readmissions and disparities. JAMA. 2011;305(7):715716.
  16. Terry DF, Moisuk S. Medicare Health Support Pilot Program. N Engl J Med. 2012;366(7):666; author reply 667–668.
  17. Fontanarosa PB, McNutt RA. Revisiting hospital readmissions. JAMA. 2013;309(4):398400.
  18. Encinosa WE, Bernard DM. Hospital finances and patient safety outcomes. Inquiry. 2005;42(1):6072.
  19. Ly DP, Jha AK, Epstein AM. The association between hospital margins, quality of care, and closure or other change in operating status. J Gen Intern Med. 2011;26(11):12911296.
  20. Bazzoli GJ, Chen HF, Zhao M, Lindrooth RC. Hospital financial condition and the quality of patient care. Health Econ. 2008;17(8):977995.
  21. State of California Office of Statewide Health Planning and Development. Healthcare Information Division. Annual financial data. Available at: http://www.oshpd.ca.gov/HID/Products/Hospitals/AnnFinanData/PivotProfles/default.asp. Accessed June 23, 2015.
  22. Centers for Medicare 4(1):1113.
  23. Ross JS, Cha SS, Epstein AJ, et al. Quality of care for acute myocardial infarction at urban safety‐net hospitals. Health Aff (Millwood). 2007;26(1):238248.
  24. Silber JH, Rosenbaum PR, Brachet TJ, et al. The Hospital Compare mortality model and the volume‐outcome relationship. Health Serv Res. 2010;45(5 Pt 1):11481167.
  25. Tukey J. Exploratory Data Analysis. Boston, MA: Addison‐Wesley; 1977.
  26. Press MJ, Scanlon DP, Ryan AM, et al. Limits of readmission rates in measuring hospital quality suggest the need for added metrics. Health Aff (Millwood). 2013;32(6):10831091.
  27. Stefan MS, Pekow PS, Nsa W, et al. Hospital performance measures and 30‐day readmission rates. J Gen Intern Med. 2013;28(3):377385.
  28. Lindenauer PK, Remus D, Roman S, et al. Public reporting and pay for performance in hospital quality improvement. N Engl J Med. 2007;356(5):486496.
  29. Spatz ES, Sheth SD, Gosch KL, et al. Usual source of care and outcomes following acute myocardial infarction. J Gen Intern Med. 2014;29(6):862869.
  30. Werner RM, Bradlow ET. Public reporting on hospital process improvements is linked to better patient outcomes. Health Aff (Millwood). 2010;29(7):13191324.
  31. Goodwin JS, Lin YL, Singh S, Kuo YF. Variation in length of stay and outcomes among hospitalized patients attributable to hospitals and hospitalists. J Gen Intern Med. 2013;28(3):370376.
  32. Singh S, Lin YL, Kuo YF, Nattinger AB, Goodwin JS. Variation in the risk of readmission among hospitals: the relative contribution of patient, hospital and inpatient provider characteristics. J Gen Intern Med. 2014;29(4):572578.
  33. Prvu Bettger J, Alexander KP, Dolor RJ, et al. Transitional care after hospitalization for acute stroke or myocardial infarction: a systematic review. Ann Intern Med. 2012;157(6):407416.
  34. Jha AK, Orav EJ, Li Z, Epstein AM. The inverse relationship between mortality rates and performance in the Hospital Quality Alliance measures. Health Aff (Millwood). 2007;26(4):11041110.
  35. Amarasingham R, Moore BJ, Tabak YP, et al. An automated model to identify heart failure patients at risk for 30‐day readmission or death using electronic medical record data. Med Care. 2010;48(11):981988.
  36. Fuller RL, Atkinson G, Hughes JS. Indications of biased risk adjustment in the hospital readmission reduction program. J Ambul Care Manage. 2015;38(1):3947.
Article PDF
Issue
Journal of Hospital Medicine - 11(7)
Page Number
481-488
Sections
Files
Files
Article PDF
Article PDF

Hospital care accounts for the single largest category of national healthcare expenditures, totaling $936.9 billion in 2013.[1] With ongoing scrutiny of US healthcare spending, hospitals are under increasing pressure to justify high costs and robust profits.[2] However, the dominant fee‐for‐service reimbursement model creates incentives for hospitals to prioritize high volume over high‐quality care to maximize profits.[3] Because hospitals may be reluctant to implement improvements if better quality is not accompanied by better payment or improved financial margins, an approach to stimulate quality improvement among hospitals has been to leverage consumer pressure through required public reporting of selected outcome metrics.[4, 5] Public reporting of outcomes is thought to influence hospital reputation; in turn, reputation affects patient perceptions and influences demand for hospital services, potentially enabling reputable hospitals to command higher prices for services to enhance hospital revenue.[6, 7]

Though improving outcomes is thought to reduce overall healthcare costs, it is unclear whether improving outcomes results in a hospital's financial return on investment.[4, 5, 8] Quality improvement can require substantial upfront investment, requiring that hospitals already have robust financial health to engage in such initiatives.[9, 10] Consequently, instead of stimulating broad efforts in quality improvement, public reporting may exacerbate existing disparities in hospital quality and finances, by rewarding already financially healthy hospitals, and by inadvertently penalizing hospitals without the means to invest in quality improvement.[11, 12, 13, 14, 15] Alternately, because fee‐for‐service remains the dominant reimbursement model for hospitals, loss of revenue through reducing readmissions may outweigh any financial gains from improved public reputation and result in worse overall financial performance, though robust evidence for this concern is lacking.[16, 17]

A small number of existing studies suggest a limited correlation between improved hospital financial performance and improved quality, patient safety, and lower readmission rates.[18, 19, 20] However, these studies had several limitations. They were conducted prior to public reporting of selected outcome metrics by the Centers for Medicare and Medicaid Services (CMS)[18, 19, 20]; used data from the Medicare Cost Report, which is not uniformly audited and thus prone to measurement error[19, 20]; used only relative measures of hospital financial performance (eg, operating margin), which do not capture the absolute amount of revenue potentially available for investment in quality improvement[18, 19]; or compared only hospitals at the extremes of financial performance, potentially exaggerating the magnitude of the relationship between hospital financial performance and quality outcomes.[19]

To address this gap in the literature, we sought to assess whether hospitals with robust financial performance have lower 30‐day risk‐standardized mortality and hospital readmission rates for acute myocardial infarction (AMI), congestive heart failure (CHF), and pneumonia (PNA). Given the concern that hospitals with the lowest mortality and readmission rates may experience a decrease in financial performance due to the lower volume of hospitalizations, we also assessed whether hospitals with the lowest readmission and mortality rates had a differential change in financial performance over time compared to hospitals with the highest rates.

METHODS

Data Sources and Study Population

This was an observational study using audited financial data from the 2008 and 2012 Hospital Annual Financial Data Files from the Office of Statewide Health Planning and Development (OSHPD) in the state of California, merged with data on outcome measures publicly reported by CMS via the Hospital Compare website for July 1, 2008 to June 30, 2011.[21, 22] We included all general acute care hospitals with available OSHPD data in 2008 and at least 1 publicly reported outcome from 2008 to 2011. We excluded hospitals without 1 year of audited financial data for 2008 and hospitals that closed during 2008 to 2011.

Measures of Financial Performance

Because we hypothesized that the absolute amount of revenue generated from clinical operations would influence investment in quality improvement programs more so than relative changes in revenue,[20] we used net revenue from operations (total operating revenue minus total operating expense) as our primary measure of hospital financial performance. We also performed 2 companion analyses using 2 commonly reported relative measures of financial performanceoperating margin (net revenue from operations divided by total operating revenue) and total margin (net total revenue divided by total revenue from all sources). Net revenue from operations for 2008 was adjusted to 2012 US dollars using the chained Consumer Price Index for all urban consumers.

Outcomes

For our primary analysis, the primary outcomes were publicly reported all‐cause 30‐day risk‐standardized mortality rates (RSMR) and readmission rates (RSRR) for AMI, CHF, and PNA aggregated over a 3‐year period. These measures were adjusted for key demographic and clinical characteristics available in Medicare data. CMS began publicly reporting 30‐day RSMR for AMI and CHF in June 2007, RSMR for PNA in June 2008, and RSRR for all 3 conditions in July 2009.[23, 24]

To assess whether public reporting had an effect on subsequent hospital financial performance, we conducted a companion analysis where the primary outcome of interest was change in hospital financial performance over time, using the same definitions of financial performance outlined above. For this companion analysis, publicly reported 30‐day RSMR and RSRR for AMI, CHF, and PNA were assessed as predictors of subsequent financial performance.

Hospital Characteristics

Hospital characteristics were ascertained from the OSHPD data. Safety‐net status was defined as hospitals with an annual Medicaid caseload (number of Medicaid discharges divided by the total number of discharges) 1 standard deviation above the mean Medicaid caseload, as defined in previous studies.[25]

Statistical Analyses

Effect of Baseline Financial Performance on Subsequent Publicly Reported Outcomes

To estimate the relationship between baseline hospital financial performance in 2008 and subsequent RSMR and RSRR for AMI, CHF, and PNA from 2008 to 2011, we used linear regression adjusted for the following hospital characteristics: teaching status, rural location, bed size, safety‐net status, ownership, Medicare caseload, and volume of cases reported for the respective outcome. We accounted for clustering of hospitals by ownership. We adjusted for hospital volume of reported cases for each condition given that the risk‐standardization models used by CMS shrink outcomes for small hospitals to the mean, and therefore do not account for a potential volume‐outcome relationship.[26] We conducted a sensitivity analysis excluding hospitals at the extremes of financial performance, defined as hospitals with extreme outlier values for each financial performance measure (eg, values more than 3 times the interquartile range above the first quartile or below the third quartile).[27] Nonlinearity of financial performance measures was assessed using restricted cubic splines. For ease of interpretation, we scaled the estimated change in RSMR and RSRR per $50 million increase in net revenue from operations, and graphed nonparametric relationships using restricted cubic splines.

Effect of Public Reporting on Subsequent Hospital Financial Performance

To assess whether public reporting had an effect on subsequent hospital financial performance, we conducted a companion hospital‐level difference‐in‐differences analysis to assess for differential changes in hospital financial performance between 2008 and 2012, stratified by tertiles of RSMR and RSRR rates from 2008 to 2011. This approach compares differences in an outcome of interest (hospital financial performance) within each group (where each group is a tertile of publicly reported rates of RSMR or RSRR), and then compares the difference in these differences between groups. Therefore, these analyses use each group as their own historical control and the opposite group as a concurrent control to account for potential secular trends. To conduct our difference‐in‐differences analysis, we compared the change in financial performance over time in the top tertile of hospitals to the change in financial performance over time in the bottom tertile of hospitals with respect to AMI, CHF, and PNA RSMR and RSRR. Our models therefore included year (2008 vs 2012), tertile of publicly reported rates for RSMR or RSRR, and the interaction between them as predictors, where the interaction was the difference‐in‐differences term and the primary predictor of interest. In addition to adjusting for hospital characteristics and accounting for clustering as mentioned above, we also included 3 separate interaction terms for year with bed size, safety‐net status, and Medicare caseload, to account for potential changes in the hospitals over time that may have independently influenced financial performance and publicly reported 30‐day measures. For sensitivity analyses, we repeated our difference‐in‐differences analyses excluding hospitals with a change in ownership and extreme outliers with respect to financial performance in 2008. We performed model diagnostics including assessment of functional form, linearity, normality, constant variance, and model misspecification. All analyses were conducted using Stata version 12.1 (StataCorp, College Station, TX). This study was deemed exempt from review by the UT Southwestern Medical Center institutional review board.

RESULTS

Among the 279 included hospitals (see Supporting Figure 1 in the online version of this article), 278 also had financial data available for 2012. In 2008, the median net revenue from operations was $1.6 million (interquartile range [IQR], $2.4 to $10.3 million), the median operating margin was 1.5% (IQR, 4.6% to 6%), and the median total margin was 2.5% (IQR, 2.2% to 7.5% (Table 1). The number of hospitals reporting each outcome, and median outcome rates, are shown in Table 2.

Hospital Characteristics and Financial Performance in 2008 and 2012
2008, n = 279 2012, n = 278
  • NOTE: Abbreviations: IQR, interquartile range; SD, standard deviation. *Medicaid caseload equivalent to 1 standard deviation above the mean (41.8% for 2008 and 42.1% for 2012). Operated by an investor‐individual, investor‐partnership, or investor‐corporation.

Hospital characteristics
Teaching, n (%) 28 (10.0) 28 (10.0)
Rural, n (%) 55 (19.7) 55 (19.7)
Bed size, n (%)
099 (small) 57 (20.4) 55 (19.8)
100299 (medium) 130 (46.6) 132 (47.5)
300 (large) 92 (33.0) 91 (32.7)
Safety‐net hospital, n (%)* 46 (16.5) 48 (17.3)
Hospital ownership, n (%)
City or county 15 (5.4) 16 (5.8)
District 42 (15.1) 39 (14.0)
Investor 66 (23.7) 66 (23.7)
Private nonprofit 156 (55.9) 157 (56.5)
Medicare caseload, mean % (SD) 41.6 (14.7) 43.6 (14.7)
Financial performance measures
Net revenue from operations, median $ in millions (IQR; range) 1.6 (2.4 to 10.3; 495.9 to 144.1) 3.2 (2.9 to 15.4; 396.2 to 276.8)
Operating margin, median % (IQR; range) 1.5 (4.6 to 6.8; 77.8 to 26.4) 2.3 (3.9 to 8.2; 134.8 to 21.1)
Total margin, median % (IQR; range) 2.5 (2.2 to 7.5; 101.0 to 26.3) 4.5 (0.7 to 9.8; 132.2 to 31.1)
Relationship Between Hospital Financial Performance and 30‐Day Mortality and Readmission Rates*
No. Median % (IQR) Adjusted % Change (95% CI) per $50 Million Increase in Net Revenue From Operations
Overall Extreme Outliers Excluded
  • NOTE: Abbreviations: CI, confidence interval; IQR, interquartile range. *Thirty‐day outcomes are risk standardized for age, sex, comorbidity count, and indicators of patient frailty.[3] Each outcome was modeled separately and adjusted for teaching status, metropolitan status (urban vs rural), bed size, safety‐net hospital status, hospital ownership type, Medicare caseload, and volume of cases reported for the respective outcome, accounting for clustering of hospitals by owner. Twenty‐three hospitals were identified as extreme outliers with respect to net revenue from operations (10 underperformers with net revenue <$49.4 million and 13 overperformers with net revenue >$52.1 million). There was a nonlinear and statistically significant relationship between net revenue from operations and readmission rate for myocardial infarction. Net revenue from operations was modeled as a cubic spline function. See Figure 1. The overall adjusted F statistic was 4.8 (P < 0.001). There was a nonlinear and statistically significant relationship between net revenue from operations and mortality rate for heart failure after exclusion of extreme outliers. Net revenue from operations was modeled as a cubic spline function. See Figure 1.The overall adjusted F statistic was 3.6 (P = 0.008).

Myocardial infarction
Mortality rate 211 15.2 (14.216.2) 0.07 (0.10 to 0.24) 0.63 (0.21 to 1.48)
Readmission rate 184 19.4 (18.520.2) Nonlinear 0.34 (1.17 to 0.50)
Congestive heart failure
Mortality rate 259 11.1 (10.112.1) 0.17 (0.01 to 0.35) Nonlinear
Readmission rate 264 24.5 (23.525.6) 0.07 (0.27 to 0.14) 0.45 (1.36 to 0.47)
Pneumonia
Mortality rate 268 11.6 (10.413.2) 0.17 (0.42 to 0.07) 0.35 (1.19 to 0.49)
Readmission rate 268 18.2 (17.319.1) 0.04 (0.20 to 0.11) 0.56 (1.27 to 0.16)

Relationship Between Financial Performance and Publicly Reported Outcomes

Acute Myocardial Infarction

We did not observe a consistent relationship between hospital financial performance and AMI mortality and readmission rates. In our overall adjusted analyses, net revenue from operations was not associated with mortality, but was significantly associated with a decrease in AMI readmissions among hospitals with net revenue from operations between approximately $5 million to $145 million (nonlinear relationship, F statistic = 4.8, P < 0.001 (Table 2, Figure 1A). However, after excluding 23 extreme outlying hospitals by net revenue from operations (10 underperformers with net revenue <$49.4 million and 13 overperformers with net revenue >$52.1 million), this relationship was no longer observed. Using operating margin instead of net revenue from operations as the measure of hospital financial performance, we observed a 0.2% increase in AMI mortality (95% confidence interval [CI]: 0.06%‐0.35%) (see Supporting Table 1 and Supporting Figure 2 in the online version of this article) for each 10% increase in operating margin, which persisted with the exclusion of 5 outlying hospitals by operating margin (all 5 were underperformers, with operating margins <38.6%). However, using total margin as the measure of financial performance, there was no significant relationship with either mortality or readmissions (see Supporting Table 2 and Supporting Figure 3 in the online version of this article).

Figure 1
Relationship between financial performance and 30‐day readmission and mortality. The open circles represent individual hospitals. The bold dashed line and the bold solid line are the unadjusted and adjusted cubic spline curves, respectively, representing the nonlinear relationship between net revenue from operations and each outcome. The shaded grey area represents the 95% confidence interval for the adjusted cubic spline curve. Thin vertical dashed lines represent median values for net revenue from operations. Multivariate models were adjusted for teaching status, metropolitan status (urban vs rural), bed size, safety‐net hospital status, hospital ownership, Medicare caseload, and volume of cases reported for the respective outcome, accounting for clustering of hospitals by owner. *Twenty‐three hospitals were identified as outliers with respect to net revenue from clinical operations (10 “underperformers” with net revenue <−$49.4 million and 13 “overperformers” with net revenue >$52.1 million.

Congestive Heart Failure

In our primary analyses, we did not observe a significant relationship between net revenue from operations and CHF mortality and readmission rates. However, after excluding 23 extreme outliers, increasing net revenue from operations was associated with a modest increase in CHF mortality among hospitals, with net revenue between approximately $35 million and $20 million (nonlinear relationship, F statistic = 3.6, P = 0.008 (Table 2, Figure 1B). Using alternate measures of financial performance, we observed a consistent relationship between increasing hospital financial performance and higher 30‐day CHF mortality rate. Using operating margin, we observed a slight increase in the mortality rate for CHF (0.26% increase in CHF RSMR for every 10% increase in operating margin) (95% CI: 0.07%‐0.45%) (see Supporting Table 1 and Supporting Figure 2 in the online version of this article), which persisted after the exclusion of 5 extreme outliers. Using total margin, we observed a significant but modest association between improved hospital financial performance and increased mortality rate for CHF (nonlinear relationship, F statistic = 2.9, P = 0.03) (see Supporting Table 2 and Supporting Figure 3 in the online version of this article), which persisted after the exclusion of 3 extreme outliers (0.32% increase in CHF RSMR for every 10% increase in total margin) (95% CI: 0.03%‐0.62%).

Pneumonia

Hospital financial performance (using net revenue, operating margin, or total margin) was not associated with 30‐day PNA mortality or readmission rates.

Relationship of Readmission and Mortality Rates on Subsequent Hospital Financial Performance

Compared to hospitals in the highest tertile of readmission and mortality rates (ie, those with the worst rates), hospitals in the lowest tertile of readmission and mortality rates (ie, those with the best rates) had a similar magnitude of increase in net revenue from operations from 2008 to 2012 (Table 3). The difference‐in‐differences analyses showed no relationship between readmission or mortality rates for AMI, CHF, and PNA and changes in net revenue from operations from 2008 to 2012 (difference‐in‐differences estimates ranged from $8.61 to $6.77 million, P > 0.3 for all). These results were robust to the exclusion of hospitals with a change in ownership and extreme outliers by net revenue from operations (data not reported).

Difference in the Differences in Financial Performance Between the Worst‐ and the Best‐Performing Hospitals
Outcome Tertile With Highest Outcome Rates (Worst Hospitals) Tertile With Lowest Outcome Rates (Best Hospitals) Difference in Net From Operations Differences Between Highest and Lowest Outcome Rate Tertiles, $ Million (95% CI) P
Outcome, Median % (IQR) Gain/Loss in Net Revenue From Operations From 2008 to 2012, $ Million* Outcome, Median % (IQR) Gain/Loss in Net Revenue from Operations From 2008 to 2012, $ Million*
  • NOTE: Abbreviations: AMI, acute myocardial infarction; CHF, congestive heart failure; CI, confidence interval; IQR, interquartile range; PNA, pneumonia. *Differences were calculated as net revenue from clinical operations in 2012 minus net revenue from clinical operations in 2008. Net revenue in 2008 was adjusted to 2012 US dollars using the chained Consumer Price Index for all urban consumers. Each outcome was modeled separately and adjusted for year, tertile of performance for the respective outcome, the interaction between year and tertile (difference‐in‐differences term), teaching status, metropolitan status (urban vs rural), bed size, safety‐net hospital status, hospital ownership type, Medicare caseload, volume of cases reported for the respective outcome, and interactions for year with bed size, safety‐net hospital status, and Medicare caseload, accounting for clustering of hospitals by owner.

AMI mortality 16.7 (16.217.4) +65.62 13.8 (13.314.2) +74.23 8.61 (27.95 to 10.73) 0.38
AMI readmit 20.7 (20.321.5) +38.62 18.3 (17.718.6) +31.85 +6.77 (13.24 to 26.77) 0.50
CHF mortality 13.0 (12.313.9) +45.66 9.6 (8.910.1) +48.60 2.94 (11.61 to 5.73) 0.50
CHF readmit 26.2 (25.726.9) +47.08 23.0 (22.323.5) +46.08 +0.99 (10.51 to 12.50) 0.87
PNA mortality 13.9 (13.314.7) +43.46 9.9 (9.310.4) +38.28 +5.18 (7.01 to 17.37) 0.40
PNA readmit 19.4 (19.120.1) +47.21 17.0 (16.517.3) +45.45 +1.76 (8.34 to 11.86) 0.73

DISCUSSION

Using audited financial data from California hospitals in 2008 and 2012, and CMS data on publicly reported outcomes from 2008 to 2011, we found no consistent relationship between hospital financial performance and publicly reported outcomes for AMI and PNA. However, better hospital financial performance was associated with a modest increase in 30‐day risk‐standardized CHF mortality rates, which was consistent across all 3 measures of hospital financial performance. Reassuringly, there was no difference in the change in net revenue from operations between 2008 and 2012 between hospitals in the highest and lowest tertiles of readmission and mortality rates for AMI, CHF, and PNA. In other words, hospitals with the lowest rates of 30‐day readmissions and mortality for AMI, CHF, and PNA did not experience a loss in net revenue from operations over time, compared to hospitals with the highest readmission and mortality rates.

Our study differs in several important ways from Ly et al., the only other study to our knowledge that investigated the relationship between hospital financial performance and outcomes for patients with AMI, CHF, and PNA.[19] First, outcomes in the Ly et al. study were ascertained in 2007, which preceded public reporting of outcomes. Second, the primary comparison was between hospitals in the bottom versus top decile of operating margin. Although Ly and colleagues also found no association between hospital financial performance and mortality rates for these 3 conditions, they found a significant absolute decrease of approximately 3% in readmission rates among hospitals in the top decile of operating margin versus those in bottom decile. However, readmission rates were comparable among the remaining 80% of hospitals, suggesting that these findings primarily reflected the influence of a few outlier hospitals. Third, the use of nonuniformly audited hospital financial data may have resulted in misclassification of financial performance. Our findings also differ from 2 previous studies that identified a modest association between improved hospital financial performance and decreased adverse patient safety events.[18, 20] However, publicly reported outcomes may not be fully representative of hospital quality and patient safety.[28, 29]

The limited association between hospital financial performance and publicly reported outcomes for AMI and PNA is noteworthy for several reasons. First, publicly reporting outcomes alone without concomitant changes to reimbursement may be inadequate to create strong financial incentives for hospital investment in quality improvement initiatives. Hospitals participating in both public reporting of outcomes and pay‐for‐performance have been shown to achieve greater improvements in outcomes than hospitals engaged only in public reporting.[30] Our time interval for ascertainment of outcomes preceded CMS implementation of the Hospital Readmissions Reduction Program (HRRP) in October 2012, which withholds up to 3% of Medicare hospital reimbursements for higher than expected mortality and readmission rates for AMI, CHF, and PNA. Once outcomes data become available for a 3‐year post‐HRRP implementation period, the impact of this combined approach can be assessed. Second, because adherence to many evidence‐based process measures for these conditions (ie, aspirin use in AMI) is already high, there may be a ceiling effect present that obviates the need for further hospital financial investment to optimize delivery of best practices.[31, 32] Third, hospitals themselves may contribute little to variation in mortality and readmission risk. Of the total variation in mortality and readmission rates among Texas Medicare beneficiaries, only about 1% is attributable to hospitals, whereas 42% to 56% of the variation is explained by differences in patient characteristics.[33, 34] Fourth, there is either low‐quality or insufficient evidence that transitional care interventions specifically targeted to patients with AMI or PNA result in better outcomes.[35] Thus, greater financial investment in hospital‐initiated and postdischarge transitional care interventions for these specific conditions may result in less than the desired effect. Lastly, many hospitalizations for these conditions are emergency hospitalizations that occur after patients present to the emergency department with unexpected and potentially life‐threatening symptoms. Thus, patients may not be able to incorporate the reputation or performance metrics of a hospital in their decisions for where they are hospitalized for AMI, CHF, or PNA despite the public reporting of outcomes.

Given the strong evidence that transitional care interventions reduce readmissions and mortality among patients hospitalized with CHF, we were surprised to find that improved hospital financial performance was associated with an increased risk‐adjusted CHF mortality rate.[36] This association held true for all 3 different measures of hospital financial performance, suggesting that this unexpected finding is unlikely to be the result of statistical chance, though potential reasons for this association remain unclear. One possibility is that the CMS model for CHF mortality may not adequately risk adjust for severity of illness.[37, 38] Thus, robust financial performance may be a marker for hospitals with more advanced heart failure services that care for more patients with severe illness.

Our findings should be interpreted in the context of certain limitations. Our study only included an analysis of outcomes for AMI, CHF, and PNA among older fee‐for‐service Medicare beneficiaries aggregated at the hospital level in California between 2008 and 2012, so generalizability to other populations, conditions, states, and time periods is uncertain. The observational design precludes a robust causal inference between financial performance and outcomes. For readmissions, rates were publicly reported for only the last 2 years of the 3‐year reporting period; thus, our findings may underestimate the association between hospital financial performance and publicly reported readmission rates.

CONCLUSION

There is no consistent relationship between hospital financial performance and subsequent publicly reported outcomes for AMI and PNA. However, for unclear reasons, hospitals with better financial performance had modestly higher CHF mortality rates. Given this limited association, public reporting of outcomes may have had less than the intended impact in motivating hospitals to invest in quality improvement. Additional financial incentives in addition to public reporting, such as readmissions penalties, may help motivate hospitals with robust financial performance to further improve outcomes. This would be a key area for future investigation once outcomes data are available for the 3‐year period following CMS implementation of readmissions penalties in 2012. Reassuringly, there was no association between low 30‐day mortality and readmissions rates and subsequent poor financial performance, suggesting that improved outcomes do not necessarily lead to loss of revenue.

Disclosures

Drs. Nguyen, Halm, and Makam were supported in part by the Agency for Healthcare Research and Quality University of Texas Southwestern Center for Patient‐Centered Outcomes Research (1R24HS022418‐01). Drs. Nguyen and Makam received funding from the University of Texas Southwestern KL2 Scholars Program (NIH/NCATS KL2 TR001103). The study sponsors had no role in design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. The authors have no conflicts of interest to disclose.

Hospital care accounts for the single largest category of national healthcare expenditures, totaling $936.9 billion in 2013.[1] With ongoing scrutiny of US healthcare spending, hospitals are under increasing pressure to justify high costs and robust profits.[2] However, the dominant fee‐for‐service reimbursement model creates incentives for hospitals to prioritize high volume over high‐quality care to maximize profits.[3] Because hospitals may be reluctant to implement improvements if better quality is not accompanied by better payment or improved financial margins, an approach to stimulate quality improvement among hospitals has been to leverage consumer pressure through required public reporting of selected outcome metrics.[4, 5] Public reporting of outcomes is thought to influence hospital reputation; in turn, reputation affects patient perceptions and influences demand for hospital services, potentially enabling reputable hospitals to command higher prices for services to enhance hospital revenue.[6, 7]

Though improving outcomes is thought to reduce overall healthcare costs, it is unclear whether improving outcomes results in a hospital's financial return on investment.[4, 5, 8] Quality improvement can require substantial upfront investment, requiring that hospitals already have robust financial health to engage in such initiatives.[9, 10] Consequently, instead of stimulating broad efforts in quality improvement, public reporting may exacerbate existing disparities in hospital quality and finances, by rewarding already financially healthy hospitals, and by inadvertently penalizing hospitals without the means to invest in quality improvement.[11, 12, 13, 14, 15] Alternately, because fee‐for‐service remains the dominant reimbursement model for hospitals, loss of revenue through reducing readmissions may outweigh any financial gains from improved public reputation and result in worse overall financial performance, though robust evidence for this concern is lacking.[16, 17]

A small number of existing studies suggest a limited correlation between improved hospital financial performance and improved quality, patient safety, and lower readmission rates.[18, 19, 20] However, these studies had several limitations. They were conducted prior to public reporting of selected outcome metrics by the Centers for Medicare and Medicaid Services (CMS)[18, 19, 20]; used data from the Medicare Cost Report, which is not uniformly audited and thus prone to measurement error[19, 20]; used only relative measures of hospital financial performance (eg, operating margin), which do not capture the absolute amount of revenue potentially available for investment in quality improvement[18, 19]; or compared only hospitals at the extremes of financial performance, potentially exaggerating the magnitude of the relationship between hospital financial performance and quality outcomes.[19]

To address this gap in the literature, we sought to assess whether hospitals with robust financial performance have lower 30‐day risk‐standardized mortality and hospital readmission rates for acute myocardial infarction (AMI), congestive heart failure (CHF), and pneumonia (PNA). Given the concern that hospitals with the lowest mortality and readmission rates may experience a decrease in financial performance due to the lower volume of hospitalizations, we also assessed whether hospitals with the lowest readmission and mortality rates had a differential change in financial performance over time compared to hospitals with the highest rates.

METHODS

Data Sources and Study Population

This was an observational study using audited financial data from the 2008 and 2012 Hospital Annual Financial Data Files from the Office of Statewide Health Planning and Development (OSHPD) in the state of California, merged with data on outcome measures publicly reported by CMS via the Hospital Compare website for July 1, 2008 to June 30, 2011.[21, 22] We included all general acute care hospitals with available OSHPD data in 2008 and at least 1 publicly reported outcome from 2008 to 2011. We excluded hospitals without 1 year of audited financial data for 2008 and hospitals that closed during 2008 to 2011.

Measures of Financial Performance

Because we hypothesized that the absolute amount of revenue generated from clinical operations would influence investment in quality improvement programs more so than relative changes in revenue,[20] we used net revenue from operations (total operating revenue minus total operating expense) as our primary measure of hospital financial performance. We also performed 2 companion analyses using 2 commonly reported relative measures of financial performanceoperating margin (net revenue from operations divided by total operating revenue) and total margin (net total revenue divided by total revenue from all sources). Net revenue from operations for 2008 was adjusted to 2012 US dollars using the chained Consumer Price Index for all urban consumers.

Outcomes

For our primary analysis, the primary outcomes were publicly reported all‐cause 30‐day risk‐standardized mortality rates (RSMR) and readmission rates (RSRR) for AMI, CHF, and PNA aggregated over a 3‐year period. These measures were adjusted for key demographic and clinical characteristics available in Medicare data. CMS began publicly reporting 30‐day RSMR for AMI and CHF in June 2007, RSMR for PNA in June 2008, and RSRR for all 3 conditions in July 2009.[23, 24]

To assess whether public reporting had an effect on subsequent hospital financial performance, we conducted a companion analysis where the primary outcome of interest was change in hospital financial performance over time, using the same definitions of financial performance outlined above. For this companion analysis, publicly reported 30‐day RSMR and RSRR for AMI, CHF, and PNA were assessed as predictors of subsequent financial performance.

Hospital Characteristics

Hospital characteristics were ascertained from the OSHPD data. Safety‐net status was defined as hospitals with an annual Medicaid caseload (number of Medicaid discharges divided by the total number of discharges) 1 standard deviation above the mean Medicaid caseload, as defined in previous studies.[25]

Statistical Analyses

Effect of Baseline Financial Performance on Subsequent Publicly Reported Outcomes

To estimate the relationship between baseline hospital financial performance in 2008 and subsequent RSMR and RSRR for AMI, CHF, and PNA from 2008 to 2011, we used linear regression adjusted for the following hospital characteristics: teaching status, rural location, bed size, safety‐net status, ownership, Medicare caseload, and volume of cases reported for the respective outcome. We accounted for clustering of hospitals by ownership. We adjusted for hospital volume of reported cases for each condition given that the risk‐standardization models used by CMS shrink outcomes for small hospitals to the mean, and therefore do not account for a potential volume‐outcome relationship.[26] We conducted a sensitivity analysis excluding hospitals at the extremes of financial performance, defined as hospitals with extreme outlier values for each financial performance measure (eg, values more than 3 times the interquartile range above the first quartile or below the third quartile).[27] Nonlinearity of financial performance measures was assessed using restricted cubic splines. For ease of interpretation, we scaled the estimated change in RSMR and RSRR per $50 million increase in net revenue from operations, and graphed nonparametric relationships using restricted cubic splines.

Effect of Public Reporting on Subsequent Hospital Financial Performance

To assess whether public reporting had an effect on subsequent hospital financial performance, we conducted a companion hospital‐level difference‐in‐differences analysis to assess for differential changes in hospital financial performance between 2008 and 2012, stratified by tertiles of RSMR and RSRR rates from 2008 to 2011. This approach compares differences in an outcome of interest (hospital financial performance) within each group (where each group is a tertile of publicly reported rates of RSMR or RSRR), and then compares the difference in these differences between groups. Therefore, these analyses use each group as their own historical control and the opposite group as a concurrent control to account for potential secular trends. To conduct our difference‐in‐differences analysis, we compared the change in financial performance over time in the top tertile of hospitals to the change in financial performance over time in the bottom tertile of hospitals with respect to AMI, CHF, and PNA RSMR and RSRR. Our models therefore included year (2008 vs 2012), tertile of publicly reported rates for RSMR or RSRR, and the interaction between them as predictors, where the interaction was the difference‐in‐differences term and the primary predictor of interest. In addition to adjusting for hospital characteristics and accounting for clustering as mentioned above, we also included 3 separate interaction terms for year with bed size, safety‐net status, and Medicare caseload, to account for potential changes in the hospitals over time that may have independently influenced financial performance and publicly reported 30‐day measures. For sensitivity analyses, we repeated our difference‐in‐differences analyses excluding hospitals with a change in ownership and extreme outliers with respect to financial performance in 2008. We performed model diagnostics including assessment of functional form, linearity, normality, constant variance, and model misspecification. All analyses were conducted using Stata version 12.1 (StataCorp, College Station, TX). This study was deemed exempt from review by the UT Southwestern Medical Center institutional review board.

RESULTS

Among the 279 included hospitals (see Supporting Figure 1 in the online version of this article), 278 also had financial data available for 2012. In 2008, the median net revenue from operations was $1.6 million (interquartile range [IQR], $2.4 to $10.3 million), the median operating margin was 1.5% (IQR, 4.6% to 6%), and the median total margin was 2.5% (IQR, 2.2% to 7.5% (Table 1). The number of hospitals reporting each outcome, and median outcome rates, are shown in Table 2.

Hospital Characteristics and Financial Performance in 2008 and 2012
2008, n = 279 2012, n = 278
  • NOTE: Abbreviations: IQR, interquartile range; SD, standard deviation. *Medicaid caseload equivalent to 1 standard deviation above the mean (41.8% for 2008 and 42.1% for 2012). Operated by an investor‐individual, investor‐partnership, or investor‐corporation.

Hospital characteristics
Teaching, n (%) 28 (10.0) 28 (10.0)
Rural, n (%) 55 (19.7) 55 (19.7)
Bed size, n (%)
099 (small) 57 (20.4) 55 (19.8)
100299 (medium) 130 (46.6) 132 (47.5)
300 (large) 92 (33.0) 91 (32.7)
Safety‐net hospital, n (%)* 46 (16.5) 48 (17.3)
Hospital ownership, n (%)
City or county 15 (5.4) 16 (5.8)
District 42 (15.1) 39 (14.0)
Investor 66 (23.7) 66 (23.7)
Private nonprofit 156 (55.9) 157 (56.5)
Medicare caseload, mean % (SD) 41.6 (14.7) 43.6 (14.7)
Financial performance measures
Net revenue from operations, median $ in millions (IQR; range) 1.6 (2.4 to 10.3; 495.9 to 144.1) 3.2 (2.9 to 15.4; 396.2 to 276.8)
Operating margin, median % (IQR; range) 1.5 (4.6 to 6.8; 77.8 to 26.4) 2.3 (3.9 to 8.2; 134.8 to 21.1)
Total margin, median % (IQR; range) 2.5 (2.2 to 7.5; 101.0 to 26.3) 4.5 (0.7 to 9.8; 132.2 to 31.1)
Relationship Between Hospital Financial Performance and 30‐Day Mortality and Readmission Rates*
No. Median % (IQR) Adjusted % Change (95% CI) per $50 Million Increase in Net Revenue From Operations
Overall Extreme Outliers Excluded
  • NOTE: Abbreviations: CI, confidence interval; IQR, interquartile range. *Thirty‐day outcomes are risk standardized for age, sex, comorbidity count, and indicators of patient frailty.[3] Each outcome was modeled separately and adjusted for teaching status, metropolitan status (urban vs rural), bed size, safety‐net hospital status, hospital ownership type, Medicare caseload, and volume of cases reported for the respective outcome, accounting for clustering of hospitals by owner. Twenty‐three hospitals were identified as extreme outliers with respect to net revenue from operations (10 underperformers with net revenue <$49.4 million and 13 overperformers with net revenue >$52.1 million). There was a nonlinear and statistically significant relationship between net revenue from operations and readmission rate for myocardial infarction. Net revenue from operations was modeled as a cubic spline function. See Figure 1. The overall adjusted F statistic was 4.8 (P < 0.001). There was a nonlinear and statistically significant relationship between net revenue from operations and mortality rate for heart failure after exclusion of extreme outliers. Net revenue from operations was modeled as a cubic spline function. See Figure 1.The overall adjusted F statistic was 3.6 (P = 0.008).

Myocardial infarction
Mortality rate 211 15.2 (14.216.2) 0.07 (0.10 to 0.24) 0.63 (0.21 to 1.48)
Readmission rate 184 19.4 (18.520.2) Nonlinear 0.34 (1.17 to 0.50)
Congestive heart failure
Mortality rate 259 11.1 (10.112.1) 0.17 (0.01 to 0.35) Nonlinear
Readmission rate 264 24.5 (23.525.6) 0.07 (0.27 to 0.14) 0.45 (1.36 to 0.47)
Pneumonia
Mortality rate 268 11.6 (10.413.2) 0.17 (0.42 to 0.07) 0.35 (1.19 to 0.49)
Readmission rate 268 18.2 (17.319.1) 0.04 (0.20 to 0.11) 0.56 (1.27 to 0.16)

Relationship Between Financial Performance and Publicly Reported Outcomes

Acute Myocardial Infarction

We did not observe a consistent relationship between hospital financial performance and AMI mortality and readmission rates. In our overall adjusted analyses, net revenue from operations was not associated with mortality, but was significantly associated with a decrease in AMI readmissions among hospitals with net revenue from operations between approximately $5 million to $145 million (nonlinear relationship, F statistic = 4.8, P < 0.001 (Table 2, Figure 1A). However, after excluding 23 extreme outlying hospitals by net revenue from operations (10 underperformers with net revenue <$49.4 million and 13 overperformers with net revenue >$52.1 million), this relationship was no longer observed. Using operating margin instead of net revenue from operations as the measure of hospital financial performance, we observed a 0.2% increase in AMI mortality (95% confidence interval [CI]: 0.06%‐0.35%) (see Supporting Table 1 and Supporting Figure 2 in the online version of this article) for each 10% increase in operating margin, which persisted with the exclusion of 5 outlying hospitals by operating margin (all 5 were underperformers, with operating margins <38.6%). However, using total margin as the measure of financial performance, there was no significant relationship with either mortality or readmissions (see Supporting Table 2 and Supporting Figure 3 in the online version of this article).

Figure 1
Relationship between financial performance and 30‐day readmission and mortality. The open circles represent individual hospitals. The bold dashed line and the bold solid line are the unadjusted and adjusted cubic spline curves, respectively, representing the nonlinear relationship between net revenue from operations and each outcome. The shaded grey area represents the 95% confidence interval for the adjusted cubic spline curve. Thin vertical dashed lines represent median values for net revenue from operations. Multivariate models were adjusted for teaching status, metropolitan status (urban vs rural), bed size, safety‐net hospital status, hospital ownership, Medicare caseload, and volume of cases reported for the respective outcome, accounting for clustering of hospitals by owner. *Twenty‐three hospitals were identified as outliers with respect to net revenue from clinical operations (10 “underperformers” with net revenue <−$49.4 million and 13 “overperformers” with net revenue >$52.1 million.

Congestive Heart Failure

In our primary analyses, we did not observe a significant relationship between net revenue from operations and CHF mortality and readmission rates. However, after excluding 23 extreme outliers, increasing net revenue from operations was associated with a modest increase in CHF mortality among hospitals, with net revenue between approximately $35 million and $20 million (nonlinear relationship, F statistic = 3.6, P = 0.008 (Table 2, Figure 1B). Using alternate measures of financial performance, we observed a consistent relationship between increasing hospital financial performance and higher 30‐day CHF mortality rate. Using operating margin, we observed a slight increase in the mortality rate for CHF (0.26% increase in CHF RSMR for every 10% increase in operating margin) (95% CI: 0.07%‐0.45%) (see Supporting Table 1 and Supporting Figure 2 in the online version of this article), which persisted after the exclusion of 5 extreme outliers. Using total margin, we observed a significant but modest association between improved hospital financial performance and increased mortality rate for CHF (nonlinear relationship, F statistic = 2.9, P = 0.03) (see Supporting Table 2 and Supporting Figure 3 in the online version of this article), which persisted after the exclusion of 3 extreme outliers (0.32% increase in CHF RSMR for every 10% increase in total margin) (95% CI: 0.03%‐0.62%).

Pneumonia

Hospital financial performance (using net revenue, operating margin, or total margin) was not associated with 30‐day PNA mortality or readmission rates.

Relationship of Readmission and Mortality Rates on Subsequent Hospital Financial Performance

Compared to hospitals in the highest tertile of readmission and mortality rates (ie, those with the worst rates), hospitals in the lowest tertile of readmission and mortality rates (ie, those with the best rates) had a similar magnitude of increase in net revenue from operations from 2008 to 2012 (Table 3). The difference‐in‐differences analyses showed no relationship between readmission or mortality rates for AMI, CHF, and PNA and changes in net revenue from operations from 2008 to 2012 (difference‐in‐differences estimates ranged from $8.61 to $6.77 million, P > 0.3 for all). These results were robust to the exclusion of hospitals with a change in ownership and extreme outliers by net revenue from operations (data not reported).

Difference in the Differences in Financial Performance Between the Worst‐ and the Best‐Performing Hospitals
Outcome Tertile With Highest Outcome Rates (Worst Hospitals) Tertile With Lowest Outcome Rates (Best Hospitals) Difference in Net From Operations Differences Between Highest and Lowest Outcome Rate Tertiles, $ Million (95% CI) P
Outcome, Median % (IQR) Gain/Loss in Net Revenue From Operations From 2008 to 2012, $ Million* Outcome, Median % (IQR) Gain/Loss in Net Revenue from Operations From 2008 to 2012, $ Million*
  • NOTE: Abbreviations: AMI, acute myocardial infarction; CHF, congestive heart failure; CI, confidence interval; IQR, interquartile range; PNA, pneumonia. *Differences were calculated as net revenue from clinical operations in 2012 minus net revenue from clinical operations in 2008. Net revenue in 2008 was adjusted to 2012 US dollars using the chained Consumer Price Index for all urban consumers. Each outcome was modeled separately and adjusted for year, tertile of performance for the respective outcome, the interaction between year and tertile (difference‐in‐differences term), teaching status, metropolitan status (urban vs rural), bed size, safety‐net hospital status, hospital ownership type, Medicare caseload, volume of cases reported for the respective outcome, and interactions for year with bed size, safety‐net hospital status, and Medicare caseload, accounting for clustering of hospitals by owner.

AMI mortality 16.7 (16.217.4) +65.62 13.8 (13.314.2) +74.23 8.61 (27.95 to 10.73) 0.38
AMI readmit 20.7 (20.321.5) +38.62 18.3 (17.718.6) +31.85 +6.77 (13.24 to 26.77) 0.50
CHF mortality 13.0 (12.313.9) +45.66 9.6 (8.910.1) +48.60 2.94 (11.61 to 5.73) 0.50
CHF readmit 26.2 (25.726.9) +47.08 23.0 (22.323.5) +46.08 +0.99 (10.51 to 12.50) 0.87
PNA mortality 13.9 (13.314.7) +43.46 9.9 (9.310.4) +38.28 +5.18 (7.01 to 17.37) 0.40
PNA readmit 19.4 (19.120.1) +47.21 17.0 (16.517.3) +45.45 +1.76 (8.34 to 11.86) 0.73

DISCUSSION

Using audited financial data from California hospitals in 2008 and 2012, and CMS data on publicly reported outcomes from 2008 to 2011, we found no consistent relationship between hospital financial performance and publicly reported outcomes for AMI and PNA. However, better hospital financial performance was associated with a modest increase in 30‐day risk‐standardized CHF mortality rates, which was consistent across all 3 measures of hospital financial performance. Reassuringly, there was no difference in the change in net revenue from operations between 2008 and 2012 between hospitals in the highest and lowest tertiles of readmission and mortality rates for AMI, CHF, and PNA. In other words, hospitals with the lowest rates of 30‐day readmissions and mortality for AMI, CHF, and PNA did not experience a loss in net revenue from operations over time, compared to hospitals with the highest readmission and mortality rates.

Our study differs in several important ways from Ly et al., the only other study to our knowledge that investigated the relationship between hospital financial performance and outcomes for patients with AMI, CHF, and PNA.[19] First, outcomes in the Ly et al. study were ascertained in 2007, which preceded public reporting of outcomes. Second, the primary comparison was between hospitals in the bottom versus top decile of operating margin. Although Ly and colleagues also found no association between hospital financial performance and mortality rates for these 3 conditions, they found a significant absolute decrease of approximately 3% in readmission rates among hospitals in the top decile of operating margin versus those in bottom decile. However, readmission rates were comparable among the remaining 80% of hospitals, suggesting that these findings primarily reflected the influence of a few outlier hospitals. Third, the use of nonuniformly audited hospital financial data may have resulted in misclassification of financial performance. Our findings also differ from 2 previous studies that identified a modest association between improved hospital financial performance and decreased adverse patient safety events.[18, 20] However, publicly reported outcomes may not be fully representative of hospital quality and patient safety.[28, 29]

The limited association between hospital financial performance and publicly reported outcomes for AMI and PNA is noteworthy for several reasons. First, publicly reporting outcomes alone without concomitant changes to reimbursement may be inadequate to create strong financial incentives for hospital investment in quality improvement initiatives. Hospitals participating in both public reporting of outcomes and pay‐for‐performance have been shown to achieve greater improvements in outcomes than hospitals engaged only in public reporting.[30] Our time interval for ascertainment of outcomes preceded CMS implementation of the Hospital Readmissions Reduction Program (HRRP) in October 2012, which withholds up to 3% of Medicare hospital reimbursements for higher than expected mortality and readmission rates for AMI, CHF, and PNA. Once outcomes data become available for a 3‐year post‐HRRP implementation period, the impact of this combined approach can be assessed. Second, because adherence to many evidence‐based process measures for these conditions (ie, aspirin use in AMI) is already high, there may be a ceiling effect present that obviates the need for further hospital financial investment to optimize delivery of best practices.[31, 32] Third, hospitals themselves may contribute little to variation in mortality and readmission risk. Of the total variation in mortality and readmission rates among Texas Medicare beneficiaries, only about 1% is attributable to hospitals, whereas 42% to 56% of the variation is explained by differences in patient characteristics.[33, 34] Fourth, there is either low‐quality or insufficient evidence that transitional care interventions specifically targeted to patients with AMI or PNA result in better outcomes.[35] Thus, greater financial investment in hospital‐initiated and postdischarge transitional care interventions for these specific conditions may result in less than the desired effect. Lastly, many hospitalizations for these conditions are emergency hospitalizations that occur after patients present to the emergency department with unexpected and potentially life‐threatening symptoms. Thus, patients may not be able to incorporate the reputation or performance metrics of a hospital in their decisions for where they are hospitalized for AMI, CHF, or PNA despite the public reporting of outcomes.

Given the strong evidence that transitional care interventions reduce readmissions and mortality among patients hospitalized with CHF, we were surprised to find that improved hospital financial performance was associated with an increased risk‐adjusted CHF mortality rate.[36] This association held true for all 3 different measures of hospital financial performance, suggesting that this unexpected finding is unlikely to be the result of statistical chance, though potential reasons for this association remain unclear. One possibility is that the CMS model for CHF mortality may not adequately risk adjust for severity of illness.[37, 38] Thus, robust financial performance may be a marker for hospitals with more advanced heart failure services that care for more patients with severe illness.

Our findings should be interpreted in the context of certain limitations. Our study only included an analysis of outcomes for AMI, CHF, and PNA among older fee‐for‐service Medicare beneficiaries aggregated at the hospital level in California between 2008 and 2012, so generalizability to other populations, conditions, states, and time periods is uncertain. The observational design precludes a robust causal inference between financial performance and outcomes. For readmissions, rates were publicly reported for only the last 2 years of the 3‐year reporting period; thus, our findings may underestimate the association between hospital financial performance and publicly reported readmission rates.

CONCLUSION

There is no consistent relationship between hospital financial performance and subsequent publicly reported outcomes for AMI and PNA. However, for unclear reasons, hospitals with better financial performance had modestly higher CHF mortality rates. Given this limited association, public reporting of outcomes may have had less than the intended impact in motivating hospitals to invest in quality improvement. Additional financial incentives in addition to public reporting, such as readmissions penalties, may help motivate hospitals with robust financial performance to further improve outcomes. This would be a key area for future investigation once outcomes data are available for the 3‐year period following CMS implementation of readmissions penalties in 2012. Reassuringly, there was no association between low 30‐day mortality and readmissions rates and subsequent poor financial performance, suggesting that improved outcomes do not necessarily lead to loss of revenue.

Disclosures

Drs. Nguyen, Halm, and Makam were supported in part by the Agency for Healthcare Research and Quality University of Texas Southwestern Center for Patient‐Centered Outcomes Research (1R24HS022418‐01). Drs. Nguyen and Makam received funding from the University of Texas Southwestern KL2 Scholars Program (NIH/NCATS KL2 TR001103). The study sponsors had no role in design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. The authors have no conflicts of interest to disclose.

References
  1. Centers for Medicare and Medicaid Services. Office of the Actuary. National Health Statistics Group. National Healthcare Expenditures Data. Baltimore, MD; 2013. https://www.cms.gov/Research-Statistics-Data-and-Systems/Statistics-Trends-and-Reports/NationalHealthExpendData/NationalHealthAccountsHistorical.html. Accessed February 16, 2016.
  2. Brill S. Bitter pill: why medical bills are killing us. Time Magazine. February 20, 2013:1655.
  3. Ginsburg PB. Fee‐for‐service will remain a feature of major payment reforms, requiring more changes in Medicare physician payment. Health Aff (Millwood). 2012;31(9):19771983.
  4. Leatherman S, Berwick D, Iles D, et al. The business case for quality: case studies and an analysis. Health Aff (Millwood). 2003;22(2):1730.
  5. Marshall MN, Shekelle PG, Davies HT, Smith PC. Public reporting on quality in the United States and the United Kingdom. Health Aff (Millwood). 2003;22(3):134148.
  6. Swensen SJ, Dilling JA, Mc Carty PM, Bolton JW, Harper CM. The business case for health‐care quality improvement. J Patient Saf. 2013;9(1):4452.
  7. Hibbard JH, Stockard J, Tusler M. Hospital performance reports: impact on quality, market share, and reputation. Health Aff (Millwood). 2005;24(4):11501160.
  8. Rauh SS, Wadsworth EB, Weeks WB, Weinstein JN. The savings illusion—why clinical quality improvement fails to deliver bottom‐line results. N Engl J Med. 2011;365(26):e48.
  9. Meyer JA, Silow‐Carroll S, Kutyla T, Stepnick LS, Rybowski LS. Hospital Quality: Ingredients for Success—Overview and Lessons Learned. New York, NY: Commonwealth Fund; 2004.
  10. Silow‐Carroll S, Alteras T, Meyer JA. Hospital Quality Improvement: Strategies and Lessons from U.S. Hospitals. New York, NY: Commonwealth Fund; 2007.
  11. Bazzoli GJ, Clement JP, Lindrooth RC, et al. Hospital financial condition and operational decisions related to the quality of hospital care. Med Care Res Rev. 2007;64(2):148168.
  12. Casalino LP, Elster A, Eisenberg A, Lewis E, Montgomery J, Ramos D. Will pay‐for‐performance and quality reporting affect health care disparities? Health Aff (Millwood). 2007;26(3):w405w414.
  13. Werner RM, Goldman LE, Dudley RA. Comparison of change in quality of care between safety‐net and non‐safety‐net hospitals. JAMA. 2008;299(18):21802187.
  14. Bhalla R, Kalkut G. Could Medicare readmission policy exacerbate health care system inequity? Ann Intern Med. 2010;152(2):114117.
  15. Hernandez AF, Curtis LH. Minding the gap between efforts to reduce readmissions and disparities. JAMA. 2011;305(7):715716.
  16. Terry DF, Moisuk S. Medicare Health Support Pilot Program. N Engl J Med. 2012;366(7):666; author reply 667–668.
  17. Fontanarosa PB, McNutt RA. Revisiting hospital readmissions. JAMA. 2013;309(4):398400.
  18. Encinosa WE, Bernard DM. Hospital finances and patient safety outcomes. Inquiry. 2005;42(1):6072.
  19. Ly DP, Jha AK, Epstein AM. The association between hospital margins, quality of care, and closure or other change in operating status. J Gen Intern Med. 2011;26(11):12911296.
  20. Bazzoli GJ, Chen HF, Zhao M, Lindrooth RC. Hospital financial condition and the quality of patient care. Health Econ. 2008;17(8):977995.
  21. State of California Office of Statewide Health Planning and Development. Healthcare Information Division. Annual financial data. Available at: http://www.oshpd.ca.gov/HID/Products/Hospitals/AnnFinanData/PivotProfles/default.asp. Accessed June 23, 2015.
  22. Centers for Medicare 4(1):1113.
  23. Ross JS, Cha SS, Epstein AJ, et al. Quality of care for acute myocardial infarction at urban safety‐net hospitals. Health Aff (Millwood). 2007;26(1):238248.
  24. Silber JH, Rosenbaum PR, Brachet TJ, et al. The Hospital Compare mortality model and the volume‐outcome relationship. Health Serv Res. 2010;45(5 Pt 1):11481167.
  25. Tukey J. Exploratory Data Analysis. Boston, MA: Addison‐Wesley; 1977.
  26. Press MJ, Scanlon DP, Ryan AM, et al. Limits of readmission rates in measuring hospital quality suggest the need for added metrics. Health Aff (Millwood). 2013;32(6):10831091.
  27. Stefan MS, Pekow PS, Nsa W, et al. Hospital performance measures and 30‐day readmission rates. J Gen Intern Med. 2013;28(3):377385.
  28. Lindenauer PK, Remus D, Roman S, et al. Public reporting and pay for performance in hospital quality improvement. N Engl J Med. 2007;356(5):486496.
  29. Spatz ES, Sheth SD, Gosch KL, et al. Usual source of care and outcomes following acute myocardial infarction. J Gen Intern Med. 2014;29(6):862869.
  30. Werner RM, Bradlow ET. Public reporting on hospital process improvements is linked to better patient outcomes. Health Aff (Millwood). 2010;29(7):13191324.
  31. Goodwin JS, Lin YL, Singh S, Kuo YF. Variation in length of stay and outcomes among hospitalized patients attributable to hospitals and hospitalists. J Gen Intern Med. 2013;28(3):370376.
  32. Singh S, Lin YL, Kuo YF, Nattinger AB, Goodwin JS. Variation in the risk of readmission among hospitals: the relative contribution of patient, hospital and inpatient provider characteristics. J Gen Intern Med. 2014;29(4):572578.
  33. Prvu Bettger J, Alexander KP, Dolor RJ, et al. Transitional care after hospitalization for acute stroke or myocardial infarction: a systematic review. Ann Intern Med. 2012;157(6):407416.
  34. Jha AK, Orav EJ, Li Z, Epstein AM. The inverse relationship between mortality rates and performance in the Hospital Quality Alliance measures. Health Aff (Millwood). 2007;26(4):11041110.
  35. Amarasingham R, Moore BJ, Tabak YP, et al. An automated model to identify heart failure patients at risk for 30‐day readmission or death using electronic medical record data. Med Care. 2010;48(11):981988.
  36. Fuller RL, Atkinson G, Hughes JS. Indications of biased risk adjustment in the hospital readmission reduction program. J Ambul Care Manage. 2015;38(1):3947.
References
  1. Centers for Medicare and Medicaid Services. Office of the Actuary. National Health Statistics Group. National Healthcare Expenditures Data. Baltimore, MD; 2013. https://www.cms.gov/Research-Statistics-Data-and-Systems/Statistics-Trends-and-Reports/NationalHealthExpendData/NationalHealthAccountsHistorical.html. Accessed February 16, 2016.
  2. Brill S. Bitter pill: why medical bills are killing us. Time Magazine. February 20, 2013:1655.
  3. Ginsburg PB. Fee‐for‐service will remain a feature of major payment reforms, requiring more changes in Medicare physician payment. Health Aff (Millwood). 2012;31(9):19771983.
  4. Leatherman S, Berwick D, Iles D, et al. The business case for quality: case studies and an analysis. Health Aff (Millwood). 2003;22(2):1730.
  5. Marshall MN, Shekelle PG, Davies HT, Smith PC. Public reporting on quality in the United States and the United Kingdom. Health Aff (Millwood). 2003;22(3):134148.
  6. Swensen SJ, Dilling JA, Mc Carty PM, Bolton JW, Harper CM. The business case for health‐care quality improvement. J Patient Saf. 2013;9(1):4452.
  7. Hibbard JH, Stockard J, Tusler M. Hospital performance reports: impact on quality, market share, and reputation. Health Aff (Millwood). 2005;24(4):11501160.
  8. Rauh SS, Wadsworth EB, Weeks WB, Weinstein JN. The savings illusion—why clinical quality improvement fails to deliver bottom‐line results. N Engl J Med. 2011;365(26):e48.
  9. Meyer JA, Silow‐Carroll S, Kutyla T, Stepnick LS, Rybowski LS. Hospital Quality: Ingredients for Success—Overview and Lessons Learned. New York, NY: Commonwealth Fund; 2004.
  10. Silow‐Carroll S, Alteras T, Meyer JA. Hospital Quality Improvement: Strategies and Lessons from U.S. Hospitals. New York, NY: Commonwealth Fund; 2007.
  11. Bazzoli GJ, Clement JP, Lindrooth RC, et al. Hospital financial condition and operational decisions related to the quality of hospital care. Med Care Res Rev. 2007;64(2):148168.
  12. Casalino LP, Elster A, Eisenberg A, Lewis E, Montgomery J, Ramos D. Will pay‐for‐performance and quality reporting affect health care disparities? Health Aff (Millwood). 2007;26(3):w405w414.
  13. Werner RM, Goldman LE, Dudley RA. Comparison of change in quality of care between safety‐net and non‐safety‐net hospitals. JAMA. 2008;299(18):21802187.
  14. Bhalla R, Kalkut G. Could Medicare readmission policy exacerbate health care system inequity? Ann Intern Med. 2010;152(2):114117.
  15. Hernandez AF, Curtis LH. Minding the gap between efforts to reduce readmissions and disparities. JAMA. 2011;305(7):715716.
  16. Terry DF, Moisuk S. Medicare Health Support Pilot Program. N Engl J Med. 2012;366(7):666; author reply 667–668.
  17. Fontanarosa PB, McNutt RA. Revisiting hospital readmissions. JAMA. 2013;309(4):398400.
  18. Encinosa WE, Bernard DM. Hospital finances and patient safety outcomes. Inquiry. 2005;42(1):6072.
  19. Ly DP, Jha AK, Epstein AM. The association between hospital margins, quality of care, and closure or other change in operating status. J Gen Intern Med. 2011;26(11):12911296.
  20. Bazzoli GJ, Chen HF, Zhao M, Lindrooth RC. Hospital financial condition and the quality of patient care. Health Econ. 2008;17(8):977995.
  21. State of California Office of Statewide Health Planning and Development. Healthcare Information Division. Annual financial data. Available at: http://www.oshpd.ca.gov/HID/Products/Hospitals/AnnFinanData/PivotProfles/default.asp. Accessed June 23, 2015.
  22. Centers for Medicare 4(1):1113.
  23. Ross JS, Cha SS, Epstein AJ, et al. Quality of care for acute myocardial infarction at urban safety‐net hospitals. Health Aff (Millwood). 2007;26(1):238248.
  24. Silber JH, Rosenbaum PR, Brachet TJ, et al. The Hospital Compare mortality model and the volume‐outcome relationship. Health Serv Res. 2010;45(5 Pt 1):11481167.
  25. Tukey J. Exploratory Data Analysis. Boston, MA: Addison‐Wesley; 1977.
  26. Press MJ, Scanlon DP, Ryan AM, et al. Limits of readmission rates in measuring hospital quality suggest the need for added metrics. Health Aff (Millwood). 2013;32(6):10831091.
  27. Stefan MS, Pekow PS, Nsa W, et al. Hospital performance measures and 30‐day readmission rates. J Gen Intern Med. 2013;28(3):377385.
  28. Lindenauer PK, Remus D, Roman S, et al. Public reporting and pay for performance in hospital quality improvement. N Engl J Med. 2007;356(5):486496.
  29. Spatz ES, Sheth SD, Gosch KL, et al. Usual source of care and outcomes following acute myocardial infarction. J Gen Intern Med. 2014;29(6):862869.
  30. Werner RM, Bradlow ET. Public reporting on hospital process improvements is linked to better patient outcomes. Health Aff (Millwood). 2010;29(7):13191324.
  31. Goodwin JS, Lin YL, Singh S, Kuo YF. Variation in length of stay and outcomes among hospitalized patients attributable to hospitals and hospitalists. J Gen Intern Med. 2013;28(3):370376.
  32. Singh S, Lin YL, Kuo YF, Nattinger AB, Goodwin JS. Variation in the risk of readmission among hospitals: the relative contribution of patient, hospital and inpatient provider characteristics. J Gen Intern Med. 2014;29(4):572578.
  33. Prvu Bettger J, Alexander KP, Dolor RJ, et al. Transitional care after hospitalization for acute stroke or myocardial infarction: a systematic review. Ann Intern Med. 2012;157(6):407416.
  34. Jha AK, Orav EJ, Li Z, Epstein AM. The inverse relationship between mortality rates and performance in the Hospital Quality Alliance measures. Health Aff (Millwood). 2007;26(4):11041110.
  35. Amarasingham R, Moore BJ, Tabak YP, et al. An automated model to identify heart failure patients at risk for 30‐day readmission or death using electronic medical record data. Med Care. 2010;48(11):981988.
  36. Fuller RL, Atkinson G, Hughes JS. Indications of biased risk adjustment in the hospital readmission reduction program. J Ambul Care Manage. 2015;38(1):3947.
Issue
Journal of Hospital Medicine - 11(7)
Issue
Journal of Hospital Medicine - 11(7)
Page Number
481-488
Page Number
481-488
Article Type
Display Headline
Relationship between hospital financial performance and publicly reported outcomes
Display Headline
Relationship between hospital financial performance and publicly reported outcomes
Sections
Article Source
© 2016 Society of Hospital Medicine
Disallow All Ads
Correspondence Location
Address for correspondence and reprint requests: Oanh Kieu Nguyen, MD, 5323 Harry Hines Blvd., Dallas, Texas 75390‐9169; Telephone: 214‐648‐3135; Fax: 214‐648‐3232; E‐mail: oanhk.nguyen@utsouthwestern.edu
Content Gating
Gated (full article locked unless allowed per User)
Gating Strategy
First Peek Free
Article PDF Media
Media Files

Identifying an Idle Line for Its Removal

Article Type
Changed
Mon, 01/02/2017 - 19:34
Display Headline
Can the identification of an idle line facilitate its removal? A comparison between a proposed guideline and clinical practice

Infections acquired in the hospital are termed healthcare‐associated infections (HAIs) and include central lineassociated blood stream infections (CLABSIs). Among HAIs, CLABSIs cause the highest number of preventable deaths.[1] Central venous catheters (CVCs) or central lines are commonly used in the hospital.[2] Each year their use is linked to 250,000 cases of CLABSIs in the United States.[3] Some CLABSIs may be prevented by the prompt removal of the line.[4] However, CVCs are often retained after their clinical indication has lapsed and are then referred to as idle lines.[5, 6] In this work, we propose and theoretically test a guideline to facilitate the safe removal of an idle line by observing the agreement and disagreement between actual practice and the proposed guideline.

METHODS

Setting

This work was conducted at a large, urban, tertiary care, academic health center in the United States as a collaborative effort to improve quality at our institution.[7]

Design and Patients

The reports linked with the electronic medical records at our institution include a daily, ward‐by‐ward listing of patients who have access other than a peripheral line in place. This central line dashboard accesses the information on intravenous access charted by bedside nurses to create a list of patients on every ward who have any kind of central access. Temporary central venous lines (CVLs), peripherally inserted central catheters (PICCs), ports, and dialysis catheters are all included. The unit charge nurses and managers use this dashboard to facilitate compliance with line care bundles. We used this source to identify patients with either type of CVC (CVLs or PICCs) on 8 days in August 2014, September 2014, and October 2014. Patients were included if they had a CVC and were on a general medical or surgical ward bed on audit day. CVLs at all sites were included (femoral, subclavian, and internal jugular). Patients in an intensive care unit (ICU) or progressive care unit on the day of the audit were excluded. Patients whose catheters were for chemotherapy and those admitted for a transplant or receiving palliative or hospice care were also excluded.

Data Collection

A protocol for data collection was written out, and a training session was held to review definitions, data sources, and methods to ensure consistency. Two authors (M.M. and J.D.) assisted by an experienced clinical nurse specialist collected data on the patients captured on audit days. Each chart was reviewed on the day of the audit, the 2 days preceding the audit day, and then followed until the patient was either discharged from the hospital or transferred to a higher level of care, died, or transitioned to palliative or hospice care. Demographics, details about the line, and the criteria for justified use were extracted from the electronic medical record.

Definitions

Justified and Idle Days

To justify the presence of a CVC on any given day, we used criteria that fell under 3 categories: intravenous (IV) access needs, unstable vitals, or meeting sepsis/systemic inflammatory response syndrome (SIRS) criteria (Table 1). For vital signs, a single abnormal reading was counted as fulfilling criteria for that day. If no criterion for justified use was met, the line was considered idle for that day.

Criteria to Justify the Presence of a Central Line
  • NOTE: If none of these criteria were met, the line was considered idle for that day. Abbreviations: IV, intravenous; TPN, total parenteral nutrition; SIRS, systemic inflammatory response syndrome; WBC, white blood count.

IV access needs
Expected duration of IV antibiotics >6 days
Administration of TPN
Anticipated requirement of home IV medications
Requirement of IV medications with documented difficult access
Hemorrhage requiring blood transfusions
Requiring more than 3 infusions
Requiring more than 2 infusions and blood transfusions
Abnormal vitals
Diastolic blood pressure >120 mm Hg
Systolic blood pressure <90 mm Hg
Systolic blood pressure >200 mm Hg
Heart rate >120 beats per minute
Heart rate <50 beats per minute
Respiratory rate >30 breaths per minute
Respiratory rate <10 breaths per minute
Oxygen saturation <90% as measured by pulse oximetry
Meeting SIRS criteria (2 or more of the following present)
Temp >38C, Temp <36C, heart rate >90 beats per minute, respiratory rate >20 breaths per minute, WBC >12,000/mm3, WBC <1,000/mm3, bandemia >10%

Qualifying IV access needs were defined similarly to those previously used,[5, 6] whereas those for SIRS followed the current consensus.[8] To determine the number of IV medications or infusions, the medication administration record was reviewed. If 3 or more infusions were found, their compatibility was checked using the same database that nurses use at our institution. Difficult IV access was inferred from the indication for line placement, coupled with the absence of documentation of a peripheral IV. Clinical progress notes were reviewed to extract information on the length of proposed IV antibiotic courses, and discharge instructions were reviewed to verify whether the line was removed prior to discharge or not. The cutoffs for diastolic blood pressure, respiratory rate, and oxygen saturation used to label patients hemodynamically labile are the same as those used by previous authors and also constitute the definition of hypertensive urgency.[5, 9] However, we diverged from the values previously used for tachycardia, bradycardia, and systolic hypotension using heart rates >120 and <50 beats per minute (compared to >130 and <40 beats per minute) and systolics <90 mm Hg (compared to <80 mm Hg) to justify the line.[5] Early warning scores have been used to identify hospitalized ward patients who are at risk for clinical deterioration. Although each score utilizes different thresholds, the risk for clinical deterioration increases as the vitals worsen.[10] Bearing this in mind, the thresholds we elected to use are more clinically conservative and also parallel the nursing call orders currently used at our institution.

Proposed Guideline

We propose the guideline that a CVC may be safely removed the day after the first idle day.

RESULTS

A total of 126 lines were observed in 126 patients. Eighty‐three (65.9%) of the lines were PICCs. The remaining 43 (34.1%) were CVLs. The indications for line placement were distributed between the need for central access, total parenteral nutrition, or antibiotics (Table 2).

Description of the Study Cohort
Description Value
  • NOTE: Abbreviations: CVL, central venous line; IV, intravenous; PICC, peripherally inserted central catheter; SD, standard deviation; TPN, total parenteral nutrition.

Age in yrs mean (SD) 55.7 (18)
Gender, n (%)
Female 66 (52.4)
Male 60 (47.6)
Type of line, n (%)
PICC 83 (65.9)
CVL 43 (34.1)
Indication for line placement, n (%)
Meds requiring central access or TPN 36 (28.6)
Antibiotics 34 (27.0)
Hemodynamic instability 30 (23.8)
Poor access with multiple IV medications 18 (14.3)
Unknown 8 (6.3)
Line removed prior to discharge, n (%)
Yes 76 (60.3)
No 50 (39.7)

Out of the 126 patients, 50 (39.7%) were discharged from the hospital, died, were transferred to a higher level of care, or transitioned to palliative or hospice care with the line in place. In the remaining 76 patients, the audit captured 635 days, out of which a line was in place for 522 (82.2%) days. Of these 522 days, the line's presence was justified by our criteria for 351 (67.2%) days. The most common reason for a line to be justified on any given day was the need for antibiotics followed by the presence of SIRS criteria (Table 3). The remaining 171 (32.7%) days were idle.

Criteria Met for the 351 Justified Line Days
Criteria N %
  • NOTE: Abbreviations: IV, intravenous; SIRS, systemic inflammatory response syndrome; TPN, total parenteral nutrition; hr: heart rate; bp. blood pressure. *Totals exceed 100% because multiple indications may exist.

No. of factors justifying use
1 184 52.4%
2 127 36.2%
>2 40 11.4
Reason for justifying line*
Anticipate home or >6 days of antibiotic use 181 51.6
SIRS criteria 124 35.3
TPN 96 27.4
Hemodynamic instability based on hr and bp 78 22.2
Poor access with need for IV medications 57 16.2
Respiratory rate (<10 or >30/minute) 25 7.1
Active hemorrhage requiring transfusions 12 3.4
>3 infusions 6 1.7

A comparison of the actual removal of the 76 central lines in practice relative to the proposed guideline of removing it the day following the first idle day is displayed in Figure 1. The central line was removed prior to our proposed guideline in 11 (14.5%) patients, and waiting for an idle day in these patients would have added 46 line days. In almost half the patients (n = 36, 47.4%), the line was removed in agreement with the proposed guideline. None of the patients in whom the line was removed prior to or in accordance with our proposed guideline required a line reinsertion. Line removal was delayed in 29 (38.2%) patients when compared to our proposed guideline. In these patients, following the guideline would have created 122 line‐free days. Most (n = 102, 83.6%) of these potential line‐free days were idle. Twenty (16.4%) were justified, of which half (n = 10) were justified by meeting SIRS criteria.

Figure 1
Pictorial demonstration of the comparison between line removal in practice and the proposed guideline of removing it the day following the first idle day. Each bar represents 1 of the 76 patients in whom the line was removed prior to discharge. The diamond represents the actual removal of the line in practice. The bar is red to indicate that the line will remain in place according to our proposed guideline. It turns to green the day following the first idle day indicating that our guideline would recommend line removal.

DISCUSSION

Approximately 1 in every 25 inpatients in the United States has at least 1 HAI on any given day.[11] The case fatality rate from a CLABSI may be as high as 12%, and up to 70% of these infections may be preventable.[1, 12] Interventions successful in decreasing CLABSIs have focused on patients in ICUs.[13] However, CVCs are increasingly prevalent outside the ICU, with over 4.5 million line days in non‐ICU beds reported to the National Healthcare Safety Network in 2012 compared to 2.5 million in 2010.[2, 14] However, adherence rates to infection control practices may be lower on the wards than in the ICUs.[6, 15] Consequently, although the number of CLABSIs has declined over the last decade, most are now occurring outside the ICU.[16] These trends underscore the need to develop strategies aimed at CLABSI prevention on the floors.

Analogous to the life cycle of a urinary catheter described by Meddings et al.,[17] strategies to prevent CLABSIs and other CVC‐related complications may be designed around the life cycle of a CVC. The life cycle starts with insertion and moves on to the maintenance, removal, and possible reinsertion of the line. The process thus starts with the decision to place the line. Over the last decade, this decision making has changed in part due to PICCs. This shift is reflected in PICC prevalence rates: in 2001, 11% of audited central lines were PICCs compared to 56% in 2007.[5, 6] In our audit, 66% of the CVCs were PICCs. This increase in the use of PICCs may be attributable to the ease and safety of their placement coupled with the increased availability of vascular access placement teams.[18] The risk of overuse that may result from such expediency may be countered by adhering to guidelines such as the Michigan Appropriateness Guide for Intravenous Catheters, which provides both clinically detailed guidance and an impetus for reflective decision making around intravenous access.[19]

The placement of CVCs for prolonged parenteral antibiotics may be a particular subset that bears further exploration. Similar to previous reports, we found that a large number of the CVCs were both inserted for and justified by the need for IV antibiotics.[5] Guidelines delineated by the Infectious Diseases Society of America regarding outpatient parenteral antibiotics weigh both the duration of therapy and the antimicrobial's potential for causing phlebitis when recommending the type of intravascular access.[20] Many courses may therefore be completed through peripheral or midline catheters. Developing strong partnerships between infectious disease specialists, hospitalists, and the facilities or home‐care services treating these patients may curtail the use of CVCs for antimicrobial administration.

The main focus of our work is on facilitating the safe removal of CVCs. The risk of CLABSIs increases each day a CVC is in place, and guidelines to prevent CLABSIs include recommendations to promptly remove nonessential catheters.[4, 21] There is also an emerging understanding that the risk of a PICC‐related CLABSI approaches that from a traditional central line in hospitalized patients, and PICCs confer an increased risk of venous thromboembolism.[18, 22] Although nearly half of surveyed hospitalists recently reported leaving PICCs in place until discharge day, our data suggest that this practice may be driven by the trajectory of a patient's recovery as much as by knowledge gaps related to the use of PICCs.[23] In nearly half the instances, clinical practice already mirrors our proposed guideline, with line removal coinciding with both the timing proposed by our guideline and discharge day. However, there is room for improvement, as line removal may have been expedited in the 29 patients in whom the line was retained after the first idle day. Maintaining an awareness of its presence and weighing its risks and benefits daily may facilitate the removal of a CVC. Based on the recent findings that up to a quarter of clinicians are unaware that their patients have a central line, the mere reminder of the presence of a line using such criteria may expedite its removal by triggering a purposeful reassessment of its ongoing need.[24] Premature CVC removal requiring line reinsertion is an unintended consequence that may emerge from the earlier removal of lines. In our sample, none of the patients who had lines removed either prior to or in accordance with our proposed guideline required a line reinsertion. In addition to line reinsertion, delays in laboratory testing and reporting due to the unavailability of access, increased patient discomfort, or increased workload on the bedside nurse or vascular access team must also be considered when implementing strategies aimed at decreasing line days.

We envisage using these criteria to both empower practitioners with knowledge and foster shared accountability between all team members by using a uniform tool. This can occur through partnerships between infection control, clinical nurse specialists, bedside nursing, and physicians. The electronic medical record could be leveraged to scan the record for the criteria and create a notification when the line becomes idle. In alignment with the Michigan Appropriateness Guide for Intravenous Catheters guidelines, we do not support the removal of lines by nursing staff without physician notification.[19] Such principles have been successfully harnessed in strategies to prevent both catheter‐associated urinary tract infections and CLABSIs in ICUs.[13, 25] In light of the complexity surrounding the decision making for CVCs, our criteria were focused on the wards and erred on the side of clinical caution. This clinical conservatism is apparent in the patients in whom lines were removed prior to what our guideline would propose, yet none of the patients required a line reinsertion. As concerns about recrudescent clinical instability may drive decision making around line removal, such conservatism may be warranted initially. However, the fidelity of these criteria in the clinical setting will need prospective validation. In particular, the inclusion of SIRS criteria may have led to an overestimation of justified days. Further studies may be needed to refine the criteria and find a clinical hierarchy that balances the risks and benefits of retaining a central line.

Our work has certain limitations. It is a single center's experience, and our findings may not therefore be generalizable. Except for when the indication for the line was for difficult access, we did not attempt to verify the presence of a peripheral IV. This, in combination with the inclusion of SIRS criteria, likely leads to an underestimation of idle days. In the interest of focusing on patients in whom the decision making around a line would be the least controversial, we did not continue to follow patients who were transferred to a higher level of care. It is possible, however, that these transfers were precipitated by line‐associated complications such as sepsis and would be important to track. We did not measure the agreement between data collectors, although definitions and methodologies were standardized and reviewed prior to data collection. As this was an observational assessment of a proposed guideline, we cannot predict how the recommendations generated by it will be received by clinicians. Although this may prove to be a barrier in adoption, we hope that the conversation it initiates leads to change.

Hospitalists are positioned to potentially influence the entire life cycle of a central line on the floor. Strategies can be enacted at each stage to help decrease the potential of harm from these devices to our patients. Creating and testing criteria and guidelines such as we propose represents just 1 such strategy in a multidisciplinary effort to provide the best possible care we can.

Acknowledgements

The authors thank Jennifer Dunscomb, Kristen Kelly, and their teams, and Deanna Sidwell, Todd Biggerstaff, Joan Miller, Rob Clark, and the tireless providers at Indiana University Health Methodist Hospital for their support.

Disclosures: This work was supported by the Indiana University Health Values Grant for research. The authors have no conflicts of interests to report.

Files
References
  1. Umscheid CA, Mitchell MD, Doshi JA, Agarwal R, Williams K, Brennan PJ. Estimating the proportion of healthcare‐associated infections that are reasonably preventable and the related mortality and costs. Infect Control Hosp Epidemiol. 2011;32(2):101114.
  2. Dudeck MA, Weiner LM, Allen‐Bridson K, et al. National Healthcare Safety Network (NHSN) report, data summary for 2012, device‐associated module. Am J Infect Control. 2013;41(12):11481166.
  3. Maki DG, Kluger DM, Crinch CJ. The risk of bloodstream infection in adults with different intravascular devices: a systematic review of 200 published prospective studies. Mayo Clin Proc. 2006;81(9):11591171.
  4. O'Grady NP, Alexander M, Burns LA, et al. Guidelines for the prevention of intravascular catheter‐related infections. Clin Infect Dis. 2011;52(9):e162e193.
  5. Chernetsky Tejedor S, Tong D, Stein J, et al. Temporary central venous catheter utilization patterns in a large tertiary care center: tracking the “idle central venous catheter.” Infect Control Hosp Epidemiol. 2012;33(1):5057.
  6. Trick WE, Vernon M, Welbel SF, Wisniewski MF, Jernigan JA, Weinstein RA. Unnecessary use of central venous catheters: the need to look outside the intensive care unit. Infect Control Hosp Epidemiol. 2004;25(3):266268.
  7. IU Health Methodist Hospital website. Available at: http://iuhealth.org/methodist/aboIut. Accessed October 20, 2014.
  8. Bone RC, Balk RA, Cerra FB, et al. Definitions for Sepsis and Organ Failure and Guidelines for the Use of Innovative Therapies in Sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest. 2009;136(5 suppl):e28.
  9. Pak KJ, Hu T, Fee C, Wang R, Smith M, Bazzano LA. Acute hypertension: a systematic review and appraisal of guidelines. Ochsner J. 2014;14(4):655663.
  10. Churpek MM, Yuen TC, Edelson DP. Predicting clinical deterioration in the hospital: the impact of outcome selection. Resuscitation. 2013;84(5):564568.
  11. Magill SS, Edwards JR, Bamberg W, et al. Multistate point‐prevalence survey of health care–associated infections. N Engl J Med. 2014;370(13):11981208.
  12. Klevens RM, Edwards JR, Richards CL, et al. Estimating health care‐associated infections and deaths in U.S. hospitals, 2002. Public Health Rep. 2007;122(2):160166.
  13. Pronovost P, Needham D, Berenholtz S, et al. An intervention to decrease catheter‐related bloodstream infections in the ICU. N Engl J Med. 2006;355(26):27252732.
  14. Dudeck MA, Horan TC, Peterson KD, et al. Data summary for 2011, device‐associated module. Centers for Disease Control and Prevention. National Healthcare Safety Network (NHSN) Report. Available at: http://www.cdc.gov/nhsn/PDFs/dataStat/NHSN‐Report‐2011‐Data‐Summary.pdf. Published April 1, 2013. Last accessed January 2015.
  15. Burdeu G, Currey J, Pilcher D. Idle central venous catheter‐days pose infection risk for patients after discharge from intensive care. Am J Infect Control. 2014;42(4):453455.
  16. Liang SY, Marschall J. Update on emerging infections: news from the Centers for Disease Control and Prevention. Vital signs: central line‐associated blood stream infections—United States, 2001, 2008, and 2009. Ann Emerg Med. 2011;58(5):447451.
  17. Meddings J, Rogers MAM, Krein SL, Fakih MG, Olmsted RN, Saint S. Reducing unnecessary urinary catheter use and other strategies to prevent catheter‐associated urinary tract infection: an integrative review. BMJ Qual Saf. 2014;23(4):277289.
  18. Chopra V, O'Horo JC, Rogers MAM, Maki DG, Safdar N. The risk of bloodstream infection associated with peripherally inserted central catheters compared with central venous catheters in adults: a systematic review and meta‐analysis. Infect Control Hosp Epidemiol. 2013;34(9):908918.
  19. Chopra V, Flanders SA, Saint S, et al. The Michigan Appropriateness Guide for Intravenous Catheters (MAGIC): results from a multispecialty panel using the RAND/UCLA Appropriateness Method. Ann Intern Med. 2015;163(6 suppl):S1S40.
  20. Tice AD, Rehm SJ, Dalovisio JR, et al. Practice guidelines for outpatient parenteral antimicrobial therapy. IDSA guidelines. Clin Infect Dis. 2004;38(12):16511672.
  21. McLaws M‐L, Berry G. Nonuniform risk of bloodstream infection with increasing central venous catheter‐days. Infect Control Hosp Epidemiol. 2005;26(8):715719.
  22. Chopra V, Anand S, Hickner A, et al. Risk of venous thromboembolism associated with peripherally inserted central catheters: a systematic review and meta‐analysis. Lancet. 2013;382(9889):311325.
  23. Chopra V, Kuhn L, Flanders SA, Saint S, Krein SL. Hospitalist experiences, practice, opinions, and knowledge regarding peripherally inserted central catheters: results of a national survey. J Hosp Med. 2013;8(11):635638.
  24. Chopra V, Govindan S, Kuhn L, et al. Do clinicians know which of their patients have central venous catheters? Ann Intern Med. 2014;161(8):562.
  25. Reilly L, Sullivan P, Ninni S, Fochesto D, Williams K, Fetherman B. Reducing foley catheter device days in an intensive care unit: using the evidence to change practice. AACN Adv Crit Care. 2006;17(3):272283.
Article PDF
Issue
Journal of Hospital Medicine - 11(7)
Page Number
489-493
Sections
Files
Files
Article PDF
Article PDF

Infections acquired in the hospital are termed healthcare‐associated infections (HAIs) and include central lineassociated blood stream infections (CLABSIs). Among HAIs, CLABSIs cause the highest number of preventable deaths.[1] Central venous catheters (CVCs) or central lines are commonly used in the hospital.[2] Each year their use is linked to 250,000 cases of CLABSIs in the United States.[3] Some CLABSIs may be prevented by the prompt removal of the line.[4] However, CVCs are often retained after their clinical indication has lapsed and are then referred to as idle lines.[5, 6] In this work, we propose and theoretically test a guideline to facilitate the safe removal of an idle line by observing the agreement and disagreement between actual practice and the proposed guideline.

METHODS

Setting

This work was conducted at a large, urban, tertiary care, academic health center in the United States as a collaborative effort to improve quality at our institution.[7]

Design and Patients

The reports linked with the electronic medical records at our institution include a daily, ward‐by‐ward listing of patients who have access other than a peripheral line in place. This central line dashboard accesses the information on intravenous access charted by bedside nurses to create a list of patients on every ward who have any kind of central access. Temporary central venous lines (CVLs), peripherally inserted central catheters (PICCs), ports, and dialysis catheters are all included. The unit charge nurses and managers use this dashboard to facilitate compliance with line care bundles. We used this source to identify patients with either type of CVC (CVLs or PICCs) on 8 days in August 2014, September 2014, and October 2014. Patients were included if they had a CVC and were on a general medical or surgical ward bed on audit day. CVLs at all sites were included (femoral, subclavian, and internal jugular). Patients in an intensive care unit (ICU) or progressive care unit on the day of the audit were excluded. Patients whose catheters were for chemotherapy and those admitted for a transplant or receiving palliative or hospice care were also excluded.

Data Collection

A protocol for data collection was written out, and a training session was held to review definitions, data sources, and methods to ensure consistency. Two authors (M.M. and J.D.) assisted by an experienced clinical nurse specialist collected data on the patients captured on audit days. Each chart was reviewed on the day of the audit, the 2 days preceding the audit day, and then followed until the patient was either discharged from the hospital or transferred to a higher level of care, died, or transitioned to palliative or hospice care. Demographics, details about the line, and the criteria for justified use were extracted from the electronic medical record.

Definitions

Justified and Idle Days

To justify the presence of a CVC on any given day, we used criteria that fell under 3 categories: intravenous (IV) access needs, unstable vitals, or meeting sepsis/systemic inflammatory response syndrome (SIRS) criteria (Table 1). For vital signs, a single abnormal reading was counted as fulfilling criteria for that day. If no criterion for justified use was met, the line was considered idle for that day.

Criteria to Justify the Presence of a Central Line
  • NOTE: If none of these criteria were met, the line was considered idle for that day. Abbreviations: IV, intravenous; TPN, total parenteral nutrition; SIRS, systemic inflammatory response syndrome; WBC, white blood count.

IV access needs
Expected duration of IV antibiotics >6 days
Administration of TPN
Anticipated requirement of home IV medications
Requirement of IV medications with documented difficult access
Hemorrhage requiring blood transfusions
Requiring more than 3 infusions
Requiring more than 2 infusions and blood transfusions
Abnormal vitals
Diastolic blood pressure >120 mm Hg
Systolic blood pressure <90 mm Hg
Systolic blood pressure >200 mm Hg
Heart rate >120 beats per minute
Heart rate <50 beats per minute
Respiratory rate >30 breaths per minute
Respiratory rate <10 breaths per minute
Oxygen saturation <90% as measured by pulse oximetry
Meeting SIRS criteria (2 or more of the following present)
Temp >38C, Temp <36C, heart rate >90 beats per minute, respiratory rate >20 breaths per minute, WBC >12,000/mm3, WBC <1,000/mm3, bandemia >10%

Qualifying IV access needs were defined similarly to those previously used,[5, 6] whereas those for SIRS followed the current consensus.[8] To determine the number of IV medications or infusions, the medication administration record was reviewed. If 3 or more infusions were found, their compatibility was checked using the same database that nurses use at our institution. Difficult IV access was inferred from the indication for line placement, coupled with the absence of documentation of a peripheral IV. Clinical progress notes were reviewed to extract information on the length of proposed IV antibiotic courses, and discharge instructions were reviewed to verify whether the line was removed prior to discharge or not. The cutoffs for diastolic blood pressure, respiratory rate, and oxygen saturation used to label patients hemodynamically labile are the same as those used by previous authors and also constitute the definition of hypertensive urgency.[5, 9] However, we diverged from the values previously used for tachycardia, bradycardia, and systolic hypotension using heart rates >120 and <50 beats per minute (compared to >130 and <40 beats per minute) and systolics <90 mm Hg (compared to <80 mm Hg) to justify the line.[5] Early warning scores have been used to identify hospitalized ward patients who are at risk for clinical deterioration. Although each score utilizes different thresholds, the risk for clinical deterioration increases as the vitals worsen.[10] Bearing this in mind, the thresholds we elected to use are more clinically conservative and also parallel the nursing call orders currently used at our institution.

Proposed Guideline

We propose the guideline that a CVC may be safely removed the day after the first idle day.

RESULTS

A total of 126 lines were observed in 126 patients. Eighty‐three (65.9%) of the lines were PICCs. The remaining 43 (34.1%) were CVLs. The indications for line placement were distributed between the need for central access, total parenteral nutrition, or antibiotics (Table 2).

Description of the Study Cohort
Description Value
  • NOTE: Abbreviations: CVL, central venous line; IV, intravenous; PICC, peripherally inserted central catheter; SD, standard deviation; TPN, total parenteral nutrition.

Age in yrs mean (SD) 55.7 (18)
Gender, n (%)
Female 66 (52.4)
Male 60 (47.6)
Type of line, n (%)
PICC 83 (65.9)
CVL 43 (34.1)
Indication for line placement, n (%)
Meds requiring central access or TPN 36 (28.6)
Antibiotics 34 (27.0)
Hemodynamic instability 30 (23.8)
Poor access with multiple IV medications 18 (14.3)
Unknown 8 (6.3)
Line removed prior to discharge, n (%)
Yes 76 (60.3)
No 50 (39.7)

Out of the 126 patients, 50 (39.7%) were discharged from the hospital, died, were transferred to a higher level of care, or transitioned to palliative or hospice care with the line in place. In the remaining 76 patients, the audit captured 635 days, out of which a line was in place for 522 (82.2%) days. Of these 522 days, the line's presence was justified by our criteria for 351 (67.2%) days. The most common reason for a line to be justified on any given day was the need for antibiotics followed by the presence of SIRS criteria (Table 3). The remaining 171 (32.7%) days were idle.

Criteria Met for the 351 Justified Line Days
Criteria N %
  • NOTE: Abbreviations: IV, intravenous; SIRS, systemic inflammatory response syndrome; TPN, total parenteral nutrition; hr: heart rate; bp. blood pressure. *Totals exceed 100% because multiple indications may exist.

No. of factors justifying use
1 184 52.4%
2 127 36.2%
>2 40 11.4
Reason for justifying line*
Anticipate home or >6 days of antibiotic use 181 51.6
SIRS criteria 124 35.3
TPN 96 27.4
Hemodynamic instability based on hr and bp 78 22.2
Poor access with need for IV medications 57 16.2
Respiratory rate (<10 or >30/minute) 25 7.1
Active hemorrhage requiring transfusions 12 3.4
>3 infusions 6 1.7

A comparison of the actual removal of the 76 central lines in practice relative to the proposed guideline of removing it the day following the first idle day is displayed in Figure 1. The central line was removed prior to our proposed guideline in 11 (14.5%) patients, and waiting for an idle day in these patients would have added 46 line days. In almost half the patients (n = 36, 47.4%), the line was removed in agreement with the proposed guideline. None of the patients in whom the line was removed prior to or in accordance with our proposed guideline required a line reinsertion. Line removal was delayed in 29 (38.2%) patients when compared to our proposed guideline. In these patients, following the guideline would have created 122 line‐free days. Most (n = 102, 83.6%) of these potential line‐free days were idle. Twenty (16.4%) were justified, of which half (n = 10) were justified by meeting SIRS criteria.

Figure 1
Pictorial demonstration of the comparison between line removal in practice and the proposed guideline of removing it the day following the first idle day. Each bar represents 1 of the 76 patients in whom the line was removed prior to discharge. The diamond represents the actual removal of the line in practice. The bar is red to indicate that the line will remain in place according to our proposed guideline. It turns to green the day following the first idle day indicating that our guideline would recommend line removal.

DISCUSSION

Approximately 1 in every 25 inpatients in the United States has at least 1 HAI on any given day.[11] The case fatality rate from a CLABSI may be as high as 12%, and up to 70% of these infections may be preventable.[1, 12] Interventions successful in decreasing CLABSIs have focused on patients in ICUs.[13] However, CVCs are increasingly prevalent outside the ICU, with over 4.5 million line days in non‐ICU beds reported to the National Healthcare Safety Network in 2012 compared to 2.5 million in 2010.[2, 14] However, adherence rates to infection control practices may be lower on the wards than in the ICUs.[6, 15] Consequently, although the number of CLABSIs has declined over the last decade, most are now occurring outside the ICU.[16] These trends underscore the need to develop strategies aimed at CLABSI prevention on the floors.

Analogous to the life cycle of a urinary catheter described by Meddings et al.,[17] strategies to prevent CLABSIs and other CVC‐related complications may be designed around the life cycle of a CVC. The life cycle starts with insertion and moves on to the maintenance, removal, and possible reinsertion of the line. The process thus starts with the decision to place the line. Over the last decade, this decision making has changed in part due to PICCs. This shift is reflected in PICC prevalence rates: in 2001, 11% of audited central lines were PICCs compared to 56% in 2007.[5, 6] In our audit, 66% of the CVCs were PICCs. This increase in the use of PICCs may be attributable to the ease and safety of their placement coupled with the increased availability of vascular access placement teams.[18] The risk of overuse that may result from such expediency may be countered by adhering to guidelines such as the Michigan Appropriateness Guide for Intravenous Catheters, which provides both clinically detailed guidance and an impetus for reflective decision making around intravenous access.[19]

The placement of CVCs for prolonged parenteral antibiotics may be a particular subset that bears further exploration. Similar to previous reports, we found that a large number of the CVCs were both inserted for and justified by the need for IV antibiotics.[5] Guidelines delineated by the Infectious Diseases Society of America regarding outpatient parenteral antibiotics weigh both the duration of therapy and the antimicrobial's potential for causing phlebitis when recommending the type of intravascular access.[20] Many courses may therefore be completed through peripheral or midline catheters. Developing strong partnerships between infectious disease specialists, hospitalists, and the facilities or home‐care services treating these patients may curtail the use of CVCs for antimicrobial administration.

The main focus of our work is on facilitating the safe removal of CVCs. The risk of CLABSIs increases each day a CVC is in place, and guidelines to prevent CLABSIs include recommendations to promptly remove nonessential catheters.[4, 21] There is also an emerging understanding that the risk of a PICC‐related CLABSI approaches that from a traditional central line in hospitalized patients, and PICCs confer an increased risk of venous thromboembolism.[18, 22] Although nearly half of surveyed hospitalists recently reported leaving PICCs in place until discharge day, our data suggest that this practice may be driven by the trajectory of a patient's recovery as much as by knowledge gaps related to the use of PICCs.[23] In nearly half the instances, clinical practice already mirrors our proposed guideline, with line removal coinciding with both the timing proposed by our guideline and discharge day. However, there is room for improvement, as line removal may have been expedited in the 29 patients in whom the line was retained after the first idle day. Maintaining an awareness of its presence and weighing its risks and benefits daily may facilitate the removal of a CVC. Based on the recent findings that up to a quarter of clinicians are unaware that their patients have a central line, the mere reminder of the presence of a line using such criteria may expedite its removal by triggering a purposeful reassessment of its ongoing need.[24] Premature CVC removal requiring line reinsertion is an unintended consequence that may emerge from the earlier removal of lines. In our sample, none of the patients who had lines removed either prior to or in accordance with our proposed guideline required a line reinsertion. In addition to line reinsertion, delays in laboratory testing and reporting due to the unavailability of access, increased patient discomfort, or increased workload on the bedside nurse or vascular access team must also be considered when implementing strategies aimed at decreasing line days.

We envisage using these criteria to both empower practitioners with knowledge and foster shared accountability between all team members by using a uniform tool. This can occur through partnerships between infection control, clinical nurse specialists, bedside nursing, and physicians. The electronic medical record could be leveraged to scan the record for the criteria and create a notification when the line becomes idle. In alignment with the Michigan Appropriateness Guide for Intravenous Catheters guidelines, we do not support the removal of lines by nursing staff without physician notification.[19] Such principles have been successfully harnessed in strategies to prevent both catheter‐associated urinary tract infections and CLABSIs in ICUs.[13, 25] In light of the complexity surrounding the decision making for CVCs, our criteria were focused on the wards and erred on the side of clinical caution. This clinical conservatism is apparent in the patients in whom lines were removed prior to what our guideline would propose, yet none of the patients required a line reinsertion. As concerns about recrudescent clinical instability may drive decision making around line removal, such conservatism may be warranted initially. However, the fidelity of these criteria in the clinical setting will need prospective validation. In particular, the inclusion of SIRS criteria may have led to an overestimation of justified days. Further studies may be needed to refine the criteria and find a clinical hierarchy that balances the risks and benefits of retaining a central line.

Our work has certain limitations. It is a single center's experience, and our findings may not therefore be generalizable. Except for when the indication for the line was for difficult access, we did not attempt to verify the presence of a peripheral IV. This, in combination with the inclusion of SIRS criteria, likely leads to an underestimation of idle days. In the interest of focusing on patients in whom the decision making around a line would be the least controversial, we did not continue to follow patients who were transferred to a higher level of care. It is possible, however, that these transfers were precipitated by line‐associated complications such as sepsis and would be important to track. We did not measure the agreement between data collectors, although definitions and methodologies were standardized and reviewed prior to data collection. As this was an observational assessment of a proposed guideline, we cannot predict how the recommendations generated by it will be received by clinicians. Although this may prove to be a barrier in adoption, we hope that the conversation it initiates leads to change.

Hospitalists are positioned to potentially influence the entire life cycle of a central line on the floor. Strategies can be enacted at each stage to help decrease the potential of harm from these devices to our patients. Creating and testing criteria and guidelines such as we propose represents just 1 such strategy in a multidisciplinary effort to provide the best possible care we can.

Acknowledgements

The authors thank Jennifer Dunscomb, Kristen Kelly, and their teams, and Deanna Sidwell, Todd Biggerstaff, Joan Miller, Rob Clark, and the tireless providers at Indiana University Health Methodist Hospital for their support.

Disclosures: This work was supported by the Indiana University Health Values Grant for research. The authors have no conflicts of interests to report.

Infections acquired in the hospital are termed healthcare‐associated infections (HAIs) and include central lineassociated blood stream infections (CLABSIs). Among HAIs, CLABSIs cause the highest number of preventable deaths.[1] Central venous catheters (CVCs) or central lines are commonly used in the hospital.[2] Each year their use is linked to 250,000 cases of CLABSIs in the United States.[3] Some CLABSIs may be prevented by the prompt removal of the line.[4] However, CVCs are often retained after their clinical indication has lapsed and are then referred to as idle lines.[5, 6] In this work, we propose and theoretically test a guideline to facilitate the safe removal of an idle line by observing the agreement and disagreement between actual practice and the proposed guideline.

METHODS

Setting

This work was conducted at a large, urban, tertiary care, academic health center in the United States as a collaborative effort to improve quality at our institution.[7]

Design and Patients

The reports linked with the electronic medical records at our institution include a daily, ward‐by‐ward listing of patients who have access other than a peripheral line in place. This central line dashboard accesses the information on intravenous access charted by bedside nurses to create a list of patients on every ward who have any kind of central access. Temporary central venous lines (CVLs), peripherally inserted central catheters (PICCs), ports, and dialysis catheters are all included. The unit charge nurses and managers use this dashboard to facilitate compliance with line care bundles. We used this source to identify patients with either type of CVC (CVLs or PICCs) on 8 days in August 2014, September 2014, and October 2014. Patients were included if they had a CVC and were on a general medical or surgical ward bed on audit day. CVLs at all sites were included (femoral, subclavian, and internal jugular). Patients in an intensive care unit (ICU) or progressive care unit on the day of the audit were excluded. Patients whose catheters were for chemotherapy and those admitted for a transplant or receiving palliative or hospice care were also excluded.

Data Collection

A protocol for data collection was written out, and a training session was held to review definitions, data sources, and methods to ensure consistency. Two authors (M.M. and J.D.) assisted by an experienced clinical nurse specialist collected data on the patients captured on audit days. Each chart was reviewed on the day of the audit, the 2 days preceding the audit day, and then followed until the patient was either discharged from the hospital or transferred to a higher level of care, died, or transitioned to palliative or hospice care. Demographics, details about the line, and the criteria for justified use were extracted from the electronic medical record.

Definitions

Justified and Idle Days

To justify the presence of a CVC on any given day, we used criteria that fell under 3 categories: intravenous (IV) access needs, unstable vitals, or meeting sepsis/systemic inflammatory response syndrome (SIRS) criteria (Table 1). For vital signs, a single abnormal reading was counted as fulfilling criteria for that day. If no criterion for justified use was met, the line was considered idle for that day.

Criteria to Justify the Presence of a Central Line
  • NOTE: If none of these criteria were met, the line was considered idle for that day. Abbreviations: IV, intravenous; TPN, total parenteral nutrition; SIRS, systemic inflammatory response syndrome; WBC, white blood count.

IV access needs
Expected duration of IV antibiotics >6 days
Administration of TPN
Anticipated requirement of home IV medications
Requirement of IV medications with documented difficult access
Hemorrhage requiring blood transfusions
Requiring more than 3 infusions
Requiring more than 2 infusions and blood transfusions
Abnormal vitals
Diastolic blood pressure >120 mm Hg
Systolic blood pressure <90 mm Hg
Systolic blood pressure >200 mm Hg
Heart rate >120 beats per minute
Heart rate <50 beats per minute
Respiratory rate >30 breaths per minute
Respiratory rate <10 breaths per minute
Oxygen saturation <90% as measured by pulse oximetry
Meeting SIRS criteria (2 or more of the following present)
Temp >38C, Temp <36C, heart rate >90 beats per minute, respiratory rate >20 breaths per minute, WBC >12,000/mm3, WBC <1,000/mm3, bandemia >10%

Qualifying IV access needs were defined similarly to those previously used,[5, 6] whereas those for SIRS followed the current consensus.[8] To determine the number of IV medications or infusions, the medication administration record was reviewed. If 3 or more infusions were found, their compatibility was checked using the same database that nurses use at our institution. Difficult IV access was inferred from the indication for line placement, coupled with the absence of documentation of a peripheral IV. Clinical progress notes were reviewed to extract information on the length of proposed IV antibiotic courses, and discharge instructions were reviewed to verify whether the line was removed prior to discharge or not. The cutoffs for diastolic blood pressure, respiratory rate, and oxygen saturation used to label patients hemodynamically labile are the same as those used by previous authors and also constitute the definition of hypertensive urgency.[5, 9] However, we diverged from the values previously used for tachycardia, bradycardia, and systolic hypotension using heart rates >120 and <50 beats per minute (compared to >130 and <40 beats per minute) and systolics <90 mm Hg (compared to <80 mm Hg) to justify the line.[5] Early warning scores have been used to identify hospitalized ward patients who are at risk for clinical deterioration. Although each score utilizes different thresholds, the risk for clinical deterioration increases as the vitals worsen.[10] Bearing this in mind, the thresholds we elected to use are more clinically conservative and also parallel the nursing call orders currently used at our institution.

Proposed Guideline

We propose the guideline that a CVC may be safely removed the day after the first idle day.

RESULTS

A total of 126 lines were observed in 126 patients. Eighty‐three (65.9%) of the lines were PICCs. The remaining 43 (34.1%) were CVLs. The indications for line placement were distributed between the need for central access, total parenteral nutrition, or antibiotics (Table 2).

Description of the Study Cohort
Description Value
  • NOTE: Abbreviations: CVL, central venous line; IV, intravenous; PICC, peripherally inserted central catheter; SD, standard deviation; TPN, total parenteral nutrition.

Age in yrs mean (SD) 55.7 (18)
Gender, n (%)
Female 66 (52.4)
Male 60 (47.6)
Type of line, n (%)
PICC 83 (65.9)
CVL 43 (34.1)
Indication for line placement, n (%)
Meds requiring central access or TPN 36 (28.6)
Antibiotics 34 (27.0)
Hemodynamic instability 30 (23.8)
Poor access with multiple IV medications 18 (14.3)
Unknown 8 (6.3)
Line removed prior to discharge, n (%)
Yes 76 (60.3)
No 50 (39.7)

Out of the 126 patients, 50 (39.7%) were discharged from the hospital, died, were transferred to a higher level of care, or transitioned to palliative or hospice care with the line in place. In the remaining 76 patients, the audit captured 635 days, out of which a line was in place for 522 (82.2%) days. Of these 522 days, the line's presence was justified by our criteria for 351 (67.2%) days. The most common reason for a line to be justified on any given day was the need for antibiotics followed by the presence of SIRS criteria (Table 3). The remaining 171 (32.7%) days were idle.

Criteria Met for the 351 Justified Line Days
Criteria N %
  • NOTE: Abbreviations: IV, intravenous; SIRS, systemic inflammatory response syndrome; TPN, total parenteral nutrition; hr: heart rate; bp. blood pressure. *Totals exceed 100% because multiple indications may exist.

No. of factors justifying use
1 184 52.4%
2 127 36.2%
>2 40 11.4
Reason for justifying line*
Anticipate home or >6 days of antibiotic use 181 51.6
SIRS criteria 124 35.3
TPN 96 27.4
Hemodynamic instability based on hr and bp 78 22.2
Poor access with need for IV medications 57 16.2
Respiratory rate (<10 or >30/minute) 25 7.1
Active hemorrhage requiring transfusions 12 3.4
>3 infusions 6 1.7

A comparison of the actual removal of the 76 central lines in practice relative to the proposed guideline of removing it the day following the first idle day is displayed in Figure 1. The central line was removed prior to our proposed guideline in 11 (14.5%) patients, and waiting for an idle day in these patients would have added 46 line days. In almost half the patients (n = 36, 47.4%), the line was removed in agreement with the proposed guideline. None of the patients in whom the line was removed prior to or in accordance with our proposed guideline required a line reinsertion. Line removal was delayed in 29 (38.2%) patients when compared to our proposed guideline. In these patients, following the guideline would have created 122 line‐free days. Most (n = 102, 83.6%) of these potential line‐free days were idle. Twenty (16.4%) were justified, of which half (n = 10) were justified by meeting SIRS criteria.

Figure 1
Pictorial demonstration of the comparison between line removal in practice and the proposed guideline of removing it the day following the first idle day. Each bar represents 1 of the 76 patients in whom the line was removed prior to discharge. The diamond represents the actual removal of the line in practice. The bar is red to indicate that the line will remain in place according to our proposed guideline. It turns to green the day following the first idle day indicating that our guideline would recommend line removal.

DISCUSSION

Approximately 1 in every 25 inpatients in the United States has at least 1 HAI on any given day.[11] The case fatality rate from a CLABSI may be as high as 12%, and up to 70% of these infections may be preventable.[1, 12] Interventions successful in decreasing CLABSIs have focused on patients in ICUs.[13] However, CVCs are increasingly prevalent outside the ICU, with over 4.5 million line days in non‐ICU beds reported to the National Healthcare Safety Network in 2012 compared to 2.5 million in 2010.[2, 14] However, adherence rates to infection control practices may be lower on the wards than in the ICUs.[6, 15] Consequently, although the number of CLABSIs has declined over the last decade, most are now occurring outside the ICU.[16] These trends underscore the need to develop strategies aimed at CLABSI prevention on the floors.

Analogous to the life cycle of a urinary catheter described by Meddings et al.,[17] strategies to prevent CLABSIs and other CVC‐related complications may be designed around the life cycle of a CVC. The life cycle starts with insertion and moves on to the maintenance, removal, and possible reinsertion of the line. The process thus starts with the decision to place the line. Over the last decade, this decision making has changed in part due to PICCs. This shift is reflected in PICC prevalence rates: in 2001, 11% of audited central lines were PICCs compared to 56% in 2007.[5, 6] In our audit, 66% of the CVCs were PICCs. This increase in the use of PICCs may be attributable to the ease and safety of their placement coupled with the increased availability of vascular access placement teams.[18] The risk of overuse that may result from such expediency may be countered by adhering to guidelines such as the Michigan Appropriateness Guide for Intravenous Catheters, which provides both clinically detailed guidance and an impetus for reflective decision making around intravenous access.[19]

The placement of CVCs for prolonged parenteral antibiotics may be a particular subset that bears further exploration. Similar to previous reports, we found that a large number of the CVCs were both inserted for and justified by the need for IV antibiotics.[5] Guidelines delineated by the Infectious Diseases Society of America regarding outpatient parenteral antibiotics weigh both the duration of therapy and the antimicrobial's potential for causing phlebitis when recommending the type of intravascular access.[20] Many courses may therefore be completed through peripheral or midline catheters. Developing strong partnerships between infectious disease specialists, hospitalists, and the facilities or home‐care services treating these patients may curtail the use of CVCs for antimicrobial administration.

The main focus of our work is on facilitating the safe removal of CVCs. The risk of CLABSIs increases each day a CVC is in place, and guidelines to prevent CLABSIs include recommendations to promptly remove nonessential catheters.[4, 21] There is also an emerging understanding that the risk of a PICC‐related CLABSI approaches that from a traditional central line in hospitalized patients, and PICCs confer an increased risk of venous thromboembolism.[18, 22] Although nearly half of surveyed hospitalists recently reported leaving PICCs in place until discharge day, our data suggest that this practice may be driven by the trajectory of a patient's recovery as much as by knowledge gaps related to the use of PICCs.[23] In nearly half the instances, clinical practice already mirrors our proposed guideline, with line removal coinciding with both the timing proposed by our guideline and discharge day. However, there is room for improvement, as line removal may have been expedited in the 29 patients in whom the line was retained after the first idle day. Maintaining an awareness of its presence and weighing its risks and benefits daily may facilitate the removal of a CVC. Based on the recent findings that up to a quarter of clinicians are unaware that their patients have a central line, the mere reminder of the presence of a line using such criteria may expedite its removal by triggering a purposeful reassessment of its ongoing need.[24] Premature CVC removal requiring line reinsertion is an unintended consequence that may emerge from the earlier removal of lines. In our sample, none of the patients who had lines removed either prior to or in accordance with our proposed guideline required a line reinsertion. In addition to line reinsertion, delays in laboratory testing and reporting due to the unavailability of access, increased patient discomfort, or increased workload on the bedside nurse or vascular access team must also be considered when implementing strategies aimed at decreasing line days.

We envisage using these criteria to both empower practitioners with knowledge and foster shared accountability between all team members by using a uniform tool. This can occur through partnerships between infection control, clinical nurse specialists, bedside nursing, and physicians. The electronic medical record could be leveraged to scan the record for the criteria and create a notification when the line becomes idle. In alignment with the Michigan Appropriateness Guide for Intravenous Catheters guidelines, we do not support the removal of lines by nursing staff without physician notification.[19] Such principles have been successfully harnessed in strategies to prevent both catheter‐associated urinary tract infections and CLABSIs in ICUs.[13, 25] In light of the complexity surrounding the decision making for CVCs, our criteria were focused on the wards and erred on the side of clinical caution. This clinical conservatism is apparent in the patients in whom lines were removed prior to what our guideline would propose, yet none of the patients required a line reinsertion. As concerns about recrudescent clinical instability may drive decision making around line removal, such conservatism may be warranted initially. However, the fidelity of these criteria in the clinical setting will need prospective validation. In particular, the inclusion of SIRS criteria may have led to an overestimation of justified days. Further studies may be needed to refine the criteria and find a clinical hierarchy that balances the risks and benefits of retaining a central line.

Our work has certain limitations. It is a single center's experience, and our findings may not therefore be generalizable. Except for when the indication for the line was for difficult access, we did not attempt to verify the presence of a peripheral IV. This, in combination with the inclusion of SIRS criteria, likely leads to an underestimation of idle days. In the interest of focusing on patients in whom the decision making around a line would be the least controversial, we did not continue to follow patients who were transferred to a higher level of care. It is possible, however, that these transfers were precipitated by line‐associated complications such as sepsis and would be important to track. We did not measure the agreement between data collectors, although definitions and methodologies were standardized and reviewed prior to data collection. As this was an observational assessment of a proposed guideline, we cannot predict how the recommendations generated by it will be received by clinicians. Although this may prove to be a barrier in adoption, we hope that the conversation it initiates leads to change.

Hospitalists are positioned to potentially influence the entire life cycle of a central line on the floor. Strategies can be enacted at each stage to help decrease the potential of harm from these devices to our patients. Creating and testing criteria and guidelines such as we propose represents just 1 such strategy in a multidisciplinary effort to provide the best possible care we can.

Acknowledgements

The authors thank Jennifer Dunscomb, Kristen Kelly, and their teams, and Deanna Sidwell, Todd Biggerstaff, Joan Miller, Rob Clark, and the tireless providers at Indiana University Health Methodist Hospital for their support.

Disclosures: This work was supported by the Indiana University Health Values Grant for research. The authors have no conflicts of interests to report.

References
  1. Umscheid CA, Mitchell MD, Doshi JA, Agarwal R, Williams K, Brennan PJ. Estimating the proportion of healthcare‐associated infections that are reasonably preventable and the related mortality and costs. Infect Control Hosp Epidemiol. 2011;32(2):101114.
  2. Dudeck MA, Weiner LM, Allen‐Bridson K, et al. National Healthcare Safety Network (NHSN) report, data summary for 2012, device‐associated module. Am J Infect Control. 2013;41(12):11481166.
  3. Maki DG, Kluger DM, Crinch CJ. The risk of bloodstream infection in adults with different intravascular devices: a systematic review of 200 published prospective studies. Mayo Clin Proc. 2006;81(9):11591171.
  4. O'Grady NP, Alexander M, Burns LA, et al. Guidelines for the prevention of intravascular catheter‐related infections. Clin Infect Dis. 2011;52(9):e162e193.
  5. Chernetsky Tejedor S, Tong D, Stein J, et al. Temporary central venous catheter utilization patterns in a large tertiary care center: tracking the “idle central venous catheter.” Infect Control Hosp Epidemiol. 2012;33(1):5057.
  6. Trick WE, Vernon M, Welbel SF, Wisniewski MF, Jernigan JA, Weinstein RA. Unnecessary use of central venous catheters: the need to look outside the intensive care unit. Infect Control Hosp Epidemiol. 2004;25(3):266268.
  7. IU Health Methodist Hospital website. Available at: http://iuhealth.org/methodist/aboIut. Accessed October 20, 2014.
  8. Bone RC, Balk RA, Cerra FB, et al. Definitions for Sepsis and Organ Failure and Guidelines for the Use of Innovative Therapies in Sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest. 2009;136(5 suppl):e28.
  9. Pak KJ, Hu T, Fee C, Wang R, Smith M, Bazzano LA. Acute hypertension: a systematic review and appraisal of guidelines. Ochsner J. 2014;14(4):655663.
  10. Churpek MM, Yuen TC, Edelson DP. Predicting clinical deterioration in the hospital: the impact of outcome selection. Resuscitation. 2013;84(5):564568.
  11. Magill SS, Edwards JR, Bamberg W, et al. Multistate point‐prevalence survey of health care–associated infections. N Engl J Med. 2014;370(13):11981208.
  12. Klevens RM, Edwards JR, Richards CL, et al. Estimating health care‐associated infections and deaths in U.S. hospitals, 2002. Public Health Rep. 2007;122(2):160166.
  13. Pronovost P, Needham D, Berenholtz S, et al. An intervention to decrease catheter‐related bloodstream infections in the ICU. N Engl J Med. 2006;355(26):27252732.
  14. Dudeck MA, Horan TC, Peterson KD, et al. Data summary for 2011, device‐associated module. Centers for Disease Control and Prevention. National Healthcare Safety Network (NHSN) Report. Available at: http://www.cdc.gov/nhsn/PDFs/dataStat/NHSN‐Report‐2011‐Data‐Summary.pdf. Published April 1, 2013. Last accessed January 2015.
  15. Burdeu G, Currey J, Pilcher D. Idle central venous catheter‐days pose infection risk for patients after discharge from intensive care. Am J Infect Control. 2014;42(4):453455.
  16. Liang SY, Marschall J. Update on emerging infections: news from the Centers for Disease Control and Prevention. Vital signs: central line‐associated blood stream infections—United States, 2001, 2008, and 2009. Ann Emerg Med. 2011;58(5):447451.
  17. Meddings J, Rogers MAM, Krein SL, Fakih MG, Olmsted RN, Saint S. Reducing unnecessary urinary catheter use and other strategies to prevent catheter‐associated urinary tract infection: an integrative review. BMJ Qual Saf. 2014;23(4):277289.
  18. Chopra V, O'Horo JC, Rogers MAM, Maki DG, Safdar N. The risk of bloodstream infection associated with peripherally inserted central catheters compared with central venous catheters in adults: a systematic review and meta‐analysis. Infect Control Hosp Epidemiol. 2013;34(9):908918.
  19. Chopra V, Flanders SA, Saint S, et al. The Michigan Appropriateness Guide for Intravenous Catheters (MAGIC): results from a multispecialty panel using the RAND/UCLA Appropriateness Method. Ann Intern Med. 2015;163(6 suppl):S1S40.
  20. Tice AD, Rehm SJ, Dalovisio JR, et al. Practice guidelines for outpatient parenteral antimicrobial therapy. IDSA guidelines. Clin Infect Dis. 2004;38(12):16511672.
  21. McLaws M‐L, Berry G. Nonuniform risk of bloodstream infection with increasing central venous catheter‐days. Infect Control Hosp Epidemiol. 2005;26(8):715719.
  22. Chopra V, Anand S, Hickner A, et al. Risk of venous thromboembolism associated with peripherally inserted central catheters: a systematic review and meta‐analysis. Lancet. 2013;382(9889):311325.
  23. Chopra V, Kuhn L, Flanders SA, Saint S, Krein SL. Hospitalist experiences, practice, opinions, and knowledge regarding peripherally inserted central catheters: results of a national survey. J Hosp Med. 2013;8(11):635638.
  24. Chopra V, Govindan S, Kuhn L, et al. Do clinicians know which of their patients have central venous catheters? Ann Intern Med. 2014;161(8):562.
  25. Reilly L, Sullivan P, Ninni S, Fochesto D, Williams K, Fetherman B. Reducing foley catheter device days in an intensive care unit: using the evidence to change practice. AACN Adv Crit Care. 2006;17(3):272283.
References
  1. Umscheid CA, Mitchell MD, Doshi JA, Agarwal R, Williams K, Brennan PJ. Estimating the proportion of healthcare‐associated infections that are reasonably preventable and the related mortality and costs. Infect Control Hosp Epidemiol. 2011;32(2):101114.
  2. Dudeck MA, Weiner LM, Allen‐Bridson K, et al. National Healthcare Safety Network (NHSN) report, data summary for 2012, device‐associated module. Am J Infect Control. 2013;41(12):11481166.
  3. Maki DG, Kluger DM, Crinch CJ. The risk of bloodstream infection in adults with different intravascular devices: a systematic review of 200 published prospective studies. Mayo Clin Proc. 2006;81(9):11591171.
  4. O'Grady NP, Alexander M, Burns LA, et al. Guidelines for the prevention of intravascular catheter‐related infections. Clin Infect Dis. 2011;52(9):e162e193.
  5. Chernetsky Tejedor S, Tong D, Stein J, et al. Temporary central venous catheter utilization patterns in a large tertiary care center: tracking the “idle central venous catheter.” Infect Control Hosp Epidemiol. 2012;33(1):5057.
  6. Trick WE, Vernon M, Welbel SF, Wisniewski MF, Jernigan JA, Weinstein RA. Unnecessary use of central venous catheters: the need to look outside the intensive care unit. Infect Control Hosp Epidemiol. 2004;25(3):266268.
  7. IU Health Methodist Hospital website. Available at: http://iuhealth.org/methodist/aboIut. Accessed October 20, 2014.
  8. Bone RC, Balk RA, Cerra FB, et al. Definitions for Sepsis and Organ Failure and Guidelines for the Use of Innovative Therapies in Sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest. 2009;136(5 suppl):e28.
  9. Pak KJ, Hu T, Fee C, Wang R, Smith M, Bazzano LA. Acute hypertension: a systematic review and appraisal of guidelines. Ochsner J. 2014;14(4):655663.
  10. Churpek MM, Yuen TC, Edelson DP. Predicting clinical deterioration in the hospital: the impact of outcome selection. Resuscitation. 2013;84(5):564568.
  11. Magill SS, Edwards JR, Bamberg W, et al. Multistate point‐prevalence survey of health care–associated infections. N Engl J Med. 2014;370(13):11981208.
  12. Klevens RM, Edwards JR, Richards CL, et al. Estimating health care‐associated infections and deaths in U.S. hospitals, 2002. Public Health Rep. 2007;122(2):160166.
  13. Pronovost P, Needham D, Berenholtz S, et al. An intervention to decrease catheter‐related bloodstream infections in the ICU. N Engl J Med. 2006;355(26):27252732.
  14. Dudeck MA, Horan TC, Peterson KD, et al. Data summary for 2011, device‐associated module. Centers for Disease Control and Prevention. National Healthcare Safety Network (NHSN) Report. Available at: http://www.cdc.gov/nhsn/PDFs/dataStat/NHSN‐Report‐2011‐Data‐Summary.pdf. Published April 1, 2013. Last accessed January 2015.
  15. Burdeu G, Currey J, Pilcher D. Idle central venous catheter‐days pose infection risk for patients after discharge from intensive care. Am J Infect Control. 2014;42(4):453455.
  16. Liang SY, Marschall J. Update on emerging infections: news from the Centers for Disease Control and Prevention. Vital signs: central line‐associated blood stream infections—United States, 2001, 2008, and 2009. Ann Emerg Med. 2011;58(5):447451.
  17. Meddings J, Rogers MAM, Krein SL, Fakih MG, Olmsted RN, Saint S. Reducing unnecessary urinary catheter use and other strategies to prevent catheter‐associated urinary tract infection: an integrative review. BMJ Qual Saf. 2014;23(4):277289.
  18. Chopra V, O'Horo JC, Rogers MAM, Maki DG, Safdar N. The risk of bloodstream infection associated with peripherally inserted central catheters compared with central venous catheters in adults: a systematic review and meta‐analysis. Infect Control Hosp Epidemiol. 2013;34(9):908918.
  19. Chopra V, Flanders SA, Saint S, et al. The Michigan Appropriateness Guide for Intravenous Catheters (MAGIC): results from a multispecialty panel using the RAND/UCLA Appropriateness Method. Ann Intern Med. 2015;163(6 suppl):S1S40.
  20. Tice AD, Rehm SJ, Dalovisio JR, et al. Practice guidelines for outpatient parenteral antimicrobial therapy. IDSA guidelines. Clin Infect Dis. 2004;38(12):16511672.
  21. McLaws M‐L, Berry G. Nonuniform risk of bloodstream infection with increasing central venous catheter‐days. Infect Control Hosp Epidemiol. 2005;26(8):715719.
  22. Chopra V, Anand S, Hickner A, et al. Risk of venous thromboembolism associated with peripherally inserted central catheters: a systematic review and meta‐analysis. Lancet. 2013;382(9889):311325.
  23. Chopra V, Kuhn L, Flanders SA, Saint S, Krein SL. Hospitalist experiences, practice, opinions, and knowledge regarding peripherally inserted central catheters: results of a national survey. J Hosp Med. 2013;8(11):635638.
  24. Chopra V, Govindan S, Kuhn L, et al. Do clinicians know which of their patients have central venous catheters? Ann Intern Med. 2014;161(8):562.
  25. Reilly L, Sullivan P, Ninni S, Fochesto D, Williams K, Fetherman B. Reducing foley catheter device days in an intensive care unit: using the evidence to change practice. AACN Adv Crit Care. 2006;17(3):272283.
Issue
Journal of Hospital Medicine - 11(7)
Issue
Journal of Hospital Medicine - 11(7)
Page Number
489-493
Page Number
489-493
Article Type
Display Headline
Can the identification of an idle line facilitate its removal? A comparison between a proposed guideline and clinical practice
Display Headline
Can the identification of an idle line facilitate its removal? A comparison between a proposed guideline and clinical practice
Sections
Article Source
© 2016 Society of Hospital Medicine
Disallow All Ads
Correspondence Location
Address for correspondence and reprint requests: Areeba Kara, MD, Inpatient Medicine, Indiana University Health Physicians, Indiana University School of Medicine, Noyes Pavilion Suite 640, 1701 N Senate Avenue, Indianapolis, IN 46202‐1239; Telephone: 317‐962‐2894; Fax number 317‐963‐5285; E‐mail: akara@iuhealth.org
Content Gating
Gated (full article locked unless allowed per User)
Gating Strategy
First Peek Free
Article PDF Media
Media Files

Discordance Between Patient and Provider

Article Type
Changed
Mon, 01/02/2017 - 19:34
Display Headline
How often are hospitalized patients and providers on the same page with regard to the patient's primary recovery goal for hospitalization?

Patient‐centered care has been recognized by the Institute of Medicine as an essential aim of the US healthcare system.[1] A fundamental component of patient‐centered care is to engage patients and caregivers in establishing preferences, needs, values, and overall goals regarding their care.[1] Prior studies have shown that delivering high‐quality patient‐centered care is associated with improved health outcomes, and in some cases, reduced costs.[2, 3, 4, 5, 6, 7] Payors, including the Centers for Medicare and Medicaid Services under the Hospital Value‐Based Purchasing program, are increasingly tying payments to measures of patient experience.[8, 9] As more emphasis is placed on public reporting of these patient‐reported outcomes, healthcare organizations are investing in efforts to engage patients and caregivers, including efforts at establishing patients' preferences for care.[10]

In the acute care setting, a prerequisite for high‐quality patient‐centered care is identifying a patient's primary goal for recovery and then delivering care consistent with that goal.[11, 12, 13] Haberle et al. previously validated patients' most common goals for recovery in the hospital setting into 7 broad categories: (1) be cured, (2) live longer, (3) improve or maintain health, (4) be comfortable, (5) accomplish a particular life goal, (6) provide support for a family member, or (7) other.[13] When providers' understanding of these recovery goals are not concordant with the patient's stated goals, patients may receive care inconsistent with their preferences; it is not uncommon for patients to receive aggressive curative treatments (eg, cardiopulmonary resuscitation) when they have expressed otherwise.[14] On the other hand, when patient goals and priorities are clearly established, patients may have better outcomes.[15] For example, earlier conversations about patient goals and priorities in serious illness can lead to realistic expectations of treatment, enhanced goal‐concordant care, improved quality of life, higher patient satisfaction, more and earlier hospice care, fewer hospitalizations, better patient and family coping, reduced burden of decision making for families, and improved bereavement outcomes.[16, 17, 18]

Although previous studies have suggested poor patient‐physician concordance with regard to the patient's plan of care,[19, 20, 21, 22, 23, 24] there are limited data regarding providers' understanding of the patient's primary recovery goal during hospitalization. The purpose of this study was to identify the patients' Haberle goal, and then determine the degree of concordance among patients and key hospital providers regarding this goal.

METHODS

Study Setting

The Partners Human Research Committee approved the study. The study was conducted on an oncology and medical intensive care unit (MICU) at a major academic medical center in Boston, Massachusetts. The oncology unit was comprised of 2 non‐localized medical teams caring for patients admitted to that unit. The MICU was comprised of a single localized medical team. Medical teams working on these units consisted of a first responder (eg, intern or a physician assistant [PA]), medical residents, and an attending physician. Both units had dedicated nursing staff.

Study Participants

All adult patients (>17 years of age) admitted to the oncology and MICU units during the study period (November 2013 through May 2014) were eligible. These units were chosen because these patients are typically complex and have multiple medical comorbidities longer lengths of stay, and many procedures and tests. In addition, a standard method for asking patients to identify a primary recovery goal for hospitalization aligned well with ongoing institutional efforts to engage these patients in goals of care discussions.

Research assistants identified all patients admitted to each study unit for at least 48 hours and approached them in a random order with a daily target of 2 to 3 patients. Only patients who demonstrated capacity (determined by medical team), or had a legally designated healthcare proxy (who spoke English and was available to participate on their behalf) were included. Research assistants then approached the patient's nurse and a physician provider (defined for this study as housestaff physician, PA, or attending) from the primary medical team to participate in the interview (within 24 hours of patient's interview). We excluded eligible patients who did not have capacity or an available caregiver or declined to participate.

Data Collection Instrument and Interviews

Research assistants administered a validated questionnaire developed by Haberle et al. to participants after 48 hours into the patient's admission to provide time to establish mutual understanding of the diagnosis and prognosis.[13] We asked patients (or the designated healthcare proxy) to select their single, most important Haberle goal (see above). Specifically, as in the original validation study,[13] patients or proxies were asked the following question: Please tell me your most important goal of care for this hospitalization. If they did not understand this question, we asked a follow‐up question: What are you expecting will be accomplished during this hospitalization? Within 24 hours of the patient/proxy interview, we independently asked the patient's nurse and physician to select what they thought was the patient's most important goal for recovery using the same questionnaire, adapted for providers. In each case, all participants were blinded to the responses of others.

Measures

We measured the frequency that each participant (patient/proxy, nurse, and physician) selected a specific Haberle recovery goal across all patients. We measured the rate of pairwise concordance by recovery goal for each participant dyad (patient/proxy‐nurse, patient/proxy‐physician, and nurse‐physician). Finally, we calculated the frequency of cases for which all 3 participants selected the same recovery goal.

Statistical Analyses

Descriptive statistics were used to report patient demographic data. The frequencies of selected responses were calculated and reported as percentages for each type of participant. The differences in rate of responses for each Haberle goal were compared across each participant group using 2 analysis. We then performed 2‐way Kappa statistical tests to measure inter‐rater agreement for each dyad.

RESULTS

Of 1436 patients (882 oncology, 554 MICU) hospitalized during the study period, 341(156 oncology, 185 MICU) were admitted for <48 hours. Of 914 potentially eligible patients (617 oncology, 297 MICU), 191 (112 oncology and 79 MICU) were approached to participate based on our sampling strategy; of these, 8 (2 oncology and 6 MICU) did not have capacity (and no proxy was available) and 2 (1 oncology and 1 MICU) declined. Of the remaining 181 patients (109 oncology and 72 MICU), we obtained a completed questionnaire from all 3 interviewees on 109 (60.2% response rate).

Of the 109 study patients, 52 (47.7%) and 57 (52.3%) were admitted to the oncology and medical intensive care units, respectively (Table 1). Patients were predominantly middle aged, Caucasian, English‐speaking, and college‐educated. Healthcare proxies were frequently interviewed on behalf of patients in the MICU. Housestaff physicians were more often interviewed in the MICU, and PAs were interviewed only on oncology units. Compared to patient responders, nonresponders tended to be male and were admitted to oncology units (see Supporting Table 1 in the online version of this article).

Patient Characteristics
Characteristics All Patients Admitted to Medical Intensive Care Units Admitted to Oncology Units
  • NOTE: Abbreviations: SD, standard deviation. *Patients were interviewed as part of a unique patient admission.

Total, no. (%) 109 (100%) 57 (52.3%) 52 (47.7%)
Gender, no. (%)
Male 55 (50.5%) 28 (49.1%) 26 (50.0%)
Female 54 (49.5%) 29 (50.9%) 26 (50.0%)
Age, y, mean SD 59.4 14 59.7 15 59.1 13
Median 61 61 60
Range 2188 2188 2285
Race, no. (%)
White 103 (94.5%) 53 (93.0%) 50 (96.2%)
Other 6 (5.5%) 4 (7.0%) 2 (3.8%)
Language, no. (%)
English 106 (97.2%) 56 (98.1%) 50 (96.2%)
Other 3 (2.8%) 1 (1.9%) 2 (3.8%)
Education level, no. (%)
Less than high school 30 (27.5%) 17 (29.8%) 13 (25.0%)
High school diploma 27 (24.5%) 18 (31.6%) 9 (17.3%)
Some college or beyond 52 (47.7%) 22 (38.6%) 30 (57.7%)
Patient or caregiver interviewed, no. (%)
Patient 68 (62.4%) 27 (47.4%) 48 (92.3%)
Caregiver 41 (37.6%) 30 (52.6%) 4 (7.7%)
Nurse interviewed, no. (unique) 109 (75) 57 (42) 52 (33)
Physician provider interviewed, no. (%); no. unique
Attending 27 (24.8%); 20 15 (26.3%); 10 12 (23.1%); 10
Housestaff 48 (44.0%); 39 42 (73.7%); 33 6 (11.5%); 6
Physician assistant 34 (31.2%); 25 0 (0%); 0 34 (65.4%); 25

The frequencies of selected Haberle recovery goals by participant type across all patients are listed in Table 2. Patients (or proxies) most often selected be cured (46.8%). Assigned nurses and physicians more commonly selected improve or maintain health (38.5% and 46.8%, respectively). Be comfortable was selected by nurses and physicians more frequently than by patients (16.5%, 16.5%, and 8.3%, respectively). The rate of responses for each Haberle goal was significantly different across all respondent groups (P < 0.0001). The frequencies of selected Haberle goals were not significantly different between patients or proxies (P = 0.67), or for patients admitted to the MICU compared to oncology units (P = 0.64).

Primary Recovery Goal Reported by Patient, Physician Provider, and Nurse
Haberle Recovery Goal Patient/Caregiver, no. (%), n = 109 Physician Provider, no. (%), n = 109* Nurse, no. (%), n = 109
  • NOTE: *Physician provider is defined as either a housestaff physician, physician assistant, or attending physician for the purposes of this study.

Be cured 51 (46.8%) 20 (18.3%) 20 (18.3%)
Be comfortable 9 (8.3%) 18 (16.5%) 18 (16.5%)
Improve or maintain health 32 (29.4%) 42 (38.5%) 51 (46.8%)
Live longer 14 (12.8%) 21 (19.3%) 12 (11%)
Accomplish personal goal 2 (1.8%) 0 (0%) 3 (2.8%)
Provide support for family 1 (0.9%) 1 (0.9%) 1 (0.9%)
Other 0 (0%) 7 (6.4%) 4 (3.7%)

Inter‐rater agreement was poor to slight for the 3 participant dyads (kappa 0.09 [0.03‐0.19], 0.19 [0.08‐0.30], and 0.20 [0.08‐0.32] for patient‐physician, patient‐nurse, and nurse‐physician, respectively). The 3 participants selected the identical recovery goal in 22 (20.2%) cases, and each selected a distinct recovery goal in 32 (29.4%) cases. Pairwise concordance between nurses and physicians was 39.4%. There were no significant differences in agreement between patients admitted to the MICU compared to oncology units (P = 0.09).

DISCUSSION

We observed poor to slight concordance among patients and key hospital providers with regard to identifying the patient's primary recovery goal during acute hospitalization. The majority of patients (or proxies), chose be cured, whereas the majority of hospital providers chose improve or maintain health. Patients were twice as likely to select be cured and half as likely to choose be comfortable compared to nurses or physicians. Strikingly, the patient (or proxy), nurse, and physician identified the same recovery goal in just 20% of cases. These findings were similar for patients admitted to either the MICU or oncology units or when healthcare proxies participated on behalf of the patient (eg, when incapacitated in the MICU).

There are many reasons why hospital providers may not correctly identify the patients' primary recovery goals. First, we do not routinely ask patients to identify recovery goals upon admission in a structured and standardized manner. In fact, clinicians often do not elicit patients' needs, concerns, and expectations regarding their care in general.[25] Second, even when recovery goals are elicited at admission, they may not be communicated effectively to all members of the care team. This could be due to geographically non‐localized teams (although we did not observe a statistically significant difference between regionalized MICU and nonregionalized oncology care units), frequent provider‐to‐provider handoffs, and siloed electronic communication (eg, email, alphanumeric pages) regarding goals of care that inevitably leaves out key providers.[26] Third, healthcare proxies who are involved in decision making on the patient's behalf may not always be available to meet with the care team in person; consequently, their input may not be considered in a timely manner or reliably communicated to all members of the care team. We observed a large discrepancy in how often patients chose be cured compared to their hospital providers. This could be explained by clinicians' unwillingness to disclose bad news or divulge accurate prognostic information that causes patients to feel depressed or lose hope, particularly for those patients with the worst prognoses.[16, 27, 28] Patients may lack sophisticated knowledge of their conditions for a variety of reasons, including low health literacy, at times choosing to hope for the best even when it is not realistic. Additionally, there may be more subtle differences in what patients and hospital providers consider the primary recovery goal in context of the main reason for hospitalization and underlying medical illness. For example, a patient with metastatic lung cancer hospitalized with recurrent postobstructive pneumonia may choose be cured as his/her primary recovery goal (thinking of the pneumonia), whereas physicians may choose improve/maintain health or comfort (thinking of the cancer). We also cannot exclude the possibility that sometimes when patients state be cured and clinicians state improve health as the primary goal, that they are really saying the same thing in different ways. However, these are 2 different constructs (cure may not be possible for many patients) that may deserve an explicit discussion for patients to have realistic expectations for their health following hospitalization.

In short, our results underscore the importance of having an open and honest dialog with patients and caregivers throughout hospitalization, and the need to provide education about the potential futility of excessive care in situations where appropriate. Simply following patients' goals without discussing their feasibility and the consequences of aggressive treatments may result in unnecessary morbidity and misuse of healthcare resources. Once goals are clearly established, communicated, and refined in hospitalized patients with serious illness, there is much reason to believe that ongoing conversation will favorably impact outcomes.[29]

We found few studies that rigorously quantified the rate of concordance of hospital recovery goals among patients and key hospital providers; however, studies that measured overall plan of care agreement have demonstrated suboptimal concordance.[20, 30, 31] Shin et al. found significant underestimation of cancer patients' needs and poor concordance between patients and oncologists in assessing perceived needs of supportive care.[20] It is also notable that nurses and physicians had low levels of concordance in our study. O'Leary and colleagues found that nurses and physicians did not reliably communicate and often did not agree on the plan of care for hospitalized patients.[30] Although geographic regionalization of care teams and multidisciplinary rounds can improve the likelihood that key members of the care team are on the same page with regard to the plan of care, there is still much room for improvement.[26, 32, 33, 34] For example, although nurses and physicians in our study independently selected individual recovery goals with similar frequencies (Table 2), we observed suboptimal concordance between nurses and providers (36.8%) for specific patients, including on our regionalized care unit (MICU). This may be due to the reasons described above.

There are several implications of these findings. As payors continue to shift payments toward value‐based metrics, largely determined by patient experience and adequate advance care planning,[9] our findings suggest that more effort should be focused on delivering care consistent with patients' primary recovery goals. As a first step, healthcare organizations can focus on efforts to systematically identify and communicate recovery goals to all members of the care team, ensuring that patients' preferences, needs, and values are captured. In addition, as innovation in patient engagement and care delivery using Web‐based and mobile technology continues to grow,[35] using these tools to capture key goals for hospitalization and recovery can play an essential role. For example, as electronic health record vendors and institutions start to implement patient portals in the acute care setting, they should consider how to configure these tools to capture key goals for hospitalization and recovery, and then communicate them to the care team; preliminary work in this area is promising.[10]

Our study has several limitations to generalizability. First, the study was conducted on 2 services (MICU and oncology) at a single institution using a sampling strategy where research assistants enrolled 2 to 3 patients per day. Although the sampling was random, the availability of patients and proxies to be interviewed may have led to selection bias. Second, the sample size was small. Third, the patients who participated were predominantly white, English‐speaking, and well educated, possibly a consequence of our sampling strategy. However, this fact makes our findings more striking; although cultural and language barriers were generally not present in our study population, large discrepancies in goal concordance still existed. Fourth, in instances when patients were unable to participate themselves, we interviewed their healthcare proxy; therefore, it is possible that the proxies' responses did not reflect those of the patient. However, we note that concordance rates did not significantly differ between the 2 services despite the fact that the proportion of proxy interviews was much higher in the MICU. Similarly, we cannot exclude the possibility that patients altered their stated goals in the presence of proxies, but patients were given the option to be interviewed alone. Patients may also have misunderstood the timing of the goals (during this hospitalization as opposed to long term), although research assistants made every effort to clarify this during the interviews. Finally, our data‐collection instrument was previously validated in hospitalized general medicine patients and not oncology or MICU patients, and it has not been used to directly ask clinicians to identify patients' recovery goals. However, there is no reason to suspect that it could not be used for this purpose in critical care as well as noncritical care settings, as the survey was developed by a multidisciplinary team that included medical professionals and was validated by clinicians who successfully identified a single, very broad goal (eg, be cured) in each case.

CONCLUSION

We report poor to slight concordance among hospitalized patients and key hospital providers with regard to the main recovery goal. Future studies should assess whether patient satisfaction and experience is adversely impacted by patient‐provider discordance regarding key recovery goals. Additionally, institutions may consider future efforts to elicit and communicate patients' primary recovery goals more effectively to all members of the care team, and address discrepancies as soon as they are discovered.

Disclosures

This work was supported by a grant from the Gordon and Betty Moore Foundation (GBMF) (grant GBMF3914). GBMF had no role in the design or conduct of the study; collection, analysis, or interpretation of data; or preparation or review of the manuscript. The authors report no conflicts of interest.

Files
References
  1. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy of Sciences; 2001.
  2. Bartlett EE, Grayson M, Barker R, Levine DM, Golden A, Libber S. The effects of physician communications skills on patient satisfaction; recall, and adherence. J Chronic Dis. 1984;37(9–10):755764.
  3. Little P, Everitt H, Williamson I, et al. Observational study of effect of patient centredness and positive approach on outcomes of general practice consultations. BMJ. 2001;323(7318):908911.
  4. Clancy CM. Reengineering hospital discharge: a protocol to improve patient safety, reduce costs, and boost patient satisfaction. Am J Med Qual. 2009;24(4):344346.
  5. Boulding W, Glickman SW, Manary MP, Schulman KA, Staelin R. Relationship between patient satisfaction with inpatient care and hospital readmission within 30 days. Am J Manag Care. 2011;17(1):4148.
  6. Jha AK, Orav EJ, Zheng J, Epstein AM. Patients' perception of hospital care in the United States. N Engl J Med. 2008;359(18):19211931.
  7. Veroff D, Marr A, Wennberg DE. Enhanced support for shared decision making reduced costs of care for patients with preference‐sensitive conditions. Health Aff (Millwood). 2013;32(2):285293.
  8. Centers for Medicare and Medicaid Services. Medicare program; hospital inpatient value‐based purchasing program. Final rule. Fed Regist. 2011;76(88):2649026547.
  9. Centers for Medicare and Medicaid Services. CMS begins implementation of key payment legislation. Available at: https://www.cms.gov/Newsroom/MediaReleaseDatabase/Press‐releases/2015‐Press‐releases‐items/2015‐07‐08.html. Published July 8, 2015.
  10. Dalal AK, Dykes PC, Collins S, et al. A web‐based, patient‐centered toolkit to engage patients and caregivers in the acute care setting: a preliminary evaluation [published online August 2, 2015]. J Am Med Informatics Assoc. doi: 10.1093/jamia/ocv093.
  11. Daly BJ, Douglas SL, O'Toole E, et al. Effectiveness trial of an intensive communication structure for families of long‐stay ICU patients. Chest. 2010;138(6):13401348.
  12. Brandt DS, Shinkunas LA, Gehlbach TG, Kaldjian LC. Understanding goals of care statements and preferences among patients and their surrogates in the medical ICU. J Hosp Palliat Nurs. 2012;14(2):126132.
  13. Haberle TH, Shinkunas LA, Erekson ZD, Kaldjian LC. Goals of care among hospitalized patients: a validation study. Am J Hosp Palliat Care. 2011;28(5):335341.
  14. Goodlin SJ, Zhong Z, Lynn J, et al. Factors associated with use of cardiopulmonary resuscitation in seriously ill hospitalized adults. JAMA. 1999;282(24):23332339.
  15. Mack JW, Weeks JC, Wright AA, Block SD, Prigerson HG. End‐of‐life discussions, goal attainment, and distress at the end of life: Predictors and outcomes of receipt of care consistent with preferences. J Clin Oncol. 2010;28(7):12031208.
  16. Mack JW, Smith TJ. Reasons why physicians do not have discussions about poor prognosis, why it matters, and what can be improved. J Clin Oncol. 2012;30(22):27152717.
  17. Wright AA, Zhang B, Ray A, et al. Associations between end‐of‐life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment. JAMA. 2008;300(14):16651673.
  18. Chiarchiaro J, Buddadhumaruk P, Arnold RM, White DB. Prior advance care planning is associated with less decisional conflict among surrogates for critically ill patients. Ann Am Thorac Soc. 2015;12(10):15281533.
  19. O'Leary KJ, Kulkarni N, Landler MP, et al. Hospitalized patients' understanding of their plan of care. Mayo Clin Proc. 2010;85(1):4752.
  20. Shin DW, Kim SY, Cho J, et al. Discordance in perceived needs between patients and physicians in oncology practice: a nationwide survey in Korea. J Clin Oncol. 2011;29(33):44244429.
  21. Dykes PC, DaDamio RR, Goldsmith D, Kim H, Ohashi K, Saba VK. Leveraging standards to support patient‐centric interdisciplinary plans of care. AMIA Annu Symp Proc. 2011;2011:356363.
  22. Desalvo KB, Muntner P. Discordance between physician and patient self‐rated health and all‐cause mortality. Ochsner J. 2011;11(3):232240.
  23. Yen JC, Abrahamowicz M, Dobkin PL, Clarke AE, Battista RN, Fortin PR. Determinants of discordance between patients and physicians in their assessment of lupus disease activity. J Rheumatol. 2003;30(9):19671976.
  24. Rothe A, Bielitzer M, Meinertz T, Limbourg T, Ladwig KH, Goette A. Predictors of discordance between physicians' and patients' appraisals of health‐related quality of life in atrial fibrillation patients: Findings from the Angiotensin II Antagonist in Paroxysmal Atrial Fibrillation Trial. Am Heart J. 2013;166(3):589596.
  25. Rozenblum R, Lisby M, Hockey PM, et al. Uncovering the blind spot of patient satisfaction: an international survey. BMJ Qual Saf. 2011;20(11):959965.
  26. O'Leary KJ, Haviley C, Slade ME, Shah HM, Lee J, Williams M. Improving teamwork: impact of structured interdisciplinary rounds on a hospitalist unit. J Hosp Med. 2011;6(2):8893.
  27. Lee SJ, Fairclough D, Antin JH, Weeks JC. Discrepancies between patient and physician estimates for the success of stem cell transplantation. JAMA. 2001;285(8):10341038.
  28. Lee SJ, Loberiza FR, Rizzo JD, Soiffer RJ, Antin JH, Weeks JC. Optimistic expectations and survival after hematopoietic stem cell transplantation. Biol Blood Marrow Transplant. 2003;9(6):389396.
  29. Bernacki R, Hutchings M, Vick J, et al. Development of the Serious Illness Care Program: a randomised controlled trial of a palliative care communication intervention. BMJ Open. 2015;5(10):e009032.
  30. O'Leary KJ, Thompson JA, Landler MP, et al. Patterns of nurse‐physician communication and agreement on the plan of care. Qual Saf Health Care. 2010;19(3):195199.
  31. O'Leary KJ, Wayne DB, Landler MP, et al. Impact of localizing physicians to hospital units on nurse‐physician communication and agreement on the plan of care. J Gen Intern Med. 2009;24(11):12231227.
  32. Starmer AJ, Spector ND, Srivastava R, et al. Changes in medical errors after implementation of a handoff program. N Engl J Med. 2014;371(19):18031812.
  33. Stein J, Payne C, Methvin A, et al. Reorganizing a hospital ward as an accountable care unit. J Hosp Med. 2015;10(1):3640.
  34. O'Leary KJ, Sehgal NL, Terrell G, Williams M. Interdisciplinary teamwork in hospitals: a review and practical recommendations for improvement. J Hosp Med. 2012;7(1):4854.
  35. Sama PR, Eapen ZJ, Weinfurt KP, Shah BR, Schulman KA. An evaluation of mobile health application tools. JMIR mHealth uHealth. 2014;2(2):e19.
Article PDF
Issue
Journal of Hospital Medicine - 11(9)
Page Number
615-619
Sections
Files
Files
Article PDF
Article PDF

Patient‐centered care has been recognized by the Institute of Medicine as an essential aim of the US healthcare system.[1] A fundamental component of patient‐centered care is to engage patients and caregivers in establishing preferences, needs, values, and overall goals regarding their care.[1] Prior studies have shown that delivering high‐quality patient‐centered care is associated with improved health outcomes, and in some cases, reduced costs.[2, 3, 4, 5, 6, 7] Payors, including the Centers for Medicare and Medicaid Services under the Hospital Value‐Based Purchasing program, are increasingly tying payments to measures of patient experience.[8, 9] As more emphasis is placed on public reporting of these patient‐reported outcomes, healthcare organizations are investing in efforts to engage patients and caregivers, including efforts at establishing patients' preferences for care.[10]

In the acute care setting, a prerequisite for high‐quality patient‐centered care is identifying a patient's primary goal for recovery and then delivering care consistent with that goal.[11, 12, 13] Haberle et al. previously validated patients' most common goals for recovery in the hospital setting into 7 broad categories: (1) be cured, (2) live longer, (3) improve or maintain health, (4) be comfortable, (5) accomplish a particular life goal, (6) provide support for a family member, or (7) other.[13] When providers' understanding of these recovery goals are not concordant with the patient's stated goals, patients may receive care inconsistent with their preferences; it is not uncommon for patients to receive aggressive curative treatments (eg, cardiopulmonary resuscitation) when they have expressed otherwise.[14] On the other hand, when patient goals and priorities are clearly established, patients may have better outcomes.[15] For example, earlier conversations about patient goals and priorities in serious illness can lead to realistic expectations of treatment, enhanced goal‐concordant care, improved quality of life, higher patient satisfaction, more and earlier hospice care, fewer hospitalizations, better patient and family coping, reduced burden of decision making for families, and improved bereavement outcomes.[16, 17, 18]

Although previous studies have suggested poor patient‐physician concordance with regard to the patient's plan of care,[19, 20, 21, 22, 23, 24] there are limited data regarding providers' understanding of the patient's primary recovery goal during hospitalization. The purpose of this study was to identify the patients' Haberle goal, and then determine the degree of concordance among patients and key hospital providers regarding this goal.

METHODS

Study Setting

The Partners Human Research Committee approved the study. The study was conducted on an oncology and medical intensive care unit (MICU) at a major academic medical center in Boston, Massachusetts. The oncology unit was comprised of 2 non‐localized medical teams caring for patients admitted to that unit. The MICU was comprised of a single localized medical team. Medical teams working on these units consisted of a first responder (eg, intern or a physician assistant [PA]), medical residents, and an attending physician. Both units had dedicated nursing staff.

Study Participants

All adult patients (>17 years of age) admitted to the oncology and MICU units during the study period (November 2013 through May 2014) were eligible. These units were chosen because these patients are typically complex and have multiple medical comorbidities longer lengths of stay, and many procedures and tests. In addition, a standard method for asking patients to identify a primary recovery goal for hospitalization aligned well with ongoing institutional efforts to engage these patients in goals of care discussions.

Research assistants identified all patients admitted to each study unit for at least 48 hours and approached them in a random order with a daily target of 2 to 3 patients. Only patients who demonstrated capacity (determined by medical team), or had a legally designated healthcare proxy (who spoke English and was available to participate on their behalf) were included. Research assistants then approached the patient's nurse and a physician provider (defined for this study as housestaff physician, PA, or attending) from the primary medical team to participate in the interview (within 24 hours of patient's interview). We excluded eligible patients who did not have capacity or an available caregiver or declined to participate.

Data Collection Instrument and Interviews

Research assistants administered a validated questionnaire developed by Haberle et al. to participants after 48 hours into the patient's admission to provide time to establish mutual understanding of the diagnosis and prognosis.[13] We asked patients (or the designated healthcare proxy) to select their single, most important Haberle goal (see above). Specifically, as in the original validation study,[13] patients or proxies were asked the following question: Please tell me your most important goal of care for this hospitalization. If they did not understand this question, we asked a follow‐up question: What are you expecting will be accomplished during this hospitalization? Within 24 hours of the patient/proxy interview, we independently asked the patient's nurse and physician to select what they thought was the patient's most important goal for recovery using the same questionnaire, adapted for providers. In each case, all participants were blinded to the responses of others.

Measures

We measured the frequency that each participant (patient/proxy, nurse, and physician) selected a specific Haberle recovery goal across all patients. We measured the rate of pairwise concordance by recovery goal for each participant dyad (patient/proxy‐nurse, patient/proxy‐physician, and nurse‐physician). Finally, we calculated the frequency of cases for which all 3 participants selected the same recovery goal.

Statistical Analyses

Descriptive statistics were used to report patient demographic data. The frequencies of selected responses were calculated and reported as percentages for each type of participant. The differences in rate of responses for each Haberle goal were compared across each participant group using 2 analysis. We then performed 2‐way Kappa statistical tests to measure inter‐rater agreement for each dyad.

RESULTS

Of 1436 patients (882 oncology, 554 MICU) hospitalized during the study period, 341(156 oncology, 185 MICU) were admitted for <48 hours. Of 914 potentially eligible patients (617 oncology, 297 MICU), 191 (112 oncology and 79 MICU) were approached to participate based on our sampling strategy; of these, 8 (2 oncology and 6 MICU) did not have capacity (and no proxy was available) and 2 (1 oncology and 1 MICU) declined. Of the remaining 181 patients (109 oncology and 72 MICU), we obtained a completed questionnaire from all 3 interviewees on 109 (60.2% response rate).

Of the 109 study patients, 52 (47.7%) and 57 (52.3%) were admitted to the oncology and medical intensive care units, respectively (Table 1). Patients were predominantly middle aged, Caucasian, English‐speaking, and college‐educated. Healthcare proxies were frequently interviewed on behalf of patients in the MICU. Housestaff physicians were more often interviewed in the MICU, and PAs were interviewed only on oncology units. Compared to patient responders, nonresponders tended to be male and were admitted to oncology units (see Supporting Table 1 in the online version of this article).

Patient Characteristics
Characteristics All Patients Admitted to Medical Intensive Care Units Admitted to Oncology Units
  • NOTE: Abbreviations: SD, standard deviation. *Patients were interviewed as part of a unique patient admission.

Total, no. (%) 109 (100%) 57 (52.3%) 52 (47.7%)
Gender, no. (%)
Male 55 (50.5%) 28 (49.1%) 26 (50.0%)
Female 54 (49.5%) 29 (50.9%) 26 (50.0%)
Age, y, mean SD 59.4 14 59.7 15 59.1 13
Median 61 61 60
Range 2188 2188 2285
Race, no. (%)
White 103 (94.5%) 53 (93.0%) 50 (96.2%)
Other 6 (5.5%) 4 (7.0%) 2 (3.8%)
Language, no. (%)
English 106 (97.2%) 56 (98.1%) 50 (96.2%)
Other 3 (2.8%) 1 (1.9%) 2 (3.8%)
Education level, no. (%)
Less than high school 30 (27.5%) 17 (29.8%) 13 (25.0%)
High school diploma 27 (24.5%) 18 (31.6%) 9 (17.3%)
Some college or beyond 52 (47.7%) 22 (38.6%) 30 (57.7%)
Patient or caregiver interviewed, no. (%)
Patient 68 (62.4%) 27 (47.4%) 48 (92.3%)
Caregiver 41 (37.6%) 30 (52.6%) 4 (7.7%)
Nurse interviewed, no. (unique) 109 (75) 57 (42) 52 (33)
Physician provider interviewed, no. (%); no. unique
Attending 27 (24.8%); 20 15 (26.3%); 10 12 (23.1%); 10
Housestaff 48 (44.0%); 39 42 (73.7%); 33 6 (11.5%); 6
Physician assistant 34 (31.2%); 25 0 (0%); 0 34 (65.4%); 25

The frequencies of selected Haberle recovery goals by participant type across all patients are listed in Table 2. Patients (or proxies) most often selected be cured (46.8%). Assigned nurses and physicians more commonly selected improve or maintain health (38.5% and 46.8%, respectively). Be comfortable was selected by nurses and physicians more frequently than by patients (16.5%, 16.5%, and 8.3%, respectively). The rate of responses for each Haberle goal was significantly different across all respondent groups (P < 0.0001). The frequencies of selected Haberle goals were not significantly different between patients or proxies (P = 0.67), or for patients admitted to the MICU compared to oncology units (P = 0.64).

Primary Recovery Goal Reported by Patient, Physician Provider, and Nurse
Haberle Recovery Goal Patient/Caregiver, no. (%), n = 109 Physician Provider, no. (%), n = 109* Nurse, no. (%), n = 109
  • NOTE: *Physician provider is defined as either a housestaff physician, physician assistant, or attending physician for the purposes of this study.

Be cured 51 (46.8%) 20 (18.3%) 20 (18.3%)
Be comfortable 9 (8.3%) 18 (16.5%) 18 (16.5%)
Improve or maintain health 32 (29.4%) 42 (38.5%) 51 (46.8%)
Live longer 14 (12.8%) 21 (19.3%) 12 (11%)
Accomplish personal goal 2 (1.8%) 0 (0%) 3 (2.8%)
Provide support for family 1 (0.9%) 1 (0.9%) 1 (0.9%)
Other 0 (0%) 7 (6.4%) 4 (3.7%)

Inter‐rater agreement was poor to slight for the 3 participant dyads (kappa 0.09 [0.03‐0.19], 0.19 [0.08‐0.30], and 0.20 [0.08‐0.32] for patient‐physician, patient‐nurse, and nurse‐physician, respectively). The 3 participants selected the identical recovery goal in 22 (20.2%) cases, and each selected a distinct recovery goal in 32 (29.4%) cases. Pairwise concordance between nurses and physicians was 39.4%. There were no significant differences in agreement between patients admitted to the MICU compared to oncology units (P = 0.09).

DISCUSSION

We observed poor to slight concordance among patients and key hospital providers with regard to identifying the patient's primary recovery goal during acute hospitalization. The majority of patients (or proxies), chose be cured, whereas the majority of hospital providers chose improve or maintain health. Patients were twice as likely to select be cured and half as likely to choose be comfortable compared to nurses or physicians. Strikingly, the patient (or proxy), nurse, and physician identified the same recovery goal in just 20% of cases. These findings were similar for patients admitted to either the MICU or oncology units or when healthcare proxies participated on behalf of the patient (eg, when incapacitated in the MICU).

There are many reasons why hospital providers may not correctly identify the patients' primary recovery goals. First, we do not routinely ask patients to identify recovery goals upon admission in a structured and standardized manner. In fact, clinicians often do not elicit patients' needs, concerns, and expectations regarding their care in general.[25] Second, even when recovery goals are elicited at admission, they may not be communicated effectively to all members of the care team. This could be due to geographically non‐localized teams (although we did not observe a statistically significant difference between regionalized MICU and nonregionalized oncology care units), frequent provider‐to‐provider handoffs, and siloed electronic communication (eg, email, alphanumeric pages) regarding goals of care that inevitably leaves out key providers.[26] Third, healthcare proxies who are involved in decision making on the patient's behalf may not always be available to meet with the care team in person; consequently, their input may not be considered in a timely manner or reliably communicated to all members of the care team. We observed a large discrepancy in how often patients chose be cured compared to their hospital providers. This could be explained by clinicians' unwillingness to disclose bad news or divulge accurate prognostic information that causes patients to feel depressed or lose hope, particularly for those patients with the worst prognoses.[16, 27, 28] Patients may lack sophisticated knowledge of their conditions for a variety of reasons, including low health literacy, at times choosing to hope for the best even when it is not realistic. Additionally, there may be more subtle differences in what patients and hospital providers consider the primary recovery goal in context of the main reason for hospitalization and underlying medical illness. For example, a patient with metastatic lung cancer hospitalized with recurrent postobstructive pneumonia may choose be cured as his/her primary recovery goal (thinking of the pneumonia), whereas physicians may choose improve/maintain health or comfort (thinking of the cancer). We also cannot exclude the possibility that sometimes when patients state be cured and clinicians state improve health as the primary goal, that they are really saying the same thing in different ways. However, these are 2 different constructs (cure may not be possible for many patients) that may deserve an explicit discussion for patients to have realistic expectations for their health following hospitalization.

In short, our results underscore the importance of having an open and honest dialog with patients and caregivers throughout hospitalization, and the need to provide education about the potential futility of excessive care in situations where appropriate. Simply following patients' goals without discussing their feasibility and the consequences of aggressive treatments may result in unnecessary morbidity and misuse of healthcare resources. Once goals are clearly established, communicated, and refined in hospitalized patients with serious illness, there is much reason to believe that ongoing conversation will favorably impact outcomes.[29]

We found few studies that rigorously quantified the rate of concordance of hospital recovery goals among patients and key hospital providers; however, studies that measured overall plan of care agreement have demonstrated suboptimal concordance.[20, 30, 31] Shin et al. found significant underestimation of cancer patients' needs and poor concordance between patients and oncologists in assessing perceived needs of supportive care.[20] It is also notable that nurses and physicians had low levels of concordance in our study. O'Leary and colleagues found that nurses and physicians did not reliably communicate and often did not agree on the plan of care for hospitalized patients.[30] Although geographic regionalization of care teams and multidisciplinary rounds can improve the likelihood that key members of the care team are on the same page with regard to the plan of care, there is still much room for improvement.[26, 32, 33, 34] For example, although nurses and physicians in our study independently selected individual recovery goals with similar frequencies (Table 2), we observed suboptimal concordance between nurses and providers (36.8%) for specific patients, including on our regionalized care unit (MICU). This may be due to the reasons described above.

There are several implications of these findings. As payors continue to shift payments toward value‐based metrics, largely determined by patient experience and adequate advance care planning,[9] our findings suggest that more effort should be focused on delivering care consistent with patients' primary recovery goals. As a first step, healthcare organizations can focus on efforts to systematically identify and communicate recovery goals to all members of the care team, ensuring that patients' preferences, needs, and values are captured. In addition, as innovation in patient engagement and care delivery using Web‐based and mobile technology continues to grow,[35] using these tools to capture key goals for hospitalization and recovery can play an essential role. For example, as electronic health record vendors and institutions start to implement patient portals in the acute care setting, they should consider how to configure these tools to capture key goals for hospitalization and recovery, and then communicate them to the care team; preliminary work in this area is promising.[10]

Our study has several limitations to generalizability. First, the study was conducted on 2 services (MICU and oncology) at a single institution using a sampling strategy where research assistants enrolled 2 to 3 patients per day. Although the sampling was random, the availability of patients and proxies to be interviewed may have led to selection bias. Second, the sample size was small. Third, the patients who participated were predominantly white, English‐speaking, and well educated, possibly a consequence of our sampling strategy. However, this fact makes our findings more striking; although cultural and language barriers were generally not present in our study population, large discrepancies in goal concordance still existed. Fourth, in instances when patients were unable to participate themselves, we interviewed their healthcare proxy; therefore, it is possible that the proxies' responses did not reflect those of the patient. However, we note that concordance rates did not significantly differ between the 2 services despite the fact that the proportion of proxy interviews was much higher in the MICU. Similarly, we cannot exclude the possibility that patients altered their stated goals in the presence of proxies, but patients were given the option to be interviewed alone. Patients may also have misunderstood the timing of the goals (during this hospitalization as opposed to long term), although research assistants made every effort to clarify this during the interviews. Finally, our data‐collection instrument was previously validated in hospitalized general medicine patients and not oncology or MICU patients, and it has not been used to directly ask clinicians to identify patients' recovery goals. However, there is no reason to suspect that it could not be used for this purpose in critical care as well as noncritical care settings, as the survey was developed by a multidisciplinary team that included medical professionals and was validated by clinicians who successfully identified a single, very broad goal (eg, be cured) in each case.

CONCLUSION

We report poor to slight concordance among hospitalized patients and key hospital providers with regard to the main recovery goal. Future studies should assess whether patient satisfaction and experience is adversely impacted by patient‐provider discordance regarding key recovery goals. Additionally, institutions may consider future efforts to elicit and communicate patients' primary recovery goals more effectively to all members of the care team, and address discrepancies as soon as they are discovered.

Disclosures

This work was supported by a grant from the Gordon and Betty Moore Foundation (GBMF) (grant GBMF3914). GBMF had no role in the design or conduct of the study; collection, analysis, or interpretation of data; or preparation or review of the manuscript. The authors report no conflicts of interest.

Patient‐centered care has been recognized by the Institute of Medicine as an essential aim of the US healthcare system.[1] A fundamental component of patient‐centered care is to engage patients and caregivers in establishing preferences, needs, values, and overall goals regarding their care.[1] Prior studies have shown that delivering high‐quality patient‐centered care is associated with improved health outcomes, and in some cases, reduced costs.[2, 3, 4, 5, 6, 7] Payors, including the Centers for Medicare and Medicaid Services under the Hospital Value‐Based Purchasing program, are increasingly tying payments to measures of patient experience.[8, 9] As more emphasis is placed on public reporting of these patient‐reported outcomes, healthcare organizations are investing in efforts to engage patients and caregivers, including efforts at establishing patients' preferences for care.[10]

In the acute care setting, a prerequisite for high‐quality patient‐centered care is identifying a patient's primary goal for recovery and then delivering care consistent with that goal.[11, 12, 13] Haberle et al. previously validated patients' most common goals for recovery in the hospital setting into 7 broad categories: (1) be cured, (2) live longer, (3) improve or maintain health, (4) be comfortable, (5) accomplish a particular life goal, (6) provide support for a family member, or (7) other.[13] When providers' understanding of these recovery goals are not concordant with the patient's stated goals, patients may receive care inconsistent with their preferences; it is not uncommon for patients to receive aggressive curative treatments (eg, cardiopulmonary resuscitation) when they have expressed otherwise.[14] On the other hand, when patient goals and priorities are clearly established, patients may have better outcomes.[15] For example, earlier conversations about patient goals and priorities in serious illness can lead to realistic expectations of treatment, enhanced goal‐concordant care, improved quality of life, higher patient satisfaction, more and earlier hospice care, fewer hospitalizations, better patient and family coping, reduced burden of decision making for families, and improved bereavement outcomes.[16, 17, 18]

Although previous studies have suggested poor patient‐physician concordance with regard to the patient's plan of care,[19, 20, 21, 22, 23, 24] there are limited data regarding providers' understanding of the patient's primary recovery goal during hospitalization. The purpose of this study was to identify the patients' Haberle goal, and then determine the degree of concordance among patients and key hospital providers regarding this goal.

METHODS

Study Setting

The Partners Human Research Committee approved the study. The study was conducted on an oncology and medical intensive care unit (MICU) at a major academic medical center in Boston, Massachusetts. The oncology unit was comprised of 2 non‐localized medical teams caring for patients admitted to that unit. The MICU was comprised of a single localized medical team. Medical teams working on these units consisted of a first responder (eg, intern or a physician assistant [PA]), medical residents, and an attending physician. Both units had dedicated nursing staff.

Study Participants

All adult patients (>17 years of age) admitted to the oncology and MICU units during the study period (November 2013 through May 2014) were eligible. These units were chosen because these patients are typically complex and have multiple medical comorbidities longer lengths of stay, and many procedures and tests. In addition, a standard method for asking patients to identify a primary recovery goal for hospitalization aligned well with ongoing institutional efforts to engage these patients in goals of care discussions.

Research assistants identified all patients admitted to each study unit for at least 48 hours and approached them in a random order with a daily target of 2 to 3 patients. Only patients who demonstrated capacity (determined by medical team), or had a legally designated healthcare proxy (who spoke English and was available to participate on their behalf) were included. Research assistants then approached the patient's nurse and a physician provider (defined for this study as housestaff physician, PA, or attending) from the primary medical team to participate in the interview (within 24 hours of patient's interview). We excluded eligible patients who did not have capacity or an available caregiver or declined to participate.

Data Collection Instrument and Interviews

Research assistants administered a validated questionnaire developed by Haberle et al. to participants after 48 hours into the patient's admission to provide time to establish mutual understanding of the diagnosis and prognosis.[13] We asked patients (or the designated healthcare proxy) to select their single, most important Haberle goal (see above). Specifically, as in the original validation study,[13] patients or proxies were asked the following question: Please tell me your most important goal of care for this hospitalization. If they did not understand this question, we asked a follow‐up question: What are you expecting will be accomplished during this hospitalization? Within 24 hours of the patient/proxy interview, we independently asked the patient's nurse and physician to select what they thought was the patient's most important goal for recovery using the same questionnaire, adapted for providers. In each case, all participants were blinded to the responses of others.

Measures

We measured the frequency that each participant (patient/proxy, nurse, and physician) selected a specific Haberle recovery goal across all patients. We measured the rate of pairwise concordance by recovery goal for each participant dyad (patient/proxy‐nurse, patient/proxy‐physician, and nurse‐physician). Finally, we calculated the frequency of cases for which all 3 participants selected the same recovery goal.

Statistical Analyses

Descriptive statistics were used to report patient demographic data. The frequencies of selected responses were calculated and reported as percentages for each type of participant. The differences in rate of responses for each Haberle goal were compared across each participant group using 2 analysis. We then performed 2‐way Kappa statistical tests to measure inter‐rater agreement for each dyad.

RESULTS

Of 1436 patients (882 oncology, 554 MICU) hospitalized during the study period, 341(156 oncology, 185 MICU) were admitted for <48 hours. Of 914 potentially eligible patients (617 oncology, 297 MICU), 191 (112 oncology and 79 MICU) were approached to participate based on our sampling strategy; of these, 8 (2 oncology and 6 MICU) did not have capacity (and no proxy was available) and 2 (1 oncology and 1 MICU) declined. Of the remaining 181 patients (109 oncology and 72 MICU), we obtained a completed questionnaire from all 3 interviewees on 109 (60.2% response rate).

Of the 109 study patients, 52 (47.7%) and 57 (52.3%) were admitted to the oncology and medical intensive care units, respectively (Table 1). Patients were predominantly middle aged, Caucasian, English‐speaking, and college‐educated. Healthcare proxies were frequently interviewed on behalf of patients in the MICU. Housestaff physicians were more often interviewed in the MICU, and PAs were interviewed only on oncology units. Compared to patient responders, nonresponders tended to be male and were admitted to oncology units (see Supporting Table 1 in the online version of this article).

Patient Characteristics
Characteristics All Patients Admitted to Medical Intensive Care Units Admitted to Oncology Units
  • NOTE: Abbreviations: SD, standard deviation. *Patients were interviewed as part of a unique patient admission.

Total, no. (%) 109 (100%) 57 (52.3%) 52 (47.7%)
Gender, no. (%)
Male 55 (50.5%) 28 (49.1%) 26 (50.0%)
Female 54 (49.5%) 29 (50.9%) 26 (50.0%)
Age, y, mean SD 59.4 14 59.7 15 59.1 13
Median 61 61 60
Range 2188 2188 2285
Race, no. (%)
White 103 (94.5%) 53 (93.0%) 50 (96.2%)
Other 6 (5.5%) 4 (7.0%) 2 (3.8%)
Language, no. (%)
English 106 (97.2%) 56 (98.1%) 50 (96.2%)
Other 3 (2.8%) 1 (1.9%) 2 (3.8%)
Education level, no. (%)
Less than high school 30 (27.5%) 17 (29.8%) 13 (25.0%)
High school diploma 27 (24.5%) 18 (31.6%) 9 (17.3%)
Some college or beyond 52 (47.7%) 22 (38.6%) 30 (57.7%)
Patient or caregiver interviewed, no. (%)
Patient 68 (62.4%) 27 (47.4%) 48 (92.3%)
Caregiver 41 (37.6%) 30 (52.6%) 4 (7.7%)
Nurse interviewed, no. (unique) 109 (75) 57 (42) 52 (33)
Physician provider interviewed, no. (%); no. unique
Attending 27 (24.8%); 20 15 (26.3%); 10 12 (23.1%); 10
Housestaff 48 (44.0%); 39 42 (73.7%); 33 6 (11.5%); 6
Physician assistant 34 (31.2%); 25 0 (0%); 0 34 (65.4%); 25

The frequencies of selected Haberle recovery goals by participant type across all patients are listed in Table 2. Patients (or proxies) most often selected be cured (46.8%). Assigned nurses and physicians more commonly selected improve or maintain health (38.5% and 46.8%, respectively). Be comfortable was selected by nurses and physicians more frequently than by patients (16.5%, 16.5%, and 8.3%, respectively). The rate of responses for each Haberle goal was significantly different across all respondent groups (P < 0.0001). The frequencies of selected Haberle goals were not significantly different between patients or proxies (P = 0.67), or for patients admitted to the MICU compared to oncology units (P = 0.64).

Primary Recovery Goal Reported by Patient, Physician Provider, and Nurse
Haberle Recovery Goal Patient/Caregiver, no. (%), n = 109 Physician Provider, no. (%), n = 109* Nurse, no. (%), n = 109
  • NOTE: *Physician provider is defined as either a housestaff physician, physician assistant, or attending physician for the purposes of this study.

Be cured 51 (46.8%) 20 (18.3%) 20 (18.3%)
Be comfortable 9 (8.3%) 18 (16.5%) 18 (16.5%)
Improve or maintain health 32 (29.4%) 42 (38.5%) 51 (46.8%)
Live longer 14 (12.8%) 21 (19.3%) 12 (11%)
Accomplish personal goal 2 (1.8%) 0 (0%) 3 (2.8%)
Provide support for family 1 (0.9%) 1 (0.9%) 1 (0.9%)
Other 0 (0%) 7 (6.4%) 4 (3.7%)

Inter‐rater agreement was poor to slight for the 3 participant dyads (kappa 0.09 [0.03‐0.19], 0.19 [0.08‐0.30], and 0.20 [0.08‐0.32] for patient‐physician, patient‐nurse, and nurse‐physician, respectively). The 3 participants selected the identical recovery goal in 22 (20.2%) cases, and each selected a distinct recovery goal in 32 (29.4%) cases. Pairwise concordance between nurses and physicians was 39.4%. There were no significant differences in agreement between patients admitted to the MICU compared to oncology units (P = 0.09).

DISCUSSION

We observed poor to slight concordance among patients and key hospital providers with regard to identifying the patient's primary recovery goal during acute hospitalization. The majority of patients (or proxies), chose be cured, whereas the majority of hospital providers chose improve or maintain health. Patients were twice as likely to select be cured and half as likely to choose be comfortable compared to nurses or physicians. Strikingly, the patient (or proxy), nurse, and physician identified the same recovery goal in just 20% of cases. These findings were similar for patients admitted to either the MICU or oncology units or when healthcare proxies participated on behalf of the patient (eg, when incapacitated in the MICU).

There are many reasons why hospital providers may not correctly identify the patients' primary recovery goals. First, we do not routinely ask patients to identify recovery goals upon admission in a structured and standardized manner. In fact, clinicians often do not elicit patients' needs, concerns, and expectations regarding their care in general.[25] Second, even when recovery goals are elicited at admission, they may not be communicated effectively to all members of the care team. This could be due to geographically non‐localized teams (although we did not observe a statistically significant difference between regionalized MICU and nonregionalized oncology care units), frequent provider‐to‐provider handoffs, and siloed electronic communication (eg, email, alphanumeric pages) regarding goals of care that inevitably leaves out key providers.[26] Third, healthcare proxies who are involved in decision making on the patient's behalf may not always be available to meet with the care team in person; consequently, their input may not be considered in a timely manner or reliably communicated to all members of the care team. We observed a large discrepancy in how often patients chose be cured compared to their hospital providers. This could be explained by clinicians' unwillingness to disclose bad news or divulge accurate prognostic information that causes patients to feel depressed or lose hope, particularly for those patients with the worst prognoses.[16, 27, 28] Patients may lack sophisticated knowledge of their conditions for a variety of reasons, including low health literacy, at times choosing to hope for the best even when it is not realistic. Additionally, there may be more subtle differences in what patients and hospital providers consider the primary recovery goal in context of the main reason for hospitalization and underlying medical illness. For example, a patient with metastatic lung cancer hospitalized with recurrent postobstructive pneumonia may choose be cured as his/her primary recovery goal (thinking of the pneumonia), whereas physicians may choose improve/maintain health or comfort (thinking of the cancer). We also cannot exclude the possibility that sometimes when patients state be cured and clinicians state improve health as the primary goal, that they are really saying the same thing in different ways. However, these are 2 different constructs (cure may not be possible for many patients) that may deserve an explicit discussion for patients to have realistic expectations for their health following hospitalization.

In short, our results underscore the importance of having an open and honest dialog with patients and caregivers throughout hospitalization, and the need to provide education about the potential futility of excessive care in situations where appropriate. Simply following patients' goals without discussing their feasibility and the consequences of aggressive treatments may result in unnecessary morbidity and misuse of healthcare resources. Once goals are clearly established, communicated, and refined in hospitalized patients with serious illness, there is much reason to believe that ongoing conversation will favorably impact outcomes.[29]

We found few studies that rigorously quantified the rate of concordance of hospital recovery goals among patients and key hospital providers; however, studies that measured overall plan of care agreement have demonstrated suboptimal concordance.[20, 30, 31] Shin et al. found significant underestimation of cancer patients' needs and poor concordance between patients and oncologists in assessing perceived needs of supportive care.[20] It is also notable that nurses and physicians had low levels of concordance in our study. O'Leary and colleagues found that nurses and physicians did not reliably communicate and often did not agree on the plan of care for hospitalized patients.[30] Although geographic regionalization of care teams and multidisciplinary rounds can improve the likelihood that key members of the care team are on the same page with regard to the plan of care, there is still much room for improvement.[26, 32, 33, 34] For example, although nurses and physicians in our study independently selected individual recovery goals with similar frequencies (Table 2), we observed suboptimal concordance between nurses and providers (36.8%) for specific patients, including on our regionalized care unit (MICU). This may be due to the reasons described above.

There are several implications of these findings. As payors continue to shift payments toward value‐based metrics, largely determined by patient experience and adequate advance care planning,[9] our findings suggest that more effort should be focused on delivering care consistent with patients' primary recovery goals. As a first step, healthcare organizations can focus on efforts to systematically identify and communicate recovery goals to all members of the care team, ensuring that patients' preferences, needs, and values are captured. In addition, as innovation in patient engagement and care delivery using Web‐based and mobile technology continues to grow,[35] using these tools to capture key goals for hospitalization and recovery can play an essential role. For example, as electronic health record vendors and institutions start to implement patient portals in the acute care setting, they should consider how to configure these tools to capture key goals for hospitalization and recovery, and then communicate them to the care team; preliminary work in this area is promising.[10]

Our study has several limitations to generalizability. First, the study was conducted on 2 services (MICU and oncology) at a single institution using a sampling strategy where research assistants enrolled 2 to 3 patients per day. Although the sampling was random, the availability of patients and proxies to be interviewed may have led to selection bias. Second, the sample size was small. Third, the patients who participated were predominantly white, English‐speaking, and well educated, possibly a consequence of our sampling strategy. However, this fact makes our findings more striking; although cultural and language barriers were generally not present in our study population, large discrepancies in goal concordance still existed. Fourth, in instances when patients were unable to participate themselves, we interviewed their healthcare proxy; therefore, it is possible that the proxies' responses did not reflect those of the patient. However, we note that concordance rates did not significantly differ between the 2 services despite the fact that the proportion of proxy interviews was much higher in the MICU. Similarly, we cannot exclude the possibility that patients altered their stated goals in the presence of proxies, but patients were given the option to be interviewed alone. Patients may also have misunderstood the timing of the goals (during this hospitalization as opposed to long term), although research assistants made every effort to clarify this during the interviews. Finally, our data‐collection instrument was previously validated in hospitalized general medicine patients and not oncology or MICU patients, and it has not been used to directly ask clinicians to identify patients' recovery goals. However, there is no reason to suspect that it could not be used for this purpose in critical care as well as noncritical care settings, as the survey was developed by a multidisciplinary team that included medical professionals and was validated by clinicians who successfully identified a single, very broad goal (eg, be cured) in each case.

CONCLUSION

We report poor to slight concordance among hospitalized patients and key hospital providers with regard to the main recovery goal. Future studies should assess whether patient satisfaction and experience is adversely impacted by patient‐provider discordance regarding key recovery goals. Additionally, institutions may consider future efforts to elicit and communicate patients' primary recovery goals more effectively to all members of the care team, and address discrepancies as soon as they are discovered.

Disclosures

This work was supported by a grant from the Gordon and Betty Moore Foundation (GBMF) (grant GBMF3914). GBMF had no role in the design or conduct of the study; collection, analysis, or interpretation of data; or preparation or review of the manuscript. The authors report no conflicts of interest.

References
  1. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy of Sciences; 2001.
  2. Bartlett EE, Grayson M, Barker R, Levine DM, Golden A, Libber S. The effects of physician communications skills on patient satisfaction; recall, and adherence. J Chronic Dis. 1984;37(9–10):755764.
  3. Little P, Everitt H, Williamson I, et al. Observational study of effect of patient centredness and positive approach on outcomes of general practice consultations. BMJ. 2001;323(7318):908911.
  4. Clancy CM. Reengineering hospital discharge: a protocol to improve patient safety, reduce costs, and boost patient satisfaction. Am J Med Qual. 2009;24(4):344346.
  5. Boulding W, Glickman SW, Manary MP, Schulman KA, Staelin R. Relationship between patient satisfaction with inpatient care and hospital readmission within 30 days. Am J Manag Care. 2011;17(1):4148.
  6. Jha AK, Orav EJ, Zheng J, Epstein AM. Patients' perception of hospital care in the United States. N Engl J Med. 2008;359(18):19211931.
  7. Veroff D, Marr A, Wennberg DE. Enhanced support for shared decision making reduced costs of care for patients with preference‐sensitive conditions. Health Aff (Millwood). 2013;32(2):285293.
  8. Centers for Medicare and Medicaid Services. Medicare program; hospital inpatient value‐based purchasing program. Final rule. Fed Regist. 2011;76(88):2649026547.
  9. Centers for Medicare and Medicaid Services. CMS begins implementation of key payment legislation. Available at: https://www.cms.gov/Newsroom/MediaReleaseDatabase/Press‐releases/2015‐Press‐releases‐items/2015‐07‐08.html. Published July 8, 2015.
  10. Dalal AK, Dykes PC, Collins S, et al. A web‐based, patient‐centered toolkit to engage patients and caregivers in the acute care setting: a preliminary evaluation [published online August 2, 2015]. J Am Med Informatics Assoc. doi: 10.1093/jamia/ocv093.
  11. Daly BJ, Douglas SL, O'Toole E, et al. Effectiveness trial of an intensive communication structure for families of long‐stay ICU patients. Chest. 2010;138(6):13401348.
  12. Brandt DS, Shinkunas LA, Gehlbach TG, Kaldjian LC. Understanding goals of care statements and preferences among patients and their surrogates in the medical ICU. J Hosp Palliat Nurs. 2012;14(2):126132.
  13. Haberle TH, Shinkunas LA, Erekson ZD, Kaldjian LC. Goals of care among hospitalized patients: a validation study. Am J Hosp Palliat Care. 2011;28(5):335341.
  14. Goodlin SJ, Zhong Z, Lynn J, et al. Factors associated with use of cardiopulmonary resuscitation in seriously ill hospitalized adults. JAMA. 1999;282(24):23332339.
  15. Mack JW, Weeks JC, Wright AA, Block SD, Prigerson HG. End‐of‐life discussions, goal attainment, and distress at the end of life: Predictors and outcomes of receipt of care consistent with preferences. J Clin Oncol. 2010;28(7):12031208.
  16. Mack JW, Smith TJ. Reasons why physicians do not have discussions about poor prognosis, why it matters, and what can be improved. J Clin Oncol. 2012;30(22):27152717.
  17. Wright AA, Zhang B, Ray A, et al. Associations between end‐of‐life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment. JAMA. 2008;300(14):16651673.
  18. Chiarchiaro J, Buddadhumaruk P, Arnold RM, White DB. Prior advance care planning is associated with less decisional conflict among surrogates for critically ill patients. Ann Am Thorac Soc. 2015;12(10):15281533.
  19. O'Leary KJ, Kulkarni N, Landler MP, et al. Hospitalized patients' understanding of their plan of care. Mayo Clin Proc. 2010;85(1):4752.
  20. Shin DW, Kim SY, Cho J, et al. Discordance in perceived needs between patients and physicians in oncology practice: a nationwide survey in Korea. J Clin Oncol. 2011;29(33):44244429.
  21. Dykes PC, DaDamio RR, Goldsmith D, Kim H, Ohashi K, Saba VK. Leveraging standards to support patient‐centric interdisciplinary plans of care. AMIA Annu Symp Proc. 2011;2011:356363.
  22. Desalvo KB, Muntner P. Discordance between physician and patient self‐rated health and all‐cause mortality. Ochsner J. 2011;11(3):232240.
  23. Yen JC, Abrahamowicz M, Dobkin PL, Clarke AE, Battista RN, Fortin PR. Determinants of discordance between patients and physicians in their assessment of lupus disease activity. J Rheumatol. 2003;30(9):19671976.
  24. Rothe A, Bielitzer M, Meinertz T, Limbourg T, Ladwig KH, Goette A. Predictors of discordance between physicians' and patients' appraisals of health‐related quality of life in atrial fibrillation patients: Findings from the Angiotensin II Antagonist in Paroxysmal Atrial Fibrillation Trial. Am Heart J. 2013;166(3):589596.
  25. Rozenblum R, Lisby M, Hockey PM, et al. Uncovering the blind spot of patient satisfaction: an international survey. BMJ Qual Saf. 2011;20(11):959965.
  26. O'Leary KJ, Haviley C, Slade ME, Shah HM, Lee J, Williams M. Improving teamwork: impact of structured interdisciplinary rounds on a hospitalist unit. J Hosp Med. 2011;6(2):8893.
  27. Lee SJ, Fairclough D, Antin JH, Weeks JC. Discrepancies between patient and physician estimates for the success of stem cell transplantation. JAMA. 2001;285(8):10341038.
  28. Lee SJ, Loberiza FR, Rizzo JD, Soiffer RJ, Antin JH, Weeks JC. Optimistic expectations and survival after hematopoietic stem cell transplantation. Biol Blood Marrow Transplant. 2003;9(6):389396.
  29. Bernacki R, Hutchings M, Vick J, et al. Development of the Serious Illness Care Program: a randomised controlled trial of a palliative care communication intervention. BMJ Open. 2015;5(10):e009032.
  30. O'Leary KJ, Thompson JA, Landler MP, et al. Patterns of nurse‐physician communication and agreement on the plan of care. Qual Saf Health Care. 2010;19(3):195199.
  31. O'Leary KJ, Wayne DB, Landler MP, et al. Impact of localizing physicians to hospital units on nurse‐physician communication and agreement on the plan of care. J Gen Intern Med. 2009;24(11):12231227.
  32. Starmer AJ, Spector ND, Srivastava R, et al. Changes in medical errors after implementation of a handoff program. N Engl J Med. 2014;371(19):18031812.
  33. Stein J, Payne C, Methvin A, et al. Reorganizing a hospital ward as an accountable care unit. J Hosp Med. 2015;10(1):3640.
  34. O'Leary KJ, Sehgal NL, Terrell G, Williams M. Interdisciplinary teamwork in hospitals: a review and practical recommendations for improvement. J Hosp Med. 2012;7(1):4854.
  35. Sama PR, Eapen ZJ, Weinfurt KP, Shah BR, Schulman KA. An evaluation of mobile health application tools. JMIR mHealth uHealth. 2014;2(2):e19.
References
  1. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy of Sciences; 2001.
  2. Bartlett EE, Grayson M, Barker R, Levine DM, Golden A, Libber S. The effects of physician communications skills on patient satisfaction; recall, and adherence. J Chronic Dis. 1984;37(9–10):755764.
  3. Little P, Everitt H, Williamson I, et al. Observational study of effect of patient centredness and positive approach on outcomes of general practice consultations. BMJ. 2001;323(7318):908911.
  4. Clancy CM. Reengineering hospital discharge: a protocol to improve patient safety, reduce costs, and boost patient satisfaction. Am J Med Qual. 2009;24(4):344346.
  5. Boulding W, Glickman SW, Manary MP, Schulman KA, Staelin R. Relationship between patient satisfaction with inpatient care and hospital readmission within 30 days. Am J Manag Care. 2011;17(1):4148.
  6. Jha AK, Orav EJ, Zheng J, Epstein AM. Patients' perception of hospital care in the United States. N Engl J Med. 2008;359(18):19211931.
  7. Veroff D, Marr A, Wennberg DE. Enhanced support for shared decision making reduced costs of care for patients with preference‐sensitive conditions. Health Aff (Millwood). 2013;32(2):285293.
  8. Centers for Medicare and Medicaid Services. Medicare program; hospital inpatient value‐based purchasing program. Final rule. Fed Regist. 2011;76(88):2649026547.
  9. Centers for Medicare and Medicaid Services. CMS begins implementation of key payment legislation. Available at: https://www.cms.gov/Newsroom/MediaReleaseDatabase/Press‐releases/2015‐Press‐releases‐items/2015‐07‐08.html. Published July 8, 2015.
  10. Dalal AK, Dykes PC, Collins S, et al. A web‐based, patient‐centered toolkit to engage patients and caregivers in the acute care setting: a preliminary evaluation [published online August 2, 2015]. J Am Med Informatics Assoc. doi: 10.1093/jamia/ocv093.
  11. Daly BJ, Douglas SL, O'Toole E, et al. Effectiveness trial of an intensive communication structure for families of long‐stay ICU patients. Chest. 2010;138(6):13401348.
  12. Brandt DS, Shinkunas LA, Gehlbach TG, Kaldjian LC. Understanding goals of care statements and preferences among patients and their surrogates in the medical ICU. J Hosp Palliat Nurs. 2012;14(2):126132.
  13. Haberle TH, Shinkunas LA, Erekson ZD, Kaldjian LC. Goals of care among hospitalized patients: a validation study. Am J Hosp Palliat Care. 2011;28(5):335341.
  14. Goodlin SJ, Zhong Z, Lynn J, et al. Factors associated with use of cardiopulmonary resuscitation in seriously ill hospitalized adults. JAMA. 1999;282(24):23332339.
  15. Mack JW, Weeks JC, Wright AA, Block SD, Prigerson HG. End‐of‐life discussions, goal attainment, and distress at the end of life: Predictors and outcomes of receipt of care consistent with preferences. J Clin Oncol. 2010;28(7):12031208.
  16. Mack JW, Smith TJ. Reasons why physicians do not have discussions about poor prognosis, why it matters, and what can be improved. J Clin Oncol. 2012;30(22):27152717.
  17. Wright AA, Zhang B, Ray A, et al. Associations between end‐of‐life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment. JAMA. 2008;300(14):16651673.
  18. Chiarchiaro J, Buddadhumaruk P, Arnold RM, White DB. Prior advance care planning is associated with less decisional conflict among surrogates for critically ill patients. Ann Am Thorac Soc. 2015;12(10):15281533.
  19. O'Leary KJ, Kulkarni N, Landler MP, et al. Hospitalized patients' understanding of their plan of care. Mayo Clin Proc. 2010;85(1):4752.
  20. Shin DW, Kim SY, Cho J, et al. Discordance in perceived needs between patients and physicians in oncology practice: a nationwide survey in Korea. J Clin Oncol. 2011;29(33):44244429.
  21. Dykes PC, DaDamio RR, Goldsmith D, Kim H, Ohashi K, Saba VK. Leveraging standards to support patient‐centric interdisciplinary plans of care. AMIA Annu Symp Proc. 2011;2011:356363.
  22. Desalvo KB, Muntner P. Discordance between physician and patient self‐rated health and all‐cause mortality. Ochsner J. 2011;11(3):232240.
  23. Yen JC, Abrahamowicz M, Dobkin PL, Clarke AE, Battista RN, Fortin PR. Determinants of discordance between patients and physicians in their assessment of lupus disease activity. J Rheumatol. 2003;30(9):19671976.
  24. Rothe A, Bielitzer M, Meinertz T, Limbourg T, Ladwig KH, Goette A. Predictors of discordance between physicians' and patients' appraisals of health‐related quality of life in atrial fibrillation patients: Findings from the Angiotensin II Antagonist in Paroxysmal Atrial Fibrillation Trial. Am Heart J. 2013;166(3):589596.
  25. Rozenblum R, Lisby M, Hockey PM, et al. Uncovering the blind spot of patient satisfaction: an international survey. BMJ Qual Saf. 2011;20(11):959965.
  26. O'Leary KJ, Haviley C, Slade ME, Shah HM, Lee J, Williams M. Improving teamwork: impact of structured interdisciplinary rounds on a hospitalist unit. J Hosp Med. 2011;6(2):8893.
  27. Lee SJ, Fairclough D, Antin JH, Weeks JC. Discrepancies between patient and physician estimates for the success of stem cell transplantation. JAMA. 2001;285(8):10341038.
  28. Lee SJ, Loberiza FR, Rizzo JD, Soiffer RJ, Antin JH, Weeks JC. Optimistic expectations and survival after hematopoietic stem cell transplantation. Biol Blood Marrow Transplant. 2003;9(6):389396.
  29. Bernacki R, Hutchings M, Vick J, et al. Development of the Serious Illness Care Program: a randomised controlled trial of a palliative care communication intervention. BMJ Open. 2015;5(10):e009032.
  30. O'Leary KJ, Thompson JA, Landler MP, et al. Patterns of nurse‐physician communication and agreement on the plan of care. Qual Saf Health Care. 2010;19(3):195199.
  31. O'Leary KJ, Wayne DB, Landler MP, et al. Impact of localizing physicians to hospital units on nurse‐physician communication and agreement on the plan of care. J Gen Intern Med. 2009;24(11):12231227.
  32. Starmer AJ, Spector ND, Srivastava R, et al. Changes in medical errors after implementation of a handoff program. N Engl J Med. 2014;371(19):18031812.
  33. Stein J, Payne C, Methvin A, et al. Reorganizing a hospital ward as an accountable care unit. J Hosp Med. 2015;10(1):3640.
  34. O'Leary KJ, Sehgal NL, Terrell G, Williams M. Interdisciplinary teamwork in hospitals: a review and practical recommendations for improvement. J Hosp Med. 2012;7(1):4854.
  35. Sama PR, Eapen ZJ, Weinfurt KP, Shah BR, Schulman KA. An evaluation of mobile health application tools. JMIR mHealth uHealth. 2014;2(2):e19.
Issue
Journal of Hospital Medicine - 11(9)
Issue
Journal of Hospital Medicine - 11(9)
Page Number
615-619
Page Number
615-619
Article Type
Display Headline
How often are hospitalized patients and providers on the same page with regard to the patient's primary recovery goal for hospitalization?
Display Headline
How often are hospitalized patients and providers on the same page with regard to the patient's primary recovery goal for hospitalization?
Sections
Article Source
© 2016 Society of Hospital Medicine
Disallow All Ads
Correspondence Location
Address for correspondence and reprint requests: Jose F. Figueroa, MD, Instructor of Medicine, Harvard Medical School, Associate Physician, Brigham Fax: 844‐861‐3477; E‐mail: jffigueroa@partners.org
Content Gating
Gated (full article locked unless allowed per User)
Gating Strategy
First Peek Free
Article PDF Media
Media Files

Discharge Preparedness and Readmission

Article Type
Changed
Mon, 01/02/2017 - 19:34
Display Headline
Preparedness for hospital discharge and prediction of readmission

In recent years, US hospitals have focused on decreasing readmission rates, incented by reimbursement penalties to hospitals having excessive readmissions.[1] Gaps in the quality of care provided during transitions likely contribute to preventable readmissions.[2] One compelling quality assessment in this setting is measuring patients' discharge preparedness, using key dimensions such as understanding their instructions for medication use and follow‐up. Patient‐reported preparedness for discharge may also be useful to identify risk of readmission.

Several patient‐reported measures of preparedness for discharge exist, and herein we describe 2 measures of interest. First, the Brief‐PREPARED (B‐PREPARED) measure was derived from the longer PREPARED instrument (Prescriptions, Ready to re‐enter community, Education, Placement, Assurance of safety, Realistic expectations, Empowerment, Directed to appropriate services), which reflects the patient's perceived needs at discharge. In previous research, the B‐PREPARED measure predicted emergency department (ED) visits for patients who had been recently hospitalized and had a high risk for readmission.[3] Second, the Care Transitions Measure‐3 (CTM‐3) was developed by Coleman et al. as a patient‐reported measure to discriminate between patients who were more likely to have an ED visit or readmission from those who did not. CTM‐3 has also been used to evaluate hospitals' level of care coordination and for public reporting purposes.[4, 5, 6] It has been endorsed by the National Quality Forum and incorporated into the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey provided to samples of recently hospitalized US patients.[7] However, recent evidence from an inpatient cohort of cardiovascular patients suggests the CTM‐3 overinflates care transition scores compared to the longer 15‐item CTM. In that cohort, the CTM‐3 could not differentiate between patients who did or did not have repeat ED visits or readmission.[8] Thus far, the B‐PREPARED and CTM‐3 measures have not been compared to one another directly.

In addition to the development of patient‐reported measures, hospitals increasingly employ administrative algorithms to predict likelihood of readmission.[9] A commonly used measure is the LACE index (Length of stay, Acuity, Comorbidity, and Emergency department use).[10] The LACE index predicted readmission and death within 30 days of discharge in a large cohort in Canada. In 2 retrospective studies of recently hospitalized patients in the United States, the LACE index's ability to discriminate between patients readmitted or not ranged from slightly better than chance to moderate (C statistic 0.56‐0.77).[11, 12]

It is unknown whether adding patient‐reported preparedness measures to commonly used readmission prediction scores increases the ability to predict readmission risk. We sought to determine whether the B‐PREPARED and CTM‐3 measures were predictive of readmission or death, as compared to the LACE index, in a large cohort of cardiovascular patients. In addition, we sought to determine the additional predictive and discriminative ability gained from administering the B‐PREPARED and CTM‐3 measures, while adjusting for the LACE index and other clinical factors. We hypothesized that: (1) higher preparedness scores on both measures would predict lower risk of readmission or death in a cohort of patients hospitalized with cardiac diagnoses; and (2) because it provides more specific and actionable information, the B‐PREPARED would discriminate readmission more accurately than CTM‐3, after controlling for clinical factors.

METHODS

Study Setting and Design

The Vanderbilt Inpatient Cohort Study (VICS) is a prospective study of patients admitted with cardiovascular disease to Vanderbilt University Hospital. The purpose of VICS is to investigate the impact of patient and social factors on postdischarge health outcomes such as quality of life, unplanned hospital utilization, and mortality. The rationale and design of VICS are detailed elsewhere.[13] Briefly, participants completed a baseline interview while hospitalized, and follow‐up phone calls were conducted within 2 to 9 days and at approximately 30 and 90 days postdischarge. During the first follow‐up call conducted by research assistants, we collected preparedness for discharge data utilizing the 2 measures described below. After the 90‐day phone call, we collected healthcare utilization since the index admission. The study was approved by the Vanderbilt University Institutional Review Board.

Patients

Eligibility screening shortly after admission identified patients with acute decompensated heart failure (ADHF) and/or an intermediate or high likelihood of acute coronary syndrome (ACS) per a physician's review of the clinical record. Exclusion criteria included: age <18 years, non‐English speaker, unstable psychiatric illness, delirium, low likelihood of follow‐up (eg, no reliable telephone number), on hospice, or otherwise too ill to complete an interview. To be included in these analyses, patients must have completed the preparedness for discharge measurements during the first follow‐up call. Patients who died before discharge or before completing the follow‐up call were excluded.

Preparedness for Discharge Measures (Patient‐Reported Data)

Preparedness for discharge was assessed using the 11‐item B‐PREPARED and the 3‐item CTM‐3.

The B‐PREPARED measures how prepared patients felt leaving the hospital with regard to: self‐care information for medications and activity, equipment/community services needed, and confidence in managing one's health after hospitalization. The B‐PREPARED measure has good internal consistency reliability (Cronbach's = 0.76) and has been validated in patients of varying age within a week of discharge. Preparedness is the sum of responses to all 11 questions, with a range of 0 to 22. Higher scores reflect increased preparedness for discharge.[3]

The CTM‐3 asks patients to rate how well their preferences were considered regarding transitional needs, as well as their understanding of postdischarge self‐management and the purpose of their medications, each on a 4‐point response scale (strongly disagree to strongly agree). The sum of the 3 responses quantifies the patient's perception of the quality of the care transition at discharge (Cronbach's = 0.86,[14] 0.92 in a cohort similar to ours[8]). Scores range from 3 to 12, with higher score indicating more preparedness. Then, the sum is transformed to a 0 to 100 scale.[15]

Clinical Readmission Risk Measures (Medical Record Data)

The LACE index, published by Van Walraven et al.,[10] takes into account 4 categories of clinical data: length of hospital stay, acuity of event, comorbidities, and ED visits in the prior 6 months. More specifically, a diagnostic code‐based, modified version of the Charlson Comorbidity Index was used to calculate the comorbidity score. These clinical criteria were obtained from an administrative database and weighted according to the methods used by Van Walraven et al. An overall score was calculated on a scale of 0 to 19, with higher scores indicating higher risk of readmission or death within 30 days.

From medical records, we also collected patients' demographic data including age, race, and gender, and diagnosis of ACS, ADHF, or both at hospital admission.

Outcome Measures

Healthcare utilization data were obtained from the index hospital as well as outside facilities. The electronic medical records from Vanderbilt University Hospital provided information about healthcare utilization at Vanderbilt 90 days after initial discharge. We also used Vanderbilt records to see if patients were transferred to Vanderbilt from other hospitals or if patients visited other hospitals before or after enrollment. We supplemented this with patient self‐report during the follow‐up telephone calls (at 30 and 90 days after initial discharge) so that any additional ED and hospital visits could be captured. Mortality data were collected from medical records, Social Security data, and family reports. The main outcome was time to first unplanned hospital readmission or death within 30 and 90 days of discharge.

Analysis

To describe our sample, we summarized categorical variables with percentages and continuous variables with percentiles. To test for evidence of unadjusted covariate‐outcome relationships, we used Pearson 2 and Wilcoxon rank sum tests for categorical and continuous covariates, respectively.

For the primary analyses we used Cox proportional hazard models to examine the independent associations between the prespecified predictors for patient‐reported preparedness and time to first unplanned readmission or death within 30 and 90 days of discharge. For each outcome (30‐ and 90‐day readmission or death), we fit marginal models separately for each of the B‐PREPARED, CTM‐3, and LACE scores. We then fit multivariable models that used both preparedness measures as well as age, gender, race, and diagnosis (ADHF and/or ACS), variables available to clinicians when patients are admitted. When fitting the multivariable models, we did not find strong evidence of nonlinear effects; therefore, only linear effects are reported. To facilitate comparison of effects, we scaled continuous variables by their interquartile range (IQR). The associated, exponentiated regression parameter estimates may therefore be interpreted as hazard ratios for readmission or death per IQR change in each predictor. In addition to parameter estimation, we computed the C index to evaluate capacity for the model to discriminate those who were and were not readmitted or died. All analyses were conducted in R version 3.1.2 (R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

From the cohort of 1239 patients (Figure 1), 64%, 28%, and 7% of patients were hospitalized with ACS, ADHF, or both, respectively (Table 1). Nearly 45% of patients were female, 83% were white, and the median age was 61 years (IQR 5269). The median length of stay was 3 days (IQR 25). The median preparedness scores were high for both B‐PREPARED (21, IQR 1822) and CTM‐3 (77.8, IQR 66.7100). A total of 211 (17%) and 380 (31%) were readmitted or died within 30 and 90 days, respectively. The completion rate for the postdischarge phone calls was 88%.

Patient Characteristics
Death or Readmission Within 30 Days Death or Readmission Within 90 Days
Not Readmitted, N = 1028 Death/Readmitted, N = 211 P Value Not Readmitted, N = 859 Death/Readmitted, N = 380 P Value
  • NOTE: Continuous variables: summarize with the 5th:25th:50th:75th:95th. Categorical variables: summarize with the percentage and (N). Abbreviations: ACS, acute coronary syndromes; ADHF, acute decompensated heart failure; B‐PREPARED, Brief PREPARED (Prescriptions, Ready to re‐enter community, Education, Placement, Assurance of safety, Realistic expectations, Empowerment, Directed to appropriate services) CTM‐3, Care Transitions Measure‐3; LACE, Length of hospital stay, Acuity of event, Comorbidities, and ED visits in the prior 6 months; LOS, length of stay. *Pearson test. Wilcoxon test.

Gender, male 55.8% (574) 53.1% (112) 0.463* 56.3% (484) 53.2% (202) 0.298*
Female 44.2% (454) 46.9% (99) 43.7% (375) 46.8% (178)
Race, white 83.9% (860) 80.6% (170) 0.237* 86.0% (737) 77.3% (293) <0.001*
Race, nonwhite 16.1% (165) 19.4% (41) 14.0% (120) 22.7% (86)
Diagnosis ACS 68.0% (699) 46.4% (98) <0.001* 72.9% (626) 45.0% (171) <0.001*
ADHF 24.8% (255) 46.0% (97) 20.3% (174) 46.8% (178)
Both 7.2% (74) 7.6% (16) 6.9% (59) 8.2% (31)
Age 39.4:52:61:68:80 37.5:53.5:62:70:82 0.301 40:52:61:68:80 38:52:61 :70:82 0.651
LOS 1:2:3:5:10 1:3: 4:7.5:17 <0.001 1:2:3:5:9 1:3:4:7:15 <0.001
CTM‐3 55.6:66.7: 77.8:100:100 55.6:66.7:77.8:100 :100 0.305 55.6:66.7:88.9:100:100 55.6:66.7:77.8:100 :100 0.080
B‐PREPARED 12:18:21:22.:22 10:17:20:22:22 0.066 12:18:21:22:22 10:17:20 :22:22 0.030
LACE 1:4: 7:10 :14 3.5:7:10:13:17 <0.001 1:4:6: 9:14 3:7:10:13:16 <0.001
Figure 1
Study flow diagram. Abbreviations: ACS, acute coronary syndrome; ADHF, acute decompensated heart failure; VICS, Vanderbilt Inpatient Cohort Study.

B‐PREPARED and CTM‐3 were moderately correlated with one another (Spearman's = 0.40, P < 0.001). In bivariate analyses (Table 1), the association between B‐PREPARED and readmission or death was significant at 90 days (P = 0.030) but not 30 days. The CTM‐3 showed no significant association with readmission or death at either time point. The LACE score was significantly associated with rates of readmission at 30 and 90 days (P < 0.001).

Outcomes Within 30 Days of Discharge

When examining readmission or death within 30 days of discharge, simple unadjusted models 2 and 3 showed that the B‐PREPARED and LACE scores, respectively, were each significantly associated with time to first readmission or death (Table 2). Specifically, a 4‐point increase in the B‐PREPARED score was associated with a 16% decrease in the hazard of readmission or death (hazard ratio [HR] = 0.84, 95% confidence interval [CI]: 0.72 to 0.97). A 5‐point increase in the LACE score was associated with a 100% increase in the hazard of readmission or death (HR = 2.00, 95% CI: 1.72 to 2.32). In the multivariable model with both preparedness scores and diagnosis (model 4), the B‐PREPARED score (HR = 0.82, 95% CI: 0.70 to 0.97) was significantly associated with time to first readmission or death. In the full 30‐day model including B‐PREPARED, CTM‐3, LACE, age, gender, race, and diagnosis (model 5), only the LACE score (HR = 1.83, 95% CI: 1.54 to 2.18) was independently associated with time to readmission or death. Finally, the CTM‐3 did not predict 30‐day readmission or death in any of the models tested.

Cox Models: Time to Death or Readmission Within 30 Days of Index Hospitalization
Models HR (95% CI)* P Value C Index
  • NOTE: Abbreviations: ADHF, acute decompensated heart failure; B‐PREPARED, Brief PREPARED (Prescriptions, Ready to re‐enter community, Education, Placement, Assurance of safety, Realistic expectations, Empowerment, Directed to appropriate services); CI, confidence interval; CTM‐3, Care Transitions Measure‐3; HR, hazard ratio; LACE, Length of hospital stay, Acuity of event, Comorbidities, and Emergency department visits in the prior 6 months.

1. CTM (per 10‐point change) 0.95 (0.88 to 1.03) 0.257 0.523
2. B‐PREPARED (per 4‐point change) 0.84 (0.72 to 0.97) 0.017 0.537
3. LACE (per 5‐point change) 2.00 (1.72 to 2.32) <0.001 0.679
4. CTM (per 10‐point change) 1.00 (0.92 to 1.10) 0.935 0.620
B‐PREPARED (per 4‐point change) 0.82 (0.70 to 0.97) 0.019
ADHF only (vs ACS only) 2.46 (1.86 to 3.26) <0.001
ADHF and ACS (vs ACS only) 1.42 (0.84 to 2.42) 0.191
5. CTM (per 10‐point change) 1.02 (0.93 to 1.11) 0.722 0.692
B‐PREPARED (per 4 point change) 0.87 (0.74 to 1.03) 0.106
LACE (per 5‐point change) 1.83 (1.54 to 2.18) <0.001
ADHF only (vs ACS only) 1.51 (1.10 to 2.08) 0.010
ADHF and ACS (vs ACS only) 0.90 (0.52 to 1.55) 0.690
Age (per 10‐year change) 1.02 (0.92 to 1.14) 0.669
Female (vs male) 1.11 (0.85 to 1.46) 0.438
Nonwhite (vs white) 0.92 (0.64 to 1.30) 0.624

Outcomes Within 90 Days of Discharge

At 90 days after discharge, again the separate unadjusted models 2 and 3 demonstrated that the B‐PREPARED and LACE scores, respectively, were each significantly associated with time to first readmission or death, whereas the CTM‐3 model only showed marginal significance (Table 3). In the multivariable model with both preparedness scores and diagnosis (model 4), results were similar to 30 days as the B‐PREPARED score was significantly associated with time to first readmission or death. Lastly, in the full model (model 5) at 90 days, again the LACE score was significantly associated with time to first readmission or death. In addition, B‐PREPARED scores were associated with a significant decrease in risk of readmission or death (HR = 0.88, 95% CI: 0.78 to 1.00); CTM‐3 scores were not independently associated with outcomes.

Cox Models: Time to Death or Readmission Within 90 Days of Index Hospitalization
Model HR (95% CI)* P Value C Index
  • NOTE: Abbreviations: ADHF, acute decompensated heart failure; B‐PREPARED, Brief PREPARED (Prescriptions, Ready to re‐enter community, Education, Placement, Assurance of safety, Realistic expectations, Empowerment, Directed to appropriate services); CI, confidence interval; CTM‐3, Care Transitions Measure‐3; HR, hazard ratio; LACE, Length of hospital stay, Acuity of event, Comorbidities, and Emergency department visits in the prior 6 months.

1. CTM (per 10‐point change) 0.94 (0.89 to 1.00) 0.051 0.526
2. B‐PREPARED (per 4‐point change) 0.84 (0.75 to 0.94) 0.002 0.533
3. LACE (per 5‐point change) 2.03 (1.82 to 2.27) <0.001 0.683
4. CTM (per 10‐point change) 0.99 (0.93 to 1.06) 0.759 0.640
B‐PREPARED (per 4‐point change) 0.83 (0.74 to 0.94) 0.003
ADHF only (vs ACS only) 2.88 (2.33 to 3.56) <0.001
ADHF and ACS (vs ACS only) 1.62 (1.11 to 2.38) 0.013
5. CTM (per 10‐point change) 1.00 (0.94 to 1.07) 0.932 0.698
B‐PREPARED (per 4‐point change) 0.88 (0.78 to 1.00) 0.043
LACE (per 5‐point change) 1.76 (1.55 to 2.00) <0.001
ADHF only (vs ACS only) 1.76 (1.39 to 2.24) <0.001
ADHF and ACS (vs ACS only) 1.00 (0.67 to 1.50) 0.980
Age (per 10‐year change) 1.00 (0.93 to 1.09) 0.894
Female (vs male) 1.10 (0.90 to 1.35) 0.341
Nonwhite (vs white) 1.14 (0.89 to 1.47) 0.288

Tables 2 and 3 also display the C indices, or the discriminative ability of the models to differentiate whether or not a patient was readmitted or died. The range of the C index is 0.5 to 1, where values closer to 0.5 indicate random predictions and values closer to 1 indicate perfect prediction. At 30 days, the individual C indices for B‐PREPARED and CTM‐3 were only slightly better than chance (0.54 and 0.52, respectively) in their discriminative abilities. However, the C indices for the LACE score alone (0.68) and the multivariable model (0.69) including all 3 measures (ie, B‐PREPARED, CTM‐3, LACE), and clinical and demographic variables, had higher utility in discriminating patients who were readmitted/died or not. The 90‐day C indices were comparable in magnitude to those at 30 days.

DISCUSSION/CONCLUSION

In this cohort of patients hospitalized with cardiovascular disease, we compared 2 patient‐reported measures of preparedness for discharge, their association with time to death or readmission at 30 and 90 days, and their ability to discriminate patients who were or were not readmitted or died. Higher preparedness as measured by higher B‐PREPARED scores was associated with lower risk of readmission or death at 30 and 90 days after discharge in unadjusted models, and at 90 days in adjusted models. CTM‐3 was not associated with the outcome in any analyses. Lastly, the individual preparedness measures were not as strongly associated with readmission or death compared to the LACE readmission index alone.

How do our findings relate to the measurement of care transition quality? We consider 2 scenarios. First, if hospitals utilize the LACE index to predict readmission, then neither self‐reported measure of preparedness adds meaningfully to its predictive ability. However, hospital management may still find the B‐PREPARED and CTM‐3 useful as a means to direct care transition quality‐improvement efforts. These measures can instruct hospitals as to what areas their patients express the greatest difficulty or lack of preparedness and closely attend to patient needs with appropriate resources. Furthermore, the patient's perception of being prepared for discharge may be different than their actual preparedness. Their perceived preparedness may be affected by cognitive impairment, dissatisfaction with medical care, depression, lower health‐related quality of life, and lower educational attainment as demonstrated by Lau et al.[16] If a patient's perception of preparedness were low, it would behoove the clinician to investigate these other issues and address those that are mutable. Additionally, perceived preparedness may not correlate with the patient's understanding of their medical conditions, so it is imperative that clinicians provide prospective guidance about their probable postdischarge trajectory. If hospitals are not utilizing the LACE index, then perhaps using the B‐PREPARED, but not the CTM‐3, may be beneficial for predicting readmission.

How do our results fit with evidence from prior studies, and what do they mean in the context of care transitions quality? First, in the psychometric evaluation of the B‐PREPARED measure in a cohort of recently hospitalized patients, the mean score was 17.3, lower than the median of 21 in our cohort.[3] Numerous studies have utilized the CTM‐3 and the longer‐version CTM‐15. Though we cannot make a direct comparison, the median in our cohort (77.8) was on par with the means from other studies, which ranged from 63 to 82.[5, 17, 18, 19] Several studies also note ceiling effects with clusters of scores at the upper end of the scale, as did we. We conjecture that our cohort's preparedness scores may be higher because our institution has made concerted efforts to improve the discharge education for cardiovascular patients.

In a comparable patient population, the TRACE‐CORE (Transitions, Risks, and Actions in Coronary Events Center for Outcomes Research and Education) study is a cohort of more than 2200 patients with ACS who were administered the CTM‐15 within 1 month of discharge.[8] In that study, the median CTM‐15 score was 66.6, which is lower than our cohort. With regard to the predictive ability of the CTM‐3, they note that CTM‐3 scores did not differentiate between patients who were or were not readmitted or had emergency department visits. Our results support their concern that the CTM‐15 and by extension the CTM‐3, though adopted widely as part of HCAHPS, may not have sufficient ability to discriminate differences in patient outcomes or the quality of care transitions.

More recently, patient‐reported preparedness for discharge was assessed in a prospective cohort in Canada.[16] Lau et al. administered a single‐item measure of readiness at the time of discharge to general medicine patients, and found that lower readiness scores were also not associated with readmission or death at 30 days, when adjusted for the LACE index as we did.

We must acknowledge the limitations of our findings. First, our sample of recently discharged patients with cardiovascular disease is different than the community‐dwelling, underserved Americans hospitalized in the prior year, which served as the sample for reducing the CTM‐15 to 3 items.[5] This fact may explain why we did not find the CTM‐3 to be associated with readmission in our sample. Second, our analyses did not include extensive adjustment for patient‐related factors. Rather, our intention was to see how well the preparedness measures performed independently and compare their abilities to predict readmission, which is particularly relevant for clinicians who may not have all possible covariates in predicting readmission. Finally, because we limited the analyses to the patients who completed the B‐PREPARED and CTM‐3 measures (88% completion rate), we may not have data for: (1) very ill patients, who had a higher risk of readmission and least prepared, and were not able to answer the postdischarge phone call; and (2) very functional patients, who had a lower risk of readmission and were too busy to answer the postdischarge phone call. This may have limited the extremes in the spectrum of our sample.

Importantly, our study has several strengths. We report on the largest sample to date with results of both B‐PREPARED and CTM‐3. Moreover, we examined how these measures compared to a widely used readmission prediction tool, the LACE index. We had very high postdischarge phone call completion rates in the week following discharge. Furthermore, we had thorough assessment of readmission data through patient report, electronic medical record documentation, and collection of outside medical records.

Further research is needed to elucidate: (1) the ideal administration time of the patient‐reported measures of preparedness (before or after discharge), and (2) the challenges to the implementation of measures in healthcare systems. Remaining research questions center on the tradeoffs and barriers to implementing a longer measure like the 11‐item B‐PREPARED compared to a shorter measure like the CTM‐3. We do not know whether longer measures preclude their use by busy clinicians, though it provides more specific information about what patients feel they need at hospital discharge. Additionally, studies need to demonstrate the mutability of preparedness and the response of measures to interventions designed to improve the hospital discharge process.

In our sample of recently hospitalized cardiovascular patients, there was a statistically significant association between patient‐reported preparedness for discharged, as measured by B‐PREPARED, and readmissions/death at 30 and 90 days, but the magnitude of the association was very small. Furthermore, another patient‐reported preparedness measure, CTM‐3, was not associated with readmissions or death at either 30 or 90 days. Lastly, neither measure discriminated well between patients who were readmitted or not, and neither measure added meaningfully to the LACE index in terms of predicting 30‐ or 90‐day readmissions.

Disclosures

This study was supported by grant R01 HL109388 from the National Heart, Lung, and Blood Institute (Dr. Kripalani) and in part by grant UL1 RR024975‐01 from the National Center for Research Resources, and grant 2 UL1 TR000445‐06 from the National Center for Advancing Translational Sciences. Dr. Kripalani is a consultant to SAI Interactive and holds equity in Bioscape Digital, and is a consultant to and holds equity in PictureRx, LLC. Dr. Bell is supported by the National Institutes of Health (K23AG048347) and by the Eisenstein Women's Heart Fund. Dr. Vasilevskis is supported by the National Institutes of Health (K23AG040157) and the Geriatric Research, Education and Clinical Center. Dr. Mixon is a Veterans Affairs Health Services Research and Development Service Career Development awardee (12‐168) at the Nashville Department of Veterans Affairs. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The funding agency was not involved in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. All authors had full access to all study data and had a significant role in writing the manuscript. The contents do not represent the views of the US Department of Veterans Affairs or the United States government. Dr. Kripalani is a consultant to and holds equity in PictureRx, LLC.

Files
References
  1. Centers for Medicare 9(9):598603.
  2. Graumlich JF, Novotny NL, Aldag JC. Brief scale measuring patient preparedness for hospital discharge to home: psychometric properties. J Hosp Med. 2008;3(6):446454.
  3. Coleman EA, Mahoney E, Parry C. Assessing the quality of preparation for posthospital care from the patient's perspective: the care transitions measure. Med Care. 2005;43(3):246255.
  4. Parry C, Mahoney E, Chalmers SA, Coleman EA. Assessing the quality of transitional care: further applications of the care transitions measure. Med Care. 2008;46(3):317322.
  5. Coleman EA, Parry C, Chalmers SA, Chugh A, Mahoney E. The central role of performance measurement in improving the quality of transitional care. Home Health Care Serv Q. 2007;26(4):93104.
  6. Centers for Medicare 3:e001053.
  7. Kansagara D, Englander H, Salanitro AH, et al. Risk prediction models for hospital readmission: a systematic review. JAMA. 2011;306(15):16881698.
  8. Walraven C, Dhalla IA, Bell C, et al. Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community. CMAJ. 2010;182(6):551557.
  9. Wang H, Robinson RD, Johnson C, et al. Using the LACE index to predict hospital readmissions in congestive heart failure patients. BMC Cardiovasc Disord. 2014;14:97.
  10. Spiva L, Hand M, VanBrackle L, McVay F. Validation of a predictive model to identify patients at high risk for hospital readmission. J Healthc Qual. 2016;38(1):3441.
  11. Meyers AG, Salanitro A, Wallston KA, et al. Determinants of health after hospital discharge: rationale and design of the Vanderbilt Inpatient Cohort Study (VICS). BMC Health Serv Res. 2014;14:10.
  12. Coleman EA. CTM frequently asked questions. Available at: http://caretransitions.org/tools-and-resources/. Accessed January 22, 2016.
  13. Coleman EA. Instructions for scoring the CTM‐3. Available at: http://caretransitions.org/tools-and-resources/. Accessed January 22, 2016.
  14. Lau D, Padwal RS, Majumdar SR, et al. Patient‐reported discharge readiness and 30‐day risk of readmission or death: a prospective cohort study. Am J Med. 2016;129:8995.
  15. Parrish MM, O'Malley K, Adams RI, Adams SR, Coleman EA. Implementaiton of the Care Transitions Intervention: sustainability and lessons learned. Prof Case Manag. 2009;14(6):282293.
  16. Englander H, Michaels L, Chan B, Kansagara D. The care transitions innovation (C‐TraIn) for socioeconomically disadvantaged adults: results of a cluster randomized controlled trial. J Gen Intern Med. 2014;29(11):14601467.
  17. Record JD, Niranjan‐Azadi A, Christmas C, et al. Telephone calls to patients after discharge from the hospital: an important part of transitions of care. Med Educ Online. 2015;29(20):26701.
Article PDF
Issue
Journal of Hospital Medicine - 11(9)
Page Number
603-609
Sections
Files
Files
Article PDF
Article PDF

In recent years, US hospitals have focused on decreasing readmission rates, incented by reimbursement penalties to hospitals having excessive readmissions.[1] Gaps in the quality of care provided during transitions likely contribute to preventable readmissions.[2] One compelling quality assessment in this setting is measuring patients' discharge preparedness, using key dimensions such as understanding their instructions for medication use and follow‐up. Patient‐reported preparedness for discharge may also be useful to identify risk of readmission.

Several patient‐reported measures of preparedness for discharge exist, and herein we describe 2 measures of interest. First, the Brief‐PREPARED (B‐PREPARED) measure was derived from the longer PREPARED instrument (Prescriptions, Ready to re‐enter community, Education, Placement, Assurance of safety, Realistic expectations, Empowerment, Directed to appropriate services), which reflects the patient's perceived needs at discharge. In previous research, the B‐PREPARED measure predicted emergency department (ED) visits for patients who had been recently hospitalized and had a high risk for readmission.[3] Second, the Care Transitions Measure‐3 (CTM‐3) was developed by Coleman et al. as a patient‐reported measure to discriminate between patients who were more likely to have an ED visit or readmission from those who did not. CTM‐3 has also been used to evaluate hospitals' level of care coordination and for public reporting purposes.[4, 5, 6] It has been endorsed by the National Quality Forum and incorporated into the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey provided to samples of recently hospitalized US patients.[7] However, recent evidence from an inpatient cohort of cardiovascular patients suggests the CTM‐3 overinflates care transition scores compared to the longer 15‐item CTM. In that cohort, the CTM‐3 could not differentiate between patients who did or did not have repeat ED visits or readmission.[8] Thus far, the B‐PREPARED and CTM‐3 measures have not been compared to one another directly.

In addition to the development of patient‐reported measures, hospitals increasingly employ administrative algorithms to predict likelihood of readmission.[9] A commonly used measure is the LACE index (Length of stay, Acuity, Comorbidity, and Emergency department use).[10] The LACE index predicted readmission and death within 30 days of discharge in a large cohort in Canada. In 2 retrospective studies of recently hospitalized patients in the United States, the LACE index's ability to discriminate between patients readmitted or not ranged from slightly better than chance to moderate (C statistic 0.56‐0.77).[11, 12]

It is unknown whether adding patient‐reported preparedness measures to commonly used readmission prediction scores increases the ability to predict readmission risk. We sought to determine whether the B‐PREPARED and CTM‐3 measures were predictive of readmission or death, as compared to the LACE index, in a large cohort of cardiovascular patients. In addition, we sought to determine the additional predictive and discriminative ability gained from administering the B‐PREPARED and CTM‐3 measures, while adjusting for the LACE index and other clinical factors. We hypothesized that: (1) higher preparedness scores on both measures would predict lower risk of readmission or death in a cohort of patients hospitalized with cardiac diagnoses; and (2) because it provides more specific and actionable information, the B‐PREPARED would discriminate readmission more accurately than CTM‐3, after controlling for clinical factors.

METHODS

Study Setting and Design

The Vanderbilt Inpatient Cohort Study (VICS) is a prospective study of patients admitted with cardiovascular disease to Vanderbilt University Hospital. The purpose of VICS is to investigate the impact of patient and social factors on postdischarge health outcomes such as quality of life, unplanned hospital utilization, and mortality. The rationale and design of VICS are detailed elsewhere.[13] Briefly, participants completed a baseline interview while hospitalized, and follow‐up phone calls were conducted within 2 to 9 days and at approximately 30 and 90 days postdischarge. During the first follow‐up call conducted by research assistants, we collected preparedness for discharge data utilizing the 2 measures described below. After the 90‐day phone call, we collected healthcare utilization since the index admission. The study was approved by the Vanderbilt University Institutional Review Board.

Patients

Eligibility screening shortly after admission identified patients with acute decompensated heart failure (ADHF) and/or an intermediate or high likelihood of acute coronary syndrome (ACS) per a physician's review of the clinical record. Exclusion criteria included: age <18 years, non‐English speaker, unstable psychiatric illness, delirium, low likelihood of follow‐up (eg, no reliable telephone number), on hospice, or otherwise too ill to complete an interview. To be included in these analyses, patients must have completed the preparedness for discharge measurements during the first follow‐up call. Patients who died before discharge or before completing the follow‐up call were excluded.

Preparedness for Discharge Measures (Patient‐Reported Data)

Preparedness for discharge was assessed using the 11‐item B‐PREPARED and the 3‐item CTM‐3.

The B‐PREPARED measures how prepared patients felt leaving the hospital with regard to: self‐care information for medications and activity, equipment/community services needed, and confidence in managing one's health after hospitalization. The B‐PREPARED measure has good internal consistency reliability (Cronbach's = 0.76) and has been validated in patients of varying age within a week of discharge. Preparedness is the sum of responses to all 11 questions, with a range of 0 to 22. Higher scores reflect increased preparedness for discharge.[3]

The CTM‐3 asks patients to rate how well their preferences were considered regarding transitional needs, as well as their understanding of postdischarge self‐management and the purpose of their medications, each on a 4‐point response scale (strongly disagree to strongly agree). The sum of the 3 responses quantifies the patient's perception of the quality of the care transition at discharge (Cronbach's = 0.86,[14] 0.92 in a cohort similar to ours[8]). Scores range from 3 to 12, with higher score indicating more preparedness. Then, the sum is transformed to a 0 to 100 scale.[15]

Clinical Readmission Risk Measures (Medical Record Data)

The LACE index, published by Van Walraven et al.,[10] takes into account 4 categories of clinical data: length of hospital stay, acuity of event, comorbidities, and ED visits in the prior 6 months. More specifically, a diagnostic code‐based, modified version of the Charlson Comorbidity Index was used to calculate the comorbidity score. These clinical criteria were obtained from an administrative database and weighted according to the methods used by Van Walraven et al. An overall score was calculated on a scale of 0 to 19, with higher scores indicating higher risk of readmission or death within 30 days.

From medical records, we also collected patients' demographic data including age, race, and gender, and diagnosis of ACS, ADHF, or both at hospital admission.

Outcome Measures

Healthcare utilization data were obtained from the index hospital as well as outside facilities. The electronic medical records from Vanderbilt University Hospital provided information about healthcare utilization at Vanderbilt 90 days after initial discharge. We also used Vanderbilt records to see if patients were transferred to Vanderbilt from other hospitals or if patients visited other hospitals before or after enrollment. We supplemented this with patient self‐report during the follow‐up telephone calls (at 30 and 90 days after initial discharge) so that any additional ED and hospital visits could be captured. Mortality data were collected from medical records, Social Security data, and family reports. The main outcome was time to first unplanned hospital readmission or death within 30 and 90 days of discharge.

Analysis

To describe our sample, we summarized categorical variables with percentages and continuous variables with percentiles. To test for evidence of unadjusted covariate‐outcome relationships, we used Pearson 2 and Wilcoxon rank sum tests for categorical and continuous covariates, respectively.

For the primary analyses we used Cox proportional hazard models to examine the independent associations between the prespecified predictors for patient‐reported preparedness and time to first unplanned readmission or death within 30 and 90 days of discharge. For each outcome (30‐ and 90‐day readmission or death), we fit marginal models separately for each of the B‐PREPARED, CTM‐3, and LACE scores. We then fit multivariable models that used both preparedness measures as well as age, gender, race, and diagnosis (ADHF and/or ACS), variables available to clinicians when patients are admitted. When fitting the multivariable models, we did not find strong evidence of nonlinear effects; therefore, only linear effects are reported. To facilitate comparison of effects, we scaled continuous variables by their interquartile range (IQR). The associated, exponentiated regression parameter estimates may therefore be interpreted as hazard ratios for readmission or death per IQR change in each predictor. In addition to parameter estimation, we computed the C index to evaluate capacity for the model to discriminate those who were and were not readmitted or died. All analyses were conducted in R version 3.1.2 (R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

From the cohort of 1239 patients (Figure 1), 64%, 28%, and 7% of patients were hospitalized with ACS, ADHF, or both, respectively (Table 1). Nearly 45% of patients were female, 83% were white, and the median age was 61 years (IQR 5269). The median length of stay was 3 days (IQR 25). The median preparedness scores were high for both B‐PREPARED (21, IQR 1822) and CTM‐3 (77.8, IQR 66.7100). A total of 211 (17%) and 380 (31%) were readmitted or died within 30 and 90 days, respectively. The completion rate for the postdischarge phone calls was 88%.

Patient Characteristics
Death or Readmission Within 30 Days Death or Readmission Within 90 Days
Not Readmitted, N = 1028 Death/Readmitted, N = 211 P Value Not Readmitted, N = 859 Death/Readmitted, N = 380 P Value
  • NOTE: Continuous variables: summarize with the 5th:25th:50th:75th:95th. Categorical variables: summarize with the percentage and (N). Abbreviations: ACS, acute coronary syndromes; ADHF, acute decompensated heart failure; B‐PREPARED, Brief PREPARED (Prescriptions, Ready to re‐enter community, Education, Placement, Assurance of safety, Realistic expectations, Empowerment, Directed to appropriate services) CTM‐3, Care Transitions Measure‐3; LACE, Length of hospital stay, Acuity of event, Comorbidities, and ED visits in the prior 6 months; LOS, length of stay. *Pearson test. Wilcoxon test.

Gender, male 55.8% (574) 53.1% (112) 0.463* 56.3% (484) 53.2% (202) 0.298*
Female 44.2% (454) 46.9% (99) 43.7% (375) 46.8% (178)
Race, white 83.9% (860) 80.6% (170) 0.237* 86.0% (737) 77.3% (293) <0.001*
Race, nonwhite 16.1% (165) 19.4% (41) 14.0% (120) 22.7% (86)
Diagnosis ACS 68.0% (699) 46.4% (98) <0.001* 72.9% (626) 45.0% (171) <0.001*
ADHF 24.8% (255) 46.0% (97) 20.3% (174) 46.8% (178)
Both 7.2% (74) 7.6% (16) 6.9% (59) 8.2% (31)
Age 39.4:52:61:68:80 37.5:53.5:62:70:82 0.301 40:52:61:68:80 38:52:61 :70:82 0.651
LOS 1:2:3:5:10 1:3: 4:7.5:17 <0.001 1:2:3:5:9 1:3:4:7:15 <0.001
CTM‐3 55.6:66.7: 77.8:100:100 55.6:66.7:77.8:100 :100 0.305 55.6:66.7:88.9:100:100 55.6:66.7:77.8:100 :100 0.080
B‐PREPARED 12:18:21:22.:22 10:17:20:22:22 0.066 12:18:21:22:22 10:17:20 :22:22 0.030
LACE 1:4: 7:10 :14 3.5:7:10:13:17 <0.001 1:4:6: 9:14 3:7:10:13:16 <0.001
Figure 1
Study flow diagram. Abbreviations: ACS, acute coronary syndrome; ADHF, acute decompensated heart failure; VICS, Vanderbilt Inpatient Cohort Study.

B‐PREPARED and CTM‐3 were moderately correlated with one another (Spearman's = 0.40, P < 0.001). In bivariate analyses (Table 1), the association between B‐PREPARED and readmission or death was significant at 90 days (P = 0.030) but not 30 days. The CTM‐3 showed no significant association with readmission or death at either time point. The LACE score was significantly associated with rates of readmission at 30 and 90 days (P < 0.001).

Outcomes Within 30 Days of Discharge

When examining readmission or death within 30 days of discharge, simple unadjusted models 2 and 3 showed that the B‐PREPARED and LACE scores, respectively, were each significantly associated with time to first readmission or death (Table 2). Specifically, a 4‐point increase in the B‐PREPARED score was associated with a 16% decrease in the hazard of readmission or death (hazard ratio [HR] = 0.84, 95% confidence interval [CI]: 0.72 to 0.97). A 5‐point increase in the LACE score was associated with a 100% increase in the hazard of readmission or death (HR = 2.00, 95% CI: 1.72 to 2.32). In the multivariable model with both preparedness scores and diagnosis (model 4), the B‐PREPARED score (HR = 0.82, 95% CI: 0.70 to 0.97) was significantly associated with time to first readmission or death. In the full 30‐day model including B‐PREPARED, CTM‐3, LACE, age, gender, race, and diagnosis (model 5), only the LACE score (HR = 1.83, 95% CI: 1.54 to 2.18) was independently associated with time to readmission or death. Finally, the CTM‐3 did not predict 30‐day readmission or death in any of the models tested.

Cox Models: Time to Death or Readmission Within 30 Days of Index Hospitalization
Models HR (95% CI)* P Value C Index
  • NOTE: Abbreviations: ADHF, acute decompensated heart failure; B‐PREPARED, Brief PREPARED (Prescriptions, Ready to re‐enter community, Education, Placement, Assurance of safety, Realistic expectations, Empowerment, Directed to appropriate services); CI, confidence interval; CTM‐3, Care Transitions Measure‐3; HR, hazard ratio; LACE, Length of hospital stay, Acuity of event, Comorbidities, and Emergency department visits in the prior 6 months.

1. CTM (per 10‐point change) 0.95 (0.88 to 1.03) 0.257 0.523
2. B‐PREPARED (per 4‐point change) 0.84 (0.72 to 0.97) 0.017 0.537
3. LACE (per 5‐point change) 2.00 (1.72 to 2.32) <0.001 0.679
4. CTM (per 10‐point change) 1.00 (0.92 to 1.10) 0.935 0.620
B‐PREPARED (per 4‐point change) 0.82 (0.70 to 0.97) 0.019
ADHF only (vs ACS only) 2.46 (1.86 to 3.26) <0.001
ADHF and ACS (vs ACS only) 1.42 (0.84 to 2.42) 0.191
5. CTM (per 10‐point change) 1.02 (0.93 to 1.11) 0.722 0.692
B‐PREPARED (per 4 point change) 0.87 (0.74 to 1.03) 0.106
LACE (per 5‐point change) 1.83 (1.54 to 2.18) <0.001
ADHF only (vs ACS only) 1.51 (1.10 to 2.08) 0.010
ADHF and ACS (vs ACS only) 0.90 (0.52 to 1.55) 0.690
Age (per 10‐year change) 1.02 (0.92 to 1.14) 0.669
Female (vs male) 1.11 (0.85 to 1.46) 0.438
Nonwhite (vs white) 0.92 (0.64 to 1.30) 0.624

Outcomes Within 90 Days of Discharge

At 90 days after discharge, again the separate unadjusted models 2 and 3 demonstrated that the B‐PREPARED and LACE scores, respectively, were each significantly associated with time to first readmission or death, whereas the CTM‐3 model only showed marginal significance (Table 3). In the multivariable model with both preparedness scores and diagnosis (model 4), results were similar to 30 days as the B‐PREPARED score was significantly associated with time to first readmission or death. Lastly, in the full model (model 5) at 90 days, again the LACE score was significantly associated with time to first readmission or death. In addition, B‐PREPARED scores were associated with a significant decrease in risk of readmission or death (HR = 0.88, 95% CI: 0.78 to 1.00); CTM‐3 scores were not independently associated with outcomes.

Cox Models: Time to Death or Readmission Within 90 Days of Index Hospitalization
Model HR (95% CI)* P Value C Index
  • NOTE: Abbreviations: ADHF, acute decompensated heart failure; B‐PREPARED, Brief PREPARED (Prescriptions, Ready to re‐enter community, Education, Placement, Assurance of safety, Realistic expectations, Empowerment, Directed to appropriate services); CI, confidence interval; CTM‐3, Care Transitions Measure‐3; HR, hazard ratio; LACE, Length of hospital stay, Acuity of event, Comorbidities, and Emergency department visits in the prior 6 months.

1. CTM (per 10‐point change) 0.94 (0.89 to 1.00) 0.051 0.526
2. B‐PREPARED (per 4‐point change) 0.84 (0.75 to 0.94) 0.002 0.533
3. LACE (per 5‐point change) 2.03 (1.82 to 2.27) <0.001 0.683
4. CTM (per 10‐point change) 0.99 (0.93 to 1.06) 0.759 0.640
B‐PREPARED (per 4‐point change) 0.83 (0.74 to 0.94) 0.003
ADHF only (vs ACS only) 2.88 (2.33 to 3.56) <0.001
ADHF and ACS (vs ACS only) 1.62 (1.11 to 2.38) 0.013
5. CTM (per 10‐point change) 1.00 (0.94 to 1.07) 0.932 0.698
B‐PREPARED (per 4‐point change) 0.88 (0.78 to 1.00) 0.043
LACE (per 5‐point change) 1.76 (1.55 to 2.00) <0.001
ADHF only (vs ACS only) 1.76 (1.39 to 2.24) <0.001
ADHF and ACS (vs ACS only) 1.00 (0.67 to 1.50) 0.980
Age (per 10‐year change) 1.00 (0.93 to 1.09) 0.894
Female (vs male) 1.10 (0.90 to 1.35) 0.341
Nonwhite (vs white) 1.14 (0.89 to 1.47) 0.288

Tables 2 and 3 also display the C indices, or the discriminative ability of the models to differentiate whether or not a patient was readmitted or died. The range of the C index is 0.5 to 1, where values closer to 0.5 indicate random predictions and values closer to 1 indicate perfect prediction. At 30 days, the individual C indices for B‐PREPARED and CTM‐3 were only slightly better than chance (0.54 and 0.52, respectively) in their discriminative abilities. However, the C indices for the LACE score alone (0.68) and the multivariable model (0.69) including all 3 measures (ie, B‐PREPARED, CTM‐3, LACE), and clinical and demographic variables, had higher utility in discriminating patients who were readmitted/died or not. The 90‐day C indices were comparable in magnitude to those at 30 days.

DISCUSSION/CONCLUSION

In this cohort of patients hospitalized with cardiovascular disease, we compared 2 patient‐reported measures of preparedness for discharge, their association with time to death or readmission at 30 and 90 days, and their ability to discriminate patients who were or were not readmitted or died. Higher preparedness as measured by higher B‐PREPARED scores was associated with lower risk of readmission or death at 30 and 90 days after discharge in unadjusted models, and at 90 days in adjusted models. CTM‐3 was not associated with the outcome in any analyses. Lastly, the individual preparedness measures were not as strongly associated with readmission or death compared to the LACE readmission index alone.

How do our findings relate to the measurement of care transition quality? We consider 2 scenarios. First, if hospitals utilize the LACE index to predict readmission, then neither self‐reported measure of preparedness adds meaningfully to its predictive ability. However, hospital management may still find the B‐PREPARED and CTM‐3 useful as a means to direct care transition quality‐improvement efforts. These measures can instruct hospitals as to what areas their patients express the greatest difficulty or lack of preparedness and closely attend to patient needs with appropriate resources. Furthermore, the patient's perception of being prepared for discharge may be different than their actual preparedness. Their perceived preparedness may be affected by cognitive impairment, dissatisfaction with medical care, depression, lower health‐related quality of life, and lower educational attainment as demonstrated by Lau et al.[16] If a patient's perception of preparedness were low, it would behoove the clinician to investigate these other issues and address those that are mutable. Additionally, perceived preparedness may not correlate with the patient's understanding of their medical conditions, so it is imperative that clinicians provide prospective guidance about their probable postdischarge trajectory. If hospitals are not utilizing the LACE index, then perhaps using the B‐PREPARED, but not the CTM‐3, may be beneficial for predicting readmission.

How do our results fit with evidence from prior studies, and what do they mean in the context of care transitions quality? First, in the psychometric evaluation of the B‐PREPARED measure in a cohort of recently hospitalized patients, the mean score was 17.3, lower than the median of 21 in our cohort.[3] Numerous studies have utilized the CTM‐3 and the longer‐version CTM‐15. Though we cannot make a direct comparison, the median in our cohort (77.8) was on par with the means from other studies, which ranged from 63 to 82.[5, 17, 18, 19] Several studies also note ceiling effects with clusters of scores at the upper end of the scale, as did we. We conjecture that our cohort's preparedness scores may be higher because our institution has made concerted efforts to improve the discharge education for cardiovascular patients.

In a comparable patient population, the TRACE‐CORE (Transitions, Risks, and Actions in Coronary Events Center for Outcomes Research and Education) study is a cohort of more than 2200 patients with ACS who were administered the CTM‐15 within 1 month of discharge.[8] In that study, the median CTM‐15 score was 66.6, which is lower than our cohort. With regard to the predictive ability of the CTM‐3, they note that CTM‐3 scores did not differentiate between patients who were or were not readmitted or had emergency department visits. Our results support their concern that the CTM‐15 and by extension the CTM‐3, though adopted widely as part of HCAHPS, may not have sufficient ability to discriminate differences in patient outcomes or the quality of care transitions.

More recently, patient‐reported preparedness for discharge was assessed in a prospective cohort in Canada.[16] Lau et al. administered a single‐item measure of readiness at the time of discharge to general medicine patients, and found that lower readiness scores were also not associated with readmission or death at 30 days, when adjusted for the LACE index as we did.

We must acknowledge the limitations of our findings. First, our sample of recently discharged patients with cardiovascular disease is different than the community‐dwelling, underserved Americans hospitalized in the prior year, which served as the sample for reducing the CTM‐15 to 3 items.[5] This fact may explain why we did not find the CTM‐3 to be associated with readmission in our sample. Second, our analyses did not include extensive adjustment for patient‐related factors. Rather, our intention was to see how well the preparedness measures performed independently and compare their abilities to predict readmission, which is particularly relevant for clinicians who may not have all possible covariates in predicting readmission. Finally, because we limited the analyses to the patients who completed the B‐PREPARED and CTM‐3 measures (88% completion rate), we may not have data for: (1) very ill patients, who had a higher risk of readmission and least prepared, and were not able to answer the postdischarge phone call; and (2) very functional patients, who had a lower risk of readmission and were too busy to answer the postdischarge phone call. This may have limited the extremes in the spectrum of our sample.

Importantly, our study has several strengths. We report on the largest sample to date with results of both B‐PREPARED and CTM‐3. Moreover, we examined how these measures compared to a widely used readmission prediction tool, the LACE index. We had very high postdischarge phone call completion rates in the week following discharge. Furthermore, we had thorough assessment of readmission data through patient report, electronic medical record documentation, and collection of outside medical records.

Further research is needed to elucidate: (1) the ideal administration time of the patient‐reported measures of preparedness (before or after discharge), and (2) the challenges to the implementation of measures in healthcare systems. Remaining research questions center on the tradeoffs and barriers to implementing a longer measure like the 11‐item B‐PREPARED compared to a shorter measure like the CTM‐3. We do not know whether longer measures preclude their use by busy clinicians, though it provides more specific information about what patients feel they need at hospital discharge. Additionally, studies need to demonstrate the mutability of preparedness and the response of measures to interventions designed to improve the hospital discharge process.

In our sample of recently hospitalized cardiovascular patients, there was a statistically significant association between patient‐reported preparedness for discharged, as measured by B‐PREPARED, and readmissions/death at 30 and 90 days, but the magnitude of the association was very small. Furthermore, another patient‐reported preparedness measure, CTM‐3, was not associated with readmissions or death at either 30 or 90 days. Lastly, neither measure discriminated well between patients who were readmitted or not, and neither measure added meaningfully to the LACE index in terms of predicting 30‐ or 90‐day readmissions.

Disclosures

This study was supported by grant R01 HL109388 from the National Heart, Lung, and Blood Institute (Dr. Kripalani) and in part by grant UL1 RR024975‐01 from the National Center for Research Resources, and grant 2 UL1 TR000445‐06 from the National Center for Advancing Translational Sciences. Dr. Kripalani is a consultant to SAI Interactive and holds equity in Bioscape Digital, and is a consultant to and holds equity in PictureRx, LLC. Dr. Bell is supported by the National Institutes of Health (K23AG048347) and by the Eisenstein Women's Heart Fund. Dr. Vasilevskis is supported by the National Institutes of Health (K23AG040157) and the Geriatric Research, Education and Clinical Center. Dr. Mixon is a Veterans Affairs Health Services Research and Development Service Career Development awardee (12‐168) at the Nashville Department of Veterans Affairs. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The funding agency was not involved in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. All authors had full access to all study data and had a significant role in writing the manuscript. The contents do not represent the views of the US Department of Veterans Affairs or the United States government. Dr. Kripalani is a consultant to and holds equity in PictureRx, LLC.

In recent years, US hospitals have focused on decreasing readmission rates, incented by reimbursement penalties to hospitals having excessive readmissions.[1] Gaps in the quality of care provided during transitions likely contribute to preventable readmissions.[2] One compelling quality assessment in this setting is measuring patients' discharge preparedness, using key dimensions such as understanding their instructions for medication use and follow‐up. Patient‐reported preparedness for discharge may also be useful to identify risk of readmission.

Several patient‐reported measures of preparedness for discharge exist, and herein we describe 2 measures of interest. First, the Brief‐PREPARED (B‐PREPARED) measure was derived from the longer PREPARED instrument (Prescriptions, Ready to re‐enter community, Education, Placement, Assurance of safety, Realistic expectations, Empowerment, Directed to appropriate services), which reflects the patient's perceived needs at discharge. In previous research, the B‐PREPARED measure predicted emergency department (ED) visits for patients who had been recently hospitalized and had a high risk for readmission.[3] Second, the Care Transitions Measure‐3 (CTM‐3) was developed by Coleman et al. as a patient‐reported measure to discriminate between patients who were more likely to have an ED visit or readmission from those who did not. CTM‐3 has also been used to evaluate hospitals' level of care coordination and for public reporting purposes.[4, 5, 6] It has been endorsed by the National Quality Forum and incorporated into the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey provided to samples of recently hospitalized US patients.[7] However, recent evidence from an inpatient cohort of cardiovascular patients suggests the CTM‐3 overinflates care transition scores compared to the longer 15‐item CTM. In that cohort, the CTM‐3 could not differentiate between patients who did or did not have repeat ED visits or readmission.[8] Thus far, the B‐PREPARED and CTM‐3 measures have not been compared to one another directly.

In addition to the development of patient‐reported measures, hospitals increasingly employ administrative algorithms to predict likelihood of readmission.[9] A commonly used measure is the LACE index (Length of stay, Acuity, Comorbidity, and Emergency department use).[10] The LACE index predicted readmission and death within 30 days of discharge in a large cohort in Canada. In 2 retrospective studies of recently hospitalized patients in the United States, the LACE index's ability to discriminate between patients readmitted or not ranged from slightly better than chance to moderate (C statistic 0.56‐0.77).[11, 12]

It is unknown whether adding patient‐reported preparedness measures to commonly used readmission prediction scores increases the ability to predict readmission risk. We sought to determine whether the B‐PREPARED and CTM‐3 measures were predictive of readmission or death, as compared to the LACE index, in a large cohort of cardiovascular patients. In addition, we sought to determine the additional predictive and discriminative ability gained from administering the B‐PREPARED and CTM‐3 measures, while adjusting for the LACE index and other clinical factors. We hypothesized that: (1) higher preparedness scores on both measures would predict lower risk of readmission or death in a cohort of patients hospitalized with cardiac diagnoses; and (2) because it provides more specific and actionable information, the B‐PREPARED would discriminate readmission more accurately than CTM‐3, after controlling for clinical factors.

METHODS

Study Setting and Design

The Vanderbilt Inpatient Cohort Study (VICS) is a prospective study of patients admitted with cardiovascular disease to Vanderbilt University Hospital. The purpose of VICS is to investigate the impact of patient and social factors on postdischarge health outcomes such as quality of life, unplanned hospital utilization, and mortality. The rationale and design of VICS are detailed elsewhere.[13] Briefly, participants completed a baseline interview while hospitalized, and follow‐up phone calls were conducted within 2 to 9 days and at approximately 30 and 90 days postdischarge. During the first follow‐up call conducted by research assistants, we collected preparedness for discharge data utilizing the 2 measures described below. After the 90‐day phone call, we collected healthcare utilization since the index admission. The study was approved by the Vanderbilt University Institutional Review Board.

Patients

Eligibility screening shortly after admission identified patients with acute decompensated heart failure (ADHF) and/or an intermediate or high likelihood of acute coronary syndrome (ACS) per a physician's review of the clinical record. Exclusion criteria included: age <18 years, non‐English speaker, unstable psychiatric illness, delirium, low likelihood of follow‐up (eg, no reliable telephone number), on hospice, or otherwise too ill to complete an interview. To be included in these analyses, patients must have completed the preparedness for discharge measurements during the first follow‐up call. Patients who died before discharge or before completing the follow‐up call were excluded.

Preparedness for Discharge Measures (Patient‐Reported Data)

Preparedness for discharge was assessed using the 11‐item B‐PREPARED and the 3‐item CTM‐3.

The B‐PREPARED measures how prepared patients felt leaving the hospital with regard to: self‐care information for medications and activity, equipment/community services needed, and confidence in managing one's health after hospitalization. The B‐PREPARED measure has good internal consistency reliability (Cronbach's = 0.76) and has been validated in patients of varying age within a week of discharge. Preparedness is the sum of responses to all 11 questions, with a range of 0 to 22. Higher scores reflect increased preparedness for discharge.[3]

The CTM‐3 asks patients to rate how well their preferences were considered regarding transitional needs, as well as their understanding of postdischarge self‐management and the purpose of their medications, each on a 4‐point response scale (strongly disagree to strongly agree). The sum of the 3 responses quantifies the patient's perception of the quality of the care transition at discharge (Cronbach's = 0.86,[14] 0.92 in a cohort similar to ours[8]). Scores range from 3 to 12, with higher score indicating more preparedness. Then, the sum is transformed to a 0 to 100 scale.[15]

Clinical Readmission Risk Measures (Medical Record Data)

The LACE index, published by Van Walraven et al.,[10] takes into account 4 categories of clinical data: length of hospital stay, acuity of event, comorbidities, and ED visits in the prior 6 months. More specifically, a diagnostic code‐based, modified version of the Charlson Comorbidity Index was used to calculate the comorbidity score. These clinical criteria were obtained from an administrative database and weighted according to the methods used by Van Walraven et al. An overall score was calculated on a scale of 0 to 19, with higher scores indicating higher risk of readmission or death within 30 days.

From medical records, we also collected patients' demographic data including age, race, and gender, and diagnosis of ACS, ADHF, or both at hospital admission.

Outcome Measures

Healthcare utilization data were obtained from the index hospital as well as outside facilities. The electronic medical records from Vanderbilt University Hospital provided information about healthcare utilization at Vanderbilt 90 days after initial discharge. We also used Vanderbilt records to see if patients were transferred to Vanderbilt from other hospitals or if patients visited other hospitals before or after enrollment. We supplemented this with patient self‐report during the follow‐up telephone calls (at 30 and 90 days after initial discharge) so that any additional ED and hospital visits could be captured. Mortality data were collected from medical records, Social Security data, and family reports. The main outcome was time to first unplanned hospital readmission or death within 30 and 90 days of discharge.

Analysis

To describe our sample, we summarized categorical variables with percentages and continuous variables with percentiles. To test for evidence of unadjusted covariate‐outcome relationships, we used Pearson 2 and Wilcoxon rank sum tests for categorical and continuous covariates, respectively.

For the primary analyses we used Cox proportional hazard models to examine the independent associations between the prespecified predictors for patient‐reported preparedness and time to first unplanned readmission or death within 30 and 90 days of discharge. For each outcome (30‐ and 90‐day readmission or death), we fit marginal models separately for each of the B‐PREPARED, CTM‐3, and LACE scores. We then fit multivariable models that used both preparedness measures as well as age, gender, race, and diagnosis (ADHF and/or ACS), variables available to clinicians when patients are admitted. When fitting the multivariable models, we did not find strong evidence of nonlinear effects; therefore, only linear effects are reported. To facilitate comparison of effects, we scaled continuous variables by their interquartile range (IQR). The associated, exponentiated regression parameter estimates may therefore be interpreted as hazard ratios for readmission or death per IQR change in each predictor. In addition to parameter estimation, we computed the C index to evaluate capacity for the model to discriminate those who were and were not readmitted or died. All analyses were conducted in R version 3.1.2 (R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

From the cohort of 1239 patients (Figure 1), 64%, 28%, and 7% of patients were hospitalized with ACS, ADHF, or both, respectively (Table 1). Nearly 45% of patients were female, 83% were white, and the median age was 61 years (IQR 5269). The median length of stay was 3 days (IQR 25). The median preparedness scores were high for both B‐PREPARED (21, IQR 1822) and CTM‐3 (77.8, IQR 66.7100). A total of 211 (17%) and 380 (31%) were readmitted or died within 30 and 90 days, respectively. The completion rate for the postdischarge phone calls was 88%.

Patient Characteristics
Death or Readmission Within 30 Days Death or Readmission Within 90 Days
Not Readmitted, N = 1028 Death/Readmitted, N = 211 P Value Not Readmitted, N = 859 Death/Readmitted, N = 380 P Value
  • NOTE: Continuous variables: summarize with the 5th:25th:50th:75th:95th. Categorical variables: summarize with the percentage and (N). Abbreviations: ACS, acute coronary syndromes; ADHF, acute decompensated heart failure; B‐PREPARED, Brief PREPARED (Prescriptions, Ready to re‐enter community, Education, Placement, Assurance of safety, Realistic expectations, Empowerment, Directed to appropriate services) CTM‐3, Care Transitions Measure‐3; LACE, Length of hospital stay, Acuity of event, Comorbidities, and ED visits in the prior 6 months; LOS, length of stay. *Pearson test. Wilcoxon test.

Gender, male 55.8% (574) 53.1% (112) 0.463* 56.3% (484) 53.2% (202) 0.298*
Female 44.2% (454) 46.9% (99) 43.7% (375) 46.8% (178)
Race, white 83.9% (860) 80.6% (170) 0.237* 86.0% (737) 77.3% (293) <0.001*
Race, nonwhite 16.1% (165) 19.4% (41) 14.0% (120) 22.7% (86)
Diagnosis ACS 68.0% (699) 46.4% (98) <0.001* 72.9% (626) 45.0% (171) <0.001*
ADHF 24.8% (255) 46.0% (97) 20.3% (174) 46.8% (178)
Both 7.2% (74) 7.6% (16) 6.9% (59) 8.2% (31)
Age 39.4:52:61:68:80 37.5:53.5:62:70:82 0.301 40:52:61:68:80 38:52:61 :70:82 0.651
LOS 1:2:3:5:10 1:3: 4:7.5:17 <0.001 1:2:3:5:9 1:3:4:7:15 <0.001
CTM‐3 55.6:66.7: 77.8:100:100 55.6:66.7:77.8:100 :100 0.305 55.6:66.7:88.9:100:100 55.6:66.7:77.8:100 :100 0.080
B‐PREPARED 12:18:21:22.:22 10:17:20:22:22 0.066 12:18:21:22:22 10:17:20 :22:22 0.030
LACE 1:4: 7:10 :14 3.5:7:10:13:17 <0.001 1:4:6: 9:14 3:7:10:13:16 <0.001
Figure 1
Study flow diagram. Abbreviations: ACS, acute coronary syndrome; ADHF, acute decompensated heart failure; VICS, Vanderbilt Inpatient Cohort Study.

B‐PREPARED and CTM‐3 were moderately correlated with one another (Spearman's = 0.40, P < 0.001). In bivariate analyses (Table 1), the association between B‐PREPARED and readmission or death was significant at 90 days (P = 0.030) but not 30 days. The CTM‐3 showed no significant association with readmission or death at either time point. The LACE score was significantly associated with rates of readmission at 30 and 90 days (P < 0.001).

Outcomes Within 30 Days of Discharge

When examining readmission or death within 30 days of discharge, simple unadjusted models 2 and 3 showed that the B‐PREPARED and LACE scores, respectively, were each significantly associated with time to first readmission or death (Table 2). Specifically, a 4‐point increase in the B‐PREPARED score was associated with a 16% decrease in the hazard of readmission or death (hazard ratio [HR] = 0.84, 95% confidence interval [CI]: 0.72 to 0.97). A 5‐point increase in the LACE score was associated with a 100% increase in the hazard of readmission or death (HR = 2.00, 95% CI: 1.72 to 2.32). In the multivariable model with both preparedness scores and diagnosis (model 4), the B‐PREPARED score (HR = 0.82, 95% CI: 0.70 to 0.97) was significantly associated with time to first readmission or death. In the full 30‐day model including B‐PREPARED, CTM‐3, LACE, age, gender, race, and diagnosis (model 5), only the LACE score (HR = 1.83, 95% CI: 1.54 to 2.18) was independently associated with time to readmission or death. Finally, the CTM‐3 did not predict 30‐day readmission or death in any of the models tested.

Cox Models: Time to Death or Readmission Within 30 Days of Index Hospitalization
Models HR (95% CI)* P Value C Index
  • NOTE: Abbreviations: ADHF, acute decompensated heart failure; B‐PREPARED, Brief PREPARED (Prescriptions, Ready to re‐enter community, Education, Placement, Assurance of safety, Realistic expectations, Empowerment, Directed to appropriate services); CI, confidence interval; CTM‐3, Care Transitions Measure‐3; HR, hazard ratio; LACE, Length of hospital stay, Acuity of event, Comorbidities, and Emergency department visits in the prior 6 months.

1. CTM (per 10‐point change) 0.95 (0.88 to 1.03) 0.257 0.523
2. B‐PREPARED (per 4‐point change) 0.84 (0.72 to 0.97) 0.017 0.537
3. LACE (per 5‐point change) 2.00 (1.72 to 2.32) <0.001 0.679
4. CTM (per 10‐point change) 1.00 (0.92 to 1.10) 0.935 0.620
B‐PREPARED (per 4‐point change) 0.82 (0.70 to 0.97) 0.019
ADHF only (vs ACS only) 2.46 (1.86 to 3.26) <0.001
ADHF and ACS (vs ACS only) 1.42 (0.84 to 2.42) 0.191
5. CTM (per 10‐point change) 1.02 (0.93 to 1.11) 0.722 0.692
B‐PREPARED (per 4 point change) 0.87 (0.74 to 1.03) 0.106
LACE (per 5‐point change) 1.83 (1.54 to 2.18) <0.001
ADHF only (vs ACS only) 1.51 (1.10 to 2.08) 0.010
ADHF and ACS (vs ACS only) 0.90 (0.52 to 1.55) 0.690
Age (per 10‐year change) 1.02 (0.92 to 1.14) 0.669
Female (vs male) 1.11 (0.85 to 1.46) 0.438
Nonwhite (vs white) 0.92 (0.64 to 1.30) 0.624

Outcomes Within 90 Days of Discharge

At 90 days after discharge, again the separate unadjusted models 2 and 3 demonstrated that the B‐PREPARED and LACE scores, respectively, were each significantly associated with time to first readmission or death, whereas the CTM‐3 model only showed marginal significance (Table 3). In the multivariable model with both preparedness scores and diagnosis (model 4), results were similar to 30 days as the B‐PREPARED score was significantly associated with time to first readmission or death. Lastly, in the full model (model 5) at 90 days, again the LACE score was significantly associated with time to first readmission or death. In addition, B‐PREPARED scores were associated with a significant decrease in risk of readmission or death (HR = 0.88, 95% CI: 0.78 to 1.00); CTM‐3 scores were not independently associated with outcomes.

Cox Models: Time to Death or Readmission Within 90 Days of Index Hospitalization
Model HR (95% CI)* P Value C Index
  • NOTE: Abbreviations: ADHF, acute decompensated heart failure; B‐PREPARED, Brief PREPARED (Prescriptions, Ready to re‐enter community, Education, Placement, Assurance of safety, Realistic expectations, Empowerment, Directed to appropriate services); CI, confidence interval; CTM‐3, Care Transitions Measure‐3; HR, hazard ratio; LACE, Length of hospital stay, Acuity of event, Comorbidities, and Emergency department visits in the prior 6 months.

1. CTM (per 10‐point change) 0.94 (0.89 to 1.00) 0.051 0.526
2. B‐PREPARED (per 4‐point change) 0.84 (0.75 to 0.94) 0.002 0.533
3. LACE (per 5‐point change) 2.03 (1.82 to 2.27) <0.001 0.683
4. CTM (per 10‐point change) 0.99 (0.93 to 1.06) 0.759 0.640
B‐PREPARED (per 4‐point change) 0.83 (0.74 to 0.94) 0.003
ADHF only (vs ACS only) 2.88 (2.33 to 3.56) <0.001
ADHF and ACS (vs ACS only) 1.62 (1.11 to 2.38) 0.013
5. CTM (per 10‐point change) 1.00 (0.94 to 1.07) 0.932 0.698
B‐PREPARED (per 4‐point change) 0.88 (0.78 to 1.00) 0.043
LACE (per 5‐point change) 1.76 (1.55 to 2.00) <0.001
ADHF only (vs ACS only) 1.76 (1.39 to 2.24) <0.001
ADHF and ACS (vs ACS only) 1.00 (0.67 to 1.50) 0.980
Age (per 10‐year change) 1.00 (0.93 to 1.09) 0.894
Female (vs male) 1.10 (0.90 to 1.35) 0.341
Nonwhite (vs white) 1.14 (0.89 to 1.47) 0.288

Tables 2 and 3 also display the C indices, or the discriminative ability of the models to differentiate whether or not a patient was readmitted or died. The range of the C index is 0.5 to 1, where values closer to 0.5 indicate random predictions and values closer to 1 indicate perfect prediction. At 30 days, the individual C indices for B‐PREPARED and CTM‐3 were only slightly better than chance (0.54 and 0.52, respectively) in their discriminative abilities. However, the C indices for the LACE score alone (0.68) and the multivariable model (0.69) including all 3 measures (ie, B‐PREPARED, CTM‐3, LACE), and clinical and demographic variables, had higher utility in discriminating patients who were readmitted/died or not. The 90‐day C indices were comparable in magnitude to those at 30 days.

DISCUSSION/CONCLUSION

In this cohort of patients hospitalized with cardiovascular disease, we compared 2 patient‐reported measures of preparedness for discharge, their association with time to death or readmission at 30 and 90 days, and their ability to discriminate patients who were or were not readmitted or died. Higher preparedness as measured by higher B‐PREPARED scores was associated with lower risk of readmission or death at 30 and 90 days after discharge in unadjusted models, and at 90 days in adjusted models. CTM‐3 was not associated with the outcome in any analyses. Lastly, the individual preparedness measures were not as strongly associated with readmission or death compared to the LACE readmission index alone.

How do our findings relate to the measurement of care transition quality? We consider 2 scenarios. First, if hospitals utilize the LACE index to predict readmission, then neither self‐reported measure of preparedness adds meaningfully to its predictive ability. However, hospital management may still find the B‐PREPARED and CTM‐3 useful as a means to direct care transition quality‐improvement efforts. These measures can instruct hospitals as to what areas their patients express the greatest difficulty or lack of preparedness and closely attend to patient needs with appropriate resources. Furthermore, the patient's perception of being prepared for discharge may be different than their actual preparedness. Their perceived preparedness may be affected by cognitive impairment, dissatisfaction with medical care, depression, lower health‐related quality of life, and lower educational attainment as demonstrated by Lau et al.[16] If a patient's perception of preparedness were low, it would behoove the clinician to investigate these other issues and address those that are mutable. Additionally, perceived preparedness may not correlate with the patient's understanding of their medical conditions, so it is imperative that clinicians provide prospective guidance about their probable postdischarge trajectory. If hospitals are not utilizing the LACE index, then perhaps using the B‐PREPARED, but not the CTM‐3, may be beneficial for predicting readmission.

How do our results fit with evidence from prior studies, and what do they mean in the context of care transitions quality? First, in the psychometric evaluation of the B‐PREPARED measure in a cohort of recently hospitalized patients, the mean score was 17.3, lower than the median of 21 in our cohort.[3] Numerous studies have utilized the CTM‐3 and the longer‐version CTM‐15. Though we cannot make a direct comparison, the median in our cohort (77.8) was on par with the means from other studies, which ranged from 63 to 82.[5, 17, 18, 19] Several studies also note ceiling effects with clusters of scores at the upper end of the scale, as did we. We conjecture that our cohort's preparedness scores may be higher because our institution has made concerted efforts to improve the discharge education for cardiovascular patients.

In a comparable patient population, the TRACE‐CORE (Transitions, Risks, and Actions in Coronary Events Center for Outcomes Research and Education) study is a cohort of more than 2200 patients with ACS who were administered the CTM‐15 within 1 month of discharge.[8] In that study, the median CTM‐15 score was 66.6, which is lower than our cohort. With regard to the predictive ability of the CTM‐3, they note that CTM‐3 scores did not differentiate between patients who were or were not readmitted or had emergency department visits. Our results support their concern that the CTM‐15 and by extension the CTM‐3, though adopted widely as part of HCAHPS, may not have sufficient ability to discriminate differences in patient outcomes or the quality of care transitions.

More recently, patient‐reported preparedness for discharge was assessed in a prospective cohort in Canada.[16] Lau et al. administered a single‐item measure of readiness at the time of discharge to general medicine patients, and found that lower readiness scores were also not associated with readmission or death at 30 days, when adjusted for the LACE index as we did.

We must acknowledge the limitations of our findings. First, our sample of recently discharged patients with cardiovascular disease is different than the community‐dwelling, underserved Americans hospitalized in the prior year, which served as the sample for reducing the CTM‐15 to 3 items.[5] This fact may explain why we did not find the CTM‐3 to be associated with readmission in our sample. Second, our analyses did not include extensive adjustment for patient‐related factors. Rather, our intention was to see how well the preparedness measures performed independently and compare their abilities to predict readmission, which is particularly relevant for clinicians who may not have all possible covariates in predicting readmission. Finally, because we limited the analyses to the patients who completed the B‐PREPARED and CTM‐3 measures (88% completion rate), we may not have data for: (1) very ill patients, who had a higher risk of readmission and least prepared, and were not able to answer the postdischarge phone call; and (2) very functional patients, who had a lower risk of readmission and were too busy to answer the postdischarge phone call. This may have limited the extremes in the spectrum of our sample.

Importantly, our study has several strengths. We report on the largest sample to date with results of both B‐PREPARED and CTM‐3. Moreover, we examined how these measures compared to a widely used readmission prediction tool, the LACE index. We had very high postdischarge phone call completion rates in the week following discharge. Furthermore, we had thorough assessment of readmission data through patient report, electronic medical record documentation, and collection of outside medical records.

Further research is needed to elucidate: (1) the ideal administration time of the patient‐reported measures of preparedness (before or after discharge), and (2) the challenges to the implementation of measures in healthcare systems. Remaining research questions center on the tradeoffs and barriers to implementing a longer measure like the 11‐item B‐PREPARED compared to a shorter measure like the CTM‐3. We do not know whether longer measures preclude their use by busy clinicians, though it provides more specific information about what patients feel they need at hospital discharge. Additionally, studies need to demonstrate the mutability of preparedness and the response of measures to interventions designed to improve the hospital discharge process.

In our sample of recently hospitalized cardiovascular patients, there was a statistically significant association between patient‐reported preparedness for discharged, as measured by B‐PREPARED, and readmissions/death at 30 and 90 days, but the magnitude of the association was very small. Furthermore, another patient‐reported preparedness measure, CTM‐3, was not associated with readmissions or death at either 30 or 90 days. Lastly, neither measure discriminated well between patients who were readmitted or not, and neither measure added meaningfully to the LACE index in terms of predicting 30‐ or 90‐day readmissions.

Disclosures

This study was supported by grant R01 HL109388 from the National Heart, Lung, and Blood Institute (Dr. Kripalani) and in part by grant UL1 RR024975‐01 from the National Center for Research Resources, and grant 2 UL1 TR000445‐06 from the National Center for Advancing Translational Sciences. Dr. Kripalani is a consultant to SAI Interactive and holds equity in Bioscape Digital, and is a consultant to and holds equity in PictureRx, LLC. Dr. Bell is supported by the National Institutes of Health (K23AG048347) and by the Eisenstein Women's Heart Fund. Dr. Vasilevskis is supported by the National Institutes of Health (K23AG040157) and the Geriatric Research, Education and Clinical Center. Dr. Mixon is a Veterans Affairs Health Services Research and Development Service Career Development awardee (12‐168) at the Nashville Department of Veterans Affairs. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The funding agency was not involved in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. All authors had full access to all study data and had a significant role in writing the manuscript. The contents do not represent the views of the US Department of Veterans Affairs or the United States government. Dr. Kripalani is a consultant to and holds equity in PictureRx, LLC.

References
  1. Centers for Medicare 9(9):598603.
  2. Graumlich JF, Novotny NL, Aldag JC. Brief scale measuring patient preparedness for hospital discharge to home: psychometric properties. J Hosp Med. 2008;3(6):446454.
  3. Coleman EA, Mahoney E, Parry C. Assessing the quality of preparation for posthospital care from the patient's perspective: the care transitions measure. Med Care. 2005;43(3):246255.
  4. Parry C, Mahoney E, Chalmers SA, Coleman EA. Assessing the quality of transitional care: further applications of the care transitions measure. Med Care. 2008;46(3):317322.
  5. Coleman EA, Parry C, Chalmers SA, Chugh A, Mahoney E. The central role of performance measurement in improving the quality of transitional care. Home Health Care Serv Q. 2007;26(4):93104.
  6. Centers for Medicare 3:e001053.
  7. Kansagara D, Englander H, Salanitro AH, et al. Risk prediction models for hospital readmission: a systematic review. JAMA. 2011;306(15):16881698.
  8. Walraven C, Dhalla IA, Bell C, et al. Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community. CMAJ. 2010;182(6):551557.
  9. Wang H, Robinson RD, Johnson C, et al. Using the LACE index to predict hospital readmissions in congestive heart failure patients. BMC Cardiovasc Disord. 2014;14:97.
  10. Spiva L, Hand M, VanBrackle L, McVay F. Validation of a predictive model to identify patients at high risk for hospital readmission. J Healthc Qual. 2016;38(1):3441.
  11. Meyers AG, Salanitro A, Wallston KA, et al. Determinants of health after hospital discharge: rationale and design of the Vanderbilt Inpatient Cohort Study (VICS). BMC Health Serv Res. 2014;14:10.
  12. Coleman EA. CTM frequently asked questions. Available at: http://caretransitions.org/tools-and-resources/. Accessed January 22, 2016.
  13. Coleman EA. Instructions for scoring the CTM‐3. Available at: http://caretransitions.org/tools-and-resources/. Accessed January 22, 2016.
  14. Lau D, Padwal RS, Majumdar SR, et al. Patient‐reported discharge readiness and 30‐day risk of readmission or death: a prospective cohort study. Am J Med. 2016;129:8995.
  15. Parrish MM, O'Malley K, Adams RI, Adams SR, Coleman EA. Implementaiton of the Care Transitions Intervention: sustainability and lessons learned. Prof Case Manag. 2009;14(6):282293.
  16. Englander H, Michaels L, Chan B, Kansagara D. The care transitions innovation (C‐TraIn) for socioeconomically disadvantaged adults: results of a cluster randomized controlled trial. J Gen Intern Med. 2014;29(11):14601467.
  17. Record JD, Niranjan‐Azadi A, Christmas C, et al. Telephone calls to patients after discharge from the hospital: an important part of transitions of care. Med Educ Online. 2015;29(20):26701.
References
  1. Centers for Medicare 9(9):598603.
  2. Graumlich JF, Novotny NL, Aldag JC. Brief scale measuring patient preparedness for hospital discharge to home: psychometric properties. J Hosp Med. 2008;3(6):446454.
  3. Coleman EA, Mahoney E, Parry C. Assessing the quality of preparation for posthospital care from the patient's perspective: the care transitions measure. Med Care. 2005;43(3):246255.
  4. Parry C, Mahoney E, Chalmers SA, Coleman EA. Assessing the quality of transitional care: further applications of the care transitions measure. Med Care. 2008;46(3):317322.
  5. Coleman EA, Parry C, Chalmers SA, Chugh A, Mahoney E. The central role of performance measurement in improving the quality of transitional care. Home Health Care Serv Q. 2007;26(4):93104.
  6. Centers for Medicare 3:e001053.
  7. Kansagara D, Englander H, Salanitro AH, et al. Risk prediction models for hospital readmission: a systematic review. JAMA. 2011;306(15):16881698.
  8. Walraven C, Dhalla IA, Bell C, et al. Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community. CMAJ. 2010;182(6):551557.
  9. Wang H, Robinson RD, Johnson C, et al. Using the LACE index to predict hospital readmissions in congestive heart failure patients. BMC Cardiovasc Disord. 2014;14:97.
  10. Spiva L, Hand M, VanBrackle L, McVay F. Validation of a predictive model to identify patients at high risk for hospital readmission. J Healthc Qual. 2016;38(1):3441.
  11. Meyers AG, Salanitro A, Wallston KA, et al. Determinants of health after hospital discharge: rationale and design of the Vanderbilt Inpatient Cohort Study (VICS). BMC Health Serv Res. 2014;14:10.
  12. Coleman EA. CTM frequently asked questions. Available at: http://caretransitions.org/tools-and-resources/. Accessed January 22, 2016.
  13. Coleman EA. Instructions for scoring the CTM‐3. Available at: http://caretransitions.org/tools-and-resources/. Accessed January 22, 2016.
  14. Lau D, Padwal RS, Majumdar SR, et al. Patient‐reported discharge readiness and 30‐day risk of readmission or death: a prospective cohort study. Am J Med. 2016;129:8995.
  15. Parrish MM, O'Malley K, Adams RI, Adams SR, Coleman EA. Implementaiton of the Care Transitions Intervention: sustainability and lessons learned. Prof Case Manag. 2009;14(6):282293.
  16. Englander H, Michaels L, Chan B, Kansagara D. The care transitions innovation (C‐TraIn) for socioeconomically disadvantaged adults: results of a cluster randomized controlled trial. J Gen Intern Med. 2014;29(11):14601467.
  17. Record JD, Niranjan‐Azadi A, Christmas C, et al. Telephone calls to patients after discharge from the hospital: an important part of transitions of care. Med Educ Online. 2015;29(20):26701.
Issue
Journal of Hospital Medicine - 11(9)
Issue
Journal of Hospital Medicine - 11(9)
Page Number
603-609
Page Number
603-609
Article Type
Display Headline
Preparedness for hospital discharge and prediction of readmission
Display Headline
Preparedness for hospital discharge and prediction of readmission
Sections
Article Source
© 2016 Society of Hospital Medicine
Disallow All Ads
Correspondence Location
Address for correspondence and reprint requests: Amanda S. Mixon, MD, Suite 6000 MCE, North Tower, 1215 21st Avenue South, Nashville, TN 37232; Telephone: 615‐936‐3710; Fax: 615‐936‐1269; E‐mail: amanda.s.mixon@vanderbilt.edu
Content Gating
Gated (full article locked unless allowed per User)
Gating Strategy
First Peek Free
Article PDF Media
Media Files

Medical scribes: How do their notes stack up?

Article Type
Changed
Thu, 03/28/2019 - 15:12
Display Headline
Medical scribes: How do their notes stack up?

ABSTRACT

Objective Medical scribes are increasingly employed to improve physician efficiency with regard to the electronic medical record (EMR). The impact of scribes on the quality of outpatient visit notes is not known. To assess the effect, we conducted a retrospective review of ambulatory progress notes written before and after 8 practice sites transitioned to the use of medical assistants as scribes.

Methods The Physician Documentation Quality Instrument 9 (PDQI-9) was used to compare the quality of outpatient progress notes written by medical assistant scribes with the quality of notes written by 18 primary care physicians working without a scribe. The notes pertained to diabetes encounters and same-day appointments and were written during the 3 to 6 months preceding the use of scribes (pre-scribe period) and the 3 to 6 months after scribes were employed (scribe period).

Results One hundred eight notes from the pre-scribe period and 109 from the scribe period were reviewed. Scribed notes were rated higher in overall quality than unscribed notes (mean total PDQI-9 score 30.3 for scribed notes vs 28.9 for nonscribed notes; P=.01) and more up-to-date, thorough, useful, and comprehensible. The differences were limited to diabetes encounters. For same-day appointments, scribed and nonscribed notes did not differ in quality. The total word count of all scribed and nonscribed notes was similar (mean words 618, standard deviation (SD) 273 for scribed notes vs 558 words, SD 289 for nonscribed notes; P=.12).

Conclusions In this retrospective review, ambulatory notes were of higher quality when medical assistants acted as scribes than when physicians wrote them alone, at least for diabetes visits. Our findings may not apply to professional scribes who are not part of the clinical care team. As the use of medical scribes expands, additional studies should examine the impact of scribes on other aspects of care quality.

Team-based models of primary care delivery may incorporate medical scribes to improve efficiency of electronic documentation.1-4 The employment of medical scribes has grown rapidly, and it is estimated that within several years there may be one scribe for every 9 physicians.3

Accurate documentation is important to providing high-quality patient care but can take a significant amount of time. Attending physicians have been estimated to spend as long as 52 minutes per day authoring notes.5 Medical scribes can help physicians improve the efficiency of electronic documentation6 and save time.2 Using scribes can also improve physician productivity7-10 and thereby potentially increase access to care. The impact of scribes on the quality of outpatient visit notes, however, is unknown.

A team-based care delivery model in our health system’s primary care clinics uses medical assistants to scribe notes during the outpatient encounter. We hypothesized that outpatient notes written by medical assistant scribes would be of similar quality to notes written by the same group of physicians without a scribe.

METHODS

Study design and sample

We conducted a retrospective review of ambulatory notes from 18 primary care physicians at 8 practice sites in our health system who had adopted a care model in which medical assistants act as scribes. Each physician works with 2 medical assistants. To train for the new model, the physician and medical assistants participated in 2 training sessions of 2 hours each and a half day of clinic observation and evaluation with a project manager.

Scribed notes were more up-to-date, thorough, useful, and comprehensible for diabetes encounters.

Of the 18 primary care physicians included in this study, none had less than one year of experience in our health system. Tenure ranged from one to 24 years with a mean of 11.3 years.

For each participating provider, we requested all available outpatient progress notes with either an International Classification of Diseases, 9th revision (ICD-9) code for diabetes or a designation of “same day” for the 3 to 6 months preceding the use of scribes (pre-scribe period) and the 3 to 6 months after employing scribes (scribe period). We chose diabetes encounters as examples of notes addressing chronic disease management and same-day encounters as examples of problem-focused notes because these 2 types of encounters are common in outpatient primary care practice.

Note quality was evaluated using the Physician Documentation Quality Instrument 9 (PDQI-9), a validated instrument designed for this purpose, comprising 9 items rated subjectively on a 5-point Likert scale (1= not at all, 5= extremely). The items assess whether notes are up-to-date, accurate, thorough, useful, organized, comprehensible, succinct, synthesized, and internally consistent.11,12 The PDQI-9 has been applied previously in inpatient12 and outpatient settings.13

While the PDQI-9 is a validated tool, it relies on subjective ratings of note quality by the reviewer. To control for the subjective nature of the ratings, an experienced internist and an internal medicine resident coded 10 progress notes separately using the PDQI-9 and discussed the results. The process was repeated for a total of 20 notes, after which consensus was reached with >70% agreement on each attribute of the PDQI-9, suggesting that the resident’s ratings were reliable when compared with those of an experienced practicing physician.

 

 

The resident then evaluated a random sample of notes written by each physician for diabetes or same-day appointments in the pre-scribe and scribe periods. Word counts for the entire note were measured. The notes used to establish the reliability of the ratings were excluded from the analysis for this study.

Data analysis

We used linear mixed-effects models to examine note quality measures by adjusting for possible correlations of notes from the same physician. Least-squares estimates were derived; the results were not adjusted for multiple comparisons.

RESULTS

One hundred eight notes from the pre-scribe period and 109 notes from the scribe period were reviewed. Compared with notes written by a physician alone, scribed notes were rated slightly higher in overall quality (mean total PDQI-9 score 30.3 for scribe notes vs 28.9 for pre-scribe notes; P=.01) and more up-to-date, thorough, useful, and comprehensible (TABLES 1 AND 2). The differences were limited to diabetes encounters. For same day appointments, scribed notes did not differ in quality from nonscribed notes (TABLE 2). Total word count did not vary significantly between all scribe and pre-scribe notes (mean words 618, SD 273 for scribed notes vs 558 words, SD 289 for nonscribed notes; P=.12).

DISCUSSION

In this retrospective review of ambulatory notes, progress notes written by medical assistant scribes were of higher quality than notes physicians wrote alone, at least for diabetes visits. Scribe and pre-scribe notes were of similar quality for problem-focused same-day visits. This is the first study of which we are aware that compares the quality of scribed notes with notes written by physicians.

Quality scribe notes can save physician time. The progress note is an important vehicle for describing care provided and transferring information among physicians caring for the same patient. Writing a note, however, adds a considerable amount of time to the physician’s workflow. Using a scribe can decrease the time burden of note writing, and if scribed notes are of similar or better quality, this practice innovation can allow the physician to focus more on clinical than clerical tasks.

Over-documentation is a possible concern. While implementation of the EMR may improve certain aspects of quality of care delivered14,15 and note quality,16 concern has been raised about over-documentation related to the connection between documentation and reimbursement.17 In our study, we found that physician notes and scribed notes for both diabetes and same-day encounters often used EMR-based note templates, which can lead to over-documentation.

Future EMR development might best focus on planned utilization by physician-scribe teams.

In general, both physician and scribed notes were rated to be of average to low quality because none of the mean scores on the 9 individual components of the PDQI-9 reached 4.0. Scribed notes were not inaccurate and had word counts similar to physician notes.

Scribing has potential drawbacks—and benefits. Drawbacks to scribing have not been well-studied. It has been suggested that using scribes to work around the EMR may actually hinder its further advancement because scribing insulates physicians from the inefficiencies of current EMRs and will not spur demands for improvements.3 Inaccurate or poor-quality notes could represent another downside to scribing, although concern about the quality of notes has not been documented. Our results suggest the opposite may be true.

Note quality has not been associated with quality of care as assessed by clinical quality scores,13 but using scribes may improve the quality of care in other ways. For example, the EMR may negatively affect patient-physician communication,18,19 and freeing the physician from documentation may improve the interaction.8,20 Incorporating scribing into practice may also improve the physician experience,9,10,21,22 a possible benefit that we did not measure.

We also did not measure the cost of using a scribe to assist in EMR documentation compared with the cost of physician time spent in performing this task. If the scribe model were associated with cost savings through increased physician productivity, as well as improved physician experience, future EMR development might best focus on planned utilization by physician-scribe teams.

Study limitations. The study was conducted in a single health system, although at 8 different practice sites. The sites all used the same EMR, but templates used for documentation could be individualized by the physician and medical assistant team, so our findings may reflect variation in template design. Our analysis did adjust for possible correlations of notes from the same physician. The selection of note types in our study may make our results less generalizable to other encounter types. Our sample was not large enough to detect variations in note quality among different providers and scribes.

 

 

The ratings on the PDQI-9 may be subjective, and the reviewers were not blinded to whether a scribe was used to write the note. The differences in PDQI-9 scores were small. Although statistically significant, they may not significantly affect clinical practice. Our care model is unique in that scribes are active members of the clinical care team; the higher quality of scribed notes we found may not apply to professional scribes who are not part of the team.

Future research directions. In our study, medical assistants acting as scribes composed progress notes of similar or higher quality than physicians who wrote notes alone, although all notes were of generally average quality. As the use of scribes in medicine expands, additional studies should examine the impact of scribes on primary care workflow, quality and cost of care delivered, and quality of physician experience.

CORRESPONDENCE
Anita D. Misra-Hebert, MD, MPH, Center for Value-Based Care Research, Medicine Institute, 9500 Euclid Avenue, G10, Cleveland, OH 44195; misraa@ccf.org.

References

1. Bodenheimer T, Willard-Grace R, Ghorob A. Expanding the roles of medical assistants: Who does what in primary care? JAMA Intern Med. 2014;174:1025-1026.

2. Reuben DB, Knudsen J, Senelick W, et al. The effect of a physician partner program on physician efficiency and patient satisfaction. JAMA Intern Med. 2014;174:1190-1193.

3. Gellert GA, Ramirea R, Webster S. The rise of the medical scribe industry: Implications for the advancement of electronic health records. JAMA. 2015;313:1315-1316.

4. Shultz CG, Holmstrom HL. The use of medical scribes in health care settings: a systematic review and future directions. J Am Board Fam Med. 2015;28:371-381.

5. Hripcsak G, Vawdrey DK, Fred MR, et al. Use of electronic clinical documentation: time spent and team interactions. J Am Med Inform Assoc. 2011;18:112-117.

6. Silverman L. Scribes Are Back, Helping Doctors Tackle Electronic Medical Records. NPR.org. Available at: www.npr.org/blogs/health/2014/04/21/303406306/scribes-are-back-helping-doctors-tackle-electronic-medical-records. Accessed April 23, 2014.

7. Arya R, Salovich DM, Ohman-Strickland P, et al. Impact of scribes on performance indicators in the emergency department. Acad Emerg Med. 2010;17:490-494.

8. Bank AJ, Obetz C, Konrardy A, et al. Impact of scribes on patient interaction, productivity, and revenue in a cardiology clinic: a prospective study. Clinicoecon Outcomes Res. 2013;5:399-406.

9. Bastani A, Shaqiri B, Palomba K, et al. An ED scribe program is able to improve throughput time and patient satisfaction. Am J Emerg Med. 2014;32:399-402.

10. Allen B, Banapoor B, Weeks EC, et al. An assessment of emergency department throughput and provider satisfaction after the implementation of a scribe program. Advances in Emergency Medicine. 2014;2014:e517319.

11. Stetson PD, Morrison FP, Bakken S, et al. Preliminary development of the Physician Documentation Quality Instrument. J Am Med Inform Assoc. 2008;15:534-541.

12. Stetson PD, Bakken S, Wrenn JO, et al. Assessing electronic note quality using the Physician Documentation Quality Instrument (PDQI-9). Appl Clin Inform. 2012;3:164-174.

13. Edwards ST, Neri PM, Volk LA, et al. Association of note quality and quality of care: a cross-sectional study. BMJ Qual Saf. 2013;23:406-413.

14. Schiff GD, Bates DW. Can electronic clinical documentation help prevent diagnostic errors? N Engl J Med. 2010;362:1066-1069.

15. Samal L, Wright A, Healey MJ, et al. Meaningful use and quality of care. JAMA Intern Med.  2014;174:997-998.

16. Burke HB, Sessums LL, Hoang A, et al. Electronic health records improve clinical note quality. J Am Med Inform Assoc. 2015;22:199-205.

17. Sheehy AM, Weissburg DJ, Dean SM. The role of copy-and-paste in the hospital electronic health record. JAMA Intern Med. 2014;174:1217-1218.

18. Shachak A, Hadas-Dayagi M, Ziv A, et al. Primary care physicians’ use of an electronic medical record system: a cognitive task analysis. J Gen Intern Med. 2009;24:341-348.

19. Shachak A, Reis S. The impact of electronic medical records on patient-doctor communication during consultation: a narrative literature review. J Eval Clin Pract. 2009;15:641-649.

20. Misra-Hebert AD, Rabovsky A, Yan C, et al. A team-based model of primary care delivery and physician-patient interaction. Am J Med. 2015;128:1025-1028.

21. Sinsky CA, Willard-Grace R, Schutzbank AM, et al. In search of joy in practice: a report of 23 high-functioning primary care practices. Ann Fam Med. 2013;11:272-278.

22. Koshy S, Feustel PJ, Hong M, et al. Scribes in an ambulatory urology practice: patient and physician satisfaction. J Urol. 2010;184:258-262.

Article PDF
Author and Disclosure Information

Anita D. Misra-Hebert, MD, MPH
Linda Amah, MD
Andrew Rabovsky, BS
Shannon Morrison, MS
Marven Cantave
Bo Hu, PhD
Christine A. Sinsky, MD
Michael B. Rothberg, MD, MPH
Center for Value-Based Care Research, Medicine Institute, Cleveland Clinic, Ohio (Drs. Misra-Hebert and Rothberg); Department of Hospital Medicine, Bridgeport Hospital-Yale New Haven Health, Bridgeport, Conn (Dr. Amah); Case Western Reserve University College of Medicine, Cleveland, Ohio (Mr. Rabovsky); Department of Quantitative Health Sciences (Ms. Morrison, Dr. Hu), Cleveland Clinic, Ohio; Case Western Reserve University, Cleveland, Ohio (Mr. Cantave); American Medical Association, Chicago, Ill (Dr. Sinsky)
misraa@ccf.org

The authors reported no potential conflict of interest relevant to this article.

The data reported here were presented as a poster presentation at the Society of General Internal Medicine’s national meeting in Toronto, Canada on April 24, 2015.

Issue
The Journal of Family Practice - 65(3)
Publications
Topics
Page Number
155-159
Legacy Keywords
medical scribes, electronic medical records, EMRs, electronic health records, EHRs, practice management, Anita D. Misra-Hebert, MD, MPH, Linda Amah, MD, Andrew Rabovsky, MD, Shannon Morrison, MS, Marven Cantave, Bo Hu, PhD, Christine A. Sinsky, MD, Michael B. Rothberg, MD, MPH
Sections
Author and Disclosure Information

Anita D. Misra-Hebert, MD, MPH
Linda Amah, MD
Andrew Rabovsky, BS
Shannon Morrison, MS
Marven Cantave
Bo Hu, PhD
Christine A. Sinsky, MD
Michael B. Rothberg, MD, MPH
Center for Value-Based Care Research, Medicine Institute, Cleveland Clinic, Ohio (Drs. Misra-Hebert and Rothberg); Department of Hospital Medicine, Bridgeport Hospital-Yale New Haven Health, Bridgeport, Conn (Dr. Amah); Case Western Reserve University College of Medicine, Cleveland, Ohio (Mr. Rabovsky); Department of Quantitative Health Sciences (Ms. Morrison, Dr. Hu), Cleveland Clinic, Ohio; Case Western Reserve University, Cleveland, Ohio (Mr. Cantave); American Medical Association, Chicago, Ill (Dr. Sinsky)
misraa@ccf.org

The authors reported no potential conflict of interest relevant to this article.

The data reported here were presented as a poster presentation at the Society of General Internal Medicine’s national meeting in Toronto, Canada on April 24, 2015.

Author and Disclosure Information

Anita D. Misra-Hebert, MD, MPH
Linda Amah, MD
Andrew Rabovsky, BS
Shannon Morrison, MS
Marven Cantave
Bo Hu, PhD
Christine A. Sinsky, MD
Michael B. Rothberg, MD, MPH
Center for Value-Based Care Research, Medicine Institute, Cleveland Clinic, Ohio (Drs. Misra-Hebert and Rothberg); Department of Hospital Medicine, Bridgeport Hospital-Yale New Haven Health, Bridgeport, Conn (Dr. Amah); Case Western Reserve University College of Medicine, Cleveland, Ohio (Mr. Rabovsky); Department of Quantitative Health Sciences (Ms. Morrison, Dr. Hu), Cleveland Clinic, Ohio; Case Western Reserve University, Cleveland, Ohio (Mr. Cantave); American Medical Association, Chicago, Ill (Dr. Sinsky)
misraa@ccf.org

The authors reported no potential conflict of interest relevant to this article.

The data reported here were presented as a poster presentation at the Society of General Internal Medicine’s national meeting in Toronto, Canada on April 24, 2015.

Article PDF
Article PDF

ABSTRACT

Objective Medical scribes are increasingly employed to improve physician efficiency with regard to the electronic medical record (EMR). The impact of scribes on the quality of outpatient visit notes is not known. To assess the effect, we conducted a retrospective review of ambulatory progress notes written before and after 8 practice sites transitioned to the use of medical assistants as scribes.

Methods The Physician Documentation Quality Instrument 9 (PDQI-9) was used to compare the quality of outpatient progress notes written by medical assistant scribes with the quality of notes written by 18 primary care physicians working without a scribe. The notes pertained to diabetes encounters and same-day appointments and were written during the 3 to 6 months preceding the use of scribes (pre-scribe period) and the 3 to 6 months after scribes were employed (scribe period).

Results One hundred eight notes from the pre-scribe period and 109 from the scribe period were reviewed. Scribed notes were rated higher in overall quality than unscribed notes (mean total PDQI-9 score 30.3 for scribed notes vs 28.9 for nonscribed notes; P=.01) and more up-to-date, thorough, useful, and comprehensible. The differences were limited to diabetes encounters. For same-day appointments, scribed and nonscribed notes did not differ in quality. The total word count of all scribed and nonscribed notes was similar (mean words 618, standard deviation (SD) 273 for scribed notes vs 558 words, SD 289 for nonscribed notes; P=.12).

Conclusions In this retrospective review, ambulatory notes were of higher quality when medical assistants acted as scribes than when physicians wrote them alone, at least for diabetes visits. Our findings may not apply to professional scribes who are not part of the clinical care team. As the use of medical scribes expands, additional studies should examine the impact of scribes on other aspects of care quality.

Team-based models of primary care delivery may incorporate medical scribes to improve efficiency of electronic documentation.1-4 The employment of medical scribes has grown rapidly, and it is estimated that within several years there may be one scribe for every 9 physicians.3

Accurate documentation is important to providing high-quality patient care but can take a significant amount of time. Attending physicians have been estimated to spend as long as 52 minutes per day authoring notes.5 Medical scribes can help physicians improve the efficiency of electronic documentation6 and save time.2 Using scribes can also improve physician productivity7-10 and thereby potentially increase access to care. The impact of scribes on the quality of outpatient visit notes, however, is unknown.

A team-based care delivery model in our health system’s primary care clinics uses medical assistants to scribe notes during the outpatient encounter. We hypothesized that outpatient notes written by medical assistant scribes would be of similar quality to notes written by the same group of physicians without a scribe.

METHODS

Study design and sample

We conducted a retrospective review of ambulatory notes from 18 primary care physicians at 8 practice sites in our health system who had adopted a care model in which medical assistants act as scribes. Each physician works with 2 medical assistants. To train for the new model, the physician and medical assistants participated in 2 training sessions of 2 hours each and a half day of clinic observation and evaluation with a project manager.

Scribed notes were more up-to-date, thorough, useful, and comprehensible for diabetes encounters.

Of the 18 primary care physicians included in this study, none had less than one year of experience in our health system. Tenure ranged from one to 24 years with a mean of 11.3 years.

For each participating provider, we requested all available outpatient progress notes with either an International Classification of Diseases, 9th revision (ICD-9) code for diabetes or a designation of “same day” for the 3 to 6 months preceding the use of scribes (pre-scribe period) and the 3 to 6 months after employing scribes (scribe period). We chose diabetes encounters as examples of notes addressing chronic disease management and same-day encounters as examples of problem-focused notes because these 2 types of encounters are common in outpatient primary care practice.

Note quality was evaluated using the Physician Documentation Quality Instrument 9 (PDQI-9), a validated instrument designed for this purpose, comprising 9 items rated subjectively on a 5-point Likert scale (1= not at all, 5= extremely). The items assess whether notes are up-to-date, accurate, thorough, useful, organized, comprehensible, succinct, synthesized, and internally consistent.11,12 The PDQI-9 has been applied previously in inpatient12 and outpatient settings.13

While the PDQI-9 is a validated tool, it relies on subjective ratings of note quality by the reviewer. To control for the subjective nature of the ratings, an experienced internist and an internal medicine resident coded 10 progress notes separately using the PDQI-9 and discussed the results. The process was repeated for a total of 20 notes, after which consensus was reached with >70% agreement on each attribute of the PDQI-9, suggesting that the resident’s ratings were reliable when compared with those of an experienced practicing physician.

 

 

The resident then evaluated a random sample of notes written by each physician for diabetes or same-day appointments in the pre-scribe and scribe periods. Word counts for the entire note were measured. The notes used to establish the reliability of the ratings were excluded from the analysis for this study.

Data analysis

We used linear mixed-effects models to examine note quality measures by adjusting for possible correlations of notes from the same physician. Least-squares estimates were derived; the results were not adjusted for multiple comparisons.

RESULTS

One hundred eight notes from the pre-scribe period and 109 notes from the scribe period were reviewed. Compared with notes written by a physician alone, scribed notes were rated slightly higher in overall quality (mean total PDQI-9 score 30.3 for scribe notes vs 28.9 for pre-scribe notes; P=.01) and more up-to-date, thorough, useful, and comprehensible (TABLES 1 AND 2). The differences were limited to diabetes encounters. For same day appointments, scribed notes did not differ in quality from nonscribed notes (TABLE 2). Total word count did not vary significantly between all scribe and pre-scribe notes (mean words 618, SD 273 for scribed notes vs 558 words, SD 289 for nonscribed notes; P=.12).

DISCUSSION

In this retrospective review of ambulatory notes, progress notes written by medical assistant scribes were of higher quality than notes physicians wrote alone, at least for diabetes visits. Scribe and pre-scribe notes were of similar quality for problem-focused same-day visits. This is the first study of which we are aware that compares the quality of scribed notes with notes written by physicians.

Quality scribe notes can save physician time. The progress note is an important vehicle for describing care provided and transferring information among physicians caring for the same patient. Writing a note, however, adds a considerable amount of time to the physician’s workflow. Using a scribe can decrease the time burden of note writing, and if scribed notes are of similar or better quality, this practice innovation can allow the physician to focus more on clinical than clerical tasks.

Over-documentation is a possible concern. While implementation of the EMR may improve certain aspects of quality of care delivered14,15 and note quality,16 concern has been raised about over-documentation related to the connection between documentation and reimbursement.17 In our study, we found that physician notes and scribed notes for both diabetes and same-day encounters often used EMR-based note templates, which can lead to over-documentation.

Future EMR development might best focus on planned utilization by physician-scribe teams.

In general, both physician and scribed notes were rated to be of average to low quality because none of the mean scores on the 9 individual components of the PDQI-9 reached 4.0. Scribed notes were not inaccurate and had word counts similar to physician notes.

Scribing has potential drawbacks—and benefits. Drawbacks to scribing have not been well-studied. It has been suggested that using scribes to work around the EMR may actually hinder its further advancement because scribing insulates physicians from the inefficiencies of current EMRs and will not spur demands for improvements.3 Inaccurate or poor-quality notes could represent another downside to scribing, although concern about the quality of notes has not been documented. Our results suggest the opposite may be true.

Note quality has not been associated with quality of care as assessed by clinical quality scores,13 but using scribes may improve the quality of care in other ways. For example, the EMR may negatively affect patient-physician communication,18,19 and freeing the physician from documentation may improve the interaction.8,20 Incorporating scribing into practice may also improve the physician experience,9,10,21,22 a possible benefit that we did not measure.

We also did not measure the cost of using a scribe to assist in EMR documentation compared with the cost of physician time spent in performing this task. If the scribe model were associated with cost savings through increased physician productivity, as well as improved physician experience, future EMR development might best focus on planned utilization by physician-scribe teams.

Study limitations. The study was conducted in a single health system, although at 8 different practice sites. The sites all used the same EMR, but templates used for documentation could be individualized by the physician and medical assistant team, so our findings may reflect variation in template design. Our analysis did adjust for possible correlations of notes from the same physician. The selection of note types in our study may make our results less generalizable to other encounter types. Our sample was not large enough to detect variations in note quality among different providers and scribes.

 

 

The ratings on the PDQI-9 may be subjective, and the reviewers were not blinded to whether a scribe was used to write the note. The differences in PDQI-9 scores were small. Although statistically significant, they may not significantly affect clinical practice. Our care model is unique in that scribes are active members of the clinical care team; the higher quality of scribed notes we found may not apply to professional scribes who are not part of the team.

Future research directions. In our study, medical assistants acting as scribes composed progress notes of similar or higher quality than physicians who wrote notes alone, although all notes were of generally average quality. As the use of scribes in medicine expands, additional studies should examine the impact of scribes on primary care workflow, quality and cost of care delivered, and quality of physician experience.

CORRESPONDENCE
Anita D. Misra-Hebert, MD, MPH, Center for Value-Based Care Research, Medicine Institute, 9500 Euclid Avenue, G10, Cleveland, OH 44195; misraa@ccf.org.

ABSTRACT

Objective Medical scribes are increasingly employed to improve physician efficiency with regard to the electronic medical record (EMR). The impact of scribes on the quality of outpatient visit notes is not known. To assess the effect, we conducted a retrospective review of ambulatory progress notes written before and after 8 practice sites transitioned to the use of medical assistants as scribes.

Methods The Physician Documentation Quality Instrument 9 (PDQI-9) was used to compare the quality of outpatient progress notes written by medical assistant scribes with the quality of notes written by 18 primary care physicians working without a scribe. The notes pertained to diabetes encounters and same-day appointments and were written during the 3 to 6 months preceding the use of scribes (pre-scribe period) and the 3 to 6 months after scribes were employed (scribe period).

Results One hundred eight notes from the pre-scribe period and 109 from the scribe period were reviewed. Scribed notes were rated higher in overall quality than unscribed notes (mean total PDQI-9 score 30.3 for scribed notes vs 28.9 for nonscribed notes; P=.01) and more up-to-date, thorough, useful, and comprehensible. The differences were limited to diabetes encounters. For same-day appointments, scribed and nonscribed notes did not differ in quality. The total word count of all scribed and nonscribed notes was similar (mean words 618, standard deviation (SD) 273 for scribed notes vs 558 words, SD 289 for nonscribed notes; P=.12).

Conclusions In this retrospective review, ambulatory notes were of higher quality when medical assistants acted as scribes than when physicians wrote them alone, at least for diabetes visits. Our findings may not apply to professional scribes who are not part of the clinical care team. As the use of medical scribes expands, additional studies should examine the impact of scribes on other aspects of care quality.

Team-based models of primary care delivery may incorporate medical scribes to improve efficiency of electronic documentation.1-4 The employment of medical scribes has grown rapidly, and it is estimated that within several years there may be one scribe for every 9 physicians.3

Accurate documentation is important to providing high-quality patient care but can take a significant amount of time. Attending physicians have been estimated to spend as long as 52 minutes per day authoring notes.5 Medical scribes can help physicians improve the efficiency of electronic documentation6 and save time.2 Using scribes can also improve physician productivity7-10 and thereby potentially increase access to care. The impact of scribes on the quality of outpatient visit notes, however, is unknown.

A team-based care delivery model in our health system’s primary care clinics uses medical assistants to scribe notes during the outpatient encounter. We hypothesized that outpatient notes written by medical assistant scribes would be of similar quality to notes written by the same group of physicians without a scribe.

METHODS

Study design and sample

We conducted a retrospective review of ambulatory notes from 18 primary care physicians at 8 practice sites in our health system who had adopted a care model in which medical assistants act as scribes. Each physician works with 2 medical assistants. To train for the new model, the physician and medical assistants participated in 2 training sessions of 2 hours each and a half day of clinic observation and evaluation with a project manager.

Scribed notes were more up-to-date, thorough, useful, and comprehensible for diabetes encounters.

Of the 18 primary care physicians included in this study, none had less than one year of experience in our health system. Tenure ranged from one to 24 years with a mean of 11.3 years.

For each participating provider, we requested all available outpatient progress notes with either an International Classification of Diseases, 9th revision (ICD-9) code for diabetes or a designation of “same day” for the 3 to 6 months preceding the use of scribes (pre-scribe period) and the 3 to 6 months after employing scribes (scribe period). We chose diabetes encounters as examples of notes addressing chronic disease management and same-day encounters as examples of problem-focused notes because these 2 types of encounters are common in outpatient primary care practice.

Note quality was evaluated using the Physician Documentation Quality Instrument 9 (PDQI-9), a validated instrument designed for this purpose, comprising 9 items rated subjectively on a 5-point Likert scale (1= not at all, 5= extremely). The items assess whether notes are up-to-date, accurate, thorough, useful, organized, comprehensible, succinct, synthesized, and internally consistent.11,12 The PDQI-9 has been applied previously in inpatient12 and outpatient settings.13

While the PDQI-9 is a validated tool, it relies on subjective ratings of note quality by the reviewer. To control for the subjective nature of the ratings, an experienced internist and an internal medicine resident coded 10 progress notes separately using the PDQI-9 and discussed the results. The process was repeated for a total of 20 notes, after which consensus was reached with >70% agreement on each attribute of the PDQI-9, suggesting that the resident’s ratings were reliable when compared with those of an experienced practicing physician.

 

 

The resident then evaluated a random sample of notes written by each physician for diabetes or same-day appointments in the pre-scribe and scribe periods. Word counts for the entire note were measured. The notes used to establish the reliability of the ratings were excluded from the analysis for this study.

Data analysis

We used linear mixed-effects models to examine note quality measures by adjusting for possible correlations of notes from the same physician. Least-squares estimates were derived; the results were not adjusted for multiple comparisons.

RESULTS

One hundred eight notes from the pre-scribe period and 109 notes from the scribe period were reviewed. Compared with notes written by a physician alone, scribed notes were rated slightly higher in overall quality (mean total PDQI-9 score 30.3 for scribe notes vs 28.9 for pre-scribe notes; P=.01) and more up-to-date, thorough, useful, and comprehensible (TABLES 1 AND 2). The differences were limited to diabetes encounters. For same day appointments, scribed notes did not differ in quality from nonscribed notes (TABLE 2). Total word count did not vary significantly between all scribe and pre-scribe notes (mean words 618, SD 273 for scribed notes vs 558 words, SD 289 for nonscribed notes; P=.12).

DISCUSSION

In this retrospective review of ambulatory notes, progress notes written by medical assistant scribes were of higher quality than notes physicians wrote alone, at least for diabetes visits. Scribe and pre-scribe notes were of similar quality for problem-focused same-day visits. This is the first study of which we are aware that compares the quality of scribed notes with notes written by physicians.

Quality scribe notes can save physician time. The progress note is an important vehicle for describing care provided and transferring information among physicians caring for the same patient. Writing a note, however, adds a considerable amount of time to the physician’s workflow. Using a scribe can decrease the time burden of note writing, and if scribed notes are of similar or better quality, this practice innovation can allow the physician to focus more on clinical than clerical tasks.

Over-documentation is a possible concern. While implementation of the EMR may improve certain aspects of quality of care delivered14,15 and note quality,16 concern has been raised about over-documentation related to the connection between documentation and reimbursement.17 In our study, we found that physician notes and scribed notes for both diabetes and same-day encounters often used EMR-based note templates, which can lead to over-documentation.

Future EMR development might best focus on planned utilization by physician-scribe teams.

In general, both physician and scribed notes were rated to be of average to low quality because none of the mean scores on the 9 individual components of the PDQI-9 reached 4.0. Scribed notes were not inaccurate and had word counts similar to physician notes.

Scribing has potential drawbacks—and benefits. Drawbacks to scribing have not been well-studied. It has been suggested that using scribes to work around the EMR may actually hinder its further advancement because scribing insulates physicians from the inefficiencies of current EMRs and will not spur demands for improvements.3 Inaccurate or poor-quality notes could represent another downside to scribing, although concern about the quality of notes has not been documented. Our results suggest the opposite may be true.

Note quality has not been associated with quality of care as assessed by clinical quality scores,13 but using scribes may improve the quality of care in other ways. For example, the EMR may negatively affect patient-physician communication,18,19 and freeing the physician from documentation may improve the interaction.8,20 Incorporating scribing into practice may also improve the physician experience,9,10,21,22 a possible benefit that we did not measure.

We also did not measure the cost of using a scribe to assist in EMR documentation compared with the cost of physician time spent in performing this task. If the scribe model were associated with cost savings through increased physician productivity, as well as improved physician experience, future EMR development might best focus on planned utilization by physician-scribe teams.

Study limitations. The study was conducted in a single health system, although at 8 different practice sites. The sites all used the same EMR, but templates used for documentation could be individualized by the physician and medical assistant team, so our findings may reflect variation in template design. Our analysis did adjust for possible correlations of notes from the same physician. The selection of note types in our study may make our results less generalizable to other encounter types. Our sample was not large enough to detect variations in note quality among different providers and scribes.

 

 

The ratings on the PDQI-9 may be subjective, and the reviewers were not blinded to whether a scribe was used to write the note. The differences in PDQI-9 scores were small. Although statistically significant, they may not significantly affect clinical practice. Our care model is unique in that scribes are active members of the clinical care team; the higher quality of scribed notes we found may not apply to professional scribes who are not part of the team.

Future research directions. In our study, medical assistants acting as scribes composed progress notes of similar or higher quality than physicians who wrote notes alone, although all notes were of generally average quality. As the use of scribes in medicine expands, additional studies should examine the impact of scribes on primary care workflow, quality and cost of care delivered, and quality of physician experience.

CORRESPONDENCE
Anita D. Misra-Hebert, MD, MPH, Center for Value-Based Care Research, Medicine Institute, 9500 Euclid Avenue, G10, Cleveland, OH 44195; misraa@ccf.org.

References

1. Bodenheimer T, Willard-Grace R, Ghorob A. Expanding the roles of medical assistants: Who does what in primary care? JAMA Intern Med. 2014;174:1025-1026.

2. Reuben DB, Knudsen J, Senelick W, et al. The effect of a physician partner program on physician efficiency and patient satisfaction. JAMA Intern Med. 2014;174:1190-1193.

3. Gellert GA, Ramirea R, Webster S. The rise of the medical scribe industry: Implications for the advancement of electronic health records. JAMA. 2015;313:1315-1316.

4. Shultz CG, Holmstrom HL. The use of medical scribes in health care settings: a systematic review and future directions. J Am Board Fam Med. 2015;28:371-381.

5. Hripcsak G, Vawdrey DK, Fred MR, et al. Use of electronic clinical documentation: time spent and team interactions. J Am Med Inform Assoc. 2011;18:112-117.

6. Silverman L. Scribes Are Back, Helping Doctors Tackle Electronic Medical Records. NPR.org. Available at: www.npr.org/blogs/health/2014/04/21/303406306/scribes-are-back-helping-doctors-tackle-electronic-medical-records. Accessed April 23, 2014.

7. Arya R, Salovich DM, Ohman-Strickland P, et al. Impact of scribes on performance indicators in the emergency department. Acad Emerg Med. 2010;17:490-494.

8. Bank AJ, Obetz C, Konrardy A, et al. Impact of scribes on patient interaction, productivity, and revenue in a cardiology clinic: a prospective study. Clinicoecon Outcomes Res. 2013;5:399-406.

9. Bastani A, Shaqiri B, Palomba K, et al. An ED scribe program is able to improve throughput time and patient satisfaction. Am J Emerg Med. 2014;32:399-402.

10. Allen B, Banapoor B, Weeks EC, et al. An assessment of emergency department throughput and provider satisfaction after the implementation of a scribe program. Advances in Emergency Medicine. 2014;2014:e517319.

11. Stetson PD, Morrison FP, Bakken S, et al. Preliminary development of the Physician Documentation Quality Instrument. J Am Med Inform Assoc. 2008;15:534-541.

12. Stetson PD, Bakken S, Wrenn JO, et al. Assessing electronic note quality using the Physician Documentation Quality Instrument (PDQI-9). Appl Clin Inform. 2012;3:164-174.

13. Edwards ST, Neri PM, Volk LA, et al. Association of note quality and quality of care: a cross-sectional study. BMJ Qual Saf. 2013;23:406-413.

14. Schiff GD, Bates DW. Can electronic clinical documentation help prevent diagnostic errors? N Engl J Med. 2010;362:1066-1069.

15. Samal L, Wright A, Healey MJ, et al. Meaningful use and quality of care. JAMA Intern Med.  2014;174:997-998.

16. Burke HB, Sessums LL, Hoang A, et al. Electronic health records improve clinical note quality. J Am Med Inform Assoc. 2015;22:199-205.

17. Sheehy AM, Weissburg DJ, Dean SM. The role of copy-and-paste in the hospital electronic health record. JAMA Intern Med. 2014;174:1217-1218.

18. Shachak A, Hadas-Dayagi M, Ziv A, et al. Primary care physicians’ use of an electronic medical record system: a cognitive task analysis. J Gen Intern Med. 2009;24:341-348.

19. Shachak A, Reis S. The impact of electronic medical records on patient-doctor communication during consultation: a narrative literature review. J Eval Clin Pract. 2009;15:641-649.

20. Misra-Hebert AD, Rabovsky A, Yan C, et al. A team-based model of primary care delivery and physician-patient interaction. Am J Med. 2015;128:1025-1028.

21. Sinsky CA, Willard-Grace R, Schutzbank AM, et al. In search of joy in practice: a report of 23 high-functioning primary care practices. Ann Fam Med. 2013;11:272-278.

22. Koshy S, Feustel PJ, Hong M, et al. Scribes in an ambulatory urology practice: patient and physician satisfaction. J Urol. 2010;184:258-262.

References

1. Bodenheimer T, Willard-Grace R, Ghorob A. Expanding the roles of medical assistants: Who does what in primary care? JAMA Intern Med. 2014;174:1025-1026.

2. Reuben DB, Knudsen J, Senelick W, et al. The effect of a physician partner program on physician efficiency and patient satisfaction. JAMA Intern Med. 2014;174:1190-1193.

3. Gellert GA, Ramirea R, Webster S. The rise of the medical scribe industry: Implications for the advancement of electronic health records. JAMA. 2015;313:1315-1316.

4. Shultz CG, Holmstrom HL. The use of medical scribes in health care settings: a systematic review and future directions. J Am Board Fam Med. 2015;28:371-381.

5. Hripcsak G, Vawdrey DK, Fred MR, et al. Use of electronic clinical documentation: time spent and team interactions. J Am Med Inform Assoc. 2011;18:112-117.

6. Silverman L. Scribes Are Back, Helping Doctors Tackle Electronic Medical Records. NPR.org. Available at: www.npr.org/blogs/health/2014/04/21/303406306/scribes-are-back-helping-doctors-tackle-electronic-medical-records. Accessed April 23, 2014.

7. Arya R, Salovich DM, Ohman-Strickland P, et al. Impact of scribes on performance indicators in the emergency department. Acad Emerg Med. 2010;17:490-494.

8. Bank AJ, Obetz C, Konrardy A, et al. Impact of scribes on patient interaction, productivity, and revenue in a cardiology clinic: a prospective study. Clinicoecon Outcomes Res. 2013;5:399-406.

9. Bastani A, Shaqiri B, Palomba K, et al. An ED scribe program is able to improve throughput time and patient satisfaction. Am J Emerg Med. 2014;32:399-402.

10. Allen B, Banapoor B, Weeks EC, et al. An assessment of emergency department throughput and provider satisfaction after the implementation of a scribe program. Advances in Emergency Medicine. 2014;2014:e517319.

11. Stetson PD, Morrison FP, Bakken S, et al. Preliminary development of the Physician Documentation Quality Instrument. J Am Med Inform Assoc. 2008;15:534-541.

12. Stetson PD, Bakken S, Wrenn JO, et al. Assessing electronic note quality using the Physician Documentation Quality Instrument (PDQI-9). Appl Clin Inform. 2012;3:164-174.

13. Edwards ST, Neri PM, Volk LA, et al. Association of note quality and quality of care: a cross-sectional study. BMJ Qual Saf. 2013;23:406-413.

14. Schiff GD, Bates DW. Can electronic clinical documentation help prevent diagnostic errors? N Engl J Med. 2010;362:1066-1069.

15. Samal L, Wright A, Healey MJ, et al. Meaningful use and quality of care. JAMA Intern Med.  2014;174:997-998.

16. Burke HB, Sessums LL, Hoang A, et al. Electronic health records improve clinical note quality. J Am Med Inform Assoc. 2015;22:199-205.

17. Sheehy AM, Weissburg DJ, Dean SM. The role of copy-and-paste in the hospital electronic health record. JAMA Intern Med. 2014;174:1217-1218.

18. Shachak A, Hadas-Dayagi M, Ziv A, et al. Primary care physicians’ use of an electronic medical record system: a cognitive task analysis. J Gen Intern Med. 2009;24:341-348.

19. Shachak A, Reis S. The impact of electronic medical records on patient-doctor communication during consultation: a narrative literature review. J Eval Clin Pract. 2009;15:641-649.

20. Misra-Hebert AD, Rabovsky A, Yan C, et al. A team-based model of primary care delivery and physician-patient interaction. Am J Med. 2015;128:1025-1028.

21. Sinsky CA, Willard-Grace R, Schutzbank AM, et al. In search of joy in practice: a report of 23 high-functioning primary care practices. Ann Fam Med. 2013;11:272-278.

22. Koshy S, Feustel PJ, Hong M, et al. Scribes in an ambulatory urology practice: patient and physician satisfaction. J Urol. 2010;184:258-262.

Issue
The Journal of Family Practice - 65(3)
Issue
The Journal of Family Practice - 65(3)
Page Number
155-159
Page Number
155-159
Publications
Publications
Topics
Article Type
Display Headline
Medical scribes: How do their notes stack up?
Display Headline
Medical scribes: How do their notes stack up?
Legacy Keywords
medical scribes, electronic medical records, EMRs, electronic health records, EHRs, practice management, Anita D. Misra-Hebert, MD, MPH, Linda Amah, MD, Andrew Rabovsky, MD, Shannon Morrison, MS, Marven Cantave, Bo Hu, PhD, Christine A. Sinsky, MD, Michael B. Rothberg, MD, MPH
Legacy Keywords
medical scribes, electronic medical records, EMRs, electronic health records, EHRs, practice management, Anita D. Misra-Hebert, MD, MPH, Linda Amah, MD, Andrew Rabovsky, MD, Shannon Morrison, MS, Marven Cantave, Bo Hu, PhD, Christine A. Sinsky, MD, Michael B. Rothberg, MD, MPH
Sections
Article Source

PURLs Copyright

Inside the Article

Article PDF Media

Regionalized Care and Adverse Events

Article Type
Changed
Mon, 01/02/2017 - 19:34
Display Headline
Impact of regionalized care on concordance of plan and preventable adverse events on general medicine services

Failures in communication among healthcare professionals are known threats to patient safety. These failures account for over 60% of root causes of sentinel events, the most serious events reported to The Joint Commission.[1] As such, identifying both patterns of effective communication as well as barriers to successful communication has been a focus of efforts aimed at improving patient safety. However, to date, the majority of this work has centered on improving communication in settings such as the operating room and intensive care unit,[2, 3, 4] or at times of care transitions.[5, 6, 7, 8]

Unique barriers exist for effective interdisciplinary communication in the hospital setting, particularly physiciannurse communication regarding shared hospitalized patients.[9] Traditionally, care of hospitalized patients is provided by physicians, nurses, and other team members working in varied workflow patterns, leading to dispersed team membership, where each team member cares for different groups of patients in different locations across the hospital. This dispersion is further heightened on teaching services, where residents' rotation schedules lead to frequent changes of care team membership, leaving inpatient care teams particularly vulnerable to ineffective communication. Evidence suggests that communication between nurses and physicians is currently suboptimal, leading to frequent disagreement regarding the patient's plan of care.[9, 10] This divergence between physician and nursing perceptions of patients' care plans may leave patients at greater risk of adverse events (AEs).

Several studies have examined the effects of regionalized inpatient care teams, where multidisciplinary team members care for the same patients on the same hospital unit, on communication and patient outcomes.[4, 11, 12, 13, 14] Results of these studies have been inconsistent, perhaps due to the particular characteristics of the care teams or to the study methodology. Thus, further rigorously done studies are required to better understand the impact of team regionalization on patient care. The goal of this study was to examine whether the implementation of regionalized inpatient care teams was associated with improvements in care team communication and preventable AEs.

METHODS

Setting, Patients, and Study Design

We performed a cohort analysis of patients at a 700‐bed tertiary care center, pre‐ and postregionalization of inpatient general medicine care teams. Our study protocol was approved by the Partners Healthcare Human Subjects Review Committee. Patients were eligible for inclusion if they were 18 years of age or older and discharged from the general medicine service (GMS) from any of the 3 participating nursing units between April 1, 2012 and June 19, 2012 (preregionalization) or April 1, 2013 and June 19, 2013 (postregionalization).

Intervention

On June 20, 2012, regionalized care was implemented on the GMS such that each of 3 GMS teams was localized to 1 of 3, 15‐bed nursing units. Prior to regionalization, the GMS physician care teams, each consisting of 1 hospitalist attending, 1 medical resident, and 2 medical interns, would care for patients on an average of 7 and up to 13 different nursing units on a given day.

Regionalized care consisted of a multifaceted intervention codeveloped by hospitalist, residency, nursing, emergency department, and hospital leadership and included: (1) regionalizing GMS teams as much as possible; (2) change in resident call structure from a traditional 4‐day call cycle to daily admitting; (3) collaborative efforts to enhance GMS patient discharges before noon to promote regionalized placement of patients without prolonging time in the emergency department (ED); (4) daily morning and postround multidisciplinary huddles to prioritize sicker patients and discharges; (5) encouragement of daily rounds at patients' bedsides with presence of physician team, nurse, and team pharmacist if available; (6) creation of unit‐ and team‐level performance reports; and (7) creation of unit‐based physician and nursing co‐leadership (Figure 1).[15]

Figure 1
Regionalization of general medical services into united‐based care teams. Regionalization of general medical services involved included localizing each physician care team to a single nursing unit. Physician care teams included shared patient care responsibilities between a day team consisting of an attending hospitalist (A), a daytime resident (DR), and 2 daytime interns (DI), and a “twilight team” consisting of a twilight resident (TR) and twilight intern (TI), limiting hours of cross‐coverage by a night‐float resident (NF‐R). In addition, structured interdisciplinary structured huddles were scheduled throughout the day to identify workflow needs (eg, calling interpreter prior to bedside rounds), create patient care plans, and anticipate patient discharges. This creates a virtuous cycle of shared responsibility between care team members to improve efficiency, create earlier bed availability, and improve regionalization. Abbreviations: AM = Morning, CC = Care Coordinator, RN = Nurse, N = Nurse, OT = Occupational Therapist, PM = Evening, PT = Physical Therapist, SW = Social Worker.

Concordance of Plan

Concordance of plan was measured via a 7‐question survey previously developed, pilot tested, and used to measure the impact of regionalized care on care team communication between inpatient nursephysician team members.[9] The survey was administered in‐person by 1 of 8 trained research assistants (RAs) (4/emntervention period) to nurse and intern pairs caring for patients on the study units pre‐ and postregionalization. GMS patients were eligible for inclusion if surveys could be administered to their nurse and intern within the first 24 hours of admission to the unit and within 48 hours of admission to the hospital, based on RA availability (thus excluding patients admitted on Fridays as surveys were not conducted over the weekend). Most often, all eligible patients admitted to the study units during time periods of data collection were included in the study. On limited occasions, the daily supply of patients surpassed RA capacity for inclusion, at which time computer‐generated randomization was utilized to randomly select patients for inclusion. Nurse and intern pairs were surveyed once during a patient's hospitalization, although they could be surveyed more than once about different patients, and patients could be included more than once if rehospitalized on the study unit and cared for by a different nurseintern pair. Of the 472 selected eligible patients, the nurses and interns of 418 patients were available and consented to survey administration, representing 361 unique nurse and intern pairs and 399 unique patients.

Each member of the pair was asked about 7 specific aspects of the patient's care plan for that day in isolation from the other team member, including: (1) the patient's primary diagnosis, (2) the patient's expressed chief concern, (3) the day's scheduled tests, (4) the day's scheduled procedures, (5) consulting services involved, (6) medication changes made that day, and (7) the patient's expected discharge date. In addition, each pair was asked the name of the other team member (ie, the nurse was asked the name of the intern and vice versa), and whether or not the patient care plan for the day had been discussed with the other team member, where concordance was defined as both members agreeing the plan had been discussed. All responses were recorded verbatim. Pairs were surveyed independently between 12 pm and 2 pm, limiting confounding by evolving plans of care over time.

Each set of surveys were then reviewed by 2 of 4 trained adjudicators, and responses to each question were scored as complete, partial, or no agreement. Rules for degree of agreement were based upon previously utilized parameters[9] as well as biweekly meetings during which common themes and disagreements in ratings were discussed, and rules generated to create consensus (see Supporting Information, Appendix, in the online version of this article).

Adverse Event Detection

Of the patients meeting eligibility criteria, 200 patients were randomly selected using computer‐generated randomization from each time period for AE outcome assessment, for a total of 400 patients.

Each patient's electronic medical record was retrospectively reviewed by a trained clinician using a previously validated screening tool to detect any possible AEs.[11] Any positive screen prompted documentation of a narrative summary including a short description of the possible AE and pertinent associated data. We defined AE as any injury due to medical management rather than the natural history of the illness, and further limited this definition to only include AEs that occurred on the study unit or as a result of care on that unit.

Two of 4 trained adjudicators, blinded to time period, then separately reviewed each narrative summary using previously validated 6‐point confidence scales to determine the presence and preventability of AE, with confidence ratings of 4 or greater used as cutoffs.[11] All AEs were also scored on a 4‐point severity scale (trivial, clinically significant, serious, or life threatening), with severe AE defined as serious or life threatening. Lastly, adjudicators grouped AEs into 1 of 10 prespecified categories.[11] Any disagreements in ratings or groupings were discussed by all 4 adjudicators to reach consensus.

Data Analysis

Patient characteristics are presented using descriptive statistics and were compared in the pre‐ and postregionalization time periods using 2 or t tests as appropriate.

To analyze whether regionalized care was associated with concordance of plan, adjudicated survey questions were assigned points of 1, 0.5, and 0 for complete, partial, and no agreement, respectively. Total mean concordance scores for any patient ranged from 0 to 7 points, and were divided by total number of answered questions (up to 7) for a range of 0 to 1. Total mean concordance scores as well as mean concordance score per survey question were compared pre‐ versus postregionalization using t tests. In sensitivity analyses, adjudicated survey responses were dichotomized with complete and partial agreement deemed concordant responses. Percent concordance for each question was then compared pre‐ versus postregionalization using 2 analysis. Questions about the name of the other team member and discussion of daily care plan with the other team member were excluded from total concordance score calculations and were compared individually pre‐ versus postregionalization, because they are not directly about the plan of care.

To analyze the association of regionalization with odds of preventable AE, we performed multivariable logistic regression adjusted for patient age, sex, race, language, and Elixhauser comorbidity score,[16] and utilized generalized estimating equations to account for clustering by hospital unit. Secondary outcomes included severe preventable AEs, nonpreventable AEs, and category of preventable AEs using similar methodology. Two‐sided P values <0.05 were considered significant, and SAS version 9.2 (SAS Institute Inc., Cary, NC) was used for all analyses.

RESULTS

The fidelity of the intervention in achieving its goal of regionalized care is discussed separately.[15] Briefly, the intervention was successful at achieving 85% regionalization by team (ie, average daily percentage of team's patients assigned to team's unit) and 87% regionalization by unit (ie, average daily percentage of unit's patients with assigned team) following implementation, compared to 20% regionalization by team and unit in the preintervention period. Importantly, the average daily census of physician care teams rose by 32%, from a mean of 10.8 patients/physician care team preregionalization to a mean of 14.3 patients/physician care team postregionalization.

Concordance of Plan

Of the 418 nurse and intern paired surveys, 4 surveys were excluded due to repeat surveys of the same patient during the same hospitalization, for a total of 197 distinct paired surveys preregionalization and 217 paired surveys postregionalization. There were no statistically significant differences in patients' age, sex, race, language, admission source, length of stay, Elixhauser comorbidity score and diagnosis‐related group weight pre‐ versus postregionalization (Table 1).

Baseline Characteristics
Characteristic Concordance of Care Plan Adverse Events
Pre, n = 197 Post, n = 217 P Value Pre, n = 198 Post, n = 194 P Value
  • NOTE: Abbreviations: DRG, diagnosis‐related group; IQR, interquartile range; SD, standard deviation.

Age, mean (SD) 60.5 (19.4) 57.6 (20.8) 0.15 60.4 (18.9) 58.0 (21.2) 0.24
Male, n (%) 77 (39.1) 92 (42.4) 0.49 94 (47.5) 85 (43.8) 0.55
Race/ethnicity, n (%) 0.34 0.12
White 134 (68.0) 141 (65.0) 132 (66.5) 121 (62.4)
Black 42 (21.3) 45 (20.7) 41 (20.8) 54 (27.8)
Hispanic 18 (9.1) 21 (9.7) 22 (11.3) 13 (6.8)
Other/unknown 3 (1.5) 10 (4.6) 3 (1.4) 6 (2.9)
Language, n (%) 0.30 0.73
English 183 (92.9) 203 (93.5) 176 (88.7) 175 (90.2)
Spanish 6 (3.0) 10 (4.6) 10 (5.2) 10 (5.3)
Other 8 (4.1) 4 (1.8) 12 (6.1) 9 (4.5)
Admitting source, n (%) 1.00 0.10
Physician office 13 (6.6) 13 (6.0) 13 (6.6) 6 (3.1)
Emergency department 136 (69.0) 150 (69.1) 126 (63.6) 127 (65.5)
Transfer from different hospital 40 (20.3) 45 (20.7) 54 (27.3) 50 (25.8)
Transfer from skilled nursing facility 8 (4.1) 9 (4.2) 5 (2.5) 11 (5.6)
Length of stay, d, median (IQR) 3.0 (4.0) 3.0 (4.0) 0.57 4.0 (5.0) 3.0 (4.0) 0.16
Elixhauser Comorbidity Score, mean (SD) 8.0 (8.8) 8.3 (9.3) 0.74 8.0 (8.6) 7.8 (8.4) 0.86
DRG weight, mean (SD) 1.6 (1.0) 1.5 (1.0) 0.37 1.5 (0.93) 1.5 (1.1) 0.96

Kappa scores for adjudications of concordance surveys (defined as both adjudicators scoring the same level of agreement (ie, both complete or partial agreement versus no agreement) ranged from 0.69 to 0.95, by question. There were no significant differences in total mean concordance scores in the care plan pre‐ versus postregionalization (0.65 vs 0.67, P = 0.26) (Table 2). Similarly, there were no significant differences in mean concordance score for each survey question, except agreement on expected date of discharge (0.56 vs 0.68, P = 0.003), knowledge of the other provider's name, and agreement that discussion of the daily plan had taken place with the other pair member. Similar results were seen when results were dichotomized (ie, partial or complete agreement vs no agreement) (Table 2).

Effect of Regionalized Care on Concordance of Care Plan between Primary Nurse and Responding Physician
Concordance Outcome Pre, n = 197 Post, n = 217 P Value
  • NOTE: Abbreviations: SD, standard deviation. *Calculation of concordance score: agree = 1 point, partial agreement = 0.5 points, disagree = 0 points. Total concordance score excluded the following survey question responses: knowledge of other team member name and plan discussed. Concordance defined as agree or partial agreement. For responding clinician knowledge of nurse's name, nurse's knowledge of responding clinician's name, and plan discussed, all paired survey responses were either agree (1) or disagree (0).

Concordance score*
Total concordance score, mean (SD) 0.65 (0.17) 0.67 (0.16) 0.26
Subgroups
Diagnosis 0.77 (0.32) 0.72 (0.35) 0.11
Patient's chief concern 0.48 (0.44) 0.48 (0.43) 0.94
Tests today 0.67 (0.40) 0.71 (0.42) 0.36
Procedures today 0.93 (0.25) 0.92 (0.25) 0.71
Medication changes today 0.56 (0.44) 0.59 (0.43) 0.54
Consulting services 0.59 (0.44) 0.60 (0.44) 0.82
Expected discharge date 0.56 (0.44) 0.68 (0.38) 0.003
Responding clinician knowledge of nurse's name 0.56 (0.50) 0.86 (0.35) <0.001
Nurse's knowledge of responding clinician's name 0.56 (0.50) 0.88 (0.33) <0.001
Plan discussed 0.73 (0.45) 0.88 (0.32) <0.001
Percent concordance, mean (SD)
Diagnosis 92.0 (27.3) 88.6 (31.9) 0.25
Patient's chief concern 59.6 (49.1) 60.6 (49.0) 0.84
Tests today 78.9 (40.9) 77.2 (42.1) 0.67
Procedures today 93.5 (24.8) 94.1 (23.7) 0.80
Medication changes today 66.3 (33.6) 69.9 (46.0) 0.44
Consulting services 69.3 (46.2) 68.9 (46.4) 0.93
Expected discharge date 67.5 (47.0) 82.6 (38.0) <0.001
Responding clinician knowledge of nurse's name 55.7 (49.8) 85.6 (35.2) <0.001
Nurse's knowledge of responding clinician's name 55.9 (49.8) 87.9 (32.8) <0.001
Plan discussed 72.9 (44.6) 88.2 (32.3) <0.001

Adverse Events

Of the 400 patients screened for AEs, 8 were excluded due to missing medical record number (5) and discharge outside of study period (3). Of the final 392 patient screens (198 pre, 194 post), there were no significant differences in patients' age, sex, race, language, length of stay, or Elixhauser score pre‐ versus postregionalization (Table 1).

Kappa scores for adjudicator agreement were 0.35 for presence of AE and 0.34 for preventability of AE. Of the 392 reviewed patient records, there were 133 total AEs detected (66 pre, 67 post), 27 preventable AEs (13 pre, 14 post), and 9 severe preventable AEs (4 pre, 5 post) (Table 3). There was no significant difference in the adjusted odds of preventable AEs post‐ versus preregionalization (adjusted odds ratio: 1.37, 95% confidence interval: 0.69, 2.69). Although the low number of AEs rated as severe or life threatening precluded adjusted analysis, unadjusted results similarly demonstrated no difference in odds of severe preventable AEs pre‐ versus postregionalization. As expected, there was no significant difference in adjusted odds of nonpreventable AE after implementation of regionalized care (Table 3).

Adjusted Effect of Regionalization on Adverse Events*
Adverse Events No. of Adverse Events Adjusted Odds Ratio Post vs Pre (95% CI)
Pre, n = 198 Post, n = 194
  • NOTE: Abbreviations: CI, confidence interval. *Adjusted for patient age, sex, race, language, and comorbidity as measured by the Elixhauser score. Number of events precluded adjusted analysis. Unadjusted odds ratio = 1.30 (0.34, 4.91).

Preventable 13 14 1.37 (0.69, 2.69)
Serious and preventable 4 5
Nonpreventable 47 50 1.20 (0.85, 1.75)

Similarly, there were no significant differences in category of preventable AE pre‐ versus postregionalization. The most frequent preventable AEs in both time periods were those related to adverse drug events and to manifestations of poor glycemic control, examples of which are illustrated (Table 4).

Examples of Preventable Adverse Events Due to Adverse Drug Events and Manifestations of Poor Glycemic Control
  • NOTE: Abbreviations: PNR, pro re nata (as needed).

Adverse drug event 29‐year‐old male with history of alcohol abuse, complicated by prior withdrawal seizures/emntensive care unit admissions, presented with alcohol withdrawal. Started on standing and PRN lorazepam, kept on home medications including standing clonidine, gabapentin, citalopram, quetiapine. Became somnolent due to polypharmacy, ultimately discontinued quetiapine as discovered took only as needed at home for insomnia
Manifestations of poor glycemic control 78‐year‐old male with recently diagnosed lymphoma, distant history of bladder and prostate cancer status post ileal loop diversion, presented status post syncopal event; during event, spilled boiling water on himself leading to second‐degree burns on 3% of his body. Initially admitted to trauma/burn service, ultimately transferred to medical service for ongoing multiple medical issues including obstructive uropathy, acute on chronic renal failure. Adverse event was hyperglycemia (>350 mg/dL on >2 consecutive readings) in the setting of holding his home insulin detemir and insulin aspart (had been placed on insulin aspart sliding scale alone). After hyperglycemic episodes, was placed back on weight‐based basal/nutritional insulin

DISCUSSION

In this study of general medicine patients at a large academic medical center, we found that regionalization of care teams on general medicine services was associated with improved recognition of care team members and agreement on estimated date of patient discharge, but was not associated with improvement in overall nurse and physician concordance of the patient care plan, or the odds of preventable AEs.

This intervention importantly addresses the barrier of dispersion of team membership, a well‐recognized barrier to interdisciplinary collaboration,[17, 18] particularly with resident physician teams due to frequently changing team membership. Localization of all team members, in addition to encouragement of daily collaborative bedside rounds as part of the regionalization initiative, likely contributed to our observed improvement in team member identification and discussion of daily care plans. Similarly, regionalization resulted in improved agreement in estimations of date of patient discharge. Focus on early patient discharges was an integral part of the implementation efforts; we therefore hypothesize that mutual focus on discharge planning by both nurses and responding clinicians may have explained this observed result.

On the other hand, regionalization did not appreciably improve the overall concordance of care plan between nurses and interns, despite a significant increase in team members agreeing that the plan had been discussed. Our findings support similar prior research demonstrating that regionalizing hospitalist attendings to single nursing units had limited impact on agreement of care plan between physicians and nurses.[13] Similarly, in settings where physicians and nurses are inherently regionalized, such as the intensive care unit[4] or the operating room,[3] communication between physicians and nurses remains difficult. Collectively, our findings suggest that colocalization of physicians and nurses alone is likely insufficient to improve measured communication between care team members. Existing literature suggests that more standardized approaches to improve communication, such as structured communication tools used during daily inpatient care[19, 20] or formalized team training,[21, 22, 23] lead to improvements in communication and collaboration. Despite these findings, it is important to highlight that this study did not assess other measures of workplace culture, such as teamwork and care team cohesiveness, which may have been positively affected by this intervention, even without measurable effect on concordance of care plan. Additionally, as noted, the average daily census on each team increased by almost a third postintervention, which may have impeded improvements in care team communication.

In addition, we found that our intervention had no significant impact on preventable AEs or severe preventable AEs. Although we cannot exclude the possibility that more subtle AEs were missed with our methodology, our results indicate that regionalized care alone may be inadequate to improve major patient safety outcomes. As discussed, the volume of patients did increase postintervention; thus, another way to state our results is that we were able to increase the daily volume of patients without any significant decreases in patient safety. Nevertheless, the results on patient safety were less than desired. A recent review of interdisciplinary team care interventions on general medical wards similarly demonstrated underwhelming improvements in patient safety outcomes, although the reviewed interventions did not specifically address preventable AEs, a gap in the literature commented on by the authors.[24] Other albeit limited literature has demonstrated improvement in patient safety outcomes via multifaceted efforts aimed at improving care team member communication. Notably, these efforts include colocalization of care team members to single units but also involve additional measures to improve communication and collaboration between care team members, such as structured communication during interdisciplinary rounds, and certification of key interdisciplinary teamwork skills.[11, 14] Although our regionalized care intervention included many similar features to these accountable care units (ACUs) including unit‐based care teams, unit‐level performance reporting, and unit‐based physician and nursing coleadership, significant differences existed. Notably, in addition to the above features, the ACU model also incorporated highly structured communication models for interdisciplinary rounding, and certification processes to ensure an appropriate communication skill base among care team members.[14] Thus, although creation of regionalized care teams is likely a necessary precursor to implementation of these additional measures, alone it may be insufficient to improve patient safety outcomes.

Importantly, in our study we identified that adverse drug events and manifestations of poor glycemic control occurred in high frequency both before and following implementation of regionalized care, supporting other literature that describes the prevalence of these AEs.[11, 25, 26, 27] These results suggest that targeted interventions to address these specific AEs are likely necessary. Notably, the intervention units in our study did not consistently employ clinical pharmacists assigned specifically to that unit's care team to allow for integration within the care team. As prior research has suggested that greater collaboration with clinical pharmacists results in reduction of adverse drug events,[28] next steps may include improved integration of team‐based pharmacists into the activities of the regionalized care teams. Inpatient management of diabetes also requires specific interventions,[29, 30, 31] only some of which may be addressable by having regionalized care and better interdisciplinary communication.

Our findings are subject to several limitations. First, this was a single‐site study and thus our findings may not be generalizable to other institutions. However, regionalized care is increasingly encouraged to optimize communication between care team members.[17, 18] Therefore, our null findings may be pertinent to other institutions looking to improve patient safety outcomes, demonstrating that additional initiatives will likely be required. Second, our modes of outcome measurement possess limitations. In measuring concordance of care plan, although previously used survey techniques were employed,[9] the concordance survey has not been formally validated, and we believe some of the questions may have led to ambiguity on the part of the responders that may have resulted in less accurate responses, thus biasing toward the null. Similarly, in measuring AEs, the screening tool relied on retrospective chart review looking for specific AE types[11] and thus may not have captured more subtle AEs. Additionally, our study may have been underpowered to demonstrate significant reduction in preventable AEs, although other studies of similar methodology demonstrated significant results with similar sample size.[11] This was due in part to our lower‐than‐expected baseline AE rate (6.6% compared with approximately 10.3% in previous studies).[11] Lastly, our study solely examined the association of regionalization with concordance of care plan and preventable AEs, but importantly excluded other clinically important outcomes that may have been positively (or negatively) impacted by these regionalization efforts, such as ED wait times, provider efficiency (eg, fewer pages, less time in transit, more time at the bedside), interdisciplinary teamwork, or patient or provider satisfaction.

CONCLUSION

In summary, our findings suggest that regionalized care teams alone may be insufficient to effectively promote communication between care team members regarding the care plan or to lead to improvements in patient safety, although we recognize that there may have been benefits (or unintended harms) not measured in this study but are nonetheless important for clinical care and workplace culture. This is an important lesson, as many hospitals move toward regionalized care in an effort to improve patient safety outcomes. However, strengthening the infrastructure by colocalizing care team members to maximize opportunity for communication is likely a necessary first step toward facilitating implementation of additional initiatives that may lead to more robust patient safety improvements, such as structured interdisciplinary bedside rounds (eg, facilitating and training all team members to fulfill specific roles), teamwork training, and certification of key interdisciplinary teamwork skills. Additionally, close examination of identified prevalent and preventable AEs can help to determine which additional initiatives are most likely to have greatest impact in improving patient safety.

Disclosures: This research was supported by funds provided by Brigham and Women's Hospital (BWH) and by funds provided by the Department of Medicine at BWH. All authors had full access to all of the data in the study and were integrally involved in the design, implementation, data collection, and analyses. The first author, Dr. Stephanie Mueller, takes responsibility for the integrity for the data and the accuracy of the data analysis. Dr. Schnipper reports grants from Sanofi Aventis, outside the submitted work.

Files
References
  1. Joint Commission on Accreditation of Healthcare Organizations. Understanding and Preventing Sentinel Events in Your Health Care Organization. Oak Brook, IL: Joint Commission; 2008.
  2. Lingard L, Espin S, Whyte S, et al. Communication failures in the operating room: an observational classification of recurrent types and effects. Qual Saf Health Care. 2004;13(5):330334.
  3. Makary MA, Sexton JB, Freischlag JA, et al. Operating room teamwork among physicians and nurses: teamwork in the eye of the beholder. J Am Coll Surg. 2006;202(5):746752.
  4. Thomas EJ, Sexton JB, Helmreich RL. Discrepant attitudes about teamwork among critical care nurses and physicians. Crit Care Med. 2003;31(3):956959.
  5. Arora V, Johnson J, Lovinger D, Humphrey HJ, Meltzer DO. Communication failures in patient sign‐out and suggestions for improvement: a critical incident analysis. Qual Saf Health Care. 2005;14(6):401407.
  6. Starmer AJ, Spector ND, Srivastava R, et al. Changes in medical errors after implementation of a handoff program. N Engl J Med. 2014;371(19):18031812.
  7. Gandara E, Moniz T, Ungar J, et al. Communication and information deficits in patients discharged to rehabilitation facilities: an evaluation of five acute care hospitals. J Hosp Med. 2009;4(8):E28E33.
  8. Kripalani S, LeFevre F, Phillips CO, Williams MV, Basaviah P, Baker DW. Deficits in communication and information transfer between hospital‐based and primary care physicians: implications for patient safety and continuity of care. JAMA. 2007;297(8):831841.
  9. O'Leary KJ, Thompson JA, Landler MP, et al. Patterns of nurse‐physician communication and agreement on the plan of care. Qual Saf Health Care. 2010;19(3):195199.
  10. Evanoff B, Potter P, Wolf L, Grayson D, Dunagan C, Boxerman S. Can we talk? Priorities for patient care differed among health care providers. In: Henriksen K, Battles JB, Marks ES, Lewin DI, eds. Advances in Patient Safety: From Research to Implementation. Vol 1. Rockville, MD: Agency for Healthcare Research and Quality; 2005.
  11. O'Leary KJ, Buck R, Fligiel HM, et al. Structured interdisciplinary rounds in a medical teaching unit: improving patient safety. Arch Intern Med. 2011;171(7):678684.
  12. O'Leary KJ, Haviley C, Slade ME, Shah HM, Lee J, Williams MV. Improving teamwork: impact of structured interdisciplinary rounds on a hospitalist unit. J Hosp Med. 2011;6(2):8893.
  13. O'Leary KJ, Wayne DB, Landler MP, et al. Impact of localizing physicians to hospital units on nurse‐physician communication and agreement on the plan of care. J Gen Intern Med. 2009;24(11):12231227.
  14. Stein J, Payne C, Methvin A, et al. Reorganizing a hospital ward as an accountable care unit. J Hosp Med. 2015;10(1):3640.
  15. Boxer R, Vitale M, Gershanik E, et al. 5th time's a charm: creation of unit‐based care teams in a high occupancy hospital [abstract]. J Hosp Med. 2015;10 (suppl. 2). Available at: http://www.shmabstracts.com/abstract/5th‐times‐a‐charm‐creation‐of‐unit‐based‐care‐teams‐in‐a‐high‐occupancy‐hospital. Accessed July 28, 2015.
  16. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998;36(1):827.
  17. O'Leary KJ, Ritter CD, Wheeler H, Szekendi MK, Brinton TS, Williams MV. Teamwork on inpatient medical units: assessing attitudes and barriers. Qual Saf Health Care. 2010;19(2):117121.
  18. Aronson MD, Neeman N, Carbo A, et al. A model for quality improvement programs in academic departments of medicine. Am J Med. 2008;121(10):922929.
  19. Narasimhan M, Eisen LA, Mahoney CD, Acerra FL, Rosen MJ. Improving nurse‐physician communication and satisfaction in the intensive care unit with a daily goals worksheet. Am J Crit Care. 2006;15(2):217222.
  20. Pronovost P, Berenholtz S, Dorman T, Lipsett PA, Simmonds T, Haraden C. Improving communication in the ICU using daily goals. J Crit Care. 2003;18(2):7175.
  21. Haller G, Garnerin P, Morales MA, et al. Effect of crew resource management training in a multidisciplinary obstetrical setting. Int J Qual Health Care. 2008;20(4):254263.
  22. Morey JC, Simon R, Jay GD, et al. Error reduction and performance improvement in the emergency department through formal teamwork training: evaluation results of the MedTeams project. Health Serv Res. 2002;37(6):15531581.
  23. Nielsen PE, Goldman MB, Mann S, et al. Effects of teamwork training on adverse outcomes and process of care in labor and delivery: a randomized controlled trial. Obstet Gynecol. 2007;109(1):4855.
  24. Pannick S, Davis R, Ashrafian H, et al. Effects of interdisciplinary team care interventions on general medical wards: a systematic review. JAMA Intern Med. 2015;175(8):12881298.
  25. Bates DW, Miller EB, Cullen DJ, et al. Patient risk factors for adverse drug events in hospitalized patients. ADE Prevention Study Group. Arch Intern Med. 1999;159(21):25532560.
  26. Kripalani S, Roumie CL, Dalal AK, et al. Effect of a pharmacist intervention on clinically important medication errors after hospital discharge: a randomized trial. Ann Intern Med. 2012;157(1):110.
  27. Donihi AC, DiNardo MM, DeVita MA, Korytkowski MT. Use of a standardized protocol to decrease medication errors and adverse events related to sliding scale insulin. Qual Saf Health Care. 2006;15(2):8991.
  28. Kaboli PJ, Hoth AB, McClimon BJ, Schnipper JL. Clinical pharmacists and inpatient medical care: a systematic review. Arch Intern Med. 2006;166(9):955964.
  29. Maynard G, Lee J, Phillips G, Fink E, Renvall M. Improved inpatient use of basal insulin, reduced hypoglycemia, and improved glycemic control: effect of structured subcutaneous insulin orders and an insulin management algorithm. J Hosp Med. 2009;4(1):315.
  30. Schnipper JL, Liang CL, Ndumele CD, Pendergrass ML. Effects of a computerized order set on the inpatient management of hyperglycemia: a cluster‐randomized controlled trial. Endocr Pract. 2010;16(2):209218.
  31. Schnipper JL, Ndumele CD, Liang CL, Pendergrass ML. Effects of a subcutaneous insulin protocol, clinical education, and computerized order set on the quality of inpatient management of hyperglycemia: results of a clinical trial. J Hosp Med. 2009;4(1):1627.
Article PDF
Issue
Journal of Hospital Medicine - 11(9)
Page Number
620-627
Sections
Files
Files
Article PDF
Article PDF

Failures in communication among healthcare professionals are known threats to patient safety. These failures account for over 60% of root causes of sentinel events, the most serious events reported to The Joint Commission.[1] As such, identifying both patterns of effective communication as well as barriers to successful communication has been a focus of efforts aimed at improving patient safety. However, to date, the majority of this work has centered on improving communication in settings such as the operating room and intensive care unit,[2, 3, 4] or at times of care transitions.[5, 6, 7, 8]

Unique barriers exist for effective interdisciplinary communication in the hospital setting, particularly physiciannurse communication regarding shared hospitalized patients.[9] Traditionally, care of hospitalized patients is provided by physicians, nurses, and other team members working in varied workflow patterns, leading to dispersed team membership, where each team member cares for different groups of patients in different locations across the hospital. This dispersion is further heightened on teaching services, where residents' rotation schedules lead to frequent changes of care team membership, leaving inpatient care teams particularly vulnerable to ineffective communication. Evidence suggests that communication between nurses and physicians is currently suboptimal, leading to frequent disagreement regarding the patient's plan of care.[9, 10] This divergence between physician and nursing perceptions of patients' care plans may leave patients at greater risk of adverse events (AEs).

Several studies have examined the effects of regionalized inpatient care teams, where multidisciplinary team members care for the same patients on the same hospital unit, on communication and patient outcomes.[4, 11, 12, 13, 14] Results of these studies have been inconsistent, perhaps due to the particular characteristics of the care teams or to the study methodology. Thus, further rigorously done studies are required to better understand the impact of team regionalization on patient care. The goal of this study was to examine whether the implementation of regionalized inpatient care teams was associated with improvements in care team communication and preventable AEs.

METHODS

Setting, Patients, and Study Design

We performed a cohort analysis of patients at a 700‐bed tertiary care center, pre‐ and postregionalization of inpatient general medicine care teams. Our study protocol was approved by the Partners Healthcare Human Subjects Review Committee. Patients were eligible for inclusion if they were 18 years of age or older and discharged from the general medicine service (GMS) from any of the 3 participating nursing units between April 1, 2012 and June 19, 2012 (preregionalization) or April 1, 2013 and June 19, 2013 (postregionalization).

Intervention

On June 20, 2012, regionalized care was implemented on the GMS such that each of 3 GMS teams was localized to 1 of 3, 15‐bed nursing units. Prior to regionalization, the GMS physician care teams, each consisting of 1 hospitalist attending, 1 medical resident, and 2 medical interns, would care for patients on an average of 7 and up to 13 different nursing units on a given day.

Regionalized care consisted of a multifaceted intervention codeveloped by hospitalist, residency, nursing, emergency department, and hospital leadership and included: (1) regionalizing GMS teams as much as possible; (2) change in resident call structure from a traditional 4‐day call cycle to daily admitting; (3) collaborative efforts to enhance GMS patient discharges before noon to promote regionalized placement of patients without prolonging time in the emergency department (ED); (4) daily morning and postround multidisciplinary huddles to prioritize sicker patients and discharges; (5) encouragement of daily rounds at patients' bedsides with presence of physician team, nurse, and team pharmacist if available; (6) creation of unit‐ and team‐level performance reports; and (7) creation of unit‐based physician and nursing co‐leadership (Figure 1).[15]

Figure 1
Regionalization of general medical services into united‐based care teams. Regionalization of general medical services involved included localizing each physician care team to a single nursing unit. Physician care teams included shared patient care responsibilities between a day team consisting of an attending hospitalist (A), a daytime resident (DR), and 2 daytime interns (DI), and a “twilight team” consisting of a twilight resident (TR) and twilight intern (TI), limiting hours of cross‐coverage by a night‐float resident (NF‐R). In addition, structured interdisciplinary structured huddles were scheduled throughout the day to identify workflow needs (eg, calling interpreter prior to bedside rounds), create patient care plans, and anticipate patient discharges. This creates a virtuous cycle of shared responsibility between care team members to improve efficiency, create earlier bed availability, and improve regionalization. Abbreviations: AM = Morning, CC = Care Coordinator, RN = Nurse, N = Nurse, OT = Occupational Therapist, PM = Evening, PT = Physical Therapist, SW = Social Worker.

Concordance of Plan

Concordance of plan was measured via a 7‐question survey previously developed, pilot tested, and used to measure the impact of regionalized care on care team communication between inpatient nursephysician team members.[9] The survey was administered in‐person by 1 of 8 trained research assistants (RAs) (4/emntervention period) to nurse and intern pairs caring for patients on the study units pre‐ and postregionalization. GMS patients were eligible for inclusion if surveys could be administered to their nurse and intern within the first 24 hours of admission to the unit and within 48 hours of admission to the hospital, based on RA availability (thus excluding patients admitted on Fridays as surveys were not conducted over the weekend). Most often, all eligible patients admitted to the study units during time periods of data collection were included in the study. On limited occasions, the daily supply of patients surpassed RA capacity for inclusion, at which time computer‐generated randomization was utilized to randomly select patients for inclusion. Nurse and intern pairs were surveyed once during a patient's hospitalization, although they could be surveyed more than once about different patients, and patients could be included more than once if rehospitalized on the study unit and cared for by a different nurseintern pair. Of the 472 selected eligible patients, the nurses and interns of 418 patients were available and consented to survey administration, representing 361 unique nurse and intern pairs and 399 unique patients.

Each member of the pair was asked about 7 specific aspects of the patient's care plan for that day in isolation from the other team member, including: (1) the patient's primary diagnosis, (2) the patient's expressed chief concern, (3) the day's scheduled tests, (4) the day's scheduled procedures, (5) consulting services involved, (6) medication changes made that day, and (7) the patient's expected discharge date. In addition, each pair was asked the name of the other team member (ie, the nurse was asked the name of the intern and vice versa), and whether or not the patient care plan for the day had been discussed with the other team member, where concordance was defined as both members agreeing the plan had been discussed. All responses were recorded verbatim. Pairs were surveyed independently between 12 pm and 2 pm, limiting confounding by evolving plans of care over time.

Each set of surveys were then reviewed by 2 of 4 trained adjudicators, and responses to each question were scored as complete, partial, or no agreement. Rules for degree of agreement were based upon previously utilized parameters[9] as well as biweekly meetings during which common themes and disagreements in ratings were discussed, and rules generated to create consensus (see Supporting Information, Appendix, in the online version of this article).

Adverse Event Detection

Of the patients meeting eligibility criteria, 200 patients were randomly selected using computer‐generated randomization from each time period for AE outcome assessment, for a total of 400 patients.

Each patient's electronic medical record was retrospectively reviewed by a trained clinician using a previously validated screening tool to detect any possible AEs.[11] Any positive screen prompted documentation of a narrative summary including a short description of the possible AE and pertinent associated data. We defined AE as any injury due to medical management rather than the natural history of the illness, and further limited this definition to only include AEs that occurred on the study unit or as a result of care on that unit.

Two of 4 trained adjudicators, blinded to time period, then separately reviewed each narrative summary using previously validated 6‐point confidence scales to determine the presence and preventability of AE, with confidence ratings of 4 or greater used as cutoffs.[11] All AEs were also scored on a 4‐point severity scale (trivial, clinically significant, serious, or life threatening), with severe AE defined as serious or life threatening. Lastly, adjudicators grouped AEs into 1 of 10 prespecified categories.[11] Any disagreements in ratings or groupings were discussed by all 4 adjudicators to reach consensus.

Data Analysis

Patient characteristics are presented using descriptive statistics and were compared in the pre‐ and postregionalization time periods using 2 or t tests as appropriate.

To analyze whether regionalized care was associated with concordance of plan, adjudicated survey questions were assigned points of 1, 0.5, and 0 for complete, partial, and no agreement, respectively. Total mean concordance scores for any patient ranged from 0 to 7 points, and were divided by total number of answered questions (up to 7) for a range of 0 to 1. Total mean concordance scores as well as mean concordance score per survey question were compared pre‐ versus postregionalization using t tests. In sensitivity analyses, adjudicated survey responses were dichotomized with complete and partial agreement deemed concordant responses. Percent concordance for each question was then compared pre‐ versus postregionalization using 2 analysis. Questions about the name of the other team member and discussion of daily care plan with the other team member were excluded from total concordance score calculations and were compared individually pre‐ versus postregionalization, because they are not directly about the plan of care.

To analyze the association of regionalization with odds of preventable AE, we performed multivariable logistic regression adjusted for patient age, sex, race, language, and Elixhauser comorbidity score,[16] and utilized generalized estimating equations to account for clustering by hospital unit. Secondary outcomes included severe preventable AEs, nonpreventable AEs, and category of preventable AEs using similar methodology. Two‐sided P values <0.05 were considered significant, and SAS version 9.2 (SAS Institute Inc., Cary, NC) was used for all analyses.

RESULTS

The fidelity of the intervention in achieving its goal of regionalized care is discussed separately.[15] Briefly, the intervention was successful at achieving 85% regionalization by team (ie, average daily percentage of team's patients assigned to team's unit) and 87% regionalization by unit (ie, average daily percentage of unit's patients with assigned team) following implementation, compared to 20% regionalization by team and unit in the preintervention period. Importantly, the average daily census of physician care teams rose by 32%, from a mean of 10.8 patients/physician care team preregionalization to a mean of 14.3 patients/physician care team postregionalization.

Concordance of Plan

Of the 418 nurse and intern paired surveys, 4 surveys were excluded due to repeat surveys of the same patient during the same hospitalization, for a total of 197 distinct paired surveys preregionalization and 217 paired surveys postregionalization. There were no statistically significant differences in patients' age, sex, race, language, admission source, length of stay, Elixhauser comorbidity score and diagnosis‐related group weight pre‐ versus postregionalization (Table 1).

Baseline Characteristics
Characteristic Concordance of Care Plan Adverse Events
Pre, n = 197 Post, n = 217 P Value Pre, n = 198 Post, n = 194 P Value
  • NOTE: Abbreviations: DRG, diagnosis‐related group; IQR, interquartile range; SD, standard deviation.

Age, mean (SD) 60.5 (19.4) 57.6 (20.8) 0.15 60.4 (18.9) 58.0 (21.2) 0.24
Male, n (%) 77 (39.1) 92 (42.4) 0.49 94 (47.5) 85 (43.8) 0.55
Race/ethnicity, n (%) 0.34 0.12
White 134 (68.0) 141 (65.0) 132 (66.5) 121 (62.4)
Black 42 (21.3) 45 (20.7) 41 (20.8) 54 (27.8)
Hispanic 18 (9.1) 21 (9.7) 22 (11.3) 13 (6.8)
Other/unknown 3 (1.5) 10 (4.6) 3 (1.4) 6 (2.9)
Language, n (%) 0.30 0.73
English 183 (92.9) 203 (93.5) 176 (88.7) 175 (90.2)
Spanish 6 (3.0) 10 (4.6) 10 (5.2) 10 (5.3)
Other 8 (4.1) 4 (1.8) 12 (6.1) 9 (4.5)
Admitting source, n (%) 1.00 0.10
Physician office 13 (6.6) 13 (6.0) 13 (6.6) 6 (3.1)
Emergency department 136 (69.0) 150 (69.1) 126 (63.6) 127 (65.5)
Transfer from different hospital 40 (20.3) 45 (20.7) 54 (27.3) 50 (25.8)
Transfer from skilled nursing facility 8 (4.1) 9 (4.2) 5 (2.5) 11 (5.6)
Length of stay, d, median (IQR) 3.0 (4.0) 3.0 (4.0) 0.57 4.0 (5.0) 3.0 (4.0) 0.16
Elixhauser Comorbidity Score, mean (SD) 8.0 (8.8) 8.3 (9.3) 0.74 8.0 (8.6) 7.8 (8.4) 0.86
DRG weight, mean (SD) 1.6 (1.0) 1.5 (1.0) 0.37 1.5 (0.93) 1.5 (1.1) 0.96

Kappa scores for adjudications of concordance surveys (defined as both adjudicators scoring the same level of agreement (ie, both complete or partial agreement versus no agreement) ranged from 0.69 to 0.95, by question. There were no significant differences in total mean concordance scores in the care plan pre‐ versus postregionalization (0.65 vs 0.67, P = 0.26) (Table 2). Similarly, there were no significant differences in mean concordance score for each survey question, except agreement on expected date of discharge (0.56 vs 0.68, P = 0.003), knowledge of the other provider's name, and agreement that discussion of the daily plan had taken place with the other pair member. Similar results were seen when results were dichotomized (ie, partial or complete agreement vs no agreement) (Table 2).

Effect of Regionalized Care on Concordance of Care Plan between Primary Nurse and Responding Physician
Concordance Outcome Pre, n = 197 Post, n = 217 P Value
  • NOTE: Abbreviations: SD, standard deviation. *Calculation of concordance score: agree = 1 point, partial agreement = 0.5 points, disagree = 0 points. Total concordance score excluded the following survey question responses: knowledge of other team member name and plan discussed. Concordance defined as agree or partial agreement. For responding clinician knowledge of nurse's name, nurse's knowledge of responding clinician's name, and plan discussed, all paired survey responses were either agree (1) or disagree (0).

Concordance score*
Total concordance score, mean (SD) 0.65 (0.17) 0.67 (0.16) 0.26
Subgroups
Diagnosis 0.77 (0.32) 0.72 (0.35) 0.11
Patient's chief concern 0.48 (0.44) 0.48 (0.43) 0.94
Tests today 0.67 (0.40) 0.71 (0.42) 0.36
Procedures today 0.93 (0.25) 0.92 (0.25) 0.71
Medication changes today 0.56 (0.44) 0.59 (0.43) 0.54
Consulting services 0.59 (0.44) 0.60 (0.44) 0.82
Expected discharge date 0.56 (0.44) 0.68 (0.38) 0.003
Responding clinician knowledge of nurse's name 0.56 (0.50) 0.86 (0.35) <0.001
Nurse's knowledge of responding clinician's name 0.56 (0.50) 0.88 (0.33) <0.001
Plan discussed 0.73 (0.45) 0.88 (0.32) <0.001
Percent concordance, mean (SD)
Diagnosis 92.0 (27.3) 88.6 (31.9) 0.25
Patient's chief concern 59.6 (49.1) 60.6 (49.0) 0.84
Tests today 78.9 (40.9) 77.2 (42.1) 0.67
Procedures today 93.5 (24.8) 94.1 (23.7) 0.80
Medication changes today 66.3 (33.6) 69.9 (46.0) 0.44
Consulting services 69.3 (46.2) 68.9 (46.4) 0.93
Expected discharge date 67.5 (47.0) 82.6 (38.0) <0.001
Responding clinician knowledge of nurse's name 55.7 (49.8) 85.6 (35.2) <0.001
Nurse's knowledge of responding clinician's name 55.9 (49.8) 87.9 (32.8) <0.001
Plan discussed 72.9 (44.6) 88.2 (32.3) <0.001

Adverse Events

Of the 400 patients screened for AEs, 8 were excluded due to missing medical record number (5) and discharge outside of study period (3). Of the final 392 patient screens (198 pre, 194 post), there were no significant differences in patients' age, sex, race, language, length of stay, or Elixhauser score pre‐ versus postregionalization (Table 1).

Kappa scores for adjudicator agreement were 0.35 for presence of AE and 0.34 for preventability of AE. Of the 392 reviewed patient records, there were 133 total AEs detected (66 pre, 67 post), 27 preventable AEs (13 pre, 14 post), and 9 severe preventable AEs (4 pre, 5 post) (Table 3). There was no significant difference in the adjusted odds of preventable AEs post‐ versus preregionalization (adjusted odds ratio: 1.37, 95% confidence interval: 0.69, 2.69). Although the low number of AEs rated as severe or life threatening precluded adjusted analysis, unadjusted results similarly demonstrated no difference in odds of severe preventable AEs pre‐ versus postregionalization. As expected, there was no significant difference in adjusted odds of nonpreventable AE after implementation of regionalized care (Table 3).

Adjusted Effect of Regionalization on Adverse Events*
Adverse Events No. of Adverse Events Adjusted Odds Ratio Post vs Pre (95% CI)
Pre, n = 198 Post, n = 194
  • NOTE: Abbreviations: CI, confidence interval. *Adjusted for patient age, sex, race, language, and comorbidity as measured by the Elixhauser score. Number of events precluded adjusted analysis. Unadjusted odds ratio = 1.30 (0.34, 4.91).

Preventable 13 14 1.37 (0.69, 2.69)
Serious and preventable 4 5
Nonpreventable 47 50 1.20 (0.85, 1.75)

Similarly, there were no significant differences in category of preventable AE pre‐ versus postregionalization. The most frequent preventable AEs in both time periods were those related to adverse drug events and to manifestations of poor glycemic control, examples of which are illustrated (Table 4).

Examples of Preventable Adverse Events Due to Adverse Drug Events and Manifestations of Poor Glycemic Control
  • NOTE: Abbreviations: PNR, pro re nata (as needed).

Adverse drug event 29‐year‐old male with history of alcohol abuse, complicated by prior withdrawal seizures/emntensive care unit admissions, presented with alcohol withdrawal. Started on standing and PRN lorazepam, kept on home medications including standing clonidine, gabapentin, citalopram, quetiapine. Became somnolent due to polypharmacy, ultimately discontinued quetiapine as discovered took only as needed at home for insomnia
Manifestations of poor glycemic control 78‐year‐old male with recently diagnosed lymphoma, distant history of bladder and prostate cancer status post ileal loop diversion, presented status post syncopal event; during event, spilled boiling water on himself leading to second‐degree burns on 3% of his body. Initially admitted to trauma/burn service, ultimately transferred to medical service for ongoing multiple medical issues including obstructive uropathy, acute on chronic renal failure. Adverse event was hyperglycemia (>350 mg/dL on >2 consecutive readings) in the setting of holding his home insulin detemir and insulin aspart (had been placed on insulin aspart sliding scale alone). After hyperglycemic episodes, was placed back on weight‐based basal/nutritional insulin

DISCUSSION

In this study of general medicine patients at a large academic medical center, we found that regionalization of care teams on general medicine services was associated with improved recognition of care team members and agreement on estimated date of patient discharge, but was not associated with improvement in overall nurse and physician concordance of the patient care plan, or the odds of preventable AEs.

This intervention importantly addresses the barrier of dispersion of team membership, a well‐recognized barrier to interdisciplinary collaboration,[17, 18] particularly with resident physician teams due to frequently changing team membership. Localization of all team members, in addition to encouragement of daily collaborative bedside rounds as part of the regionalization initiative, likely contributed to our observed improvement in team member identification and discussion of daily care plans. Similarly, regionalization resulted in improved agreement in estimations of date of patient discharge. Focus on early patient discharges was an integral part of the implementation efforts; we therefore hypothesize that mutual focus on discharge planning by both nurses and responding clinicians may have explained this observed result.

On the other hand, regionalization did not appreciably improve the overall concordance of care plan between nurses and interns, despite a significant increase in team members agreeing that the plan had been discussed. Our findings support similar prior research demonstrating that regionalizing hospitalist attendings to single nursing units had limited impact on agreement of care plan between physicians and nurses.[13] Similarly, in settings where physicians and nurses are inherently regionalized, such as the intensive care unit[4] or the operating room,[3] communication between physicians and nurses remains difficult. Collectively, our findings suggest that colocalization of physicians and nurses alone is likely insufficient to improve measured communication between care team members. Existing literature suggests that more standardized approaches to improve communication, such as structured communication tools used during daily inpatient care[19, 20] or formalized team training,[21, 22, 23] lead to improvements in communication and collaboration. Despite these findings, it is important to highlight that this study did not assess other measures of workplace culture, such as teamwork and care team cohesiveness, which may have been positively affected by this intervention, even without measurable effect on concordance of care plan. Additionally, as noted, the average daily census on each team increased by almost a third postintervention, which may have impeded improvements in care team communication.

In addition, we found that our intervention had no significant impact on preventable AEs or severe preventable AEs. Although we cannot exclude the possibility that more subtle AEs were missed with our methodology, our results indicate that regionalized care alone may be inadequate to improve major patient safety outcomes. As discussed, the volume of patients did increase postintervention; thus, another way to state our results is that we were able to increase the daily volume of patients without any significant decreases in patient safety. Nevertheless, the results on patient safety were less than desired. A recent review of interdisciplinary team care interventions on general medical wards similarly demonstrated underwhelming improvements in patient safety outcomes, although the reviewed interventions did not specifically address preventable AEs, a gap in the literature commented on by the authors.[24] Other albeit limited literature has demonstrated improvement in patient safety outcomes via multifaceted efforts aimed at improving care team member communication. Notably, these efforts include colocalization of care team members to single units but also involve additional measures to improve communication and collaboration between care team members, such as structured communication during interdisciplinary rounds, and certification of key interdisciplinary teamwork skills.[11, 14] Although our regionalized care intervention included many similar features to these accountable care units (ACUs) including unit‐based care teams, unit‐level performance reporting, and unit‐based physician and nursing coleadership, significant differences existed. Notably, in addition to the above features, the ACU model also incorporated highly structured communication models for interdisciplinary rounding, and certification processes to ensure an appropriate communication skill base among care team members.[14] Thus, although creation of regionalized care teams is likely a necessary precursor to implementation of these additional measures, alone it may be insufficient to improve patient safety outcomes.

Importantly, in our study we identified that adverse drug events and manifestations of poor glycemic control occurred in high frequency both before and following implementation of regionalized care, supporting other literature that describes the prevalence of these AEs.[11, 25, 26, 27] These results suggest that targeted interventions to address these specific AEs are likely necessary. Notably, the intervention units in our study did not consistently employ clinical pharmacists assigned specifically to that unit's care team to allow for integration within the care team. As prior research has suggested that greater collaboration with clinical pharmacists results in reduction of adverse drug events,[28] next steps may include improved integration of team‐based pharmacists into the activities of the regionalized care teams. Inpatient management of diabetes also requires specific interventions,[29, 30, 31] only some of which may be addressable by having regionalized care and better interdisciplinary communication.

Our findings are subject to several limitations. First, this was a single‐site study and thus our findings may not be generalizable to other institutions. However, regionalized care is increasingly encouraged to optimize communication between care team members.[17, 18] Therefore, our null findings may be pertinent to other institutions looking to improve patient safety outcomes, demonstrating that additional initiatives will likely be required. Second, our modes of outcome measurement possess limitations. In measuring concordance of care plan, although previously used survey techniques were employed,[9] the concordance survey has not been formally validated, and we believe some of the questions may have led to ambiguity on the part of the responders that may have resulted in less accurate responses, thus biasing toward the null. Similarly, in measuring AEs, the screening tool relied on retrospective chart review looking for specific AE types[11] and thus may not have captured more subtle AEs. Additionally, our study may have been underpowered to demonstrate significant reduction in preventable AEs, although other studies of similar methodology demonstrated significant results with similar sample size.[11] This was due in part to our lower‐than‐expected baseline AE rate (6.6% compared with approximately 10.3% in previous studies).[11] Lastly, our study solely examined the association of regionalization with concordance of care plan and preventable AEs, but importantly excluded other clinically important outcomes that may have been positively (or negatively) impacted by these regionalization efforts, such as ED wait times, provider efficiency (eg, fewer pages, less time in transit, more time at the bedside), interdisciplinary teamwork, or patient or provider satisfaction.

CONCLUSION

In summary, our findings suggest that regionalized care teams alone may be insufficient to effectively promote communication between care team members regarding the care plan or to lead to improvements in patient safety, although we recognize that there may have been benefits (or unintended harms) not measured in this study but are nonetheless important for clinical care and workplace culture. This is an important lesson, as many hospitals move toward regionalized care in an effort to improve patient safety outcomes. However, strengthening the infrastructure by colocalizing care team members to maximize opportunity for communication is likely a necessary first step toward facilitating implementation of additional initiatives that may lead to more robust patient safety improvements, such as structured interdisciplinary bedside rounds (eg, facilitating and training all team members to fulfill specific roles), teamwork training, and certification of key interdisciplinary teamwork skills. Additionally, close examination of identified prevalent and preventable AEs can help to determine which additional initiatives are most likely to have greatest impact in improving patient safety.

Disclosures: This research was supported by funds provided by Brigham and Women's Hospital (BWH) and by funds provided by the Department of Medicine at BWH. All authors had full access to all of the data in the study and were integrally involved in the design, implementation, data collection, and analyses. The first author, Dr. Stephanie Mueller, takes responsibility for the integrity for the data and the accuracy of the data analysis. Dr. Schnipper reports grants from Sanofi Aventis, outside the submitted work.

Failures in communication among healthcare professionals are known threats to patient safety. These failures account for over 60% of root causes of sentinel events, the most serious events reported to The Joint Commission.[1] As such, identifying both patterns of effective communication as well as barriers to successful communication has been a focus of efforts aimed at improving patient safety. However, to date, the majority of this work has centered on improving communication in settings such as the operating room and intensive care unit,[2, 3, 4] or at times of care transitions.[5, 6, 7, 8]

Unique barriers exist for effective interdisciplinary communication in the hospital setting, particularly physiciannurse communication regarding shared hospitalized patients.[9] Traditionally, care of hospitalized patients is provided by physicians, nurses, and other team members working in varied workflow patterns, leading to dispersed team membership, where each team member cares for different groups of patients in different locations across the hospital. This dispersion is further heightened on teaching services, where residents' rotation schedules lead to frequent changes of care team membership, leaving inpatient care teams particularly vulnerable to ineffective communication. Evidence suggests that communication between nurses and physicians is currently suboptimal, leading to frequent disagreement regarding the patient's plan of care.[9, 10] This divergence between physician and nursing perceptions of patients' care plans may leave patients at greater risk of adverse events (AEs).

Several studies have examined the effects of regionalized inpatient care teams, where multidisciplinary team members care for the same patients on the same hospital unit, on communication and patient outcomes.[4, 11, 12, 13, 14] Results of these studies have been inconsistent, perhaps due to the particular characteristics of the care teams or to the study methodology. Thus, further rigorously done studies are required to better understand the impact of team regionalization on patient care. The goal of this study was to examine whether the implementation of regionalized inpatient care teams was associated with improvements in care team communication and preventable AEs.

METHODS

Setting, Patients, and Study Design

We performed a cohort analysis of patients at a 700‐bed tertiary care center, pre‐ and postregionalization of inpatient general medicine care teams. Our study protocol was approved by the Partners Healthcare Human Subjects Review Committee. Patients were eligible for inclusion if they were 18 years of age or older and discharged from the general medicine service (GMS) from any of the 3 participating nursing units between April 1, 2012 and June 19, 2012 (preregionalization) or April 1, 2013 and June 19, 2013 (postregionalization).

Intervention

On June 20, 2012, regionalized care was implemented on the GMS such that each of 3 GMS teams was localized to 1 of 3, 15‐bed nursing units. Prior to regionalization, the GMS physician care teams, each consisting of 1 hospitalist attending, 1 medical resident, and 2 medical interns, would care for patients on an average of 7 and up to 13 different nursing units on a given day.

Regionalized care consisted of a multifaceted intervention codeveloped by hospitalist, residency, nursing, emergency department, and hospital leadership and included: (1) regionalizing GMS teams as much as possible; (2) change in resident call structure from a traditional 4‐day call cycle to daily admitting; (3) collaborative efforts to enhance GMS patient discharges before noon to promote regionalized placement of patients without prolonging time in the emergency department (ED); (4) daily morning and postround multidisciplinary huddles to prioritize sicker patients and discharges; (5) encouragement of daily rounds at patients' bedsides with presence of physician team, nurse, and team pharmacist if available; (6) creation of unit‐ and team‐level performance reports; and (7) creation of unit‐based physician and nursing co‐leadership (Figure 1).[15]

Figure 1
Regionalization of general medical services into united‐based care teams. Regionalization of general medical services involved included localizing each physician care team to a single nursing unit. Physician care teams included shared patient care responsibilities between a day team consisting of an attending hospitalist (A), a daytime resident (DR), and 2 daytime interns (DI), and a “twilight team” consisting of a twilight resident (TR) and twilight intern (TI), limiting hours of cross‐coverage by a night‐float resident (NF‐R). In addition, structured interdisciplinary structured huddles were scheduled throughout the day to identify workflow needs (eg, calling interpreter prior to bedside rounds), create patient care plans, and anticipate patient discharges. This creates a virtuous cycle of shared responsibility between care team members to improve efficiency, create earlier bed availability, and improve regionalization. Abbreviations: AM = Morning, CC = Care Coordinator, RN = Nurse, N = Nurse, OT = Occupational Therapist, PM = Evening, PT = Physical Therapist, SW = Social Worker.

Concordance of Plan

Concordance of plan was measured via a 7‐question survey previously developed, pilot tested, and used to measure the impact of regionalized care on care team communication between inpatient nursephysician team members.[9] The survey was administered in‐person by 1 of 8 trained research assistants (RAs) (4/emntervention period) to nurse and intern pairs caring for patients on the study units pre‐ and postregionalization. GMS patients were eligible for inclusion if surveys could be administered to their nurse and intern within the first 24 hours of admission to the unit and within 48 hours of admission to the hospital, based on RA availability (thus excluding patients admitted on Fridays as surveys were not conducted over the weekend). Most often, all eligible patients admitted to the study units during time periods of data collection were included in the study. On limited occasions, the daily supply of patients surpassed RA capacity for inclusion, at which time computer‐generated randomization was utilized to randomly select patients for inclusion. Nurse and intern pairs were surveyed once during a patient's hospitalization, although they could be surveyed more than once about different patients, and patients could be included more than once if rehospitalized on the study unit and cared for by a different nurseintern pair. Of the 472 selected eligible patients, the nurses and interns of 418 patients were available and consented to survey administration, representing 361 unique nurse and intern pairs and 399 unique patients.

Each member of the pair was asked about 7 specific aspects of the patient's care plan for that day in isolation from the other team member, including: (1) the patient's primary diagnosis, (2) the patient's expressed chief concern, (3) the day's scheduled tests, (4) the day's scheduled procedures, (5) consulting services involved, (6) medication changes made that day, and (7) the patient's expected discharge date. In addition, each pair was asked the name of the other team member (ie, the nurse was asked the name of the intern and vice versa), and whether or not the patient care plan for the day had been discussed with the other team member, where concordance was defined as both members agreeing the plan had been discussed. All responses were recorded verbatim. Pairs were surveyed independently between 12 pm and 2 pm, limiting confounding by evolving plans of care over time.

Each set of surveys were then reviewed by 2 of 4 trained adjudicators, and responses to each question were scored as complete, partial, or no agreement. Rules for degree of agreement were based upon previously utilized parameters[9] as well as biweekly meetings during which common themes and disagreements in ratings were discussed, and rules generated to create consensus (see Supporting Information, Appendix, in the online version of this article).

Adverse Event Detection

Of the patients meeting eligibility criteria, 200 patients were randomly selected using computer‐generated randomization from each time period for AE outcome assessment, for a total of 400 patients.

Each patient's electronic medical record was retrospectively reviewed by a trained clinician using a previously validated screening tool to detect any possible AEs.[11] Any positive screen prompted documentation of a narrative summary including a short description of the possible AE and pertinent associated data. We defined AE as any injury due to medical management rather than the natural history of the illness, and further limited this definition to only include AEs that occurred on the study unit or as a result of care on that unit.

Two of 4 trained adjudicators, blinded to time period, then separately reviewed each narrative summary using previously validated 6‐point confidence scales to determine the presence and preventability of AE, with confidence ratings of 4 or greater used as cutoffs.[11] All AEs were also scored on a 4‐point severity scale (trivial, clinically significant, serious, or life threatening), with severe AE defined as serious or life threatening. Lastly, adjudicators grouped AEs into 1 of 10 prespecified categories.[11] Any disagreements in ratings or groupings were discussed by all 4 adjudicators to reach consensus.

Data Analysis

Patient characteristics are presented using descriptive statistics and were compared in the pre‐ and postregionalization time periods using 2 or t tests as appropriate.

To analyze whether regionalized care was associated with concordance of plan, adjudicated survey questions were assigned points of 1, 0.5, and 0 for complete, partial, and no agreement, respectively. Total mean concordance scores for any patient ranged from 0 to 7 points, and were divided by total number of answered questions (up to 7) for a range of 0 to 1. Total mean concordance scores as well as mean concordance score per survey question were compared pre‐ versus postregionalization using t tests. In sensitivity analyses, adjudicated survey responses were dichotomized with complete and partial agreement deemed concordant responses. Percent concordance for each question was then compared pre‐ versus postregionalization using 2 analysis. Questions about the name of the other team member and discussion of daily care plan with the other team member were excluded from total concordance score calculations and were compared individually pre‐ versus postregionalization, because they are not directly about the plan of care.

To analyze the association of regionalization with odds of preventable AE, we performed multivariable logistic regression adjusted for patient age, sex, race, language, and Elixhauser comorbidity score,[16] and utilized generalized estimating equations to account for clustering by hospital unit. Secondary outcomes included severe preventable AEs, nonpreventable AEs, and category of preventable AEs using similar methodology. Two‐sided P values <0.05 were considered significant, and SAS version 9.2 (SAS Institute Inc., Cary, NC) was used for all analyses.

RESULTS

The fidelity of the intervention in achieving its goal of regionalized care is discussed separately.[15] Briefly, the intervention was successful at achieving 85% regionalization by team (ie, average daily percentage of team's patients assigned to team's unit) and 87% regionalization by unit (ie, average daily percentage of unit's patients with assigned team) following implementation, compared to 20% regionalization by team and unit in the preintervention period. Importantly, the average daily census of physician care teams rose by 32%, from a mean of 10.8 patients/physician care team preregionalization to a mean of 14.3 patients/physician care team postregionalization.

Concordance of Plan

Of the 418 nurse and intern paired surveys, 4 surveys were excluded due to repeat surveys of the same patient during the same hospitalization, for a total of 197 distinct paired surveys preregionalization and 217 paired surveys postregionalization. There were no statistically significant differences in patients' age, sex, race, language, admission source, length of stay, Elixhauser comorbidity score and diagnosis‐related group weight pre‐ versus postregionalization (Table 1).

Baseline Characteristics
Characteristic Concordance of Care Plan Adverse Events
Pre, n = 197 Post, n = 217 P Value Pre, n = 198 Post, n = 194 P Value
  • NOTE: Abbreviations: DRG, diagnosis‐related group; IQR, interquartile range; SD, standard deviation.

Age, mean (SD) 60.5 (19.4) 57.6 (20.8) 0.15 60.4 (18.9) 58.0 (21.2) 0.24
Male, n (%) 77 (39.1) 92 (42.4) 0.49 94 (47.5) 85 (43.8) 0.55
Race/ethnicity, n (%) 0.34 0.12
White 134 (68.0) 141 (65.0) 132 (66.5) 121 (62.4)
Black 42 (21.3) 45 (20.7) 41 (20.8) 54 (27.8)
Hispanic 18 (9.1) 21 (9.7) 22 (11.3) 13 (6.8)
Other/unknown 3 (1.5) 10 (4.6) 3 (1.4) 6 (2.9)
Language, n (%) 0.30 0.73
English 183 (92.9) 203 (93.5) 176 (88.7) 175 (90.2)
Spanish 6 (3.0) 10 (4.6) 10 (5.2) 10 (5.3)
Other 8 (4.1) 4 (1.8) 12 (6.1) 9 (4.5)
Admitting source, n (%) 1.00 0.10
Physician office 13 (6.6) 13 (6.0) 13 (6.6) 6 (3.1)
Emergency department 136 (69.0) 150 (69.1) 126 (63.6) 127 (65.5)
Transfer from different hospital 40 (20.3) 45 (20.7) 54 (27.3) 50 (25.8)
Transfer from skilled nursing facility 8 (4.1) 9 (4.2) 5 (2.5) 11 (5.6)
Length of stay, d, median (IQR) 3.0 (4.0) 3.0 (4.0) 0.57 4.0 (5.0) 3.0 (4.0) 0.16
Elixhauser Comorbidity Score, mean (SD) 8.0 (8.8) 8.3 (9.3) 0.74 8.0 (8.6) 7.8 (8.4) 0.86
DRG weight, mean (SD) 1.6 (1.0) 1.5 (1.0) 0.37 1.5 (0.93) 1.5 (1.1) 0.96

Kappa scores for adjudications of concordance surveys (defined as both adjudicators scoring the same level of agreement (ie, both complete or partial agreement versus no agreement) ranged from 0.69 to 0.95, by question. There were no significant differences in total mean concordance scores in the care plan pre‐ versus postregionalization (0.65 vs 0.67, P = 0.26) (Table 2). Similarly, there were no significant differences in mean concordance score for each survey question, except agreement on expected date of discharge (0.56 vs 0.68, P = 0.003), knowledge of the other provider's name, and agreement that discussion of the daily plan had taken place with the other pair member. Similar results were seen when results were dichotomized (ie, partial or complete agreement vs no agreement) (Table 2).

Effect of Regionalized Care on Concordance of Care Plan between Primary Nurse and Responding Physician
Concordance Outcome Pre, n = 197 Post, n = 217 P Value
  • NOTE: Abbreviations: SD, standard deviation. *Calculation of concordance score: agree = 1 point, partial agreement = 0.5 points, disagree = 0 points. Total concordance score excluded the following survey question responses: knowledge of other team member name and plan discussed. Concordance defined as agree or partial agreement. For responding clinician knowledge of nurse's name, nurse's knowledge of responding clinician's name, and plan discussed, all paired survey responses were either agree (1) or disagree (0).

Concordance score*
Total concordance score, mean (SD) 0.65 (0.17) 0.67 (0.16) 0.26
Subgroups
Diagnosis 0.77 (0.32) 0.72 (0.35) 0.11
Patient's chief concern 0.48 (0.44) 0.48 (0.43) 0.94
Tests today 0.67 (0.40) 0.71 (0.42) 0.36
Procedures today 0.93 (0.25) 0.92 (0.25) 0.71
Medication changes today 0.56 (0.44) 0.59 (0.43) 0.54
Consulting services 0.59 (0.44) 0.60 (0.44) 0.82
Expected discharge date 0.56 (0.44) 0.68 (0.38) 0.003
Responding clinician knowledge of nurse's name 0.56 (0.50) 0.86 (0.35) <0.001
Nurse's knowledge of responding clinician's name 0.56 (0.50) 0.88 (0.33) <0.001
Plan discussed 0.73 (0.45) 0.88 (0.32) <0.001
Percent concordance, mean (SD)
Diagnosis 92.0 (27.3) 88.6 (31.9) 0.25
Patient's chief concern 59.6 (49.1) 60.6 (49.0) 0.84
Tests today 78.9 (40.9) 77.2 (42.1) 0.67
Procedures today 93.5 (24.8) 94.1 (23.7) 0.80
Medication changes today 66.3 (33.6) 69.9 (46.0) 0.44
Consulting services 69.3 (46.2) 68.9 (46.4) 0.93
Expected discharge date 67.5 (47.0) 82.6 (38.0) <0.001
Responding clinician knowledge of nurse's name 55.7 (49.8) 85.6 (35.2) <0.001
Nurse's knowledge of responding clinician's name 55.9 (49.8) 87.9 (32.8) <0.001
Plan discussed 72.9 (44.6) 88.2 (32.3) <0.001

Adverse Events

Of the 400 patients screened for AEs, 8 were excluded due to missing medical record number (5) and discharge outside of study period (3). Of the final 392 patient screens (198 pre, 194 post), there were no significant differences in patients' age, sex, race, language, length of stay, or Elixhauser score pre‐ versus postregionalization (Table 1).

Kappa scores for adjudicator agreement were 0.35 for presence of AE and 0.34 for preventability of AE. Of the 392 reviewed patient records, there were 133 total AEs detected (66 pre, 67 post), 27 preventable AEs (13 pre, 14 post), and 9 severe preventable AEs (4 pre, 5 post) (Table 3). There was no significant difference in the adjusted odds of preventable AEs post‐ versus preregionalization (adjusted odds ratio: 1.37, 95% confidence interval: 0.69, 2.69). Although the low number of AEs rated as severe or life threatening precluded adjusted analysis, unadjusted results similarly demonstrated no difference in odds of severe preventable AEs pre‐ versus postregionalization. As expected, there was no significant difference in adjusted odds of nonpreventable AE after implementation of regionalized care (Table 3).

Adjusted Effect of Regionalization on Adverse Events*
Adverse Events No. of Adverse Events Adjusted Odds Ratio Post vs Pre (95% CI)
Pre, n = 198 Post, n = 194
  • NOTE: Abbreviations: CI, confidence interval. *Adjusted for patient age, sex, race, language, and comorbidity as measured by the Elixhauser score. Number of events precluded adjusted analysis. Unadjusted odds ratio = 1.30 (0.34, 4.91).

Preventable 13 14 1.37 (0.69, 2.69)
Serious and preventable 4 5
Nonpreventable 47 50 1.20 (0.85, 1.75)

Similarly, there were no significant differences in category of preventable AE pre‐ versus postregionalization. The most frequent preventable AEs in both time periods were those related to adverse drug events and to manifestations of poor glycemic control, examples of which are illustrated (Table 4).

Examples of Preventable Adverse Events Due to Adverse Drug Events and Manifestations of Poor Glycemic Control
  • NOTE: Abbreviations: PNR, pro re nata (as needed).

Adverse drug event 29‐year‐old male with history of alcohol abuse, complicated by prior withdrawal seizures/emntensive care unit admissions, presented with alcohol withdrawal. Started on standing and PRN lorazepam, kept on home medications including standing clonidine, gabapentin, citalopram, quetiapine. Became somnolent due to polypharmacy, ultimately discontinued quetiapine as discovered took only as needed at home for insomnia
Manifestations of poor glycemic control 78‐year‐old male with recently diagnosed lymphoma, distant history of bladder and prostate cancer status post ileal loop diversion, presented status post syncopal event; during event, spilled boiling water on himself leading to second‐degree burns on 3% of his body. Initially admitted to trauma/burn service, ultimately transferred to medical service for ongoing multiple medical issues including obstructive uropathy, acute on chronic renal failure. Adverse event was hyperglycemia (>350 mg/dL on >2 consecutive readings) in the setting of holding his home insulin detemir and insulin aspart (had been placed on insulin aspart sliding scale alone). After hyperglycemic episodes, was placed back on weight‐based basal/nutritional insulin

DISCUSSION

In this study of general medicine patients at a large academic medical center, we found that regionalization of care teams on general medicine services was associated with improved recognition of care team members and agreement on estimated date of patient discharge, but was not associated with improvement in overall nurse and physician concordance of the patient care plan, or the odds of preventable AEs.

This intervention importantly addresses the barrier of dispersion of team membership, a well‐recognized barrier to interdisciplinary collaboration,[17, 18] particularly with resident physician teams due to frequently changing team membership. Localization of all team members, in addition to encouragement of daily collaborative bedside rounds as part of the regionalization initiative, likely contributed to our observed improvement in team member identification and discussion of daily care plans. Similarly, regionalization resulted in improved agreement in estimations of date of patient discharge. Focus on early patient discharges was an integral part of the implementation efforts; we therefore hypothesize that mutual focus on discharge planning by both nurses and responding clinicians may have explained this observed result.

On the other hand, regionalization did not appreciably improve the overall concordance of care plan between nurses and interns, despite a significant increase in team members agreeing that the plan had been discussed. Our findings support similar prior research demonstrating that regionalizing hospitalist attendings to single nursing units had limited impact on agreement of care plan between physicians and nurses.[13] Similarly, in settings where physicians and nurses are inherently regionalized, such as the intensive care unit[4] or the operating room,[3] communication between physicians and nurses remains difficult. Collectively, our findings suggest that colocalization of physicians and nurses alone is likely insufficient to improve measured communication between care team members. Existing literature suggests that more standardized approaches to improve communication, such as structured communication tools used during daily inpatient care[19, 20] or formalized team training,[21, 22, 23] lead to improvements in communication and collaboration. Despite these findings, it is important to highlight that this study did not assess other measures of workplace culture, such as teamwork and care team cohesiveness, which may have been positively affected by this intervention, even without measurable effect on concordance of care plan. Additionally, as noted, the average daily census on each team increased by almost a third postintervention, which may have impeded improvements in care team communication.

In addition, we found that our intervention had no significant impact on preventable AEs or severe preventable AEs. Although we cannot exclude the possibility that more subtle AEs were missed with our methodology, our results indicate that regionalized care alone may be inadequate to improve major patient safety outcomes. As discussed, the volume of patients did increase postintervention; thus, another way to state our results is that we were able to increase the daily volume of patients without any significant decreases in patient safety. Nevertheless, the results on patient safety were less than desired. A recent review of interdisciplinary team care interventions on general medical wards similarly demonstrated underwhelming improvements in patient safety outcomes, although the reviewed interventions did not specifically address preventable AEs, a gap in the literature commented on by the authors.[24] Other albeit limited literature has demonstrated improvement in patient safety outcomes via multifaceted efforts aimed at improving care team member communication. Notably, these efforts include colocalization of care team members to single units but also involve additional measures to improve communication and collaboration between care team members, such as structured communication during interdisciplinary rounds, and certification of key interdisciplinary teamwork skills.[11, 14] Although our regionalized care intervention included many similar features to these accountable care units (ACUs) including unit‐based care teams, unit‐level performance reporting, and unit‐based physician and nursing coleadership, significant differences existed. Notably, in addition to the above features, the ACU model also incorporated highly structured communication models for interdisciplinary rounding, and certification processes to ensure an appropriate communication skill base among care team members.[14] Thus, although creation of regionalized care teams is likely a necessary precursor to implementation of these additional measures, alone it may be insufficient to improve patient safety outcomes.

Importantly, in our study we identified that adverse drug events and manifestations of poor glycemic control occurred in high frequency both before and following implementation of regionalized care, supporting other literature that describes the prevalence of these AEs.[11, 25, 26, 27] These results suggest that targeted interventions to address these specific AEs are likely necessary. Notably, the intervention units in our study did not consistently employ clinical pharmacists assigned specifically to that unit's care team to allow for integration within the care team. As prior research has suggested that greater collaboration with clinical pharmacists results in reduction of adverse drug events,[28] next steps may include improved integration of team‐based pharmacists into the activities of the regionalized care teams. Inpatient management of diabetes also requires specific interventions,[29, 30, 31] only some of which may be addressable by having regionalized care and better interdisciplinary communication.

Our findings are subject to several limitations. First, this was a single‐site study and thus our findings may not be generalizable to other institutions. However, regionalized care is increasingly encouraged to optimize communication between care team members.[17, 18] Therefore, our null findings may be pertinent to other institutions looking to improve patient safety outcomes, demonstrating that additional initiatives will likely be required. Second, our modes of outcome measurement possess limitations. In measuring concordance of care plan, although previously used survey techniques were employed,[9] the concordance survey has not been formally validated, and we believe some of the questions may have led to ambiguity on the part of the responders that may have resulted in less accurate responses, thus biasing toward the null. Similarly, in measuring AEs, the screening tool relied on retrospective chart review looking for specific AE types[11] and thus may not have captured more subtle AEs. Additionally, our study may have been underpowered to demonstrate significant reduction in preventable AEs, although other studies of similar methodology demonstrated significant results with similar sample size.[11] This was due in part to our lower‐than‐expected baseline AE rate (6.6% compared with approximately 10.3% in previous studies).[11] Lastly, our study solely examined the association of regionalization with concordance of care plan and preventable AEs, but importantly excluded other clinically important outcomes that may have been positively (or negatively) impacted by these regionalization efforts, such as ED wait times, provider efficiency (eg, fewer pages, less time in transit, more time at the bedside), interdisciplinary teamwork, or patient or provider satisfaction.

CONCLUSION

In summary, our findings suggest that regionalized care teams alone may be insufficient to effectively promote communication between care team members regarding the care plan or to lead to improvements in patient safety, although we recognize that there may have been benefits (or unintended harms) not measured in this study but are nonetheless important for clinical care and workplace culture. This is an important lesson, as many hospitals move toward regionalized care in an effort to improve patient safety outcomes. However, strengthening the infrastructure by colocalizing care team members to maximize opportunity for communication is likely a necessary first step toward facilitating implementation of additional initiatives that may lead to more robust patient safety improvements, such as structured interdisciplinary bedside rounds (eg, facilitating and training all team members to fulfill specific roles), teamwork training, and certification of key interdisciplinary teamwork skills. Additionally, close examination of identified prevalent and preventable AEs can help to determine which additional initiatives are most likely to have greatest impact in improving patient safety.

Disclosures: This research was supported by funds provided by Brigham and Women's Hospital (BWH) and by funds provided by the Department of Medicine at BWH. All authors had full access to all of the data in the study and were integrally involved in the design, implementation, data collection, and analyses. The first author, Dr. Stephanie Mueller, takes responsibility for the integrity for the data and the accuracy of the data analysis. Dr. Schnipper reports grants from Sanofi Aventis, outside the submitted work.

References
  1. Joint Commission on Accreditation of Healthcare Organizations. Understanding and Preventing Sentinel Events in Your Health Care Organization. Oak Brook, IL: Joint Commission; 2008.
  2. Lingard L, Espin S, Whyte S, et al. Communication failures in the operating room: an observational classification of recurrent types and effects. Qual Saf Health Care. 2004;13(5):330334.
  3. Makary MA, Sexton JB, Freischlag JA, et al. Operating room teamwork among physicians and nurses: teamwork in the eye of the beholder. J Am Coll Surg. 2006;202(5):746752.
  4. Thomas EJ, Sexton JB, Helmreich RL. Discrepant attitudes about teamwork among critical care nurses and physicians. Crit Care Med. 2003;31(3):956959.
  5. Arora V, Johnson J, Lovinger D, Humphrey HJ, Meltzer DO. Communication failures in patient sign‐out and suggestions for improvement: a critical incident analysis. Qual Saf Health Care. 2005;14(6):401407.
  6. Starmer AJ, Spector ND, Srivastava R, et al. Changes in medical errors after implementation of a handoff program. N Engl J Med. 2014;371(19):18031812.
  7. Gandara E, Moniz T, Ungar J, et al. Communication and information deficits in patients discharged to rehabilitation facilities: an evaluation of five acute care hospitals. J Hosp Med. 2009;4(8):E28E33.
  8. Kripalani S, LeFevre F, Phillips CO, Williams MV, Basaviah P, Baker DW. Deficits in communication and information transfer between hospital‐based and primary care physicians: implications for patient safety and continuity of care. JAMA. 2007;297(8):831841.
  9. O'Leary KJ, Thompson JA, Landler MP, et al. Patterns of nurse‐physician communication and agreement on the plan of care. Qual Saf Health Care. 2010;19(3):195199.
  10. Evanoff B, Potter P, Wolf L, Grayson D, Dunagan C, Boxerman S. Can we talk? Priorities for patient care differed among health care providers. In: Henriksen K, Battles JB, Marks ES, Lewin DI, eds. Advances in Patient Safety: From Research to Implementation. Vol 1. Rockville, MD: Agency for Healthcare Research and Quality; 2005.
  11. O'Leary KJ, Buck R, Fligiel HM, et al. Structured interdisciplinary rounds in a medical teaching unit: improving patient safety. Arch Intern Med. 2011;171(7):678684.
  12. O'Leary KJ, Haviley C, Slade ME, Shah HM, Lee J, Williams MV. Improving teamwork: impact of structured interdisciplinary rounds on a hospitalist unit. J Hosp Med. 2011;6(2):8893.
  13. O'Leary KJ, Wayne DB, Landler MP, et al. Impact of localizing physicians to hospital units on nurse‐physician communication and agreement on the plan of care. J Gen Intern Med. 2009;24(11):12231227.
  14. Stein J, Payne C, Methvin A, et al. Reorganizing a hospital ward as an accountable care unit. J Hosp Med. 2015;10(1):3640.
  15. Boxer R, Vitale M, Gershanik E, et al. 5th time's a charm: creation of unit‐based care teams in a high occupancy hospital [abstract]. J Hosp Med. 2015;10 (suppl. 2). Available at: http://www.shmabstracts.com/abstract/5th‐times‐a‐charm‐creation‐of‐unit‐based‐care‐teams‐in‐a‐high‐occupancy‐hospital. Accessed July 28, 2015.
  16. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998;36(1):827.
  17. O'Leary KJ, Ritter CD, Wheeler H, Szekendi MK, Brinton TS, Williams MV. Teamwork on inpatient medical units: assessing attitudes and barriers. Qual Saf Health Care. 2010;19(2):117121.
  18. Aronson MD, Neeman N, Carbo A, et al. A model for quality improvement programs in academic departments of medicine. Am J Med. 2008;121(10):922929.
  19. Narasimhan M, Eisen LA, Mahoney CD, Acerra FL, Rosen MJ. Improving nurse‐physician communication and satisfaction in the intensive care unit with a daily goals worksheet. Am J Crit Care. 2006;15(2):217222.
  20. Pronovost P, Berenholtz S, Dorman T, Lipsett PA, Simmonds T, Haraden C. Improving communication in the ICU using daily goals. J Crit Care. 2003;18(2):7175.
  21. Haller G, Garnerin P, Morales MA, et al. Effect of crew resource management training in a multidisciplinary obstetrical setting. Int J Qual Health Care. 2008;20(4):254263.
  22. Morey JC, Simon R, Jay GD, et al. Error reduction and performance improvement in the emergency department through formal teamwork training: evaluation results of the MedTeams project. Health Serv Res. 2002;37(6):15531581.
  23. Nielsen PE, Goldman MB, Mann S, et al. Effects of teamwork training on adverse outcomes and process of care in labor and delivery: a randomized controlled trial. Obstet Gynecol. 2007;109(1):4855.
  24. Pannick S, Davis R, Ashrafian H, et al. Effects of interdisciplinary team care interventions on general medical wards: a systematic review. JAMA Intern Med. 2015;175(8):12881298.
  25. Bates DW, Miller EB, Cullen DJ, et al. Patient risk factors for adverse drug events in hospitalized patients. ADE Prevention Study Group. Arch Intern Med. 1999;159(21):25532560.
  26. Kripalani S, Roumie CL, Dalal AK, et al. Effect of a pharmacist intervention on clinically important medication errors after hospital discharge: a randomized trial. Ann Intern Med. 2012;157(1):110.
  27. Donihi AC, DiNardo MM, DeVita MA, Korytkowski MT. Use of a standardized protocol to decrease medication errors and adverse events related to sliding scale insulin. Qual Saf Health Care. 2006;15(2):8991.
  28. Kaboli PJ, Hoth AB, McClimon BJ, Schnipper JL. Clinical pharmacists and inpatient medical care: a systematic review. Arch Intern Med. 2006;166(9):955964.
  29. Maynard G, Lee J, Phillips G, Fink E, Renvall M. Improved inpatient use of basal insulin, reduced hypoglycemia, and improved glycemic control: effect of structured subcutaneous insulin orders and an insulin management algorithm. J Hosp Med. 2009;4(1):315.
  30. Schnipper JL, Liang CL, Ndumele CD, Pendergrass ML. Effects of a computerized order set on the inpatient management of hyperglycemia: a cluster‐randomized controlled trial. Endocr Pract. 2010;16(2):209218.
  31. Schnipper JL, Ndumele CD, Liang CL, Pendergrass ML. Effects of a subcutaneous insulin protocol, clinical education, and computerized order set on the quality of inpatient management of hyperglycemia: results of a clinical trial. J Hosp Med. 2009;4(1):1627.
References
  1. Joint Commission on Accreditation of Healthcare Organizations. Understanding and Preventing Sentinel Events in Your Health Care Organization. Oak Brook, IL: Joint Commission; 2008.
  2. Lingard L, Espin S, Whyte S, et al. Communication failures in the operating room: an observational classification of recurrent types and effects. Qual Saf Health Care. 2004;13(5):330334.
  3. Makary MA, Sexton JB, Freischlag JA, et al. Operating room teamwork among physicians and nurses: teamwork in the eye of the beholder. J Am Coll Surg. 2006;202(5):746752.
  4. Thomas EJ, Sexton JB, Helmreich RL. Discrepant attitudes about teamwork among critical care nurses and physicians. Crit Care Med. 2003;31(3):956959.
  5. Arora V, Johnson J, Lovinger D, Humphrey HJ, Meltzer DO. Communication failures in patient sign‐out and suggestions for improvement: a critical incident analysis. Qual Saf Health Care. 2005;14(6):401407.
  6. Starmer AJ, Spector ND, Srivastava R, et al. Changes in medical errors after implementation of a handoff program. N Engl J Med. 2014;371(19):18031812.
  7. Gandara E, Moniz T, Ungar J, et al. Communication and information deficits in patients discharged to rehabilitation facilities: an evaluation of five acute care hospitals. J Hosp Med. 2009;4(8):E28E33.
  8. Kripalani S, LeFevre F, Phillips CO, Williams MV, Basaviah P, Baker DW. Deficits in communication and information transfer between hospital‐based and primary care physicians: implications for patient safety and continuity of care. JAMA. 2007;297(8):831841.
  9. O'Leary KJ, Thompson JA, Landler MP, et al. Patterns of nurse‐physician communication and agreement on the plan of care. Qual Saf Health Care. 2010;19(3):195199.
  10. Evanoff B, Potter P, Wolf L, Grayson D, Dunagan C, Boxerman S. Can we talk? Priorities for patient care differed among health care providers. In: Henriksen K, Battles JB, Marks ES, Lewin DI, eds. Advances in Patient Safety: From Research to Implementation. Vol 1. Rockville, MD: Agency for Healthcare Research and Quality; 2005.
  11. O'Leary KJ, Buck R, Fligiel HM, et al. Structured interdisciplinary rounds in a medical teaching unit: improving patient safety. Arch Intern Med. 2011;171(7):678684.
  12. O'Leary KJ, Haviley C, Slade ME, Shah HM, Lee J, Williams MV. Improving teamwork: impact of structured interdisciplinary rounds on a hospitalist unit. J Hosp Med. 2011;6(2):8893.
  13. O'Leary KJ, Wayne DB, Landler MP, et al. Impact of localizing physicians to hospital units on nurse‐physician communication and agreement on the plan of care. J Gen Intern Med. 2009;24(11):12231227.
  14. Stein J, Payne C, Methvin A, et al. Reorganizing a hospital ward as an accountable care unit. J Hosp Med. 2015;10(1):3640.
  15. Boxer R, Vitale M, Gershanik E, et al. 5th time's a charm: creation of unit‐based care teams in a high occupancy hospital [abstract]. J Hosp Med. 2015;10 (suppl. 2). Available at: http://www.shmabstracts.com/abstract/5th‐times‐a‐charm‐creation‐of‐unit‐based‐care‐teams‐in‐a‐high‐occupancy‐hospital. Accessed July 28, 2015.
  16. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998;36(1):827.
  17. O'Leary KJ, Ritter CD, Wheeler H, Szekendi MK, Brinton TS, Williams MV. Teamwork on inpatient medical units: assessing attitudes and barriers. Qual Saf Health Care. 2010;19(2):117121.
  18. Aronson MD, Neeman N, Carbo A, et al. A model for quality improvement programs in academic departments of medicine. Am J Med. 2008;121(10):922929.
  19. Narasimhan M, Eisen LA, Mahoney CD, Acerra FL, Rosen MJ. Improving nurse‐physician communication and satisfaction in the intensive care unit with a daily goals worksheet. Am J Crit Care. 2006;15(2):217222.
  20. Pronovost P, Berenholtz S, Dorman T, Lipsett PA, Simmonds T, Haraden C. Improving communication in the ICU using daily goals. J Crit Care. 2003;18(2):7175.
  21. Haller G, Garnerin P, Morales MA, et al. Effect of crew resource management training in a multidisciplinary obstetrical setting. Int J Qual Health Care. 2008;20(4):254263.
  22. Morey JC, Simon R, Jay GD, et al. Error reduction and performance improvement in the emergency department through formal teamwork training: evaluation results of the MedTeams project. Health Serv Res. 2002;37(6):15531581.
  23. Nielsen PE, Goldman MB, Mann S, et al. Effects of teamwork training on adverse outcomes and process of care in labor and delivery: a randomized controlled trial. Obstet Gynecol. 2007;109(1):4855.
  24. Pannick S, Davis R, Ashrafian H, et al. Effects of interdisciplinary team care interventions on general medical wards: a systematic review. JAMA Intern Med. 2015;175(8):12881298.
  25. Bates DW, Miller EB, Cullen DJ, et al. Patient risk factors for adverse drug events in hospitalized patients. ADE Prevention Study Group. Arch Intern Med. 1999;159(21):25532560.
  26. Kripalani S, Roumie CL, Dalal AK, et al. Effect of a pharmacist intervention on clinically important medication errors after hospital discharge: a randomized trial. Ann Intern Med. 2012;157(1):110.
  27. Donihi AC, DiNardo MM, DeVita MA, Korytkowski MT. Use of a standardized protocol to decrease medication errors and adverse events related to sliding scale insulin. Qual Saf Health Care. 2006;15(2):8991.
  28. Kaboli PJ, Hoth AB, McClimon BJ, Schnipper JL. Clinical pharmacists and inpatient medical care: a systematic review. Arch Intern Med. 2006;166(9):955964.
  29. Maynard G, Lee J, Phillips G, Fink E, Renvall M. Improved inpatient use of basal insulin, reduced hypoglycemia, and improved glycemic control: effect of structured subcutaneous insulin orders and an insulin management algorithm. J Hosp Med. 2009;4(1):315.
  30. Schnipper JL, Liang CL, Ndumele CD, Pendergrass ML. Effects of a computerized order set on the inpatient management of hyperglycemia: a cluster‐randomized controlled trial. Endocr Pract. 2010;16(2):209218.
  31. Schnipper JL, Ndumele CD, Liang CL, Pendergrass ML. Effects of a subcutaneous insulin protocol, clinical education, and computerized order set on the quality of inpatient management of hyperglycemia: results of a clinical trial. J Hosp Med. 2009;4(1):1627.
Issue
Journal of Hospital Medicine - 11(9)
Issue
Journal of Hospital Medicine - 11(9)
Page Number
620-627
Page Number
620-627
Article Type
Display Headline
Impact of regionalized care on concordance of plan and preventable adverse events on general medicine services
Display Headline
Impact of regionalized care on concordance of plan and preventable adverse events on general medicine services
Sections
Article Source
© 2016 Society of Hospital Medicine
Disallow All Ads
Correspondence Location
Address for correspondence and reprint requests: Stephanie Mueller, MD, Division of General Internal Medicine, Brigham and Women's Hospital, 1620 Tremont Street, Roxbury, MA 02120; Telephone: 617‐278‐0628; Fax: 617‐732‐7072; E‐mail: smueller1@partners.org
Content Gating
Gated (full article locked unless allowed per User)
Gating Strategy
First Peek Free
Article PDF Media
Media Files