Original Research

Patient complexity is associated with greater hospital readmission rates,^1,2 poorer quality of care,³ and lower patient satisfaction.⁴ Improving outcomes for complex patients is a global priority,⁵ and local initiatives such as Ontario’s Health Links are being developed, yet evidence to inform care is lacking.^6-8

The prevalence of patients living with multiple comorbidities is increasing as advances in medicine enable people to live and manage chronic diseases.^9-11 However, these medical gains have resulted in an increased burden on both patients and healthcare systems. Socioeconomic status and co-occurring psychosocial challenges further complicate health and healthcare in marginalized populations.^12,13

Human immunodeficiency virus/acquired immunodeficiency syndrome (HIV/AIDS) is one example of a disease that medicine has transformed. Individuals living with HIV today, on antiretroviral medications, may be able to manage their chronic illness for decades.^14,15 However, in addition to social determinants of health that influence ongoing adherence and engagement in care, these medications do not completely eradicate the impact of HIV and, as a result, HIV-positive individuals are at a greater risk of developing additional comorbidities.¹⁵ People living with HIV may, therefore, represent an important patient population in which healthcare interventions and system improvements for complex patients should be explored.

Improving health systems and better supporting complex patients requires a broader understanding of the patient experience and the challenges encountered, especially during high-risk periods such as hospital discharge. Qualitative research approaches are designed to help us understand social phenomena in their “natural” settings,¹⁶ and thus suited to achieve this goal, providing critical insight to inform healthcare systems and policies.^17,18 This study sought to answer the question, “What are the obstacles and challenges faced by complex patients during hospital discharge and post-discharge transition?” We approached patient complexity holistically, using a unified Complexity Framework⁶ that connects 5 health dimensions—social capital, mental health, demographics, health and social experiences, and physical health—identified as important to understanding complex patients and their interaction with healthcare. A longitudinal case study approach was used, with multiple sources of data, to understand the clinical context and discharge plans in relation to the lived experience of patients over time, exploring potential misalignment and areas for improvement.

This community-based research study was conducted at Casey House, a 13-bed subacute care hospital in Toronto, Canada that provides in-patient and community programs to a complex patient group. All patients are HIV-positive. Inpatient hospital care is provided by an interdisciplinary team, including physicians, social workers, nurses, and healthcare aides. A harm reduction approach is taken to substance use. Twelve beds are for general admission. Patients may be transferred from acute-care hospitals or referred by community-based providers. One bed is reserved for scheduled 2-week respite stays.

The primary research team for this community-based project consisted of clinicians and community and academic researchers. The study was conducted in collaboration with housing, healthcare, and HIV service providers and was advised by 2 individuals with lived experience of discharge from Casey House. Community members with lived experience attended team meetings, provided feedback on all stages of the project (ie, interview guides, recruitment, analysis and dissemination), and helped facilitate community engagement sessions with other patients at the start and the end of the project.

Standard practice for discharge planning involves clinicians determining a tentative discharge date and identifying strategies to support the patient. Planning is informed by knowledge gathered by the interdisciplinary team throughout the admission, including social determinants of health (ie, housing, social support, food security). Patients are encouraged to invite an individual from their social support network to attend a discharge meeting, where the care team reviews goals for admission, course of treatment, referrals, and important follow-up dates.

We used a multi-case study approach to explore the discharge process and post-discharge period. A case was defined as the discharge and transition of a patient from hospital to community. Data were collected through serial interviews with patients (n = 4), medical chart abstraction, and review of discharge summaries. Serial interviews, although not frequently used in clinical research, have been proposed as a strong approach for exploring complex processes and to build trust between researcher and participant,¹⁹ both of which were relevant in this study. Patient interviews were conducted by the Master’s trained research coordinator (SM) using tailored semi-structured interview guides for 4 time points: before the discharge meeting (I1); after the discharge meeting but before discharge (I2); within a week of discharge (I3); and approximately 30 days after discharge (I4). Interviews were audio recorded and transcribed verbatim.

Cases were eligible if the patient had a general admission and a planned discharge to the community, and was able to communicate in English and direct his/her own care. Patient-initiated discharges and discharges to another healthcare facility were excluded. Casey House clinical staff approached consecutive potentially eligible patients for their willingness to speak with the researcher coordinator. The research coordinator met with patients to assess eligibility and obtain informed consent to participate. All participants provided informed written consent. The study was approved by the University of Toronto HIV Research Ethics Board.

Interview data, managed with MAXQDA software (VERBI GmbH, Berlin, Germany), were analyzed using a framework analysis approach.^20,21 At least 3 authors read each transcript in its entirety. Priority questions/topics identified a priori by stakeholders as important to inform change in care and practices were used as the first draft of the coding framework. The framework was modified through team discussion in the analysis phase to integrate emerging themes. Participant demographic and clinical data were extracted using a structured data collection form.

Preliminary data analysis was completed for the separate data sources including inter- and intra-case comparisons: exploring how experiences and perceptions changed over time and themes that emerged across cases at the same time point. Data sources were combined to strengthen the understanding of the cases and identify relationships and discrepancies across sources.²² Audit trails, reflexive journaling, group coding and analysis meetings and member-checking, were used to enhance analytical rigor.

The results focus on the patient experience of the “discharge plan” and are presented in terms of 3 pre-identified categories: 1) social support; 2) discharge process and transition experience; and 3) post-discharge follow-up and referrals; and 1 emergent theme, patient priorities.

Participants experienced complex medical and psychosocial challenges (Table 1, participant characteristics). All participants were living with HIV plus a mean of 5 additional comorbidities, the most common being hepatitis C (n = 3), chronic obstructive pulmonary disease (n = 2), herpes (n = 2) and opportunistic infections (n = 2). Eight of 9 participants had a history of an Axis 1 diagnosis, most commonly mood disorder (n = 4). Substance use was identified in all participants. An overview of each case is presented in Table 2.

Social Support

For the purposes of this paper, we define “social support” as the emotional or instrumental assistance an individual perceives and experiences from people in his/her self-identified network (ie, family, friends). Participants’ discharge-related experience of social support did not align, in most cases, with the information from their medical charts or their expectations. At admission, 8 of 9 participants identified at least 1 person in their social support network, yet only 1 participant had someone attend the discharge meeting. One participant said she had expected “my daughter, my mother, my brother, somebody. At least somebody. But they never show up.” (P5, I2).

The complexity of her relationship with her family and her unmet needs for support continued after discharge:

I try and be as independent as possible. I don’t have to call them for nothing. Because, even the other day, I called my mom and I asked her, I said, “Mom, I’m going to give you $400 [to pay back a personal loan] and I’m going to give you an extra $100, you could buy me some food.” And she goes “Okay.” But, I didn’t give it to her yet. I don’t know, she seems money hungry right now, so I’m like no, I’ll wait. (P5, I4)

In the hospital, participants frequently spoke about discharge and transition planning that was inclusive of their social support networks. However, a sense of isolation and loneliness was common post-discharge. Often, friends and family members did not provide the support that participants anticipated, but instead were sources of anxiety and stress. One participant conveyed his experience with a friend he listed as a social support:

I gave him some money to get me some groceries, to make sure I had some food in the house when I got home. He didn’t do that. All of a sudden he was called away to [another city]. He told me his father had a heart attack. He told [others] his father had a slip. I still have yet to receive my money. (P7, I4)

Discharge Process and Transition Experience

While some participants were excited about the thought of freedom of being home, others were anxious about the burdens of returning to life outside of the hospital.

I kind of feel like, yeah, I want to go home, but then I think to myself what am I going to do when I get home. Am I just going to go back to what I’ve been doing? Am I going to really change? Am I going to forget to take my pill one day because I’m home and stuff like that. (P4, I1)

The discharge process was often perceived by participants to be rushed. Some participants found the discharge meetings helpful, while others did not feel the process empowered them to engage in a meaningful conversation with hospital staff.

There was no one there with me to even help me with my brain, to think. But it’s afterwards I’m like why didn’t I say that, like that’s what I meant to say. The brain just doesn’t function that way. (P8, I2).

This participant struggled with the transition. One week after discharge when she was asked how her health was she replied:

Terrible. I’ve got no energy. I haven’t eaten for 3 days. I haven’t drank for 3 days. I’ve got diarrhea galore […] Just no appetite whatsoever. I can’t even make it up the stairs without losing my breath. If I make it up the stairs, I have to sit for 15 or 20 minutes… (P8, I3)

The weight of maintaining activities of daily living was prominent in all post-discharge interviews, in many cases accentuated by declining health. The transition to home was more challenging than participants expected; the experience was strongly influenced by the stability of their health, their environment, and the complexity of their lives.

Follow-up and Referrals

Discharge summaries included a mean of 7 referrals. All participants were referred to a case coordinator, nurse, and family physician. Other referrals included pharmacist (n = 8); personal support worker (n = 6); housing (n = 5); and food-support programs (n = 5).

Several factors led to challenges accessing and receiving services. Participants identified: difficulty with requisite paperwork; mobility and financial constraints; personal and logistical challenges with home-care providers; and competing priorities, such as caring for family. These experiences were frequently accompanied by frustration and anxiety.

Because, if I’m in [city where girlfriend lives], I will not get the support that I get when I’m home. Like my nurse comes. [She] was supposed to come and see me twice and I missed that. I missed like 4 [appointments]. You understand? Certain things I’ve been missing. (P6, I4)

When one participant was asked if she had followed up with the food support program she had been referred to, she responded:

Oh, baby, no. I’ve been so confused. I’ve had ODSP [referring to Ontario Disability Support Program, a government disability program] on my case. I’ve got all the files all mixed up. My worker’s a real bitch. She hates me, big time. I was supposed to go bring in papers today, but I couldn’t get out of bed. I don’t know how much trouble I’m going to be in with ODSP now. (P8, I3)

Despite comprehensive discharge plans and referrals, all participants experienced delays and difficulties in accessing and receiving services. In most cases, there was no single contributing factor to these challenges; the unique experiences were a result of the complex interplay of multiple factors for each individual.

Patient Priorities

In the hospital, participants primarily identified goals of improving physical health and medication adherence. However, these goals often shifted to meeting basic living necessities and supporting others upon discharge. Barriers to adequate food and mobility were prominent themes.

One participant spoke about the challenges of supporting her son while struggling with her own health after discharge:

Well, I’ve been dying, I can’t even walk, and yet I’m the one that still has to go to WalMart, to grab milk and bread for my kid. It’s not like I need any of that stuff, because I don’t even eat. (P8, I3)

Participants were admitted on a mean of 6 medications and discharged with a mean of 14 (Table 1). In the hospital, medications are dispensed directly to patients; however, maintaining optimal adherence at home was complex. When 1 participant was asked about her medications after being home for a week, she said:

My meds, you know I have the cream that I’m supposed to put … and I can’t find it. I lost it yesterday. I used it yesterday morning and all day yesterday I’m looking, like, did it fall behind there? But, obviously, I can’t look over there [because of mobility challenges] … I don’t think I can get it covered [by insurance to replace it]. (P5, I3)

Participants found it difficult to follow a specific dosing schedule, ensure food intake corresponded to medication guidelines, and navigate the impact of substance use. Substance use for some was associated with nonadherence. A participant, explaining his quickly declining health, spoke about the impact of using crack cocaine:

Yeah, when I use I don’t think about medicating, taking my pills or anything like that. That’s not even on your mind. It doesn’t come across your mind. […] I guess, that’s part of the addictive personality. It wants to grab hold of you and say “no, focus on me, focus on me.” (P7, I4)

Others used marijuana as an appetite stimulant and a critical piece of their medication adherence routine.

This study followed complex patients through hospital discharge and transition back into the community. In the hospital, participants focused on medical goals, but following discharge basic living needs became the priority. Despite a comprehensive plan to provide support upon discharge, participants found executing and following up with referrals, services, and medication adherence was often overwhelming and not achieved in the month post-hospitalization.

Our study provides depth and context to support and understand the findings of reviews evaluating interventions to improve transitions in care.^23,24 A systematic review of interventions to decrease 30-day readmission rates concluded that comprehensive support interventions (with many components) contributed to the greatest reduction in risk of readmission.¹⁶ Components that showed the greatest impact were those that were designed to improve patients’ capacity for self-care (including their ability to access and follow through with post-discharge care plans) and those that involved more individuals in the delivery of care.²³

Our results also support and expand on other qualitative findings of complex patients. Kangovi et al.²⁵ interviewed patients with low socioeconomic status at a single time point post-discharge to identify common experiences. They summarized their findings in 6 themes: powerlessness during hospitalization; incongruence of patient and clinical team goals; competing issues influencing prominence of health behaviors; socioeconomic constraints on patients’ ability to perform recommended behaviors; sense of abandonment after discharge; and loss of self-efficacy resulting from the “failure” to follow the discharge plan. Our findings tell a very similar story but provide the additional context and understanding of the lived experience over time. We found that the transition experience was most challenging when the home environment was unstable, resulting in a shift in priorities from those set during hospitalization.

While increased support may improve outcomes, there is a need to improve awareness, integration, and support for building capacity within complex patients.²⁶ Capacity is defined here as the sum of resources and abilities that a patient can draw on, and includes physical and mental as well as social, financial, personal, and environmental capabilities and resources.²⁷ This includes understanding the potential negative impact of developing a clinical plan which, in order to operationalize, requires resources in excess of the patient’s capacity at that time.²⁷ Minimally disruptive medicine, a promising theoretical approach for improving the care of complex clients, embodies the awareness of capacity in achieving patient-centered care while “imposing the smallest possible treatment burden on patients’ lives.”²⁸

This study, although not without its limitations, provides an in-depth exploration of the experiences of a small number of patients living with HIV, recruited from a single facility in Toronto, Canada after relatively long hospital stays. There are specific context issues related to HIV, such as stigma and severe consequences for suboptimal medication adherence. Furthermore, this study took place where many urban health resources exist; complex patients in rural settings or in environments less tailored to the needs associated with complex medical, psychiatric, and social conditions may experience greater barriers in the transition process. Although this study captured data from medical charts and documents relevant to the cases, further exploration of the clinician decision-making process in creating the discharge plans and additional sources of data on health outcomes post-discharge would be beneficial.

Despite its limitations, this study provides detail and depth to understand some of the most complex patients who suffer from significant challenges in the health system and who are amongst the highest-cost healthcare users. The case study approach, with serial interviews, is an important strength of this study, enabling meaningful insight into hospital discharge processes and challenges experienced by complex patients that can inform individual-level care practice and the development of new programs and interventions.

This study builds on recent research with complex patients in calling for a new approach to clinical care.^6,29,30 In order to support complex patients through discharge, clinical goals and referrals must be made in light of a patient’s capacity in the community. Structural changes may be made to improve coordination and access to services, decreasing the burden and improving the healthcare experience. Albreht et al.³¹ highlight a number of promising programs across Europe (such as the Clinic for Multimorbidity and Polypharmacy in Denmark) designed to improve the health and healthcare for individuals living with multiple chronic conditions. Small-scale changes are also important such as increasing conversations about the capacity and limitations of individuals listed as social supports, and making appropriate and realistic referrals based on an understanding of a patient’s capacity and motivation for follow-up. Shippee et al.³² identify a list of approaches in line with minimally disruptive medicine that can be integrated into existing systems as part of a developing “toolkit” (eg, elicitation of transcendent patient goals, and integration of patient-reported outcome tracking of challenges and burdens associated with health and daily living). The findings of this study suggest that the elements of the toolkit may provide a foundation for future interventions and research to improve hospital care and discharge outcomes for complex patients.

Disclosures

This project was funded by a Canadian Institutes of Health Research (CIHR) HIV/AIDS Community-based Research Catalyst Grant (#126669). Dr. Brennan’s research is supported by an Ontario HIV Treatment Network (OHTN) Applied HIV Research Chair. Dr. Chan Carusone reports grants from Canadian Institutes of Health Research during the conduct of the study.

Patient complexity is associated with greater hospital readmission rates,^1,2 poorer quality of care,³ and lower patient satisfaction.⁴ Improving outcomes for complex patients is a global priority,⁵ and local initiatives such as Ontario’s Health Links are being developed, yet evidence to inform care is lacking.^6-8

The prevalence of patients living with multiple comorbidities is increasing as advances in medicine enable people to live and manage chronic diseases.^9-11 However, these medical gains have resulted in an increased burden on both patients and healthcare systems. Socioeconomic status and co-occurring psychosocial challenges further complicate health and healthcare in marginalized populations.^12,13

Human immunodeficiency virus/acquired immunodeficiency syndrome (HIV/AIDS) is one example of a disease that medicine has transformed. Individuals living with HIV today, on antiretroviral medications, may be able to manage their chronic illness for decades.^14,15 However, in addition to social determinants of health that influence ongoing adherence and engagement in care, these medications do not completely eradicate the impact of HIV and, as a result, HIV-positive individuals are at a greater risk of developing additional comorbidities.¹⁵ People living with HIV may, therefore, represent an important patient population in which healthcare interventions and system improvements for complex patients should be explored.

Improving health systems and better supporting complex patients requires a broader understanding of the patient experience and the challenges encountered, especially during high-risk periods such as hospital discharge. Qualitative research approaches are designed to help us understand social phenomena in their “natural” settings,¹⁶ and thus suited to achieve this goal, providing critical insight to inform healthcare systems and policies.^17,18 This study sought to answer the question, “What are the obstacles and challenges faced by complex patients during hospital discharge and post-discharge transition?” We approached patient complexity holistically, using a unified Complexity Framework⁶ that connects 5 health dimensions—social capital, mental health, demographics, health and social experiences, and physical health—identified as important to understanding complex patients and their interaction with healthcare. A longitudinal case study approach was used, with multiple sources of data, to understand the clinical context and discharge plans in relation to the lived experience of patients over time, exploring potential misalignment and areas for improvement.

This community-based research study was conducted at Casey House, a 13-bed subacute care hospital in Toronto, Canada that provides in-patient and community programs to a complex patient group. All patients are HIV-positive. Inpatient hospital care is provided by an interdisciplinary team, including physicians, social workers, nurses, and healthcare aides. A harm reduction approach is taken to substance use. Twelve beds are for general admission. Patients may be transferred from acute-care hospitals or referred by community-based providers. One bed is reserved for scheduled 2-week respite stays.

The primary research team for this community-based project consisted of clinicians and community and academic researchers. The study was conducted in collaboration with housing, healthcare, and HIV service providers and was advised by 2 individuals with lived experience of discharge from Casey House. Community members with lived experience attended team meetings, provided feedback on all stages of the project (ie, interview guides, recruitment, analysis and dissemination), and helped facilitate community engagement sessions with other patients at the start and the end of the project.

Standard practice for discharge planning involves clinicians determining a tentative discharge date and identifying strategies to support the patient. Planning is informed by knowledge gathered by the interdisciplinary team throughout the admission, including social determinants of health (ie, housing, social support, food security). Patients are encouraged to invite an individual from their social support network to attend a discharge meeting, where the care team reviews goals for admission, course of treatment, referrals, and important follow-up dates.

We used a multi-case study approach to explore the discharge process and post-discharge period. A case was defined as the discharge and transition of a patient from hospital to community. Data were collected through serial interviews with patients (n = 4), medical chart abstraction, and review of discharge summaries. Serial interviews, although not frequently used in clinical research, have been proposed as a strong approach for exploring complex processes and to build trust between researcher and participant,¹⁹ both of which were relevant in this study. Patient interviews were conducted by the Master’s trained research coordinator (SM) using tailored semi-structured interview guides for 4 time points: before the discharge meeting (I1); after the discharge meeting but before discharge (I2); within a week of discharge (I3); and approximately 30 days after discharge (I4). Interviews were audio recorded and transcribed verbatim.

Cases were eligible if the patient had a general admission and a planned discharge to the community, and was able to communicate in English and direct his/her own care. Patient-initiated discharges and discharges to another healthcare facility were excluded. Casey House clinical staff approached consecutive potentially eligible patients for their willingness to speak with the researcher coordinator. The research coordinator met with patients to assess eligibility and obtain informed consent to participate. All participants provided informed written consent. The study was approved by the University of Toronto HIV Research Ethics Board.

Interview data, managed with MAXQDA software (VERBI GmbH, Berlin, Germany), were analyzed using a framework analysis approach.^20,21 At least 3 authors read each transcript in its entirety. Priority questions/topics identified a priori by stakeholders as important to inform change in care and practices were used as the first draft of the coding framework. The framework was modified through team discussion in the analysis phase to integrate emerging themes. Participant demographic and clinical data were extracted using a structured data collection form.

Preliminary data analysis was completed for the separate data sources including inter- and intra-case comparisons: exploring how experiences and perceptions changed over time and themes that emerged across cases at the same time point. Data sources were combined to strengthen the understanding of the cases and identify relationships and discrepancies across sources.²² Audit trails, reflexive journaling, group coding and analysis meetings and member-checking, were used to enhance analytical rigor.

The results focus on the patient experience of the “discharge plan” and are presented in terms of 3 pre-identified categories: 1) social support; 2) discharge process and transition experience; and 3) post-discharge follow-up and referrals; and 1 emergent theme, patient priorities.

Participants experienced complex medical and psychosocial challenges (Table 1, participant characteristics). All participants were living with HIV plus a mean of 5 additional comorbidities, the most common being hepatitis C (n = 3), chronic obstructive pulmonary disease (n = 2), herpes (n = 2) and opportunistic infections (n = 2). Eight of 9 participants had a history of an Axis 1 diagnosis, most commonly mood disorder (n = 4). Substance use was identified in all participants. An overview of each case is presented in Table 2.

Social Support

For the purposes of this paper, we define “social support” as the emotional or instrumental assistance an individual perceives and experiences from people in his/her self-identified network (ie, family, friends). Participants’ discharge-related experience of social support did not align, in most cases, with the information from their medical charts or their expectations. At admission, 8 of 9 participants identified at least 1 person in their social support network, yet only 1 participant had someone attend the discharge meeting. One participant said she had expected “my daughter, my mother, my brother, somebody. At least somebody. But they never show up.” (P5, I2).

The complexity of her relationship with her family and her unmet needs for support continued after discharge:

I try and be as independent as possible. I don’t have to call them for nothing. Because, even the other day, I called my mom and I asked her, I said, “Mom, I’m going to give you $400 [to pay back a personal loan] and I’m going to give you an extra $100, you could buy me some food.” And she goes “Okay.” But, I didn’t give it to her yet. I don’t know, she seems money hungry right now, so I’m like no, I’ll wait. (P5, I4)

In the hospital, participants frequently spoke about discharge and transition planning that was inclusive of their social support networks. However, a sense of isolation and loneliness was common post-discharge. Often, friends and family members did not provide the support that participants anticipated, but instead were sources of anxiety and stress. One participant conveyed his experience with a friend he listed as a social support:

I gave him some money to get me some groceries, to make sure I had some food in the house when I got home. He didn’t do that. All of a sudden he was called away to [another city]. He told me his father had a heart attack. He told [others] his father had a slip. I still have yet to receive my money. (P7, I4)

Discharge Process and Transition Experience

While some participants were excited about the thought of freedom of being home, others were anxious about the burdens of returning to life outside of the hospital.

I kind of feel like, yeah, I want to go home, but then I think to myself what am I going to do when I get home. Am I just going to go back to what I’ve been doing? Am I going to really change? Am I going to forget to take my pill one day because I’m home and stuff like that. (P4, I1)

The discharge process was often perceived by participants to be rushed. Some participants found the discharge meetings helpful, while others did not feel the process empowered them to engage in a meaningful conversation with hospital staff.

There was no one there with me to even help me with my brain, to think. But it’s afterwards I’m like why didn’t I say that, like that’s what I meant to say. The brain just doesn’t function that way. (P8, I2).

This participant struggled with the transition. One week after discharge when she was asked how her health was she replied:

Terrible. I’ve got no energy. I haven’t eaten for 3 days. I haven’t drank for 3 days. I’ve got diarrhea galore […] Just no appetite whatsoever. I can’t even make it up the stairs without losing my breath. If I make it up the stairs, I have to sit for 15 or 20 minutes… (P8, I3)

The weight of maintaining activities of daily living was prominent in all post-discharge interviews, in many cases accentuated by declining health. The transition to home was more challenging than participants expected; the experience was strongly influenced by the stability of their health, their environment, and the complexity of their lives.

Follow-up and Referrals

Discharge summaries included a mean of 7 referrals. All participants were referred to a case coordinator, nurse, and family physician. Other referrals included pharmacist (n = 8); personal support worker (n = 6); housing (n = 5); and food-support programs (n = 5).

Several factors led to challenges accessing and receiving services. Participants identified: difficulty with requisite paperwork; mobility and financial constraints; personal and logistical challenges with home-care providers; and competing priorities, such as caring for family. These experiences were frequently accompanied by frustration and anxiety.

Because, if I’m in [city where girlfriend lives], I will not get the support that I get when I’m home. Like my nurse comes. [She] was supposed to come and see me twice and I missed that. I missed like 4 [appointments]. You understand? Certain things I’ve been missing. (P6, I4)

When one participant was asked if she had followed up with the food support program she had been referred to, she responded:

Oh, baby, no. I’ve been so confused. I’ve had ODSP [referring to Ontario Disability Support Program, a government disability program] on my case. I’ve got all the files all mixed up. My worker’s a real bitch. She hates me, big time. I was supposed to go bring in papers today, but I couldn’t get out of bed. I don’t know how much trouble I’m going to be in with ODSP now. (P8, I3)

Despite comprehensive discharge plans and referrals, all participants experienced delays and difficulties in accessing and receiving services. In most cases, there was no single contributing factor to these challenges; the unique experiences were a result of the complex interplay of multiple factors for each individual.

Patient Priorities

In the hospital, participants primarily identified goals of improving physical health and medication adherence. However, these goals often shifted to meeting basic living necessities and supporting others upon discharge. Barriers to adequate food and mobility were prominent themes.

One participant spoke about the challenges of supporting her son while struggling with her own health after discharge:

Well, I’ve been dying, I can’t even walk, and yet I’m the one that still has to go to WalMart, to grab milk and bread for my kid. It’s not like I need any of that stuff, because I don’t even eat. (P8, I3)

Participants were admitted on a mean of 6 medications and discharged with a mean of 14 (Table 1). In the hospital, medications are dispensed directly to patients; however, maintaining optimal adherence at home was complex. When 1 participant was asked about her medications after being home for a week, she said:

My meds, you know I have the cream that I’m supposed to put … and I can’t find it. I lost it yesterday. I used it yesterday morning and all day yesterday I’m looking, like, did it fall behind there? But, obviously, I can’t look over there [because of mobility challenges] … I don’t think I can get it covered [by insurance to replace it]. (P5, I3)

Participants found it difficult to follow a specific dosing schedule, ensure food intake corresponded to medication guidelines, and navigate the impact of substance use. Substance use for some was associated with nonadherence. A participant, explaining his quickly declining health, spoke about the impact of using crack cocaine:

Yeah, when I use I don’t think about medicating, taking my pills or anything like that. That’s not even on your mind. It doesn’t come across your mind. […] I guess, that’s part of the addictive personality. It wants to grab hold of you and say “no, focus on me, focus on me.” (P7, I4)

Others used marijuana as an appetite stimulant and a critical piece of their medication adherence routine.

This study followed complex patients through hospital discharge and transition back into the community. In the hospital, participants focused on medical goals, but following discharge basic living needs became the priority. Despite a comprehensive plan to provide support upon discharge, participants found executing and following up with referrals, services, and medication adherence was often overwhelming and not achieved in the month post-hospitalization.

Our study provides depth and context to support and understand the findings of reviews evaluating interventions to improve transitions in care.^23,24 A systematic review of interventions to decrease 30-day readmission rates concluded that comprehensive support interventions (with many components) contributed to the greatest reduction in risk of readmission.¹⁶ Components that showed the greatest impact were those that were designed to improve patients’ capacity for self-care (including their ability to access and follow through with post-discharge care plans) and those that involved more individuals in the delivery of care.²³

Our results also support and expand on other qualitative findings of complex patients. Kangovi et al.²⁵ interviewed patients with low socioeconomic status at a single time point post-discharge to identify common experiences. They summarized their findings in 6 themes: powerlessness during hospitalization; incongruence of patient and clinical team goals; competing issues influencing prominence of health behaviors; socioeconomic constraints on patients’ ability to perform recommended behaviors; sense of abandonment after discharge; and loss of self-efficacy resulting from the “failure” to follow the discharge plan. Our findings tell a very similar story but provide the additional context and understanding of the lived experience over time. We found that the transition experience was most challenging when the home environment was unstable, resulting in a shift in priorities from those set during hospitalization.

While increased support may improve outcomes, there is a need to improve awareness, integration, and support for building capacity within complex patients.²⁶ Capacity is defined here as the sum of resources and abilities that a patient can draw on, and includes physical and mental as well as social, financial, personal, and environmental capabilities and resources.²⁷ This includes understanding the potential negative impact of developing a clinical plan which, in order to operationalize, requires resources in excess of the patient’s capacity at that time.²⁷ Minimally disruptive medicine, a promising theoretical approach for improving the care of complex clients, embodies the awareness of capacity in achieving patient-centered care while “imposing the smallest possible treatment burden on patients’ lives.”²⁸

This study, although not without its limitations, provides an in-depth exploration of the experiences of a small number of patients living with HIV, recruited from a single facility in Toronto, Canada after relatively long hospital stays. There are specific context issues related to HIV, such as stigma and severe consequences for suboptimal medication adherence. Furthermore, this study took place where many urban health resources exist; complex patients in rural settings or in environments less tailored to the needs associated with complex medical, psychiatric, and social conditions may experience greater barriers in the transition process. Although this study captured data from medical charts and documents relevant to the cases, further exploration of the clinician decision-making process in creating the discharge plans and additional sources of data on health outcomes post-discharge would be beneficial.

Despite its limitations, this study provides detail and depth to understand some of the most complex patients who suffer from significant challenges in the health system and who are amongst the highest-cost healthcare users. The case study approach, with serial interviews, is an important strength of this study, enabling meaningful insight into hospital discharge processes and challenges experienced by complex patients that can inform individual-level care practice and the development of new programs and interventions.

This study builds on recent research with complex patients in calling for a new approach to clinical care.^6,29,30 In order to support complex patients through discharge, clinical goals and referrals must be made in light of a patient’s capacity in the community. Structural changes may be made to improve coordination and access to services, decreasing the burden and improving the healthcare experience. Albreht et al.³¹ highlight a number of promising programs across Europe (such as the Clinic for Multimorbidity and Polypharmacy in Denmark) designed to improve the health and healthcare for individuals living with multiple chronic conditions. Small-scale changes are also important such as increasing conversations about the capacity and limitations of individuals listed as social supports, and making appropriate and realistic referrals based on an understanding of a patient’s capacity and motivation for follow-up. Shippee et al.³² identify a list of approaches in line with minimally disruptive medicine that can be integrated into existing systems as part of a developing “toolkit” (eg, elicitation of transcendent patient goals, and integration of patient-reported outcome tracking of challenges and burdens associated with health and daily living). The findings of this study suggest that the elements of the toolkit may provide a foundation for future interventions and research to improve hospital care and discharge outcomes for complex patients.

Disclosures

This project was funded by a Canadian Institutes of Health Research (CIHR) HIV/AIDS Community-based Research Catalyst Grant (#126669). Dr. Brennan’s research is supported by an Ontario HIV Treatment Network (OHTN) Applied HIV Research Chair. Dr. Chan Carusone reports grants from Canadian Institutes of Health Research during the conduct of the study.

Patient complexity is associated with greater hospital readmission rates,^1,2 poorer quality of care,³ and lower patient satisfaction.⁴ Improving outcomes for complex patients is a global priority,⁵ and local initiatives such as Ontario’s Health Links are being developed, yet evidence to inform care is lacking.^6-8

The prevalence of patients living with multiple comorbidities is increasing as advances in medicine enable people to live and manage chronic diseases.^9-11 However, these medical gains have resulted in an increased burden on both patients and healthcare systems. Socioeconomic status and co-occurring psychosocial challenges further complicate health and healthcare in marginalized populations.^12,13

Human immunodeficiency virus/acquired immunodeficiency syndrome (HIV/AIDS) is one example of a disease that medicine has transformed. Individuals living with HIV today, on antiretroviral medications, may be able to manage their chronic illness for decades.^14,15 However, in addition to social determinants of health that influence ongoing adherence and engagement in care, these medications do not completely eradicate the impact of HIV and, as a result, HIV-positive individuals are at a greater risk of developing additional comorbidities.¹⁵ People living with HIV may, therefore, represent an important patient population in which healthcare interventions and system improvements for complex patients should be explored.

Improving health systems and better supporting complex patients requires a broader understanding of the patient experience and the challenges encountered, especially during high-risk periods such as hospital discharge. Qualitative research approaches are designed to help us understand social phenomena in their “natural” settings,¹⁶ and thus suited to achieve this goal, providing critical insight to inform healthcare systems and policies.^17,18 This study sought to answer the question, “What are the obstacles and challenges faced by complex patients during hospital discharge and post-discharge transition?” We approached patient complexity holistically, using a unified Complexity Framework⁶ that connects 5 health dimensions—social capital, mental health, demographics, health and social experiences, and physical health—identified as important to understanding complex patients and their interaction with healthcare. A longitudinal case study approach was used, with multiple sources of data, to understand the clinical context and discharge plans in relation to the lived experience of patients over time, exploring potential misalignment and areas for improvement.

This community-based research study was conducted at Casey House, a 13-bed subacute care hospital in Toronto, Canada that provides in-patient and community programs to a complex patient group. All patients are HIV-positive. Inpatient hospital care is provided by an interdisciplinary team, including physicians, social workers, nurses, and healthcare aides. A harm reduction approach is taken to substance use. Twelve beds are for general admission. Patients may be transferred from acute-care hospitals or referred by community-based providers. One bed is reserved for scheduled 2-week respite stays.

The primary research team for this community-based project consisted of clinicians and community and academic researchers. The study was conducted in collaboration with housing, healthcare, and HIV service providers and was advised by 2 individuals with lived experience of discharge from Casey House. Community members with lived experience attended team meetings, provided feedback on all stages of the project (ie, interview guides, recruitment, analysis and dissemination), and helped facilitate community engagement sessions with other patients at the start and the end of the project.

Standard practice for discharge planning involves clinicians determining a tentative discharge date and identifying strategies to support the patient. Planning is informed by knowledge gathered by the interdisciplinary team throughout the admission, including social determinants of health (ie, housing, social support, food security). Patients are encouraged to invite an individual from their social support network to attend a discharge meeting, where the care team reviews goals for admission, course of treatment, referrals, and important follow-up dates.

We used a multi-case study approach to explore the discharge process and post-discharge period. A case was defined as the discharge and transition of a patient from hospital to community. Data were collected through serial interviews with patients (n = 4), medical chart abstraction, and review of discharge summaries. Serial interviews, although not frequently used in clinical research, have been proposed as a strong approach for exploring complex processes and to build trust between researcher and participant,¹⁹ both of which were relevant in this study. Patient interviews were conducted by the Master’s trained research coordinator (SM) using tailored semi-structured interview guides for 4 time points: before the discharge meeting (I1); after the discharge meeting but before discharge (I2); within a week of discharge (I3); and approximately 30 days after discharge (I4). Interviews were audio recorded and transcribed verbatim.

Cases were eligible if the patient had a general admission and a planned discharge to the community, and was able to communicate in English and direct his/her own care. Patient-initiated discharges and discharges to another healthcare facility were excluded. Casey House clinical staff approached consecutive potentially eligible patients for their willingness to speak with the researcher coordinator. The research coordinator met with patients to assess eligibility and obtain informed consent to participate. All participants provided informed written consent. The study was approved by the University of Toronto HIV Research Ethics Board.

Interview data, managed with MAXQDA software (VERBI GmbH, Berlin, Germany), were analyzed using a framework analysis approach.^20,21 At least 3 authors read each transcript in its entirety. Priority questions/topics identified a priori by stakeholders as important to inform change in care and practices were used as the first draft of the coding framework. The framework was modified through team discussion in the analysis phase to integrate emerging themes. Participant demographic and clinical data were extracted using a structured data collection form.

Preliminary data analysis was completed for the separate data sources including inter- and intra-case comparisons: exploring how experiences and perceptions changed over time and themes that emerged across cases at the same time point. Data sources were combined to strengthen the understanding of the cases and identify relationships and discrepancies across sources.²² Audit trails, reflexive journaling, group coding and analysis meetings and member-checking, were used to enhance analytical rigor.

The results focus on the patient experience of the “discharge plan” and are presented in terms of 3 pre-identified categories: 1) social support; 2) discharge process and transition experience; and 3) post-discharge follow-up and referrals; and 1 emergent theme, patient priorities.

Participants experienced complex medical and psychosocial challenges (Table 1, participant characteristics). All participants were living with HIV plus a mean of 5 additional comorbidities, the most common being hepatitis C (n = 3), chronic obstructive pulmonary disease (n = 2), herpes (n = 2) and opportunistic infections (n = 2). Eight of 9 participants had a history of an Axis 1 diagnosis, most commonly mood disorder (n = 4). Substance use was identified in all participants. An overview of each case is presented in Table 2.

Social Support

For the purposes of this paper, we define “social support” as the emotional or instrumental assistance an individual perceives and experiences from people in his/her self-identified network (ie, family, friends). Participants’ discharge-related experience of social support did not align, in most cases, with the information from their medical charts or their expectations. At admission, 8 of 9 participants identified at least 1 person in their social support network, yet only 1 participant had someone attend the discharge meeting. One participant said she had expected “my daughter, my mother, my brother, somebody. At least somebody. But they never show up.” (P5, I2).

The complexity of her relationship with her family and her unmet needs for support continued after discharge:

I try and be as independent as possible. I don’t have to call them for nothing. Because, even the other day, I called my mom and I asked her, I said, “Mom, I’m going to give you $400 [to pay back a personal loan] and I’m going to give you an extra $100, you could buy me some food.” And she goes “Okay.” But, I didn’t give it to her yet. I don’t know, she seems money hungry right now, so I’m like no, I’ll wait. (P5, I4)

In the hospital, participants frequently spoke about discharge and transition planning that was inclusive of their social support networks. However, a sense of isolation and loneliness was common post-discharge. Often, friends and family members did not provide the support that participants anticipated, but instead were sources of anxiety and stress. One participant conveyed his experience with a friend he listed as a social support:

I gave him some money to get me some groceries, to make sure I had some food in the house when I got home. He didn’t do that. All of a sudden he was called away to [another city]. He told me his father had a heart attack. He told [others] his father had a slip. I still have yet to receive my money. (P7, I4)

Discharge Process and Transition Experience

While some participants were excited about the thought of freedom of being home, others were anxious about the burdens of returning to life outside of the hospital.

I kind of feel like, yeah, I want to go home, but then I think to myself what am I going to do when I get home. Am I just going to go back to what I’ve been doing? Am I going to really change? Am I going to forget to take my pill one day because I’m home and stuff like that. (P4, I1)

The discharge process was often perceived by participants to be rushed. Some participants found the discharge meetings helpful, while others did not feel the process empowered them to engage in a meaningful conversation with hospital staff.

There was no one there with me to even help me with my brain, to think. But it’s afterwards I’m like why didn’t I say that, like that’s what I meant to say. The brain just doesn’t function that way. (P8, I2).

This participant struggled with the transition. One week after discharge when she was asked how her health was she replied:

Terrible. I’ve got no energy. I haven’t eaten for 3 days. I haven’t drank for 3 days. I’ve got diarrhea galore […] Just no appetite whatsoever. I can’t even make it up the stairs without losing my breath. If I make it up the stairs, I have to sit for 15 or 20 minutes… (P8, I3)

The weight of maintaining activities of daily living was prominent in all post-discharge interviews, in many cases accentuated by declining health. The transition to home was more challenging than participants expected; the experience was strongly influenced by the stability of their health, their environment, and the complexity of their lives.

Follow-up and Referrals

Discharge summaries included a mean of 7 referrals. All participants were referred to a case coordinator, nurse, and family physician. Other referrals included pharmacist (n = 8); personal support worker (n = 6); housing (n = 5); and food-support programs (n = 5).

Several factors led to challenges accessing and receiving services. Participants identified: difficulty with requisite paperwork; mobility and financial constraints; personal and logistical challenges with home-care providers; and competing priorities, such as caring for family. These experiences were frequently accompanied by frustration and anxiety.

Because, if I’m in [city where girlfriend lives], I will not get the support that I get when I’m home. Like my nurse comes. [She] was supposed to come and see me twice and I missed that. I missed like 4 [appointments]. You understand? Certain things I’ve been missing. (P6, I4)

When one participant was asked if she had followed up with the food support program she had been referred to, she responded:

Oh, baby, no. I’ve been so confused. I’ve had ODSP [referring to Ontario Disability Support Program, a government disability program] on my case. I’ve got all the files all mixed up. My worker’s a real bitch. She hates me, big time. I was supposed to go bring in papers today, but I couldn’t get out of bed. I don’t know how much trouble I’m going to be in with ODSP now. (P8, I3)

Despite comprehensive discharge plans and referrals, all participants experienced delays and difficulties in accessing and receiving services. In most cases, there was no single contributing factor to these challenges; the unique experiences were a result of the complex interplay of multiple factors for each individual.

Patient Priorities

In the hospital, participants primarily identified goals of improving physical health and medication adherence. However, these goals often shifted to meeting basic living necessities and supporting others upon discharge. Barriers to adequate food and mobility were prominent themes.

One participant spoke about the challenges of supporting her son while struggling with her own health after discharge:

Well, I’ve been dying, I can’t even walk, and yet I’m the one that still has to go to WalMart, to grab milk and bread for my kid. It’s not like I need any of that stuff, because I don’t even eat. (P8, I3)

Participants were admitted on a mean of 6 medications and discharged with a mean of 14 (Table 1). In the hospital, medications are dispensed directly to patients; however, maintaining optimal adherence at home was complex. When 1 participant was asked about her medications after being home for a week, she said:

My meds, you know I have the cream that I’m supposed to put … and I can’t find it. I lost it yesterday. I used it yesterday morning and all day yesterday I’m looking, like, did it fall behind there? But, obviously, I can’t look over there [because of mobility challenges] … I don’t think I can get it covered [by insurance to replace it]. (P5, I3)

Participants found it difficult to follow a specific dosing schedule, ensure food intake corresponded to medication guidelines, and navigate the impact of substance use. Substance use for some was associated with nonadherence. A participant, explaining his quickly declining health, spoke about the impact of using crack cocaine:

Yeah, when I use I don’t think about medicating, taking my pills or anything like that. That’s not even on your mind. It doesn’t come across your mind. […] I guess, that’s part of the addictive personality. It wants to grab hold of you and say “no, focus on me, focus on me.” (P7, I4)

Others used marijuana as an appetite stimulant and a critical piece of their medication adherence routine.

This study followed complex patients through hospital discharge and transition back into the community. In the hospital, participants focused on medical goals, but following discharge basic living needs became the priority. Despite a comprehensive plan to provide support upon discharge, participants found executing and following up with referrals, services, and medication adherence was often overwhelming and not achieved in the month post-hospitalization.

Our study provides depth and context to support and understand the findings of reviews evaluating interventions to improve transitions in care.^23,24 A systematic review of interventions to decrease 30-day readmission rates concluded that comprehensive support interventions (with many components) contributed to the greatest reduction in risk of readmission.¹⁶ Components that showed the greatest impact were those that were designed to improve patients’ capacity for self-care (including their ability to access and follow through with post-discharge care plans) and those that involved more individuals in the delivery of care.²³

Our results also support and expand on other qualitative findings of complex patients. Kangovi et al.²⁵ interviewed patients with low socioeconomic status at a single time point post-discharge to identify common experiences. They summarized their findings in 6 themes: powerlessness during hospitalization; incongruence of patient and clinical team goals; competing issues influencing prominence of health behaviors; socioeconomic constraints on patients’ ability to perform recommended behaviors; sense of abandonment after discharge; and loss of self-efficacy resulting from the “failure” to follow the discharge plan. Our findings tell a very similar story but provide the additional context and understanding of the lived experience over time. We found that the transition experience was most challenging when the home environment was unstable, resulting in a shift in priorities from those set during hospitalization.

While increased support may improve outcomes, there is a need to improve awareness, integration, and support for building capacity within complex patients.²⁶ Capacity is defined here as the sum of resources and abilities that a patient can draw on, and includes physical and mental as well as social, financial, personal, and environmental capabilities and resources.²⁷ This includes understanding the potential negative impact of developing a clinical plan which, in order to operationalize, requires resources in excess of the patient’s capacity at that time.²⁷ Minimally disruptive medicine, a promising theoretical approach for improving the care of complex clients, embodies the awareness of capacity in achieving patient-centered care while “imposing the smallest possible treatment burden on patients’ lives.”²⁸

This study, although not without its limitations, provides an in-depth exploration of the experiences of a small number of patients living with HIV, recruited from a single facility in Toronto, Canada after relatively long hospital stays. There are specific context issues related to HIV, such as stigma and severe consequences for suboptimal medication adherence. Furthermore, this study took place where many urban health resources exist; complex patients in rural settings or in environments less tailored to the needs associated with complex medical, psychiatric, and social conditions may experience greater barriers in the transition process. Although this study captured data from medical charts and documents relevant to the cases, further exploration of the clinician decision-making process in creating the discharge plans and additional sources of data on health outcomes post-discharge would be beneficial.

Despite its limitations, this study provides detail and depth to understand some of the most complex patients who suffer from significant challenges in the health system and who are amongst the highest-cost healthcare users. The case study approach, with serial interviews, is an important strength of this study, enabling meaningful insight into hospital discharge processes and challenges experienced by complex patients that can inform individual-level care practice and the development of new programs and interventions.

This study builds on recent research with complex patients in calling for a new approach to clinical care.^6,29,30 In order to support complex patients through discharge, clinical goals and referrals must be made in light of a patient’s capacity in the community. Structural changes may be made to improve coordination and access to services, decreasing the burden and improving the healthcare experience. Albreht et al.³¹ highlight a number of promising programs across Europe (such as the Clinic for Multimorbidity and Polypharmacy in Denmark) designed to improve the health and healthcare for individuals living with multiple chronic conditions. Small-scale changes are also important such as increasing conversations about the capacity and limitations of individuals listed as social supports, and making appropriate and realistic referrals based on an understanding of a patient’s capacity and motivation for follow-up. Shippee et al.³² identify a list of approaches in line with minimally disruptive medicine that can be integrated into existing systems as part of a developing “toolkit” (eg, elicitation of transcendent patient goals, and integration of patient-reported outcome tracking of challenges and burdens associated with health and daily living). The findings of this study suggest that the elements of the toolkit may provide a foundation for future interventions and research to improve hospital care and discharge outcomes for complex patients.

Disclosures

This project was funded by a Canadian Institutes of Health Research (CIHR) HIV/AIDS Community-based Research Catalyst Grant (#126669). Dr. Brennan’s research is supported by an Ontario HIV Treatment Network (OHTN) Applied HIV Research Chair. Dr. Chan Carusone reports grants from Canadian Institutes of Health Research during the conduct of the study.

Central line-associated bloodstream infections (CLABSIs) are common and serious occurrences across healthcare systems, with an attributable mortality of 12% to 25%.^1,2 Given this burden,^3–5 CLABSI is a focus for both high-profile public reporting and quality improvement interventions. An integral component of such interventions is audit and feedback via quality metrics. These measures are intended to allow decision makers to assess their own performance and appropriately allocate resources. Quality metrics present a substantial cost to health systems, with an estimated $15.4 billion dollars spent annually simply for reporting.⁶ Despite this toll, “audit and feedback” interventions have proven to be variably successful.^7–9 The mechanisms that limit the effectiveness of these interventions remain
poorly understood.

One plausible explanation for limited efficacy of quality metrics is inadequate clinician numeracy—that is, “the ability to understand the quantitative aspects of clinical medicine, original research, quality improvement, and financial matters.”¹⁰ Indeed, clinicians are not consistently able to interpret probabilities and or clinical test characteristics. For example, Wegwarth et al. identified shortcomings in physician application of lead-time bias toward cancer screening.¹¹ Additionally, studies have demonstrated systematic misinterpretations of probabilistic information in clinical settings, along with misconceptions regarding the impact of prevalence on post-test probabilities.^12,13 Effective interpretation of rates may be a key—if unstated—requirement of many CLABSI quality improvement efforts.^14–19 Our broader hypothesis is that clinicians who can more accurately interpret quality data, even if only from their own institution, are more likely to act on it appropriately and persistently than those who feel they must depend on a preprocessed interpretation of that same data by some other expert.

Therefore, we designed a survey to assess the numeracy of clinicians on CLABSI data presented in a prototypical feedback report. We studied 3 domains of comprehension: (1) basic numeracy: numerical tasks related to simple data; (2) risk-adjustment numeracy: numerical tasks related to risk-adjusted data; and (3) risk-adjustment interpretation: inferential tasks concerning risk-adjusted data. We hypothesized that clinician performance would vary substantially across domains, with the poorest performance in risk-
adjusted data.

We conducted a cross-sectional survey of clinician numeracy regarding CLABSI feedback data. Respondents were also asked to provide demographic information and opinions regarding the reliability of quality metric data. Survey recruitment occurred on Twitter, a novel approach that leveraged social media to facilitate rapid recruitment of participants. The study instrument was administered using a web survey with randomized question order to preclude any possibility of order effects between questions. The study was deemed Institutional Review Board exempt by the University of Michigan: protocol HUM00106696.

Data Presentation Method

To determine the optimal mode of presenting data, we reviewed the literature on quality metric numeracy and presentation methods. Additionally, we evaluated quality metric presentation methods used by the Centers for Disease Control and Prevention (CDC), Centers for Medicare & Medicaid Services (CMS), and a tertiary academic medical center. After assessing the available literature and options, we adapted a CLABSI data presentation array from a study that had qualitatively validated the format using physician feedback (Appendix).²⁰ We used hypothetical CLABSI data for our survey.

Survey Development

We developed a survey that included an 11-item test regarding CLABSI numeracy and data interpretation. Additional questions related to quality metric reliability and demographic information were included. No preexisting assessment tools existed for our areas of interest. Therefore, we developed a novel instrument using a broad, exploratory approach as others have employed.²¹ 

First, we defined 3 conceptual categories related to CLABSI data. Within this conceptual framework, an iterative process of development and revision was used to assemble a question bank from which the survey would be constructed. A series of think-aloud sessions were held to evaluate each prompt for precision, clarity, and accuracy in assessing the conceptual categories. Correct and incorrect answers were defined based on literature review in conjunction with input from methodological and content experts (TJI and VC) (see Appendix for answer explanations). 

Within the conceptual categories related to CLABSI risk-adjustment, a key measure is the standardized infection ratio (SIR). This value is defined as the ratio of observed number of CLABSI over the expected number of CLABSIs.²² This is the primary measure to stratify hospital performance, and it was used in our assessment of risk-adjustment comprehension. In total, 54 question prompts were developed and subsequently narrowed to 11 study questions for the initial survey. 

The instrument was then pretested in a cohort of 8 hospitalists and intensivists to ensure appropriate comprehension, retrieval, and judgment processes.²³ Questions were revised based on feedback from this cognitive testing to constitute the final instrument. During the survey, the data table was reshown on each page directly above each question and so was always on the same screen for the respondents.

Survey Sample

We innovated by using Twitter as an online platform for recruiting participants; we used Survey Monkey to host the electronic instrument. Two authors (TJI, VC) systematically sent out solicitation tweets to their followers. These tweets clearly indicated that the recruitment was for the purpose of a research study, and participants would receive no financial reward/incentive (Appendix). A link to the survey was provided in each tweet, and the period of recruitment was 30 days. To ensure respondents were clinicians, they needed to first answer a screening question recognizing that central lines were placed in the subclavian site but not the aorta, iliac, or radial sites.

To prevent systematic or anchoring biases, the order of questions was electronically randomized for each respondent. The primary outcome was the percentage correct of attempted questions.

Statistical Analysis

Descriptive statistics were calculated for all demographic variables. The primary outcome was evaluated as a dichotomous variable for each question (correct vs. incorrect response), and as a continuous variable when assessing mean percent correct on the overall survey. Demographic and conceptual associations were assessed via t-tests, chi-square, or Fisher exact tests. Point biserial correlations were calculated to assess for associations between response to a single question and overall performance on the survey. 

To evaluate the association between various respondent characteristics and responses, logistic regression analyses were performed. An ANOVA was performed to assess the association between self-reported reliability of quality metric data and the overall performance on attempted items. Analyses were conducted using STATA MP 14.0 (College Station, TX); P <0.05 was considered statistically significant.

A total of 97 respondents attempted at least 1 question on the survey, and 72 respondents attempted all 11 questions, yielding 939 unique responses for analysis. Seventy respondents (87%) identified as doctors or nurses, and 44 (55%) reported having 6 to 20 years of experience; the survey cohort also came from 6 nations (Table 1). All respondents answered the CLABSI knowledge filter question correctly.

Primary Outcome

The mean percent correct of attempted questions was 61% (standard deviation 21%, interquartile range 50%-75%) (Figure 1). Of those who answered all 11 CLABSI questions, the mean percent correct was 63% (95% CI, 59%-67%). Some questions were answered correctly more often than others—ranging from 17% to 95% (Table 2). Doctors answered 68% of questions correctly (95% CI, 63%-73%), while nurses and other respondents answered 57% of questions correctly (95% CI, 52%-62%) (P = 0.003). Other demographic variables—including self-reported involvement in a quality improvement committee and being from the United States versus elsewhere—were not associated with survey performance. The point biserial correlations for each individual question with overall performance were all more than 0.2 (range 0.24–0.62) and all statistically significant at P < 0.05.

Concept-Specific Performance

Average percent correct declined across categories as numeracy requirements increased (P < 0.05 for all pairwise comparisons). In the area of basic numeracy, respondents’ mean percent correct was 82% (95% CI, 77%-87%) of attempted. This category had 4 questions, with a performance range of 77% to 90%. For example, on the question, “Which hospital has the lowest CLABSI rate?”, 80% of respondents answered correctly. For risk-adjustment numeracy, the mean percent correct was 70% (95% CI, 64%-76%); 2 items assessed this category. For “Which is better: a higher or lower SIR?”, 95% of the cohort answered correctly. However, on “If hospital B had its number of projected infection halved, what is its SIR?”, only 46% of those who attempted the question answered correctly.

Questions featuring risk-adjustment interpretation had an average percent correct of 43% (95% CI, 37%-49%). Five questions made up this category, with a percent correct range of 17% to 75%. For example, on the question, “Which hospital’s patients are the most predisposed to developing CLABSI?”, only 32% of respondents answered this correctly. In contrast, for the question “Which hospital is most effective at preventing CLABSI?”, 51% answered correctly. Figure 2 illustrates the cohort’s performance on each conceptual category while Table 2 displays question-by-question results.

Opinions Regarding CLABSI Data Reliability

Respondents were also asked about their opinion regarding the reliability of CLABSI quality metric data. Forty-three percent of respondents stated that such data were reliable at best 50% of the time. Notably, 10% of respondents indicated that CLABSI quality metric data were rarely or never reliable. There was no association between perceived reliability of quality metric data and survey performance (P = 0.87).

This Twitter-based study found wide variation in clinician interpretation of CLABSI quality data, with low overall performance. In particular, comprehension and interpretation of risk-adjusted data were substantially worse than unadjusted data. Although doctors performed somewhat better than nurses and other respondents, those involved in quality improvement initiatives performed no better than respondents who were not. Collectively, these findings suggest clinicians may not reliably comprehend quality metric data, potentially affecting their ability to utilize audit and feedback data. These results may have important implications for policy efforts that seek to leverage quality metric data to improve patient safety.

An integral component of many contemporary quality improvement initiatives is audit and feedback through metrics.⁶ Unfortunately, formal audit and feedback, along with other similar methods that benchmark data, have not consistently improved outcomes.^24–27 A recent meta-analysis noted that audit and feedback interventions are not becoming more efficacious over time; the study further asserted that “new trials have provided little new knowledge regarding key effect modifiers.”⁹ Our findings suggest that numeracy and comprehension of quality metrics may be important candidate effect modifiers not previously considered. Simply put: we hypothesize that without intrinsic comprehension of data, impetus or insight to change practice might be diminished. In other words, clinicians may be more apt to act on insights they themselves derive from the data than when they are simply told what the data “mean.”

The present study further demonstrates that clinicians do not understand risk-adjusted data as well as raw data. Risk-adjustment has long been recognized as necessary to compare outcomes among hospitals.^28,29 However, risk-adjustment is complex and, by its nature, difficult to understand. Although efforts have focused on improving the statistical reliability of quality metrics, this may represent but one half of the equation. Numeracy and interpretation of the data by decision makers are potentially equally important to effecting change. Because clinicians seem to have difficulty understanding risk-adjusted data, this deficit may be of growing importance as our risk-adjustment techniques become more sophisticated.

We note that clinicians expressed concerns regarding the reliability of quality metric feedback. These findings corroborate recent research that has reported reservations from hospital leaders concerning quality data.^30,31 However, as shown in the context of patients and healthcare decisions, the aversion associated with quality metrics may be related to incomplete understanding of the data.³² Whether perceptions of unreliability drive lack of understanding or, conversely, whether lack of understanding fuels perceived unreliability is an important question that requires further study.

This study has several strengths. First, we used rigorous survey development techniques to evaluate the understudied issue of quality metric numeracy. Second, our sample size was sufficient to show statistically significant differences in numeracy and comprehension of CLABSI quality metric data. Third, we leveraged social media to rapidly acquire this sample. Finally, our results provided new insights that may have important implications in the area of quality metrics.

There were also limitations to our study. First, the Twitter-derived sample precludes the calculation of a response rate and may not be representative of individuals engaged in CLABSI prevention. However, respondents were solicited from the Twitter-followers of 2 health services researchers (TJI, VC) who are actively engaged in scholarly activities pertaining to critically ill patients and hospital-acquired complications. Thus, our sample likely represents a highly motivated subset that engages in these topics on a regular basis—potentially making them more numerate than average clinicians. Second, we did not ask whether the respondents had previously seen CLABSI data specifically, so we cannot stratify by exposure to such data. Third, this study assessed only CLABSI quality metric data; generalizations regarding numeracy with other metrics should be made with caution. However, as many such data are presented in similar formats, we suspect our findings are applicable to similar audit-and-feedback initiatives.

The findings of this study serve as a stimulus for further inquiry. Research of this nature needs to be carried out in samples drawn from specific, policy-relevant populations (eg, infection control practitioners, bedside nurses, intensive care unit directors). Such studies should include longitudinal assessments of numeracy that attempt to mechanistically examine its impact on CLABSI prevention efforts and outcomes. The latter is an important issue as the link between numeracy and behavioral response, while plausible, cannot be assumed, particularly given the complexity of issues related to behavioral modification.³³ Additionally, whether alternate presentations of quality data affect numeracy, interpretation, and performance is worthy of further testing; indeed, this has been shown to be the case in other forms of communication.^34–37 Until data from larger samples are available, it may be prudent for quality improvement leaders to assess the comprehension of local clinicians regarding feedback and whether lack of adequate comprehension is a barrier to deploying quality improvement interventions.

Quality measurement is a cornerstone of patient safety as it seeks to assess and improve the care delivered at the bedside. Rigorous metric development is important; however, ensuring that decision makers understand complex quality metrics may be equally fundamental. Given the cost of examining quality, elucidating the mechanisms of numeracy and interpretation as decision makers engage with quality metric data is necessary, along with whether improved comprehension leads to behavior change. Such inquiry may provide an evidence-base to shape alterations in quality metric deployment that will ensure maximal efficacy in driving practice change.

Disclosures

This work was supported by VA HSR&D IIR-13-079 (TJI). Dr. Chopra is supported by a career development award from the Agency of Healthcare Research and Quality (1-K08-HS022835-01). The views expressed here are the authors’ own and do not necessarily represent the view of the US Government or the Department of Veterans’ Affairs. The authors report no conflicts of interest.

Central line-associated bloodstream infections (CLABSIs) are common and serious occurrences across healthcare systems, with an attributable mortality of 12% to 25%.^1,2 Given this burden,^3–5 CLABSI is a focus for both high-profile public reporting and quality improvement interventions. An integral component of such interventions is audit and feedback via quality metrics. These measures are intended to allow decision makers to assess their own performance and appropriately allocate resources. Quality metrics present a substantial cost to health systems, with an estimated $15.4 billion dollars spent annually simply for reporting.⁶ Despite this toll, “audit and feedback” interventions have proven to be variably successful.^7–9 The mechanisms that limit the effectiveness of these interventions remain
poorly understood.

One plausible explanation for limited efficacy of quality metrics is inadequate clinician numeracy—that is, “the ability to understand the quantitative aspects of clinical medicine, original research, quality improvement, and financial matters.”¹⁰ Indeed, clinicians are not consistently able to interpret probabilities and or clinical test characteristics. For example, Wegwarth et al. identified shortcomings in physician application of lead-time bias toward cancer screening.¹¹ Additionally, studies have demonstrated systematic misinterpretations of probabilistic information in clinical settings, along with misconceptions regarding the impact of prevalence on post-test probabilities.^12,13 Effective interpretation of rates may be a key—if unstated—requirement of many CLABSI quality improvement efforts.^14–19 Our broader hypothesis is that clinicians who can more accurately interpret quality data, even if only from their own institution, are more likely to act on it appropriately and persistently than those who feel they must depend on a preprocessed interpretation of that same data by some other expert.

Therefore, we designed a survey to assess the numeracy of clinicians on CLABSI data presented in a prototypical feedback report. We studied 3 domains of comprehension: (1) basic numeracy: numerical tasks related to simple data; (2) risk-adjustment numeracy: numerical tasks related to risk-adjusted data; and (3) risk-adjustment interpretation: inferential tasks concerning risk-adjusted data. We hypothesized that clinician performance would vary substantially across domains, with the poorest performance in risk-
adjusted data.

We conducted a cross-sectional survey of clinician numeracy regarding CLABSI feedback data. Respondents were also asked to provide demographic information and opinions regarding the reliability of quality metric data. Survey recruitment occurred on Twitter, a novel approach that leveraged social media to facilitate rapid recruitment of participants. The study instrument was administered using a web survey with randomized question order to preclude any possibility of order effects between questions. The study was deemed Institutional Review Board exempt by the University of Michigan: protocol HUM00106696.

Data Presentation Method

To determine the optimal mode of presenting data, we reviewed the literature on quality metric numeracy and presentation methods. Additionally, we evaluated quality metric presentation methods used by the Centers for Disease Control and Prevention (CDC), Centers for Medicare & Medicaid Services (CMS), and a tertiary academic medical center. After assessing the available literature and options, we adapted a CLABSI data presentation array from a study that had qualitatively validated the format using physician feedback (Appendix).²⁰ We used hypothetical CLABSI data for our survey.

Survey Development

We developed a survey that included an 11-item test regarding CLABSI numeracy and data interpretation. Additional questions related to quality metric reliability and demographic information were included. No preexisting assessment tools existed for our areas of interest. Therefore, we developed a novel instrument using a broad, exploratory approach as others have employed.²¹ 

First, we defined 3 conceptual categories related to CLABSI data. Within this conceptual framework, an iterative process of development and revision was used to assemble a question bank from which the survey would be constructed. A series of think-aloud sessions were held to evaluate each prompt for precision, clarity, and accuracy in assessing the conceptual categories. Correct and incorrect answers were defined based on literature review in conjunction with input from methodological and content experts (TJI and VC) (see Appendix for answer explanations). 

Within the conceptual categories related to CLABSI risk-adjustment, a key measure is the standardized infection ratio (SIR). This value is defined as the ratio of observed number of CLABSI over the expected number of CLABSIs.²² This is the primary measure to stratify hospital performance, and it was used in our assessment of risk-adjustment comprehension. In total, 54 question prompts were developed and subsequently narrowed to 11 study questions for the initial survey. 

The instrument was then pretested in a cohort of 8 hospitalists and intensivists to ensure appropriate comprehension, retrieval, and judgment processes.²³ Questions were revised based on feedback from this cognitive testing to constitute the final instrument. During the survey, the data table was reshown on each page directly above each question and so was always on the same screen for the respondents.

Survey Sample

We innovated by using Twitter as an online platform for recruiting participants; we used Survey Monkey to host the electronic instrument. Two authors (TJI, VC) systematically sent out solicitation tweets to their followers. These tweets clearly indicated that the recruitment was for the purpose of a research study, and participants would receive no financial reward/incentive (Appendix). A link to the survey was provided in each tweet, and the period of recruitment was 30 days. To ensure respondents were clinicians, they needed to first answer a screening question recognizing that central lines were placed in the subclavian site but not the aorta, iliac, or radial sites.

To prevent systematic or anchoring biases, the order of questions was electronically randomized for each respondent. The primary outcome was the percentage correct of attempted questions.

Statistical Analysis

Descriptive statistics were calculated for all demographic variables. The primary outcome was evaluated as a dichotomous variable for each question (correct vs. incorrect response), and as a continuous variable when assessing mean percent correct on the overall survey. Demographic and conceptual associations were assessed via t-tests, chi-square, or Fisher exact tests. Point biserial correlations were calculated to assess for associations between response to a single question and overall performance on the survey. 

To evaluate the association between various respondent characteristics and responses, logistic regression analyses were performed. An ANOVA was performed to assess the association between self-reported reliability of quality metric data and the overall performance on attempted items. Analyses were conducted using STATA MP 14.0 (College Station, TX); P <0.05 was considered statistically significant.

A total of 97 respondents attempted at least 1 question on the survey, and 72 respondents attempted all 11 questions, yielding 939 unique responses for analysis. Seventy respondents (87%) identified as doctors or nurses, and 44 (55%) reported having 6 to 20 years of experience; the survey cohort also came from 6 nations (Table 1). All respondents answered the CLABSI knowledge filter question correctly.

Primary Outcome

The mean percent correct of attempted questions was 61% (standard deviation 21%, interquartile range 50%-75%) (Figure 1). Of those who answered all 11 CLABSI questions, the mean percent correct was 63% (95% CI, 59%-67%). Some questions were answered correctly more often than others—ranging from 17% to 95% (Table 2). Doctors answered 68% of questions correctly (95% CI, 63%-73%), while nurses and other respondents answered 57% of questions correctly (95% CI, 52%-62%) (P = 0.003). Other demographic variables—including self-reported involvement in a quality improvement committee and being from the United States versus elsewhere—were not associated with survey performance. The point biserial correlations for each individual question with overall performance were all more than 0.2 (range 0.24–0.62) and all statistically significant at P < 0.05.

Concept-Specific Performance

Average percent correct declined across categories as numeracy requirements increased (P < 0.05 for all pairwise comparisons). In the area of basic numeracy, respondents’ mean percent correct was 82% (95% CI, 77%-87%) of attempted. This category had 4 questions, with a performance range of 77% to 90%. For example, on the question, “Which hospital has the lowest CLABSI rate?”, 80% of respondents answered correctly. For risk-adjustment numeracy, the mean percent correct was 70% (95% CI, 64%-76%); 2 items assessed this category. For “Which is better: a higher or lower SIR?”, 95% of the cohort answered correctly. However, on “If hospital B had its number of projected infection halved, what is its SIR?”, only 46% of those who attempted the question answered correctly.

Questions featuring risk-adjustment interpretation had an average percent correct of 43% (95% CI, 37%-49%). Five questions made up this category, with a percent correct range of 17% to 75%. For example, on the question, “Which hospital’s patients are the most predisposed to developing CLABSI?”, only 32% of respondents answered this correctly. In contrast, for the question “Which hospital is most effective at preventing CLABSI?”, 51% answered correctly. Figure 2 illustrates the cohort’s performance on each conceptual category while Table 2 displays question-by-question results.

Opinions Regarding CLABSI Data Reliability

Respondents were also asked about their opinion regarding the reliability of CLABSI quality metric data. Forty-three percent of respondents stated that such data were reliable at best 50% of the time. Notably, 10% of respondents indicated that CLABSI quality metric data were rarely or never reliable. There was no association between perceived reliability of quality metric data and survey performance (P = 0.87).

This Twitter-based study found wide variation in clinician interpretation of CLABSI quality data, with low overall performance. In particular, comprehension and interpretation of risk-adjusted data were substantially worse than unadjusted data. Although doctors performed somewhat better than nurses and other respondents, those involved in quality improvement initiatives performed no better than respondents who were not. Collectively, these findings suggest clinicians may not reliably comprehend quality metric data, potentially affecting their ability to utilize audit and feedback data. These results may have important implications for policy efforts that seek to leverage quality metric data to improve patient safety.

An integral component of many contemporary quality improvement initiatives is audit and feedback through metrics.⁶ Unfortunately, formal audit and feedback, along with other similar methods that benchmark data, have not consistently improved outcomes.^24–27 A recent meta-analysis noted that audit and feedback interventions are not becoming more efficacious over time; the study further asserted that “new trials have provided little new knowledge regarding key effect modifiers.”⁹ Our findings suggest that numeracy and comprehension of quality metrics may be important candidate effect modifiers not previously considered. Simply put: we hypothesize that without intrinsic comprehension of data, impetus or insight to change practice might be diminished. In other words, clinicians may be more apt to act on insights they themselves derive from the data than when they are simply told what the data “mean.”

The present study further demonstrates that clinicians do not understand risk-adjusted data as well as raw data. Risk-adjustment has long been recognized as necessary to compare outcomes among hospitals.^28,29 However, risk-adjustment is complex and, by its nature, difficult to understand. Although efforts have focused on improving the statistical reliability of quality metrics, this may represent but one half of the equation. Numeracy and interpretation of the data by decision makers are potentially equally important to effecting change. Because clinicians seem to have difficulty understanding risk-adjusted data, this deficit may be of growing importance as our risk-adjustment techniques become more sophisticated.

We note that clinicians expressed concerns regarding the reliability of quality metric feedback. These findings corroborate recent research that has reported reservations from hospital leaders concerning quality data.^30,31 However, as shown in the context of patients and healthcare decisions, the aversion associated with quality metrics may be related to incomplete understanding of the data.³² Whether perceptions of unreliability drive lack of understanding or, conversely, whether lack of understanding fuels perceived unreliability is an important question that requires further study.

This study has several strengths. First, we used rigorous survey development techniques to evaluate the understudied issue of quality metric numeracy. Second, our sample size was sufficient to show statistically significant differences in numeracy and comprehension of CLABSI quality metric data. Third, we leveraged social media to rapidly acquire this sample. Finally, our results provided new insights that may have important implications in the area of quality metrics.

There were also limitations to our study. First, the Twitter-derived sample precludes the calculation of a response rate and may not be representative of individuals engaged in CLABSI prevention. However, respondents were solicited from the Twitter-followers of 2 health services researchers (TJI, VC) who are actively engaged in scholarly activities pertaining to critically ill patients and hospital-acquired complications. Thus, our sample likely represents a highly motivated subset that engages in these topics on a regular basis—potentially making them more numerate than average clinicians. Second, we did not ask whether the respondents had previously seen CLABSI data specifically, so we cannot stratify by exposure to such data. Third, this study assessed only CLABSI quality metric data; generalizations regarding numeracy with other metrics should be made with caution. However, as many such data are presented in similar formats, we suspect our findings are applicable to similar audit-and-feedback initiatives.

The findings of this study serve as a stimulus for further inquiry. Research of this nature needs to be carried out in samples drawn from specific, policy-relevant populations (eg, infection control practitioners, bedside nurses, intensive care unit directors). Such studies should include longitudinal assessments of numeracy that attempt to mechanistically examine its impact on CLABSI prevention efforts and outcomes. The latter is an important issue as the link between numeracy and behavioral response, while plausible, cannot be assumed, particularly given the complexity of issues related to behavioral modification.³³ Additionally, whether alternate presentations of quality data affect numeracy, interpretation, and performance is worthy of further testing; indeed, this has been shown to be the case in other forms of communication.^34–37 Until data from larger samples are available, it may be prudent for quality improvement leaders to assess the comprehension of local clinicians regarding feedback and whether lack of adequate comprehension is a barrier to deploying quality improvement interventions.

Quality measurement is a cornerstone of patient safety as it seeks to assess and improve the care delivered at the bedside. Rigorous metric development is important; however, ensuring that decision makers understand complex quality metrics may be equally fundamental. Given the cost of examining quality, elucidating the mechanisms of numeracy and interpretation as decision makers engage with quality metric data is necessary, along with whether improved comprehension leads to behavior change. Such inquiry may provide an evidence-base to shape alterations in quality metric deployment that will ensure maximal efficacy in driving practice change.

Disclosures

This work was supported by VA HSR&D IIR-13-079 (TJI). Dr. Chopra is supported by a career development award from the Agency of Healthcare Research and Quality (1-K08-HS022835-01). The views expressed here are the authors’ own and do not necessarily represent the view of the US Government or the Department of Veterans’ Affairs. The authors report no conflicts of interest.

Central line-associated bloodstream infections (CLABSIs) are common and serious occurrences across healthcare systems, with an attributable mortality of 12% to 25%.^1,2 Given this burden,^3–5 CLABSI is a focus for both high-profile public reporting and quality improvement interventions. An integral component of such interventions is audit and feedback via quality metrics. These measures are intended to allow decision makers to assess their own performance and appropriately allocate resources. Quality metrics present a substantial cost to health systems, with an estimated $15.4 billion dollars spent annually simply for reporting.⁶ Despite this toll, “audit and feedback” interventions have proven to be variably successful.^7–9 The mechanisms that limit the effectiveness of these interventions remain
poorly understood.

One plausible explanation for limited efficacy of quality metrics is inadequate clinician numeracy—that is, “the ability to understand the quantitative aspects of clinical medicine, original research, quality improvement, and financial matters.”¹⁰ Indeed, clinicians are not consistently able to interpret probabilities and or clinical test characteristics. For example, Wegwarth et al. identified shortcomings in physician application of lead-time bias toward cancer screening.¹¹ Additionally, studies have demonstrated systematic misinterpretations of probabilistic information in clinical settings, along with misconceptions regarding the impact of prevalence on post-test probabilities.^12,13 Effective interpretation of rates may be a key—if unstated—requirement of many CLABSI quality improvement efforts.^14–19 Our broader hypothesis is that clinicians who can more accurately interpret quality data, even if only from their own institution, are more likely to act on it appropriately and persistently than those who feel they must depend on a preprocessed interpretation of that same data by some other expert.

Therefore, we designed a survey to assess the numeracy of clinicians on CLABSI data presented in a prototypical feedback report. We studied 3 domains of comprehension: (1) basic numeracy: numerical tasks related to simple data; (2) risk-adjustment numeracy: numerical tasks related to risk-adjusted data; and (3) risk-adjustment interpretation: inferential tasks concerning risk-adjusted data. We hypothesized that clinician performance would vary substantially across domains, with the poorest performance in risk-
adjusted data.

We conducted a cross-sectional survey of clinician numeracy regarding CLABSI feedback data. Respondents were also asked to provide demographic information and opinions regarding the reliability of quality metric data. Survey recruitment occurred on Twitter, a novel approach that leveraged social media to facilitate rapid recruitment of participants. The study instrument was administered using a web survey with randomized question order to preclude any possibility of order effects between questions. The study was deemed Institutional Review Board exempt by the University of Michigan: protocol HUM00106696.

Data Presentation Method

To determine the optimal mode of presenting data, we reviewed the literature on quality metric numeracy and presentation methods. Additionally, we evaluated quality metric presentation methods used by the Centers for Disease Control and Prevention (CDC), Centers for Medicare & Medicaid Services (CMS), and a tertiary academic medical center. After assessing the available literature and options, we adapted a CLABSI data presentation array from a study that had qualitatively validated the format using physician feedback (Appendix).²⁰ We used hypothetical CLABSI data for our survey.

Survey Development

We developed a survey that included an 11-item test regarding CLABSI numeracy and data interpretation. Additional questions related to quality metric reliability and demographic information were included. No preexisting assessment tools existed for our areas of interest. Therefore, we developed a novel instrument using a broad, exploratory approach as others have employed.²¹ 

First, we defined 3 conceptual categories related to CLABSI data. Within this conceptual framework, an iterative process of development and revision was used to assemble a question bank from which the survey would be constructed. A series of think-aloud sessions were held to evaluate each prompt for precision, clarity, and accuracy in assessing the conceptual categories. Correct and incorrect answers were defined based on literature review in conjunction with input from methodological and content experts (TJI and VC) (see Appendix for answer explanations). 

Within the conceptual categories related to CLABSI risk-adjustment, a key measure is the standardized infection ratio (SIR). This value is defined as the ratio of observed number of CLABSI over the expected number of CLABSIs.²² This is the primary measure to stratify hospital performance, and it was used in our assessment of risk-adjustment comprehension. In total, 54 question prompts were developed and subsequently narrowed to 11 study questions for the initial survey. 

The instrument was then pretested in a cohort of 8 hospitalists and intensivists to ensure appropriate comprehension, retrieval, and judgment processes.²³ Questions were revised based on feedback from this cognitive testing to constitute the final instrument. During the survey, the data table was reshown on each page directly above each question and so was always on the same screen for the respondents.

Survey Sample

We innovated by using Twitter as an online platform for recruiting participants; we used Survey Monkey to host the electronic instrument. Two authors (TJI, VC) systematically sent out solicitation tweets to their followers. These tweets clearly indicated that the recruitment was for the purpose of a research study, and participants would receive no financial reward/incentive (Appendix). A link to the survey was provided in each tweet, and the period of recruitment was 30 days. To ensure respondents were clinicians, they needed to first answer a screening question recognizing that central lines were placed in the subclavian site but not the aorta, iliac, or radial sites.

To prevent systematic or anchoring biases, the order of questions was electronically randomized for each respondent. The primary outcome was the percentage correct of attempted questions.

Statistical Analysis

Descriptive statistics were calculated for all demographic variables. The primary outcome was evaluated as a dichotomous variable for each question (correct vs. incorrect response), and as a continuous variable when assessing mean percent correct on the overall survey. Demographic and conceptual associations were assessed via t-tests, chi-square, or Fisher exact tests. Point biserial correlations were calculated to assess for associations between response to a single question and overall performance on the survey. 

To evaluate the association between various respondent characteristics and responses, logistic regression analyses were performed. An ANOVA was performed to assess the association between self-reported reliability of quality metric data and the overall performance on attempted items. Analyses were conducted using STATA MP 14.0 (College Station, TX); P <0.05 was considered statistically significant.

A total of 97 respondents attempted at least 1 question on the survey, and 72 respondents attempted all 11 questions, yielding 939 unique responses for analysis. Seventy respondents (87%) identified as doctors or nurses, and 44 (55%) reported having 6 to 20 years of experience; the survey cohort also came from 6 nations (Table 1). All respondents answered the CLABSI knowledge filter question correctly.

Primary Outcome

The mean percent correct of attempted questions was 61% (standard deviation 21%, interquartile range 50%-75%) (Figure 1). Of those who answered all 11 CLABSI questions, the mean percent correct was 63% (95% CI, 59%-67%). Some questions were answered correctly more often than others—ranging from 17% to 95% (Table 2). Doctors answered 68% of questions correctly (95% CI, 63%-73%), while nurses and other respondents answered 57% of questions correctly (95% CI, 52%-62%) (P = 0.003). Other demographic variables—including self-reported involvement in a quality improvement committee and being from the United States versus elsewhere—were not associated with survey performance. The point biserial correlations for each individual question with overall performance were all more than 0.2 (range 0.24–0.62) and all statistically significant at P < 0.05.

Concept-Specific Performance

Average percent correct declined across categories as numeracy requirements increased (P < 0.05 for all pairwise comparisons). In the area of basic numeracy, respondents’ mean percent correct was 82% (95% CI, 77%-87%) of attempted. This category had 4 questions, with a performance range of 77% to 90%. For example, on the question, “Which hospital has the lowest CLABSI rate?”, 80% of respondents answered correctly. For risk-adjustment numeracy, the mean percent correct was 70% (95% CI, 64%-76%); 2 items assessed this category. For “Which is better: a higher or lower SIR?”, 95% of the cohort answered correctly. However, on “If hospital B had its number of projected infection halved, what is its SIR?”, only 46% of those who attempted the question answered correctly.

Questions featuring risk-adjustment interpretation had an average percent correct of 43% (95% CI, 37%-49%). Five questions made up this category, with a percent correct range of 17% to 75%. For example, on the question, “Which hospital’s patients are the most predisposed to developing CLABSI?”, only 32% of respondents answered this correctly. In contrast, for the question “Which hospital is most effective at preventing CLABSI?”, 51% answered correctly. Figure 2 illustrates the cohort’s performance on each conceptual category while Table 2 displays question-by-question results.

Opinions Regarding CLABSI Data Reliability

Respondents were also asked about their opinion regarding the reliability of CLABSI quality metric data. Forty-three percent of respondents stated that such data were reliable at best 50% of the time. Notably, 10% of respondents indicated that CLABSI quality metric data were rarely or never reliable. There was no association between perceived reliability of quality metric data and survey performance (P = 0.87).

This Twitter-based study found wide variation in clinician interpretation of CLABSI quality data, with low overall performance. In particular, comprehension and interpretation of risk-adjusted data were substantially worse than unadjusted data. Although doctors performed somewhat better than nurses and other respondents, those involved in quality improvement initiatives performed no better than respondents who were not. Collectively, these findings suggest clinicians may not reliably comprehend quality metric data, potentially affecting their ability to utilize audit and feedback data. These results may have important implications for policy efforts that seek to leverage quality metric data to improve patient safety.

An integral component of many contemporary quality improvement initiatives is audit and feedback through metrics.⁶ Unfortunately, formal audit and feedback, along with other similar methods that benchmark data, have not consistently improved outcomes.^24–27 A recent meta-analysis noted that audit and feedback interventions are not becoming more efficacious over time; the study further asserted that “new trials have provided little new knowledge regarding key effect modifiers.”⁹ Our findings suggest that numeracy and comprehension of quality metrics may be important candidate effect modifiers not previously considered. Simply put: we hypothesize that without intrinsic comprehension of data, impetus or insight to change practice might be diminished. In other words, clinicians may be more apt to act on insights they themselves derive from the data than when they are simply told what the data “mean.”

The present study further demonstrates that clinicians do not understand risk-adjusted data as well as raw data. Risk-adjustment has long been recognized as necessary to compare outcomes among hospitals.^28,29 However, risk-adjustment is complex and, by its nature, difficult to understand. Although efforts have focused on improving the statistical reliability of quality metrics, this may represent but one half of the equation. Numeracy and interpretation of the data by decision makers are potentially equally important to effecting change. Because clinicians seem to have difficulty understanding risk-adjusted data, this deficit may be of growing importance as our risk-adjustment techniques become more sophisticated.

We note that clinicians expressed concerns regarding the reliability of quality metric feedback. These findings corroborate recent research that has reported reservations from hospital leaders concerning quality data.^30,31 However, as shown in the context of patients and healthcare decisions, the aversion associated with quality metrics may be related to incomplete understanding of the data.³² Whether perceptions of unreliability drive lack of understanding or, conversely, whether lack of understanding fuels perceived unreliability is an important question that requires further study.

This study has several strengths. First, we used rigorous survey development techniques to evaluate the understudied issue of quality metric numeracy. Second, our sample size was sufficient to show statistically significant differences in numeracy and comprehension of CLABSI quality metric data. Third, we leveraged social media to rapidly acquire this sample. Finally, our results provided new insights that may have important implications in the area of quality metrics.

There were also limitations to our study. First, the Twitter-derived sample precludes the calculation of a response rate and may not be representative of individuals engaged in CLABSI prevention. However, respondents were solicited from the Twitter-followers of 2 health services researchers (TJI, VC) who are actively engaged in scholarly activities pertaining to critically ill patients and hospital-acquired complications. Thus, our sample likely represents a highly motivated subset that engages in these topics on a regular basis—potentially making them more numerate than average clinicians. Second, we did not ask whether the respondents had previously seen CLABSI data specifically, so we cannot stratify by exposure to such data. Third, this study assessed only CLABSI quality metric data; generalizations regarding numeracy with other metrics should be made with caution. However, as many such data are presented in similar formats, we suspect our findings are applicable to similar audit-and-feedback initiatives.

The findings of this study serve as a stimulus for further inquiry. Research of this nature needs to be carried out in samples drawn from specific, policy-relevant populations (eg, infection control practitioners, bedside nurses, intensive care unit directors). Such studies should include longitudinal assessments of numeracy that attempt to mechanistically examine its impact on CLABSI prevention efforts and outcomes. The latter is an important issue as the link between numeracy and behavioral response, while plausible, cannot be assumed, particularly given the complexity of issues related to behavioral modification.³³ Additionally, whether alternate presentations of quality data affect numeracy, interpretation, and performance is worthy of further testing; indeed, this has been shown to be the case in other forms of communication.^34–37 Until data from larger samples are available, it may be prudent for quality improvement leaders to assess the comprehension of local clinicians regarding feedback and whether lack of adequate comprehension is a barrier to deploying quality improvement interventions.

Quality measurement is a cornerstone of patient safety as it seeks to assess and improve the care delivered at the bedside. Rigorous metric development is important; however, ensuring that decision makers understand complex quality metrics may be equally fundamental. Given the cost of examining quality, elucidating the mechanisms of numeracy and interpretation as decision makers engage with quality metric data is necessary, along with whether improved comprehension leads to behavior change. Such inquiry may provide an evidence-base to shape alterations in quality metric deployment that will ensure maximal efficacy in driving practice change.

Disclosures

This work was supported by VA HSR&D IIR-13-079 (TJI). Dr. Chopra is supported by a career development award from the Agency of Healthcare Research and Quality (1-K08-HS022835-01). The views expressed here are the authors’ own and do not necessarily represent the view of the US Government or the Department of Veterans’ Affairs. The authors report no conflicts of interest.

The transfer of inpatients between hospitals for specialized services is common, affecting nearly 10% of all Medicare admissions¹ and 4.5% of all critical care hospitalizations.² At tertiary referral centers, 49% of medical intensive care unit (ICU) admissions are transferred from another hospital.³

Transfer patients have longer length of stay (LOS) than patients admitted directly from the emergency department or clinic. Among patients initially admitted to an ICU, transfer patients spend 1 day to 2.2 more days in the ICU and an additional 2 days to 4 more days total at the receiving hospital.^4,5 Furthermore, transfer patients have higher mortality than nontransferred patients by 4% to 8%.^3-5 Even after adjustment for case mix and comorbid disease, interhospital transfer is an independent predictor of both ICU death and LOS.^6,7 As a result, interhospital transfer has been associated with a $9600 increase (on average) in hospital costs.⁴

Despite evidence detailing patient handovers as a key time when poor communication can lead to delays in care and significant patient risk, ^8-10 most studies have focused on hospital discharge or change of shift, and scant effort has been dedicated to improving the interhospital handover. The process of interhospital transfer is often prolonged and discontinuous,¹¹ commonly including delays of more than 24 hours between initiation and completion. This frequently precludes direct physician-to-physician contact at the time of transfer, and physicians rely on the discharge/transfer summary.¹² Yet discharge summaries are frequently absent or incomplete,¹³ and often lack information for high-risk treatments such as systemic anticoagulation.¹⁴ The traditional reliance on discharge summaries for handover communication requires interpretation of unstandardized documentation and increases the risk for miscommunication, delays, and error.

To improve communication, we developed a 1-page handover tool for all inbound adult interhospital transfers to our academic medical center. We sought to determine whether implementation of this standardized handover tool improved the timeliness of initial care, LOS, and mortality among interhospital transfer patients. 

Study Design, Setting, Population

We conducted a retrospective cohort study of patients transferred into Vanderbilt University Hospital (VUH), an adult 626-bed quaternary care academic medical center in Nashville, Tennessee. The Vanderbilt University Institutional Review Board approved this study.  

Population

We selected for inclusion all patients age 18 or older who were transferred into VUH between July 1, 2009 and December 31, 2010. We excluded patients whose transfer was routed outside the main VUH Patient Flow Center as well as patients who did not arrive alive at VUH. We also excluded patients transferred to the emergency department and patients admitted to obstetrics, burn, or trauma services, because these admitting services did not initially use the handover tool. Patients were followed for the duration of their hospitalization at VUH.

Intervention

The 1-page handover tool was developed with multidisciplinary physician input from house staff; medical directors from intensive care, neurology, and surgery; and the chief of staff. The tool was structured on the SBAR model (Situation, Background, Assessment, and Recommendation).¹⁵ Fields on the handover tool were limited to those deemed critical for immediate patient care and designed for 1 tool to be used for both ICU and non-ICU transfers. Fields included primary diagnosis; allergies; use and last dose of anticoagulants, vasopressors, sedative/paralytics, and antibiotics; isolation needs; indwelling devices; recent operations/procedures; code status; emergency contact information; problem list; active medication list; vital signs; pertinent exam; imaging; lab findings; and overall reason for transfer. 

The handover tool was completed by the physician at the transferring hospital, faxed to VUH, and immediately scanned into the electronic record, allowing the receiving physicians to review information before patient arrival. Use of the tool was piloted first with 2 referring hospitals in April 2010 and universally recommended but not compulsory for all adult patients routed through the main VUH Patient Flow Center beginning July 1, 2010. Immediately before full implementation, the chief of staff sent letters to leadership of the 40 highest volume referral hospitals, highlighting the institutional goal of improving hand-off communication, framing completion of the tool as a step in the transfer acceptance process, and providing contact information for questions, feedback, or concerns. To ensure the tool was a standard part of the transfer process, the VUH Patient Flow Center maintained the responsibility of faxing the form to the outside facility and monitoring its receipt. The tool was processed in the same manner as other faxed patient records and treated as a part of the formal medical record to meet all standards for the Health Insurance Portability and Accountability Act (HIPAA) and medicolegal compliance. The medical center also has a separate cardiac transfer center where the handover tool was not implemented owing to its specialized workflow.

Data Source

The VUH Patient Flow Center maintains a database of all patients for whom transfer to VUH is requested, including information on the requesting hospital and the duration of transfer process. Outcome data and patient characteristics were extracted from the Enterprise Data Warehouse. Data related to comorbid illness were extracted from the Perioperative Data Warehouse, an IRB-approved data registry.

Measures

We evaluated 3 outcomes. First, we defined 2 measures of the timeliness of initial care, the time from arrival at VUH until entry of an admission order, and the time from arrival until entry of the first antibiotic order. Only antibiotics ordered within the first 36 hours of admission were included. Second, we evaluated the total LOS after transfer to VUH and the ICU LOS for patients transferred into an ICU setting. Finally, we examined in-hospital mortality at VUH. These metrics were chosen for their broad applicability across patient groups and feasibility of data capture. Length of stay and mortality also represent final common pathways for avoidance of complications. Specific patient safety indicators and complications were not abstracted due to their low frequency and burden of data collection. Due to system changes in our cost accounting systems, we were not able to obtain cost data pre- and postimplementation that provided meaningful comparisons.

Patient covariates included age, gender, payer, and Elixhauser comorbidity index as modified by van Walraven,¹⁶ calculated based on the admission of interest and the previous 365 days. We also examined admission characteristics including location (ICU vs. non-ICU), admitting service (medicine, surgery, neurology, or gynecology), and shift of arrival (day, 7:00 am to 6:00 pm; or night, 6:00 pm to 7:00 pm). Finally, we examined duration of the transfer process (ie, time between transfer request and arrival at VUH) and the volume of the transferring hospital (high was defined as 3 or more transfers to VUH per year). 

Statistical analysis

Patient characteristics before and after implementation of the handover tool were compared using Pearson’s chi-square test and Fisher exact test for categorical variables and using Student t test and the Wilcoxon rank sum test for continuous variables. The outcome variables of time to admission order entry, time to antibiotic order entry, LOS, ICU LOS, and in-hospital mortality were compared between the before- and after-intervention time periods, using the Wilcoxon rank sum test for continuous outcomes and Pearson’s chi-square  test for in-hospital mortality. 

To account for temporal trends, the effect of the handover tool on time-to-admission order entry, hospital LOS, and mortality was measured using an interrupted time-series design with segmented linear regression analysis.¹⁷ The study period was divided into 2-week intervals, with 26 time periods in the pre-intervention period and 13 time periods in the postintervention period. Expected rates for the postintervention time periods were projected from the pre-intervention data using a linear regression model. To assess the observed effect of the intervention, rates from the postintervention periods were compared with these projected rates, assuming continuation of the trend. Restricted cubic spline models were also fit for time-to-admission order and hospital LOS; however, the F-statistics for these models were not significant, suggesting the linear regression provided a more appropriate model.

To further account for potential confounding of outcomes by comorbid disease and other patient factors, multivariate linear regression models assessed change in timeliness and LOS with implementation of the intervention. A multivariate logistic regression model was used to assess change in mortality with intervention implementation. All models adjusted for age, gender, payer, comorbid illness, admitting team, shift of arrival (day vs. night), transfer duration, volume of transferring hospital, and ICU status. Outcomes were further adjusted for calendar month to account for temporal trends in house staff efficiency. Because the cardiac transfer center did not adopt the use of the transfer tool, we evaluated adjusted in-hospital mortality for these patients as a concurrent control group. 

All statistical testing was 2-sided at a significance level of 0.05. All analyses were conducted using STATA 12.1 statistical software (StataCorp LP, College Station, Texas).

Of 10,325 patients for whom transfer to VUH was requested during the study period, 1715 met inclusion criteria, including 798 patients (46.5%) initially admitted to an ICU setting. Specific patient exclusions are detailed in the Supplemental Figure; the majority of exclusions were due to patients being transferred directly to the emergency department setting. Table 1 summarizes patient characteristics before and after implementation of the handover tool. The median age was 57 years, with 48.6% male patients. Accepting services included medicine (56%), surgery (34%), neurology (9%), and gynecology (1%). The median duration of transfer was 8 hours, and the majority (93%) of patients came from higher volume transferring hospitals. Most (65%) of patients were admitted during night shift. The median modified Elixhauser comorbidity index was 11 (range of possible scores, -19 to 89). A slightly higher proportion of patients admitted postimplementation of the handover tool came from higher volume transferring hospitals; otherwise, there were no significant differences between the pre- and postintervention groups.

Vanderbilt University Hospital received transfers from more than 350 unique facilities in more than 25 U.S. states during the overall study period. During the postintervention period, adherence to the handover process was excellent, with more than 85% of patients having a completed handover tool available in their medical record at the time of transfer. The remaining 15% had either incomplete forms or no form. 

Timeliness of Initial Care

There was no change in either the median time-to-admission order entry after implementation (47 vs. 45 minutes, unadjusted P = 0.36) or time to antibiotic order entry (199 vs. 202 minutes; unadjusted P = 0.81; Table 2). 

In the time-series analysis, the pre-intervention period did not have a significant temporal trend in median time-to-admission order entry (ß-coefficient = -0.27; 95% confidence interval [CI] -0.85 to 0.31; R2 = 0.04; P = 0.34; Figure 1A). The postintervention period showed a trend toward a reduction in median time-to-admission order entry (ß-coefficient = -1.39; 95% CI -2.92 to 0.15; R2 = 0.27; P = 0.07). There was no significant difference between the actual time-to-admission order entry in the postintervention period when compared to the projected rates from the pre-intervention period (P = 0.18). 

After multivariate adjustment, the postintervention time period was not associated with any significant change in the median time-to-admission order entry (P = 0.94, R² = 0.09) nor time-to-antibiotic order entry (P = 0.91; R² = 0.08; Table 2). 

Length of Stay

Hospital LOS demonstrated a nonstatistically significant decline after implementation of the handover tool from 6.47 days to 5.81 days (unadjusted P = 0.18; Table 2). There was no significant change in ICU LOS postintervention (4.34 days to 4.55 days; P = 0.38).  

In time series analysis, hospital LOS did not have a significant temporal trend in either the pre-intervention period (ß-coefficient = 0.00094; 95% CI, -0.07 to 0.07; R2 = 0.00; P = 0.98) or the postintervention period (ß-coefficient = 0.09; 95% CI, -0.07 to 0.25; R2 = 0.13; P = 0.23; Figure 1B). Similarly, there was no significant difference between the actual and projected hospital LOS after implementation of the handover tool (P = 0.31).

After multivariate adjustment, the postintervention time period was associated with a trend toward reduction in overall LOS (P = 0.06; R² = 0.07) but no significant change in ICU LOS (P = 0.99; R² = 0.09). 

Mortality

In-hospital mortality declined significantly from 12.0% in the pre-intervention period to 8.9% in the postintervention period (P = 0.04; Table 2). In time-series analysis, mortality did not have a significant trend in the pre-intervention period (ß-coefficient = 0.00017, 95% CI, -0.0020 to 0.0024; P = 0.878) and had a trend toward reduction in the postintervention period (ß-coefficient = -0.0032; 95% CI, -0.0091 to 0.0027; P = 0.255; Figure 1C) but did not reach statistical significance due to relatively small numbers of deaths in each individual time period.

After multivariate adjustment, the postintervention period was associated with overall lower odds of mortality among transfer patients when compared with the pre-intervention period (adjusted OR 0.68; 95% CI, 0.47 – 0.99; R² = 0.21; P = 0.04; Figure 2). Among the concurrent control group of patients routed through the cardiac transfer center, there was no significant change in mortality between the pre- and postintervention periods (adjusted OR 1.31; 95% CI, 0.88 – 1.93; R² = 0.28; P = 0.18). 

We developed a simple 1-page handover tool for interhospital transfer patients and aimed to improve timeliness, efficiency, and outcomes of care at the receiving hospital. Implementation of the handover tool was feasible and well accepted by transferring physicians despite a geographically large and diverse transfer network. Although implementation did not substantially improve measures of the timeliness of initial care among transfer patients, we noted a nonsignificant trend toward reduced LOS postintervention.

We observed a substantial and statistically significant reduction in mortality among transfer patients after implementation of the handover tool that persisted after controlling for time trends, comorbid illness, and several other patient factors. This effect was not seen in a concurrent control group of cardiac transfer patients for whom the handover tool was not implemented. Standardizing communication regarding high-risk clinical care processes may be responsible for the observed mortality reduction, similar to improvements seen in other small pilot studies.¹⁸ We acknowledge that the magnitude of the improvement in mortality is more than might be expected from the handover tool alone and could be due to chance. 

In this initial evaluation, it was not feasible to determine whether information provided in the handover tool helped avert specific complications that could affect mortality, such as complications related to the use of ventilators, high-risk medications, or indwelling devices. Assessment of additional patient safety indices such as code events, unplanned ICU transfers, and medication errors could also help clarify the effect of the handover tool on patient-safety outcomes, and future work should include these metrics as well. Alternately, the improvement in mortality may result from other unmeasured processes that occurred concurrently and verification of this finding should be completed in other settings.

More work is needed to determine suitable process and outcome measures for interhospital transfers. Most literature has focused on cost and LOS at the exclusion of more proximal measures of initial care.^3-7 The Institute of Medicine has identified timeliness as 1 of the 6 aims for care delivery redesign,¹⁹ yet standardized timeliness outcomes do not exist across broad inpatient populations. We chose to monitor the time-to-admission order entry and time-to-antibiotic order entry as 2 indicators of timeliness that would be applicable to a variety of patients. The lack of change in these selected measures should prompt examination for other measures of efficiency, including those that affect nontransferred patients. It is possible that nontransferred patients cared for by the same physician also benefit from fewer delays or disruptions and experience increased efficiency of care if transfer patient communication is improved. Further work is necessary to understand whether other measures of timely initial patient care may be more suitable.

The use of a time-series design to account for temporal trends adds substantial rigor to this study, since the majority of these patients were cared for by house staff whose experience and efficiency vary throughout the academic year. Furthermore, subsequent multivariate analysis demonstrated consistent findings after adjustment for comorbid illness and several other hospital and patient-level confounders. 

This study has several limitations. The primary limitation is its nonrandomized design. Patient characteristics were stable across multiple variables before and after implementation, but it is possible that another confounding factor was responsible for observed improvements. Likewise, we collected data for only 6 rather than 12 months during the postintervention time period, which limited our sample size and statistical power. This was chosen because a significant restructuring of resident duty hours occurred in spring 2011 that had the potential to affect all measures studied.^20,21 Finally, we did not collect data on accuracy of the information provided in the handover tool or end-user utilization and were unable to account for effects of these.  

Since implementation in 2010, this process for interhospital transfers at VUH remains the same, although the volume of incoming transfers has significantly increased. Electronic handover tools with similar structure and content have since been adopted for patients being transferred to the emergency department or directly admitted from clinic. As VUH moves in the coming years from a locally developed electronic medical record to a national vendor, there will be an opportunity to transform this tool into an electronic template that will easily share data between institutions and further enhance communication.

Interhospital transfer patients represent a high-risk population whose unique handover needs have not been adequately measured or addressed. Our investigation demonstrated that a standardized handover aid can be implemented across a broad transfer network and may contribute to reductions in LOS and mortality. Further study is warranted to confirm these findings and assess the effect on other clinical outcomes.

Disclosures

This material is based upon work supported by the Office of Academic Affiliations, Department of Veterans Affairs, VA National Quality Scholars Program, and was made possible by the use of the facilities at VA Tennessee Valley Healthcare System, Nashville, Tennessee. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the US government. Additionally, this publication was supported in part by CTSA award No. UL1TR000445 from the National Center for Advancing Translational Sciences. Its contents are solely the responsibility of the authors and do not necessarily represent official views of the National Center for Advancing Translational Sciences or the National Institutes of Health.  

The transfer of inpatients between hospitals for specialized services is common, affecting nearly 10% of all Medicare admissions¹ and 4.5% of all critical care hospitalizations.² At tertiary referral centers, 49% of medical intensive care unit (ICU) admissions are transferred from another hospital.³

Transfer patients have longer length of stay (LOS) than patients admitted directly from the emergency department or clinic. Among patients initially admitted to an ICU, transfer patients spend 1 day to 2.2 more days in the ICU and an additional 2 days to 4 more days total at the receiving hospital.^4,5 Furthermore, transfer patients have higher mortality than nontransferred patients by 4% to 8%.^3-5 Even after adjustment for case mix and comorbid disease, interhospital transfer is an independent predictor of both ICU death and LOS.^6,7 As a result, interhospital transfer has been associated with a $9600 increase (on average) in hospital costs.⁴

Despite evidence detailing patient handovers as a key time when poor communication can lead to delays in care and significant patient risk, ^8-10 most studies have focused on hospital discharge or change of shift, and scant effort has been dedicated to improving the interhospital handover. The process of interhospital transfer is often prolonged and discontinuous,¹¹ commonly including delays of more than 24 hours between initiation and completion. This frequently precludes direct physician-to-physician contact at the time of transfer, and physicians rely on the discharge/transfer summary.¹² Yet discharge summaries are frequently absent or incomplete,¹³ and often lack information for high-risk treatments such as systemic anticoagulation.¹⁴ The traditional reliance on discharge summaries for handover communication requires interpretation of unstandardized documentation and increases the risk for miscommunication, delays, and error.

To improve communication, we developed a 1-page handover tool for all inbound adult interhospital transfers to our academic medical center. We sought to determine whether implementation of this standardized handover tool improved the timeliness of initial care, LOS, and mortality among interhospital transfer patients. 

Study Design, Setting, Population

We conducted a retrospective cohort study of patients transferred into Vanderbilt University Hospital (VUH), an adult 626-bed quaternary care academic medical center in Nashville, Tennessee. The Vanderbilt University Institutional Review Board approved this study.  

Population

We selected for inclusion all patients age 18 or older who were transferred into VUH between July 1, 2009 and December 31, 2010. We excluded patients whose transfer was routed outside the main VUH Patient Flow Center as well as patients who did not arrive alive at VUH. We also excluded patients transferred to the emergency department and patients admitted to obstetrics, burn, or trauma services, because these admitting services did not initially use the handover tool. Patients were followed for the duration of their hospitalization at VUH.

Intervention

The 1-page handover tool was developed with multidisciplinary physician input from house staff; medical directors from intensive care, neurology, and surgery; and the chief of staff. The tool was structured on the SBAR model (Situation, Background, Assessment, and Recommendation).¹⁵ Fields on the handover tool were limited to those deemed critical for immediate patient care and designed for 1 tool to be used for both ICU and non-ICU transfers. Fields included primary diagnosis; allergies; use and last dose of anticoagulants, vasopressors, sedative/paralytics, and antibiotics; isolation needs; indwelling devices; recent operations/procedures; code status; emergency contact information; problem list; active medication list; vital signs; pertinent exam; imaging; lab findings; and overall reason for transfer. 

The handover tool was completed by the physician at the transferring hospital, faxed to VUH, and immediately scanned into the electronic record, allowing the receiving physicians to review information before patient arrival. Use of the tool was piloted first with 2 referring hospitals in April 2010 and universally recommended but not compulsory for all adult patients routed through the main VUH Patient Flow Center beginning July 1, 2010. Immediately before full implementation, the chief of staff sent letters to leadership of the 40 highest volume referral hospitals, highlighting the institutional goal of improving hand-off communication, framing completion of the tool as a step in the transfer acceptance process, and providing contact information for questions, feedback, or concerns. To ensure the tool was a standard part of the transfer process, the VUH Patient Flow Center maintained the responsibility of faxing the form to the outside facility and monitoring its receipt. The tool was processed in the same manner as other faxed patient records and treated as a part of the formal medical record to meet all standards for the Health Insurance Portability and Accountability Act (HIPAA) and medicolegal compliance. The medical center also has a separate cardiac transfer center where the handover tool was not implemented owing to its specialized workflow.

Data Source

The VUH Patient Flow Center maintains a database of all patients for whom transfer to VUH is requested, including information on the requesting hospital and the duration of transfer process. Outcome data and patient characteristics were extracted from the Enterprise Data Warehouse. Data related to comorbid illness were extracted from the Perioperative Data Warehouse, an IRB-approved data registry.

Measures

We evaluated 3 outcomes. First, we defined 2 measures of the timeliness of initial care, the time from arrival at VUH until entry of an admission order, and the time from arrival until entry of the first antibiotic order. Only antibiotics ordered within the first 36 hours of admission were included. Second, we evaluated the total LOS after transfer to VUH and the ICU LOS for patients transferred into an ICU setting. Finally, we examined in-hospital mortality at VUH. These metrics were chosen for their broad applicability across patient groups and feasibility of data capture. Length of stay and mortality also represent final common pathways for avoidance of complications. Specific patient safety indicators and complications were not abstracted due to their low frequency and burden of data collection. Due to system changes in our cost accounting systems, we were not able to obtain cost data pre- and postimplementation that provided meaningful comparisons.

Patient covariates included age, gender, payer, and Elixhauser comorbidity index as modified by van Walraven,¹⁶ calculated based on the admission of interest and the previous 365 days. We also examined admission characteristics including location (ICU vs. non-ICU), admitting service (medicine, surgery, neurology, or gynecology), and shift of arrival (day, 7:00 am to 6:00 pm; or night, 6:00 pm to 7:00 pm). Finally, we examined duration of the transfer process (ie, time between transfer request and arrival at VUH) and the volume of the transferring hospital (high was defined as 3 or more transfers to VUH per year). 

Statistical analysis

Patient characteristics before and after implementation of the handover tool were compared using Pearson’s chi-square test and Fisher exact test for categorical variables and using Student t test and the Wilcoxon rank sum test for continuous variables. The outcome variables of time to admission order entry, time to antibiotic order entry, LOS, ICU LOS, and in-hospital mortality were compared between the before- and after-intervention time periods, using the Wilcoxon rank sum test for continuous outcomes and Pearson’s chi-square  test for in-hospital mortality. 

To account for temporal trends, the effect of the handover tool on time-to-admission order entry, hospital LOS, and mortality was measured using an interrupted time-series design with segmented linear regression analysis.¹⁷ The study period was divided into 2-week intervals, with 26 time periods in the pre-intervention period and 13 time periods in the postintervention period. Expected rates for the postintervention time periods were projected from the pre-intervention data using a linear regression model. To assess the observed effect of the intervention, rates from the postintervention periods were compared with these projected rates, assuming continuation of the trend. Restricted cubic spline models were also fit for time-to-admission order and hospital LOS; however, the F-statistics for these models were not significant, suggesting the linear regression provided a more appropriate model.

To further account for potential confounding of outcomes by comorbid disease and other patient factors, multivariate linear regression models assessed change in timeliness and LOS with implementation of the intervention. A multivariate logistic regression model was used to assess change in mortality with intervention implementation. All models adjusted for age, gender, payer, comorbid illness, admitting team, shift of arrival (day vs. night), transfer duration, volume of transferring hospital, and ICU status. Outcomes were further adjusted for calendar month to account for temporal trends in house staff efficiency. Because the cardiac transfer center did not adopt the use of the transfer tool, we evaluated adjusted in-hospital mortality for these patients as a concurrent control group. 

All statistical testing was 2-sided at a significance level of 0.05. All analyses were conducted using STATA 12.1 statistical software (StataCorp LP, College Station, Texas).

Of 10,325 patients for whom transfer to VUH was requested during the study period, 1715 met inclusion criteria, including 798 patients (46.5%) initially admitted to an ICU setting. Specific patient exclusions are detailed in the Supplemental Figure; the majority of exclusions were due to patients being transferred directly to the emergency department setting. Table 1 summarizes patient characteristics before and after implementation of the handover tool. The median age was 57 years, with 48.6% male patients. Accepting services included medicine (56%), surgery (34%), neurology (9%), and gynecology (1%). The median duration of transfer was 8 hours, and the majority (93%) of patients came from higher volume transferring hospitals. Most (65%) of patients were admitted during night shift. The median modified Elixhauser comorbidity index was 11 (range of possible scores, -19 to 89). A slightly higher proportion of patients admitted postimplementation of the handover tool came from higher volume transferring hospitals; otherwise, there were no significant differences between the pre- and postintervention groups.

Vanderbilt University Hospital received transfers from more than 350 unique facilities in more than 25 U.S. states during the overall study period. During the postintervention period, adherence to the handover process was excellent, with more than 85% of patients having a completed handover tool available in their medical record at the time of transfer. The remaining 15% had either incomplete forms or no form. 

Timeliness of Initial Care

There was no change in either the median time-to-admission order entry after implementation (47 vs. 45 minutes, unadjusted P = 0.36) or time to antibiotic order entry (199 vs. 202 minutes; unadjusted P = 0.81; Table 2). 

In the time-series analysis, the pre-intervention period did not have a significant temporal trend in median time-to-admission order entry (ß-coefficient = -0.27; 95% confidence interval [CI] -0.85 to 0.31; R2 = 0.04; P = 0.34; Figure 1A). The postintervention period showed a trend toward a reduction in median time-to-admission order entry (ß-coefficient = -1.39; 95% CI -2.92 to 0.15; R2 = 0.27; P = 0.07). There was no significant difference between the actual time-to-admission order entry in the postintervention period when compared to the projected rates from the pre-intervention period (P = 0.18). 

After multivariate adjustment, the postintervention time period was not associated with any significant change in the median time-to-admission order entry (P = 0.94, R² = 0.09) nor time-to-antibiotic order entry (P = 0.91; R² = 0.08; Table 2). 

Length of Stay

Hospital LOS demonstrated a nonstatistically significant decline after implementation of the handover tool from 6.47 days to 5.81 days (unadjusted P = 0.18; Table 2). There was no significant change in ICU LOS postintervention (4.34 days to 4.55 days; P = 0.38).  

In time series analysis, hospital LOS did not have a significant temporal trend in either the pre-intervention period (ß-coefficient = 0.00094; 95% CI, -0.07 to 0.07; R2 = 0.00; P = 0.98) or the postintervention period (ß-coefficient = 0.09; 95% CI, -0.07 to 0.25; R2 = 0.13; P = 0.23; Figure 1B). Similarly, there was no significant difference between the actual and projected hospital LOS after implementation of the handover tool (P = 0.31).

After multivariate adjustment, the postintervention time period was associated with a trend toward reduction in overall LOS (P = 0.06; R² = 0.07) but no significant change in ICU LOS (P = 0.99; R² = 0.09). 

Mortality

In-hospital mortality declined significantly from 12.0% in the pre-intervention period to 8.9% in the postintervention period (P = 0.04; Table 2). In time-series analysis, mortality did not have a significant trend in the pre-intervention period (ß-coefficient = 0.00017, 95% CI, -0.0020 to 0.0024; P = 0.878) and had a trend toward reduction in the postintervention period (ß-coefficient = -0.0032; 95% CI, -0.0091 to 0.0027; P = 0.255; Figure 1C) but did not reach statistical significance due to relatively small numbers of deaths in each individual time period.

After multivariate adjustment, the postintervention period was associated with overall lower odds of mortality among transfer patients when compared with the pre-intervention period (adjusted OR 0.68; 95% CI, 0.47 – 0.99; R² = 0.21; P = 0.04; Figure 2). Among the concurrent control group of patients routed through the cardiac transfer center, there was no significant change in mortality between the pre- and postintervention periods (adjusted OR 1.31; 95% CI, 0.88 – 1.93; R² = 0.28; P = 0.18). 

We developed a simple 1-page handover tool for interhospital transfer patients and aimed to improve timeliness, efficiency, and outcomes of care at the receiving hospital. Implementation of the handover tool was feasible and well accepted by transferring physicians despite a geographically large and diverse transfer network. Although implementation did not substantially improve measures of the timeliness of initial care among transfer patients, we noted a nonsignificant trend toward reduced LOS postintervention.

We observed a substantial and statistically significant reduction in mortality among transfer patients after implementation of the handover tool that persisted after controlling for time trends, comorbid illness, and several other patient factors. This effect was not seen in a concurrent control group of cardiac transfer patients for whom the handover tool was not implemented. Standardizing communication regarding high-risk clinical care processes may be responsible for the observed mortality reduction, similar to improvements seen in other small pilot studies.¹⁸ We acknowledge that the magnitude of the improvement in mortality is more than might be expected from the handover tool alone and could be due to chance. 

In this initial evaluation, it was not feasible to determine whether information provided in the handover tool helped avert specific complications that could affect mortality, such as complications related to the use of ventilators, high-risk medications, or indwelling devices. Assessment of additional patient safety indices such as code events, unplanned ICU transfers, and medication errors could also help clarify the effect of the handover tool on patient-safety outcomes, and future work should include these metrics as well. Alternately, the improvement in mortality may result from other unmeasured processes that occurred concurrently and verification of this finding should be completed in other settings.

More work is needed to determine suitable process and outcome measures for interhospital transfers. Most literature has focused on cost and LOS at the exclusion of more proximal measures of initial care.^3-7 The Institute of Medicine has identified timeliness as 1 of the 6 aims for care delivery redesign,¹⁹ yet standardized timeliness outcomes do not exist across broad inpatient populations. We chose to monitor the time-to-admission order entry and time-to-antibiotic order entry as 2 indicators of timeliness that would be applicable to a variety of patients. The lack of change in these selected measures should prompt examination for other measures of efficiency, including those that affect nontransferred patients. It is possible that nontransferred patients cared for by the same physician also benefit from fewer delays or disruptions and experience increased efficiency of care if transfer patient communication is improved. Further work is necessary to understand whether other measures of timely initial patient care may be more suitable.

The use of a time-series design to account for temporal trends adds substantial rigor to this study, since the majority of these patients were cared for by house staff whose experience and efficiency vary throughout the academic year. Furthermore, subsequent multivariate analysis demonstrated consistent findings after adjustment for comorbid illness and several other hospital and patient-level confounders. 

This study has several limitations. The primary limitation is its nonrandomized design. Patient characteristics were stable across multiple variables before and after implementation, but it is possible that another confounding factor was responsible for observed improvements. Likewise, we collected data for only 6 rather than 12 months during the postintervention time period, which limited our sample size and statistical power. This was chosen because a significant restructuring of resident duty hours occurred in spring 2011 that had the potential to affect all measures studied.^20,21 Finally, we did not collect data on accuracy of the information provided in the handover tool or end-user utilization and were unable to account for effects of these.  

Since implementation in 2010, this process for interhospital transfers at VUH remains the same, although the volume of incoming transfers has significantly increased. Electronic handover tools with similar structure and content have since been adopted for patients being transferred to the emergency department or directly admitted from clinic. As VUH moves in the coming years from a locally developed electronic medical record to a national vendor, there will be an opportunity to transform this tool into an electronic template that will easily share data between institutions and further enhance communication.

Interhospital transfer patients represent a high-risk population whose unique handover needs have not been adequately measured or addressed. Our investigation demonstrated that a standardized handover aid can be implemented across a broad transfer network and may contribute to reductions in LOS and mortality. Further study is warranted to confirm these findings and assess the effect on other clinical outcomes.

Disclosures

This material is based upon work supported by the Office of Academic Affiliations, Department of Veterans Affairs, VA National Quality Scholars Program, and was made possible by the use of the facilities at VA Tennessee Valley Healthcare System, Nashville, Tennessee. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the US government. Additionally, this publication was supported in part by CTSA award No. UL1TR000445 from the National Center for Advancing Translational Sciences. Its contents are solely the responsibility of the authors and do not necessarily represent official views of the National Center for Advancing Translational Sciences or the National Institutes of Health.  

The transfer of inpatients between hospitals for specialized services is common, affecting nearly 10% of all Medicare admissions¹ and 4.5% of all critical care hospitalizations.² At tertiary referral centers, 49% of medical intensive care unit (ICU) admissions are transferred from another hospital.³

Transfer patients have longer length of stay (LOS) than patients admitted directly from the emergency department or clinic. Among patients initially admitted to an ICU, transfer patients spend 1 day to 2.2 more days in the ICU and an additional 2 days to 4 more days total at the receiving hospital.^4,5 Furthermore, transfer patients have higher mortality than nontransferred patients by 4% to 8%.^3-5 Even after adjustment for case mix and comorbid disease, interhospital transfer is an independent predictor of both ICU death and LOS.^6,7 As a result, interhospital transfer has been associated with a $9600 increase (on average) in hospital costs.⁴

Despite evidence detailing patient handovers as a key time when poor communication can lead to delays in care and significant patient risk, ^8-10 most studies have focused on hospital discharge or change of shift, and scant effort has been dedicated to improving the interhospital handover. The process of interhospital transfer is often prolonged and discontinuous,¹¹ commonly including delays of more than 24 hours between initiation and completion. This frequently precludes direct physician-to-physician contact at the time of transfer, and physicians rely on the discharge/transfer summary.¹² Yet discharge summaries are frequently absent or incomplete,¹³ and often lack information for high-risk treatments such as systemic anticoagulation.¹⁴ The traditional reliance on discharge summaries for handover communication requires interpretation of unstandardized documentation and increases the risk for miscommunication, delays, and error.

To improve communication, we developed a 1-page handover tool for all inbound adult interhospital transfers to our academic medical center. We sought to determine whether implementation of this standardized handover tool improved the timeliness of initial care, LOS, and mortality among interhospital transfer patients. 

Study Design, Setting, Population

We conducted a retrospective cohort study of patients transferred into Vanderbilt University Hospital (VUH), an adult 626-bed quaternary care academic medical center in Nashville, Tennessee. The Vanderbilt University Institutional Review Board approved this study.  

Population

We selected for inclusion all patients age 18 or older who were transferred into VUH between July 1, 2009 and December 31, 2010. We excluded patients whose transfer was routed outside the main VUH Patient Flow Center as well as patients who did not arrive alive at VUH. We also excluded patients transferred to the emergency department and patients admitted to obstetrics, burn, or trauma services, because these admitting services did not initially use the handover tool. Patients were followed for the duration of their hospitalization at VUH.

Intervention

The 1-page handover tool was developed with multidisciplinary physician input from house staff; medical directors from intensive care, neurology, and surgery; and the chief of staff. The tool was structured on the SBAR model (Situation, Background, Assessment, and Recommendation).¹⁵ Fields on the handover tool were limited to those deemed critical for immediate patient care and designed for 1 tool to be used for both ICU and non-ICU transfers. Fields included primary diagnosis; allergies; use and last dose of anticoagulants, vasopressors, sedative/paralytics, and antibiotics; isolation needs; indwelling devices; recent operations/procedures; code status; emergency contact information; problem list; active medication list; vital signs; pertinent exam; imaging; lab findings; and overall reason for transfer. 

The handover tool was completed by the physician at the transferring hospital, faxed to VUH, and immediately scanned into the electronic record, allowing the receiving physicians to review information before patient arrival. Use of the tool was piloted first with 2 referring hospitals in April 2010 and universally recommended but not compulsory for all adult patients routed through the main VUH Patient Flow Center beginning July 1, 2010. Immediately before full implementation, the chief of staff sent letters to leadership of the 40 highest volume referral hospitals, highlighting the institutional goal of improving hand-off communication, framing completion of the tool as a step in the transfer acceptance process, and providing contact information for questions, feedback, or concerns. To ensure the tool was a standard part of the transfer process, the VUH Patient Flow Center maintained the responsibility of faxing the form to the outside facility and monitoring its receipt. The tool was processed in the same manner as other faxed patient records and treated as a part of the formal medical record to meet all standards for the Health Insurance Portability and Accountability Act (HIPAA) and medicolegal compliance. The medical center also has a separate cardiac transfer center where the handover tool was not implemented owing to its specialized workflow.

Data Source

The VUH Patient Flow Center maintains a database of all patients for whom transfer to VUH is requested, including information on the requesting hospital and the duration of transfer process. Outcome data and patient characteristics were extracted from the Enterprise Data Warehouse. Data related to comorbid illness were extracted from the Perioperative Data Warehouse, an IRB-approved data registry.

Measures

We evaluated 3 outcomes. First, we defined 2 measures of the timeliness of initial care, the time from arrival at VUH until entry of an admission order, and the time from arrival until entry of the first antibiotic order. Only antibiotics ordered within the first 36 hours of admission were included. Second, we evaluated the total LOS after transfer to VUH and the ICU LOS for patients transferred into an ICU setting. Finally, we examined in-hospital mortality at VUH. These metrics were chosen for their broad applicability across patient groups and feasibility of data capture. Length of stay and mortality also represent final common pathways for avoidance of complications. Specific patient safety indicators and complications were not abstracted due to their low frequency and burden of data collection. Due to system changes in our cost accounting systems, we were not able to obtain cost data pre- and postimplementation that provided meaningful comparisons.

Patient covariates included age, gender, payer, and Elixhauser comorbidity index as modified by van Walraven,¹⁶ calculated based on the admission of interest and the previous 365 days. We also examined admission characteristics including location (ICU vs. non-ICU), admitting service (medicine, surgery, neurology, or gynecology), and shift of arrival (day, 7:00 am to 6:00 pm; or night, 6:00 pm to 7:00 pm). Finally, we examined duration of the transfer process (ie, time between transfer request and arrival at VUH) and the volume of the transferring hospital (high was defined as 3 or more transfers to VUH per year). 

Statistical analysis

Patient characteristics before and after implementation of the handover tool were compared using Pearson’s chi-square test and Fisher exact test for categorical variables and using Student t test and the Wilcoxon rank sum test for continuous variables. The outcome variables of time to admission order entry, time to antibiotic order entry, LOS, ICU LOS, and in-hospital mortality were compared between the before- and after-intervention time periods, using the Wilcoxon rank sum test for continuous outcomes and Pearson’s chi-square  test for in-hospital mortality. 

To account for temporal trends, the effect of the handover tool on time-to-admission order entry, hospital LOS, and mortality was measured using an interrupted time-series design with segmented linear regression analysis.¹⁷ The study period was divided into 2-week intervals, with 26 time periods in the pre-intervention period and 13 time periods in the postintervention period. Expected rates for the postintervention time periods were projected from the pre-intervention data using a linear regression model. To assess the observed effect of the intervention, rates from the postintervention periods were compared with these projected rates, assuming continuation of the trend. Restricted cubic spline models were also fit for time-to-admission order and hospital LOS; however, the F-statistics for these models were not significant, suggesting the linear regression provided a more appropriate model.

To further account for potential confounding of outcomes by comorbid disease and other patient factors, multivariate linear regression models assessed change in timeliness and LOS with implementation of the intervention. A multivariate logistic regression model was used to assess change in mortality with intervention implementation. All models adjusted for age, gender, payer, comorbid illness, admitting team, shift of arrival (day vs. night), transfer duration, volume of transferring hospital, and ICU status. Outcomes were further adjusted for calendar month to account for temporal trends in house staff efficiency. Because the cardiac transfer center did not adopt the use of the transfer tool, we evaluated adjusted in-hospital mortality for these patients as a concurrent control group. 

All statistical testing was 2-sided at a significance level of 0.05. All analyses were conducted using STATA 12.1 statistical software (StataCorp LP, College Station, Texas).

Of 10,325 patients for whom transfer to VUH was requested during the study period, 1715 met inclusion criteria, including 798 patients (46.5%) initially admitted to an ICU setting. Specific patient exclusions are detailed in the Supplemental Figure; the majority of exclusions were due to patients being transferred directly to the emergency department setting. Table 1 summarizes patient characteristics before and after implementation of the handover tool. The median age was 57 years, with 48.6% male patients. Accepting services included medicine (56%), surgery (34%), neurology (9%), and gynecology (1%). The median duration of transfer was 8 hours, and the majority (93%) of patients came from higher volume transferring hospitals. Most (65%) of patients were admitted during night shift. The median modified Elixhauser comorbidity index was 11 (range of possible scores, -19 to 89). A slightly higher proportion of patients admitted postimplementation of the handover tool came from higher volume transferring hospitals; otherwise, there were no significant differences between the pre- and postintervention groups.

Vanderbilt University Hospital received transfers from more than 350 unique facilities in more than 25 U.S. states during the overall study period. During the postintervention period, adherence to the handover process was excellent, with more than 85% of patients having a completed handover tool available in their medical record at the time of transfer. The remaining 15% had either incomplete forms or no form. 

Timeliness of Initial Care

There was no change in either the median time-to-admission order entry after implementation (47 vs. 45 minutes, unadjusted P = 0.36) or time to antibiotic order entry (199 vs. 202 minutes; unadjusted P = 0.81; Table 2). 

In the time-series analysis, the pre-intervention period did not have a significant temporal trend in median time-to-admission order entry (ß-coefficient = -0.27; 95% confidence interval [CI] -0.85 to 0.31; R2 = 0.04; P = 0.34; Figure 1A). The postintervention period showed a trend toward a reduction in median time-to-admission order entry (ß-coefficient = -1.39; 95% CI -2.92 to 0.15; R2 = 0.27; P = 0.07). There was no significant difference between the actual time-to-admission order entry in the postintervention period when compared to the projected rates from the pre-intervention period (P = 0.18). 

After multivariate adjustment, the postintervention time period was not associated with any significant change in the median time-to-admission order entry (P = 0.94, R² = 0.09) nor time-to-antibiotic order entry (P = 0.91; R² = 0.08; Table 2). 

Length of Stay

Hospital LOS demonstrated a nonstatistically significant decline after implementation of the handover tool from 6.47 days to 5.81 days (unadjusted P = 0.18; Table 2). There was no significant change in ICU LOS postintervention (4.34 days to 4.55 days; P = 0.38).  

In time series analysis, hospital LOS did not have a significant temporal trend in either the pre-intervention period (ß-coefficient = 0.00094; 95% CI, -0.07 to 0.07; R2 = 0.00; P = 0.98) or the postintervention period (ß-coefficient = 0.09; 95% CI, -0.07 to 0.25; R2 = 0.13; P = 0.23; Figure 1B). Similarly, there was no significant difference between the actual and projected hospital LOS after implementation of the handover tool (P = 0.31).

After multivariate adjustment, the postintervention time period was associated with a trend toward reduction in overall LOS (P = 0.06; R² = 0.07) but no significant change in ICU LOS (P = 0.99; R² = 0.09). 

Mortality

In-hospital mortality declined significantly from 12.0% in the pre-intervention period to 8.9% in the postintervention period (P = 0.04; Table 2). In time-series analysis, mortality did not have a significant trend in the pre-intervention period (ß-coefficient = 0.00017, 95% CI, -0.0020 to 0.0024; P = 0.878) and had a trend toward reduction in the postintervention period (ß-coefficient = -0.0032; 95% CI, -0.0091 to 0.0027; P = 0.255; Figure 1C) but did not reach statistical significance due to relatively small numbers of deaths in each individual time period.

After multivariate adjustment, the postintervention period was associated with overall lower odds of mortality among transfer patients when compared with the pre-intervention period (adjusted OR 0.68; 95% CI, 0.47 – 0.99; R² = 0.21; P = 0.04; Figure 2). Among the concurrent control group of patients routed through the cardiac transfer center, there was no significant change in mortality between the pre- and postintervention periods (adjusted OR 1.31; 95% CI, 0.88 – 1.93; R² = 0.28; P = 0.18). 

We developed a simple 1-page handover tool for interhospital transfer patients and aimed to improve timeliness, efficiency, and outcomes of care at the receiving hospital. Implementation of the handover tool was feasible and well accepted by transferring physicians despite a geographically large and diverse transfer network. Although implementation did not substantially improve measures of the timeliness of initial care among transfer patients, we noted a nonsignificant trend toward reduced LOS postintervention.

We observed a substantial and statistically significant reduction in mortality among transfer patients after implementation of the handover tool that persisted after controlling for time trends, comorbid illness, and several other patient factors. This effect was not seen in a concurrent control group of cardiac transfer patients for whom the handover tool was not implemented. Standardizing communication regarding high-risk clinical care processes may be responsible for the observed mortality reduction, similar to improvements seen in other small pilot studies.¹⁸ We acknowledge that the magnitude of the improvement in mortality is more than might be expected from the handover tool alone and could be due to chance. 

In this initial evaluation, it was not feasible to determine whether information provided in the handover tool helped avert specific complications that could affect mortality, such as complications related to the use of ventilators, high-risk medications, or indwelling devices. Assessment of additional patient safety indices such as code events, unplanned ICU transfers, and medication errors could also help clarify the effect of the handover tool on patient-safety outcomes, and future work should include these metrics as well. Alternately, the improvement in mortality may result from other unmeasured processes that occurred concurrently and verification of this finding should be completed in other settings.

More work is needed to determine suitable process and outcome measures for interhospital transfers. Most literature has focused on cost and LOS at the exclusion of more proximal measures of initial care.^3-7 The Institute of Medicine has identified timeliness as 1 of the 6 aims for care delivery redesign,¹⁹ yet standardized timeliness outcomes do not exist across broad inpatient populations. We chose to monitor the time-to-admission order entry and time-to-antibiotic order entry as 2 indicators of timeliness that would be applicable to a variety of patients. The lack of change in these selected measures should prompt examination for other measures of efficiency, including those that affect nontransferred patients. It is possible that nontransferred patients cared for by the same physician also benefit from fewer delays or disruptions and experience increased efficiency of care if transfer patient communication is improved. Further work is necessary to understand whether other measures of timely initial patient care may be more suitable.

The use of a time-series design to account for temporal trends adds substantial rigor to this study, since the majority of these patients were cared for by house staff whose experience and efficiency vary throughout the academic year. Furthermore, subsequent multivariate analysis demonstrated consistent findings after adjustment for comorbid illness and several other hospital and patient-level confounders. 

This study has several limitations. The primary limitation is its nonrandomized design. Patient characteristics were stable across multiple variables before and after implementation, but it is possible that another confounding factor was responsible for observed improvements. Likewise, we collected data for only 6 rather than 12 months during the postintervention time period, which limited our sample size and statistical power. This was chosen because a significant restructuring of resident duty hours occurred in spring 2011 that had the potential to affect all measures studied.^20,21 Finally, we did not collect data on accuracy of the information provided in the handover tool or end-user utilization and were unable to account for effects of these.  

Since implementation in 2010, this process for interhospital transfers at VUH remains the same, although the volume of incoming transfers has significantly increased. Electronic handover tools with similar structure and content have since been adopted for patients being transferred to the emergency department or directly admitted from clinic. As VUH moves in the coming years from a locally developed electronic medical record to a national vendor, there will be an opportunity to transform this tool into an electronic template that will easily share data between institutions and further enhance communication.

Interhospital transfer patients represent a high-risk population whose unique handover needs have not been adequately measured or addressed. Our investigation demonstrated that a standardized handover aid can be implemented across a broad transfer network and may contribute to reductions in LOS and mortality. Further study is warranted to confirm these findings and assess the effect on other clinical outcomes.

Disclosures

This material is based upon work supported by the Office of Academic Affiliations, Department of Veterans Affairs, VA National Quality Scholars Program, and was made possible by the use of the facilities at VA Tennessee Valley Healthcare System, Nashville, Tennessee. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the US government. Additionally, this publication was supported in part by CTSA award No. UL1TR000445 from the National Center for Advancing Translational Sciences. Its contents are solely the responsibility of the authors and do not necessarily represent official views of the National Center for Advancing Translational Sciences or the National Institutes of Health.  

Although much has been written about pediatric discharge and readmissions^1-5 over the past several years, surprisingly little is known about which care practices are most effective at preventing postdischarge utilization.⁵ Major collaborations across the U.S. are currently focused on improving pediatric discharge processes,^6-8 although the impact that these efforts will have on readmissions remains to be seen.

Research on handoffs between hospitals and primary care has mixed associations with postdischarge utilization. Although some studies observe positive relationships between specific activities and reduced postdischarge utilization,¹ others suggest no relationship^9-12 or, paradoxically, more utilization.^13,14 Brittan et al¹⁵ found that outpatient visits were associated with more readmissions when occurring less than 4 days after discharge, but fewer readmissions when occurring 4 days to  29 days after discharge. Most studies, however, investigate single or limited sets of care activities, such as having an outpatient visit,¹⁵ timeliness of that visit,¹⁶ or receipt of a discharge summary.¹¹ Inclusion of a more comprehensive set of hospital- to primary-care communication practices may better unravel this complex relationship between discharge care and postdischarge outcomes for children.

The purpose of this study was to characterize a set of traditional discharge handoff practices between hospital and primary care providers (PCPs) and to explore their relationships to readmissions. We hypothesized that handoff practices would be associated with fewer unplanned readmissions.

Study Design, Setting, Participants

This project was part of a prospective cohort study with 2 aims: to investigate relationships between medical home experience and postdischarge utilization,¹⁷ and to identify relationships between common discharge communication practices and postdischarge utilization. This manuscript is focused on the second aim. Randomly selected pediatric patients and their caregivers were enrolled from any medical or surgical service during an acute hospitalization lasting more than 24 hours from October 1, 2012 to January 1, 2014, at a 100-bed tertiary children’s hospital. Patients who transferred to another facility, died, were older than 18 years or in neonatal care (ie, newborn nursery or neonatal intensive care unit) were excluded since their discharge experiences would be significantly distinct from the population of interest. Patients were enrolled once in the study.

Outcome

The study’s primary outcome was 30-day unplanned readmissions, defined as a hospitalization occurring within 30 days of the index (ie, study enrollment) hospitalization, identified through caregiver report or administrative sources.¹⁷ Although the study site is a single hospital system, readmissions could have occurred to any hospital reported by caregivers, (ie, readmissions could have occurred within or outside our health system). Readmissions for chemotherapy, radiation, dialysis, rehabilitation, or labor and delivery were excluded. If caregivers reported an admission as planned or chart review of the index discharge summary noted that a rehospitalization was scheduled in the subsequent 30 days, the readmission was labeled “planned” and excluded. 

Discharge Handoff Communication

Transitional care is a set of actions designed to ensure continuity and coordination of healthcare during transfer from 1 location or level of care to another.^18,19 The study team, comprised of a division chief of general pediatrics, a division chief of hospital medicine, 2 departmental vice-chairs, and the medical director for quality at the study site, identified 11 common handoff activities and reporting sources. These consensus-based activities were expected by the study team to improve continuity and coordination during hospital-to-home transfer, and included:

• verifying PCP identity during the hospitalization (caregiver report); 
• notifying the PCP of admission, discharge, and providing updates during the hospitalization (PCP report); 
• PCP follow-up appointment set prior to discharge (caregiver report); 
• documenting planned PCP and subspecialty follow-up in the discharge summary (chart review); 
• completing the discharge summary within 48 hours (chart review); 
• providing a verbal or written handoff to the PCP prior to follow-up (PCP report); and 
• having a PCP follow-up visit within 30 days of discharge (caregiver report). 

We also asked PCPs whether they thought the follow-up interval was appropriate and whether phone follow-up with the patient would have been as appropriate as a face-to-face visit. 

Covariates

Patient demographics that might confound the relationship between handoff practices and readmissions based on pediatric research^20,21 were included. Medical complexity was accounted for by length-of-index stay, the number of hospitalizations and emergency department (ED) visits in past 12 months, complex chronic conditions,^22,23 and seeing 3 or more subspecialists.^24,25 Variables from related work included PCP scope (general pediatrics or subspecialist) and presence of a usual source for well and sick care.¹⁷

The Care Transitions Measure-3 (CTM-3), originally developed to assess the patient-centeredness of hospital transition,^26,27 can discriminate adult patients at risk for readmission.²⁶ We adapted the original CTM-3 to be answered by caregiver respondents after pilot testing with 5 caregivers not enrolled in the study: 1) “The hospital staff took my preferences and those of my family into account in deciding what my child’s health care needs would be when I left the hospital;” 2) “When I left the hospital, I had a good understanding of the things I was responsible for in managing my child’s health;” and 3) “When I left the hospital, I clearly understood the purpose for giving each of my child’s medications.” We analyzed the adapted CTM-3 on a transformed 0-100 scale as designed,²⁶ initially hypothesizing that the CTM-3 would mediate the relationship between handoff practices and readmissions.

We assessed caregiver confidence to avoid a readmission, based on a strong independent association with readmissions described in Coller et al.¹⁷ Using questions developed for this study, caregivers were asked to rate “How confident are you that [child’s name] will stay out of the hospital for the next 30 days?” with instructions to refer to unplanned hospital visits only. Responses were reported on a 4-point Likert scale (1 = very confident, 4 = not very confident). Responses were dichotomized into very confident (ie, “1”) or not very confident (ie, “2-4”).

Enrollment and Data Collection

Computer-generated random numbers were assigned to patients admitted the previous day, and families were enrolled sequentially until the daily enrollment target was reached. Data were obtained from 3 sources: medical record, caregiver report, and PCP report. Trained research assistants systematically extracted chart review data documenting the transitions practices above, while a hospital information technology analyst extracted claims and demographic data to complement what was reported by parents and PCPs. After study conclusion, these medical record data were merged with caregiver and PCP-reported data.

Trained bilingual research assistants collected caregiver- and PCP-reported data using structured questionnaires in English or Spanish, according to preference. Timing of data collection differed by data source; caregiver-reported data were collected immediately after discharge and at 30 days postdischarge; PCP-reported data were collected at 30 days postdischarge. 

Caregiver-reported data were collected through 2 separate phone calls following index discharge: immediately after discharge (caregiver confidence and CTM-3 measures) and at 30 days (readmission measures). Caregiver confidence questions were asked after (rather than immediately before) discharge to avoid biasing clinical care and revisit risk, consistent with previous work.²⁸ 

PCP-reported data were collected using structured questionnaires with the PCP who was identified by the family during study enrollment. PCP-reported data were collected by telephone or fax 30 days after discharge, with up to 5 telephone attempts and 3 fax attempts. At the beginning of the questionnaire, PCPs were asked if they agreed with the designation, although they were asked to complete the questionnaire regardless. 

Analyses

Descriptive statistics compared differences in handoff practices and 30-day unplanned readmissions. Exploratory factor analysis assessed whether certain handoff practices were sufficiently correlated to allow grouping of items and construction of scales. Relationships between handoff practices and readmissions were examined using bivariate, followed by multivariate, logistic regression adjusting for the covariates described. Collinearity was tested before constructing final models. Because no relationship was observed between CTM-3 and readmissions, additional mediation analyses were not pursued. All analyses were completed using STATA (SE version 14.0, StataCorp LP, College Station, Texas). This study was approved by the Institutional Review Boards at UCLA (study site) and University of Wisconsin (lead author site).

This study enrolled 701 of 816 eligible participants (85.9%) between October 2012 and January 2014. More than 99% of administrative data and 97% of caregiver questionnaires were complete. Of 685 patients with a reported PCP, we obtained responses from 577 PCPs (84.2%). Patient characteristics and outcomes were not significantly different for patients with and without a responding PCP; however, patients of nonresponding PCPs were more often publicly insured (64.5% vs. 48.2% for responding PCPs, P = 0.004) or seen by a subspecialist as opposed to a generalist (28.1% vs. 13.8% for responding PCPs, P = 0.001). 

The overall population characteristics are summarized in Table 1: 27.4% of the cohort was younger 2 years, 49.2% were Hispanic, and the majority (51.1%) had public insurance. The average length of the index hospitalization for the overall population was 4.8 days (standard deviation = 9.6), and 53.5% had at least 1 complex chronic condition. Eighty-four percent of the cohort reported using a generalist (vs. subspecialist) for primary care. 

Discharge Handoff Communication

Practices varied widely (Figure 1a). Verbal handoffs between hospital-based and PCPs were least common (10.7%), whereas discharge summary completion within 48 hours was most common (84.9%). Of variables measuring direct communication with PCPs, only notification of admission occurred at least half the time (50.8%). 

Exploratory factor analysis identified 5 well-correlated items (Cronbach α = 0.77), which were combined and labeled the Hospital and Primary Care Provider Communication scale (Figure 1b). Items included PCP notification of admission, discharge, and receipt of updates during hospitalization, as well as receipt of verbal and written handoffs prior to follow-up. While these 5 items were analyzed only in this scale, other practices were analyzed as independent variables. In this assessment, 42.1% of patients had a scale score of 0 (no items performed), while 5% had all 5 items completed 

Readmissions

The 30-day unplanned readmission rate to any hospital was 12.4%. Demographic characteristics were similar in patients with and without an unplanned readmission (Table 1); however, patients with a readmission were more often younger (P = 0.03) and used a subspecialist for primary care (P = 0.03). Fewer than 60% of those with an unplanned readmission had a usual source of sick and well care compared with 77.5% of those without a readmission (P < 0.001). The length of index stay was nearly 4 days longer for those with an unplanned readmission (9.3 days vs. 4.4 days, P < 0.001). These patients also had more hospitalizations or ED visits in the past year (P = 0.002 and P = 0.04, respectively) and saw more subspecialists (P < 0.001). 

Frequencies of communication practices between those with and without an unplanned readmission are illustrated in Table 2. Nearly three-quarters of caregivers whose children were readmitted reported having follow-up appointments scheduled before discharge, compared to 48.9% without a readmission (P < 0.001). In 71% of discharges followed by a readmission, caregivers were not very confident about avoiding readmission, vs. 44.8% of discharges with no readmission (P < 0.001). 

Readmissions were largely unrelated to handoff practices in multivariate analyses (Table 3). Having a follow-up visit scheduled prior to discharge was the only activity with a statistically significant association; however, it was actually associated with more than double the odds of readmission (adjusted odds ratio 2.20, 95% confidence interval 1.08-4.46). 

The complex nature of hospital discharge care has led to general optimism that improved handoff processes might reduce readmissions for pediatric patients. Although the current literature linking transition practices to readmissions in pediatrics has mixed results,^1,4,5 most studies are fragmented—investigating a single or small number of transitional care activities, such as outpatient follow-up visits, postdischarge caregiver phone calls, or PCP receipt of discharge summaries. Despite finding limited relationships with readmissions, a strength of our study was its inclusion of a more comprehensive set of traditional communication practices that the study team anticipates many primary care and hospital medicine providers would expect to be carried out for most, if not all, patients during the hospital-to-home transition. 

Although our study was developed earlier, the variables in our analyses align with each domain of the conceptual model for readmission risk proposed by the Seamless Transitions and Re(admissions) Network (STARNet).⁶ This model identifies 7 elements believed to directly impact readmission risk in children: hospital and ED utilization, underlying diseases, ability to care for diseases, access to outpatient care, discharge processes, and discharge readiness. For example, our study included ED and hospital visits in the past year, complex chronic conditions, number of subspecialists, caregiver confidence, having a usual source of care, insurance status, and the 11 consensus-based handoff practices identified by our study team. Therefore, although the included handoff practices we included were a limited set, our models provide a relatively comprehensive analysis of readmission risk, confirming caregiver confidence, usual source of care, and hospitalizations to be associated with unplanned readmissions.

With the exception of having scheduled follow-up appointments before discharge – which was associated with more rather than fewer readmissions—the included care practices were not associated with readmissions. We suspect that these findings likely represent selection bias, with hospital providers taking additional steps in communicating with outpatient providers when they are most concerned about a patient’s vulnerability at discharge, eg, due to severity of illness, sociodemographics, health literacy, access to care, or other factors. Such selection bias could have 2 potential effects: (1) creating associations between the performance of certain handoff practices and higher readmission risk (eg, hospital providers are more likely to set follow-up appointments with the sickest patients who are also most likely to be readmitted, or (2) negating weakly effective communication practices that have small effect sizes. The currently mixed literature suggests that if associations between these handoff practices and postdischarge outcomes exist, they are often opposite to our expectation and likely driven by selection bias. If there are real effects that are hidden by this selection bias, they may be weak or inconsistent.

Recent qualitative research highlights the needs and preferences of caregivers of children with chronic or complex conditions to promote their sense of self-efficacy at discharge.²⁹ Such needs include support from within and beyond the health system, comprehensive discharge education, and written instructions, ultimately leading to confidence and comfort in executing the home-management plan. Consistent with our work,¹⁷ a strong independent relationship between caregiver confidence and postdischarge outcomes remained even after accounting for these conventional handoff activities. 

Transitions research in pediatrics has started only recently to move beyond traditional handoff communication between hospital and outpatient providers. Over the last several years, more ambitious conceptualizations of hospital discharge care have evolved² and include constructs such as family-centeredness,^4,28,29 discharge readiness,³⁰ and social determinants of health.³¹ Interventions targeting these constructs are largely missing from the literature and are greatly needed. If transitions are to have an effect on downstream utilization, their focus likely needs to evolve to address such areas.

Finally, our study underscores the need to identify relevant outcomes of improved transitional care. Although the preventability of postdischarge utilization continues to be debated, most would agree that this should not detract from the importance of high-quality transitional care. The STARNet collaborative provides some examples of outcomes potentially impacted through improved transitional care,⁶ although the authors note that reliability, validity, and feasibility of the measures are not well understood. High-quality transitional care presumably would lead to improvements in patient and family experience and perhaps safer care. Although caregiver experience measured by an adapted CTM-3 was neither a mediator nor a predictor of postdischarge utilization for children in our study, use of more rigorously developed tools for pediatric patients³² may provide a better assessment of caregiver experience. Finally, given the well-described risks of poor communication between hospital and outpatient providers,^33-35 safety events may be a better outcome of high-quality transitional care than readmissions. Investment in transitional care initiatives would be well justified if the positive patient, provider, and health system impacts can be better demonstrated through improved outcomes.

Future readmissions research should aim to accomplish several goals. Because observational studies will continue to be challenged by the selection biases described above, more rigorously designed and controlled experimental pediatric studies are needed. Family, social, and primary care characteristics should continue to be incorporated into pediatric readmission analyses given their increasingly recognized critical role. These variables, some of which could be modifiable, might represent potential targets for innovative readmission reduction interventions. Recently published conceptual models^6,29,36 provide a useful starting framework. 

Limitations

Because of the observational study design, we cannot draw conclusions about causal relationships between handoff practices and the measured outcomes. The tertiary care single-center nature of the study limits generalizability. Response biases are possible given that we often could not verify accuracy of PCP and caregiver responses. As noted above, we suspect that handoff practices were driven by important selection bias, not all of which could be controlled by the measured patient and clinical characteristics. The handoff practices included in this study were a limited set primarily focused on communication between hospital providers and PCPs. Therefore, the study does not rule out the possibility that other aspects of transitional care may reduce readmissions. Subsequent work investigating innovative interventions may find reductions in readmissions and other important outcomes. Additionally, not all practices have standardized definitions, eg, what 1 PCP considers a verbal handoff may be different from that of another provider. Although we assessed whether communication occurred, we were not able to assess the content or quality of communication, which may have important implications for its effectiveness.^37,38 

Improvements in handoffs between hospital and PCPs may have an important impact on postdischarge outcomes, but it is not clear that unplanned 30-day readmissions is among them. Efforts to reduce postdischarge utilization, if possible, likely need to focus on broader constructs such as caregiver self-efficacy, discharge readiness, and social determinants of health.

Disclosures

This study was supported by a grant from the Lucile Packard Foundation for Children’s Health, Palo Alto, California, as well as grant R40MC25677 Maternal and Child Health Research Program, Maternal and Child Health Bureau (Title V, Social Security Act), Health Resources and Services Administration, Department of Health and Human Services. The authors report no financial conflicts of interest.

Although much has been written about pediatric discharge and readmissions^1-5 over the past several years, surprisingly little is known about which care practices are most effective at preventing postdischarge utilization.⁵ Major collaborations across the U.S. are currently focused on improving pediatric discharge processes,^6-8 although the impact that these efforts will have on readmissions remains to be seen.

Research on handoffs between hospitals and primary care has mixed associations with postdischarge utilization. Although some studies observe positive relationships between specific activities and reduced postdischarge utilization,¹ others suggest no relationship^9-12 or, paradoxically, more utilization.^13,14 Brittan et al¹⁵ found that outpatient visits were associated with more readmissions when occurring less than 4 days after discharge, but fewer readmissions when occurring 4 days to  29 days after discharge. Most studies, however, investigate single or limited sets of care activities, such as having an outpatient visit,¹⁵ timeliness of that visit,¹⁶ or receipt of a discharge summary.¹¹ Inclusion of a more comprehensive set of hospital- to primary-care communication practices may better unravel this complex relationship between discharge care and postdischarge outcomes for children.

The purpose of this study was to characterize a set of traditional discharge handoff practices between hospital and primary care providers (PCPs) and to explore their relationships to readmissions. We hypothesized that handoff practices would be associated with fewer unplanned readmissions.

Study Design, Setting, Participants

This project was part of a prospective cohort study with 2 aims: to investigate relationships between medical home experience and postdischarge utilization,¹⁷ and to identify relationships between common discharge communication practices and postdischarge utilization. This manuscript is focused on the second aim. Randomly selected pediatric patients and their caregivers were enrolled from any medical or surgical service during an acute hospitalization lasting more than 24 hours from October 1, 2012 to January 1, 2014, at a 100-bed tertiary children’s hospital. Patients who transferred to another facility, died, were older than 18 years or in neonatal care (ie, newborn nursery or neonatal intensive care unit) were excluded since their discharge experiences would be significantly distinct from the population of interest. Patients were enrolled once in the study.

Outcome

The study’s primary outcome was 30-day unplanned readmissions, defined as a hospitalization occurring within 30 days of the index (ie, study enrollment) hospitalization, identified through caregiver report or administrative sources.¹⁷ Although the study site is a single hospital system, readmissions could have occurred to any hospital reported by caregivers, (ie, readmissions could have occurred within or outside our health system). Readmissions for chemotherapy, radiation, dialysis, rehabilitation, or labor and delivery were excluded. If caregivers reported an admission as planned or chart review of the index discharge summary noted that a rehospitalization was scheduled in the subsequent 30 days, the readmission was labeled “planned” and excluded. 

Discharge Handoff Communication

Transitional care is a set of actions designed to ensure continuity and coordination of healthcare during transfer from 1 location or level of care to another.^18,19 The study team, comprised of a division chief of general pediatrics, a division chief of hospital medicine, 2 departmental vice-chairs, and the medical director for quality at the study site, identified 11 common handoff activities and reporting sources. These consensus-based activities were expected by the study team to improve continuity and coordination during hospital-to-home transfer, and included:

• verifying PCP identity during the hospitalization (caregiver report); 
• notifying the PCP of admission, discharge, and providing updates during the hospitalization (PCP report); 
• PCP follow-up appointment set prior to discharge (caregiver report); 
• documenting planned PCP and subspecialty follow-up in the discharge summary (chart review); 
• completing the discharge summary within 48 hours (chart review); 
• providing a verbal or written handoff to the PCP prior to follow-up (PCP report); and 
• having a PCP follow-up visit within 30 days of discharge (caregiver report). 

We also asked PCPs whether they thought the follow-up interval was appropriate and whether phone follow-up with the patient would have been as appropriate as a face-to-face visit. 

Covariates

Patient demographics that might confound the relationship between handoff practices and readmissions based on pediatric research^20,21 were included. Medical complexity was accounted for by length-of-index stay, the number of hospitalizations and emergency department (ED) visits in past 12 months, complex chronic conditions,^22,23 and seeing 3 or more subspecialists.^24,25 Variables from related work included PCP scope (general pediatrics or subspecialist) and presence of a usual source for well and sick care.¹⁷

The Care Transitions Measure-3 (CTM-3), originally developed to assess the patient-centeredness of hospital transition,^26,27 can discriminate adult patients at risk for readmission.²⁶ We adapted the original CTM-3 to be answered by caregiver respondents after pilot testing with 5 caregivers not enrolled in the study: 1) “The hospital staff took my preferences and those of my family into account in deciding what my child’s health care needs would be when I left the hospital;” 2) “When I left the hospital, I had a good understanding of the things I was responsible for in managing my child’s health;” and 3) “When I left the hospital, I clearly understood the purpose for giving each of my child’s medications.” We analyzed the adapted CTM-3 on a transformed 0-100 scale as designed,²⁶ initially hypothesizing that the CTM-3 would mediate the relationship between handoff practices and readmissions.

We assessed caregiver confidence to avoid a readmission, based on a strong independent association with readmissions described in Coller et al.¹⁷ Using questions developed for this study, caregivers were asked to rate “How confident are you that [child’s name] will stay out of the hospital for the next 30 days?” with instructions to refer to unplanned hospital visits only. Responses were reported on a 4-point Likert scale (1 = very confident, 4 = not very confident). Responses were dichotomized into very confident (ie, “1”) or not very confident (ie, “2-4”).

Enrollment and Data Collection

Computer-generated random numbers were assigned to patients admitted the previous day, and families were enrolled sequentially until the daily enrollment target was reached. Data were obtained from 3 sources: medical record, caregiver report, and PCP report. Trained research assistants systematically extracted chart review data documenting the transitions practices above, while a hospital information technology analyst extracted claims and demographic data to complement what was reported by parents and PCPs. After study conclusion, these medical record data were merged with caregiver and PCP-reported data.

Trained bilingual research assistants collected caregiver- and PCP-reported data using structured questionnaires in English or Spanish, according to preference. Timing of data collection differed by data source; caregiver-reported data were collected immediately after discharge and at 30 days postdischarge; PCP-reported data were collected at 30 days postdischarge. 

Caregiver-reported data were collected through 2 separate phone calls following index discharge: immediately after discharge (caregiver confidence and CTM-3 measures) and at 30 days (readmission measures). Caregiver confidence questions were asked after (rather than immediately before) discharge to avoid biasing clinical care and revisit risk, consistent with previous work.²⁸ 

PCP-reported data were collected using structured questionnaires with the PCP who was identified by the family during study enrollment. PCP-reported data were collected by telephone or fax 30 days after discharge, with up to 5 telephone attempts and 3 fax attempts. At the beginning of the questionnaire, PCPs were asked if they agreed with the designation, although they were asked to complete the questionnaire regardless. 

Analyses

Descriptive statistics compared differences in handoff practices and 30-day unplanned readmissions. Exploratory factor analysis assessed whether certain handoff practices were sufficiently correlated to allow grouping of items and construction of scales. Relationships between handoff practices and readmissions were examined using bivariate, followed by multivariate, logistic regression adjusting for the covariates described. Collinearity was tested before constructing final models. Because no relationship was observed between CTM-3 and readmissions, additional mediation analyses were not pursued. All analyses were completed using STATA (SE version 14.0, StataCorp LP, College Station, Texas). This study was approved by the Institutional Review Boards at UCLA (study site) and University of Wisconsin (lead author site).

This study enrolled 701 of 816 eligible participants (85.9%) between October 2012 and January 2014. More than 99% of administrative data and 97% of caregiver questionnaires were complete. Of 685 patients with a reported PCP, we obtained responses from 577 PCPs (84.2%). Patient characteristics and outcomes were not significantly different for patients with and without a responding PCP; however, patients of nonresponding PCPs were more often publicly insured (64.5% vs. 48.2% for responding PCPs, P = 0.004) or seen by a subspecialist as opposed to a generalist (28.1% vs. 13.8% for responding PCPs, P = 0.001). 

The overall population characteristics are summarized in Table 1: 27.4% of the cohort was younger 2 years, 49.2% were Hispanic, and the majority (51.1%) had public insurance. The average length of the index hospitalization for the overall population was 4.8 days (standard deviation = 9.6), and 53.5% had at least 1 complex chronic condition. Eighty-four percent of the cohort reported using a generalist (vs. subspecialist) for primary care. 

Discharge Handoff Communication

Practices varied widely (Figure 1a). Verbal handoffs between hospital-based and PCPs were least common (10.7%), whereas discharge summary completion within 48 hours was most common (84.9%). Of variables measuring direct communication with PCPs, only notification of admission occurred at least half the time (50.8%). 

Exploratory factor analysis identified 5 well-correlated items (Cronbach α = 0.77), which were combined and labeled the Hospital and Primary Care Provider Communication scale (Figure 1b). Items included PCP notification of admission, discharge, and receipt of updates during hospitalization, as well as receipt of verbal and written handoffs prior to follow-up. While these 5 items were analyzed only in this scale, other practices were analyzed as independent variables. In this assessment, 42.1% of patients had a scale score of 0 (no items performed), while 5% had all 5 items completed 

Readmissions

The 30-day unplanned readmission rate to any hospital was 12.4%. Demographic characteristics were similar in patients with and without an unplanned readmission (Table 1); however, patients with a readmission were more often younger (P = 0.03) and used a subspecialist for primary care (P = 0.03). Fewer than 60% of those with an unplanned readmission had a usual source of sick and well care compared with 77.5% of those without a readmission (P < 0.001). The length of index stay was nearly 4 days longer for those with an unplanned readmission (9.3 days vs. 4.4 days, P < 0.001). These patients also had more hospitalizations or ED visits in the past year (P = 0.002 and P = 0.04, respectively) and saw more subspecialists (P < 0.001). 

Frequencies of communication practices between those with and without an unplanned readmission are illustrated in Table 2. Nearly three-quarters of caregivers whose children were readmitted reported having follow-up appointments scheduled before discharge, compared to 48.9% without a readmission (P < 0.001). In 71% of discharges followed by a readmission, caregivers were not very confident about avoiding readmission, vs. 44.8% of discharges with no readmission (P < 0.001). 

Readmissions were largely unrelated to handoff practices in multivariate analyses (Table 3). Having a follow-up visit scheduled prior to discharge was the only activity with a statistically significant association; however, it was actually associated with more than double the odds of readmission (adjusted odds ratio 2.20, 95% confidence interval 1.08-4.46). 

The complex nature of hospital discharge care has led to general optimism that improved handoff processes might reduce readmissions for pediatric patients. Although the current literature linking transition practices to readmissions in pediatrics has mixed results,^1,4,5 most studies are fragmented—investigating a single or small number of transitional care activities, such as outpatient follow-up visits, postdischarge caregiver phone calls, or PCP receipt of discharge summaries. Despite finding limited relationships with readmissions, a strength of our study was its inclusion of a more comprehensive set of traditional communication practices that the study team anticipates many primary care and hospital medicine providers would expect to be carried out for most, if not all, patients during the hospital-to-home transition. 

Although our study was developed earlier, the variables in our analyses align with each domain of the conceptual model for readmission risk proposed by the Seamless Transitions and Re(admissions) Network (STARNet).⁶ This model identifies 7 elements believed to directly impact readmission risk in children: hospital and ED utilization, underlying diseases, ability to care for diseases, access to outpatient care, discharge processes, and discharge readiness. For example, our study included ED and hospital visits in the past year, complex chronic conditions, number of subspecialists, caregiver confidence, having a usual source of care, insurance status, and the 11 consensus-based handoff practices identified by our study team. Therefore, although the included handoff practices we included were a limited set, our models provide a relatively comprehensive analysis of readmission risk, confirming caregiver confidence, usual source of care, and hospitalizations to be associated with unplanned readmissions.

With the exception of having scheduled follow-up appointments before discharge – which was associated with more rather than fewer readmissions—the included care practices were not associated with readmissions. We suspect that these findings likely represent selection bias, with hospital providers taking additional steps in communicating with outpatient providers when they are most concerned about a patient’s vulnerability at discharge, eg, due to severity of illness, sociodemographics, health literacy, access to care, or other factors. Such selection bias could have 2 potential effects: (1) creating associations between the performance of certain handoff practices and higher readmission risk (eg, hospital providers are more likely to set follow-up appointments with the sickest patients who are also most likely to be readmitted, or (2) negating weakly effective communication practices that have small effect sizes. The currently mixed literature suggests that if associations between these handoff practices and postdischarge outcomes exist, they are often opposite to our expectation and likely driven by selection bias. If there are real effects that are hidden by this selection bias, they may be weak or inconsistent.

Recent qualitative research highlights the needs and preferences of caregivers of children with chronic or complex conditions to promote their sense of self-efficacy at discharge.²⁹ Such needs include support from within and beyond the health system, comprehensive discharge education, and written instructions, ultimately leading to confidence and comfort in executing the home-management plan. Consistent with our work,¹⁷ a strong independent relationship between caregiver confidence and postdischarge outcomes remained even after accounting for these conventional handoff activities. 

Transitions research in pediatrics has started only recently to move beyond traditional handoff communication between hospital and outpatient providers. Over the last several years, more ambitious conceptualizations of hospital discharge care have evolved² and include constructs such as family-centeredness,^4,28,29 discharge readiness,³⁰ and social determinants of health.³¹ Interventions targeting these constructs are largely missing from the literature and are greatly needed. If transitions are to have an effect on downstream utilization, their focus likely needs to evolve to address such areas.

Finally, our study underscores the need to identify relevant outcomes of improved transitional care. Although the preventability of postdischarge utilization continues to be debated, most would agree that this should not detract from the importance of high-quality transitional care. The STARNet collaborative provides some examples of outcomes potentially impacted through improved transitional care,⁶ although the authors note that reliability, validity, and feasibility of the measures are not well understood. High-quality transitional care presumably would lead to improvements in patient and family experience and perhaps safer care. Although caregiver experience measured by an adapted CTM-3 was neither a mediator nor a predictor of postdischarge utilization for children in our study, use of more rigorously developed tools for pediatric patients³² may provide a better assessment of caregiver experience. Finally, given the well-described risks of poor communication between hospital and outpatient providers,^33-35 safety events may be a better outcome of high-quality transitional care than readmissions. Investment in transitional care initiatives would be well justified if the positive patient, provider, and health system impacts can be better demonstrated through improved outcomes.

Future readmissions research should aim to accomplish several goals. Because observational studies will continue to be challenged by the selection biases described above, more rigorously designed and controlled experimental pediatric studies are needed. Family, social, and primary care characteristics should continue to be incorporated into pediatric readmission analyses given their increasingly recognized critical role. These variables, some of which could be modifiable, might represent potential targets for innovative readmission reduction interventions. Recently published conceptual models^6,29,36 provide a useful starting framework. 

Limitations

Because of the observational study design, we cannot draw conclusions about causal relationships between handoff practices and the measured outcomes. The tertiary care single-center nature of the study limits generalizability. Response biases are possible given that we often could not verify accuracy of PCP and caregiver responses. As noted above, we suspect that handoff practices were driven by important selection bias, not all of which could be controlled by the measured patient and clinical characteristics. The handoff practices included in this study were a limited set primarily focused on communication between hospital providers and PCPs. Therefore, the study does not rule out the possibility that other aspects of transitional care may reduce readmissions. Subsequent work investigating innovative interventions may find reductions in readmissions and other important outcomes. Additionally, not all practices have standardized definitions, eg, what 1 PCP considers a verbal handoff may be different from that of another provider. Although we assessed whether communication occurred, we were not able to assess the content or quality of communication, which may have important implications for its effectiveness.^37,38 

Improvements in handoffs between hospital and PCPs may have an important impact on postdischarge outcomes, but it is not clear that unplanned 30-day readmissions is among them. Efforts to reduce postdischarge utilization, if possible, likely need to focus on broader constructs such as caregiver self-efficacy, discharge readiness, and social determinants of health.

Disclosures

This study was supported by a grant from the Lucile Packard Foundation for Children’s Health, Palo Alto, California, as well as grant R40MC25677 Maternal and Child Health Research Program, Maternal and Child Health Bureau (Title V, Social Security Act), Health Resources and Services Administration, Department of Health and Human Services. The authors report no financial conflicts of interest.

Although much has been written about pediatric discharge and readmissions^1-5 over the past several years, surprisingly little is known about which care practices are most effective at preventing postdischarge utilization.⁵ Major collaborations across the U.S. are currently focused on improving pediatric discharge processes,^6-8 although the impact that these efforts will have on readmissions remains to be seen.

Research on handoffs between hospitals and primary care has mixed associations with postdischarge utilization. Although some studies observe positive relationships between specific activities and reduced postdischarge utilization,¹ others suggest no relationship^9-12 or, paradoxically, more utilization.^13,14 Brittan et al¹⁵ found that outpatient visits were associated with more readmissions when occurring less than 4 days after discharge, but fewer readmissions when occurring 4 days to  29 days after discharge. Most studies, however, investigate single or limited sets of care activities, such as having an outpatient visit,¹⁵ timeliness of that visit,¹⁶ or receipt of a discharge summary.¹¹ Inclusion of a more comprehensive set of hospital- to primary-care communication practices may better unravel this complex relationship between discharge care and postdischarge outcomes for children.

The purpose of this study was to characterize a set of traditional discharge handoff practices between hospital and primary care providers (PCPs) and to explore their relationships to readmissions. We hypothesized that handoff practices would be associated with fewer unplanned readmissions.

Study Design, Setting, Participants

This project was part of a prospective cohort study with 2 aims: to investigate relationships between medical home experience and postdischarge utilization,¹⁷ and to identify relationships between common discharge communication practices and postdischarge utilization. This manuscript is focused on the second aim. Randomly selected pediatric patients and their caregivers were enrolled from any medical or surgical service during an acute hospitalization lasting more than 24 hours from October 1, 2012 to January 1, 2014, at a 100-bed tertiary children’s hospital. Patients who transferred to another facility, died, were older than 18 years or in neonatal care (ie, newborn nursery or neonatal intensive care unit) were excluded since their discharge experiences would be significantly distinct from the population of interest. Patients were enrolled once in the study.

Outcome

The study’s primary outcome was 30-day unplanned readmissions, defined as a hospitalization occurring within 30 days of the index (ie, study enrollment) hospitalization, identified through caregiver report or administrative sources.¹⁷ Although the study site is a single hospital system, readmissions could have occurred to any hospital reported by caregivers, (ie, readmissions could have occurred within or outside our health system). Readmissions for chemotherapy, radiation, dialysis, rehabilitation, or labor and delivery were excluded. If caregivers reported an admission as planned or chart review of the index discharge summary noted that a rehospitalization was scheduled in the subsequent 30 days, the readmission was labeled “planned” and excluded. 

Discharge Handoff Communication

Transitional care is a set of actions designed to ensure continuity and coordination of healthcare during transfer from 1 location or level of care to another.^18,19 The study team, comprised of a division chief of general pediatrics, a division chief of hospital medicine, 2 departmental vice-chairs, and the medical director for quality at the study site, identified 11 common handoff activities and reporting sources. These consensus-based activities were expected by the study team to improve continuity and coordination during hospital-to-home transfer, and included:

• verifying PCP identity during the hospitalization (caregiver report); 
• notifying the PCP of admission, discharge, and providing updates during the hospitalization (PCP report); 
• PCP follow-up appointment set prior to discharge (caregiver report); 
• documenting planned PCP and subspecialty follow-up in the discharge summary (chart review); 
• completing the discharge summary within 48 hours (chart review); 
• providing a verbal or written handoff to the PCP prior to follow-up (PCP report); and 
• having a PCP follow-up visit within 30 days of discharge (caregiver report). 

We also asked PCPs whether they thought the follow-up interval was appropriate and whether phone follow-up with the patient would have been as appropriate as a face-to-face visit. 

Covariates

Patient demographics that might confound the relationship between handoff practices and readmissions based on pediatric research^20,21 were included. Medical complexity was accounted for by length-of-index stay, the number of hospitalizations and emergency department (ED) visits in past 12 months, complex chronic conditions,^22,23 and seeing 3 or more subspecialists.^24,25 Variables from related work included PCP scope (general pediatrics or subspecialist) and presence of a usual source for well and sick care.¹⁷

The Care Transitions Measure-3 (CTM-3), originally developed to assess the patient-centeredness of hospital transition,^26,27 can discriminate adult patients at risk for readmission.²⁶ We adapted the original CTM-3 to be answered by caregiver respondents after pilot testing with 5 caregivers not enrolled in the study: 1) “The hospital staff took my preferences and those of my family into account in deciding what my child’s health care needs would be when I left the hospital;” 2) “When I left the hospital, I had a good understanding of the things I was responsible for in managing my child’s health;” and 3) “When I left the hospital, I clearly understood the purpose for giving each of my child’s medications.” We analyzed the adapted CTM-3 on a transformed 0-100 scale as designed,²⁶ initially hypothesizing that the CTM-3 would mediate the relationship between handoff practices and readmissions.

We assessed caregiver confidence to avoid a readmission, based on a strong independent association with readmissions described in Coller et al.¹⁷ Using questions developed for this study, caregivers were asked to rate “How confident are you that [child’s name] will stay out of the hospital for the next 30 days?” with instructions to refer to unplanned hospital visits only. Responses were reported on a 4-point Likert scale (1 = very confident, 4 = not very confident). Responses were dichotomized into very confident (ie, “1”) or not very confident (ie, “2-4”).

Enrollment and Data Collection

Computer-generated random numbers were assigned to patients admitted the previous day, and families were enrolled sequentially until the daily enrollment target was reached. Data were obtained from 3 sources: medical record, caregiver report, and PCP report. Trained research assistants systematically extracted chart review data documenting the transitions practices above, while a hospital information technology analyst extracted claims and demographic data to complement what was reported by parents and PCPs. After study conclusion, these medical record data were merged with caregiver and PCP-reported data.

Trained bilingual research assistants collected caregiver- and PCP-reported data using structured questionnaires in English or Spanish, according to preference. Timing of data collection differed by data source; caregiver-reported data were collected immediately after discharge and at 30 days postdischarge; PCP-reported data were collected at 30 days postdischarge. 

Caregiver-reported data were collected through 2 separate phone calls following index discharge: immediately after discharge (caregiver confidence and CTM-3 measures) and at 30 days (readmission measures). Caregiver confidence questions were asked after (rather than immediately before) discharge to avoid biasing clinical care and revisit risk, consistent with previous work.²⁸ 

PCP-reported data were collected using structured questionnaires with the PCP who was identified by the family during study enrollment. PCP-reported data were collected by telephone or fax 30 days after discharge, with up to 5 telephone attempts and 3 fax attempts. At the beginning of the questionnaire, PCPs were asked if they agreed with the designation, although they were asked to complete the questionnaire regardless. 

Analyses

Descriptive statistics compared differences in handoff practices and 30-day unplanned readmissions. Exploratory factor analysis assessed whether certain handoff practices were sufficiently correlated to allow grouping of items and construction of scales. Relationships between handoff practices and readmissions were examined using bivariate, followed by multivariate, logistic regression adjusting for the covariates described. Collinearity was tested before constructing final models. Because no relationship was observed between CTM-3 and readmissions, additional mediation analyses were not pursued. All analyses were completed using STATA (SE version 14.0, StataCorp LP, College Station, Texas). This study was approved by the Institutional Review Boards at UCLA (study site) and University of Wisconsin (lead author site).

This study enrolled 701 of 816 eligible participants (85.9%) between October 2012 and January 2014. More than 99% of administrative data and 97% of caregiver questionnaires were complete. Of 685 patients with a reported PCP, we obtained responses from 577 PCPs (84.2%). Patient characteristics and outcomes were not significantly different for patients with and without a responding PCP; however, patients of nonresponding PCPs were more often publicly insured (64.5% vs. 48.2% for responding PCPs, P = 0.004) or seen by a subspecialist as opposed to a generalist (28.1% vs. 13.8% for responding PCPs, P = 0.001). 

The overall population characteristics are summarized in Table 1: 27.4% of the cohort was younger 2 years, 49.2% were Hispanic, and the majority (51.1%) had public insurance. The average length of the index hospitalization for the overall population was 4.8 days (standard deviation = 9.6), and 53.5% had at least 1 complex chronic condition. Eighty-four percent of the cohort reported using a generalist (vs. subspecialist) for primary care. 

Discharge Handoff Communication

Practices varied widely (Figure 1a). Verbal handoffs between hospital-based and PCPs were least common (10.7%), whereas discharge summary completion within 48 hours was most common (84.9%). Of variables measuring direct communication with PCPs, only notification of admission occurred at least half the time (50.8%). 

Exploratory factor analysis identified 5 well-correlated items (Cronbach α = 0.77), which were combined and labeled the Hospital and Primary Care Provider Communication scale (Figure 1b). Items included PCP notification of admission, discharge, and receipt of updates during hospitalization, as well as receipt of verbal and written handoffs prior to follow-up. While these 5 items were analyzed only in this scale, other practices were analyzed as independent variables. In this assessment, 42.1% of patients had a scale score of 0 (no items performed), while 5% had all 5 items completed 

Readmissions

The 30-day unplanned readmission rate to any hospital was 12.4%. Demographic characteristics were similar in patients with and without an unplanned readmission (Table 1); however, patients with a readmission were more often younger (P = 0.03) and used a subspecialist for primary care (P = 0.03). Fewer than 60% of those with an unplanned readmission had a usual source of sick and well care compared with 77.5% of those without a readmission (P < 0.001). The length of index stay was nearly 4 days longer for those with an unplanned readmission (9.3 days vs. 4.4 days, P < 0.001). These patients also had more hospitalizations or ED visits in the past year (P = 0.002 and P = 0.04, respectively) and saw more subspecialists (P < 0.001). 

Frequencies of communication practices between those with and without an unplanned readmission are illustrated in Table 2. Nearly three-quarters of caregivers whose children were readmitted reported having follow-up appointments scheduled before discharge, compared to 48.9% without a readmission (P < 0.001). In 71% of discharges followed by a readmission, caregivers were not very confident about avoiding readmission, vs. 44.8% of discharges with no readmission (P < 0.001). 

Readmissions were largely unrelated to handoff practices in multivariate analyses (Table 3). Having a follow-up visit scheduled prior to discharge was the only activity with a statistically significant association; however, it was actually associated with more than double the odds of readmission (adjusted odds ratio 2.20, 95% confidence interval 1.08-4.46). 

The complex nature of hospital discharge care has led to general optimism that improved handoff processes might reduce readmissions for pediatric patients. Although the current literature linking transition practices to readmissions in pediatrics has mixed results,^1,4,5 most studies are fragmented—investigating a single or small number of transitional care activities, such as outpatient follow-up visits, postdischarge caregiver phone calls, or PCP receipt of discharge summaries. Despite finding limited relationships with readmissions, a strength of our study was its inclusion of a more comprehensive set of traditional communication practices that the study team anticipates many primary care and hospital medicine providers would expect to be carried out for most, if not all, patients during the hospital-to-home transition. 

Although our study was developed earlier, the variables in our analyses align with each domain of the conceptual model for readmission risk proposed by the Seamless Transitions and Re(admissions) Network (STARNet).⁶ This model identifies 7 elements believed to directly impact readmission risk in children: hospital and ED utilization, underlying diseases, ability to care for diseases, access to outpatient care, discharge processes, and discharge readiness. For example, our study included ED and hospital visits in the past year, complex chronic conditions, number of subspecialists, caregiver confidence, having a usual source of care, insurance status, and the 11 consensus-based handoff practices identified by our study team. Therefore, although the included handoff practices we included were a limited set, our models provide a relatively comprehensive analysis of readmission risk, confirming caregiver confidence, usual source of care, and hospitalizations to be associated with unplanned readmissions.

With the exception of having scheduled follow-up appointments before discharge – which was associated with more rather than fewer readmissions—the included care practices were not associated with readmissions. We suspect that these findings likely represent selection bias, with hospital providers taking additional steps in communicating with outpatient providers when they are most concerned about a patient’s vulnerability at discharge, eg, due to severity of illness, sociodemographics, health literacy, access to care, or other factors. Such selection bias could have 2 potential effects: (1) creating associations between the performance of certain handoff practices and higher readmission risk (eg, hospital providers are more likely to set follow-up appointments with the sickest patients who are also most likely to be readmitted, or (2) negating weakly effective communication practices that have small effect sizes. The currently mixed literature suggests that if associations between these handoff practices and postdischarge outcomes exist, they are often opposite to our expectation and likely driven by selection bias. If there are real effects that are hidden by this selection bias, they may be weak or inconsistent.

Recent qualitative research highlights the needs and preferences of caregivers of children with chronic or complex conditions to promote their sense of self-efficacy at discharge.²⁹ Such needs include support from within and beyond the health system, comprehensive discharge education, and written instructions, ultimately leading to confidence and comfort in executing the home-management plan. Consistent with our work,¹⁷ a strong independent relationship between caregiver confidence and postdischarge outcomes remained even after accounting for these conventional handoff activities. 

Transitions research in pediatrics has started only recently to move beyond traditional handoff communication between hospital and outpatient providers. Over the last several years, more ambitious conceptualizations of hospital discharge care have evolved² and include constructs such as family-centeredness,^4,28,29 discharge readiness,³⁰ and social determinants of health.³¹ Interventions targeting these constructs are largely missing from the literature and are greatly needed. If transitions are to have an effect on downstream utilization, their focus likely needs to evolve to address such areas.

Finally, our study underscores the need to identify relevant outcomes of improved transitional care. Although the preventability of postdischarge utilization continues to be debated, most would agree that this should not detract from the importance of high-quality transitional care. The STARNet collaborative provides some examples of outcomes potentially impacted through improved transitional care,⁶ although the authors note that reliability, validity, and feasibility of the measures are not well understood. High-quality transitional care presumably would lead to improvements in patient and family experience and perhaps safer care. Although caregiver experience measured by an adapted CTM-3 was neither a mediator nor a predictor of postdischarge utilization for children in our study, use of more rigorously developed tools for pediatric patients³² may provide a better assessment of caregiver experience. Finally, given the well-described risks of poor communication between hospital and outpatient providers,^33-35 safety events may be a better outcome of high-quality transitional care than readmissions. Investment in transitional care initiatives would be well justified if the positive patient, provider, and health system impacts can be better demonstrated through improved outcomes.

Future readmissions research should aim to accomplish several goals. Because observational studies will continue to be challenged by the selection biases described above, more rigorously designed and controlled experimental pediatric studies are needed. Family, social, and primary care characteristics should continue to be incorporated into pediatric readmission analyses given their increasingly recognized critical role. These variables, some of which could be modifiable, might represent potential targets for innovative readmission reduction interventions. Recently published conceptual models^6,29,36 provide a useful starting framework. 

Limitations

Because of the observational study design, we cannot draw conclusions about causal relationships between handoff practices and the measured outcomes. The tertiary care single-center nature of the study limits generalizability. Response biases are possible given that we often could not verify accuracy of PCP and caregiver responses. As noted above, we suspect that handoff practices were driven by important selection bias, not all of which could be controlled by the measured patient and clinical characteristics. The handoff practices included in this study were a limited set primarily focused on communication between hospital providers and PCPs. Therefore, the study does not rule out the possibility that other aspects of transitional care may reduce readmissions. Subsequent work investigating innovative interventions may find reductions in readmissions and other important outcomes. Additionally, not all practices have standardized definitions, eg, what 1 PCP considers a verbal handoff may be different from that of another provider. Although we assessed whether communication occurred, we were not able to assess the content or quality of communication, which may have important implications for its effectiveness.^37,38 

Improvements in handoffs between hospital and PCPs may have an important impact on postdischarge outcomes, but it is not clear that unplanned 30-day readmissions is among them. Efforts to reduce postdischarge utilization, if possible, likely need to focus on broader constructs such as caregiver self-efficacy, discharge readiness, and social determinants of health.

Disclosures

This study was supported by a grant from the Lucile Packard Foundation for Children’s Health, Palo Alto, California, as well as grant R40MC25677 Maternal and Child Health Research Program, Maternal and Child Health Bureau (Title V, Social Security Act), Health Resources and Services Administration, Department of Health and Human Services. The authors report no financial conflicts of interest.