Original Research

It was estimated that physicians would diagnose a form of invasive cancer > 1.7 million times in 2019. As the second most common cause of death in the US, > 600,000 people were projected to die from cancer in 2019.¹ Many individuals with cancer endure distress, which the National Comprehensive Cancer Network (NCCN) defines as a “multifactorial unpleasant experience of a psychological (ie, cognitive, behavioral, emotional), social, spiritual, and/or physical nature that may interfere with the ability to cope effectively with cancer, its physical symptoms, and its treatment.”^2,3 Distress in people living with cancer has been attributed to various psychosocial concerns, such as family problems, whichinclude dealing with partners and children; emotional problems, such as depression and anxiety; and physical symptoms, such as pain and fatigue.^4-9 Certain factors associated with distress may increase a patient’s risk for suicide.⁴

Veterans are at particularly high risk for suicide.¹⁰ In 2014, veterans accounted for 18% of completed suicides in the US but only were 8.5% of the total population that same year.¹⁰ Yet, little research has been done on the relationship between distress and suicide in veterans living with cancer. Aboumrad and colleagues found that 45% of veterans with cancer who completed suicide reported family issues and 41% endorsed chronic pain.¹¹ This study recommended continued efforts to assess and treat distress to lessen risk of suicide in veterans living with cancer; however, to date, only 1 study has specifically evaluated distress and problems endorsed among veterans living with cancer.⁷

Suicide prevention is of the highest priority to the US Department of Veterans Affairs (VA).¹² Consistent with the VA mission to end veteran suicide, the current study aimed to better understand the relationship between distress and suicide within a sample of veterans living with cancer. Findings would additionally be used to tailor clinical assessments and interventions for veterans living with cancer.

This study had 3 primary goals. First, we sought to understand demographic and clinical factors associated with low, moderate, and severe levels of distress in veterans living with cancer who were referred for psychology services. Second, the study investigated the most commonly endorsed problems by veterans living with cancer. Finally, we examined which problems were related to suicidal ideation (SI). It was hypothesized that veterans who reported severe distress would be significantly more likely to endorse SI when compared with veterans who reported mild or moderate distress. Based on existing literature, it was further hypothesized that family, emotional, and physical problems would be significantly associated with SI.^7,11

Methods

The current study was conducted at James A. Haley Veterans’ Hospital (JAHVH) in Tampa, Florida. Inclusion criteria included veterans who were diagnosed with cancer, attended an outpatient psychology-oncology evaluation, and completed mental health screening measures provided during their evaluation. Exclusion criteria included veterans who: were seen in response to an inpatient consult, were seen solely for a stem cell transplant evaluation, or did not complete the screening measures.

Measures

A veteran’s demographic (eg, age, sex, ethnicity) and clinical (eg, cancer type, stage of disease, recurrence, cancer treatments received) information was abstracted from their VA medical records. Marital status was assessed during a clinical interview and documented as part of the standardized suicide risk assessment.

The Distress Thermometer (DT) is a subjective measure developed by the NCCN.² The DT provides a visual representation of a thermometer and asks patients to rate their level of distress over the past week with 0 indicating no distress and 10 indicating extreme distress. A distress rating of 4 or higher is clinically significant.^4,6 Distress may be categorized into 3 levels of severity: mild distress (< 4), moderate distress (4-7), or severe distress (8-10). The DT has been found to have good face validity, sensitivity and specificity, and is user-friendly.^2,6,7,13

The measurement additionally lists 39 problems nested within 5 domains: practical, family, emotional, spiritual/religious, and physical. Patients may endorse listed items under each problem domain by indicating yes or no. Endorsement of various items are intended to provide more detailed information about sources of distress. Due to the predominantly male and mostly older population included in this study the ability to have children measure was removed from the family problem domain.

SI was assessed in 2 ways. First, by patients’ self-report through item-9 of the Patient Health Questionnaire-9 (PHQ-9).¹⁴ Item-9 asks “over the last 2 weeks, how often have you been bothered by thoughts that you would be better off dead or of hurting yourself in some way?” Responses range from 0 (not at all) to 3 (nearly every day).¹⁴ Responses > 0 were considered a positive screen for SI. The process of administering the PHQ-9 item-9 is part of a national VA directive for standardizing assessment of suicide risk (Steve Young, personal communication, May 23, 2018). The PHQ-9 has been found to have good construct validity when used with both medical samples and the general population along with good internal and test-retest reliability.^14,15 Second, all veterans also were asked directly about SI during clinical interview, the results of which were documented in health records using a standardized format for risk assessment.

Procedure

Participants were a sample of veterans who were referred for psychology-oncology services. The NCCN DT and Problems List were administered prior to the start of clinical interviews, which followed a checklist and included standardized assessments of SI and history of a suicide attempt(s). A licensed clinical psychologist or a postdoctoral resident conducted these assessments under the supervision of a licensed psychologist. Data gathered during the clinical interview and from the DT and problems list were documented in health records, which were retrospectively reviewed for relevant information (eg, cancer diagnosis, SI). Therefore, informed consent was waived. This study was approved by the JAHVH Institutional Review Board.

Analysis

Data were analyzed using SPSS Version 25. Data analysis proceeded in 3 steps. First, descriptive statistics included the demographic and clinical factors present in the current sample. Difference between those with and without suicidal ideation were compared using F-statistic for continuous variables and χ² analyses for categorical variables. Second, to examine relationships between each DT problem domain and SI, χ² analyses were conducted. Third, DT problem domains that had a significant relationship with SI were entered in a logistic regression. This analysis determined which items, if any, from a DT problem domain predicted SI. In the logistic regression model, history of suicide attempts was entered into the first block, as history of suicide attempts is a well-established risk factor for subsequent suicidal ideation. In the second block, other variables that were significantly related to suicidal ideation in the second step of analyses were included. Before interpreting the results of the logistic regression, model fit was tested using the Hosmer-Lemeshow test. Significance of each individual predictor variable in the model is reported using the Wald χ² statistic; each Wald statistic is compared with a χ² distribution with 1 degree of freedom (df). Results of logistic regression models also provide information about the effect of each predictor variable in the regression equation (beta weight), odds a veteran who endorsed each predictor variable in the model would also endorse SI (as indicated by the odds ratio), and an estimate of the amount of variance accounted for by each predictor variable (using Nagelkerke’s pseudo R², which ranges in value from 0 to 1 with higher values indicating more variance explained). For all analyses, P value of .05 (2-tailed) was used for statistical significance.

Results

The sample consisted of 174 veterans (Table 1). The majority (77.6%) were male with a mean age of nearly 62 years (range, 29-87). Most identified as white (74.1%) with half reporting they were either married or living with a partner.

Prostate cancer (19.0%) was the most common type of cancer among study participants followed by head and neck (18.4%), lymphoma/leukemia (11.5%), lung (11.5%), and breast (10.9%); 31.6% had metastatic disease and 14.9% had recurrent disease. Chemotherapy (42.5%) was the most common treatment modality, followed by surgery (38.5%) and radiation (31.6%). The sample was distributed among the 3 distress DT categories: mild (18.4%), moderate (42.5%), and severe (39.1%).

Problems Endorsed

Treatment decisions (44.3%) and insurance/financial concerns (35.1%) were the most frequently endorsed practical problems (Figure 1). Family health issues (33.9%) and dealing with partner (23.0%) were the most frequently endorsed family problems (Figure 2). Worry (73.0%) and depression (69.5%) were the most frequent emotional problems; of note, all emotional problems were endorsed by at least 50% of veterans (Figure 3). Fatigue (71.3%), sleep (70.7%), and pain (69%), were the most frequently endorsed physical problems (Figure 4). Spiritual/religious problems were endorsed by 15% of veterans.

Suicidal Ideation

Overall, 25.3% of veterans endorsed SI. About 20% of veterans reported a history of ≥ 1 suicide attempts in their lifetime. A significant relationship among distress categories and SI was found (χ² = 18.36, P < .001). Veterans with severe distress were more likely to endorse SI (42.7%) when compared with veterans with mild (9.4%) or moderate (16.2%) distress.

Similarly, a significant relationship among distress categories and a history of a suicide attempt(s) was found (χ² = 6.08, P = .048). Veterans with severe distress were more likely to have attempted suicide (29.4%) when compared with veterans with mild (12.5%) or moderate (14.9%) distress.

Discussion

This study aimed to understand factors associated with low, moderate, and severe levels of distress in veterans living with cancer who were referred for psychology services. As hypothesized, veterans who endorsed severe distress were significantly more likely to endorse SI. They also were more likely to have a history of a suicide attempt(s) when compared with those with mild or moderate distress.

A second aim of this study was to understand the most commonly endorsed problems. Consistent with prior literature, treatment decisions were the most commonly endorsed practical problem; worry and depression were the most common emotional problems; and fatigue, sleep, and pain were the most common physical problems.⁷

A finding unique to the current study is that family health issues and dealing with partner were specified as the most common family problems. However, a study by Smith and colleagues did not provide information about the rank of most frequently reported problems within this domain.⁷

The third aim was to understand which problems were related to SI. It was hypothesized that family, emotional, and physical problems would be related to SI. However, results indicated that only family problems (specifically, problems dealing with a partner) were significantly associated with SI among veterans living with cancer.

Contrary to expectations, emotional and physical problems were not found to have a significant relationship with SI. This is likely because veterans endorsed items nested within these problem domains with similar frequency. The lack of significant findings does not suggest that emotional and physical problems are not significant predictors of SI for veterans living with cancer, but that no specific emotional or physical symptom stood out as a predictor of suicidal ideation above the others.

The finding of a significant relationship between family problems (specifically, problems dealing with a partner) and SI in this study is consistent with findings of Aboumrad and colleagues in a study that examined root-cause analyses of completed suicides by veterans living with cancer.¹¹ They found that nearly half the sample endorsed family problems prior to their death, and a small but notable percentage of veterans who completed suicide reported divorce as a stressor prior to their death.

This finding may be explained by Thomas Joiner's interpersonal-psychological theory of suicidal behavior (IPT), which suggests that completed suicide may result from a thwarted sense of belonging, perceived burdensomeness, and acquired capability for suicide.¹⁶ Problems dealing with a partner may impact a veteran’s sense of belonging or social connectedness. Problems dealing with a partner also may be attributed to perceived burdens due to limitations imposed by living with cancer and/or undergoing treatment. In both circumstances, the veteran’s social support system may be negatively impacted, and perceived social support is a well-established protective factor against suicide.¹⁷

Partner distress is a second consideration. It is likely that veterans’ partners experienced their own distress in response to the veteran’s cancer diagnosis and/or treatment. The partner’s cause, severity, and expression of distress may contribute to problems for the couple.

Finally, the latter point of the IPT refers to acquired capability, or the ability to inflict deadly harm to oneself.¹⁸ A military sample was found to have more acquired capability for suicide when compared with a college undergraduate sample.¹⁹ A history of a suicide attempt(s) and male gender have been found to significantly predict acquired capability to complete suicide.¹⁸ Furthermore, because veterans living with cancer often are in pain, fear of pain associated with suicide may be reduced and, therefore, acquired capability increased. This suggests that male veterans living with cancer who are in pain, have a history of a suicide attempt(s), and current problems with their partner may be an extremely vulnerable population at-risk for suicide. Results from the current study emphasize the importance of veterans having access to mental health and crisis resources for problems dealing with their partner. Partner problems may foreshadow a potentially lethal type of distress.

Strengths

This study’s aims are consistent with the VA’s mission to end veteran suicide and contributes to literature in several important ways.¹² First, veterans living with cancer are an understudied population. The current study addresses a gap in existing literature by researching veterans living with cancer and aims to better understand the relationship between cancer-related distress and SI. Second, to the best of the authors’ knowledge, this study is the first to find that problems dealing with a partner significantly increases a veteran’s risk for SI above a history of a suicide attempt(s). Risk assessments now may be more comprehensive through inclusion of this distress factor.

It is recommended that future research use IPT to further investigate the relationship between problems dealing with a partner and SI.¹⁶ Future research may do so by including specific measures to assess for the tenants of the theory, including measurements of burdensomeness and belongingness. An expanded knowledge base about what makes problems dealing with a partner a significant suicide risk factor (eg, increased conflict, lack of support, etc.) would better enable clinicians to intervene effectively. Effective intervention may lessen suicidal behaviors or deaths from suicides within the Veteran population.

Limitations

One limitation is the focus on patients who accepted a mental health referral. This study design may limit the generalizability of results to veterans who would not accept mental health treatment. The homogenous sample of veterans is a second limitation. Most participants were male, white, and had a mean age of 62 years. These demographics are representative of the veterans that most typically utilize VA services; however, more research is needed on veterans living with cancer who are female and of diverse racial and ethnic backgrounds. There are likely differences in problems endorsed and factors associated with SI based on age, race, sex, and other socioeconomic factors. A third limitation is the cross-sectional, retrospective nature of this study. Future studies are advised to assess for distress at multiple time points. This is consistent with NCCN Standards of Care for Distress Management.² Longitudinal data would enable more findings about distress and SI throughout the course of cancer diagnosis and treatment, therefore enhancing clinical implications and informing future research.

Conclusion

This is among the first of studies to investigate distress and factors associated with SI in veterans living with cancer who were referred for psychology services. The prevalence of distress caused by psychosocial factors (including treatment decisions, worry, and depression) highlights the importance of including mental health services as part of comprehensive cancer treatment.

Distress due to treatment decisions may be attributed to a litany of factors such as a veteran’s consideration of adverse effects, effectiveness of treatments, changes to quality of life or functioning, and inclusion of alternative or complimentary treatments. These types of decisions often are reported to be difficult conversations to have with family members or loved ones, who are likely experiencing distress of their own. The role of a mental health provider to assist veterans in exploring their treatment decisions and the implications of such decisions appears important to lessening distress.

Early intervention for emotional symptoms would likely benefit veterans’ management of distress and may lessen suicide risk as depression is known to place veterans at-risk for SI.²⁰ This underscores the importance of timely distress assessment to prevent mild emotional distress from progressing to potentially severe or life-threatening emotional distress. For veterans with a psychiatric history, timely assessment and intervention is essential because psychiatric history is an established suicide risk factor that may be exacerbated by cancer-related distress.¹²

Furthermore, management of intolerable physical symptoms may lessen risk for suicide.⁴ Under medical guidance, fatigue may be improved using exercise.²¹ Behavioral intervention is commonly used as first-line treatment for sleep problems.²² While pain may be lessened through medication or nonpharmacological interventions.²³

Considering the numerous ways that distress may present itself (eg, practical, emotional, or physical) and increase risk for SI, it is essential that all veterans living with cancer are assessed for distress and SI, regardless of their presentation. Although veterans may not outwardly express distress, this does not indicate the absence of either distress or risk for suicide. For example, a veteran may be distressed due to financial concerns, transportation issues, and the health of his/her partner or spouse. This veteran may not exhibit visible symptoms of distress, as would be expected when the source of distress is emotional (eg, depression, anxiety). However, this veteran is equally vulnerable to impairing distress and SI as someone who exhibits emotional distress. Distress assessments should be further developed to capture both the visible and less apparent sources of distress, while also serving the imperative function of screening for suicide. Other researchers also have noted the necessity of this development.²⁴ Currently, the NCCN DT and Problems List does not include any assessment of SI or behavior.

Finally, this study identified a potentially critical factor to include in distress assessment: problems dealing with a partner. Problems dealing with a partner have been noted as a source of distress in existing literature, but this is the first study to find problems dealing with a partner to be a predictor of SI in veterans living with cancer.^4-6

Because partners often attend appointments with veterans, it is not surprising that problems dealing with their partner are not disclosed more readily. It is recommended that clinicians ask veterans about potential problems with their partner when they are alone. Directly gathering information about such problems while assessing for distress may assist health care workers in providing the most effective, accurate type of intervention in a timely manner, and potentially mitigate risk for suicide.

As recommended by the NCCN and numerous researchers, findings from the current study underscore the importance of accurate, timely assessment of distress.^2,4,8 This study makes several important recommendations about how distress assessment may be strengthened and further developed, specifically for the veteran population. This study also expands the current knowledge base of what is known about veterans living with cancer, and has begun to fill a gap in the existing literature. Consistent with the VA mission to end veteran suicide, results suggest that veterans living with cancer should be regularly screened for distress, asked about distress related to their partner, and assessed for SI. Continued efforts to enhance assessment of and response to distress may lessen suicide risk in veterans with cancer.¹¹

Acknowledgements

This study is the result of work supported with resources and the use of facilities at the James A. Haley Veterans’ Hospital.

It was estimated that physicians would diagnose a form of invasive cancer > 1.7 million times in 2019. As the second most common cause of death in the US, > 600,000 people were projected to die from cancer in 2019.¹ Many individuals with cancer endure distress, which the National Comprehensive Cancer Network (NCCN) defines as a “multifactorial unpleasant experience of a psychological (ie, cognitive, behavioral, emotional), social, spiritual, and/or physical nature that may interfere with the ability to cope effectively with cancer, its physical symptoms, and its treatment.”^2,3 Distress in people living with cancer has been attributed to various psychosocial concerns, such as family problems, whichinclude dealing with partners and children; emotional problems, such as depression and anxiety; and physical symptoms, such as pain and fatigue.^4-9 Certain factors associated with distress may increase a patient’s risk for suicide.⁴

Veterans are at particularly high risk for suicide.¹⁰ In 2014, veterans accounted for 18% of completed suicides in the US but only were 8.5% of the total population that same year.¹⁰ Yet, little research has been done on the relationship between distress and suicide in veterans living with cancer. Aboumrad and colleagues found that 45% of veterans with cancer who completed suicide reported family issues and 41% endorsed chronic pain.¹¹ This study recommended continued efforts to assess and treat distress to lessen risk of suicide in veterans living with cancer; however, to date, only 1 study has specifically evaluated distress and problems endorsed among veterans living with cancer.⁷

Suicide prevention is of the highest priority to the US Department of Veterans Affairs (VA).¹² Consistent with the VA mission to end veteran suicide, the current study aimed to better understand the relationship between distress and suicide within a sample of veterans living with cancer. Findings would additionally be used to tailor clinical assessments and interventions for veterans living with cancer.

This study had 3 primary goals. First, we sought to understand demographic and clinical factors associated with low, moderate, and severe levels of distress in veterans living with cancer who were referred for psychology services. Second, the study investigated the most commonly endorsed problems by veterans living with cancer. Finally, we examined which problems were related to suicidal ideation (SI). It was hypothesized that veterans who reported severe distress would be significantly more likely to endorse SI when compared with veterans who reported mild or moderate distress. Based on existing literature, it was further hypothesized that family, emotional, and physical problems would be significantly associated with SI.^7,11

Methods

The current study was conducted at James A. Haley Veterans’ Hospital (JAHVH) in Tampa, Florida. Inclusion criteria included veterans who were diagnosed with cancer, attended an outpatient psychology-oncology evaluation, and completed mental health screening measures provided during their evaluation. Exclusion criteria included veterans who: were seen in response to an inpatient consult, were seen solely for a stem cell transplant evaluation, or did not complete the screening measures.

Measures

A veteran’s demographic (eg, age, sex, ethnicity) and clinical (eg, cancer type, stage of disease, recurrence, cancer treatments received) information was abstracted from their VA medical records. Marital status was assessed during a clinical interview and documented as part of the standardized suicide risk assessment.

The Distress Thermometer (DT) is a subjective measure developed by the NCCN.² The DT provides a visual representation of a thermometer and asks patients to rate their level of distress over the past week with 0 indicating no distress and 10 indicating extreme distress. A distress rating of 4 or higher is clinically significant.^4,6 Distress may be categorized into 3 levels of severity: mild distress (< 4), moderate distress (4-7), or severe distress (8-10). The DT has been found to have good face validity, sensitivity and specificity, and is user-friendly.^2,6,7,13

The measurement additionally lists 39 problems nested within 5 domains: practical, family, emotional, spiritual/religious, and physical. Patients may endorse listed items under each problem domain by indicating yes or no. Endorsement of various items are intended to provide more detailed information about sources of distress. Due to the predominantly male and mostly older population included in this study the ability to have children measure was removed from the family problem domain.

SI was assessed in 2 ways. First, by patients’ self-report through item-9 of the Patient Health Questionnaire-9 (PHQ-9).¹⁴ Item-9 asks “over the last 2 weeks, how often have you been bothered by thoughts that you would be better off dead or of hurting yourself in some way?” Responses range from 0 (not at all) to 3 (nearly every day).¹⁴ Responses > 0 were considered a positive screen for SI. The process of administering the PHQ-9 item-9 is part of a national VA directive for standardizing assessment of suicide risk (Steve Young, personal communication, May 23, 2018). The PHQ-9 has been found to have good construct validity when used with both medical samples and the general population along with good internal and test-retest reliability.^14,15 Second, all veterans also were asked directly about SI during clinical interview, the results of which were documented in health records using a standardized format for risk assessment.

Procedure

Participants were a sample of veterans who were referred for psychology-oncology services. The NCCN DT and Problems List were administered prior to the start of clinical interviews, which followed a checklist and included standardized assessments of SI and history of a suicide attempt(s). A licensed clinical psychologist or a postdoctoral resident conducted these assessments under the supervision of a licensed psychologist. Data gathered during the clinical interview and from the DT and problems list were documented in health records, which were retrospectively reviewed for relevant information (eg, cancer diagnosis, SI). Therefore, informed consent was waived. This study was approved by the JAHVH Institutional Review Board.

Analysis

Data were analyzed using SPSS Version 25. Data analysis proceeded in 3 steps. First, descriptive statistics included the demographic and clinical factors present in the current sample. Difference between those with and without suicidal ideation were compared using F-statistic for continuous variables and χ² analyses for categorical variables. Second, to examine relationships between each DT problem domain and SI, χ² analyses were conducted. Third, DT problem domains that had a significant relationship with SI were entered in a logistic regression. This analysis determined which items, if any, from a DT problem domain predicted SI. In the logistic regression model, history of suicide attempts was entered into the first block, as history of suicide attempts is a well-established risk factor for subsequent suicidal ideation. In the second block, other variables that were significantly related to suicidal ideation in the second step of analyses were included. Before interpreting the results of the logistic regression, model fit was tested using the Hosmer-Lemeshow test. Significance of each individual predictor variable in the model is reported using the Wald χ² statistic; each Wald statistic is compared with a χ² distribution with 1 degree of freedom (df). Results of logistic regression models also provide information about the effect of each predictor variable in the regression equation (beta weight), odds a veteran who endorsed each predictor variable in the model would also endorse SI (as indicated by the odds ratio), and an estimate of the amount of variance accounted for by each predictor variable (using Nagelkerke’s pseudo R², which ranges in value from 0 to 1 with higher values indicating more variance explained). For all analyses, P value of .05 (2-tailed) was used for statistical significance.

Results

The sample consisted of 174 veterans (Table 1). The majority (77.6%) were male with a mean age of nearly 62 years (range, 29-87). Most identified as white (74.1%) with half reporting they were either married or living with a partner.

Prostate cancer (19.0%) was the most common type of cancer among study participants followed by head and neck (18.4%), lymphoma/leukemia (11.5%), lung (11.5%), and breast (10.9%); 31.6% had metastatic disease and 14.9% had recurrent disease. Chemotherapy (42.5%) was the most common treatment modality, followed by surgery (38.5%) and radiation (31.6%). The sample was distributed among the 3 distress DT categories: mild (18.4%), moderate (42.5%), and severe (39.1%).

Problems Endorsed

Treatment decisions (44.3%) and insurance/financial concerns (35.1%) were the most frequently endorsed practical problems (Figure 1). Family health issues (33.9%) and dealing with partner (23.0%) were the most frequently endorsed family problems (Figure 2). Worry (73.0%) and depression (69.5%) were the most frequent emotional problems; of note, all emotional problems were endorsed by at least 50% of veterans (Figure 3). Fatigue (71.3%), sleep (70.7%), and pain (69%), were the most frequently endorsed physical problems (Figure 4). Spiritual/religious problems were endorsed by 15% of veterans.

Suicidal Ideation

Overall, 25.3% of veterans endorsed SI. About 20% of veterans reported a history of ≥ 1 suicide attempts in their lifetime. A significant relationship among distress categories and SI was found (χ² = 18.36, P < .001). Veterans with severe distress were more likely to endorse SI (42.7%) when compared with veterans with mild (9.4%) or moderate (16.2%) distress.

Similarly, a significant relationship among distress categories and a history of a suicide attempt(s) was found (χ² = 6.08, P = .048). Veterans with severe distress were more likely to have attempted suicide (29.4%) when compared with veterans with mild (12.5%) or moderate (14.9%) distress.

Discussion

This study aimed to understand factors associated with low, moderate, and severe levels of distress in veterans living with cancer who were referred for psychology services. As hypothesized, veterans who endorsed severe distress were significantly more likely to endorse SI. They also were more likely to have a history of a suicide attempt(s) when compared with those with mild or moderate distress.

A second aim of this study was to understand the most commonly endorsed problems. Consistent with prior literature, treatment decisions were the most commonly endorsed practical problem; worry and depression were the most common emotional problems; and fatigue, sleep, and pain were the most common physical problems.⁷

A finding unique to the current study is that family health issues and dealing with partner were specified as the most common family problems. However, a study by Smith and colleagues did not provide information about the rank of most frequently reported problems within this domain.⁷

The third aim was to understand which problems were related to SI. It was hypothesized that family, emotional, and physical problems would be related to SI. However, results indicated that only family problems (specifically, problems dealing with a partner) were significantly associated with SI among veterans living with cancer.

Contrary to expectations, emotional and physical problems were not found to have a significant relationship with SI. This is likely because veterans endorsed items nested within these problem domains with similar frequency. The lack of significant findings does not suggest that emotional and physical problems are not significant predictors of SI for veterans living with cancer, but that no specific emotional or physical symptom stood out as a predictor of suicidal ideation above the others.

The finding of a significant relationship between family problems (specifically, problems dealing with a partner) and SI in this study is consistent with findings of Aboumrad and colleagues in a study that examined root-cause analyses of completed suicides by veterans living with cancer.¹¹ They found that nearly half the sample endorsed family problems prior to their death, and a small but notable percentage of veterans who completed suicide reported divorce as a stressor prior to their death.

This finding may be explained by Thomas Joiner's interpersonal-psychological theory of suicidal behavior (IPT), which suggests that completed suicide may result from a thwarted sense of belonging, perceived burdensomeness, and acquired capability for suicide.¹⁶ Problems dealing with a partner may impact a veteran’s sense of belonging or social connectedness. Problems dealing with a partner also may be attributed to perceived burdens due to limitations imposed by living with cancer and/or undergoing treatment. In both circumstances, the veteran’s social support system may be negatively impacted, and perceived social support is a well-established protective factor against suicide.¹⁷

Partner distress is a second consideration. It is likely that veterans’ partners experienced their own distress in response to the veteran’s cancer diagnosis and/or treatment. The partner’s cause, severity, and expression of distress may contribute to problems for the couple.

Finally, the latter point of the IPT refers to acquired capability, or the ability to inflict deadly harm to oneself.¹⁸ A military sample was found to have more acquired capability for suicide when compared with a college undergraduate sample.¹⁹ A history of a suicide attempt(s) and male gender have been found to significantly predict acquired capability to complete suicide.¹⁸ Furthermore, because veterans living with cancer often are in pain, fear of pain associated with suicide may be reduced and, therefore, acquired capability increased. This suggests that male veterans living with cancer who are in pain, have a history of a suicide attempt(s), and current problems with their partner may be an extremely vulnerable population at-risk for suicide. Results from the current study emphasize the importance of veterans having access to mental health and crisis resources for problems dealing with their partner. Partner problems may foreshadow a potentially lethal type of distress.

Strengths

This study’s aims are consistent with the VA’s mission to end veteran suicide and contributes to literature in several important ways.¹² First, veterans living with cancer are an understudied population. The current study addresses a gap in existing literature by researching veterans living with cancer and aims to better understand the relationship between cancer-related distress and SI. Second, to the best of the authors’ knowledge, this study is the first to find that problems dealing with a partner significantly increases a veteran’s risk for SI above a history of a suicide attempt(s). Risk assessments now may be more comprehensive through inclusion of this distress factor.

It is recommended that future research use IPT to further investigate the relationship between problems dealing with a partner and SI.¹⁶ Future research may do so by including specific measures to assess for the tenants of the theory, including measurements of burdensomeness and belongingness. An expanded knowledge base about what makes problems dealing with a partner a significant suicide risk factor (eg, increased conflict, lack of support, etc.) would better enable clinicians to intervene effectively. Effective intervention may lessen suicidal behaviors or deaths from suicides within the Veteran population.

Limitations

One limitation is the focus on patients who accepted a mental health referral. This study design may limit the generalizability of results to veterans who would not accept mental health treatment. The homogenous sample of veterans is a second limitation. Most participants were male, white, and had a mean age of 62 years. These demographics are representative of the veterans that most typically utilize VA services; however, more research is needed on veterans living with cancer who are female and of diverse racial and ethnic backgrounds. There are likely differences in problems endorsed and factors associated with SI based on age, race, sex, and other socioeconomic factors. A third limitation is the cross-sectional, retrospective nature of this study. Future studies are advised to assess for distress at multiple time points. This is consistent with NCCN Standards of Care for Distress Management.² Longitudinal data would enable more findings about distress and SI throughout the course of cancer diagnosis and treatment, therefore enhancing clinical implications and informing future research.

Conclusion

This is among the first of studies to investigate distress and factors associated with SI in veterans living with cancer who were referred for psychology services. The prevalence of distress caused by psychosocial factors (including treatment decisions, worry, and depression) highlights the importance of including mental health services as part of comprehensive cancer treatment.

Distress due to treatment decisions may be attributed to a litany of factors such as a veteran’s consideration of adverse effects, effectiveness of treatments, changes to quality of life or functioning, and inclusion of alternative or complimentary treatments. These types of decisions often are reported to be difficult conversations to have with family members or loved ones, who are likely experiencing distress of their own. The role of a mental health provider to assist veterans in exploring their treatment decisions and the implications of such decisions appears important to lessening distress.

Early intervention for emotional symptoms would likely benefit veterans’ management of distress and may lessen suicide risk as depression is known to place veterans at-risk for SI.²⁰ This underscores the importance of timely distress assessment to prevent mild emotional distress from progressing to potentially severe or life-threatening emotional distress. For veterans with a psychiatric history, timely assessment and intervention is essential because psychiatric history is an established suicide risk factor that may be exacerbated by cancer-related distress.¹²

Furthermore, management of intolerable physical symptoms may lessen risk for suicide.⁴ Under medical guidance, fatigue may be improved using exercise.²¹ Behavioral intervention is commonly used as first-line treatment for sleep problems.²² While pain may be lessened through medication or nonpharmacological interventions.²³

Considering the numerous ways that distress may present itself (eg, practical, emotional, or physical) and increase risk for SI, it is essential that all veterans living with cancer are assessed for distress and SI, regardless of their presentation. Although veterans may not outwardly express distress, this does not indicate the absence of either distress or risk for suicide. For example, a veteran may be distressed due to financial concerns, transportation issues, and the health of his/her partner or spouse. This veteran may not exhibit visible symptoms of distress, as would be expected when the source of distress is emotional (eg, depression, anxiety). However, this veteran is equally vulnerable to impairing distress and SI as someone who exhibits emotional distress. Distress assessments should be further developed to capture both the visible and less apparent sources of distress, while also serving the imperative function of screening for suicide. Other researchers also have noted the necessity of this development.²⁴ Currently, the NCCN DT and Problems List does not include any assessment of SI or behavior.

Finally, this study identified a potentially critical factor to include in distress assessment: problems dealing with a partner. Problems dealing with a partner have been noted as a source of distress in existing literature, but this is the first study to find problems dealing with a partner to be a predictor of SI in veterans living with cancer.^4-6

Because partners often attend appointments with veterans, it is not surprising that problems dealing with their partner are not disclosed more readily. It is recommended that clinicians ask veterans about potential problems with their partner when they are alone. Directly gathering information about such problems while assessing for distress may assist health care workers in providing the most effective, accurate type of intervention in a timely manner, and potentially mitigate risk for suicide.

As recommended by the NCCN and numerous researchers, findings from the current study underscore the importance of accurate, timely assessment of distress.^2,4,8 This study makes several important recommendations about how distress assessment may be strengthened and further developed, specifically for the veteran population. This study also expands the current knowledge base of what is known about veterans living with cancer, and has begun to fill a gap in the existing literature. Consistent with the VA mission to end veteran suicide, results suggest that veterans living with cancer should be regularly screened for distress, asked about distress related to their partner, and assessed for SI. Continued efforts to enhance assessment of and response to distress may lessen suicide risk in veterans with cancer.¹¹

Acknowledgements

This study is the result of work supported with resources and the use of facilities at the James A. Haley Veterans’ Hospital.

It was estimated that physicians would diagnose a form of invasive cancer > 1.7 million times in 2019. As the second most common cause of death in the US, > 600,000 people were projected to die from cancer in 2019.¹ Many individuals with cancer endure distress, which the National Comprehensive Cancer Network (NCCN) defines as a “multifactorial unpleasant experience of a psychological (ie, cognitive, behavioral, emotional), social, spiritual, and/or physical nature that may interfere with the ability to cope effectively with cancer, its physical symptoms, and its treatment.”^2,3 Distress in people living with cancer has been attributed to various psychosocial concerns, such as family problems, whichinclude dealing with partners and children; emotional problems, such as depression and anxiety; and physical symptoms, such as pain and fatigue.^4-9 Certain factors associated with distress may increase a patient’s risk for suicide.⁴

Veterans are at particularly high risk for suicide.¹⁰ In 2014, veterans accounted for 18% of completed suicides in the US but only were 8.5% of the total population that same year.¹⁰ Yet, little research has been done on the relationship between distress and suicide in veterans living with cancer. Aboumrad and colleagues found that 45% of veterans with cancer who completed suicide reported family issues and 41% endorsed chronic pain.¹¹ This study recommended continued efforts to assess and treat distress to lessen risk of suicide in veterans living with cancer; however, to date, only 1 study has specifically evaluated distress and problems endorsed among veterans living with cancer.⁷

Suicide prevention is of the highest priority to the US Department of Veterans Affairs (VA).¹² Consistent with the VA mission to end veteran suicide, the current study aimed to better understand the relationship between distress and suicide within a sample of veterans living with cancer. Findings would additionally be used to tailor clinical assessments and interventions for veterans living with cancer.

This study had 3 primary goals. First, we sought to understand demographic and clinical factors associated with low, moderate, and severe levels of distress in veterans living with cancer who were referred for psychology services. Second, the study investigated the most commonly endorsed problems by veterans living with cancer. Finally, we examined which problems were related to suicidal ideation (SI). It was hypothesized that veterans who reported severe distress would be significantly more likely to endorse SI when compared with veterans who reported mild or moderate distress. Based on existing literature, it was further hypothesized that family, emotional, and physical problems would be significantly associated with SI.^7,11

Methods

The current study was conducted at James A. Haley Veterans’ Hospital (JAHVH) in Tampa, Florida. Inclusion criteria included veterans who were diagnosed with cancer, attended an outpatient psychology-oncology evaluation, and completed mental health screening measures provided during their evaluation. Exclusion criteria included veterans who: were seen in response to an inpatient consult, were seen solely for a stem cell transplant evaluation, or did not complete the screening measures.

Measures

A veteran’s demographic (eg, age, sex, ethnicity) and clinical (eg, cancer type, stage of disease, recurrence, cancer treatments received) information was abstracted from their VA medical records. Marital status was assessed during a clinical interview and documented as part of the standardized suicide risk assessment.

The Distress Thermometer (DT) is a subjective measure developed by the NCCN.² The DT provides a visual representation of a thermometer and asks patients to rate their level of distress over the past week with 0 indicating no distress and 10 indicating extreme distress. A distress rating of 4 or higher is clinically significant.^4,6 Distress may be categorized into 3 levels of severity: mild distress (< 4), moderate distress (4-7), or severe distress (8-10). The DT has been found to have good face validity, sensitivity and specificity, and is user-friendly.^2,6,7,13

The measurement additionally lists 39 problems nested within 5 domains: practical, family, emotional, spiritual/religious, and physical. Patients may endorse listed items under each problem domain by indicating yes or no. Endorsement of various items are intended to provide more detailed information about sources of distress. Due to the predominantly male and mostly older population included in this study the ability to have children measure was removed from the family problem domain.

SI was assessed in 2 ways. First, by patients’ self-report through item-9 of the Patient Health Questionnaire-9 (PHQ-9).¹⁴ Item-9 asks “over the last 2 weeks, how often have you been bothered by thoughts that you would be better off dead or of hurting yourself in some way?” Responses range from 0 (not at all) to 3 (nearly every day).¹⁴ Responses > 0 were considered a positive screen for SI. The process of administering the PHQ-9 item-9 is part of a national VA directive for standardizing assessment of suicide risk (Steve Young, personal communication, May 23, 2018). The PHQ-9 has been found to have good construct validity when used with both medical samples and the general population along with good internal and test-retest reliability.^14,15 Second, all veterans also were asked directly about SI during clinical interview, the results of which were documented in health records using a standardized format for risk assessment.

Procedure

Participants were a sample of veterans who were referred for psychology-oncology services. The NCCN DT and Problems List were administered prior to the start of clinical interviews, which followed a checklist and included standardized assessments of SI and history of a suicide attempt(s). A licensed clinical psychologist or a postdoctoral resident conducted these assessments under the supervision of a licensed psychologist. Data gathered during the clinical interview and from the DT and problems list were documented in health records, which were retrospectively reviewed for relevant information (eg, cancer diagnosis, SI). Therefore, informed consent was waived. This study was approved by the JAHVH Institutional Review Board.

Analysis

Data were analyzed using SPSS Version 25. Data analysis proceeded in 3 steps. First, descriptive statistics included the demographic and clinical factors present in the current sample. Difference between those with and without suicidal ideation were compared using F-statistic for continuous variables and χ² analyses for categorical variables. Second, to examine relationships between each DT problem domain and SI, χ² analyses were conducted. Third, DT problem domains that had a significant relationship with SI were entered in a logistic regression. This analysis determined which items, if any, from a DT problem domain predicted SI. In the logistic regression model, history of suicide attempts was entered into the first block, as history of suicide attempts is a well-established risk factor for subsequent suicidal ideation. In the second block, other variables that were significantly related to suicidal ideation in the second step of analyses were included. Before interpreting the results of the logistic regression, model fit was tested using the Hosmer-Lemeshow test. Significance of each individual predictor variable in the model is reported using the Wald χ² statistic; each Wald statistic is compared with a χ² distribution with 1 degree of freedom (df). Results of logistic regression models also provide information about the effect of each predictor variable in the regression equation (beta weight), odds a veteran who endorsed each predictor variable in the model would also endorse SI (as indicated by the odds ratio), and an estimate of the amount of variance accounted for by each predictor variable (using Nagelkerke’s pseudo R², which ranges in value from 0 to 1 with higher values indicating more variance explained). For all analyses, P value of .05 (2-tailed) was used for statistical significance.

Results

The sample consisted of 174 veterans (Table 1). The majority (77.6%) were male with a mean age of nearly 62 years (range, 29-87). Most identified as white (74.1%) with half reporting they were either married or living with a partner.

Prostate cancer (19.0%) was the most common type of cancer among study participants followed by head and neck (18.4%), lymphoma/leukemia (11.5%), lung (11.5%), and breast (10.9%); 31.6% had metastatic disease and 14.9% had recurrent disease. Chemotherapy (42.5%) was the most common treatment modality, followed by surgery (38.5%) and radiation (31.6%). The sample was distributed among the 3 distress DT categories: mild (18.4%), moderate (42.5%), and severe (39.1%).

Problems Endorsed

Treatment decisions (44.3%) and insurance/financial concerns (35.1%) were the most frequently endorsed practical problems (Figure 1). Family health issues (33.9%) and dealing with partner (23.0%) were the most frequently endorsed family problems (Figure 2). Worry (73.0%) and depression (69.5%) were the most frequent emotional problems; of note, all emotional problems were endorsed by at least 50% of veterans (Figure 3). Fatigue (71.3%), sleep (70.7%), and pain (69%), were the most frequently endorsed physical problems (Figure 4). Spiritual/religious problems were endorsed by 15% of veterans.

Suicidal Ideation

Overall, 25.3% of veterans endorsed SI. About 20% of veterans reported a history of ≥ 1 suicide attempts in their lifetime. A significant relationship among distress categories and SI was found (χ² = 18.36, P < .001). Veterans with severe distress were more likely to endorse SI (42.7%) when compared with veterans with mild (9.4%) or moderate (16.2%) distress.

Similarly, a significant relationship among distress categories and a history of a suicide attempt(s) was found (χ² = 6.08, P = .048). Veterans with severe distress were more likely to have attempted suicide (29.4%) when compared with veterans with mild (12.5%) or moderate (14.9%) distress.

Discussion

This study aimed to understand factors associated with low, moderate, and severe levels of distress in veterans living with cancer who were referred for psychology services. As hypothesized, veterans who endorsed severe distress were significantly more likely to endorse SI. They also were more likely to have a history of a suicide attempt(s) when compared with those with mild or moderate distress.

A second aim of this study was to understand the most commonly endorsed problems. Consistent with prior literature, treatment decisions were the most commonly endorsed practical problem; worry and depression were the most common emotional problems; and fatigue, sleep, and pain were the most common physical problems.⁷

A finding unique to the current study is that family health issues and dealing with partner were specified as the most common family problems. However, a study by Smith and colleagues did not provide information about the rank of most frequently reported problems within this domain.⁷

The third aim was to understand which problems were related to SI. It was hypothesized that family, emotional, and physical problems would be related to SI. However, results indicated that only family problems (specifically, problems dealing with a partner) were significantly associated with SI among veterans living with cancer.

Contrary to expectations, emotional and physical problems were not found to have a significant relationship with SI. This is likely because veterans endorsed items nested within these problem domains with similar frequency. The lack of significant findings does not suggest that emotional and physical problems are not significant predictors of SI for veterans living with cancer, but that no specific emotional or physical symptom stood out as a predictor of suicidal ideation above the others.

The finding of a significant relationship between family problems (specifically, problems dealing with a partner) and SI in this study is consistent with findings of Aboumrad and colleagues in a study that examined root-cause analyses of completed suicides by veterans living with cancer.¹¹ They found that nearly half the sample endorsed family problems prior to their death, and a small but notable percentage of veterans who completed suicide reported divorce as a stressor prior to their death.

This finding may be explained by Thomas Joiner's interpersonal-psychological theory of suicidal behavior (IPT), which suggests that completed suicide may result from a thwarted sense of belonging, perceived burdensomeness, and acquired capability for suicide.¹⁶ Problems dealing with a partner may impact a veteran’s sense of belonging or social connectedness. Problems dealing with a partner also may be attributed to perceived burdens due to limitations imposed by living with cancer and/or undergoing treatment. In both circumstances, the veteran’s social support system may be negatively impacted, and perceived social support is a well-established protective factor against suicide.¹⁷

Partner distress is a second consideration. It is likely that veterans’ partners experienced their own distress in response to the veteran’s cancer diagnosis and/or treatment. The partner’s cause, severity, and expression of distress may contribute to problems for the couple.

Finally, the latter point of the IPT refers to acquired capability, or the ability to inflict deadly harm to oneself.¹⁸ A military sample was found to have more acquired capability for suicide when compared with a college undergraduate sample.¹⁹ A history of a suicide attempt(s) and male gender have been found to significantly predict acquired capability to complete suicide.¹⁸ Furthermore, because veterans living with cancer often are in pain, fear of pain associated with suicide may be reduced and, therefore, acquired capability increased. This suggests that male veterans living with cancer who are in pain, have a history of a suicide attempt(s), and current problems with their partner may be an extremely vulnerable population at-risk for suicide. Results from the current study emphasize the importance of veterans having access to mental health and crisis resources for problems dealing with their partner. Partner problems may foreshadow a potentially lethal type of distress.

Strengths

This study’s aims are consistent with the VA’s mission to end veteran suicide and contributes to literature in several important ways.¹² First, veterans living with cancer are an understudied population. The current study addresses a gap in existing literature by researching veterans living with cancer and aims to better understand the relationship between cancer-related distress and SI. Second, to the best of the authors’ knowledge, this study is the first to find that problems dealing with a partner significantly increases a veteran’s risk for SI above a history of a suicide attempt(s). Risk assessments now may be more comprehensive through inclusion of this distress factor.

It is recommended that future research use IPT to further investigate the relationship between problems dealing with a partner and SI.¹⁶ Future research may do so by including specific measures to assess for the tenants of the theory, including measurements of burdensomeness and belongingness. An expanded knowledge base about what makes problems dealing with a partner a significant suicide risk factor (eg, increased conflict, lack of support, etc.) would better enable clinicians to intervene effectively. Effective intervention may lessen suicidal behaviors or deaths from suicides within the Veteran population.

Limitations

One limitation is the focus on patients who accepted a mental health referral. This study design may limit the generalizability of results to veterans who would not accept mental health treatment. The homogenous sample of veterans is a second limitation. Most participants were male, white, and had a mean age of 62 years. These demographics are representative of the veterans that most typically utilize VA services; however, more research is needed on veterans living with cancer who are female and of diverse racial and ethnic backgrounds. There are likely differences in problems endorsed and factors associated with SI based on age, race, sex, and other socioeconomic factors. A third limitation is the cross-sectional, retrospective nature of this study. Future studies are advised to assess for distress at multiple time points. This is consistent with NCCN Standards of Care for Distress Management.² Longitudinal data would enable more findings about distress and SI throughout the course of cancer diagnosis and treatment, therefore enhancing clinical implications and informing future research.

Conclusion

This is among the first of studies to investigate distress and factors associated with SI in veterans living with cancer who were referred for psychology services. The prevalence of distress caused by psychosocial factors (including treatment decisions, worry, and depression) highlights the importance of including mental health services as part of comprehensive cancer treatment.

Distress due to treatment decisions may be attributed to a litany of factors such as a veteran’s consideration of adverse effects, effectiveness of treatments, changes to quality of life or functioning, and inclusion of alternative or complimentary treatments. These types of decisions often are reported to be difficult conversations to have with family members or loved ones, who are likely experiencing distress of their own. The role of a mental health provider to assist veterans in exploring their treatment decisions and the implications of such decisions appears important to lessening distress.

Early intervention for emotional symptoms would likely benefit veterans’ management of distress and may lessen suicide risk as depression is known to place veterans at-risk for SI.²⁰ This underscores the importance of timely distress assessment to prevent mild emotional distress from progressing to potentially severe or life-threatening emotional distress. For veterans with a psychiatric history, timely assessment and intervention is essential because psychiatric history is an established suicide risk factor that may be exacerbated by cancer-related distress.¹²

Furthermore, management of intolerable physical symptoms may lessen risk for suicide.⁴ Under medical guidance, fatigue may be improved using exercise.²¹ Behavioral intervention is commonly used as first-line treatment for sleep problems.²² While pain may be lessened through medication or nonpharmacological interventions.²³

Considering the numerous ways that distress may present itself (eg, practical, emotional, or physical) and increase risk for SI, it is essential that all veterans living with cancer are assessed for distress and SI, regardless of their presentation. Although veterans may not outwardly express distress, this does not indicate the absence of either distress or risk for suicide. For example, a veteran may be distressed due to financial concerns, transportation issues, and the health of his/her partner or spouse. This veteran may not exhibit visible symptoms of distress, as would be expected when the source of distress is emotional (eg, depression, anxiety). However, this veteran is equally vulnerable to impairing distress and SI as someone who exhibits emotional distress. Distress assessments should be further developed to capture both the visible and less apparent sources of distress, while also serving the imperative function of screening for suicide. Other researchers also have noted the necessity of this development.²⁴ Currently, the NCCN DT and Problems List does not include any assessment of SI or behavior.

Finally, this study identified a potentially critical factor to include in distress assessment: problems dealing with a partner. Problems dealing with a partner have been noted as a source of distress in existing literature, but this is the first study to find problems dealing with a partner to be a predictor of SI in veterans living with cancer.^4-6

Because partners often attend appointments with veterans, it is not surprising that problems dealing with their partner are not disclosed more readily. It is recommended that clinicians ask veterans about potential problems with their partner when they are alone. Directly gathering information about such problems while assessing for distress may assist health care workers in providing the most effective, accurate type of intervention in a timely manner, and potentially mitigate risk for suicide.

As recommended by the NCCN and numerous researchers, findings from the current study underscore the importance of accurate, timely assessment of distress.^2,4,8 This study makes several important recommendations about how distress assessment may be strengthened and further developed, specifically for the veteran population. This study also expands the current knowledge base of what is known about veterans living with cancer, and has begun to fill a gap in the existing literature. Consistent with the VA mission to end veteran suicide, results suggest that veterans living with cancer should be regularly screened for distress, asked about distress related to their partner, and assessed for SI. Continued efforts to enhance assessment of and response to distress may lessen suicide risk in veterans with cancer.¹¹

Acknowledgements

This study is the result of work supported with resources and the use of facilities at the James A. Haley Veterans’ Hospital.

Chronic lymphocytic leukemia (CLL) is the most common leukemia diagnosed in developed countries, with an estimated 21,040 new diagnoses of CLL expected in the US in 2020. ^1-3 CLL is an indolent cancer characterized by the accumulation of B-lymphocytes in the blood, marrow, and lymphoid tissues. ⁴ It has a heterogeneous clinical course; the majority of patients are observed or receive delayed treatment following diagnosis, while a minority of patients require immediate treatment. After first-line treatment, some patients experience prolonged remissions while others require retreatment within 1 or 2 years. Fortunately, advances in cancer biology and therapeutics in the last decade have increased the number of treatment options available for patients with CLL.

Until recently, most CLL treatments relied on a chemotherapy or a chemoimmunotherapy backbone; however, the last few years have seen novel therapies introduced, such as small molecule inhibitors to target molecular pathways that promote the normal development, expansion, and survival of B-cells.⁵ One such therapy is ibrutinib, a targeted Bruton tyrosine kinase inhibitor that received accelerated approval by the US Food and Drug Administration (FDA) in February 2014 for patients with CLL who received at least 1 prior therapy. The FDA later expanded this approval to include use of ibrutinib in patients with CLL with relapsed or refractory disease, with or without chromosome 17p deletion. In 2016, based on data from the RESONATE-17 study, the FDA approved ibrutinib for first-line therapy in patients with CLL.⁶

Ibrutinib’s efficacy, ease of administration and dosing (all doses are oral and fixed, rather than based on weight or body surface area), and relatively favorable safety profile have resulted in a rapid growth in its adoption.⁷ Since its adverse event (AE) profile is generally more tolerable than that of a typical chemoimmunotherapy, its use in older patients with CLL and patients with significant comorbidities is particularly appealing.⁸

However, the results of some clinical trials suggest an association between treatment with ibrutinib and an increased risk of bleeding-related events of any grade (44%) and major bleeding events (4%).^7,8 The incidence of major bleeding events was reported to be higher (9%) in one clinical trial and at 5-year follow-up, although this trial did not exclude patients receiving concomitant oral anticoagulation with warfarin.^6,9

Heterogeneity in clinical trials’ definitions of major bleeding confounded the ability to calculate bleeding risk in patients treated with ibrutinib in a systematic review and meta-analysis that called for more data.¹⁰ Additionally, patients with factors that might increase the risk of major bleeding with ibrutinib treatment were likely underrepresented in clinical trials, given the carefully selected nature of clinical trial subjects. These factors include renal or hepatic disease, gastrointestinal disease, and use of a number of concomitant medications such as antiplatelets or anticoagulant medications. Accounting for use of the latter is particularly important because patients who develop atrial fibrillation (Afib), one of the recognized AEs of treatment with ibrutinib, often are treated with anticoagulant medications in order to decrease the risk of stroke or other thromboembolic complications.

Methods

The VHA maintains the centralized US Department of Veterans Affairs Cancer Registry System (VACRS), with electronic medical record data and other sources captured in its Corporate Data Warehouse (CDW). The VHA CDW is a national repository comprising data from several VHA clinical and administrative systems. The CDW includes patient identifiers; demographics; vital status; lab information; administrative information (such as diagnostic International Statistical Classification of Diseases and Related Health Problems [ICD-9] codes); medication dispensation tables (such as outpatient fill); IV package information; and notes from radiology, pathology, outpatient and inpatient admission, discharge, and daily progress.

Registrars abstract all cancer cases within the VHA system (or diagnosed outside the VHA, if patients subsequently receive treatment in the VHA). It is estimated that VACRS captures 3% of cancer cases in the US.¹² Like most registries, VACRS captures data such as diagnosis, age, gender, race, and vital status.

Study Definitions

Prevalence of comorbidities that could increase bleeding risk was determined using administrative ICD-9-CM codes. Liver disease was identified by presence of cirrhosis, hepatitis C virus, or alcoholic liver disease using administrative codes validated by Kramer and colleagues, who reported positive and negative predictive values of 90% and 87% for cirrhosis, 93% and 92% for hepatitis C virus, and 71% and 98% for alcoholic liver disease.¹³ Similarly, end-stage liver disease was identified using a validated coding algorithm developed by Goldberg and colleagues, with a positive predictive value of 89.3%.¹⁴ The presence of controlled or uncontrolled diabetes mellitus (DM) was identified using the procedure described by Guzman and colleagues.¹⁵ Quan’s algorithm was used to calculate Charlson Comorbidity Index (CCI) based on ICD-9-CM codes for inpatient and outpatient visits within a 6-month lookback period prior to treatment initiation.¹⁶

A major bleeding event was defined as a hospitalization with an ICD-9-CM code suggestive of major bleeding as the primary reason, as defined by Lane and colleagues in their study of major bleeding related to warfarin in a cohort of patients treated within the VHA.¹⁷ Incidence rates of major bleeding events were identified during the first 6 months of treatment. Incidence of Afib—defined as an inpatient or outpatient encounter with the 427.31 ICD-9-CM code—also was examined within the first 6 months after starting treatment. The period of 6 months was chosen because bendamustine must be discontinued after 6 months.

Study Analysis

Descriptive statistics were used to examine patient demographics, disease characteristics, and treatment history from initial CLL diagnosis through end of study observation period. Categorical variables were summarized using frequencies and accompanying proportions, while a mean and standard deviation were used to summarize continuous variables. For the means of continuous variables and of categorical data, 95% CIs were used. Proportions and accompanying 95% CIs characterized treatment patterns, including line of therapy, comorbidities, and bleeding events. Treatment duration was described using mean and accompanying 95% CI. Statistical tests were not conducted for comparisons among treatment groups. Patients were censored at the end of follow-up, defined as the earliest of the following scenarios: (1) end of study observation period (December 31, 2016); (2) development of a secondary cancer; or (3) last day of contact given absence of care within the VHA for ≥ 18 months (with care defined as oncology and/or oncology/hematology visit with an associated note). Analysis was performed using R 3.4.0.

Results

Between 2010 and 2014, 2,796 patients were diagnosed and received care for CLL within the VHA. Overall, all 172 patients who were treated with ibrutinib during our inclusion period were selected. These patients were treated between January 1, 2014 and December 31, 2016, following ibrutinib’s approval in early 2014. An additional 291 patients were selected who received BR (Table). Reflecting the predominantly male population of the VHA, 282 (97%) BR patients and 167 (97%) ibrutinib patients were male. The median age at diagnosis was 67 years for BR patients and 69 years for ibrutinib patients. About 76% of patients who received ibrutinib and 82% of patients who received BR were non-Hispanic white; 17% and 14% were African American, respectively.

Less than 10% of patients receiving either ibrutinib or BR had liver disease per criteria used by Kramer and colleagues, or end-stage liver disease using criteria developed by Goldberg and colleagues.^12,13 About 5% of patients had a history of previous bleeding in the 6-month period prior to initiating either therapy. Mean CCI (excluding malignancy) score was 1.5 (range, 0-11) for the ibrutinib group, and 2.1 (range, 0-9) for the BR group. About 16% of the ibrutinib group had controlled DM and fewer than 10% had uncontrolled DM, while 4% of patients in the BR group met the criteria for controlled DM and another 4% met the criteria for uncontrolled DM.

There was very low utilization of anticoagulant or antiplatelet medication prior to initiation of ibrutinib (2.9% and 2.3%, respectively) or BR (< 1% each). In the first 6 months after treatment initiation, about 8% of patients in both ibrutinib and BR cohorts received anticoagulant medication while antiplatelet utilization was < 5% in either group.

In the BR group, 8 patients (2.7%) experienced a major bleeding event, while 14 patients (8.1%) in the ibrutinib group experienced a bleeding event (P = .008). While these numbers were too low to perform a formal statistical analysis of the association between clinical covariates and bleeding in either group, there did not seem to be an association between bleeding and liver disease or DM. Of patients who experienced a bleeding event, about 1 in 4 patients had had a prior bleeding event in both the ibrutinib and the BR groups. Interestingly, while none of the patients who experienced a bleeding event while receiving BR were taking concomitant anticoagulant medication, 3 of the 14 patients who experienced a bleeding event in the ibrutinib group showed evidence of anticoagulant utilization. Finally, the incidence of Afib (defined as patients with no evidence of Afib in the 6 months prior to treatment but with evidence of Afib in the 6 months following treatment initiation) was 4% in the BR group, and about 8% in the ibrutinib group (P = .003).

Discussion

To the authors’ knowledge, this study is the first to examine the real-world incidence of bleeding and Afib in veterans who received ibrutinib for CLL in the first year of its availability. The study found minimal use of anticoagulants and/or antiplatelet agents prior to receiving first-line ibrutinib or BR, and very low use of these agents in the first 6 months following the initiation of first-line treatment. This finding suggests a high awareness among VA providers of potential adverse effects (AEs) of ibrutinib and chemotherapy, and a careful selection of patients that lack risk factors for AEs.

In patients treated with first-line ibrutinib when compared with patients treated with first-line BR, moderate increases in bleeding (2.7% vs 8.1%, P = .008) and Afib (10.5% vs 3%, P = .003) also were observed. These results are concordant with previous findings examining the use of ibrutinib in patients with CLL.^18-20

Limitations

The results of this study should be interpreted with caution, as some limitations must be considered. The study was conducted in the early days of ibrutinib adoption. Since then, more patients have been treated with ibrutinib and for longer durations. As clinicians gain more familiarity and with ibrutinib, and as additional novel therapeutics emerge, it is possible that the initial awareness about risks for possible AEs may diminish; patients with high comorbidity burdens and concomitant medications would be especially vulnerable in cases of reduced physician vigilance.

Another limitation of this study stems from the potential for dual system use among patients treated in the VHA. Concurrent or alternating use of multiple health care systems (use of VHA and private-sector facilities) may present gaps in the reconstruction of patient histories, resulting in missing data as patients transition between commercial, the Centers for Medicare and Medicaid Services, and VHA care. As a result, the results presented here do not reflect instances where a patient experienced a bleeding event treated outside the VA.

Problems with missing data also may occur due to incomplete extraction from the electronic health record; these issues were addressed by leveraging an understanding of the multiple data marts within the CDW environment to harmonize missing and/or erroneous information through use of other data marts when possible. Lastly, this research represents a population-level study of the VHA, thus all findings are directly relevant to the VHA. The generalizability of the findings outside the VHA would depend on the characteristics of the external population.

Conclusion

Real-world evidence from a nationwide cohort of veteran patients with CLL treated with ibrutinib suggest that, while there is an association of increased bleeding-related events and Afib, the risk is comparable to those reported in previous studies.^18-20 These findings suggest that patients in real-world clinical care settings with higher levels of comorbidities may be at a slight increased risk for bleeding events and Afib.

Chronic lymphocytic leukemia (CLL) is the most common leukemia diagnosed in developed countries, with an estimated 21,040 new diagnoses of CLL expected in the US in 2020. ^1-3 CLL is an indolent cancer characterized by the accumulation of B-lymphocytes in the blood, marrow, and lymphoid tissues. ⁴ It has a heterogeneous clinical course; the majority of patients are observed or receive delayed treatment following diagnosis, while a minority of patients require immediate treatment. After first-line treatment, some patients experience prolonged remissions while others require retreatment within 1 or 2 years. Fortunately, advances in cancer biology and therapeutics in the last decade have increased the number of treatment options available for patients with CLL.

Until recently, most CLL treatments relied on a chemotherapy or a chemoimmunotherapy backbone; however, the last few years have seen novel therapies introduced, such as small molecule inhibitors to target molecular pathways that promote the normal development, expansion, and survival of B-cells.⁵ One such therapy is ibrutinib, a targeted Bruton tyrosine kinase inhibitor that received accelerated approval by the US Food and Drug Administration (FDA) in February 2014 for patients with CLL who received at least 1 prior therapy. The FDA later expanded this approval to include use of ibrutinib in patients with CLL with relapsed or refractory disease, with or without chromosome 17p deletion. In 2016, based on data from the RESONATE-17 study, the FDA approved ibrutinib for first-line therapy in patients with CLL.⁶

Ibrutinib’s efficacy, ease of administration and dosing (all doses are oral and fixed, rather than based on weight or body surface area), and relatively favorable safety profile have resulted in a rapid growth in its adoption.⁷ Since its adverse event (AE) profile is generally more tolerable than that of a typical chemoimmunotherapy, its use in older patients with CLL and patients with significant comorbidities is particularly appealing.⁸

However, the results of some clinical trials suggest an association between treatment with ibrutinib and an increased risk of bleeding-related events of any grade (44%) and major bleeding events (4%).^7,8 The incidence of major bleeding events was reported to be higher (9%) in one clinical trial and at 5-year follow-up, although this trial did not exclude patients receiving concomitant oral anticoagulation with warfarin.^6,9

Heterogeneity in clinical trials’ definitions of major bleeding confounded the ability to calculate bleeding risk in patients treated with ibrutinib in a systematic review and meta-analysis that called for more data.¹⁰ Additionally, patients with factors that might increase the risk of major bleeding with ibrutinib treatment were likely underrepresented in clinical trials, given the carefully selected nature of clinical trial subjects. These factors include renal or hepatic disease, gastrointestinal disease, and use of a number of concomitant medications such as antiplatelets or anticoagulant medications. Accounting for use of the latter is particularly important because patients who develop atrial fibrillation (Afib), one of the recognized AEs of treatment with ibrutinib, often are treated with anticoagulant medications in order to decrease the risk of stroke or other thromboembolic complications.

Methods

The VHA maintains the centralized US Department of Veterans Affairs Cancer Registry System (VACRS), with electronic medical record data and other sources captured in its Corporate Data Warehouse (CDW). The VHA CDW is a national repository comprising data from several VHA clinical and administrative systems. The CDW includes patient identifiers; demographics; vital status; lab information; administrative information (such as diagnostic International Statistical Classification of Diseases and Related Health Problems [ICD-9] codes); medication dispensation tables (such as outpatient fill); IV package information; and notes from radiology, pathology, outpatient and inpatient admission, discharge, and daily progress.

Registrars abstract all cancer cases within the VHA system (or diagnosed outside the VHA, if patients subsequently receive treatment in the VHA). It is estimated that VACRS captures 3% of cancer cases in the US.¹² Like most registries, VACRS captures data such as diagnosis, age, gender, race, and vital status.

Study Definitions

Prevalence of comorbidities that could increase bleeding risk was determined using administrative ICD-9-CM codes. Liver disease was identified by presence of cirrhosis, hepatitis C virus, or alcoholic liver disease using administrative codes validated by Kramer and colleagues, who reported positive and negative predictive values of 90% and 87% for cirrhosis, 93% and 92% for hepatitis C virus, and 71% and 98% for alcoholic liver disease.¹³ Similarly, end-stage liver disease was identified using a validated coding algorithm developed by Goldberg and colleagues, with a positive predictive value of 89.3%.¹⁴ The presence of controlled or uncontrolled diabetes mellitus (DM) was identified using the procedure described by Guzman and colleagues.¹⁵ Quan’s algorithm was used to calculate Charlson Comorbidity Index (CCI) based on ICD-9-CM codes for inpatient and outpatient visits within a 6-month lookback period prior to treatment initiation.¹⁶

A major bleeding event was defined as a hospitalization with an ICD-9-CM code suggestive of major bleeding as the primary reason, as defined by Lane and colleagues in their study of major bleeding related to warfarin in a cohort of patients treated within the VHA.¹⁷ Incidence rates of major bleeding events were identified during the first 6 months of treatment. Incidence of Afib—defined as an inpatient or outpatient encounter with the 427.31 ICD-9-CM code—also was examined within the first 6 months after starting treatment. The period of 6 months was chosen because bendamustine must be discontinued after 6 months.

Study Analysis

Descriptive statistics were used to examine patient demographics, disease characteristics, and treatment history from initial CLL diagnosis through end of study observation period. Categorical variables were summarized using frequencies and accompanying proportions, while a mean and standard deviation were used to summarize continuous variables. For the means of continuous variables and of categorical data, 95% CIs were used. Proportions and accompanying 95% CIs characterized treatment patterns, including line of therapy, comorbidities, and bleeding events. Treatment duration was described using mean and accompanying 95% CI. Statistical tests were not conducted for comparisons among treatment groups. Patients were censored at the end of follow-up, defined as the earliest of the following scenarios: (1) end of study observation period (December 31, 2016); (2) development of a secondary cancer; or (3) last day of contact given absence of care within the VHA for ≥ 18 months (with care defined as oncology and/or oncology/hematology visit with an associated note). Analysis was performed using R 3.4.0.

Results

Between 2010 and 2014, 2,796 patients were diagnosed and received care for CLL within the VHA. Overall, all 172 patients who were treated with ibrutinib during our inclusion period were selected. These patients were treated between January 1, 2014 and December 31, 2016, following ibrutinib’s approval in early 2014. An additional 291 patients were selected who received BR (Table). Reflecting the predominantly male population of the VHA, 282 (97%) BR patients and 167 (97%) ibrutinib patients were male. The median age at diagnosis was 67 years for BR patients and 69 years for ibrutinib patients. About 76% of patients who received ibrutinib and 82% of patients who received BR were non-Hispanic white; 17% and 14% were African American, respectively.

Less than 10% of patients receiving either ibrutinib or BR had liver disease per criteria used by Kramer and colleagues, or end-stage liver disease using criteria developed by Goldberg and colleagues.^12,13 About 5% of patients had a history of previous bleeding in the 6-month period prior to initiating either therapy. Mean CCI (excluding malignancy) score was 1.5 (range, 0-11) for the ibrutinib group, and 2.1 (range, 0-9) for the BR group. About 16% of the ibrutinib group had controlled DM and fewer than 10% had uncontrolled DM, while 4% of patients in the BR group met the criteria for controlled DM and another 4% met the criteria for uncontrolled DM.

There was very low utilization of anticoagulant or antiplatelet medication prior to initiation of ibrutinib (2.9% and 2.3%, respectively) or BR (< 1% each). In the first 6 months after treatment initiation, about 8% of patients in both ibrutinib and BR cohorts received anticoagulant medication while antiplatelet utilization was < 5% in either group.

In the BR group, 8 patients (2.7%) experienced a major bleeding event, while 14 patients (8.1%) in the ibrutinib group experienced a bleeding event (P = .008). While these numbers were too low to perform a formal statistical analysis of the association between clinical covariates and bleeding in either group, there did not seem to be an association between bleeding and liver disease or DM. Of patients who experienced a bleeding event, about 1 in 4 patients had had a prior bleeding event in both the ibrutinib and the BR groups. Interestingly, while none of the patients who experienced a bleeding event while receiving BR were taking concomitant anticoagulant medication, 3 of the 14 patients who experienced a bleeding event in the ibrutinib group showed evidence of anticoagulant utilization. Finally, the incidence of Afib (defined as patients with no evidence of Afib in the 6 months prior to treatment but with evidence of Afib in the 6 months following treatment initiation) was 4% in the BR group, and about 8% in the ibrutinib group (P = .003).

Discussion

To the authors’ knowledge, this study is the first to examine the real-world incidence of bleeding and Afib in veterans who received ibrutinib for CLL in the first year of its availability. The study found minimal use of anticoagulants and/or antiplatelet agents prior to receiving first-line ibrutinib or BR, and very low use of these agents in the first 6 months following the initiation of first-line treatment. This finding suggests a high awareness among VA providers of potential adverse effects (AEs) of ibrutinib and chemotherapy, and a careful selection of patients that lack risk factors for AEs.

In patients treated with first-line ibrutinib when compared with patients treated with first-line BR, moderate increases in bleeding (2.7% vs 8.1%, P = .008) and Afib (10.5% vs 3%, P = .003) also were observed. These results are concordant with previous findings examining the use of ibrutinib in patients with CLL.^18-20

Limitations

The results of this study should be interpreted with caution, as some limitations must be considered. The study was conducted in the early days of ibrutinib adoption. Since then, more patients have been treated with ibrutinib and for longer durations. As clinicians gain more familiarity and with ibrutinib, and as additional novel therapeutics emerge, it is possible that the initial awareness about risks for possible AEs may diminish; patients with high comorbidity burdens and concomitant medications would be especially vulnerable in cases of reduced physician vigilance.

Another limitation of this study stems from the potential for dual system use among patients treated in the VHA. Concurrent or alternating use of multiple health care systems (use of VHA and private-sector facilities) may present gaps in the reconstruction of patient histories, resulting in missing data as patients transition between commercial, the Centers for Medicare and Medicaid Services, and VHA care. As a result, the results presented here do not reflect instances where a patient experienced a bleeding event treated outside the VA.

Problems with missing data also may occur due to incomplete extraction from the electronic health record; these issues were addressed by leveraging an understanding of the multiple data marts within the CDW environment to harmonize missing and/or erroneous information through use of other data marts when possible. Lastly, this research represents a population-level study of the VHA, thus all findings are directly relevant to the VHA. The generalizability of the findings outside the VHA would depend on the characteristics of the external population.

Conclusion

Real-world evidence from a nationwide cohort of veteran patients with CLL treated with ibrutinib suggest that, while there is an association of increased bleeding-related events and Afib, the risk is comparable to those reported in previous studies.^18-20 These findings suggest that patients in real-world clinical care settings with higher levels of comorbidities may be at a slight increased risk for bleeding events and Afib.

Chronic lymphocytic leukemia (CLL) is the most common leukemia diagnosed in developed countries, with an estimated 21,040 new diagnoses of CLL expected in the US in 2020. ^1-3 CLL is an indolent cancer characterized by the accumulation of B-lymphocytes in the blood, marrow, and lymphoid tissues. ⁴ It has a heterogeneous clinical course; the majority of patients are observed or receive delayed treatment following diagnosis, while a minority of patients require immediate treatment. After first-line treatment, some patients experience prolonged remissions while others require retreatment within 1 or 2 years. Fortunately, advances in cancer biology and therapeutics in the last decade have increased the number of treatment options available for patients with CLL.

Until recently, most CLL treatments relied on a chemotherapy or a chemoimmunotherapy backbone; however, the last few years have seen novel therapies introduced, such as small molecule inhibitors to target molecular pathways that promote the normal development, expansion, and survival of B-cells.⁵ One such therapy is ibrutinib, a targeted Bruton tyrosine kinase inhibitor that received accelerated approval by the US Food and Drug Administration (FDA) in February 2014 for patients with CLL who received at least 1 prior therapy. The FDA later expanded this approval to include use of ibrutinib in patients with CLL with relapsed or refractory disease, with or without chromosome 17p deletion. In 2016, based on data from the RESONATE-17 study, the FDA approved ibrutinib for first-line therapy in patients with CLL.⁶

Ibrutinib’s efficacy, ease of administration and dosing (all doses are oral and fixed, rather than based on weight or body surface area), and relatively favorable safety profile have resulted in a rapid growth in its adoption.⁷ Since its adverse event (AE) profile is generally more tolerable than that of a typical chemoimmunotherapy, its use in older patients with CLL and patients with significant comorbidities is particularly appealing.⁸

However, the results of some clinical trials suggest an association between treatment with ibrutinib and an increased risk of bleeding-related events of any grade (44%) and major bleeding events (4%).^7,8 The incidence of major bleeding events was reported to be higher (9%) in one clinical trial and at 5-year follow-up, although this trial did not exclude patients receiving concomitant oral anticoagulation with warfarin.^6,9

Heterogeneity in clinical trials’ definitions of major bleeding confounded the ability to calculate bleeding risk in patients treated with ibrutinib in a systematic review and meta-analysis that called for more data.¹⁰ Additionally, patients with factors that might increase the risk of major bleeding with ibrutinib treatment were likely underrepresented in clinical trials, given the carefully selected nature of clinical trial subjects. These factors include renal or hepatic disease, gastrointestinal disease, and use of a number of concomitant medications such as antiplatelets or anticoagulant medications. Accounting for use of the latter is particularly important because patients who develop atrial fibrillation (Afib), one of the recognized AEs of treatment with ibrutinib, often are treated with anticoagulant medications in order to decrease the risk of stroke or other thromboembolic complications.

Methods

The VHA maintains the centralized US Department of Veterans Affairs Cancer Registry System (VACRS), with electronic medical record data and other sources captured in its Corporate Data Warehouse (CDW). The VHA CDW is a national repository comprising data from several VHA clinical and administrative systems. The CDW includes patient identifiers; demographics; vital status; lab information; administrative information (such as diagnostic International Statistical Classification of Diseases and Related Health Problems [ICD-9] codes); medication dispensation tables (such as outpatient fill); IV package information; and notes from radiology, pathology, outpatient and inpatient admission, discharge, and daily progress.

Registrars abstract all cancer cases within the VHA system (or diagnosed outside the VHA, if patients subsequently receive treatment in the VHA). It is estimated that VACRS captures 3% of cancer cases in the US.¹² Like most registries, VACRS captures data such as diagnosis, age, gender, race, and vital status.

Study Definitions

Prevalence of comorbidities that could increase bleeding risk was determined using administrative ICD-9-CM codes. Liver disease was identified by presence of cirrhosis, hepatitis C virus, or alcoholic liver disease using administrative codes validated by Kramer and colleagues, who reported positive and negative predictive values of 90% and 87% for cirrhosis, 93% and 92% for hepatitis C virus, and 71% and 98% for alcoholic liver disease.¹³ Similarly, end-stage liver disease was identified using a validated coding algorithm developed by Goldberg and colleagues, with a positive predictive value of 89.3%.¹⁴ The presence of controlled or uncontrolled diabetes mellitus (DM) was identified using the procedure described by Guzman and colleagues.¹⁵ Quan’s algorithm was used to calculate Charlson Comorbidity Index (CCI) based on ICD-9-CM codes for inpatient and outpatient visits within a 6-month lookback period prior to treatment initiation.¹⁶

A major bleeding event was defined as a hospitalization with an ICD-9-CM code suggestive of major bleeding as the primary reason, as defined by Lane and colleagues in their study of major bleeding related to warfarin in a cohort of patients treated within the VHA.¹⁷ Incidence rates of major bleeding events were identified during the first 6 months of treatment. Incidence of Afib—defined as an inpatient or outpatient encounter with the 427.31 ICD-9-CM code—also was examined within the first 6 months after starting treatment. The period of 6 months was chosen because bendamustine must be discontinued after 6 months.

Study Analysis

Descriptive statistics were used to examine patient demographics, disease characteristics, and treatment history from initial CLL diagnosis through end of study observation period. Categorical variables were summarized using frequencies and accompanying proportions, while a mean and standard deviation were used to summarize continuous variables. For the means of continuous variables and of categorical data, 95% CIs were used. Proportions and accompanying 95% CIs characterized treatment patterns, including line of therapy, comorbidities, and bleeding events. Treatment duration was described using mean and accompanying 95% CI. Statistical tests were not conducted for comparisons among treatment groups. Patients were censored at the end of follow-up, defined as the earliest of the following scenarios: (1) end of study observation period (December 31, 2016); (2) development of a secondary cancer; or (3) last day of contact given absence of care within the VHA for ≥ 18 months (with care defined as oncology and/or oncology/hematology visit with an associated note). Analysis was performed using R 3.4.0.

Results

Between 2010 and 2014, 2,796 patients were diagnosed and received care for CLL within the VHA. Overall, all 172 patients who were treated with ibrutinib during our inclusion period were selected. These patients were treated between January 1, 2014 and December 31, 2016, following ibrutinib’s approval in early 2014. An additional 291 patients were selected who received BR (Table). Reflecting the predominantly male population of the VHA, 282 (97%) BR patients and 167 (97%) ibrutinib patients were male. The median age at diagnosis was 67 years for BR patients and 69 years for ibrutinib patients. About 76% of patients who received ibrutinib and 82% of patients who received BR were non-Hispanic white; 17% and 14% were African American, respectively.

Less than 10% of patients receiving either ibrutinib or BR had liver disease per criteria used by Kramer and colleagues, or end-stage liver disease using criteria developed by Goldberg and colleagues.^12,13 About 5% of patients had a history of previous bleeding in the 6-month period prior to initiating either therapy. Mean CCI (excluding malignancy) score was 1.5 (range, 0-11) for the ibrutinib group, and 2.1 (range, 0-9) for the BR group. About 16% of the ibrutinib group had controlled DM and fewer than 10% had uncontrolled DM, while 4% of patients in the BR group met the criteria for controlled DM and another 4% met the criteria for uncontrolled DM.

There was very low utilization of anticoagulant or antiplatelet medication prior to initiation of ibrutinib (2.9% and 2.3%, respectively) or BR (< 1% each). In the first 6 months after treatment initiation, about 8% of patients in both ibrutinib and BR cohorts received anticoagulant medication while antiplatelet utilization was < 5% in either group.

In the BR group, 8 patients (2.7%) experienced a major bleeding event, while 14 patients (8.1%) in the ibrutinib group experienced a bleeding event (P = .008). While these numbers were too low to perform a formal statistical analysis of the association between clinical covariates and bleeding in either group, there did not seem to be an association between bleeding and liver disease or DM. Of patients who experienced a bleeding event, about 1 in 4 patients had had a prior bleeding event in both the ibrutinib and the BR groups. Interestingly, while none of the patients who experienced a bleeding event while receiving BR were taking concomitant anticoagulant medication, 3 of the 14 patients who experienced a bleeding event in the ibrutinib group showed evidence of anticoagulant utilization. Finally, the incidence of Afib (defined as patients with no evidence of Afib in the 6 months prior to treatment but with evidence of Afib in the 6 months following treatment initiation) was 4% in the BR group, and about 8% in the ibrutinib group (P = .003).

Discussion

To the authors’ knowledge, this study is the first to examine the real-world incidence of bleeding and Afib in veterans who received ibrutinib for CLL in the first year of its availability. The study found minimal use of anticoagulants and/or antiplatelet agents prior to receiving first-line ibrutinib or BR, and very low use of these agents in the first 6 months following the initiation of first-line treatment. This finding suggests a high awareness among VA providers of potential adverse effects (AEs) of ibrutinib and chemotherapy, and a careful selection of patients that lack risk factors for AEs.

In patients treated with first-line ibrutinib when compared with patients treated with first-line BR, moderate increases in bleeding (2.7% vs 8.1%, P = .008) and Afib (10.5% vs 3%, P = .003) also were observed. These results are concordant with previous findings examining the use of ibrutinib in patients with CLL.^18-20

Limitations

The results of this study should be interpreted with caution, as some limitations must be considered. The study was conducted in the early days of ibrutinib adoption. Since then, more patients have been treated with ibrutinib and for longer durations. As clinicians gain more familiarity and with ibrutinib, and as additional novel therapeutics emerge, it is possible that the initial awareness about risks for possible AEs may diminish; patients with high comorbidity burdens and concomitant medications would be especially vulnerable in cases of reduced physician vigilance.

Another limitation of this study stems from the potential for dual system use among patients treated in the VHA. Concurrent or alternating use of multiple health care systems (use of VHA and private-sector facilities) may present gaps in the reconstruction of patient histories, resulting in missing data as patients transition between commercial, the Centers for Medicare and Medicaid Services, and VHA care. As a result, the results presented here do not reflect instances where a patient experienced a bleeding event treated outside the VA.

Problems with missing data also may occur due to incomplete extraction from the electronic health record; these issues were addressed by leveraging an understanding of the multiple data marts within the CDW environment to harmonize missing and/or erroneous information through use of other data marts when possible. Lastly, this research represents a population-level study of the VHA, thus all findings are directly relevant to the VHA. The generalizability of the findings outside the VHA would depend on the characteristics of the external population.

Conclusion

Real-world evidence from a nationwide cohort of veteran patients with CLL treated with ibrutinib suggest that, while there is an association of increased bleeding-related events and Afib, the risk is comparable to those reported in previous studies.^18-20 These findings suggest that patients in real-world clinical care settings with higher levels of comorbidities may be at a slight increased risk for bleeding events and Afib.

Lung cancer is the leading cause of cancer mortality both in the US and worldwide.¹ Many patients diagnosed with lung cancer present with advanced disease with thoracic symptoms such as cough, hemoptysis, dyspnea, and chest pain.^2-4 Palliative radiotherapy is routinely used in patients with locally advanced and metastatic lung cancer with the goal of relieving these symptoms and improving quality of life. Guidelines published by the American Society for Radiation Oncology (ASTRO) in 2011, and updated in 2018, provide recommendations on palliation of lung cancer with external beam radiotherapy (EBRT) and clarify the roles of concurrent chemotherapy and endobronchial brachytherapy (EBB) for palliation.^5,6

After prostate cancer, lung cancer is the second most frequently diagnosed cancer in the Veterans Health Administration (VHA).⁷ The VHA consists of 172 medical centers and is the largest integrated health care system in the US. At the time of this study, 40 of these centers had onsite radiation facilities. The VHA Palliative Radiation Taskforce has conducted a series of surveys to evaluate use of palliative radiotherapy in the VHA, determine VHA practice concordance with ASTRO and American College of Radiology (ACR) guidelines, and direct educational efforts towards addressing gaps in knowledge. These efforts are directed at ensuring best practices throughout this large and heterogeneous healthcare system. In 2016 a survey was conducted to evaluate concordance of VHA radiation oncologist (RO) practice with the 2011 ASTRO guidelines on palliative thoracic radiotherapy for non-small cell lung cancer (NSCLC).

Methods

A survey instrument was generated by VHA National Palliative Radiotherapy Taskforce members. It was reviewed and approved for use by the VHA Patient Care Services office. In May of 2016, the online survey was sent to the 88 VHA ROs practicing at the 40 sites with onsite radiation facilities. The survey aimed to determine patterns of practice for palliation of thoracic symptoms secondary to lung cancer.

Demographic information obtained included years in practice, employment status, academic appointment, board certification, and familiarity with ASTRO lung cancer guidelines. Two clinical scenarios were presented to glean opinions on dose/fractionation schemes preferred, use of concurrent chemotherapy, and use of EBB and/or yttrium aluminum garnet (YAG) laser technology. Survey questions also assessed use of EBRT for palliation of hemoptysis, chest wall pain, and/or stridor as well as use of stereotactic body radiotherapy (SBRT) for palliation.

Survey results were assessed for concordance with published ASTRO guidelines. χ² tests were run to test for associations between demographic factors such as academic appointment, years of practice, full time vs part time employment, and familiarity with ASTRO palliative lung cancer guidelines, with use of EBRT for palliation, dose and fractionation preference, use of concurrent chemotherapy, and strategy for management of endobronchial lesions.

Results

Of the 88 physicians surveyed, 54 responded for a response rate of 61%. Respondents represented 37 of the 40 (93%) VHA radiation oncology departments (Table 1). Among respondents, most were board certified (96%), held academic appointments (91%), and were full-time employees (85%). Forty-four percent of respondents were in practice for > 20 years, 19% for 11 to 20 years, 20% for 6 to 10 years, and 17% for < 6 years. A majority reported familiarity with the ASTRO guidelines (64%), while just 11% reported no familiarity with the guidelines.

When asked about use of SBRT for palliation of hemoptysis, stridor, and/or chest pain, the majority (87%) preferred conventional EBRT. Of the 13% who reported use of SBRT, most (11%) performed it onsite, with 2% of respondents referring offsite to non-VHA centers for the service. When asked about use of EBB for palliation, only 2% reported use of that procedure at their facilities, while 26% reported referral to non-VHA facilities for EBB. The remaining 72% of respondents favor use of conventional EBRT.

Respondents were presented with a case of a male patient aged 70 years who smoked and had widely metastatic NSCLC, a life expectancy of about 3 months, and 10/10 chest wall pain from direct tumor invasion. All respondents recommended palliative radiotherapy. The preferred fractionation was 20 Gray (Gy) in 5 fractions, which was recommended by 69% of respondents. The remainder recommended 30 Gy in 10 fractions (22%) or a single fraction of 10 Gy (9%). No respondent recommended the longer fractionation options of 60 Gy in 30 fractions, 45 Gy in 15 fractions, or 40 Gy in 20 fractions. The majority (98%) did not recommend concurrent chemotherapy.

When the above case was modified for an endobronchial lesion requiring palliation with associated lung collapse, rather than chest wall invasion, 20 respondents (38%) reported they would refer for EBB, and 20 respondents reported they would refer for YAG laser. As > 1 answer could be selected for this question, there were 12 respondents who selected both EBB and YAG laser; 8 selected only EBB, and 8 selected only YAG laser. Many respondents added comments about treating with EBRT, which had not been presented as an answer choice. Nearly half of respondents (49%) were amenable to referral for the use of EBB or YAG laser for lung reexpansion prior to radiotherapy. Three respondents mentioned referral for an endobronchial stent prior to palliative radiotherapy to address this question.

Discussion

This survey was conducted to evaluate concordance of management of metastatic lung cancer in the VHA with ASTRO guidelines. The relationship between respondents’ familiarity with the guidelines and responses also was evaluated to determine the impact such guidelines have on decision-making. The ASTRO guidelines for palliative thoracic radiation make recommendations regarding 3 issues: (1) radiation doses and fractionations for palliation; (2) the role of EBB; and (3) the use of concurrent chemotherapy.^5,6

Radiation Dose and Fractionation for Palliation

A variety of dose/fractionation schemes are considered appropriate in the ASTRO guideline statement, including more prolonged courses such as 30 Gy/10 fractions as well as more hypofractionated regimens (ie, 20 Gy/5 fractions, 17 Gy/2 fractions, and a single fraction of 10 Gy). Higher dose regimens, such as 30 Gy/10 fractions, have been associated with prolonged survival, as well as increased toxicities such as radiation esophagitis.⁸ Therefore, the guidelines support use of 30 Gy/10 fractions for patients with good performance status while encouraging use of more hypofractionated regimens for patients with poor performance status. In considering more hypofractionated regimens, one must consider the possibility of adverse effects that can be associated with higher dose per fraction. For instance, 17 Gy/2 fractions has been associated with myelopathy; therefore it should be used with caution and careful treatment planning.⁹

For the survey case example (a male aged 70 years with a 3-month life expectancy who required palliation for chest wall pain), all respondents selected hypofractionated regimens; with no respondent selected the more prolonged fractionations of 60 Gy/30 fractions, 45 Gy/15 fractions, or 40 Gy/20 fractions. These more prolonged fractionations are not endorsed by the guidelines in general, and particularly not for a patient with poor life expectancy. All responses for this case selected by survey respondents are considered appropriate per the consensus guideline statement.

Role of Concurrent Chemotherapy

The ASTRO guidelines do not support use of concurrent chemotherapy for palliation of stage IV NSCLC.^5,6 The 2018 updated guidelines established a role for concurrent chemotherapy for patients with stage III NSCLC with good performance status and life expectancy of > 3 months. This updated recommendation is based on data from 2 randomized trials demonstrating improvement in overall survival with the addition of chemotherapy for patients with stage III NSCLC undergoing palliative radiotherapy.^10-12

These newer studies are in contrast to an older randomized study by Ball and colleagues that demonstrated greater toxicity from concurrent chemotherapy, with no improvement in outcomes such as palliation of symptoms, overall survival, or progression free survival.¹³ In contrast to the newer studies that included only patients with stage III NSCLC, about half of the patients in the Ball and colleagues study had known metastatic disease.^10-13 Of note, staging for metastatic disease was not carried out routinely, so it is possible that a greater proportion of patients had metastatic disease that would have been seen on imaging. In concordance with the guidelines, 98% of the survey respondents did not recommend concurrent chemotherapy for palliation of intrathoracic symptom; only 1 respondent recommended use of chemotherapy for palliation.

Role of Endobronchial Brachytherapy

EBB involves implantation of radioactive sources for treatment of endobronchial lesions causing obstructive symptoms.¹⁴ Given the lack of randomized data that demonstrate a benefit of EBB over EBRT, the ASTRO guidelines do not endorse routine use of EBB for initial palliative management.^15,16 The ASTRO guidelines reference a Cochrane Review of 13 trials that concluded that EBRT alone is superior to EBB alone for initial palliation of symptoms from endobronchial NSCLC.¹⁷

Of respondents surveyed, only 1 facility offered onsite EBB. The majority of respondents (72%) preferred the use of conventional EBRT techniques, while 26% refer to non-VHA centers for EBB. Lack of incorporation of EBB into routine VHA practice likely is a reflection of the unclear role of this technology based on the available literature and ASTRO guidelines. In the setting of a right lower lung collapse, more respondents (49%) would consider use of EBB or YAG laser technology for lung reexpansion prior to EBRT.

The ASTRO guidelines recommend that initial EBB in conjunction with EBRT be considered based on randomized data demonstrating significant improvement in lung reexpansion and in patient reported dyspnea with addition of EBB to EBRT over EBRT alone.¹⁸ However, the guidelines do not mandate the use of EBB in this situation. It is possible that targeted education regarding the role of EBB would improve knowledge of the potential benefit in the setting of lung collapse and increase the percentage of VHA ROs who would recommend this procedure.

Limitations

The study is limited by lack of generalizability of these findings to all ROs in the country. It is also possible that physician responses do not represent practice patterns with complete accuracy. The use of EBB varied among practitioners. Further study of this technology is necessary to clarify its role in the management of endobronchial obstructive symptoms and to determine whether efforts should be made to increase access to EBB within the VHA.

Conclusions

Most of the ROs who responded to our survey were cognizant and compliant with current ASTRO guidelines on management of lung cancer. Furthermore, familiarity with ASTRO guidelines and management choices were not associated with the respondents’ years in practice, academic appointment, full-time vs part-time status, or familiarity with ASTRO guidelines. This study is a nationwide survey of ROs in the VHA system that reflects the radiation-related care received by veterans with metastatic lung cancer. Responses were obtained from 93% of the 40 radiation oncology centers, so it is likely that the survey accurately represents the decision-making process at the majority of centers. It is possible that those who did not respond to the survey do not treat thoracic cases.

Lung cancer is the leading cause of cancer mortality both in the US and worldwide.¹ Many patients diagnosed with lung cancer present with advanced disease with thoracic symptoms such as cough, hemoptysis, dyspnea, and chest pain.^2-4 Palliative radiotherapy is routinely used in patients with locally advanced and metastatic lung cancer with the goal of relieving these symptoms and improving quality of life. Guidelines published by the American Society for Radiation Oncology (ASTRO) in 2011, and updated in 2018, provide recommendations on palliation of lung cancer with external beam radiotherapy (EBRT) and clarify the roles of concurrent chemotherapy and endobronchial brachytherapy (EBB) for palliation.^5,6

After prostate cancer, lung cancer is the second most frequently diagnosed cancer in the Veterans Health Administration (VHA).⁷ The VHA consists of 172 medical centers and is the largest integrated health care system in the US. At the time of this study, 40 of these centers had onsite radiation facilities. The VHA Palliative Radiation Taskforce has conducted a series of surveys to evaluate use of palliative radiotherapy in the VHA, determine VHA practice concordance with ASTRO and American College of Radiology (ACR) guidelines, and direct educational efforts towards addressing gaps in knowledge. These efforts are directed at ensuring best practices throughout this large and heterogeneous healthcare system. In 2016 a survey was conducted to evaluate concordance of VHA radiation oncologist (RO) practice with the 2011 ASTRO guidelines on palliative thoracic radiotherapy for non-small cell lung cancer (NSCLC).

Methods

A survey instrument was generated by VHA National Palliative Radiotherapy Taskforce members. It was reviewed and approved for use by the VHA Patient Care Services office. In May of 2016, the online survey was sent to the 88 VHA ROs practicing at the 40 sites with onsite radiation facilities. The survey aimed to determine patterns of practice for palliation of thoracic symptoms secondary to lung cancer.

Demographic information obtained included years in practice, employment status, academic appointment, board certification, and familiarity with ASTRO lung cancer guidelines. Two clinical scenarios were presented to glean opinions on dose/fractionation schemes preferred, use of concurrent chemotherapy, and use of EBB and/or yttrium aluminum garnet (YAG) laser technology. Survey questions also assessed use of EBRT for palliation of hemoptysis, chest wall pain, and/or stridor as well as use of stereotactic body radiotherapy (SBRT) for palliation.

Survey results were assessed for concordance with published ASTRO guidelines. χ² tests were run to test for associations between demographic factors such as academic appointment, years of practice, full time vs part time employment, and familiarity with ASTRO palliative lung cancer guidelines, with use of EBRT for palliation, dose and fractionation preference, use of concurrent chemotherapy, and strategy for management of endobronchial lesions.

Results

Of the 88 physicians surveyed, 54 responded for a response rate of 61%. Respondents represented 37 of the 40 (93%) VHA radiation oncology departments (Table 1). Among respondents, most were board certified (96%), held academic appointments (91%), and were full-time employees (85%). Forty-four percent of respondents were in practice for > 20 years, 19% for 11 to 20 years, 20% for 6 to 10 years, and 17% for < 6 years. A majority reported familiarity with the ASTRO guidelines (64%), while just 11% reported no familiarity with the guidelines.

When asked about use of SBRT for palliation of hemoptysis, stridor, and/or chest pain, the majority (87%) preferred conventional EBRT. Of the 13% who reported use of SBRT, most (11%) performed it onsite, with 2% of respondents referring offsite to non-VHA centers for the service. When asked about use of EBB for palliation, only 2% reported use of that procedure at their facilities, while 26% reported referral to non-VHA facilities for EBB. The remaining 72% of respondents favor use of conventional EBRT.

Respondents were presented with a case of a male patient aged 70 years who smoked and had widely metastatic NSCLC, a life expectancy of about 3 months, and 10/10 chest wall pain from direct tumor invasion. All respondents recommended palliative radiotherapy. The preferred fractionation was 20 Gray (Gy) in 5 fractions, which was recommended by 69% of respondents. The remainder recommended 30 Gy in 10 fractions (22%) or a single fraction of 10 Gy (9%). No respondent recommended the longer fractionation options of 60 Gy in 30 fractions, 45 Gy in 15 fractions, or 40 Gy in 20 fractions. The majority (98%) did not recommend concurrent chemotherapy.

When the above case was modified for an endobronchial lesion requiring palliation with associated lung collapse, rather than chest wall invasion, 20 respondents (38%) reported they would refer for EBB, and 20 respondents reported they would refer for YAG laser. As > 1 answer could be selected for this question, there were 12 respondents who selected both EBB and YAG laser; 8 selected only EBB, and 8 selected only YAG laser. Many respondents added comments about treating with EBRT, which had not been presented as an answer choice. Nearly half of respondents (49%) were amenable to referral for the use of EBB or YAG laser for lung reexpansion prior to radiotherapy. Three respondents mentioned referral for an endobronchial stent prior to palliative radiotherapy to address this question.

Discussion

This survey was conducted to evaluate concordance of management of metastatic lung cancer in the VHA with ASTRO guidelines. The relationship between respondents’ familiarity with the guidelines and responses also was evaluated to determine the impact such guidelines have on decision-making. The ASTRO guidelines for palliative thoracic radiation make recommendations regarding 3 issues: (1) radiation doses and fractionations for palliation; (2) the role of EBB; and (3) the use of concurrent chemotherapy.^5,6

Radiation Dose and Fractionation for Palliation

A variety of dose/fractionation schemes are considered appropriate in the ASTRO guideline statement, including more prolonged courses such as 30 Gy/10 fractions as well as more hypofractionated regimens (ie, 20 Gy/5 fractions, 17 Gy/2 fractions, and a single fraction of 10 Gy). Higher dose regimens, such as 30 Gy/10 fractions, have been associated with prolonged survival, as well as increased toxicities such as radiation esophagitis.⁸ Therefore, the guidelines support use of 30 Gy/10 fractions for patients with good performance status while encouraging use of more hypofractionated regimens for patients with poor performance status. In considering more hypofractionated regimens, one must consider the possibility of adverse effects that can be associated with higher dose per fraction. For instance, 17 Gy/2 fractions has been associated with myelopathy; therefore it should be used with caution and careful treatment planning.⁹

For the survey case example (a male aged 70 years with a 3-month life expectancy who required palliation for chest wall pain), all respondents selected hypofractionated regimens; with no respondent selected the more prolonged fractionations of 60 Gy/30 fractions, 45 Gy/15 fractions, or 40 Gy/20 fractions. These more prolonged fractionations are not endorsed by the guidelines in general, and particularly not for a patient with poor life expectancy. All responses for this case selected by survey respondents are considered appropriate per the consensus guideline statement.

Role of Concurrent Chemotherapy

The ASTRO guidelines do not support use of concurrent chemotherapy for palliation of stage IV NSCLC.^5,6 The 2018 updated guidelines established a role for concurrent chemotherapy for patients with stage III NSCLC with good performance status and life expectancy of > 3 months. This updated recommendation is based on data from 2 randomized trials demonstrating improvement in overall survival with the addition of chemotherapy for patients with stage III NSCLC undergoing palliative radiotherapy.^10-12

These newer studies are in contrast to an older randomized study by Ball and colleagues that demonstrated greater toxicity from concurrent chemotherapy, with no improvement in outcomes such as palliation of symptoms, overall survival, or progression free survival.¹³ In contrast to the newer studies that included only patients with stage III NSCLC, about half of the patients in the Ball and colleagues study had known metastatic disease.^10-13 Of note, staging for metastatic disease was not carried out routinely, so it is possible that a greater proportion of patients had metastatic disease that would have been seen on imaging. In concordance with the guidelines, 98% of the survey respondents did not recommend concurrent chemotherapy for palliation of intrathoracic symptom; only 1 respondent recommended use of chemotherapy for palliation.

Role of Endobronchial Brachytherapy

EBB involves implantation of radioactive sources for treatment of endobronchial lesions causing obstructive symptoms.¹⁴ Given the lack of randomized data that demonstrate a benefit of EBB over EBRT, the ASTRO guidelines do not endorse routine use of EBB for initial palliative management.^15,16 The ASTRO guidelines reference a Cochrane Review of 13 trials that concluded that EBRT alone is superior to EBB alone for initial palliation of symptoms from endobronchial NSCLC.¹⁷

Of respondents surveyed, only 1 facility offered onsite EBB. The majority of respondents (72%) preferred the use of conventional EBRT techniques, while 26% refer to non-VHA centers for EBB. Lack of incorporation of EBB into routine VHA practice likely is a reflection of the unclear role of this technology based on the available literature and ASTRO guidelines. In the setting of a right lower lung collapse, more respondents (49%) would consider use of EBB or YAG laser technology for lung reexpansion prior to EBRT.

The ASTRO guidelines recommend that initial EBB in conjunction with EBRT be considered based on randomized data demonstrating significant improvement in lung reexpansion and in patient reported dyspnea with addition of EBB to EBRT over EBRT alone.¹⁸ However, the guidelines do not mandate the use of EBB in this situation. It is possible that targeted education regarding the role of EBB would improve knowledge of the potential benefit in the setting of lung collapse and increase the percentage of VHA ROs who would recommend this procedure.

Limitations

The study is limited by lack of generalizability of these findings to all ROs in the country. It is also possible that physician responses do not represent practice patterns with complete accuracy. The use of EBB varied among practitioners. Further study of this technology is necessary to clarify its role in the management of endobronchial obstructive symptoms and to determine whether efforts should be made to increase access to EBB within the VHA.

Conclusions

Most of the ROs who responded to our survey were cognizant and compliant with current ASTRO guidelines on management of lung cancer. Furthermore, familiarity with ASTRO guidelines and management choices were not associated with the respondents’ years in practice, academic appointment, full-time vs part-time status, or familiarity with ASTRO guidelines. This study is a nationwide survey of ROs in the VHA system that reflects the radiation-related care received by veterans with metastatic lung cancer. Responses were obtained from 93% of the 40 radiation oncology centers, so it is likely that the survey accurately represents the decision-making process at the majority of centers. It is possible that those who did not respond to the survey do not treat thoracic cases.

Lung cancer is the leading cause of cancer mortality both in the US and worldwide.¹ Many patients diagnosed with lung cancer present with advanced disease with thoracic symptoms such as cough, hemoptysis, dyspnea, and chest pain.^2-4 Palliative radiotherapy is routinely used in patients with locally advanced and metastatic lung cancer with the goal of relieving these symptoms and improving quality of life. Guidelines published by the American Society for Radiation Oncology (ASTRO) in 2011, and updated in 2018, provide recommendations on palliation of lung cancer with external beam radiotherapy (EBRT) and clarify the roles of concurrent chemotherapy and endobronchial brachytherapy (EBB) for palliation.^5,6

After prostate cancer, lung cancer is the second most frequently diagnosed cancer in the Veterans Health Administration (VHA).⁷ The VHA consists of 172 medical centers and is the largest integrated health care system in the US. At the time of this study, 40 of these centers had onsite radiation facilities. The VHA Palliative Radiation Taskforce has conducted a series of surveys to evaluate use of palliative radiotherapy in the VHA, determine VHA practice concordance with ASTRO and American College of Radiology (ACR) guidelines, and direct educational efforts towards addressing gaps in knowledge. These efforts are directed at ensuring best practices throughout this large and heterogeneous healthcare system. In 2016 a survey was conducted to evaluate concordance of VHA radiation oncologist (RO) practice with the 2011 ASTRO guidelines on palliative thoracic radiotherapy for non-small cell lung cancer (NSCLC).

Methods

A survey instrument was generated by VHA National Palliative Radiotherapy Taskforce members. It was reviewed and approved for use by the VHA Patient Care Services office. In May of 2016, the online survey was sent to the 88 VHA ROs practicing at the 40 sites with onsite radiation facilities. The survey aimed to determine patterns of practice for palliation of thoracic symptoms secondary to lung cancer.

Demographic information obtained included years in practice, employment status, academic appointment, board certification, and familiarity with ASTRO lung cancer guidelines. Two clinical scenarios were presented to glean opinions on dose/fractionation schemes preferred, use of concurrent chemotherapy, and use of EBB and/or yttrium aluminum garnet (YAG) laser technology. Survey questions also assessed use of EBRT for palliation of hemoptysis, chest wall pain, and/or stridor as well as use of stereotactic body radiotherapy (SBRT) for palliation.

Survey results were assessed for concordance with published ASTRO guidelines. χ² tests were run to test for associations between demographic factors such as academic appointment, years of practice, full time vs part time employment, and familiarity with ASTRO palliative lung cancer guidelines, with use of EBRT for palliation, dose and fractionation preference, use of concurrent chemotherapy, and strategy for management of endobronchial lesions.

Results

Of the 88 physicians surveyed, 54 responded for a response rate of 61%. Respondents represented 37 of the 40 (93%) VHA radiation oncology departments (Table 1). Among respondents, most were board certified (96%), held academic appointments (91%), and were full-time employees (85%). Forty-four percent of respondents were in practice for > 20 years, 19% for 11 to 20 years, 20% for 6 to 10 years, and 17% for < 6 years. A majority reported familiarity with the ASTRO guidelines (64%), while just 11% reported no familiarity with the guidelines.

When asked about use of SBRT for palliation of hemoptysis, stridor, and/or chest pain, the majority (87%) preferred conventional EBRT. Of the 13% who reported use of SBRT, most (11%) performed it onsite, with 2% of respondents referring offsite to non-VHA centers for the service. When asked about use of EBB for palliation, only 2% reported use of that procedure at their facilities, while 26% reported referral to non-VHA facilities for EBB. The remaining 72% of respondents favor use of conventional EBRT.

Respondents were presented with a case of a male patient aged 70 years who smoked and had widely metastatic NSCLC, a life expectancy of about 3 months, and 10/10 chest wall pain from direct tumor invasion. All respondents recommended palliative radiotherapy. The preferred fractionation was 20 Gray (Gy) in 5 fractions, which was recommended by 69% of respondents. The remainder recommended 30 Gy in 10 fractions (22%) or a single fraction of 10 Gy (9%). No respondent recommended the longer fractionation options of 60 Gy in 30 fractions, 45 Gy in 15 fractions, or 40 Gy in 20 fractions. The majority (98%) did not recommend concurrent chemotherapy.

When the above case was modified for an endobronchial lesion requiring palliation with associated lung collapse, rather than chest wall invasion, 20 respondents (38%) reported they would refer for EBB, and 20 respondents reported they would refer for YAG laser. As > 1 answer could be selected for this question, there were 12 respondents who selected both EBB and YAG laser; 8 selected only EBB, and 8 selected only YAG laser. Many respondents added comments about treating with EBRT, which had not been presented as an answer choice. Nearly half of respondents (49%) were amenable to referral for the use of EBB or YAG laser for lung reexpansion prior to radiotherapy. Three respondents mentioned referral for an endobronchial stent prior to palliative radiotherapy to address this question.

Discussion

This survey was conducted to evaluate concordance of management of metastatic lung cancer in the VHA with ASTRO guidelines. The relationship between respondents’ familiarity with the guidelines and responses also was evaluated to determine the impact such guidelines have on decision-making. The ASTRO guidelines for palliative thoracic radiation make recommendations regarding 3 issues: (1) radiation doses and fractionations for palliation; (2) the role of EBB; and (3) the use of concurrent chemotherapy.^5,6

Radiation Dose and Fractionation for Palliation

A variety of dose/fractionation schemes are considered appropriate in the ASTRO guideline statement, including more prolonged courses such as 30 Gy/10 fractions as well as more hypofractionated regimens (ie, 20 Gy/5 fractions, 17 Gy/2 fractions, and a single fraction of 10 Gy). Higher dose regimens, such as 30 Gy/10 fractions, have been associated with prolonged survival, as well as increased toxicities such as radiation esophagitis.⁸ Therefore, the guidelines support use of 30 Gy/10 fractions for patients with good performance status while encouraging use of more hypofractionated regimens for patients with poor performance status. In considering more hypofractionated regimens, one must consider the possibility of adverse effects that can be associated with higher dose per fraction. For instance, 17 Gy/2 fractions has been associated with myelopathy; therefore it should be used with caution and careful treatment planning.⁹

For the survey case example (a male aged 70 years with a 3-month life expectancy who required palliation for chest wall pain), all respondents selected hypofractionated regimens; with no respondent selected the more prolonged fractionations of 60 Gy/30 fractions, 45 Gy/15 fractions, or 40 Gy/20 fractions. These more prolonged fractionations are not endorsed by the guidelines in general, and particularly not for a patient with poor life expectancy. All responses for this case selected by survey respondents are considered appropriate per the consensus guideline statement.

Role of Concurrent Chemotherapy

The ASTRO guidelines do not support use of concurrent chemotherapy for palliation of stage IV NSCLC.^5,6 The 2018 updated guidelines established a role for concurrent chemotherapy for patients with stage III NSCLC with good performance status and life expectancy of > 3 months. This updated recommendation is based on data from 2 randomized trials demonstrating improvement in overall survival with the addition of chemotherapy for patients with stage III NSCLC undergoing palliative radiotherapy.^10-12

These newer studies are in contrast to an older randomized study by Ball and colleagues that demonstrated greater toxicity from concurrent chemotherapy, with no improvement in outcomes such as palliation of symptoms, overall survival, or progression free survival.¹³ In contrast to the newer studies that included only patients with stage III NSCLC, about half of the patients in the Ball and colleagues study had known metastatic disease.^10-13 Of note, staging for metastatic disease was not carried out routinely, so it is possible that a greater proportion of patients had metastatic disease that would have been seen on imaging. In concordance with the guidelines, 98% of the survey respondents did not recommend concurrent chemotherapy for palliation of intrathoracic symptom; only 1 respondent recommended use of chemotherapy for palliation.

Role of Endobronchial Brachytherapy

EBB involves implantation of radioactive sources for treatment of endobronchial lesions causing obstructive symptoms.¹⁴ Given the lack of randomized data that demonstrate a benefit of EBB over EBRT, the ASTRO guidelines do not endorse routine use of EBB for initial palliative management.^15,16 The ASTRO guidelines reference a Cochrane Review of 13 trials that concluded that EBRT alone is superior to EBB alone for initial palliation of symptoms from endobronchial NSCLC.¹⁷

Of respondents surveyed, only 1 facility offered onsite EBB. The majority of respondents (72%) preferred the use of conventional EBRT techniques, while 26% refer to non-VHA centers for EBB. Lack of incorporation of EBB into routine VHA practice likely is a reflection of the unclear role of this technology based on the available literature and ASTRO guidelines. In the setting of a right lower lung collapse, more respondents (49%) would consider use of EBB or YAG laser technology for lung reexpansion prior to EBRT.

The ASTRO guidelines recommend that initial EBB in conjunction with EBRT be considered based on randomized data demonstrating significant improvement in lung reexpansion and in patient reported dyspnea with addition of EBB to EBRT over EBRT alone.¹⁸ However, the guidelines do not mandate the use of EBB in this situation. It is possible that targeted education regarding the role of EBB would improve knowledge of the potential benefit in the setting of lung collapse and increase the percentage of VHA ROs who would recommend this procedure.

Limitations

The study is limited by lack of generalizability of these findings to all ROs in the country. It is also possible that physician responses do not represent practice patterns with complete accuracy. The use of EBB varied among practitioners. Further study of this technology is necessary to clarify its role in the management of endobronchial obstructive symptoms and to determine whether efforts should be made to increase access to EBB within the VHA.

Conclusions

Most of the ROs who responded to our survey were cognizant and compliant with current ASTRO guidelines on management of lung cancer. Furthermore, familiarity with ASTRO guidelines and management choices were not associated with the respondents’ years in practice, academic appointment, full-time vs part-time status, or familiarity with ASTRO guidelines. This study is a nationwide survey of ROs in the VHA system that reflects the radiation-related care received by veterans with metastatic lung cancer. Responses were obtained from 93% of the 40 radiation oncology centers, so it is likely that the survey accurately represents the decision-making process at the majority of centers. It is possible that those who did not respond to the survey do not treat thoracic cases.

SARS-CoV-2 has infected 141 million people worldwide and 31 million people in the United States as of April 20, 2021.^1,2 The influx of hospital admissions and deaths has severely strained healthcare systems worldwide and placed healthcare workers (HCWs) at increased risk for acquiring COVID-19.^3-5

Several studies have described the impact of COVID-19 on this heterogeneous group of HCWs. Shields et al reported a seroprevalence of 24.4% in HCWs at University Hospitals Birmingham (UK), with the highest rate, 34.5%, in housekeeping staff.⁶ Steensels et al reported a lower prevalence of 6.4% at a tertiary care center in Belgium, and showed no increased risk for HCWs when directly involved in clinical care.⁷ The authors attributed this to adequate use of personal protective equipment (PPE). Other studies have reported seroprevalences ranging from 1.6% to 18%.^8-11 In the New York City (NYC) metro area, Jeremias et al reported a seroprevalence of 9.8% in HCWs and found no difference by job title or work location,¹² whereas Moscola et al reported a seroprevalence of 13.7% and demonstrated a 3% increased risk for those working in service or maintenance.¹³ Antibody tests were conducted between March and April 2020 in all but two of these studies; testing in these two studies was performed between April 13 and June 23, 2020, with one reporting a seroprevalence of 6%¹¹ and the other, 13.7%.¹³

NYC became the earliest pandemic epicenter in the United States following untracked transmission from ongoing circulation of SARS-CoV-2 in Europe.¹⁴ As a result, the COVID-19 surge in NYC commenced in March and largely subsided by the end of May 2020. Most HCW data reported to date do not reflect the situation at the end of the surge, and may underestimate true seroprevalence. We describe SARS-CoV-2 seroprevalence in HCWs in a large inner-city hospital in NYC, with antibody testing conducted from May 18 to June 26, 2020, at the subsidence of the surge. To further our understanding of occupational risk among different groups of HCWs, we examined associations of seroprevalence with HCWs’ job function and work location.

This was a cross-sectional seroprevalence study conducted in the BronxCare Health System located in South and Central Bronx, an area that experienced one of the highest incidences of SARS-CoV-2 infections within NYC’s five boroughs.

HCWs were offered voluntary testing for serum antibodies to SARS-CoV-2 between May 18 and June 26, 2020. Testing occurred in the institution’s auditorium, a central and easily accessible location. Weekly emails were sent to all employees and department heads during the testing period, offering antibody testing and providing location and testing time information. The Elecsys Anti-SARS-CoV-2 (Roche) assay measuring total qualitative antibodies was used; the assay has a reported sensitivity of 97.1% 14 days after a positive SARS-CoV-2 RNA polymerase chain reaction (PCR) test result and a specificity of 100%.¹⁵

Demographic and work-related information was abstracted from electronic medical records, including all comorbid conditions that affected 30 or more HCWs. Pulmonary diagnoses, including asthma and chronic obstructive pulmonary disease, were grouped as chronic lung disease, and cardiovascular diseases, including hypertension, as chronic heart disease. Personal identifiers and data were delinked upon completion of data abstraction. The study was approved by the hospital’s institutional review board.

Job Function and Work Location

HCWs were grouped by job function as follows: physicians; nurses (including physician assistants and nurse practitioners); allied HCW I (medical assistants, patient care, and electrocardiogram, radiology, and ear, nose and throat technicians); allied HCW II (social workers, dieticians and nutritionists, registration clerks and unit associates, physical and occupational therapists); nonclinical staff (patient transporters, housekeeping staff, and security staff); pharmacists; engineering; and administrative staff. Respiratory therapists were considered as a separate group as their work placed them at high risk for respiratory diseases.

Work locations were as follows: clinics (including dental, outpatient, and satellite clinics), emergency departments (ED), inpatient units (including floors and intensive care units [ICU]), radiology suite, laboratory and pharmacy, and offices.

Statistical Analysis

Descriptive statistics were calculated using χ² analyses. All demographic variables were tested against serology status (positive/negative). A binary logistic regression analysis was used to calculate odds ratios (ORs). Eight separate univariate unadjusted ORs were calculated by running each predictor variable against serology status (dependent variable), which included the six categorical variables—race, ethnicity, age, sex, body mass index (BMI), and prior SARS-CoV-2 PCR results—and the two main predictors—job function and work location. To obtain adjusted ORs, two final separate multivariable logistic regression analyses were executed including the six covariates listed. Due to high collinearity between job function and work location (χ² = 3030.13, df = 35 [6 levels of work location – 1]*[8 levels of job function – 1]; P < .001), we included only one of the main predictors in each model. The regressions were specified such that the reference groups for the work location and job function variables were office work and administration, respectively. This choice was made based on the fact that their nonclinical functions do not confer an exposure risk in excess of that experienced by typical community populations. Sensitivity analyses were performed on the subset of HCWs whose address zip codes indicated residence within NYC to exclude the effect of different community seroprevalences in areas outside of NYC. The 95% CI for seroprevalence of antibodies within tested HCWs was estimated using the Clopper-Pearson binomial method.

Among all HCWs in the institution (N = 4,807), 2,749 (57.2%) underwent voluntary testing. Of those who underwent testing, 831 were positive for antibodies to SARS-CoV-2 (Figure 1), a seroprevalence of 30.2% (95% CI, 29%-32%). Among the age groups, the 45-to-64−year group had the highest seropositivity at 33% (400/1203), and those ≥75 years of age, the lowest at 16.7% (2/12) (P < .009).

Among all tested HCWs, 70.1% (1,928/2,749) resided in NYC. SARS-CoV-2 seroprevalence in this subset was 32% (616/1,928) (Figure 1). Demographic and comorbid conditions in HCWs who lived in NYC were similar to those of the whole group (Appendix Table 1).

HCWs who underwent voluntary antibody testing (Appendix Table 2) had a higher percentage of persons in the 45-to-64−year age group (43.8% vs 40.9%) and a lower percentage of persons in the 65-to-74−year age group (3.3% vs 5.3%) compared with the group of HCWs that did not undergo testing (P < .001). Gender, race, ethnicity, comorbid conditions, SARS-CoV-2 PCR testing, and work locations were not different between groups. The tested group had higher proportions of clinicians (physicians, nurses, allied HCWs I and II) than the untested nonparticipant group (P = .014).

SARS-CoV-2 PCR Tests on HCWs

More than one-third (34.1%; 938/2,749) of HCWs had a documented nasopharyngeal PCR test between March 23 and June 26, 2020 (Table). Of all PCRs performed, 262 were positive, giving an overall PCR positivity rate of 27.9%. Positivity was 51.4% in March and 36.6% in April. The reasons for PCR testing were not available, but likely represent a combination of exposure-related testing among asymptomatic individuals and diagnostic testing of symptomatic HCWs. In contrast, serology testing was indicative of prior infection and yielded a cumulative seroprevalence at the end of the surge. Findings were similar among HCWs residing in NYC (Appendix Table 1).

Work Location and Job Function

Among all HCWs (Table, Figure 2), there were differences in seropositivity by work location (P = .001). The largest number of HCWs worked in inpatient units (1,348/2,749, 49%), and the second largest in offices (554/2,749, 20%). The highest seropositivity rate was in the EDs, at 36.4% (64/176), followed by radiology suites, at 32.7% (17/52); the seropositivity rate in office locations was 25.8% (143/554). Among HCWs residing in NYC (Appendix Table 1, Appendix Figure 1), the rank order according to proportion seropositive by work location was similar to that of the whole group (P = .004), except that the second highest seropositivity rate was in the inpatient units (33.9% [323/953]). In the group of HCWs residing in NYC, office locations had a seropositivity of 27.4% (102/372). The seropositivity rates for both groups working in office locations were slightly higher than the 22% community seroprevalence in NYC reported for the same period.¹⁶

Among all HCWs, there were differences in seropositivity by job function (P = .001). The greatest proportion of HCWs were allied HCW II (23% [631/2,749]), followed by nurses (22.2% [611/2,749]) and physicians (21.3% [585/2,749] ). Seropositivity was highest for nonclinical staff (44.0% [51/116]), followed by nurses (37.5% [229/611]) and allied clinical HCW I and II (34.5% [143/414] and 32.0% [202/631], respectively). It was lowest for administrative staff (20.9% [42/201]) and pharmacists (11.1% [5/45]). Among HCWs residing in NYC, the rank order according to proportion seropositive by location was similar to that of the whole group. Administrative staff seropositivity was 18.3% (20/109). Administrative staff seropositivity for both groups was marginally lower than the 22% community seroprevalence in NYC for the same period.¹⁶

Odds Ratios for SARS-CoV-2 Seropositivity

For all HCWs, in unadjusted models (Appendix Table 3), age 45 to 64 years and Black race were associated with increased odds of being seropositive (1.26; 95% CI, 1.07-1.49 and 2.26; 95% CI, 1.51-3.37, respectively). Increased odds were seen for HCWs working in the ED (1.64; 95% CI, 1.14-2.36) and inpatient units (1.35; 95% CI, 1.08-1.69), and decreased odds were seen for those working in the laboratory and pharmacy (0.47; 95% CI, 0.26-0.86). Increased odds for seropositivity were found for nurses (2.27; 95% CI, 1.56-3.31), allied HCW I (2.00; 95% CI, 1.34-2.97), allied HCW II (1.78; 95% CI, 1.22-2.60), and nonclinical staff (2.97; 95% CI,1.80-4.90).

After adjusting for all covariates, HCWs who were Black remained at increased odds for being seropositive in the two final models (adjusted OR, 2.29; 95% CI, 1.38-3.81 and adjusted OR, 2.94; 95% CI, 1.78-4.85), as did those who had a BMI >30 kg/m², with an adjusted OR of 1.36 (95% CI, 1.05-1.77) in one of the final models (Appendix Table 3). None of the other comorbid conditions had increased ORs. Those who worked in the ED and inpatient units also remained at increased odds after adjusting for covariates (2.27; 95% CI, 1.53-3.37 and 1.48; 95% CI, 1.14-1.92, respectively; Figure 3). Other job functions that had increased odds for seropositivity were nurses (2.54; 95% CI, 1.64-3.94), allied HCW I (1.83; 95% CI, 1.15-2.89) and II (1.70; 95% CI, 1.10-2.63), and nonclinical staff (2.51; 95% CI, 1.42-4.43).

Having a positive PCR for SAR-CoV-2 on nasopharyngeal swabs was strongly associated with seropositivity (OR, 47.26; 95% CI, 29.30-76.23 and OR, 44.79; 95% CI, 27.87-72.00) in the two multivariate-adjusted models. These findings were confirmed when the analyses were performed on HCWs who resided in NYC (Appendix Table 4 and Appendix Figure 2).

In a large inner-city New York hospital, we report a cumulative SARS-CoV-2 seroprevalence of 30.2% in HCWs at the end of the first surge of SARS-CoV-2 infections in NYC. We identified the highest seropositivity rates for nonclinical staff and nurses, followed by allied HCWs, with the odds of being seropositive ranging from 1.7 to 2.5. The work locations with the highest seroprevalences were the ED and inpatient units, with 2.3-fold and 1.5-fold increased odds of seropositivity, respectively.

Serosurveillance studies have reported the trajectory of community prevalence in NYC over the first wave. A 6.3% prevalence was reported in samples collected between March 23 and April 1, 2020.¹⁷ In a study by Rosenberg et al¹⁸ with testing performed from April 9 through April 28, 2020, prevalence increased to 22.7%. Serosurveillance data from the NYC Department of Health show prevalence ranging from 20.1% to 23.3% (average 22%) during the study period.¹⁶ Compared to the estimated seroprevalence of 9.3% in the United States,¹⁹ these rates established NYC as an early epicenter for the COVID-19 pandemic, with our institution’s HCW seroprevalence considerably higher than NYC community serosurveillance rates, 2.2 times higher than reported in the earlier HCW study in the greater NYC area,¹³ and higher than the 27% rate during May 2020 recently reported in another NYC hospital.²⁰

Data from studies of hospital transmission and effects of mitigation measures, such as a universal masking policy for HCWs and patients, clearly demonstrate the high effectiveness of these measures in reducing hospital transmissions.^21,22 This suggests HCW seroprevalence in institutions with well-implemented infection control and universal masking policies may not be a consequence of workplace exposures, but rather may be reflective of community rates.²³ Our institution’s response commenced February 3, 2020, with implementation of social distancing, a universal masking policy, transmission-based precautions, and use of fitted N95 masks. Mid-March, elective surgeries were canceled, and inpatient visitation suspended. During the surge, these measures were widely and consistently implemented for all categories of HCWs throughout the work environment, based on emerging guidelines from the Centers for Disease Control and Prevention (CDC) and NYC Department of Health. Our overall observed HCW seroprevalence, well above that of the community, with differences in categories of job function and work locations, is therefore an important finding. Our sample of 2,749 HCWs lived in NYC and its surrounding suburbs and nearby states. There is heterogeneity in community seroprevalence between areas outside of NYC and NYC (an epicenter) itself. We therefore analyzed our data in the subset with NYC zip codes, confirming a similar overall prevalence and increased odds of seropositivity in nurses, allied HCWs, and nonclinical staff.

Physicians and administrative and office staff had seropositivity rates of 18.1%, 20.9%, and 25.8%, respectively, consistent with community rates and illustrating the effectiveness of PPE in the hospital setting. Since PPE use was part of a universal policy applied to all HCWs in our institution, other possible reasons may explain the differences we found. We speculate that the close working relationship nurses have with their patients resulted in a longer duration and higher frequency of daily interactions, increasing the risk for transmission and causing breakthrough infections.^24,25 This increased risk is reflected in a study in which 28% of hospitalized patients were nurses and 9% certified nursing assistants.²⁶

The CDC recently redefined close contact with someone with COVID-19 as a cumulative total of >15 minutes over 24 hours.²⁵ Thus, several multiple short periods of exposure can increase risk for infection with SARS-CoV-2; such exposure is characteristic of the job function of nurses, nursing staff, and nonclinical staff. Further, housekeeping, transportation, and security officers are all nonclinical staff with significant and multiple exposures to COVID-19 patients during the surge, and for security officers, to continuous public traffic in and out of the hospital. SARS-CoV-2 spreads by virus shedding in large droplets and aerosols, with droplet nuclei <5 microns in size efficiently dispersed in air, an important additional mode of transmission.^27-30 Airborne transmission coupled with virus shedding in asymptomatic and presymptomatic persons, which has been shown to cause secondary attack rates of up to 32%, are other factors that likely contributed to the increased seroprevalence in this group.³¹ Our observation is consistent with the Birmingham study, which reported the highest rate in housekeeping staff, with a prevalence of 34.5%, compared to 44% in this study.⁶ Similar reasons for high seropositivity rates apply to the two groups of allied HCWs (eg, medical assistants and patient care technicians, social workers, nutritionists and therapists), whose job functions place them in intermittent but significant proximity with inpatients and outpatients.

Consistent with public health data showing that minorities are disproportionately affected by this disease, we found that Black HCWs were three times more likely to be seropositive.³² However, an unexpected observation was the association between obesity and SARS-CoV-2 seropositivity. A possible explanation for this association may be inability to achieve optimal fit testing for N95 masks, thereby increasing the risk of exposure to droplet nuclei. This is important given that obesity is associated with poorer outcomes from COVID-19.

During the height of the first wave in NYC, EDs and inpatient units handled a large volume of COVID-19 patients with high PCR positivity rates (peak of 51% in March in our hospital). It was not unexpected that we observed increased odds of seropositivity in these work locations. As ICUs were at capacity, inpatient units cared for critically ill patients they would not normally have. HCWs in these locations coped with an increased workload, increased demand on PPE supplies, and work fatigue, which contributed to increased risk for hospital-acquired SARS-CoV-2 infections.

Reporting seroprevalence at a single institution was a limitation of the study. Approximately 57% of the hospital’s total HCW population was tested for antibodies. It is possible their risk profile influenced their decision to volunteer for testing when it became available, introducing selection bias. A comparison between tested and untested HCWs showed similarity in all demographic measures, including nasopharyngeal PCR testing, except for age. We did not have information on symptoms that would prompt PCR testing. HCWs who underwent voluntary testing were younger compared to those who did not undergo testing. Current NYC serosurveillance data showed higher seropositivity in the 45-to-64–year age group (27.8%-28.6%) compared to the 65-to-74–year age group (24.3%), which suggests that the tested group may overestimate seroprevalence among HCWs relative to a randomly selected sample.³³ Similarly, there were more nurses, allied HCWs, physicians, and administrative staff in the tested group, with the former two having higher SARS-CoV-2 seropositivity compared to community prevalence, which could also overestimate seroprevalence. Our large sample size provided us with the power to detect differences within several different job functions and work locations, a strength of this study. It was not possible to differentiate community- from hospital-acquired infection in our HCWs, a limitation in many observational HCW seroprevalence studies. However, when we analyzed data restricted only to HCWs in NYC, to reduce the effect of differing community prevalences outside the city, our results were unchanged. Since it is possible that nonclinical HCWs are of a lower socioeconomic status compared to others (nurses and allied HCWs), we cannot exclude the possibility that higher SARS-CoV-2 seroprevalence associated with lower status explains, partly or completely, the increased odds of seropositivity we observed.³⁴ Due to the high proportion of missing data for race (61.3%), we advise caution in interpreting our finding that the odds of seropositivity were three times higher for Black race, even though consistent with prior literature.³⁴ Healthcare organizations have similar job function and work location categories incorporated in their infrastructure, suggesting that our observations may be generalizable to other hospitals in the United States.

These findings show that during the first surge in NYC, with its increased burden of disease, hospitalizations, morbidity, and mortality, seroprevalences varied based on job function and work location within this institution. Nurses were at highest risk for SARS-CoV-2 infection, as were those who worked in the ED. In preparation for subsequent waves of SARS-CoV-2 and other highly contagious respiratory infections, major medical centers need to enhance efforts aimed at protecting HCWs, with particular attention to these groups. This study also strongly supports the recent CDC guideline prioritizing HCWs to receive COVID-19 mRNA and adenovirus vector vaccines that have obtained emergency use authorization by the US Food and Drug Administration.³⁵

The authors thank all the residents, nurses, and staff of the Department of Family Medicine for their contribution to this work.

SARS-CoV-2 has infected 141 million people worldwide and 31 million people in the United States as of April 20, 2021.^1,2 The influx of hospital admissions and deaths has severely strained healthcare systems worldwide and placed healthcare workers (HCWs) at increased risk for acquiring COVID-19.^3-5

Several studies have described the impact of COVID-19 on this heterogeneous group of HCWs. Shields et al reported a seroprevalence of 24.4% in HCWs at University Hospitals Birmingham (UK), with the highest rate, 34.5%, in housekeeping staff.⁶ Steensels et al reported a lower prevalence of 6.4% at a tertiary care center in Belgium, and showed no increased risk for HCWs when directly involved in clinical care.⁷ The authors attributed this to adequate use of personal protective equipment (PPE). Other studies have reported seroprevalences ranging from 1.6% to 18%.^8-11 In the New York City (NYC) metro area, Jeremias et al reported a seroprevalence of 9.8% in HCWs and found no difference by job title or work location,¹² whereas Moscola et al reported a seroprevalence of 13.7% and demonstrated a 3% increased risk for those working in service or maintenance.¹³ Antibody tests were conducted between March and April 2020 in all but two of these studies; testing in these two studies was performed between April 13 and June 23, 2020, with one reporting a seroprevalence of 6%¹¹ and the other, 13.7%.¹³

NYC became the earliest pandemic epicenter in the United States following untracked transmission from ongoing circulation of SARS-CoV-2 in Europe.¹⁴ As a result, the COVID-19 surge in NYC commenced in March and largely subsided by the end of May 2020. Most HCW data reported to date do not reflect the situation at the end of the surge, and may underestimate true seroprevalence. We describe SARS-CoV-2 seroprevalence in HCWs in a large inner-city hospital in NYC, with antibody testing conducted from May 18 to June 26, 2020, at the subsidence of the surge. To further our understanding of occupational risk among different groups of HCWs, we examined associations of seroprevalence with HCWs’ job function and work location.

This was a cross-sectional seroprevalence study conducted in the BronxCare Health System located in South and Central Bronx, an area that experienced one of the highest incidences of SARS-CoV-2 infections within NYC’s five boroughs.

HCWs were offered voluntary testing for serum antibodies to SARS-CoV-2 between May 18 and June 26, 2020. Testing occurred in the institution’s auditorium, a central and easily accessible location. Weekly emails were sent to all employees and department heads during the testing period, offering antibody testing and providing location and testing time information. The Elecsys Anti-SARS-CoV-2 (Roche) assay measuring total qualitative antibodies was used; the assay has a reported sensitivity of 97.1% 14 days after a positive SARS-CoV-2 RNA polymerase chain reaction (PCR) test result and a specificity of 100%.¹⁵

Demographic and work-related information was abstracted from electronic medical records, including all comorbid conditions that affected 30 or more HCWs. Pulmonary diagnoses, including asthma and chronic obstructive pulmonary disease, were grouped as chronic lung disease, and cardiovascular diseases, including hypertension, as chronic heart disease. Personal identifiers and data were delinked upon completion of data abstraction. The study was approved by the hospital’s institutional review board.

Job Function and Work Location

HCWs were grouped by job function as follows: physicians; nurses (including physician assistants and nurse practitioners); allied HCW I (medical assistants, patient care, and electrocardiogram, radiology, and ear, nose and throat technicians); allied HCW II (social workers, dieticians and nutritionists, registration clerks and unit associates, physical and occupational therapists); nonclinical staff (patient transporters, housekeeping staff, and security staff); pharmacists; engineering; and administrative staff. Respiratory therapists were considered as a separate group as their work placed them at high risk for respiratory diseases.

Work locations were as follows: clinics (including dental, outpatient, and satellite clinics), emergency departments (ED), inpatient units (including floors and intensive care units [ICU]), radiology suite, laboratory and pharmacy, and offices.

Statistical Analysis

Descriptive statistics were calculated using χ² analyses. All demographic variables were tested against serology status (positive/negative). A binary logistic regression analysis was used to calculate odds ratios (ORs). Eight separate univariate unadjusted ORs were calculated by running each predictor variable against serology status (dependent variable), which included the six categorical variables—race, ethnicity, age, sex, body mass index (BMI), and prior SARS-CoV-2 PCR results—and the two main predictors—job function and work location. To obtain adjusted ORs, two final separate multivariable logistic regression analyses were executed including the six covariates listed. Due to high collinearity between job function and work location (χ² = 3030.13, df = 35 [6 levels of work location – 1]*[8 levels of job function – 1]; P < .001), we included only one of the main predictors in each model. The regressions were specified such that the reference groups for the work location and job function variables were office work and administration, respectively. This choice was made based on the fact that their nonclinical functions do not confer an exposure risk in excess of that experienced by typical community populations. Sensitivity analyses were performed on the subset of HCWs whose address zip codes indicated residence within NYC to exclude the effect of different community seroprevalences in areas outside of NYC. The 95% CI for seroprevalence of antibodies within tested HCWs was estimated using the Clopper-Pearson binomial method.

Among all HCWs in the institution (N = 4,807), 2,749 (57.2%) underwent voluntary testing. Of those who underwent testing, 831 were positive for antibodies to SARS-CoV-2 (Figure 1), a seroprevalence of 30.2% (95% CI, 29%-32%). Among the age groups, the 45-to-64−year group had the highest seropositivity at 33% (400/1203), and those ≥75 years of age, the lowest at 16.7% (2/12) (P < .009).

Among all tested HCWs, 70.1% (1,928/2,749) resided in NYC. SARS-CoV-2 seroprevalence in this subset was 32% (616/1,928) (Figure 1). Demographic and comorbid conditions in HCWs who lived in NYC were similar to those of the whole group (Appendix Table 1).

HCWs who underwent voluntary antibody testing (Appendix Table 2) had a higher percentage of persons in the 45-to-64−year age group (43.8% vs 40.9%) and a lower percentage of persons in the 65-to-74−year age group (3.3% vs 5.3%) compared with the group of HCWs that did not undergo testing (P < .001). Gender, race, ethnicity, comorbid conditions, SARS-CoV-2 PCR testing, and work locations were not different between groups. The tested group had higher proportions of clinicians (physicians, nurses, allied HCWs I and II) than the untested nonparticipant group (P = .014).

SARS-CoV-2 PCR Tests on HCWs

More than one-third (34.1%; 938/2,749) of HCWs had a documented nasopharyngeal PCR test between March 23 and June 26, 2020 (Table). Of all PCRs performed, 262 were positive, giving an overall PCR positivity rate of 27.9%. Positivity was 51.4% in March and 36.6% in April. The reasons for PCR testing were not available, but likely represent a combination of exposure-related testing among asymptomatic individuals and diagnostic testing of symptomatic HCWs. In contrast, serology testing was indicative of prior infection and yielded a cumulative seroprevalence at the end of the surge. Findings were similar among HCWs residing in NYC (Appendix Table 1).

Work Location and Job Function

Among all HCWs (Table, Figure 2), there were differences in seropositivity by work location (P = .001). The largest number of HCWs worked in inpatient units (1,348/2,749, 49%), and the second largest in offices (554/2,749, 20%). The highest seropositivity rate was in the EDs, at 36.4% (64/176), followed by radiology suites, at 32.7% (17/52); the seropositivity rate in office locations was 25.8% (143/554). Among HCWs residing in NYC (Appendix Table 1, Appendix Figure 1), the rank order according to proportion seropositive by work location was similar to that of the whole group (P = .004), except that the second highest seropositivity rate was in the inpatient units (33.9% [323/953]). In the group of HCWs residing in NYC, office locations had a seropositivity of 27.4% (102/372). The seropositivity rates for both groups working in office locations were slightly higher than the 22% community seroprevalence in NYC reported for the same period.¹⁶

Among all HCWs, there were differences in seropositivity by job function (P = .001). The greatest proportion of HCWs were allied HCW II (23% [631/2,749]), followed by nurses (22.2% [611/2,749]) and physicians (21.3% [585/2,749] ). Seropositivity was highest for nonclinical staff (44.0% [51/116]), followed by nurses (37.5% [229/611]) and allied clinical HCW I and II (34.5% [143/414] and 32.0% [202/631], respectively). It was lowest for administrative staff (20.9% [42/201]) and pharmacists (11.1% [5/45]). Among HCWs residing in NYC, the rank order according to proportion seropositive by location was similar to that of the whole group. Administrative staff seropositivity was 18.3% (20/109). Administrative staff seropositivity for both groups was marginally lower than the 22% community seroprevalence in NYC for the same period.¹⁶

Odds Ratios for SARS-CoV-2 Seropositivity

For all HCWs, in unadjusted models (Appendix Table 3), age 45 to 64 years and Black race were associated with increased odds of being seropositive (1.26; 95% CI, 1.07-1.49 and 2.26; 95% CI, 1.51-3.37, respectively). Increased odds were seen for HCWs working in the ED (1.64; 95% CI, 1.14-2.36) and inpatient units (1.35; 95% CI, 1.08-1.69), and decreased odds were seen for those working in the laboratory and pharmacy (0.47; 95% CI, 0.26-0.86). Increased odds for seropositivity were found for nurses (2.27; 95% CI, 1.56-3.31), allied HCW I (2.00; 95% CI, 1.34-2.97), allied HCW II (1.78; 95% CI, 1.22-2.60), and nonclinical staff (2.97; 95% CI,1.80-4.90).

After adjusting for all covariates, HCWs who were Black remained at increased odds for being seropositive in the two final models (adjusted OR, 2.29; 95% CI, 1.38-3.81 and adjusted OR, 2.94; 95% CI, 1.78-4.85), as did those who had a BMI >30 kg/m², with an adjusted OR of 1.36 (95% CI, 1.05-1.77) in one of the final models (Appendix Table 3). None of the other comorbid conditions had increased ORs. Those who worked in the ED and inpatient units also remained at increased odds after adjusting for covariates (2.27; 95% CI, 1.53-3.37 and 1.48; 95% CI, 1.14-1.92, respectively; Figure 3). Other job functions that had increased odds for seropositivity were nurses (2.54; 95% CI, 1.64-3.94), allied HCW I (1.83; 95% CI, 1.15-2.89) and II (1.70; 95% CI, 1.10-2.63), and nonclinical staff (2.51; 95% CI, 1.42-4.43).

Having a positive PCR for SAR-CoV-2 on nasopharyngeal swabs was strongly associated with seropositivity (OR, 47.26; 95% CI, 29.30-76.23 and OR, 44.79; 95% CI, 27.87-72.00) in the two multivariate-adjusted models. These findings were confirmed when the analyses were performed on HCWs who resided in NYC (Appendix Table 4 and Appendix Figure 2).

In a large inner-city New York hospital, we report a cumulative SARS-CoV-2 seroprevalence of 30.2% in HCWs at the end of the first surge of SARS-CoV-2 infections in NYC. We identified the highest seropositivity rates for nonclinical staff and nurses, followed by allied HCWs, with the odds of being seropositive ranging from 1.7 to 2.5. The work locations with the highest seroprevalences were the ED and inpatient units, with 2.3-fold and 1.5-fold increased odds of seropositivity, respectively.

Serosurveillance studies have reported the trajectory of community prevalence in NYC over the first wave. A 6.3% prevalence was reported in samples collected between March 23 and April 1, 2020.¹⁷ In a study by Rosenberg et al¹⁸ with testing performed from April 9 through April 28, 2020, prevalence increased to 22.7%. Serosurveillance data from the NYC Department of Health show prevalence ranging from 20.1% to 23.3% (average 22%) during the study period.¹⁶ Compared to the estimated seroprevalence of 9.3% in the United States,¹⁹ these rates established NYC as an early epicenter for the COVID-19 pandemic, with our institution’s HCW seroprevalence considerably higher than NYC community serosurveillance rates, 2.2 times higher than reported in the earlier HCW study in the greater NYC area,¹³ and higher than the 27% rate during May 2020 recently reported in another NYC hospital.²⁰

Data from studies of hospital transmission and effects of mitigation measures, such as a universal masking policy for HCWs and patients, clearly demonstrate the high effectiveness of these measures in reducing hospital transmissions.^21,22 This suggests HCW seroprevalence in institutions with well-implemented infection control and universal masking policies may not be a consequence of workplace exposures, but rather may be reflective of community rates.²³ Our institution’s response commenced February 3, 2020, with implementation of social distancing, a universal masking policy, transmission-based precautions, and use of fitted N95 masks. Mid-March, elective surgeries were canceled, and inpatient visitation suspended. During the surge, these measures were widely and consistently implemented for all categories of HCWs throughout the work environment, based on emerging guidelines from the Centers for Disease Control and Prevention (CDC) and NYC Department of Health. Our overall observed HCW seroprevalence, well above that of the community, with differences in categories of job function and work locations, is therefore an important finding. Our sample of 2,749 HCWs lived in NYC and its surrounding suburbs and nearby states. There is heterogeneity in community seroprevalence between areas outside of NYC and NYC (an epicenter) itself. We therefore analyzed our data in the subset with NYC zip codes, confirming a similar overall prevalence and increased odds of seropositivity in nurses, allied HCWs, and nonclinical staff.

Physicians and administrative and office staff had seropositivity rates of 18.1%, 20.9%, and 25.8%, respectively, consistent with community rates and illustrating the effectiveness of PPE in the hospital setting. Since PPE use was part of a universal policy applied to all HCWs in our institution, other possible reasons may explain the differences we found. We speculate that the close working relationship nurses have with their patients resulted in a longer duration and higher frequency of daily interactions, increasing the risk for transmission and causing breakthrough infections.^24,25 This increased risk is reflected in a study in which 28% of hospitalized patients were nurses and 9% certified nursing assistants.²⁶

The CDC recently redefined close contact with someone with COVID-19 as a cumulative total of >15 minutes over 24 hours.²⁵ Thus, several multiple short periods of exposure can increase risk for infection with SARS-CoV-2; such exposure is characteristic of the job function of nurses, nursing staff, and nonclinical staff. Further, housekeeping, transportation, and security officers are all nonclinical staff with significant and multiple exposures to COVID-19 patients during the surge, and for security officers, to continuous public traffic in and out of the hospital. SARS-CoV-2 spreads by virus shedding in large droplets and aerosols, with droplet nuclei <5 microns in size efficiently dispersed in air, an important additional mode of transmission.^27-30 Airborne transmission coupled with virus shedding in asymptomatic and presymptomatic persons, which has been shown to cause secondary attack rates of up to 32%, are other factors that likely contributed to the increased seroprevalence in this group.³¹ Our observation is consistent with the Birmingham study, which reported the highest rate in housekeeping staff, with a prevalence of 34.5%, compared to 44% in this study.⁶ Similar reasons for high seropositivity rates apply to the two groups of allied HCWs (eg, medical assistants and patient care technicians, social workers, nutritionists and therapists), whose job functions place them in intermittent but significant proximity with inpatients and outpatients.

Consistent with public health data showing that minorities are disproportionately affected by this disease, we found that Black HCWs were three times more likely to be seropositive.³² However, an unexpected observation was the association between obesity and SARS-CoV-2 seropositivity. A possible explanation for this association may be inability to achieve optimal fit testing for N95 masks, thereby increasing the risk of exposure to droplet nuclei. This is important given that obesity is associated with poorer outcomes from COVID-19.

During the height of the first wave in NYC, EDs and inpatient units handled a large volume of COVID-19 patients with high PCR positivity rates (peak of 51% in March in our hospital). It was not unexpected that we observed increased odds of seropositivity in these work locations. As ICUs were at capacity, inpatient units cared for critically ill patients they would not normally have. HCWs in these locations coped with an increased workload, increased demand on PPE supplies, and work fatigue, which contributed to increased risk for hospital-acquired SARS-CoV-2 infections.

Reporting seroprevalence at a single institution was a limitation of the study. Approximately 57% of the hospital’s total HCW population was tested for antibodies. It is possible their risk profile influenced their decision to volunteer for testing when it became available, introducing selection bias. A comparison between tested and untested HCWs showed similarity in all demographic measures, including nasopharyngeal PCR testing, except for age. We did not have information on symptoms that would prompt PCR testing. HCWs who underwent voluntary testing were younger compared to those who did not undergo testing. Current NYC serosurveillance data showed higher seropositivity in the 45-to-64–year age group (27.8%-28.6%) compared to the 65-to-74–year age group (24.3%), which suggests that the tested group may overestimate seroprevalence among HCWs relative to a randomly selected sample.³³ Similarly, there were more nurses, allied HCWs, physicians, and administrative staff in the tested group, with the former two having higher SARS-CoV-2 seropositivity compared to community prevalence, which could also overestimate seroprevalence. Our large sample size provided us with the power to detect differences within several different job functions and work locations, a strength of this study. It was not possible to differentiate community- from hospital-acquired infection in our HCWs, a limitation in many observational HCW seroprevalence studies. However, when we analyzed data restricted only to HCWs in NYC, to reduce the effect of differing community prevalences outside the city, our results were unchanged. Since it is possible that nonclinical HCWs are of a lower socioeconomic status compared to others (nurses and allied HCWs), we cannot exclude the possibility that higher SARS-CoV-2 seroprevalence associated with lower status explains, partly or completely, the increased odds of seropositivity we observed.³⁴ Due to the high proportion of missing data for race (61.3%), we advise caution in interpreting our finding that the odds of seropositivity were three times higher for Black race, even though consistent with prior literature.³⁴ Healthcare organizations have similar job function and work location categories incorporated in their infrastructure, suggesting that our observations may be generalizable to other hospitals in the United States.

These findings show that during the first surge in NYC, with its increased burden of disease, hospitalizations, morbidity, and mortality, seroprevalences varied based on job function and work location within this institution. Nurses were at highest risk for SARS-CoV-2 infection, as were those who worked in the ED. In preparation for subsequent waves of SARS-CoV-2 and other highly contagious respiratory infections, major medical centers need to enhance efforts aimed at protecting HCWs, with particular attention to these groups. This study also strongly supports the recent CDC guideline prioritizing HCWs to receive COVID-19 mRNA and adenovirus vector vaccines that have obtained emergency use authorization by the US Food and Drug Administration.³⁵

The authors thank all the residents, nurses, and staff of the Department of Family Medicine for their contribution to this work.

SARS-CoV-2 has infected 141 million people worldwide and 31 million people in the United States as of April 20, 2021.^1,2 The influx of hospital admissions and deaths has severely strained healthcare systems worldwide and placed healthcare workers (HCWs) at increased risk for acquiring COVID-19.^3-5

Several studies have described the impact of COVID-19 on this heterogeneous group of HCWs. Shields et al reported a seroprevalence of 24.4% in HCWs at University Hospitals Birmingham (UK), with the highest rate, 34.5%, in housekeeping staff.⁶ Steensels et al reported a lower prevalence of 6.4% at a tertiary care center in Belgium, and showed no increased risk for HCWs when directly involved in clinical care.⁷ The authors attributed this to adequate use of personal protective equipment (PPE). Other studies have reported seroprevalences ranging from 1.6% to 18%.^8-11 In the New York City (NYC) metro area, Jeremias et al reported a seroprevalence of 9.8% in HCWs and found no difference by job title or work location,¹² whereas Moscola et al reported a seroprevalence of 13.7% and demonstrated a 3% increased risk for those working in service or maintenance.¹³ Antibody tests were conducted between March and April 2020 in all but two of these studies; testing in these two studies was performed between April 13 and June 23, 2020, with one reporting a seroprevalence of 6%¹¹ and the other, 13.7%.¹³

NYC became the earliest pandemic epicenter in the United States following untracked transmission from ongoing circulation of SARS-CoV-2 in Europe.¹⁴ As a result, the COVID-19 surge in NYC commenced in March and largely subsided by the end of May 2020. Most HCW data reported to date do not reflect the situation at the end of the surge, and may underestimate true seroprevalence. We describe SARS-CoV-2 seroprevalence in HCWs in a large inner-city hospital in NYC, with antibody testing conducted from May 18 to June 26, 2020, at the subsidence of the surge. To further our understanding of occupational risk among different groups of HCWs, we examined associations of seroprevalence with HCWs’ job function and work location.

This was a cross-sectional seroprevalence study conducted in the BronxCare Health System located in South and Central Bronx, an area that experienced one of the highest incidences of SARS-CoV-2 infections within NYC’s five boroughs.

HCWs were offered voluntary testing for serum antibodies to SARS-CoV-2 between May 18 and June 26, 2020. Testing occurred in the institution’s auditorium, a central and easily accessible location. Weekly emails were sent to all employees and department heads during the testing period, offering antibody testing and providing location and testing time information. The Elecsys Anti-SARS-CoV-2 (Roche) assay measuring total qualitative antibodies was used; the assay has a reported sensitivity of 97.1% 14 days after a positive SARS-CoV-2 RNA polymerase chain reaction (PCR) test result and a specificity of 100%.¹⁵

Demographic and work-related information was abstracted from electronic medical records, including all comorbid conditions that affected 30 or more HCWs. Pulmonary diagnoses, including asthma and chronic obstructive pulmonary disease, were grouped as chronic lung disease, and cardiovascular diseases, including hypertension, as chronic heart disease. Personal identifiers and data were delinked upon completion of data abstraction. The study was approved by the hospital’s institutional review board.

Job Function and Work Location

HCWs were grouped by job function as follows: physicians; nurses (including physician assistants and nurse practitioners); allied HCW I (medical assistants, patient care, and electrocardiogram, radiology, and ear, nose and throat technicians); allied HCW II (social workers, dieticians and nutritionists, registration clerks and unit associates, physical and occupational therapists); nonclinical staff (patient transporters, housekeeping staff, and security staff); pharmacists; engineering; and administrative staff. Respiratory therapists were considered as a separate group as their work placed them at high risk for respiratory diseases.

Work locations were as follows: clinics (including dental, outpatient, and satellite clinics), emergency departments (ED), inpatient units (including floors and intensive care units [ICU]), radiology suite, laboratory and pharmacy, and offices.

Statistical Analysis

Descriptive statistics were calculated using χ² analyses. All demographic variables were tested against serology status (positive/negative). A binary logistic regression analysis was used to calculate odds ratios (ORs). Eight separate univariate unadjusted ORs were calculated by running each predictor variable against serology status (dependent variable), which included the six categorical variables—race, ethnicity, age, sex, body mass index (BMI), and prior SARS-CoV-2 PCR results—and the two main predictors—job function and work location. To obtain adjusted ORs, two final separate multivariable logistic regression analyses were executed including the six covariates listed. Due to high collinearity between job function and work location (χ² = 3030.13, df = 35 [6 levels of work location – 1]*[8 levels of job function – 1]; P < .001), we included only one of the main predictors in each model. The regressions were specified such that the reference groups for the work location and job function variables were office work and administration, respectively. This choice was made based on the fact that their nonclinical functions do not confer an exposure risk in excess of that experienced by typical community populations. Sensitivity analyses were performed on the subset of HCWs whose address zip codes indicated residence within NYC to exclude the effect of different community seroprevalences in areas outside of NYC. The 95% CI for seroprevalence of antibodies within tested HCWs was estimated using the Clopper-Pearson binomial method.

Among all HCWs in the institution (N = 4,807), 2,749 (57.2%) underwent voluntary testing. Of those who underwent testing, 831 were positive for antibodies to SARS-CoV-2 (Figure 1), a seroprevalence of 30.2% (95% CI, 29%-32%). Among the age groups, the 45-to-64−year group had the highest seropositivity at 33% (400/1203), and those ≥75 years of age, the lowest at 16.7% (2/12) (P < .009).

Among all tested HCWs, 70.1% (1,928/2,749) resided in NYC. SARS-CoV-2 seroprevalence in this subset was 32% (616/1,928) (Figure 1). Demographic and comorbid conditions in HCWs who lived in NYC were similar to those of the whole group (Appendix Table 1).

HCWs who underwent voluntary antibody testing (Appendix Table 2) had a higher percentage of persons in the 45-to-64−year age group (43.8% vs 40.9%) and a lower percentage of persons in the 65-to-74−year age group (3.3% vs 5.3%) compared with the group of HCWs that did not undergo testing (P < .001). Gender, race, ethnicity, comorbid conditions, SARS-CoV-2 PCR testing, and work locations were not different between groups. The tested group had higher proportions of clinicians (physicians, nurses, allied HCWs I and II) than the untested nonparticipant group (P = .014).

SARS-CoV-2 PCR Tests on HCWs

More than one-third (34.1%; 938/2,749) of HCWs had a documented nasopharyngeal PCR test between March 23 and June 26, 2020 (Table). Of all PCRs performed, 262 were positive, giving an overall PCR positivity rate of 27.9%. Positivity was 51.4% in March and 36.6% in April. The reasons for PCR testing were not available, but likely represent a combination of exposure-related testing among asymptomatic individuals and diagnostic testing of symptomatic HCWs. In contrast, serology testing was indicative of prior infection and yielded a cumulative seroprevalence at the end of the surge. Findings were similar among HCWs residing in NYC (Appendix Table 1).

Work Location and Job Function

Among all HCWs (Table, Figure 2), there were differences in seropositivity by work location (P = .001). The largest number of HCWs worked in inpatient units (1,348/2,749, 49%), and the second largest in offices (554/2,749, 20%). The highest seropositivity rate was in the EDs, at 36.4% (64/176), followed by radiology suites, at 32.7% (17/52); the seropositivity rate in office locations was 25.8% (143/554). Among HCWs residing in NYC (Appendix Table 1, Appendix Figure 1), the rank order according to proportion seropositive by work location was similar to that of the whole group (P = .004), except that the second highest seropositivity rate was in the inpatient units (33.9% [323/953]). In the group of HCWs residing in NYC, office locations had a seropositivity of 27.4% (102/372). The seropositivity rates for both groups working in office locations were slightly higher than the 22% community seroprevalence in NYC reported for the same period.¹⁶

Among all HCWs, there were differences in seropositivity by job function (P = .001). The greatest proportion of HCWs were allied HCW II (23% [631/2,749]), followed by nurses (22.2% [611/2,749]) and physicians (21.3% [585/2,749] ). Seropositivity was highest for nonclinical staff (44.0% [51/116]), followed by nurses (37.5% [229/611]) and allied clinical HCW I and II (34.5% [143/414] and 32.0% [202/631], respectively). It was lowest for administrative staff (20.9% [42/201]) and pharmacists (11.1% [5/45]). Among HCWs residing in NYC, the rank order according to proportion seropositive by location was similar to that of the whole group. Administrative staff seropositivity was 18.3% (20/109). Administrative staff seropositivity for both groups was marginally lower than the 22% community seroprevalence in NYC for the same period.¹⁶

Odds Ratios for SARS-CoV-2 Seropositivity

For all HCWs, in unadjusted models (Appendix Table 3), age 45 to 64 years and Black race were associated with increased odds of being seropositive (1.26; 95% CI, 1.07-1.49 and 2.26; 95% CI, 1.51-3.37, respectively). Increased odds were seen for HCWs working in the ED (1.64; 95% CI, 1.14-2.36) and inpatient units (1.35; 95% CI, 1.08-1.69), and decreased odds were seen for those working in the laboratory and pharmacy (0.47; 95% CI, 0.26-0.86). Increased odds for seropositivity were found for nurses (2.27; 95% CI, 1.56-3.31), allied HCW I (2.00; 95% CI, 1.34-2.97), allied HCW II (1.78; 95% CI, 1.22-2.60), and nonclinical staff (2.97; 95% CI,1.80-4.90).

After adjusting for all covariates, HCWs who were Black remained at increased odds for being seropositive in the two final models (adjusted OR, 2.29; 95% CI, 1.38-3.81 and adjusted OR, 2.94; 95% CI, 1.78-4.85), as did those who had a BMI >30 kg/m², with an adjusted OR of 1.36 (95% CI, 1.05-1.77) in one of the final models (Appendix Table 3). None of the other comorbid conditions had increased ORs. Those who worked in the ED and inpatient units also remained at increased odds after adjusting for covariates (2.27; 95% CI, 1.53-3.37 and 1.48; 95% CI, 1.14-1.92, respectively; Figure 3). Other job functions that had increased odds for seropositivity were nurses (2.54; 95% CI, 1.64-3.94), allied HCW I (1.83; 95% CI, 1.15-2.89) and II (1.70; 95% CI, 1.10-2.63), and nonclinical staff (2.51; 95% CI, 1.42-4.43).

Having a positive PCR for SAR-CoV-2 on nasopharyngeal swabs was strongly associated with seropositivity (OR, 47.26; 95% CI, 29.30-76.23 and OR, 44.79; 95% CI, 27.87-72.00) in the two multivariate-adjusted models. These findings were confirmed when the analyses were performed on HCWs who resided in NYC (Appendix Table 4 and Appendix Figure 2).

In a large inner-city New York hospital, we report a cumulative SARS-CoV-2 seroprevalence of 30.2% in HCWs at the end of the first surge of SARS-CoV-2 infections in NYC. We identified the highest seropositivity rates for nonclinical staff and nurses, followed by allied HCWs, with the odds of being seropositive ranging from 1.7 to 2.5. The work locations with the highest seroprevalences were the ED and inpatient units, with 2.3-fold and 1.5-fold increased odds of seropositivity, respectively.

Serosurveillance studies have reported the trajectory of community prevalence in NYC over the first wave. A 6.3% prevalence was reported in samples collected between March 23 and April 1, 2020.¹⁷ In a study by Rosenberg et al¹⁸ with testing performed from April 9 through April 28, 2020, prevalence increased to 22.7%. Serosurveillance data from the NYC Department of Health show prevalence ranging from 20.1% to 23.3% (average 22%) during the study period.¹⁶ Compared to the estimated seroprevalence of 9.3% in the United States,¹⁹ these rates established NYC as an early epicenter for the COVID-19 pandemic, with our institution’s HCW seroprevalence considerably higher than NYC community serosurveillance rates, 2.2 times higher than reported in the earlier HCW study in the greater NYC area,¹³ and higher than the 27% rate during May 2020 recently reported in another NYC hospital.²⁰

Data from studies of hospital transmission and effects of mitigation measures, such as a universal masking policy for HCWs and patients, clearly demonstrate the high effectiveness of these measures in reducing hospital transmissions.^21,22 This suggests HCW seroprevalence in institutions with well-implemented infection control and universal masking policies may not be a consequence of workplace exposures, but rather may be reflective of community rates.²³ Our institution’s response commenced February 3, 2020, with implementation of social distancing, a universal masking policy, transmission-based precautions, and use of fitted N95 masks. Mid-March, elective surgeries were canceled, and inpatient visitation suspended. During the surge, these measures were widely and consistently implemented for all categories of HCWs throughout the work environment, based on emerging guidelines from the Centers for Disease Control and Prevention (CDC) and NYC Department of Health. Our overall observed HCW seroprevalence, well above that of the community, with differences in categories of job function and work locations, is therefore an important finding. Our sample of 2,749 HCWs lived in NYC and its surrounding suburbs and nearby states. There is heterogeneity in community seroprevalence between areas outside of NYC and NYC (an epicenter) itself. We therefore analyzed our data in the subset with NYC zip codes, confirming a similar overall prevalence and increased odds of seropositivity in nurses, allied HCWs, and nonclinical staff.

Physicians and administrative and office staff had seropositivity rates of 18.1%, 20.9%, and 25.8%, respectively, consistent with community rates and illustrating the effectiveness of PPE in the hospital setting. Since PPE use was part of a universal policy applied to all HCWs in our institution, other possible reasons may explain the differences we found. We speculate that the close working relationship nurses have with their patients resulted in a longer duration and higher frequency of daily interactions, increasing the risk for transmission and causing breakthrough infections.^24,25 This increased risk is reflected in a study in which 28% of hospitalized patients were nurses and 9% certified nursing assistants.²⁶

The CDC recently redefined close contact with someone with COVID-19 as a cumulative total of >15 minutes over 24 hours.²⁵ Thus, several multiple short periods of exposure can increase risk for infection with SARS-CoV-2; such exposure is characteristic of the job function of nurses, nursing staff, and nonclinical staff. Further, housekeeping, transportation, and security officers are all nonclinical staff with significant and multiple exposures to COVID-19 patients during the surge, and for security officers, to continuous public traffic in and out of the hospital. SARS-CoV-2 spreads by virus shedding in large droplets and aerosols, with droplet nuclei <5 microns in size efficiently dispersed in air, an important additional mode of transmission.^27-30 Airborne transmission coupled with virus shedding in asymptomatic and presymptomatic persons, which has been shown to cause secondary attack rates of up to 32%, are other factors that likely contributed to the increased seroprevalence in this group.³¹ Our observation is consistent with the Birmingham study, which reported the highest rate in housekeeping staff, with a prevalence of 34.5%, compared to 44% in this study.⁶ Similar reasons for high seropositivity rates apply to the two groups of allied HCWs (eg, medical assistants and patient care technicians, social workers, nutritionists and therapists), whose job functions place them in intermittent but significant proximity with inpatients and outpatients.

Consistent with public health data showing that minorities are disproportionately affected by this disease, we found that Black HCWs were three times more likely to be seropositive.³² However, an unexpected observation was the association between obesity and SARS-CoV-2 seropositivity. A possible explanation for this association may be inability to achieve optimal fit testing for N95 masks, thereby increasing the risk of exposure to droplet nuclei. This is important given that obesity is associated with poorer outcomes from COVID-19.

During the height of the first wave in NYC, EDs and inpatient units handled a large volume of COVID-19 patients with high PCR positivity rates (peak of 51% in March in our hospital). It was not unexpected that we observed increased odds of seropositivity in these work locations. As ICUs were at capacity, inpatient units cared for critically ill patients they would not normally have. HCWs in these locations coped with an increased workload, increased demand on PPE supplies, and work fatigue, which contributed to increased risk for hospital-acquired SARS-CoV-2 infections.

Reporting seroprevalence at a single institution was a limitation of the study. Approximately 57% of the hospital’s total HCW population was tested for antibodies. It is possible their risk profile influenced their decision to volunteer for testing when it became available, introducing selection bias. A comparison between tested and untested HCWs showed similarity in all demographic measures, including nasopharyngeal PCR testing, except for age. We did not have information on symptoms that would prompt PCR testing. HCWs who underwent voluntary testing were younger compared to those who did not undergo testing. Current NYC serosurveillance data showed higher seropositivity in the 45-to-64–year age group (27.8%-28.6%) compared to the 65-to-74–year age group (24.3%), which suggests that the tested group may overestimate seroprevalence among HCWs relative to a randomly selected sample.³³ Similarly, there were more nurses, allied HCWs, physicians, and administrative staff in the tested group, with the former two having higher SARS-CoV-2 seropositivity compared to community prevalence, which could also overestimate seroprevalence. Our large sample size provided us with the power to detect differences within several different job functions and work locations, a strength of this study. It was not possible to differentiate community- from hospital-acquired infection in our HCWs, a limitation in many observational HCW seroprevalence studies. However, when we analyzed data restricted only to HCWs in NYC, to reduce the effect of differing community prevalences outside the city, our results were unchanged. Since it is possible that nonclinical HCWs are of a lower socioeconomic status compared to others (nurses and allied HCWs), we cannot exclude the possibility that higher SARS-CoV-2 seroprevalence associated with lower status explains, partly or completely, the increased odds of seropositivity we observed.³⁴ Due to the high proportion of missing data for race (61.3%), we advise caution in interpreting our finding that the odds of seropositivity were three times higher for Black race, even though consistent with prior literature.³⁴ Healthcare organizations have similar job function and work location categories incorporated in their infrastructure, suggesting that our observations may be generalizable to other hospitals in the United States.

These findings show that during the first surge in NYC, with its increased burden of disease, hospitalizations, morbidity, and mortality, seroprevalences varied based on job function and work location within this institution. Nurses were at highest risk for SARS-CoV-2 infection, as were those who worked in the ED. In preparation for subsequent waves of SARS-CoV-2 and other highly contagious respiratory infections, major medical centers need to enhance efforts aimed at protecting HCWs, with particular attention to these groups. This study also strongly supports the recent CDC guideline prioritizing HCWs to receive COVID-19 mRNA and adenovirus vector vaccines that have obtained emergency use authorization by the US Food and Drug Administration.³⁵

The authors thank all the residents, nurses, and staff of the Department of Family Medicine for their contribution to this work.

India is emerging as one of the world’s largest hotspots for SARS-CoV-2 infection (COVID-19)—second only to the United States—with more than 13,000,000 documented infections since the first case was recorded on January 30, 2020.^1,2 Kashmir, a northern territory of India, reported its first case of COVID-19 on March 18, 2020, from the central District Srinagar; this region has accounted for more cases of COVID-19 than any other district throughout the pandemic.³ The large majority of healthcare in District Srinagar is provided by three tertiary care institutions, one district hospital, two subdistrict hospitals, and 70 primary healthcare centers. Potential occupational exposures place healthcare workers (HCWs) at higher risk of acquiring SARS-CoV-2 infection, which in turn may serve as an important source of infection for their families and other community members.^4-6 Given the high frequency and geographic variability of asymptomatic infection, growing evidence suggests this hidden reservoir is a source of infection for the general population.^7,8

Many countries have started testing for antibodies against SARS-CoV-2, both at the population level and in specific groups, such as HCWs. Seroepidemiological studies are crucial to understanding the dynamics of SARS-CoV-2 infection. Many seroepidemiological studies have been conducted among community populations, but there are insufficient data on HCWs. The World Health Organization also encouraged its member states to conduct seroepidemiological studies to attain a better understanding of COVID-19 infection prevalence and distribution.^9-11 Therefore, to quantify the prevalence of SARS-CoV-2 infection among HCWs, we conducted a seroepidemiological study by testing for SARS-CoV-2–specific immunoglobulin (IgG) to gain insight into the extent of infection among specific subgroups of HCWs and to identify risk-factor profiles associated with seropositivity.

Study Design and Settings

We conducted this seroepidemiological study to ascertain the presence of IgG antibodies against SARS-CoV-2 among HCWs in the District Srinagar of Kashmir, India. The 2-week period of data collection began on June 15, 2020. As part of healthcare system pandemic preparedness efforts, India’s Ministry of Health provided specific guidelines for health facilities to manage COVID-19. Hospitals were categorized as dedicated COVID and non-COVID hospitals. Dedicated COVID hospitals provided comprehensive care exclusively to patients with COVID-19 and were equipped with fully functional intensive care units, ventilators, and beds with reliable access to oxygen support.¹² In addition, infection prevention and control strategies to limit the transmission of SARS-CoV-2 infection were implemented according to guidelines specified by India’s National Center for Disease Control.¹³ To strengthen service provision, HCWs from other hospitals, including resident physicians, were relocated to these dedicated COVID hospitals. The additional staff were selected by administrative leadership, without input from HCWs.

Study Population and Data Collection

We approached administrative heads of the hospitals in District Srinagar for permission to conduct our study and to invite their HCWs to participate in the study. As Figure 1 shows, we were denied permission by the administrative heads of two tertiary care hospitals. Finally, with a point person serving as a study liaison at each institution, HCWs from three dedicated COVID and seven non-COVID tertiary care hospitals, two subdistrict hospitals, and six primary healthcare centers across the District Srinagar were invited to participate. The sample primary healthcare centers were each selected randomly, after stratification, from six major regions of the district. All frontline HCWs, including physicians, administrative and laboratory personnel, technicians, field workers involved in surveillance activity, and other supporting staff were eligible for the study.

We collected information on an interview form using Epicollect5, a free data-gathering tool widely used in health research.¹⁴ Physicians specifically trained in the use of Epicollect5 conducted the face-to-face interview on a prespecified day and recorded the collected information through mobile phones. This information included the participants’ role in providing care to patients with COVID-19 and risk factors for SARS-CoV-2 infection (eg, history of travel since January 1, 2020, symptoms of an influenza-like illness [ILI] in the 4 weeks prior to the interview, close contact with a COVID-19 case). We defined close contact as an unmasked exposure within 6 feet of an infected individual for at least 15 minutes, irrespective of location (ie, community or the hospital).

Following the interview, trained phlebotomists collected 3 to 5 mL of venous blood under aseptic conditions. We strictly adhered to standard operating procedures during collection, transportation, and testing of blood samples. Following collection, the blood samples remained undisturbed for at least 30 minutes before centrifugation, which was performed at the collection site (or at the central laboratory for sites lacking the capability). The samples were then transported for further processing and testing through a cold chain supply line, using vaccine carriers with conditioned icepacks. All testing procedures were conducted with strict adherence to the manufacturers’ guidelines.

Laboratory Procedure

In accordance with the manufacturer’s recommendations, we used a chemiluminescent microparticle immunoassay to detect SARS-CoV-2–specific IgG antibodies in serum samples. The assay is an automated two-step immunoassay for the qualitative detection of IgG antibodies against the nucleocapsid of SARS-CoV-2 in human serum and plasma. The sensitivity and specificity of this test are 100% and 99%, respectively. The test result was considered positive for SARS-CoV-2 IgG if the index value was ≥1.4, consistent with guidance provided by the manufacturer.¹⁵

The IgG values were also entered into Epicollect5. Two trained medical interns independently entered the laboratory results in two separate forms. A third medical intern reviewed these forms for discrepancies, in response to which they referenced the source data for adjudication. The information gathered during the interview and the laboratory results were linked with the help of a unique identification number, which was generated at the time of the interview.

Statistical Analysis

We estimated the proportion (and logit-transformed 95% CI) of HCWs with a positive SARS-CoV-2–specific IgG antibody level, the primary outcome of interest. We compared seroprevalence rates by gender, age group, specific occupational group, and type of health facility (dedicated COVID hospital vs non-COVID hospital). Seroprevalence was also estimated separately for HCWs who reported symptoms in the past 4 weeks, had a history of exposure to a known case of COVID-19, or had undergone testing by reverse transcriptase-polymerase chain reaction (RT-PCR). In the case of zero seroprevalences, Jeffreys 95% CIs were reported. We used a chi-square test to report two-sided P values for comparison of seroprevalence between groups. When the expected frequency was <5 in more than 20% of the cells, the exact test was used instead of the chi-square test. We additionally performed multivariable logistic regression analysis to evaluate the independent association between place of work (primary independent variable) and seropositivity (dependent variable). We adjusted for the following observable covariates by including them as categorical variables: age, gender, occupational group, and history of close contact with a patient who was COVID-positive. We performed data analysis using Stata, version 15.1 (StataCorp LP). The Institutional Ethics Committee of Government Medical College, Srinagar, approved the study (Reference No. 1003/ETH/GMC dated 13-05-2020). We obtained written, informed consent from all participants.

Of the 7,346 HCWs we were granted permission to approach, 2,915 (39.7%) agreed to participate in the study. The participation rate was 49% at the dedicated COVID hospitals (57% physicians and 47% nonphysicians) and 39% at the non-COVID hospitals (46% physicians and 36% nonphysicians). We analyzed information gathered from 2,905 HCWs (Epicollect5 interview forms were missing for nine participants, and the laboratory report was missing for one participant).

The mean age of the participants was 38.6 years, and 35.8% of participants identified as female (Table 1). One third (33.7%) of the participants were physicians, nearly half of whom were residents. In our sample, the overall seroprevalence of SARS-CoV-2–specific antibodies was 2.5% (95% CI, 2.0%-3.1%).

Of the 2,905 participating HCWs, 123 (4.2%) reported an ILI (ie, fever and cough) in the 4 weeks preceding the interview, and 339 (11.7%) reported close contact with a person with COVID-19 (Table 2). A total of 760 (26.2%) HCWs had undergone RT-PCR testing, 29 (3.8%) of whom had a positive result. Stratifying by workplace, history of nasopharyngeal RT-PCR positivity was reported by 4 of 77 (5.1%) participants from dedicated COVID hospitals compared to (3.7%) participants from the non-COVID hospital (P = .528).

As Table 2 also demonstrates, we found a significantly higher seropositivity rate among HCWs who had a history of ILI (P < .001), a history of positive RT-PCR (P < .001), history of ever being put under quarantine (P = .009), and a self-reported history of close contact with a person with COVID-19 (P = .014). Healthcare workers who had ever worked at a dedicated COVID hospital had a significantly lower seroprevalence of infection (P = .004).

Among HCWs who reported no ILI symptoms in the 4 weeks prior to the interview but who had positive RT-PCR test, 20.8% were seropositive. Of HCWs who reported both ILI and a positive RT-PCR test result, 60.0% were seropositive. Compared to employment at a non-COVID hospital, HCWs working in dedicated COVID hospitals had a reduced multivariate-adjusted risk of seropositivity (odds ratio, 0.21; 95% CI, 0.06-0.66).

We aimed to estimate the seroprevalence of SARS-CoV-2 infection in HCWs in different hospital settings in the District Srinagar of Kashmir, India. In general, seroprevalence was low (2.5%), with little difference across gender or occupational group.

Seroprevalence studies of HCWs across divergent workplace environments have revealed estimates ranging from 1% to 10.2%.^16-19 Generally, the seroprevalence rates among HCWs are not significantly different from those of the general population, which reflects how different the dynamics of COVID-19 are compared to other infections in healthcare settings. The low seroprevalence observed in our study coincides with the overall low infection rate in the community population. During the study period, District Srinagar reported a median of 28 new infections daily (interquartile range, 17-46), which is indicative of the early phase of the pandemic in the population at the time of the study.²⁰

Among the HCW occupational groups, ambulance drivers and housekeeping staff had the highest seroprevalence rates, followed by nurses and physicians. Possible explanations for higher seropositivity in these groups are improper use or inadequate supply of protective gear and lack of training on the use of personal protective equipment (PPE), resulting in increased exposure risk.²¹ Concordance of HCW and community infection rates in specific geographic areas suggests that community exposure may be the dominant source of healthcare exposure and infection. Additionally, careful in-hospital behavior of HCWs in dedicated COVID hospitals may have had a spillover effect on their out-of-hospital behavior, which may partially explain our finding that employment at dedicated COVID hospitals was associated with a markedly lower chance of seropositivity. A study of 6,510 HCWs in Chicago, Illinois, showed high seropositivity rates among support service workers, medical assistants, and nurses, with nurses identified as having a markedly higher adjusted odds of seropositivity relative to administrators. The authors of the study concluded that exposure in the community setting plays a crucial role in transmission among HCWs.²² Similarly, higher seroprevalence among housekeeping, nonadministrative staff, and other support service staff has been reported elsewhere.²³ Certain underlying factors related to socioeconomic status and lifestyle may also contribute to higher seroprevalence in some occupational groups.²⁴ Nonadherence to masking, social distancing, and proper hand hygiene outside the hospital setting could result in community-acquired infection.

Interestingly, participants who were working in a dedicated COVID hospital or who had ever worked at one had a seroprevalence of 0.6%, much lower than the 2.8% observed among other participants. This difference remained statistically significant after controlling for age, sex, place of work, and occupational group. As these facilities were dedicated to the management and care of patients with COVID-19, the hospital staff strictly adhered to safety precautions, with particular vigilance during patient contact. These hospitals also strictly adhered to infection prevention and control practices based on the latest guidelines released by India’s Ministry of Health and Family Welfare.¹³

A commitment was made to provide adequate PPE to the dedicated COVID hospitals and staff, commensurate with expected infected patient volumes and associated exposure risks. Healthcare workers were specifically trained on proper donning and doffing of PPE, self-health monitoring, and protocols for reporting symptoms and PPE breaches during patient encounters. Healthcare workers were regularly tested for COVID-19 using nasopharyngeal RT-PCR. Of critical importance, these hospitals implemented a buddy system wherein a team of two or more staff members was responsible for ensuring each other’s safety, proper PPE use, conformance to other protective measures, and reporting breaches of PPE compliance.²⁵ Universal masking was mandatory for all hospital staff and patients at the COVID-focused facilities, with the additional use of N-95 masks, gloves, and face shields during times of patient contact. Administrative measures, including visitor restrictions and environmental sanitation, were rigorously enforced. Also, being a potentially high-risk area for transmission of infection, these facilities implemented staff-rationing to reduce the duration of exposure to the healthcare staff. Third, the HCWs of COVID-dedicated hospitals were provided with separate living accommodations during the period in which they were employed at a dedicated COVID hospital.

In contrast, in non-COVID hospitals, with the exception of HCWs, patients and the hospital visitors were not subject to a masking policy. Moreover, an adequate and timely supply of PPE was not prioritized at the non-COVID facilities due to resource constraints. Further, lack of testing of asymptomatic patients at non-COVID hospitals may have resulted in nosocomial transmission from asymptomatic carriers. Though routine infection prevention and control activities were performed at non-COVID hospitals, we did not assess adherence to infection prevention and control guidelines in the two different categories of hospitals. Our results are also supported by evidence from studies conducted in different hospital settings, the findings of which reiterate the importance of fundamental principles of prevention (eg, proper masking, hand hygiene, and distancing) and are of particular importance in resource-limited settings.^17,26,27 The only published study quantifying seroprevalence among HCWs in India was performed in a single hospital setting with separate COVID and non-COVID units. The authors of that study reported a higher seroprevalence among HCWs in the COVID unit. However, this difference seems to be confounded by other factors as revealed by the multivariable analysis result.²³

We found a two-fold higher seroprevalence (4.4%) in HCWs who reported close contact with a patient with COVID-19. Respiratory infections pose a greater health risk to HCWs in an occupational setting. Substantial evidence has emerged demonstrating that the respiratory system is the dominant route of SARS-CoV-2 transmission, with proximity and ventilation as key predictive factors.²⁸ Globally, among thousands of HCWs infected with SARS-CoV-2, one of the leading risk factors identified was close contact with a patient with COVID-19; other identified risk factors were lack of PPE, poor infection prevention and control practices, work overload, and a preexisting health condition.²⁹

The seroprevalence estimate among participants who reported an ILI in the 4 weeks preceding the interview was only 12.2%, suggesting an alternative etiology of these symptoms. Among those who reported a previously positive RT-PCR for SARS-CoV-2, only 27.6% showed the presence of SARS-CoV-2–specific IgG antibodies. The inability to mount an antibody-mediated immune response or early conversion to seronegative status during the convalescence phase has been suggested as an explanation for such discordant findings.³⁰ On the contrary, seropositivity among participants who reported having a negative RT-PCR test was 1.9%. There are few plausible explanations for such observations. First, several studies have reported false-negative result rates from RT-PCR testing ranging from 2% to 29%.^31-33 Second, the sensitivity of the SARS-CoV-2 assay is influenced by the timing of the test after the onset of symptoms or RT-PCR positivity. The sensitivity of the assay we used varies from 53.1% at day 7 to 100% at day 17 postinfection.³⁴ Variable viral load and differences in duration of viral shedding are other possible reasons for false-negative RT-PCR results.^35,36

In our study, seroconversion among asymptomatic HCWs who were RT-PCR-positive was 20.8%. Among HCWs who reported an ILI and were RT-PCR-positive, seropositivity was 60%. In one study, 40% of asymptomatic and 13% of symptomatic patients who tested positive for COVID-19 became seronegative after initial seropositivity—that is, 8 weeks after hospital discharge.³⁷

Serological testing offers insight into both the exposure history and residual COVID-19 susceptibility of HCWs. However, current immunological knowledge does not allow us to conclude that seropositivity conveys high-level immunity against reinfection. As the epidemic evolves, HCWs will continue to be exposed to COVID-19 in the community and the workplace. Serial cross-sectional serosurveys can help monitor the progression of the pandemic within the healthcare setting and guide hospital authorities in resource allocation.

Strengths and Limitations

We used the Abbott Architect SARS-CoV-2 IgG assay, which has exhibited a high level of consistency and performance characteristics when tested in different patient populations. The participation rate was acceptable compared to similar studies, and we included all the major hospitals in the District Srinagar. The findings from our study can therefore be considered representative of the HCWs in the district.

The study results should be interpreted in the context of the following limitations. First, information on risk factors for seropositivity were based on participant report. Also, we did not collect information on the timing of symptoms or the date on which a participant became RT-PCR-positive. Second, information regarding place of exposure (ie, community or hospital setting) was not recorded, limiting conclusions regarding the effect of workplace exposures. Third, given the voluntary nature of participation in the study, there is a possibility of selection bias that may have limited the generalizability of our findings. For example, some HCWs with a recent exposure to COVID-19 or those who were symptomatic at the time of the study might not have participated based on the absence of an individual benefit from IgG testing in the early phase of infection. Conversely, some HCWs who had symptoms in the distant past might have been more likely to have participated in the study. However, we believe that selection bias does not vitiate the validity of the associations based on the plausible assumption that infection risk should be similar between respondents and nonrespondents due to comparable work environments. Finally, with a cross-sectional study design, we cannot ascertain the reconversion from an initial positive-IgG to negative-IgG status, which warrants a cohort study.

We conclude that the seroprevalence of SARS-CoV-2 infection was low among HCWs of District Srinagar at the time of the study. Healthcare workers in a dedicated COVID hospital or HCWs who had ever worked in such a facility had lower seroprevalence, suggesting both adherence to and effectiveness of standard protective measures during contact with patients who had COVID-19. Nonetheless, the careful in-hospital behavior of the HCWs at the COVID hospitals may have had a spillover effect on their out-of-hospital behaviors, which lead to community-acquired infection. On the contrary, lack of testing of asymptomatic patients at non-COVID hospitals may have resulted in nosocomial transmission from asymptomatic carriers. We believe that our findings highlight the value of implementing infection prevention and control measures in the hospital setting. Moreover, training and retraining of sanitation and other housekeeping staff on standard hygienic practices and appropriate use of the protective gear may further help reduce their rates of exposure.

The authors thank Principal and Dean of the Government Medical College, Srinagar, Professor Samia Rashid, and District Commissioner, Srinagar, Shahid Iqbal Chowdhary for their support. We also acknowledge the support rendered by the Directorate of Health Services, Kashmir; Chief Medical Officer Srinagar; Block Medical Officers; and Zonal Medical Officers of District Srinagar, Kashmir, and extend our appreciation to the medical interns for their efforts in data collection, and to laboratory in-charge Gulzar Ahmad Wani, PhD scholar, Biochemistry, and his staff, who were involved in this study. Finally, we thank the study participants for their understanding of the importance of this study and for their time and participation.

Data shall be made available on request through the corresponding author.

India is emerging as one of the world’s largest hotspots for SARS-CoV-2 infection (COVID-19)—second only to the United States—with more than 13,000,000 documented infections since the first case was recorded on January 30, 2020.^1,2 Kashmir, a northern territory of India, reported its first case of COVID-19 on March 18, 2020, from the central District Srinagar; this region has accounted for more cases of COVID-19 than any other district throughout the pandemic.³ The large majority of healthcare in District Srinagar is provided by three tertiary care institutions, one district hospital, two subdistrict hospitals, and 70 primary healthcare centers. Potential occupational exposures place healthcare workers (HCWs) at higher risk of acquiring SARS-CoV-2 infection, which in turn may serve as an important source of infection for their families and other community members.^4-6 Given the high frequency and geographic variability of asymptomatic infection, growing evidence suggests this hidden reservoir is a source of infection for the general population.^7,8

Many countries have started testing for antibodies against SARS-CoV-2, both at the population level and in specific groups, such as HCWs. Seroepidemiological studies are crucial to understanding the dynamics of SARS-CoV-2 infection. Many seroepidemiological studies have been conducted among community populations, but there are insufficient data on HCWs. The World Health Organization also encouraged its member states to conduct seroepidemiological studies to attain a better understanding of COVID-19 infection prevalence and distribution.^9-11 Therefore, to quantify the prevalence of SARS-CoV-2 infection among HCWs, we conducted a seroepidemiological study by testing for SARS-CoV-2–specific immunoglobulin (IgG) to gain insight into the extent of infection among specific subgroups of HCWs and to identify risk-factor profiles associated with seropositivity.

Study Design and Settings

We conducted this seroepidemiological study to ascertain the presence of IgG antibodies against SARS-CoV-2 among HCWs in the District Srinagar of Kashmir, India. The 2-week period of data collection began on June 15, 2020. As part of healthcare system pandemic preparedness efforts, India’s Ministry of Health provided specific guidelines for health facilities to manage COVID-19. Hospitals were categorized as dedicated COVID and non-COVID hospitals. Dedicated COVID hospitals provided comprehensive care exclusively to patients with COVID-19 and were equipped with fully functional intensive care units, ventilators, and beds with reliable access to oxygen support.¹² In addition, infection prevention and control strategies to limit the transmission of SARS-CoV-2 infection were implemented according to guidelines specified by India’s National Center for Disease Control.¹³ To strengthen service provision, HCWs from other hospitals, including resident physicians, were relocated to these dedicated COVID hospitals. The additional staff were selected by administrative leadership, without input from HCWs.

Study Population and Data Collection

We approached administrative heads of the hospitals in District Srinagar for permission to conduct our study and to invite their HCWs to participate in the study. As Figure 1 shows, we were denied permission by the administrative heads of two tertiary care hospitals. Finally, with a point person serving as a study liaison at each institution, HCWs from three dedicated COVID and seven non-COVID tertiary care hospitals, two subdistrict hospitals, and six primary healthcare centers across the District Srinagar were invited to participate. The sample primary healthcare centers were each selected randomly, after stratification, from six major regions of the district. All frontline HCWs, including physicians, administrative and laboratory personnel, technicians, field workers involved in surveillance activity, and other supporting staff were eligible for the study.

We collected information on an interview form using Epicollect5, a free data-gathering tool widely used in health research.¹⁴ Physicians specifically trained in the use of Epicollect5 conducted the face-to-face interview on a prespecified day and recorded the collected information through mobile phones. This information included the participants’ role in providing care to patients with COVID-19 and risk factors for SARS-CoV-2 infection (eg, history of travel since January 1, 2020, symptoms of an influenza-like illness [ILI] in the 4 weeks prior to the interview, close contact with a COVID-19 case). We defined close contact as an unmasked exposure within 6 feet of an infected individual for at least 15 minutes, irrespective of location (ie, community or the hospital).

Following the interview, trained phlebotomists collected 3 to 5 mL of venous blood under aseptic conditions. We strictly adhered to standard operating procedures during collection, transportation, and testing of blood samples. Following collection, the blood samples remained undisturbed for at least 30 minutes before centrifugation, which was performed at the collection site (or at the central laboratory for sites lacking the capability). The samples were then transported for further processing and testing through a cold chain supply line, using vaccine carriers with conditioned icepacks. All testing procedures were conducted with strict adherence to the manufacturers’ guidelines.

Laboratory Procedure

In accordance with the manufacturer’s recommendations, we used a chemiluminescent microparticle immunoassay to detect SARS-CoV-2–specific IgG antibodies in serum samples. The assay is an automated two-step immunoassay for the qualitative detection of IgG antibodies against the nucleocapsid of SARS-CoV-2 in human serum and plasma. The sensitivity and specificity of this test are 100% and 99%, respectively. The test result was considered positive for SARS-CoV-2 IgG if the index value was ≥1.4, consistent with guidance provided by the manufacturer.¹⁵

The IgG values were also entered into Epicollect5. Two trained medical interns independently entered the laboratory results in two separate forms. A third medical intern reviewed these forms for discrepancies, in response to which they referenced the source data for adjudication. The information gathered during the interview and the laboratory results were linked with the help of a unique identification number, which was generated at the time of the interview.

Statistical Analysis

We estimated the proportion (and logit-transformed 95% CI) of HCWs with a positive SARS-CoV-2–specific IgG antibody level, the primary outcome of interest. We compared seroprevalence rates by gender, age group, specific occupational group, and type of health facility (dedicated COVID hospital vs non-COVID hospital). Seroprevalence was also estimated separately for HCWs who reported symptoms in the past 4 weeks, had a history of exposure to a known case of COVID-19, or had undergone testing by reverse transcriptase-polymerase chain reaction (RT-PCR). In the case of zero seroprevalences, Jeffreys 95% CIs were reported. We used a chi-square test to report two-sided P values for comparison of seroprevalence between groups. When the expected frequency was <5 in more than 20% of the cells, the exact test was used instead of the chi-square test. We additionally performed multivariable logistic regression analysis to evaluate the independent association between place of work (primary independent variable) and seropositivity (dependent variable). We adjusted for the following observable covariates by including them as categorical variables: age, gender, occupational group, and history of close contact with a patient who was COVID-positive. We performed data analysis using Stata, version 15.1 (StataCorp LP). The Institutional Ethics Committee of Government Medical College, Srinagar, approved the study (Reference No. 1003/ETH/GMC dated 13-05-2020). We obtained written, informed consent from all participants.

Of the 7,346 HCWs we were granted permission to approach, 2,915 (39.7%) agreed to participate in the study. The participation rate was 49% at the dedicated COVID hospitals (57% physicians and 47% nonphysicians) and 39% at the non-COVID hospitals (46% physicians and 36% nonphysicians). We analyzed information gathered from 2,905 HCWs (Epicollect5 interview forms were missing for nine participants, and the laboratory report was missing for one participant).

The mean age of the participants was 38.6 years, and 35.8% of participants identified as female (Table 1). One third (33.7%) of the participants were physicians, nearly half of whom were residents. In our sample, the overall seroprevalence of SARS-CoV-2–specific antibodies was 2.5% (95% CI, 2.0%-3.1%).

Of the 2,905 participating HCWs, 123 (4.2%) reported an ILI (ie, fever and cough) in the 4 weeks preceding the interview, and 339 (11.7%) reported close contact with a person with COVID-19 (Table 2). A total of 760 (26.2%) HCWs had undergone RT-PCR testing, 29 (3.8%) of whom had a positive result. Stratifying by workplace, history of nasopharyngeal RT-PCR positivity was reported by 4 of 77 (5.1%) participants from dedicated COVID hospitals compared to (3.7%) participants from the non-COVID hospital (P = .528).

As Table 2 also demonstrates, we found a significantly higher seropositivity rate among HCWs who had a history of ILI (P < .001), a history of positive RT-PCR (P < .001), history of ever being put under quarantine (P = .009), and a self-reported history of close contact with a person with COVID-19 (P = .014). Healthcare workers who had ever worked at a dedicated COVID hospital had a significantly lower seroprevalence of infection (P = .004).

Among HCWs who reported no ILI symptoms in the 4 weeks prior to the interview but who had positive RT-PCR test, 20.8% were seropositive. Of HCWs who reported both ILI and a positive RT-PCR test result, 60.0% were seropositive. Compared to employment at a non-COVID hospital, HCWs working in dedicated COVID hospitals had a reduced multivariate-adjusted risk of seropositivity (odds ratio, 0.21; 95% CI, 0.06-0.66).

We aimed to estimate the seroprevalence of SARS-CoV-2 infection in HCWs in different hospital settings in the District Srinagar of Kashmir, India. In general, seroprevalence was low (2.5%), with little difference across gender or occupational group.

Seroprevalence studies of HCWs across divergent workplace environments have revealed estimates ranging from 1% to 10.2%.^16-19 Generally, the seroprevalence rates among HCWs are not significantly different from those of the general population, which reflects how different the dynamics of COVID-19 are compared to other infections in healthcare settings. The low seroprevalence observed in our study coincides with the overall low infection rate in the community population. During the study period, District Srinagar reported a median of 28 new infections daily (interquartile range, 17-46), which is indicative of the early phase of the pandemic in the population at the time of the study.²⁰

Among the HCW occupational groups, ambulance drivers and housekeeping staff had the highest seroprevalence rates, followed by nurses and physicians. Possible explanations for higher seropositivity in these groups are improper use or inadequate supply of protective gear and lack of training on the use of personal protective equipment (PPE), resulting in increased exposure risk.²¹ Concordance of HCW and community infection rates in specific geographic areas suggests that community exposure may be the dominant source of healthcare exposure and infection. Additionally, careful in-hospital behavior of HCWs in dedicated COVID hospitals may have had a spillover effect on their out-of-hospital behavior, which may partially explain our finding that employment at dedicated COVID hospitals was associated with a markedly lower chance of seropositivity. A study of 6,510 HCWs in Chicago, Illinois, showed high seropositivity rates among support service workers, medical assistants, and nurses, with nurses identified as having a markedly higher adjusted odds of seropositivity relative to administrators. The authors of the study concluded that exposure in the community setting plays a crucial role in transmission among HCWs.²² Similarly, higher seroprevalence among housekeeping, nonadministrative staff, and other support service staff has been reported elsewhere.²³ Certain underlying factors related to socioeconomic status and lifestyle may also contribute to higher seroprevalence in some occupational groups.²⁴ Nonadherence to masking, social distancing, and proper hand hygiene outside the hospital setting could result in community-acquired infection.

Interestingly, participants who were working in a dedicated COVID hospital or who had ever worked at one had a seroprevalence of 0.6%, much lower than the 2.8% observed among other participants. This difference remained statistically significant after controlling for age, sex, place of work, and occupational group. As these facilities were dedicated to the management and care of patients with COVID-19, the hospital staff strictly adhered to safety precautions, with particular vigilance during patient contact. These hospitals also strictly adhered to infection prevention and control practices based on the latest guidelines released by India’s Ministry of Health and Family Welfare.¹³

A commitment was made to provide adequate PPE to the dedicated COVID hospitals and staff, commensurate with expected infected patient volumes and associated exposure risks. Healthcare workers were specifically trained on proper donning and doffing of PPE, self-health monitoring, and protocols for reporting symptoms and PPE breaches during patient encounters. Healthcare workers were regularly tested for COVID-19 using nasopharyngeal RT-PCR. Of critical importance, these hospitals implemented a buddy system wherein a team of two or more staff members was responsible for ensuring each other’s safety, proper PPE use, conformance to other protective measures, and reporting breaches of PPE compliance.²⁵ Universal masking was mandatory for all hospital staff and patients at the COVID-focused facilities, with the additional use of N-95 masks, gloves, and face shields during times of patient contact. Administrative measures, including visitor restrictions and environmental sanitation, were rigorously enforced. Also, being a potentially high-risk area for transmission of infection, these facilities implemented staff-rationing to reduce the duration of exposure to the healthcare staff. Third, the HCWs of COVID-dedicated hospitals were provided with separate living accommodations during the period in which they were employed at a dedicated COVID hospital.

In contrast, in non-COVID hospitals, with the exception of HCWs, patients and the hospital visitors were not subject to a masking policy. Moreover, an adequate and timely supply of PPE was not prioritized at the non-COVID facilities due to resource constraints. Further, lack of testing of asymptomatic patients at non-COVID hospitals may have resulted in nosocomial transmission from asymptomatic carriers. Though routine infection prevention and control activities were performed at non-COVID hospitals, we did not assess adherence to infection prevention and control guidelines in the two different categories of hospitals. Our results are also supported by evidence from studies conducted in different hospital settings, the findings of which reiterate the importance of fundamental principles of prevention (eg, proper masking, hand hygiene, and distancing) and are of particular importance in resource-limited settings.^17,26,27 The only published study quantifying seroprevalence among HCWs in India was performed in a single hospital setting with separate COVID and non-COVID units. The authors of that study reported a higher seroprevalence among HCWs in the COVID unit. However, this difference seems to be confounded by other factors as revealed by the multivariable analysis result.²³

We found a two-fold higher seroprevalence (4.4%) in HCWs who reported close contact with a patient with COVID-19. Respiratory infections pose a greater health risk to HCWs in an occupational setting. Substantial evidence has emerged demonstrating that the respiratory system is the dominant route of SARS-CoV-2 transmission, with proximity and ventilation as key predictive factors.²⁸ Globally, among thousands of HCWs infected with SARS-CoV-2, one of the leading risk factors identified was close contact with a patient with COVID-19; other identified risk factors were lack of PPE, poor infection prevention and control practices, work overload, and a preexisting health condition.²⁹

The seroprevalence estimate among participants who reported an ILI in the 4 weeks preceding the interview was only 12.2%, suggesting an alternative etiology of these symptoms. Among those who reported a previously positive RT-PCR for SARS-CoV-2, only 27.6% showed the presence of SARS-CoV-2–specific IgG antibodies. The inability to mount an antibody-mediated immune response or early conversion to seronegative status during the convalescence phase has been suggested as an explanation for such discordant findings.³⁰ On the contrary, seropositivity among participants who reported having a negative RT-PCR test was 1.9%. There are few plausible explanations for such observations. First, several studies have reported false-negative result rates from RT-PCR testing ranging from 2% to 29%.^31-33 Second, the sensitivity of the SARS-CoV-2 assay is influenced by the timing of the test after the onset of symptoms or RT-PCR positivity. The sensitivity of the assay we used varies from 53.1% at day 7 to 100% at day 17 postinfection.³⁴ Variable viral load and differences in duration of viral shedding are other possible reasons for false-negative RT-PCR results.^35,36

In our study, seroconversion among asymptomatic HCWs who were RT-PCR-positive was 20.8%. Among HCWs who reported an ILI and were RT-PCR-positive, seropositivity was 60%. In one study, 40% of asymptomatic and 13% of symptomatic patients who tested positive for COVID-19 became seronegative after initial seropositivity—that is, 8 weeks after hospital discharge.³⁷

Serological testing offers insight into both the exposure history and residual COVID-19 susceptibility of HCWs. However, current immunological knowledge does not allow us to conclude that seropositivity conveys high-level immunity against reinfection. As the epidemic evolves, HCWs will continue to be exposed to COVID-19 in the community and the workplace. Serial cross-sectional serosurveys can help monitor the progression of the pandemic within the healthcare setting and guide hospital authorities in resource allocation.

Strengths and Limitations

We used the Abbott Architect SARS-CoV-2 IgG assay, which has exhibited a high level of consistency and performance characteristics when tested in different patient populations. The participation rate was acceptable compared to similar studies, and we included all the major hospitals in the District Srinagar. The findings from our study can therefore be considered representative of the HCWs in the district.

The study results should be interpreted in the context of the following limitations. First, information on risk factors for seropositivity were based on participant report. Also, we did not collect information on the timing of symptoms or the date on which a participant became RT-PCR-positive. Second, information regarding place of exposure (ie, community or hospital setting) was not recorded, limiting conclusions regarding the effect of workplace exposures. Third, given the voluntary nature of participation in the study, there is a possibility of selection bias that may have limited the generalizability of our findings. For example, some HCWs with a recent exposure to COVID-19 or those who were symptomatic at the time of the study might not have participated based on the absence of an individual benefit from IgG testing in the early phase of infection. Conversely, some HCWs who had symptoms in the distant past might have been more likely to have participated in the study. However, we believe that selection bias does not vitiate the validity of the associations based on the plausible assumption that infection risk should be similar between respondents and nonrespondents due to comparable work environments. Finally, with a cross-sectional study design, we cannot ascertain the reconversion from an initial positive-IgG to negative-IgG status, which warrants a cohort study.

We conclude that the seroprevalence of SARS-CoV-2 infection was low among HCWs of District Srinagar at the time of the study. Healthcare workers in a dedicated COVID hospital or HCWs who had ever worked in such a facility had lower seroprevalence, suggesting both adherence to and effectiveness of standard protective measures during contact with patients who had COVID-19. Nonetheless, the careful in-hospital behavior of the HCWs at the COVID hospitals may have had a spillover effect on their out-of-hospital behaviors, which lead to community-acquired infection. On the contrary, lack of testing of asymptomatic patients at non-COVID hospitals may have resulted in nosocomial transmission from asymptomatic carriers. We believe that our findings highlight the value of implementing infection prevention and control measures in the hospital setting. Moreover, training and retraining of sanitation and other housekeeping staff on standard hygienic practices and appropriate use of the protective gear may further help reduce their rates of exposure.

The authors thank Principal and Dean of the Government Medical College, Srinagar, Professor Samia Rashid, and District Commissioner, Srinagar, Shahid Iqbal Chowdhary for their support. We also acknowledge the support rendered by the Directorate of Health Services, Kashmir; Chief Medical Officer Srinagar; Block Medical Officers; and Zonal Medical Officers of District Srinagar, Kashmir, and extend our appreciation to the medical interns for their efforts in data collection, and to laboratory in-charge Gulzar Ahmad Wani, PhD scholar, Biochemistry, and his staff, who were involved in this study. Finally, we thank the study participants for their understanding of the importance of this study and for their time and participation.

Data shall be made available on request through the corresponding author.

India is emerging as one of the world’s largest hotspots for SARS-CoV-2 infection (COVID-19)—second only to the United States—with more than 13,000,000 documented infections since the first case was recorded on January 30, 2020.^1,2 Kashmir, a northern territory of India, reported its first case of COVID-19 on March 18, 2020, from the central District Srinagar; this region has accounted for more cases of COVID-19 than any other district throughout the pandemic.³ The large majority of healthcare in District Srinagar is provided by three tertiary care institutions, one district hospital, two subdistrict hospitals, and 70 primary healthcare centers. Potential occupational exposures place healthcare workers (HCWs) at higher risk of acquiring SARS-CoV-2 infection, which in turn may serve as an important source of infection for their families and other community members.^4-6 Given the high frequency and geographic variability of asymptomatic infection, growing evidence suggests this hidden reservoir is a source of infection for the general population.^7,8

Many countries have started testing for antibodies against SARS-CoV-2, both at the population level and in specific groups, such as HCWs. Seroepidemiological studies are crucial to understanding the dynamics of SARS-CoV-2 infection. Many seroepidemiological studies have been conducted among community populations, but there are insufficient data on HCWs. The World Health Organization also encouraged its member states to conduct seroepidemiological studies to attain a better understanding of COVID-19 infection prevalence and distribution.^9-11 Therefore, to quantify the prevalence of SARS-CoV-2 infection among HCWs, we conducted a seroepidemiological study by testing for SARS-CoV-2–specific immunoglobulin (IgG) to gain insight into the extent of infection among specific subgroups of HCWs and to identify risk-factor profiles associated with seropositivity.

Study Design and Settings

We conducted this seroepidemiological study to ascertain the presence of IgG antibodies against SARS-CoV-2 among HCWs in the District Srinagar of Kashmir, India. The 2-week period of data collection began on June 15, 2020. As part of healthcare system pandemic preparedness efforts, India’s Ministry of Health provided specific guidelines for health facilities to manage COVID-19. Hospitals were categorized as dedicated COVID and non-COVID hospitals. Dedicated COVID hospitals provided comprehensive care exclusively to patients with COVID-19 and were equipped with fully functional intensive care units, ventilators, and beds with reliable access to oxygen support.¹² In addition, infection prevention and control strategies to limit the transmission of SARS-CoV-2 infection were implemented according to guidelines specified by India’s National Center for Disease Control.¹³ To strengthen service provision, HCWs from other hospitals, including resident physicians, were relocated to these dedicated COVID hospitals. The additional staff were selected by administrative leadership, without input from HCWs.

Study Population and Data Collection

We approached administrative heads of the hospitals in District Srinagar for permission to conduct our study and to invite their HCWs to participate in the study. As Figure 1 shows, we were denied permission by the administrative heads of two tertiary care hospitals. Finally, with a point person serving as a study liaison at each institution, HCWs from three dedicated COVID and seven non-COVID tertiary care hospitals, two subdistrict hospitals, and six primary healthcare centers across the District Srinagar were invited to participate. The sample primary healthcare centers were each selected randomly, after stratification, from six major regions of the district. All frontline HCWs, including physicians, administrative and laboratory personnel, technicians, field workers involved in surveillance activity, and other supporting staff were eligible for the study.

We collected information on an interview form using Epicollect5, a free data-gathering tool widely used in health research.¹⁴ Physicians specifically trained in the use of Epicollect5 conducted the face-to-face interview on a prespecified day and recorded the collected information through mobile phones. This information included the participants’ role in providing care to patients with COVID-19 and risk factors for SARS-CoV-2 infection (eg, history of travel since January 1, 2020, symptoms of an influenza-like illness [ILI] in the 4 weeks prior to the interview, close contact with a COVID-19 case). We defined close contact as an unmasked exposure within 6 feet of an infected individual for at least 15 minutes, irrespective of location (ie, community or the hospital).

Following the interview, trained phlebotomists collected 3 to 5 mL of venous blood under aseptic conditions. We strictly adhered to standard operating procedures during collection, transportation, and testing of blood samples. Following collection, the blood samples remained undisturbed for at least 30 minutes before centrifugation, which was performed at the collection site (or at the central laboratory for sites lacking the capability). The samples were then transported for further processing and testing through a cold chain supply line, using vaccine carriers with conditioned icepacks. All testing procedures were conducted with strict adherence to the manufacturers’ guidelines.

Laboratory Procedure

In accordance with the manufacturer’s recommendations, we used a chemiluminescent microparticle immunoassay to detect SARS-CoV-2–specific IgG antibodies in serum samples. The assay is an automated two-step immunoassay for the qualitative detection of IgG antibodies against the nucleocapsid of SARS-CoV-2 in human serum and plasma. The sensitivity and specificity of this test are 100% and 99%, respectively. The test result was considered positive for SARS-CoV-2 IgG if the index value was ≥1.4, consistent with guidance provided by the manufacturer.¹⁵

The IgG values were also entered into Epicollect5. Two trained medical interns independently entered the laboratory results in two separate forms. A third medical intern reviewed these forms for discrepancies, in response to which they referenced the source data for adjudication. The information gathered during the interview and the laboratory results were linked with the help of a unique identification number, which was generated at the time of the interview.

Statistical Analysis

We estimated the proportion (and logit-transformed 95% CI) of HCWs with a positive SARS-CoV-2–specific IgG antibody level, the primary outcome of interest. We compared seroprevalence rates by gender, age group, specific occupational group, and type of health facility (dedicated COVID hospital vs non-COVID hospital). Seroprevalence was also estimated separately for HCWs who reported symptoms in the past 4 weeks, had a history of exposure to a known case of COVID-19, or had undergone testing by reverse transcriptase-polymerase chain reaction (RT-PCR). In the case of zero seroprevalences, Jeffreys 95% CIs were reported. We used a chi-square test to report two-sided P values for comparison of seroprevalence between groups. When the expected frequency was <5 in more than 20% of the cells, the exact test was used instead of the chi-square test. We additionally performed multivariable logistic regression analysis to evaluate the independent association between place of work (primary independent variable) and seropositivity (dependent variable). We adjusted for the following observable covariates by including them as categorical variables: age, gender, occupational group, and history of close contact with a patient who was COVID-positive. We performed data analysis using Stata, version 15.1 (StataCorp LP). The Institutional Ethics Committee of Government Medical College, Srinagar, approved the study (Reference No. 1003/ETH/GMC dated 13-05-2020). We obtained written, informed consent from all participants.

Of the 7,346 HCWs we were granted permission to approach, 2,915 (39.7%) agreed to participate in the study. The participation rate was 49% at the dedicated COVID hospitals (57% physicians and 47% nonphysicians) and 39% at the non-COVID hospitals (46% physicians and 36% nonphysicians). We analyzed information gathered from 2,905 HCWs (Epicollect5 interview forms were missing for nine participants, and the laboratory report was missing for one participant).

The mean age of the participants was 38.6 years, and 35.8% of participants identified as female (Table 1). One third (33.7%) of the participants were physicians, nearly half of whom were residents. In our sample, the overall seroprevalence of SARS-CoV-2–specific antibodies was 2.5% (95% CI, 2.0%-3.1%).

Of the 2,905 participating HCWs, 123 (4.2%) reported an ILI (ie, fever and cough) in the 4 weeks preceding the interview, and 339 (11.7%) reported close contact with a person with COVID-19 (Table 2). A total of 760 (26.2%) HCWs had undergone RT-PCR testing, 29 (3.8%) of whom had a positive result. Stratifying by workplace, history of nasopharyngeal RT-PCR positivity was reported by 4 of 77 (5.1%) participants from dedicated COVID hospitals compared to (3.7%) participants from the non-COVID hospital (P = .528).

As Table 2 also demonstrates, we found a significantly higher seropositivity rate among HCWs who had a history of ILI (P < .001), a history of positive RT-PCR (P < .001), history of ever being put under quarantine (P = .009), and a self-reported history of close contact with a person with COVID-19 (P = .014). Healthcare workers who had ever worked at a dedicated COVID hospital had a significantly lower seroprevalence of infection (P = .004).

Among HCWs who reported no ILI symptoms in the 4 weeks prior to the interview but who had positive RT-PCR test, 20.8% were seropositive. Of HCWs who reported both ILI and a positive RT-PCR test result, 60.0% were seropositive. Compared to employment at a non-COVID hospital, HCWs working in dedicated COVID hospitals had a reduced multivariate-adjusted risk of seropositivity (odds ratio, 0.21; 95% CI, 0.06-0.66).

We aimed to estimate the seroprevalence of SARS-CoV-2 infection in HCWs in different hospital settings in the District Srinagar of Kashmir, India. In general, seroprevalence was low (2.5%), with little difference across gender or occupational group.

Seroprevalence studies of HCWs across divergent workplace environments have revealed estimates ranging from 1% to 10.2%.^16-19 Generally, the seroprevalence rates among HCWs are not significantly different from those of the general population, which reflects how different the dynamics of COVID-19 are compared to other infections in healthcare settings. The low seroprevalence observed in our study coincides with the overall low infection rate in the community population. During the study period, District Srinagar reported a median of 28 new infections daily (interquartile range, 17-46), which is indicative of the early phase of the pandemic in the population at the time of the study.²⁰

Among the HCW occupational groups, ambulance drivers and housekeeping staff had the highest seroprevalence rates, followed by nurses and physicians. Possible explanations for higher seropositivity in these groups are improper use or inadequate supply of protective gear and lack of training on the use of personal protective equipment (PPE), resulting in increased exposure risk.²¹ Concordance of HCW and community infection rates in specific geographic areas suggests that community exposure may be the dominant source of healthcare exposure and infection. Additionally, careful in-hospital behavior of HCWs in dedicated COVID hospitals may have had a spillover effect on their out-of-hospital behavior, which may partially explain our finding that employment at dedicated COVID hospitals was associated with a markedly lower chance of seropositivity. A study of 6,510 HCWs in Chicago, Illinois, showed high seropositivity rates among support service workers, medical assistants, and nurses, with nurses identified as having a markedly higher adjusted odds of seropositivity relative to administrators. The authors of the study concluded that exposure in the community setting plays a crucial role in transmission among HCWs.²² Similarly, higher seroprevalence among housekeeping, nonadministrative staff, and other support service staff has been reported elsewhere.²³ Certain underlying factors related to socioeconomic status and lifestyle may also contribute to higher seroprevalence in some occupational groups.²⁴ Nonadherence to masking, social distancing, and proper hand hygiene outside the hospital setting could result in community-acquired infection.

Interestingly, participants who were working in a dedicated COVID hospital or who had ever worked at one had a seroprevalence of 0.6%, much lower than the 2.8% observed among other participants. This difference remained statistically significant after controlling for age, sex, place of work, and occupational group. As these facilities were dedicated to the management and care of patients with COVID-19, the hospital staff strictly adhered to safety precautions, with particular vigilance during patient contact. These hospitals also strictly adhered to infection prevention and control practices based on the latest guidelines released by India’s Ministry of Health and Family Welfare.¹³

A commitment was made to provide adequate PPE to the dedicated COVID hospitals and staff, commensurate with expected infected patient volumes and associated exposure risks. Healthcare workers were specifically trained on proper donning and doffing of PPE, self-health monitoring, and protocols for reporting symptoms and PPE breaches during patient encounters. Healthcare workers were regularly tested for COVID-19 using nasopharyngeal RT-PCR. Of critical importance, these hospitals implemented a buddy system wherein a team of two or more staff members was responsible for ensuring each other’s safety, proper PPE use, conformance to other protective measures, and reporting breaches of PPE compliance.²⁵ Universal masking was mandatory for all hospital staff and patients at the COVID-focused facilities, with the additional use of N-95 masks, gloves, and face shields during times of patient contact. Administrative measures, including visitor restrictions and environmental sanitation, were rigorously enforced. Also, being a potentially high-risk area for transmission of infection, these facilities implemented staff-rationing to reduce the duration of exposure to the healthcare staff. Third, the HCWs of COVID-dedicated hospitals were provided with separate living accommodations during the period in which they were employed at a dedicated COVID hospital.

In contrast, in non-COVID hospitals, with the exception of HCWs, patients and the hospital visitors were not subject to a masking policy. Moreover, an adequate and timely supply of PPE was not prioritized at the non-COVID facilities due to resource constraints. Further, lack of testing of asymptomatic patients at non-COVID hospitals may have resulted in nosocomial transmission from asymptomatic carriers. Though routine infection prevention and control activities were performed at non-COVID hospitals, we did not assess adherence to infection prevention and control guidelines in the two different categories of hospitals. Our results are also supported by evidence from studies conducted in different hospital settings, the findings of which reiterate the importance of fundamental principles of prevention (eg, proper masking, hand hygiene, and distancing) and are of particular importance in resource-limited settings.^17,26,27 The only published study quantifying seroprevalence among HCWs in India was performed in a single hospital setting with separate COVID and non-COVID units. The authors of that study reported a higher seroprevalence among HCWs in the COVID unit. However, this difference seems to be confounded by other factors as revealed by the multivariable analysis result.²³

We found a two-fold higher seroprevalence (4.4%) in HCWs who reported close contact with a patient with COVID-19. Respiratory infections pose a greater health risk to HCWs in an occupational setting. Substantial evidence has emerged demonstrating that the respiratory system is the dominant route of SARS-CoV-2 transmission, with proximity and ventilation as key predictive factors.²⁸ Globally, among thousands of HCWs infected with SARS-CoV-2, one of the leading risk factors identified was close contact with a patient with COVID-19; other identified risk factors were lack of PPE, poor infection prevention and control practices, work overload, and a preexisting health condition.²⁹

The seroprevalence estimate among participants who reported an ILI in the 4 weeks preceding the interview was only 12.2%, suggesting an alternative etiology of these symptoms. Among those who reported a previously positive RT-PCR for SARS-CoV-2, only 27.6% showed the presence of SARS-CoV-2–specific IgG antibodies. The inability to mount an antibody-mediated immune response or early conversion to seronegative status during the convalescence phase has been suggested as an explanation for such discordant findings.³⁰ On the contrary, seropositivity among participants who reported having a negative RT-PCR test was 1.9%. There are few plausible explanations for such observations. First, several studies have reported false-negative result rates from RT-PCR testing ranging from 2% to 29%.^31-33 Second, the sensitivity of the SARS-CoV-2 assay is influenced by the timing of the test after the onset of symptoms or RT-PCR positivity. The sensitivity of the assay we used varies from 53.1% at day 7 to 100% at day 17 postinfection.³⁴ Variable viral load and differences in duration of viral shedding are other possible reasons for false-negative RT-PCR results.^35,36

In our study, seroconversion among asymptomatic HCWs who were RT-PCR-positive was 20.8%. Among HCWs who reported an ILI and were RT-PCR-positive, seropositivity was 60%. In one study, 40% of asymptomatic and 13% of symptomatic patients who tested positive for COVID-19 became seronegative after initial seropositivity—that is, 8 weeks after hospital discharge.³⁷

Serological testing offers insight into both the exposure history and residual COVID-19 susceptibility of HCWs. However, current immunological knowledge does not allow us to conclude that seropositivity conveys high-level immunity against reinfection. As the epidemic evolves, HCWs will continue to be exposed to COVID-19 in the community and the workplace. Serial cross-sectional serosurveys can help monitor the progression of the pandemic within the healthcare setting and guide hospital authorities in resource allocation.

Strengths and Limitations

We used the Abbott Architect SARS-CoV-2 IgG assay, which has exhibited a high level of consistency and performance characteristics when tested in different patient populations. The participation rate was acceptable compared to similar studies, and we included all the major hospitals in the District Srinagar. The findings from our study can therefore be considered representative of the HCWs in the district.

The study results should be interpreted in the context of the following limitations. First, information on risk factors for seropositivity were based on participant report. Also, we did not collect information on the timing of symptoms or the date on which a participant became RT-PCR-positive. Second, information regarding place of exposure (ie, community or hospital setting) was not recorded, limiting conclusions regarding the effect of workplace exposures. Third, given the voluntary nature of participation in the study, there is a possibility of selection bias that may have limited the generalizability of our findings. For example, some HCWs with a recent exposure to COVID-19 or those who were symptomatic at the time of the study might not have participated based on the absence of an individual benefit from IgG testing in the early phase of infection. Conversely, some HCWs who had symptoms in the distant past might have been more likely to have participated in the study. However, we believe that selection bias does not vitiate the validity of the associations based on the plausible assumption that infection risk should be similar between respondents and nonrespondents due to comparable work environments. Finally, with a cross-sectional study design, we cannot ascertain the reconversion from an initial positive-IgG to negative-IgG status, which warrants a cohort study.

We conclude that the seroprevalence of SARS-CoV-2 infection was low among HCWs of District Srinagar at the time of the study. Healthcare workers in a dedicated COVID hospital or HCWs who had ever worked in such a facility had lower seroprevalence, suggesting both adherence to and effectiveness of standard protective measures during contact with patients who had COVID-19. Nonetheless, the careful in-hospital behavior of the HCWs at the COVID hospitals may have had a spillover effect on their out-of-hospital behaviors, which lead to community-acquired infection. On the contrary, lack of testing of asymptomatic patients at non-COVID hospitals may have resulted in nosocomial transmission from asymptomatic carriers. We believe that our findings highlight the value of implementing infection prevention and control measures in the hospital setting. Moreover, training and retraining of sanitation and other housekeeping staff on standard hygienic practices and appropriate use of the protective gear may further help reduce their rates of exposure.

The authors thank Principal and Dean of the Government Medical College, Srinagar, Professor Samia Rashid, and District Commissioner, Srinagar, Shahid Iqbal Chowdhary for their support. We also acknowledge the support rendered by the Directorate of Health Services, Kashmir; Chief Medical Officer Srinagar; Block Medical Officers; and Zonal Medical Officers of District Srinagar, Kashmir, and extend our appreciation to the medical interns for their efforts in data collection, and to laboratory in-charge Gulzar Ahmad Wani, PhD scholar, Biochemistry, and his staff, who were involved in this study. Finally, we thank the study participants for their understanding of the importance of this study and for their time and participation.

Data shall be made available on request through the corresponding author.

Febrile infants aged 0 to 60 days often undergo diagnostic testing to evaluate for invasive bacterial infections (IBI; ie, bacteremia and meningitis) and are subsequently hospitalized pending culture results. Only 1% to 2% of infants 0 to 60 days old have an IBI,^1-3 and most hospitalized infants are discharged once physicians feel confident that pathogens are unlikely to be isolated from blood and cerebrospinal fluid (CSF) cultures. Practice regarding duration of hospitalization while awaiting blood and CSF culture results is not standardized in this population. Longer hospitalizations can lead to increased costs and familial stress, including difficulty with breastfeeding and anxiety in newly postpartum mothers.^4,5

In 2010, an institutional evidence-based guideline for the management of febrile infants aged 0 to 60 days recommended discharge after 36 hours of observation if all cultures were negative.⁶ However, recent studies demonstrate that 85% to 93% of pathogens in blood and CSF cultures grow within 24 hours of incubation.^7-9 Assuming a 2% prevalence of IBI, if 15% of pathogens were identified after 24 hours of incubation, only one out of 333 infants would have an IBI identified after 24 hours of hospital observation.⁷

Furthermore, a review of our institution’s electronic health records (EHR) over the past 5 years revealed that an observation period of 24 hours would have resulted in the discharge of three infants with an IBI. Two infants had bacteremia; both were discharged from the emergency department (ED) without antibiotics, returned to care after cultures were reported positive at 27 hours, and had no adverse outcomes. The third infant had meningitis, but also had an abnormal CSF Gram stain, which led to a longer hospitalization.

In 2019, our institution appraised the emerging literature and institutional data supporting the low absolute risk of missed IBI, and also leveraged local consensus among key stakeholders to update its evidence-based guideline for the evaluation and management of febrile infants aged 60 days and younger. The updated guideline recommends that clinicians consider discharging well-appearing neonates and infants if blood and CSF cultures remain negative at 24 hours.¹⁰ The objective of this study was to decrease the average hospital culture observation time (COT; culture incubation to hospital discharge) from 38 to 30 hours over a 12-month period in febrile infants aged 0 to 60 days.

Context

Improvement efforts were conducted at Cincinnati Children’s Hospital Medical Center (CCHMC), a large, urban, academic hospital that admitted more than 8,000 noncritically ill patients to the hospital medicine (HM) service from July 1, 2018, through June 30, 2019. Hospital medicine teams, located at both the main and satellite campuses, are staffed by attending physicians, fellows, residents, medical students, and nurse practitioners. The two campuses, which are about 20 miles apart, share clinician providers but have distinct nursing pools.

Microbiology services for all CCHMC patients are provided at the main campus. Blood and CSF cultures at the satellite campus are transported to the main campus for incubation and monitoring via an urgent courier service. The microbiology laboratory at CCHMC uses a continuous monitoring system for blood cultures (BACT/ALERT Virtuo, BioMérieux). The system automatically alerts laboratory technicians of positive cultures; these results are reported to clinical providers within 30 minutes of detection. Laboratory technicians manually evaluate CSF cultures once daily for 5 days.

Improvement Team

Our improvement team included three HM attending physicians; two HM fellows; a pediatric chief resident; two nurses, who represented nursing pools at the main and satellite campuses; and a clinical pharmacist, who is a co-leader of the antimicrobial stewardship program at CCHMC. Supporting members for the improvement team included the CCHMC laboratory director; the microbiology laboratory director; an infectious disease physician, who is a co-leader of the antimicrobial stewardship program; and nursing directors of the HM units at both campuses.

Evidence-Based Guideline

Our improvement initiative was based on recommendations from the updated CCHMC Evidence-Based Care Guideline for Management of Infants 0 to 60 days with Fever of Unknown Source.¹⁰ This guideline, published in May 2019, was developed by a multidisciplinary working group composed of key stakeholders from HM, community pediatrics, emergency medicine, the pediatric residency program, infectious disease, and laboratory medicine. Several improvement team members were participants on the committee that published the evidence-based guideline. The committee first performed a systematic literature review and critical appraisal of the literature. Care recommendations were formulated via a consensus process directed by best evidence, patient and family preferences, and clinical expertise; the recommendations were subsequently reviewed and approved by clinical experts who were not involved in the development process.

Based on evidence review and multistakeholder consensus, the updated guideline recommends clinicians consider discharging neonates and infants aged 60 days and younger if there is no culture growth after an observation period of 24 hours (as documented in the EHR) and patients are otherwise medically ready for discharge (ie, well appearing with adequate oral intake).^10,11 In addition, prior to discharge, there must be a documented working phone number on file for the patient’s parents/guardians, an established outpatient follow-up plan within 24 hours, and communication with the primary pediatrician who is in agreement with discharge at 24 hours.

Study Population

Infants 0 to 60 days old who had a documented or reported fever without an apparent source based on history and physical exam upon presentation to the ED, and who were subsequently admitted to the HM service at CCHMC between October 30, 2018, and July 10, 2020, were eligible for inclusion. We excluded infants who were admitted to other clinical services (eg, intensive care unit); had organisms identified on blood, urine, or CSF culture within 24 hours of incubation; had positive herpes simplex virus testing; had skin/soft tissue infections or another clearly documented source of bacterial infection; or had an alternative indication for hospitalization (eg, need for intravenous fluid or deep suctioning) after cultures had incubated for 24 hours. Infants who had a positive blood, urine, or CSF culture result after 24 hours of incubation were included in the study population. Organisms were classified as pathogen or contaminant based on treatment decisions made by the care team.

Improvement Activities

Key drivers critical to success of the improvement efforts were: (1) clearly defined standard of care for duration of observation in febrile infants 0 to 60 days old; (2) improved understanding of microbiology lab procedures; (3) effective communication of discharge criteria between providers and nurses; and (4) transparency of data with feedback (Figure 1).

Education and Structured Dissemination of Evidence-Based Guideline

The CCHMC febrile infant guideline¹⁰ was disseminated to HM physicians, residents, and nurses via the following means: (1) in-person announcements at staff meetings and educational conferences, (2) published highlights from the guideline in weekly newsletters, and (3) email announcements. Additionally, members of the study team educated HM attending physicians, nursing staff from the medical units at both campuses, and resident physicians about recent studies demonstrating safety of shorter length of stay (LOS) in febrile infants aged 0 to 60 days. The study team also provided residents, physicians, and nurses with data on the number of positive blood and CSF cultures and outcomes of patients at CCHMC within the past 5 years. In addition, team members led a journal club for residents discussing an article⁷ describing time-to-positivity of blood and CSF cultures in febrile infants. For ongoing engagement, the evidence-based guideline and a detailed explanation of microbiology procedures were published in the resident handbook, an internal resource that includes vital clinical pearls and practice guidelines across specialties. (Each resident receives an updated hard copy each year, and there is also an online link to the resource in the EHR.) Information about the guideline and COT was also included in the monthly chief resident’s orientation script, which is relayed to all residents on the first day of their HM rotation.

Clear Communication of Microbiology Procedures

Team members created a detailed process map describing the processing protocols for blood and CSF cultures collected at both CCHMC campuses. This information was shared with HM attending physicians and nurses via in-person announcements at staff meetings, flyers in team workrooms, and email communications. Residents received information on microbiology protocols via in-person announcements at educational conferences and dissemination in the weekly residency newsletter.Important information communicated included:

1. Definition of culture start time. We conveyed that there may be a delay of up to 4 hours between culture collection at the satellite campus and culture incubation at the main campus laboratory. As a result, the time of blood or CSF sample arrival to the main campus laboratory was a more accurate reflection of the culture incubation start time than the culture collection time.

2. Explanation of CSF culture processing. We discussed the process by which these cultures are plated upon arrival at the microbiology laboratory and read once per day in the morning. Therefore, a culture incubated at midnight would be evaluated once at 9 hours and not again until 33 hours.

Modification of Febrile Infant Order Set

Enhancements to the febrile infant order set improved communication and cultivated a shared mental model regarding discharge goals among all members of the care team. The EHR order set for febrile infants was updated as follows: (1) mandatory free-text fields that established the culture start time for blood and CSF cultures were added, (2) culture start time was clearly defined (ie, the time culture arrives at the main campus laboratory), and (3) a change was made in the default discharge criteria¹¹ to “culture observation for 24 hours,” with the ability to modify COT (Appendix Figure 1). We embedded hyperlinks to the guideline and microbiology process map within the updated order set, which allowed providers to easily access this information and refresh their knowledge of the recommendations (Appendix Figure 1).

Identification of Failures and Follow-up With Near-Time Feedback

All cases of febrile infants were tracked weekly. For infants hospitalized longer than 24 hours, the study team contacted the discharging clinicians to discuss reasons for prolonged hospitalization, with an emphasis on identifying system-level barriers to earlier discharge.

Study of the Interventions

The institutional microbiology database was queried weekly to identify all infants 0 to 60 days old who had a blood culture obtained and were hospitalized on the HM service. Study team members conducted targeted EHR review to determine whether patients met exclusion criteria and to identify reasons for prolonged COT. Baseline data were collected retrospectively for a 3-month period prior to initiation of improvement activities. During the study period, queries were conducted weekly and reviewed by study team members to evaluate the impact of improvement activities and to inform new interventions.

Measures

Our primary outcome measure was COT, defined as the hours between final culture incubation and hospital discharge. The operational definition for “final culture incubation” was the documented time of arrival of the last collected culture to the microbiology laboratory. Our goal COT was 30 hours to account for a subset of patients whose blood and/or CSF culture were obtained overnight (ie, after 9 pm), since subsequent discharge times would likely and practically be delayed beyond 24 hours. Our secondary outcome measure was LOS, defined as the time between ED arrival and hospital discharge. Process measures included the proportion of patients for whom the febrile infant EHR order set was used and the proportion of patients for whom medical discharge criteria (ie, blood and CSF culture observed for ”xx” hours) and culture incubation start times were entered using the order set. Balancing measures included identification of IBI after hospital discharge, 48-hour ED revisits, and 7-day hospital readmissions.

Analysis

Measures were evaluated using statistical process control charts and run charts, and Western Electric rules were employed to determine special cause variation.¹² Annotated X-bar S control charts tracked the impact of improvement activities on average COT and LOS for all infants. Given that a relatively small number of patients (ie, two to four) met inclusion criteria each week, average COT was calculated per five patients.

This study was considered exempt from review by the CCHMC Institutional Review Board.

Of the 184 infants in this study, 46 were included as part of baseline data collection, and 138 were included during the intervention period. The median age was 26.6 days (range, 3-59 days); 52% of patients were female; two-thirds were non-Hispanic White; 22% were Black, and 5% were Hispanic (Appendix Table).

Average COT decreased from 38 hours to 32 hours with improvement activities (Figure 2) and was sustained for a total of 17 months. There were small decreases in COT after initial education was provided to attendings, nurses, and residents.

After the launch of the updated order set, median usage of the EHR order set increased from 50% to 80%. Medical discharge criteria were entered for 80 (96%) of the 83 patients for whom the updated order set was applied; culture incubation start times were entered for 78 (94%) of these patients.

No infants in our cohort were found to have IBI after hospital discharge. There were no ED revisits within 48 hours of discharge, and there were no hospital readmissions within 7 days of index discharge. Furthermore, none of the patients included in the study had growth of a pathogenic organism after 24 hours.

Of the 138 infants hospitalized during the intervention period, 77 (56%) had a COT greater than 30 hours. Among these 77 patients, 49 (64%) had their final culture incubated between 9 pm and 4 am; Furthermore, 11 (14%) had missing, abnormal, pretreated, or uninterpretable CSF studies, 7 (9%) had ongoing fevers, and 4 (5%) remained hospitalized due to family preference or inability to obtain timely outpatient follow-up.

Our study aimed to decrease the average COT from 38 hours to 30 hours among hospitalized infants aged 60 days and younger over a period of 12 months. An intervention featuring implementation of an evidence-based guideline through education, laboratory procedure transparency, creation of a standardized EHR order set, and near-time feedback was associated with a shorter average COT of 32 hours, sustained over a 17-month period. No infants with bacteremia or meningitis were inappropriately discharged during this study.

Interpretation

Prior to our improvement efforts, most febrile infants at CCHMC were observed for at least 36 hours based on a prior institutional guideline,⁶ despite recent evidence suggesting that most pathogens in blood and CSF cultures grow within 24 hours of incubation.^7-9 The goal of this improvement initiative was to bridge the gap between emerging evidence and clinical practice by developing and disseminating an updated evidence-based guideline to safely decrease the hospital observation time in febrile infants aged 60 days and younger.

Similar to previous studies aimed at improving diagnosis and management among febrile infants,^13-16 generation and structured dissemination of an institutional evidence-based guideline was crucial to safely shortening COT in our population. These prior studies established a goal COT of 36 to 42 hours for hospitalized febrile infants.^13,15,16 Our study incorporated emerging evidence and local experience into an updated evidence-based practice guideline to further reduce COT to 32 hours for hospitalized infants. Key factors contributing to our success included multidisciplinary engagement, specifically partnering with nurses and resident physicians in designing and implementing our initiatives. Furthermore, improved transparency of culture monitoring practices allowed clinicians to better understand the recommended observation periods. Finally, we employed a standardized EHR order set as a no-cost, one-time, high-reliability intervention to establish 24 hours of culture monitoring as the default and to enhance transparency around start time for culture incubation.

Average COT remained stable at 32 hours for 17 months after initiation of the intervention. During the intervention period, 64% patients with hospital stays longer than 30 hours had cultures obtained between 9 pm to 4 am. These patients often remained hospitalized for longer than 30 hours to allow for a daytime hospital discharge. Additionally, CSF cultures were only monitored manually once per day between 8 am and 10 am. As a result, CSF cultures obtained in the evening (eg, 9 pm) would be evaluated once at roughly 12 hours of incubation, and then the following morning at 36 hours of incubation. In cases where CSF studies (eg, cell count, protein, Gram stain) were abnormal, uninterpretable, or could not be obtained, clinicians monitored CSF cultures closer to 36 hours from incubation. While evidence-based guidelines and local data support safe early discharge of febrile infants, clinicians presented with incomplete or uninterpretable data were appropriately more likely to observe infants for longer periods to confirm negative cultures.

Limitations

The study has several limitations. First, this single-center study was conducted at a quaternary care medical center with a robust quality improvement infrastructure. Our interventions took advantage of the existing processes in place that ensure timely discharge of medically ready patients.¹¹ Furthermore, microbiology laboratory practices are unique to our institution. These factors limit the generalizability of this work. Second, due to small numbers of eligible infants, analyses were conducted per five patients. Infrequent hospitalizations limited our ability to learn quickly from PDSA cycles. Finally, we did not measure cost savings attributable to shorter hospital stays. However, in addition to financial savings from charges and decreased nonmedical costs such as lost earnings and childcare,¹⁷ shorter hospitalizations have many additional benefits, such as promoting bonding and breastfeeding and decreasing exposure to nosocomial infections. Shorter hospitalizations, with clearly communicated discharge times, also serve to optimize patient throughput.

Implementation of a clinical practice guideline resulted in reduction of average COT from 38 to 32 hours in febrile infants aged 60 days and younger, with no cases of missed IBI. Engagement of multidisciplinary stakeholders in the generation and structured dissemination of the evidence-based guideline, improved transparency of the microbiological blood and CSF culture process, and standardization of EHR order sets were crucial to the success of this work. Cultures incubated overnight and daily CSF culture-monitoring practices primarily contributed to an average LOS of more than 30 hours.

Future work will include collaboration with emergency physicians to improve evaluation efficiency and decrease LOS in the ED for febrile infants. Additionally, creation of an automated data dashboard of COT and LOS will provide clinicians with real-time feedback on hospitalization practices.

The authors thank Dr Jeffrey Simmons, MD, MSc, as well as the members of the 2019 Fever of Uncertain Source Evidence-Based Guideline Committee. We also thank the James M Anderson Center for Health System Excellence and the Rapid Cycle Improvement Collaborative for their support with guideline development as well as design and execution of our improvement efforts.

Febrile infants aged 0 to 60 days often undergo diagnostic testing to evaluate for invasive bacterial infections (IBI; ie, bacteremia and meningitis) and are subsequently hospitalized pending culture results. Only 1% to 2% of infants 0 to 60 days old have an IBI,^1-3 and most hospitalized infants are discharged once physicians feel confident that pathogens are unlikely to be isolated from blood and cerebrospinal fluid (CSF) cultures. Practice regarding duration of hospitalization while awaiting blood and CSF culture results is not standardized in this population. Longer hospitalizations can lead to increased costs and familial stress, including difficulty with breastfeeding and anxiety in newly postpartum mothers.^4,5

In 2010, an institutional evidence-based guideline for the management of febrile infants aged 0 to 60 days recommended discharge after 36 hours of observation if all cultures were negative.⁶ However, recent studies demonstrate that 85% to 93% of pathogens in blood and CSF cultures grow within 24 hours of incubation.^7-9 Assuming a 2% prevalence of IBI, if 15% of pathogens were identified after 24 hours of incubation, only one out of 333 infants would have an IBI identified after 24 hours of hospital observation.⁷

Furthermore, a review of our institution’s electronic health records (EHR) over the past 5 years revealed that an observation period of 24 hours would have resulted in the discharge of three infants with an IBI. Two infants had bacteremia; both were discharged from the emergency department (ED) without antibiotics, returned to care after cultures were reported positive at 27 hours, and had no adverse outcomes. The third infant had meningitis, but also had an abnormal CSF Gram stain, which led to a longer hospitalization.

In 2019, our institution appraised the emerging literature and institutional data supporting the low absolute risk of missed IBI, and also leveraged local consensus among key stakeholders to update its evidence-based guideline for the evaluation and management of febrile infants aged 60 days and younger. The updated guideline recommends that clinicians consider discharging well-appearing neonates and infants if blood and CSF cultures remain negative at 24 hours.¹⁰ The objective of this study was to decrease the average hospital culture observation time (COT; culture incubation to hospital discharge) from 38 to 30 hours over a 12-month period in febrile infants aged 0 to 60 days.

Context

Improvement efforts were conducted at Cincinnati Children’s Hospital Medical Center (CCHMC), a large, urban, academic hospital that admitted more than 8,000 noncritically ill patients to the hospital medicine (HM) service from July 1, 2018, through June 30, 2019. Hospital medicine teams, located at both the main and satellite campuses, are staffed by attending physicians, fellows, residents, medical students, and nurse practitioners. The two campuses, which are about 20 miles apart, share clinician providers but have distinct nursing pools.

Microbiology services for all CCHMC patients are provided at the main campus. Blood and CSF cultures at the satellite campus are transported to the main campus for incubation and monitoring via an urgent courier service. The microbiology laboratory at CCHMC uses a continuous monitoring system for blood cultures (BACT/ALERT Virtuo, BioMérieux). The system automatically alerts laboratory technicians of positive cultures; these results are reported to clinical providers within 30 minutes of detection. Laboratory technicians manually evaluate CSF cultures once daily for 5 days.

Improvement Team

Our improvement team included three HM attending physicians; two HM fellows; a pediatric chief resident; two nurses, who represented nursing pools at the main and satellite campuses; and a clinical pharmacist, who is a co-leader of the antimicrobial stewardship program at CCHMC. Supporting members for the improvement team included the CCHMC laboratory director; the microbiology laboratory director; an infectious disease physician, who is a co-leader of the antimicrobial stewardship program; and nursing directors of the HM units at both campuses.

Evidence-Based Guideline

Our improvement initiative was based on recommendations from the updated CCHMC Evidence-Based Care Guideline for Management of Infants 0 to 60 days with Fever of Unknown Source.¹⁰ This guideline, published in May 2019, was developed by a multidisciplinary working group composed of key stakeholders from HM, community pediatrics, emergency medicine, the pediatric residency program, infectious disease, and laboratory medicine. Several improvement team members were participants on the committee that published the evidence-based guideline. The committee first performed a systematic literature review and critical appraisal of the literature. Care recommendations were formulated via a consensus process directed by best evidence, patient and family preferences, and clinical expertise; the recommendations were subsequently reviewed and approved by clinical experts who were not involved in the development process.

Based on evidence review and multistakeholder consensus, the updated guideline recommends clinicians consider discharging neonates and infants aged 60 days and younger if there is no culture growth after an observation period of 24 hours (as documented in the EHR) and patients are otherwise medically ready for discharge (ie, well appearing with adequate oral intake).^10,11 In addition, prior to discharge, there must be a documented working phone number on file for the patient’s parents/guardians, an established outpatient follow-up plan within 24 hours, and communication with the primary pediatrician who is in agreement with discharge at 24 hours.

Study Population

Infants 0 to 60 days old who had a documented or reported fever without an apparent source based on history and physical exam upon presentation to the ED, and who were subsequently admitted to the HM service at CCHMC between October 30, 2018, and July 10, 2020, were eligible for inclusion. We excluded infants who were admitted to other clinical services (eg, intensive care unit); had organisms identified on blood, urine, or CSF culture within 24 hours of incubation; had positive herpes simplex virus testing; had skin/soft tissue infections or another clearly documented source of bacterial infection; or had an alternative indication for hospitalization (eg, need for intravenous fluid or deep suctioning) after cultures had incubated for 24 hours. Infants who had a positive blood, urine, or CSF culture result after 24 hours of incubation were included in the study population. Organisms were classified as pathogen or contaminant based on treatment decisions made by the care team.

Improvement Activities

Key drivers critical to success of the improvement efforts were: (1) clearly defined standard of care for duration of observation in febrile infants 0 to 60 days old; (2) improved understanding of microbiology lab procedures; (3) effective communication of discharge criteria between providers and nurses; and (4) transparency of data with feedback (Figure 1).

Education and Structured Dissemination of Evidence-Based Guideline

The CCHMC febrile infant guideline¹⁰ was disseminated to HM physicians, residents, and nurses via the following means: (1) in-person announcements at staff meetings and educational conferences, (2) published highlights from the guideline in weekly newsletters, and (3) email announcements. Additionally, members of the study team educated HM attending physicians, nursing staff from the medical units at both campuses, and resident physicians about recent studies demonstrating safety of shorter length of stay (LOS) in febrile infants aged 0 to 60 days. The study team also provided residents, physicians, and nurses with data on the number of positive blood and CSF cultures and outcomes of patients at CCHMC within the past 5 years. In addition, team members led a journal club for residents discussing an article⁷ describing time-to-positivity of blood and CSF cultures in febrile infants. For ongoing engagement, the evidence-based guideline and a detailed explanation of microbiology procedures were published in the resident handbook, an internal resource that includes vital clinical pearls and practice guidelines across specialties. (Each resident receives an updated hard copy each year, and there is also an online link to the resource in the EHR.) Information about the guideline and COT was also included in the monthly chief resident’s orientation script, which is relayed to all residents on the first day of their HM rotation.

Clear Communication of Microbiology Procedures

Team members created a detailed process map describing the processing protocols for blood and CSF cultures collected at both CCHMC campuses. This information was shared with HM attending physicians and nurses via in-person announcements at staff meetings, flyers in team workrooms, and email communications. Residents received information on microbiology protocols via in-person announcements at educational conferences and dissemination in the weekly residency newsletter.Important information communicated included:

1. Definition of culture start time. We conveyed that there may be a delay of up to 4 hours between culture collection at the satellite campus and culture incubation at the main campus laboratory. As a result, the time of blood or CSF sample arrival to the main campus laboratory was a more accurate reflection of the culture incubation start time than the culture collection time.

2. Explanation of CSF culture processing. We discussed the process by which these cultures are plated upon arrival at the microbiology laboratory and read once per day in the morning. Therefore, a culture incubated at midnight would be evaluated once at 9 hours and not again until 33 hours.

Modification of Febrile Infant Order Set

Enhancements to the febrile infant order set improved communication and cultivated a shared mental model regarding discharge goals among all members of the care team. The EHR order set for febrile infants was updated as follows: (1) mandatory free-text fields that established the culture start time for blood and CSF cultures were added, (2) culture start time was clearly defined (ie, the time culture arrives at the main campus laboratory), and (3) a change was made in the default discharge criteria¹¹ to “culture observation for 24 hours,” with the ability to modify COT (Appendix Figure 1). We embedded hyperlinks to the guideline and microbiology process map within the updated order set, which allowed providers to easily access this information and refresh their knowledge of the recommendations (Appendix Figure 1).

Identification of Failures and Follow-up With Near-Time Feedback

All cases of febrile infants were tracked weekly. For infants hospitalized longer than 24 hours, the study team contacted the discharging clinicians to discuss reasons for prolonged hospitalization, with an emphasis on identifying system-level barriers to earlier discharge.

Study of the Interventions

The institutional microbiology database was queried weekly to identify all infants 0 to 60 days old who had a blood culture obtained and were hospitalized on the HM service. Study team members conducted targeted EHR review to determine whether patients met exclusion criteria and to identify reasons for prolonged COT. Baseline data were collected retrospectively for a 3-month period prior to initiation of improvement activities. During the study period, queries were conducted weekly and reviewed by study team members to evaluate the impact of improvement activities and to inform new interventions.

Measures

Our primary outcome measure was COT, defined as the hours between final culture incubation and hospital discharge. The operational definition for “final culture incubation” was the documented time of arrival of the last collected culture to the microbiology laboratory. Our goal COT was 30 hours to account for a subset of patients whose blood and/or CSF culture were obtained overnight (ie, after 9 pm), since subsequent discharge times would likely and practically be delayed beyond 24 hours. Our secondary outcome measure was LOS, defined as the time between ED arrival and hospital discharge. Process measures included the proportion of patients for whom the febrile infant EHR order set was used and the proportion of patients for whom medical discharge criteria (ie, blood and CSF culture observed for ”xx” hours) and culture incubation start times were entered using the order set. Balancing measures included identification of IBI after hospital discharge, 48-hour ED revisits, and 7-day hospital readmissions.

Analysis

Measures were evaluated using statistical process control charts and run charts, and Western Electric rules were employed to determine special cause variation.¹² Annotated X-bar S control charts tracked the impact of improvement activities on average COT and LOS for all infants. Given that a relatively small number of patients (ie, two to four) met inclusion criteria each week, average COT was calculated per five patients.

This study was considered exempt from review by the CCHMC Institutional Review Board.

Of the 184 infants in this study, 46 were included as part of baseline data collection, and 138 were included during the intervention period. The median age was 26.6 days (range, 3-59 days); 52% of patients were female; two-thirds were non-Hispanic White; 22% were Black, and 5% were Hispanic (Appendix Table).

Average COT decreased from 38 hours to 32 hours with improvement activities (Figure 2) and was sustained for a total of 17 months. There were small decreases in COT after initial education was provided to attendings, nurses, and residents.

After the launch of the updated order set, median usage of the EHR order set increased from 50% to 80%. Medical discharge criteria were entered for 80 (96%) of the 83 patients for whom the updated order set was applied; culture incubation start times were entered for 78 (94%) of these patients.

No infants in our cohort were found to have IBI after hospital discharge. There were no ED revisits within 48 hours of discharge, and there were no hospital readmissions within 7 days of index discharge. Furthermore, none of the patients included in the study had growth of a pathogenic organism after 24 hours.

Of the 138 infants hospitalized during the intervention period, 77 (56%) had a COT greater than 30 hours. Among these 77 patients, 49 (64%) had their final culture incubated between 9 pm and 4 am; Furthermore, 11 (14%) had missing, abnormal, pretreated, or uninterpretable CSF studies, 7 (9%) had ongoing fevers, and 4 (5%) remained hospitalized due to family preference or inability to obtain timely outpatient follow-up.

Our study aimed to decrease the average COT from 38 hours to 30 hours among hospitalized infants aged 60 days and younger over a period of 12 months. An intervention featuring implementation of an evidence-based guideline through education, laboratory procedure transparency, creation of a standardized EHR order set, and near-time feedback was associated with a shorter average COT of 32 hours, sustained over a 17-month period. No infants with bacteremia or meningitis were inappropriately discharged during this study.

Interpretation

Prior to our improvement efforts, most febrile infants at CCHMC were observed for at least 36 hours based on a prior institutional guideline,⁶ despite recent evidence suggesting that most pathogens in blood and CSF cultures grow within 24 hours of incubation.^7-9 The goal of this improvement initiative was to bridge the gap between emerging evidence and clinical practice by developing and disseminating an updated evidence-based guideline to safely decrease the hospital observation time in febrile infants aged 60 days and younger.

Similar to previous studies aimed at improving diagnosis and management among febrile infants,^13-16 generation and structured dissemination of an institutional evidence-based guideline was crucial to safely shortening COT in our population. These prior studies established a goal COT of 36 to 42 hours for hospitalized febrile infants.^13,15,16 Our study incorporated emerging evidence and local experience into an updated evidence-based practice guideline to further reduce COT to 32 hours for hospitalized infants. Key factors contributing to our success included multidisciplinary engagement, specifically partnering with nurses and resident physicians in designing and implementing our initiatives. Furthermore, improved transparency of culture monitoring practices allowed clinicians to better understand the recommended observation periods. Finally, we employed a standardized EHR order set as a no-cost, one-time, high-reliability intervention to establish 24 hours of culture monitoring as the default and to enhance transparency around start time for culture incubation.

Average COT remained stable at 32 hours for 17 months after initiation of the intervention. During the intervention period, 64% patients with hospital stays longer than 30 hours had cultures obtained between 9 pm to 4 am. These patients often remained hospitalized for longer than 30 hours to allow for a daytime hospital discharge. Additionally, CSF cultures were only monitored manually once per day between 8 am and 10 am. As a result, CSF cultures obtained in the evening (eg, 9 pm) would be evaluated once at roughly 12 hours of incubation, and then the following morning at 36 hours of incubation. In cases where CSF studies (eg, cell count, protein, Gram stain) were abnormal, uninterpretable, or could not be obtained, clinicians monitored CSF cultures closer to 36 hours from incubation. While evidence-based guidelines and local data support safe early discharge of febrile infants, clinicians presented with incomplete or uninterpretable data were appropriately more likely to observe infants for longer periods to confirm negative cultures.

Limitations

The study has several limitations. First, this single-center study was conducted at a quaternary care medical center with a robust quality improvement infrastructure. Our interventions took advantage of the existing processes in place that ensure timely discharge of medically ready patients.¹¹ Furthermore, microbiology laboratory practices are unique to our institution. These factors limit the generalizability of this work. Second, due to small numbers of eligible infants, analyses were conducted per five patients. Infrequent hospitalizations limited our ability to learn quickly from PDSA cycles. Finally, we did not measure cost savings attributable to shorter hospital stays. However, in addition to financial savings from charges and decreased nonmedical costs such as lost earnings and childcare,¹⁷ shorter hospitalizations have many additional benefits, such as promoting bonding and breastfeeding and decreasing exposure to nosocomial infections. Shorter hospitalizations, with clearly communicated discharge times, also serve to optimize patient throughput.

Implementation of a clinical practice guideline resulted in reduction of average COT from 38 to 32 hours in febrile infants aged 60 days and younger, with no cases of missed IBI. Engagement of multidisciplinary stakeholders in the generation and structured dissemination of the evidence-based guideline, improved transparency of the microbiological blood and CSF culture process, and standardization of EHR order sets were crucial to the success of this work. Cultures incubated overnight and daily CSF culture-monitoring practices primarily contributed to an average LOS of more than 30 hours.

Future work will include collaboration with emergency physicians to improve evaluation efficiency and decrease LOS in the ED for febrile infants. Additionally, creation of an automated data dashboard of COT and LOS will provide clinicians with real-time feedback on hospitalization practices.

The authors thank Dr Jeffrey Simmons, MD, MSc, as well as the members of the 2019 Fever of Uncertain Source Evidence-Based Guideline Committee. We also thank the James M Anderson Center for Health System Excellence and the Rapid Cycle Improvement Collaborative for their support with guideline development as well as design and execution of our improvement efforts.

Febrile infants aged 0 to 60 days often undergo diagnostic testing to evaluate for invasive bacterial infections (IBI; ie, bacteremia and meningitis) and are subsequently hospitalized pending culture results. Only 1% to 2% of infants 0 to 60 days old have an IBI,^1-3 and most hospitalized infants are discharged once physicians feel confident that pathogens are unlikely to be isolated from blood and cerebrospinal fluid (CSF) cultures. Practice regarding duration of hospitalization while awaiting blood and CSF culture results is not standardized in this population. Longer hospitalizations can lead to increased costs and familial stress, including difficulty with breastfeeding and anxiety in newly postpartum mothers.^4,5

In 2010, an institutional evidence-based guideline for the management of febrile infants aged 0 to 60 days recommended discharge after 36 hours of observation if all cultures were negative.⁶ However, recent studies demonstrate that 85% to 93% of pathogens in blood and CSF cultures grow within 24 hours of incubation.^7-9 Assuming a 2% prevalence of IBI, if 15% of pathogens were identified after 24 hours of incubation, only one out of 333 infants would have an IBI identified after 24 hours of hospital observation.⁷

Furthermore, a review of our institution’s electronic health records (EHR) over the past 5 years revealed that an observation period of 24 hours would have resulted in the discharge of three infants with an IBI. Two infants had bacteremia; both were discharged from the emergency department (ED) without antibiotics, returned to care after cultures were reported positive at 27 hours, and had no adverse outcomes. The third infant had meningitis, but also had an abnormal CSF Gram stain, which led to a longer hospitalization.

In 2019, our institution appraised the emerging literature and institutional data supporting the low absolute risk of missed IBI, and also leveraged local consensus among key stakeholders to update its evidence-based guideline for the evaluation and management of febrile infants aged 60 days and younger. The updated guideline recommends that clinicians consider discharging well-appearing neonates and infants if blood and CSF cultures remain negative at 24 hours.¹⁰ The objective of this study was to decrease the average hospital culture observation time (COT; culture incubation to hospital discharge) from 38 to 30 hours over a 12-month period in febrile infants aged 0 to 60 days.

Context

Improvement efforts were conducted at Cincinnati Children’s Hospital Medical Center (CCHMC), a large, urban, academic hospital that admitted more than 8,000 noncritically ill patients to the hospital medicine (HM) service from July 1, 2018, through June 30, 2019. Hospital medicine teams, located at both the main and satellite campuses, are staffed by attending physicians, fellows, residents, medical students, and nurse practitioners. The two campuses, which are about 20 miles apart, share clinician providers but have distinct nursing pools.

Microbiology services for all CCHMC patients are provided at the main campus. Blood and CSF cultures at the satellite campus are transported to the main campus for incubation and monitoring via an urgent courier service. The microbiology laboratory at CCHMC uses a continuous monitoring system for blood cultures (BACT/ALERT Virtuo, BioMérieux). The system automatically alerts laboratory technicians of positive cultures; these results are reported to clinical providers within 30 minutes of detection. Laboratory technicians manually evaluate CSF cultures once daily for 5 days.

Improvement Team

Our improvement team included three HM attending physicians; two HM fellows; a pediatric chief resident; two nurses, who represented nursing pools at the main and satellite campuses; and a clinical pharmacist, who is a co-leader of the antimicrobial stewardship program at CCHMC. Supporting members for the improvement team included the CCHMC laboratory director; the microbiology laboratory director; an infectious disease physician, who is a co-leader of the antimicrobial stewardship program; and nursing directors of the HM units at both campuses.

Evidence-Based Guideline

Our improvement initiative was based on recommendations from the updated CCHMC Evidence-Based Care Guideline for Management of Infants 0 to 60 days with Fever of Unknown Source.¹⁰ This guideline, published in May 2019, was developed by a multidisciplinary working group composed of key stakeholders from HM, community pediatrics, emergency medicine, the pediatric residency program, infectious disease, and laboratory medicine. Several improvement team members were participants on the committee that published the evidence-based guideline. The committee first performed a systematic literature review and critical appraisal of the literature. Care recommendations were formulated via a consensus process directed by best evidence, patient and family preferences, and clinical expertise; the recommendations were subsequently reviewed and approved by clinical experts who were not involved in the development process.

Based on evidence review and multistakeholder consensus, the updated guideline recommends clinicians consider discharging neonates and infants aged 60 days and younger if there is no culture growth after an observation period of 24 hours (as documented in the EHR) and patients are otherwise medically ready for discharge (ie, well appearing with adequate oral intake).^10,11 In addition, prior to discharge, there must be a documented working phone number on file for the patient’s parents/guardians, an established outpatient follow-up plan within 24 hours, and communication with the primary pediatrician who is in agreement with discharge at 24 hours.

Study Population

Infants 0 to 60 days old who had a documented or reported fever without an apparent source based on history and physical exam upon presentation to the ED, and who were subsequently admitted to the HM service at CCHMC between October 30, 2018, and July 10, 2020, were eligible for inclusion. We excluded infants who were admitted to other clinical services (eg, intensive care unit); had organisms identified on blood, urine, or CSF culture within 24 hours of incubation; had positive herpes simplex virus testing; had skin/soft tissue infections or another clearly documented source of bacterial infection; or had an alternative indication for hospitalization (eg, need for intravenous fluid or deep suctioning) after cultures had incubated for 24 hours. Infants who had a positive blood, urine, or CSF culture result after 24 hours of incubation were included in the study population. Organisms were classified as pathogen or contaminant based on treatment decisions made by the care team.

Improvement Activities

Key drivers critical to success of the improvement efforts were: (1) clearly defined standard of care for duration of observation in febrile infants 0 to 60 days old; (2) improved understanding of microbiology lab procedures; (3) effective communication of discharge criteria between providers and nurses; and (4) transparency of data with feedback (Figure 1).

Education and Structured Dissemination of Evidence-Based Guideline

The CCHMC febrile infant guideline¹⁰ was disseminated to HM physicians, residents, and nurses via the following means: (1) in-person announcements at staff meetings and educational conferences, (2) published highlights from the guideline in weekly newsletters, and (3) email announcements. Additionally, members of the study team educated HM attending physicians, nursing staff from the medical units at both campuses, and resident physicians about recent studies demonstrating safety of shorter length of stay (LOS) in febrile infants aged 0 to 60 days. The study team also provided residents, physicians, and nurses with data on the number of positive blood and CSF cultures and outcomes of patients at CCHMC within the past 5 years. In addition, team members led a journal club for residents discussing an article⁷ describing time-to-positivity of blood and CSF cultures in febrile infants. For ongoing engagement, the evidence-based guideline and a detailed explanation of microbiology procedures were published in the resident handbook, an internal resource that includes vital clinical pearls and practice guidelines across specialties. (Each resident receives an updated hard copy each year, and there is also an online link to the resource in the EHR.) Information about the guideline and COT was also included in the monthly chief resident’s orientation script, which is relayed to all residents on the first day of their HM rotation.

Clear Communication of Microbiology Procedures

Team members created a detailed process map describing the processing protocols for blood and CSF cultures collected at both CCHMC campuses. This information was shared with HM attending physicians and nurses via in-person announcements at staff meetings, flyers in team workrooms, and email communications. Residents received information on microbiology protocols via in-person announcements at educational conferences and dissemination in the weekly residency newsletter.Important information communicated included:

1. Definition of culture start time. We conveyed that there may be a delay of up to 4 hours between culture collection at the satellite campus and culture incubation at the main campus laboratory. As a result, the time of blood or CSF sample arrival to the main campus laboratory was a more accurate reflection of the culture incubation start time than the culture collection time.

2. Explanation of CSF culture processing. We discussed the process by which these cultures are plated upon arrival at the microbiology laboratory and read once per day in the morning. Therefore, a culture incubated at midnight would be evaluated once at 9 hours and not again until 33 hours.

Modification of Febrile Infant Order Set

Enhancements to the febrile infant order set improved communication and cultivated a shared mental model regarding discharge goals among all members of the care team. The EHR order set for febrile infants was updated as follows: (1) mandatory free-text fields that established the culture start time for blood and CSF cultures were added, (2) culture start time was clearly defined (ie, the time culture arrives at the main campus laboratory), and (3) a change was made in the default discharge criteria¹¹ to “culture observation for 24 hours,” with the ability to modify COT (Appendix Figure 1). We embedded hyperlinks to the guideline and microbiology process map within the updated order set, which allowed providers to easily access this information and refresh their knowledge of the recommendations (Appendix Figure 1).

Identification of Failures and Follow-up With Near-Time Feedback

All cases of febrile infants were tracked weekly. For infants hospitalized longer than 24 hours, the study team contacted the discharging clinicians to discuss reasons for prolonged hospitalization, with an emphasis on identifying system-level barriers to earlier discharge.

Study of the Interventions

The institutional microbiology database was queried weekly to identify all infants 0 to 60 days old who had a blood culture obtained and were hospitalized on the HM service. Study team members conducted targeted EHR review to determine whether patients met exclusion criteria and to identify reasons for prolonged COT. Baseline data were collected retrospectively for a 3-month period prior to initiation of improvement activities. During the study period, queries were conducted weekly and reviewed by study team members to evaluate the impact of improvement activities and to inform new interventions.

Measures

Our primary outcome measure was COT, defined as the hours between final culture incubation and hospital discharge. The operational definition for “final culture incubation” was the documented time of arrival of the last collected culture to the microbiology laboratory. Our goal COT was 30 hours to account for a subset of patients whose blood and/or CSF culture were obtained overnight (ie, after 9 pm), since subsequent discharge times would likely and practically be delayed beyond 24 hours. Our secondary outcome measure was LOS, defined as the time between ED arrival and hospital discharge. Process measures included the proportion of patients for whom the febrile infant EHR order set was used and the proportion of patients for whom medical discharge criteria (ie, blood and CSF culture observed for ”xx” hours) and culture incubation start times were entered using the order set. Balancing measures included identification of IBI after hospital discharge, 48-hour ED revisits, and 7-day hospital readmissions.

Analysis

Measures were evaluated using statistical process control charts and run charts, and Western Electric rules were employed to determine special cause variation.¹² Annotated X-bar S control charts tracked the impact of improvement activities on average COT and LOS for all infants. Given that a relatively small number of patients (ie, two to four) met inclusion criteria each week, average COT was calculated per five patients.

This study was considered exempt from review by the CCHMC Institutional Review Board.

Of the 184 infants in this study, 46 were included as part of baseline data collection, and 138 were included during the intervention period. The median age was 26.6 days (range, 3-59 days); 52% of patients were female; two-thirds were non-Hispanic White; 22% were Black, and 5% were Hispanic (Appendix Table).

Average COT decreased from 38 hours to 32 hours with improvement activities (Figure 2) and was sustained for a total of 17 months. There were small decreases in COT after initial education was provided to attendings, nurses, and residents.

After the launch of the updated order set, median usage of the EHR order set increased from 50% to 80%. Medical discharge criteria were entered for 80 (96%) of the 83 patients for whom the updated order set was applied; culture incubation start times were entered for 78 (94%) of these patients.

No infants in our cohort were found to have IBI after hospital discharge. There were no ED revisits within 48 hours of discharge, and there were no hospital readmissions within 7 days of index discharge. Furthermore, none of the patients included in the study had growth of a pathogenic organism after 24 hours.

Of the 138 infants hospitalized during the intervention period, 77 (56%) had a COT greater than 30 hours. Among these 77 patients, 49 (64%) had their final culture incubated between 9 pm and 4 am; Furthermore, 11 (14%) had missing, abnormal, pretreated, or uninterpretable CSF studies, 7 (9%) had ongoing fevers, and 4 (5%) remained hospitalized due to family preference or inability to obtain timely outpatient follow-up.

Our study aimed to decrease the average COT from 38 hours to 30 hours among hospitalized infants aged 60 days and younger over a period of 12 months. An intervention featuring implementation of an evidence-based guideline through education, laboratory procedure transparency, creation of a standardized EHR order set, and near-time feedback was associated with a shorter average COT of 32 hours, sustained over a 17-month period. No infants with bacteremia or meningitis were inappropriately discharged during this study.

Interpretation

Prior to our improvement efforts, most febrile infants at CCHMC were observed for at least 36 hours based on a prior institutional guideline,⁶ despite recent evidence suggesting that most pathogens in blood and CSF cultures grow within 24 hours of incubation.^7-9 The goal of this improvement initiative was to bridge the gap between emerging evidence and clinical practice by developing and disseminating an updated evidence-based guideline to safely decrease the hospital observation time in febrile infants aged 60 days and younger.

Similar to previous studies aimed at improving diagnosis and management among febrile infants,^13-16 generation and structured dissemination of an institutional evidence-based guideline was crucial to safely shortening COT in our population. These prior studies established a goal COT of 36 to 42 hours for hospitalized febrile infants.^13,15,16 Our study incorporated emerging evidence and local experience into an updated evidence-based practice guideline to further reduce COT to 32 hours for hospitalized infants. Key factors contributing to our success included multidisciplinary engagement, specifically partnering with nurses and resident physicians in designing and implementing our initiatives. Furthermore, improved transparency of culture monitoring practices allowed clinicians to better understand the recommended observation periods. Finally, we employed a standardized EHR order set as a no-cost, one-time, high-reliability intervention to establish 24 hours of culture monitoring as the default and to enhance transparency around start time for culture incubation.

Average COT remained stable at 32 hours for 17 months after initiation of the intervention. During the intervention period, 64% patients with hospital stays longer than 30 hours had cultures obtained between 9 pm to 4 am. These patients often remained hospitalized for longer than 30 hours to allow for a daytime hospital discharge. Additionally, CSF cultures were only monitored manually once per day between 8 am and 10 am. As a result, CSF cultures obtained in the evening (eg, 9 pm) would be evaluated once at roughly 12 hours of incubation, and then the following morning at 36 hours of incubation. In cases where CSF studies (eg, cell count, protein, Gram stain) were abnormal, uninterpretable, or could not be obtained, clinicians monitored CSF cultures closer to 36 hours from incubation. While evidence-based guidelines and local data support safe early discharge of febrile infants, clinicians presented with incomplete or uninterpretable data were appropriately more likely to observe infants for longer periods to confirm negative cultures.

Limitations

The study has several limitations. First, this single-center study was conducted at a quaternary care medical center with a robust quality improvement infrastructure. Our interventions took advantage of the existing processes in place that ensure timely discharge of medically ready patients.¹¹ Furthermore, microbiology laboratory practices are unique to our institution. These factors limit the generalizability of this work. Second, due to small numbers of eligible infants, analyses were conducted per five patients. Infrequent hospitalizations limited our ability to learn quickly from PDSA cycles. Finally, we did not measure cost savings attributable to shorter hospital stays. However, in addition to financial savings from charges and decreased nonmedical costs such as lost earnings and childcare,¹⁷ shorter hospitalizations have many additional benefits, such as promoting bonding and breastfeeding and decreasing exposure to nosocomial infections. Shorter hospitalizations, with clearly communicated discharge times, also serve to optimize patient throughput.

Implementation of a clinical practice guideline resulted in reduction of average COT from 38 to 32 hours in febrile infants aged 60 days and younger, with no cases of missed IBI. Engagement of multidisciplinary stakeholders in the generation and structured dissemination of the evidence-based guideline, improved transparency of the microbiological blood and CSF culture process, and standardization of EHR order sets were crucial to the success of this work. Cultures incubated overnight and daily CSF culture-monitoring practices primarily contributed to an average LOS of more than 30 hours.

Future work will include collaboration with emergency physicians to improve evaluation efficiency and decrease LOS in the ED for febrile infants. Additionally, creation of an automated data dashboard of COT and LOS will provide clinicians with real-time feedback on hospitalization practices.

The authors thank Dr Jeffrey Simmons, MD, MSc, as well as the members of the 2019 Fever of Uncertain Source Evidence-Based Guideline Committee. We also thank the James M Anderson Center for Health System Excellence and the Rapid Cycle Improvement Collaborative for their support with guideline development as well as design and execution of our improvement efforts.

Bronchiolitis is the leading cause of hospitalization in infants aged <1 year in the United States.^1-3 Estimates suggest that 1.5% to 2.0% of US infants require hospitalization every year, with a median (interquartile range) length of stay of 2 days (1-4),³ incurring direct medical costs of $555 million annually.¹ Evidence suggests that few interventions, aside from supportive care, are effective for bronchiolitis.^4-7 Adherence to standardized clinical guidelines could improve outcomes and resource use by streamlining care and limiting ineffective interventions, thereby decreasing hospital length of stay, which is a major medical cost.^8-13 For this reason, many hospitals have adopted bronchiolitis guidelines, although institutional practices vary.^14,15

Two relatively unexplored methods to reduce the inpatient burden of bronchiolitis are the use of observation units (OU) and home oxygen therapy (HOT). Motivated by research demonstrating the safety and effectiveness of an emergency department (ED)–based HOT protocol,¹⁶ where 36 of 37 patients with mild hypoxemia discharged on HOT avoided hospital admission, our institution implemented an observation unit and home oxygen therapy (OU-HOT) protocol designed to return children with bronchiolitis home earlier from the hospital. In the first winter season of implementation (2010 to 2011), the OU-HOT protocol was associated with significant reductions in length of stay and substantial cost savings, without an increase in return visits to the ED or inpatient readmissions.¹⁷ The objectives of this study were to determine whether these encouraging initial findings persisted and to measure the long-term impact of the OU-HOT protocol.

We conducted a retrospective cohort study of children hospitalized with bronchiolitis at Primary Children’s Hospital, a freestanding children’s hospital in Salt Lake City, Utah. Discharge diagnosis and procedures codes, as well as laboratory, imaging, pharmacy, and supply costs, were obtained from the Intermountain Healthcare enterprise data warehouse. A crosswalk available from the Centers for Medicare and Medicaid Services was used to convert International Classification of Diseases (ICD)-10 discharge diagnosis and procedure codes to ICD-9 equivalents.¹⁸ This study was approved by the University of Utah institutional review board (00110419).

Patients

Children aged 3 to 24 months who were discharged with a diagnosis of bronchiolitis (466.xx) during winter seasons from 2007 to 2019 were included. A winter season was defined as November 1 to April 30. Both observation and inpatient encounters were included in the cohort. We excluded patients with discharge diagnosis or procedure codes indicating tracheostomy (519.0-519.09, V44.0, V55.0, 31.1, 31.21, 31.41, 31.74, 97.23), ventilator dependence (V46.1x), chronic lung disease (518.83, 770.7), or pulmonary hypertension (416.xx). Patients with both bronchiolitis and a concurrent diagnosis, such as otitis media or pneumonia, were included unless exclusion criteria were met.

Intervention and Process Measures

Our institution implemented the OU-HOT protocol at the start of the 2010-2011 winter season.¹⁷ The aim of the OU-HOT protocol was to discharge children with bronchiolitis home sooner by increasing use of both an OU, with frequent assessment of discharge readiness, and HOT to help children become ready for discharge. Similar to most OUs, admission to our unit was limited to patients who met hospital admission criteria, and had a short anticipated length of stay (<48 hours). As a self-contained 20-bed unit providing 24-hour dedicated pediatrician/pediatric emergency medicine physician and nursing coverage, the OU actively monitored patients’ discharge readiness, with a goal to facilitate patient throughput more akin to an ED rather than a traditional inpatient unit. Patients who could not be discharged from the OU within 48 hours were transferred to the inpatient unit. Although the OU existed at the time of protocol implementation, its use for patients with bronchiolitis was not actively encouraged until implementation.

Hospitalized patients—in either inpatient or observation units—were eligible for discharge on HOT if they met the following criteria: hypoxemia was the only indication for continued hospitalization, the child’s oxygen requirement was <0.5 L/min for at least 6 hours (0.8 L/min for children aged >1 year), the child’s caregiver(s) were willing to manage oxygen at home, and the child had reliable access to primary care provider follow up. We used two process measures across winter seasons: (1) the percentage of patients discharged from the OU, and (2) the percentage of patients discharged with HOT. The percentage of patients discharged on HOT was estimated by a manual chart review and an electronic medical record (EMR) HOT flag that came into existence with our hospital system’s adoption of a new EMR (2017-2019). Chart review randomly sampled patients from 2007-2017, totaling 457 patients. To estimate the reliability of this method, we calculated the sensitivity, specificity, positive predictive value, and negative predictive value of the EMR HOT flag using chart review as the gold standard.

Outcome Measures

The main outcome measure was mean hospital length of stay. Balancing measures were revisit rates (stratified into ED visits and readmissions) and annual per-population bronchiolitis admission rates. Visits were considered revisits if they occurred within 7 days of initial hospital discharge, and included visits to Primary Children’s Hospital as well as 22 other Intermountain Healthcare hospitals. Population estimates from the Utah Department of Health were used to calculate the annual population-based rate of bronchiolitis admissions to Primary Children’s Hospital.¹⁹ Annual admission rates were calculated per 10,000 children aged 3 to 24 months who resided in Utah each year of the study period, and were evaluated to determine if patients were admitted more frequently after OU-HOT implementation. Secondary outcome measures included the percentage of patients discharged within 24 hours and mean inflation-adjusted cost per episode of care (in 2019 dollars). Hospitalization costs were determined using Intermountain Healthcare’s internal cost accounting system, an activity-based method that aggregates costs of individual resources according to date of service.²⁰ Costs were adjusted to 2019 dollars and were defined as the total costs of a patient’s initial hospitalization as well as any 7-day revisit encounters.

Data Analysis

Demographic data were compared before and after OU-HOT protocol implementation using Pearson chi-square tests. Multivariable linear or logistic regression models were used to compare measures before and after OU-HOT protocol implementation via an interrupted time-series approach. The interrupted time-series analysis measured two types of changes after protocol implementation during the 2010-2011 winter season: (1) any immediate change in the level of an outcome (immediate effect) and (2) any change of an outcome going forward over time (change in slope).²¹ Covariates in the regression models included patient age, sex, race, ethnicity, and insurance type, as well as presence of an underlying complex chronic condition, mechanical ventilation use, and pediatric intensive care unit (PICU) admission during hospitalization. Data were analyzed in STATA 15 (StataCorp LLC).²²

A total of 7,116 patients met inclusion criteria over the study period (2,061 pre-implementation, 5,055 post-implementation). A comparison of patient characteristics before and after HOT protocol implementation is presented in Table 1. Patients were similar in terms of age, sex, and insurance type. Patients in the postimplementation period were more likely to have a complex chronic condition, require admission to the PICU, and need mechanical ventilation (P < .01). Differences between cohorts with regard to race/ethnicity distribution largely were a result of improved capture of these data elements in the postimplementation period. For example, 30% of patients were classified as “race/ethnicity unknown” in the preimplementation cohort, compared with 4% of patients in the postimplementation period.

Process Measures

Figure 1 shows trends in OU and HOT use by winter season. The percentage of patients discharged from the OU increased immediately after OU-HOT protocol implementation (absolute 26.9% immediate increase; 95% CI, 21.9-42.2). The change in the proportion of OU use per season also increased (change in slope +3.9% per season; 95% CI, 3.4%-4.4%). The percentage of patients discharged with HOT increased immediately after OU-HOT protocol implementation (26.0% immediate change; 95% CI, 18.9%-33.1%); however, the immediate increase in HOT discharges was coupled with a declining rate of HOT discharges per season in the postprotocol period compared with the preprotocol period (change in slope –4.5% per season; 95% CI, –7.5% to –1.5%). Our chart review and EMR flag included 1,354 patients, or 19.0% of our cohort. Our EMR flag for HOT in the last two seasons of the study had a positive predictive value of 100% (5 of 5 identified by EMR flag as receiving HOT were confirmed by chart review) and negative predictive value of 89% (31 of 35 identified by EMR flag as not receiving HOT were confirmed by chart review). The specificity of the EMR flag was 100% (31 of 31 of those confirmed by chart review as not receiving HOT, who were correctly identified by EMR) and the sensitivity was 55% (5 of 9 of those confirmed by chart review as receiving HOT, who were correctly identified by EMR).

Primary and Secondary Outcomes

Trends in length of stay across winter seasons are presented in Figure 2. The OU-HOT protocol was associated with an immediate reduction of 30.6 hours in mean length of stay (95% CI, –37.1 to –24.2). The rate of change in length of stay postimplementation did not differ significantly from the rate of change preimplementation (change in slope –0.6 hours per season; 95% CI, –2.3 to 1.1 hours). The percentage of patients discharged within 24 hours of admission rose immediately after protocol implementation, by 23.8 absolute percentage points (95% CI, 11.7-28.8). Slopes of the preintervention and postintervention regression lines did not differ significantly (change in slope –0.1% per season; 95% CI, –1.4% to 1.1%). Immediate decreases in length of stay were accompanied by an immediate decrease in mean cost per episode of care (–$4,181; 95% CI, –$4,829 to –$3,533). Protocol implementation also was associated with a decreased slope in cost postimplementation (change in slope –$403 per season; 95% CI, –$543 to –$264). The total cost savings, estimated by the product of the average cost savings per episode of care and the number of bronchiolitis admissions included in the study after OU-HOT implementation, amounted to $21.1 million over the 9-year period, or $2.3 million per winter season.

Balancing Measures

We observed an immediate reduction in 7-day hospital revisits (–1.1% immediate change; 95% CI, –1.8% to –0.4%), but an increasing slope in revisits after implementation (change in slope 0.4% per season; 95% CI, 0.1%-0.8%) (Figure 3). Stratifying revisits into ED visits and readmissions revealed that the revisit findings reflected changes in ED return visits, for which there was an immediate reduction at the time of implementation (–1.0% immediate change; 95% CI, –1.6% to –0.4%), but an increasing slope postimplementation (change in slope 0.5% per season; 95% CI, 0.2-0.8). Neither an immediate intervention effect (0.0% immediate change; 95% CI, –0.5% to 0.4%) nor a change in slope (change in slope 0.0% per season; 95% CI, –0.1% to 0.1%) were observed for inpatient readmissions alone. The annual rate of bronchiolitis admissions to Primary Children’s Hospital per 10,000 children who reside in Utah decreased after implementation of the OU-HOT protocol (immediate intervention effect –6.2 admissions; 95% CI, –10.8 to –1.6; change in slope –1.8 admissions per season; 95% CI, –2.8 to –0.69).

Our OU-HOT protocol was associated with immediate improvements in care delivered to children hospitalized for bronchiolitis, including decreased length of stay and cost savings. These improvements in outcomes largely have been sustained over a 9-year period. The OU-HOT protocol also appears to be safe as evidenced by a stable rate of readmissions over the study period and only a small increase in revisits to EDs across Intermountain Healthcare facilities, which see most children in the catchment area. Our OU-HOT protocol represents a combination of two interventions: (1) the creation of an OU focused on discharge within 24 to 48 hours of admission and (2) encouragement to discharge children with HOT. We found that use of the OU and a commitment to timely discharges has been sustained in recent years, while the commitment to HOT has appeared to wane.

Earlier investigations have evaluated the efficacy of HOT in the ED setting to prevent hospital admissions, finding high levels of caregiver comfort, estimating $1,300 per patient cost savings, and reporting readmission rates of approximately 5%.^16,23-25 Our study is unique in addressing HOT among a population of patients already hospitalized with bronchiolitis. The cost reductions we observed with our OU-HOT protocol were similar to those noted in the ED-based HOT protocols. However, we recorded lower readmission rates, likely because of the additional time allotted to caregivers to better gauge illness trajectory in the inpatient setting vs the ED, as well as additional time for hospitalized patients to reach the plateau or convalescent phase of illness. The small increase in ED revisits that we measured in recent years might be related to the concurrent rise in patient acuity and complexity.

Considering that length of stay has remained low despite less commitment to HOT, our results suggest that the OU might be the more impactful of the two interventions, and these data support the use of such a unit for a subset of patients with bronchiolitis. However, it is important to note that while the EMR HOT flag demonstrated high specificity, positive predictive value, and negative predictive value, the sensitivity was low (56%). As a result, it is possible that we have underestimated HOT use in the 2017-2018 and 2018-2019 seasons, the final two years of the study. Alternatively, the discrepancy between sustained outcomes and lagging use of HOT could be explained by improved identification of patients who would experience the greatest benefit with oxygen in terms of length of stay reductions, with fewer patients discharged on HOT but greater per-patient benefit. Finally, in an era that encourages reduced monitor use and less aggressive response to transient mild desaturations,^13,26,27 it is possible that fewer patients are identified with clinically actionable hypoxemia around the time they would be otherwise discharged.

Our OU-HOT model is not unprecedented. Increasingly, other formerly inpatient indications are being successfully managed in the observation, outpatient, and home setting, such as parenteral antibiotic treatment^28,29 and chemotherapy administration.³⁰ Considering the inpatient burden of bronchiolitis, similar strategies to expedite discharge are needed. Although outpatient intravenous antibiotic and chemotherapy administration have been widely adopted, we are aware of only one other pediatric health care system in the United States (Children’s Hospital Colorado) that routinely discharges inpatients with bronchiolitis on HOT.

This study has several limitations. First, although the interrupted time-series analysis is designed to account for trends that precede an intervention and covariates that differ before and after the intervention, it is possible that important unmeasured patient factors or changes in practice patterns differed between the pre- and post-intervention cohorts. There were no major changes to the OU-HOT protocol or discharge criteria after implementation, but individual practice management of bronchiolitis during the study period likely has evolved as new evidence emerges. Second, one could postulate that the increase in discharges within 24 hours and accompanying decreases in average length of stay and cost could be achieved by hospitalizing healthier patients over time, which the presence of an OU might incentivize. To the contrary, we found that population-based bronchiolitis admission rates have declined and disease severity appears to be increased since implementation of the OU-HOT protocol. The increase in medically complex children and PICU use in our postimplementation cohort aligns with recently published data suggesting these are national trends.^3,31 Third, HOT use was estimated from a sample of the cohort using a chart review and a newly available EMR flag. A low sensitivity and a small sample for the positive predictive value are limitations of the EMR flag.

Additionally, there are almost certainly unmeasured ambulatory burdens of HOT not captured by this study. ED-based protocols have estimated that patients discharged with HOT have a median of two follow-up ambulatory visits before oxygen is discontinued³²; however, the ambulatory burden associated with discharge on HOT after a hospitalization and the extent to which demographic factors affect that burden is unknown. Furthermore, one insurance company charged $94 for a month of HOT in 2019; paying even a portion of this charge represents a nontrivial financial burden for many families, even considering inpatient cost savings. Although the decision to discharge on oxygen or remain hospitalized until the child did not need oxygen was left to the parents, their posthospitalization perspectives were not assessed in this study. Although reports indicate that families largely feel positive about HOT after discharge from an ED setting, with 90% of caregivers preferring HOT use to inpatient admission and most reporting no difficulty with home management,²³ it is uncertain whether this would also apply after inpatient hospitalization.

The OU-HOT bronchiolitis protocol was associated with decreases in inpatient length of stay and cost while appearing safe to implement. The sustained use of the OU combined with declining use of HOT suggests that the OU might be the more impactful intervention. As previously inpatient indications such as parenteral antibiotics and chemotherapy increasingly have been administered in observation and outpatient settings, bronchiolitis appears ideal for a similar strategy that allows patients to spend less time in the hospital. Studies are needed to understand the outpatient burden of HOT and the generalizability of our findings.

Bronchiolitis is the leading cause of hospitalization in infants aged <1 year in the United States.^1-3 Estimates suggest that 1.5% to 2.0% of US infants require hospitalization every year, with a median (interquartile range) length of stay of 2 days (1-4),³ incurring direct medical costs of $555 million annually.¹ Evidence suggests that few interventions, aside from supportive care, are effective for bronchiolitis.^4-7 Adherence to standardized clinical guidelines could improve outcomes and resource use by streamlining care and limiting ineffective interventions, thereby decreasing hospital length of stay, which is a major medical cost.^8-13 For this reason, many hospitals have adopted bronchiolitis guidelines, although institutional practices vary.^14,15

Two relatively unexplored methods to reduce the inpatient burden of bronchiolitis are the use of observation units (OU) and home oxygen therapy (HOT). Motivated by research demonstrating the safety and effectiveness of an emergency department (ED)–based HOT protocol,¹⁶ where 36 of 37 patients with mild hypoxemia discharged on HOT avoided hospital admission, our institution implemented an observation unit and home oxygen therapy (OU-HOT) protocol designed to return children with bronchiolitis home earlier from the hospital. In the first winter season of implementation (2010 to 2011), the OU-HOT protocol was associated with significant reductions in length of stay and substantial cost savings, without an increase in return visits to the ED or inpatient readmissions.¹⁷ The objectives of this study were to determine whether these encouraging initial findings persisted and to measure the long-term impact of the OU-HOT protocol.

We conducted a retrospective cohort study of children hospitalized with bronchiolitis at Primary Children’s Hospital, a freestanding children’s hospital in Salt Lake City, Utah. Discharge diagnosis and procedures codes, as well as laboratory, imaging, pharmacy, and supply costs, were obtained from the Intermountain Healthcare enterprise data warehouse. A crosswalk available from the Centers for Medicare and Medicaid Services was used to convert International Classification of Diseases (ICD)-10 discharge diagnosis and procedure codes to ICD-9 equivalents.¹⁸ This study was approved by the University of Utah institutional review board (00110419).

Patients

Children aged 3 to 24 months who were discharged with a diagnosis of bronchiolitis (466.xx) during winter seasons from 2007 to 2019 were included. A winter season was defined as November 1 to April 30. Both observation and inpatient encounters were included in the cohort. We excluded patients with discharge diagnosis or procedure codes indicating tracheostomy (519.0-519.09, V44.0, V55.0, 31.1, 31.21, 31.41, 31.74, 97.23), ventilator dependence (V46.1x), chronic lung disease (518.83, 770.7), or pulmonary hypertension (416.xx). Patients with both bronchiolitis and a concurrent diagnosis, such as otitis media or pneumonia, were included unless exclusion criteria were met.

Intervention and Process Measures

Our institution implemented the OU-HOT protocol at the start of the 2010-2011 winter season.¹⁷ The aim of the OU-HOT protocol was to discharge children with bronchiolitis home sooner by increasing use of both an OU, with frequent assessment of discharge readiness, and HOT to help children become ready for discharge. Similar to most OUs, admission to our unit was limited to patients who met hospital admission criteria, and had a short anticipated length of stay (<48 hours). As a self-contained 20-bed unit providing 24-hour dedicated pediatrician/pediatric emergency medicine physician and nursing coverage, the OU actively monitored patients’ discharge readiness, with a goal to facilitate patient throughput more akin to an ED rather than a traditional inpatient unit. Patients who could not be discharged from the OU within 48 hours were transferred to the inpatient unit. Although the OU existed at the time of protocol implementation, its use for patients with bronchiolitis was not actively encouraged until implementation.

Hospitalized patients—in either inpatient or observation units—were eligible for discharge on HOT if they met the following criteria: hypoxemia was the only indication for continued hospitalization, the child’s oxygen requirement was <0.5 L/min for at least 6 hours (0.8 L/min for children aged >1 year), the child’s caregiver(s) were willing to manage oxygen at home, and the child had reliable access to primary care provider follow up. We used two process measures across winter seasons: (1) the percentage of patients discharged from the OU, and (2) the percentage of patients discharged with HOT. The percentage of patients discharged on HOT was estimated by a manual chart review and an electronic medical record (EMR) HOT flag that came into existence with our hospital system’s adoption of a new EMR (2017-2019). Chart review randomly sampled patients from 2007-2017, totaling 457 patients. To estimate the reliability of this method, we calculated the sensitivity, specificity, positive predictive value, and negative predictive value of the EMR HOT flag using chart review as the gold standard.

Outcome Measures

The main outcome measure was mean hospital length of stay. Balancing measures were revisit rates (stratified into ED visits and readmissions) and annual per-population bronchiolitis admission rates. Visits were considered revisits if they occurred within 7 days of initial hospital discharge, and included visits to Primary Children’s Hospital as well as 22 other Intermountain Healthcare hospitals. Population estimates from the Utah Department of Health were used to calculate the annual population-based rate of bronchiolitis admissions to Primary Children’s Hospital.¹⁹ Annual admission rates were calculated per 10,000 children aged 3 to 24 months who resided in Utah each year of the study period, and were evaluated to determine if patients were admitted more frequently after OU-HOT implementation. Secondary outcome measures included the percentage of patients discharged within 24 hours and mean inflation-adjusted cost per episode of care (in 2019 dollars). Hospitalization costs were determined using Intermountain Healthcare’s internal cost accounting system, an activity-based method that aggregates costs of individual resources according to date of service.²⁰ Costs were adjusted to 2019 dollars and were defined as the total costs of a patient’s initial hospitalization as well as any 7-day revisit encounters.

Data Analysis

Demographic data were compared before and after OU-HOT protocol implementation using Pearson chi-square tests. Multivariable linear or logistic regression models were used to compare measures before and after OU-HOT protocol implementation via an interrupted time-series approach. The interrupted time-series analysis measured two types of changes after protocol implementation during the 2010-2011 winter season: (1) any immediate change in the level of an outcome (immediate effect) and (2) any change of an outcome going forward over time (change in slope).²¹ Covariates in the regression models included patient age, sex, race, ethnicity, and insurance type, as well as presence of an underlying complex chronic condition, mechanical ventilation use, and pediatric intensive care unit (PICU) admission during hospitalization. Data were analyzed in STATA 15 (StataCorp LLC).²²

A total of 7,116 patients met inclusion criteria over the study period (2,061 pre-implementation, 5,055 post-implementation). A comparison of patient characteristics before and after HOT protocol implementation is presented in Table 1. Patients were similar in terms of age, sex, and insurance type. Patients in the postimplementation period were more likely to have a complex chronic condition, require admission to the PICU, and need mechanical ventilation (P < .01). Differences between cohorts with regard to race/ethnicity distribution largely were a result of improved capture of these data elements in the postimplementation period. For example, 30% of patients were classified as “race/ethnicity unknown” in the preimplementation cohort, compared with 4% of patients in the postimplementation period.

Process Measures

Figure 1 shows trends in OU and HOT use by winter season. The percentage of patients discharged from the OU increased immediately after OU-HOT protocol implementation (absolute 26.9% immediate increase; 95% CI, 21.9-42.2). The change in the proportion of OU use per season also increased (change in slope +3.9% per season; 95% CI, 3.4%-4.4%). The percentage of patients discharged with HOT increased immediately after OU-HOT protocol implementation (26.0% immediate change; 95% CI, 18.9%-33.1%); however, the immediate increase in HOT discharges was coupled with a declining rate of HOT discharges per season in the postprotocol period compared with the preprotocol period (change in slope –4.5% per season; 95% CI, –7.5% to –1.5%). Our chart review and EMR flag included 1,354 patients, or 19.0% of our cohort. Our EMR flag for HOT in the last two seasons of the study had a positive predictive value of 100% (5 of 5 identified by EMR flag as receiving HOT were confirmed by chart review) and negative predictive value of 89% (31 of 35 identified by EMR flag as not receiving HOT were confirmed by chart review). The specificity of the EMR flag was 100% (31 of 31 of those confirmed by chart review as not receiving HOT, who were correctly identified by EMR) and the sensitivity was 55% (5 of 9 of those confirmed by chart review as receiving HOT, who were correctly identified by EMR).

Primary and Secondary Outcomes

Trends in length of stay across winter seasons are presented in Figure 2. The OU-HOT protocol was associated with an immediate reduction of 30.6 hours in mean length of stay (95% CI, –37.1 to –24.2). The rate of change in length of stay postimplementation did not differ significantly from the rate of change preimplementation (change in slope –0.6 hours per season; 95% CI, –2.3 to 1.1 hours). The percentage of patients discharged within 24 hours of admission rose immediately after protocol implementation, by 23.8 absolute percentage points (95% CI, 11.7-28.8). Slopes of the preintervention and postintervention regression lines did not differ significantly (change in slope –0.1% per season; 95% CI, –1.4% to 1.1%). Immediate decreases in length of stay were accompanied by an immediate decrease in mean cost per episode of care (–$4,181; 95% CI, –$4,829 to –$3,533). Protocol implementation also was associated with a decreased slope in cost postimplementation (change in slope –$403 per season; 95% CI, –$543 to –$264). The total cost savings, estimated by the product of the average cost savings per episode of care and the number of bronchiolitis admissions included in the study after OU-HOT implementation, amounted to $21.1 million over the 9-year period, or $2.3 million per winter season.

Balancing Measures

We observed an immediate reduction in 7-day hospital revisits (–1.1% immediate change; 95% CI, –1.8% to –0.4%), but an increasing slope in revisits after implementation (change in slope 0.4% per season; 95% CI, 0.1%-0.8%) (Figure 3). Stratifying revisits into ED visits and readmissions revealed that the revisit findings reflected changes in ED return visits, for which there was an immediate reduction at the time of implementation (–1.0% immediate change; 95% CI, –1.6% to –0.4%), but an increasing slope postimplementation (change in slope 0.5% per season; 95% CI, 0.2-0.8). Neither an immediate intervention effect (0.0% immediate change; 95% CI, –0.5% to 0.4%) nor a change in slope (change in slope 0.0% per season; 95% CI, –0.1% to 0.1%) were observed for inpatient readmissions alone. The annual rate of bronchiolitis admissions to Primary Children’s Hospital per 10,000 children who reside in Utah decreased after implementation of the OU-HOT protocol (immediate intervention effect –6.2 admissions; 95% CI, –10.8 to –1.6; change in slope –1.8 admissions per season; 95% CI, –2.8 to –0.69).

Our OU-HOT protocol was associated with immediate improvements in care delivered to children hospitalized for bronchiolitis, including decreased length of stay and cost savings. These improvements in outcomes largely have been sustained over a 9-year period. The OU-HOT protocol also appears to be safe as evidenced by a stable rate of readmissions over the study period and only a small increase in revisits to EDs across Intermountain Healthcare facilities, which see most children in the catchment area. Our OU-HOT protocol represents a combination of two interventions: (1) the creation of an OU focused on discharge within 24 to 48 hours of admission and (2) encouragement to discharge children with HOT. We found that use of the OU and a commitment to timely discharges has been sustained in recent years, while the commitment to HOT has appeared to wane.

Earlier investigations have evaluated the efficacy of HOT in the ED setting to prevent hospital admissions, finding high levels of caregiver comfort, estimating $1,300 per patient cost savings, and reporting readmission rates of approximately 5%.^16,23-25 Our study is unique in addressing HOT among a population of patients already hospitalized with bronchiolitis. The cost reductions we observed with our OU-HOT protocol were similar to those noted in the ED-based HOT protocols. However, we recorded lower readmission rates, likely because of the additional time allotted to caregivers to better gauge illness trajectory in the inpatient setting vs the ED, as well as additional time for hospitalized patients to reach the plateau or convalescent phase of illness. The small increase in ED revisits that we measured in recent years might be related to the concurrent rise in patient acuity and complexity.

Considering that length of stay has remained low despite less commitment to HOT, our results suggest that the OU might be the more impactful of the two interventions, and these data support the use of such a unit for a subset of patients with bronchiolitis. However, it is important to note that while the EMR HOT flag demonstrated high specificity, positive predictive value, and negative predictive value, the sensitivity was low (56%). As a result, it is possible that we have underestimated HOT use in the 2017-2018 and 2018-2019 seasons, the final two years of the study. Alternatively, the discrepancy between sustained outcomes and lagging use of HOT could be explained by improved identification of patients who would experience the greatest benefit with oxygen in terms of length of stay reductions, with fewer patients discharged on HOT but greater per-patient benefit. Finally, in an era that encourages reduced monitor use and less aggressive response to transient mild desaturations,^13,26,27 it is possible that fewer patients are identified with clinically actionable hypoxemia around the time they would be otherwise discharged.

Our OU-HOT model is not unprecedented. Increasingly, other formerly inpatient indications are being successfully managed in the observation, outpatient, and home setting, such as parenteral antibiotic treatment^28,29 and chemotherapy administration.³⁰ Considering the inpatient burden of bronchiolitis, similar strategies to expedite discharge are needed. Although outpatient intravenous antibiotic and chemotherapy administration have been widely adopted, we are aware of only one other pediatric health care system in the United States (Children’s Hospital Colorado) that routinely discharges inpatients with bronchiolitis on HOT.

This study has several limitations. First, although the interrupted time-series analysis is designed to account for trends that precede an intervention and covariates that differ before and after the intervention, it is possible that important unmeasured patient factors or changes in practice patterns differed between the pre- and post-intervention cohorts. There were no major changes to the OU-HOT protocol or discharge criteria after implementation, but individual practice management of bronchiolitis during the study period likely has evolved as new evidence emerges. Second, one could postulate that the increase in discharges within 24 hours and accompanying decreases in average length of stay and cost could be achieved by hospitalizing healthier patients over time, which the presence of an OU might incentivize. To the contrary, we found that population-based bronchiolitis admission rates have declined and disease severity appears to be increased since implementation of the OU-HOT protocol. The increase in medically complex children and PICU use in our postimplementation cohort aligns with recently published data suggesting these are national trends.^3,31 Third, HOT use was estimated from a sample of the cohort using a chart review and a newly available EMR flag. A low sensitivity and a small sample for the positive predictive value are limitations of the EMR flag.

Additionally, there are almost certainly unmeasured ambulatory burdens of HOT not captured by this study. ED-based protocols have estimated that patients discharged with HOT have a median of two follow-up ambulatory visits before oxygen is discontinued³²; however, the ambulatory burden associated with discharge on HOT after a hospitalization and the extent to which demographic factors affect that burden is unknown. Furthermore, one insurance company charged $94 for a month of HOT in 2019; paying even a portion of this charge represents a nontrivial financial burden for many families, even considering inpatient cost savings. Although the decision to discharge on oxygen or remain hospitalized until the child did not need oxygen was left to the parents, their posthospitalization perspectives were not assessed in this study. Although reports indicate that families largely feel positive about HOT after discharge from an ED setting, with 90% of caregivers preferring HOT use to inpatient admission and most reporting no difficulty with home management,²³ it is uncertain whether this would also apply after inpatient hospitalization.

The OU-HOT bronchiolitis protocol was associated with decreases in inpatient length of stay and cost while appearing safe to implement. The sustained use of the OU combined with declining use of HOT suggests that the OU might be the more impactful intervention. As previously inpatient indications such as parenteral antibiotics and chemotherapy increasingly have been administered in observation and outpatient settings, bronchiolitis appears ideal for a similar strategy that allows patients to spend less time in the hospital. Studies are needed to understand the outpatient burden of HOT and the generalizability of our findings.

Bronchiolitis is the leading cause of hospitalization in infants aged <1 year in the United States.^1-3 Estimates suggest that 1.5% to 2.0% of US infants require hospitalization every year, with a median (interquartile range) length of stay of 2 days (1-4),³ incurring direct medical costs of $555 million annually.¹ Evidence suggests that few interventions, aside from supportive care, are effective for bronchiolitis.^4-7 Adherence to standardized clinical guidelines could improve outcomes and resource use by streamlining care and limiting ineffective interventions, thereby decreasing hospital length of stay, which is a major medical cost.^8-13 For this reason, many hospitals have adopted bronchiolitis guidelines, although institutional practices vary.^14,15

Two relatively unexplored methods to reduce the inpatient burden of bronchiolitis are the use of observation units (OU) and home oxygen therapy (HOT). Motivated by research demonstrating the safety and effectiveness of an emergency department (ED)–based HOT protocol,¹⁶ where 36 of 37 patients with mild hypoxemia discharged on HOT avoided hospital admission, our institution implemented an observation unit and home oxygen therapy (OU-HOT) protocol designed to return children with bronchiolitis home earlier from the hospital. In the first winter season of implementation (2010 to 2011), the OU-HOT protocol was associated with significant reductions in length of stay and substantial cost savings, without an increase in return visits to the ED or inpatient readmissions.¹⁷ The objectives of this study were to determine whether these encouraging initial findings persisted and to measure the long-term impact of the OU-HOT protocol.

We conducted a retrospective cohort study of children hospitalized with bronchiolitis at Primary Children’s Hospital, a freestanding children’s hospital in Salt Lake City, Utah. Discharge diagnosis and procedures codes, as well as laboratory, imaging, pharmacy, and supply costs, were obtained from the Intermountain Healthcare enterprise data warehouse. A crosswalk available from the Centers for Medicare and Medicaid Services was used to convert International Classification of Diseases (ICD)-10 discharge diagnosis and procedure codes to ICD-9 equivalents.¹⁸ This study was approved by the University of Utah institutional review board (00110419).

Patients

Children aged 3 to 24 months who were discharged with a diagnosis of bronchiolitis (466.xx) during winter seasons from 2007 to 2019 were included. A winter season was defined as November 1 to April 30. Both observation and inpatient encounters were included in the cohort. We excluded patients with discharge diagnosis or procedure codes indicating tracheostomy (519.0-519.09, V44.0, V55.0, 31.1, 31.21, 31.41, 31.74, 97.23), ventilator dependence (V46.1x), chronic lung disease (518.83, 770.7), or pulmonary hypertension (416.xx). Patients with both bronchiolitis and a concurrent diagnosis, such as otitis media or pneumonia, were included unless exclusion criteria were met.

Intervention and Process Measures

Our institution implemented the OU-HOT protocol at the start of the 2010-2011 winter season.¹⁷ The aim of the OU-HOT protocol was to discharge children with bronchiolitis home sooner by increasing use of both an OU, with frequent assessment of discharge readiness, and HOT to help children become ready for discharge. Similar to most OUs, admission to our unit was limited to patients who met hospital admission criteria, and had a short anticipated length of stay (<48 hours). As a self-contained 20-bed unit providing 24-hour dedicated pediatrician/pediatric emergency medicine physician and nursing coverage, the OU actively monitored patients’ discharge readiness, with a goal to facilitate patient throughput more akin to an ED rather than a traditional inpatient unit. Patients who could not be discharged from the OU within 48 hours were transferred to the inpatient unit. Although the OU existed at the time of protocol implementation, its use for patients with bronchiolitis was not actively encouraged until implementation.

Hospitalized patients—in either inpatient or observation units—were eligible for discharge on HOT if they met the following criteria: hypoxemia was the only indication for continued hospitalization, the child’s oxygen requirement was <0.5 L/min for at least 6 hours (0.8 L/min for children aged >1 year), the child’s caregiver(s) were willing to manage oxygen at home, and the child had reliable access to primary care provider follow up. We used two process measures across winter seasons: (1) the percentage of patients discharged from the OU, and (2) the percentage of patients discharged with HOT. The percentage of patients discharged on HOT was estimated by a manual chart review and an electronic medical record (EMR) HOT flag that came into existence with our hospital system’s adoption of a new EMR (2017-2019). Chart review randomly sampled patients from 2007-2017, totaling 457 patients. To estimate the reliability of this method, we calculated the sensitivity, specificity, positive predictive value, and negative predictive value of the EMR HOT flag using chart review as the gold standard.

Outcome Measures

The main outcome measure was mean hospital length of stay. Balancing measures were revisit rates (stratified into ED visits and readmissions) and annual per-population bronchiolitis admission rates. Visits were considered revisits if they occurred within 7 days of initial hospital discharge, and included visits to Primary Children’s Hospital as well as 22 other Intermountain Healthcare hospitals. Population estimates from the Utah Department of Health were used to calculate the annual population-based rate of bronchiolitis admissions to Primary Children’s Hospital.¹⁹ Annual admission rates were calculated per 10,000 children aged 3 to 24 months who resided in Utah each year of the study period, and were evaluated to determine if patients were admitted more frequently after OU-HOT implementation. Secondary outcome measures included the percentage of patients discharged within 24 hours and mean inflation-adjusted cost per episode of care (in 2019 dollars). Hospitalization costs were determined using Intermountain Healthcare’s internal cost accounting system, an activity-based method that aggregates costs of individual resources according to date of service.²⁰ Costs were adjusted to 2019 dollars and were defined as the total costs of a patient’s initial hospitalization as well as any 7-day revisit encounters.

Data Analysis

Demographic data were compared before and after OU-HOT protocol implementation using Pearson chi-square tests. Multivariable linear or logistic regression models were used to compare measures before and after OU-HOT protocol implementation via an interrupted time-series approach. The interrupted time-series analysis measured two types of changes after protocol implementation during the 2010-2011 winter season: (1) any immediate change in the level of an outcome (immediate effect) and (2) any change of an outcome going forward over time (change in slope).²¹ Covariates in the regression models included patient age, sex, race, ethnicity, and insurance type, as well as presence of an underlying complex chronic condition, mechanical ventilation use, and pediatric intensive care unit (PICU) admission during hospitalization. Data were analyzed in STATA 15 (StataCorp LLC).²²

A total of 7,116 patients met inclusion criteria over the study period (2,061 pre-implementation, 5,055 post-implementation). A comparison of patient characteristics before and after HOT protocol implementation is presented in Table 1. Patients were similar in terms of age, sex, and insurance type. Patients in the postimplementation period were more likely to have a complex chronic condition, require admission to the PICU, and need mechanical ventilation (P < .01). Differences between cohorts with regard to race/ethnicity distribution largely were a result of improved capture of these data elements in the postimplementation period. For example, 30% of patients were classified as “race/ethnicity unknown” in the preimplementation cohort, compared with 4% of patients in the postimplementation period.

Process Measures

Figure 1 shows trends in OU and HOT use by winter season. The percentage of patients discharged from the OU increased immediately after OU-HOT protocol implementation (absolute 26.9% immediate increase; 95% CI, 21.9-42.2). The change in the proportion of OU use per season also increased (change in slope +3.9% per season; 95% CI, 3.4%-4.4%). The percentage of patients discharged with HOT increased immediately after OU-HOT protocol implementation (26.0% immediate change; 95% CI, 18.9%-33.1%); however, the immediate increase in HOT discharges was coupled with a declining rate of HOT discharges per season in the postprotocol period compared with the preprotocol period (change in slope –4.5% per season; 95% CI, –7.5% to –1.5%). Our chart review and EMR flag included 1,354 patients, or 19.0% of our cohort. Our EMR flag for HOT in the last two seasons of the study had a positive predictive value of 100% (5 of 5 identified by EMR flag as receiving HOT were confirmed by chart review) and negative predictive value of 89% (31 of 35 identified by EMR flag as not receiving HOT were confirmed by chart review). The specificity of the EMR flag was 100% (31 of 31 of those confirmed by chart review as not receiving HOT, who were correctly identified by EMR) and the sensitivity was 55% (5 of 9 of those confirmed by chart review as receiving HOT, who were correctly identified by EMR).

Primary and Secondary Outcomes

Trends in length of stay across winter seasons are presented in Figure 2. The OU-HOT protocol was associated with an immediate reduction of 30.6 hours in mean length of stay (95% CI, –37.1 to –24.2). The rate of change in length of stay postimplementation did not differ significantly from the rate of change preimplementation (change in slope –0.6 hours per season; 95% CI, –2.3 to 1.1 hours). The percentage of patients discharged within 24 hours of admission rose immediately after protocol implementation, by 23.8 absolute percentage points (95% CI, 11.7-28.8). Slopes of the preintervention and postintervention regression lines did not differ significantly (change in slope –0.1% per season; 95% CI, –1.4% to 1.1%). Immediate decreases in length of stay were accompanied by an immediate decrease in mean cost per episode of care (–$4,181; 95% CI, –$4,829 to –$3,533). Protocol implementation also was associated with a decreased slope in cost postimplementation (change in slope –$403 per season; 95% CI, –$543 to –$264). The total cost savings, estimated by the product of the average cost savings per episode of care and the number of bronchiolitis admissions included in the study after OU-HOT implementation, amounted to $21.1 million over the 9-year period, or $2.3 million per winter season.

Balancing Measures

We observed an immediate reduction in 7-day hospital revisits (–1.1% immediate change; 95% CI, –1.8% to –0.4%), but an increasing slope in revisits after implementation (change in slope 0.4% per season; 95% CI, 0.1%-0.8%) (Figure 3). Stratifying revisits into ED visits and readmissions revealed that the revisit findings reflected changes in ED return visits, for which there was an immediate reduction at the time of implementation (–1.0% immediate change; 95% CI, –1.6% to –0.4%), but an increasing slope postimplementation (change in slope 0.5% per season; 95% CI, 0.2-0.8). Neither an immediate intervention effect (0.0% immediate change; 95% CI, –0.5% to 0.4%) nor a change in slope (change in slope 0.0% per season; 95% CI, –0.1% to 0.1%) were observed for inpatient readmissions alone. The annual rate of bronchiolitis admissions to Primary Children’s Hospital per 10,000 children who reside in Utah decreased after implementation of the OU-HOT protocol (immediate intervention effect –6.2 admissions; 95% CI, –10.8 to –1.6; change in slope –1.8 admissions per season; 95% CI, –2.8 to –0.69).

Our OU-HOT protocol was associated with immediate improvements in care delivered to children hospitalized for bronchiolitis, including decreased length of stay and cost savings. These improvements in outcomes largely have been sustained over a 9-year period. The OU-HOT protocol also appears to be safe as evidenced by a stable rate of readmissions over the study period and only a small increase in revisits to EDs across Intermountain Healthcare facilities, which see most children in the catchment area. Our OU-HOT protocol represents a combination of two interventions: (1) the creation of an OU focused on discharge within 24 to 48 hours of admission and (2) encouragement to discharge children with HOT. We found that use of the OU and a commitment to timely discharges has been sustained in recent years, while the commitment to HOT has appeared to wane.

Earlier investigations have evaluated the efficacy of HOT in the ED setting to prevent hospital admissions, finding high levels of caregiver comfort, estimating $1,300 per patient cost savings, and reporting readmission rates of approximately 5%.^16,23-25 Our study is unique in addressing HOT among a population of patients already hospitalized with bronchiolitis. The cost reductions we observed with our OU-HOT protocol were similar to those noted in the ED-based HOT protocols. However, we recorded lower readmission rates, likely because of the additional time allotted to caregivers to better gauge illness trajectory in the inpatient setting vs the ED, as well as additional time for hospitalized patients to reach the plateau or convalescent phase of illness. The small increase in ED revisits that we measured in recent years might be related to the concurrent rise in patient acuity and complexity.

Considering that length of stay has remained low despite less commitment to HOT, our results suggest that the OU might be the more impactful of the two interventions, and these data support the use of such a unit for a subset of patients with bronchiolitis. However, it is important to note that while the EMR HOT flag demonstrated high specificity, positive predictive value, and negative predictive value, the sensitivity was low (56%). As a result, it is possible that we have underestimated HOT use in the 2017-2018 and 2018-2019 seasons, the final two years of the study. Alternatively, the discrepancy between sustained outcomes and lagging use of HOT could be explained by improved identification of patients who would experience the greatest benefit with oxygen in terms of length of stay reductions, with fewer patients discharged on HOT but greater per-patient benefit. Finally, in an era that encourages reduced monitor use and less aggressive response to transient mild desaturations,^13,26,27 it is possible that fewer patients are identified with clinically actionable hypoxemia around the time they would be otherwise discharged.

Our OU-HOT model is not unprecedented. Increasingly, other formerly inpatient indications are being successfully managed in the observation, outpatient, and home setting, such as parenteral antibiotic treatment^28,29 and chemotherapy administration.³⁰ Considering the inpatient burden of bronchiolitis, similar strategies to expedite discharge are needed. Although outpatient intravenous antibiotic and chemotherapy administration have been widely adopted, we are aware of only one other pediatric health care system in the United States (Children’s Hospital Colorado) that routinely discharges inpatients with bronchiolitis on HOT.

This study has several limitations. First, although the interrupted time-series analysis is designed to account for trends that precede an intervention and covariates that differ before and after the intervention, it is possible that important unmeasured patient factors or changes in practice patterns differed between the pre- and post-intervention cohorts. There were no major changes to the OU-HOT protocol or discharge criteria after implementation, but individual practice management of bronchiolitis during the study period likely has evolved as new evidence emerges. Second, one could postulate that the increase in discharges within 24 hours and accompanying decreases in average length of stay and cost could be achieved by hospitalizing healthier patients over time, which the presence of an OU might incentivize. To the contrary, we found that population-based bronchiolitis admission rates have declined and disease severity appears to be increased since implementation of the OU-HOT protocol. The increase in medically complex children and PICU use in our postimplementation cohort aligns with recently published data suggesting these are national trends.^3,31 Third, HOT use was estimated from a sample of the cohort using a chart review and a newly available EMR flag. A low sensitivity and a small sample for the positive predictive value are limitations of the EMR flag.

Additionally, there are almost certainly unmeasured ambulatory burdens of HOT not captured by this study. ED-based protocols have estimated that patients discharged with HOT have a median of two follow-up ambulatory visits before oxygen is discontinued³²; however, the ambulatory burden associated with discharge on HOT after a hospitalization and the extent to which demographic factors affect that burden is unknown. Furthermore, one insurance company charged $94 for a month of HOT in 2019; paying even a portion of this charge represents a nontrivial financial burden for many families, even considering inpatient cost savings. Although the decision to discharge on oxygen or remain hospitalized until the child did not need oxygen was left to the parents, their posthospitalization perspectives were not assessed in this study. Although reports indicate that families largely feel positive about HOT after discharge from an ED setting, with 90% of caregivers preferring HOT use to inpatient admission and most reporting no difficulty with home management,²³ it is uncertain whether this would also apply after inpatient hospitalization.

The OU-HOT bronchiolitis protocol was associated with decreases in inpatient length of stay and cost while appearing safe to implement. The sustained use of the OU combined with declining use of HOT suggests that the OU might be the more impactful intervention. As previously inpatient indications such as parenteral antibiotics and chemotherapy increasingly have been administered in observation and outpatient settings, bronchiolitis appears ideal for a similar strategy that allows patients to spend less time in the hospital. Studies are needed to understand the outpatient burden of HOT and the generalizability of our findings.

Across the country, residents are bound to a set of rules from the Accreditation Council for Graduate Medical Education (ACGME) designed to mini mize fatigue, maintain quality of life, and reduce fatigue-related patient safety events. Adherence to work hours regulations is required to maintain accreditation. Among other guidelines, residents are required to work fewer than 80 hours per week on average over 4 consecutive weeks.¹ When work hour violations occur, programs risk citation, penalties, and harm to the program’s reputation.

Residents self-report their adherence to program regulations in an annual survey conducted by the ACGME.² To collect more frequent data, most training programs monitor resident work hours through self-report on an electronic tracking platform.³ These data generally are used internally to identify problems and opportunities for improvement. However, self-report approaches are subject to imperfect recall and incomplete reporting, and require time and effort to complete.⁴

The widespread adoption of electronic health records (EHRs) brings new opportunity to measure and promote adherence to work hours. EHR log data capture when users log in and out of the system, along with their location and specific actions. These data offer a compelling alternative to self-report because they are already being collected and can be analyzed almost immediately. Recent studies using EHR log data to approximate resident work hours in a pediatric hospital successfully approximated scheduled hours, but the approach was customized to their hospital’s workflows and might not generalize to other settings.⁵ Furthermore, earlier studies have not captured evening out-of-hospital work, which contributes to total work hours and is associated with physician burnout.⁶

We developed a computational method that sought to accurately capture work hours, including out-of-hospital work, which could be used as a screening tool to identify at-risk residents and rotations in near real-time. We estimated work hours, including EHR and non-EHR work, from these EHR data and compared these daily estimations to self-report. We then used a heuristic to estimate the frequency of exceeding the 80-hour workweek in a large internal medicine residency program.

The population included 82 internal medicine interns (PGY-1) and 121 residents (PGY-2 = 60, PGY-3 = 61) who rotated through University of California, San Francisco Medical Center (UCSFMC) between July 1, 2018, and June 30, 2019, on inpatient rotations. In the UCSF internal medicine residency program, interns spend an average of 5 months per year and residents spend an average of 2 months per year on inpatient rotations at UCSFMC. Scheduled inpatient rotations generally are in 1-month blocks and include general medical wards, cardiology, liver transplant, night-float, and a procedures and jeopardy rotation where interns perform procedures at UCSFMC and serve as backup for their colleagues across sites. Although expected shift duration differs by rotation, types of shifts include regular length days, call days that are not overnight (but expected duration of work is into the late evening), 28-hour overnight call (PGY-2 and PGY-3), and night-float.

Data Source

This computational method was developed at UCSFMC. This study was approved by the University of California, San Francisco institutional review board. Using the UCSF Epic Clarity database, EHR access log data were obtained, including all Epic logins/logoffs, times, and access devices. Access devices identified included medical center computers, personal computers, and mobile devices.

Trainees self-report their work hours in MedHub, a widely used electronic tracking platform for self-report of resident work hours.⁷ Data were extracted from this database for interns and residents who matched the criteria above. The self-report data were considered the gold standard for comparison, because it is the best available despite its known limitations.

We used data collected from UCSF’s physician scheduling platform, AMiON, to identify interns and residents assigned to rotations at UCSF hospitals.⁸ AMiON also was used to capture half-days of off-site scheduled clinics and teaching, which count toward the workday but would not be associated with on-campus logins.

Developing a Computational Method to Measure Work Hours

We developed a heuristic to accomplish two goals: (1) infer the duration of continuous in-hospital work hours while providing clinical care and (2) measure “out-of-hospital” work. Logins from medical center computers were considered to be “on-campus” work. Logins from personal computers were considered to be “out-of-hospital.” “Out-of-hospital” login sessions were further subdivided into “out-of-hospital work” and “out-of-hospital study” based on activity during the session; if any work activities listed in Appendix Table 1 were performed, the session was attributed to work. If only chart review was performed, the session was attributed to study and did not count towards total hours worked. Logins from mobile devices also did not count towards total hours worked.

We inferred continuous in-hospital work by linking on-campus EHR sessions from the first on-campus login until the last on-campus logoff (Figure 1).

If there was overlapping time measurement between on-campus work and personal computer logins (for example, a resident was inferred to be doing on-campus work based on frequent medical center computer logins but there were also logins from personal computers), we inferred this to indicate that a personal device had been brought on-campus and the time was only attributed to on-campus work and was not double counted as out-of-hospital work. Out-of-hospital work that did not overlap with inferred on-campus work time contributed to the total hours worked in a week, consistent with ACGME guidelines.

Our internal medicine residents work at three hospitals: UCSFMC and two affiliated teaching hospitals. Although this study measured work hours while the residents were on an inpatient rotation at UCSFMC, trainees also might have occasional half-day clinics or teaching activities at other sites not captured by these EHR log data. The allocated time for that scheduled activity (extracted from AMiON) was counted as work hours. If the trainee was assigned to a morning half-day of off-site work (eg, didactics), this was counted the same as an 8 am to noon on-campus EHR session. If a trainee was assigned an afternoon half-day of off-site work (eg, a non-UCSF clinic), this was counted the same as a 1 pm to 5 pm on-campus EHR session. Counting this scheduled time as an on-campus EHR session allowed half-days of off-site work to be linked with inferred in-hospital work.

Comparison of EHR-Derived Work Hours Heuristic to Self-Report

Because resident adherence with daily self-report is imperfect, we compared EHR-derived work to self-report on days when both were available. We generated scatter plots of EHR-derived work hours compared with self-report and calculated the mean absolute error of estimation. We fit a linear mixed-effect model for each PGY, modeling self-reported hours as a linear function of estimated hours (fixed effect) with a random intercept (random effect) for each trainee to account for variations among individuals. StatsModels, version 0.11.1, was used for statistical analyses.⁹

We reviewed detailed data from outlier clusters to understand situations where the heuristic might not perform optimally. To assess whether EHR-derived work hours reasonably overlapped with expected shifts, 20 8-day blocks from separate interns and residents were randomly selected for qualitative detail review in comparison with AMiON schedule data.

Estimating Hours Worked and Work Hours Violations

After validating against self-report on a daily basis, we used our heuristic to infer the average rate at which the 80-hour workweek was exceeded across all inpatient rotations at UCSFMC. This was determined both including “out-of-hospital” work as derived from logins on personal computers and excluding it. Using the estimated daily hours worked, we built a near real-time dashboard to assist program leadership with identifying at-risk trainees and trends across the program.

Data from 82 interns (PGY-1) and 121 internal medicine residents (PGY-2 and PGY-3) who rotated at UCSFMC between July 1, 2018, and June 30, 2019, were included in the study. Table 1 shows the number of days and rotations worked at UCSFMC as well as the frequency of self-report of work hours according to program year.

Qualitative review of EHR-derived data compared with schedule data showed that, although residents often reported homogenous daily work hours, EHR-derived work hours often varied as expected on a day-to-day basis according to the schedule (Appendix Table 2).

Because out-of-hospital EHR use does not count as work if done for educational purposes, we evaluated the proportion of out-of-hospital EHR use that is considered work and found that 67% of PGY-1, 50% of PGY-2, and 53% of PGY-3 out-of-hospital sessions included at least one work activity, as denoted in Appendix Table 1. Out-of-hospital work therefore represented 85% of PGY-1, 66% of PGY-2, and 73% of PGY-3 time spent in the EHR out-of-hospital. These sessions were counted towards work hours in accordance with ACGME rules and included 29% of PGY-1 workdays and 21% of PGY-2 and PGY-3 workdays. This amounted to a median of 1.0 hours per day (95% CI, 0.1-4.6 hours) of out-of-hospital work for PGY-1, 0.9 hours per day (95% CI, 0.1-4.1 hours) for PGY-2, and 0.8 hours per day (95% CI, 0.1-4.7 hours) for PGY-3 residents. Out-of-hospital logins that did not include work activities, as denoted in Appendix Table 1, were labeled out-of-hospital study and did not count towards work hours; this amounted to a median of 0.3 hours per day (95% CI, 0.02-1.6 hours) for PGY-1, 0.5 hours per day (95% CI, 0.04-0.25 hours) for PGY-2, and 0.3 hours per day (95% CI, 0.03-1.7 hours) for PGY-3. Mobile device logins also were not counted towards total work hours, with a median of 3 minutes per day for PGY-1, 6 minutes per day for PGY-2, and 5 minutes per day for PGY-3.

The percentage of rotation months where average hours worked exceeded 80 hours weekly is shown in Table 2. Inclusion of out-of-hospital work hours substantially increased the frequency at which the 80-hour workweek was exceeded. The frequency of individual residents working more than 80 hours weekly on average is shown in Appendix Figure 3. A narrow majority of PGY-1 and PGY-2 trainees and a larger majority of PGY-3 trainees never worked in excess of 80 hours per week when averaged over the course of a rotation, but several trainees did on several occasions.

Estimations from the computational method were built into a dashboard for use as screening tool by residency program directors (Appendix Figure 4).

EHR log data can be used to automate measurement of trainee work hours, providing timely data to program directors for identifying residents at risk of exceeding work hours limits. We demonstrated this by developing a data-driven approach to link on-campus logins that can be replicated in other training programs. We further demonstrated that out-of-hospital work substantially contributed to resident work hours and the frequency with which they exceed the 80-hour workweek, making it a critical component of any work hour estimation approach. Inclusive of out-of-hospital work, our computational method found that residents exceeded the 80-hour workweek 10% to 21% of the time, depending on their year in residency, with a small majority of residents never exceeding the 80-hour workweek.

Historically, most ACGME residency programs have relied on resident self-report to determine work hours.³ The validity of this method has been extensively studied and results remain mixed; in some surveys, residents admit to underreporting their hours while other validation studies, including the use of clock-in and clock-out or time-stamped parking data, align with self-report relatively well.^10-12 Regardless of the reliability of self-report, it is a cumbersome task that residents have difficulty adhering to, as shown in our study, where only slightly more than one-half of the days worked had associated self-report. By relying on resident self-report, we are adding to the burden of clerical work, which is associated with physician burnout.¹³ Furthermore, because self-report typically does not happen in real-time, it limits a program’s ability to intervene on recent or impending work-hour violations. Our computational method enabled us to build a dashboard that is updated daily and provides critical insight into resident work hours at any time, without waiting for retrospective self-report.

Our study builds on previous work by Dziorny et al using EHR log data to algorithmically measure in-hospital work.⁵ In their study, the authors isolated shifts with a login gap of 4 hours and then combined shifts according to a set of heuristics. However, their logic integrated an extensive workflow analysis of trainee shifts, which might limit generalizability.⁵ Our approach computationally derives the temporal threshold for linking EHR sessions, which in our data was 5 hours but might differ at other sites. Automated derivation of this threshold will support generalizability to other programs and sites, although programs will still need to manually account for off-site work such as didactics. In a subsequent study evaluating the 80-hour workweek, Dziorny et al evaluated shift duration and appropriate time-off between shifts and found systematic underreporting of work.¹⁴ In our study, we prioritized evaluation of the 80-hour workweek and found general alignment between self-report and EHR-derived work-hour estimates, with a tendency to underestimate at lower reported work hours and overestimate at higher reported work hours (potentially because of underreporting as illustrated by Dziorny et al). We included the important out-of-hospital logins as discrete work events because out-of-hospital work contributes to the total hours worked and to the number of workweeks that exceed the 80-hour workweek, and might contribute to burnout.¹⁵ The incidence of exceeding the 80-hour workweek increased by 7% to 8% across all residents when out-of-hospital work was included, demonstrating that tools such as ResQ (ResQ Medical) that rely primarily on geolocation data might not sufficiently capture the ways in which residents spend their time working.¹⁶

Our approach has limitations. We determined on-campus vs out-of-hospital locations based on whether the login device belonged to the medical center or was a personal computer. Consequently, if trainees exclusively used a personal computer while on-campus and never used a medical center computer, we would have captured this work done while logged into the EHR but would not have inferred on-campus work. Although nearly all trainees in our organization use medical center computers throughout the day, this might impact generalizability for programs where trainees use personal computers exclusively in the hospital. Our approach also assumes trainees will use the EHR at the beginning and end of their workdays, which could lead to underestimation of work hours in trainees who do not employ this practice. With regards to work done on personal computers, our heuristic required that at least one work activity (as denoted in Appendix Table 1) be included in the session in order for it to count as work. Although this approach allows us to exclude sessions where trainees might be reviewing charts exclusively for educational purposes, it is difficult to infer the true intent of chart review.

There might be periods of time where residents are doing in-hospital work but more than 5 hours elapsed between EHR user sessions. As we have started adapting this computational method for other residency programs, we have added logic that allows for long periods of time in the operating room to be considered part of a continuous workday. There also are limitations to assigning blocks of time to off-site clinics; clinics that are associated with after-hours work but use a different EHR would not be captured in total out-of-hospital work.

Although correlation with self-report was good, we identified clusters of inaccuracy. This likely resulted from our residency program covering three medical centers, two of which were not included in the data set. For example, if a resident had an off-site clinic that was not accounted for in AMiON, EHR-derived work hours might have been underestimated relative to self-report. Operationally leveraging an automated system for measuring work hours in the form of dashboards and other tools could provide the impetus to ensure accurate documentation of schedule anomalies.

Implementation of our EHR-derived work-hour model will allow ACGME residency programs to understand and act upon trainee work-hour violations closer to real time, as the data extraction is daily and automated. Automation will save busy residents a cumbersome task, provide more complete data than self-report, and empower residency programs to intervene quickly to support overworked trainees.

The authors thank Drs Bradley Monash, Larissa Thomas, and Rebecca Berman for providing residency program input.

Across the country, residents are bound to a set of rules from the Accreditation Council for Graduate Medical Education (ACGME) designed to mini mize fatigue, maintain quality of life, and reduce fatigue-related patient safety events. Adherence to work hours regulations is required to maintain accreditation. Among other guidelines, residents are required to work fewer than 80 hours per week on average over 4 consecutive weeks.¹ When work hour violations occur, programs risk citation, penalties, and harm to the program’s reputation.

Residents self-report their adherence to program regulations in an annual survey conducted by the ACGME.² To collect more frequent data, most training programs monitor resident work hours through self-report on an electronic tracking platform.³ These data generally are used internally to identify problems and opportunities for improvement. However, self-report approaches are subject to imperfect recall and incomplete reporting, and require time and effort to complete.⁴

The widespread adoption of electronic health records (EHRs) brings new opportunity to measure and promote adherence to work hours. EHR log data capture when users log in and out of the system, along with their location and specific actions. These data offer a compelling alternative to self-report because they are already being collected and can be analyzed almost immediately. Recent studies using EHR log data to approximate resident work hours in a pediatric hospital successfully approximated scheduled hours, but the approach was customized to their hospital’s workflows and might not generalize to other settings.⁵ Furthermore, earlier studies have not captured evening out-of-hospital work, which contributes to total work hours and is associated with physician burnout.⁶

We developed a computational method that sought to accurately capture work hours, including out-of-hospital work, which could be used as a screening tool to identify at-risk residents and rotations in near real-time. We estimated work hours, including EHR and non-EHR work, from these EHR data and compared these daily estimations to self-report. We then used a heuristic to estimate the frequency of exceeding the 80-hour workweek in a large internal medicine residency program.

The population included 82 internal medicine interns (PGY-1) and 121 residents (PGY-2 = 60, PGY-3 = 61) who rotated through University of California, San Francisco Medical Center (UCSFMC) between July 1, 2018, and June 30, 2019, on inpatient rotations. In the UCSF internal medicine residency program, interns spend an average of 5 months per year and residents spend an average of 2 months per year on inpatient rotations at UCSFMC. Scheduled inpatient rotations generally are in 1-month blocks and include general medical wards, cardiology, liver transplant, night-float, and a procedures and jeopardy rotation where interns perform procedures at UCSFMC and serve as backup for their colleagues across sites. Although expected shift duration differs by rotation, types of shifts include regular length days, call days that are not overnight (but expected duration of work is into the late evening), 28-hour overnight call (PGY-2 and PGY-3), and night-float.

Data Source

This computational method was developed at UCSFMC. This study was approved by the University of California, San Francisco institutional review board. Using the UCSF Epic Clarity database, EHR access log data were obtained, including all Epic logins/logoffs, times, and access devices. Access devices identified included medical center computers, personal computers, and mobile devices.

Trainees self-report their work hours in MedHub, a widely used electronic tracking platform for self-report of resident work hours.⁷ Data were extracted from this database for interns and residents who matched the criteria above. The self-report data were considered the gold standard for comparison, because it is the best available despite its known limitations.

We used data collected from UCSF’s physician scheduling platform, AMiON, to identify interns and residents assigned to rotations at UCSF hospitals.⁸ AMiON also was used to capture half-days of off-site scheduled clinics and teaching, which count toward the workday but would not be associated with on-campus logins.

Developing a Computational Method to Measure Work Hours

We developed a heuristic to accomplish two goals: (1) infer the duration of continuous in-hospital work hours while providing clinical care and (2) measure “out-of-hospital” work. Logins from medical center computers were considered to be “on-campus” work. Logins from personal computers were considered to be “out-of-hospital.” “Out-of-hospital” login sessions were further subdivided into “out-of-hospital work” and “out-of-hospital study” based on activity during the session; if any work activities listed in Appendix Table 1 were performed, the session was attributed to work. If only chart review was performed, the session was attributed to study and did not count towards total hours worked. Logins from mobile devices also did not count towards total hours worked.

We inferred continuous in-hospital work by linking on-campus EHR sessions from the first on-campus login until the last on-campus logoff (Figure 1).

If there was overlapping time measurement between on-campus work and personal computer logins (for example, a resident was inferred to be doing on-campus work based on frequent medical center computer logins but there were also logins from personal computers), we inferred this to indicate that a personal device had been brought on-campus and the time was only attributed to on-campus work and was not double counted as out-of-hospital work. Out-of-hospital work that did not overlap with inferred on-campus work time contributed to the total hours worked in a week, consistent with ACGME guidelines.

Our internal medicine residents work at three hospitals: UCSFMC and two affiliated teaching hospitals. Although this study measured work hours while the residents were on an inpatient rotation at UCSFMC, trainees also might have occasional half-day clinics or teaching activities at other sites not captured by these EHR log data. The allocated time for that scheduled activity (extracted from AMiON) was counted as work hours. If the trainee was assigned to a morning half-day of off-site work (eg, didactics), this was counted the same as an 8 am to noon on-campus EHR session. If a trainee was assigned an afternoon half-day of off-site work (eg, a non-UCSF clinic), this was counted the same as a 1 pm to 5 pm on-campus EHR session. Counting this scheduled time as an on-campus EHR session allowed half-days of off-site work to be linked with inferred in-hospital work.

Comparison of EHR-Derived Work Hours Heuristic to Self-Report

Because resident adherence with daily self-report is imperfect, we compared EHR-derived work to self-report on days when both were available. We generated scatter plots of EHR-derived work hours compared with self-report and calculated the mean absolute error of estimation. We fit a linear mixed-effect model for each PGY, modeling self-reported hours as a linear function of estimated hours (fixed effect) with a random intercept (random effect) for each trainee to account for variations among individuals. StatsModels, version 0.11.1, was used for statistical analyses.⁹

We reviewed detailed data from outlier clusters to understand situations where the heuristic might not perform optimally. To assess whether EHR-derived work hours reasonably overlapped with expected shifts, 20 8-day blocks from separate interns and residents were randomly selected for qualitative detail review in comparison with AMiON schedule data.

Estimating Hours Worked and Work Hours Violations

After validating against self-report on a daily basis, we used our heuristic to infer the average rate at which the 80-hour workweek was exceeded across all inpatient rotations at UCSFMC. This was determined both including “out-of-hospital” work as derived from logins on personal computers and excluding it. Using the estimated daily hours worked, we built a near real-time dashboard to assist program leadership with identifying at-risk trainees and trends across the program.

Data from 82 interns (PGY-1) and 121 internal medicine residents (PGY-2 and PGY-3) who rotated at UCSFMC between July 1, 2018, and June 30, 2019, were included in the study. Table 1 shows the number of days and rotations worked at UCSFMC as well as the frequency of self-report of work hours according to program year.

Qualitative review of EHR-derived data compared with schedule data showed that, although residents often reported homogenous daily work hours, EHR-derived work hours often varied as expected on a day-to-day basis according to the schedule (Appendix Table 2).

Because out-of-hospital EHR use does not count as work if done for educational purposes, we evaluated the proportion of out-of-hospital EHR use that is considered work and found that 67% of PGY-1, 50% of PGY-2, and 53% of PGY-3 out-of-hospital sessions included at least one work activity, as denoted in Appendix Table 1. Out-of-hospital work therefore represented 85% of PGY-1, 66% of PGY-2, and 73% of PGY-3 time spent in the EHR out-of-hospital. These sessions were counted towards work hours in accordance with ACGME rules and included 29% of PGY-1 workdays and 21% of PGY-2 and PGY-3 workdays. This amounted to a median of 1.0 hours per day (95% CI, 0.1-4.6 hours) of out-of-hospital work for PGY-1, 0.9 hours per day (95% CI, 0.1-4.1 hours) for PGY-2, and 0.8 hours per day (95% CI, 0.1-4.7 hours) for PGY-3 residents. Out-of-hospital logins that did not include work activities, as denoted in Appendix Table 1, were labeled out-of-hospital study and did not count towards work hours; this amounted to a median of 0.3 hours per day (95% CI, 0.02-1.6 hours) for PGY-1, 0.5 hours per day (95% CI, 0.04-0.25 hours) for PGY-2, and 0.3 hours per day (95% CI, 0.03-1.7 hours) for PGY-3. Mobile device logins also were not counted towards total work hours, with a median of 3 minutes per day for PGY-1, 6 minutes per day for PGY-2, and 5 minutes per day for PGY-3.

The percentage of rotation months where average hours worked exceeded 80 hours weekly is shown in Table 2. Inclusion of out-of-hospital work hours substantially increased the frequency at which the 80-hour workweek was exceeded. The frequency of individual residents working more than 80 hours weekly on average is shown in Appendix Figure 3. A narrow majority of PGY-1 and PGY-2 trainees and a larger majority of PGY-3 trainees never worked in excess of 80 hours per week when averaged over the course of a rotation, but several trainees did on several occasions.

Estimations from the computational method were built into a dashboard for use as screening tool by residency program directors (Appendix Figure 4).

EHR log data can be used to automate measurement of trainee work hours, providing timely data to program directors for identifying residents at risk of exceeding work hours limits. We demonstrated this by developing a data-driven approach to link on-campus logins that can be replicated in other training programs. We further demonstrated that out-of-hospital work substantially contributed to resident work hours and the frequency with which they exceed the 80-hour workweek, making it a critical component of any work hour estimation approach. Inclusive of out-of-hospital work, our computational method found that residents exceeded the 80-hour workweek 10% to 21% of the time, depending on their year in residency, with a small majority of residents never exceeding the 80-hour workweek.

Historically, most ACGME residency programs have relied on resident self-report to determine work hours.³ The validity of this method has been extensively studied and results remain mixed; in some surveys, residents admit to underreporting their hours while other validation studies, including the use of clock-in and clock-out or time-stamped parking data, align with self-report relatively well.^10-12 Regardless of the reliability of self-report, it is a cumbersome task that residents have difficulty adhering to, as shown in our study, where only slightly more than one-half of the days worked had associated self-report. By relying on resident self-report, we are adding to the burden of clerical work, which is associated with physician burnout.¹³ Furthermore, because self-report typically does not happen in real-time, it limits a program’s ability to intervene on recent or impending work-hour violations. Our computational method enabled us to build a dashboard that is updated daily and provides critical insight into resident work hours at any time, without waiting for retrospective self-report.

Our study builds on previous work by Dziorny et al using EHR log data to algorithmically measure in-hospital work.⁵ In their study, the authors isolated shifts with a login gap of 4 hours and then combined shifts according to a set of heuristics. However, their logic integrated an extensive workflow analysis of trainee shifts, which might limit generalizability.⁵ Our approach computationally derives the temporal threshold for linking EHR sessions, which in our data was 5 hours but might differ at other sites. Automated derivation of this threshold will support generalizability to other programs and sites, although programs will still need to manually account for off-site work such as didactics. In a subsequent study evaluating the 80-hour workweek, Dziorny et al evaluated shift duration and appropriate time-off between shifts and found systematic underreporting of work.¹⁴ In our study, we prioritized evaluation of the 80-hour workweek and found general alignment between self-report and EHR-derived work-hour estimates, with a tendency to underestimate at lower reported work hours and overestimate at higher reported work hours (potentially because of underreporting as illustrated by Dziorny et al). We included the important out-of-hospital logins as discrete work events because out-of-hospital work contributes to the total hours worked and to the number of workweeks that exceed the 80-hour workweek, and might contribute to burnout.¹⁵ The incidence of exceeding the 80-hour workweek increased by 7% to 8% across all residents when out-of-hospital work was included, demonstrating that tools such as ResQ (ResQ Medical) that rely primarily on geolocation data might not sufficiently capture the ways in which residents spend their time working.¹⁶

Our approach has limitations. We determined on-campus vs out-of-hospital locations based on whether the login device belonged to the medical center or was a personal computer. Consequently, if trainees exclusively used a personal computer while on-campus and never used a medical center computer, we would have captured this work done while logged into the EHR but would not have inferred on-campus work. Although nearly all trainees in our organization use medical center computers throughout the day, this might impact generalizability for programs where trainees use personal computers exclusively in the hospital. Our approach also assumes trainees will use the EHR at the beginning and end of their workdays, which could lead to underestimation of work hours in trainees who do not employ this practice. With regards to work done on personal computers, our heuristic required that at least one work activity (as denoted in Appendix Table 1) be included in the session in order for it to count as work. Although this approach allows us to exclude sessions where trainees might be reviewing charts exclusively for educational purposes, it is difficult to infer the true intent of chart review.

There might be periods of time where residents are doing in-hospital work but more than 5 hours elapsed between EHR user sessions. As we have started adapting this computational method for other residency programs, we have added logic that allows for long periods of time in the operating room to be considered part of a continuous workday. There also are limitations to assigning blocks of time to off-site clinics; clinics that are associated with after-hours work but use a different EHR would not be captured in total out-of-hospital work.

Although correlation with self-report was good, we identified clusters of inaccuracy. This likely resulted from our residency program covering three medical centers, two of which were not included in the data set. For example, if a resident had an off-site clinic that was not accounted for in AMiON, EHR-derived work hours might have been underestimated relative to self-report. Operationally leveraging an automated system for measuring work hours in the form of dashboards and other tools could provide the impetus to ensure accurate documentation of schedule anomalies.

Implementation of our EHR-derived work-hour model will allow ACGME residency programs to understand and act upon trainee work-hour violations closer to real time, as the data extraction is daily and automated. Automation will save busy residents a cumbersome task, provide more complete data than self-report, and empower residency programs to intervene quickly to support overworked trainees.

The authors thank Drs Bradley Monash, Larissa Thomas, and Rebecca Berman for providing residency program input.

Across the country, residents are bound to a set of rules from the Accreditation Council for Graduate Medical Education (ACGME) designed to mini mize fatigue, maintain quality of life, and reduce fatigue-related patient safety events. Adherence to work hours regulations is required to maintain accreditation. Among other guidelines, residents are required to work fewer than 80 hours per week on average over 4 consecutive weeks.¹ When work hour violations occur, programs risk citation, penalties, and harm to the program’s reputation.

Residents self-report their adherence to program regulations in an annual survey conducted by the ACGME.² To collect more frequent data, most training programs monitor resident work hours through self-report on an electronic tracking platform.³ These data generally are used internally to identify problems and opportunities for improvement. However, self-report approaches are subject to imperfect recall and incomplete reporting, and require time and effort to complete.⁴

The widespread adoption of electronic health records (EHRs) brings new opportunity to measure and promote adherence to work hours. EHR log data capture when users log in and out of the system, along with their location and specific actions. These data offer a compelling alternative to self-report because they are already being collected and can be analyzed almost immediately. Recent studies using EHR log data to approximate resident work hours in a pediatric hospital successfully approximated scheduled hours, but the approach was customized to their hospital’s workflows and might not generalize to other settings.⁵ Furthermore, earlier studies have not captured evening out-of-hospital work, which contributes to total work hours and is associated with physician burnout.⁶

We developed a computational method that sought to accurately capture work hours, including out-of-hospital work, which could be used as a screening tool to identify at-risk residents and rotations in near real-time. We estimated work hours, including EHR and non-EHR work, from these EHR data and compared these daily estimations to self-report. We then used a heuristic to estimate the frequency of exceeding the 80-hour workweek in a large internal medicine residency program.

The population included 82 internal medicine interns (PGY-1) and 121 residents (PGY-2 = 60, PGY-3 = 61) who rotated through University of California, San Francisco Medical Center (UCSFMC) between July 1, 2018, and June 30, 2019, on inpatient rotations. In the UCSF internal medicine residency program, interns spend an average of 5 months per year and residents spend an average of 2 months per year on inpatient rotations at UCSFMC. Scheduled inpatient rotations generally are in 1-month blocks and include general medical wards, cardiology, liver transplant, night-float, and a procedures and jeopardy rotation where interns perform procedures at UCSFMC and serve as backup for their colleagues across sites. Although expected shift duration differs by rotation, types of shifts include regular length days, call days that are not overnight (but expected duration of work is into the late evening), 28-hour overnight call (PGY-2 and PGY-3), and night-float.

Data Source

This computational method was developed at UCSFMC. This study was approved by the University of California, San Francisco institutional review board. Using the UCSF Epic Clarity database, EHR access log data were obtained, including all Epic logins/logoffs, times, and access devices. Access devices identified included medical center computers, personal computers, and mobile devices.

Trainees self-report their work hours in MedHub, a widely used electronic tracking platform for self-report of resident work hours.⁷ Data were extracted from this database for interns and residents who matched the criteria above. The self-report data were considered the gold standard for comparison, because it is the best available despite its known limitations.

We used data collected from UCSF’s physician scheduling platform, AMiON, to identify interns and residents assigned to rotations at UCSF hospitals.⁸ AMiON also was used to capture half-days of off-site scheduled clinics and teaching, which count toward the workday but would not be associated with on-campus logins.

Developing a Computational Method to Measure Work Hours

We developed a heuristic to accomplish two goals: (1) infer the duration of continuous in-hospital work hours while providing clinical care and (2) measure “out-of-hospital” work. Logins from medical center computers were considered to be “on-campus” work. Logins from personal computers were considered to be “out-of-hospital.” “Out-of-hospital” login sessions were further subdivided into “out-of-hospital work” and “out-of-hospital study” based on activity during the session; if any work activities listed in Appendix Table 1 were performed, the session was attributed to work. If only chart review was performed, the session was attributed to study and did not count towards total hours worked. Logins from mobile devices also did not count towards total hours worked.

We inferred continuous in-hospital work by linking on-campus EHR sessions from the first on-campus login until the last on-campus logoff (Figure 1).

If there was overlapping time measurement between on-campus work and personal computer logins (for example, a resident was inferred to be doing on-campus work based on frequent medical center computer logins but there were also logins from personal computers), we inferred this to indicate that a personal device had been brought on-campus and the time was only attributed to on-campus work and was not double counted as out-of-hospital work. Out-of-hospital work that did not overlap with inferred on-campus work time contributed to the total hours worked in a week, consistent with ACGME guidelines.

Our internal medicine residents work at three hospitals: UCSFMC and two affiliated teaching hospitals. Although this study measured work hours while the residents were on an inpatient rotation at UCSFMC, trainees also might have occasional half-day clinics or teaching activities at other sites not captured by these EHR log data. The allocated time for that scheduled activity (extracted from AMiON) was counted as work hours. If the trainee was assigned to a morning half-day of off-site work (eg, didactics), this was counted the same as an 8 am to noon on-campus EHR session. If a trainee was assigned an afternoon half-day of off-site work (eg, a non-UCSF clinic), this was counted the same as a 1 pm to 5 pm on-campus EHR session. Counting this scheduled time as an on-campus EHR session allowed half-days of off-site work to be linked with inferred in-hospital work.

Comparison of EHR-Derived Work Hours Heuristic to Self-Report

Because resident adherence with daily self-report is imperfect, we compared EHR-derived work to self-report on days when both were available. We generated scatter plots of EHR-derived work hours compared with self-report and calculated the mean absolute error of estimation. We fit a linear mixed-effect model for each PGY, modeling self-reported hours as a linear function of estimated hours (fixed effect) with a random intercept (random effect) for each trainee to account for variations among individuals. StatsModels, version 0.11.1, was used for statistical analyses.⁹

We reviewed detailed data from outlier clusters to understand situations where the heuristic might not perform optimally. To assess whether EHR-derived work hours reasonably overlapped with expected shifts, 20 8-day blocks from separate interns and residents were randomly selected for qualitative detail review in comparison with AMiON schedule data.

Estimating Hours Worked and Work Hours Violations

After validating against self-report on a daily basis, we used our heuristic to infer the average rate at which the 80-hour workweek was exceeded across all inpatient rotations at UCSFMC. This was determined both including “out-of-hospital” work as derived from logins on personal computers and excluding it. Using the estimated daily hours worked, we built a near real-time dashboard to assist program leadership with identifying at-risk trainees and trends across the program.

Data from 82 interns (PGY-1) and 121 internal medicine residents (PGY-2 and PGY-3) who rotated at UCSFMC between July 1, 2018, and June 30, 2019, were included in the study. Table 1 shows the number of days and rotations worked at UCSFMC as well as the frequency of self-report of work hours according to program year.

Qualitative review of EHR-derived data compared with schedule data showed that, although residents often reported homogenous daily work hours, EHR-derived work hours often varied as expected on a day-to-day basis according to the schedule (Appendix Table 2).

Because out-of-hospital EHR use does not count as work if done for educational purposes, we evaluated the proportion of out-of-hospital EHR use that is considered work and found that 67% of PGY-1, 50% of PGY-2, and 53% of PGY-3 out-of-hospital sessions included at least one work activity, as denoted in Appendix Table 1. Out-of-hospital work therefore represented 85% of PGY-1, 66% of PGY-2, and 73% of PGY-3 time spent in the EHR out-of-hospital. These sessions were counted towards work hours in accordance with ACGME rules and included 29% of PGY-1 workdays and 21% of PGY-2 and PGY-3 workdays. This amounted to a median of 1.0 hours per day (95% CI, 0.1-4.6 hours) of out-of-hospital work for PGY-1, 0.9 hours per day (95% CI, 0.1-4.1 hours) for PGY-2, and 0.8 hours per day (95% CI, 0.1-4.7 hours) for PGY-3 residents. Out-of-hospital logins that did not include work activities, as denoted in Appendix Table 1, were labeled out-of-hospital study and did not count towards work hours; this amounted to a median of 0.3 hours per day (95% CI, 0.02-1.6 hours) for PGY-1, 0.5 hours per day (95% CI, 0.04-0.25 hours) for PGY-2, and 0.3 hours per day (95% CI, 0.03-1.7 hours) for PGY-3. Mobile device logins also were not counted towards total work hours, with a median of 3 minutes per day for PGY-1, 6 minutes per day for PGY-2, and 5 minutes per day for PGY-3.

The percentage of rotation months where average hours worked exceeded 80 hours weekly is shown in Table 2. Inclusion of out-of-hospital work hours substantially increased the frequency at which the 80-hour workweek was exceeded. The frequency of individual residents working more than 80 hours weekly on average is shown in Appendix Figure 3. A narrow majority of PGY-1 and PGY-2 trainees and a larger majority of PGY-3 trainees never worked in excess of 80 hours per week when averaged over the course of a rotation, but several trainees did on several occasions.

Estimations from the computational method were built into a dashboard for use as screening tool by residency program directors (Appendix Figure 4).

EHR log data can be used to automate measurement of trainee work hours, providing timely data to program directors for identifying residents at risk of exceeding work hours limits. We demonstrated this by developing a data-driven approach to link on-campus logins that can be replicated in other training programs. We further demonstrated that out-of-hospital work substantially contributed to resident work hours and the frequency with which they exceed the 80-hour workweek, making it a critical component of any work hour estimation approach. Inclusive of out-of-hospital work, our computational method found that residents exceeded the 80-hour workweek 10% to 21% of the time, depending on their year in residency, with a small majority of residents never exceeding the 80-hour workweek.

Historically, most ACGME residency programs have relied on resident self-report to determine work hours.³ The validity of this method has been extensively studied and results remain mixed; in some surveys, residents admit to underreporting their hours while other validation studies, including the use of clock-in and clock-out or time-stamped parking data, align with self-report relatively well.^10-12 Regardless of the reliability of self-report, it is a cumbersome task that residents have difficulty adhering to, as shown in our study, where only slightly more than one-half of the days worked had associated self-report. By relying on resident self-report, we are adding to the burden of clerical work, which is associated with physician burnout.¹³ Furthermore, because self-report typically does not happen in real-time, it limits a program’s ability to intervene on recent or impending work-hour violations. Our computational method enabled us to build a dashboard that is updated daily and provides critical insight into resident work hours at any time, without waiting for retrospective self-report.

Our study builds on previous work by Dziorny et al using EHR log data to algorithmically measure in-hospital work.⁵ In their study, the authors isolated shifts with a login gap of 4 hours and then combined shifts according to a set of heuristics. However, their logic integrated an extensive workflow analysis of trainee shifts, which might limit generalizability.⁵ Our approach computationally derives the temporal threshold for linking EHR sessions, which in our data was 5 hours but might differ at other sites. Automated derivation of this threshold will support generalizability to other programs and sites, although programs will still need to manually account for off-site work such as didactics. In a subsequent study evaluating the 80-hour workweek, Dziorny et al evaluated shift duration and appropriate time-off between shifts and found systematic underreporting of work.¹⁴ In our study, we prioritized evaluation of the 80-hour workweek and found general alignment between self-report and EHR-derived work-hour estimates, with a tendency to underestimate at lower reported work hours and overestimate at higher reported work hours (potentially because of underreporting as illustrated by Dziorny et al). We included the important out-of-hospital logins as discrete work events because out-of-hospital work contributes to the total hours worked and to the number of workweeks that exceed the 80-hour workweek, and might contribute to burnout.¹⁵ The incidence of exceeding the 80-hour workweek increased by 7% to 8% across all residents when out-of-hospital work was included, demonstrating that tools such as ResQ (ResQ Medical) that rely primarily on geolocation data might not sufficiently capture the ways in which residents spend their time working.¹⁶

Our approach has limitations. We determined on-campus vs out-of-hospital locations based on whether the login device belonged to the medical center or was a personal computer. Consequently, if trainees exclusively used a personal computer while on-campus and never used a medical center computer, we would have captured this work done while logged into the EHR but would not have inferred on-campus work. Although nearly all trainees in our organization use medical center computers throughout the day, this might impact generalizability for programs where trainees use personal computers exclusively in the hospital. Our approach also assumes trainees will use the EHR at the beginning and end of their workdays, which could lead to underestimation of work hours in trainees who do not employ this practice. With regards to work done on personal computers, our heuristic required that at least one work activity (as denoted in Appendix Table 1) be included in the session in order for it to count as work. Although this approach allows us to exclude sessions where trainees might be reviewing charts exclusively for educational purposes, it is difficult to infer the true intent of chart review.

There might be periods of time where residents are doing in-hospital work but more than 5 hours elapsed between EHR user sessions. As we have started adapting this computational method for other residency programs, we have added logic that allows for long periods of time in the operating room to be considered part of a continuous workday. There also are limitations to assigning blocks of time to off-site clinics; clinics that are associated with after-hours work but use a different EHR would not be captured in total out-of-hospital work.

Although correlation with self-report was good, we identified clusters of inaccuracy. This likely resulted from our residency program covering three medical centers, two of which were not included in the data set. For example, if a resident had an off-site clinic that was not accounted for in AMiON, EHR-derived work hours might have been underestimated relative to self-report. Operationally leveraging an automated system for measuring work hours in the form of dashboards and other tools could provide the impetus to ensure accurate documentation of schedule anomalies.

Implementation of our EHR-derived work-hour model will allow ACGME residency programs to understand and act upon trainee work-hour violations closer to real time, as the data extraction is daily and automated. Automation will save busy residents a cumbersome task, provide more complete data than self-report, and empower residency programs to intervene quickly to support overworked trainees.

The authors thank Drs Bradley Monash, Larissa Thomas, and Rebecca Berman for providing residency program input.

Veterans have a higher prevalence of type 2 diabetes mellitus (T2DM) when compared with their civilian counterparts with an overall prevalence rate of 25%.¹ This higher prevalence is similar to other major chronic health conditions, including heart disease and arthritis, with additional costs for disease self-management.² Psychological and behavioral change strategies are a principal means of limiting the severity and even restoring function once T2DM is diagnosed.³ More broadly, there is mounting evidence that addressing distress and behavior change are important across many conditions, particularly T2DM.⁴ Therefore, the US Department of Veterans Affairs (VA) has established patient education and multidisciplinary interventions to optimize engagement in T2DM self-management and health behavior change.⁵

Traditional T2DM education programs aim to meet the American Diabetes Association (ADA) standards of medical care and include a T2DM educator and other allied health professionals. ADA Standard 1.2 emphasizes “productive interactions between a prepared, proactive care team and an informed, activated patient.”⁶ Thus, to attain ADA accreditation, educational programs require instructors to teach about T2DM while engaging patients to help them set and achieve recommended changes. The requirements emphasize setting specific goals, (ie, eating wisely, being physically active, monitoring blood sugars or taking medications). The care team also helps to identify barriers, and at a required follow-up class, patients evaluate how well they met goals and make modifications if needed. The impact of traditional patient education programs to improve glycemic levels is well established.⁷ Importantly, veterans with comorbid mental health conditions may not experience the same beneficial outcomes if or when they participate in traditional diabetes or self-management programs.^8,9 Veterans with T2DM may be particularly vulnerable to chronic stress and effects of comorbid mental health diagnoses.¹⁰ Furthermore, when individuals experience T2DM-related distress, associations with poor health outcomes, including elevated hemoglobin A_1c (HbA_1c), are observed independent of depression.¹¹

Health psychology services integrate into medical settings and strive to reach veterans who may not engage in traditional mental health clinical offerings.¹² These collaborative interventions focus less on diagnostic or screening procedures and more on a patient’s understanding of illness and ability and willingness to carry out treatment regimens. Given the significant roles of distress and co-occurring conditions, health psychology services further aim to provide psychoeducation about stress management in order to explore and enhance motivation for making a wide range of health behavior changes.

The purpose of this study was to evaluate baseline and follow-up HbA_1c, weight, and psychosocial measures, namely, health-related self-efficacy and T2DM-related distress among a small sample that engaged in integrated health psychology services. The focus of this evidence-based psychotherapy service was to improve T2DM self-care and physical health. The participants were offered cognitive and behavioral strategies for setting and meeting personalized T2DM self-management goals. Importantly, motivational interviewing was used throughout to adapt to the participants’ preferences and needs as well as to maintain engagement.

Methods

Primary care providers referred veterans with T2DM to the Health Psychology service at VA Ann Arbor Healthcare System (VAAAHS). A T2DM diagnosis was verified through electronic health record review. Most common referrals included addressing coping with chronic illness and improving glycemic levels. Veterans were invited to participate in a program evaluation project to monitor health-related changes. All participants provided written informed consent and did not receive incentive or payment for participating. The VAAAHS Institutional Review Board reviewed and approved this study.

Intervention

Veterans met individually with a health psychologist or health psychology trainee to create personalized health and behavioral goals for improving T2DM self-management, overall health, and psychological well-being. This intervention included motivational interviewing, SMART (specific, measurable, action-oriented, realistic, timely) goal setting, behavioral activation, acceptance of T2DM-related physical changes, problem-solving therapy, challenging maladaptive disease-related cognitions, and incorporating values to help find motivation for change. Interventionists took a flexible approach and met in-person in primary or specialty care clinics, over the phone, and through telehealth 1 to 4 times a month, meeting participant preferences, with sessions typically ranging from 45 to 60 minutes. The goal of the study was to disseminate and implement evidence-based behavioral change strategies into a multidisciplinary team format without excluding veterans who would benefit from receiving active treatment. Due to this translational approach, a control group was not included for comparison.

Data Collection

Participants completed study measures at the beginning and end of the T2DM-focused intervention sessions. Demographic variables collected included age, sex, race/ethnicity, highest educational attainment, and whether a veteran was prescribed insulin, service connected for T2DM, concurrent enrollment in other educational programs, and time since T2DM diagnosis. Measures were selected based on their relevance to T2DM psychosocial care and diabetes health outcomes.¹³

Body mass index, low-density lipoprotein cholesterol, blood pressure (BP), HbA_1c within 3 months of the pre- and postmeasures were collected by reviewing medical records. T2DM complications were collected by self-report, and comorbid physical and mental health conditions were collected by review of the most recent primary care note. The Diabetes Empowerment Scale-Short Form (DES-SF) is a well-validated measure that was used to measure T2DM-related psychosocial self-efficacy.¹⁴ Scores ranged from 8 to 40 with higher scores indicating higher diabetes T2DM empowerment. The Patient Health Questionnaire 9-item (PHQ-9) was used to assess the frequency of somatic (fatigue, appetite, psychomotor) and cognitive symptoms (anhedonia, low mood) of depression over the past 2 weeks.¹⁵ The Generalized Anxiety Disorder 7-item (GAD-7) was used to assess the frequency of common anxiety symptoms, including feelings of worry, difficulty controlling worry, and trouble relaxing.¹⁶ Veterans were also asked to rate their general health on a 5-point Likert scale. Self-rated health is a well-established indicator of disability and risk of future T2DM complications in older adults.^17,18 The Diabetes Distress Scale (DDS) was used to measure emotional burden, physician-related distress, regimen-related distress, and T2DM-related interpersonal distress.¹⁹ Scores > 2.0 suggest clinical significant diabetes distress.²⁰ Medication questionnaires were adapted from Wilson and colleagues, 2013.²¹

Statistical Analyses

Descriptive statistics, including mean and standard deviation (SD) or frequency distributions, as appropriate, were used to characterize the sample. For pre- and postintervention within-group comparisons, a paired samples Student t test analysis was used to evaluate baseline and follow-up measures for statistically significant differences between continuous variables; scores also were evaluated for clinically meaningful change.

Results

This sample (N = 13) of older adults was predominately male, white, with HbA_1c > 7.0, and prescribed insulin (Table). On average, participants were at higher risk for future complications due to high BP, hyperlipidemia, and BMI > 30.0. Regarding participation, veterans were seen for an average of 7.8 sessions (range, 4-13) with 46% service connected for T2DM. Of note, 4 veterans received other T2DM-specific self-management support within the same year of their participation with health psychology, such as attending a T2DM education class or T2DM shared medical appointment.²² Reliability in the current sample for the DES-SF was high (Cronbach α = 0.90), PHQ-9 was good (Cronbach α = 0.81), and GAD-7 was very good (Cronbach α = 0.86).

Among the 13 older adults, the most common T2DM-related complications included peripheral neuropathy (n = 7), heart pain or heart attack (n = 5), and retinopathy (n = 4). Recent primary care notes showed a mean (SD) 7 (2.2) comorbid chronic medical conditions with a high prevalence of cardiometabolic illnesses including hypertension, hyperlipidemia, obstructive sleep apnea, and a diagnosis of chronic pain. Eleven veterans were diagnosed with a mental health condition, including bipolar disorder, depression, anxiety, trauma-related disorder, and sleep disorders. Veterans reported high T2DM emotional distress (mean [SD] 3.1 [1.2]), moderate regimen-related distress (mean [SD] 2.9 [1.1]), and moderate total T2DM distress (mean [SD] 2.4 [0.7]). Physician distress (mean [SD] 1.3 [0.55]) and interpersonal T2DM distress (mean [SD] 1.6 [0.9]) subscales indicated little to no distress. The sample reported mild symptoms of depression (PHQ-9 mean [SD] 8.8 [4.6]); mild symptoms of anxiety (GAD-7 mean, 7.1; SD, 4.4), and Diabetes Empowerment (mean, 31.2; SD, 6.0). Participants described missing an average of 2.4 days within the past 30 days of their T2DM oral medications.

Twelve veterans (92.7%) completed the Follow-up questionnaires. The Figure illustrates statistically significant changes in patient-reported outcomes between baseline and follow-up. Clinically meaningful reductions were shown in total T2DM distress (t₁₁ = 5.03, P < .01), T2DM emotional burden (t₁₁ = 4.83, P = .01), and T2DM regimen-related distress (t₁₁ = 5.14, P < .01). There was a significant increase in T2DM self-efficacy (t₁₁ = 0.32, P = .008) as well. A statistically significant reduction was seen in depressive symptoms (t₁₁ = 2.22, P = .048). While HbA_1c fell by .56 percentage points (standard error of the mean [SEM], 31; P = .10), this change was not statistically significant. Follow-up analyses also showed a clinically, though not statistically, significant reduction in weight loss by 6.9 lb. (SEM, 3.8; P = .20), and reductions of generalized anxiety by 1.2 points (SEM, 1.4; P = .42). Pre- and postanalyses did not show differences among self-rated health, physician-related burden, interpersonal-related burden, and indicators of medication taking behavior.

Discussion

This observational study evaluated change among patient-reported T2DM-specific and general distress measures and health outcomes among a small sample of veterans at VAAAHS medical center that engaged in an episode of individual care with health psychology. Statistically significant decreases were observed in T2DM-related distress. Noteworthy, these decreases were observed for the emotional burden and regimen subscales, and each of these was clinically meaningful, falling below a score of 2.0 on the T2DM-specific scale. This is important given that T2DM distress may interfere with the ability to understand and find motivation for engaging in health behavior change. Incorporating stress management interventions into interdisciplinary health programs has been demonstrated to improve not only levels of distress, but also other health outcomes, such as health related quality of life and cardiac events in heart disease.²³ Thus, behavioral health interventions that incorporate cognitive-behavioral strategies to enhance distress-specific coping may prove important to include among individuals with T2DM.

Reductions in T2DM-related distress also converged with increases observed in the T2DM empowerment scale. These significant improvements in perceived ability suggest increased self-efficacy and willingness to follow a daily T2DM regimen. This finding aligns with the social support literature that demonstrates how instrumental and other aspects of autonomous social support mediate improvements in health-related outcomes and reduced T2DM distress.^24,25 Health psychology interventions strive to both provide social support as well as enhance participants’ perceptions and use of existing support as a cognitive-behavioral strategy. Adding in assessments of social support could shed light on such mediating factors.

The ADA standards of care encourage heath care providers to engage patients in conversations in order to better understand the barriers of T2DM self-care.¹³ How to best support patients within a primary care multidisciplinary team remains unclear.²⁶ T2DM distress and negative reactions to T2DM, including symptoms of anxiety and depression, are common and may require specific referral to a mental health provider if repeated attempts at T2DM education do not improve self-management and illness biomarkers.²⁷ Thus, integrating these providers and services within the medical setting aims to reach more veterans and potentially meet these standards of care. With our health psychology integrated services, clinically significant decreases in anxiety and statistically significant decreases in depressive symptoms were observed that approached “mild to no” symptoms. Although this was not measured formally, the veterans were not engaging in mental health specialty care historically or during the year of the health psychology intervention. This suggests that health psychology services helped bridge the gap and address these psychosocial needs within the small sample.

For clinical measures, modest decreases were observed for HbA_1c and weight. The authors recognize that these changes may not be optimal in terms of health status. A review of the specific patient-centered goals may illuminate this finding. For example, 1 participant had a goal to consume fewer sugary beverages and achieved this behavior change. Yet this change alone may not equate to actual weight loss or a lower HbA_1c. Furthermore, in the context of T2DM-related distress, maintaining current weight and/or blood sugar levels may be a more realistic goal. An evaluation of the specific patient-oriented action goals and observed progress may be important outcomes to include in larger studies. Moreover, while not significant, the average HbA_1c decrease of about 1% is comparable with traditional T2DM education and should be considered in light of the sample’s significant mental health comorbidities. While landmark intensive glucose control trials illustrate significant benefits in reductions of hyperglycemia and nonfatal cardiovascular disease, these reductions are associated with an approximate 2-fold risk of hypoglycemia.^28-30 Thus, the focus on improved glycemic control has been criticized as lacking meaning to patients in contrast to preventing T2DM complications and persevering quality of life.³¹

Limitations and Future Directions

Noted limitations include small sample size, the range of time, and a broad number of sessions given that the intervention was tailored to each veteran. Conclusions drawn from a small sample may be influenced by individual outliers. Given co-occurring conditions and moderate levels of distress, all participants may benefit from additional support resources.

In addition to these considerations, having a comparison group could further strengthen the study as part of an observational database. A between-group comparison could help clinicians better understand what the interventions offer as well as some individual factors that relate to participation and success with behavior change. In the future, studies with a priori hypotheses could also consider the trajectories of weight and blood sugar levels for extended periods; for example, 6 months before the intervention and 6 months following.³² Given the complexity of comorbid mental health and chronic medical conditions in this sample, it also may be important to measure the relationships between chronic physical symptoms as an additional barrier for veterans to make health behavior changes.

Conclusions

The authors believe that the health psychology interventions offered important support and motivation for engagement in health behavior change that led to reduced distress in this patient group. It remains a challenge to engage veterans with psychiatric conditions in mental health care, and simultaneously for health care systems that strive to reduce costs and complications associated with chronic illness management.³³ Aligned with these broader health care goals, the ADA aims to reduce complications and cost and improve outcomes for T2DM with guidelines requiring mental and behavioral health interventions. The authors believe that health psychology interventions are a personalized and feasible bridge to address engagement, illness-related distress while improving patient-satisfaction and T2DM self-management.

Acknowledgments

The authors thank the veterans who participated in the observational study. We thank the VA Ann Arbor Healthcare System Institutional Review Board. For instrumental support for health psychology integrated services, we acknowledge Adam Tremblay, MD, Primary Care Chief, and R.J. Schildhouse, MD, Acting Associate Chief of Staff, Ambulatory Care. The work was supported by the Ambulatory Care Service at the VA Ann Arbor Healthcare System and the VA Office of Academic Affiliations.

Veterans have a higher prevalence of type 2 diabetes mellitus (T2DM) when compared with their civilian counterparts with an overall prevalence rate of 25%.¹ This higher prevalence is similar to other major chronic health conditions, including heart disease and arthritis, with additional costs for disease self-management.² Psychological and behavioral change strategies are a principal means of limiting the severity and even restoring function once T2DM is diagnosed.³ More broadly, there is mounting evidence that addressing distress and behavior change are important across many conditions, particularly T2DM.⁴ Therefore, the US Department of Veterans Affairs (VA) has established patient education and multidisciplinary interventions to optimize engagement in T2DM self-management and health behavior change.⁵

Traditional T2DM education programs aim to meet the American Diabetes Association (ADA) standards of medical care and include a T2DM educator and other allied health professionals. ADA Standard 1.2 emphasizes “productive interactions between a prepared, proactive care team and an informed, activated patient.”⁶ Thus, to attain ADA accreditation, educational programs require instructors to teach about T2DM while engaging patients to help them set and achieve recommended changes. The requirements emphasize setting specific goals, (ie, eating wisely, being physically active, monitoring blood sugars or taking medications). The care team also helps to identify barriers, and at a required follow-up class, patients evaluate how well they met goals and make modifications if needed. The impact of traditional patient education programs to improve glycemic levels is well established.⁷ Importantly, veterans with comorbid mental health conditions may not experience the same beneficial outcomes if or when they participate in traditional diabetes or self-management programs.^8,9 Veterans with T2DM may be particularly vulnerable to chronic stress and effects of comorbid mental health diagnoses.¹⁰ Furthermore, when individuals experience T2DM-related distress, associations with poor health outcomes, including elevated hemoglobin A_1c (HbA_1c), are observed independent of depression.¹¹

Health psychology services integrate into medical settings and strive to reach veterans who may not engage in traditional mental health clinical offerings.¹² These collaborative interventions focus less on diagnostic or screening procedures and more on a patient’s understanding of illness and ability and willingness to carry out treatment regimens. Given the significant roles of distress and co-occurring conditions, health psychology services further aim to provide psychoeducation about stress management in order to explore and enhance motivation for making a wide range of health behavior changes.

The purpose of this study was to evaluate baseline and follow-up HbA_1c, weight, and psychosocial measures, namely, health-related self-efficacy and T2DM-related distress among a small sample that engaged in integrated health psychology services. The focus of this evidence-based psychotherapy service was to improve T2DM self-care and physical health. The participants were offered cognitive and behavioral strategies for setting and meeting personalized T2DM self-management goals. Importantly, motivational interviewing was used throughout to adapt to the participants’ preferences and needs as well as to maintain engagement.

Methods

Primary care providers referred veterans with T2DM to the Health Psychology service at VA Ann Arbor Healthcare System (VAAAHS). A T2DM diagnosis was verified through electronic health record review. Most common referrals included addressing coping with chronic illness and improving glycemic levels. Veterans were invited to participate in a program evaluation project to monitor health-related changes. All participants provided written informed consent and did not receive incentive or payment for participating. The VAAAHS Institutional Review Board reviewed and approved this study.

Intervention

Veterans met individually with a health psychologist or health psychology trainee to create personalized health and behavioral goals for improving T2DM self-management, overall health, and psychological well-being. This intervention included motivational interviewing, SMART (specific, measurable, action-oriented, realistic, timely) goal setting, behavioral activation, acceptance of T2DM-related physical changes, problem-solving therapy, challenging maladaptive disease-related cognitions, and incorporating values to help find motivation for change. Interventionists took a flexible approach and met in-person in primary or specialty care clinics, over the phone, and through telehealth 1 to 4 times a month, meeting participant preferences, with sessions typically ranging from 45 to 60 minutes. The goal of the study was to disseminate and implement evidence-based behavioral change strategies into a multidisciplinary team format without excluding veterans who would benefit from receiving active treatment. Due to this translational approach, a control group was not included for comparison.

Data Collection

Participants completed study measures at the beginning and end of the T2DM-focused intervention sessions. Demographic variables collected included age, sex, race/ethnicity, highest educational attainment, and whether a veteran was prescribed insulin, service connected for T2DM, concurrent enrollment in other educational programs, and time since T2DM diagnosis. Measures were selected based on their relevance to T2DM psychosocial care and diabetes health outcomes.¹³

Body mass index, low-density lipoprotein cholesterol, blood pressure (BP), HbA_1c within 3 months of the pre- and postmeasures were collected by reviewing medical records. T2DM complications were collected by self-report, and comorbid physical and mental health conditions were collected by review of the most recent primary care note. The Diabetes Empowerment Scale-Short Form (DES-SF) is a well-validated measure that was used to measure T2DM-related psychosocial self-efficacy.¹⁴ Scores ranged from 8 to 40 with higher scores indicating higher diabetes T2DM empowerment. The Patient Health Questionnaire 9-item (PHQ-9) was used to assess the frequency of somatic (fatigue, appetite, psychomotor) and cognitive symptoms (anhedonia, low mood) of depression over the past 2 weeks.¹⁵ The Generalized Anxiety Disorder 7-item (GAD-7) was used to assess the frequency of common anxiety symptoms, including feelings of worry, difficulty controlling worry, and trouble relaxing.¹⁶ Veterans were also asked to rate their general health on a 5-point Likert scale. Self-rated health is a well-established indicator of disability and risk of future T2DM complications in older adults.^17,18 The Diabetes Distress Scale (DDS) was used to measure emotional burden, physician-related distress, regimen-related distress, and T2DM-related interpersonal distress.¹⁹ Scores > 2.0 suggest clinical significant diabetes distress.²⁰ Medication questionnaires were adapted from Wilson and colleagues, 2013.²¹

Statistical Analyses

Descriptive statistics, including mean and standard deviation (SD) or frequency distributions, as appropriate, were used to characterize the sample. For pre- and postintervention within-group comparisons, a paired samples Student t test analysis was used to evaluate baseline and follow-up measures for statistically significant differences between continuous variables; scores also were evaluated for clinically meaningful change.

Results

This sample (N = 13) of older adults was predominately male, white, with HbA_1c > 7.0, and prescribed insulin (Table). On average, participants were at higher risk for future complications due to high BP, hyperlipidemia, and BMI > 30.0. Regarding participation, veterans were seen for an average of 7.8 sessions (range, 4-13) with 46% service connected for T2DM. Of note, 4 veterans received other T2DM-specific self-management support within the same year of their participation with health psychology, such as attending a T2DM education class or T2DM shared medical appointment.²² Reliability in the current sample for the DES-SF was high (Cronbach α = 0.90), PHQ-9 was good (Cronbach α = 0.81), and GAD-7 was very good (Cronbach α = 0.86).

Among the 13 older adults, the most common T2DM-related complications included peripheral neuropathy (n = 7), heart pain or heart attack (n = 5), and retinopathy (n = 4). Recent primary care notes showed a mean (SD) 7 (2.2) comorbid chronic medical conditions with a high prevalence of cardiometabolic illnesses including hypertension, hyperlipidemia, obstructive sleep apnea, and a diagnosis of chronic pain. Eleven veterans were diagnosed with a mental health condition, including bipolar disorder, depression, anxiety, trauma-related disorder, and sleep disorders. Veterans reported high T2DM emotional distress (mean [SD] 3.1 [1.2]), moderate regimen-related distress (mean [SD] 2.9 [1.1]), and moderate total T2DM distress (mean [SD] 2.4 [0.7]). Physician distress (mean [SD] 1.3 [0.55]) and interpersonal T2DM distress (mean [SD] 1.6 [0.9]) subscales indicated little to no distress. The sample reported mild symptoms of depression (PHQ-9 mean [SD] 8.8 [4.6]); mild symptoms of anxiety (GAD-7 mean, 7.1; SD, 4.4), and Diabetes Empowerment (mean, 31.2; SD, 6.0). Participants described missing an average of 2.4 days within the past 30 days of their T2DM oral medications.

Twelve veterans (92.7%) completed the Follow-up questionnaires. The Figure illustrates statistically significant changes in patient-reported outcomes between baseline and follow-up. Clinically meaningful reductions were shown in total T2DM distress (t₁₁ = 5.03, P < .01), T2DM emotional burden (t₁₁ = 4.83, P = .01), and T2DM regimen-related distress (t₁₁ = 5.14, P < .01). There was a significant increase in T2DM self-efficacy (t₁₁ = 0.32, P = .008) as well. A statistically significant reduction was seen in depressive symptoms (t₁₁ = 2.22, P = .048). While HbA_1c fell by .56 percentage points (standard error of the mean [SEM], 31; P = .10), this change was not statistically significant. Follow-up analyses also showed a clinically, though not statistically, significant reduction in weight loss by 6.9 lb. (SEM, 3.8; P = .20), and reductions of generalized anxiety by 1.2 points (SEM, 1.4; P = .42). Pre- and postanalyses did not show differences among self-rated health, physician-related burden, interpersonal-related burden, and indicators of medication taking behavior.

Discussion

This observational study evaluated change among patient-reported T2DM-specific and general distress measures and health outcomes among a small sample of veterans at VAAAHS medical center that engaged in an episode of individual care with health psychology. Statistically significant decreases were observed in T2DM-related distress. Noteworthy, these decreases were observed for the emotional burden and regimen subscales, and each of these was clinically meaningful, falling below a score of 2.0 on the T2DM-specific scale. This is important given that T2DM distress may interfere with the ability to understand and find motivation for engaging in health behavior change. Incorporating stress management interventions into interdisciplinary health programs has been demonstrated to improve not only levels of distress, but also other health outcomes, such as health related quality of life and cardiac events in heart disease.²³ Thus, behavioral health interventions that incorporate cognitive-behavioral strategies to enhance distress-specific coping may prove important to include among individuals with T2DM.

Reductions in T2DM-related distress also converged with increases observed in the T2DM empowerment scale. These significant improvements in perceived ability suggest increased self-efficacy and willingness to follow a daily T2DM regimen. This finding aligns with the social support literature that demonstrates how instrumental and other aspects of autonomous social support mediate improvements in health-related outcomes and reduced T2DM distress.^24,25 Health psychology interventions strive to both provide social support as well as enhance participants’ perceptions and use of existing support as a cognitive-behavioral strategy. Adding in assessments of social support could shed light on such mediating factors.

The ADA standards of care encourage heath care providers to engage patients in conversations in order to better understand the barriers of T2DM self-care.¹³ How to best support patients within a primary care multidisciplinary team remains unclear.²⁶ T2DM distress and negative reactions to T2DM, including symptoms of anxiety and depression, are common and may require specific referral to a mental health provider if repeated attempts at T2DM education do not improve self-management and illness biomarkers.²⁷ Thus, integrating these providers and services within the medical setting aims to reach more veterans and potentially meet these standards of care. With our health psychology integrated services, clinically significant decreases in anxiety and statistically significant decreases in depressive symptoms were observed that approached “mild to no” symptoms. Although this was not measured formally, the veterans were not engaging in mental health specialty care historically or during the year of the health psychology intervention. This suggests that health psychology services helped bridge the gap and address these psychosocial needs within the small sample.

For clinical measures, modest decreases were observed for HbA_1c and weight. The authors recognize that these changes may not be optimal in terms of health status. A review of the specific patient-centered goals may illuminate this finding. For example, 1 participant had a goal to consume fewer sugary beverages and achieved this behavior change. Yet this change alone may not equate to actual weight loss or a lower HbA_1c. Furthermore, in the context of T2DM-related distress, maintaining current weight and/or blood sugar levels may be a more realistic goal. An evaluation of the specific patient-oriented action goals and observed progress may be important outcomes to include in larger studies. Moreover, while not significant, the average HbA_1c decrease of about 1% is comparable with traditional T2DM education and should be considered in light of the sample’s significant mental health comorbidities. While landmark intensive glucose control trials illustrate significant benefits in reductions of hyperglycemia and nonfatal cardiovascular disease, these reductions are associated with an approximate 2-fold risk of hypoglycemia.^28-30 Thus, the focus on improved glycemic control has been criticized as lacking meaning to patients in contrast to preventing T2DM complications and persevering quality of life.³¹

Limitations and Future Directions

Noted limitations include small sample size, the range of time, and a broad number of sessions given that the intervention was tailored to each veteran. Conclusions drawn from a small sample may be influenced by individual outliers. Given co-occurring conditions and moderate levels of distress, all participants may benefit from additional support resources.

In addition to these considerations, having a comparison group could further strengthen the study as part of an observational database. A between-group comparison could help clinicians better understand what the interventions offer as well as some individual factors that relate to participation and success with behavior change. In the future, studies with a priori hypotheses could also consider the trajectories of weight and blood sugar levels for extended periods; for example, 6 months before the intervention and 6 months following.³² Given the complexity of comorbid mental health and chronic medical conditions in this sample, it also may be important to measure the relationships between chronic physical symptoms as an additional barrier for veterans to make health behavior changes.

Conclusions

The authors believe that the health psychology interventions offered important support and motivation for engagement in health behavior change that led to reduced distress in this patient group. It remains a challenge to engage veterans with psychiatric conditions in mental health care, and simultaneously for health care systems that strive to reduce costs and complications associated with chronic illness management.³³ Aligned with these broader health care goals, the ADA aims to reduce complications and cost and improve outcomes for T2DM with guidelines requiring mental and behavioral health interventions. The authors believe that health psychology interventions are a personalized and feasible bridge to address engagement, illness-related distress while improving patient-satisfaction and T2DM self-management.

Acknowledgments

The authors thank the veterans who participated in the observational study. We thank the VA Ann Arbor Healthcare System Institutional Review Board. For instrumental support for health psychology integrated services, we acknowledge Adam Tremblay, MD, Primary Care Chief, and R.J. Schildhouse, MD, Acting Associate Chief of Staff, Ambulatory Care. The work was supported by the Ambulatory Care Service at the VA Ann Arbor Healthcare System and the VA Office of Academic Affiliations.

Veterans have a higher prevalence of type 2 diabetes mellitus (T2DM) when compared with their civilian counterparts with an overall prevalence rate of 25%.¹ This higher prevalence is similar to other major chronic health conditions, including heart disease and arthritis, with additional costs for disease self-management.² Psychological and behavioral change strategies are a principal means of limiting the severity and even restoring function once T2DM is diagnosed.³ More broadly, there is mounting evidence that addressing distress and behavior change are important across many conditions, particularly T2DM.⁴ Therefore, the US Department of Veterans Affairs (VA) has established patient education and multidisciplinary interventions to optimize engagement in T2DM self-management and health behavior change.⁵

Traditional T2DM education programs aim to meet the American Diabetes Association (ADA) standards of medical care and include a T2DM educator and other allied health professionals. ADA Standard 1.2 emphasizes “productive interactions between a prepared, proactive care team and an informed, activated patient.”⁶ Thus, to attain ADA accreditation, educational programs require instructors to teach about T2DM while engaging patients to help them set and achieve recommended changes. The requirements emphasize setting specific goals, (ie, eating wisely, being physically active, monitoring blood sugars or taking medications). The care team also helps to identify barriers, and at a required follow-up class, patients evaluate how well they met goals and make modifications if needed. The impact of traditional patient education programs to improve glycemic levels is well established.⁷ Importantly, veterans with comorbid mental health conditions may not experience the same beneficial outcomes if or when they participate in traditional diabetes or self-management programs.^8,9 Veterans with T2DM may be particularly vulnerable to chronic stress and effects of comorbid mental health diagnoses.¹⁰ Furthermore, when individuals experience T2DM-related distress, associations with poor health outcomes, including elevated hemoglobin A_1c (HbA_1c), are observed independent of depression.¹¹

Health psychology services integrate into medical settings and strive to reach veterans who may not engage in traditional mental health clinical offerings.¹² These collaborative interventions focus less on diagnostic or screening procedures and more on a patient’s understanding of illness and ability and willingness to carry out treatment regimens. Given the significant roles of distress and co-occurring conditions, health psychology services further aim to provide psychoeducation about stress management in order to explore and enhance motivation for making a wide range of health behavior changes.

The purpose of this study was to evaluate baseline and follow-up HbA_1c, weight, and psychosocial measures, namely, health-related self-efficacy and T2DM-related distress among a small sample that engaged in integrated health psychology services. The focus of this evidence-based psychotherapy service was to improve T2DM self-care and physical health. The participants were offered cognitive and behavioral strategies for setting and meeting personalized T2DM self-management goals. Importantly, motivational interviewing was used throughout to adapt to the participants’ preferences and needs as well as to maintain engagement.

Methods

Primary care providers referred veterans with T2DM to the Health Psychology service at VA Ann Arbor Healthcare System (VAAAHS). A T2DM diagnosis was verified through electronic health record review. Most common referrals included addressing coping with chronic illness and improving glycemic levels. Veterans were invited to participate in a program evaluation project to monitor health-related changes. All participants provided written informed consent and did not receive incentive or payment for participating. The VAAAHS Institutional Review Board reviewed and approved this study.

Intervention

Veterans met individually with a health psychologist or health psychology trainee to create personalized health and behavioral goals for improving T2DM self-management, overall health, and psychological well-being. This intervention included motivational interviewing, SMART (specific, measurable, action-oriented, realistic, timely) goal setting, behavioral activation, acceptance of T2DM-related physical changes, problem-solving therapy, challenging maladaptive disease-related cognitions, and incorporating values to help find motivation for change. Interventionists took a flexible approach and met in-person in primary or specialty care clinics, over the phone, and through telehealth 1 to 4 times a month, meeting participant preferences, with sessions typically ranging from 45 to 60 minutes. The goal of the study was to disseminate and implement evidence-based behavioral change strategies into a multidisciplinary team format without excluding veterans who would benefit from receiving active treatment. Due to this translational approach, a control group was not included for comparison.

Data Collection

Participants completed study measures at the beginning and end of the T2DM-focused intervention sessions. Demographic variables collected included age, sex, race/ethnicity, highest educational attainment, and whether a veteran was prescribed insulin, service connected for T2DM, concurrent enrollment in other educational programs, and time since T2DM diagnosis. Measures were selected based on their relevance to T2DM psychosocial care and diabetes health outcomes.¹³

Body mass index, low-density lipoprotein cholesterol, blood pressure (BP), HbA_1c within 3 months of the pre- and postmeasures were collected by reviewing medical records. T2DM complications were collected by self-report, and comorbid physical and mental health conditions were collected by review of the most recent primary care note. The Diabetes Empowerment Scale-Short Form (DES-SF) is a well-validated measure that was used to measure T2DM-related psychosocial self-efficacy.¹⁴ Scores ranged from 8 to 40 with higher scores indicating higher diabetes T2DM empowerment. The Patient Health Questionnaire 9-item (PHQ-9) was used to assess the frequency of somatic (fatigue, appetite, psychomotor) and cognitive symptoms (anhedonia, low mood) of depression over the past 2 weeks.¹⁵ The Generalized Anxiety Disorder 7-item (GAD-7) was used to assess the frequency of common anxiety symptoms, including feelings of worry, difficulty controlling worry, and trouble relaxing.¹⁶ Veterans were also asked to rate their general health on a 5-point Likert scale. Self-rated health is a well-established indicator of disability and risk of future T2DM complications in older adults.^17,18 The Diabetes Distress Scale (DDS) was used to measure emotional burden, physician-related distress, regimen-related distress, and T2DM-related interpersonal distress.¹⁹ Scores > 2.0 suggest clinical significant diabetes distress.²⁰ Medication questionnaires were adapted from Wilson and colleagues, 2013.²¹

Statistical Analyses

Descriptive statistics, including mean and standard deviation (SD) or frequency distributions, as appropriate, were used to characterize the sample. For pre- and postintervention within-group comparisons, a paired samples Student t test analysis was used to evaluate baseline and follow-up measures for statistically significant differences between continuous variables; scores also were evaluated for clinically meaningful change.

Results

This sample (N = 13) of older adults was predominately male, white, with HbA_1c > 7.0, and prescribed insulin (Table). On average, participants were at higher risk for future complications due to high BP, hyperlipidemia, and BMI > 30.0. Regarding participation, veterans were seen for an average of 7.8 sessions (range, 4-13) with 46% service connected for T2DM. Of note, 4 veterans received other T2DM-specific self-management support within the same year of their participation with health psychology, such as attending a T2DM education class or T2DM shared medical appointment.²² Reliability in the current sample for the DES-SF was high (Cronbach α = 0.90), PHQ-9 was good (Cronbach α = 0.81), and GAD-7 was very good (Cronbach α = 0.86).

Among the 13 older adults, the most common T2DM-related complications included peripheral neuropathy (n = 7), heart pain or heart attack (n = 5), and retinopathy (n = 4). Recent primary care notes showed a mean (SD) 7 (2.2) comorbid chronic medical conditions with a high prevalence of cardiometabolic illnesses including hypertension, hyperlipidemia, obstructive sleep apnea, and a diagnosis of chronic pain. Eleven veterans were diagnosed with a mental health condition, including bipolar disorder, depression, anxiety, trauma-related disorder, and sleep disorders. Veterans reported high T2DM emotional distress (mean [SD] 3.1 [1.2]), moderate regimen-related distress (mean [SD] 2.9 [1.1]), and moderate total T2DM distress (mean [SD] 2.4 [0.7]). Physician distress (mean [SD] 1.3 [0.55]) and interpersonal T2DM distress (mean [SD] 1.6 [0.9]) subscales indicated little to no distress. The sample reported mild symptoms of depression (PHQ-9 mean [SD] 8.8 [4.6]); mild symptoms of anxiety (GAD-7 mean, 7.1; SD, 4.4), and Diabetes Empowerment (mean, 31.2; SD, 6.0). Participants described missing an average of 2.4 days within the past 30 days of their T2DM oral medications.

Twelve veterans (92.7%) completed the Follow-up questionnaires. The Figure illustrates statistically significant changes in patient-reported outcomes between baseline and follow-up. Clinically meaningful reductions were shown in total T2DM distress (t₁₁ = 5.03, P < .01), T2DM emotional burden (t₁₁ = 4.83, P = .01), and T2DM regimen-related distress (t₁₁ = 5.14, P < .01). There was a significant increase in T2DM self-efficacy (t₁₁ = 0.32, P = .008) as well. A statistically significant reduction was seen in depressive symptoms (t₁₁ = 2.22, P = .048). While HbA_1c fell by .56 percentage points (standard error of the mean [SEM], 31; P = .10), this change was not statistically significant. Follow-up analyses also showed a clinically, though not statistically, significant reduction in weight loss by 6.9 lb. (SEM, 3.8; P = .20), and reductions of generalized anxiety by 1.2 points (SEM, 1.4; P = .42). Pre- and postanalyses did not show differences among self-rated health, physician-related burden, interpersonal-related burden, and indicators of medication taking behavior.

Discussion

This observational study evaluated change among patient-reported T2DM-specific and general distress measures and health outcomes among a small sample of veterans at VAAAHS medical center that engaged in an episode of individual care with health psychology. Statistically significant decreases were observed in T2DM-related distress. Noteworthy, these decreases were observed for the emotional burden and regimen subscales, and each of these was clinically meaningful, falling below a score of 2.0 on the T2DM-specific scale. This is important given that T2DM distress may interfere with the ability to understand and find motivation for engaging in health behavior change. Incorporating stress management interventions into interdisciplinary health programs has been demonstrated to improve not only levels of distress, but also other health outcomes, such as health related quality of life and cardiac events in heart disease.²³ Thus, behavioral health interventions that incorporate cognitive-behavioral strategies to enhance distress-specific coping may prove important to include among individuals with T2DM.

Reductions in T2DM-related distress also converged with increases observed in the T2DM empowerment scale. These significant improvements in perceived ability suggest increased self-efficacy and willingness to follow a daily T2DM regimen. This finding aligns with the social support literature that demonstrates how instrumental and other aspects of autonomous social support mediate improvements in health-related outcomes and reduced T2DM distress.^24,25 Health psychology interventions strive to both provide social support as well as enhance participants’ perceptions and use of existing support as a cognitive-behavioral strategy. Adding in assessments of social support could shed light on such mediating factors.

The ADA standards of care encourage heath care providers to engage patients in conversations in order to better understand the barriers of T2DM self-care.¹³ How to best support patients within a primary care multidisciplinary team remains unclear.²⁶ T2DM distress and negative reactions to T2DM, including symptoms of anxiety and depression, are common and may require specific referral to a mental health provider if repeated attempts at T2DM education do not improve self-management and illness biomarkers.²⁷ Thus, integrating these providers and services within the medical setting aims to reach more veterans and potentially meet these standards of care. With our health psychology integrated services, clinically significant decreases in anxiety and statistically significant decreases in depressive symptoms were observed that approached “mild to no” symptoms. Although this was not measured formally, the veterans were not engaging in mental health specialty care historically or during the year of the health psychology intervention. This suggests that health psychology services helped bridge the gap and address these psychosocial needs within the small sample.

For clinical measures, modest decreases were observed for HbA_1c and weight. The authors recognize that these changes may not be optimal in terms of health status. A review of the specific patient-centered goals may illuminate this finding. For example, 1 participant had a goal to consume fewer sugary beverages and achieved this behavior change. Yet this change alone may not equate to actual weight loss or a lower HbA_1c. Furthermore, in the context of T2DM-related distress, maintaining current weight and/or blood sugar levels may be a more realistic goal. An evaluation of the specific patient-oriented action goals and observed progress may be important outcomes to include in larger studies. Moreover, while not significant, the average HbA_1c decrease of about 1% is comparable with traditional T2DM education and should be considered in light of the sample’s significant mental health comorbidities. While landmark intensive glucose control trials illustrate significant benefits in reductions of hyperglycemia and nonfatal cardiovascular disease, these reductions are associated with an approximate 2-fold risk of hypoglycemia.^28-30 Thus, the focus on improved glycemic control has been criticized as lacking meaning to patients in contrast to preventing T2DM complications and persevering quality of life.³¹

Limitations and Future Directions

Noted limitations include small sample size, the range of time, and a broad number of sessions given that the intervention was tailored to each veteran. Conclusions drawn from a small sample may be influenced by individual outliers. Given co-occurring conditions and moderate levels of distress, all participants may benefit from additional support resources.

In addition to these considerations, having a comparison group could further strengthen the study as part of an observational database. A between-group comparison could help clinicians better understand what the interventions offer as well as some individual factors that relate to participation and success with behavior change. In the future, studies with a priori hypotheses could also consider the trajectories of weight and blood sugar levels for extended periods; for example, 6 months before the intervention and 6 months following.³² Given the complexity of comorbid mental health and chronic medical conditions in this sample, it also may be important to measure the relationships between chronic physical symptoms as an additional barrier for veterans to make health behavior changes.

Conclusions

The authors believe that the health psychology interventions offered important support and motivation for engagement in health behavior change that led to reduced distress in this patient group. It remains a challenge to engage veterans with psychiatric conditions in mental health care, and simultaneously for health care systems that strive to reduce costs and complications associated with chronic illness management.³³ Aligned with these broader health care goals, the ADA aims to reduce complications and cost and improve outcomes for T2DM with guidelines requiring mental and behavioral health interventions. The authors believe that health psychology interventions are a personalized and feasible bridge to address engagement, illness-related distress while improving patient-satisfaction and T2DM self-management.

Acknowledgments

The authors thank the veterans who participated in the observational study. We thank the VA Ann Arbor Healthcare System Institutional Review Board. For instrumental support for health psychology integrated services, we acknowledge Adam Tremblay, MD, Primary Care Chief, and R.J. Schildhouse, MD, Acting Associate Chief of Staff, Ambulatory Care. The work was supported by the Ambulatory Care Service at the VA Ann Arbor Healthcare System and the VA Office of Academic Affiliations.