Mario A Reyes, MD

The COVID-19 pandemic has led to more than 40 million infections and more than 650,000 deaths in the United States alone.¹ Morbidity and mortality have disproportionately affected older adults.^2-4 However, acute infection and delayed effects, such as multisystem inflammatory syndrome in children (MIS-C), occur and can lead to severe complications, hospitalization, and death in pediatric patients.^5,6 Due to higher clinical disease prevalence and morbidity in the adult population, we have learned much about the clinical factors associated with severe adult COVID-19 disease.^5,7-9 Such clinical factors include older age, concurrent comorbidities, smoke exposure, and Black race or Hispanic ethnicity, among others.^5,7-10 However, there is a paucity of data on severe COVID-19 disease in pediatric patients.^5,11,12 In addition, most immunization strategies and pharmacologic treatments for COVID-19 have not been evaluated or approved for use in children.¹³ To guide targeted prevention and treatment strategies, there is a critical need to identify children and adolescents—who are among the most vulnerable patient populations—at high risk for severe disease.

Identifying the clinical factors associated with severe COVID-19 disease will help with prioritizing and allocating vaccines when they are approved for use in patients younger than 12 years. It also can provide insight for clinicians and families faced with decisions wherein individual risk assessment is crucial (eg, in-person schooling, other group activities). The objective of this study was to determine the clinical factors associated with severe COVID-19 among children and adolescents in the United States.

Study Design

We conducted a multicenter retrospective cohort study of patients presenting for care at pediatric hospitals that report data to the Pediatric Health Information System (PHIS) database. The PHIS administrative database includes billing and utilization data from 45 US tertiary care hospitals affiliated with the Children’s Hospital Association (Lenexa, Kansas). Data quality and reliability are ensured through a joint validation effort between the Children’s Hospital Association and participating hospitals. Hospitals submit discharge data, including demographics, diagnoses, and procedures using International Classification of Diseases, 10th Revision (ICD-10) codes, along with daily detailed information on pharmacy, location of care, and other services.

Study Population

Patients 30 days to 18 years of age discharged from the emergency department (ED) or inpatient setting with a primary diagnosis of COVID-19 (ICD-10 codes U.071 and U.072) between April 1, 2020, and September 30, 2020, were eligible for inclusion.¹⁴ In a prior study, the positive predictive value of an ICD-10–coded diagnosis of COVID-19 among hospitalized pediatric patients was 95.5%, compared with reverse transcription polymerase reaction results or presence of MIS-C.¹⁵ The diagnostic code for COVID-19 (ICD-10-CM) also had a high sensitivity (98.0%) in the hospitalized population.¹⁶ Acknowledging the increasing practice of screening patients upon admission, and in an attempt to minimize potential misclassification, we did not include encounters with secondary diagnoses of COVID-19 in our primary analyses. Pediatric patients with surgical diagnoses and neonates who never left the hospital were also excluded.

Factors Associated With Severe COVID-19 Disease

Exposures of interest were determined a priori based on current evidence in the literature and included patient age (0-4 years, 5-11 years, and 12-18 years), sex, race and ethnicity (non-Hispanic White, non-Hispanic Black, Hispanic, Asian, other non-White race [defined as Pacific Islander, Native American, or other]), payor type, cardiovascular complex chronic conditions (CCC), neuromuscular CCC, obesity/type 2 diabetes mellitus (DM), pulmonary CCC, asthma (defined using ICD-10 codes¹⁷), and immunocompromised CCC. Race and ethnicity were included as covariates based on previous studies reporting differences in COVID-19 outcomes among racial and ethnic groups.⁹ The CCC covariates were defined using the pediatric CCC ICD-10 classification system version 2.¹⁸

Pediatric Complications and Conditions Associated With COVID-19

Based on current evidence and expert opinion of study members, associated diagnoses and complications co-occurring with a COVID-19 diagnosis were defined a priori and identified through ICD-10 codes (Appendix Table 1). These included acute kidney injury, acute liver injury, aseptic meningitis, asthma exacerbation, bronchiolitis, cerebral infarction, croup, encephalitis, encephalopathy, infant fever, febrile seizure, gastroenteritis/dehydration, Kawasaki disease/MIS-C, myocarditis/pericarditis, pneumonia, lung effusion or empyema, respiratory failure, sepsis, nonfebrile seizure, pancreatitis, sickle cell complications, and thrombotic complications.

Outcomes

COVID-19 severity outcomes were assessed as follows: (1) mild = ED discharge; (2) moderate = inpatient admission; (3) severe = intensive care unit (ICU) admission without mechanical ventilation, shock, or death; and (4) very severe = ICU admission with mechanical ventilation, shock, or death.¹⁹ This ordinal ranking system did not violate the proportional odds assumption. Potential reasons for admission to the ICU without mechanical ventilation, shock, or death include, but are not limited to, need for noninvasive ventilation, vital sign instability, dysrhythmias, respiratory insufficiency, or complications arising from concurrent conditions (eg, thrombotic events, need for continuous albuterol therapy). We examined several secondary, hospital-based outcomes, including associated diagnoses and complications, all-cause 30-day healthcare reutilization (ED visit or rehospitalization), length of stay (LOS), and ICU LOS.

Statistical Analysis

Demographic characteristics were summarized using frequencies and percentages for categorical variables and geometric means with SD and medians with interquartile ranges (IQR) for continuous variables, as appropriate. Factors associated with hospitalization (encompassing severity levels 2-4) vs ED discharge (severity level 1) were assessed using logistic regression. Factors associated with increasing severity among hospitalized pediatric patients (severity levels 2, 3, and 4) were assessed using ordinal logistic regression. Covariates in these analyses included race and ethnicity, age, sex, payor, cardiovascular CCC, neurologic/neuromuscular CCC, obesity/type 2 DM, pulmonary CCC, asthma, and immunocompromised CCC. Adjusted odds ratios (aOR) and corresponding 95% CI for each risk factor were generated using generalized linear mixed effects models and random intercepts for each hospital. Given the potential for diagnostic misclassification of pediatric patients with COVID-19 based on primary vs secondary diagnoses, we performed sensitivity analyses defining the study population as those with a primary diagnosis of COVID-19 and those with a secondary diagnosis of COVID-19 plus a concurrent primary diagnosis of a condition associated with COVID-19 (Appendix Table 1).

All analyses were performed using SAS version 9.4 (SAS Institute, Inc), and P < .05 was considered statistically significant. The Institutional Review Board at Vanderbilt University Medical Center determined that this study of de-identified data did not meet the criteria for human subjects research.

Study Population

A total of 19,976 encounters were included in the study. Of those, 15,913 (79.7%) were discharged from the ED and 4063 (20.3%) were hospitalized (Table 1). The most common race/ethnicity was Hispanic (9741, 48.8%), followed by non-Hispanic White (4217, 21.1%). Reference race/ethnicity data for the overall 2019 PHIS population can be found in Appendix Table 2.

The severity distribution among the hospitalized population was moderate (3222, 79.3%), severe (431, 11.3%), and very severe (380, 9.4%). The frequency of COVID-19 diagnoses increased late in the study period (Figure). Among those hospitalized, the median LOS for the index admission was 2 days (IQR, 1-4), while among those admitted to the ICU, the median LOS was 3 days (IQR, 2-5).

Overall, 10.1% (n = 2020) of the study population had an all-cause repeat encounter (ie, subsequent ED encounter or hospitalization) within 30 days following the index discharge. Repeat encounters were more frequent among patients hospitalized than among those discharged from the ED (Appendix Table 3).

Prevalence of Conditions and Complications Associated With COVID-19

Overall, 3257 (16.3%) patients had one or more co-occurring diagnoses categorized as a COVID-19–associated condition or complication. The most frequent diagnoses included lower respiratory tract disease (pneumonia, lung effusion, or empyema; n = 1415, 7.1%), gastroenteritis/dehydration (n = 1068, 5.3%), respiratory failure (n = 731, 3.7%), febrile infant (n = 413, 2.1%), and nonfebrile seizure (n = 425, 2.1%). Aside from nonfebrile seizure, neurological complications were less frequent and included febrile seizure (n = 155, 0.8%), encephalopathy (n = 63, 0.3%), aseptic meningitis (n = 16, 0.1%), encephalitis (n = 11, 0.1%), and cerebral infarction (n = 6, <0.1%). Kawasaki disease and MIS-C comprised 1.7% (n = 346) of diagnoses. Thrombotic complications occurred in 0.1% (n = 13) of patients. Overall, these conditions and complications associated with COVID-19 were more frequent in hospitalized patients than in those discharged from the ED (P < .001) (Table 2).

Factors Associated With COVID-19 Disease Severity

Compared to pediatric patients with COVID-19 discharged from the ED, factors associated with increased odds of hospitalization included private payor insurance; obesity/type 2 DM; asthma; and cardiovascular, immunocompromised, neurologic/neuromuscular, and pulmonary CCCs (Table 3). Factors associated with decreased risk of hospitalization included Black race or Hispanic ethnicity compared with White race; female sex; and age 5 to 11 years and age 12 to 17 years (vs age 0-4 years). Among children and adolescents hospitalized with COVID-19, factors associated with greater disease severity included Black or other non-White race; age 5 to 11 years; age 12 to 17 years; obesity/type 2 DM; immunocompromised conditions; and cardiovascular, neurologic/neuromuscular, and pulmonary CCCs (Table 3).

Sensitivity Analysis

We performed a sensitivity analysis that expanded the study population to include those with a secondary diagnosis of COVID-19 plus a diagnosis of a COVID-19–associated condition or complication. Analyses using the expanded population (N = 21,247) were similar to the primary analyses (Appendix Table 4 and Appendix Table 5).

In this large multicenter study evaluating COVID-19 disease severity in more than 19,000 patients presenting for emergency care at US pediatric hospitals, approximately 20% were hospitalized, and among those hospitalized almost a quarter required ICU care. Clinical risk factors associated with increased risk of hospitalization include private payor status and selected comorbidities (obesity/type 2 DM; asthma; and cardiovascular, pulmonary, immunocompromised, neurologic/neuromuscular CCCs), while those associated with decreased risk of hospitalization include older age, female sex, and Black race or Hispanic ethnicity. Factors associated with severe disease among hospitalized pediatric patients include Black or other non-White race, school age (≥5 years), and certain chronic conditions (cardiovascular disease, obesity/type 2 DM, neurologic or neuromuscular disease). Sixteen percent of patients had a concurrent diagnosis for a condition or complication associated with COVID-19.

While the study population (ie, children and adolescents presenting to the ED) represents a small fraction of children and adolescents in the community with SARS-CoV-2 infection, the results provide important insight into factors of severe COVID-19 in the pediatric population. A report from France suggested ventilatory or hemodynamic support or death were independently associated with older age (≥10 years), elevated C-reactive protein, and hypoxemia.¹² An Italian study found that younger age (0-4 years) was associated with less severe disease, while preexisting conditions were more likely in patients with severe disease.¹¹ A single-center case series of 50 patients (aged ≤21 years) hospitalized at a children’s hospital in New York City found respiratory failure (n = 9) was more common in children older than 1 year, patients with elevated inflammatory markers, and patients with obesity.²⁰

Our study confirms several factors for severe COVID-19 found in these studies, including older age,^11,12,20 obesity,²⁰ and preexisting conditions.¹¹ Our findings also expand on these reports, including identification of factors associated with hospitalization. Given the rate of 30-day re-encounters among pediatric patients with COVID-19 (10.1%), identifying risk factors for hospitalization may aid ED providers in determining optimal disposition (eg, home, hospital admission, ICU). We also identified specific comorbidities associated with more severe disease in those hospitalized with COVID-19, such as cardiovascular disease, obesity/type 2 DM, and pulmonary, neurologic, or neuromuscular conditions. We also found that asthma increased the risk for hospitalization but not more severe disease among those hospitalized. This latter finding also aligns with recent single-center studies,^21,22 whereas a Turkish study of pediatric patients aged 0 to 18 years found no association between asthma and COVID-19 hospitalizations.²³We also examined payor type and racial/ethnic factors in our analysis. In 2019, patients who identified as Black or Hispanic comprised 52.3% of all encounters and 40.7% of hospitalizations recorded in the PHIS database. During the same year, encounters for influenza among Black or Hispanic pediatric patients comprised 58.7% of all influenza diagnoses and 47.0% of pediatric influenza hospitalizations (Appendix Table 2). In this study, patients who identified as Black or Hispanic race represented a disproportionately large share of patients presenting to children’s hospitals (68.5%) and of those hospitalized (60.8%). Hispanic ethnicity, in particular, represented a disproportionate share of patients seeking care for COVID-19 compared to the overall PHIS population (47.7% and 27.1%, respectively). After accounting for other factors, we found Black and other non-White race—but not of Hispanic ethnicity—were independently associated with more disease severity among those hospitalized. This contrasts with findings from a recent adult study by Yehia et al,²⁴ who found (after adjusting for other clinical factors) no significant difference in mortality between Black patients and White patients among adults hospitalized due to COVID-19. It also contrasts with a recent large population-based UK study wherein pediatric patients identifying as Asian, but not Black or mixed race or ethnicity, had an increased risk of hospital admission and admission to the ICU compared to children identifying as White. Children identifying as Black or mixed race had longer hospital admissions.²⁵ However, as the authors of the study note, residual confounders and ascertainment bias due to differences in COVID testing may have influenced these findings.

Our findings of differences in hospitalization and disease severity among those hospitalized by race and ethnicity should be interpreted carefully. These may reflect a constellation of factors that are difficult to measure, including differences in healthcare access, inequalities in care (including hospital admission inequalities), and implicit bias—all of which may reflect structural racism. For example, it is possible that children who identify as Black or Hispanic have different access to care compared to children who identify as White, and this may affect disease severity on presentation.² Alternatively, it is possible that White pediatric patients are more likely to be hospitalized as compared to non-White pediatric patients with similar illness severity. Our finding that pediatric patients who identify as Hispanic or Black had a lower risk of hospitalization should be also interpreted carefully, as this may reflect higher utilization of the ED for SARS-CoV-2 testing, increased use of nonemergency services among those without access to primary care, or systematic differences in provider decision-making among this segment of the population.² Further study is needed to determine specific drivers for racial and ethnic differences in healthcare utilization in children and adolescents with COVID-19.²⁶

Complications and co-occurring diagnoses in adults with COVID-19 are well documented.^27-30 However, there is little information to date on the co-occurring diagnoses and complications associated with COVID-19 in children and adolescents. We found that complications and co-occurring conditions occurred in 16.3% of the study population, with the most frequent conditions including known complications of viral infections such as pneumonia, respiratory failure, and seizures. Acute kidney and liver injury, as well as thrombotic complications, occurred less commonly than in adults.^26-29 Interestingly, neurologic complications were also uncommon compared to adult reports^8,31 and less frequent than in other viral illnesses in children and adolescents. For example, neurologic complications occur in approximately 7.5% of children and adolescents hospitalized with influenza.³²

Limitations of the present study include the retrospective design, as well as incomplete patient-level clinical data in the PHIS database. The PHIS database only includes children’s hospitals, which may limit the generalizability of findings to community hospitals. We also excluded newborns, and our findings may not be generalizable to this population. We only included children and adolescents with a primary diagnosis of COVID-19, which has the potential for misclassification in cases where COVID-19 was a secondary diagnosis. However, results of our sensitivity analysis, which incorporated secondary diagnoses of COVID-19, were consistent with findings from our main analyses. Our study was designed to examine associations between certain prespecified factors and COVID-19 severity among pediatric patients who visited the ED or were admitted to the hospital during the COVID-19 pandemic. Thus, our findings must be interpreted in light of these considerations and may not be generalizable outside the ED or hospital setting. For example, it could be that some segments of the population utilized ED resources for testing, whereas others avoided the ED and other healthcare settings for fear of exposure to SARS-CoV-2. We also relied on diagnosis codes to identify concurrent diagnoses, as well as mechanical ventilation in our very severe outcome cohort, which resulted in this classification for some of these diagnoses. Despite these limitations, our findings represent an important step in understanding the risk factors associated with severe clinical COVID-19 disease in pediatric patients.

Our findings may inform future research and clinical interventions. Future studies on antiviral therapies and immune modulators targeting SARS-CoV-2 infection in children and adolescents should focus on high-risk populations, such as those identified in the study, as these patients are most likely to benefit from therapeutic interventions. Similarly, vaccine-development efforts may benefit from additional evaluation in high-risk populations, some of which may have altered immune responses. Furthermore, with increasing vaccination among adults and changes in recommendations, societal mitigation efforts (eg, masking, physical distancing) will diminish. Continued vigilance and COVID-19–mitigation efforts among high-risk children, for whom vaccines are not yet available, are critical during this transition.

Among children with COVID-19 who received care at children’s hospitals and EDs, 20% were hospitalized, and, of those, 21% were admitted to the ICU. Older children and adolescent patients had a lower risk of hospitalization; however, when hospitalized, they had greater illness severity. Those with selected comorbidities (eg, cardiovascular, obesity/type 2 DM, pulmonary and neurologic or neuromuscular disease) had both increased odds of hospitalization and in-hospital illness severity. While there were observed differences in COVID-19 severity by race and ethnicity, additional research is needed to clarify the drivers of such disparities. These factors should be considered when prioritizing mitigation strategies to prevent infection (eg, remote learning, avoidance of group activities, prioritization of COVID-19 vaccine when approved for children aged <12 years).

The COVID-19 pandemic has led to more than 40 million infections and more than 650,000 deaths in the United States alone.¹ Morbidity and mortality have disproportionately affected older adults.^2-4 However, acute infection and delayed effects, such as multisystem inflammatory syndrome in children (MIS-C), occur and can lead to severe complications, hospitalization, and death in pediatric patients.^5,6 Due to higher clinical disease prevalence and morbidity in the adult population, we have learned much about the clinical factors associated with severe adult COVID-19 disease.^5,7-9 Such clinical factors include older age, concurrent comorbidities, smoke exposure, and Black race or Hispanic ethnicity, among others.^5,7-10 However, there is a paucity of data on severe COVID-19 disease in pediatric patients.^5,11,12 In addition, most immunization strategies and pharmacologic treatments for COVID-19 have not been evaluated or approved for use in children.¹³ To guide targeted prevention and treatment strategies, there is a critical need to identify children and adolescents—who are among the most vulnerable patient populations—at high risk for severe disease.

Identifying the clinical factors associated with severe COVID-19 disease will help with prioritizing and allocating vaccines when they are approved for use in patients younger than 12 years. It also can provide insight for clinicians and families faced with decisions wherein individual risk assessment is crucial (eg, in-person schooling, other group activities). The objective of this study was to determine the clinical factors associated with severe COVID-19 among children and adolescents in the United States.

Study Design

We conducted a multicenter retrospective cohort study of patients presenting for care at pediatric hospitals that report data to the Pediatric Health Information System (PHIS) database. The PHIS administrative database includes billing and utilization data from 45 US tertiary care hospitals affiliated with the Children’s Hospital Association (Lenexa, Kansas). Data quality and reliability are ensured through a joint validation effort between the Children’s Hospital Association and participating hospitals. Hospitals submit discharge data, including demographics, diagnoses, and procedures using International Classification of Diseases, 10th Revision (ICD-10) codes, along with daily detailed information on pharmacy, location of care, and other services.

Study Population

Patients 30 days to 18 years of age discharged from the emergency department (ED) or inpatient setting with a primary diagnosis of COVID-19 (ICD-10 codes U.071 and U.072) between April 1, 2020, and September 30, 2020, were eligible for inclusion.¹⁴ In a prior study, the positive predictive value of an ICD-10–coded diagnosis of COVID-19 among hospitalized pediatric patients was 95.5%, compared with reverse transcription polymerase reaction results or presence of MIS-C.¹⁵ The diagnostic code for COVID-19 (ICD-10-CM) also had a high sensitivity (98.0%) in the hospitalized population.¹⁶ Acknowledging the increasing practice of screening patients upon admission, and in an attempt to minimize potential misclassification, we did not include encounters with secondary diagnoses of COVID-19 in our primary analyses. Pediatric patients with surgical diagnoses and neonates who never left the hospital were also excluded.

Factors Associated With Severe COVID-19 Disease

Exposures of interest were determined a priori based on current evidence in the literature and included patient age (0-4 years, 5-11 years, and 12-18 years), sex, race and ethnicity (non-Hispanic White, non-Hispanic Black, Hispanic, Asian, other non-White race [defined as Pacific Islander, Native American, or other]), payor type, cardiovascular complex chronic conditions (CCC), neuromuscular CCC, obesity/type 2 diabetes mellitus (DM), pulmonary CCC, asthma (defined using ICD-10 codes¹⁷), and immunocompromised CCC. Race and ethnicity were included as covariates based on previous studies reporting differences in COVID-19 outcomes among racial and ethnic groups.⁹ The CCC covariates were defined using the pediatric CCC ICD-10 classification system version 2.¹⁸

Pediatric Complications and Conditions Associated With COVID-19

Based on current evidence and expert opinion of study members, associated diagnoses and complications co-occurring with a COVID-19 diagnosis were defined a priori and identified through ICD-10 codes (Appendix Table 1). These included acute kidney injury, acute liver injury, aseptic meningitis, asthma exacerbation, bronchiolitis, cerebral infarction, croup, encephalitis, encephalopathy, infant fever, febrile seizure, gastroenteritis/dehydration, Kawasaki disease/MIS-C, myocarditis/pericarditis, pneumonia, lung effusion or empyema, respiratory failure, sepsis, nonfebrile seizure, pancreatitis, sickle cell complications, and thrombotic complications.

Outcomes

COVID-19 severity outcomes were assessed as follows: (1) mild = ED discharge; (2) moderate = inpatient admission; (3) severe = intensive care unit (ICU) admission without mechanical ventilation, shock, or death; and (4) very severe = ICU admission with mechanical ventilation, shock, or death.¹⁹ This ordinal ranking system did not violate the proportional odds assumption. Potential reasons for admission to the ICU without mechanical ventilation, shock, or death include, but are not limited to, need for noninvasive ventilation, vital sign instability, dysrhythmias, respiratory insufficiency, or complications arising from concurrent conditions (eg, thrombotic events, need for continuous albuterol therapy). We examined several secondary, hospital-based outcomes, including associated diagnoses and complications, all-cause 30-day healthcare reutilization (ED visit or rehospitalization), length of stay (LOS), and ICU LOS.

Statistical Analysis

Demographic characteristics were summarized using frequencies and percentages for categorical variables and geometric means with SD and medians with interquartile ranges (IQR) for continuous variables, as appropriate. Factors associated with hospitalization (encompassing severity levels 2-4) vs ED discharge (severity level 1) were assessed using logistic regression. Factors associated with increasing severity among hospitalized pediatric patients (severity levels 2, 3, and 4) were assessed using ordinal logistic regression. Covariates in these analyses included race and ethnicity, age, sex, payor, cardiovascular CCC, neurologic/neuromuscular CCC, obesity/type 2 DM, pulmonary CCC, asthma, and immunocompromised CCC. Adjusted odds ratios (aOR) and corresponding 95% CI for each risk factor were generated using generalized linear mixed effects models and random intercepts for each hospital. Given the potential for diagnostic misclassification of pediatric patients with COVID-19 based on primary vs secondary diagnoses, we performed sensitivity analyses defining the study population as those with a primary diagnosis of COVID-19 and those with a secondary diagnosis of COVID-19 plus a concurrent primary diagnosis of a condition associated with COVID-19 (Appendix Table 1).

All analyses were performed using SAS version 9.4 (SAS Institute, Inc), and P < .05 was considered statistically significant. The Institutional Review Board at Vanderbilt University Medical Center determined that this study of de-identified data did not meet the criteria for human subjects research.

Study Population

A total of 19,976 encounters were included in the study. Of those, 15,913 (79.7%) were discharged from the ED and 4063 (20.3%) were hospitalized (Table 1). The most common race/ethnicity was Hispanic (9741, 48.8%), followed by non-Hispanic White (4217, 21.1%). Reference race/ethnicity data for the overall 2019 PHIS population can be found in Appendix Table 2.

The severity distribution among the hospitalized population was moderate (3222, 79.3%), severe (431, 11.3%), and very severe (380, 9.4%). The frequency of COVID-19 diagnoses increased late in the study period (Figure). Among those hospitalized, the median LOS for the index admission was 2 days (IQR, 1-4), while among those admitted to the ICU, the median LOS was 3 days (IQR, 2-5).

Overall, 10.1% (n = 2020) of the study population had an all-cause repeat encounter (ie, subsequent ED encounter or hospitalization) within 30 days following the index discharge. Repeat encounters were more frequent among patients hospitalized than among those discharged from the ED (Appendix Table 3).

Prevalence of Conditions and Complications Associated With COVID-19

Overall, 3257 (16.3%) patients had one or more co-occurring diagnoses categorized as a COVID-19–associated condition or complication. The most frequent diagnoses included lower respiratory tract disease (pneumonia, lung effusion, or empyema; n = 1415, 7.1%), gastroenteritis/dehydration (n = 1068, 5.3%), respiratory failure (n = 731, 3.7%), febrile infant (n = 413, 2.1%), and nonfebrile seizure (n = 425, 2.1%). Aside from nonfebrile seizure, neurological complications were less frequent and included febrile seizure (n = 155, 0.8%), encephalopathy (n = 63, 0.3%), aseptic meningitis (n = 16, 0.1%), encephalitis (n = 11, 0.1%), and cerebral infarction (n = 6, <0.1%). Kawasaki disease and MIS-C comprised 1.7% (n = 346) of diagnoses. Thrombotic complications occurred in 0.1% (n = 13) of patients. Overall, these conditions and complications associated with COVID-19 were more frequent in hospitalized patients than in those discharged from the ED (P < .001) (Table 2).

Factors Associated With COVID-19 Disease Severity

Compared to pediatric patients with COVID-19 discharged from the ED, factors associated with increased odds of hospitalization included private payor insurance; obesity/type 2 DM; asthma; and cardiovascular, immunocompromised, neurologic/neuromuscular, and pulmonary CCCs (Table 3). Factors associated with decreased risk of hospitalization included Black race or Hispanic ethnicity compared with White race; female sex; and age 5 to 11 years and age 12 to 17 years (vs age 0-4 years). Among children and adolescents hospitalized with COVID-19, factors associated with greater disease severity included Black or other non-White race; age 5 to 11 years; age 12 to 17 years; obesity/type 2 DM; immunocompromised conditions; and cardiovascular, neurologic/neuromuscular, and pulmonary CCCs (Table 3).

Sensitivity Analysis

We performed a sensitivity analysis that expanded the study population to include those with a secondary diagnosis of COVID-19 plus a diagnosis of a COVID-19–associated condition or complication. Analyses using the expanded population (N = 21,247) were similar to the primary analyses (Appendix Table 4 and Appendix Table 5).

In this large multicenter study evaluating COVID-19 disease severity in more than 19,000 patients presenting for emergency care at US pediatric hospitals, approximately 20% were hospitalized, and among those hospitalized almost a quarter required ICU care. Clinical risk factors associated with increased risk of hospitalization include private payor status and selected comorbidities (obesity/type 2 DM; asthma; and cardiovascular, pulmonary, immunocompromised, neurologic/neuromuscular CCCs), while those associated with decreased risk of hospitalization include older age, female sex, and Black race or Hispanic ethnicity. Factors associated with severe disease among hospitalized pediatric patients include Black or other non-White race, school age (≥5 years), and certain chronic conditions (cardiovascular disease, obesity/type 2 DM, neurologic or neuromuscular disease). Sixteen percent of patients had a concurrent diagnosis for a condition or complication associated with COVID-19.

While the study population (ie, children and adolescents presenting to the ED) represents a small fraction of children and adolescents in the community with SARS-CoV-2 infection, the results provide important insight into factors of severe COVID-19 in the pediatric population. A report from France suggested ventilatory or hemodynamic support or death were independently associated with older age (≥10 years), elevated C-reactive protein, and hypoxemia.¹² An Italian study found that younger age (0-4 years) was associated with less severe disease, while preexisting conditions were more likely in patients with severe disease.¹¹ A single-center case series of 50 patients (aged ≤21 years) hospitalized at a children’s hospital in New York City found respiratory failure (n = 9) was more common in children older than 1 year, patients with elevated inflammatory markers, and patients with obesity.²⁰

Our study confirms several factors for severe COVID-19 found in these studies, including older age,^11,12,20 obesity,²⁰ and preexisting conditions.¹¹ Our findings also expand on these reports, including identification of factors associated with hospitalization. Given the rate of 30-day re-encounters among pediatric patients with COVID-19 (10.1%), identifying risk factors for hospitalization may aid ED providers in determining optimal disposition (eg, home, hospital admission, ICU). We also identified specific comorbidities associated with more severe disease in those hospitalized with COVID-19, such as cardiovascular disease, obesity/type 2 DM, and pulmonary, neurologic, or neuromuscular conditions. We also found that asthma increased the risk for hospitalization but not more severe disease among those hospitalized. This latter finding also aligns with recent single-center studies,^21,22 whereas a Turkish study of pediatric patients aged 0 to 18 years found no association between asthma and COVID-19 hospitalizations.²³We also examined payor type and racial/ethnic factors in our analysis. In 2019, patients who identified as Black or Hispanic comprised 52.3% of all encounters and 40.7% of hospitalizations recorded in the PHIS database. During the same year, encounters for influenza among Black or Hispanic pediatric patients comprised 58.7% of all influenza diagnoses and 47.0% of pediatric influenza hospitalizations (Appendix Table 2). In this study, patients who identified as Black or Hispanic race represented a disproportionately large share of patients presenting to children’s hospitals (68.5%) and of those hospitalized (60.8%). Hispanic ethnicity, in particular, represented a disproportionate share of patients seeking care for COVID-19 compared to the overall PHIS population (47.7% and 27.1%, respectively). After accounting for other factors, we found Black and other non-White race—but not of Hispanic ethnicity—were independently associated with more disease severity among those hospitalized. This contrasts with findings from a recent adult study by Yehia et al,²⁴ who found (after adjusting for other clinical factors) no significant difference in mortality between Black patients and White patients among adults hospitalized due to COVID-19. It also contrasts with a recent large population-based UK study wherein pediatric patients identifying as Asian, but not Black or mixed race or ethnicity, had an increased risk of hospital admission and admission to the ICU compared to children identifying as White. Children identifying as Black or mixed race had longer hospital admissions.²⁵ However, as the authors of the study note, residual confounders and ascertainment bias due to differences in COVID testing may have influenced these findings.

Our findings of differences in hospitalization and disease severity among those hospitalized by race and ethnicity should be interpreted carefully. These may reflect a constellation of factors that are difficult to measure, including differences in healthcare access, inequalities in care (including hospital admission inequalities), and implicit bias—all of which may reflect structural racism. For example, it is possible that children who identify as Black or Hispanic have different access to care compared to children who identify as White, and this may affect disease severity on presentation.² Alternatively, it is possible that White pediatric patients are more likely to be hospitalized as compared to non-White pediatric patients with similar illness severity. Our finding that pediatric patients who identify as Hispanic or Black had a lower risk of hospitalization should be also interpreted carefully, as this may reflect higher utilization of the ED for SARS-CoV-2 testing, increased use of nonemergency services among those without access to primary care, or systematic differences in provider decision-making among this segment of the population.² Further study is needed to determine specific drivers for racial and ethnic differences in healthcare utilization in children and adolescents with COVID-19.²⁶

Complications and co-occurring diagnoses in adults with COVID-19 are well documented.^27-30 However, there is little information to date on the co-occurring diagnoses and complications associated with COVID-19 in children and adolescents. We found that complications and co-occurring conditions occurred in 16.3% of the study population, with the most frequent conditions including known complications of viral infections such as pneumonia, respiratory failure, and seizures. Acute kidney and liver injury, as well as thrombotic complications, occurred less commonly than in adults.^26-29 Interestingly, neurologic complications were also uncommon compared to adult reports^8,31 and less frequent than in other viral illnesses in children and adolescents. For example, neurologic complications occur in approximately 7.5% of children and adolescents hospitalized with influenza.³²

Limitations of the present study include the retrospective design, as well as incomplete patient-level clinical data in the PHIS database. The PHIS database only includes children’s hospitals, which may limit the generalizability of findings to community hospitals. We also excluded newborns, and our findings may not be generalizable to this population. We only included children and adolescents with a primary diagnosis of COVID-19, which has the potential for misclassification in cases where COVID-19 was a secondary diagnosis. However, results of our sensitivity analysis, which incorporated secondary diagnoses of COVID-19, were consistent with findings from our main analyses. Our study was designed to examine associations between certain prespecified factors and COVID-19 severity among pediatric patients who visited the ED or were admitted to the hospital during the COVID-19 pandemic. Thus, our findings must be interpreted in light of these considerations and may not be generalizable outside the ED or hospital setting. For example, it could be that some segments of the population utilized ED resources for testing, whereas others avoided the ED and other healthcare settings for fear of exposure to SARS-CoV-2. We also relied on diagnosis codes to identify concurrent diagnoses, as well as mechanical ventilation in our very severe outcome cohort, which resulted in this classification for some of these diagnoses. Despite these limitations, our findings represent an important step in understanding the risk factors associated with severe clinical COVID-19 disease in pediatric patients.

Our findings may inform future research and clinical interventions. Future studies on antiviral therapies and immune modulators targeting SARS-CoV-2 infection in children and adolescents should focus on high-risk populations, such as those identified in the study, as these patients are most likely to benefit from therapeutic interventions. Similarly, vaccine-development efforts may benefit from additional evaluation in high-risk populations, some of which may have altered immune responses. Furthermore, with increasing vaccination among adults and changes in recommendations, societal mitigation efforts (eg, masking, physical distancing) will diminish. Continued vigilance and COVID-19–mitigation efforts among high-risk children, for whom vaccines are not yet available, are critical during this transition.

Among children with COVID-19 who received care at children’s hospitals and EDs, 20% were hospitalized, and, of those, 21% were admitted to the ICU. Older children and adolescent patients had a lower risk of hospitalization; however, when hospitalized, they had greater illness severity. Those with selected comorbidities (eg, cardiovascular, obesity/type 2 DM, pulmonary and neurologic or neuromuscular disease) had both increased odds of hospitalization and in-hospital illness severity. While there were observed differences in COVID-19 severity by race and ethnicity, additional research is needed to clarify the drivers of such disparities. These factors should be considered when prioritizing mitigation strategies to prevent infection (eg, remote learning, avoidance of group activities, prioritization of COVID-19 vaccine when approved for children aged <12 years).

The COVID-19 pandemic has led to more than 40 million infections and more than 650,000 deaths in the United States alone.¹ Morbidity and mortality have disproportionately affected older adults.^2-4 However, acute infection and delayed effects, such as multisystem inflammatory syndrome in children (MIS-C), occur and can lead to severe complications, hospitalization, and death in pediatric patients.^5,6 Due to higher clinical disease prevalence and morbidity in the adult population, we have learned much about the clinical factors associated with severe adult COVID-19 disease.^5,7-9 Such clinical factors include older age, concurrent comorbidities, smoke exposure, and Black race or Hispanic ethnicity, among others.^5,7-10 However, there is a paucity of data on severe COVID-19 disease in pediatric patients.^5,11,12 In addition, most immunization strategies and pharmacologic treatments for COVID-19 have not been evaluated or approved for use in children.¹³ To guide targeted prevention and treatment strategies, there is a critical need to identify children and adolescents—who are among the most vulnerable patient populations—at high risk for severe disease.

Identifying the clinical factors associated with severe COVID-19 disease will help with prioritizing and allocating vaccines when they are approved for use in patients younger than 12 years. It also can provide insight for clinicians and families faced with decisions wherein individual risk assessment is crucial (eg, in-person schooling, other group activities). The objective of this study was to determine the clinical factors associated with severe COVID-19 among children and adolescents in the United States.

Study Design

We conducted a multicenter retrospective cohort study of patients presenting for care at pediatric hospitals that report data to the Pediatric Health Information System (PHIS) database. The PHIS administrative database includes billing and utilization data from 45 US tertiary care hospitals affiliated with the Children’s Hospital Association (Lenexa, Kansas). Data quality and reliability are ensured through a joint validation effort between the Children’s Hospital Association and participating hospitals. Hospitals submit discharge data, including demographics, diagnoses, and procedures using International Classification of Diseases, 10th Revision (ICD-10) codes, along with daily detailed information on pharmacy, location of care, and other services.

Study Population

Patients 30 days to 18 years of age discharged from the emergency department (ED) or inpatient setting with a primary diagnosis of COVID-19 (ICD-10 codes U.071 and U.072) between April 1, 2020, and September 30, 2020, were eligible for inclusion.¹⁴ In a prior study, the positive predictive value of an ICD-10–coded diagnosis of COVID-19 among hospitalized pediatric patients was 95.5%, compared with reverse transcription polymerase reaction results or presence of MIS-C.¹⁵ The diagnostic code for COVID-19 (ICD-10-CM) also had a high sensitivity (98.0%) in the hospitalized population.¹⁶ Acknowledging the increasing practice of screening patients upon admission, and in an attempt to minimize potential misclassification, we did not include encounters with secondary diagnoses of COVID-19 in our primary analyses. Pediatric patients with surgical diagnoses and neonates who never left the hospital were also excluded.

Factors Associated With Severe COVID-19 Disease

Exposures of interest were determined a priori based on current evidence in the literature and included patient age (0-4 years, 5-11 years, and 12-18 years), sex, race and ethnicity (non-Hispanic White, non-Hispanic Black, Hispanic, Asian, other non-White race [defined as Pacific Islander, Native American, or other]), payor type, cardiovascular complex chronic conditions (CCC), neuromuscular CCC, obesity/type 2 diabetes mellitus (DM), pulmonary CCC, asthma (defined using ICD-10 codes¹⁷), and immunocompromised CCC. Race and ethnicity were included as covariates based on previous studies reporting differences in COVID-19 outcomes among racial and ethnic groups.⁹ The CCC covariates were defined using the pediatric CCC ICD-10 classification system version 2.¹⁸

Pediatric Complications and Conditions Associated With COVID-19

Based on current evidence and expert opinion of study members, associated diagnoses and complications co-occurring with a COVID-19 diagnosis were defined a priori and identified through ICD-10 codes (Appendix Table 1). These included acute kidney injury, acute liver injury, aseptic meningitis, asthma exacerbation, bronchiolitis, cerebral infarction, croup, encephalitis, encephalopathy, infant fever, febrile seizure, gastroenteritis/dehydration, Kawasaki disease/MIS-C, myocarditis/pericarditis, pneumonia, lung effusion or empyema, respiratory failure, sepsis, nonfebrile seizure, pancreatitis, sickle cell complications, and thrombotic complications.

Outcomes

COVID-19 severity outcomes were assessed as follows: (1) mild = ED discharge; (2) moderate = inpatient admission; (3) severe = intensive care unit (ICU) admission without mechanical ventilation, shock, or death; and (4) very severe = ICU admission with mechanical ventilation, shock, or death.¹⁹ This ordinal ranking system did not violate the proportional odds assumption. Potential reasons for admission to the ICU without mechanical ventilation, shock, or death include, but are not limited to, need for noninvasive ventilation, vital sign instability, dysrhythmias, respiratory insufficiency, or complications arising from concurrent conditions (eg, thrombotic events, need for continuous albuterol therapy). We examined several secondary, hospital-based outcomes, including associated diagnoses and complications, all-cause 30-day healthcare reutilization (ED visit or rehospitalization), length of stay (LOS), and ICU LOS.

Statistical Analysis

Demographic characteristics were summarized using frequencies and percentages for categorical variables and geometric means with SD and medians with interquartile ranges (IQR) for continuous variables, as appropriate. Factors associated with hospitalization (encompassing severity levels 2-4) vs ED discharge (severity level 1) were assessed using logistic regression. Factors associated with increasing severity among hospitalized pediatric patients (severity levels 2, 3, and 4) were assessed using ordinal logistic regression. Covariates in these analyses included race and ethnicity, age, sex, payor, cardiovascular CCC, neurologic/neuromuscular CCC, obesity/type 2 DM, pulmonary CCC, asthma, and immunocompromised CCC. Adjusted odds ratios (aOR) and corresponding 95% CI for each risk factor were generated using generalized linear mixed effects models and random intercepts for each hospital. Given the potential for diagnostic misclassification of pediatric patients with COVID-19 based on primary vs secondary diagnoses, we performed sensitivity analyses defining the study population as those with a primary diagnosis of COVID-19 and those with a secondary diagnosis of COVID-19 plus a concurrent primary diagnosis of a condition associated with COVID-19 (Appendix Table 1).

All analyses were performed using SAS version 9.4 (SAS Institute, Inc), and P < .05 was considered statistically significant. The Institutional Review Board at Vanderbilt University Medical Center determined that this study of de-identified data did not meet the criteria for human subjects research.

Study Population

A total of 19,976 encounters were included in the study. Of those, 15,913 (79.7%) were discharged from the ED and 4063 (20.3%) were hospitalized (Table 1). The most common race/ethnicity was Hispanic (9741, 48.8%), followed by non-Hispanic White (4217, 21.1%). Reference race/ethnicity data for the overall 2019 PHIS population can be found in Appendix Table 2.

The severity distribution among the hospitalized population was moderate (3222, 79.3%), severe (431, 11.3%), and very severe (380, 9.4%). The frequency of COVID-19 diagnoses increased late in the study period (Figure). Among those hospitalized, the median LOS for the index admission was 2 days (IQR, 1-4), while among those admitted to the ICU, the median LOS was 3 days (IQR, 2-5).

Overall, 10.1% (n = 2020) of the study population had an all-cause repeat encounter (ie, subsequent ED encounter or hospitalization) within 30 days following the index discharge. Repeat encounters were more frequent among patients hospitalized than among those discharged from the ED (Appendix Table 3).

Prevalence of Conditions and Complications Associated With COVID-19

Overall, 3257 (16.3%) patients had one or more co-occurring diagnoses categorized as a COVID-19–associated condition or complication. The most frequent diagnoses included lower respiratory tract disease (pneumonia, lung effusion, or empyema; n = 1415, 7.1%), gastroenteritis/dehydration (n = 1068, 5.3%), respiratory failure (n = 731, 3.7%), febrile infant (n = 413, 2.1%), and nonfebrile seizure (n = 425, 2.1%). Aside from nonfebrile seizure, neurological complications were less frequent and included febrile seizure (n = 155, 0.8%), encephalopathy (n = 63, 0.3%), aseptic meningitis (n = 16, 0.1%), encephalitis (n = 11, 0.1%), and cerebral infarction (n = 6, <0.1%). Kawasaki disease and MIS-C comprised 1.7% (n = 346) of diagnoses. Thrombotic complications occurred in 0.1% (n = 13) of patients. Overall, these conditions and complications associated with COVID-19 were more frequent in hospitalized patients than in those discharged from the ED (P < .001) (Table 2).

Factors Associated With COVID-19 Disease Severity

Compared to pediatric patients with COVID-19 discharged from the ED, factors associated with increased odds of hospitalization included private payor insurance; obesity/type 2 DM; asthma; and cardiovascular, immunocompromised, neurologic/neuromuscular, and pulmonary CCCs (Table 3). Factors associated with decreased risk of hospitalization included Black race or Hispanic ethnicity compared with White race; female sex; and age 5 to 11 years and age 12 to 17 years (vs age 0-4 years). Among children and adolescents hospitalized with COVID-19, factors associated with greater disease severity included Black or other non-White race; age 5 to 11 years; age 12 to 17 years; obesity/type 2 DM; immunocompromised conditions; and cardiovascular, neurologic/neuromuscular, and pulmonary CCCs (Table 3).

Sensitivity Analysis

We performed a sensitivity analysis that expanded the study population to include those with a secondary diagnosis of COVID-19 plus a diagnosis of a COVID-19–associated condition or complication. Analyses using the expanded population (N = 21,247) were similar to the primary analyses (Appendix Table 4 and Appendix Table 5).

In this large multicenter study evaluating COVID-19 disease severity in more than 19,000 patients presenting for emergency care at US pediatric hospitals, approximately 20% were hospitalized, and among those hospitalized almost a quarter required ICU care. Clinical risk factors associated with increased risk of hospitalization include private payor status and selected comorbidities (obesity/type 2 DM; asthma; and cardiovascular, pulmonary, immunocompromised, neurologic/neuromuscular CCCs), while those associated with decreased risk of hospitalization include older age, female sex, and Black race or Hispanic ethnicity. Factors associated with severe disease among hospitalized pediatric patients include Black or other non-White race, school age (≥5 years), and certain chronic conditions (cardiovascular disease, obesity/type 2 DM, neurologic or neuromuscular disease). Sixteen percent of patients had a concurrent diagnosis for a condition or complication associated with COVID-19.

While the study population (ie, children and adolescents presenting to the ED) represents a small fraction of children and adolescents in the community with SARS-CoV-2 infection, the results provide important insight into factors of severe COVID-19 in the pediatric population. A report from France suggested ventilatory or hemodynamic support or death were independently associated with older age (≥10 years), elevated C-reactive protein, and hypoxemia.¹² An Italian study found that younger age (0-4 years) was associated with less severe disease, while preexisting conditions were more likely in patients with severe disease.¹¹ A single-center case series of 50 patients (aged ≤21 years) hospitalized at a children’s hospital in New York City found respiratory failure (n = 9) was more common in children older than 1 year, patients with elevated inflammatory markers, and patients with obesity.²⁰

Our study confirms several factors for severe COVID-19 found in these studies, including older age,^11,12,20 obesity,²⁰ and preexisting conditions.¹¹ Our findings also expand on these reports, including identification of factors associated with hospitalization. Given the rate of 30-day re-encounters among pediatric patients with COVID-19 (10.1%), identifying risk factors for hospitalization may aid ED providers in determining optimal disposition (eg, home, hospital admission, ICU). We also identified specific comorbidities associated with more severe disease in those hospitalized with COVID-19, such as cardiovascular disease, obesity/type 2 DM, and pulmonary, neurologic, or neuromuscular conditions. We also found that asthma increased the risk for hospitalization but not more severe disease among those hospitalized. This latter finding also aligns with recent single-center studies,^21,22 whereas a Turkish study of pediatric patients aged 0 to 18 years found no association between asthma and COVID-19 hospitalizations.²³We also examined payor type and racial/ethnic factors in our analysis. In 2019, patients who identified as Black or Hispanic comprised 52.3% of all encounters and 40.7% of hospitalizations recorded in the PHIS database. During the same year, encounters for influenza among Black or Hispanic pediatric patients comprised 58.7% of all influenza diagnoses and 47.0% of pediatric influenza hospitalizations (Appendix Table 2). In this study, patients who identified as Black or Hispanic race represented a disproportionately large share of patients presenting to children’s hospitals (68.5%) and of those hospitalized (60.8%). Hispanic ethnicity, in particular, represented a disproportionate share of patients seeking care for COVID-19 compared to the overall PHIS population (47.7% and 27.1%, respectively). After accounting for other factors, we found Black and other non-White race—but not of Hispanic ethnicity—were independently associated with more disease severity among those hospitalized. This contrasts with findings from a recent adult study by Yehia et al,²⁴ who found (after adjusting for other clinical factors) no significant difference in mortality between Black patients and White patients among adults hospitalized due to COVID-19. It also contrasts with a recent large population-based UK study wherein pediatric patients identifying as Asian, but not Black or mixed race or ethnicity, had an increased risk of hospital admission and admission to the ICU compared to children identifying as White. Children identifying as Black or mixed race had longer hospital admissions.²⁵ However, as the authors of the study note, residual confounders and ascertainment bias due to differences in COVID testing may have influenced these findings.

Our findings of differences in hospitalization and disease severity among those hospitalized by race and ethnicity should be interpreted carefully. These may reflect a constellation of factors that are difficult to measure, including differences in healthcare access, inequalities in care (including hospital admission inequalities), and implicit bias—all of which may reflect structural racism. For example, it is possible that children who identify as Black or Hispanic have different access to care compared to children who identify as White, and this may affect disease severity on presentation.² Alternatively, it is possible that White pediatric patients are more likely to be hospitalized as compared to non-White pediatric patients with similar illness severity. Our finding that pediatric patients who identify as Hispanic or Black had a lower risk of hospitalization should be also interpreted carefully, as this may reflect higher utilization of the ED for SARS-CoV-2 testing, increased use of nonemergency services among those without access to primary care, or systematic differences in provider decision-making among this segment of the population.² Further study is needed to determine specific drivers for racial and ethnic differences in healthcare utilization in children and adolescents with COVID-19.²⁶

Complications and co-occurring diagnoses in adults with COVID-19 are well documented.^27-30 However, there is little information to date on the co-occurring diagnoses and complications associated with COVID-19 in children and adolescents. We found that complications and co-occurring conditions occurred in 16.3% of the study population, with the most frequent conditions including known complications of viral infections such as pneumonia, respiratory failure, and seizures. Acute kidney and liver injury, as well as thrombotic complications, occurred less commonly than in adults.^26-29 Interestingly, neurologic complications were also uncommon compared to adult reports^8,31 and less frequent than in other viral illnesses in children and adolescents. For example, neurologic complications occur in approximately 7.5% of children and adolescents hospitalized with influenza.³²

Limitations of the present study include the retrospective design, as well as incomplete patient-level clinical data in the PHIS database. The PHIS database only includes children’s hospitals, which may limit the generalizability of findings to community hospitals. We also excluded newborns, and our findings may not be generalizable to this population. We only included children and adolescents with a primary diagnosis of COVID-19, which has the potential for misclassification in cases where COVID-19 was a secondary diagnosis. However, results of our sensitivity analysis, which incorporated secondary diagnoses of COVID-19, were consistent with findings from our main analyses. Our study was designed to examine associations between certain prespecified factors and COVID-19 severity among pediatric patients who visited the ED or were admitted to the hospital during the COVID-19 pandemic. Thus, our findings must be interpreted in light of these considerations and may not be generalizable outside the ED or hospital setting. For example, it could be that some segments of the population utilized ED resources for testing, whereas others avoided the ED and other healthcare settings for fear of exposure to SARS-CoV-2. We also relied on diagnosis codes to identify concurrent diagnoses, as well as mechanical ventilation in our very severe outcome cohort, which resulted in this classification for some of these diagnoses. Despite these limitations, our findings represent an important step in understanding the risk factors associated with severe clinical COVID-19 disease in pediatric patients.

Our findings may inform future research and clinical interventions. Future studies on antiviral therapies and immune modulators targeting SARS-CoV-2 infection in children and adolescents should focus on high-risk populations, such as those identified in the study, as these patients are most likely to benefit from therapeutic interventions. Similarly, vaccine-development efforts may benefit from additional evaluation in high-risk populations, some of which may have altered immune responses. Furthermore, with increasing vaccination among adults and changes in recommendations, societal mitigation efforts (eg, masking, physical distancing) will diminish. Continued vigilance and COVID-19–mitigation efforts among high-risk children, for whom vaccines are not yet available, are critical during this transition.

Among children with COVID-19 who received care at children’s hospitals and EDs, 20% were hospitalized, and, of those, 21% were admitted to the ICU. Older children and adolescent patients had a lower risk of hospitalization; however, when hospitalized, they had greater illness severity. Those with selected comorbidities (eg, cardiovascular, obesity/type 2 DM, pulmonary and neurologic or neuromuscular disease) had both increased odds of hospitalization and in-hospital illness severity. While there were observed differences in COVID-19 severity by race and ethnicity, additional research is needed to clarify the drivers of such disparities. These factors should be considered when prioritizing mitigation strategies to prevent infection (eg, remote learning, avoidance of group activities, prioritization of COVID-19 vaccine when approved for children aged <12 years).

In the United States, respiratory illnesses are the most common cause of emergency department (ED) visits and hospitalizations in children.¹ In response to the ongoing COVID-19 pandemic, several public health interventions, including school and business closures, stay-at-home orders, and mask mandates, were implemented to limit transmission of SARS-CoV-2.^2,3 Studies have shown that children can contribute to the spread of SARS-CoV-2 infections, especially within households.^4-6 Recent data suggest that COVID-19, and the associated public health measures enacted to slow its spread, may have affected the transmission of other respiratory pathogens.⁷ Similarly, the pandemic has likely affected healthcare utilization for nonrespiratory illnesses through adoption of social distancing recommendations, suspension and delays in nonemergent elective care, avoidance of healthcare settings, and the effect of decreased respiratory disease on exacerbation of chronic illness.⁸ The objective of this study was to examine associations between the COVID-19 pandemic and healthcare utilization for pediatric respiratory and nonrespiratory illnesses at US pediatric hospitals.

Study Design

This is a multicenter, cross-sectional study of encounters at 44 pediatric hospitals that reported data to the Pediatric Health Information System (PHIS) database maintained by the Children’s Hospital Association (Lenexa, Kansas).

Study Population

Children 2 months to 18 years of age discharged from ED or inpatient settings with a nonsurgical diagnosis from January 1 to September 30 over a 4-year period (2017-2020) were included.

Exposure

The primary exposure was the 2020 COVID-19 pandemic time, divided into three periods: pre-COVID-19 (January-February 2020, the period prior to the pandemic in the United States), early COVID-19 (March-April 2020, coinciding with the first reported US pediatric case of COVID-19 on March 2, 2020), and COVID-19 (May-September 2020, marked by the implementation of at least two of the following containment measures in every US state: stay-at-home/shelter orders, school closures, nonessential business closures, restaurant closures, or prohibition of gatherings of more than 10 people).²

Outcomes

Respiratory illness diagnoses were classified into mutually exclusive subgroups following a prespecified hierarchy: influenza, pneumonia, croup, bronchiolitis, asthma, unspecified influenza-like illness, and “other respiratory diagnoses” (Appendix Table 1). To assess the impact of COVID-19 after its International Classification of Diseases, Tenth Revision code was established on March 25, 2020, the “other respiratory” subgroup was divided into other respiratory illnesses with and without COVID-19. Nonrespiratory illness diagnoses were defined as all diagnoses not included in the respiratory illness cohort.

Statistical Analysis

Categorical variables were summarized using frequencies and percentages and compared using chi-square tests. Continuous variables were summarized as median and interquartile range (IQR) and compared using Wilcoxon rank sum tests. Weekly observed-to-expected (O:E) ratios were calculated for each hospital by dividing the number of observed respiratory illness and nonrespiratory illness encounters in a given week in 2020 (observed) by the average number of encounters for that same week during 2017-2019 (expected). O:E ratios were then aggregated over the three COVID-19 study periods, and 95% confidence intervals were established around mean O:E ratios across individual hospitals. Outcomes were then stratified by respiratory illness subgroups, geographic region, and age. Additional details can be found in the Supplemental Methods in the Appendix.

Study Population

A total of 9,051,980 encounters were included in the study, 6,811,799 with nonrespiratory illnesses and 2,240,181 with respiratory illnesses. Median age was 5 years (IQR, 1-11 years), and 52.7% of the population was male (Appendix Table 2 and Appendix Table 3).

Respiratory vs Nonrespiratory Illness During the COVID-19 Pandemic

Over the study period, fewer respiratory and nonrespiratory illness encounters were observed than expected, with a larger decrease in respiratory illness encounters (Table, Appendix Table 4).

Respiratory Subgroup Analyses

The O:E ratio decreased for all respiratory subgroups over the study period (Table, Appendix Table 4). There were significant differences in specific respiratory subgroups, including asthma, bronchiolitis, croup, influenza, and pneumonia (Appendix Figure 1A). Temporal trends in respiratory encounters were consistent across hospital settings, ages, and geographic regions (Appendix Figure 1B-D). When comparing the with and without COVID-19 subgroups in the “other respiratory illnesses” cohort, other respiratory illness without COVID-19 decreased and remained lower than expected over the rest of the study period, while other respiratory illness with COVID-19 increased markedly during the summer months and declined thereafter (Appendix Figure 2).

All age groups had reductions in respiratory illness encounters during the early COVID-19 and COVID-19 periods, although the decline was less pronounced in the 12- to 17-year-old group (Appendix Figure 1B). Similarly, while all age groups experienced increases in encounters for respiratory illnesses during the summer months, only children in the 12- to 17-year-old group experienced increases beyond pre-COVID-19 levels. Importantly, this increase in respiratory encounters was largely driven by COVID-19 diagnoses (Appendix Figure 3). The trend in nonrespiratory illness encounters stratified by age is shown in Appendix Figure 4.

When patients were stratified by hospital setting, there were no differences between those hospitalized and those discharged from the ED (Appendix Figure 1C). Patterns in respiratory illnesses by geographic location were qualitatively similar until the beginning of the summer 2020, after which geographical variation became more evident (Appendix Figure 1D).

In this large, multicenter study evaluating ED visits and hospitalizations for respiratory and nonrespiratory illnesses at US pediatric hospitals during the 2020 COVID-19 pandemic, we found a significant and substantial decrease in healthcare encounters for respiratory illnesses. A rapid and marked decline in encounters for respiratory illness in a relatively short period of time (March 12-April 2) was observed across all hospitals and US regions. Declines were consistent across common respiratory illnesses. More modest, yet still substantial, declines were also observed for nonrespiratory illnesses.

There are likely multiple underlying reasons for the observed reductions. Social distancing measures almost certainly played an important role in interrupting respiratory infection transmission. Rapid reduction in influenza transmission during the early COVID-19 period has been attributed to social distancing measures,³ and influenza transmission in children decreases with school closures.⁹ It is also possible that some families delayed seeking care at hospitals due to COVID-19, leading to less frequent encounters but more severe illness. The similar decrease in O:E ratio for ED visits and hospitalizations, however, is inconsistent with this explanation. It is also possible that nonurgent conditions cared for in the hospital settings were diverted to other care settings. For example, during this pandemic, telehealth and telephone visits for pediatric asthma increased by 61% and 19%, respectively, while ED and outpatient visits decreased concurrently.¹⁰Similar changes in location of care may also contribute to the decline in nonrespiratory illness encounters. Decreased use of hospital resources for nonurgent care diagnoses during the pandemic would suggest that, prior to COVID-19, there was overutilization of ambulatory services at children’s hospitals. Therefore, the pandemic may be driving care to more appropriate settings.

We also found relative differences in changes in encounters for respiratory illness by age. Adolescents’ levels of respiratory healthcare use declined less and recovered at a faster rate than those of younger children, returning to pre-COVID-19 levels by the end of the study period. The reason for this age differential is likely multifaceted. Infections, such as bronchiolitis and pneumonia, are more likely to be a source of respiratory illness in younger than in older children. It is also likely that disproportionate relaxation of social distancing measures among adolescents, who are known to have a stronger pattern of social interaction, contributed to the faster rise in respiratory illness–related encounters in this age group.¹¹ Adolescents have been reported to be more susceptible to, and more likely to transmit, SARS-CoV-2 compared to younger age groups.¹² More modest, albeit similar, age-based changes were observed in encounters for nonrespiratory illnesses. It is possible that pandemic-related stressors resulted in a subsequent increase in mental health encounters among this age group.¹³ While the reason for this also is likely multifactorial, adolescent behavior, as well as transmission of infectious illness that can exacerbate nonrespiratory conditions, may be a factor.

Emerging evidence suggests that school-age children may play an important role in SARS-CoV-2 transmission in the community.^4,14 Our finding that, compared to younger children, adolescents had significantly fewer reductions in respiratory illness encounters is concerning. These findings suggest that community-based efforts to help prevent respiratory illnesses, especially COVID-19, should focus on adolescents, who are most likely to maintain social interactions and transmit respiratory infections in the school setting and their households.

This study is limited by the inclusion of only tertiary care children’s hospitals, which may not be nationally representative, and the inability to assess the precise timing of when specific public health interventions were introduced. Moreover, previous studies suggest that social distancing behaviors may have changed even before formal recommendations were enacted.¹⁵ Future studies should investigate the local impact of state- and municipality-specific mandates on the burden of COVID-19 and other respiratory illnesses.

The COVID-19 pandemic was associated with substantial reductions in encounters for respiratory diseases, and also with more modest but still sizable reductions in encounters for nonrespiratory diseases. These reductions varied by age. Encounters among adolescents declined less and returned to previous levels faster compared with those of younger children.

This publication is dedicated to the memory of our coauthor, Dr. Michael Bendel-Stenzel. Dr. Bendel-Stenzel was dedicated to bettering the lives of children and advancing our knowledge of pediatrics through his research.

In the United States, respiratory illnesses are the most common cause of emergency department (ED) visits and hospitalizations in children.¹ In response to the ongoing COVID-19 pandemic, several public health interventions, including school and business closures, stay-at-home orders, and mask mandates, were implemented to limit transmission of SARS-CoV-2.^2,3 Studies have shown that children can contribute to the spread of SARS-CoV-2 infections, especially within households.^4-6 Recent data suggest that COVID-19, and the associated public health measures enacted to slow its spread, may have affected the transmission of other respiratory pathogens.⁷ Similarly, the pandemic has likely affected healthcare utilization for nonrespiratory illnesses through adoption of social distancing recommendations, suspension and delays in nonemergent elective care, avoidance of healthcare settings, and the effect of decreased respiratory disease on exacerbation of chronic illness.⁸ The objective of this study was to examine associations between the COVID-19 pandemic and healthcare utilization for pediatric respiratory and nonrespiratory illnesses at US pediatric hospitals.

Study Design

This is a multicenter, cross-sectional study of encounters at 44 pediatric hospitals that reported data to the Pediatric Health Information System (PHIS) database maintained by the Children’s Hospital Association (Lenexa, Kansas).

Study Population

Children 2 months to 18 years of age discharged from ED or inpatient settings with a nonsurgical diagnosis from January 1 to September 30 over a 4-year period (2017-2020) were included.

Exposure

The primary exposure was the 2020 COVID-19 pandemic time, divided into three periods: pre-COVID-19 (January-February 2020, the period prior to the pandemic in the United States), early COVID-19 (March-April 2020, coinciding with the first reported US pediatric case of COVID-19 on March 2, 2020), and COVID-19 (May-September 2020, marked by the implementation of at least two of the following containment measures in every US state: stay-at-home/shelter orders, school closures, nonessential business closures, restaurant closures, or prohibition of gatherings of more than 10 people).²

Outcomes

Respiratory illness diagnoses were classified into mutually exclusive subgroups following a prespecified hierarchy: influenza, pneumonia, croup, bronchiolitis, asthma, unspecified influenza-like illness, and “other respiratory diagnoses” (Appendix Table 1). To assess the impact of COVID-19 after its International Classification of Diseases, Tenth Revision code was established on March 25, 2020, the “other respiratory” subgroup was divided into other respiratory illnesses with and without COVID-19. Nonrespiratory illness diagnoses were defined as all diagnoses not included in the respiratory illness cohort.

Statistical Analysis

Categorical variables were summarized using frequencies and percentages and compared using chi-square tests. Continuous variables were summarized as median and interquartile range (IQR) and compared using Wilcoxon rank sum tests. Weekly observed-to-expected (O:E) ratios were calculated for each hospital by dividing the number of observed respiratory illness and nonrespiratory illness encounters in a given week in 2020 (observed) by the average number of encounters for that same week during 2017-2019 (expected). O:E ratios were then aggregated over the three COVID-19 study periods, and 95% confidence intervals were established around mean O:E ratios across individual hospitals. Outcomes were then stratified by respiratory illness subgroups, geographic region, and age. Additional details can be found in the Supplemental Methods in the Appendix.

Study Population

A total of 9,051,980 encounters were included in the study, 6,811,799 with nonrespiratory illnesses and 2,240,181 with respiratory illnesses. Median age was 5 years (IQR, 1-11 years), and 52.7% of the population was male (Appendix Table 2 and Appendix Table 3).

Respiratory vs Nonrespiratory Illness During the COVID-19 Pandemic

Over the study period, fewer respiratory and nonrespiratory illness encounters were observed than expected, with a larger decrease in respiratory illness encounters (Table, Appendix Table 4).

Respiratory Subgroup Analyses

The O:E ratio decreased for all respiratory subgroups over the study period (Table, Appendix Table 4). There were significant differences in specific respiratory subgroups, including asthma, bronchiolitis, croup, influenza, and pneumonia (Appendix Figure 1A). Temporal trends in respiratory encounters were consistent across hospital settings, ages, and geographic regions (Appendix Figure 1B-D). When comparing the with and without COVID-19 subgroups in the “other respiratory illnesses” cohort, other respiratory illness without COVID-19 decreased and remained lower than expected over the rest of the study period, while other respiratory illness with COVID-19 increased markedly during the summer months and declined thereafter (Appendix Figure 2).

All age groups had reductions in respiratory illness encounters during the early COVID-19 and COVID-19 periods, although the decline was less pronounced in the 12- to 17-year-old group (Appendix Figure 1B). Similarly, while all age groups experienced increases in encounters for respiratory illnesses during the summer months, only children in the 12- to 17-year-old group experienced increases beyond pre-COVID-19 levels. Importantly, this increase in respiratory encounters was largely driven by COVID-19 diagnoses (Appendix Figure 3). The trend in nonrespiratory illness encounters stratified by age is shown in Appendix Figure 4.

When patients were stratified by hospital setting, there were no differences between those hospitalized and those discharged from the ED (Appendix Figure 1C). Patterns in respiratory illnesses by geographic location were qualitatively similar until the beginning of the summer 2020, after which geographical variation became more evident (Appendix Figure 1D).

In this large, multicenter study evaluating ED visits and hospitalizations for respiratory and nonrespiratory illnesses at US pediatric hospitals during the 2020 COVID-19 pandemic, we found a significant and substantial decrease in healthcare encounters for respiratory illnesses. A rapid and marked decline in encounters for respiratory illness in a relatively short period of time (March 12-April 2) was observed across all hospitals and US regions. Declines were consistent across common respiratory illnesses. More modest, yet still substantial, declines were also observed for nonrespiratory illnesses.

There are likely multiple underlying reasons for the observed reductions. Social distancing measures almost certainly played an important role in interrupting respiratory infection transmission. Rapid reduction in influenza transmission during the early COVID-19 period has been attributed to social distancing measures,³ and influenza transmission in children decreases with school closures.⁹ It is also possible that some families delayed seeking care at hospitals due to COVID-19, leading to less frequent encounters but more severe illness. The similar decrease in O:E ratio for ED visits and hospitalizations, however, is inconsistent with this explanation. It is also possible that nonurgent conditions cared for in the hospital settings were diverted to other care settings. For example, during this pandemic, telehealth and telephone visits for pediatric asthma increased by 61% and 19%, respectively, while ED and outpatient visits decreased concurrently.¹⁰Similar changes in location of care may also contribute to the decline in nonrespiratory illness encounters. Decreased use of hospital resources for nonurgent care diagnoses during the pandemic would suggest that, prior to COVID-19, there was overutilization of ambulatory services at children’s hospitals. Therefore, the pandemic may be driving care to more appropriate settings.

We also found relative differences in changes in encounters for respiratory illness by age. Adolescents’ levels of respiratory healthcare use declined less and recovered at a faster rate than those of younger children, returning to pre-COVID-19 levels by the end of the study period. The reason for this age differential is likely multifaceted. Infections, such as bronchiolitis and pneumonia, are more likely to be a source of respiratory illness in younger than in older children. It is also likely that disproportionate relaxation of social distancing measures among adolescents, who are known to have a stronger pattern of social interaction, contributed to the faster rise in respiratory illness–related encounters in this age group.¹¹ Adolescents have been reported to be more susceptible to, and more likely to transmit, SARS-CoV-2 compared to younger age groups.¹² More modest, albeit similar, age-based changes were observed in encounters for nonrespiratory illnesses. It is possible that pandemic-related stressors resulted in a subsequent increase in mental health encounters among this age group.¹³ While the reason for this also is likely multifactorial, adolescent behavior, as well as transmission of infectious illness that can exacerbate nonrespiratory conditions, may be a factor.

Emerging evidence suggests that school-age children may play an important role in SARS-CoV-2 transmission in the community.^4,14 Our finding that, compared to younger children, adolescents had significantly fewer reductions in respiratory illness encounters is concerning. These findings suggest that community-based efforts to help prevent respiratory illnesses, especially COVID-19, should focus on adolescents, who are most likely to maintain social interactions and transmit respiratory infections in the school setting and their households.

This study is limited by the inclusion of only tertiary care children’s hospitals, which may not be nationally representative, and the inability to assess the precise timing of when specific public health interventions were introduced. Moreover, previous studies suggest that social distancing behaviors may have changed even before formal recommendations were enacted.¹⁵ Future studies should investigate the local impact of state- and municipality-specific mandates on the burden of COVID-19 and other respiratory illnesses.

The COVID-19 pandemic was associated with substantial reductions in encounters for respiratory diseases, and also with more modest but still sizable reductions in encounters for nonrespiratory diseases. These reductions varied by age. Encounters among adolescents declined less and returned to previous levels faster compared with those of younger children.

This publication is dedicated to the memory of our coauthor, Dr. Michael Bendel-Stenzel. Dr. Bendel-Stenzel was dedicated to bettering the lives of children and advancing our knowledge of pediatrics through his research.

In the United States, respiratory illnesses are the most common cause of emergency department (ED) visits and hospitalizations in children.¹ In response to the ongoing COVID-19 pandemic, several public health interventions, including school and business closures, stay-at-home orders, and mask mandates, were implemented to limit transmission of SARS-CoV-2.^2,3 Studies have shown that children can contribute to the spread of SARS-CoV-2 infections, especially within households.^4-6 Recent data suggest that COVID-19, and the associated public health measures enacted to slow its spread, may have affected the transmission of other respiratory pathogens.⁷ Similarly, the pandemic has likely affected healthcare utilization for nonrespiratory illnesses through adoption of social distancing recommendations, suspension and delays in nonemergent elective care, avoidance of healthcare settings, and the effect of decreased respiratory disease on exacerbation of chronic illness.⁸ The objective of this study was to examine associations between the COVID-19 pandemic and healthcare utilization for pediatric respiratory and nonrespiratory illnesses at US pediatric hospitals.

Study Design

This is a multicenter, cross-sectional study of encounters at 44 pediatric hospitals that reported data to the Pediatric Health Information System (PHIS) database maintained by the Children’s Hospital Association (Lenexa, Kansas).

Study Population

Children 2 months to 18 years of age discharged from ED or inpatient settings with a nonsurgical diagnosis from January 1 to September 30 over a 4-year period (2017-2020) were included.

Exposure

The primary exposure was the 2020 COVID-19 pandemic time, divided into three periods: pre-COVID-19 (January-February 2020, the period prior to the pandemic in the United States), early COVID-19 (March-April 2020, coinciding with the first reported US pediatric case of COVID-19 on March 2, 2020), and COVID-19 (May-September 2020, marked by the implementation of at least two of the following containment measures in every US state: stay-at-home/shelter orders, school closures, nonessential business closures, restaurant closures, or prohibition of gatherings of more than 10 people).²

Outcomes

Respiratory illness diagnoses were classified into mutually exclusive subgroups following a prespecified hierarchy: influenza, pneumonia, croup, bronchiolitis, asthma, unspecified influenza-like illness, and “other respiratory diagnoses” (Appendix Table 1). To assess the impact of COVID-19 after its International Classification of Diseases, Tenth Revision code was established on March 25, 2020, the “other respiratory” subgroup was divided into other respiratory illnesses with and without COVID-19. Nonrespiratory illness diagnoses were defined as all diagnoses not included in the respiratory illness cohort.

Statistical Analysis

Categorical variables were summarized using frequencies and percentages and compared using chi-square tests. Continuous variables were summarized as median and interquartile range (IQR) and compared using Wilcoxon rank sum tests. Weekly observed-to-expected (O:E) ratios were calculated for each hospital by dividing the number of observed respiratory illness and nonrespiratory illness encounters in a given week in 2020 (observed) by the average number of encounters for that same week during 2017-2019 (expected). O:E ratios were then aggregated over the three COVID-19 study periods, and 95% confidence intervals were established around mean O:E ratios across individual hospitals. Outcomes were then stratified by respiratory illness subgroups, geographic region, and age. Additional details can be found in the Supplemental Methods in the Appendix.

Study Population

A total of 9,051,980 encounters were included in the study, 6,811,799 with nonrespiratory illnesses and 2,240,181 with respiratory illnesses. Median age was 5 years (IQR, 1-11 years), and 52.7% of the population was male (Appendix Table 2 and Appendix Table 3).

Respiratory vs Nonrespiratory Illness During the COVID-19 Pandemic

Over the study period, fewer respiratory and nonrespiratory illness encounters were observed than expected, with a larger decrease in respiratory illness encounters (Table, Appendix Table 4).

Respiratory Subgroup Analyses

The O:E ratio decreased for all respiratory subgroups over the study period (Table, Appendix Table 4). There were significant differences in specific respiratory subgroups, including asthma, bronchiolitis, croup, influenza, and pneumonia (Appendix Figure 1A). Temporal trends in respiratory encounters were consistent across hospital settings, ages, and geographic regions (Appendix Figure 1B-D). When comparing the with and without COVID-19 subgroups in the “other respiratory illnesses” cohort, other respiratory illness without COVID-19 decreased and remained lower than expected over the rest of the study period, while other respiratory illness with COVID-19 increased markedly during the summer months and declined thereafter (Appendix Figure 2).

All age groups had reductions in respiratory illness encounters during the early COVID-19 and COVID-19 periods, although the decline was less pronounced in the 12- to 17-year-old group (Appendix Figure 1B). Similarly, while all age groups experienced increases in encounters for respiratory illnesses during the summer months, only children in the 12- to 17-year-old group experienced increases beyond pre-COVID-19 levels. Importantly, this increase in respiratory encounters was largely driven by COVID-19 diagnoses (Appendix Figure 3). The trend in nonrespiratory illness encounters stratified by age is shown in Appendix Figure 4.

When patients were stratified by hospital setting, there were no differences between those hospitalized and those discharged from the ED (Appendix Figure 1C). Patterns in respiratory illnesses by geographic location were qualitatively similar until the beginning of the summer 2020, after which geographical variation became more evident (Appendix Figure 1D).

In this large, multicenter study evaluating ED visits and hospitalizations for respiratory and nonrespiratory illnesses at US pediatric hospitals during the 2020 COVID-19 pandemic, we found a significant and substantial decrease in healthcare encounters for respiratory illnesses. A rapid and marked decline in encounters for respiratory illness in a relatively short period of time (March 12-April 2) was observed across all hospitals and US regions. Declines were consistent across common respiratory illnesses. More modest, yet still substantial, declines were also observed for nonrespiratory illnesses.

There are likely multiple underlying reasons for the observed reductions. Social distancing measures almost certainly played an important role in interrupting respiratory infection transmission. Rapid reduction in influenza transmission during the early COVID-19 period has been attributed to social distancing measures,³ and influenza transmission in children decreases with school closures.⁹ It is also possible that some families delayed seeking care at hospitals due to COVID-19, leading to less frequent encounters but more severe illness. The similar decrease in O:E ratio for ED visits and hospitalizations, however, is inconsistent with this explanation. It is also possible that nonurgent conditions cared for in the hospital settings were diverted to other care settings. For example, during this pandemic, telehealth and telephone visits for pediatric asthma increased by 61% and 19%, respectively, while ED and outpatient visits decreased concurrently.¹⁰Similar changes in location of care may also contribute to the decline in nonrespiratory illness encounters. Decreased use of hospital resources for nonurgent care diagnoses during the pandemic would suggest that, prior to COVID-19, there was overutilization of ambulatory services at children’s hospitals. Therefore, the pandemic may be driving care to more appropriate settings.

We also found relative differences in changes in encounters for respiratory illness by age. Adolescents’ levels of respiratory healthcare use declined less and recovered at a faster rate than those of younger children, returning to pre-COVID-19 levels by the end of the study period. The reason for this age differential is likely multifaceted. Infections, such as bronchiolitis and pneumonia, are more likely to be a source of respiratory illness in younger than in older children. It is also likely that disproportionate relaxation of social distancing measures among adolescents, who are known to have a stronger pattern of social interaction, contributed to the faster rise in respiratory illness–related encounters in this age group.¹¹ Adolescents have been reported to be more susceptible to, and more likely to transmit, SARS-CoV-2 compared to younger age groups.¹² More modest, albeit similar, age-based changes were observed in encounters for nonrespiratory illnesses. It is possible that pandemic-related stressors resulted in a subsequent increase in mental health encounters among this age group.¹³ While the reason for this also is likely multifactorial, adolescent behavior, as well as transmission of infectious illness that can exacerbate nonrespiratory conditions, may be a factor.

Emerging evidence suggests that school-age children may play an important role in SARS-CoV-2 transmission in the community.^4,14 Our finding that, compared to younger children, adolescents had significantly fewer reductions in respiratory illness encounters is concerning. These findings suggest that community-based efforts to help prevent respiratory illnesses, especially COVID-19, should focus on adolescents, who are most likely to maintain social interactions and transmit respiratory infections in the school setting and their households.

This study is limited by the inclusion of only tertiary care children’s hospitals, which may not be nationally representative, and the inability to assess the precise timing of when specific public health interventions were introduced. Moreover, previous studies suggest that social distancing behaviors may have changed even before formal recommendations were enacted.¹⁵ Future studies should investigate the local impact of state- and municipality-specific mandates on the burden of COVID-19 and other respiratory illnesses.

The COVID-19 pandemic was associated with substantial reductions in encounters for respiratory diseases, and also with more modest but still sizable reductions in encounters for nonrespiratory diseases. These reductions varied by age. Encounters among adolescents declined less and returned to previous levels faster compared with those of younger children.

This publication is dedicated to the memory of our coauthor, Dr. Michael Bendel-Stenzel. Dr. Bendel-Stenzel was dedicated to bettering the lives of children and advancing our knowledge of pediatrics through his research.

The Choosing Wisely^® Campaign (CWC) was launched in 2012. This ongoing national initiative encourages conversations among patients and clinicians about the need —or the lack thereof—for frequent tests, treatments, and procedures in healthcare. More than 80 professional societies have developed short lists of evidence-based recommendations aimed at avoiding unnecessary, “low-value” care. More than 550 recommendations are currently available.¹ The Society of Hospital Medicine (SHM) Pediatric Committee published a list of five recommendations for the CWC in 2013.²

After seven years, the campaign has posted several success stories highlighting the increase in clinicians’ awareness about the recommendations. Several local, regional, and national initiatives and quality improvement (QI) projects have been inspired by the CWC and its tenants.^1,3 However, limited research has been performed on the true impact of these recommendations on avoiding “low-value” services. A more comprehensive approach is required to “measure wisely” the impact of the campaign on bedside clinical practice.⁴ Stakeholders in healthcare value have been challenged to collaborate in creating high-impact lists of “low-value” interventions and designing effective tools to measure their impact on clinical practice and costs.⁵

We initially developed a report card with five metrics derived from the CWC-SHM pediatric recommendations to help individual institutions and group practices to measure their performance and benchmark their results with peers.⁶ The report card is available for hospital members of the Children’s Hospital Association (CHA).⁷

The current study analyzes the frequency of utilization and trends of five metrics included in the CHA/Pediatric Health Information System^® (PHIS) CWC report card in tertiary children’s hospitals in the United States. We analyzed data from five years before and five years after the CWC-PHM recommendations were published in 2013. We hypothesize that the publication and dissemination of the CWC-PHM recommendations—the intervention—will result in either an immediate decrease in the use of the “low-value” services studied and/or a change in the trend of utilization over time.

Study Design

We conducted an observational, longitudinal retrospective study aimed at evaluating the impact of the CWC-PHM recommendations on clinical practice in tertiary children’s hospitals in the US.

Study Population

The population included inpatient and observation stays for children aged 0-18 years admitted to the 36 children’s hospitals consistently providing data from 2008 to 2017 to the PHIS administrative database (CHA, Lenexa, Kansas). This database contains inpatient, emergency department, ambulatory, and observation encounter–level data from more than 50 not-for-profit, tertiary care pediatric hospitals and accounts for ~20% of all pediatric hospitalizations in the US every year.

A joint effort between the CHA and the participating hospitals ensures the quality of the data submitted, as previously described.⁸ These data are subjected to a routine quality check with each submission and within each report. Data were fully deidentified for this study. In total, 36 PHIS hospitals met the strict quality standards for inclusion of submitted data. The remaining hospitals were excluded because they did not have complete data or had incomplete billing information.

For external benchmarking purposes, PHIS participating hospitals provide encounter data, including demographics, diagnoses, and procedures (International Classification of Diseases versions 9 and 10).^9,10 The transition from ICD-9 to ICD-10 in the US took place during the study period. However, the CHA completed a process of translating and mapping all ICD-9 codes to every possible equivalent ICD-10 code in the PHIS database. Thus, the change from ICD-9 to ICD-10 should not have had any significant effect on population definition and data analytics, including trend analysis.

For each condition, the study population was divided into the following two cohorts for comparison of the trends: all admissions from January 1, 2008 to December 31, 2012 (before) and all admissions from January 1, 2013 to December 31, 2017 (after) the CWC-PHM recommendations were published.

This study was determined to be nonhuman subject research and was therefore exempted by Nicklaus Children’s Hospital Human Research Protection Program.

Outcomes

The outcomes for this study were the percentages of patients receiving the not-recommended “low-value” services targeted by the CWC-PHM recommendations. For this purpose, four of the five recommendations were translated into the following five metrics, operationalized in the PHIS database and displayed in the “Choosing Wisely” report card:⁶

1. Percentage of patients with uncomplicated asthma receiving chest radiograph (CXR).

2. Percentage of patients with uncomplicated bronchiolitis receiving CXR.

3. Percentage of patients with uncomplicated bronchiolitis receiving bronchodilators.

4. Percentage of patients with lower respiratory tract infection (LRTI) receiving systemic corticosteroids (relievers).

5. Percentage of patients with uncomplicated gastroesophageal reflux (GER) receiving acid suppressor therapy.

The fifth recommendation—limiting the use of continuous pulse oximetry unless the patient is receiving supplemental oxygen—could not be operationalized in the PHIS database because of inconsistent reporting of these resources.⁶

The resulting percentages represent nonadherence to the recommendations, suggesting overuse of the specific “low-value” intervention. As such, a decreasing trend over time is the desired direction of improvement.

The definition of “uncomplicated” conditions and the metrics are presented in Table 1. A complete list of the inclusion and exclusion criteria to define “uncomplicated” conditions and the complete list of the clinical translation codes used in PHIS to identify the “low-value” services are presented as an electronic supplement.

Statistical Analyses

We compared the demographic and clinical characteristics of the various cohorts before and after the release of the CWC-PHM recommendations—the intervention—using chi-square statistics. To assess the individual hospital-level trends over time for each measure, we modeled the patient-level data of each hospital using generalized linear mixed effects models with a binomial distribution. These models were adjusted for patient demographic and clinical factors that were found to be significantly different (P < .01) before and after the intervention on bivariate analyses. From these models, we generated adjusted estimates for the quarterly percentages for each hospital. We then conducted an interrupted time series (ITS) using these estimates to compare trends in the five years before (2008-2012) and five years after (2013-2017) the publication of the CWC-PHM recommendations. For the ITS analysis, we used a generalized linear mixed effects model with the quarterly adjusted hospital-level utilization rates of “low-value” services for each cohort as the unit of analysis and a random intercept for each hospital. The model used an autoregressive(1) covariance structure to account for autocorrelation. The ITS allowed us to test our hypothesis by assessing the following two important features: (a) if a significant decrease occurred right after the CWC-PHM recommendations were published (level-change) and/or (b) if the intervention altered the secular trend (slope-change). All statistical analyses were performed using SAS v. 9.4 (SAS Institute, Cary, North Carolina), and P values <.01 were considered to be statistically significant.

Table 2 presents the demographic characteristics of the cohorts before (2008-2012) and after (2013-2017) the publication of the CWC-PHM recommendations. Hospitalizations due to asthma represented the largest cohort with 142,067 cases, followed by hospitalizations due to bronchiolitis with 94,253 cases. Hospitalizations due to GER comprised the smallest cohort with 13,635 cases. Most of the children had government insurance and had “minor” severity according to the All Patient Revised Diagnosis Related Group (APR-DRG) system.

We found statistically significant differences in most of the demographic characteristics for the cohorts when comparing cases before and after the introduction of the CWC-PHM recommendations.

After adjusting for demographic characteristics, we estimated the percentages of the utilization of the “low-value” services from 2008 to 2017. We observed a steady decrease in overutilization of all services over time. The absolute percentage decrease was more evident in the reduction of the utilization of relievers by 36.6% and that of CXR by 31.5% for bronchiolitis. We also observed a 20.8% absolute reduction in the use of CXR for asthma.

The use of systemic steroids in LRTI revealed the lowest utilization among the “low-value” services studied, with 15.1% in 2008 and 12.2% in 2017, a 2.9% absolute reduction. However, the prescription of acid suppressors for GER showed the highest utilization among all the overuse metrics studied, ie, 63% in 2008 and 48.9% in 2017, with an absolute decrease of 24.1%. The yearly adjusted estimated percentages of utilization for each “low-value” service are presented in Appendix Table A.

Table 3 and the Figure (attached as supplemental online graphic) respectively present the risk-adjusted ITS parameter estimates and the graphic representation before and after the inception of the CWC-PHM recommendations for the trend analysis.

We found a steady reduction in the frequency of overutilization of five “low-value” services described in the CWC-PHM recommendations from 2008 to 2017 in 36 tertiary children’s hospitals in the US. This trend was more evident in the utilization of relievers and CXR for bronchiolitis. The ITS analysis demonstrated that immediately after the publication of the CWC-PHM recommendations, only the use of CXR for asthma decreased significantly. Then, only the use of relievers for bronchiolitis decreased significantly over time in comparison with the secular trend.

These results support our hypothesis for two of the five metrics studied, suggesting that the publication of the CWC-PHM recommendations had a modest impact in clinical practices related to those services in tertiary children’s hospitals.

These findings align with a limited number of published studies that have consistently found a modest decrease in the use of “low-value” services before 2012^11-13 and a limited impact of the CWC in clinical practices on the use of “low-value” services after the inception of the campaign.^14-17

For instance, in a cross-sectional analysis of the 1999 and 2009 samples of ambulatory care practices in the US, only two of 11 overuse quality indicators showed improvement.¹¹ The authors recognized that reducing inappropriate care will require the same attention to guideline development and performance measurement that was directed at reducing the underuse of needed therapies. However, determining whether a patient received inappropriate care generally requires a much more detailed analysis of clinical information than what is required for assessments of underuse.¹¹

Another study designed claims-based algorithms to measure the prevalence of 11 Choosing Wisely-identified “low-value” services in fee-for-service Medicare patients aged >65 years from 2006 to 2011.¹² The annual prevalence of selected CWC “low-value” services ranged from 1.2% (upper urinary tract imaging in men with benign prostatic hyperplasia) to 46.5% (preoperative cardiac testing for low-risk, noncardiac procedures). The study concluded that identifying and measuring “low-value” health services is a prerequisite for improving quality and eliminating waste.¹²

In pediatric medicine, the authors investigated a large cohort of infants aged one to 24 months hospitalized with bronchiolitis to 41 tertiary children’s hospitals reporting data to the PHIS database from 2004 to 2012.¹³ The trend analysis revealed a decrease in the utilization of diagnostics and treatment interventions before the publication of the American Academy of Pediatrics 2006 Bronchiolitis Guidelines.¹⁸ There was an additional reduction in the use of CXR, steroids, and bronchodilators after the publication of the guidelines.¹³

After the CWC was launched in 2012, several surveys have demonstrated a tangible increase in awareness of the CWC and its goals, mostly among primary care physicians and subspecialists. Clinicians who were aware of the campaign found the recommendations to be useful as a legitimate source of guidance and were more likely to reduce the indication of unnecessary care and “low-value” clinical services included in the CWC.^1,3,19,20

Few studies in adults have focused on measuring the trends in overuse metrics derived from the CWC recommendations.^14-16 The initial studies have found limited reduction on the use of “low-value” care after the inception of the CWC. They suggest that clinician education, awareness, and public promotion alone do not appear to be sufficient to achieve widespread changes in clinical practice. Additional interventions are necessary for the wider implementation and success of the CWC recommendations.^{11,14,15,19,21,22}

However, a more recent study was conducted in 91 academic centers from 2013 through 2016, before and after the publication of a CWC recommendation on the use of troponin-only testing for the diagnosis of acute myocardial infarction. Hospitals with low rates of troponin-only testing before the publication of the recommendation demonstrated a statistically significant increase over time in the rate of adherence. The authors postulated that the impact of the CWC might have been significant because of the increase in the institutional and provider attention to “high-value” care as a result of the campaign.¹⁶

In pediatrics, a cross-sectional study defined 20 “low-value” services from a list of more than 400 items from the CWC and other sources of highly regarded, evidence-based pediatrics healthcare recommendations. The list included six diagnostic tests, five imaging tests, and nine prescription drugs ordered in a robust cohort of 4.4 million children nationwide in 2014. The study concluded that approximately one in 10 children received a “low-value” service. The majority (59.4%) were related to prescription drugs, specifically the inappropriate use of antibiotics for a variety of conditions. The estimated combined cost of these unnecessary services was approximately $27 million, with one-third of the cost being paid out of pocket, arguing for significant financial harm. However, this study did not perform a trend analysis.¹⁷

Our results are comparable with these studies, reporting an initial increase in awareness and beliefs, followed by progressive changes in clinical practice among pediatric hospital-based clinicians in delivering evidence-based, high-value care after the CWC.

The attribution of the steady reduction in the absolute percentages of overuse/waste in the five metrics related to the CWC observed in this study, including the significant changes noted in two of the overuse indicators after the publication of the CWC-PHM recommendations, should be interpreted with caution. For example, the significant decrease in the use of “low-value” services in bronchiolitis could be attributed to multiple factors such as national guidelines released in 2014 after the campaign,²³ national multicenter QI collaborative projects,^24,25 and multiple local QI efforts.^26,27 The increase in the awareness and impact of the CWC recommendations among pediatric providers could also be a contributing factor, but this association cannot be established in the light of our findings.

On the other hand, despite extensive evidence for the lack of efficacy and the potential harm associated with the use of acid suppressors for uncomplicated GER in infants,^28-30 the frequency of this “low-value” therapeutic intervention remains high (~50%). The trend in utilization was not impacted by the CWC-PHM recommendations. This finding could be explained by several factors, including the possibility that several hospitalized patients may suffer from GER disease requiring acid suppressors. Another possibility is that acid suppressors are generally prescribed as an outpatient medication, and physicians treating inpatients may be reluctant to discontinue it during hospitalization. Nevertheless, this recommendation represents a target for review, update, and QI interventions in the near future.

The delivery of inappropriate “low-value” care represents the most significant dimension of waste in healthcare.³¹ The development of quality measures of “low-value” services representing overuse and waste is the most needed step toward assessing the magnitude of the problem. Overuse metrics could be incorporated into QI interventions to decrease the provision of such services. However, systematic efforts aimed at developing quality indicators of overuse based on the CWC recommendations have been limited. To our knowledge, this is the first study on the trends of metrics derived from the CWC recommendations in pediatric medicine.

Future research is needed to develop overuse metrics further to assess the specific outcomes related to the implementation of the CWC. How much has clinical practice changed as a result of the campaign? What are the outcomes and savings attributable to these efforts? These are critical questions for the immediate future that should be answered to sustain the ongoing efforts and results and to validate that the efforts are worthwhile.

This study has several limitations. First, this is a retrospective and observational study. It cannot prove a direct causal relationship between the publication of the CWC-PHM and the observed trends, as other potential factors may have contributed to the outcomes. Second, in administrative databases, the data quality is dependent on proper documentation and coding that may vary among reporting institutions. These data lack clinical information, and a fair assessment of “appropriateness” could be questioned. In addition, the study included only 36 academic, tertiary children’s hospitals. Because approximately two-thirds of all pediatric hospitalizations in the US occur in community settings,³² this study may not fully represent clinical practice in the majority of pediatric hospitalizations in the US. Finally, the validity of the ITS analysis has inherent limitations due to the variability of the data in some metrics that may affect the power of the analysis. This fact could lead to inaccurate conclusions regarding intervention effectiveness due to the data-driven model applied, as well as the lack of control for other time-varying confounders.³³

After seven years, the CWC faces important challenges. Critical to the success of the campaign is to “measure wisely” by developing quality indicators of overuse and operationalizing them into administrative and clinical data sources to assess the impact on clinical practice. Our study highlights some limited but steady reduction in the use of some “low-value” services before the campaign. It also demonstrates a modest impact of the campaign on clinical practices in tertiary care children’s hospitals in the US. Clinicians and institutions still have a long way to go in reducing the use of “low-value” interventions in pediatric medicine. These observations challenge us to step up our efforts to implement QI interventions aimed at incorporating these professional, society-endorsed recommendations into our clinical practice.

Acknowledgments

The authors thank Dr. Kristine De La Torre and Dr. Jennifer McCafferty-Fernandez and the Research Institute of Nicklaus Children’s Hospital for medical writing assistance. They also acknowledge Tatiana Consuegra, library technician, for her clerical assistance in the preparation and submission of this article.

The Choosing Wisely^® Campaign (CWC) was launched in 2012. This ongoing national initiative encourages conversations among patients and clinicians about the need —or the lack thereof—for frequent tests, treatments, and procedures in healthcare. More than 80 professional societies have developed short lists of evidence-based recommendations aimed at avoiding unnecessary, “low-value” care. More than 550 recommendations are currently available.¹ The Society of Hospital Medicine (SHM) Pediatric Committee published a list of five recommendations for the CWC in 2013.²

After seven years, the campaign has posted several success stories highlighting the increase in clinicians’ awareness about the recommendations. Several local, regional, and national initiatives and quality improvement (QI) projects have been inspired by the CWC and its tenants.^1,3 However, limited research has been performed on the true impact of these recommendations on avoiding “low-value” services. A more comprehensive approach is required to “measure wisely” the impact of the campaign on bedside clinical practice.⁴ Stakeholders in healthcare value have been challenged to collaborate in creating high-impact lists of “low-value” interventions and designing effective tools to measure their impact on clinical practice and costs.⁵

We initially developed a report card with five metrics derived from the CWC-SHM pediatric recommendations to help individual institutions and group practices to measure their performance and benchmark their results with peers.⁶ The report card is available for hospital members of the Children’s Hospital Association (CHA).⁷

The current study analyzes the frequency of utilization and trends of five metrics included in the CHA/Pediatric Health Information System^® (PHIS) CWC report card in tertiary children’s hospitals in the United States. We analyzed data from five years before and five years after the CWC-PHM recommendations were published in 2013. We hypothesize that the publication and dissemination of the CWC-PHM recommendations—the intervention—will result in either an immediate decrease in the use of the “low-value” services studied and/or a change in the trend of utilization over time.

Study Design

We conducted an observational, longitudinal retrospective study aimed at evaluating the impact of the CWC-PHM recommendations on clinical practice in tertiary children’s hospitals in the US.

Study Population

The population included inpatient and observation stays for children aged 0-18 years admitted to the 36 children’s hospitals consistently providing data from 2008 to 2017 to the PHIS administrative database (CHA, Lenexa, Kansas). This database contains inpatient, emergency department, ambulatory, and observation encounter–level data from more than 50 not-for-profit, tertiary care pediatric hospitals and accounts for ~20% of all pediatric hospitalizations in the US every year.

A joint effort between the CHA and the participating hospitals ensures the quality of the data submitted, as previously described.⁸ These data are subjected to a routine quality check with each submission and within each report. Data were fully deidentified for this study. In total, 36 PHIS hospitals met the strict quality standards for inclusion of submitted data. The remaining hospitals were excluded because they did not have complete data or had incomplete billing information.

For external benchmarking purposes, PHIS participating hospitals provide encounter data, including demographics, diagnoses, and procedures (International Classification of Diseases versions 9 and 10).^9,10 The transition from ICD-9 to ICD-10 in the US took place during the study period. However, the CHA completed a process of translating and mapping all ICD-9 codes to every possible equivalent ICD-10 code in the PHIS database. Thus, the change from ICD-9 to ICD-10 should not have had any significant effect on population definition and data analytics, including trend analysis.

For each condition, the study population was divided into the following two cohorts for comparison of the trends: all admissions from January 1, 2008 to December 31, 2012 (before) and all admissions from January 1, 2013 to December 31, 2017 (after) the CWC-PHM recommendations were published.

This study was determined to be nonhuman subject research and was therefore exempted by Nicklaus Children’s Hospital Human Research Protection Program.

Outcomes

The outcomes for this study were the percentages of patients receiving the not-recommended “low-value” services targeted by the CWC-PHM recommendations. For this purpose, four of the five recommendations were translated into the following five metrics, operationalized in the PHIS database and displayed in the “Choosing Wisely” report card:⁶

1. Percentage of patients with uncomplicated asthma receiving chest radiograph (CXR).

2. Percentage of patients with uncomplicated bronchiolitis receiving CXR.

3. Percentage of patients with uncomplicated bronchiolitis receiving bronchodilators.

4. Percentage of patients with lower respiratory tract infection (LRTI) receiving systemic corticosteroids (relievers).

5. Percentage of patients with uncomplicated gastroesophageal reflux (GER) receiving acid suppressor therapy.

The fifth recommendation—limiting the use of continuous pulse oximetry unless the patient is receiving supplemental oxygen—could not be operationalized in the PHIS database because of inconsistent reporting of these resources.⁶

The resulting percentages represent nonadherence to the recommendations, suggesting overuse of the specific “low-value” intervention. As such, a decreasing trend over time is the desired direction of improvement.

The definition of “uncomplicated” conditions and the metrics are presented in Table 1. A complete list of the inclusion and exclusion criteria to define “uncomplicated” conditions and the complete list of the clinical translation codes used in PHIS to identify the “low-value” services are presented as an electronic supplement.

Statistical Analyses

We compared the demographic and clinical characteristics of the various cohorts before and after the release of the CWC-PHM recommendations—the intervention—using chi-square statistics. To assess the individual hospital-level trends over time for each measure, we modeled the patient-level data of each hospital using generalized linear mixed effects models with a binomial distribution. These models were adjusted for patient demographic and clinical factors that were found to be significantly different (P < .01) before and after the intervention on bivariate analyses. From these models, we generated adjusted estimates for the quarterly percentages for each hospital. We then conducted an interrupted time series (ITS) using these estimates to compare trends in the five years before (2008-2012) and five years after (2013-2017) the publication of the CWC-PHM recommendations. For the ITS analysis, we used a generalized linear mixed effects model with the quarterly adjusted hospital-level utilization rates of “low-value” services for each cohort as the unit of analysis and a random intercept for each hospital. The model used an autoregressive(1) covariance structure to account for autocorrelation. The ITS allowed us to test our hypothesis by assessing the following two important features: (a) if a significant decrease occurred right after the CWC-PHM recommendations were published (level-change) and/or (b) if the intervention altered the secular trend (slope-change). All statistical analyses were performed using SAS v. 9.4 (SAS Institute, Cary, North Carolina), and P values <.01 were considered to be statistically significant.

Table 2 presents the demographic characteristics of the cohorts before (2008-2012) and after (2013-2017) the publication of the CWC-PHM recommendations. Hospitalizations due to asthma represented the largest cohort with 142,067 cases, followed by hospitalizations due to bronchiolitis with 94,253 cases. Hospitalizations due to GER comprised the smallest cohort with 13,635 cases. Most of the children had government insurance and had “minor” severity according to the All Patient Revised Diagnosis Related Group (APR-DRG) system.

We found statistically significant differences in most of the demographic characteristics for the cohorts when comparing cases before and after the introduction of the CWC-PHM recommendations.

After adjusting for demographic characteristics, we estimated the percentages of the utilization of the “low-value” services from 2008 to 2017. We observed a steady decrease in overutilization of all services over time. The absolute percentage decrease was more evident in the reduction of the utilization of relievers by 36.6% and that of CXR by 31.5% for bronchiolitis. We also observed a 20.8% absolute reduction in the use of CXR for asthma.

The use of systemic steroids in LRTI revealed the lowest utilization among the “low-value” services studied, with 15.1% in 2008 and 12.2% in 2017, a 2.9% absolute reduction. However, the prescription of acid suppressors for GER showed the highest utilization among all the overuse metrics studied, ie, 63% in 2008 and 48.9% in 2017, with an absolute decrease of 24.1%. The yearly adjusted estimated percentages of utilization for each “low-value” service are presented in Appendix Table A.

Table 3 and the Figure (attached as supplemental online graphic) respectively present the risk-adjusted ITS parameter estimates and the graphic representation before and after the inception of the CWC-PHM recommendations for the trend analysis.

We found a steady reduction in the frequency of overutilization of five “low-value” services described in the CWC-PHM recommendations from 2008 to 2017 in 36 tertiary children’s hospitals in the US. This trend was more evident in the utilization of relievers and CXR for bronchiolitis. The ITS analysis demonstrated that immediately after the publication of the CWC-PHM recommendations, only the use of CXR for asthma decreased significantly. Then, only the use of relievers for bronchiolitis decreased significantly over time in comparison with the secular trend.

These results support our hypothesis for two of the five metrics studied, suggesting that the publication of the CWC-PHM recommendations had a modest impact in clinical practices related to those services in tertiary children’s hospitals.

These findings align with a limited number of published studies that have consistently found a modest decrease in the use of “low-value” services before 2012^11-13 and a limited impact of the CWC in clinical practices on the use of “low-value” services after the inception of the campaign.^14-17

For instance, in a cross-sectional analysis of the 1999 and 2009 samples of ambulatory care practices in the US, only two of 11 overuse quality indicators showed improvement.¹¹ The authors recognized that reducing inappropriate care will require the same attention to guideline development and performance measurement that was directed at reducing the underuse of needed therapies. However, determining whether a patient received inappropriate care generally requires a much more detailed analysis of clinical information than what is required for assessments of underuse.¹¹

Another study designed claims-based algorithms to measure the prevalence of 11 Choosing Wisely-identified “low-value” services in fee-for-service Medicare patients aged >65 years from 2006 to 2011.¹² The annual prevalence of selected CWC “low-value” services ranged from 1.2% (upper urinary tract imaging in men with benign prostatic hyperplasia) to 46.5% (preoperative cardiac testing for low-risk, noncardiac procedures). The study concluded that identifying and measuring “low-value” health services is a prerequisite for improving quality and eliminating waste.¹²

In pediatric medicine, the authors investigated a large cohort of infants aged one to 24 months hospitalized with bronchiolitis to 41 tertiary children’s hospitals reporting data to the PHIS database from 2004 to 2012.¹³ The trend analysis revealed a decrease in the utilization of diagnostics and treatment interventions before the publication of the American Academy of Pediatrics 2006 Bronchiolitis Guidelines.¹⁸ There was an additional reduction in the use of CXR, steroids, and bronchodilators after the publication of the guidelines.¹³

After the CWC was launched in 2012, several surveys have demonstrated a tangible increase in awareness of the CWC and its goals, mostly among primary care physicians and subspecialists. Clinicians who were aware of the campaign found the recommendations to be useful as a legitimate source of guidance and were more likely to reduce the indication of unnecessary care and “low-value” clinical services included in the CWC.^1,3,19,20

Few studies in adults have focused on measuring the trends in overuse metrics derived from the CWC recommendations.^14-16 The initial studies have found limited reduction on the use of “low-value” care after the inception of the CWC. They suggest that clinician education, awareness, and public promotion alone do not appear to be sufficient to achieve widespread changes in clinical practice. Additional interventions are necessary for the wider implementation and success of the CWC recommendations.^{11,14,15,19,21,22}

However, a more recent study was conducted in 91 academic centers from 2013 through 2016, before and after the publication of a CWC recommendation on the use of troponin-only testing for the diagnosis of acute myocardial infarction. Hospitals with low rates of troponin-only testing before the publication of the recommendation demonstrated a statistically significant increase over time in the rate of adherence. The authors postulated that the impact of the CWC might have been significant because of the increase in the institutional and provider attention to “high-value” care as a result of the campaign.¹⁶

In pediatrics, a cross-sectional study defined 20 “low-value” services from a list of more than 400 items from the CWC and other sources of highly regarded, evidence-based pediatrics healthcare recommendations. The list included six diagnostic tests, five imaging tests, and nine prescription drugs ordered in a robust cohort of 4.4 million children nationwide in 2014. The study concluded that approximately one in 10 children received a “low-value” service. The majority (59.4%) were related to prescription drugs, specifically the inappropriate use of antibiotics for a variety of conditions. The estimated combined cost of these unnecessary services was approximately $27 million, with one-third of the cost being paid out of pocket, arguing for significant financial harm. However, this study did not perform a trend analysis.¹⁷

Our results are comparable with these studies, reporting an initial increase in awareness and beliefs, followed by progressive changes in clinical practice among pediatric hospital-based clinicians in delivering evidence-based, high-value care after the CWC.

The attribution of the steady reduction in the absolute percentages of overuse/waste in the five metrics related to the CWC observed in this study, including the significant changes noted in two of the overuse indicators after the publication of the CWC-PHM recommendations, should be interpreted with caution. For example, the significant decrease in the use of “low-value” services in bronchiolitis could be attributed to multiple factors such as national guidelines released in 2014 after the campaign,²³ national multicenter QI collaborative projects,^24,25 and multiple local QI efforts.^26,27 The increase in the awareness and impact of the CWC recommendations among pediatric providers could also be a contributing factor, but this association cannot be established in the light of our findings.

On the other hand, despite extensive evidence for the lack of efficacy and the potential harm associated with the use of acid suppressors for uncomplicated GER in infants,^28-30 the frequency of this “low-value” therapeutic intervention remains high (~50%). The trend in utilization was not impacted by the CWC-PHM recommendations. This finding could be explained by several factors, including the possibility that several hospitalized patients may suffer from GER disease requiring acid suppressors. Another possibility is that acid suppressors are generally prescribed as an outpatient medication, and physicians treating inpatients may be reluctant to discontinue it during hospitalization. Nevertheless, this recommendation represents a target for review, update, and QI interventions in the near future.

The delivery of inappropriate “low-value” care represents the most significant dimension of waste in healthcare.³¹ The development of quality measures of “low-value” services representing overuse and waste is the most needed step toward assessing the magnitude of the problem. Overuse metrics could be incorporated into QI interventions to decrease the provision of such services. However, systematic efforts aimed at developing quality indicators of overuse based on the CWC recommendations have been limited. To our knowledge, this is the first study on the trends of metrics derived from the CWC recommendations in pediatric medicine.

Future research is needed to develop overuse metrics further to assess the specific outcomes related to the implementation of the CWC. How much has clinical practice changed as a result of the campaign? What are the outcomes and savings attributable to these efforts? These are critical questions for the immediate future that should be answered to sustain the ongoing efforts and results and to validate that the efforts are worthwhile.

This study has several limitations. First, this is a retrospective and observational study. It cannot prove a direct causal relationship between the publication of the CWC-PHM and the observed trends, as other potential factors may have contributed to the outcomes. Second, in administrative databases, the data quality is dependent on proper documentation and coding that may vary among reporting institutions. These data lack clinical information, and a fair assessment of “appropriateness” could be questioned. In addition, the study included only 36 academic, tertiary children’s hospitals. Because approximately two-thirds of all pediatric hospitalizations in the US occur in community settings,³² this study may not fully represent clinical practice in the majority of pediatric hospitalizations in the US. Finally, the validity of the ITS analysis has inherent limitations due to the variability of the data in some metrics that may affect the power of the analysis. This fact could lead to inaccurate conclusions regarding intervention effectiveness due to the data-driven model applied, as well as the lack of control for other time-varying confounders.³³

After seven years, the CWC faces important challenges. Critical to the success of the campaign is to “measure wisely” by developing quality indicators of overuse and operationalizing them into administrative and clinical data sources to assess the impact on clinical practice. Our study highlights some limited but steady reduction in the use of some “low-value” services before the campaign. It also demonstrates a modest impact of the campaign on clinical practices in tertiary care children’s hospitals in the US. Clinicians and institutions still have a long way to go in reducing the use of “low-value” interventions in pediatric medicine. These observations challenge us to step up our efforts to implement QI interventions aimed at incorporating these professional, society-endorsed recommendations into our clinical practice.

Acknowledgments

The authors thank Dr. Kristine De La Torre and Dr. Jennifer McCafferty-Fernandez and the Research Institute of Nicklaus Children’s Hospital for medical writing assistance. They also acknowledge Tatiana Consuegra, library technician, for her clerical assistance in the preparation and submission of this article.

The Choosing Wisely^® Campaign (CWC) was launched in 2012. This ongoing national initiative encourages conversations among patients and clinicians about the need —or the lack thereof—for frequent tests, treatments, and procedures in healthcare. More than 80 professional societies have developed short lists of evidence-based recommendations aimed at avoiding unnecessary, “low-value” care. More than 550 recommendations are currently available.¹ The Society of Hospital Medicine (SHM) Pediatric Committee published a list of five recommendations for the CWC in 2013.²

After seven years, the campaign has posted several success stories highlighting the increase in clinicians’ awareness about the recommendations. Several local, regional, and national initiatives and quality improvement (QI) projects have been inspired by the CWC and its tenants.^1,3 However, limited research has been performed on the true impact of these recommendations on avoiding “low-value” services. A more comprehensive approach is required to “measure wisely” the impact of the campaign on bedside clinical practice.⁴ Stakeholders in healthcare value have been challenged to collaborate in creating high-impact lists of “low-value” interventions and designing effective tools to measure their impact on clinical practice and costs.⁵

We initially developed a report card with five metrics derived from the CWC-SHM pediatric recommendations to help individual institutions and group practices to measure their performance and benchmark their results with peers.⁶ The report card is available for hospital members of the Children’s Hospital Association (CHA).⁷

The current study analyzes the frequency of utilization and trends of five metrics included in the CHA/Pediatric Health Information System^® (PHIS) CWC report card in tertiary children’s hospitals in the United States. We analyzed data from five years before and five years after the CWC-PHM recommendations were published in 2013. We hypothesize that the publication and dissemination of the CWC-PHM recommendations—the intervention—will result in either an immediate decrease in the use of the “low-value” services studied and/or a change in the trend of utilization over time.

Study Design

We conducted an observational, longitudinal retrospective study aimed at evaluating the impact of the CWC-PHM recommendations on clinical practice in tertiary children’s hospitals in the US.

Study Population

The population included inpatient and observation stays for children aged 0-18 years admitted to the 36 children’s hospitals consistently providing data from 2008 to 2017 to the PHIS administrative database (CHA, Lenexa, Kansas). This database contains inpatient, emergency department, ambulatory, and observation encounter–level data from more than 50 not-for-profit, tertiary care pediatric hospitals and accounts for ~20% of all pediatric hospitalizations in the US every year.

A joint effort between the CHA and the participating hospitals ensures the quality of the data submitted, as previously described.⁸ These data are subjected to a routine quality check with each submission and within each report. Data were fully deidentified for this study. In total, 36 PHIS hospitals met the strict quality standards for inclusion of submitted data. The remaining hospitals were excluded because they did not have complete data or had incomplete billing information.

For external benchmarking purposes, PHIS participating hospitals provide encounter data, including demographics, diagnoses, and procedures (International Classification of Diseases versions 9 and 10).^9,10 The transition from ICD-9 to ICD-10 in the US took place during the study period. However, the CHA completed a process of translating and mapping all ICD-9 codes to every possible equivalent ICD-10 code in the PHIS database. Thus, the change from ICD-9 to ICD-10 should not have had any significant effect on population definition and data analytics, including trend analysis.

For each condition, the study population was divided into the following two cohorts for comparison of the trends: all admissions from January 1, 2008 to December 31, 2012 (before) and all admissions from January 1, 2013 to December 31, 2017 (after) the CWC-PHM recommendations were published.

This study was determined to be nonhuman subject research and was therefore exempted by Nicklaus Children’s Hospital Human Research Protection Program.

Outcomes

The outcomes for this study were the percentages of patients receiving the not-recommended “low-value” services targeted by the CWC-PHM recommendations. For this purpose, four of the five recommendations were translated into the following five metrics, operationalized in the PHIS database and displayed in the “Choosing Wisely” report card:⁶

1. Percentage of patients with uncomplicated asthma receiving chest radiograph (CXR).

2. Percentage of patients with uncomplicated bronchiolitis receiving CXR.

3. Percentage of patients with uncomplicated bronchiolitis receiving bronchodilators.

4. Percentage of patients with lower respiratory tract infection (LRTI) receiving systemic corticosteroids (relievers).

5. Percentage of patients with uncomplicated gastroesophageal reflux (GER) receiving acid suppressor therapy.

The fifth recommendation—limiting the use of continuous pulse oximetry unless the patient is receiving supplemental oxygen—could not be operationalized in the PHIS database because of inconsistent reporting of these resources.⁶

The resulting percentages represent nonadherence to the recommendations, suggesting overuse of the specific “low-value” intervention. As such, a decreasing trend over time is the desired direction of improvement.

The definition of “uncomplicated” conditions and the metrics are presented in Table 1. A complete list of the inclusion and exclusion criteria to define “uncomplicated” conditions and the complete list of the clinical translation codes used in PHIS to identify the “low-value” services are presented as an electronic supplement.

Statistical Analyses

We compared the demographic and clinical characteristics of the various cohorts before and after the release of the CWC-PHM recommendations—the intervention—using chi-square statistics. To assess the individual hospital-level trends over time for each measure, we modeled the patient-level data of each hospital using generalized linear mixed effects models with a binomial distribution. These models were adjusted for patient demographic and clinical factors that were found to be significantly different (P < .01) before and after the intervention on bivariate analyses. From these models, we generated adjusted estimates for the quarterly percentages for each hospital. We then conducted an interrupted time series (ITS) using these estimates to compare trends in the five years before (2008-2012) and five years after (2013-2017) the publication of the CWC-PHM recommendations. For the ITS analysis, we used a generalized linear mixed effects model with the quarterly adjusted hospital-level utilization rates of “low-value” services for each cohort as the unit of analysis and a random intercept for each hospital. The model used an autoregressive(1) covariance structure to account for autocorrelation. The ITS allowed us to test our hypothesis by assessing the following two important features: (a) if a significant decrease occurred right after the CWC-PHM recommendations were published (level-change) and/or (b) if the intervention altered the secular trend (slope-change). All statistical analyses were performed using SAS v. 9.4 (SAS Institute, Cary, North Carolina), and P values <.01 were considered to be statistically significant.

Table 2 presents the demographic characteristics of the cohorts before (2008-2012) and after (2013-2017) the publication of the CWC-PHM recommendations. Hospitalizations due to asthma represented the largest cohort with 142,067 cases, followed by hospitalizations due to bronchiolitis with 94,253 cases. Hospitalizations due to GER comprised the smallest cohort with 13,635 cases. Most of the children had government insurance and had “minor” severity according to the All Patient Revised Diagnosis Related Group (APR-DRG) system.

We found statistically significant differences in most of the demographic characteristics for the cohorts when comparing cases before and after the introduction of the CWC-PHM recommendations.

After adjusting for demographic characteristics, we estimated the percentages of the utilization of the “low-value” services from 2008 to 2017. We observed a steady decrease in overutilization of all services over time. The absolute percentage decrease was more evident in the reduction of the utilization of relievers by 36.6% and that of CXR by 31.5% for bronchiolitis. We also observed a 20.8% absolute reduction in the use of CXR for asthma.

The use of systemic steroids in LRTI revealed the lowest utilization among the “low-value” services studied, with 15.1% in 2008 and 12.2% in 2017, a 2.9% absolute reduction. However, the prescription of acid suppressors for GER showed the highest utilization among all the overuse metrics studied, ie, 63% in 2008 and 48.9% in 2017, with an absolute decrease of 24.1%. The yearly adjusted estimated percentages of utilization for each “low-value” service are presented in Appendix Table A.

Table 3 and the Figure (attached as supplemental online graphic) respectively present the risk-adjusted ITS parameter estimates and the graphic representation before and after the inception of the CWC-PHM recommendations for the trend analysis.

We found a steady reduction in the frequency of overutilization of five “low-value” services described in the CWC-PHM recommendations from 2008 to 2017 in 36 tertiary children’s hospitals in the US. This trend was more evident in the utilization of relievers and CXR for bronchiolitis. The ITS analysis demonstrated that immediately after the publication of the CWC-PHM recommendations, only the use of CXR for asthma decreased significantly. Then, only the use of relievers for bronchiolitis decreased significantly over time in comparison with the secular trend.

These results support our hypothesis for two of the five metrics studied, suggesting that the publication of the CWC-PHM recommendations had a modest impact in clinical practices related to those services in tertiary children’s hospitals.

These findings align with a limited number of published studies that have consistently found a modest decrease in the use of “low-value” services before 2012^11-13 and a limited impact of the CWC in clinical practices on the use of “low-value” services after the inception of the campaign.^14-17

For instance, in a cross-sectional analysis of the 1999 and 2009 samples of ambulatory care practices in the US, only two of 11 overuse quality indicators showed improvement.¹¹ The authors recognized that reducing inappropriate care will require the same attention to guideline development and performance measurement that was directed at reducing the underuse of needed therapies. However, determining whether a patient received inappropriate care generally requires a much more detailed analysis of clinical information than what is required for assessments of underuse.¹¹

Another study designed claims-based algorithms to measure the prevalence of 11 Choosing Wisely-identified “low-value” services in fee-for-service Medicare patients aged >65 years from 2006 to 2011.¹² The annual prevalence of selected CWC “low-value” services ranged from 1.2% (upper urinary tract imaging in men with benign prostatic hyperplasia) to 46.5% (preoperative cardiac testing for low-risk, noncardiac procedures). The study concluded that identifying and measuring “low-value” health services is a prerequisite for improving quality and eliminating waste.¹²

In pediatric medicine, the authors investigated a large cohort of infants aged one to 24 months hospitalized with bronchiolitis to 41 tertiary children’s hospitals reporting data to the PHIS database from 2004 to 2012.¹³ The trend analysis revealed a decrease in the utilization of diagnostics and treatment interventions before the publication of the American Academy of Pediatrics 2006 Bronchiolitis Guidelines.¹⁸ There was an additional reduction in the use of CXR, steroids, and bronchodilators after the publication of the guidelines.¹³

After the CWC was launched in 2012, several surveys have demonstrated a tangible increase in awareness of the CWC and its goals, mostly among primary care physicians and subspecialists. Clinicians who were aware of the campaign found the recommendations to be useful as a legitimate source of guidance and were more likely to reduce the indication of unnecessary care and “low-value” clinical services included in the CWC.^1,3,19,20

Few studies in adults have focused on measuring the trends in overuse metrics derived from the CWC recommendations.^14-16 The initial studies have found limited reduction on the use of “low-value” care after the inception of the CWC. They suggest that clinician education, awareness, and public promotion alone do not appear to be sufficient to achieve widespread changes in clinical practice. Additional interventions are necessary for the wider implementation and success of the CWC recommendations.^{11,14,15,19,21,22}

However, a more recent study was conducted in 91 academic centers from 2013 through 2016, before and after the publication of a CWC recommendation on the use of troponin-only testing for the diagnosis of acute myocardial infarction. Hospitals with low rates of troponin-only testing before the publication of the recommendation demonstrated a statistically significant increase over time in the rate of adherence. The authors postulated that the impact of the CWC might have been significant because of the increase in the institutional and provider attention to “high-value” care as a result of the campaign.¹⁶

In pediatrics, a cross-sectional study defined 20 “low-value” services from a list of more than 400 items from the CWC and other sources of highly regarded, evidence-based pediatrics healthcare recommendations. The list included six diagnostic tests, five imaging tests, and nine prescription drugs ordered in a robust cohort of 4.4 million children nationwide in 2014. The study concluded that approximately one in 10 children received a “low-value” service. The majority (59.4%) were related to prescription drugs, specifically the inappropriate use of antibiotics for a variety of conditions. The estimated combined cost of these unnecessary services was approximately $27 million, with one-third of the cost being paid out of pocket, arguing for significant financial harm. However, this study did not perform a trend analysis.¹⁷

Our results are comparable with these studies, reporting an initial increase in awareness and beliefs, followed by progressive changes in clinical practice among pediatric hospital-based clinicians in delivering evidence-based, high-value care after the CWC.

The attribution of the steady reduction in the absolute percentages of overuse/waste in the five metrics related to the CWC observed in this study, including the significant changes noted in two of the overuse indicators after the publication of the CWC-PHM recommendations, should be interpreted with caution. For example, the significant decrease in the use of “low-value” services in bronchiolitis could be attributed to multiple factors such as national guidelines released in 2014 after the campaign,²³ national multicenter QI collaborative projects,^24,25 and multiple local QI efforts.^26,27 The increase in the awareness and impact of the CWC recommendations among pediatric providers could also be a contributing factor, but this association cannot be established in the light of our findings.

On the other hand, despite extensive evidence for the lack of efficacy and the potential harm associated with the use of acid suppressors for uncomplicated GER in infants,^28-30 the frequency of this “low-value” therapeutic intervention remains high (~50%). The trend in utilization was not impacted by the CWC-PHM recommendations. This finding could be explained by several factors, including the possibility that several hospitalized patients may suffer from GER disease requiring acid suppressors. Another possibility is that acid suppressors are generally prescribed as an outpatient medication, and physicians treating inpatients may be reluctant to discontinue it during hospitalization. Nevertheless, this recommendation represents a target for review, update, and QI interventions in the near future.

The delivery of inappropriate “low-value” care represents the most significant dimension of waste in healthcare.³¹ The development of quality measures of “low-value” services representing overuse and waste is the most needed step toward assessing the magnitude of the problem. Overuse metrics could be incorporated into QI interventions to decrease the provision of such services. However, systematic efforts aimed at developing quality indicators of overuse based on the CWC recommendations have been limited. To our knowledge, this is the first study on the trends of metrics derived from the CWC recommendations in pediatric medicine.

Future research is needed to develop overuse metrics further to assess the specific outcomes related to the implementation of the CWC. How much has clinical practice changed as a result of the campaign? What are the outcomes and savings attributable to these efforts? These are critical questions for the immediate future that should be answered to sustain the ongoing efforts and results and to validate that the efforts are worthwhile.

This study has several limitations. First, this is a retrospective and observational study. It cannot prove a direct causal relationship between the publication of the CWC-PHM and the observed trends, as other potential factors may have contributed to the outcomes. Second, in administrative databases, the data quality is dependent on proper documentation and coding that may vary among reporting institutions. These data lack clinical information, and a fair assessment of “appropriateness” could be questioned. In addition, the study included only 36 academic, tertiary children’s hospitals. Because approximately two-thirds of all pediatric hospitalizations in the US occur in community settings,³² this study may not fully represent clinical practice in the majority of pediatric hospitalizations in the US. Finally, the validity of the ITS analysis has inherent limitations due to the variability of the data in some metrics that may affect the power of the analysis. This fact could lead to inaccurate conclusions regarding intervention effectiveness due to the data-driven model applied, as well as the lack of control for other time-varying confounders.³³

After seven years, the CWC faces important challenges. Critical to the success of the campaign is to “measure wisely” by developing quality indicators of overuse and operationalizing them into administrative and clinical data sources to assess the impact on clinical practice. Our study highlights some limited but steady reduction in the use of some “low-value” services before the campaign. It also demonstrates a modest impact of the campaign on clinical practices in tertiary care children’s hospitals in the US. Clinicians and institutions still have a long way to go in reducing the use of “low-value” interventions in pediatric medicine. These observations challenge us to step up our efforts to implement QI interventions aimed at incorporating these professional, society-endorsed recommendations into our clinical practice.

Acknowledgments

The authors thank Dr. Kristine De La Torre and Dr. Jennifer McCafferty-Fernandez and the Research Institute of Nicklaus Children’s Hospital for medical writing assistance. They also acknowledge Tatiana Consuegra, library technician, for her clerical assistance in the preparation and submission of this article.