John Levander
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
John Levander has a total of 229 co-authors. Here are the top co-authors by number of shared publications:
- Alessandro Vespignani
Network Science Institute, Northeastern University, Boston, MA, USA
5 shared publications Amanda Wilson
University of Virginia, Charlottesville, USA
5 shared publications- Bryan Lewis
University of Virginia, Charlottesville, VA, USA
5 shared publications - Emily Howerton
Department of Ecology and Evolutionary Biology, Princeton University, Princeton, USA
5 shared publications - Harry Hochheiser
Department of Biomedical Informatics, University of Pittsburgh , Pittsburgh, PA
5 shared publications - Jean-Claude Thill
Department of Earth, Environmental and Geographical Sciences University of North Carolina at Charlotte Charlotte North Carolina USA
5 shared publications
Latest Publications
- Open Access
- 3 September 2025
- Published by JMIR Publications Inc. in JMIR Bioinformatics and Biotechnology
- Vol. 6, e67801
Systemic Anticancer Therapy Timelines Extraction From Electronic Medical Records Text: Algorithm Development and Validation
Background: The systemic treatment of cancer typically requires the use of multiple anticancer agents in combination or sequentially. Clinical narrative texts often contain extensive descriptions of the temporal sequencing of systemic anticancer therapy (SACT), setting up an important task that may be amenable to automated extraction of SACT timelines. Objective: We aimed to explore automatic methods for extracting patient-level SACT timelines from clinical narratives in the electronic medical records (EMRs). Methods: We used two datasets from two institutions: (1) a colorectal cancer (CRC) dataset including the entire EMR of the 199 patients in the THYME (Temporal Histories of Your Medical Event) dataset and (2) the 2024 ChemoTimelines shared task dataset including 149 patients with ovarian cancer, breast cancer, and melanoma. We explored finetuning smaller language models trained to attend to events and time expressions, and few-shot prompting of large language models (LLMs). Evaluation used the 2024 ChemoTimelines shared task configuration—Subtask1 involving the construction of SACT timelines from manually annotated SACT event and time expression mentions provided as input in addition to the patient’s notes and Subtask2 requiring extraction of SACT timelines directly from the patient’s notes. Results: Our task-specific finetuned EntityBERT model achieved 93% F1-score, outperforming the best results in Subtask1 of the 2024 ChemoTimelines shared task (90%). It ranked second in Subtask2. LLM (LLaMA2, LLaMA3.1, and Mixtral) performance lagged the task-specific finetuned model performance for both the THYME and shared task datasets. On the shared task datasets, the best LLM performance was 77% macro F1-score, 16% points lower than the task-specific finetuned system (Subtask1). Conclusions: In this paper, we explored approaches for patient-level timeline extraction through the SACT timeline extraction task. Our results and analysis add to the knowledge of extracting treatment timelines from EMR clinical narratives using language modeling methods.
- 3 September 2025
- Published by JMIR Publications Inc. in JMIR Bioinformatics and Biotechnology
- Vol. 6, e67801
Open AccessSystemic Anticancer Therapy Timelines Extraction From Electronic Medical Records Text: Algorithm Development and Validation
Background: The systemic treatment of cancer typically requires the use of multiple anticancer agents in combination or sequentially. Clinical narrative texts often contain extensive descriptions of the temporal sequencing of systemic anticancer therapy (SACT), setting up an important task that may be amenable to automated extraction of SACT timelines. Objective: We aimed to explore automatic methods for extracting patient-level SACT timelines from clinical narratives in the electronic medical records (EMRs). Methods: We used two datasets from two institutions: (1) a colorectal cancer (CRC) dataset including the entire EMR of the 199 patients in the THYME (Temporal Histories of Your Medical Event) dataset and (2) the 2024 ChemoTimelines shared task dataset including 149 patients with ovarian cancer, breast cancer, and melanoma. We explored finetuning smaller language models trained to attend to events and time expressions, and few-shot prompting of large language models (LLMs). Evaluation used the 2024 ChemoTimelines shared task configuration—Subtask1 involving the construction of SACT timelines from manually annotated SACT event and time expression mentions provided as input in addition to the patient’s notes and Subtask2 requiring extraction of SACT timelines directly from the patient’s notes. Results: Our task-specific finetuned EntityBERT model achieved 93% F1-score, outperforming the best results in Subtask1 of the 2024 ChemoTimelines shared task (90%). It ranked second in Subtask2. LLM (LLaMA2, LLaMA3.1, and Mixtral) performance lagged the task-specific finetuned model performance for both the THYME and shared task datasets. On the shared task datasets, the best LLM performance was 77% macro F1-score, 16% points lower than the task-specific finetuned system (Subtask1). Conclusions: In this paper, we explored approaches for patient-level timeline extraction through the SACT timeline extraction task. Our results and analysis add to the knowledge of extracting treatment timelines from EMR clinical narratives using language modeling methods.
- Top Cited PapersOpen Access
- 20 November 2023
- Published by Springer Nature in Nature Communications
- Vol. 14 (1) , 1-15
Evaluation of the US COVID-19 Scenario Modeling Hub for informing pandemic response under uncertainty
Our ability to forecast epidemics far into the future is constrained by the many complexities of disease systems. Realistic longer-term projections may, however, be possible under well-defined scenarios that specify the future state of critical epidemic drivers. Since December 2020, the U.S. COVID-19 Scenario Modeling Hub (SMH) has convened multiple modeling teams to make months ahead projections of SARS-CoV-2 burden, totaling nearly 1.8 million national and state-level projections. Here, we find SMH performance varied widely as a function of both scenario validity and model calibration. We show scenarios remained close to reality for 22 weeks on average before the arrival of unanticipated SARS-CoV-2 variants invalidated key assumptions. An ensemble of participating models that preserved variation between models (using the linear opinion pool method) was consistently more reliable than any single model in periods of valid scenario assumptions, while projection interval coverage was near target levels. SMH projections were used to guide pandemic response, illustrating the value of collaborative hubs for longer-term scenario projections.
- 20 November 2023
- Published by Springer Nature in Nature Communications
- Vol. 14 (1) , 1-15
Top Cited PapersOpen AccessEvaluation of the US COVID-19 Scenario Modeling Hub for informing pandemic response under uncertainty
Our ability to forecast epidemics far into the future is constrained by the many complexities of disease systems. Realistic longer-term projections may, however, be possible under well-defined scenarios that specify the future state of critical epidemic drivers. Since December 2020, the U.S. COVID-19 Scenario Modeling Hub (SMH) has convened multiple modeling teams to make months ahead projections of SARS-CoV-2 burden, totaling nearly 1.8 million national and state-level projections. Here, we find SMH performance varied widely as a function of both scenario validity and model calibration. We show scenarios remained close to reality for 22 weeks on average before the arrival of unanticipated SARS-CoV-2 variants invalidated key assumptions. An ensemble of participating models that preserved variation between models (using the linear opinion pool method) was consistently more reliable than any single model in periods of valid scenario assumptions, while projection interval coverage was near target levels. SMH projections were used to guide pandemic response, illustrating the value of collaborative hubs for longer-term scenario projections.
- Open Access
- 26 October 2023
- Published by Cold Spring Harbor Laboratory in medRxiv
Potential impact of annual vaccination with reformulated COVID-19 vaccines: lessons from the U.S. COVID-19 Scenario Modeling Hub
Importance: COVID-19 continues to cause significant hospitalizations and deaths in the United States. Its continued burden and the impact of annually reformulated vaccines remain unclear.Objective: To project COVID-19 hospitalizations and deaths from April 2023–April 2025 under two plausible assumptions about immune escape (20% per year and 50% per year) and three possible CDC recommendations for the use of annually reformulated vaccines (no vaccine recommendation, vaccination for those aged 65+, vaccination for all eligible groups).Design: The COVID-19 Scenario Modeling Hub solicited projections of COVID-19 hospitalization and deaths between April 15, 2023–April 15, 2025 under six scenarios representing the intersection of considered levels of immune escape and vaccination. State and national projections from eight modeling teams were ensembled to produce projections for each scenario.Setting: The entire United States.Participants: None.Exposure: Annually reformulated vaccines assumed to be 65% effective against strains circulating on June 15 of each year and to become available on September 1. Age and state specific coverage in recommended groups was assumed to match that seen for the first (fall 2021) COVID-19 booster.Main outcomes and measures: Ensemble estimates of weekly and cumulative COVID-19 hospitalizations and deaths. Expected relative and absolute reductions in hospitalizations and deaths due to vaccination over the projection period.Results: From April 15, 2023–April 15, 2025, COVID-19 is projected to cause annual epidemics peaking November–January. In the most pessimistic scenario (high immune escape, no vaccination recommendation), we project 2.1 million (90% PI: 1,438,000–4,270,000) hospitalizations and 209,000 (90% PI: 139,000–461,000) deaths, exceeding pre-pandemic mortality of influenza and pneumonia. In high immune escape scenarios, vaccination of those aged 65+ results in 230,000 (95% CI: 104,000–355,000) fewer hospitalizations and 33,000 (95% CI: 12,000–54,000) fewer deaths, while vaccination of all eligible individuals results in 431,000 (95% CI: 264,000–598,000) fewer hospitalizations and 49,000 (95% CI: 29,000–69,000) fewer deaths.Conclusion and Relevance: COVID-19 is projected to be a significant public health threat over the coming two years. Broad vaccination has the potential to substantially reduce the burden of this disease.Key points: Question: What is the likely impact of COVID-19 from April 2023–April 2025 and to what extent can vaccination reduce hospitalizations and deaths?Findings: Under plausible assumptions about viral evolution and waning immunity, COVID-19 will likely cause annual epidemics peaking in November–January over the two-year projection period. Though significant, hospitalizations and deaths are unlikely to reach levels seen in previous winters. The projected health impacts of COVID-19 are reduced by 10–20% through moderate use of reformulated vaccines.Meaning: COVID-19 is projected to remain a significant public health threat. Annual vaccination can reduce morbidity, mortality, and strain on health systems.
- 26 October 2023
- Published by Cold Spring Harbor Laboratory in medRxiv
Open AccessPotential impact of annual vaccination with reformulated COVID-19 vaccines: lessons from the U.S. COVID-19 Scenario Modeling Hub
Importance: COVID-19 continues to cause significant hospitalizations and deaths in the United States. Its continued burden and the impact of annually reformulated vaccines remain unclear.Objective: To project COVID-19 hospitalizations and deaths from April 2023–April 2025 under two plausible assumptions about immune escape (20% per year and 50% per year) and three possible CDC recommendations for the use of annually reformulated vaccines (no vaccine recommendation, vaccination for those aged 65+, vaccination for all eligible groups).Design: The COVID-19 Scenario Modeling Hub solicited projections of COVID-19 hospitalization and deaths between April 15, 2023–April 15, 2025 under six scenarios representing the intersection of considered levels of immune escape and vaccination. State and national projections from eight modeling teams were ensembled to produce projections for each scenario.Setting: The entire United States.Participants: None.Exposure: Annually reformulated vaccines assumed to be 65% effective against strains circulating on June 15 of each year and to become available on September 1. Age and state specific coverage in recommended groups was assumed to match that seen for the first (fall 2021) COVID-19 booster.Main outcomes and measures: Ensemble estimates of weekly and cumulative COVID-19 hospitalizations and deaths. Expected relative and absolute reductions in hospitalizations and deaths due to vaccination over the projection period.Results: From April 15, 2023–April 15, 2025, COVID-19 is projected to cause annual epidemics peaking November–January. In the most pessimistic scenario (high immune escape, no vaccination recommendation), we project 2.1 million (90% PI: 1,438,000–4,270,000) hospitalizations and 209,000 (90% PI: 139,000–461,000) deaths, exceeding pre-pandemic mortality of influenza and pneumonia. In high immune escape scenarios, vaccination of those aged 65+ results in 230,000 (95% CI: 104,000–355,000) fewer hospitalizations and 33,000 (95% CI: 12,000–54,000) fewer deaths, while vaccination of all eligible individuals results in 431,000 (95% CI: 264,000–598,000) fewer hospitalizations and 49,000 (95% CI: 29,000–69,000) fewer deaths.Conclusion and Relevance: COVID-19 is projected to be a significant public health threat over the coming two years. Broad vaccination has the potential to substantially reduce the burden of this disease.Key points: Question: What is the likely impact of COVID-19 from April 2023–April 2025 and to what extent can vaccination reduce hospitalizations and deaths?Findings: Under plausible assumptions about viral evolution and waning immunity, COVID-19 will likely cause annual epidemics peaking in November–January over the two-year projection period. Though significant, hospitalizations and deaths are unlikely to reach levels seen in previous winters. The projected health impacts of COVID-19 are reduced by 10–20% through moderate use of reformulated vaccines.Meaning: COVID-19 is projected to remain a significant public health threat. Annual vaccination can reduce morbidity, mortality, and strain on health systems.
- Open Access
Informing pandemic response in the face of uncertainty. An evaluation of the U.S. COVID-19 Scenario Modeling Hub
Our ability to forecast epidemics more than a few weeks into the future is constrained by the complexity of disease systems, our limited ability to measure the current state of an epidemic, and uncertainties in how human action will affect transmission. Realistic longer-term projections (spanning more than a few weeks) may, however, be possible under defined scenarios that specify the future state of critical epidemic drivers, with the additional benefit that such scenarios can be used to anticipate the comparative effect of control measures. Since December 2020, the U.S. COVID-19 Scenario Modeling Hub (SMH) has convened multiple modeling teams to make 6-month ahead projections of the number of SARS-CoV-2 cases, hospitalizations and deaths. The SMH released nearly 1.8 million national and state-level projections between February 2021 and November 2022. SMH performance varied widely as a function of both scenario validity and model calibration. Scenario assumptions were periodically invalidated by the arrival of unanticipated SARS-CoV-2 variants, but SMH still provided projections on average 22 weeks before changes in assumptions (such as virus transmissibility) invalidated scenarios and their corresponding projections. During these periods, before emergence of a novel variant, a linear opinion pool ensemble of contributed models was consistently more reliable than any single model, and projection interval coverage was near target levels for the most plausible scenarios (e.g., 79% coverage for 95% projection interval). SMH projections were used operationally to guide planning and policy at different stages of the pandemic, illustrating the value of the hub approach for long-term scenario projections.
Open AccessInforming pandemic response in the face of uncertainty. An evaluation of the U.S. COVID-19 Scenario Modeling Hub
Our ability to forecast epidemics more than a few weeks into the future is constrained by the complexity of disease systems, our limited ability to measure the current state of an epidemic, and uncertainties in how human action will affect transmission. Realistic longer-term projections (spanning more than a few weeks) may, however, be possible under defined scenarios that specify the future state of critical epidemic drivers, with the additional benefit that such scenarios can be used to anticipate the comparative effect of control measures. Since December 2020, the U.S. COVID-19 Scenario Modeling Hub (SMH) has convened multiple modeling teams to make 6-month ahead projections of the number of SARS-CoV-2 cases, hospitalizations and deaths. The SMH released nearly 1.8 million national and state-level projections between February 2021 and November 2022. SMH performance varied widely as a function of both scenario validity and model calibration. Scenario assumptions were periodically invalidated by the arrival of unanticipated SARS-CoV-2 variants, but SMH still provided projections on average 22 weeks before changes in assumptions (such as virus transmissibility) invalidated scenarios and their corresponding projections. During these periods, before emergence of a novel variant, a linear opinion pool ensemble of contributed models was consistently more reliable than any single model, and projection interval coverage was near target levels for the most plausible scenarios (e.g., 79% coverage for 95% projection interval). SMH projections were used operationally to guide planning and policy at different stages of the pandemic, illustrating the value of the hub approach for long-term scenario projections.
- Top Cited PapersOpen Access
- 25 April 2023
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 120 (18)
Multiple models for outbreak decision support in the face of uncertainty
Policymakers must make management decisions despite incomplete knowledge and conflicting model projections. Little guidance exists for the rapid, representative, and unbiased collection of policy-relevant scientific input from independent modeling teams. Integrating approaches from decision analysis, expert judgment, and model aggregation, we convened multiple modeling teams to evaluate COVID-19 reopening strategies for a mid-sized United States county early in the pandemic. Projections from seventeen distinct models were inconsistent in magnitude but highly consistent in ranking interventions. The 6-mo-ahead aggregate projections were well in line with observed outbreaks in mid-sized US counties. The aggregate results showed that up to half the population could be infected with full workplace reopening, while workplace restrictions reduced median cumulative infections by 82%. Rankings of interventions were consistent across public health objectives, but there was a strong trade-off between public health outcomes and duration of workplace closures, and no win-win intermediate reopening strategies were identified. Between-model variation was high; the aggregate results thus provide valuable risk quantification for decision making. This approach can be applied to the evaluation of management interventions in any setting where models are used to inform decision making. This case study demonstrated the utility of our approach and was one of several multimodel efforts that laid the groundwork for the COVID-19 Scenario Modeling Hub, which has provided multiple rounds of real-time scenario projections for situational awareness and decision making to the Centers for Disease Control and Prevention since December 2020.
- 25 April 2023
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 120 (18)
Top Cited PapersOpen AccessMultiple models for outbreak decision support in the face of uncertainty
Policymakers must make management decisions despite incomplete knowledge and conflicting model projections. Little guidance exists for the rapid, representative, and unbiased collection of policy-relevant scientific input from independent modeling teams. Integrating approaches from decision analysis, expert judgment, and model aggregation, we convened multiple modeling teams to evaluate COVID-19 reopening strategies for a mid-sized United States county early in the pandemic. Projections from seventeen distinct models were inconsistent in magnitude but highly consistent in ranking interventions. The 6-mo-ahead aggregate projections were well in line with observed outbreaks in mid-sized US counties. The aggregate results showed that up to half the population could be infected with full workplace reopening, while workplace restrictions reduced median cumulative infections by 82%. Rankings of interventions were consistent across public health objectives, but there was a strong trade-off between public health outcomes and duration of workplace closures, and no win-win intermediate reopening strategies were identified. Between-model variation was high; the aggregate results thus provide valuable risk quantification for decision making. This approach can be applied to the evaluation of management interventions in any setting where models are used to inform decision making. This case study demonstrated the utility of our approach and was one of several multimodel efforts that laid the groundwork for the COVID-19 Scenario Modeling Hub, which has provided multiple rounds of real-time scenario projections for situational awareness and decision making to the Centers for Disease Control and Prevention since December 2020.
- Open Access
- 22 November 2022
- Published by Elsevier in The Lancet Regional Health - Americas
- Vol. 17, 100398
Impact of SARS-CoV-2 vaccination of children ages 5–11 years on COVID-19 disease burden and resilience to new variants in the United States, November 2021–March 2022: a multi-model study
- 22 November 2022
- Published by Elsevier in The Lancet Regional Health - Americas
- Vol. 17, 100398
Open AccessImpact of SARS-CoV-2 vaccination of children ages 5–11 years on COVID-19 disease burden and resilience to new variants in the United States, November 2021–March 2022: a multi-model study
- Open Access
Impact of SARS-CoV-2 vaccination of children ages 5-11 years on COVID-19 disease burden and resilience to new variants in the United States, November 2021-March 2022: a multi-model study
Background SARS-CoV-2 vaccination of persons aged 12 years and older has reduced disease burden in the United States. The COVID-19 Scenario Modeling Hub convened multiple modeling teams in September 2021 to project the impact of expanding vaccine administration to children 5-11 years old on anticipated COVID-19 burden and resilience against variant strains. Methods Nine modeling teams contributed state- and national-level projections for weekly counts of cases, hospitalizations, and deaths in the United States for the period September 12, 2021 to March 12, 2022. Four scenarios covered all combinations of: 1) presence vs. absence of vaccination of children ages 5-11 years starting on November 1, 2021; and 2) continued dominance of the Delta variant vs. emergence of a hypothetical more transmissible variant on November 15, 2021. Individual team projections were combined using linear pooling. The effect of childhood vaccination on overall and age-specific outcomes was estimated by meta-analysis approaches. Findings Absent a new variant, COVID-19 cases, hospitalizations, and deaths among all ages were projected to decrease nationally through mid-March 2022. Under a set of specific assumptions, models projected that vaccination of children 5-11 years old was associated with reductions in all-age cumulative cases (7.2%, mean incidence ratio [IR] 0.928, 95% confidence interval [CI] 0.880-0.977), hospitalizations (8.7%, mean IR 0.913, 95% CI 0.834-0.992), and deaths (9.2%, mean IR 0.908, 95% CI 0.797-1.020) compared with scenarios where children were not vaccinated. This projected effect of vaccinating children 5-11 years old increased in the presence of a more transmissible variant, assuming no change in vaccine effectiveness by variant. Larger relative reductions in cumulative cases, hospitalizations, and deaths were observed for children than for the entire U.S. population. Substantial state-level variation was projected in epidemic trajectories, vaccine benefits, and variant impacts. Conclusions Results from this multi-model aggregation study suggest that, under a specific set of scenario assumptions, expanding vaccination to children 5-11 years old would provide measurable direct benefits to this age group and indirect benefits to the all-age U.S. population, including resilience to more transmissible variants.
Open AccessImpact of SARS-CoV-2 vaccination of children ages 5-11 years on COVID-19 disease burden and resilience to new variants in the United States, November 2021-March 2022: a multi-model study
Background SARS-CoV-2 vaccination of persons aged 12 years and older has reduced disease burden in the United States. The COVID-19 Scenario Modeling Hub convened multiple modeling teams in September 2021 to project the impact of expanding vaccine administration to children 5-11 years old on anticipated COVID-19 burden and resilience against variant strains. Methods Nine modeling teams contributed state- and national-level projections for weekly counts of cases, hospitalizations, and deaths in the United States for the period September 12, 2021 to March 12, 2022. Four scenarios covered all combinations of: 1) presence vs. absence of vaccination of children ages 5-11 years starting on November 1, 2021; and 2) continued dominance of the Delta variant vs. emergence of a hypothetical more transmissible variant on November 15, 2021. Individual team projections were combined using linear pooling. The effect of childhood vaccination on overall and age-specific outcomes was estimated by meta-analysis approaches. Findings Absent a new variant, COVID-19 cases, hospitalizations, and deaths among all ages were projected to decrease nationally through mid-March 2022. Under a set of specific assumptions, models projected that vaccination of children 5-11 years old was associated with reductions in all-age cumulative cases (7.2%, mean incidence ratio [IR] 0.928, 95% confidence interval [CI] 0.880-0.977), hospitalizations (8.7%, mean IR 0.913, 95% CI 0.834-0.992), and deaths (9.2%, mean IR 0.908, 95% CI 0.797-1.020) compared with scenarios where children were not vaccinated. This projected effect of vaccinating children 5-11 years old increased in the presence of a more transmissible variant, assuming no change in vaccine effectiveness by variant. Larger relative reductions in cumulative cases, hospitalizations, and deaths were observed for children than for the entire U.S. population. Substantial state-level variation was projected in epidemic trajectories, vaccine benefits, and variant impacts. Conclusions Results from this multi-model aggregation study suggest that, under a specific set of scenario assumptions, expanding vaccination to children 5-11 years old would provide measurable direct benefits to this age group and indirect benefits to the all-age U.S. population, including resilience to more transmissible variants.