Applying Machine Learning Models with An Ensemble Approach for Accurate Real-Time Influenza Forecasting in Taiwan: Development and Validation Study (Preprint)
Authors:
- Hao-Yuan ChengHCHao-Yuan Cheng
- Yu-Chun WuYWYu-Chun Wu
- Min-Hau LinMLMin-Hau Lin
- Yu-Lun LiuYLYu-Lun Liu
- Yue-Yang TsaiYTYue-Yang Tsai
- Jo-Hua WuJWJo-Hua Wu
- Ke-Han PanKPKe-Han Pan
- Chih-Jung KeCKChih-Jung Ke
- Chiu-Mei ChenCCChiu-Mei Chen
- Ding-Ping LiuDLDing-Ping Liu
- I-Feng LinILI-Feng Lin
- Jen-Hsiang ChuangJCJen-Hsiang Chuang
Preprint
- 8 July 2019
- preprint
- Published by JMIR Publications Inc. in JMIR Preprints
Abstract
BACKGROUND Changeful seasonal influenza activity in subtropical areas such as Taiwan causes problems in epidemic preparedness. The Taiwan Centers for Disease Control has maintained real-time national influenza surveillance systems since 2004. Except for timely monitoring, epidemic forecasting using the national influenza surveillance data can provide pivotal information for public health response. OBJECTIVE We aimed to develop predictive models using machine learning to provide real-time influenza-like illness forecasts. METHODS Using surveillance data of influenza-like illness visits from emergency departments (from the Real-Time Outbreak and Disease Surveillance System), outpatient departments (from the National Health Insurance database), and the records of patients with severe influenza with complications (from the National Notifiable Disease Surveillance System), we developed 4 machine learning models (autoregressive integrated moving average, random forest, support vector regression, and extreme gradient boosting) to produce weekly influenza-like illness predictions for a given week and 3 subsequent weeks. We established a framework of the machine learning models and used an ensemble approach called stacking to integrate these predictions. We trained the models using historical data from 2008-2014. We evaluated their predictive ability during 2015-2017 for each of the 4-week time periods using Pearson correlation, mean absolute percentage error (MAPE), and hit rate of trend prediction. A dashboard website was built to visualize the forecasts, and the results of real-world implementation of this forecasting framework in 2018 were evaluated using the same metrics. RESULTS All models could accurately predict the timing and magnitudes of the seasonal peaks in the then-current week (nowcast) (ρ=0.802-0.965; MAPE: 5.2%-9.2%; hit rate: 0.577-0.756), 1-week (ρ=0.803-0.918; MAPE: 8.3%-11.8%; hit rate: 0.643-0.747), 2-week (ρ=0.783-0.867; MAPE: 10.1%-15.3%; hit rate: 0.669-0.734), and 3-week forecasts (ρ=0.676-0.801; MAPE: 12.0%-18.9%; hit rate: 0.643-0.786), especially the ensemble model. In real-world implementation in 2018, the forecasting performance was still accurate in nowcasts (ρ=0.875-0.969; MAPE: 5.3%-8.0%; hit rate: 0.582-0.782) and remained satisfactory in 3-week forecasts (ρ=0.721-0.908; MAPE: 7.6%-13.5%; hit rate: 0.596-0.904). CONCLUSIONS This machine learning and ensemble approach can make accurate, real-time influenza-like illness forecasts for a 4-week period, and thus, facilitate decision making.Keywords
All Related Versions
- Published version: Journal of Medical Internet Research, 22 (8), e15394.
This publication has 27 references indexed in Scilit:
- Seasonal influenza prevention and control in Taiwan—Strategies revisitedJournal of the Formosan Medical Association, 2019
- Centennial review of influenza in TaiwanBiomedical Journal, 2018
- Forecasting influenza epidemics by integrating internet search queries and traditional surveillance data with the support vector machine regression model in Liaoning, from 2011 to 2015PeerJ, 2018
- An epidemic surge of influenza A(H3N2) virus at the end of the 2016–2017 season in Taiwan with an increased viral genetic heterogeneityJournal of Clinical Virology, 2018
- Modeling and Forecasting Influenza-like Illness (ILI) in Houston, Texas Using Three Surveillance Data Capture MechanismsOnline Journal of Public Health Informatics, 2017
- Real-Time Surveillance of Infectious Diseases: Taiwan's ExperienceHealth Security, 2017
- Cost-effectiveness analysis of quadrivalent versus trivalent influenza vaccine in Taiwan: A lifetime multi-cohort modelHuman Vaccines & Immunotherapeutics, 2016
- A systematic review of studies on forecasting the dynamics of influenza outbreaksInfluenza and Other Respiratory Viruses, 2013
- Nationwide Surveillance of Influenza during the Pandemic (2009–10) and Post-Pandemic (2010–11) Periods in TaiwanPLOS ONE, 2012
- Improving the Evidence Base for Decision Making During a Pandemic: The Example of 2009 Influenza A/H1N1Biosecurity and Bioterrorism: Biodefense Strategy, Practice, and Science, 2011