Identifying tweets of personal health experience through word embedding and LSTM neural network
Open Access
- 13 June 2018
- journal article
- conference paper
- Published by Springer Nature in BMC Bioinformatics
- Vol. 19 (S8) , 210-74
Abstract
As Twitter has become an active data source for health surveillance research, it is important that efficient and effective methods are developed to identify tweets related to personal health experience. Conventional classification algorithms rely on features engineered by human domain experts, and engineering such features is a challenging task and requires much human intelligence. The resultant features may not be optimal for the classification problem, and can make it challenging for conventional classifiers to correctly predict personal experience tweets (PETs) due to the various ways to express and/or describe personal experience in tweets. In this study, we developed a method that combines word embedding and long short-term memory (LSTM) model without the need to engineer any specific features. Through word embedding, tweet texts were represented as dense vectors which in turn were fed to the LSTM neural network as sequences. Statistical analyses of the results of 10-fold cross-validations of our method and conventional methods indicate that there exist significant differences (p < 0.01) in performance measures of accuracy, precision, recall, F1-score, and ROC/AUC, demonstrating that our approach outperforms the conventional methods in identifying PETs. We presented an efficient and effective method of identifying health-related personal experience tweets by combining word embedding and an LSTM neural network. It is conceivable that our method can help accelerate and scale up analyzing textual data of social media for health surveillance purposes, because of no need for the laborious and costly process of engineering features.Keywords
This publication has 26 references indexed in Scilit:
- Utilizing social media data for pharmacovigilance: A reviewJournal of Biomedical Informatics, 2015
- Deep learning in neural networks: An overviewNeural Networks, 2015
- Digital Drug Safety Surveillance: Monitoring Pharmaceutical Products in TwitterDrug Safety, 2014
- Sources of Information and Behavioral Patterns in Online Health Forums: Qualitative StudyJournal of Medical Internet Research, 2014
- Twitter Improves Influenza ForecastingPLoS Currents, 2014
- Using Twitter to Examine Smoking Behavior and Perceptions of Emerging Tobacco ProductsJournal of Medical Internet Research, 2013
- The presentation of seizures and epilepsy in YouTube videosEpilepsy & Behavior, 2013
- Mining Twitter Data for Potential Drug Effects2013
- Multi-column deep neural network for traffic sign classificationNeural Networks, 2012
- Learning to Forget: Continual Prediction with LSTMNeural Computation, 2000