Time series discord detection in medical data using a parallel relational database
Authors:
- 1 November 2015
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1420-1426
Abstract
Recent advances in sensor technology have made continuous real-time health monitoring available in both hospital and non-hospital settings. Since data collected from high frequency medical sensors includes a huge amount of data, storing and processing continuous medical data is an emerging big data area. Especially detecting anomaly in real time is important for patients' emergency detection and prevention. A time series discord indicates a subsequence that has the maximum difference to the rest of the time series subsequences, meaning that it has abnormal or unusual data trends. In this study, we implemented two versions of time series discord detection algorithms on a high performance parallel database management system (DBMS) and applied them to 240 Hz waveform data collected from 9,723 patients. The initial brute force version of the discord detection algorithm takes each possible subsequence and calculates a distance to the nearest non-self match to find the biggest discords in time series. For the heuristic version of the algorithm, a combination of an array and a trie structure was applied to order time series data for enhancing time efficiency. The study results showed efficient data loading, decoding and discord searches in a large amount of data, benefiting from the time series discord detection algorithm and the architectural characteristics of the parallel DBMS including data compression, data pipe-lining, and task scheduling.Keywords
This publication has 12 references indexed in Scilit:
- Design and Development of a Medical Big Data Processing System Based on HadoopJournal of Medical Systems, 2015
- Some Novel Heuristics for Finding the Most Unusual Time Series Subsequences2010
- MapReduce and parallel DBMSsCommunications of the ACM, 2010
- MapReduceCommunications of the ACM, 2008
- ECG Anomaly Detection via Time Series Analysis2007
- Finding the most unusual time series subsequence: algorithms and applicationsKnowledge and Information Systems, 2006
- HOT SAX: Efficiently Finding the Most Unusual Time Series Subsequence2006
- Discovering Representative Models in Large Time Series Databases2004
- A symbolic representation of time series, with implications for streaming algorithms2003
- Compressed Text Databases with Efficient Query Algorithms Based on the Compressed Suffix ArrayLecture Notes in Computer Science, 2000