Time Series Representation for Visualization in Apache IoTDB
Open Access
- 12 March 2024
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Management of Data
- Vol. 2 (1) , 1-26
Abstract
When analyzing time series, often interactively, the analysts frequently demand to visualize instantly large-scale data stored in databases. M4 visualization selects the first, last, bottom and top data points in each pixel column to ensure pixel-perfectness of the two-color line chart visualization. While M4 already shows its preciseness of encasing time series in different scales into a fixed size of pixels, how to efficiently support M4 representation in a time series native database is still absent. It is worth noting that, to enable fast writes, the commodity time series database systems, such as Apache IoTDB or InfluxDB, employ LSM-Tree based storage. That is, a time series is segmented and stored in a number of chunks, with possibly out-of-order arrivals, i.e., disordered on timestamps. To implement M4, a natural idea is to merge online the chunks as a whole series, with costly merge sort on timestamps, and then perform M4 representation as in relational databases. In this study, we propose a novel chunk merge free approach called M4-LSM to accelerate M4 representation and visualization. In particular, we utilize the metadata of chunks to prune and avoid the costly merging of any chunk. Moreover, intra-chunk indexing and pruning are enabled for efficiently accessing the representation points, referring to the special properties of time series. Remarkably, the time series database native operator M4-LSM has been implemented in Apache IoTDB, an open-source time series database, and deployed in companies across various industries. In the experiments over real-world datasets, the proposed M4-LSM operator demonstrates high efficiency without sacrificing preciseness.Keywords
Funding Information
- National Key Research and Development Plan (2021YFB3300500)
This publication has 29 references indexed in Scilit:
- A Prefetch-Adaptive Intelligent Cache Replacement Policy Based on Machine LearningJournal of Computer Science and Technology, 2023
- Construct Trip Graphs by Using Taxi Trajectory DataData Science and Engineering, 2023
- Plotly-Resampler: Effective Visual Analytics for Large Time Series2022
- KD-Box: Line-segment-based KD-tree for Interactive Exploration of Large-scale Time-Series DataIEEE Transactions on Visualization and Computer Graphics, 2021
- Time Series Management Systems: A SurveyIEEE Transactions on Knowledge and Data Engineering, 2017
- The Effects of Interactive Latency on Exploratory Visual AnalysisIEEE Transactions on Visualization and Computer Graphics, 2014
- Experiencing SAX: a novel symbolic representation of time seriesData Mining and Knowledge Discovery, 2007
- Dimensionality Reduction for Fast Similarity Search in Large Time Series DatabasesKnowledge and Information Systems, 2001
- The log-structured merge-tree (LSM-tree)Acta Informatica, 1996
- Efficient similarity search in sequence databases1993