Apache IoTDB: A Time Series Database for Large Scale IoT Applications
Open Access
- 14 May 2025
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Database Systems
- Vol. 50 (2) , 1-45
Abstract
A typical industrial scenario encounters thousands of devices with millions of sensors, consistently generating billions of data points. It poses new requirements of time series data management, not well addressed in existing solutions, including (1) device-defined ever-evolving schema, (2) mostly periodical data collection, (3) strongly correlated series, (4) variously delayed data arrival, and (5) highly concurrent data ingestion. In this paper, we present a time series database management system, Apache IoTDB. It consists of (i) a time series native file format, TsFile, with specially designed data encoding, and (ii) an IoTDB engine for efficiently handling delayed data arrivals and processing queries. We introduce a native distributed solution with distributed queries optimized by parallel operators. We also explore efficient TsFile synchronization mechanisms, ensuring seamless data integration without the need for ETL processes. The system achieves a throughput of 10 million inserted values per second. Queries such as 1-day data selection of 0.1 million points and 3-year data aggregation over 10 million points can be processed in 100 ms. Comparisons with InfluxDB, TimescaleDB, KairosDB, Parquet and ORC over real world data loads demonstrate the superiority of IoTDB and TsFile.Keywords
Funding Information
- National Natural Science Foundation of China (92267203, 62021002, 62072265, 62232005)
- National Key Research and Development Plan (2021YFB3300500)
- Beijing National Research Center For Information Science And Technology (BNR2025RC01011, 31511130201)
- Alibaba Group through Alibaba Innovative Research (AIR) Program
This publication has 25 references indexed in Scilit:
- Apache IoTDB: A Time Series Database for IoT ApplicationsProceedings of the ACM on Management of Data, 2023
- SOCA-DOM: A Mobile System-on-Chip Array System for Analyzing Big Data on the MoveJournal of Computer Science and Technology, 2022
- Confidence Bounded Replica Currency Estimation2022
- ByteSeries2020
- Time Series Management Systems: A SurveyIEEE Transactions on Knowledge and Data Engineering, 2017
- CassandraACM SIGOPS Operating Systems Review, 2010
- MapReduceCommunications of the ACM, 2008
- The log-structured merge-tree (LSM-tree)Acta Informatica, 1996
- Differential filesACM Transactions on Database Systems, 1976
- Run-length encodings (Corresp.)IEEE Transactions on Information Theory, 1966