Dejiao Niu

The Log‐Structured Merge Tree (LSM‐tree) is widely adopted in key‐value (KV) storage systems because it transforms random writes into sequential writes. It has also become an important storage structure for supporting highly concurrent data access in high‐performance computing, cloud computing, and large‐scale data‐intensive applications. However, LSM‐tree still suffers from write amplification and periodic performance degradation, especially on emerging storage devices with strict sequential‐write constraints, such as Zoned Namespace Solid State Drives (ZNS SSDs). Meanwhile, Non‐Volatile Memory (NVM) provides new opportunities for optimizing the internal organization of LSM‐tree‐based storage systems. We observe that traditional SSTables tightly couple metadata with data blocks, causing frequent metadata updates to trigger unnecessary rewrites of large data regions and further exacerbating write amplification. To address this problem, we propose DS‐LSM‐tree, an LSM‐tree design based on SSTable decoupling. In DS‐LSM‐tree, large and low‐update‐frequency data blocks are sequentially stored on ZNS SSDs, while frequently updated fine‐grained metadata are migrated to NVM, thereby alleviating the mismatch between update granularity and storage device characteristics. Based on the decoupled SSTable design, we further develop a ZNS‐aware SSTable metadata structure and propose an NVM‐based logical merge strategy for SSTable data blocks, which reduces unnecessary physical data migration during compaction. We implement a prototype system named SPMS based on DS‐LSM‐tree and evaluate it using YCSB benchmarks. Experimental results show that, compared with ListDB, SPMS improves write throughput by up to 51.2%, increases read throughput by 5.1%, and reduces write amplification by 53.1%.

The LSM‐tree based on extendible hashing has been adopted in key‐value storage systems due to its high write throughput, good scalability, and balanced read‐write performance. However, it faces several challenges when accessing SSTables, including low cache efficiency, uneven data distribution across hash buckets, and frequent directory expansions. To address these issues, this paper proposes a cache‐friendly LSM‐tree based on extendible hashing. To solve the problem that similar keys in SSTables are not stored adjacently in SSTables, a key‐to‐value mapping strategy based on Locality‐Sensitive Hashing (LSH) is employed. Second, to address the uneven data distribution across hash buckets in extendible hashing, a logically uniform extendible hashing scheme is designed, along with a novel HTable structure to replace the traditional SSTables in LSM‐tree. In addition, an HTable indexing strategy based on the LSH‐Cuckoo filter is proposed to accurately locate the target HTable. Based on the Intel Optane DC Persistent Memory driver, a prototype of a cache‐friendly key‐value storage system named DLMS was implemented on non‐volatile memory (NVM), and evaluated using the YCSB benchmark. Experimental results show that, compared to the LSM‐tree‐based storage system RocksDB, DLMS achieves an average improvement of 9.8% in read throughput and 11.9% in write throughput, while reducing insertion latency by 7.8%.

Numerous IoT devices in IoT systems collect data concurrently, which brings great challenges to IoT time series databases to store and manage these data. NVM device has high read–write speed compared with HDD and Flash-based SSD, and it is a possible way to solve the storage bottleneck. However, there are some limitations that should be solved such as the overhead of the I/O software stack for NVM devices and the lack of optimization for IoT time series databases in a Linux environment. By analyzing the characteristics of IoT time series databases and NVM devices, we optimized the device driver of NVM in Linux and provide a new structure of a NVM device driver for IoT time series databases. A multi-queue management strategy and a lightweight load balance mechanism based on frequency were designed to improve the concurrency and efficiency of NVM device drivers. The prototype of an IoT-oriented NVM device driver named TS-PMEM was implemented based on an open-source NVM device driver. Six prototypes were used for evaluation with YCSB-TS, a test tool for time series databases. Results showed that TS-PMEM can improve write throughput of the time series databases by 18.6%, query throughput by 10.6%, and reduce the write latency by 8.3% and query latency by 6.4%.

As a human-cortex-inspired computing model, hierarchical temporal memory (HTM) has shown great promise in sequence learning and has been applied to various time-series applications. HTM uses the combination of columns and neurons to learn the temporal patterns within the sequence. However, the conventional HTM model compacts the input into two naive column states—active and nonactive, and uses a fixed learning strategy. This simplicity limits the representation capability of HTM and ignores the impacts of active columns on learning the temporal context. To address these issues, we propose a new HTM algorithm based on activation intensity. By introducing the column activation intensity, more useful and fine-grained information from the input is retained for sequence learning. Furthermore, a self-adaptive nonlinear learning strategy is proposed where the synaptic connections are dynamically adjusted according to the activation intensity of columns. Extensive experiments are carried out on two real-world time-series datasets. Compared to the conventional HTM and LSTM model, our method achieved higher accuracy and less time overhead.

IoT time series data is an important form of big data. How to improve the efficiency of storage system is crucial for IoT time series database to store and manage massive IoT time series data from various IoT devices. Mixing NVM and SSD is an effective method to improve the I/O performance of storage systems. However, there are great differences between HDD and NVM or SSD. As a result, NVM and SSD cannot be directly used in the current time series database effectively. We design an IoT time series database with an embedded engine in storage device drivers for the hybrid solid-state storage system consisting of NVM and SSD. The I/O software stack of storing and managing IoT time series data can be shortened to improve the efficiency. Based upon the intrinsic characteristics of IoT time series data and different features of NVM and SSD, a redundancy elimination and compression fusion strategy, a hierarchical management strategy, and a heterogeneous time series data index are designed to improve the efficiency. Finally, a prototype of embedded IoT time series database named TS-NSM is implemented, and YCSB-TS is used to measure the IOPS. The results show that TS-NSM can improve the write IOPS up to 243.6 times and 174.3 times, respectively, compared with InfluxDB and OpenTSDB, and improve the read IOPS up to 10.1 times and 14.4 times, respectively.