Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects
Preprint
Abstract
Self-supervised learning (SSL) has recently achieved impressive performance on various time series tasks. The most prominent advantage of SSL is that it reduces the dependence on labeled data. Based on the pre-training and fine-tuning strategy, even a small amount of labeled data can achieve high performance. Compared with many published self-supervised surveys on computer vision and natural language processing, a comprehensive survey for time series SSL is still missing. To fill this gap, we review current state-of-the-art SSL methods for time series data in this article. To this end, we first comprehensively review existing surveys related to SSL and time series, and then provide a new taxonomy of existing time series SSL methods by summarizing them from three perspectives: generative-based, contrastive-based, and adversarial-based. These methods are further divided into ten subcategories with detailed reviews and discussions about their key intuitions, main frameworks, advantages and disadvantages. To facilitate the experiments and validation of time series SSL methods, we also summarize datasets commonly used in time series forecasting, classification, anomaly detection, and clustering tasks. Finally, we present the future directions of SSL for time series analysis.Keywords
All Related Versions
This publication has 48 references indexed in Scilit:
- Time series classification based on convolutional network with a Gated Linear Units kernelEngineering Applications of Artificial Intelligence, 2023
- Data Augmentation techniques in time series domain: a survey and taxonomyNeural Computing and Applications, 2023
- Semi-supervised Time Series Classification Model with Self-supervised LearningEngineering Applications of Artificial Intelligence, 2022
- Small perturbations are enough: Adversarial attacks on time series predictionInformation Sciences, 2022
- Contrastive learning based self-supervised time-series analysisApplied Soft Computing, 2022
- Pre-trained models for natural language processing: A surveyScience China Technological Sciences, 2020
- Financial time series forecasting with deep learning : A systematic literature review: 2005–2019Applied Soft Computing, 2020
- A survey on face data augmentation for the training of deep neural networksNeural Computing and Applications, 2020
- Deep learning for time series classification: a reviewData Mining and Knowledge Discovery, 2019
- A review on distance based time series classificationData Mining and Knowledge Discovery, 2018