CoST: contrastive learning of disentangled seasonal-trend representations for time series forecasting

• Main Authors: WOO, Gerald, LIU, Chenghao, SAHOO, Doyen, KUMAR, Akshat, HOI, Steven
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: Self-supervised learning; Forecasting; Representation learning; Time series; Databases and Information Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/7702; https://ink.library.smu.edu.sg/context/sis_research/article/8705/viewcontent/cost.pdf
• Institution: Singapore Management University

Deep learning has been actively studied for time series forecasting, and the mainstream paradigm is based on the end-to-end training of neural network architectures, ranging from classical LSTM/RNNs to more recent TCNs and Transformers. Motivated by the recent success of representation learning in computer vision and natural language processing, we argue that a more promising paradigm for time series forecasting, is to first learn disentangled feature representations, followed by a simple regression fine-tuning step – we justify such a paradigm from a causal perspective. Following this principle, we propose a new time series representation learning framework for long sequence time series forecasting named CoST, which applies contrastive learning methods to learn disentangled seasonal-trend representations. CoST comprises both time domain and frequency domain contrastive losses to learn discriminative trend and seasonal representations, respectively. Extensive experiments on real-world datasets show that CoST consistently outperforms the state-of-the-art methods by a considerable margin, achieving a 21.3% improvement in MSE on multivariate benchmarks. It is also robust to various choices of backbone encoders, as well as downstream regressors.