However, the variations of real-world time series always involve intricate temporal patterns, where multiple variations (e.g. rising, falling, fluctuation and etc.) mix and overlap with each other, making the temporal variation modeling extremely challenging.
However, the variations of real-world time series are usually too complex to be covered by pre-defined patterns, limiting the practical applicability of these classical methods.
Also, because of the locality property of the one-dimension convolution kernels, they can only model the variations among adjacent time points, thereby still failing in long-term dependencies.
But it is hard for attention mechanism to find out reliable dependencies directly from scattered time points, since the temporal dependencies can be obscured deeply in intricate temporal patterns.
工作的构思（创新点）
Overall design of TimesNet
多周期视角（Multi-periodicity）来看待现实世界中的时间序列（overlap and interaction），多周期耦合性。
modular architecture将模型分成多个模块，分开处理，但是如此处理将无法处理多个模块之间的耦合关系（1D time series has limitations in representation capability），多个模块之间需要连续的看
Temporal 2D-variation建模，使用新的数据结构，在二维空间内进行分析
从intraperiod（adjacent area, short-term variations）和interperiod（same phase in adjacent periods，long-term variations）视角来看
5 mainstream time series analysis tasks; 36 datasets, 81 settings, 20+ baseline
现阶段是在卷ETTm1/ETTm2/ETTh1/ETTh2/Electricity/Traffic/Weather/Exchange/ILI这9个数据集，但这9个数据集的temporal variation相对较为简单，本文使用了M4数据集（其变化相对而言较为复杂），“Simple linear methods degenerate a lot”
数据集
Especially for the long-term setting, we follow the benchmarks used in Autoformer (2021), including ETT (Zhou et al., 2021), Electricity (UCI), Traffic (PeMS), Weather (Wetterstation), Exchange (Lai et al., 2018) and ILI (CDC, Influenza-like Illness), covering five real-world applications. For the short-term dataset, we adopt the M4 (Spyros Makridakis, 2018), which contains the yearly, quarterly and monthly collected univariate marketing data.
指标
baseline
Model/Method
Year
Description
LSTM
1997
A recurrent neural network (RNN) model known for its ability to capture long-term dependencies in sequence data.
LSTNet
2018
Combines CNN and RNN layers to capture both spatial and temporal correlations in time series data.
LSSL
2022
An RNN-based approach that emphasizes long and short-term memory components for time series analysis.
TCN (Temporal Convolutional Network)
2019
Utilizes causal convolutions to handle sequence modeling tasks, offering an alternative to RNNs.
LightTS
2022
A lightweight MLP-based model designed for efficient and scalable time series forecasting.
DLinear
2023
An MLP-based model focusing on linear transformations for time series analysis.
Reformer
2020
A Transformer variant that reduces memory usage and computation time, suitable for long sequences.
Informer
2021
Optimizes the Transformer architecture for long-sequence time series forecasting by reducing computational cost.
Pyraformer
2021
Leverages a pyramidal structure to efficiently process long sequences in time series analysis.
Autoformer
2021
Incorporates an automatic decomposition mechanism to enhance forecasting performance in time series data.
FEDformer
2022
Employs Fourier and wavelet transformations within the Transformer framework for enhanced time series forecasting.
Non-stationary Transformer
2022
Designed to handle non-stationary time series data by adapting the Transformer architecture.
ETSformer
2022
A Transformer-based model that integrates exponential smoothing techniques for time series forecasting.
N-HiTS
2022
Focuses on short-term forecasting with a hierarchical architecture for multi-step time series prediction.
N-BEATS
2019
A purely MLP-based model, notable for its stackable and interpretable architecture in forecasting tasks.
Anomaly Transformer
2021
Specialized for anomaly detection in time series, leveraging Transformer's attention mechanism.
Rocket
2020
A classification-oriented approach using random convolutional kernels for feature extraction from time series.
Flowformer
2022
Aims at classification tasks in time series data, integrating flow-based models with the Transformer structure.
In the future, we will further explore large-scale pre-training methods in time series, which utilize TimesNet as the backbone and can generally benefit extensive downstream tasks.
