time series representation learning with consistency
Temporal consistency
Tonekaboni, Sana, Danny Eytan, and Anna Goldenberg. “Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding.” International Conference on Learning Representations. 2020
Subseries consistency
Franceschi, Jean-Yves, Aymeric Dieuleveut, and Martin Jaggi. “Unsupervised scalable representation learning for multivariate time series.” Advances in neural information processing systems 32 (2019)
Transformation consistency
Eldele, Emadeldeen, et al. “Time-series representation learning via temporal and contextual contrasting.” arXiv preprint arXiv:2106.14112 (IJCAI 2021).
- weak augmentation : jitter-and scale
- strong augmentation : permutation-and-jitter
- a tough cross-view prediction task by using the context of the strong augmentation \(c^s_t\) to predict the future timesteps of the weak augmentation \(z^w _{t+k}\) and vice versa (temporal contrasting module)
- \(c_t^{i+}\) as the positive sample of \(c_t^{i}\) that comes from the other augmented view of the same inut (contextual contrasting module)
Contextual consistency
Yue, Zhihan, et al. “Ts2vec: Towards universal representation of time series.” Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. No. 8. 2022.
- random cropping : randomly samples two overlapping time segments [a1, b1], [a2, b2] a1<=a2<=b1<=b2 (the overlapped segment [a2, b1] should be consistent for two context views)
- timestamp masking : randomly mask the latent vector \(z_i\)
- hierarchical constrative loss encapsulate representations in different levels of granularity
Frequency consistency
Zhang, Xiang, et al. “Self-supervised contrastive pre-training for time series via time-frequency consistency.” Advances in Neural Information Processing Systems 35 (2022): 3988-4003.
TBU
Comments