SOM-CPC: Unsupervised Contrastive Learning with Self-Organizing Maps for Structured Representations of High-Rate Time Series

Huijben, Iris AM, et al. (ICLR poster 2023)

self-organizing map (SOM)

• an unsupervised machine learning technique used to produce a low-dimensional (typically two-dimensional) representation of a higher dimensional data set while preserving the topological structure of the data
• K개의 node\(\phi\)를 가정하고 각 데이터 포인트 \(z\)를 1개의 카운터파트 노드(winning node) \(q_{\phi}(z)\)로 할당함

\[q_{\phi}(z) = \phi[argmin_i(||\phi, z||_2^2)] \\\]

At training, each ϕi is updated as follows

\[\phi(i)^{(n+1)} = \phi(i)^{(n)} + \eta^{(n)}\mathcal{S}_i(q_{\phi}(z))(z - \phi(i)^{(n)} ) \\ where \ \mathcal{S}_i(q_{\phi}(z)) = \text{exp}(-\frac{d_i^{(n)}}{2(\sigma^{(n)})^2}) \\ d_i^{(n)} = ||\mathcal{P}[q_{\phi}(z)],\mathcal{P}[\phi_i^{(n)}]||_2^{2} \\ \sigma^{(n)} =\sigma^{(0)}\text{exp}(-\frac{n}{\lambda})\]

• winning node와 그와 가가운 노드들은 데이터 z와 가까운 쪽으로 업데이트됨
• \(\eta^{(n)}\) : decreasing learning rate
• \mathcal{S}i(q{\phi}(z)) : distance from the winning node

SOM-CPC

• a representation learning model that learns to map windows of time series data to a structured 2D grid for the purpose of pattern discovery
  

TBU