Momentum Contrast (MoCo) for Unsupervised Visual Representation Learning

1 Introduction

1.1 Instance discrimination (樣本判別)

制定了一種劃分正樣本和負樣本的規則

1.2 InfoNCE Loss

1.3 Momentum

動量在數學上可以理解為是一種指數移動平均(Exponential Moving Average)
\(m\)為動量係數，目的是為了 \(Y_t\) 不僅僅依賴於當前時刻的輸入 \(X_t\)

$Y_t = mY_{t-1} + (1-m)X_t$

• \(Y_t\): 當前時刻的輸出
• \(Y_{t-1}\): 上一時刻的輸出
• \(X_t\): 當前時刻的輸入

\(m\)越大，對當前時刻的輸入 \(X_t\) 的依賴越小

1.4 Momentum Contrast (MoCo)

2 Related Work

the relationships between Supervised Learning and Unsupervised Learning/Self-supervised Learning

2.1 Loss Function(目標函式)

2.2 Pretext tasks(代理任務)