DualGNN: Dual Graph Neural Network for Multimedia Recommendation

馒头and花卷發表於2024-03-23
GNN

目錄
  • 概
  • 符號說明
  • DualGCN
  • 程式碼

Wang Q., Wei Y., Yin J., Wu J., Song X. and Nie L. DualGNN: Dual graph neural network for multimedia recommendation. IEEE Transactions on Multimedia, 2023.

概

多模態 + user co-occureence graph -> recommendation.

文章中提到的 modality-missing 問題挺好的, 但是我並沒有看到有什麼特別的設計是解決這個問題的.

符號說明

  • \(\mathcal{U}\), users, \(|\mathcal{U}| = N\);
  • \(\mathcal{I}\), micor-videos, \(|\mathcal{I}| = M\);
  • \(\mathcal{G} = (\mathcal{V}, \mathcal{E})\), interaction graph, \(\mathcal{V} = \mathcal{U} \cup \mathcal{I}\);
  • \(m \in \mathcal{M} = \{v, a, t\}\), modality, \(v\): visual, \(a\): acoustic, \(t\): textual;
  • \(\bm{u}_m^{(0)}\), 隨機初始化的 user embedding for modality \(m\);
  • \(\bm{i}_m^{(0)}\), modality features.

DualGCN

  • 首先, 透過 user-item interaction graph, 對 user/item 的資訊進行初步的轉換, 這部分對於每個模態是獨立的:

    \[\bm{u}_m^{(l+1)} = \sum_{i \in \mathcal{N}_u} \frac{ 1 }{ \sqrt{|\mathcal{N}_u|} \sqrt{|\mathcal{N}_i|} }\bm{i}_m^{(l)}, \\ \bm{i}_m^{(l+1)} = \sum_{i \in \mathcal{N}_u} \frac{ 1 }{ \sqrt{|\mathcal{N}_u|} \sqrt{|\mathcal{N}_i|} }\bm{i}_m^{(l)}, \]

    然後, 對各層求和:

    \[\bm{u}_m = \sum_{l=0}^L \bm{u}_m^{(l)}, \quad \bm{i}_m = \sum_{l=0}^L \bm{i}_m^{(l)}. \]

  • 然後是, 多模態資訊的一個融合, 用於得到使用者的一個表徵, 作者給了三種方案:

    1. Attentively concatenation construction:

      \[\bm{u}_{mul} = \bm{W}_m \bm{h}_u + \bm{b}_m, \\ \bm{h}_u = \alpha_{u, v} \bm{u}_v \| \alpha_{u, a} \bm{u}_a \| \alpha_{u, t} \bm{u}_t. \]

    2. Attentively sum construction:

      \[\bm{u}_{mul} = \alpha_{u,v} \bm{u}_v + \alpha_{u, a} \bm{u}_a + \alpha_{u, t} \bm{u}_t. \]

    3. Attentively maximum construction:

      \[\bm{u}_{mul} = \max( \alpha_{u, v} \bm{u}_v, \alpha_{u,a} \bm{u}_a, \alpha_{u, t} \bm{u}_t ). \]

  • 接著 \(\bm{u}_{mul}\) 還要透過 user-user graph 來進行進一步的轉換. 構造 co-occurrence graph, 對於每個 user, 作者取它的 top-K 個 frequent users 作為鄰居, 便的權重賦為 \(1\), 其它為 0. 接下來透過下列二者之一的方式進行轉換:

    1. Mean aggregation:

      \[\bm{u}_{mul}^{(l'+1)} = \bm{u}_{mul}^{(l')} + \sum_{u' \in \mathcal{N}_{u, c}} \frac{1}{|\mathcal{N}_{u, c}|} {\bm{u}_{mul}'}^{(l')}, \]

      其中 \(\mathcal{N}_{u,c}\) 表示 user \(u\) 的 top-\(K\) 鄰居.
    2. Softmax weighted aggregation:

      \[\bm{u}_{mul}^{(l'+1)} = \bm{u}_{mul}^{(l')} + \sum_{u' \in \mathcal{N}_{u, c}} \frac{ \exp(C_{u, u'}) }{ \sum_{u' \in \mathcal{N}_{u, c}} \exp(C_{u, u'}) } {\bm{u}_{mul}'}^{(l')}, \]

      其中 \(C_{u, u'}\) 表示 users \(u, u'\) 的 co-occurrence times.

  • 最後得到最終 user/item 的表示:

    \[\bm{u}^* = \bm{u}_{mul}^{(L')}, \quad \bm{i}^* = \bm{i}_v + \bm{i}_a + \bm{i}_t. \]

    然後 score 的預測透過內積得到

    \[y_{u,i} = {\bm{u}^*}^T \bm{i}^*. \]

  • 訓練透過 BPR 損失.

程式碼

[official]

相關文章