感悟
看完圖卷積一代、二代，深感圖卷積的強大，剛開始接觸圖卷積的時候完全不懂為什麼要使用拉普拉斯矩陣（ $L=D-W$），主要是其背後的物理意義。通過借鑑前輩們的論文、部落格、評論逐漸對圖卷積有了一定的瞭解，作為一個剛上研的博士生，深感得對圖神經網路進行一個系統的學習。
本篇論文得感謝論文 David I Shuman 作者及博主：純牛奶愛酸牛奶
Paper Information
Authors：D. Shuman, S. Narang, P. Frossard, Antonio Ortega, P. Vandergheynst
Sources：2012, IEEE Signal Processing Magazine
Paper：Download chrome-extension://ibllepbpahcoppkjjllbabhnigcbffpi/https://arxiv.org/pdf/1211.0053.pdf
Code：Download
2528 Citations, 75 References
Abstract
The emerging field of signal processing on graphs merges algebraic and spectral graph theoretic concepts with computational harmonic analysis to process such signals on graphs.
圖上訊號處理的新興領域，將代數和譜圖理論的概念與計算諧波分析相結合，以處理圖上的訊號。
本文將闡述該領域上的一些挑戰，以及定義在 graph spectral domains 上的一些方法，這些方法和經典的 frequency domain 相似。本文同時還點明瞭在處理 graph signal 融合使用 irregular data of graph 的重要性（每每看論文都出現結合結構特性，但並沒有有人能完全說清楚）。然後介紹一些 common operator ，比如說 filtering、translation、modulation、dilation、downsampling to the graph setting 。對已經提出的 localized, multiscale transforms 用於提取圖資料的高維特徵做了總結。
1 Introduction
- 圖中每條邊的權重，經常表示為兩個頂點之間的相似度。對於節點之間的連線性以及權重大小，一般為資料本身就有的，如社交關係，另外一種為自己根據需求構建的，比如使用KNN 構建權重矩陣 $W$。這在本部落格《譜聚類原理總結》已經做了詳細介紹。
- 這裡相似性通常和距離成反比，採用聚類的思想：距離較遠的兩個點之間的邊權重值較低，而距離較近的兩個點之間的邊權重值較高。
- graph signal：The data on these graphs can be visualized as a finite collection of samples, with one sample at each vertex in the graph. Collectively, we refer to these samples as a graph signal.示例： Fig. 1.
1.1 The Main Challenges of Signal Processing on Graphs
- 目前的研究現狀：使用 wavelet, time-frequency, curvelet and other localized transforms 可以很好的將 Euclidean space 上的高維不同類資料區分開來，但對 non-Euclidean space 上的資料卻無有效手段。
- 傳統的 signal processing techniques 忽略了 irregular data domain 。
Challenges on Graph:
-
- The unavoidable fact is that weighted graphs are irregular structures that lack a shift-invariant notion of translation.
- 平移不變性（translation invariance）：在歐式空間中，平移是一種幾何變換，表示把一幅影像或空間中的每一個點在相同方向移動相同距離。用基礎的分類結構比如 ResNet、Inception 給一隻貓分類時，無論貓怎麼扭曲、平移，最終識別出來的都是貓，輸入怎麼變形輸出都不變這就是平移不變性，網路的層次越深這個特性會越明顯。
- 平移可變性（translation variance）：針對目標檢測的，比如一隻貓從圖片左側移到了右側，檢測出的貓的座標會發生變化就稱為平移可變性。當卷積網路變深後最後一層卷積輸出的 feature map 變小，物體在輸入上的小偏移，經過 $N$ 層 pooling 後在最後的小 feature map 上會感知不到，這就是為什麼 R-FCN 原文會說網路變深平移可變性變差。
- CNN 無法處理 Non Euclidean Structure 的資料，通俗理解就是在拓撲圖中每個頂點的相鄰頂點數目都可能不同，那麼當然無法用一個同樣尺寸的卷積核來進行卷積運算。
- The unavoidable fact is that weighted graphs are irregular structures that lack a shift-invariant notion of translation.
- Tramsform thr spatial domain data into spectral domain date.$F\left(\lambda_{i}\right)=\hat{f}\left(\lambda_{i}\right)=\sum \limits _{i=1}^{N} f(i) u(i)$
- We need a method to generate a coarser version of the graph that somehow captures the structural properties embedded in the original graph.
- We intuitively downsample a discrete-time signal by deleting every other data point,the process is as Fig.1.
處理資料域的不規則性、圖結構，在之前提到的應用中，可以表示很多頂點的特徵。為了能很好地對資料的尺度進行縮放，對於圖訊號的處理技術應該使用區域性操作，通過對每個頂點，計算頂點的鄰居，或是和它很近的頂點的資訊。
The overarching challenges of processing signals on graphs:
- How to construct a weighted graph that captures the geometric structure of the underlying data domain;
- Incorporating the graph structure into localized transform methods;
- Leveraging the invaluable intuitions in Euclidean domains to deal with processing signals on graphs;
- Consider to improve the efficient of localized transforms.
2 The graph spectral domain
2.1 Weighted Graphs and Graph Signals
Defitions of graph：
-
- Graph type：undirected, connected, weighted graph $\mathcal{G}=\{\mathcal{V}, \mathcal{E}, \mathbf{W}\} $,
- Vertice set：a finite set of vertices $\mathcal{V}$ with $|\mathcal{V}|=N$ ,
- Edge set：a set of edges $\mathcal{E} $,
- Weighted adjacency matrix：Symbol as $\mathbf{W}$ . If there is an edge $e=(i, j)$ connecting vertices $i$ and $j$ , the entry $W_{i, j}$ represents the weight of the edge; otherwise, $W_{i, j}=0$ .
- SSC（連通分量）：If the graph $\mathcal{G}$ is not connected and has $M$ connected components $ (M>1)$ , we can separate signals on $\mathcal{G}$ into $M$ pieces corresponding to the $M$ connected components, and independently process the separated signals on each of the subgraphs.
According the need of application,a common way to construct a weight matrix is full connect which for construct a symmetric matrx.More detail you can refer to my blog 《譜聚類原理總結》.
$W_{i, j}=\left\{\begin{array}{ll}\exp \left(-\frac{[\operatorname{dist}(i, j)]^{2}}{2 \theta^{2}}\right) & \text { if } \operatorname{dist}(i, j) \leq \kappa \\0 & \text { otherwise }\end{array}\right.$
Where, $ dist(i, j)$ can be Euclidean distance between two vector $i$ and $j$. two leanable parmeter $\theta$ and $\kappa$.
Another way two generate a weight matrix is KNN .
$f: \mathcal{V} \rightarrow \mathbb{R}$ defined the value of verticles of the graph,the $i^{t h}$ component $f_i$ mean the $i^{t h}$ vertices value .
2.2 The Non-Normalized Graph Laplacian
The non-normalized graph Laplacian, also called the combinatorial graph Laplacian, is defined as
$L:=\mathbf{D}-\mathbf{W}$
Where
-
- $D$ is degree matrix which a diagonal matrix ;
- $\mathbf{W}$ is weighted adjacency matrix .
The graph Laplacian matrix is define as :
$(L f)(i)=\sum \limits _{j \in \mathcal{N}_{i}} W_{i, j}[f(i)-f(j)]$
If you want to know the background physical meaning of this Eq, you can refer to my another blog 《圖神經網路基礎二：譜圖理論》.
Where
-
- The neighborhood $\mathcal{N}_{i}$ is the set of vertices connected to vertex $i$ by an edge. More generally, we denote by $\mathcal{N}(i, k)$ the set of vertices connected to vertex $i$ by a path of $k$ or fewer edges.（鄰居定義為小於等於k-step可到達的點）
Due to graph Laplacian matrix $L$ is a real symmetric matrix ,so it can do Laplace Spectral Decomposition as the following:
$L \mu_{k}=\lambda_{k} \mu_{k}$
Because $L$ is real symmetric matrix ,so it can be transformed as：
$L=U \Lambda U^{-1}=U \Lambda U^{T}$
Some note in here：
-
- Orthonormal eigenvectors $\left\{\mathbf{u}_{l}\right\}_{l=0,1, \ldots, N-1}$ ;
- Non-negative eigenvalues $\left\{\lambda_{l}\right\}_{l=0,1, \ldots, N-1}$ ;
- $\Lambda $ is consist of the non-negative eigenvalues that are ordered as $0=\lambda_{0}<\lambda_{1} \leq \lambda_{2} \cdots \leq \lambda_{N-1}:=\lambda_{\max }$
-
We denote the entire spectrum by $\sigma(L):=\left\{\lambda_{0}, \lambda_{1}, \ldots, \lambda_{N-1}\right\}$ .
- Orthonormal eigenvectors $\left\{\mathbf{u}_{l}\right\}_{l=0,1, \ldots, N-1}$ ;
2.3 A Graph Fourier Transform and Notion of Frequency
Classical fourier tramsform is ：
$\hat{f}(\xi):=\left\langle f, e^{2 \pi i \xi t}\right\rangle=\int_{\mathbb{R}} f(t) e^{-2 \pi i \xi t} d t$
即：將函式 $f$ 在特徵函式（eigenfunctions）上表示出來，類似座標點在平面上的形式。
Eigenfunction with Laplace operator ：
$-\Delta\left(e^{2 \pi i \xi t}\right)=-\frac{\partial^{2}}{\partial t^{2}} e^{2 \pi i \xi t}=(2 \pi \xi)^{2} e^{2 \pi i \xi t}\quad \quad \quad \quad(2)$
即：Eigenfunction 的負散度。參考《圖神經網路基礎二：譜圖理論》
Analogously,we can define the Graph Fourier tramsform $\hat{f}$ of any function $\mathbf{f} \in \mathbb{R}^{N}$ on the vertices of the $G$ as the expansion of f in terms of the eigenvectors of the graph Laplacian:
$\hat{f}\left(\lambda_{l}\right):=\left\langle\mathbf{f}, \mathbf{u}_{l}\right\rangle=\sum\limits _{i=1}^{N} f(i) u_{l}^{*}(i)\quad \quad \quad \quad(3)$
The inverse of Graph Fourier tramsform is :
$f(i)=\sum \limits _{l=0}^{N-1} \hat{f}\left(\lambda_{l}\right) u_{l}(i)\quad \quad \quad \quad(4)$
More analysis about Classical Fourier tramsform：
-
- The eigenvalues $\left\{(2 \pi \xi)^{2}\right\}_{\xi \in \mathbb{R}}$ in Eq.2 carry a specific notion of frequency $\xi$.
- If $\xi$ close to zero (low frequencies), the associated complex exponential eigenfunctions are smooth
In graph Laplacian, the eigenvalues and eigenvectors provide a similar notion of frequency.
-
- The Laplacian eigenvector $\mathbf{u}_{0}$ associated with the eigenvalue $0$ is constant and equal to $\frac{1}{\sqrt{N}}$ at each vertex. 特徵值為 $0$ 時，對應的特徵向量為歸一化的單位向量，即每個分量為 $\frac{1}{\sqrt{N}}$ 。
- 將 特徵值 $\lambda_{l}$ 看作頻率，$\lambda_{l}$ 越小代表變化慢和周圍節點相差不大。（其實就是平滑性度量）
- The graph Laplacian eigenvectors associated with low frequencies $\lambda_{l}$ vary slowly across the graph.即不同graph signal 的特徵向量受特徵值的影響。圖中相似的節點，特徵向量差別不大；不相似的節點，特徵向量差別很大，而判斷相似性是通過特徵值來判斷，特徵值越小，節點和鄰居節點的相似性越大。
This is demonstrated in both Figure 2.
分析
-
- 對於第一個圖，此時特徵值 $\lambda_{0} $ 為 $0$，特徵向量為 $u_0$ ,可以看到每個 graph signal 都是 $\frac{1}{\sqrt{N}} $ （用高度衡量）。
- 對於第二個圖，此時特徵向量為 $u_1$，可以看到每個 graph signal 區域附近 graph signal 的高度差不多。
- 對於第三個圖，此時特徵向量為 $u_{50}$，可以看到每個 graph signal 區域附近 graph signal 的高度差很多。
Figure 3, which shows the number of zero crossings of each graph Laplacian eigenvector. The set of zero crossings of a signal f on a graph G is defined as:
$\mathcal{Z}_{\mathcal{G}}(\mathbf{f}):=\{e=(i, j) \in \mathcal{E}: f(i) f(j)<0\}$
It means that if $\lambda $ is more bigger ,the bigger the change.
2.4 Graph Signal Representations in Two Domains
The graph Fourier transform (3) and its inverse (4) give us a way to equivalently represent a signal in two different domains: the vertex domain and the graph spectral domain.
- A signal g in the vertex domain.
- Define a signal $\hat{g}$ directly in the graph spectral domain.
Figure 4 will show you the graph Fourier coefficients with different $\lambda_{l}$.
Figure 4 告訴我們一個 graph signal 在不同的域上的等價表現形式。圖4中展示的一個平緩訊號的圖傅立葉係數衰減的很快。這樣的訊號是可壓縮的(compressible)，因為可以通過調整一些圖傅立葉係數來趨近他們。
2.5 Discrete Calculus and Signal Smoothness with Respect to the Intrinsic Structure of the Graph
When we analyze signals, it is important to emphasize that properties such as smoothness are with respect to the intrinsic structure of the data domain.
下面將介紹 smoothness function 。至於推導完全可以看《圖神經網路基礎二：譜圖理論》
The edge derivative of a signal $\mathbf{f}$ with respect to edge $e=(i, j)$ at vertex $i$ is defined as
$\left.\frac{\partial \mathbf{f}}{\partial e}\right|_{i}:=\sqrt{W_{i, j}}[f(j)-f(i)]$
The graph gradient of $f$ at vertex $i$ is the vector
$\nabla_{i} \mathbf{f}:=\left[\left\{\left.\frac{\partial \mathbf{f}}{\partial e}\right|_{i}\right\}_{e \in \mathcal{E} \text { s.t. } e=(i, j) \text { for some }_{j \in \mathcal{V}}}\right]$
Then the local variation at vertex $i$：
$\begin{aligned}\left\|\nabla_{i} \mathbf{f}\right\|_{2} &:=\left[\sum \limits _{e \in \mathcal{E} \text { s.t. } e=(i, j) \text { for some } j \in \mathcal{V}}\left(\left.\frac{\partial \mathbf{f}}{\partial e}\right|_{i}\right)^{2}\right]^{\frac{1}{2}} \\&=\left[\sum \limits_{j \in \mathcal{N}_{i}} W_{i, j}[f(j)-f(i)]^{2}\right]^{\frac{1}{2}}\end{aligned}$
This provides a measure of local smoothness of $f$ around vertex $i$, as it is small when the function $f$ has similar values at $i$ and all neighboring vertices of $i$.
For global smoothness of all nodes in the graph,we can define a discrete p-Dirichlet form of $f$ ：
$S_{p}(\mathbf{f}):=\frac{1}{p} \sum \limits _{i \in V}\left\|\nabla_{i} \mathbf{f}\right\|_{2}^{p}=\frac{1}{p} \sum\limits_{i \in V}\left[\sum_{j \in \mathcal{N}_{i}} W_{i, j}[f(j)-f(i)]^{2}\right]^{\frac{p}{2}}\quad \quad \quad \quad(5)$
When $p=1$, $S_{1}(\mathbf{f})$ is the total variation of the signal with respect to the graph. When $p=2$ , we have
$\begin{aligned}S_{2}(\mathbf{f}) &=\frac{1}{2} \sum\limits _{i \in V} \sum\limits_{j \in \mathcal{N}_{i}} W_{i, j}[f(j)-f(i)]^{2} \\&=\sum\limits_{(i, j) \in \mathcal{E}} W_{i, j}[f(j)-f(i)]^{2}\\&=\mathbf{f}^{\mathrm{T}} \text { Lf }\end{aligned}\quad \quad \quad \quad(6)$
拉普拉斯運算元
$\begin{aligned}\Delta f(x) &=\frac{\partial^{2} f}{\partial x^{2}} \\&=f^{\prime \prime}(x) \\& \approx f^{\prime}(x)-f^{\prime}(x-1) \\& \approx[f(x+1)-f(x)]-[f(x)-f(x-1)] \\&=f(x+1)+f(x-1)-2 f(x)\end{aligned}$
圖的拉普拉斯運算元
$\Delta f_{i}=\sum \limits _{j \in N_{i}}\left(f_{i}-f_{j}\right)$
而如果邊 $E_{i j}$ 具有權重 $W_{i j}$ 時，則有:
$\Delta f_{i}=\sum\limits_{j \in N} W_{i j}\left(f_{i}-f_{j}\right)$
對於任意向量 $f$，有：
$\begin{aligned}f^{T} L f &=f^{T} D f-f^{T} W f \\&=\sum\limits_{i=1}^{N} d_{i} f_{i}^{2}-\sum\limits_{i=1}^{N} \sum\limits_{j=1}^{N} w_{i j} f_{i} f_{j} \\&=\frac{1}{2}\left(\sum\limits_{i=1}^{N} d_{i} f_{i}^{2}-2 \sum\limits_{i=1}^{N} \sum\limits_{j=1}^{N} w_{i j} f_{i} f_{j}+\sum\limits_{j=1}^{N} d_{j} f_{j}^{2}\right) \\&=\frac{1}{2}\left(\sum\limits_{i=1}^{N} \sum\limits_{j=1}^{N} w_{i j} f_{i}^{2}-2 \sum\limits_{i=1}^{N} \sum\limits_{j=1}^{N} w_{i j} f_{i} f_{j}+\sum\limits_{i=1}^{N} \sum\limits_{j=1}^{N} w_{i j} f_{j}^{2}\right) \\&=\frac{1}{2} \sum\limits_{i=1}^{N} \sum\limits_{j=1}^{N} w_{i j}\left(f_{i}-f_{j}\right)^{2}\end{aligned}$
$S_{2}(\mathbf{f})$ is known as the graph Laplacian quadratic form and the semi-norm $\|\mathbf{f}\|_{\text {L }} $ （準範數）is defined as
$\|\mathbf{f}\|_{L}:=\left\|L^{\frac{1}{2}} \mathbf{f}\right\|_{2}=\sqrt{\mathbf{f}^{\mathrm{T}} L \mathbf{f}}=\sqrt{S_{2}(\mathbf{f})}$
$S_{2}(f)$ is small when the signal $f$ has similar values at neighboring vertices connected by an edge with a large weight.
現在回到 graph Laplacian eigenvalues 和 eigenvectors ：（Courant-Fischer Theorem）可以參考《極大極小定理》
$\lambda_{0}=\underset{\mathbf{f} \in \mathbb{R}^{N} \atop\|\mathbf{R}\|_{2}=1}{min} \;\left\{\mathbf{f}^{\mathrm{T}} L \mathbf{f}\right\}\quad \quad \quad \quad(7)$
and
${\large \lambda_{l}=\underset{\mathbf{f} \in \mathbb{R}^{N} \atop \underset{\mathbf{f} \perp \operatorname{span}\left\{\mathbf{u}_{0}, \ldots, \mathbf{u}_{l-1}\right\}}{\|\mathbf{f}\|_{2}=1} }{min} \left\{\mathbf{f}^{\mathrm{T}} L \mathbf{f}\right\}, l=1,2, \ldots, N-1}\quad \quad \quad \quad(8)$
拉普拉斯矩陣可以定義圖傅立葉變換（通過特徵向量）以及平滑性的不同表示。並且圖的連通性也編碼進了拉普拉斯矩陣。 Example 1 展示了 smoothness 和一個圖訊號的譜內容是如何依賴於圖的。
Discussion： 上面的三個圖，點同邊不同，頂點域的圖 the vertex domains 下面的三個圖是對應的譜域 graph spectral domains 可以看出，smoothness and graph spectral content of the signa 取決於圖的結構的，$G_1$ 的 signal $f$ 最平滑(smoothest)，$G_3$ 最不平滑。
2.6 Other Graph Matrices
Method1:
A second popular option is to normalize each weight $W_{i, j}$ by a factor of $\frac{1}{\sqrt{d_{i} d_{j}}}$ .
Doing so leads to the normalized graph Laplacian, which is defined as
$\tilde{L}:=\mathbf{D}^{-\frac{1}{2}} \text { LD }^{-\frac{1}{2}}$
equivalently
$(\tilde{L}f)(i) = \frac{1}{\sqrt{d_i}} \sum \limits_{j \in \mathcal{N}_i} W_{i,j} \LARGE[\normalsize \frac{f(i)}{\sqrt{d_i}} - \frac{f(j)}{\sqrt{d_j}} \LARGE]$
The eigenvalues satisfy
$0 = \tilde{\lambda}_0 < \tilde{\lambda}_1 \leq ... \leq \tilde{\lambda}_{\text{max}} \leq 2$
with $\tilde{\lambda}_{\text{max}} = 2$ if and only if $G$ is bipartite（二部圖）.
The normalized and non-normalized graph Laplacians are Generalized graph Laplacians.
A generalized graph Laplacian of a graph G is any symmetric matrix:
- 如果這個矩陣中有邊連線頂點 $i$ 和頂點 $j$ ，那麼這個矩陣的 $entry([i,j])$ 是負的，
- 如果 $i \ne j$ 並且 $i$ 和頂點 $j$ 不相連，那麼為 $0$；
- 如果 $i = j$ ，那麼有可能是任何值。
Method2: random walk matrix
$P:=D^{-1} W$
$P_{ij}$ describes the probability of going from vertex $i$ to vertex $j$ in one step of a Markov random walk on the graph $G$.
For connected, aperiodic graphs, each row of $P^{t}$ converges to the stationary distribution of the random walk as $t$ goes to infinity.
Another type random walk matrix ：asymmetric graph Laplacian
$L_{a}:=\mathbf{I}_{N}-\mathbf{P}$
Where
$\mathbf{I}_{N}$ is the $N \times N$ identity matrix.
Note that $L_{a}$ has the same set of eigenvalues as $\tilde{L}$ , and if $\tilde{\mathbf{u}}_{l}$ is an eigenvector of $\tilde{L}$ associated with $\tilde{\lambda}_{l}$ , then $\mathbf{D}_{}^{-\frac{1}{2}} \tilde{\mathbf{u}}_{l}$ is an eigenvector of $L_{a}$ associated with the eigenvalue $\tilde{\lambda}_{l}$ .
The normalized graph Laplacian has the nice properties that its spectrum is always contained in the interval [0, 2] and, for bipartite graphs.
3 Generalized Operators For Signals on Graphs
In this section ,we will review different fundamental operations such as filtering, translation, modulation, dilation, and downsampling to the graph setting.
3.1 Filtering
3.1.1 Frequency Filtering
In classical signal processing,
$\hat{f}_{\text {out }}(\xi)=\hat{f}_{\text {in }}(\xi) \hat{h}(\xi)$
Where
$\hat{h}(\cdot)$ is the transfer function of the filter.
This frequency filtering will combine the input signal as a linear combination of complex exponentials.其實就是將輸入的 signal 使用基函式線性組合來表示。
The inverse Fourier transform is as following：
$f_{\text {out }}(t)={\mathcal{F}}^{-1}\left\{\hat{f}_{\text {in }}(\xi) \hat{h}(\xi)\right\}$
equivalently
$\begin{array}{l}f_{\text {out }}(t)&=\int_{\mathbb{R}} \hat{f}_{\text {in }}(\xi) \hat{h}(\xi) e^{2 \pi i \xi t} d \xi \quad \quad\quad \quad\quad\quad\quad\quad(10) \\&=\int_{\mathbb{R}} f_{i n}(\tau) h(t-\tau) d \tau=:\left(f_{i n} * h\right)(t)\;\;\quad\quad(11) \end{array}$
Refer to my blog《圖神經網路基礎一：傅立葉級數與傅立葉變換》
$F(W)$ 是 $f(t) $ 的傅立葉變換
$F(W)=\int_{-\infty}^{+\infty} f(t) e^{-i W t} \mathrm{~d} t$
$f(t) $ 是傅立葉變換的逆變換。
$f(t)=\frac{1}{2 \pi} \int_{-\infty}^{+\infty} F(W) e^{i W t} \mathrm{~d} W$
Frequency filtering or graph spectral filtering definition:
$\hat{f}_{\text {out }}\left(\lambda_{l}\right)=\hat{f}_{\text {in }}\left(\lambda_{l}\right) \hat{h}\left(\lambda_{l}\right)\quad\quad\quad\quad(12)$
Inverse graph Fourier transform
$f_{\text {out }}(i)=\sum\limits _{l=0}^{N-1} \hat{f}_{i n}\left(\lambda_{l}\right) \hat{h}\left(\lambda_{l}\right) u_{l}(i)\quad\quad\quad\quad(13)$
Graph Laplacian
$ L=U\left[\begin{array}{cccc}\lambda_{0} & 0 & \cdots & 0 \\0 & \lambda_{1} & \cdots & 0 \\\vdots & \vdots & \ddots & \vdots \\0 & 0 & \cdots & \lambda_{N-1}\end{array}\right] U^{-1}=U\left[\begin{array}{cccc}\lambda_{0} & 0 & \cdots & 0 \\0 & \lambda_{1} & \cdots & 0 \\\vdots & \vdots & \ddots & \vdots \\0 & 0 & \cdots & \lambda_{N-1}\end{array}\right] U^{T}$
Graph Laplacian another expression:
$\hat{h}(\mathbf{L}):=\mathbf{U}\left[\begin{array}{ccc}\hat{h}\left(\lambda_{0}\right) & & \mathbf{0} \\& \ddots & \\\mathbf{0} & & \hat{h}\left(\lambda_{N-1}\right)\end{array}\right] \mathbf{U}^{\mathrm{T}} .$
$\mathbf{f}_{\text {out }}=\hat{h}(L) \mathbf{f}_{i n}$
The basic graph spectral filtering in Eq.12 can be used to Gaussian smoothing, bilateral filtering, total variation filtering, anisotropic diffusion, and non-local means filtering.
Example.2 中比較了 the classical Gaussian filter 與 graph spectral filtering。前者在平滑處理時，會將圖片的邊緣也 smooth ；而後者沒有，這是因為圖拉普拉斯矩陣包含了幾何結構資訊。
Here,we talk about the common minimize mode (discrete regularization framework):
$\min _{\mathbf{f}}\left\{\|\mathbf{f}-\mathbf{y}\|_{2}^{2}+\gamma S_{p}(\mathbf{f})\right\}$
Where
-
- $S_{p}(\mathbf{f})$ is the p-Dirichlet form of (5b).
- $f$ is the classfier
- $y$ is the ground-trueth
3.1.2 Filtering in the Vertex Domain
In the vertex domain,the output $f_{out}(i)$ at vertex is :
$f_{\text {out }}(i)=b_{i, i} f_{\text {in }}(i)+\sum \limits _{j \in \mathcal{N}(i, K)} b_{i, j} f_{\text {in }}(j)\quad\quad\quad\quad(18)$
It is a linear combination of the components of the input signal at vertices within a K-hop local neighborhood of vertex $i$:
It also mean a localized linear transform.
Now,we relate the spectral domain with vertex domain .When Eq.12 frequency filter is a order $K$ polynomial kernal,
$\hat{h}\left(\lambda_{l}\right)=\sum\limits _{k=0}^{K} a_{k} \lambda_{l}^{k}$
Where
Some constants $\left\{a_{k}\right\}_{k=0,1, \ldots, K}$
Attempt it to vertex domain in Eq.13:
$\begin{aligned}f_{\text {out }}(i) &=\sum\limits_{l=0}^{N-1} \hat{f}_{\text {in }}\left(\lambda_{l}\right) \hat{h}\left(\lambda_{l}\right) u_{l}(i) \\&=\sum\limits_{j=1}^{N} f_{\text {in }}(j) \sum\limits_{k=0}^{K} a_{k} \sum\limits_{l=0}^{N-1} \lambda_{l}^{k} u_{l}^{*}(j) u_{l}(i) \\&=\sum\limits_{j=1}^{N} f_{\text {in }}(j) \sum\limits_{k=0}^{K} a_{k}\left(L^{k}\right)_{i, j}\end{aligned}$
當 vertex $i$ 和 vertex $j$ 的距離大於 $K$時 ，有
$\left(\mathrm{L}^{k}\right)_{i, j}=0$
此時將 Eq18 中的係數寫成
$b_{i, j}:=\sum \limits _{k=d_{\mathcal{G}}(i, j)}^{K} a_{k}\left(L^{k}\right)_{i, j}$
3.2 Convolution
We can not dirrectly define a convolution product into graph setting($h(t-\tau)$),so we try to replace the complex exponentials with graph Laplacian eigenvectors.
$(f * h)(i):=\sum\limits _{l=0}^{N-1} \hat{f}\left(\lambda_{l}\right) \hat{h}\left(\lambda_{l}\right) u_{l}(i)\quad\quad\quad\quad(20)$
3.3 Translation
在圖域中沒法直接定義“平移”概念，因此仍需要通過譜域定義平移。
時域平移可以視為訊號與在延時 $t$ 上的脈衝 $δ$的卷積結果，因此頂點域的平移 $n$ 可以視為訊號與在頂點 $n$ 上的脈衝 $δ$ 的卷積，而 $δ$ 的圖傅立葉變換即為頂點 $n$ 上特徵向量之和。
$\left(T_{n} g\right)(i):=\sqrt{N}\left(g * \delta_{n}\right)(i) \stackrel{20}{=} \sqrt{N} \sum \limits _{l=0}^{N-1} \hat{g}\left(\lambda_{l}\right) u_{l}^{*}(n) u_{l}(i)\quad\quad\quad(21)$
where
$\delta_{n}(i)=\left\{\begin{array}{ll}1 & \text { if } i=n \\0 & \text { otherwise }\end{array}\right.\quad\quad\quad(22)$
但是，我們一般不認為這是“圖上的平移”，而是認為這是圖譜域上的核（核即前文提到的訊號 kernel ）操作，要將核 $g(·)$ 平移到頂點 $n$ 上，則需要在核 $g$ 的每一項上乘上對應的特徵向量 $u_l（n）$，再反變換回頂點域。其次，係數 $\sqrt{N}$ 是為了保證平移運算元保持了原訊號的均值不變。再次，訊號 $g(·)$ 的平滑程度控制了平移後訊號在頂點n附近的區域性性（localization），即隨著頂點 $i$ 與 $n$ 距離增大（最短路徑跳數），其幅值下降的程度。最後，廣義平移運算元並非等距對映，due to the possible localization of the graph Laplacian eigenvectors（誰能解釋一下）。
看不下去了...........心情好了接著看