PostgreSQL 14在MVCC上面有所增強，以保證在大批次客戶端連線場景下效能不會出現顯著下降（詳細可參見參考資料中的連結）。本節介紹了提交記錄“snapshot scalability: Introduce dense array of in-progress xids.”中的修改以及涉及的相關資料結構。

### 一、資料結構
**PROC_HDR**
```

/*
 * There is one ProcGlobal struct for the whole database cluster.
 * 全域性的ProcGlobal(PROC_HDR)結構體
 *
 * Adding/Removing an entry into the procarray requires holding *both*
 * ProcArrayLock and XidGenLock in exclusive mode (in that order). Both are
 * needed because the dense arrays (see below) are accessed from
 * GetNewTransactionId() and GetSnapshotData(), and we don't want to add
 * further contention by both using the same lock. Adding/Removing a procarray
 * entry is much less frequent.
 * 對proc陣列的插入和刪除,需要以獨佔的模式持有ProcArrayLock和XidGenLock,
 * 原因是GetNewTransactionId()和GetSnapshotData()函式需要訪問,並且我們不希望使用同樣的鎖來增加爭用.
 * 其實,插入和刪除proc陣列並不頻繁.
 *
 * Some fields in PGPROC are mirrored into more densely packed arrays (e.g.
 * xids), with one entry for each backend. These arrays only contain entries
 * for PGPROCs that have been added to the shared array with ProcArrayAdd()
 * (in contrast to PGPROC array which has unused PGPROCs interspersed).
 * PGPROC中的一些欄位映象到更稠密的陣列中(如xids),每一個後臺程式都有一項.
 * 這些陣列只包含使用ProcArrayAdd()加入到共享陣列的PGPROCs.
 *
 * The dense arrays are indexed by PGPROC->pgxactoff. Any concurrent
 * ProcArrayAdd() / ProcArrayRemove() can lead to pgxactoff of a procarray
 * member to change.  Therefore it is only safe to use PGPROC->pgxactoff to
 * access the dense array while holding either ProcArrayLock or XidGenLock.
 * 稠密陣列透過PGPROC->pgxactoff欄位進行索引,所有同步ProcArrayAdd和ProcArrayRemove操作
 * 會改變procarray中的pgxactoff成員變數.
 *
 * As long as a PGPROC is in the procarray, the mirrored values need to be
 * maintained in both places in a coherent manner.
 * 由於PGPROC在proc陣列中,映象值需要以一致的方式進行運維.
 *
 * The denser separate arrays are beneficial for three main reasons: First, to
 * allow for as tight loops accessing the data as possible. Second, to prevent
 * updates of frequently changing data (e.g. xmin) from invalidating
 * cachelines also containing less frequently changing data (e.g. xid,
 * statusFlags). Third to condense frequently accessed data into as few
 * cachelines as possible.
 * 基於以下3點原因,更稠密單獨的陣列會帶來不少好處.
 * 1.緊密的迴圈訪問資料成為可能;
 * 2.避免了對修改頻度不高的資料進行修改導致CPU快取無效的情況出現;
 * 3.使頻繁訪問的資料放在小量的CPU快取中成為可能.
 *
 * There are two main reasons to have the data mirrored between these dense
 * arrays and PGPROC. First, as explained above, a PGPROC's array entries can
 * only be accessed with either ProcArrayLock or XidGenLock held, whereas the
 * PGPROC entries do not require that (obviously there may still be locking
 * requirements around the individual field, separate from the concerns
 * here). That is particularly important for a backend to efficiently checks
 * it own values, which it often can safely do without locking.  Second, the
 * PGPROC fields allow to avoid unnecessary accesses and modification to the
 * dense arrays. A backend's own PGPROC is more likely to be in a local cache,
 * whereas the cachelines for the dense array will be modified by other
 * backends (often removing it from the cache for other cores/sockets). At
 * commit/abort time a check of the PGPROC value can avoid accessing/dirtying
 * the corresponding array value.
 * 讓這些稠密陣列和PGPROC之間生成資料映象,有兩個主要原因:
 * 1.如上所述,PGPROC陣列條目需要在持有ProcArrayLock和XidGenLock的前提下才能訪問,但實際上並不需要.
 *   在沒有鎖的情況下,後臺程式有效且安全的檢查自己的值是十分重要的.
 * 2.PGPROC中的欄位避免不必要的訪問和修改.後臺程式自己的PGPROC更應該在自己的本地快取中,
 *   而無需理會陣列會被其他程式修改.在事務提交或回滾時,檢查PGPROC值可以避免訪問或者弄髒相應的陣列值
 *
 * Basically it makes sense to access the PGPROC variable when checking a
 * single backend's data, especially when already looking at the PGPROC for
 * other reasons already.  It makes sense to look at the "dense" arrays if we
 * need to look at many / most entries, because we then benefit from the
 * reduced indirection and better cross-process cache-ability.
 * 檢查單個後臺程式資料時,訪問PGPROC值是比較敏感的,特別是在已經因為其他原因檢索過PGPROC的情況下
 *
 * When entering a PGPROC for 2PC transactions with ProcArrayAdd(), the data
 * in the dense arrays is initialized from the PGPROC while it already holds
 * ProcArrayLock.
 * 在2PC事務的情況下,透過ProcArrayAdd方法進入PGPROC時,
 *   陣列中的資料在持有ProcArrayLock鎖的情況下初始化陣列.
 */
typedef struct PROC_HDR
{
	/* Array of PGPROC structures (not including dummies for prepared txns) */
	//PGPROC結構體陣列
	PGPROC	   *allProcs;

/* Array mirroring PGPROC.xid for each PGPROC currently in the procarray */
	// 映象每一個PGPROC.xid
	TransactionId *xids;

/*
	 * Array mirroring PGPROC.subxidStatus for each PGPROC currently in the
	 * procarray.
	 */
	//映象PGPROC.subxidStatus
	XidCacheStatus *subxidStates;

/*
	 * Array mirroring PGPROC.statusFlags for each PGPROC currently in the
	 * procarray.
	 */
	//映象PGPROC.statusFlags
	uint8	   *statusFlags;

/* Length of allProcs array */
	//allProcs陣列的大小
	uint32		allProcCount;
	/* Head of list of free PGPROC structures */
	//空閒PGPROC結構體連結串列頭
	PGPROC	   *freeProcs;
	/* Head of list of autovacuum's free PGPROC structures */
	//autovacuum程式空閒PGPROC結構體連結串列頭
	PGPROC	   *autovacFreeProcs;
	/* Head of list of bgworker free PGPROC structures */
	//bgworker程式
	PGPROC	   *bgworkerFreeProcs;
	/* Head of list of walsender free PGPROC structures */
	//walsender
	PGPROC	   *walsenderFreeProcs;
	/* First pgproc waiting for group XID clear */
	//等待XID組清理的第一個pgproc
	pg_atomic_uint32 procArrayGroupFirst;
	/* First pgproc waiting for group transaction status update */
	//等待事務狀態組更新的首個pgproc
	pg_atomic_uint32 clogGroupFirst;
	/* WALWriter process's latch */
	//WALWriter程式latch
	Latch	   *walwriterLatch;
	/* Checkpointer process's latch */
	//檢查點程式latch
	Latch	   *checkpointerLatch;
	/* Current shared estimate of appropriate spins_per_delay value */
	//spins_per_delay
	int			spins_per_delay;
	/* The proc of the Startup process, since not in ProcArray */
	//Startup程式的proc結構體,不再Proc陣列中
	PGPROC	   *startupProc;
	int			startupProcPid;
	/* Buffer id of the buffer that Startup process waits for pin on, or -1 */
	//Startup程式等待pin的快取ID
	int			startupBufferPinWaitBufId;
} PROC_HDR;

extern PGDLLIMPORT PROC_HDR *ProcGlobal;

```

### 二、原始碼解讀
提交記錄和相關解析如下:
```
The new array contains the xids for all connected backends / in-use
PGPROC entries in a dense manner (in contrast to the PGPROC/PGXACT
arrays which can have unused entries interspersed).
主要的改動是對於處理中的xids使用稠密陣列儲存(相對於稀疏陣列而言,每個陣列項都是有效的).

This improves performance because GetSnapshotData() always needs to
scan the xids of all live procarray entries and now there's no need to
go through the procArray->pgprocnos indirection anymore.
這樣的做法之所以有助於提升效能,原因是GetSnapshotData()函式經常需要掃描所有活動procarray條目中的xids,
修改之後不再需要透過procArray->pgprocnos進行間接訪問.

As the set of running top-level xids changes rarely, compared to the
number of snapshots taken, this substantially increases the likelihood
of most data required for a snapshot being in l2 cache.  In
read-mostly workloads scanning the xids[] array will sufficient to
build a snapshot, as most backends will not have an xid assigned.
同時,相對於快照的數目,由於正在執行的頂層xids很少改動,
這樣的處理可以提高快照需要的資料更多的留存在CPU L2快取中.
在讀為主的場景中,由於大多數後臺程式不會分配xid,因此掃描xids[]陣列對於構建快照已經足夠了

To keep the xid array dense ProcArrayRemove() needs to move entries
behind the to-be-removed proc's one further up in the array. Obviously
moving array entries cannot happen while a backend sets it
xid. I.e. locking needs to prevent that array entries are moved while
a backend modifies its xid.
為了保持xid陣列的稠密,函式ProcArrayRemove()需要移動在待移除的proc之後的條目,
顯然,在設定xid時是不能移動陣列條目的,因此需要鎖定機制.

To avoid locking ProcArrayLock in GetNewTransactionId() - a fairly hot
spot already - ProcArrayAdd() / ProcArrayRemove() now needs to hold
XidGenLock in addition to ProcArrayLock. Adding / Removing a procarray
entry is not a very frequent operation, even taking 2PC into account.
為了避免在GetNewTransactionId()函式中鎖定熱點ProcArrayLock,
ProcArrayAdd()和ProcArrayRemove除了ProcArrayLock之外需要持有XidGenLock.
就算考慮到2PC,新增和移除procarray條目也不會頻繁發生.

Due to the above, the dense array entries can only be read or modified
while holding ProcArrayLock and/or XidGenLock. This prevents a
concurrent ProcArrayRemove() from shifting the dense array while it is
accessed concurrently.
綜上,稠密陣列中的條目在持有ProcArrayLock和/或XidGenLock時才能讀和修改.
這可以阻止在同步訪問時ProcArrayRemove()函式同步移動稠密陣列

While the new dense array is very good when needing to look at all
xids it is less suitable when accessing a single backend's xid. In
particular it would be problematic to have to acquire a lock to access
a backend's own xid. Therefore a backend's xid is not just stored in
the dense array, but also in PGPROC. This also allows a backend to
only access the shared xid value when the backend had acquired an
xid.
新的稠密陣列在需要檢索所有xids時表現得很好,在訪問單個程式xid時則一般.
特別在不得不請求鎖來訪問後臺程式自己的xid時表現尤其不好.
因此後臺程式的xid不單單儲存在稠密陣列中,還儲存在PGPROC中.
這允許在請求了事務號後臺程式只需要訪問共享的xid.

The infrastructure added in this commit will be used for the remaining
PGXACT fields in subsequent commits. They are kept separate to make
review easier.
這次提交增加的"基建"在後續的提交中用於PGXACT仍遺留的欄位,
之所以還儲存在這裡是因為便於複審.
```
原始碼可參見參考資料中的連結。

### 三、跟蹤分析
N/A

### 四、參考資料
1.[Improving connection scalability: GetSnapshotData()](https://www.postgresql.org/message-id/20200301083601.ews6hz5dduc3w2se%40alap3.anarazel.de)
2.[snapshot scalability: Introduce dense array of in-progress xids.](https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=941697c3c1ae5d6ee153065adb96e1e63ee11224)

PostgreSQL 原始碼解讀（261）- PG 14（Improving connection scalability）#13

相關文章