Redis基礎知識（學習筆記16--持久化 (2)）

三. AOF持久化

AOF，Append Only File，是指Redis將每一次的寫操作（命令）都以日誌的形式記錄到一個AOF檔案中的持久化技術。當需要恢復記憶體資料時，將這些寫操作重新執行一次，便會恢復到之前的記憶體資料狀態。

3.1 AOF基礎配置

（1）AOF的開啟

直接修改配置檔案的方式

# Please check https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/ for more information.

appendonly no

預設情況下AOF持久化是沒有開啟的，透過修改配置檔案中的appendonly的屬性為yes。這種方式需要重啟生效。

命令的檢視修改方式

狀態檢視的，可透過下面的命令：

> config get appendonly

修改的話

>config set appendonly yes

注意，這個命令只是修改了記憶體中的配置。這種命令修改的方式，不需要重啟生效。

如果要重新整理到配置檔案中

>config rewrite

（2）檔名的配置

# - appendonly.aof.1.base.rdb as a base file.
# - appendonly.aof.1.incr.aof, appendonly.aof.2.incr.aof as incremental files.
# - appendonly.aof.manifest as a manifest file.

appendfilename "appendonly.aof"  ###注意：這只是定義了這類檔案的字首

Redis 7 在這裡發生了重大變化。原來只有一個appendonly.aof檔案，現在具有了三類多個檔案：

*** 基礎檔案：可以是RDF格式也可以是AOF格式。其存放的內容是由RDB轉為AOF當時記憶體的快照資料。該檔案可以有多個。

*** 增量檔案：以操作日誌形式記錄轉為AOF後的寫入操作。該檔案可以有多個。

*** 清單檔案：用於維護AOF檔案的建立順序，保障啟用時的應用順序，該檔案只有一個。

（3）混合式持久化開啟

# Redis can create append-only base files in either RDB or AOF formats. Using
# the RDB format is always faster and more efficient, and disabling it is only
# supported for backward compatibility purposes.
aof-use-rdb-preamble yes

對於基本檔案(例如：appendonly.aof.1.base.rdb as a base file) 可以是RDF格式也可以是AOF格式，透過aof-use-rdb-preamble 屬性可以選擇。其預設值為yes，即預設AOF持久化的基本檔案為rdb格式檔案，也就是預設採用混合式持久化。

（4）AOF完整配置引數

############################## APPEND ONLY MODE ###############################

# By default Redis asynchronously【非同步】 dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).###概況說RDB存在的風險
#
# The Append Only File is an alternative【可供替代的】 persistence mode that provides
# much better durability【永續性】. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic【巨大的，引人注目的】 event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.###概況說AOF開啟帶來的好處
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Note that changing this value in a config file of an existing database and
# restarting the server can lead to data loss. A conversion needs to be done
# by setting it via CONFIG command on a live server first.
#
# Please check https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/ for more information.

appendonly no

# The base name of the append only file.
#
# Redis 7 and newer use a set of append-only files to persist the dataset
# and changes applied to it. There are two basic types of files in use:
#
# - Base files, which are a snapshot representing the complete state of the
#   dataset at the time the file was created. Base files can be either in
#   the form of RDB (binary serialized) or AOF (textual commands).
# - Incremental files, which contain additional commands that were applied
#   to the dataset following the previous file.
#
# In addition, manifest【貨單；清單】 files are used to track the files and the order in
# which they were created and should be applied.
#
# Append-only file names are created by Redis following a specific pattern.
# The file name's prefix is based on the 'appendfilename' configuration
# parameter, followed by additional information about the sequence【序號】 and type【型別】.
#
# For example, if appendfilename is set to appendonly.aof, the following file
# names could be derived【產生；獲得】:
#
# - appendonly.aof.1.base.rdb as a base file.【大多數情況下，這個型別的檔案就一個；如果記憶體裡面的資料量很大時，也可能會不止一個】
# - appendonly.aof.1.incr.aof, appendonly.aof.2.incr.aof as incremental files.【這個型別的檔案會同時有多個】
# - appendonly.aof.manifest as a manifest file.

appendfilename "appendonly.aof"

# For convenience, Redis stores all persistent append-only files in a dedicated【專門的；特定的】
# directory. The name of the directory is determined by the appenddirname
# configuration parameter.

appenddirname "appendonlydir"

# The fsync() call tells the Operating System to actually write data on disk
# instead of waiting for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log. Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary, use "always" that's very slow but a bit safer than
# everysec.
#
# More details please check the following article:
# http://antirez.com/post/redis-persistence-demystified.html
#
# If unsure, use "everysec".

# appendfsync always
appendfsync everysec
# appendfsync no

# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate this problem it's possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability of Redis is
# the same as "appendfsync no". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.

no-appendfsync-on-rewrite no

# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling【隱式呼叫】
# BGREWRITEAOF when the AOF log size grows by the specified percentage【指定的百分比】.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

# An AOF file may be found to be truncated at the end during the Redis
# startup process, when the AOF data gets loaded back into memory.
# This may happen when the system where Redis is running
# crashes, especially when an ext4 filesystem is mounted without the
# data=ordered option (however this can't happen when Redis itself
# crashes or aborts but the operating system still works correctly).
#
# Redis can either exit with an error when this happens, or load as much
# data as possible (the default now) and start if the AOF file is found
# to be truncated at the end. The following option controls this behavior.
#
# If aof-load-truncated is set to yes, a truncated AOF file is loaded and
# the Redis server starts emitting a log to inform the user of the event.
# Otherwise if the option is set to no, the server aborts with an error
# and refuses to start. When the option is set to no, the user requires
# to fix the AOF file using the "redis-check-aof" utility before to restart
# the server.
#
# Note that if the AOF file will be found to be corrupted in the middle
# the server will still exit with an error. This option only applies when
# Redis will try to read more data from the AOF file but not enough bytes
# will be found.
aof-load-truncated yes

# Redis can create append-only base files in either RDB or AOF formats. Using
# the RDB format is always faster and more efficient, and disabling it is only
# supported for backward compatibility purposes.
aof-use-rdb-preamble yes

# Redis supports recording timestamp annotations in the AOF to support restoring
# the data from a specific point-in-time. However, using this capability changes
# the AOF format in a way that may not be compatible with existing AOF parsers.
aof-timestamp-enabled no

################################ SHUTDOWN #####################################

3.2 AOF 檔案格式

AOF檔案包含三類檔案：基本檔案、增量檔案與清單檔案。其中基本檔案一般為rdb格式，不再贅述，下面主要看下增量檔案和清單檔案的內容格式。

（1）Redis 協議

增量副檔名為.aof，採用AOF格式。AOF格式其實就是redis通訊協議格式，AOF持久化檔案的本質就是基於redis通訊協議的文字，將命令以純文字的方式寫入到檔案中。

redis協議規定，redis文字是以行來劃分，每行以\r\n行結束。每一行都有一個訊息頭，以表示訊息型別。訊息頭由六種不同的符號表示，其意義如下：

• （+）表示一個正確的狀態訊息；
• （-）表示一個錯誤的訊息；
• （*）表示訊息體總共有多少行，不包括當前行；
• （$）表示下一行訊息資料的長度，不包括換行符長度\r\n；
• （空）表示一個訊息資料；
• （:）表示返回一個數值。

（2）檢視AOF檔案

開啟appendonly.aof.1.incr.aof檔案，可以看到格式如下：

（3）清單檔案

開啟appendonly.aof.manifest檔案，其格式如下：

tpye有兩種型別：b（base file） 和 i（incremental file）。

該檔案首先會按照seq序號列舉出所有基本檔案，基本檔案type型別為b，然後再按照seq序號再列舉出所有增量檔案，增量檔案type為i。

對redis啟動時的資料恢復，也會按照該檔案從上到下依次載入他們中的資料。

3.3 Rewrite 機制

隨著使用時間的推移，AOF檔案會越來越大。為了防止AOF檔案由於太大而佔用大量的磁碟空間，降低效能，redis引入了rewrite機制來對AOF檔案進行壓縮。

（1） rewrite的定義

所謂rewrite其實就是對AOF檔案進行重寫整理。當rewrite開啟後，主程序redis-server建立出一個子程序bgrewriteaof，由該子程序完成rewrite過程。其首先對現有aof檔案進行rewrite計算，將計算結果寫入到一個臨時檔案，寫入完畢後，再rename該臨時檔案為原aof檔名，覆蓋原有檔案。

（2）rewrite計算

rewrite計算也稱為rewrite策略。rewrite計算遵循以下策略：

• 讀操作命令不寫入檔案；
• 無效命令不寫入檔案；
• 過期資料不寫入檔案；
• 多條命令合併寫入檔案。

（3）手動開啟rewrite

rewrite過程的執行有兩種方式。一種是透過bgrewriteaof命令手動開啟，一種是透過設定條件自動開啟。

手動啟動（觸發）的命令：

> bgrewriteaof

該命令會使主程序redis-server建立出一個子程序bgrewriteaof，由該子程序完成rewrite過程。而在rewrite期間，redis-server仍是可以對外提供讀寫服務的 。

（4）自動開啟rewrite

其實就是配置檔案中的引數，如下：

# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size grows by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

學習參閱特別宣告

【Redis影片從入門到高階】

【https://www.bilibili.com/video/BV1U24y1y7jF?p=11&vd_source=0e347fbc6c2b049143afaa5a15abfc1c】