Redis基礎知識（學習筆記17--持久化 (3)）

东山絮柳仔發表於2024-07-13
Redis筆記持久化

3.4 引數最佳化

（1）appendfsync

# The fsync() call tells the Operating System to actually write data on disk
# instead of waiting for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster. ##這種模式比較快
# always: fsync after every write to the append only log. Slow, Safest.  ##這種模式比較安全
# everysec: fsync only one time every second. Compromise. ##妥協或折中的方案
#
# The default is "everysec"【預設值】, as that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer【輸出快取】 when
# it wants, for better performances (but if you can live with【接受】 the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary【相反】, use "always" that's very slow but a bit safer than
# everysec.
#
# More details please check the following article:
# http://antirez.com/post/redis-persistence-demystified.html
#
# If unsure, use "everysec".

# appendfsync always
appendfsync everysec
# appendfsync no

當客戶端提交寫命令後，該命令就會寫入到aof_buf中，而aof_buf中的資料持久化到磁碟AOF檔案的過程稱為資料同步。

何時將aof_buf中的資料同步到AOF檔案？採用不同的資料同步策略，同步的時機是不同的，有三種策略：

  • alwasy：寫操作命令寫入aof_buf後會立即呼叫fsync()系統函式，將其追加到AOF檔案。該策略效率較低，但是相對較安全，不會丟失太多資料。最多就是剛剛執行過的寫操作在尚未同步時出現當機或重啟，將這一操作丟失。
  • no：寫操作命令寫入aof_buf後什麼也不做，不會呼叫fsync()函式。而將aof_buf中的資料同步到磁碟的操作由作業系統負責。Linux系統預設同步週期為30秒。效率較高。
  • everysec：預設策略。寫操作命令寫入aof_buf後並不直接呼叫fsync()，而是每秒呼叫一次fsync()系統函式來完成同步。該策略兼顧到了效能與安全，是一種折中方案。

（2）no-appendfsync-on-rewrite

# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix【沒有修復】 for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate【緩和；減輕；緩解】 this problem it's possible to use the following option
# that will prevent【阻塞；阻止；】 fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability【永續性】 of Redis is
# the same as "appendfsync no". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency【延遲】 problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.

no-appendfsync-on-rewrite no

該屬性用於指定，當AOF fsync策略設定為always 或 everysec，當主程序建立了子程序正在執行bgsave或bgrewriteaof時，主程序是否不呼叫fsync()來做資料同步。設定為no，雙重否定即肯定，主程序會呼叫fsync()做同步。而yes則不會呼叫fsync()做資料同步。

如果呼叫了fsync()，在需要同步的資料量非常大時，會阻塞主程序對外提供服務，即會存在延遲問題。如果不呼叫fsync()，則AOF fsync策略相當於設定為了no，可能會存在高達30秒日誌的丟失。

（3）aof-rewrite-incremental-fsync 和 rdb-save-incremental-fsync

注意：這一步的定義在【########## ADVANCED CONFIG #######】部分

# When a child rewrites the AOF file, if the following option is enabled
# the file will be fsync-ed every 4 MB of data generated. This is useful
# in order to commit the file to the disk more incrementally and avoid
# big latency【延遲】 spikes【尖刺】.
aof-rewrite-incremental-fsync yes

# When redis saves RDB file, if the following option is enabled
# the file will be fsync-ed every 4 MB of data generated. This is useful
# in order to commit the file to the disk more incrementally and avoid
# big latency spikes.
rdb-save-incremental-fsync yes

當bgrewriteaof在執行過程中也是先將rewrite計算的結果寫入到了aof_rewrite_buf快取中，然後當快取中資料達到一定量後，就會呼叫fsync()進行刷盤操作，即資料同步，將資料寫入到臨時檔案。該屬性用於控制fsync()每次刷盤的資料量最大不超過4MB。這樣可以避免由於單次刷盤量過大而引發長時間阻塞。

（4）aof-load-truncated

# An AOF file may be found to be truncated【截斷的；縮減的】 at the end during the Redis
# startup process, when the AOF data gets loaded back into memory.
# This may happen when the system【指作業系統】 where Redis is running
# crashes, especially when an ext4 filesystem is mounted without the
# data=ordered option (however this can't happen when Redis itself
# crashes or aborts but the operating system still works correctly).
#
# Redis can either exit with an error when this happens, or load as much
# data as possible (the default now) and start if the AOF file is found
# to be truncated at the end. The following option controls this behavior.
#
# If aof-load-truncated is set to yes, a truncated AOF file is loaded and
# the Redis server starts emitting【發出；發射】 a log to inform the user of the event.
# Otherwise if the option is set to no, the server aborts【終止】 with an error
# and refuses to start. When the option is set to no, the user requires
# to fix the AOF file using the "redis-check-aof" utility before to restart
# the server.
#
# Note that if the AOF file will be found to be corrupted【損壞的】 in the middle
# the server will still exit【退出】 with an error. This option only applies when
# Redis will try to read more data from the AOF file but not enough bytes
# will be found.
aof-load-truncated yes

（5）aof-timestamp-enabled

# Redis supports recording timestamp annotations【註解】 in the AOF to support restoring # the data from a specific point-in-time. However, using this capability changes # the AOF format in a way that may not be compatible【相容的】 with existing AOF parsers.【當前的AOF解析器】 aof-timestamp-enabled no

該屬性設定為yes則會開啟在AOF檔案中增加時間戳的顯示功能，可方便按照時間對資料進行恢復。但該方式可能會與AOF解析器不相容，所以預設值為no，不開啟。

3.5 AOF持久化過程

AOF詳細的持久化過程如下：

1）Redis接受到寫操作命令並不是直接追加到磁碟的AOF檔案的，而是將每一條寫命令按照redis通訊協議格式暫時新增到AOF緩衝區aof_buf。

2）根據設定的資料同步策略，當同步條件滿足時，再將緩衝區中的資料一次性寫入磁碟的AOF檔案，以減少磁碟IO次數，提高效能。

3）當磁碟的AOF檔案大小達到rewrite條件時，redis-server主程序會fork出一個子程序bgwriteaof，由該子程序完成rewrite過程。

4）子程序bgwriteaof首先對磁碟AOF檔案進行rewrite計算，將計算結果寫入到一個臨時檔案，全部寫入完畢後，再rename該臨時檔案為磁碟檔案的原名稱，覆蓋原檔案。

5）如果在rewrite過程中又有寫操作命令寫入到臨時檔案後，那麼這些資料會暫時寫入aof_rewrite_buf緩衝區。等將全部rewrite計算結果寫入臨時檔案後，會先將aof_rewrite_buf緩衝區中的資料寫入臨時檔案，然後再rename為磁碟檔案的原名稱，覆蓋原檔案。

概況如下：

四. RDB 與 AOF 對比

4.4.1 RDB優勢與不足

（1）RDB優勢

  • RDB檔案較小（記憶體資料）；
  • 資料恢復較快。

（2）RDB不足

  • 在一次性寫入量較大的情況下，資料丟失風險較大；持久化過程相對較長，較重的磁碟IO；因此資料永續性相對差；
  • 寫時複製會降低效能；
  • RDB檔案可讀性較差。

4.4.2 AOF優勢與不足

（1）AOF優勢

  • 資料及時永續性強（資料丟失的可能性小）；
  • AOF檔案可讀性強。

（2）AOF不足

  • AOF檔案相對較大（日誌型）；
  • 寫操作會影響效能；
  • 資料恢復相對慢。

4.4.3 持久化技術的選型

  • 官方推薦使用RDB與AOF混合式持久化。
  • 若對資料永續性要求不高，則推薦使用純RDB持久化方式。
  • 官方不推薦使用純AOF持久化方式（檔案大；恢復慢）。
  • 若Redis僅用於快取，則無需使用任何持久化技術。

學習參閱特別宣告

【Redis影片從入門到高階】

【https://www.bilibili.com/video/BV1U24y1y7jF?p=11&vd_source=0e347fbc6c2b049143afaa5a15abfc1c】

相關文章