Rsync服務
【1】、rsync定義
作用:
1.本地的備份 類似cp命令 意義不大
2.遠端的複製 類似scp命令 意義不大 (增量備份)
如果第一次遠端執行檔案複製動作使用scp和rsync一樣
如果第二次遠端執行,有新增加的檔案可以使用rsync命令
3.守護程序模式 持續不斷地在後臺執行,接收客戶端傳送的重要 資料。類似於百度網盤。
scp 每次都是全量複製
rsync 每次都是增量複製
# 全量備份
[root@web01 ~]# scp -r test backup:~/
Authorized users only. All activities may be monitored and reported.
01.txt 100% 0 0.0KB/s 00:00
02.txt 100% 0 0.0KB/s 00:00
03.txt 100% 0 0.0KB/s 00:00
04.txt 100% 0 0.0KB/s 00:00
05.txt 100% 0 0.0KB/s 00:00
[root@web01 ~]# touch test/{06..09}.txt
[root@web01 ~]# scp -r test backup:~/
# 全量備份再有新增加的內容時，不僅會備份新增的內容，還會再備份一次就的資料
Authorized users only. All activities may be monitored and reported.
01.txt 100% 0 0.0KB/s 00:00
02.txt 100% 0 0.0KB/s 00:00
03.txt 100% 0 0.0KB/s 00:00
04.txt 100% 0 0.0KB/s 00:00
05.txt 100% 0 0.0KB/s 00:00
06.txt 100% 0 0.0KB/s 00:00
07.txt 100% 0 0.0KB/s 00:00
08.txt 100% 0 0.0KB/s 00:00
09.txt 100% 0 0.0KB/s 00:00
# 增量備份
[root@web01 ~]# rsync -avz test backup:~/
The authenticity of host 'backup (192.168.121.41)' can't be established.
ECDSA key fingerprint is SHA256:ojGuz7nAhVgYaZgcnxgbxZOsIwQOL7DkBGqc38t7hXw.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'backup' (ECDSA) to the list of known hosts.
Authorized users only. All activities may be monitored and reported.
sending incremental file list
test/
test/01.txt
test/02.txt
test/03.txt
test/04.txt
test/05.txt
test/06.txt
test/07.txt
test/08.txt
test/09.txt
test/10.txt
sent 595 bytes received 210 bytes 230.00 bytes/sec
total size is 0 speedup is 0.00
[root@web01 ~]# touch test/new.txt
[root@web01 ~]# rsync -avz test backup:~/
# 增量備份之備份第一備份後新增加的內容
Authorized users only. All activities may be monitored and reported.
sending incremental file list
test/
test/new.txt
sent 296 bytes received 39 byte
【2】、rsync三種工作模式
SYNOPSIS
Local: rsync [OPTION...] SRC... [DEST]
Access via remote shell:
Pull: rsync [OPTION...] [USER@]HOST:SRC... [DEST]
Push: rsync [OPTION...] SRC... [USER@]HOST:DEST
Access via rsync daemon:
Pull: rsync [OPTION...] [USER@]HOST::SRC... [DEST]
rsync [OPTION...] rsync://[USER@]HOST[:PORT]/SRC... [DEST]
Push: rsync [OPTION...] SRC... [USER@]HOST::DEST
rsync [OPTION...] SRC... rsync://[USER@]HOST[:PORT]/DEST
1、本地模式
類似於cp命令，瞭解即可
2、遠端模式
類似於scp命令(在有增量需求複製的時候或者有資料同步的要求才使用)
# 注意rsync的目錄後面加/表示複製目錄下的檔案 不加/表示複製目錄及下所有內容
[root@web01 ~]# rsync -avz test1/ backup:~/
Authorized users only. All activities may be monitored and reported.
sending incremental file list
./
01.txt
02.txt
03.txt
04.txt
sent 258 bytes received 95 bytes 706.00 bytes/sec
total size is 0 speedup is 0.00
[root@web01 ~]# rsync -avz test backup:~/
Authorized users only. All activities may be monitored and reported.
sending incremental file list
test/
test/01.txt
test/02.txt
test/03.txt
test/04.txt
test/05.txt
sent 329 bytes received 115 bytes 296.00 bytes/sec
total size is 0 speedup is 0.00
# rsync拉取模式，將遠端上的東西拉取到本地
[root@backup ~]# rsync -avz web01:~/test ./
Authorized users only. All activities may be monitored and reported.
receiving incremental file list
test/
test/01.txt
test/02.txt
test/03.txt
test/04.txt
test/05.txt
sent 123 bytes received 329 bytes 129.14 bytes/sec
total size is 0 speedup is 0.00
在使用rsync時，如果dest path沒有加使用者，預設使用本地登入的使用者去登入遠端
[oldboy@web01 ~]$ rsync -avz /etc/hosts backup:~/
# 我們在傳遞時，需要保證本地和遠端都有相同的使用者
The authenticity of host 'backup (192.168.121.41)' can't be established.
ECDSA key fingerprint is SHA256:ojGuz7nAhVgYaZgcnxgbxZOsIwQOL7DkBGqc38t7hXw.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'backup,192.168.121.41' (ECDSA) to the list of known hosts.
Authorized users only. All activities may be monitored and reported.
oldboy@backup's password:
3、守護程序模式
主機角色 外網IP(WAN) 內網IP(LAN) 主機名稱
Rsync服務端 192.168.121.41 172.16.1.41 backup
Rsync客戶端 192.168.121.7 172.16.1.7 web01
1、服務端安裝rsync服務
[root@backup ~]# yum install -y rsync
2、服務端配置rsync
[root@backup ~]# rpm -qc rsync
/etc/rsyncd.conf
/etc/sysconfig/rsyncd
[root@backup ~]# cat /etc/rsyncd.conf
uid = rsync # 執行程序的使用者
gid = rsync # 執行程序的使用者組
port = 873 # 監聽埠
fake super = yes # 無需讓rsync以root的身份執行，允許接受檔案的完整屬性
use chroot = no # 禁錮推送的資料到某一個目錄，不允許跳出該目錄
max connections = 200 # 最大連線數
timeout = 600 # 超時時間
ignore errors # 忽略錯誤資訊
read only = false # 對備份資料不是隻讀的，可讀可寫
list = false #不許檢視模組資訊
auth users = rsync_backup # 定義虛擬使用者，作為連線認證使用者
secrets file = /etc/rsync.passwd # 定義rsync服務使用者連線認證密碼檔案路徑
log file = /var/log/rsyncd.log
#####################################
[backup] # 定義模組資訊
path = /backup # 定義接收備份資料的目錄
3.根據配置檔案建立必要的資料資訊
[root@backup ~]# grep rsync /etc/passwd
[root@backup ~]#
# 建立虛擬使用者rsync
[root@backup ~]# useradd -M -s /sbin/nologin rsync
# 建立密碼檔案
[root@backup ~]# cat /etc/rsync.passwd
rsync_backup:123456
[root@backup ~]# chmod 600 /etc/rsync.passwd
# 建立接收目錄
[root@backup ~]# mkdir /backup
[root@backup ~]# ll /backup -d
drwxr-xr-x 2 root root 6 Dec 3 16:17 /backup
# rsync是以啟動程序的使用者身份往/backup目錄中寫入
[root@backup ~]# chown rsync:rsync /backup/
4.服務端啟動rsync服務
[root@backup ~]# systemctl enable rsyncd --now
Created symlink /etc/systemd/system/multi-user.target.wants/rsyncd.service → /usr/lib/systemd/system/rsyncd.service.
[root@backup ~]# ss -tunlp | grep rsync
tcp LISTEN 0 5 0.0.0.0:873 0.0.0.0:* users:(("rsync",pid=78018,fd=3))
tcp LISTEN 0 5 [::]:873 [::]:* users:(("rsync",pid=78018,fd=5))
需要使用守護程序模式的語法格式推送檔案
Push: rsync [OPTION...] SRC... [USER@]HOST::DEST
推送: rsync -avz /etc/passwd rsync_backup@10.0.0.41::模組的名稱
[root@web01 ~]# rsync -avz /etc/passwd rsync_backup@backup::backup
Password:
sending incremental file list
passwd
sent 847 bytes received 43 bytes 356.00 bytes/sec
total size is 1,848 speedup is 2.08
客戶端引數：--password-file，指定密碼檔案在哪
[root@web01 ~]# echo 123456 > /etc/pa.txt
[root@web01 ~]# chmod 600 /etc/pa.txt
[root@web01 ~]# rsync -avz /etc/hosts rsync_backup@backup::backup --password-file=/etc/pa.txt
sending incremental file list
hosts
sent 176 bytes received 43 bytes 438.00 bytes/sec
total size is 200 speedup is 0.91
rsync客戶端密碼內建變數
[root@web01 ~]# echo $RSYNC_PASSWORD [root@web01 ~]# export RSYNC_PASSWORD=123456 [root@web01 ~]# echo $RSYNC_PASSWORD 123456 [root@web01 ~]# rsync -avz /etc/rc.local rsync_backup@backup::backup sending incremental file list rc.local -> rc.d/rc.local sent 62 bytes received 23 bytes 170.00 bytes/sec total size is 13 speedup is 0.15rsync在推送資料時，首先會查RSYNC_PASSWORD
# 案例
1.客戶端提前準備存放的備份的目錄，目錄規則如 下:/backup/web01_172.16.1.7_2018-09-02 date +%F 2.客戶端在本地打包備份(系統配置檔案、應用配置等)複製 至/backup/web01_172.16.1.7_2018-09-02
3.客戶端最後將備份的資料進行推送至備份伺服器 守護程序
4.客戶端每天凌晨1點定時執行該指令碼
5.客戶端伺服器本地保留最近7天的資料, 避免浪費磁碟空間
#!/bin/bash
mkdir -p /backup
IP=`hostname -I | awk -F" " '{print $1}'`
path=/backup/web01_${IP}_`date +%F`
tar -zcvf $path /etc/
rsync -avz $path rsync_backup@backup::backup
find /backup -mtime +7 -exec rm -f {} \;
# 拉取資料
[root@web01 ~]# ls
backup.sh html
[root@web01 ~]# rsync -avz rsync_backup@backup::backup/hosts .
receiving incremental file list
hosts
sent 43 bytes received 180 bytes 446.00 bytes/sec
total size is 200 speedup is 0.90
[root@web01 ~]# ls
backup.sh hosts html
新增模組，可以儲存到多個服務的目錄下
[root@backup ~]# vim /etc/rsyncd.conf
# 新增一個模組即可
[data]
path = /data
[root@backup ~]# systemctl restart rsyncd
[root@backup ~]# mkdir /data
[root@backup ~]# chown rsync:rsync /data/
# 在客戶端進行傳遞，指定新加的模組名
[root@web01 ~]# rsync -avz /etc/rc.local rsync_backup@backup::data
sending incremental file list
rc.local -> rc.d/rc.local
sent 62 bytes received 23 bytes 170.00 bytes/sec
total size is 13 speedup is 0.15
【3】、rsync引數
-a #歸檔模式傳輸, 等於-tropgDl
-v #詳細模式輸出, 列印速率, 檔案數量等
-z #傳輸時進行壓縮以提高效率
-r #遞迴傳輸目錄及子目錄，即目錄下得所有目錄 都同樣傳輸。
-t #保持檔案時間資訊
-o #保持檔案屬主資訊
-p #保持檔案許可權
-g #保持檔案屬組資訊
-l #保留軟連線
-P #顯示同步的過程及傳輸時的進度等資訊
-D #保持裝置檔案資訊
-L #保留軟連線指向的目標檔案
-e #使用的通道協議,指定替代rsh的shell程式
--exclude=PATTERN #指定排除不需要傳輸的檔案模式
--exclude-from=file #檔名所在的目錄檔案
--bwlimit=100 #限速傳輸
--partial #斷點續傳
--delete #讓目標目錄和源目錄資料保持一致
--password-file=xxx #使用密碼檔案，守護程序模式下才可以使用
# 使用--exclude 排除檔案，不參與傳輸
[root@backup ~]# rsync -avz test/ web01:~/ --exclude=43.txt
Authorized users only. All activities may be monitored and reported.
sending incremental file list
./
hosts
passwd
sent 992 bytes received 57 bytes 2,098.00 bytes/sec
total size is 2,005 speedup is 1.91
# 使用{} 排除多個檔案
[root@backup ~]# rsync -avz test web01:~/ --exclude={43.txt,passwd}
Authorized users only. All activities may be monitored and reported.
sending incremental file list
test/
test/hosts
sent 208 bytes received 39 bytes 164.67 bytes/sec
total size is 200 speedup is 0.81
--exclude-from=file 可以指定檔案，安裝指定的檔案中的名字排除不需要進行傳輸的檔案
[root@backup ~]# vim /opt/ex.txt
[root@backup ~]# cat /opt/ex.txt
passwd
hosts
[root@backup ~]# rsync -avc test web01:~/ --exclude-from=/opt/ex.txt
Authorized users only. All activities may be monitored and reported.
sending incremental file list
test/
test/43.txt
sent 136 bytes received 39 bytes 350.00 bytes/sec
total size is 0 speedup is 0.00
--bwlimit 限速傳遞
[root@backup ~]# dd if=/dev/zero of=/1g.txt bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 10.4023 s, 101 MB/s
[root@backup ~]# rsync -avcP 1g.txt web01:~/
Authorized users only. All activities may be monitored and reported.
sending incremental file list
1g.txt
448,299,008 42% 26.47MB/s 0:00:22
[root@backup ~]# rsync -avcP 1g.txt web01:~/ --bwlimit=1m
Authorized users only. All activities may be monitored and reported.
sending incremental file list
1g.txt
26,509,312 2% 1.00MB/s 0:16:37
--password-file只能在守護程序模式下使用
--delete 可以讓兩端伺服器的內容保持一致
# 誰在前就以誰為準
[root@backup ~]# rsync -avz web01:~/ . --delete
Authorized users only. All activities may be monitored and reported.
receiving incremental file list
deleting test/passwd
deleting test/hosts
deleting test/43.txt
deleting test/
deleting passwd.txt
deleting 1g.txt
./
.bash_history
.lesshst
.viminfo
.ssh/
.ssh/authorized_keys
.ssh/id_rsa
.ssh/id_rsa.pub
.ssh/known_hosts
sent 364 bytes received 4,451 bytes 9,630.00 bytes/sec
total size is 9,732 speedup is 2.02
# 可以用於快速同步資訊，一般用在網站被篡改，可以透過一條命令進行同步
[root@backup ~]# rsync -avz html web01:~/
Authorized users only. All activities may be monitored and reported.
root@web01's password:
sending incremental file list
html/
html/1.html
html/2.html
html/3.html
sent 287 bytes received 77 bytes 104.00 bytes/sec
total size is 45 speedup is 0.12
# 模擬網站被篡改
[root@web01 ~]# find html/ -type f -name "*.html" -exec sed -i 's#haha#???#g' {} \;
[root@web01 ~]# cat html/1.html
www.???ha.com
[root@web01 ~]# cat html/2.html
www.???ha.com
[root@web01 ~]# cat html/3.html
www.???ha.com
# 使用rsync進行同步
[root@backup ~]# rsync -avz html/ web01:~/html --delete
Authorized users only. All activities may be monitored and reported.
root@web01's password:
sending incremental file list
./
1.html
2.html
3.html
sent 276 bytes received 94 bytes 105.71 bytes/sec
total size is 45 speedup is 0.12
[root@web01 ~]# cat html/1.html
www.hahaha.com
[root@web01 ~]# cat html/2.html
www.hahaha.com
[root@web01 ~]# cat html/3.html
www.hahaha.com