keepalived+MySQL實現高可用

gegeman發表於2020-08-03

Snipaste_2020-08-03_23-21-47



(一)keepalived概述

Keepalived通過VRRP(虛擬路由冗餘協議)協議實現虛擬IP的漂移。當master故障後,VIP會自動漂移到backup,這時通知下端主機重新整理ARP表,如果業務是通過VIP連線到伺服器的,則此時依然能夠連線到正常執行的主機,RedHat給出的VRRP工作原理如下圖:

wps1


本來對VIP漂移有一定了解的我,看了上面的圖後,越來越懵了。因此只能根據我的個人理解,來對keepalived的VIP漂移做一個解釋了,假設我現在有一套這樣的環境:
主機A的IP地址為:192.168.10.11
主機B的IP地址為:192.168.10.12

我們再單獨定義一個keepalived使用的VIP:192.168.10.10

當2臺主機安裝了keepalive並正常執行時,keepalive會選擇一個節點做為主節點(這裡假設為主機A,IP為192.168.10.11),由於A是主節點,所以主機A上還會生成一個IP地址192.168.10.10,即虛擬IP(Virtual IP,也稱VIP),此時我們使用192.168.10.10訪問主機,訪問到的主機是A;假如A主機上的keepalived由於某些原因(例如伺服器當機、使用者主動關閉…)關閉了,keepalived備用節點會檢查與主節點keepalived的通訊是否正常,檢測到不正常,則會提升一個備節點為主節點,相應的虛擬IP也會在對應的主機上生成,從而實現高可用的目的。

Snipaste_2020-08-03_18-54-27


(二)MySQL是如何結合keepalived實現高可用的

在MySQL中,通過搭建MySQL雙主複製,保持2臺主機上的MySQL資料庫一模一樣,並在2臺主機上安裝keepalived軟體,啟用VIP,使用者應用程式通過VIP訪問資料庫。當包含VIP的主機上的資料庫發生故障時,關閉keepalived,從而將VIP漂移到另一個節點,使用者依然可以正常訪問資料庫。 (這裡需要注意,雖然MySQL架構雙主複製,2個節點都可以寫入資料,但是我們在使用的時候,是通過VIP訪問其中一個例項,並沒有2個資料庫例項一起使用)。這裡我簡單畫了一個流程圖,來說明keepalive與MySQL實現高可用的過程:

Snipaste_2020-08-03_19-30-48


(三)keepalived+MySQL實現高可用過程實現

基礎環境規劃:

主機名 IP地址 備註
伺服器A hosta 192.168.10.11 keepalive主節點
伺服器B hostb 192.168.10.12 keepalive備節點
192.168.10.10 虛擬IP,會在keepalive啟動後分配到上面2臺機器的主節點上


(3.1)搭建MySQL雙主複製環境

STEP1:安裝MySQL過程見:https://www.cnblogs.com/lijiaman/p/10743102.html

STEP2:配置雙主複製引數

伺服器A 伺服器B

[mysqld]
basedir=/usr/local/mysql
datadir=/mysql/data

server_id = 1
binlog_format=ROW
log_bin=/mysql/binlog/master-bin
auto-increment-increment = 2            #欄位變化增量值
auto-increment-offset = 1               #初始欄位ID為1
slave-skip-errors = all                 #忽略所有複製產生的錯誤
gtid_mode=ON
enforce-gtid-consistency=ON


[mysqld]
basedir=/usr/local/mysql
datadir=/mysql/data

server_id = 2
binlog_format=ROW
log_bin=/mysql/binlog/master-bin
auto-increment-increment = 2            #欄位變化增量值
auto-increment-offset = 2               #初始欄位ID為2
slave-skip-errors = all                 #忽略所有複製產生的錯誤
gtid_mode=ON
enforce-gtid-consistency=ON

STEP3:建立複製使用者,2個資料庫上都要建立

grant replication slave on *.* to 'rep'@'%' identified by '123';

STEP4:將hosta的資料拷貝到hostb,並應用

[root@hostb ~]# mysqldump -uroot -p123456 -h 192.168.10.11 --single-transaction --all-databases --master-data=2  > hosta.sql
[root@hostb ~]# mysql -uroot -p123456 < hosta.sql

STEP5:hostb上開啟複製,以下指令碼在hostb上執行

-- 配置複製
mysql> CHANGE MASTER TO
     ->   master_host='192.168.10.11',
     ->   master_port=3306,
     ->   master_user='rep',
     ->   master_password='123',
     ->   MASTER_AUTO_POSITION = 1;
Query OK, 0 rows affected, 2 warnings (0.01 sec)

-- 開啟複製
mysql> start slave;
 Query OK, 0 rows affected (0.00 sec)


-- 檢視複製狀態
mysql> show slave status \G
 *************************** 1. row ***************************
                Slave_IO_State: Waiting for master to send event
                   Master_Host: 192.168.10.11
                   Master_User: rep
                   Master_Port: 3306
                 Connect_Retry: 60
               Master_Log_File: master-bin.000001
           Read_Master_Log_Pos: 322
                Relay_Log_File: hostb-relay-bin.000002
                 Relay_Log_Pos: 417
         Relay_Master_Log_File: master-bin.000001
              Slave_IO_Running: Yes

             Slave_SQL_Running: Yes

STEP6:hosta上開啟複製,以下指令碼在hosta上執行

mysql> CHANGE MASTER TO
    ->    master_host='192.168.10.12',
    ->    master_port=3306,
    ->    master_user='rep',
    ->    master_password='123',
    ->    MASTER_AUTO_POSITION = 1;
Query OK, 0 rows affected, 2 warnings (0.01 sec)

mysql> start slave;
Query OK, 0 rows affected (0.01 sec)

mysql> show slave status \G;
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.168.10.12
                  Master_User: rep
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: master-bin.000001
          Read_Master_Log_Pos: 154
               Relay_Log_File: hosta-relay-bin.000002
                Relay_Log_Pos: 369
        Relay_Master_Log_File: master-bin.000001
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes

STEP7:測試雙主複製

在hosta上建立資料庫testdb,到hostb伺服器上檢視資料庫是否已經建立

-- hosta上建立資料庫
create database testdb;


--hostb上檢視資料庫,發現已經建立
mysql> show databases;
 +--------------------+
 | Database           |
 +--------------------+
 | information_schema |
 | db1                |
 | lijiamandb         |
 | mysql              |
 | performance_schema |
 | sbtest             |
 | sys                |
 | testdb             |
 +--------------------+
 8 rows in set (0.01 sec)

在hostb的testdb資料庫上建立表t1,並插入資料,到hosta上檢視是否複製過來

-- 在hostb上建立表並插入資料
mysql> use testdb
Database changed
mysql> create table t1(id int,name varchar(20));
Query OK, 0 rows affected (0.01 sec)

mysql> insert into t1 values(1,'a');
Query OK, 1 row affected (0.01 sec)


-- 在hosta上檢視資料,資料已經過來
mysql> select * from testdb.t1;
+------+------+
| id   | name |
+------+------+
|    1 | a    |
+------+------+
1 row in set (0.00 sec)

到這,雙主複製已經搭建完成,接下來安裝配置keepalived。


(3.2)安裝配置keepalived

(3.2.1)keepalived的安裝與管理

keepalived可以使用原始碼安裝,也可以使用yum線上安裝,這裡直接使用yum線上安裝:

[root@hosta data]# yum install -y keepalived

使用如下命令檢視安裝路徑:

[root@hosta data]# rpm -ql keepalived
/etc/keepalived
/etc/keepalived/keepalived.conf
/etc/sysconfig/keepalived
/usr/bin/genhash
/usr/lib/systemd/system/keepalived.service
/usr/libexec/keepalived
/usr/sbin/keepalived
/usr/share/doc/keepalived-1.3.5
… 略


使用如下命令管理keepalived

# 開啟keepalived
systemctl start keepalived 或者 service keepalived start 

# 關閉keepalived
systemctl stop keepalived 或者 service keepalived stop 

# 檢視keepalived執行狀態
systemctl status keepalived 或者 service keepalived status

# 重新啟動keepalived
systemctl restart keepalived 或者 service keepalived restart


(3.2.2)keepalived的配置

keepalived的配置檔案為:/etc/keepalived/keepalived.conf,我的配置檔案如下:

【hosta主機的配置檔案】

[root@hosta keepalived]# cat keepalived.conf
! Configuration File for keepalived
       
global_defs {
notification_email {
ops@wangshibo.cn
tech@wangshibo.cn
}
       
notification_email_from ops@wangshibo.cn
smtp_server 127.0.0.1 
smtp_connect_timeout 30
router_id MASTER-HA
}
       
vrrp_script chk_mysql_port {       #檢測mysql服務是否在執行。有很多方式,比如程式,用指令碼檢測等等
    script "/mysql/chk_mysql.sh"   #這裡通過指令碼監測
    interval 2                     #指令碼執行間隔,每2s檢測一次
    weight –5                      #指令碼結果導致的優先順序變更,檢測失敗(指令碼返回非0)則優先順序 -5
    fall 2                         #檢測連續2次失敗才算確定是真失敗。會用weight減少優先順序(1-255之間)
    rise 1                         #檢測1次成功就算成功。但不修改優先順序
}
       
vrrp_instance VI_1 {
    state BACKUP                  #這裡所有節點都定義為BACKUP
    interface ens34               #指定虛擬ip的網路卡介面
    mcast_src_ip 192.168.10.11    #本地IP 
    virtual_router_id 51          #路由器標識,MASTER和BACKUP必須是一致的
    priority 101                  #定義優先順序,數字越大,優先順序越高,在同一個vrrp_instance下,MASTER的優先順序必須大於BACKUP的優先順序。 
    advert_int 1
    nopreempt                     #不搶佔模式,在優先順序高的機器上設定即可,優先順序低的機器可不設定         
    authentication {   
        auth_type PASS 
        auth_pass 1111     
    }
    virtual_ipaddress {    
        192.168.10.10             #虛擬IP 
    }
      
track_script {               
   chk_mysql_port             
}
}


【hostb主機的配置檔案】

[root@hostb keepalived]# cat keepalived.conf
! Configuration File for keepalived
       
global_defs {
notification_email {
ops@wangshibo.cn
tech@wangshibo.cn
}
       
notification_email_from ops@wangshibo.cn
smtp_server 127.0.0.1 
smtp_connect_timeout 30
router_id MASTER-HA
}
       
vrrp_script chk_mysql_port {
    script "/mysql/chk_mysql.sh"
    interval 2            
    weight -5                 
    fall 2                 
    rise 1               
}
       
vrrp_instance VI_1 {
    state BACKUP
    interface ens34 
    mcast_src_ip 192.168.10.12
    virtual_router_id 51    
    priority 99          
    advert_int 1         
    authentication {   
        auth_type PASS 
        auth_pass 1111     
    }
    virtual_ipaddress {    
        192.168.10.10
    }
      
track_script {               
   chk_mysql_port             
}
}

需要特別注意:nopreempt這個引數只能用於state為BACKUP的情況,所以在配置的時候要把master和backup的state都設定成BACKUP,這樣才會實現keepalived的非搶佔模式!


在配置完成之後,啟動MySQL資料庫和keepalive,需要注意,先啟動MySQL,再啟動keepalive,因為keepalive啟動後會檢測MySQL的執行狀態,如果MySQL執行異常,keepalive會自動關閉。


(3.3)高可用測試

時間軸

(時間遞增)

hosta操作hostb操作
時間1

# 資料庫執行正常
[root@hosta ~]# service mysqld status
  SUCCESS! MySQL running (8530)

#keepalived執行正常
[root@hosta ~]# service keepalived status
    Active: active (running) since Mon 2020-08-03 22:29:10 CST; 1min 32s ago

# 此時虛擬IP在hosta上
[root@hosta ~]# ip addr
3: ens34: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
     link/ether 00:0c:29:fb:62:0a brd ff:ff:ff:ff:ff:ff
     inet 192.168.10.11/24 brd 192.168.10.255 scope global ens34
        valid_lft forever preferred_lft forever
     inet 192.168.10.10/32 scope global ens34
        valid_lft forever preferred_lft forever
     inet6 fe80::20c:29ff:fefb:620a/64 scope link
        valid_lft forever preferred_lft forever

# 資料庫執行正常

[root@hostb ~]# service mysqld status
  SUCCESS! MySQL running (8022)
 

#keepalived執行正常

[root@hostb ~]# service keepalived status
    Active: active (running) since Mon 2020-08-03 22:31:09 CST; 8s ago

[root@hostb ~]# ip addr
3: ens34: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
     link/ether 00:0c:29:92:3d:5a brd ff:ff:ff:ff:ff:ff
     inet 192.168.10.12/24 brd 192.168.10.255 scope global ens34
        valid_lft forever preferred_lft forever
     inet6 fe80::20c:29ff:fe92:3d5a/64 scope link
        valid_lft forever preferred_lft forever

時間2

# 通過VIP訪問資料庫,訪問到的都是hosta上的例項
[root@hosta keepalived]# mysql -uroot -p123456 -h192.168.10.10 -e "select @@hostname"
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| hosta      |
+------------+

# 通過VIP訪問資料庫,訪問到的都是hosta上的例項
[root@hostb ~]# mysql -uroot -p123456 -h192.168.10.10 -e "select @@hostname"
  mysql: [Warning] Using a password on the command line interface can be insecure.
  +------------+
  | @@hostname |
  +------------+
  | hosta      |
  +------------+


時間3

# 主節點關閉MySQL
[root@hosta keepalived]# service mysqld stop
Shutting down MySQL............ SUCCESS!


時間4

# keepalived檢測到MySQL關閉後,會自動關閉
[root@hosta keepalived]# service keepalived status
    Active: inactive (dead)

# 此時VIP已經不存在
[root@hosta keepalived]# ip addr
3: ens34: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
     link/ether 00:0c:29:fb:62:0a brd ff:ff:ff:ff:ff:ff
     inet 192.168.10.11/24 brd 192.168.10.255 scope global ens34
        valid_lft forever preferred_lft forever
     inet6 fe80::20c:29ff:fefb:620a/64 scope link
        valid_lft forever preferred_lft forever

# hostb上的keepalived執行正常
[root@hostb ~]# service keepalived status
    Active: active (running) since Mon 2020-08-03 22:31:09 CST; 6min ago

# 發現VIP已經切換到了hostb
[root@hostb ~]#  ip addr
3: ens34: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
     link/ether 00:0c:29:92:3d:5a brd ff:ff:ff:ff:ff:ff
     inet 192.168.10.12/24 brd 192.168.10.255 scope global ens34
        valid_lft forever preferred_lft forever
     inet 192.168.10.10/32 scope global ens34
        valid_lft forever preferred_lft forever
     inet6 fe80::20c:29ff:fe92:3d5a/64 scope link
        valid_lft forever preferred_lft forever

時間5

# 通過VIP訪問資料庫,訪問到的都是hosta上的例項
[root@hosta keepalived]# mysql -uroot -p123456 -h192.168.10.10 -e "select @@hostname"
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| hostb      |
+------------+

# 通過VIP訪問資料庫,訪問到的都是hosta上的例項
[root@hostb ~]# mysql -uroot -p123456 -h192.168.10.10 -e "select @@hostname"
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| hostb      |
+------------+

時間6

# 重啟MySQL
[root@hosta keepalived]# service mysqld start
Starting MySQL. SUCCESS!

# 重啟keepalived
[root@hosta keepalived]# service keepalived start
Redirecting to /bin/systemctl start keepalived.service
[root@hosta keepalived]#
[root@hosta keepalived]# service keepalived status
Redirecting to /bin/systemctl status keepalived.service
    Active: active (running) since Mon 2020-08-03 22:40:08 CST; 6s ago

# 因為使用了非搶佔模式,VIP不會漂回來
# 如果要把VIP漂回來,可以關閉hostb上的資料庫或者直接關閉hostb伺服器

[root@hosta keepalived]# ip addr
3: ens34: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
     link/ether 00:0c:29:fb:62:0a brd ff:ff:ff:ff:ff:ff
     inet 192.168.10.11/24 brd 192.168.10.255 scope global ens34
        valid_lft forever preferred_lft forever
     inet6 fe80::20c:29ff:fefb:620a/64 scope link
        valid_lft forever preferred_lft forever



【完】

相關文章