Mysql MHA部署-06手動切換

Mysql MHA部署-06手動切換

一 Mysql MHA部署-01介紹

二 Mysql MHA部署-02主從複製

三 Mysql MHA部署-03MHA軟體安裝

四 Mysql MHA部署-04MHA配置

五 Mysql MHA部署-05故障轉移

六 Mysql MHA部署-06手動切換

七 Mysql MHA部署-07常見問題

架構說明：

參考： http://www.zhaibibei.cn/mysql/mha/

1 檢查從庫現有狀態(188,223)

---show slave status\G

2 檢視管理節點日誌(222)

通過如下命令事實檢視MHA當前狀態

tail -f /etc/mha/manager/mha.log

3 關閉MHA的管理程式(222)

[root@rac4 ~]# masterha_stop -conf=/etc/mha/mha.conf 

MHA Manager is not running on mha(2:NOT_RUNNING).

4.手動切換

相關命令如下：

masterha_master_switch   -master_state=alive –orig_master_is_new_slave –conf=/etc/mha/mha.conf

-master_state=alive 代表告訴MHA原master還是存活的，不需要將其從配置檔案刪除

–orig_master_is_new_slave 引數代表原master會自動同步新的master

--還有一些其他的引數如下

-running_updates_limit 如果主庫的寫操作時間超過了該引數，則退出切換

–interactive=0 代表直接確認，不需要輸入YES

切換過程如下：

[root@rac4 ~]# masterha_master_switch -master_state=alive -orig_master_is_new_slave -conf=/etc/mha/mha.conf

Sat Mar 14 22:17:37 2020 - [info] MHA::MasterRotate version 0.56.

Sat Mar 14 22:17:37 2020 - [info] Starting online master switch..

Sat Mar 14 22:17:37 2020 - [info] 

Sat Mar 14 22:17:37 2020 - [info] * Phase 1: Configuration Check Phase..

Sat Mar 14 22:17:37 2020 - [info] 

Sat Mar 14 22:17:37 2020 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.

Sat Mar 14 22:17:37 2020 - [info] Reading application default configuration from /etc/mha/mha.conf..

Sat Mar 14 22:17:37 2020 - [info] Reading server configuration from /etc/mha/mha.conf..

Sat Mar 14 22:17:38 2020 - [info] GTID failover mode = 1

Sat Mar 14 22:17:38 2020 - [info] Current Alive Master: rac1(192.168.2.187:3306)

Sat Mar 14 22:17:38 2020 - [info] Alive Slaves:

Sat Mar 14 22:17:38 2020 - [info]   rac2(192.168.2.188:3306)  Version=5.7.28-log (oldest major version between slaves) log-bin:enabled

Sat Mar 14 22:17:38 2020 - [info]     GTID ON

Sat Mar 14 22:17:38 2020 - [info]     Replicating from 192.168.2.187(192.168.2.187:3306)

Sat Mar 14 22:17:38 2020 - [info]     Primary candidate for the new Master (candidate_master is set)

Sat Mar 14 22:17:38 2020 - [info]   rac3(192.168.2.223:3306)  Version=5.7.28-log (oldest major version between slaves) log-bin:enabled

Sat Mar 14 22:17:38 2020 - [info]     GTID ON

Sat Mar 14 22:17:38 2020 - [info]     Replicating from 192.168.2.187(192.168.2.187:3306)

Sat Mar 14 22:17:38 2020 - [info]     Not candidate for the new Master (no_master is set)

It is better to execute FLUSH NO_WRITE_TO_BINLOG TABLES on the master before switching. Is it ok to execute on rac1(192.168.2.187:3306)? (YES/no): YES

Sat Mar 14 22:19:01 2020 - [info] Executing FLUSH NO_WRITE_TO_BINLOG TABLES. This may take long time..

Sat Mar 14 22:19:01 2020 - [info]  ok.

Sat Mar 14 22:19:01 2020 - [info] Checking MHA is not monitoring or doing failover..

Sat Mar 14 22:19:01 2020 - [info] Checking replication health on rac2..

Sat Mar 14 22:19:01 2020 - [info]  ok.

Sat Mar 14 22:19:01 2020 - [info] Checking replication health on rac3..

Sat Mar 14 22:19:01 2020 - [info]  ok.

Sat Mar 14 22:19:01 2020 - [info] Searching new master from slaves..

Sat Mar 14 22:19:01 2020 - [info]  Candidate masters from the configuration file:

Sat Mar 14 22:19:01 2020 - [info]   rac1(192.168.2.187:3306)  Version=5.7.28-log log-bin:enabled

Sat Mar 14 22:19:01 2020 - [info]     GTID ON

Sat Mar 14 22:19:01 2020 - [info]   rac2(192.168.2.188:3306)  Version=5.7.28-log (oldest major version between slaves) log-bin:enabled

Sat Mar 14 22:19:01 2020 - [info]     GTID ON

Sat Mar 14 22:19:01 2020 - [info]     Replicating from 192.168.2.187(192.168.2.187:3306)

Sat Mar 14 22:19:01 2020 - [info]     Primary candidate for the new Master (candidate_master is set)

Sat Mar 14 22:19:01 2020 - [info]  Non-candidate masters:

Sat Mar 14 22:19:01 2020 - [info]   rac3(192.168.2.223:3306)  Version=5.7.28-log (oldest major version between slaves) log-bin:enabled

Sat Mar 14 22:19:01 2020 - [info]     GTID ON

Sat Mar 14 22:19:01 2020 - [info]     Replicating from 192.168.2.187(192.168.2.187:3306)

Sat Mar 14 22:19:01 2020 - [info]     Not candidate for the new Master (no_master is set)

Sat Mar 14 22:19:01 2020 - [info]  Searching from candidate_master slaves which have received the latest relay log events..

Sat Mar 14 22:19:01 2020 - [info] 

From:

rac1(192.168.2.187:3306) (current master)

 +--rac2(192.168.2.188:3306)

 +--rac3(192.168.2.223:3306)

To:

rac2(192.168.2.188:3306) (new master)

 +--rac3(192.168.2.223:3306)

 +--rac1(192.168.2.187:3306)

Starting master switch from rac1(192.168.2.187:3306) to rac2(192.168.2.188:3306)? (yes/NO): yes

Sat Mar 14 22:19:32 2020 - [info] Checking whether rac2(192.168.2.188:3306) is ok for the new master..

Sat Mar 14 22:19:32 2020 - [info]  ok.

Sat Mar 14 22:19:32 2020 - [info] rac1(192.168.2.187:3306): SHOW SLAVE STATUS returned empty result. To check replication filtering rules, temporarily executing CHANGE MASTER to a dummy host.

Sat Mar 14 22:19:32 2020 - [info] rac1(192.168.2.187:3306): Resetting slave pointing to the dummy host.

Sat Mar 14 22:19:32 2020 - [info] ** Phase 1: Configuration Check Phase completed.

Sat Mar 14 22:19:32 2020 - [info] 

Sat Mar 14 22:19:32 2020 - [info] * Phase 2: Rejecting updates Phase..

Sat Mar 14 22:19:32 2020 - [info] 

Sat Mar 14 22:19:32 2020 - [info] Executing master ip online change script to disable write on the current master:

Sat Mar 14 22:19:32 2020 - [info]   /etc/mha/script/master_ip_online_change --command=stop --orig_master_host=rac1 --orig_master_ip=192.168.2.187 --orig_master_port=3306 --orig_master_user='monitor' --orig_master_password='123456' --new_master_host=rac2 --new_master_ip=192.168.2.188 --new_master_port=3306 --new_master_user='monitor' --new_master_password='123456' --orig_master_ssh_user=root --new_master_ssh_user=root   --orig_master_is_new_slave

Sat Mar 14 22:19:32 2020 756531 Set read_only on the new master.. ok.

Sat Mar 14 22:19:32 2020 761639 Set read_only=1 on the orig master.. ok.

Sat Mar 14 22:19:32 2020 763740 Killing all application threads..

Sat Mar 14 22:19:32 2020 763785 done.

Disabling the VIP an old master: rac1 

eth0:2: ERROR while getting interface flags: No such device

Sat Mar 14 22:19:33 2020 - [info]  ok.

Sat Mar 14 22:19:33 2020 - [info] Locking all tables on the orig master to reject updates from everybody (including root):

Sat Mar 14 22:19:33 2020 - [info] Executing FLUSH TABLES WITH READ LOCK..

Sat Mar 14 22:19:33 2020 - [info]  ok.

Sat Mar 14 22:19:33 2020 - [info] Orig master binlog:pos is mysql-bin.000009:194.

Sat Mar 14 22:19:33 2020 - [info]  Waiting to execute all relay logs on rac2(192.168.2.188:3306)..

Sat Mar 14 22:19:33 2020 - [info]  master_pos_wait(mysql-bin.000009:194) completed on rac2(192.168.2.188:3306). Executed 0 events.

Sat Mar 14 22:19:33 2020 - [info]   done.

Sat Mar 14 22:19:33 2020 - [info] Getting new master's binlog name and position..

Sat Mar 14 22:19:33 2020 - [info]  mysql-bin.000006:3938

Sat Mar 14 22:19:33 2020 - [info]  All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='rac2 or 192.168.2.188', MASTER_PORT=3306, MASTER_AUTO_POSITION=1, MASTER_USER='repl', MASTER_PASSWORD='xxx';

Sat Mar 14 22:19:33 2020 - [info] Executing master ip online change script to allow write on the new master:

Sat Mar 14 22:19:33 2020 - [info]   /etc/mha/script/master_ip_online_change --command=start --orig_master_host=rac1 --orig_master_ip=192.168.2.187 --orig_master_port=3306 --orig_master_user='monitor' --orig_master_password='123456' --new_master_host=rac2 --new_master_ip=192.168.2.188 --new_master_port=3306 --new_master_user='monitor' --new_master_password='123456' --orig_master_ssh_user=root --new_master_ssh_user=root   --orig_master_is_new_slave

Sat Mar 14 22:19:33 2020 227880 Set read_only=0 on the new master.

Enabling the VIP 192.168.2.189 on the new master: rac2 

SIOCSIFADDR: No such device

eth0:2: ERROR while getting interface flags: No such device

SIOCSIFNETMASK: No such device

arping: Device eth0 not available.

Sat Mar 14 22:19:33 2020 - [info]  ok.

Sat Mar 14 22:19:33 2020 - [info] 

Sat Mar 14 22:19:33 2020 - [info] * Switching slaves in parallel..

Sat Mar 14 22:19:33 2020 - [info] 

Sat Mar 14 22:19:33 2020 - [info] -- Slave switch on host rac3(192.168.2.223:3306) started, pid: 12424

Sat Mar 14 22:19:33 2020 - [info] 

Sat Mar 14 22:19:34 2020 - [info] Log messages from rac3 ...

Sat Mar 14 22:19:34 2020 - [info] 

Sat Mar 14 22:19:33 2020 - [info]  Waiting to execute all relay logs on rac3(192.168.2.223:3306)..

Sat Mar 14 22:19:33 2020 - [info]  master_pos_wait(mysql-bin.000009:194) completed on rac3(192.168.2.223:3306). Executed 0 events.

Sat Mar 14 22:19:33 2020 - [info]   done.

Sat Mar 14 22:19:33 2020 - [info]  Resetting slave rac3(192.168.2.223:3306) and starting replication from the new master rac2(192.168.2.188:3306)..

Sat Mar 14 22:19:33 2020 - [info]  Executed CHANGE MASTER.

Sat Mar 14 22:19:33 2020 - [info]  Slave started.

Sat Mar 14 22:19:34 2020 - [info] End of log messages from rac3 ...

Sat Mar 14 22:19:34 2020 - [info] 

Sat Mar 14 22:19:34 2020 - [info] -- Slave switch on host rac3(192.168.2.223:3306) succeeded.

Sat Mar 14 22:19:34 2020 - [info] Unlocking all tables on the orig master:

Sat Mar 14 22:19:34 2020 - [info] Executing UNLOCK TABLES..

Sat Mar 14 22:19:34 2020 - [info]  ok.

Sat Mar 14 22:19:34 2020 - [info] Starting orig master as a new slave..

Sat Mar 14 22:19:34 2020 - [info]  Resetting slave rac1(192.168.2.187:3306) and starting replication from the new master rac2(192.168.2.188:3306)..

Sat Mar 14 22:19:34 2020 - [info]  Executed CHANGE MASTER.

Sat Mar 14 22:19:34 2020 - [info]  Slave started.

Sat Mar 14 22:19:34 2020 - [info] All new slave servers switched successfully.

Sat Mar 14 22:19:34 2020 - [info] 

Sat Mar 14 22:19:34 2020 - [info] * Phase 5: New master cleanup phase..

Sat Mar 14 22:19:34 2020 - [info] 

Sat Mar 14 22:19:34 2020 - [info]  rac2: Resetting slave info succeeded.

Sat Mar 14 22:19:34 2020 - [info] Switching master to rac2(192.168.2.188:3306) completed successfully.

5.切換階段

執行master_ip_online_change時當前master不可寫

新的主庫設定只讀

舊的主庫設定只讀

禁用原主庫VIP

舊主庫設全域性鎖

獲取新主庫master資訊

新主庫設定VIP

新主庫取消只讀

非同步從庫重新同步至新主庫

原主庫釋放全域性鎖

原主庫同步至新主庫

6.檢視切換後狀態

---223

mysql> show slave status \G;

*************************** 1. row ***************************

               Slave_IO_State: Waiting for master to send event

                  Master_Host: 192.168.2.188

                  Master_User: repl

                  Master_Port: 3306

                Connect_Retry: 60

              Master_Log_File: mysql-bin.000006

          Read_Master_Log_Pos: 3938

               Relay_Log_File: mysql-relay.000005

                Relay_Log_Pos: 454

        Relay_Master_Log_File: mysql-bin.000006

             Slave_IO_Running: Yes

            Slave_SQL_Running: Yes

----187

mysql> show slave status \G;

*************************** 1. row ***************************

               Slave_IO_State: Waiting for master to send event

                  Master_Host: rac2

                  Master_User: repl

                  Master_Port: 3306

                Connect_Retry: 60

              Master_Log_File: mysql-bin.000006

          Read_Master_Log_Pos: 3938

               Relay_Log_File: mysql-relay.000007

                Relay_Log_Pos: 454

        Relay_Master_Log_File: mysql-bin.000006

             Slave_IO_Running: Yes

            Slave_SQL_Running: Yes

              Replicate_Do_DB: 

          Replicate_Ignore_DB: 

......

7.資料測試

---188：

mysql> use jumptest

mysql> insert into cjc01 values(3000);

Query OK, 1 row affected (0.02 sec)

mysql> select * from cjc01;

+------+

| id   |

+------+

|  100 |

|    3 |

| 3000 |

+------+

3 rows in set (0.00 sec)

---187：

mysql> select * from cjc01;

+------+

| id   |

+------+

|  100 |

|    3 |

| 3000 |

+------+

3 rows in set (0.01 sec)

---223：

mysql> select * from cjc01;

+------+

| id   |

+------+

|  100 |

|    3 |

| 3000 |

+------+

3 rows in set (0.01 sec)

歡迎關注我的微信公眾號"IT小Chen"，共同學習，共同成長！！！

來自 “ ITPUB部落格 ” ，連結：http://blog.itpub.net/29785807/viewspace-2680453/，如需轉載，請註明出處，否則將追究法律責任。