MHA配置搭建
1、一、MHA部署環境：
MHA_Manager： 192.168.1.100
Mysql_Master：10.20.20.12
Mysql_Slave-1：10.20.20.13
Mysql_Slave-2：10.20.20.14
軟體包下載地址：
訪問密碼 5ea4
MHA_Manager： 192.168.1.100
Mysql_Master：10.20.20.12
Mysql_Slave-1：10.20.20.13
Mysql_Slave-2：10.20.20.14
軟體包下載地址：
訪問密碼 5ea4
注意：關閉防火牆、selinux
二、準備階段：
step1：建立信任關係：
在Node：10.20.20.12上：
[root@mysqlmaster /]#ssh-keygen -t rsa
[root@mysqlmaster /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.12
[root@mysqlmaster /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.13
[root@mysqlmaster /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.14
在Node：10.20.20.13上：
[root@mysqlslave1 /]#ssh-keygen -t rsa
[root@mysqlslave1 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.12
[root@mysqlslave1 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.13
[root@mysqlslave1 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.14
在Node：10.20.20.14上：
[root@mysqlslave2 /]#ssh-keygen -t rsa
[root@mysqlslave2 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.12
[root@mysqlslave2 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.13
[root@mysqlslave2 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.14
step2：部署mysql服務(原始碼搭建)
1、安裝編譯程式碼需要的包
yum -y install make gcc-c++ cmake bison-devel ncurses-devel
2、下載MySQL 5.6.14
wget
tar xvf mysql-5.6.17.tar.gz
cd mysql-5.6.27
3、編譯安裝
cmake \
-DCMAKE_INSTALL_PREFIX=/usr/local/mysql \
-DMYSQL_DATADIR=/data/mysql \
-DSYSCONFDIR=/etc \
-DWITH_MYISAM_STORAGE_ENGINE=1 \
-DWITH_INNOBASE_STORAGE_ENGINE=1 \
-DWITH_MEMORY_STORAGE_ENGINE=1 \
-DWITH_READLINE=1 \
-DMYSQL_UNIX_ADDR=/usr/local/mysql/mysql.sock \
-DMYSQL_TCP_PORT=3306 \
-DENABLED_LOCAL_INFILE=1 \
-DWITH_PARTITION_STORAGE_ENGINE=1 \
-DEXTRA_CHARSETS=all \
-DDEFAULT_CHARSET=utf8 \
-DDEFAULT_COLLATION=utf8_general_ci
make && make install
配置MySQL
建立使用者組
groupadd mysql
useradd -g mysql mysql
修改/usr/local/mysql許可權
chown -R mysql:mysql /usr/local/mysql
chown -R mysql:mysql /data
配置my.cnf(詳見conf檔案)
以下是簡單的my.cnf配置
[root@mysqlmaster /]# cat /etc/my.cnf
[mysqld]
log-bin=mysql-bin
server-id=12
skip-name-resolve
[root@mysqlslave1 ~]# cat /etc/my.cnf
[mysqld]
log-bin=mysql-bin
server-id=13
read-only=1
relay_log_purge=0
skip-name-resolve
[root@mysqlslave2 .ssh]# cat /etc/my.cnf
[mysqld]
log-bin=mysql-bin
server-id=14
read-only=1
relay_log_purge=0
skip-name-resolve
初始化配置
cd /usr/local/mysql
進入安裝路徑，執行初始化配置指令碼，建立系統自帶的資料庫和表
scripts/mysql_install_db --basedir=/usr/local/mysql --datadir=/data/mysql --user=mysql
啟動MySQL
cp support-files/mysql.server /etc/init.d/mysql
chkconfig mysql on
service mysql start
配置環境變數
vi /etc/profile
export PATH=/usr/local/mysql/bin:$PATH
生效環境變數
source /etc/profile
執行下面的命令修改root密碼
mysql -uroot
mysql> SET PASSWORD = PASSWORD('123456');
若要設定root使用者可以遠端訪問，執行
mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'172.16.%' IDENTIFIED BY 'password' WITH GRANT OPTION;
三、建立MySQL主從複製
step1：建立資料庫管理賬號和複製賬號（主庫、備庫 都執行）
grant all privileges on *.* to 'root'@'10.20.20.%' identified by '123456';
grant Replication slave,super on *.* to 'repler'@'10.20.20.%' identified by '123.com';
grant all privileges on *.* to 'root'@'192.168.1.%' identified by '123456';
flush privileges;
step2：建立mysql之間的主從複製
登入主伺服器的mysql，查詢master的狀態
mysql>show master status;
+------------------+----------+--------------+------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+------------------+----------+--------------+------------------+
| mysql-bin.000001 | 308 | | |
+------------------+----------+--------------+------------------+
配置從伺服器Slave
mysql>change master to master_host='10.20.20.12',master_user='repler',master_password='123.com',master_log_file='mysql-bin.000001',master_log_pos=308;
四、建立軟連結和設定從庫為read only
step1：ln -s /usr/local/mysql/bin/* /usr/local/bin/
注意：需要做這個軟連線，否則再執行masterha_check_repl校驗的時候，會報Can't exec "mysqlbinlog": 沒有那個檔案或目錄 的錯誤。
step2：所有從伺服器上my.cnf裡面新增：read-only=1和relay_log_purge=0
五、安裝MHA_Manager
1. 安裝epel源
rpm -Uvh
yum --disablerepo=epel -y update ca-certificates
2. 安裝依賴包
yum -y install perl-DBD-MySQL perl-Config-Tiny perl-Log-Dispatch perl-Params-Validate perl-TimeDate perl-ExtUtils-CBuilder perl-ExtUtils-MakeMaker perl-CPAN perl-Parallel-ForkManager perl-Log-Message
yum -y install perl-NTLM.noarch --如果要發郵件send_report，需要安裝這個包，否則就不發郵件
3、 安裝MHA_Node(MHA_Manager需要)
tar -zxvf mha4mysql-manager-0.57.tar.gz
cd mha4mysql-manager-0.57
perl Makefile.PL
make
make install
tar -zxf Test-Simple-1.001014.tar.gz
cd Test-Simple-1.001014
perl Makefile.PL
make
make install
tar -zxf CPAN-Perl-Releases-2.48.tar.gz
cd CPAN-Perl-Releases-2.48
perl Makefile.PL
make
make install
tar -zxf Log-Dispatch-Perl-0.04.tar.gz
cd Log-Dispatch-Perl-0.04
perl Makefile.PL
make
make install
mkdir /etc/masterha/
mkdir /etc/masterha/scripts
mkdir /masterha/app1 -p
六、安裝MHA_Node(在所有的Mysql伺服器上)
1. 安裝epel源
rpm -Uvh
yum --disablerepo=epel -y update ca-certificates
2. 安裝依賴包
yum -y install perl-DBD-MySQL perl-Config-Tiny perl-Log-Dispatch perl-Params-Validate perl-TimeDate perl-ExtUtils-CBuilder perl-ExtUtils-MakeMaker perl-CPAN perl-Parallel-ForkManager
3、 安裝MHA_Node
tar -zxf mha4mysql-node-0.57.tar.gz
cd mha4mysql-node-0.57
perl Makefile.PL
make
make install
mkdir /masterha/app1 -p
七、配置MHA_Manager(192.168.1.100)：
vim /etc/masterha/app1.cnf
[server default]
#workdir on the management server
manager_workdir=/masterha/app1/
manager_log=/masterha/app1/manager.log
#workdir on the node for mysql server
remote_workdir=/masterha/app1
master_binlog_dir=/data/mysql
check_repl_delay=0
#mysql user and password
user=root
password=123456
#replication_user
repl_user=repler
repl_password=123.com
#checking master every second
ping_interval=1
#ssh user
ssh_user=root
#檢測master的可用性
secondary_check_script=masterha_secondary_check -s 10.20.20.13 -s 10.20.20.14
#自動故障VIP切換呼叫指令碼
master_ip_failover_script=/etc/masterha/scripts/master_ip_failover
#手動故障切換呼叫指令碼
master_ip_online_change_script=/etc/masterha/scripts/master_ip_online_change
#傳送郵件指令碼
report_script=/mha/scripts/send_report
[server1]
hostname=10.20.20.12
candidate_master=1
[server2]
hostname=10.20.20.13
candidate_master=1
[server3]
hostname=10.20.20.14
no_master=1
2、自動故障VIP切換呼叫指令碼
vi /etc/masterha/scripts/master_ip_failover （在MHA_manager 節點上）
#!/usr/bin/env perl
use strict;
use warnings FATAL => 'all';
use Getopt::Long;
my (
$command, $ssh_user, $orig_master_host, $orig_master_ip,
$orig_master_port, $new_master_host, $new_master_ip, $new_master_port
);
my $vip = '192.168.1.100'; #Virtual IP
my $gateway = '192.168.1.1'; #Gateway IP
my $interface = 'eth0'; #bind to interface
my $key = "1";
my $ssh_stop_vip = "/sbin/ifconfig $interface:$key down";
GetOptions(
'command=s' => \$command,
'ssh_user=s' => \$ssh_user,
'orig_master_host=s' => \$orig_master_host,
'orig_master_ip=s' => \$orig_master_ip,
'orig_master_port=i' => \$orig_master_port,
'new_master_host=s' => \$new_master_host,
'new_master_ip=s' => \$new_master_ip,
'new_master_port=i' => \$new_master_port,
);
exit &main();
sub main {
print "\n\nIN SCRIPT TEST====$ssh_stop_vip==$ssh_start_vip===\n\n";
if ( $command eq "stop" || $command eq "stopssh" ) {
# $orig_master_host, $orig_master_ip, $orig_master_port are passed.
# If you manage master ip address at global catalog database,
# invalidate orig_master_ip here.
my $exit_code = 1;
eval {
print "Disabling the VIP on old master: $orig_master_host \n";
&stop_vip();
$exit_code = 0;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "start" ) {
# all arguments are passed.
# If you manage master ip address at global catalog database,
# activate new_master_ip here.
# You can also grant write access (create user, set read_only=0, etc) here.
my $exit_code = 10;
eval {
print "Enabling the VIP - $vip on the new master - $new_master_host \n";
&start_vip();
$exit_code = 0;
};
if ($@) {
warn $@;
exit $exit_code;
}
elsif ( $command eq "status" ) {
print "Checking the Status of the script.. OK \n";
`ssh $ssh_user\@$orig_master_host \" $ssh_start_vip \"`;
exit 0;
}
else {
&usage();
exit 1;
}
}
# A simple system call that enable the VIP on the new master
sub start_vip() {
`ssh $ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}
# A simple system call that disable the VIP on the old_master
sub stop_vip() {
`ssh $ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}
sub usage {
print
"Usage: master_ip_failover --command=start|stop|stopssh|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n";
}
3、手動故障切換呼叫指令碼 （在MHA_manager）
vi /etc/masterha/scripts/master_ip_online_change
#!/usr/bin/env perl
# Copyright (C) 2011 DeNA Co.,Ltd.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
## Note: This is a sample script and is not complete. Modify the script based on your environment.
use strict;
use warnings FATAL => 'all';
use Getopt::Long;
use MHA::DBHelper;
use MHA::NodeUtil;
use Time::HiRes qw( sleep gettimeofday tv_interval );
use Data::Dumper;
my $_tstart;
my $_running_interval = 0.1;
my (
$command, $orig_master_host, $orig_master_ip,
$orig_master_port, $orig_master_user, $orig_master_ssh_user,
$new_master_host, $new_master_ip, $new_master_port,
$new_master_user, $new_master_ssh_user, $orig_master_is_new_slave,
$orig_master_password, $new_master_password,
);
my $vip = '192.168.1.100'; ###Virtual IP
my $gateway = '192.168.1.1'; ###Gateway IP
my $interface = 'eth0'; ###interface
my $key = "1";
my $ssh_start_vip = "/sbin/ifconfig $interface:$key $vip/24";
my $flush_arp = "/sbin/arping -I $interface -c 2 -s $vip $gateway";
my $ssh_stop_vip = "/sbin/ifconfig $interface:$key down";
my $ssh_user = "root";
GetOptions(
'command=s' => \$command,
'ssh_user=s' => \$ssh_user,
'orig_master_host=s' => \$orig_master_host,
'orig_master_ip=s' => \$orig_master_ip,
'orig_master_port=i' => \$orig_master_port,
'orig_master_user=s' => \$orig_master_user,
'orig_master_password=s' => \$orig_master_password,
'new_master_host=s' => \$new_master_host,
'new_master_ip=s' => \$new_master_ip,
'new_master_port=i' => \$new_master_port,
'new_master_user=s' => \$new_master_user,
'new_master_password=s' => \$new_master_password,
'orig_master_ssh_user=s' => \$orig_master_ssh_user,
'new_master_ssh_user=s' => \$new_master_ssh_user,
'orig_master_is_new_slave' => \$orig_master_is_new_slave,
);
exit &main();
sub current_time_us {
my ( $sec, $microsec ) = gettimeofday();
my $curdate = localtime($sec);
return $curdate . " " . sprintf( "%06d", $microsec );
}
sub sleep_until {
my $elapsed = tv_interval($_tstart);
if ( $_running_interval > $elapsed ) {
sleep( $_running_interval - $elapsed );
}
}
sub get_threads_util {
my $dbh = shift;
my $my_connection_id = shift;
my $running_time_threshold = shift;
my $type = shift;
$running_time_threshold = 0 unless ($running_time_threshold);
$type = 0 unless ($type);
my @threads;
my $sth = $dbh->prepare("SHOW PROCESSLIST");
$sth->execute();
while ( my $ref = $sth->fetchrow_hashref() ) {
my $id = $ref->{Id};
my $user = $ref->{User};
my $host = $ref->{Host};
my $command = $ref->{Command};
my $state = $ref->{State};
my $query_time = $ref->{Time};
my $info = $ref->{Info};
$info =~ s/^\s*(.*?)\s*$/$1/ if defined($info);
next if ( $my_connection_id == $id );
next if ( defined($query_time) && $query_time < $running_time_threshold );
next if ( defined($command) && $command eq "Binlog Dump" );
next if ( defined($user) && $user eq "system user" );
next
if ( defined($command)
&& $command eq "Sleep"
&& defined($query_time)
&& $query_time >= 1 );
if ( $type >= 1 ) {
next if ( defined($command) && $command eq "Sleep" );
next if ( defined($command) && $command eq "Connect" );
}
if ( $type >= 2 ) {
next if ( defined($info) && $info =~ m/^select/i );
next if ( defined($info) && $info =~ m/^show/i );
}
push @threads, $ref;
}
return @threads;
}
sub main {
if ( $command eq "stop" ) {
## Gracefully killing connections on the current master
# 1. Set read_only= 1 on the new master
# 2. DROP USER so that no app user can establish new connections
# 3. Set read_only= 1 on the current master
# 4. Kill current queries
# * Any database access failure will result in script die.
my $exit_code = 1;
eval {
## Setting read_only=1 on the new master (to avoid accident)
my $new_master_handler = new MHA::DBHelper();
# args: hostname, port, user, password, raise_error(die_on_error)_or_not
$new_master_handler->connect( $new_master_ip, $new_master_port,
$new_master_user, $new_master_password, 1 );
print current_time_us() . " Set read_only on the new master.. ";
$new_master_handler->enable_read_only();
if ( $new_master_handler->is_read_only() ) {
print "ok.\n";
}
else {
die "Failed!\n";
}
$new_master_handler->disconnect();
# Connecting to the orig master, die if any database error happens
my $orig_master_handler = new MHA::DBHelper();
$orig_master_handler->connect( $orig_master_ip, $orig_master_port,
$orig_master_user, $orig_master_password, 1 );
## Drop application user so that nobody can connect. Disabling per-session binlog beforehand
#$orig_master_handler->disable_log_bin_local();
#print current_time_us() . " Drpping app user on the orig master..\n";
#FIXME_xxx_drop_app_user($orig_master_handler);
## Waiting for N * 100 milliseconds so that current connections can exit
my $time_until_read_only = 15;
$_tstart = [gettimeofday];
my @threads = get_threads_util( $orig_master_handler->{dbh},
$orig_master_handler->{connection_id} );
while ( $time_until_read_only > 0 && $#threads >= 0 ) {
if ( $time_until_read_only % 5 == 0 ) {
printf
"%s Waiting all running %d threads are disconnected.. (max %d milliseconds)\n",
current_time_us(), $#threads + 1, $time_until_read_only * 100;
if ( $#threads < 5 ) {
print Data::Dumper->new( [$_] )->Indent(0)->Terse(1)->Dump . "\n"
foreach (@threads);
}
}
sleep_until();
$_tstart = [gettimeofday];
$time_until_read_only--;
@threads = get_threads_util( $orig_master_handler->{dbh},
$orig_master_handler->{connection_id} );
}
## Setting read_only=1 on the current master so that nobody(except SUPER) can write
print current_time_us() . " Set read_only=1 on the orig master.. ";
$orig_master_handler->enable_read_only();
if ( $orig_master_handler->is_read_only() ) {
print "ok.\n";
}
else {
die "Failed!\n";
}
## Waiting for M * 100 milliseconds so that current update queries can complete
my $time_until_kill_threads = 5;
@threads = get_threads_util( $orig_master_handler->{dbh},
$orig_master_handler->{connection_id} );
while ( $time_until_kill_threads > 0 && $#threads >= 0 ) {
if ( $time_until_kill_threads % 5 == 0 ) {
printf
"%s Waiting all running %d queries are disconnected.. (max %d milliseconds)\n",
current_time_us(), $#threads + 1, $time_until_kill_threads * 100;
if ( $#threads < 5 ) {
print Data::Dumper->new( [$_] )->Indent(0)->Terse(1)->Dump . "\n"
foreach (@threads);
}
}
sleep_until();
$_tstart = [gettimeofday];
$time_until_kill_threads--;
@threads = get_threads_util( $orig_master_handler->{dbh},
$orig_master_handler->{connection_id} );
&stop_vip();
## Terminating all threads
print current_time_us() . " Killing all application threads..\n";
$orig_master_handler->kill_threads(@threads) if ( $#threads >= 0 );
print current_time_us() . " done.\n";
#$orig_master_handler->enable_log_bin_local();
$orig_master_handler->disconnect();
## After finishing the script, MHA executes FLUSH TABLES WITH READ LOCK
$exit_code = 0;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "start" ) {
## Activating master ip on the new master
# 1. Create app user with write privileges
# 2. Moving backup script if needed
# 3. Register new master's ip to the catalog database
# If exit code is 0 or 10, MHA does not abort
my $exit_code = 10;
eval {
my $new_master_handler = new MHA::DBHelper();
# args: hostname, port, user, password, raise_error_or_not
$new_master_handler->connect( $new_master_ip, $new_master_port,
$new_master_user, $new_master_password, 1 );
## Set read_only=0 on the new master
#$new_master_handler->disable_log_bin_local();
print current_time_us() . " Set read_only=0 on the new master.\n";
$new_master_handler->disable_read_only();
## Creating an app user on the new master
#print current_time_us() . " Creating app user on the new master..\n";
#FIXME_xxx_create_app_user($new_master_handler);
#$new_master_handler->enable_log_bin_local();
$new_master_handler->disconnect();
## Update master ip on the catalog database, etc
print "Enabling the VIP - $vip on the new master - $new_master_host \n";
&start_vip();
&flush_arp();
$exit_code = 0;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
exit 0;
}
else {
&usage();
exit 1;
}
}
# A simple system call that enable the VIP on the new master
sub start_vip() {
`ssh $ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}
# A simple system call that disable the VIP on the old_master
sub stop_vip() {
`ssh $ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}
#flush arp
sub flush_arp() {
`ssh $ssh_user\@$new_master_host \" $flush_arp \"`;
}
sub usage {
print
"Usage: master_ip_online_change --command=start|stop|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n";
die;
}
4、自動發郵件指令碼（自動切換後會觸發郵件， 在MHA_manager）
vi /etc/masterha/scripts/send_report
#!/usr/bin/perl
# Copyright (C) 2011 DeNA Co.,Ltd.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
## Note: This is a sample script and is not complete. Modify the script based on your environment.
use strict;
use warnings FATAL => 'all';
use Mail::Sender;
use Getopt::Long;
#new_master_host and new_slave_hosts are set only when recovering master succeeded
my ( $dead_master_host, $new_master_host, $new_slave_hosts, $subject, $body );
my $smtp='smtp.chinadaas.com';
my $mail_from='support@chinadaas.com';
my $mail_user='support@chinadaas.com';
my $mail_pass='ZS_123456';
#my $mail_to=['chenzhibin@chinadaas.com'];
my $mail_to='chenzhibin@chinadaas.com';
GetOptions(
'orig_master_host=s' => \$dead_master_host,
'new_master_host=s' => \$new_master_host,
'new_slave_hosts=s' => \$new_slave_hosts,
'subject=s' => \$subject,
'body=s' => \$body,
);
# Do whatever you want here
mailToContacts($smtp,$mail_from,$mail_user,$mail_pass,$mail_to,$subject,$body);
sub mailToContacts {
my ($smtp, $mail_from, $mail_user, $mail_pass, $mail_to, $subject, $msg ) = @_;
open my $DEBUG, ">/var/log/masterha/app1/mail.log"
or die "Can't open the debug file:$!\n";
my $sender = new Mail::Sender {
ctype => 'text/plain;charset=utf-8',
encoding => 'utf-8',
smtp => $smtp,
from => $mail_from,
auth => 'LOGIN',
TLS_allowed => '0',
authid => $mail_user,
authpwd => $mail_pass,
to => $mail_to,
subject => $subject,
debug => $DEBUG
};
$sender->MailMsg(
{
msg => $msg,
debug => $DEBUG
}
) or print $Mail::Sender::Error;
return 1;
}
exit 0;
5、賦予執行許可權
chmod +x /etc/masterha/scripts/master_ip_failover
chmod +x /etc/masterha/scripts/master_ip_online_change
chmod +x /etc/masterha/scripts/send_report
八、檢查狀態
檢查ssh認證狀態（在mha_manager上）
masterha_check_ssh --conf=/etc/masterha/app1.cnf
檢查複製狀態（在mha_manager上）
masterha_check_repl --conf=/etc/masterha/app1.cnf
九、啟動和關閉管理節點
可以透過如下命令啟動mha：
nohup masterha_manager --conf=/etc/masterha/app1.cnf >> /dev/null 2>&1 &
可以透過如下命令停止mha
masterha_stop --conf=/etc/app1.cnf
十、檢視管理節點啟動後的服務：
ps -ef |grep master_ip_
ps -ef |grep masterha
或
masterha_check_status -conf=/etc/masterha/app1.cnf
或
檢視管理節點日誌資訊
tail -f /masterha/app1/manager.log
十一、自動切換測試
在主上建立VIP(只有在故障時，才會觸發指令碼，才會生成VIP)：
第一次需要在主庫Mysql_Master：10.101.110.90上手工繫結vip
ifconfig eth0:1 10.101.110.10/24
宕掉主庫（service mysql stop），可以透過日誌（tail -f /masterha/app1/manager.log），觀察到自動提升10.20.20.13為主，並將其餘的從庫的主庫切為了10.20.20.13
同時，VIP在新主10.20.20.13上自動建立
注意一下事項：
發生主備切換後，manager服務會自動停掉，且在/masterha/app1下面生成
app1.failover.complete，若再次發生切換需要刪除app1.failover.complete檔案
rm -fr /masterha/app1/app1.failover.complete
*****如果自動切換後，需要手工執行mha manager啟動指令碼，進行監控master 節點，並且刪除 rm -fr /masterha/app1/app1.failover.complete
常情況下自動切換以後，原master可能已經廢棄掉，待原master主機修復後，如果資料完整的情況下，
想把原來master重新作為新主庫的slave，這時我們可以藉助當時自動切換時刻的MHA日誌來完成對原master的修復
下面是提取相關日誌的命令：（在mha_manager上）
原主庫修復，變為備庫需要以下步驟
mha manager節點
[root@mha manager]# grep -i "All other slaves should start" /masterha/app1/manager.log
Mon Apr 21 22:28:33 2014 - [info] All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='10.20.20.13', （新主庫的ip）MASTER_PORT=3306, MASTER_LOG_FILE='mysql-bin.000022', MASTER_LOG_POS=506716, MASTER_USER='repl', MASTER_PASSWORD='xxx';
獲取上述資訊以後，就可以直接在修復後的原master上執行change master to相關操作，重新作為從庫了
十二、手工切換
有時候需要手動切換MySQL主伺服器，可以使用masterha_master_switch命令，不過它呼叫的不是master_ip_failover_script指令碼，而是master_ip_online_change_script指令碼，但呼叫引數類似，指令碼可以互用。
在mha_manager機器上：
masterha_master_switch --master_state=alive --conf=/etc/masterha/app1.cnf --new_master_host=10.20.20.12 --new_master_port=3306 --orig_master_is_new_slave --running_updates_limit=10000 --interactive=0
十三清除relay log （備庫恢復的binlog日誌）
對於MHA場景下，對於某些滯後從庫的恢復依賴於其他從庫的relay log，因此採取禁用自動刪除功能以及定期清理的辦法。對於清理過多過大的relay log需要注意引起的複製延遲資源開銷等。MHA可透過purge_relay_logs指令碼及配合cronjob來完成此項任務
在幾個MHA node節點上(MHA manager不需要)：
新增如下指令碼和任務執行計劃：
[root@JBWDB03 shell]# cat /usr/local/shell/purge_relay_logs.sh
#!/bin/bash
user=root
passwd='-s4)SVqJ'
port=3306
log_dir='/etc/masterha/log'
work_dir='/etc/masterha/data'
purge='/usr/local/bin/purge_relay_logs'
if [ ! -d $log_dir ]
then
mkdir $log_dir -p
fi
if [ ! -d $work_dir ]
then
mkdir $work_dir -p
fi
$purge --user=$user --password=$passwd --disable_relay_log_purge --port=$port --workdir=$work_dir >> $log_dir/purge_relay_logs.log 2>&1
[root@JBWDB03 shell]# crontab -l
##每天凌晨1點清空relay log
00 01 */1 * * /bin/bash /usr/local/shell/purge_relay_logs.sh
二、準備階段：
step1：建立信任關係：
在Node：10.20.20.12上：
[root@mysqlmaster /]#ssh-keygen -t rsa
[root@mysqlmaster /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.12
[root@mysqlmaster /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.13
[root@mysqlmaster /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.14
在Node：10.20.20.13上：
[root@mysqlslave1 /]#ssh-keygen -t rsa
[root@mysqlslave1 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.12
[root@mysqlslave1 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.13
[root@mysqlslave1 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.14
在Node：10.20.20.14上：
[root@mysqlslave2 /]#ssh-keygen -t rsa
[root@mysqlslave2 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.12
[root@mysqlslave2 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.13
[root@mysqlslave2 /]#ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.20.20.14
1、安裝編譯程式碼需要的包
yum -y install make gcc-c++ cmake bison-devel ncurses-devel
2、下載MySQL 5.6.14
wget
tar xvf mysql-5.6.17.tar.gz
cd mysql-5.6.27
3、編譯安裝
cmake \
-DCMAKE_INSTALL_PREFIX=/usr/local/mysql \
-DMYSQL_DATADIR=/data/mysql \
-DSYSCONFDIR=/etc \
-DWITH_MYISAM_STORAGE_ENGINE=1 \
-DWITH_INNOBASE_STORAGE_ENGINE=1 \
-DWITH_MEMORY_STORAGE_ENGINE=1 \
-DWITH_READLINE=1 \
-DMYSQL_UNIX_ADDR=/usr/local/mysql/mysql.sock \
-DMYSQL_TCP_PORT=3306 \
-DENABLED_LOCAL_INFILE=1 \
-DWITH_PARTITION_STORAGE_ENGINE=1 \
-DEXTRA_CHARSETS=all \
-DDEFAULT_CHARSET=utf8 \
-DDEFAULT_COLLATION=utf8_general_ci
make && make install
配置MySQL
建立使用者組
groupadd mysql
useradd -g mysql mysql
修改/usr/local/mysql許可權
chown -R mysql:mysql /usr/local/mysql
chown -R mysql:mysql /data
配置my.cnf(詳見conf檔案)
以下是簡單的my.cnf配置
[root@mysqlmaster /]# cat /etc/my.cnf
[mysqld]
log-bin=mysql-bin
server-id=12
skip-name-resolve
[root@mysqlslave1 ~]# cat /etc/my.cnf
[mysqld]
log-bin=mysql-bin
server-id=13
read-only=1
relay_log_purge=0
skip-name-resolve
[root@mysqlslave2 .ssh]# cat /etc/my.cnf
[mysqld]
log-bin=mysql-bin
server-id=14
read-only=1
relay_log_purge=0
skip-name-resolve
初始化配置
cd /usr/local/mysql
進入安裝路徑，執行初始化配置指令碼，建立系統自帶的資料庫和表
scripts/mysql_install_db --basedir=/usr/local/mysql --datadir=/data/mysql --user=mysql
啟動MySQL
cp support-files/mysql.server /etc/init.d/mysql
chkconfig mysql on
service mysql start
配置環境變數
vi /etc/profile
export PATH=/usr/local/mysql/bin:$PATH
生效環境變數
source /etc/profile
執行下面的命令修改root密碼
mysql -uroot
mysql> SET PASSWORD = PASSWORD('123456');
若要設定root使用者可以遠端訪問，執行
mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'172.16.%' IDENTIFIED BY 'password' WITH GRANT OPTION;
三、建立MySQL主從複製
step1：建立資料庫管理賬號和複製賬號（主庫、備庫 都執行）
grant all privileges on *.* to 'root'@'10.20.20.%' identified by '123456';
grant Replication slave,super on *.* to 'repler'@'10.20.20.%' identified by '123.com';
grant all privileges on *.* to 'root'@'192.168.1.%' identified by '123456';
flush privileges;
step2：建立mysql之間的主從複製
登入主伺服器的mysql，查詢master的狀態
mysql>show master status;
+------------------+----------+--------------+------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+------------------+----------+--------------+------------------+
| mysql-bin.000001 | 308 | | |
+------------------+----------+--------------+------------------+
配置從伺服器Slave
mysql>change master to master_host='10.20.20.12',master_user='repler',master_password='123.com',master_log_file='mysql-bin.000001',master_log_pos=308;
四、建立軟連結和設定從庫為read only
step1：ln -s /usr/local/mysql/bin/* /usr/local/bin/
注意：需要做這個軟連線，否則再執行masterha_check_repl校驗的時候，會報Can't exec "mysqlbinlog": 沒有那個檔案或目錄 的錯誤。
step2：所有從伺服器上my.cnf裡面新增：read-only=1和relay_log_purge=0
五、安裝MHA_Manager
1. 安裝epel源
rpm -Uvh
yum --disablerepo=epel -y update ca-certificates
2. 安裝依賴包
yum -y install perl-DBD-MySQL perl-Config-Tiny perl-Log-Dispatch perl-Params-Validate perl-TimeDate perl-ExtUtils-CBuilder perl-ExtUtils-MakeMaker perl-CPAN perl-Parallel-ForkManager perl-Log-Message
yum -y install perl-NTLM.noarch --如果要發郵件send_report，需要安裝這個包，否則就不發郵件
3、 安裝MHA_Node(MHA_Manager需要)
tar -zxvf mha4mysql-manager-0.57.tar.gz
cd mha4mysql-manager-0.57
perl Makefile.PL
make
make install
tar -zxf Test-Simple-1.001014.tar.gz
cd Test-Simple-1.001014
perl Makefile.PL
make
make install
tar -zxf CPAN-Perl-Releases-2.48.tar.gz
cd CPAN-Perl-Releases-2.48
perl Makefile.PL
make
make install
tar -zxf Log-Dispatch-Perl-0.04.tar.gz
cd Log-Dispatch-Perl-0.04
perl Makefile.PL
make
make install
mkdir /etc/masterha/
mkdir /etc/masterha/scripts
mkdir /masterha/app1 -p
六、安裝MHA_Node(在所有的Mysql伺服器上)
1. 安裝epel源
rpm -Uvh
yum --disablerepo=epel -y update ca-certificates
2. 安裝依賴包
yum -y install perl-DBD-MySQL perl-Config-Tiny perl-Log-Dispatch perl-Params-Validate perl-TimeDate perl-ExtUtils-CBuilder perl-ExtUtils-MakeMaker perl-CPAN perl-Parallel-ForkManager
3、 安裝MHA_Node
tar -zxf mha4mysql-node-0.57.tar.gz
cd mha4mysql-node-0.57
perl Makefile.PL
make
make install
mkdir /masterha/app1 -p
七、配置MHA_Manager(192.168.1.100)：
vim /etc/masterha/app1.cnf
[server default]
#workdir on the management server
manager_workdir=/masterha/app1/
manager_log=/masterha/app1/manager.log
#workdir on the node for mysql server
remote_workdir=/masterha/app1
master_binlog_dir=/data/mysql
check_repl_delay=0
#mysql user and password
user=root
password=123456
#replication_user
repl_user=repler
repl_password=123.com
#checking master every second
ping_interval=1
#ssh user
ssh_user=root
#檢測master的可用性
secondary_check_script=masterha_secondary_check -s 10.20.20.13 -s 10.20.20.14
#自動故障VIP切換呼叫指令碼
master_ip_failover_script=/etc/masterha/scripts/master_ip_failover
#手動故障切換呼叫指令碼
master_ip_online_change_script=/etc/masterha/scripts/master_ip_online_change
#傳送郵件指令碼
report_script=/mha/scripts/send_report
[server1]
hostname=10.20.20.12
candidate_master=1
[server2]
hostname=10.20.20.13
candidate_master=1
[server3]
hostname=10.20.20.14
no_master=1
2、自動故障VIP切換呼叫指令碼
vi /etc/masterha/scripts/master_ip_failover （在MHA_manager 節點上）
#!/usr/bin/env perl
use strict;
use warnings FATAL => 'all';
use Getopt::Long;
my (
$command, $ssh_user, $orig_master_host, $orig_master_ip,
$orig_master_port, $new_master_host, $new_master_ip, $new_master_port
);
my $vip = '192.168.1.100'; #Virtual IP
my $gateway = '192.168.1.1'; #Gateway IP
my $interface = 'eth0'; #bind to interface
my $key = "1";
my $ssh_stop_vip = "/sbin/ifconfig $interface:$key down";
GetOptions(
'command=s' => \$command,
'ssh_user=s' => \$ssh_user,
'orig_master_host=s' => \$orig_master_host,
'orig_master_ip=s' => \$orig_master_ip,
'orig_master_port=i' => \$orig_master_port,
'new_master_host=s' => \$new_master_host,
'new_master_ip=s' => \$new_master_ip,
'new_master_port=i' => \$new_master_port,
);
exit &main();
sub main {
print "\n\nIN SCRIPT TEST====$ssh_stop_vip==$ssh_start_vip===\n\n";
if ( $command eq "stop" || $command eq "stopssh" ) {
# $orig_master_host, $orig_master_ip, $orig_master_port are passed.
# If you manage master ip address at global catalog database,
# invalidate orig_master_ip here.
my $exit_code = 1;
eval {
print "Disabling the VIP on old master: $orig_master_host \n";
&stop_vip();
$exit_code = 0;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "start" ) {
# all arguments are passed.
# If you manage master ip address at global catalog database,
# activate new_master_ip here.
# You can also grant write access (create user, set read_only=0, etc) here.
my $exit_code = 10;
eval {
print "Enabling the VIP - $vip on the new master - $new_master_host \n";
&start_vip();
$exit_code = 0;
};
if ($@) {
warn $@;
exit $exit_code;
}
elsif ( $command eq "status" ) {
print "Checking the Status of the script.. OK \n";
`ssh $ssh_user\@$orig_master_host \" $ssh_start_vip \"`;
exit 0;
}
else {
&usage();
exit 1;
}
}
# A simple system call that enable the VIP on the new master
sub start_vip() {
`ssh $ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}
# A simple system call that disable the VIP on the old_master
sub stop_vip() {
`ssh $ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}
sub usage {
"Usage: master_ip_failover --command=start|stop|stopssh|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n";
}
3、手動故障切換呼叫指令碼 （在MHA_manager）
vi /etc/masterha/scripts/master_ip_online_change
#!/usr/bin/env perl
# Copyright (C) 2011 DeNA Co.,Ltd.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
## Note: This is a sample script and is not complete. Modify the script based on your environment.
use strict;
use warnings FATAL => 'all';
use Getopt::Long;
use MHA::DBHelper;
use MHA::NodeUtil;
use Time::HiRes qw( sleep gettimeofday tv_interval );
use Data::Dumper;
my $_tstart;
my $_running_interval = 0.1;
my (
$command, $orig_master_host, $orig_master_ip,
$orig_master_port, $orig_master_user, $orig_master_ssh_user,
$new_master_host, $new_master_ip, $new_master_port,
$new_master_user, $new_master_ssh_user, $orig_master_is_new_slave,
$orig_master_password, $new_master_password,
);
my $vip = '192.168.1.100'; ###Virtual IP
my $gateway = '192.168.1.1'; ###Gateway IP
my $interface = 'eth0'; ###interface
my $key = "1";
my $ssh_start_vip = "/sbin/ifconfig $interface:$key $vip/24";
my $flush_arp = "/sbin/arping -I $interface -c 2 -s $vip $gateway";
my $ssh_stop_vip = "/sbin/ifconfig $interface:$key down";
my $ssh_user = "root";
GetOptions(
'command=s' => \$command,
'ssh_user=s' => \$ssh_user,
'orig_master_host=s' => \$orig_master_host,
'orig_master_ip=s' => \$orig_master_ip,
'orig_master_port=i' => \$orig_master_port,
'orig_master_user=s' => \$orig_master_user,
'orig_master_password=s' => \$orig_master_password,
'new_master_host=s' => \$new_master_host,
'new_master_ip=s' => \$new_master_ip,
'new_master_port=i' => \$new_master_port,
'new_master_user=s' => \$new_master_user,
'new_master_password=s' => \$new_master_password,
'orig_master_ssh_user=s' => \$orig_master_ssh_user,
'new_master_ssh_user=s' => \$new_master_ssh_user,
'orig_master_is_new_slave' => \$orig_master_is_new_slave,
);
exit &main();
sub current_time_us {
my ( $sec, $microsec ) = gettimeofday();
my $curdate = localtime($sec);
return $curdate . " " . sprintf( "%06d", $microsec );
}
sub sleep_until {
my $elapsed = tv_interval($_tstart);
if ( $_running_interval > $elapsed ) {
sleep( $_running_interval - $elapsed );
}
}
sub get_threads_util {
my $dbh = shift;
my $my_connection_id = shift;
my $running_time_threshold = shift;
my $type = shift;
$running_time_threshold = 0 unless ($running_time_threshold);
$type = 0 unless ($type);
my @threads;
my $sth = $dbh->prepare("SHOW PROCESSLIST");
$sth->execute();
while ( my $ref = $sth->fetchrow_hashref() ) {
my $id = $ref->{Id};
my $user = $ref->{User};
my $host = $ref->{Host};
my $command = $ref->{Command};
my $state = $ref->{State};
my $query_time = $ref->{Time};
my $info = $ref->{Info};
$info =~ s/^\s*(.*?)\s*$/$1/ if defined($info);
next if ( $my_connection_id == $id );
next if ( defined($query_time) && $query_time < $running_time_threshold );
next if ( defined($command) && $command eq "Binlog Dump" );
next if ( defined($user) && $user eq "system user" );
next
if ( defined($command)
&& $command eq "Sleep"
&& defined($query_time)
&& $query_time >= 1 );
if ( $type >= 1 ) {
next if ( defined($command) && $command eq "Sleep" );
next if ( defined($command) && $command eq "Connect" );
}
if ( $type >= 2 ) {
next if ( defined($info) && $info =~ m/^select/i );
next if ( defined($info) && $info =~ m/^show/i );
}
push @threads, $ref;
}
return @threads;
}
sub main {
if ( $command eq "stop" ) {
## Gracefully killing connections on the current master
# 1. Set read_only= 1 on the new master
# 2. DROP USER so that no app user can establish new connections
# 3. Set read_only= 1 on the current master
# 4. Kill current queries
# * Any database access failure will result in script die.
my $exit_code = 1;
eval {
## Setting read_only=1 on the new master (to avoid accident)
my $new_master_handler = new MHA::DBHelper();
# args: hostname, port, user, password, raise_error(die_on_error)_or_not
$new_master_handler->connect( $new_master_ip, $new_master_port,
$new_master_user, $new_master_password, 1 );
print current_time_us() . " Set read_only on the new master.. ";
$new_master_handler->enable_read_only();
if ( $new_master_handler->is_read_only() ) {
print "ok.\n";
}
else {
die "Failed!\n";
}
$new_master_handler->disconnect();
# Connecting to the orig master, die if any database error happens
my $orig_master_handler = new MHA::DBHelper();
$orig_master_handler->connect( $orig_master_ip, $orig_master_port,
$orig_master_user, $orig_master_password, 1 );
## Drop application user so that nobody can connect. Disabling per-session binlog beforehand
#$orig_master_handler->disable_log_bin_local();
#print current_time_us() . " Drpping app user on the orig master..\n";
#FIXME_xxx_drop_app_user($orig_master_handler);
## Waiting for N * 100 milliseconds so that current connections can exit
my $time_until_read_only = 15;
$_tstart = [gettimeofday];
my @threads = get_threads_util( $orig_master_handler->{dbh},
$orig_master_handler->{connection_id} );
while ( $time_until_read_only > 0 && $#threads >= 0 ) {
if ( $time_until_read_only % 5 == 0 ) {
printf
"%s Waiting all running %d threads are disconnected.. (max %d milliseconds)\n",
current_time_us(), $#threads + 1, $time_until_read_only * 100;
if ( $#threads < 5 ) {
print Data::Dumper->new( [$_] )->Indent(0)->Terse(1)->Dump . "\n"
foreach (@threads);
}
}
sleep_until();
$_tstart = [gettimeofday];
$time_until_read_only--;
@threads = get_threads_util( $orig_master_handler->{dbh},
$orig_master_handler->{connection_id} );
}
## Setting read_only=1 on the current master so that nobody(except SUPER) can write
print current_time_us() . " Set read_only=1 on the orig master.. ";
$orig_master_handler->enable_read_only();
if ( $orig_master_handler->is_read_only() ) {
print "ok.\n";
}
else {
die "Failed!\n";
}
## Waiting for M * 100 milliseconds so that current update queries can complete
my $time_until_kill_threads = 5;
@threads = get_threads_util( $orig_master_handler->{dbh},
$orig_master_handler->{connection_id} );
while ( $time_until_kill_threads > 0 && $#threads >= 0 ) {
if ( $time_until_kill_threads % 5 == 0 ) {
printf
"%s Waiting all running %d queries are disconnected.. (max %d milliseconds)\n",
current_time_us(), $#threads + 1, $time_until_kill_threads * 100;
if ( $#threads < 5 ) {
print Data::Dumper->new( [$_] )->Indent(0)->Terse(1)->Dump . "\n"
foreach (@threads);
}
}
sleep_until();
$_tstart = [gettimeofday];
$time_until_kill_threads--;
@threads = get_threads_util( $orig_master_handler->{dbh},
$orig_master_handler->{connection_id} );
&stop_vip();
## Terminating all threads
print current_time_us() . " Killing all application threads..\n";
$orig_master_handler->kill_threads(@threads) if ( $#threads >= 0 );
print current_time_us() . " done.\n";
#$orig_master_handler->enable_log_bin_local();
$orig_master_handler->disconnect();
## After finishing the script, MHA executes FLUSH TABLES WITH READ LOCK
$exit_code = 0;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "start" ) {
## Activating master ip on the new master
# 1. Create app user with write privileges
# 2. Moving backup script if needed
# 3. Register new master's ip to the catalog database
# If exit code is 0 or 10, MHA does not abort
my $exit_code = 10;
eval {
my $new_master_handler = new MHA::DBHelper();
# args: hostname, port, user, password, raise_error_or_not
$new_master_handler->connect( $new_master_ip, $new_master_port,
$new_master_user, $new_master_password, 1 );
## Set read_only=0 on the new master
#$new_master_handler->disable_log_bin_local();
print current_time_us() . " Set read_only=0 on the new master.\n";
$new_master_handler->disable_read_only();
## Creating an app user on the new master
#print current_time_us() . " Creating app user on the new master..\n";
#FIXME_xxx_create_app_user($new_master_handler);
#$new_master_handler->enable_log_bin_local();
$new_master_handler->disconnect();
## Update master ip on the catalog database, etc
print "Enabling the VIP - $vip on the new master - $new_master_host \n";
&start_vip();
&flush_arp();
$exit_code = 0;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
exit 0;
}
else {
&usage();
exit 1;
}
}
# A simple system call that enable the VIP on the new master
sub start_vip() {
`ssh $ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}
# A simple system call that disable the VIP on the old_master
sub stop_vip() {
`ssh $ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}
#flush arp
sub flush_arp() {
`ssh $ssh_user\@$new_master_host \" $flush_arp \"`;
}
sub usage {
"Usage: master_ip_online_change --command=start|stop|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n";
die;
}
4、自動發郵件指令碼（自動切換後會觸發郵件， 在MHA_manager）
vi /etc/masterha/scripts/send_report
#!/usr/bin/perl
# Copyright (C) 2011 DeNA Co.,Ltd.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
## Note: This is a sample script and is not complete. Modify the script based on your environment.
use strict;
use warnings FATAL => 'all';
use Mail::Sender;
use Getopt::Long;
#new_master_host and new_slave_hosts are set only when recovering master succeeded
my ( $dead_master_host, $new_master_host, $new_slave_hosts, $subject, $body );
my $smtp='smtp.chinadaas.com';
my $mail_from='support@chinadaas.com';
my $mail_user='support@chinadaas.com';
my $mail_pass='ZS_123456';
#my $mail_to=['chenzhibin@chinadaas.com'];
my $mail_to='chenzhibin@chinadaas.com';
GetOptions(
'orig_master_host=s' => \$dead_master_host,
'new_master_host=s' => \$new_master_host,
'new_slave_hosts=s' => \$new_slave_hosts,
'subject=s' => \$subject,
'body=s' => \$body,
);
# Do whatever you want here
mailToContacts($smtp,$mail_from,$mail_user,$mail_pass,$mail_to,$subject,$body);
sub mailToContacts {
my ($smtp, $mail_from, $mail_user, $mail_pass, $mail_to, $subject, $msg ) = @_;
open my $DEBUG, ">/var/log/masterha/app1/mail.log"
or die "Can't open the debug file:$!\n";
my $sender = new Mail::Sender {
ctype => 'text/plain;charset=utf-8',
encoding => 'utf-8',
smtp => $smtp,
from => $mail_from,
auth => 'LOGIN',
TLS_allowed => '0',
authid => $mail_user,
authpwd => $mail_pass,
to => $mail_to,
subject => $subject,
debug => $DEBUG
};
$sender->MailMsg(
{
msg => $msg,
debug => $DEBUG
}
) or print $Mail::Sender::Error;
return 1;
}
exit 0;
5、賦予執行許可權
chmod +x /etc/masterha/scripts/master_ip_failover
chmod +x /etc/masterha/scripts/master_ip_online_change
chmod +x /etc/masterha/scripts/send_report
八、檢查狀態
檢查ssh認證狀態（在mha_manager上）
masterha_check_ssh --conf=/etc/masterha/app1.cnf
檢查複製狀態（在mha_manager上）
masterha_check_repl --conf=/etc/masterha/app1.cnf
九、啟動和關閉管理節點
可以透過如下命令啟動mha：
nohup masterha_manager --conf=/etc/masterha/app1.cnf >> /dev/null 2>&1 &
可以透過如下命令停止mha
masterha_stop --conf=/etc/app1.cnf
十、檢視管理節點啟動後的服務：
ps -ef |grep master_ip_
ps -ef |grep masterha
或
masterha_check_status -conf=/etc/masterha/app1.cnf
或
檢視管理節點日誌資訊
tail -f /masterha/app1/manager.log
十一、自動切換測試
在主上建立VIP(只有在故障時，才會觸發指令碼，才會生成VIP)：
第一次需要在主庫Mysql_Master：10.101.110.90上手工繫結vip
ifconfig eth0:1 10.101.110.10/24
宕掉主庫（service mysql stop），可以透過日誌（tail -f /masterha/app1/manager.log），觀察到自動提升10.20.20.13為主，並將其餘的從庫的主庫切為了10.20.20.13
同時，VIP在新主10.20.20.13上自動建立
注意一下事項：
發生主備切換後，manager服務會自動停掉，且在/masterha/app1下面生成
app1.failover.complete，若再次發生切換需要刪除app1.failover.complete檔案
rm -fr /masterha/app1/app1.failover.complete
*****如果自動切換後，需要手工執行mha manager啟動指令碼，進行監控master 節點，並且刪除 rm -fr /masterha/app1/app1.failover.complete
常情況下自動切換以後，原master可能已經廢棄掉，待原master主機修復後，如果資料完整的情況下，
想把原來master重新作為新主庫的slave，這時我們可以藉助當時自動切換時刻的MHA日誌來完成對原master的修復
下面是提取相關日誌的命令：（在mha_manager上）
原主庫修復，變為備庫需要以下步驟
mha manager節點
[root@mha manager]# grep -i "All other slaves should start" /masterha/app1/manager.log
Mon Apr 21 22:28:33 2014 - [info] All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='10.20.20.13', （新主庫的ip）MASTER_PORT=3306, MASTER_LOG_FILE='mysql-bin.000022', MASTER_LOG_POS=506716, MASTER_USER='repl', MASTER_PASSWORD='xxx';
獲取上述資訊以後，就可以直接在修復後的原master上執行change master to相關操作，重新作為從庫了
十二、手工切換
有時候需要手動切換MySQL主伺服器，可以使用masterha_master_switch命令，不過它呼叫的不是master_ip_failover_script指令碼，而是master_ip_online_change_script指令碼，但呼叫引數類似，指令碼可以互用。
在mha_manager機器上：
masterha_master_switch --master_state=alive --conf=/etc/masterha/app1.cnf --new_master_host=10.20.20.12 --new_master_port=3306 --orig_master_is_new_slave --running_updates_limit=10000 --interactive=0
十三清除relay log （備庫恢復的binlog日誌）
對於MHA場景下，對於某些滯後從庫的恢復依賴於其他從庫的relay log，因此採取禁用自動刪除功能以及定期清理的辦法。對於清理過多過大的relay log需要注意引起的複製延遲資源開銷等。MHA可透過purge_relay_logs指令碼及配合cronjob來完成此項任務
在幾個MHA node節點上(MHA manager不需要)：
新增如下指令碼和任務執行計劃：
[root@JBWDB03 shell]# cat /usr/local/shell/purge_relay_logs.sh
#!/bin/bash
user=root
passwd='-s4)SVqJ'
port=3306
log_dir='/etc/masterha/log'
work_dir='/etc/masterha/data'
purge='/usr/local/bin/purge_relay_logs'
if [ ! -d $log_dir ]
then
mkdir $log_dir -p
fi
if [ ! -d $work_dir ]
then
mkdir $work_dir -p
fi
$purge --user=$user --password=$passwd --disable_relay_log_purge --port=$port --workdir=$work_dir >> $log_dir/purge_relay_logs.log 2>&1
[root@JBWDB03 shell]# crontab -l
##每天凌晨1點清空relay log
00 01 */1 * * /bin/bash /usr/local/shell/purge_relay_logs.sh
來自 “ ITPUB部落格 ” ，連結：http://blog.itpub.net/30345407/viewspace-2088055/，如需轉載，請註明出處，否則將追究法律責任。
相關文章
- Mysql MHA部署-04MHA配置MySql
- MySQL MHA配置MySql
- MHA搭建步驟
- MYSQL + MHA +keepalive + VIP安裝配置(二)--MHA的配置MySql
- mysql5.7MHA配置MySql
- MHA搭建及故障維護
- MySQL MHA詳細搭建過程MySql
- mysql高可用架構MHA搭建MySql架構
- MHA安裝搭建和測試
- 【Mysql】MHA配置虛擬ipMySql
- mysql MHA搭建和切換測試MySql
- MHA架構搭建中遇到的問題架構
- 【MySQL】MHA的基本配置及註釋MySql
- MHA高可用配置及故障切換
- MYSQL實戰-MHA搭建及問題總結MySql
- mha+keepalived安裝配置過程
- MYSQL + MHA +keepalive + VIP安裝配置(一)--MYSQL安裝配置MySql
- 【MHA】mha日常維護命令
- 【MHA】mysql高可用之MHAMySql
- MYSQL+複製+MHA+VIP配置總結MySql
- MYSQL + MHA +keepalive + VIP安裝配置(三)--keepalived安裝配置MySql
- mysql高可用架構MHA搭建（centos7+mysql5.7.28）MySql架構CentOS
- MHA+MySQL主從配置實現MySQL高可用MySql
- MySQL MHA部署 Part 5 MHA部署指南MySql
- MySQL-HA MHA(Master High Availability manager)搭建手冊MySqlASTAI
- MHA初探
- Mysql MHA部署-03MHA軟體安裝MySql
- mysql之MHAMySql
- MySQL MHA部署 Part 6 MHA故障轉移測試MySql
- MySQL MHA部署 Part 7 MHA手動切換測試MySql
- MHA原始碼分析原始碼
- 【Mysql】MHA的原理MySql
- MySQL MHA介紹MySql
- 【MySQL】MHA安裝MySql
- Java環境搭建與配置、以及Tomcat搭建與配置JavaTomcat
- DG搭建配置方案
- Mysql 5.7 MHA 高可用MySql
- MySQL MHA部署實戰MySql