關於mysqldump備份非事務表的注意事項

aaron8219發表於2018-07-29
 
Preface
 
    We`re used to get a logical backup set(whole instance) by simply specifying “–single-transaction” and “–master-data” options.For innodb engine tables,we can get a consistent backup.But the non-innodb(not supporting transaction) tables may be not backed up consistently in MySQL 5.7.
 
Procedure
 
Check and open general log(if have not enabled it yet).
 1 (zlm@192.168.1.101 3306)[(none)]>show variables like `general_log`;
 2 +---------------+-------+
 3 | Variable_name | Value |
 4 +---------------+-------+
 5 | general_log   | OFF   |
 6 +---------------+-------+
 7 1 row in set (0.00 sec)
 8 
 9 (zlm@192.168.1.101 3306)[(none)]>set @@global.general_log=on;
10 Query OK, 0 rows affected (0.01 sec)
11 
12 (zlm@192.168.1.101 3306)[(none)]>show variables like `general_log`;
13 +---------------+-------+
14 | Variable_name | Value |
15 +---------------+-------+
16 | general_log   | ON    |
17 +---------------+-------+
18 1 row in set (0.00 sec)

 

Clear the content of general log and tail it.

1 [root@zlm2 09:35:07 /data/mysql/mysql3306/data]
2 #echo ``> zlm2.log
3 
4 [root@zlm2 09:35:13 /data/mysql/mysql3306/data]
5 #tail -f zlm2.log

 

Create test tables.

 1 (zlm@192.168.1.101 3306)[(none)]>create database zlm;
 2 Query OK, 1 row affected (0.00 sec)
 3 
 4 (zlm@192.168.1.101 3306)[(none)]>use zlm;
 5 Database changed
 6 
 7 (zlm@192.168.1.101 3306)[(none)]>create table test_trx(
 8     -> id int,
 9     -> name varchar(10)
10     -> ) engine=innodb;
11 Query OK, 0 rows affected (0.01 sec)
12 
13 (zlm@192.168.1.101 3306)[zlm]>insert into test_trx values(1,`zlm`);
14 Query OK, 1 row affected (0.01 sec)
15 
16 (zlm@192.168.1.101 3306)[zlm]>create table test_notrx like test_trx;
17 Query OK, 0 rows affected (0.01 sec)
18 
19 (zlm@192.168.1.101 3306)[zlm]>insert into test_notrx values(2,`aaron8219`);
20 Query OK, 1 row affected (0.00 sec)
21 
22 (zlm@192.168.1.101 3306)[zlm]>alter table test_notrx engine=MyISAM;
23 Query OK, 1 row affected (0.01 sec)
24 Records: 1  Duplicates: 0  Warnings: 0
25 
26 (zlm@192.168.1.101 3306)[zlm]>show create table test_trxG
27 *************************** 1. row ***************************
28        Table: test_trx
29 Create Table: CREATE TABLE `test_trx` (
30   `id` int(11) DEFAULT NULL,
31   `name` varchar(10) DEFAULT NULL
32 ) ENGINE=InnoDB DEFAULT CHARSET=utf8
33 1 row in set (0.00 sec)
34 
35 (zlm@192.168.1.101 3306)[zlm]>show create table test_notrxG
36 *************************** 1. row ***************************
37        Table: test_notrx
38 Create Table: CREATE TABLE `test_notrx` (
39   `id` int(11) DEFAULT NULL,
40   `name` varchar(10) DEFAULT NULL
41 ) ENGINE=MyISAM DEFAULT CHARSET=utf8
42 1 row in set (0.00 sec)
43 
44 (zlm@192.168.1.101 3306)[zlm]>select * from test_trx;
45 +------+------+
46 | id   | name |
47 +------+------+
48 |    1 | zlm  |
49 +------+------+
50 1 row in set (0.00 sec)
51 
52 (zlm@192.168.1.101 3306)[zlm]>select * from test_notrx;
53 +------+-----------+
54 | id   | name      |
55 +------+-----------+
56 |    2 | aaron8219 |
57 +------+-----------+
58 1 row in set (0.00 sec)

 

Generate a backup with mysqldump specifying common option of “–single-transaction” together with “–master-data=2”.

 1 [root@zlm2 09:36:23 ~]
 2 #mysqldump --single-transaction --master-data=2 zlm > test1.sql
 3 
 4 //General log output below.
 5 2018-07-30T09:45:17.815021+01:00        4 Connect    zlm@zlm2 on  using TCP/IP
 6 2018-07-30T09:45:17.815290+01:00        4 Query    /*!40100 SET @@SQL_MODE=`` */
 7 2018-07-30T09:45:17.815445+01:00        4 Query    /*!40103 SET TIME_ZONE=`+00:00` */
 8 2018-07-30T09:45:17.815862+01:00        4 Query    FLUSH /*!40101 LOCAL */ TABLES
 9 2018-07-30T09:45:17.823118+01:00        4 Query    FLUSH TABLES WITH READ LOCK //Generate a FTWRL.
10 2018-07-30T09:45:17.823435+01:00        4 Query    SET SESSION TRANSACTION ISOLATION LEVEL REPEATABLE READ
11 2018-07-30T09:45:17.823684+01:00        4 Query    START TRANSACTION /*!40100 WITH CONSISTENT SNAPSHOT */
12 2018-07-30T09:45:17.823904+01:00        4 Query    SHOW VARIABLES LIKE `gtid\_mode`
13 2018-07-30T09:45:17.826626+01:00        4 Query    SELECT @@GLOBAL.GTID_EXECUTED
14 2018-07-30T09:45:17.827032+01:00        4 Query    SHOW MASTER STATUS
15 2018-07-30T09:45:17.827278+01:00        4 Query    UNLOCK TABLES //Release the locks on all tables in whole instance.
16 2018-07-30T09:45:17.827480+01:00        4 Query    SELECT LOGFILE_GROUP_NAME, FILE_NAME, TOTAL_EXTENTS, INITIAL_SIZE, ENGINE, EXTRA FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = `UNDO LOG` AND FILE_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IN (SELECT DISTINCT LOGFILE_GROUP_NAME FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = `DATAFILE` AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA IN (`zlm`))) GROUP BY LOGFILE_GROUP_NAME, FILE_NAME, ENGINE, TOTAL_EXTENTS, INITIAL_SIZE ORDER BY LOGFILE_GROUP_NAME
17 2018-07-30T09:45:17.828912+01:00        4 Query    SELECT DISTINCT TABLESPACE_NAME, FILE_NAME, LOGFILE_GROUP_NAME, EXTENT_SIZE, INITIAL_SIZE, ENGINE FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = `DATAFILE` AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA IN (`zlm`)) ORDER BY TABLESPACE_NAME, LOGFILE_GROUP_NAME
18 2018-07-30T09:45:17.830942+01:00        4 Query    SHOW VARIABLES LIKE `ndbinfo\_version`
19 2018-07-30T09:45:17.833548+01:00        4 Init DB    zlm
20 2018-07-30T09:45:17.833785+01:00        4 Query    SAVEPOINT sp
21 2018-07-30T09:45:17.833963+01:00        4 Query    show tables
22 2018-07-30T09:45:17.834252+01:00        4 Query    show table status like `test\_notrx`
23 2018-07-30T09:45:17.834449+01:00        4 Query    SET SQL_QUOTE_SHOW_CREATE=1
24 2018-07-30T09:45:17.834582+01:00        4 Query    SET SESSION character_set_results = `binary`
25 2018-07-30T09:45:17.834781+01:00        4 Query    show create table `test_notrx`
26 2018-07-30T09:45:17.835621+01:00        4 Query    SET SESSION character_set_results = `utf8`
27 2018-07-30T09:45:17.835851+01:00        4 Query    show fields from `test_notrx`
28 2018-07-30T09:45:17.836362+01:00        4 Query    show fields from `test_notrx`
29 2018-07-30T09:45:17.836843+01:00        4 Query    SELECT /*!40001 SQL_NO_CACHE */ * FROM `test_notrx`
30 2018-07-30T09:45:17.837327+01:00        4 Query    SET SESSION character_set_results = `binary`
31 2018-07-30T09:45:17.838289+01:00        4 Query    use `zlm`
32 2018-07-30T09:45:17.838723+01:00        4 Query    select @@collation_database
33 2018-07-30T09:45:17.838908+01:00        4 Query    SHOW TRIGGERS LIKE `test\_notrx`
34 2018-07-30T09:45:17.839457+01:00        4 Query    SET SESSION character_set_results = `utf8`
35 2018-07-30T09:45:17.839678+01:00        4 Query    ROLLBACK TO SAVEPOINT sp
36 2018-07-30T09:45:17.839821+01:00        4 Query    show table status like `test\_trx`
37 2018-07-30T09:45:17.840112+01:00        4 Query    SET SQL_QUOTE_SHOW_CREATE=1
38 2018-07-30T09:45:17.840248+01:00        4 Query    SET SESSION character_set_results = `binary`
39 2018-07-30T09:45:17.840364+01:00        4 Query    show create table `test_trx`
40 2018-07-30T09:45:17.840675+01:00        4 Query    SET SESSION character_set_results = `utf8`
41 2018-07-30T09:45:17.840828+01:00        4 Query    show fields from `test_trx`
42 2018-07-30T09:45:17.841258+01:00        4 Query    show fields from `test_trx`
43 2018-07-30T09:45:17.841774+01:00        4 Query    SELECT /*!40001 SQL_NO_CACHE */ * FROM `test_trx`
44 2018-07-30T09:45:17.842205+01:00        4 Query    SET SESSION character_set_results = `binary`
45 2018-07-30T09:45:17.842370+01:00        4 Query    use `zlm`
46 2018-07-30T09:45:17.842646+01:00        4 Query    select @@collation_database
47 2018-07-30T09:45:17.842814+01:00        4 Query    SHOW TRIGGERS LIKE `test\_trx`
48 2018-07-30T09:45:17.844200+01:00        4 Query    SET SESSION character_set_results = `utf8`
49 2018-07-30T09:45:17.844540+01:00        4 Query    ROLLBACK TO SAVEPOINT sp
50 2018-07-30T09:45:17.844803+01:00        4 Query    RELEASE SAVEPOINT sp
51 2018-07-30T09:45:17.847871+01:00        4 Quit
52 
53 //The "unlock tables" operation was executed very soon after a short period of time of getting gtid information.
54 //After the releasing lock operation,the consistency of transaction table can be guaranteed while the non-transaction table cannot be.

 

This time merely use “–master-data=2”.

 1 [root@zlm2 09:45:17 ~]
 2 #mysqldump --master-data=2 zlm > test2.sql
 3 
 4 //General log output below.
 5 2018-07-30T10:06:41.766819+01:00        5 Connect    zlm@zlm2 on  using TCP/IP
 6 2018-07-30T10:06:41.767265+01:00        5 Query    /*!40100 SET @@SQL_MODE=`` */
 7 2018-07-30T10:06:41.767495+01:00        5 Query    /*!40103 SET TIME_ZONE=`+00:00` */
 8 2018-07-30T10:06:41.767852+01:00        5 Query    FLUSH /*!40101 LOCAL */ TABLES
 9 2018-07-30T10:06:41.768067+01:00        5 Query    FLUSH TABLES WITH READ LOCK //It still generate a FTWRL.
10 2018-07-30T10:06:41.768237+01:00        5 Query    SHOW VARIABLES LIKE `gtid\_mode`
11 2018-07-30T10:06:41.770245+01:00        5 Query    SELECT @@GLOBAL.GTID_EXECUTED
12 2018-07-30T10:06:41.770659+01:00        5 Query    SHOW MASTER STATUS //No releasing lock operation here.
13 2018-07-30T10:06:41.770955+01:00        5 Query    SELECT LOGFILE_GROUP_NAME, FILE_NAME, TOTAL_EXTENTS, INITIAL_SIZE, ENGINE, EXTRA FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = `UNDO LOG` AND FILE_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IN (SELECT DISTINCT LOGFILE_GROUP_NAME FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = `DATAFILE` AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA IN (`zlm`))) GROUP BY LOGFILE_GROUP_NAME, FILE_NAME, ENGINE, TOTAL_EXTENTS, INITIAL_SIZE ORDER BY LOGFILE_GROUP_NAME
14 2018-07-30T10:06:41.773632+01:00        5 Query    SELECT DISTINCT TABLESPACE_NAME, FILE_NAME, LOGFILE_GROUP_NAME, EXTENT_SIZE, INITIAL_SIZE, ENGINE FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = `DATAFILE` AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA IN (`zlm`)) ORDER BY TABLESPACE_NAME, LOGFILE_GROUP_NAME
15 2018-07-30T10:06:41.775734+01:00        5 Query    SHOW VARIABLES LIKE `ndbinfo\_version`
16 2018-07-30T10:06:41.777953+01:00        5 Init DB    zlm
17 2018-07-30T10:06:41.778223+01:00        5 Query    show tables
18 2018-07-30T10:06:41.778529+01:00        5 Query    show table status like `test\_notrx`
19 2018-07-30T10:06:41.778898+01:00        5 Query    SET SQL_QUOTE_SHOW_CREATE=1
20 2018-07-30T10:06:41.779149+01:00        5 Query    SET SESSION character_set_results = `binary`
21 2018-07-30T10:06:41.780362+01:00        5 Query    show create table `test_notrx`
22 2018-07-30T10:06:41.780722+01:00        5 Query    SET SESSION character_set_results = `utf8`
23 2018-07-30T10:06:41.780907+01:00        5 Query    show fields from `test_notrx`
24 2018-07-30T10:06:41.781665+01:00        5 Query    show fields from `test_notrx`
25 2018-07-30T10:06:41.782447+01:00        5 Query    SELECT /*!40001 SQL_NO_CACHE */ * FROM `test_notrx`
26 2018-07-30T10:06:41.782788+01:00        5 Query    SET SESSION character_set_results = `binary`
27 2018-07-30T10:06:41.782945+01:00        5 Query    use `zlm`
28 2018-07-30T10:06:41.783298+01:00        5 Query    select @@collation_database
29 2018-07-30T10:06:41.783425+01:00        5 Query    SHOW TRIGGERS LIKE `test\_notrx`
30 2018-07-30T10:06:41.784086+01:00        5 Query    SET SESSION character_set_results = `utf8`
31 2018-07-30T10:06:41.790618+01:00        5 Query    show table status like `test\_trx`
32 2018-07-30T10:06:41.791015+01:00        5 Query    SET SQL_QUOTE_SHOW_CREATE=1
33 2018-07-30T10:06:41.791094+01:00        5 Query    SET SESSION character_set_results = `binary`
34 2018-07-30T10:06:41.791192+01:00        5 Query    show create table `test_trx`
35 2018-07-30T10:06:41.791285+01:00        5 Query    SET SESSION character_set_results = `utf8`
36 2018-07-30T10:06:41.791360+01:00        5 Query    show fields from `test_trx`
37 2018-07-30T10:06:41.792328+01:00        5 Query    show fields from `test_trx`
38 2018-07-30T10:06:41.792837+01:00        5 Query    SELECT /*!40001 SQL_NO_CACHE */ * FROM `test_trx`
39 2018-07-30T10:06:41.793036+01:00        5 Query    SET SESSION character_set_results = `binary`
40 2018-07-30T10:06:41.793147+01:00        5 Query    use `zlm`
41 2018-07-30T10:06:41.793277+01:00        5 Query    select @@collation_database
42 2018-07-30T10:06:41.793381+01:00        5 Query    SHOW TRIGGERS LIKE `test\_trx`
43 2018-07-30T10:06:41.794223+01:00        5 Query    SET SESSION character_set_results = `utf8`
44 2018-07-30T10:06:41.799961+01:00        5 Quit
45 
46 //No RR isolation level and consistent snapshot this time what means the consistency of transaction table cannot be guaranteed,too.
47 //As for the non-transaction table,it will hold the FTWRL until the backup finishes.That means it`s consistent in the course.
48 //The bad influence is that the locking time will be extremly long.Because option of "--master-data" will enbale "--lock-all-tables" or "-x" option.

 

This time execute mysqldump without any option.

 1 [root@zlm2 10:16:20 ~]
 2 #mysqldump zlm > test3.sql
 3 
 4 //General log output below.
 5 2018-07-30T10:24:04.371650+01:00        6 Connect    zlm@zlm2 on  using TCP/IP
 6 2018-07-30T10:24:04.371651+01:00        6 Query    /*!40100 SET @@SQL_MODE=`` */
 7 2018-07-30T10:24:04.371885+01:00        6 Query    /*!40103 SET TIME_ZONE=`+00:00` */
 8 2018-07-30T10:24:04.372048+01:00        6 Query    SHOW VARIABLES LIKE `gtid\_mode`
 9 2018-07-30T10:24:04.374903+01:00        6 Query    SELECT @@GLOBAL.GTID_EXECUTED
10 2018-07-30T10:24:04.376278+01:00        6 Query    SELECT LOGFILE_GROUP_NAME, FILE_NAME, TOTAL_EXTENTS, INITIAL_SIZE, ENGINE, EXTRA FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = `UNDO LOG` AND FILE_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IN (SELECT DISTINCT LOGFILE_GROUP_NAME FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = `DATAFILE` AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA IN (`zlm`))) GROUP BY LOGFILE_GROUP_NAME, FILE_NAME, ENGINE, TOTAL_EXTENTS, INITIAL_SIZE ORDER BY LOGFILE_GROUP_NAME
11 2018-07-30T10:24:04.378718+01:00        6 Query    SELECT DISTINCT TABLESPACE_NAME, FILE_NAME, LOGFILE_GROUP_NAME, EXTENT_SIZE, INITIAL_SIZE, ENGINE FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = `DATAFILE` AND TABLESPACE_NAME IN (SELECT DISTINCT TABLESPACE_NAME FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA IN (`zlm`)) ORDER BY TABLESPACE_NAME, LOGFILE_GROUP_NAME
12 2018-07-30T10:24:04.380381+01:00        6 Query    SHOW VARIABLES LIKE `ndbinfo\_version`
13 2018-07-30T10:24:04.383136+01:00        6 Init DB    zlm
14 2018-07-30T10:24:04.383512+01:00        6 Query    show tables
15 2018-07-30T10:24:04.383922+01:00        6 Query    LOCK TABLES `test_notrx` READ /*!32311 LOCAL */,`test_trx` READ /*!32311 LOCAL */ //There`re two individual locks here.
16 2018-07-30T10:24:04.384267+01:00        6 Query    show table status like `test\_notrx`
17 2018-07-30T10:24:04.384870+01:00        6 Query    SET SQL_QUOTE_SHOW_CREATE=1
18 2018-07-30T10:24:04.385115+01:00        6 Query    SET SESSION character_set_results = `binary`
19 2018-07-30T10:24:04.385241+01:00        6 Query    show create table `test_notrx`
20 2018-07-30T10:24:04.385479+01:00        6 Query    SET SESSION character_set_results = `utf8`
21 2018-07-30T10:24:04.385651+01:00        6 Query    show fields from `test_notrx`
22 2018-07-30T10:24:04.386440+01:00        6 Query    show fields from `test_notrx`
23 2018-07-30T10:24:04.387011+01:00        6 Query    SELECT /*!40001 SQL_NO_CACHE */ * FROM `test_notrx`
24 2018-07-30T10:24:04.387314+01:00        6 Query    SET SESSION character_set_results = `binary`
25 2018-07-30T10:24:04.387446+01:00        6 Query    use `zlm`
26 2018-07-30T10:24:04.387704+01:00        6 Query    select @@collation_database
27 2018-07-30T10:24:04.388009+01:00        6 Query    SHOW TRIGGERS LIKE `test\_notrx`
28 2018-07-30T10:24:04.388492+01:00        6 Query    SET SESSION character_set_results = `utf8`
29 2018-07-30T10:24:04.388576+01:00        6 Query    show table status like `test\_trx`
30 2018-07-30T10:24:04.389393+01:00        6 Query    SET SQL_QUOTE_SHOW_CREATE=1
31 2018-07-30T10:24:04.389691+01:00        6 Query    SET SESSION character_set_results = `binary`
32 2018-07-30T10:24:04.390024+01:00        6 Query    show create table `test_trx`
33 2018-07-30T10:24:04.390330+01:00        6 Query    SET SESSION character_set_results = `utf8`
34 2018-07-30T10:24:04.392180+01:00        6 Query    show fields from `test_trx`
35 2018-07-30T10:24:04.393918+01:00        6 Query    show fields from `test_trx`
36 2018-07-30T10:24:04.394497+01:00        6 Query    SELECT /*!40001 SQL_NO_CACHE */ * FROM `test_trx`
37 2018-07-30T10:24:04.394847+01:00        6 Query    SET SESSION character_set_results = `binary`
38 2018-07-30T10:24:04.395108+01:00        6 Query    use `zlm`
39 2018-07-30T10:24:04.395955+01:00        6 Query    select @@collation_database
40 2018-07-30T10:24:04.396372+01:00        6 Query    SHOW TRIGGERS LIKE `test\_trx`
41 2018-07-30T10:24:04.397040+01:00        6 Query    SET SESSION character_set_results = `utf8`
42 2018-07-30T10:24:04.397295+01:00        6 Query    UNLOCK TABLES //The lock was released after finished backup.
43 2018-07-30T10:24:04.401336+01:00        6 Quit
44 
45 //If you don not specify any option of mysqldump,default option "--lock-tables" or "-l" will be enable.
46 //The "--lock-tables" option will add a read-only lock by executing "lock table xxx read;" individually on which table you want to backup.
47 //There`s neither RR isolation leve nor consistent snapshot.Therefore,only the non-transaction table was guaranteed a consistent backup.

 

Summary
  • We cannot get a consistent backup of non-transaction table with simply using option of “–singel-transaction” and “–master-data” simultaneously.
  • If you merely specify the option “–master-data”.It can guarantee a consistent backup of non-transaction table since it will enable a FTWRL.That is,it will enable the option of “–lock-all-tables”(short option is “-x”).The global read lock on table will sustain until to the end of backup.
  • If you do not specify any option of mysqldump,the default option of “–lock-table”(short option is “-l”) will be enabled.That will also guarantee a consisten backup of non-transaction table.
  • Notice the above situations when backup your non-transaction tables when using mysqldump otherwise you`ll get an inconsistent backup.That`s really an awful thing.

 

相關文章