MYSQL INNODB replace into 死鎖 及 next key lock 淺析

gaopengtttt發表於2017-06-29
原創:全文帶入了大量自我認知和理解,可能錯誤,因為水平有限,但是代表我努力分析過。



一、問題提出
問題是由姜大師提出的、問題如下:
表:
mysql> show create table c \G
*************************** 1. row ***************************
       Table: c
Create Table: CREATE TABLE `c` (
  `a` int(11) NOT NULL AUTO_INCREMENT,
  `b` int(11) DEFAULT NULL,
  PRIMARY KEY (`a`),
  UNIQUE KEY `b` (`b`)
) ENGINE=InnoDB 
1 row in set (0.01 sec)
開啟兩個會話不斷的執行
replace into c values(NULL,1);
會觸發死鎖。問死鎖觸發的原因。

我使用的環境:
MYSQL 5.7.14 debug版本、隔離級別RR、自動提交,很顯然這裡的c表中的可以select出來的記錄始終是1條
只是a列不斷的增大,但是這裡實際儲存空間確不止1條,因為從heap no來看二級索引中,heap no 已經到了
7,也就是有至少7(7-1)條記錄,只是其他記錄標記為del並且被purge執行緒放到了page free_list中。

二、準備工作和使用方法
1、稍微修改了原始碼關於鎖的列印部分,我們知道每個事物下顯示鎖記憶體結構lock 
   struct會連線成一個連結串列,只要按照順序列印出記憶體lock struct就列印出了
   所有關於這個事物顯示鎖全部資訊和加鎖順序如下:

點選(此處)摺疊或開啟

  1. ---TRANSACTION 184771, ACTIVE 45 sec
  2. 4 lock struct(s), heap size 1160, 3 row lock(s)
  3. MySQL thread id 2, OS thread handle 140737154311936, query id 642 localhost root cleaning up
  4. ---lock strcut(1):(Add by gaopeng) In modify Version I force check all REC_LOCK/TAB_LOCK for this Trx
  5. TABLE LOCK table `test`.`c4` trx id 184771 lock mode IX
  6. ---lock strcut(2):(Add by gaopeng) In modify Version I force check all REC_LOCK/TAB_LOCK for this Trx
  7. RECORD LOCKS space id 413 page no 4 n bits 72 index id2 of table `test`.`c4` trx id 184771 lock_mode X(LOCK_X)
  8. Record lock, heap no 4 PHYSICAL RECORD: n_fields 2; compact format; info bits 0
  9.  0: len 4; hex 80000014; asc ;;
  10.  1: len 4; hex 80000014; asc ;;
  11. ---lock strcut(3):(Add by gaopeng) In modify Version I force check all REC_LOCK/TAB_LOCK for this Trx
  12. RECORD LOCKS space id 413 page no 3 n bits 72 index PRIMARY of table `test`.`c4` trx id 184771 lock_mode X(LOCK_X) locks rec but not gap(LOCK_REC_NOT_GAP)
  13. Record lock, heap no 4 PHYSICAL RECORD: n_fields 4; compact format; info bits 0
  14.  0: len 4; hex 80000014; asc ;;
  15.  1: len 6; hex 00000002d1bd; asc ;;
  16.  2: len 7; hex a600000e230110; asc # ;;
  17.  3: len 4; hex 80000014; asc ;;
  18. ---lock strcut(4):(Add by gaopeng) In modify Version I force check all REC_LOCK/TAB_LOCK for this Trx
  19. RECORD LOCKS space id 413 page no 4 n bits 72 index id2 of table `test`.`c4` trx id 184771 lock_mode X(LOCK_X) locks gap before rec(LOCK_GAP)
  20. Record lock, heap no 5 PHYSICAL RECORD: n_fields 2; compact format; info bits 0
  21.  0: len 4; hex 8000001e; asc ;;
  22.  1: len 4; hex 8000001e; asc ;;


正常的版本只有

點選(此處)摺疊或開啟

  1. ---TRANSACTION 184771, ACTIVE 45 sec
  2. 4 lock struct(s), heap size 1160, 3 row lock(s)
  3. MySQL thread id 2, OS thread handle 140737154311936, query id 642 localhost root cleaning up
部分後面的都是我加上的,其實修改很簡單,innodb其實自己寫好了只是沒有開啟,我開啟後加上了序號來表示順序。
上面是一個 select * from c where  id2= 20 for update; b列為輔助索引的所有4 lock struct(s),可以看到有了這些資訊分析
不那麼難了。
這裡稍微分析一下
表結構為:
mysql> show create table c4;
+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table | Create Table                                                                                                                                                  |
+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+
| c4    | CREATE TABLE `c4` (
  `id1` int(11) NOT NULL,
  `id2` int(11) DEFAULT NULL,
  PRIMARY KEY (`id1`),
  KEY `id2` (`id2`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 |
+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
資料為:
mysql> select * from c4;
+-----+------+
| id1 | id2  |
+-----+------+
|   1 |    1 |
|  10 |   10 |
|  20 |   20 |
|  30 |   30 |
+-----+------+
4 rows in set (0.00 sec)
語句為:
 select * from c where  id2= 20 for update;
RR模式
從鎖結構連結串列來看,這個語句在輔助索引分別鎖定了
id2:20 id1:20 LOCK_X|LOCK_ORDINARY 也就是NEXT KEY LOCK
同時鎖定了
id2:30 id1:30 LOCK_X|LOCK_GAP也就是gap lock不包含這一列
那麼畫個圖容易理解黃色部分為鎖定部分:

是不是一目瞭然?如果是rc那麼鎖定的只有記錄了兩個黃色箭頭
表示gap沒有了就不在畫圖了

2、在死鎖檢測回滾前呼叫這個列印函式列印到err日誌檔案中,列印出全部的事物的顯示記憶體lock struct如下,這裡就
不給出了,後面會有replace觸發死鎖千事物鎖結構的一個輸出


3、使用MYSQL TRACE SQL語句得到大部分的函式呼叫來分析replace的過程

修改出現的問題:修改原始碼列印出所有lock struct 線上上顯然是不能用的。因為列印出來後show engine innodb status 會非常
長,甚至引發其他問題,但是測試是可以,其次修改了列印死鎖事物鎖連結串列到日誌後,每次只要遇到死鎖資訊可以列印
到日誌,但是每次MYSQLD都會掛掉,但是不影響分析了。

三、預備知識(自我理解)
1、
Precise modes:
#define LOCK_ORDINARY 0 /*!< this flag denotes an ordinary
next-key lock in contrast to LOCK_GAP
or LOCK_REC_NOT_GAP */
預設是LOCK_ORDINARY及普通的next_key_lock,鎖住行及以前的間隙
#define LOCK_GAP 512 /*!< when this bit is set, it means that the
lock holds only on the gap before the record;
for instance, an x-lock on the gap does not
give permission to modify the record on which
the bit is set; locks of this type are created
when records are removed from the index chain
of records */
間隙鎖,鎖住行以前的間隙,不鎖住本行
#define LOCK_REC_NOT_GAP 1024 /*!< this bit means that the lock is only on
the index record and does NOT block inserts
to the gap before the index record; this is
used in the case when we retrieve a record
with a unique key, and is also used in
locking plain SELECTs (not part of UPDATE
or DELETE) when the user has set the READ
COMMITTED isolation level */
行鎖,鎖住行而不鎖住任何間隙
#define LOCK_INSERT_INTENTION 2048 /*!< this bit is set when we place a waiting
gap type record lock request in order to let
an insert of an index record to wait until
there are no conflicting locks by other
transactions on the gap; note that this flag
remains set when the waiting lock is granted,
or if the lock is inherited record */
插入意向鎖,如果插入的記錄在某個已經鎖定的間隙內為這個鎖
2、引數innodb_autoinc_lock_mode的值為1,也許不能保證replace into的順序。
3、infimum和supremum
   一個page中包含這兩個偽列,頁中所有的行未刪除(刪除未purge)的行都連線到這兩個虛列之間,其中
   supremum偽列的鎖始終為next_key_lock。
4、heap no
   此行在page中的heap no heap no儲存在fixed_extrasize 中,heap no 為物理儲存填充的序號,頁的空閒空間掛載在page free連結串列中(頭插法)可以重用,
   但是重用此heap no不變,如果一直是insert 則heap no 不斷增加,並非按照KEY大小排序的邏輯連結串列順序,而是物理填充順序 
5、n bits
   和這個page相關的鎖點陣圖的大小如果我的表有9條資料 還包含2個infimum和supremum虛擬列 及 64+11 bits,及75bits但是必須被8整除為一個位元組就是
   80 bits
6、隱含鎖(Implicit lock)和顯示鎖(explict)
鎖有隱含和顯示之分。隱含鎖通常發生在 insert 的時候對cluster index和second index 都加隱含鎖,如果是UPDATE(DELETE)對cluster index加顯示鎖 輔助
索引加隱含鎖。目的在於減少鎖結構的記憶體開銷,如果有事務需要和這個隱含鎖而不相容,這個事務需要幫助 insert或者update(delete)事物將隱含
鎖變為顯示鎖,然後給自己加鎖,通常insert主鍵檢查會給自己加上S鎖,REPLACE、delete、update通常會給自己加上X鎖。

四、replace過程分析
透過replace的trace找到了這些步驟的大概呼叫:
首先我們假設
TRX1:replace 不提交
TRX2:replace 堵塞
TRX1:replace 提交
TRX2:replace 繼續執行直到完成
這樣做的目的在於透過trace找到TRX2在哪裡等待,確實如我所願我找到了。

1、檢查是否衝突,插入主鍵 

點選(此處)摺疊或開啟

  1.     569 T@4: | | | | | | | | >row_ins
  2.     570 T@4: | | | | | | | | | row_ins: table: test/c
  3.     571 T@4: | | | | | | | | | >row_ins_index_entry_step
  4.     572 T@4: | | | | | | | | | | >row_ins_clust_index_entry
  5.     573 T@4: | | | | | | | | | | | >row_ins_clust_index_entry_low
  6.     574 T@4: | | | | | | | | | | | | >btr_cur_search_to_nth_level
  7.     575 T@4: | | | | | | | | | | | | <btr_cur_search_to_nth_level 2005
  8.     576 T@4: | | | | | | | | | | | | ib_cur: insert PRIMARY (366) by 183808: TUPLE (info_bits=0, 4 fields): {[4] %(0x00000005),[6] (0x000000020E00),[7] (0x 0A000001010100),[4] (0x00000001)}
  9.     577 T@4: | | | | | | | | | | | <row_ins_clust_index_entry_low 2649
  10.     578 T@4: | | | | | | | | | | <row_ins_clust_index_entry 3313
  11.     579 T@4: | | | | | | | | | <row_ins_index_entry_step 3589
2、檢查是否衝突,插入輔助索引,這裡實際上就是會話2被堵塞的地方,如下解釋
(如果衝突回滾先前插入的主鍵內容)

點選(此處)摺疊或開啟

  1.     580 T@4: | | | | | | | | | >row_ins_index_entry_step 3589
  2.     581 T@4: | | | | | | | | | | >row_ins_sec_index_entry_low
  3.     582 T@4: | | | | | | | | | | | >btr_cur_search_to_nth_level
  4.     583 T@4: | | | | | | | | | | | <btr_cur_search_to_nth_level 2005
  5.     584 T@4: | | | | | | | | | | | >row_ins_scan_sec_index_for_duplicate
  6.     585 T@4: | | | | | | | | | | | | >btr_cur_search_to_nth_level
  7.     586 T@4: | | | | | | | | | | | | <btr_cur_search_to_nth_level 2005
  8.     587 T@4: | | | | | | | | | | | | >btr_cur_search_to_nth_level
  9.     588 T@4: | | | | | | | | | | | | <btr_cur_search_to_nth_level 2005
  10.     589 T@4: | | | | | | | | | | | | >row_vers_impl_x_locked_low
  11.     590 T@4: | | | | | | | | | | | | | info: Implicit lock is held by trx:183803
  12.     591 T@4: | | | | | | | | | | | | <row_vers_impl_x_locked_low 329
  13.     592 T@4: | | | | | | | | | | | | >thd_report_row_lock_wait
  14.     593 T@4: | | | | | | | | | | | | <thd_report_row_lock_wait 4246
  15.     594 T@4: | | | | | | | | | | | <row_ins_scan_sec_index_for_duplicate 2148
  16.     595 T@4: | | | | | | | | | | <row_ins_sec_index_entry_low 3043
  17.     596 T@4: | | | | | | | | | <row_ins_index_entry_step 3589
  18.     597 T@4: | | | | | | | | <row_ins 3758 
  19.     598 //wait here
  20.     這裡我做trace的時候事物的trace停止在了這裡我特意加上了598//wait here從下面的輸出
  21.     我們也能肯定確實這裡觸發了鎖等待 
  22.     >row_vers_impl_x_locked_low
  23.     | info: Implicit lock is held by trx:183803
  24.     <row_vers_impl_x_locked_low 329
  25.     >thd_report_row_lock_wait
  26.     <thd_report_row_lock_wait 4246
  27.     等待獲得鎖過後重新檢查:
  28.     599 T@4: | | | | | | | | >row_ins
  29.     600 T@4: | | | | | | | | | row_ins: table: test/c
  30.     601 T@4: | | | | | | | | | >row_ins_index_entry_step
  31.     602 T@4: | | | | | | | | | | >row_ins_sec_index_entry_low
  32.     603 T@4: | | | | | | | | | | | >btr_cur_search_to_nth_level
  33.     604 T@4: | | | | | | | | | | | <btr_cur_search_to_nth_level 2005
  34.     605 T@4: | | | | | | | | | | | >row_ins_scan_sec_index_for_duplicate
  35.     606 T@4: | | | | | | | | | | | | >btr_cur_search_to_nth_level
  36.     607 T@4: | | | | | | | | | | | | <btr_cur_search_to_nth_level 2005
  37.     608 T@4: | | | | | | | | | | | <row_ins_scan_sec_index_for_duplicate 2148
  38.     609 T@4: | | | | | | | | | | <row_ins_sec_index_entry_low 3043
  39.     610 T@4: | | | | | | | | | <row_ins_index_entry_step 3589
  40.     611 T@4: | | | | | | | | <row_ins 3810
我們可以隱隱約約看到row_ins_sec_index_entry_low和row_ins_clust_index_entry_low回檢查是否有重複的行
    分別代表是二級索引和聚集索引的相關檢查,因為就這個案例主鍵不可能出現重複值,而二級索引這個例子中肯定是
    重複的,索引row_ins_sec_index_entry_low觸發了等待,其實我們知道這裡的鎖方式如下列子:
    
---lock strcut(2):(Add by gaopeng) In modify Version I force check all REC_LOCK/TAB_LOCK chain! for this Trx
RECORD LOCKS space id 406 page no 4 n bits 72 index b of table `test`.`c` trx id 177891 lock_mode X(LOCK_X) waiting(LOCK_WAIT)
Record lock, heap no 3 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
 0: len 4; hex 80000001; asc     ;;
 1: len 4; hex 80000006; asc     ;;
 
LOCK_X|LOCK_ORDINARY|LOCK_WAIT:需要X的next_key lock處於等待狀態他需要鎖定(infimum,{1,6}]這個區間。
這也是死鎖發生的關鍵一個環節。

3、這裡涉及到了回滾操作,從下面的trace輸出我們也能看到確實做了回滾
   實際上事物2會堵塞在這裡,因為我做trace的時候他一直停在
   這裡不動了。為此我還加上598行說明在這裡wait了

點選(此處)摺疊或開啟

  1.     612 T@4: | | | | | | | | ib_que: Execute 9 (QUERY THREAD) at 0x7fffe804b7d8
  2.     613 T@4: | | | | | | | | ib_que: Execute 12 (ROLLBACK) at 0x7fffe804b6b0
  3.     614 T@4: | | | | | | | | ib_que: Execute 12 (ROLLBACK) at 0x7fffe804b6b0
  4.     615 T@4: | | | | | | | | ib_que: Execute 9 (QUERY THREAD) at 0x7fffe804b7d8
  5.     616 T@4: | | | | | | | | ib_que: Execute 9 (QUERY THREAD) at 0x7fffe800eec8
  6.     617 T@4: | | | | | | | | ib_que: Execute 10 (UNDO ROW) at 0x7fffe801b090
  7.     618 T@4: | | | | | | | | >btr_cur_search_to_nth_level
  8.     619 T@4: | | | | | | | | <btr_cur_search_to_nth_level 2005
  9.     620 T@4: | | | | | | | | >btr_cur_search_to_nth_level
  10.     621 T@4: | | | | | | | | <btr_cur_search_to_nth_level 2005
  11.     622 T@4: | | | | | | | | ib_que: Execute 10 (UNDO ROW) at 0x7fffe801b090
  12.     623 T@4: | | | | | | | | ib_que: Execute 9 (QUERY THREAD) at 0x7fffe800eec8
4、這個重複key會傳遞給SERVER層次,並且貌似重新初始化了事物(只是從trace猜測)

點選(此處)摺疊或開啟

  1. 639 T@4: | | | | | | >handler::get_dup_key
  2.     640 T@4: | | | | | | | >info
  3.     641 T@4: | | | | | | | | >ha_innobase::update_thd
  4.     642 T@4: | | | | | | | | | ha_innobase::update_thd: user_thd: 0x7fffe8000b90 -> 0x7fffe8000b90
  5.     643 T@4: | | | | | | | | | >innobase_trx_init
  6.     644 T@4: | | | | | | | | | <innobase_trx_init 2765
  7.     645 T@4: | | | | | | | | <ha_innobase::update_thd 3073
  8.     646 T@4: | | | | | | | <info 14717
  9.     647 T@4: | | | | | | <handler::get_dup_key 4550
  10.     648 T@4: | | | | | | >column_bitmaps_signal
  11.     649 T@4: | | | | | | | info: read_set: 0x7fffc8941da0 write_set: 0x7fffc8941da0
  12.     650 T@4: | | | | | | <column_bitmaps_signal 3846
  13.     651 T@4: | | | | | | >innobase_trx_init
  14.     652 T@4: | | | | | | <innobase_trx_init 2765
  15.     653 T@4: | | | | | | >index_init
  16.     654 T@4: | | | | | | <index_init 8864
5、接下就是真正刪除插入主鍵

點選(此處)摺疊或開啟

  1.     689 T@4: | | | | | | | | >row_update_for_mysql_using_upd_graph
  2.     690 T@4: | | | | | | | | | >row_upd_step
  3.     691 T@4: | | | | | | | | | | >row_upd
  4.     692 T@4: | | | | | | | | | | | row_upd: table: test/c
  5.     693 T@4: | | | | | | | | | | | row_upd: info bits in update vector: 0x0
  6.     694 T@4: | | | | | | | | | | | row_upd: foreign_id: NULL
  7.     695 T@4: | | | | | | | | | | | ib_cur: delete-mark clust test/(366) by 183808: COMPACT RECORD(info_bits=32, 4 fields): {[4] $(0x00000004),[6] (0x000000020D 0B),[7] (0x00000001090100),[4] (0x00000001)}
  8.     696 T@4: | | | | | | | | | | | >row_ins_clust_index_entry
  9.     697 T@4: | | | | | | | | | | | | >row_ins_clust_index_entry_low
  10.     698 T@4: | | | | | | | | | | | | | >btr_cur_search_to_nth_level
  11.     699 T@4: | | | | | | | | | | | | | <btr_cur_search_to_nth_level 2005
  12.     700 T@4: | | | | | | | | | | | | | ib_cur: insert PRIMARY (366) by 183808: TUPLE (info_bits=0, 4 fields): {[4] %(0x00000005),[6](0x000000020E00),[7] ( 0x00000001090100),[4] (0x00000001)}
  13.     701 T@4: | | | | | | | | | | | | <row_ins_clust_index_entry_low 2649
  14.     702 T@4: | | | | | | | | | | | <row_ins_clust_index_entry 3313
  15.     703 T@4: | | | | | | | | | | | >btr_cur_search_to_nth_level
  16.     704 T@4: | | | | | | | | | | | <btr_cur_search_to_nth_level 2005
  17.     705 T@4: | | | | | | | | | | | ib_cur: delete-mark=1 sec 406:4:in b(367) by 183808
6、接下就是真正插入輔助索引

點選(此處)摺疊或開啟

  1.     706 T@4: | | | | | | | | | | | >row_ins_sec_index_entry_low
  2.     707 T@4: | | | | | | | | | | | | >btr_cur_search_to_nth_level
  3.     708 T@4: | | | | | | | | | | | | <btr_cur_search_to_nth_level 2005
  4.     709 T@4: | | | | | | | | | | | | >row_ins_scan_sec_index_for_duplicate
  5.     710 T@4: | | | | | | | | | | | | | >btr_cur_search_to_nth_level
  6.     711 T@4: | | | | | | | | | | | | | <btr_cur_search_to_nth_level 2005
  7.     712 T@4: | | | | | | | | | | | | | >btr_cur_search_to_nth_level
  8.     713 T@4: | | | | | | | | | | | | | <btr_cur_search_to_nth_level 2005
  9.     714 T@4: | | | | | | | | | | | | | >row_vers_impl_x_locked_low
  10.     715 T@4: | | | | | | | | | | | | | <row_vers_impl_x_locked_low 123
  11.     716 T@4: | | | | | | | | | | | | | >btr_cur_search_to_nth_level
  12.     717 T@4: | | | | | | | | | | | | | <btr_cur_search_to_nth_level 2005
  13.     718 T@4: | | | | | | | | | | | | | >row_vers_impl_x_locked_low
  14.     719 T@4: | | | | | | | | | | | | | | info: Implicit lock is held by trx:183808
  15.     720 T@4: | | | | | | | | | | | | | <row_vers_impl_x_locked_low 329
  16.     721 T@4: | | | | | | | | | | | | <row_ins_scan_sec_index_for_duplicate 2148
  17.     722 T@4: | | | | | | | | | | | | >btr_cur_search_to_nth_level
  18.     723 T@4: | | | | | | | | | | | | <btr_cur_search_to_nth_level 2005
  19.     724 T@4: | | | | | | | | | | | | ib_cur: insert b (367) by 183808: TUPLE (info_bits=0, 2 fields): {[4] (0x00000001),[4] %(0x00000005)}
  20.     725 T@4: | | | | | | | | | | | <row_ins_sec_index_entry_low 3194
  21.     726 T@4: | | | | | | | | | | <row_upd 3066
  22.     727 T@4: | | | | | | | | | <row_upd_step 3181
  23.     728 T@4: | | | | | | | | <row_update_for_mysql_using_upd_graph 2670
  24.     729 T@4: | | | | | | | <ha_innobase::update_row 8656
注意:上面只是看trace出來的過程,很多是根據函式呼叫進行的猜測。

五、死鎖前事物鎖資訊列印分析
列印出死鎖前事物的全部資訊

點選(此處)摺疊或開啟

  1. 2017-06-28T00:25:20.202052Z 76 [Note] InnoDB: (DeadlockChecker::check_and_resolve(T):Add by gaopeng before rollback print all trx info here:)
  2. LIST OF TRANSACTIONS FOR EACH SESSION:
  3. ---TRANSACTION 422212176319952, not started
  4. 0 lock struct(s), heap size 1160, 0 row lock(s)
  5. ---TRANSACTION 182592, ACTIVE 0 sec inserting
  6. mysql tables in use 1, locked 1
  7. LOCK WAIT 2 lock struct(s), heap size 1160, 1 row lock(s), undo log entries 1
  8. MySQL thread id 77, OS thread handle 140737155630848, query id 3627 localhost root update
  9. replace into c values(NULL,1)
  10. ---lock strcut(1):(Add by gaopeng) In modify Version I force check all REC_LOCK/TAB_LOCK for this Trx
  11. TABLE LOCK table `mysqlslap`.`c` trx id 182592 lock mode IX
  12. ---lock strcut(2):(Add by gaopeng) In modify Version I force check all REC_LOCK/TAB_LOCK for this Trx
  13. RECORD LOCKS space id 407 page no 4 n bits 80 index b of table `mysqlslap`.`c` trx id 182592 lock_mode X(LOCK_X) waiting(LOCK_WAIT)
  14. Record lock, heap no 7 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
  15.  0: len 4; hex 80000001; asc ;;
  16.  1: len 4; hex 80000911; asc ;;

  17. ---TRANSACTION 182588, ACTIVE 0 sec updating or deleting
  18. mysql tables in use 1, locked 1
  19. 5 lock struct(s), heap size 1160, 5 row lock(s), undo log entries 2
  20. MySQL thread id 76, OS thread handle 140737156429568, query id 3623 localhost root update
  21. replace into c values(NULL,1)
  22. ---lock strcut(1):(Add by gaopeng) In modify Version I force check all REC_LOCK/TAB_LOCK for this Trx
  23. TABLE LOCK table `mysqlslap`.`c` trx id 182588 lock mode IX
  24. ---lock strcut(2):(Add by gaopeng) In modify Version I force check all REC_LOCK/TAB_LOCK for this Trx
  25. RECORD LOCKS space id 407 page no 4 n bits 80 index b of table `mysqlslap`.`c` trx id 182588 lock_mode X(LOCK_X)
  26. Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
  27.  0: len 8; hex 73757072656d756d; asc supremum;;
  28. Record lock, heap no 7 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
  29.  0: len 4; hex 80000001; asc ;;
  30.  1: len 4; hex 80000911; asc ;;

  31. ---lock strcut(3):(Add by gaopeng) In modify Version I force check all REC_LOCK/TAB_LOCK for this Trx
  32. RECORD LOCKS space id 407 page no 4 n bits 80 index b of table `mysqlslap`.`c` trx id 182588 lock_mode X(LOCK_X) locks gap before rec(LOCK_GAP)
  33. Record lock, heap no 7 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
  34.  0: len 4; hex 80000001; asc ;;
  35.  1: len 4; hex 80000911; asc ;;

  36. ---lock strcut(4):(Add by gaopeng) In modify Version I force check all REC_LOCK/TAB_LOCK for this Trx
  37. RECORD LOCKS space id 407 page no 3 n bits 88 index PRIMARY of table `mysqlslap`.`c` trx id 182588 lock_mode X(LOCK_X) locks rec but not gap(LOCK_REC_NOT_GAP)
  38. Record lock, heap no 2 PHYSICAL RECORD: n_fields 4; compact format; info bits 32
  39.  0: len 4; hex 80000911; asc ;;
  40.  1: len 6; hex 00000002c93c; asc <;;
  41.  2: len 7; hex 2c0000402e0c97; asc , @. ;;
  42.  3: len 4; hex 80000001; asc ;;

  43. ---lock strcut(5):(Add by gaopeng) In modify Version I force check all REC_LOCK/TAB_LOCK for this Trx
  44. RECORD LOCKS space id 407 page no 4 n bits 80 index b of table `mysqlslap`.`c` trx id 182588 lock_mode X(LOCK_X) locks gap before rec(LOCK_GAP) insert intention(LOCK_INSERT_INTENTION) waiting(LOCK_WAIT)
  45. Record lock, heap no 7 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
  46.  0: len 4; hex 80000001; asc ;;
  47.  1: len 4; hex 80000911; asc ;;
這裡涉及兩個事物
1、TRANSACTION 182592
2、TRANSACTION 182588
事物182588先於182592執行。
在上一次事物結束後:
留下的資料為:
a=0X911 b=0X1
那麼開始輔助索引上的邏輯連結串列 如下(這裡只畫單項鍊表)




死鎖流程流程開始:
時刻N:

--- TRX:182588   lock strcut(2)
  
 supremum和a=0X911 b=0X1均加LOCK_X|LOCK_ORDINARY
    為獨佔NEXT KEY LOCK,
    換句話說整個範圍都加鎖了,就這個列子replace期間不能
    插入任何資料,這點比較狠。
    如下圖黃色部分都是這裡加鎖了:


時刻N+1:
---TRX:182592 lock strcut(2)
檢查輔助索引唯一性,這裡在上面的trace中已經說明過了

等待發生,鎖型別為  LOCK_X|LOCK_ORDINARY|LOCK_WAIT
同樣為獨佔NEXT KEY LOCK處於WAIT範圍是0X1 0X911如下圖 紅色箭頭部分都
是要想加鎖的其實就是0X1 0X911和infimum到0X1 0X911的範圍:                                                                                                                                      

時刻N+2:
---TRX:182588 
   這裡涉及一個重點和lock strcut(5)

    (引數innodb_autoinc_lock_mode的值為1,也許不能保證replace into的順序有關),
   顯然這個時候如果
我要插入的新資料是a=0X908 b=0X1,只要小於a=0X911 b=0X1
    就會落到TRX182592鎖定的範圍那鎖的型別為:
    LOCK_X|LOCK_GAP|LOCK_INSERT_INTENTION|LOCK_WAIT
    因為這個範圍已經被其他事物TRX:182592鎖定本事物WAIT
   插入意向GAP鎖不能獲得

    如下圖綠色箭頭就表示a=0X908 b=0X1需要插入到的位置,顯然
   已經被TRX:182592鎖定:                                                       


                       

這樣一來 TRX:182588等待TRX:182592等待TRX:182588的流程就出來了,死鎖觸發。
從整個死鎖的關鍵來看視乎innodb_autoinc_lock_mode能夠保證併發事物獲得自增值的
順序就成了關鍵,如果能夠保證順序顯然a=0X908 b=0X1這樣的記錄不可能出現,因為0X911
後面應該是0X912,如果是a=0X912 b=0X1自然不會落到infimum到a=0X911 b=0X1之間不會被TRX182592堵塞
死鎖也不會出現,所以這裡死鎖觸發還是需要一定條件的。
如下圖:


六、實驗初次驗證

我使用如下:
CREATE TABLE `c` (
  `a` int(11) NOT NULL ,
  `b` int(11) DEFAULT NULL,
  PRIMARY KEY (`a`),
  UNIQUE KEY `b` (`b`)
) ENGINE=InnoDB ;

這個儲存過程增加奇數
DELIMITER //  
CREATE PROCEDURE test1()
begin 
  declare num int;
set num = 1; 
while num <= 100000000 do
  replace into c values(num,1);
  set num = num+2;
end while;
end //
這個儲存過程增加偶數
CREATE PROCEDURE test2()
begin 
  declare num int;
set num = 2; 
while num <= 100000000 do
  replace into c values(num,1);
  set num = num+2;
end while;
end //
DELIMITER ;

TRX1:時間N
mysql> call test1();        

TRX2:時間N+0.5分鐘
 mysql> call test2();繼續執行

TRX1:(幾乎同時)時間N+0.5分鐘
mysql> call test1();    
ERROR 1213 (40001): Deadlock found when trying to get lock; try restarting transaction

TRX2:
 mysql> call test2();繼續執行

可以看到這兩個儲存過程不會產生同樣的主鍵一個為奇數一個為偶數
然後讓test1先跑一會,然後再跑test2()目的在於模擬上面主鍵自增
取值小於當前值的情況,果然基本是馬上就出現死鎖了。
例如某一時刻 test1() 中主鍵的值為1001
此時test2()才開始跑這個時候主鍵值為2
那麼就模擬出來上面判斷的0X911後事物取值為0X908的情況,只是這裡不再是偶然事件,
而是必然,而原題中為偶然事件。
由於家裡沒有修改了原始碼的環境就先看show engine innodb status中
的死鎖如下:

點選(此處)摺疊或開啟

  1. ------------------------
  2. LATEST DETECTED DEADLOCK
  3. ------------------------
  4. 2017-06-29 14:10:30 0x7fa48148b700
  5. *** (1) TRANSACTION:
  6. TRANSACTION 4912797, ACTIVE 0 sec inserting
  7. mysql tables in use 1, locked 1
  8. LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s), undo log entries 1
  9. MySQL thread id 2, OS thread handle 140344520656640, query id 3371 localhost root update
  10. replace into c values(num,1)
  11. *** (1) WAITING FOR THIS LOCK TO BE GRANTED:
  12. RECORD LOCKS space id 598 page no 4 n bits 80 index b of table `test`.`c` trx id 4912797 lock_mode X waiting
  13. Record lock, heap no 3 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
  14.  0: len 4; hex 80000001; asc ;;
  15.  1: len 4; hex 800007d5; asc ;;

  16. *** (2) TRANSACTION:
  17. TRANSACTION 4912793, ACTIVE 0 sec updating or deleting
  18. mysql tables in use 1, locked 1
  19. 6 lock struct(s), heap size 1136, 6 row lock(s), undo log entries 2
  20. MySQL thread id 3, OS thread handle 140344520390400, query id 3365 localhost root update
  21. replace into c values(num,1)
  22. *** (2) HOLDS THE LOCK(S):
  23. RECORD LOCKS space id 598 page no 4 n bits 80 index b of table `test`.`c` trx id 4912793 lock_mode X
  24. Record lock, heap no 3 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
  25.  0: len 4; hex 80000001; asc ;;
  26.  1: len 4; hex 800007d5; asc ;;

  27. *** (2) WAITING FOR THIS LOCK TO BE GRANTED:
  28. RECORD LOCKS space id 598 page no 4 n bits 80 index b of table `test`.`c` trx id 4912793 lock_mode X locks gap before rec insert intention waiting
  29. Record lock, heap no 3 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
  30.  0: len 4; hex 80000001; asc ;;
  31.  1: len 4; hex 800007d5; asc ;;

  32. *** WE ROLL BACK TRANSACTION (1)
可以看到完全一致,證明問題判斷正確。

七、疑問
1、
二級索引中heap no執行一段時間後如下:
00200010000e 80000001800008af
002000180054 8000000180000712
00200020000e 8000000180000014
00240028002a 8000000180000017
002000300070 80000001800008e0
00200038fff2 8000000180000911
00200040002a 800000018000001a
00200048002a 800000018000001b
00000050ff82 8000000180000912
002000580000 800000018000001d
00200060ff90 800000018000001e
00200068ffd6 800000018000090e
00200070ff58 80000001800008df
很顯然這裡只有8000000180000912 是當前資料,其他都標記為了del,按理說長度一樣的資料進行刪除插入,空間應該
會不斷重用,為什麼有時候重用不了呢?
2、在整個replace加鎖流程中,我並沒有完全搞懂,譬如182588的lock strcut(3)和lock strcut(1)分別是什麼時候加
     的用於保護什麼操作,這裡只是從死鎖現象進行了分析。

八、RC模式下我做了同樣的測試得到如下的死鎖前事物LOCK STRUCT連結串列和RR模式基本無異。不在分析給出即可。
mysql> show variables like '%tx_isolation%';
+---------------+----------------+
| Variable_name | Value          |
+---------------+----------------+
| tx_isolation  | READ-COMMITTED |
+---------------+----------------+
1 row in set (0.09 sec)

點選(此處)摺疊或開啟

  1. ------- TRX HAS BEEN WAITING 0 SEC FOR THIS LOCK TO BE GRANTED:
  2. RECORD LOCKS space id 407 page no 4 n bits 104 index b of table `mysqlslap`.`c` trx id 289638 lock_mode X(LOCK_X) waiting(LOCK_WAIT)
  3. Record lock, heap no 20 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
  4.  0: len 4; hex 80000001; asc ;;
  5.  1: len 4; hex 80001220; asc ;;

  6. ------------------
  7. ---lock strcut(1):(Add by gaopeng) In modify Version I force check all REC_LOCK/TAB_LOCK for this Trx
  8. TABLE LOCK table `mysqlslap`.`c` trx id 289638 lock mode IX
  9. ---lock strcut(2):(Add by gaopeng) In modify Version I force check all REC_LOCK/TAB_LOCK for this Trx
  10. RECORD LOCKS space id 407 page no 4 n bits 104 index b of table `mysqlslap`.`c` trx id 289638 lock_mode X(LOCK_X) waiting(LOCK_WAIT)
  11. Record lock, heap no 20 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
  12.  0: len 4; hex 80000001; asc ;;
  13.  1: len 4; hex 80001220; asc ;;

  14. ---TRANSACTION 289636, ACTIVE 0 sec updating or deleting
  15. mysql tables in use 1, locked 1
  16. 7 lock struct(s), heap size 1160, 7 row lock(s), undo log entries 2
  17. MySQL thread id 5, OS thread handle 140734658983680, query id 4646 localhost root update
  18. replace into c values(null,1)
  19. ---lock strcut(1):(Add by gaopeng) In modify Version I force check all REC_LOCK/TAB_LOCK for this Trx
  20. TABLE LOCK table `mysqlslap`.`c` trx id 289636 lock mode IX
  21. ---lock strcut(2):(Add by gaopeng) In modify Version I force check all REC_LOCK/TAB_LOCK for this Trx
  22. RECORD LOCKS space id 407 page no 4 n bits 104 index b of table `mysqlslap`.`c` trx id 289636 lock_mode X(LOCK_X)
  23. Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
  24.  0: len 8; hex 73757072656d756d; asc supremum;;
  25. Record lock, heap no 3 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
  26.  0: len 4; hex 80000001; asc ;;
  27.  1: len 4; hex 80001221; asc !;;

  28. ---lock strcut(3):(Add by gaopeng) In modify Version I force check all REC_LOCK/TAB_LOCK for this Trx
  29. RECORD LOCKS space id 407 page no 4 n bits 104 index b of table `mysqlslap`.`c` trx id 289636 lock_mode X(LOCK_X) locks gap before rec(LOCK_GAP)
  30. Record lock, heap no 20 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
  31.  0: len 4; hex 80000001; asc ;;
  32.  1: len 4; hex 80001220; asc ;;

  33. ---lock strcut(4):(Add by gaopeng) In modify Version I force check all REC_LOCK/TAB_LOCK for this Trx
  34. RECORD LOCKS space id 407 page no 4 n bits 104 index b of table `mysqlslap`.`c` trx id 289636 lock_mode X(LOCK_X)

  35. ---lock strcut(5):(Add by gaopeng) In modify Version I force check all REC_LOCK/TAB_LOCK for this Trx
  36. RECORD LOCKS space id 407 page no 4 n bits 104 index b of table `mysqlslap`.`c` trx id 289636 lock_mode X(LOCK_X)
  37. Record lock, heap no 20 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
  38.  0: len 4; hex 80000001; asc ;;
  39.  1: len 4; hex 80001220; asc ;;

  40. ---lock strcut(6):(Add by gaopeng) In modify Version I force check all REC_LOCK/TAB_LOCK for this Trx
  41. RECORD LOCKS space id 407 page no 3 n bits 104 index PRIMARY of table `mysqlslap`.`c` trx id 289636 lock_mode X(LOCK_X) locks rec butnot gap(LOCK_REC_NOT_GAP)
  42. Record lock, heap no 8 PHYSICAL RECORD: n_fields 4; compact format; info bits 32
  43.  0: len 4; hex 80001221; asc !;;
  44.  1: len 6; hex 000000046b64; asc kd;;
  45.  2: len 7; hex 30000001f00c97; asc 0 ;;
  46.  3: len 4; hex 80000001; asc ;;

  47. ---lock strcut(7):(Add by gaopeng) In modify Version I force check all REC_LOCK/TAB_LOCK for this Trx
  48. RECORD LOCKS space id 407 page no 4 n bits 104 index b of table `mysqlslap`.`c` trx id 289636 lock_mode X(LOCK_X) locks gap before rec(LOCK_GAP) insert intention(LOCK_INSERT_INTENTION) waiting(LOCK_WAIT)
  49. Record lock, heap no 20 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
  50.  0: len 4; hex 80000001; asc ;;
  51.  1: len 4; hex 80001220; asc ;;

本文某些關鍵點參考了文章,最大的提示就是自增值不是有序的,這點以後要驗證一下,但是實驗也證明了這一點:



作者微信:

MYSQL INNODB replace into 死鎖 及 next key lock 淺析

來自 “ ITPUB部落格 ” ,連結:http://blog.itpub.net/7728585/viewspace-2141409/,如需轉載,請註明出處,否則將追究法律責任。

相關文章