併發插入引起的死鎖問題排查

壹頁書發表於2017-02-17
2017大年初一晚上,線上報死鎖了..
大家掛著VPN排查問題,也是有點意思.
不過當時並沒有排查出原因

上班之後,發現是客戶端的一個bug.本來應該傳送一個請求,但是卻傳送了大量的請求,應用層面又沒有做冪等設計,所以所有的請求都落到了資料庫層面。
資料庫是一個過程

環境
MySQL 5.6.14
事務隔離級別 讀提交

引起問題的邏輯大致如下:

  1.     if not exists (select UserID from award_free_firecracker_watch_common where UserID=pUserID and AwardDate=pToday and GiftID=pGift) then
  2.       insert into award_free_firecracker_watch_common (UserID, AwardDate,GiftID) values (pUserID, pToday,pGift);
  3.     end if;
  4.     
  5.     select AwardTotal into pOld from award_free_firecracker_common where UserID=pUserID and GiftID=pGift for update;
  6.     update award_free_firecracker_common set AwardTotal=AwardTotal+1,AwardLeft=AwardLeft+1 where UserID=pUserID and GiftID=pGift;
  7.     
  8.     SELECT ROW_COUNT() into pUpdateCount;
  9.     if pUpdateCount>0 then
  10.       update award_free_firecracker_watch_common set WatchCount=WatchCount+1 where UserID=pUserID and AwardDate=pToday and GiftID=pGift ;
  11.     end if;


因為這個過程也不是我寫的,大致的意思是先查詢award_free_firecracker_watch_common表,如果沒有這個使用者的記錄,則插入.
然後修改award_free_firecracker_common的記錄,最後在修改award_free_firecracker_watch_common的記錄.

MySQL的加鎖過程
https://dev.mysql.com/doc/refman/5.7/en/innodb-locks-set.html

INSERT sets an exclusive lock on the inserted row. This lock is an index-record lock, not a next-key lock (that is, there is no gap lock) and does not prevent other sessions from inserting into the gap before the inserted row.

Prior to inserting the row, a type of gap lock called an insert intention gap lock is set. This lock signals the intent to insert in such a way that multiple transactions inserting into the same index gap need not wait for each other if they are not inserting at the same position within the gap. Suppose that there are index records with values of 4 and 7. Separate transactions that attempt to insert values of 5 and 6 each lock the gap between 4 and 7 with insert intention locks prior to obtaining the exclusive lock on the inserted row, but do not block each other because the rows are nonconflicting.

If a duplicate-key error occurs, a shared lock on the duplicate index record is set. This use of a shared lock can result in deadlock should there be multiple sessions trying to insert the same row if another session already has an exclusive lock. This can occur if another session deletes the row. Suppose that an InnoDB table t1 has the following structure:

CREATE TABLE t1 (i INT, PRIMARY KEY (i)) ENGINE = InnoDB;

Now suppose that three sessions perform the following operations in order:

Session 1:

START TRANSACTION;
INSERT INTO t1 VALUES(1);

Session 2:

START TRANSACTION;
INSERT INTO t1 VALUES(1);

Session 3:

START TRANSACTION;
INSERT INTO t1 VALUES(1);

Session 1:

ROLLBACK;

The first operation by session 1 acquires an exclusive lock for the row. The operations by sessions 2 and 3 both result in a duplicate-key error and they both request a shared lock for the row. When session 1 rolls back, it releases its exclusive lock on the row and the queued shared lock requests for sessions 2 and 3 are granted. At this point, sessions 2 and 3 deadlock: Neither can acquire an exclusive lock for the row because of the shared lock held by the other.

A similar situation occurs if the table already contains a row with key value 1 and three sessions perform the following operations in order:

Session 1:

START TRANSACTION;
DELETE FROM t1 WHERE i = 1;

Session 2:

START TRANSACTION;
INSERT INTO t1 VALUES(1);

Session 3:

START TRANSACTION;
INSERT INTO t1 VALUES(1);

Session 1:

COMMIT;

The first operation by session 1 acquires an exclusive lock for the row. The operations by sessions 2 and 3 both result in a duplicate-key error and they both request a shared lock for the row. When session 1 commits, it releases its exclusive lock on the row and the queued shared lock requests for sessions 2 and 3 are granted. At this point, sessions 2 and 3 deadlock: Neither can acquire an exclusive lock for the row because of the shared lock held by the other.


可以看到,Insert的時候,對記錄上排它鎖和插入意向鎖.
併發的情況下,如果這個記錄已經上了排它鎖,則會嘗試給這個記錄上共享鎖.
如果有三個以上的併發執行緒,
第一個執行緒上了排它鎖,第二和第三個執行緒,看到該記錄有排他鎖,則嘗試給這個記錄上共享鎖。
一旦第一個執行緒回滾,則第二,第三執行緒擁有共享鎖的同時,都在申請排它鎖.這時候就出現了死鎖.

需要注意的是,假如第一個執行緒提交了,則第二個,第三個執行緒會報重複主鍵的錯誤,但是這時候,第二個,第三個執行緒,還是擁有這個記錄的共享鎖.第二,第三執行緒必須回滾.否則他們擁有的共享鎖不釋放.

回到最開始的問題.
三個執行緒同時insert award_free_firecracker_watch_common表,一個執行緒成功獲取排它鎖,其他兩個執行緒上共享鎖.
等獲取排他鎖的執行緒提交,兩個上共享鎖的執行緒,最後一步 update award_free_firecracker_watch_common表,則產生了死鎖。
他們都是在獲取了共享鎖的同時,申請排它鎖.

參考:
https://dev.mysql.com/doc/refman/5.7/en/innodb-locks-set.html
http://www.cnblogs.com/renolei/p/4673842.html
http://feiyang21687.github.io/Mysql-Transaction/
%20insert%E9%94%81%E6%9C%BA%E5%88%B6/
http://blog.csdn.net/and1kaney/article/details/51214001

來自 “ ITPUB部落格 ” ,連結:http://blog.itpub.net/29254281/viewspace-2133705/,如需轉載,請註明出處,否則將追究法律責任。

相關文章