Mysql 5.7中資料量更改統計資料收集的邏輯

今天一個朋友在問Mysql資料修改後什麼時候收集統計資料，我就簡單的找了一下原始碼，現總結如下。如有錯誤請指出，因為我只是簡單做了一下除錯。

一、持久化(PERSISTENT))與非持久化統計資料(TRANSIENT)

Mysql統計資料分為持久化和非持久化

• 持久化統計資料
  儲存在mysql.innodb_index_stats和mysql.innodb_table_stats中
• 非持久化統計資料
  儲存在information_schema.indexes和information_schema.tables中

前者是innodb表後者是memory表。他們受到引數innodb_stats_persistent的控制，預設為ON。
關於這裡還參考官方文件：

• 14.6.12 Configuring Optimizer Statistics for InnoDB

二、持久化統計資料的更改方式。

我在看更改邏輯的時候，發現非持久化統計資料是直接更新的或者說是同步更新的。但是持久化統計資料應該是非同步更新的，看起來是典型的生產者消費者模型。Mysql中有一個專門的後臺執行緒來負責收集統計資料如下：

mysql> select name,thread_id from performance_schema.threads where NAME like '%dict_stats_thread%' \G
*************************** 1. row ***************************
     name: thread/innodb/dict_stats_thread
thread_id: 25 

這個執行緒會從一個vcoter陣列中取出需要更新統計資料的表的table_id然後更新統計資料，而生產者就是我們的當前執行緒他透過邏輯判斷將需要更新統計資料表的table_id加入到這個陣列，什麼邏輯呢？我們稍後在討論。現在來看一個簡單的圖：

三、更改邏輯

其實更改邏輯還是相當簡單基本就是兩個函式就總結了，他們是row_update_statistics_if_needed函式和dict_stats_process_entry_from_recalc_pool函式，但是我不知道有沒有漏掉的。

持久化邏輯(PERSISTENT)

• 1、資料修改超過10% ，入口函式row_update_statistics_if_needed邏輯如下：

 if (dict_stats_is_persistent_enabled(table)) {
        if (counter > n_rows / 10 /* 10% */
            && dict_stats_auto_recalc_is_enabled(table)) {

            dict_stats_recalc_pool_add(table);
            table->stat_modified_counter = 0;
        }
        return;
    } 

• 2、兩次申請統計資料收集超過10S，入口函式dict_stats_process_entry_from_recalc_pool，邏輯如下：

if (ut_difftime(ut_time(), table->stats_last_recalc)
        < MIN_RECALC_INTERVAL) {

        /* Stats were (re)calculated not long ago. To avoid
        too frequent stats updates we put back the table on
        the auto recalc list and do nothing. */

        dict_stats_recalc_pool_add(table);

    } else {

        dict_stats_update(table, DICT_STATS_RECALC_PERSISTENT);
    } 

其中MIN_RECALC_INTERVAL 為宏定義

#define MIN_RECALC_INTERVAL 10 /* seconds */ 

非持久化邏輯(TRANSIENT)

- 修改超過1/16進行收集，入口函式row_update_statistics_if_needed，邏輯如下：

if (counter > 16 + n_rows / 16 /* 6.25% */) {

        ut_ad(!mutex_own(&dict_sys->mutex));
        /* this will reset table->stat_modified_counter to 0 */
        dict_stats_update(table, DICT_STATS_RECALC_TRANSIENT);
    } 

四、總結

這裡我只是對觸發邏輯進行了簡單的梳理，但是很多其他引數並沒有進行介紹，還是參考官方文件：

14.6.12 Configuring Optimizer Statistics for InnoDB 

其次我以前寫的一篇文章可以參考如下：

http://blog.itpub.net/7728585/viewspace-1795516/ 

五、備用棧幀

這是留個我自己除錯用的

• 當前執行緒棧幀：

#0  dict_stats_recalc_pool_add (table=0x7fff1805a790) at /root/mysql5.7.14/percona-server-5.7.14-7/storage/innobase/dict/dict0stats_bg.cc:129
#1  0x0000000001b0f4e3 in row_update_statistics_if_needed (table=0x7fff1805a790) at /root/mysql5.7.14/percona-server-5.7.14-7/storage/innobase/row/row0mysql.cc:1206
#2  0x0000000001b10c34 in row_insert_for_mysql_using_ins_graph (mysql_rec=0x7fff180479a0 "\375\036", prebuilt=0x7fff18098c10)
    at /root/mysql5.7.14/percona-server-5.7.14-7/storage/innobase/row/row0mysql.cc:1831
#3  0x0000000001b10cbf in row_insert_for_mysql (mysql_rec=0x7fff180479a0 "\375\036", prebuilt=0x7fff18098c10)
    at /root/mysql5.7.14/percona-server-5.7.14-7/storage/innobase/row/row0mysql.cc:1857
#4  0x00000000019b04dd in ha_innobase::write_row (this=0x7fff18047460, record=0x7fff180479a0 "\375\036")
    at /root/mysql5.7.14/percona-server-5.7.14-7/storage/innobase/handler/ha_innodb.cc:7923
#5  0x0000000000f732af in handler::ha_write_row (this=0x7fff18047460, buf=0x7fff180479a0 "\375\036") at /root/mysql5.7.14/percona-server-5.7.14-7/sql/handler.cc:8228
#6  0x00000000017d1764 in write_record (thd=0x7fff18000b70, table=0x7fff18036420, info=0x7ffff0255a30, update=0x7ffff02559b0)
    at /root/mysql5.7.14/percona-server-5.7.14-7/sql/sql_insert.cc:1864
#7  0x00000000017ce909 in Sql_cmd_insert::mysql_insert (this=0x7fff18006b00, thd=0x7fff18000b70, table_list=0x7fff18006570)
    at /root/mysql5.7.14/percona-server-5.7.14-7/sql/sql_insert.cc:780
#8  0x00000000017d5349 in Sql_cmd_insert::execute (this=0x7fff18006b00, thd=0x7fff18000b70) at /root/mysql5.7.14/percona-server-5.7.14-7/sql/sql_insert.cc:3100
#9  0x00000000015a773e in mysql_execute_command (thd=0x7fff18000b70, first_level=true) at /root/mysql5.7.14/percona-server-5.7.14-7/sql/sql_parse.cc:3719
#10 0x00000000015adcae in mysql_parse (thd=0x7fff18000b70, parser_state=0x7ffff0257600) at /root/mysql5.7.14/percona-server-5.7.14-7/sql/sql_parse.cc:5836
#11 0x00000000015a1b6d in dispatch_command (thd=0x7fff18000b70, com_data=0x7ffff0257d70, command=COM_QUERY)
    at /root/mysql5.7.14/percona-server-5.7.14-7/sql/sql_parse.cc:1447
#12 0x00000000015a099e in do_command (thd=0x7fff18000b70) at /root/mysql5.7.14/percona-server-5.7.14-7/sql/sql_parse.cc:1010
#13 0x00000000016e28f0 in handle_connection (arg=0x68d7220) at /root/mysql5.7.14/percona-server-5.7.14-7/sql/conn_handler/connection_handler_per_thread.cc:312
#14 0x0000000001d7a514 in pfs_spawn_thread (arg=0x37fcbc0) at /root/mysql5.7.14/percona-server-5.7.14-7/storage/perfschema/pfs.cc:2188
#15 0x0000003f74807aa1 in start_thread () from /lib64/libpthread.so.0
#16 0x0000003f740e8bcd in clone () from /lib64/libc.so.6 

• 統計資料收集執行緒棧幀：

#0  dict_stats_update (table=0x7fff1805a790, stats_upd_option=DICT_STATS_RECALC_PERSISTENT)
    at /root/mysql5.7.14/percona-server-5.7.14-7/storage/innobase/dict/dict0stats.cc:3052
#1  0x0000000001cd0362 in dict_stats_process_entry_from_recalc_pool () at /root/mysql5.7.14/percona-server-5.7.14-7/storage/innobase/dict/dict0stats_bg.cc:349
#2  0x0000000001cd05a9 in dict_stats_thread (arg=0x0) at /root/mysql5.7.14/percona-server-5.7.14-7/storage/innobase/dict/dict0stats_bg.cc:438
#3  0x0000003f74807aa1 in start_thread () from /lib64/libpthread.so.0
#4  0x0000003f740e8bcd in clone () from /lib64/libc.so.6 

作者微信：