篤行雜記之ZookeeperSessionTimeOut分析
0.前言
本文為篤行日常工作記錄,爛筆頭系列。
原始碼前面,了無祕密 — by 侯傑
近期的一個C++專案裡使用了Zookeeper做服務發現,期間遇到了SessionTimeOut問題的困擾,明明通過zookeeper c client設定了超時時間,但無效。
請原諒我一開始對zookeeper不熟悉。最終通過分析原始碼瞭解到SessionTimeOut最終的確定是一個協商的過程,而不是簡單的配置生效。
在這裡記錄下Session超時時間的有關分析,基於zookeeper 3.4.8
1.zookeeper client SessionTimeOut
專案中使用的是 C client,通過zookeer_init
建立zk session,呼叫了zookeeper_init其實就開始了建立連結到ZK叢集的過程,這裡設定的recv_timeout 為客戶端所期望的session超時時間,單位為毫秒。
ZOOAPI zhandle_t *zookeeper_init(const char *host, watcher_fn fn,
int recv_timeout, const clientid_t *clientid, void *context, int flags);
連線成功之後客戶端發起握手協議,可以看到之前設定的recv_timeout隨握手協議一起傳送給服務端,zookeeper.c#L1485。
static int prime_connection(zhandle_t *zh)
{
int rc;
/*this is the size of buffer to serialize req into*/
char buffer_req[HANDSHAKE_REQ_SIZE];
int len = sizeof(buffer_req);
int hlen = 0;
struct connect_req req;
req.protocolVersion = 0;
req.sessionId = zh->seen_rw_server_before ? zh->client_id.client_id : 0;
req.passwd_len = sizeof(req.passwd);
memcpy(req.passwd, zh->client_id.passwd, sizeof(zh->client_id.passwd));
req.timeOut = zh->recv_timeout; <-這裡設定timeOut
req.lastZxidSeen = zh->last_zxid;
req.readOnly = zh->allow_read_only;
hlen = htonl(len);
/* We are running fast and loose here, but this string should fit in the initial buffer! */
rc=zookeeper_send(zh->fd, &hlen, sizeof(len));
serialize_prime_connect(&req, buffer_req);
rc=rc<0 ? rc : zookeeper_send(zh->fd, buffer_req, len);
if (rc<0) {
return handle_socket_error_msg(zh, __LINE__, ZCONNECTIONLOSS,
"failed to send a handshake packet: %s", strerror(errno));
}
再來看看處理握手協議Resp的邏輯 zookeeper.c L1767
static int check_events(zhandle_t *zh, int events)
{
if (zh->fd == -1)
return ZINVALIDSTATE;
……
……
……
deserialize_prime_response(&zh->primer_storage, zh->primer_buffer.buffer);
/* We are processing the primer_buffer, so we need to finish
* the connection handshake */
oldid = zh->client_id.client_id;
newid = zh->primer_storage.sessionId;
if (oldid != 0 && oldid != newid) {
zh->state = ZOO_EXPIRED_SESSION_STATE;
errno = ESTALE;
return handle_socket_error_msg(zh,__LINE__,ZSESSIONEXPIRED,
"sessionId=%#llx has expired.",oldid);
} else {
zh->recv_timeout = zh->primer_storage.timeOut; //設定為Resp的Timeout
zh->client_id.client_id = newid;
}
至此可以發現,最終客戶端的SessionTimeOut時間實際是經過服務端下發之後的,並不一定是最先設定的。
2.Zookeeper Server SessionTimeOut
2.1協商客戶端上報的SessionTimeOut
來看看服務端握手的處理邏輯ZooKeeperServer.java#L876。
public void processConnectRequest(ServerCnxn cnxn, ByteBuffer incomingBuffer) throws IOException {
BinaryInputArchive bia = BinaryInputArchive.getArchive(new ByteBufferInputStream(incomingBuffer));
ConnectRequest connReq = new ConnectRequest();
connReq.deserialize(bia, "connect");
……
……
……
//根據客戶端上報的timeout和服務端自身的minSessionTimeOut。
//如果上報的timeout小於minSessionTimeOut則 設定timeout為minSessionTimeOut.
//如果上報的timeout大於maxSessionTimeOut則 設定timeout為maxSessionTimeOut.
//如果介於兩則之間,則以上報的時間為準。
int sessionTimeout = connReq.getTimeOut();
byte passwd[] = connReq.getPasswd();
int minSessionTimeout = getMinSessionTimeout();
if (sessionTimeout < minSessionTimeout) {
sessionTimeout = minSessionTimeout;
}
int maxSessionTimeout = getMaxSessionTimeout();
if (sessionTimeout > maxSessionTimeout) {
sessionTimeout = maxSessionTimeout;
}
cnxn.setSessionTimeout(sessionTimeout);
……
……
……
}
可以一句話概括,客戶端上報的期望timeout一定要在服務端設定的上下界之間,如果越過邊界,則以邊界為準。
2.2 服務端MinSessionTimeOut和MaxSessionTimeOut的確定
繼續看ZooKeeperServer.java#L104和ZooKeeperServer.java#L791
public static final int DEFAULT_TICK_TIME = 3000;
protected int tickTime = DEFAULT_TICK_TIME;
/** value of -1 indicates unset, use default */
protected int minSessionTimeout = -1;
/** value of -1 indicates unset, use default */
protected int maxSessionTimeout = -1;
protected SessionTracker sessionTracker;
tickTime為3000毫秒,minSessionTimeOut和maxSessionTimeOut預設值為-1
public int getTickTime() {
return tickTime;
}
public void setTickTime(int tickTime) {
LOG.info("tickTime set to " + tickTime);
this.tickTime = tickTime;
}
public int getMinSessionTimeout() {
return minSessionTimeout == -1 ? tickTime * 2 : minSessionTimeout;
//如果minSessionTimeOut為預設值這設定minSessionTimeOut為2倍tickTime
}
public void setMinSessionTimeout(int min) {
LOG.info("minSessionTimeout set to " + min);
this.minSessionTimeout = min;
}
public int getMaxSessionTimeout() {
return maxSessionTimeout == -1 ? tickTime * 20 : maxSessionTimeout;
//如果maxSessionTimeout為預設值則設定maxSessionTimeout為20倍tickTime
}
public void setMaxSessionTimeout(int max) {
LOG.info("maxSessionTimeout set to " + max);
this.maxSessionTimeout = max;
}
可以知道minSessionTimeOut和maxSessionTimeOut在預設的時候則跟tickTime有關,分別為2和20倍tickTime,繼續分析。ZooKeeperServer.java#L160
public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime,
int minSessionTimeout, int maxSessionTimeout,
DataTreeBuilder treeBuilder, ZKDatabase zkDb) {
serverStats = new ServerStats(this);
this.txnLogFactory = txnLogFactory;
this.zkDb = zkDb;
this.tickTime = tickTime;
this.minSessionTimeout = minSessionTimeout;
this.maxSessionTimeout = maxSessionTimeout;
LOG.info("Created server with tickTime " + tickTime
+ " minSessionTimeout " + getMinSessionTimeout()
+ " maxSessionTimeout " + getMaxSessionTimeout()
+ " datadir " + txnLogFactory.getDataDir()
+ " snapdir " + txnLogFactory.getSnapDir());
}
tickTime、minSessionTimeOut、maxSessionTimeOut實際建構函式傳入,當然還有一個無參建構函式以及一些setter和getter可以設定這幾個引數。
繼續分析ZooKeeperServerMain.java#L94
public void runFromConfig(ServerConfig config) throws IOException {
LOG.info("Starting server");
FileTxnSnapLog txnLog = null;
try {
// Note that this thread isn`t going to be doing anything else,
// so rather than spawning another thread, we will just call
// run() in this thread.
// create a file logger url from the command line args
ZooKeeperServer zkServer = new ZooKeeperServer();
txnLog = new FileTxnSnapLog(new File(config.dataLogDir), new File(
config.dataDir));
zkServer.setTxnLogFactory(txnLog);
zkServer.setTickTime(config.tickTime);
//我們可以發現實際執行的幾個引數除了預設值以外,可以通過配置檔案來配置生效。
zkServer.setMinSessionTimeout(config.minSessionTimeout);
zkServer.setMaxSessionTimeout(config.maxSessionTimeout);
cnxnFactory = ServerCnxnFactory.createFactory();
cnxnFactory.configure(config.getClientPortAddress(),
config.getMaxClientCnxns());
cnxnFactory.startup(zkServer);
cnxnFactory.join();
if (zkServer.isRunning()) {
zkServer.shutdown();
}
} catch (InterruptedException e) {
// warn, but generally this is ok
LOG.warn("Server interrupted", e);
} finally {
if (txnLog != null) {
txnLog.close();
}
}
}
到此問題就明瞭了,我們可以通過配置來修改SessionTimeOut,預設配置檔案只配置了tickTime,如下。
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/tmp/zookeeper
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
3.總結
經過原始碼分析,得出SessionTimeOut的協商如下:
- 情況1: 配置檔案配置了maxSessionTimeOut和minSessionTimeOut
最終SessionTimeOut,必須在minSessionTimeOut和maxSessionTimeOut區間裡,如果跨越上下界,則以跨越的上屆或下界為準。
- 情況2:配置檔案沒有配置maxSessionTimeOut和minSessionTimeOut
maxSessionTimeout沒配置則 maxSessionTimeOut設定為 20 * tickTime
minSessionTimeOut沒配置則 minSessionTimeOut設定為 2 * tickTime
也就是預設情況下, SessionTimeOut的合法範圍為 4秒~40秒,預設配置中tickTime為2秒。
如果tickTime也沒配置,那麼tickTime預設為3秒。
遇到問題從原始碼分析一定是最好的,能使得理解更深入記憶更深刻。
最後 ending…如有不足請指點,亦可留言或聯絡 fobcrackgp@163.com.
相關文章
- ThinkJS作者李成銀:擇善而從之,篤行致遠(圖靈訪談)JS圖靈
- robot framework學習筆記之九-雜記Framework筆記
- JavaScript 演算法之複雜度分析JavaScript演算法複雜度
- pwn雜項之linux命令執行Linux
- 小前端學演算法之複雜度分析前端演算法複雜度
- Java雜記3—流程控制之條件Java
- 分散式系統之CAP理論雜記分散式
- Effective c++(筆記) 之 雜項討論C++筆記
- 如何進行演算法的複雜度分析?演算法複雜度
- 執行緒池之ThreadPoolExecutor執行緒池原始碼分析筆記執行緒thread原始碼筆記
- 執行緒池之ScheduledThreadPoolExecutor執行緒池原始碼分析筆記執行緒thread原始碼筆記
- 複雜度分析複雜度
- Java集合原始碼分析之基礎(二):雜湊表Java原始碼
- html雜記HTML
- webserver 雜記WebServer
- oracle雜記Oracle
- cpp雜記
- PostgreSQL-並行雜湊JOIN分析查詢效能爆炸SQL並行
- 記憶體安全週報|1019日積月累,篤學不倦記憶體
- Spring AMQP雜記之Spring實現簡述SpringMQ
- 資料分析雜談
- redis之雜湊Redis
- Nodejs 雜記NodeJS
- HTTPS雜記HTTP
- 網路--------雜記
- 雜題記錄
- 演算法分析__時間複雜度的五個記號演算法時間複雜度
- “犇放計劃”全新升級,用友專業服務生態篤行致遠
- nginx聽課隨記雜記Nginx
- 雜記-本週工作記錄
- ceph-pg雜湊分析
- Shell排序複雜度分析排序複雜度
- 前端模組化雜記前端
- React-Router 雜記React
- React-setState雜記React
- Linux雜記1Linux
- Linux雜記4Linux
- Linux雜記5Linux