當Kafka啟動時,會啟動這個SocketServer來接收客戶端的連線,處理客戶端請求,傳送響應。
這個類的註釋說明了這個socket server的結構
/**
* An NIO socket server. The threading model is
* 1 Acceptor thread that handles new connections
* N Processor threads that each have their own selector and read requests from sockets
* M Handler threads that handle requests and produce responses back to the processor threads for writing.
*/
即
其中:
其中使用的類主要包括:
Acceptor : 它是一個SocketServer, 接受新的連線,並且分配連線給Processor
Processor: 讀取請求,傳送響應
Handler: 處理請求,產生響應。這裡的Handler由kafka.server.RequestHandler實現。
RequestChannel: 它包括了一個request queue 和 一個 response queue. 是Handler和Processsor互動時使用的佇列。Request由Processor放入RequestChannel, 由Handler取出,然後把Response放回RequestChannel.
Acceptor在接受連線後,就把相當的SocketChannel設成非阻塞模式。因此Processor對這些SocketChannel的讀寫都是使用Selector,採用非阻塞的處理模式。
問題:
(1) Acceptor是如何把新來的連線分配給對應的Processor,這個演算法是什麼?是round robin嗎?
(2) Processor應該把對應的SocketChannel在自己的Selector上如何註冊?
(3) Processor如何讀取請求?
/* * Process reads from ready sockets */ def read(key: SelectionKey) { val socketChannel = channelFor(key) //獲取可讀的SocketChannel var receive = key.attachment.asInstanceOf[Receive] //獲取attach到SelectionKey的Receive物件 if(key.attachment == null) { //如果attachment是空,說明這是第一次讀,就新建一個Receive物件,attach到這個SocketChannel的SelectionKey上。如果不是空,說明之前已經從中讀了一些資料,只是沒讀完。 receive = new BoundedByteBufferReceive(maxRequestSize) key.attach(receive) } val read = receive.readFrom(socketChannel) //從SocketChannel中讀資料 val address = socketChannel.socket.getRemoteSocketAddress(); trace(read + " bytes read from " + address) if(read < 0) { //如果讀的資料數小於0,就關閉socket連線。實際上從BoundedByteBufferReceive的實現來看,read的值不會小於0 close(key) } else if(receive.complete) {//如果讀完了,就構造request,傳送給requestChannel val req = RequestChannel.Request(processor = id, requestKey = key, buffer = receive.buffer, startTimeMs = time.milliseconds, remoteAddress = address) requestChannel.sendRequest(req) key.attach(null) //取消attach的Receive物件 // explicitly reset interest ops to not READ, no need to wake up the selector just yet key.interestOps(key.interestOps & (~SelectionKey.OP_READ))//顯示地把這個SocketChannel設為非OP_READ,等到Response發給這個SocketChannel以後,它會被再設為OP_READ,以繼續處理來自這個SocketChannel的請求。 } else {//如果沒有讀完,就把這個SocketChannel註冊為OP_READ,然後wakeup對應的selector,繼續從SocketChannel中讀資料。所以下一次再處理這個SocketChannel時,attach到它的SelectionKey的Receive物件就不是空了。 // more reading to be done trace("Did not finish reading, registering for read again on connection " + socketChannel.socket.getRemoteSocketAddress()) key.interestOps(SelectionKey.OP_READ) wakeup() } }
那麼BoundedByteBufferReceive是如何知道一個請求讀沒讀完呢?
原來每個Request的前4個位元組標識了這個Request有多長,BoundedByteBufferReceive從SocketChannel中讀取前4個位元組,轉換成整形,以這個整數為大小構造一個ByteBuffer,如果這個ByteBuffer沒有寫滿,就說明請求的內容還沒有讀完。receive.complete就不被設為true,否則就說明這個Request已經從channel中完全讀出。
if(!contentBuffer.hasRemaining) { contentBuffer.rewind() complete = true }
Request Header (all single non-multi requests begin with this)
0
1
2
3
0
1
2
3
4
5
6
7
8
9
0
1
2
3
4
5
6
7
8
9
0
1
2
3
4
5
6
7
8
9
0
1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| REQUEST_LENGTH |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| REQUEST_TYPE | TOPIC_LENGTH |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
/ /
/ TOPIC (variable length) /
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PARTITION |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
(4) Processor如何接收Handler產生的response?