muduo網路庫學習之EventLoop(六):TcpConnection::send()、shutdown()、handleRead()、handleWrite()

s1mba發表於2013-11-10

首先在EventLoop(五)基礎上,在TcpConnection 建構函式中新增:

 C++ Code 
1
2
3
 
// 通道可寫事件到來的時候,回撥TcpConnection::handleWrite
channel_->setWriteCallback(
    boost::bind(&TcpConnection::handleWrite, this));

多了兩個應用層緩衝區成員:

 C++ Code 
1
2
 
Buffer inputBuffer_;            // 應用層接收緩衝區
Buffer outputBuffer_;           // 應用層傳送緩衝區

在 TcpServer::newConnection() 中再新增:

 C++ Code 
1
 
conn->setWriteCompleteCallback(writeCompleteCallback_);

將TcpConnection::handleRead() 修改為:

 C++ Code 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
 
void TcpConnection::handleRead(Timestamp receiveTime)
{
    loop_->assertInLoopThread();
    int savedErrno = 0;
    ssize_t n = inputBuffer_.readFd(channel_->fd(), &savedErrno);
    if (n > 0)
    {
        messageCallback_(shared_from_this(), &inputBuffer_, receiveTime);
    }
    else if (n == 0)
    {
        handleClose();
    }
    else
    {
        errno = savedErrno;
        LOG_SYSERR << "TcpConnection::handleRead";
        handleError();
    }
}

現在當某個TcpConnection 發生可讀事件,呼叫TcpConnection::handleRead() , 先呼叫inputBuffer_.readFd() 

將核心接收緩衝區資料讀取到inputBuffer_ 中,接著呼叫messageCallback_ , 使用者程式碼可以按訊息界限從

inputBuffer_ 中讀取資料。


使用者程式碼想要傳送資料時,呼叫TcpConnection::send() ,過載了3個版本,都是執行緒安全的,內部最終都是呼叫TcpConnection::sendInLoop()(如果不是在當前IO執行緒呼叫send 時,sendInLoop 會在當前IO執行緒處理doPendingFunctors 時被呼叫)

 C++ Code 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
 
void TcpConnection::sendInLoop(const void *data, size_t len)
{
    loop_->assertInLoopThread();
    ssize_t nwrote = 0;
    size_t remaining = len;
    bool error = false;
    if (state_ == kDisconnected)
    {
        LOG_WARN << "disconnected, give up writing";
        return;
    }
    // if no thing in output queue, try writing directly
    // 通道沒有關注可寫事件並且應用層傳送緩衝區沒有資料,直接write
    if (!channel_->isWriting() && outputBuffer_.readableBytes() == 0)
    {
        nwrote = sockets::write(channel_->fd(), data, len);
        if (nwrote >= 0)
        {
            remaining = len - nwrote;
            // 寫完了,回撥writeCompleteCallback_
            if (remaining == 0 && writeCompleteCallback_)
            {
                loop_->queueInLoop(boost::bind(writeCompleteCallback_, shared_from_this()));
            }
        }
        else // nwrote < 0
        {
            nwrote = 0;
            if (errno != EWOULDBLOCK)
            {
                LOG_SYSERR << "TcpConnection::sendInLoop";
                if (errno == EPIPE) // FIXME: any others?
                {
                    error = true;
                }
            }
        }
    }

    assert(remaining <= len);
    // 沒有錯誤,並且還有未寫完的資料(說明核心傳送緩衝區滿,要將未寫完的資料新增到output buffer中)
    if (!error && remaining > 0)
    {
        LOG_TRACE << "I am going to write more data";
        size_t oldLen = outputBuffer_.readableBytes();
        // 如果超過highWaterMark_(高水位標),回撥highWaterMarkCallback_
        if (oldLen + remaining >= highWaterMark_
                && oldLen < highWaterMark_
                && highWaterMarkCallback_)
        {
            loop_->queueInLoop(boost::bind(highWaterMarkCallback_, shared_from_this(), oldLen + remaining));
        }
        outputBuffer_.append(static_cast<const char *>(data) + nwrote, remaining);
        if (!channel_->isWriting())
        {
            channel_->enableWriting();      // 關注POLLOUT事件
        }
    }
}

即首先嚐試write入核心傳送緩衝區,如果核心傳送緩衝區滿則將未寫完的資料新增到outputBuffer_ 中(注意,只要第一次沒寫完,

下次呼叫send 也會將資料新增到outputBuffer_ 的末尾而不直接write),並關注POLLOUT 事件,當核心傳送緩衝區不為滿,即發生

可寫事件,調TcpConnection::handleWrite() 

 C++ Code 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
 
// 核心傳送緩衝區有空間了,回撥該函式
void TcpConnection::handleWrite()
{
    loop_->assertInLoopThread();
    if (channel_->isWriting())
    {
        ssize_t n = sockets::write(channel_->fd(),
                                   outputBuffer_.peek(),
                                   outputBuffer_.readableBytes());
        if (n > 0)
        {
            outputBuffer_.retrieve(n);
            if (outputBuffer_.readableBytes() == 0)  // 應用層傳送緩衝區已清空
            {
                channel_->disableWriting();     // 停止關注POLLOUT事件,以免出現busy loop
                if (writeCompleteCallback_)     // 回撥writeCompleteCallback_
                {
                    // 應用層傳送緩衝區被清空,就回撥用writeCompleteCallback_
                    loop_->queueInLoop(boost::bind(writeCompleteCallback_, shared_from_this()));
                }
                if (state_ == kDisconnecting)   // 應用層傳送緩衝區已清空並且連線狀態是kDisconnecting, 要關閉連線
                {
                    shutdownInLoop();       // 關閉連線
                }
            }
            else
            {
                LOG_TRACE << "I am going to write more data";
            }
        }
        else
        {
            LOG_SYSERR << "TcpConnection::handleWrite";
            // if (state_ == kDisconnecting)
            // {
            //   shutdownInLoop();
            // }
        }
    }
    else
    {
        LOG_TRACE << "Connection fd = " << channel_->fd()
                  << " is down, no more writing";
    }
}

即從outputBuffer_ 中取出資料寫入核心傳送緩衝區,當然也許此次並不能完全寫入,但只要應用層傳送緩衝區不為空,就一直關注

POLLOUT事件,當核心傳送緩衝區不為滿時觸發再次寫入。


如果output buffer 裡還有待傳送的資料,而程式又想關閉連線(對程式而言,呼叫TcpConnection::send() 之後他就認為資料遲早會發出去),那麼這時候網路庫不能立刻關閉連線,而要等資料傳送完畢,而Muduo TcpConnection 沒有提供close,而只提供shutdown ,這麼做是為了收發資料的完整性。如下所示

 C++ Code 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
 
void DaytimeServer::onConnection(const muduo::net::TcpConnectionPtr &conn)
{
    if (conn->connected())
    {
        conn->send(Timestamp::now().toFormattedString() + ”\n”);
        conn->shutdown(); // 呼叫TcpConnection::shutdown()
    }
}

void TcpConnection::shutdown()
{
    if (state_ == kConnected)
    {
        setState(kDisconnecting);
        // 呼叫TcpConnection::shutdownInLoop()
        loop_->runInLoop(boost::bind(&TcpConnection::shutdownInLoop, this));
    }
}

void TcpConnection::shutdownInLoop()
{
    loop_->assertInLoopThread();
    if (!channel_->isWriting())
    {
        // we are not writing
        socket_->shutdownWrite(); // 呼叫Socket::shutdownWrite()
    }
}

void Socket::shutdownWrite()
{
    sockets::shutdownWrite(sockfd_);
}

void sockets::shutdownWrite(int sockfd)
{
    int ret = ::shutdown(sockfd, SHUT_WR);
    // 檢查錯誤
}

此時如果應用層緩衝區資料還沒發完,即還在關注POLLOUT事件,那麼shutdown() 中只是先設定state_ = kDisconnecting; 而 shutdownInLoop() 中判斷 isWriting() 為true, 故不會執行shutdownWrite(),回顧handleWrite() 函式,當應用層緩衝區資料發完,判斷狀態為kDisconnecting 而且已經disableWriting(),就繼續呼叫
shutdownInLoop() ,此時就會真正關閉寫的這一端。

用shutdown 而不用close 的效果是,如果對方已經傳送了資料,這些資料還“在路上”,那麼muduo 不會漏收這些資料。我們發完了資料,於是shutdownWrite,傳送TCP FIN 分節,對方會讀到0 位元組,然後對方通常會關閉連線(無論shutdownWrite() 還是close()),可讀事件發生呼叫handleRead(),這樣muduo 會讀到0 位元組,呼叫handleClose(),進而呼叫connectionCallback_, 這樣客戶程式碼就知道對方斷開連線了(判斷是否connected()),最後呼叫closeCallback_ (TcpServer::removeConnection())。


那麼muduo 什麼時候真正close socket 呢?在TcpConnection 物件析構的時候。TcpConnection 持有一個Socket 物件,Socket 是一個RAII handler,它的析構函式會close(sockfd_)。TcpConnection 物件生存期參考

測試程式碼:
客戶端 nc 127.0.0.1 8888

 C++ Code 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
 
#include <muduo/net/TcpServer.h>
#include <muduo/net/EventLoop.h>
#include <muduo/net/InetAddress.h>

#include <boost/bind.hpp>

#include <stdio.h>

using namespace muduo;
using namespace muduo::net;

class TestServer
{
public:
    TestServer(EventLoop *loop,
               const InetAddress &listenAddr)
        : loop_(loop),
          server_(loop, listenAddr, "TestServer")
    {
        server_.setConnectionCallback(
            boost::bind(&TestServer::onConnection, this, _1));
        server_.setMessageCallback(
            boost::bind(&TestServer::onMessage, this, _1, _2, _3));

        message1_.resize(100);
        message2_.resize(200);
        std::fill(message1_.begin(), message1_.end(), 'A');
        std::fill(message2_.begin(), message2_.end(), 'B');
    }

    void start()
    {
        server_.start();
    }

private:
    void onConnection(const TcpConnectionPtr &conn)
    {
        if (conn->connected())
        {
            printf("onConnection(): new connection [%s] from %s\n",
                   conn->name().c_str(),
                   conn->peerAddress().toIpPort().c_str());
            conn->send(message1_);
            conn->send(message2_);
            conn->shutdown();
        }
        else
        {
            printf("onConnection(): connection [%s] is down\n",
                   conn->name().c_str());
        }
    }

    void onMessage(const TcpConnectionPtr &conn,
                   Buffer *buf,
                   Timestamp receiveTime)
    {
        muduo::string msg(buf->retrieveAllAsString());
        printf("onMessage(): received %zd bytes from connection [%s] at %s\n",
               msg.size(),
               conn->name().c_str(),
               receiveTime.toFormattedString().c_str());

        conn->send(msg);
    }

    EventLoop *loop_;
    TcpServer server_;

    muduo::string message1_;
    muduo::string message2_;
};


int main()
{
    printf("main(): pid = %d\n", getpid());

    InetAddress listenAddr(8888);
    EventLoop loop;

    TestServer server(&loop, listenAddr);
    server.start();

    loop.loop();
}

執行結果如下:
simba@ubuntu:~/Documents/build/debug/bin$ ./reactor_test12
20131110 04:47:24.913096Z  2330 TRACE IgnoreSigPipe Ignore SIGPIPE - EventLoop.cc:51
main(): pid = 2330
20131110 04:47:24.916700Z  2330 TRACE updateChannel fd = 4 events = 3 - EPollPoller.cc:104
20131110 04:47:24.917170Z  2330 TRACE EventLoop EventLoop created 0xBFCB2CE4 in thread 2330 - EventLoop.cc:76
20131110 04:47:24.917487Z  2330 TRACE updateChannel fd = 5 events = 3 - EPollPoller.cc:104
20131110 04:47:24.918344Z  2330 TRACE updateChannel fd = 6 events = 3 - EPollPoller.cc:104
20131110 04:47:24.918942Z  2330 TRACE loop EventLoop 0xBFCB2CE4 start looping - EventLoop.cc:108
20131110 04:47:26.868111Z  2330 TRACE poll 1 events happended - EPollPoller.cc:65
20131110 04:47:26.868584Z  2330 TRACE printActiveChannels {6: IN }  - EventLoop.cc:271
20131110 04:47:26.868688Z  2330 INFO  TcpServer::newConnection [TestServer] - new connection [TestServer:0.0.0.0:8888#1] from 127.0.0.1:54898 - TcpServer.cc:93
20131110 04:47:26.868831Z  2330 DEBUG TcpConnection TcpConnection::ctor[TestServer:0.0.0.0:8888#1] at 0x84AE920 fd=8 - TcpConnection.cc:65
20131110 04:47:26.868847Z  2330 TRACE newConnection [1] usecount=1 - TcpServer.cc:111
20131110 04:47:26.868894Z  2330 TRACE newConnection [2] usecount=2 - TcpServer.cc:113
20131110 04:47:26.868931Z  2330 TRACE connectEstablished [3] usecount=6 - TcpConnection.cc:238
20131110 04:47:26.868941Z  2330 TRACE updateChannel fd = 8 events = 3 - EPollPoller.cc:104
onConnection(): new connection [TestServer:0.0.0.0:8888#1] from 127.0.0.1:54898
20131110 04:47:26.869098Z  2330 TRACE connectEstablished [4] usecount=6 - TcpConnection.cc:243
20131110 04:47:26.869109Z  2330 TRACE newConnection [5] usecount=2 - TcpServer.cc:123
20131110 04:47:26.869800Z  2330 TRACE poll 1 events happended - EPollPoller.cc:65
20131110 04:47:26.869831Z  2330 TRACE printActiveChannels {8: IN HUP }  - EventLoop.cc:271
20131110 04:47:26.869841Z  2330 TRACE handleEvent [6] usecount=2 - Channel.cc:67
20131110 04:47:26.869899Z  2330 TRACE handleClose fd = 8 state = 3 - TcpConnection.cc:369
20131110 04:47:26.869909Z  2330 TRACE updateChannel fd = 8 events = 0 - EPollPoller.cc:104
onConnection(): connection [TestServer:0.0.0.0:8888#1] is down
20131110 04:47:26.869925Z  2330 TRACE handleClose [7] usecount=3 - TcpConnection.cc:377
20131110 04:47:26.869935Z  2330 INFO  TcpServer::removeConnectionInLoop [TestServer] - connection TestServer:0.0.0.0:8888#1 - TcpServer.cc:154
20131110 04:47:26.869943Z  2330 TRACE removeConnectionInLoop [8] usecount=6 - TcpServer.cc:158
20131110 04:47:26.869978Z  2330 TRACE removeConnectionInLoop [9] usecount=5 - TcpServer.cc:160
20131110 04:47:26.869992Z  2330 TRACE removeConnectionInLoop [10] usecount=6 - TcpServer.cc:171
20131110 04:47:26.870000Z  2330 TRACE handleClose [11] usecount=3 - TcpConnection.cc:380
20131110 04:47:26.870007Z  2330 TRACE handleEvent [12] usecount=2 - Channel.cc:69
20131110 04:47:26.870015Z  2330 TRACE removeChannel fd = 8 - EPollPoller.cc:147
20131110 04:47:26.870053Z  2330 DEBUG ~TcpConnection TcpConnection::dtor[TestServer:0.0.0.0:8888#1] at 0x84AE920 fd=8 - TcpConnection.cc:72
20131110 04:47:36.880508Z  2330 TRACE poll  nothing happended - EPollPoller.cc:74


程式中一旦連線建立,呼叫onConnection(),send(message1), send(message2),然後立馬shutdown()。由前面分析可知會一直等到outputBuffer_ 資料全部寫到核心傳送緩衝區才會真正關閉寫端,客戶端讀到資料後最後read 返回0,客戶端close導致服務端最終removeConnection。可以看到在handleEvent()處理完畢後TcpConnection 才會析構,對照 EventLoop(五)可以理解。


WriteCompleteCallback_ & highWaterMarkCallback_:

如果我們會向一個連線傳送send()大流量的資料,傳送頻率不能太快,因為如果對等方接收不及時,則核心傳送緩衝區會堆積資料,根據前面的分析,我們會將資料新增到outputBuffer_,導致outputBuffer_ 增長太快,對此可以關注WriteCompleteCallback_ ,當它被呼叫時表示outputBuffer_ 已經被清空,此時再次send(),否則outputBuffer_ 可能一直增長直到撐爆。
從這個角度看,可以把WriteCompleteCallback_ 當作是“低水位標”回撥函式,相應地,highWaterMarkCallback_ 可以當作是”高水位標“ 回撥函式,即如果對等方接收不及時,outputBuffer_ 會一直增大,當增長到highWaterMark_ (具體數值)時,回撥highWaterMarkCallback_  函式,很可能在函式內主動shutdown。

TcpConnection 中 boost::any context_;  // 繫結一個未知型別的上下文物件比如HttpContext

可變型別解決方案
void*. 這種方法不是型別安全的
boost::any
任意型別的型別安全儲存以及安全的取回
在標準庫容器中存放不同型別的方法,比如說vector<boost::any>

下面的程式會不斷地傳送不同的字元資料,類似chargen 協議(DDos):
 C++ Code 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
 
#include <muduo/net/TcpServer.h>
#include <muduo/net/EventLoop.h>
#include <muduo/net/InetAddress.h>

#include <boost/bind.hpp>

#include <stdio.h>

using namespace muduo;
using namespace muduo::net;

class TestServer
{
public:
    TestServer(EventLoop *loop,
               const InetAddress &listenAddr)
        : loop_(loop),
          server_(loop, listenAddr, "TestServer")
    {
        server_.setConnectionCallback(
            boost::bind(&TestServer::onConnection, this, _1));
        server_.setMessageCallback(
            boost::bind(&TestServer::onMessage, this, _1, _2, _3));
        server_.setWriteCompleteCallback(
            boost::bind(&TestServer::onWriteComplete, this, _1));

        // 生成資料
        string line;
        for (int i = 33; i < 127; ++i)
        {
            line.push_back(char(i));
        }
        line += line;

        for (size_t i = 0; i < 127 - 33; ++i)
        {
            message_ += line.substr(i, 72) + '\n';
        }
    }

    void start()
    {
        server_.start();
    }

private:
    void onConnection(const TcpConnectionPtr &conn)
    {
        if (conn->connected())
        {
            printf("onConnection(): new connection [%s] from %s\n",
                   conn->name().c_str(),
                   conn->peerAddress().toIpPort().c_str());

            conn->setTcpNoDelay(true);
            conn->send(message_);
        }
        else
        {
            printf("onConnection(): connection [%s] is down\n",
                   conn->name().c_str());
        }
    }

    void onMessage(const TcpConnectionPtr &conn,
                   Buffer *buf,
                   Timestamp receiveTime)
    {
        muduo::string msg(buf->retrieveAllAsString());
        printf("onMessage(): received %zd bytes from connection [%s] at %s\n",
               msg.size(),
               conn->name().c_str(),
               receiveTime.toFormattedString().c_str());

        conn->send(msg);
    }

    void onWriteComplete(const TcpConnectionPtr &conn)
    {
        conn->send(message_);
    }

    EventLoop *loop_;
    TcpServer server_;

    muduo::string message_;
};


int main()
{
    printf("main(): pid = %d\n", getpid());

    InetAddress listenAddr(8888);
    EventLoop loop;

    TestServer server(&loop, listenAddr);
    server.start();

    loop.loop();
}

程式中一旦連線建立就開始send,當outputBuffer_ 資料全部拷貝到核心傳送緩衝區後,回撥OnWriteComplete(), 繼續send,類似大流量的ddos攻擊。客戶端 nc 127.0.0.1 8888 > aa 執行後立馬ctrl+c 掉,但此時aa檔案已經是很大的了,檔案的內容部分如下:
simba@ubuntu:~$ ls -lh aa
-rw-rw-r-- 1 simba simba 28M Nov  9 21:01 aa


ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*
BCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+
CDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,
DEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-
EFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-.
FGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./
GHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0
HIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./01
IJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./012
JKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123
KLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./01234
LMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./012345
MNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456
NOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./01234567
OPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./012345678
PQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789
QRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:
RSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;
STUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<
TUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=
UVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>
VWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?
WXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@
XYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@A
YZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@AB
Z[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABC
[\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCD
\]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDE
]^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEF
^_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFG
_`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGH
`abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHI
abcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJ
bcdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJK
cdefghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKL
defghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLM
efghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMN
fghijklmnopqrstuvwxyz{|}~!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNO


參考:
《UNP》
muduo manual.pdf
《linux 多執行緒伺服器程式設計:使用muduo c++網路庫》

相關文章