HttpWebRequest多執行緒效能問題，請求超時的錯誤

轉自：http://hi.baidu.com/ju_feng/blog/item/b1c41dbf09ad9e0119d81fb0.html

透過Netstat –abn | find “:443” 如果有兩個Time_Wait 和CLOSE_WAIT狀態的網路連線，則該服務就不可用

解決方法：

1 HttpWebRequest的預設連線只有兩個，因此在多執行緒併發的情況下只有兩個併發請求

可以透過 System.Net.ServicePointManager.DefaultConnectionLimit = 20

設定網路HttpWebRequest的網路連線池的個數

2 HttpWebRequest 的請求因為網路問題導致連線沒有被釋放則會佔用連線池中的連線個數，導致並非連線數量減少

可以透過如下方法來修改此問題 ：

2.1 給ServicePointManager設定KeepLive檢測，在網路沒有響應的情況下關閉該連線

System.Net.ServicePointManager.SetTcpKeepAlive(true, _keepLiveTime, _intervalTime);

3 定時檢測HttpWebRequest的ServicePoint的健康狀況

//ServicePoint point = webReq.ServicePoint;

//string connectionsGroupName = webReq.ConnectionGroupName;

//if (point != null)

//{

// if (point. > 15)

// point.CloseConnectionGroup(connectionsGroupName);

//}

4 使用HttpWebRequest 一定要保證GetRequestStream和GetResponse物件關閉，否則容易造成連線處於CLOSE_WAIT狀態

using (Stream stream = webReq.GetRequestStream())

{

stream.Write(buffer, 0, buffer.Length);

}

HttpWebResponse response = null;

try

{

response = webReq.GetResponse() as HttpWebResponse;

}

finally

{

try

{

if (response != null)

response.Close();

//ServicePoint point = webReq.ServicePoint;

//string connectionsGroupName = webReq.ConnectionGroupName;

//if (point != null)

//{

// if (point. > 15)

// point.CloseConnectionGroup(connectionsGroupName);

//}

}

catch

{

}

}

I am wondering whether there is a way to handle broken (or better: closed) TCP connections that reside in the connection pool (ServicePoint) used by a HttpWebRequest instance. What I have seen on Windows 2003 Server systems is that under certain circumstances, all sockets used to connect to a web service are in CLOSE_WAIT state and all subsequent HttpWebRequests fail with a timeout, no new connections to the web service were opened. Looking at the problem with WireShark, I couldn't see any HTTP traffic even though the application kept on making requests that all ended in a timeout.

I have tried to reproduce this behaviour on a development system by writing a simple HTTP server that closes the socket after it has sent a proper HTTP 1.1 response. If you do this, you will see the same behavior that I have described above. Of course, HTTP 1.1 assumes that by default connections are kept alive unless the server explicitly sends a "Connection: Close" header. Thus, the client-side would assume that the connection should be kept alive although the server (that does not function according to HTTP 1.1) already closed it. What stuns me is that the HttpWebRequest, or more specifically the ServicePoint does not recover from such a situation (on Windows Vista/2003 Server the client-side sockets stay in CLOSE_WAIT until they're killed by the OS, on another development machine running the Windows 7 RC they're killed by the OS rather quickly and are just gone, no new ones will be created in either case).

I have tried various things on the client-side, so far I could not solve this problem (except by disabling keep alive in each HttpWebRequest by setting the "KeepAlive" property to "false"). The following list compiles what I've tried to recover from the situation described above:

- Various combinations of using() blocks to dispose all resources (WebResponse object, stream retrieved by GetResponseStream(), StreamReader that was used to read from the stream), as well as using the WebResponse.Close() method call that do everything required from my understanding of the MSDN documentation.
- Calling CloseConnectionGroup() on the ServicePoint used by the HttpWebRequest to somehow "kill" the connections
- Temporarily disabling keep alives to recover
- Increasing the number of connections in the pool of the ServicePoint so that it doesn't stop the system from functioning properly if some connections die
- Enabling TCP keep alives on the ServicePoint
- Using a lower ConnectionLeaseTimeout on the ServicePoint (works correctly in general, fails to do anything when connections are closed)

Thus, I'd like to ask here whether someone faced similar problems and might know a way to overcome such issues. I'd also like to know whether I am hunting ghosts here, although I still doubt it at the moment. I also don't see how a socket could get into a CLOSE_WAIT state without the remote communication partner closing the connection (i.e. sending FIN) and I don't understand why Vista/2003 Server do not seem to respond to this by sending a FIN and receiving an ACK to transition from CLOSE_WAIT to LAST_ACK and then to CLOSED. The HTTP server I wrote does not tinker with the LingerState of the TcpClient, default behavior should apply when it comes down to gracefully closing sockets.

Maybe I am missing some very important point about this whole thing, I'd love to be enlightened :-). If you need any further info, I'm happy to supply it.

PS: I have done all of my tests in Visual Studio 2008 linking both .NET 2.0 and .NET 3.5, it doesn't make a difference from what I can tell.

Cheers!

HTTP連線狀態：

// //伺服器開通KeepLive. 第一個引數是客戶端開啟Keeplive，
// //第二個引數是客戶端是否關閉網路連線
////Server: Close_Wait Client:Fin_Wait_2
//HttpRequestTest.TestHttpConnect(false, false);
////Server: Close Client: Time_Wait
//HttpRequestTest.TestHttpConnect(false, true);
////Server: ESTABLISHED Client: ESTABLISHED
//HttpRequestTest.TestHttpConnect(true, true);
//Server: Close Client: Time_Wait
// HttpRequestTest.TestHttpConnect(false, true);

//伺服器關閉KeepLive. 第一個引數是客戶端開啟Keeplive，
//第二個引數是客戶端是否關閉網路連線
////Server: Close_Wait Client:Fin_Wait_2
//HttpRequestTest.TestHttpConnect(false, false);
////Server: Close Client: Time_Wait
//HttpRequestTest.TestHttpConnect(false, true);
////Server: Close Client: Time_Wait
//HttpRequestTest.TestHttpConnect(true, true);
////Server: Close Client: Time_Wait
//HttpRequestTest.TestHttpConnect(false, true);

REF:

http://www.cnblogs.com/zealic/archive/2008/05/01/1107942.html

[@more@]