http的302 redirect的一個問題

今天在curl一個網站的時候遇到一個奇怪的問題，下面是輸出：

lxg@lxg-X240:~$ curl -L http://www.yngs.gov.cn/ -v
* Hostname was NOT found in DNS cache
* Trying 116.52.12.163…
* Connected to www.yngs.gov.cn (116.52.12.163) port 80 (#0)
GET / HTTP/1.1
User-Agent: curl/7.38.0
Host: www.yngs.gov.cn
Accept: /

< HTTP/1.1 302 Moved Temporarily
< Date: Wed, 04 Nov 2015 14:08:49 GMT
< Transfer-Encoding: chunked
< Location: http://www.yngs.gov.cn/newWeb/template/index.jsp
< Content-Type: text/html; charset=UTF-8
< Set-Cookie: JSESSIONID=SLyTW6RR3R7zPNkkvzvpj12Q1snzzvNFQjYPDbDhYbvgTXWhSnff!-995202664; path=/; HttpOnly
< X-Powered-By: *********
< Set-Cookie: SANGFOR_AD=20111157; path=/
<
* Ignoring the response-body
* Connection #0 to host www.yngs.gov.cn left intact
* Issue another request to this URL: ‘http://www.yngs.gov.cn/newWeb/template/index.jsp’
* Found bundle for host www.yngs.gov.cn: 0xb89840c0
* Re-using existing connection! (#0) with host www.yngs.gov.cn
* Connected to www.yngs.gov.cn (116.52.12.163) port 80 (#0)
GET /newWeb/template/index.jsp HTTP/1.1
User-Agent: curl/7.38.0
Host: www.yngs.gov.cn
Accept: /
……… //上面的輸出一直重複
* Ignoring the response-body
* Connection #0 to host www.yngs.gov.cn left intact
* Maximum (50) redirects followed
curl: (47) Maximum (50) redirects followed

最後的錯誤顯示超過了curl設定的最大50次跳轉。
從上面的輸出來看訪問http://www.yngs.gov.cn/的時候返回302跳轉，跳轉的url為http://www.yngs.gov.cn/newWeb/template/index.jsp，但是接著訪問 http://www.yngs.gov.cn/newWeb/template/index.jsp的時候還是返回同樣的302跳轉，跳轉後的地址是目標自身，這樣肯定就會一直在 http://www.yngs.gov.cn/newWeb/template/index.jsp這個url上跳轉，當超過curl設定的預設最大跳轉次數50以後就異常結束了。
既然curl有問題那麼試一下wget命令看看吧，看這個命令是否也是會遇到同樣的錯誤結果：

lxg@lxg-X240:~$ wget http://www.yngs.gov.cn/ –debug

—request begin—
GET / HTTP/1.1
User-Agent: Wget/1.16.1 (linux-gnu)
Accept: /
Accept-Encoding: identity
Host: www.yngs.gov.cn
Connection: Keep-Alive
—request end—

—response begin—
HTTP/1.1 302 Moved Temporarily
Date: Wed, 04 Nov 2015 14:18:51 GMT
Transfer-Encoding: chunked
Location: http://www.yngs.gov.cn/newWeb/template/index.jsp
Content-Type: text/html; charset=UTF-8
Set-Cookie: JSESSIONID=7JJTW6TLpKRF0vyNXtRpQrnZffkgDfB0vh6vDzQ9jhGNvRsmZxyv!-1122044597; path=/; HttpOnly
X-Powered-By: *********
Set-Cookie: SANGFOR_AD=20111151; path=/
—response end—

302 Moved Temporarily

Stored cookie www.yngs.gov.cn -1 (ANY) / <session> <insecure> [expiry none] JSESSIONID 7JJTW6TLpKRF0vyNXtRpQrnZffkgDfB0vh6vDzQ9jhGNvRsmZxyv!-1122044597

Stored cookie www.yngs.gov.cn -1 (ANY) / <session> <insecure> [expiry none] SANGFOR_AD 20111151
Registered socket 3 for persistent reuse.
URI content encoding = “UTF-8”
位置：http://www.yngs.gov.cn/newWeb/template/index.jsp [跟隨至新的 URL]

URI content encoding = None
–2015-11-04 22:23:09– http://www.yngs.gov.cn/newWeb/template/index.jsp
再次使用存在的到 www.yngs.gov.cn:80 的連線。
Reusing fd 3.

—request begin—
GET /newWeb/template/index.jsp HTTP/1.1
User-Agent: Wget/1.16.1 (linux-gnu)
Accept: /
Accept-Encoding: identity
Host: www.yngs.gov.cn
Connection: Keep-Alive
Cookie: JSESSIONID=7JJTW6TLpKRF0vyNXtRpQrnZffkgDfB0vh6vDzQ9jhGNvRsmZxyv!-1122044597; SANGFOR_AD=20111151
—request end—

—response begin—
HTTP/1.1 200 OK
Date: Wed, 04 Nov 2015 14:18:51 GMT
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8
X-Powered-By: *********
—response end—
200 OK

上面是wget的執行結果（去掉了一些多餘的輸出），我們看到wget能正常的獲取到http://www.yngs.gov.cn/的結果，並沒有出現curl遇到的死迴圈跳轉的問題。那麼也就是說這個網站本身是沒有問題的，只是我們訪問的時候可能缺少了一些引數。
接著我對比了一下curl和wget的request、response資訊，我發現兩者在第一次請求http://www.yngs.gov.cn/的時候request、response都是差不多的，不同的可能就是user-agent。但是當再次請求302返回回來的redirect url http://www.yngs.gov.cn/newWeb/template/index.jsp的時候兩者的request中的引數就有一些不一樣了，wget的請求中是把第一次響應返回Cookie帶上了，但是curl卻是Ignoring the response-body，忽略了第一次響應返回的資料，第二次請求的時候沒有帶上第一次請求返回的Cookie。
此時基本可以判斷是因為curl訪問的時候預設忽略了response返回的資料，redirect url的時候沒有設定Cookie導致的，那麼怎麼來驗證呢？
第一種方法就是禁掉wget的Cookie看還能否正常獲取內容：

lxg@lxg-X240:~$ wget http://www.yngs.gov.cn/ –debug –no-cookies
Setting –cookies (cookies) to 0
—request begin—
GET / HTTP/1.1
User-Agent: Wget/1.16.1 (linux-gnu)
Accept: /
Accept-Encoding: identity
Host: www.yngs.gov.cn
Connection: Keep-Alive
—request end—

—response begin—
HTTP/1.1 302 Moved Temporarily
Date: Wed, 04 Nov 2015 14:43:41 GMT
Transfer-Encoding: chunked
Location: http://www.yngs.gov.cn/newWeb/template/index.jsp
Content-Type: text/html; charset=UTF-8
Set-Cookie: JSESSIONID=SDLtW6ZdvQwPpqGR5mBf2N1TxChNlySvTN8lhDBTQpyP3KvDdr0R!-170174379; path=/; HttpOnly
X-Powered-By: *********
Set-Cookie: SANGFOR_AD=20111158; path=/

—response end—
–2015-11-04 22:46:33– http://www.yngs.gov.cn/newWeb/template/index.jsp
再次使用存在的到 www.yngs.gov.cn:80 的連線。
Reusing fd 3.

—request begin—
GET /newWeb/template/index.jsp HTTP/1.1
User-Agent: Wget/1.16.1 (linux-gnu)
Accept: /
Accept-Encoding: identity
Host: www.yngs.gov.cn
Connection: Keep-Alive
—request end—

—response begin—
HTTP/1.1 302 Moved Temporarily
Date: Wed, 04 Nov 2015 14:42:16 GMT
Transfer-Encoding: chunked
Location: http://www.yngs.gov.cn/newWeb/template/index.jsp
Content-Type: text/html; charset=UTF-8
Set-Cookie: JSESSIONID=YQJTW6ZLpWhm7pfr3LzL6lkQdQ1XbnBMCHQhjn7vZ2yptMJvsJvW!-1122044597; path=/; HttpOnly
X-Powered-By: *********
Set-Cookie: SANGFOR_AD=20111151; path=/
—response end—

…………….
URI content encoding = None
已超過 20 次重定向。

我們看到wget最後也是以嘗試20次跳轉失敗結束。
第二中方法就是開啟curl的Cookie：

lxg@lxg-X240:~$ curl -L -b /tmp/curl.cookies http://www.yngs.gov.cn/
* Hostname was NOT found in DNS cache
* Trying 116.52.12.163…
* Connected to www.yngs.gov.cn (116.52.12.163) port 80 (#0)
GET / HTTP/1.1
User-Agent: curl/7.38.0
Host: www.yngs.gov.cn
Accept: /

< HTTP/1.1 302 Moved Temporarily
< Date: Wed, 04 Nov 2015 14:55:53 GMT
< Transfer-Encoding: chunked
< Location: http://www.yngs.gov.cn/newWeb/template/index.jsp
< Content-Type: text/html; charset=UTF-8
* Added cookie JSESSIONID=”lswQW6cZzRtvyGkkJm0hL8RscHT98bcC3YD4f4V1RCJvLLwb2ZMJ!-1122044597” for domain www.yngs.gov.cn, path /, expire 0
< Set-Cookie: JSESSIONID=lswQW6cZzRtvyGkkJm0hL8RscHT98bcC3YD4f4V1RCJvLLwb2ZMJ!-1122044597; path=/; HttpOnly
< X-Powered-By: *********
* Added cookie SANGFOR_AD=”20111151” for domain www.yngs.gov.cn, path /, expire 0
< Set-Cookie: SANGFOR_AD=20111151; path=/
<
* Ignoring the response-body
* Connection #0 to host www.yngs.gov.cn left intact
* Issue another request to this URL: ‘http://www.yngs.gov.cn/newWeb/template/index.jsp’
* Found bundle for host www.yngs.gov.cn: 0xb8b74108
* Re-using existing connection! (#0) with host www.yngs.gov.cn
* Connected to www.yngs.gov.cn (116.52.12.163) port 80 (#0)
GET /newWeb/template/index.jsp HTTP/1.1
User-Agent: curl/7.38.0
Host: www.yngs.gov.cn
Accept: /
Cookie: JSESSIONID=lswQW6cZzRtvyGkkJm0hL8RscHT98bcC3YD4f4V1RCJvLLwb2ZMJ!-1122044597; SANGFOR_AD=20111151

< HTTP/1.1 200 OK
< Date: Wed, 04 Nov 2015 14:55:53 GMT
< Transfer-Encoding: chunked
< Content-Type: text/html; charset=UTF-8
< X-Powered-By: *********
<
<!DOCTYPE html>
<html xmlns=”http://www.w3.org/1999/xhtml”>
<head>
………………..

Connection #0 to host www.yngs.gov.cn left intact
</html>

curl成功獲取到了結果。
從上面的結果也就驗證了之前的猜想。