[20200304]測試網路狀態TIME_WAIT.txt

lfree發表於2020-03-05

[20200304]測試網路狀態TIME_WAIT.txt

--//前幾天在做IPTABLE測試時,遇到的情況,看到網路TIME_WAIT狀態。想起以前遇到的一次網路故障。
--//我當時執行如下:
#  netstat -tnop 2>/dev/null  |grep TIME_WAIT|wc
    392    3528   50506

--//看到狀態是TIME_WAIT的連線數量很大,比上面的還要大,以為網路出了問題,實際上是DNS解析緩慢導致的問題。
--//今天測試看看網路狀態是TIME_WAIT的情況要等多久消失。

https://benohead.com/blog/2013/07/21/tcp-about-fin_wait_2-time_wait-and-close_wait/#TIME_WAIT

TIME_WAIT

The TIME-WAIT state means that from the local end-point point of view, the connection is closed but we're still waiting
before accepting a new connection in order to prevent delayed duplicate packets from the previous connection from being
accepted by the new connection.

In this state, TCP blocks any second connection between these address/port pairs until the TIME_WAIT state is exited
after waiting for twice the maximum segment lifetime (MSL).

In most cases, seeing many TIME_WAIT connection doesn't show any issue. You only have to start worrying when the number
of TIME_WAIT connections cause performance problems or a memory overflow.

--//還是看不懂...,我還是不理解我們內網的生產系統為什麼出現大量的TIME_WAIT。
--//僅僅理解新的連線不是使用原來的IP:port直到timewait減少到0的狀態結束.

1.環境:
SYS@book> @ ver1

PORT_STRING                    VERSION        BANNER
------------------------------ -------------- --------------------------------------------------------------------------------
x86_64/Linux 2.4.xx            11.2.0.4.0     Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production

$ grep -i expire sqlnet.ora
#SQLNET.EXPIRE_TIME = 1

$ echo /proc/sys/net/ipv4/tcp_keepalive* | xargs   -n 1  strings -1 -f
/proc/sys/net/ipv4/tcp_keepalive_intvl: 10
/proc/sys/net/ipv4/tcp_keepalive_probes: 4
/proc/sys/net/ipv4/tcp_keepalive_time: 33

$ echo /proc/sys/net/ipv4/tcp_syncookies  /proc/sys/net/ipv4/tcp_tw_* /proc/sys/net/ipv4/tcp_fin_timeout | xargs  -n 1 strings -1 -f
/proc/sys/net/ipv4/tcp_syncookies: 1
/proc/sys/net/ipv4/tcp_tw_recycle: 0
/proc/sys/net/ipv4/tcp_tw_reuse: 0
/proc/sys/net/ipv4/tcp_fin_timeout: 60

$ cat /usr/local/bin/ts.awk
# /bin/bash
gawk '{ print strftime("[%Y-%m-%d %H:%M:%S]"), $0 }'

$ alias zdate
alias zdate='date +'\''%Y/%m/%d %T'\'''

2.測試:
--//在服務端執行:
$ seq 10000 | xargs -IQ bash -c " netstat -notp 2>/dev/null | egrep -i 'time_wait'| ts.awk ;sleep 1"
--//正常我的測試伺服器很乾淨,不會出現任何輸出。

--//在客戶端執行:
$ echo @spid | sqlplus  -s -l scott/book@192.168.100.78:1521/book:DEDICATED
       SID    SERIAL# PROCESS                  SERVER    SPID       PID  P_SERIAL# C50
---------- ---------- ------------------------ --------- ------ ------- ---------- --------------------------------------------------
        30        241 19134                    DEDICATED 46825       26        115 alter system kill session '30,241' immediate;

--//在服務端觀察,沒有任何輸出。也就是正常退出不會出現time_wait的情況。
--//我還在windows的客戶端做了一個測試,就是登入連線伺服器,然後直接ctrl+c或者點選X按鈕,情況也一樣,服務端沒有任何輸出。
--//在linux下的客戶端還做了kill -9 PROCESS, 情況也一樣,服務端沒有任何輸出。

--//也就是正常關閉或者客戶端kill的情況下連線會斷開,不會出現State=TIME_WAIT的情況。

--//在客戶端執行:
$ zdate ;tnsping 192.168.100.78 1 >/dev/null
2020/03/04 09:43:06

--//在服務端觀察:
$ seq 10000 | xargs -IQ bash -c " netstat -notp 2>/dev/null | egrep -i 'time_wait'| ts.awk ;sleep 1"
[2020-03-04 09:43:07] tcp        0      0 192.168.100.78:1521         192.168.100.40:53052        TIME_WAIT   -                   timewait (59.85/0/0)
[2020-03-04 09:43:08] tcp        0      0 192.168.100.78:1521         192.168.100.40:53052        TIME_WAIT   -                   timewait (58.83/0/0)
[2020-03-04 09:43:09] tcp        0      0 192.168.100.78:1521         192.168.100.40:53052        TIME_WAIT   -                   timewait (57.81/0/0)
[2020-03-04 09:43:10] tcp        0      0 192.168.100.78:1521         192.168.100.40:53052        TIME_WAIT   -                   timewait (56.79/0/0)
[2020-03-04 09:43:11] tcp        0      0 192.168.100.78:1521         192.168.100.40:53052        TIME_WAIT   -                   timewait (55.77/0/0)
[2020-03-04 09:43:12] tcp        0      0 192.168.100.78:1521         192.168.100.40:53052        TIME_WAIT   -                   timewait (54.75/0/0)
--//時間基本對上,也可以看出TIME_WAIT從60秒開始遞減到0後消失。

--//如果使用strace跟蹤:
$ strace -f  sqlplus scott/book@book <<<quit
..
setsockopt(9, SOL_SOCKET, SO_SNDTIMEO, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 16) = 0
setsockopt(9, SOL_SOCKET, SO_RCVTIMEO, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 16) = 0
write(9, "\0\n\0\0\6\0\0\0\0@", 10)     = 10
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
close(9)                                = 0
--//在關閉socket控制程式碼9槽前執行write。我估計這個向服務端發起關閉連線的資訊。

$ strace -f   tnsping 192.168.100.78 1
...
getsockopt(4, SOL_SOCKET, SO_SNDBUF, [-4611755699976781824], [4]) = 0
getsockopt(4, SOL_SOCKET, SO_RCVBUF, [-4611755699976710828], [4]) = 0
setsockopt(4, SOL_TCP, TCP_NODELAY, [1], 4) = 0
fcntl(4, F_SETFD, FD_CLOEXEC)           = 0
rt_sigaction(SIGPIPE, {SIG_IGN}, {SIG_DFL}, 8) = 0
gettimeofday({1583286911, 730815}, NULL) = 0
write(4, "\0W\0\0\1\0\0\0\1:\1,\0\0 \0\377\377\177\10\0\0\1\0\0\35"..., 87) = 87
read(4, "\0A\0\0\4\0\0\0\"\0\0005(DESCRIPTION=(TMP=)("..., 8208) = 65
setsockopt(4, SOL_SOCKET, SO_SNDTIMEO, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 16) = 0
setsockopt(4, SOL_SOCKET, SO_RCVTIMEO, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 16) = 0
close(4)                                = 0

--//在close前沒有wtite操作。"

3.看看能否縮短TIME_WAIT時間。
--//網上大量連結提示修改核心引數/proc/sys/net/ipv4/tcp_fin_timeout可以減少TIME_WAIT時間,我做了N多測試,根本沒用。
--//實際上這個時間是寫死在net/tcp.h標頭檔案裡面的,參考:/usr/src/kernels/2.6.39-300.26.1.el5uek/include/net/tcp.h可以發現如下:

#define TCP_TIMEWAIT_LEN (60*HZ) /* how long to wait to destroy TIME-WAIT
                  * state, about 60 seconds */
#define TCP_FIN_TIMEOUT TCP_TIMEWAIT_LEN
                                 /* BSD style FIN_WAIT2 deadlock breaker.
                  * It used to be 3min, new value is 60sec,
                  * to combine FIN-WAIT-2 timeout with
                  * TIME-WAIT timer.
                  */
--//除非你修改標頭檔案並且重新編譯核心,否則都是無效的。
--//如果設定/proc/sys/net/ipv4/tcp_tw_recycle=1,很快回收。
$ echo /proc/sys/net/ipv4/tcp_syncookies  /proc/sys/net/ipv4/tcp_tw_* /proc/sys/net/ipv4/tcp_fin_timeout | xargs  -n 1 strings -1 -f
/proc/sys/net/ipv4/tcp_syncookies: 1
/proc/sys/net/ipv4/tcp_tw_recycle: 1
/proc/sys/net/ipv4/tcp_tw_reuse: 0
/proc/sys/net/ipv4/tcp_fin_timeout: 60

--//在客戶端執行:
$ zdate ;tnsping 192.168.100.78 1 >/dev/null
2020/03/04 10:22:01

$ seq 10000 | xargs -IQ bash -c " netstat -notp 2>/dev/null | egrep -i 'time_wait'| ts.awk ;sleep 1"
[2020-03-04 10:22:02] tcp        0      0 192.168.100.78:1521         192.168.100.40:53537        TIME_WAIT   -                   timewait (0.25/0/0)
--//很快消失。

--//檢查生產系統發現。
#  netstat -tnop 2>/dev/null |  grep "TIME_WAIT" | wc
   1275    2475   35459

# netstat -tnop 2>/dev/null  |grep TIME_WAIT| awk '{print $5}' | cut -f1 -d":" | sort|uniq -c | sort -nr
--//結果不在貼出.
--//我發現對應IP執行的程式是httpd.exe或者文w3wp.exe程式。我感覺這些程式屬於短連線查詢,連上馬上斷開,感覺像這些應用沒有發出斷開的資訊。

4.繼續測試:
$ seq 10000 | xargs -IQ bash -c " netstat -notp 2>/dev/null | egrep  -v -i 'time_wait|ESTABLISHED|Active Internet connection|Proto Rec'| ts.awk ;sleep 1"

--//並行發起65000個連線.消耗完全部埠看看:
$ seq 10 | xargs -IQ -P 10 bash -c "tnsping 127.0.0.1 6500 >/dev/null"

--//僅僅偶爾能看到少量的輸出。也許127.0.0.1 的lo介面很特殊.
$ seq 10000 | xargs -IQ bash -c " netstat -notp 2>/dev/null | egrep  -v -i 'time_wait|ESTABLISHED|Active Internet connection|Proto Rec'| ts.awk ;sleep 1"
^[[A
[2020-03-05 09:07:38] tcp        1      1 127.0.0.1:20005             127.0.0.1:1521              LAST_ACK    36320/tnsping       off (0.00/0/0)
[2020-03-05 09:07:38] tcp        0      1 127.0.0.1:43473             127.0.0.1:1521              SYN_SENT    -                   off (0.00/0/0)
^C

--//換成如下:
$ seq 10 | xargs -IQ -P 10 bash -c "tnsping 192.168.100.78 6500 >/dev/null"

--//可以發現。埠占用不會馬上斷開出現state=SYN_SENT。
$ seq 10000 | xargs -IQ bash -c " netstat -notp 2>/dev/null | egrep  -v -i 'time_wait|ESTABLISHED|Active Internet connection|Proto Rec'| ts.awk ;sleep 1"
[2020-03-05 09:12:19] tcp        0      1 192.168.100.78:40820        192.168.100.78:1521         SYN_SENT    -                   on (2.99/0/0)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[2020-03-05 09:12:19] tcp        0      1 192.168.100.78:40828        192.168.100.78:1521         SYN_SENT    -                   on (2.99/0/0)
[2020-03-05 09:12:19] tcp        0      1 192.168.100.78:40826        192.168.100.78:1521         SYN_SENT    -                   on (2.89/0/0)
[2020-03-05 09:12:19] tcp        0      1 192.168.100.78:40821        192.168.100.78:1521         SYN_SENT    -                   on (2.88/0/0)
[2020-03-05 09:12:19] tcp        0      1 192.168.100.78:40829        192.168.100.78:1521         SYN_SENT    -                   on (2.87/0/0)
[2020-03-05 09:12:19] tcp        0      1 192.168.100.78:40825        192.168.100.78:1521         SYN_SENT    -                   on (2.86/0/0)
[2020-03-05 09:12:19] tcp        0      1 192.168.100.78:40827        192.168.100.78:1521         SYN_SENT    -                   on (2.85/0/0)
[2020-03-05 09:12:20] tcp        0      1 192.168.100.78:40824        192.168.100.78:1521         SYN_SENT    37361/tnsping       on (1.81/0/0)
[2020-03-05 09:12:20] tcp        0      1 192.168.100.78:40823        192.168.100.78:1521         SYN_SENT    37366/tnsping       on (1.81/0/0)
[2020-03-05 09:12:20] tcp        0      1 192.168.100.78:40822        192.168.100.78:1521         SYN_SENT    37368/tnsping       on (1.79/0/0)
[2020-03-05 09:12:20] tcp        0      1 192.168.100.78:40820        192.168.100.78:1521         SYN_SENT    37367/tnsping       on (1.77/0/0)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[2020-03-05 09:12:20] tcp        0      1 192.168.100.78:40828        192.168.100.78:1521         SYN_SENT    37365/tnsping       on (1.77/0/0)
[2020-03-05 09:12:20] tcp        0      1 192.168.100.78:40826        192.168.100.78:1521         SYN_SENT    37362/tnsping       on (1.65/0/0)
[2020-03-05 09:12:20] tcp        0      1 192.168.100.78:40821        192.168.100.78:1521         SYN_SENT    37364/tnsping       on (1.65/0/0)
[2020-03-05 09:12:20] tcp        0      1 192.168.100.78:40829        192.168.100.78:1521         SYN_SENT    37369/tnsping       on (1.64/0/0)
[2020-03-05 09:12:20] tcp        0      1 192.168.100.78:40825        192.168.100.78:1521         SYN_SENT    37370/tnsping       on (1.62/0/0)
[2020-03-05 09:12:20] tcp        0      1 192.168.100.78:40827        192.168.100.78:1521         SYN_SENT    37363/tnsping       on (1.61/0/0)
[2020-03-05 09:12:21] tcp        0      1 192.168.100.78:40824        192.168.100.78:1521         SYN_SENT    37361/tnsping       on (0.58/0/0)
[2020-03-05 09:12:21] tcp        0      1 192.168.100.78:40823        192.168.100.78:1521         SYN_SENT    37366/tnsping       on (0.58/0/0)
[2020-03-05 09:12:21] tcp        0      1 192.168.100.78:40822        192.168.100.78:1521         SYN_SENT    37368/tnsping       on (0.55/0/0)
[2020-03-05 09:12:21] tcp        0      1 192.168.100.78:40820        192.168.100.78:1521         SYN_SENT    37367/tnsping       on (0.54/0/0)
[2020-03-05 09:12:21] tcp        0      1 192.168.100.78:40828        192.168.100.78:1521         SYN_SENT    37365/tnsping       on (0.54/0/0)
[2020-03-05 09:12:21] tcp        0      1 192.168.100.78:40826        192.168.100.78:1521         SYN_SENT    37362/tnsping       on (0.41/0/0)
[2020-03-05 09:12:21] tcp        0      1 192.168.100.78:40821        192.168.100.78:1521         SYN_SENT    37364/tnsping       on (0.40/0/0)
[2020-03-05 09:12:21] tcp        0      1 192.168.100.78:40829        192.168.100.78:1521         SYN_SENT    37369/tnsping       on (0.39/0/0)
[2020-03-05 09:12:21] tcp        0      1 192.168.100.78:40825        192.168.100.78:1521         SYN_SENT    37370/tnsping       on (0.38/0/0)
[2020-03-05 09:12:21] tcp        0      1 192.168.100.78:40827        192.168.100.78:1521         SYN_SENT    37363/tnsping       on (0.37/0/0)

--//你可以發現大量發起連線馬上斷開,time_wait需要60秒斷開。這樣其它執行tnsping操作state=SYN_SENT.有點不能理解的是,如果第2次重複執行,
--//就很少或者基本看不到上面的輸出,而且非常快的執行完成。

$ seq 10000 | xargs -IQ bash -c " netstat -notp 2>/dev/null | egrep  -v -i 'time_wait|ESTABLISHED|Active Internet connection|Proto Rec'| ts.awk ;sleep 1"
[2020-03-05 09:20:28] tcp        1      1 192.168.100.78:59575        192.168.100.78:1521         LAST_ACK    -                   off (0.00/0/0)
[2020-03-05 09:21:39] tcp        0      1 192.168.100.78:52801        192.168.100.78:1521         SYN_SENT    -                   off (0.00/0/0)

--//再換一個IP又大量出現:
$ seq 10 | xargs -IQ -P 10 bash -c "tnsping 127.0.0.1 6500 >/dev/null"
$ seq 10000 | xargs -IQ bash -c " netstat -notp 2>/dev/null | egrep  -v -i 'time_wait|ESTABLISHED|Active Internet connection|Proto Rec'| ts.awk ;sleep 1"
[2020-03-05 09:20:28] tcp        1      1 192.168.100.78:59575        192.168.100.78:1521         LAST_ACK    -                   off (0.00/0/0)
[2020-03-05 09:21:39] tcp        0      1 192.168.100.78:52801        192.168.100.78:1521         SYN_SENT    -                   off (0.00/0/0)
[2020-03-05 09:22:42] tcp        0      1 127.0.0.1:57158             127.0.0.1:1521              SYN_SENT    -                   on (1.81/0/0)
[2020-03-05 09:22:51] tcp        0      1 127.0.0.1:57153             127.0.0.1:1521              SYN_SENT    39110/tnsping       on (11.25/2/0)
[2020-03-05 09:22:51] tcp        0      1 127.0.0.1:57148             127.0.0.1:1521              SYN_SENT    39116/tnsping       on (10.72/2/0)
[2020-03-05 09:22:51] tcp        0      1 127.0.0.1:57149             127.0.0.1:1521              SYN_SENT    39118/tnsping       on (10.64/2/0)
[2020-03-05 09:22:51] tcp        0      1 127.0.0.1:57154             127.0.0.1:1521              SYN_SENT    39114/tnsping       on (9.95/2/0)
[2020-03-05 09:22:51] tcp        0      1 127.0.0.1:57152             127.0.0.1:1521              SYN_SENT    39112/tnsping       on (9.89/2/0)
[2020-03-05 09:22:51] tcp        0      1 127.0.0.1:57147             127.0.0.1:1521              SYN_SENT    39108/tnsping       on (8.89/2/0)
[2020-03-05 09:22:51] tcp        0      1 127.0.0.1:57151             127.0.0.1:1521              SYN_SENT    39111/tnsping       on (5.98/2/0)
--//對於網路的許多知識很匱乏,不懂。

5.停止監聽看看:
$ seq 10000 | xargs -IQ bash -c " netstat -notp 2>/dev/null | egrep  -v -i 'time_wait' ;sleep 1"
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name    Timer
tcp        0      0 192.168.100.78:63637        192.168.100.40:1521         ESTABLISHED 23221/ora_arc1_book off (0.00/0/0)
tcp        0      0 192.168.100.78:1521         192.168.100.78:5684         ESTABLISHED 17528/tnslsnr       keepalive (19.06/0/0)
tcp        0      0 192.168.100.78:1521         192.168.98.6:51794          ESTABLISHED 23209/ora_d000_book keepalive (0.25/0/0)
tcp        0      0 192.168.100.78:22           192.168.98.6:53993          ESTABLISHED -                   keepalive (5.05/0/0)
tcp        0      0 192.168.100.78:63636        192.168.100.40:1521         ESTABLISHED 41372/ora_nsa2_book off (0.00/0/0)
tcp        0     76 192.168.100.78:22           192.168.98.6:51134          ESTABLISHED -                   on (0.44/0/0)
tcp        0      0 192.168.100.78:5684         192.168.100.78:1521         ESTABLISHED 23173/ora_pmon_book off (0.00/0/0)

$ lsnrctl stop
LSNRCTL for Linux: Version 11.2.0.4.0 - Production on 05-MAR-2020 09:27:32
Copyright (c) 1991, 2013, Oracle.  All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=0.0.0.0)(PORT=1521)))
The command completed successfully

$ seq 10000 | xargs -IQ bash -c " netstat -notp 2>/dev/null | egrep   -i 'time_wait' ;sleep 1"
tcp        0      0 127.0.0.1:1521              127.0.0.1:56737             TIME_WAIT   -                   timewait (59.94/0/0)
tcp        0      0 127.0.0.1:1521              127.0.0.1:56737             TIME_WAIT   -                   timewait (58.92/0/0)
tcp        0      0 127.0.0.1:1521              127.0.0.1:56737             TIME_WAIT   -                   timewait (57.90/0/0)
tcp        0      0 127.0.0.1:1521              127.0.0.1:56737             TIME_WAIT   -                   timewait (56.88/0/0)
tcp        0      0 192.168.100.78:1521         192.168.100.78:5684         TIME_WAIT   -                   timewait (59.15/0/0)
tcp        0      0 127.0.0.1:1521              127.0.0.1:56737             TIME_WAIT   -                   timewait (55.86/0/0)
tcp        0      0 192.168.100.78:1521         192.168.100.78:5684         TIME_WAIT   -                   timewait (58.13/0/0)
tcp        0      0 127.0.0.1:1521              127.0.0.1:56737             TIME_WAIT   -                   timewait (54.84/0/0)
tcp        0      0 192.168.100.78:1521         192.168.100.78:5684         TIME_WAIT   -                   timewait (57.11/0/0)
tcp        0      0 127.0.0.1:1521              127.0.0.1:56737             TIME_WAIT   -                   timewait (53.82/0/0)
tcp        0      0 192.168.100.78:1521         192.168.100.78:5684         TIME_WAIT   -                   timewait (56.09/0/0)
tcp        0      0 127.0.0.1:1521              127.0.0.1:56737             TIME_WAIT   -                   timewait (52.80/0/0)
tcp        0      0 192.168.100.78:1521         192.168.100.78:5684         TIME_WAIT   -                   timewait (55.07/0/0)
tcp        0      0 127.0.0.1:1521              127.0.0.1:56737             TIME_WAIT   -                   timewait (51.78/0/0)
tcp        0      0 192.168.100.78:1521         192.168.100.78:5684         TIME_WAIT   -                   timewait (54.05/0/0)
tcp        0      0 127.0.0.1:1521              127.0.0.1:56737             TIME_WAIT   -                   timewait (50.76/0/0)
tcp        0      0 192.168.100.78:1521         192.168.100.78:5684         TIME_WAIT   -                   timewait (53.03/0/0)
tcp        0      0 127.0.0.1:1521              127.0.0.1:56737             TIME_WAIT   -                   timewait (49.74/0/0)
^C

6.修改核心引數:

$ echo /proc/sys/net/ipv4/tcp_syncookies  /proc/sys/net/ipv4/tcp_tw_* /proc/sys/net/ipv4/tcp_fin_timeout | xargs  -n 1 strings -1 -f
/proc/sys/net/ipv4/tcp_syncookies: 1
/proc/sys/net/ipv4/tcp_tw_recycle: 1
/proc/sys/net/ipv4/tcp_tw_reuse: 0
/proc/sys/net/ipv4/tcp_fin_timeout: 60

$ seq 10 | xargs -IQ -P 10 bash -c "tnsping 192.168.100.78 6500 >/dev/null"

$ seq 10000 | xargs -IQ bash -c " netstat -notp 2>/dev/null | egrep  -v -i 'time_wait|ESTABLISHED|Active Internet connection|Proto Rec'| ts.awk ;sleep 1"
[2020-03-05 09:45:06] tcp        1      0 192.168.100.78:36972        192.168.100.78:1521         CLOSE_WAIT  -                   off (0.00/0/0)
[2020-03-05 09:45:06] tcp        1      0 192.168.100.78:39849        192.168.100.78:1521         CLOSE_WAIT  -                   off (0.00/0/0)
[2020-03-05 09:45:07] tcp        0      1 192.168.100.78:46608        192.168.100.78:1521         SYN_SENT    -                   off (0.00/0/0)
[2020-03-05 09:45:09] tcp        1      0 192.168.100.78:52063        192.168.100.78:1521         CLOSE_WAIT  -                   off (0.00/0/0)
[2020-03-05 09:45:09] tcp        0      1 192.168.100.78:52134        192.168.100.78:1521         SYN_SENT    -                   off (0.00/0/0)
[2020-03-05 09:45:12] tcp        1      1 192.168.100.78:7511         192.168.100.78:1521         LAST_ACK    -                   off (0.00/0/0)

總結:
--//我僅僅得出結論出現TIME_WAIT要等60秒消失。而且寫死在核心裡面,修改引數/proc/sys/net/ipv4/tcp_fin_timeout無效。
--//至於生產系統出現大量的情況,我自己還是無法探究。而且還有一些情況我無法解析:
#  netstat -tnop 2>/dev/null  |grep probe|wc
     62     558    7819

--//而且我查詢程式發現STIME竟然是Jan14,2019年的。
#  ps -eLf | egrep "6681[1]|UI[D]|7415[0]"
UID         PID   PPID    LWP  C NLWP STIME TTY          TIME CMD
oracle    66811      1  66811  0    1 Jan14 ?        00:00:01 oraclexxxx1 (LOCAL=NO)
oracle    74150      1  74150  0    1  2019 ?        00:00:00 oraclexxxx1 (LOCAL=NO)

#  netstat -tnop 2>/dev/null  | grep "probe" |grep 74150
tcp        0  32120 192.168.xxx.xxx:1521        192.168.yyy.yyy:55868        ESTABLISHED 74150/oraclexxxx1   probe (89.44/0/0)


來自 “ ITPUB部落格 ” ,連結:http://blog.itpub.net/267265/viewspace-2678684/,如需轉載,請註明出處,否則將追究法律責任。

相關文章