Nginx access.log日誌分析shell命令

Nginx 版本資訊:

nginx version: nginx/0.8.53

Nginx日誌配置項:

access_log      /data0/logs/access.log  combined;

Nginx日誌格式:

$remote_addr – $remote_user [$time_local]  $request $status $apache_bytes_sent $http_referer $http_user_agent

127.0.0.1 - - [24/Mar/2011:12:45:07 +0800] "GET /fcgi_bin/xxx.fcgi?id=xxx HTTP/1.0" 200 160 "-" "Mozilla/4.0"

透過日誌檢視當天訪問頁面排前10的url: #>cat access.log | grep "24/Mar/2011" | awk '{print $7}' | sort | uniq -c | sort -nr | head -n 10

透過日誌檢視當天ip連線數,統計ip地址的總連線數 #>cat access.log | grep "24/Mar/2011" | awk '{print $1}' | sort | uniq -c | sort –nr      
38 112.97.192.16      20 117.136.31.145      19 112.97.192.31        3 61.156.31.20        2 209.213.40.6        1 222.76.85.28
透過日誌檢視當天訪問次數最多的10個IP ,只需要在上一個命令後加上head命令 #>cat access.log | grep "24/Mar/2011" |awk '{print $3}'|sort |uniq -c|sort -nr|head –n 10     
 38 112.97.192.16      20 117.136.31.145      19 112.97.192.31        3 61.156.31.20        2 209.213.40.6        1 222.76.85.28

透過日誌檢視當天訪問次數最多的10個IP #>awk '{print $1}' access.log |sort |uniq -c|sort -nr|head   10680 10.0.21.17     1702 10.0.20.167       823 10.0.20.51       504 10.0.20.255       215 58.60.188.61       192 183.17.161.216         38 112.97.192.16         20 117.136.31.145         19 112.97.192.31           6 113.106.88.10

透過日誌檢視當天指定ip訪問次數過的url和訪問次數: #>cat access.log | grep "10.0.21.17" | awk '{print $7}' | sort | uniq -c | sort –nr     
224 /test/themes/default/img/logo_index.gif     224 /test/themes/default/img/bg_index_head.jpg     224 /test/themes/default/img/bg_index.gif     219 /test/vc.php     219 /     213 /misc/js/global.js     211 /misc/jsext/popup.ext.js     211 /misc/js/common.js     210 /sladmin/home     197 /misc/js/flib.js

透過日誌檢視當天訪問次數最多的時間段 #>awk '{print $4}' access.log | grep "24/Mar/2011" |cut -c 14-18|sort|uniq -c|sort -nr|head      
24 16:49      19 16:17      16 16:51      11 16:48        4 16:50        3 16:52        1 20:09        1 20:05        1 20:03        1 19:55 還有什麼其他的,大家繼續補充啦


取出所有的狀態碼
cat /opt/logs/nginx/access.log | grep -ioE "HTTP\/1\.[1|0]\"[[:blank:]][0-9]{3}"
grep中不用加"\"嗎？