goaccess安裝和使用

一、Debian下安裝goaccess
cat /etc/debian_version 
6.0.10
apt-cache search goaccess
goaccess - log analyzer and interactive viewer for the Apache Webserver
apt-get -y install goaccess

二、Centos下安裝goaccess
cat /etc/redhat-release 
CentOS release 6.6 (Final)
yum list|grep goaccess
goaccess.x86_64                            0.8.5-1.el6                   epel
yum -y install goaccess
沒有epel源的話需要安裝
rpm -Uvh

三、引數說明

-b Enable total bandwidth consumption. To achieve faster parsing, 
do not enable this flag.  This has been disabled by default.(流量統計)
-f input-file path to input log filename(日誌檔名)
-s Enable/report HTTP status codes. To avoid overhead while parsing, 
this has been disabled by default.(HTTP響應程式碼統計)
-e Exclude an IP from being count it under the HOST module. 
This has been disabled by default.(指定IP地址統計，預設禁用)指定還是排除？
-c Prompt log/date configuration window.(配置)
-a Enable a List of User-Agents by host.(使用者代理統計)

四、功能說明
1、General Statistics(統計概況，流量消耗)
Number  of valid requests, number of invalid requests, time to analyze the data, 
unique visitors, unique requested files, unique static files (css, ico, jpg,
js, swf, gif, png) unique HTTP referrers (URLs), unique 404s (not found), 
size of the parsed log file, bandwidth consumption.
2、Unique visitors(訪客排名)
HTTP requests having the same IP, same date and same agent will be considered a 
unique visit. This includes crawlers.
3、Requested files(動態Web請求)
Hit totals are based on total requests. This module will display hits, percent, 
bandwidth [time served], [protocol] and [method].
4、Requested static files(靜態web請求，如圖片、樣式表、指令碼等)
Hit totals are based on total requests. Includes files such as: jpg, css, swf, js, 
gif, png etc. 
This module will display hits,percent,bandwidth,[time served],[protocol] and [method].
5、404 or Not Found(404 錯誤)
Hit totals are based on total requests. This module will display hits, percent, 
bandwidth, [time served], [protocol] and [method].
6、Operating Systems(作業系統)
Hit totals are based on unique visitors. This module will display hits and percent. 
The expanded module shows all available versions of the parent node.
7、Browsers(瀏覽器和搜尋引擎)
Hit totals are based on unique visitors. This module will display hits and percent. 
The expanded module shows all available versions of the parent node.
8、HTTP Status Codes(HTTP 響應程式碼)
The values of the numeric status code to HTTP requests. 
Hit totals are based on total requests. This module will display hits and percent.
9、Keyphrases(鍵盤佈局)
This  module  will report keyphrases used on Google search, Google cache, 
and Google translate. Hit totals are based on total requests. This module will dis-
play hits and percent.

• -f – 日誌檔名
• -b – 開啟流量統計，如果希望加快分析速度不建議使用該引數
• -s – 開啟HTTP響應程式碼統計
• -a – 開啟使用者代理統計
• -e – 開啟指定IP地址統計，預設禁用
• -p -指定goacces使用者配置檔案路徑

• F1 幫助選單
• F5 重新整理
• q 退出當前口窗、選單、或是當前檢視的選項
• o 開啟當前的選項、選單
• c 改變視窗配色（目前只有兩種 預設和綠色）
• SHIFT + TAB 從當前選定模組向後切換
• RIGHT 開啟當前選中模組，檢視詳細資訊
• s 透過日期排序，只會在訪問請求模組起作用
• S 透過點選次數排序，只會在訪問請求模組起作用
• / 檢視詳細資訊的視窗進行搜尋
• n 透過/進行查詢後，查詢下個匹配的內容的位置，如果沒有則在視窗底部顯示“search hit BOTTOM”
• t 在檢視詳細資訊視窗，移動指標到最頂部
• b 在檢視詳細資訊視窗，移動指標到最底部

apache - what is? - how in goaccess

%v - vhost - %^ ignore it

%h - host - %h

%l - hyphen - %^ ignore it

%u - user - %^ ignore it

%t - timestamp - [%d:t% %^] ignore the zone from apache log

%r - request - %r

%s - status - %s

%b - size - %b

%{referer}i - request header - %R

%{UAgent}i - user agent - %u

%d date field matching the date_format variable.

%h host (the client IP address, either IPv4 or IPv6) IT網，

%r The request line from the client. This requires specific delimiters around the request (as single quotes, double quotes, or anything else) to be parsable. If not, we
have to use a combination of special format specifiers as %m %U %H.

%m The request method.

%U The URL path requested (including any query string).

%H The request protocol.

%s The status code that the server sends back to the client. IT網，

%b The size of the object returned to the client.

%R The “Referrer” HTTP request header.

%u The user-agent HTTP request header. Linux學習，http:// linux.it.net.cn

%D The time taken to serve the request, in microseconds.

%T The time taken to serve the request, in seconds or milliseconds. Note: %D will take priority over %T if both are used.

%^ Ignore this field. Lin

使用bash/sed/awk手動查詢Nginx日誌

#按日期查詢時間段 sed -n "/14//Jul//2015:00:00:00/,/15//Jul//2015:15:00:00/"p access.log > time_access.log  

#查詢504錯誤的頁面和數量 awk '($9 ~ /504/)' time_access.log | awk '{print $7}' | sort | uniq -c | sort -rn > 504.log  

#查詢訪問最多的20個IP及訪問次數 awk '{print $1}' time_access.log | sort | uniq -c | sort -n -k 1 -r | head -n 20 > top.log