1.ELK簡介
ELK是三個開源軟體的縮寫，分別表示：Elasticsearch , Logstash, Kibana , 它們都是開源軟體。新增了一個FileBeat，它是一個輕量級的日誌收集處理工具(Agent)，Filebeat佔用資源少，適合於在各個伺服器上搜集日誌後傳輸給Logstash，官方也推薦此工具。
Elasticsearch是個開源分散式搜尋引擎，提供蒐集、分析、儲存資料三大功能。它的特點有：分散式，零配置，自動發現，索引自動分片，索引副本機制，restful風格介面，多資料來源，自動搜尋負載等。
Logstash 主要是用來日誌的蒐集、分析、過濾日誌的工具，支援大量的資料獲取方式。一般工作方式為c/s架構，client端安裝在需要收集日誌的主機上，server端負責將收到的各節點日誌進行過濾、修改等操作在一併發往elasticsearch上去。
Kibana 也是一個開源和免費的工具，Kibana可以為 Logstash 和 ElasticSearch 提供的日誌分析友好的 Web 介面，可以幫助彙總、分析和搜尋重要資料日誌。
Filebeat隸屬於Beats。目前Beats包含四種工具：
2.Topbeat（蒐集系統、程式和檔案系統級別的 CPU 和記憶體使用情況等資料）
3.Filebeat（蒐集檔案資料）
4.Winlogbeat（蒐集 Windows 事件日誌資料）
3.環境準備
| 名稱 | IP |
|---|---|
| es01 | 10.0.0.267 |
| es02 | 10.0.0.268 |
| web01 | 10.0.0.269 |
4.搭建elk
4.1關閉防火牆和selinux
systemctl stop firewalld systemctl disable firewalld sed -i 's/=enforcing/=disabled/g' /etc/selinux/config setenforce 0
[root@es01-10.0.0.237 ~]# cd /etc/yum.repos.d/ [root@es01-10.0.0.237 ~]# wget http://mirrors.aliyun.com/repo/Centos-7.repo
[root@es01-10.0.0.237 /elk_soft]# yum install java-1.8.0-openjdk -y
[root@es01-10.0.0.237 ~]# cd /elk_soft/ [root@es01-10.0.0.237 /elk_soft]# ls elasticsearch-6.4.1.rpm kibana-6.4.1-x86_64.rpm logstash-6.4.1.rpm [root@es01-10.0.0.237 /elk_soft]# yum localinstall elasticsearch-6.4.1.rpm -y
[root@es02-10.0.0.238 /elk_soft]# vim /etc/elasticsearch/elasticsearch.yml cluster.name: my-cluster #叢集名稱 node.name: es02 #節點名稱 path.data: /var/lib/elasticsearch #存放資料的目錄（如果修改將目錄的屬組屬主改為elasticearch） path.logs: /var/log/elasticsearch #日誌目錄（如果修改將目錄的屬組屬主改為elasticearch） network.host: 10.0.0.238 #本機IP地址 http.port: 9200 #預設埠 discovery.zen.ping.unicast.hosts: ["10.0.0.237", "10.0.0.238"] #叢集中每個node的IP地址 #在配置檔案倒數第二行加入，如果不新增是不能發現別的主機，只能發現自己 http.cors.enabled: true http.cors.allow-origin: "*"
[root@es01-10.0.0.237 /elk_soft]# systemctl start elasticsearch.service [root@es01-10.0.0.237 /elk_soft]# systemctl enable elasticsearch.service [root@es01-10.0.0.237 /elk_soft]# netstat -lntup Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 6141/sshd tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 6222/master tcp6 0 0 10.0.0.237:9200 :::* LISTEN 8998/java tcp6 0 0 10.0.0.237:9300 :::* LISTEN 8998/java
5.安裝head外掛來管理叢集
5.1在es01上安裝head外掛，配置epel源
wget http://mirrors.aliyun.com/repo/epel-7.repo
[root@es01-10.0.0.237 ~]# yum install nodejs npm openssl screen -y
[root@es01-10.0.0.237 ~]# node -v v6.17.1 [root@es01-10.0.0.237 ~]# npm -v 3.10.10
[root@es01-10.0.0.237 ~]# npm install -g cnpm --registry=https://registry.npm.taobao.org
[root@es01-10.0.0.237 ~]# yum install git -y [root@es01-10.0.0.237 ~]# cd /opt/ [root@es01-10.0.0.237 /opt]# git clone git://github.com/mobz/elasticsearch-head.git [root@es01-10.0.0.237 /opt]# cd elasticsearch-head/ [root@es01-10.0.0.237 /opt/elasticsearch-head]# ls crx elasticsearch-head.sublime-project index.html plugin-descriptor.properties _site Dockerfile Gruntfile.js LICENCE proxy src Dockerfile-alpine grunt_fileSets.js package.json README.textile test [root@es01-10.0.0.237 /opt/elasticsearch-head]# cnmp install [root@es01-10.0.0.237 /opt/elasticsearch-head]# screen -S es-head #切屏指令 [root@es01-10.0.0.237 /opt/elasticsearch-head]# cnpm run start > elasticsearch-head@0.0.0 start /opt/elasticsearch-head > grunt server Running "connect:server" (connect) task Waiting forever... Started connect web server on http://localhost:9100 ###Ctrl+A+D #切換快捷鍵 後臺執行程式
5.7叢集的三種健康狀態
-
綠色：表示叢集健康
-
黃色：表示亞健康狀態，勉強可以執行
-
紅色：表示已經很危險，有可能資料已經丟失
6.elasticearch資料庫操作
6.1查入三條資料
curl -XPUT '10.0.0.237:9200/megacorp/employee/1?pretty' -H 'Content-Type: application/json' -d' { "first_name" : "Li", "last_name": "si", "age" : 28, "about" : "I love cat", "interests": [ "swimming" ] } ' curl -XPUT '10.0.0.237:9200/megacorp/employee/2?pretty' -H 'Content-Type: application/json' -d' { "first_name" : "Zhang", "last_name": "san", "age" : 22, "about" : "I love dog", "interests": [ "swimming" ] } ' curl -XPUT '10.0.0.237:9200/megacorp/employee/3?pretty' -H 'Content-Type: application/json' -d' { "first_name" : "Wang", "last_name": "wu", "age" : 30, "about" : "I love beautiful girl", "interests": [ "swimming" ] } '
6.3索引介紹
| es資料庫 | |
|---|---|
| index（索引） | database（資料庫） |
| type （型別） | tables（資料庫中的表） |
| id （id號） | 對應表中的欄位 |
6.4如何查詢一條資料
[root@es01-10.0.0.237 /opt/elasticsearch-head]# curl -XGET '10.0.0.237:9200/megacorp/employee/1?pretty' { "_index" : "megacorp", "_type" : "employee", "_id" : "1", "_version" : 1, "found" : true, "_source" : { "first_name" : "Li", "last_name" : "si", "age" : 28, "about" : "I love cat", "interests" : [ "swimming" ] } }
[root@es01-10.0.0.237 /opt/elasticsearch-head]# curl -XDELETE '10.0.0.237:9200/megacorp/employee/1?pretty' { "_index" : "megacorp", "_type" : "employee", "_id" : "1", "_version" : 2, "result" : "deleted", "_shards" : { "total" : 2, "successful" : 2, "failed" : 0 }, "_seq_no" : 1, "_primary_term" : 1 }
[root@es01-10.0.0.237 /opt/elasticsearch-head]# curl -XPUT '10.0.0.237:9200/student_message?pretty' { "acknowledged" : true, "shards_acknowledged" : true, "index" : "student_message" }
curl -XPUT '10.0.0.237:9200/my_temp_index?pretty' -H 'Content-Type: application/json' -d' { "settings": { "number_of_shards" : 2, "number_of_replicas" : 1 } } '
7.kibana安裝配置
7.1在es01上安裝kibana
[root@es01-10.0.0.237 /elk_soft]# yum localinstall kibana-6.4.1-x86_64.rpm -y
[root@es01-10.0.0.237 /elk_soft]# vim /etc/kibana/kibana.yml server.port: 5601 #服務監聽埠 server.host: "10.0.0.237" #服務IP地址 elasticsearch.url: "http://10.0.0.237:9200" #elasticsearch的地址 elasticsearch.pingTimeout: 30000000 logging.dest: stdout #如果指定日誌儲存路徑，需要修改檔案的所有者及所屬組
[root@es01-10.0.0.237 /elk_soft]# systemctl start kibana.service [root@es01-10.0.0.237 /elk_soft]# systemctl enable kibana.service [root@es01-10.0.0.237 /elk_soft]# ss -lnt State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 128 *:9100 *:* LISTEN 0 128 *:22 *:* LISTEN 0 100 127.0.0.1:25 *:* LISTEN 0 128 10.0.0.237:5601 *:*
8.在web01上安裝nginx+logstash
8.1配置nginx的yum源，安裝nginx
[root@web01-10.0.0.239 /etc/yum.repos.d]# vim /etc/yum.repos.d/nginx.repo [nginx] name=nginx repo baseurl=http://nginx.org/packages/centos/7/$basearch/ gpgcheck=0 enabled=1 [root@web01-10.0.0.239 ~]# yum install nginx -y
[root@web01-10.0.0.239 ~]# vim /etc/nginx/conf.d/elk.conf server { listen 80; server_name elk.test.com; location / { proxy_pass http://10.0.0.237:5601; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } # access_log /var/log/nginx/elk_access.log main; }
[root@web01-10.0.0.239 ~]# vim /etc/nginx/nginx.conf log_format access_json '{"@timestamp":"$time_iso8601",' '"host":"$server_addr",' '"clientip":"$remote_addr",' '"size":$body_bytes_sent,' '"responsetime":$request_time,' '"upstreamtime":"$upstream_response_time",' '"upstreamhost":"$upstream_addr",' '"http_host":"$host",' '"url":"$uri",' '"domain":"$host",' '"xff":"$http_x_forwarded_for",' '"referer":"$http_referer",' '"status":"$status"}'; access_log /var/log/nginx/ access_json;
[root@web01-10.0.0.239 ~]# systemctl start nginx [root@web01-10.0.0.239 ~]# systemctl enable nginx
10.0.0.239 elk.tast.com
[root@web01-10.0.0.239 /elk_soft]# yum install java-1.8.0-openjdk -y
[root@web01-10.0.0.239 /elk_soft]# ls filebeat-6.4.1-x86_64.rpm logstash-6.4.1.rpm
[root@web01-10.0.0.239 /elk_soft]# vim /etc/logstash/logstash.yml path.data: /var/lib/logstash #資料存放目錄（如需修改記得修改屬主屬組） http.host: "10.0.0.239" #http的IP地址 http.port: 9600 #監聽埠 path.logs: /var/log/logstash #日誌存放目錄（如需修改記得修改屬主屬組）
[root@web01-10.0.0.239 ~]# vim /etc/logstash/conf.d/nginx_access.conf input { file { path => "/var/log/nginx/access_json.log" start_position => "end" type => "nginx_access" codec => json } } output { elasticsearch { hosts => ["10.0.0.237:9200"] index => "nginx_access-%{+YYYY.MM.dd}" } }
[root@web01-10.0.0.239 /etc/logstash]# sudo /usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd Using provided startup.options file: /etc/logstash/startup.options Manually creating startup for specified platform: systemd Successfully created system startup script for Logstash [root@web01-10.0.0.239 /etc/logstash]# systemctl start logstash [root@web01-10.0.0.239 /etc/logstash]# systemctl enable logstash
[root@es01-10.0.0.237 /elk_soft]# systemctl restart elasticsearch.service
9.2.kibana平臺展示所收到的日誌資訊
9.2.1首先新增一個索引模式
9.2.2新增成功
9.2.3新增一個餅狀圖
9.2.4建立排班表
9.2.5通過dashboard進行詳細展示
9.通過rdis安裝部署新的架構
9.1停掉web01上的logstash服務
[root@web01-10.0.0.239 ~]# systemctl stop logstash.service
9.2在web01上安裝filebeat
[root@web01-10.0.0.239 /elk_soft]# yum localinstall filebeat-6.4.1-x86_64.rpm -y
9.3修改filebeat配置檔案
24 enabled: true 27 paths: 28 - /var/log/nginx/access_json.log ##配置檔案最後新增 output.redis: hosts: ["10.0.0.237"] #資料要發給的主機 port: 6379 #Redis的埠 key: "nginx-log" #日誌型別 db: 0 #兩端要一樣 timeout: 5 #超時時間5秒
9.4啟動filebeat服務
[root@web01-10.0.0.239 ~]# systemctl start filebeat.service [root@web01-10.0.0.239 ~]# systemctl enable filebeat.service
9.5在es01上安裝Redis
[root@es01-10.0.0.237 ~]# yum install redis -y
9.6修改Redis配置檔案
[root@es01-10.0.0.237 ~]# vim /etc/redis.conf 61 bind 10.0.0.237 #redis本機ip地址 port 6379 #redis預設埠 128 daemonize yes #開啟守護程式
9.7啟動Redis
[root@es01-10.0.0.237 ~]# systemctl start redis [root@es01-10.0.0.237 ~]# systemctl enable redis [root@es01-10.0.0.237 ~]# redis-cli -h 10.0.0.237 10.0.0.237:6379>
9.8檢查Redis是否取到日誌
10.0.0.237:6379> keys * 1) "nginx-log" 10.0.0.237:6379> info keyspace # Keyspace db0:keys=1,expires=0,avg_ttl=0
9.8在es01上安裝logstash
[root@es01-10.0.0.237 /elk_soft]# yum localinstall logstash-6.4.1.rpm -y
10修改logstash的配置檔案
http.host: "10.0.0.237" #http的IP地址 http.port: 9600 #監聽埠
11在/etc/logstash/conf.d/下新增redis_input.conf檔案
[root@es01-10.0.0.237 /etc/logstash/conf.d]# vim redis_input.conf input { redis { host => "10.0.0.237" port => "6379" db => "0" data_type => "list" key => "nginx-log" } } output { stdout { codec => rubydebug } elasticsearch { hosts => ["10.0.0.237:9200"] index => "nginx-redis-%{+YYYY.MM.dd}" } }
12啟動logstash
[root@es01-10.0.0.237 /etc/logstash/conf.d]# /usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd [root@es01-10.0.0.237 /etc/logstash/conf.d]# systemctl start logstash [root@es01-10.0.0.237 /etc/logstash/conf.d]# systemctl enable logstash
13kibana展示
