CentOS6.9安裝Filebeat監控Nginx的訪問日誌傳送到Kafka

繽紛世界wb發表於2018-02-10
原文網址 : https://flycode.co/archives/163097
CentOSNginxKafka

一、下載地址：

官方：https://www.elastic.co/cn/downloads/beats/filebeat

百度雲盤：https://pan.baidu.com/s/1dvhqb0

 

二、安裝

tar zxvf filebeat-6.2.1-linux-x86_64.tar.gz
mv filebeat-6.2.1-linux-x86_64 filebeat

也可yum安裝

下載和安裝key檔案

rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch

建立yum原始檔

vi /etc/yum.repos.d/elastic.repo
[elastic-6.x]
name=Elastic repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

開始安裝

yum install filebeat

 

三、配置

vi kafka.yml
filebeat.prospectors:
- input_type: log
  paths:
    - /usr/local/openresty/nginx/logs/access.log
output.kafka:
  hosts: ["10.10.6.225:9092"]
  topic: need_process_nginx_access_log
  partition.round_robin:
    reachable_only: false
  required_acks: 1

注：output到kafka的partitions策略：分為random、round_robin、hash，預設是hash

具體說明：https://www.elastic.co/guide/en/beats/filebeat/current/kafka-output.html#_literal_partition_literal

執行

./filebeat -e -c kafka.yml

 後臺執行

nohup /usr/local/filebeat/filebeat -e -c /usr/local/filebeat/kafka.yml >/dev/null 2>&1 &

 


相關文章