flume + elasticsearch

flume-1.9.0 + elasticsearch-7.7.1 + kibana-7.7.1

下載地址

flume-ng  

http://flume.apache.org/download.html

elasticsearch  

https://www.elastic.co/cn/downloads/past-releases#elasticsearch

kibana        

https://www.elastic.co/cn/downloads/past-releases/kibana-7-7-1

 

預設flume支援0.x 1.x的elasticsearch
如使用高版本的elasticsearch 需要修改

The elasticsearch and lucene-core jars required for your environment must be placed in the lib directory of the Apache Flume installation. Elasticsearch requires that the major version of the client JAR match that of the server and that both are running the same minor version of the JVM. SerializationExceptions will appear if this is incorrect. To select the required version first determine the version of elasticsearch and the JVM version the target cluster is running. Then select an elasticsearch client library which matches the major version. A 0.19.x client can talk to a 0.19.x cluster; 0.20.x can talk to 0.20.x and 0.90.x can talk to 0.90.x. Once the elasticsearch version has been determined then read the pom.xml file to determine the correct lucene-core JAR version to use. The Flume agent which is running the ElasticSearchSink should also match the JVM the target cluster is running down to the minor version.

這個是網上找到的一個7.7.0版本的

https://github.com/bitilandu/flume-ng-elasticsearch-sink

 

 

 

###一些報錯
給出的建議是要刪除index .kibana_2 然後重啟 kibana
curl -XDELETE http://localhost:9200/.kibana_1