es 5.5.3叢集環境搭建

weixin_34393428發表於2018-07-14

es 5.5.3叢集環境搭建

前提配置

Java version

1.8.0_131+

ES-KIBANA-LOGSTASH 版本對應關係

ES-KIBANA-LOGSTASH-VERSION

阿里雲es版本：5.5.3 對應版本如下:

Elasticsearch Kibana X-Pack Logstash* Beats
5.5.x 5.5.x 5.5.x 2.4.x-5.6.x 1.3.x-5.6.x

Elasticsearch

Elasticsearch官方文件

安裝

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.5.3.tar.gz
sha1sum elasticsearch-5.5.3.tar.gz 
tar -xzf elasticsearch-5.5.3.tar.gz

es叢集配置

/config/elasticsearch.yml                            #es配置檔案
/config/jvm.options                                  #JVM相關配置，記憶體大小等等
/config/log4j2.properties                            #es日誌配置
/data                                                #資料存放位置

cluster.name: my-es       
node.name: node-2         

network.host: 10.10.10.12
http.port: 9200

http.cors.enabled: true
http.cors.allow-origin: "*"

#node.master: true
node.data: true
discovery.zen.ping.unicast.hosts: ["10.10.10.12","10.10.10.13","10.10.10.14"]
discovery.zen.minimum_master_nodes:2   

bootstrap.system_call_filter: false
cluster.routing.allocation.disk.threshold_enabled: true
cluster.routing.allocation.disk.watermark.low: 2gb
cluster.routing.allocation.disk.watermark.high: 10gb

#thread_pool.bulk.queue_size: 1000  #此執行緒池用於批量操作。它的型別預設為fixed，size預設為可用處理器的數量，佇列的size預設為50

scp -r /app/elasticsearch-5.5.3 root@10.10.10.12:/app/
scp -r /app/elasticsearch-5.5.3 root@10.10.10.14:/app/



cluster.name: my-es       #叢集名稱
node.name: node-1         #節點名稱

network.host: 10.10.10.12
http.port: 9200

http.cors.enabled: true
http.cors.allow-origin: "*"

#node.master: true
node.data: true
discovery.zen.ping.unicast.hosts: ["10.10.10.12","10.10.10.13","10.10.10.14"]
discovery.zen.minimum_master_nodes:2   

cluster.name: my-es       #叢集名稱
node.name: node-3         #節點名稱

network.host: 10.10.10.14 #
http.port: 9200

http.cors.enabled: true
http.cors.allow-origin: "*"

#node.master: true
node.data: true
discovery.zen.ping.unicast.hosts: ["10.10.10.12","10.10.10.13","10.10.10.14"]
discovery.zen.minimum_master_nodes:2   #(master_eligible_nodes / 2) + 1

啟動及驗證

# 啟動
sh elasticsearch -d

# 關閉
kill -9 pid

# 驗證
curl http://network.host:http.prot


[elasticsearch@vm12 bin]$ curl http://10.10.10.12:9200
{
  "name" : "node-1",
  "cluster_name" : "my-es",
  "cluster_uuid" : "47Z3EUq3SX6hTDeZEAZI1Q",
  "version" : {
    "number" : "5.5.3",
    "build_hash" : "9305a5e",
    "build_date" : "2017-09-07T15:56:59.599Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.0"
  },
  "tagline" : "You Know, for Search"
}




curl http://10.10.10.14:9200/_cat/health?v

epoch      timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1531297304 16:21:44  my-es   green           3         3      0   0    0    0        0             0                  -                100.0%

[root@vm14 ~]# curl http://10.10.10.14:9200/_cat
=^.^=
/_cat/allocation
/_cat/shards
/_cat/shards/{index}
/_cat/master
/_cat/nodes
/_cat/tasks
/_cat/indices
/_cat/indices/{index}
/_cat/segments
/_cat/segments/{index}
/_cat/count
/_cat/count/{index}
/_cat/recovery
/_cat/recovery/{index}
/_cat/health
/_cat/pending_tasks
/_cat/aliases
/_cat/aliases/{alias}
/_cat/thread_pool
/_cat/thread_pool/{thread_pools}
/_cat/plugins
/_cat/fielddata
/_cat/fielddata/{fields}
/_cat/nodeattrs
/_cat/repositories
/_cat/snapshots/{repository}
/_cat/templates

安裝head 外掛

安裝node.js環境 伺服器：10.10.10.12

node -v

npm install -g grunt-cli

npm install -g grunt

# 下載github elasticsearch-head外掛
wget https://github.com/mobz/elasticsearch-head/archive/master.zip


unzip master.zip 

cd elasticsearch-head-master/

vi Gruntfile.js 
 
   connect: {
                        server: {
                                options: {
                                        port: 9100,
                                        hostname:'*',
                                        base: '.',
                                        keepalive: true
                                }
                        }
                }

cd _site
vi app.js
this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "http://localhost:9200";

localhost:9200 --> 10.10.10.12.9200

cd elasticsearch-head-master/
npm install

grunt server &

head-web-ui

10.10.10.12:9100

中文分詞器 elasticsearch-analysis-ik

elasticsearch-analysis-ik

ES IK
5.5.3 5.5.3

create plugin folder cd your-es-root/plugins/ && mkdir ik
unzip plugin to folder your-es-root/plugins/ik

wget https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v5.5.3/elasticsearch-analysis-ik-5.5.3.zip

解壓後每臺node複製一份，重啟es

FAQ

can not run elasticsearch as root

# elasticsearch 不能使用root使用者登入
useradd -m elasticsearch
passwd elasticsearch
#許可權，用root使用者執行 ： chown -R 資料夾名 使用者名稱

permission denied

chown -R data elasticsearch
chown -R logs elasticsearch

#chmod -R 777 logs

[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
[2]: max number of threads [1024] for user [elasticsearch] is too low, increase to at least [2048]
[3]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144

max_map_count
最大虛擬記憶體太小

檢視 sysctl -a|grep vm.max_map_count

sysctl -w vm.max_map_count=655360

使用者最大可建立檔案數太小:
/etc/security/limits.conf

* soft nofile 65536

* hard nofile 131072
* soft nproc 2048

* hard nproc 4096

使用者最大可建立執行緒數太小:
vi /etc/security/limits.d/90-nproc.conf

* soft nproc 1024

#修改為

* soft nproc 2048

安裝Kinaba

Kinaba官方文件

安裝位置：10.10.10.14:5601

wget https://artifacts.elastic.co/downloads/kibana/kibana-5.5.3-linux-x86_64.tar.gz

tar -xzf kibana-5.5.3-linux-x86_64.tar.gz

配置

預設埠 5601

config/kibana.yml

server.host: "10.10.10.13"
elasticsearch.url: "http://10.10.10.13:9200"
logging.dest: /apps/svr/kibana-5.5.3-linux-x86_64/logs/kibana.log

啟動

bin/kibana &

netstat -tnlp | grep 5601
kill -9 pid

相關文章