etcd介紹，以及適用場景，參考：http://www.infoq.com/cn/articles/etcd-interpretation-application-scenario-implement-principle
etcd的專案：https://github.com/coreos/etcd/releases/tag/v3.2.10 並且下載最新版本etcd
環境：
master：192.168.101.14，node1：192.168.101.15，node2：192.168.101.19
1、在三個節點上下載etcd-v3.2.10-linux-amd64.tar.gz，並提供etcd命令，具體操作如下
[root@docker ~]# tar xf etcd-v3.2.10-linux-amd64.tar.gz [root@docker ~]# cd etcd-v3.2.10-linux-amd64 [root@docker etcd-v3.2.10-linux-amd64]# cp etcd etcdctl /usr/local/bin/
2、在三個節點上建立etcd資料目錄：
# mkdir -p /var/lib/etcd
3、在每個節點上建立etcd的systemd unit檔案/usr/lib/systemd/system/etcd.service
master節點： [root@docker ~]# cat /usr/lib/systemd/system/etcd.service [Unit] Description=etcd server After=network.target After=network-online.target Wants=network-online.target [Service] Type=notify WorkingDirectory=/var/lib/etcd/ EnvironmentFile=-/etc/etcd/etcd.conf ExecStart=/usr/local/bin/etcd --name master --initial-advertise-peer-urls http://192.168.101.14:2380 --listen-peer-urls http://192.168.101.14:2380 --listen-client-urls http://192.168.101.14:2379,http://127.0.0.1:2379 --advertise-client-urls http://192.168.101.14:2379 --initial-cluster-token etcd-cluster-1 --initial-cluster master=http://192.168.101.14:2380,node1=http://192.168.101.15:2380,node2=http://192.168.101.19:2380 --initial-cluster-state new --data-dir=/var/lib/etcd Restart=on-failure RestartSec=5 LimitNOFILE=65536 [Install] WantedBy=multi-user.target
node1節點： [root@localhost ~]# cat /usr/lib/systemd/system/etcd.service [Unit] Description=etcd server After=network.target After=network-online.target Wants=network-online.target [Service] Type=notify WorkingDirectory=/var/lib/etcd/ EnvironmentFile=-/etc/etcd/etcd.conf ExecStart=/usr/local/bin/etcd --name node1 --initial-advertise-peer-urls http://192.168.101.15:2380 --listen-peer-urls http://192.168.101.15:2380 --listen-client-urls http://192.168.101.15:2379,http://127.0.0.1:2379 --advertise-client-urls http://192.168.101.15:2379 --initial-cluster-token etcd-cluster-1 --initial-cluster master=http://192.168.101.14:2380,node1=http://192.168.101.15:2380,node2=http://192.168.101.19:2380 --initial-cluster-state new --data-dir=/var/lib/etcd Restart=on-failure RestartSec=5 LimitNOFILE=65536 [Install] WantedBy=multi-user.target
node2節點： [root@localhost ~]# cat /usr/lib/systemd/system/etcd.service [Unit] Description=etcd server After=network.target After=network-online.target Wants=network-online.target [Service] Type=notify WorkingDirectory=/var/lib/etcd/ EnvironmentFile=-/etc/etcd/etcd.conf ExecStart=/usr/local/bin/etcd --name node2 --initial-advertise-peer-urls http://192.168.101.19:2380 --listen-peer-urls http://192.168.101.19:2380 --listen-client-urls http://192.168.101.19:2379,http://127.0.0.1:2379 --advertise-client-urls http://192.168.101.19:2379 --initial-cluster-token etcd-cluster-1 --initial-cluster master=http://192.168.101.14:2380,node1=http://192.168.101.15:2380,node2=http://192.168.101.19:2380 --initial-cluster-state new --data-dir=/var/lib/etcd Restart=on-failure RestartSec=5 LimitNOFILE=65536 [Install] WantedBy=multi-user.target
由於並沒有配置TSL認證，所以都是http而不是https，etcd客戶端監聽在2379，服務端監聽在2380
三個節點配置完成了服務檔案，於是載入該檔案並啟動服務：
# systemctl daemon-reload
# systemctl enable etcd
# systemctl start etcd
# systemctl status etcd
檢視三個節點的狀態：
[root@docker ~]# systemctl status etcd ● etcd.service - etcd server Loaded: loaded (/usr/lib/systemd/system/etcd.service; enabled; vendor preset: disabled) Active: active (running) since Thu 2017-11-23 13:56:05 CST; 12s ago Main PID: 21562 (etcd) CGroup: /system.slice/etcd.service └─21562 /usr/local/bin/etcd --name master --initial-advertise-peer-urls http://192.168.101.14:2380 --listen-peer-urls http://192.168.101.... Nov 23 13:56:05 docker etcd[21562]: enabled capabilities for version 3.0 Nov 23 13:56:06 docker etcd[21562]: health check for peer 192d36c71643c39d could not connect: dial tcp 192.168.101.19:2380: getsockopt: con...n refused Nov 23 13:56:07 docker etcd[21562]: peer 192d36c71643c39d became active Nov 23 13:56:07 docker etcd[21562]: established a TCP streaming connection with peer 192d36c71643c39d (stream Message writer) Nov 23 13:56:07 docker etcd[21562]: established a TCP streaming connection with peer 192d36c71643c39d (stream MsgApp v2 writer) Nov 23 13:56:07 docker etcd[21562]: established a TCP streaming connection with peer 192d36c71643c39d (stream MsgApp v2 reader) Nov 23 13:56:07 docker etcd[21562]: established a TCP streaming connection with peer 192d36c71643c39d (stream Message reader) Nov 23 13:56:09 docker etcd[21562]: updating the cluster version from 3.0 to 3.2 Nov 23 13:56:09 docker etcd[21562]: updated the cluster version from 3.0 to 3.2 Nov 23 13:56:09 docker etcd[21562]: enabled capabilities for version 3.2 Hint: Some lines were ellipsized, use -l to show in full.
[root@localhost ~]# systemctl status etcd ● etcd.service - etcd server Loaded: loaded (/usr/lib/systemd/system/etcd.service; enabled; vendor preset: disabled) Active: active (running) since Thu 2017-11-23 13:56:05 CST; 15s ago Main PID: 21504 (etcd) CGroup: /system.slice/etcd.service └─21504 /usr/local/bin/etcd --name node1 --initial-advertise-peer-urls http://192.168.101.15:2380 --listen-peer-urls http://192.168.101.1... Nov 23 13:56:05 localhost.localdomain systemd[1]: Started etcd server. Nov 23 13:56:05 localhost.localdomain etcd[21504]: set the initial cluster version to 3.0 Nov 23 13:56:05 localhost.localdomain etcd[21504]: enabled capabilities for version 3.0 Nov 23 13:56:07 localhost.localdomain etcd[21504]: peer 192d36c71643c39d became active Nov 23 13:56:07 localhost.localdomain etcd[21504]: established a TCP streaming connection with peer 192d36c71643c39d (stream Message writer) Nov 23 13:56:07 localhost.localdomain etcd[21504]: established a TCP streaming connection with peer 192d36c71643c39d (stream MsgApp v2 reader) Nov 23 13:56:07 localhost.localdomain etcd[21504]: established a TCP streaming connection with peer 192d36c71643c39d (stream MsgApp v2 writer) Nov 23 13:56:07 localhost.localdomain etcd[21504]: established a TCP streaming connection with peer 192d36c71643c39d (stream Message reader) Nov 23 13:56:09 localhost.localdomain etcd[21504]: updated the cluster version from 3.0 to 3.2 Nov 23 13:56:09 localhost.localdomain etcd[21504]: enabled capabilities for version 3.2
[root@localhost ~]# systemctl status etcd ● etcd.service - etcd server Loaded: loaded (/usr/lib/systemd/system/etcd.service; enabled; vendor preset: disabled) Active: active (running) since Thu 2017-11-23 13:56:07 CST; 17s ago Main PID: 21508 (etcd) CGroup: /system.slice/etcd.service └─21508 /usr/local/bin/etcd --name node2 --initial-advertise-peer-urls http://192.168.101.19:2380 --listen-peer-urls http://192.168.101.1... Nov 23 13:56:07 localhost.localdomain etcd[21508]: dialing to target with scheme: "" Nov 23 13:56:07 localhost.localdomain etcd[21508]: could not get resolver for scheme: "" Nov 23 13:56:07 localhost.localdomain etcd[21508]: serving insecure client requests on 127.0.0.1:2379, this is strongly discouraged! Nov 23 13:56:07 localhost.localdomain etcd[21508]: ready to serve client requests Nov 23 13:56:07 localhost.localdomain etcd[21508]: dialing to target with scheme: "" Nov 23 13:56:07 localhost.localdomain etcd[21508]: could not get resolver for scheme: "" Nov 23 13:56:07 localhost.localdomain etcd[21508]: serving insecure client requests on 192.168.101.19:2379, this is strongly discouraged! Nov 23 13:56:07 localhost.localdomain systemd[1]: Started etcd server. Nov 23 13:56:09 localhost.localdomain etcd[21508]: updated the cluster version from 3.0 to 3.2 Nov 23 13:56:09 localhost.localdomain etcd[21508]: enabled capabilities for version 3.2
沒有報錯問題後檢視叢集中的成員，任意節點執行就行：
[root@docker ~]# etcdctl member list 192d36c71643c39d: name=node2 peerURLs=http://192.168.101.19:2380 clientURLs=http://192.168.101.19:2379 isLeader=false 5f3835545a5f41e4: name=master peerURLs=http://192.168.101.14:2380 clientURLs=http://192.168.101.14:2379 isLeader=true 77c1ac60c5100363: name=node1 peerURLs=http://192.168.101.15:2380 clientURLs=http://192.168.101.15:2379 isLeader=false
可以看見叢集中自動推選了一個節點作為leader，然後檢視叢集健康狀態：
[root@docker ~]# etcdctl cluster-health member 192d36c71643c39d is healthy: got healthy result from http://192.168.101.19:2379 member 5f3835545a5f41e4 is healthy: got healthy result from http://192.168.101.14:2379 member 77c1ac60c5100363 is healthy: got healthy result from http://192.168.101.15:2379 cluster is healthy
使用etcd進行運算元據：
[root@docker ~]# etcdctl set name wadeson
wadeson
在node1、node2節點上進行檢視：
[root@localhost ~]# etcdctl get name
wadeson
[root@localhost ~]# etcdctl get name
wadeson
下面也是檢視叢集健康狀態的一種：
[root@docker ~]# curl http://192.168.101.14:2379/health {"health": "true"}[root@docker ~]# curl http://192.168.101.15:2379/health {"health": "true"}[root@docker ~]# curl http://192.168.101.19:2379/health {"health": "true"}
4、對etcd叢集中的member進行操作：
4.1、更新advertise client urls
直接修改引數--advertise-client-urls或者ETCD_ADVERTISE_CLIENT_URLS，然後重啟該member
4.2、更新advertise peer urls：
$ etcdctl member update 192d36c71643c39d http://192.168.101.15:2380
更新要修改的peer urls的member id和peer urls，然後重啟member
4.3、移除叢集中的某一個節點member：
首先檢視member列表：
[root@docker ~]# etcdctl member list 192d36c71643c39d: name=node2 peerURLs=http://192.168.101.19:2380 clientURLs=http://192.168.101.19:2379 isLeader=false 5f3835545a5f41e4: name=master peerURLs=http://192.168.101.14:2380 clientURLs=http://192.168.101.14:2379 isLeader=true 77c1ac60c5100363: name=node1 peerURLs=http://192.168.101.15:2380 clientURLs=http://192.168.101.15:2379 isLeader=false
現在將node2也就是192.168.101.19這個節點移除該叢集：
[root@docker ~]# etcdctl member remove 192d36c71643c39d Removed member 192d36c71643c39d from cluster [root@docker ~]# etcdctl member list 5f3835545a5f41e4: name=master peerURLs=http://192.168.101.14:2380 clientURLs=http://192.168.101.14:2379 isLeader=true 77c1ac60c5100363: name=node1 peerURLs=http://192.168.101.15:2380 clientURLs=http://192.168.101.15:2379 isLeader=false
叢集針對的操作是對member_id進行操作
4.4、向叢集增加一個member，現在叢集只有兩個member：
[root@docker ~]# etcdctl member list 5f3835545a5f41e4: name=master peerURLs=http://192.168.101.14:2380 clientURLs=http://192.168.101.14:2379 isLeader=true 77c1ac60c5100363: name=node1 peerURLs=http://192.168.101.15:2380 clientURLs=http://192.168.101.15:2379 isLeader=false
現在向該叢集增加一個member，也就是增加一個節點：
[root@docker ~]# etcdctl member add node2 http://192.168.101.19:2380 Added member named node2 with ID 4edc521f6598ba03 to cluster ETCD_NAME="node2" ETCD_INITIAL_CLUSTER="node2=http://192.168.101.19:2380,master=http://192.168.101.14:2380,node1=http://192.168.101.15:2380" ETCD_INITIAL_CLUSTER_STATE="existing"
然後安裝提示操作：
[root@docker ~]# export ETCD_NAME="node2" [root@docker ~]# export ETCD_INITIAL_CLUSTER="node2=http://192.168.101.19:2380,master=http://192.168.101.14:2380,node1=http://192.168.101.15:2380" [root@docker ~]# export ETCD_INITIAL_CLUSTER_STATE="existing"
[root@docker ~]# etcd --listen-client-urls http://192.168.101.19:2379 --advertise-client-urls http://192.168.101.19:2379 --listen-peer-urls http://192.168.101.19:2380 --initial-advertise-peer-urls http://192.168.101.19:2380 --data-dir %data_dir% 2017-11-23 14:30:33.649640 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_CLUSTER=node2=http://192.168.101.19:2380,master=http://192.168.101.14:2380,node1=http://192.168.101.15:2380 2017-11-23 14:30:33.649694 I | pkg/flags: recognized and used environment variable ETCD_INITIAL_CLUSTER_STATE=existing 2017-11-23 14:30:33.649706 I | pkg/flags: recognized and used environment variable ETCD_NAME=node2 2017-11-23 14:30:33.649748 I | etcdmain: etcd Version: 3.2.10 2017-11-23 14:30:33.649752 I | etcdmain: Git SHA: 694728c 2017-11-23 14:30:33.649754 I | etcdmain: Go Version: go1.8.5 2017-11-23 14:30:33.649757 I | etcdmain: Go OS/Arch: linux/amd64 2017-11-23 14:30:33.649760 I | etcdmain: setting maximum number of CPUs to 2, total number of available CPUs is 2 2017-11-23 14:30:33.649850 C | etcdmain: listen tcp 192.168.101.19:2380: bind: cannot assign requested address
不知道為何最後繫結不到192.168.101.19上，這裡是一個問題。。。。。。。待後面解決