Kubernetes安裝之三：etcd叢集的配置

dessler發表於2019-02-14

為什麼要首先配置etcd叢集呢，因為etcd叢集是Kubernetes的基礎，所有資料都儲存在這裡

1.生成etcd證書 

其實這裡建議使用主機名，避免ip變化而導致的資料庫不可用,但是我習慣用ip，為避免證書看起來不那麼亂，所以我將不同的證書放置到不同的目錄裡面

cat > /etc/ssl/etcd/etcd-csr.json <<EOF
{
  "CN": "etcd",
  "hosts": [
    "127.0.0.1",
    "192.168.1.40",
    "192.168.1.41",
    "192.168.1.42"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "ChengDu",
      "L": "ChengDu",
      "O": "k8s",
      "OU": "dessler"
    }
  ]
}
EOF
複製程式碼

cfssl gencert -ca=/etc/ssl/ca.pem \
    -ca-key=/etc/ssl/ca-key.pem \
    -config=/etc/ssl/ca-config.json \
    -profile=kubernetes etcd-csr.json | cfssljson -bare etcd複製程式碼

ls
etcd.csr  etcd-csr.json  etcd-key.pem  etcd.pem
複製程式碼

2.分發證書及二進位制檔案

過程我就省略了，都是些基本的檔案傳輸操作

3.準備證書環境

（自己注意資料儲存目錄，根據自己的情況決定）

chmod 644 /etc/ssl/etcd/etcd-key.pem
useradd -s /sbin/nologin -M etcd
mkdir -p /var/lib/etcd/
chown -R etcd:etcd /var/lib/etcd/
複製程式碼

4.配置etcd的服務

這裡需要特別注意就是etcd的名字，叢集的3個節點是不能重複的，還有和這個名字和叢集的配置（比如我這裡的etcd01，etcd02，etcd03）

4.1伺服器a配置etcd服務

cat > /usr/lib/systemd/system/etcd.service <<EOF
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
Documentation=https://github.com/coreos

[Service]
User=etcd
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/usr/bin/etcd \\
  --data-dir=/var/lib/etcd \\
  --name=etcd01 \\
  --cert-file=/etc/ssl/etcd/etcd.pem \\
  --key-file=/etc/ssl/etcd/etcd-key.pem \\
  --trusted-ca-file=/etc/ssl/ca.pem \\
  --peer-cert-file=/etc/ssl/etcd/etcd.pem \\
  --peer-key-file=/etc/ssl/etcd/etcd-key.pem \\
  --peer-trusted-ca-file=/etc/ssl/ca.pem \\
  --peer-client-cert-auth \\
  --client-cert-auth \\
  --listen-peer-urls=https://192.168.1.40:2380 \\
  --initial-advertise-peer-urls=https://192.168.1.40:2380 \\
  --listen-client-urls=https://192.168.1.40:2379,http://127.0.0.1:2379 \\
  --advertise-client-urls=https://192.168.1.40:2379 \\
  --initial-cluster-token=etcd-cluster-0 \\
  --initial-cluster=etcd01=https://192.168.1.40:2380,etcd02=https://192.168.1.41:2380,etcd03=https://192.168.1.42:2380 \\
  --initial-cluster-state=new
Restart=on-failure
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF複製程式碼

4.2伺服器a配置etcd服務

cat > /usr/lib/systemd/system/etcd.service <<EOF
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
Documentation=https://github.com/coreos

[Service]
User=etcd
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/usr/bin/etcd \\
  --data-dir=/var/lib/etcd \\
  --name=etcd02 \\
  --cert-file=/etc/ssl/etcd/etcd.pem \\
  --key-file=/etc/ssl/etcd/etcd-key.pem \\
  --trusted-ca-file=/etc/ssl/ca.pem \\
  --peer-cert-file=/etc/ssl/etcd/etcd.pem \\
  --peer-key-file=/etc/ssl/etcd/etcd-key.pem \\
  --peer-trusted-ca-file=/etc/ssl/ca.pem \\
  --peer-client-cert-auth \\
  --client-cert-auth \\
  --listen-peer-urls=https://192.168.1.41:2380 \\
  --initial-advertise-peer-urls=https://192.168.1.41:2380 \\
  --listen-client-urls=https://192.168.1.41:2379,http://127.0.0.1:2379 \\
  --advertise-client-urls=https://192.168.1.41:2379 \\
  --initial-cluster-token=etcd-cluster-0 \\
  --initial-cluster=etcd01=https://192.168.1.40:2380,etcd02=https://192.168.1.41:2380,etcd03=https://192.168.1.42:2380 \\
  --initial-cluster-state=new
Restart=on-failure
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF複製程式碼

4.3伺服器a配置etcd服務

cat > /usr/lib/systemd/system/etcd.service <<EOF
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
Documentation=https://github.com/coreos

[Service]
User=etcd
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/usr/bin/etcd \\
  --data-dir=/var/lib/etcd \\
  --name=etcd03 \\
  --cert-file=/etc/ssl/etcd/etcd.pem \\
  --key-file=/etc/ssl/etcd/etcd-key.pem \\
  --trusted-ca-file=/etc/ssl/ca.pem \\
  --peer-cert-file=/etc/ssl/etcd/etcd.pem \\
  --peer-key-file=/etc/ssl/etcd/etcd-key.pem \\
  --peer-trusted-ca-file=/etc/ssl/ca.pem \\
  --peer-client-cert-auth \\
  --client-cert-auth \\
  --listen-peer-urls=https://192.168.1.42:2380 \\
  --initial-advertise-peer-urls=https://192.168.1.42:2380 \\
  --listen-client-urls=https://192.168.1.42:2379,http://127.0.0.1:2379 \\
  --advertise-client-urls=https://192.168.1.42:2379 \\
  --initial-cluster-token=etcd-cluster-0 \\
  --initial-cluster=etcd01=https://192.168.1.40:2380,etcd02=https://192.168.1.41:2380,etcd03=https://192.168.1.42:2380 \\
  --initial-cluster-state=new
Restart=on-failure
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF
複製程式碼

5.檢查叢集狀態

etcdctl --endpoints=https://192.168.1.40:2379,https://192.168.1.41:2379,https://192.168.1.42:2379 \
>         --cert-file=/etc/ssl/etcd/etcd.pem \
>         --ca-file=/etc/ssl/ca.pem \
>         --key-file=/etc/ssl/etcd/etcd-key.pem \
>         cluster-health
member e42b05792291d17 is healthy: got healthy result from https://192.168.1.41:2379
member 68326dea8aa5233d is healthy: got healthy result from https://192.168.1.40:2379
member e10fc861b3b13bc0 is healthy: got healthy result from https://192.168.1.42:2379複製程式碼

注意的幾個地方：name 需要一一對應，ip地址也容易出錯

叢集狀態正常，etd叢集建立成果

  • 說明：
  • User：指定以 k8s 賬戶執行
  • WorkingDirectory、--data-dir：指定工作目錄和資料目錄為/var/lib/etcd，需在啟動服務前建立這個目錄
  • --name：指定節點名稱，當--initial-cluster-state 值為 new 時，--name 的引數值必須位於 --initial-cluster 列表中
  • --cert-file、--key-file：etcd server 與 client 通訊時使用的證書和私鑰
  • --trusted-ca-file：簽名 client 證書的 CA 證書，用於驗證 client 證書
  • --peer-cert-file、--peer-key-file：etcd 與 peer 通訊使用的證書和私鑰
  • --peer-trusted-ca-file：簽名 peer 證書的 CA 證書，用於驗證 peer 證書


相關文章