Kubernetes (k8s) 叢集部署（二） 完整版

第二步：Etcd

在開始安裝 Kubernetes 之前，需要先將一些必要系統建立完成，其中 Etcd 就是 Kubernetes 最重要的一環，Kubernetes 會將大部分資訊儲存於 Etcd 上，來提供給其他節點索取，以確保整個叢集運作與溝通正常。

在這部分，將會需要產生 client 與 server 的各元件 certificates，並且替 Kubernetes admin user 產生 client 證照。建立/etc/etcd/ssl資料夾，然後進入目錄完成以下操作。

1、在master01需要安裝CFSSL工具，這將會用來建立 TLS certificates

$ export CFSSL_URL="https://pkg.cfssl.org/R1.2"
$ wget "${CFSSL_URL}/cfssl_linux-amd64" -O /usr/local/bin/cfssl
$ wget "${CFSSL_URL}/cfssljson_linux-amd64" -O /usr/local/bin/cfssljson
$ wget "${CFSSL_URL}/cfssl-certinfo_linux-amd64" -O /usr/local/bin/cfssl-certinfo
$ chmod +x /usr/local/bin/cfssl /usr/local/bin/cfssljson /usr/local/bin/cfssl-certinfo
$ scp -r /usr/local/bin/ 192.168.184.29:/usr/local/
$ scp -r /usr/local/bin/ 192.168.184.30:/usr/local/

2、建立ca證照與祕鑰

• 建立目錄ssl臨時目錄

$ cd /etc/etcd/ssl_tmp

• 建立用來生成 CA 檔案的 JSON 配置檔案

$ cfssl print-defaults config > config.json && cfssl print-defaults csr > csr.json
$ cat >  ca-config.json <<EOF
{
  "signing": {
    "default": {
      "expiry": "8760h"
    },
    "profiles": {
      "kubernetes": {
        "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ],
        "expiry": "8760h"
      }
    }
  }
}
EOF

• 建立用來生成 CA 證照籤名請求（CSR）的 JSON 配置檔案

$ cat > ca-csr.json <<EOF
{
  "CN": "kubernetes",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}
EOF

• 生成CA證照和金鑰

$ cfssl gencert -initca ca-csr.json | cfssljson -bare ca

• 校驗證照

$ openssl x509 -noout -text -in ca.pem

• 分發證照

$ cp ca.csr ca.pem ca-key.pem ca-config.json ../ssl
$ scp ca.csr ca.pem ca-key.pem ca-config.json 192.168.184.29:/etc/etcd/ssl/
$ scp ca.csr ca.pem ca-key.pem ca-config.json 192.168.184.30:/etc/etcd/ssl/

3、Etcd 安裝與設定

• 首先在master1節點下載 Etcd：

$ export ETCD_URL="https://github.com/coreos/etcd/releases/download"
$ cd && wget "${ETCD_URL}/v3.2.9/etcd-v3.2.9-linux-amd64.tar.gz" 
$ tar -zxf etcd-v3.2.9-linux-amd64.tar.gz
$ mv etcd-v3.2.9-linux-amd64/etcd* /usr/local/bin/ && rm -rf etcd-v3.2.9-linux-amd64
$ scp  /usr/local/bin/etcd* 192.168.184.29:/usr/local/bin/
$ scp  /usr/local/bin/etcd* 192.168.184.30:/usr/local/bin/

• 建立etcd證照籤名請求

$ cd /etc/etcd/ssl_tmp
$ cat > etcd-csr.json <<EOF
{
  "CN": "etcd",
  "hosts": [
    "127.0.0.1",
    "192.168.184.28",
    "192.168.184.29",
    "192.168.184.30"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}
EOF

• 產生 kube-apiserver certificate 證照：

$ cfssl gencert -ca=/etc/etcd/ssl/ca.pem -ca-key=/etc/etcd/ssl/ca-key.pem -config=/etc/etcd/ssl/ca-config.json -profile=kubernetes etcd-csr.json | cfssljson -bare etcd

• 分發

$ cp   etcd*.pem ../ssl
$ scp  etcd*.pem 192.168.184.29:/etc/etcd/ssl/
$ scp  etcd*.pem 192.168.184.30:/etc/etcd/ssl/

• etcd相關檔案

$ cat > /etc/etcd/cfg/etcd.conf <<EOF
#[member]
ETCD_NAME="master01"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
#ETCD_SNAPSHOT_COUNTER="10000"
#ETCD_HEARTBEAT_INTERVAL="100"
#ETCD_ELECTION_TIMEOUT="1000"
ETCD_LISTEN_PEER_URLS="https://192.168.184.28:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.184.28:2379,https://127.0.0.1:2379"
#ETCD_MAX_SNAPSHOTS="5"
#ETCD_MAX_WALS="5"
#ETCD_CORS=""
#[cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.184.28:2380"
# if you use different ETCD_NAME (e.g. test),
# set ETCD_INITIAL_CLUSTER value for this name, i.e. "test=http://..."
ETCD_INITIAL_CLUSTER="master01=https://192.168.184.28:2380,work01=https://192.168.184.29:2380,work02=https://192.168.184.30:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.184.28:2379"
#[security]
CLIENT_CERT_AUTH="true"
ETCD_CA_FILE="/etc/etcd/ssl/ca.pem"
ETCD_CERT_FILE="/etc/etcd/ssl/etcd.pem"
ETCD_KEY_FILE="/etc/etcd/ssl/etcd-key.pem"
PEER_CLIENT_CERT_AUTH="true"
ETCD_PEER_CA_FILE="/etc/etcd/ssl/ca.pem"
ETCD_PEER_CERT_FILE="/etc/etcd/ssl/etcd.pem"
ETCD_PEER_KEY_FILE="/etc/etcd/ssl/etcd-key.pem"

EOF

$ cat > /lib/systemd/system/etcd.service <<EOF
[Unit]
Description=Etcd Server
After=network.target

[Service]
Type=simple
WorkingDirectory=/var/lib/etcd
EnvironmentFile=-/etc/etcd/cfg/etcd.conf
# set GOMAXPROCS to number of processors
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/local/bin/etcd"
Type=notify

[Install]
WantedBy=multi-user.target
EOF

• 建立 var 存放資訊，然後啟動 Etcd 服務:

$  scp /etc/etcd/cfg/* 192.168.184.29:/etc/etcd/cfg/
$  scp /etc/etcd/cfg/* 192.168.184.30:/etc/etcd/cfg/
$  scp /lib/systemd/system/etcd.service 192.168.184.29:/lib/systemd/system/etcd.service
$  scp /lib/systemd/system/etcd.service 192.168.184.30:/lib/systemd/system/etcd.service

• 修改192.168.184.29和192.168.184.30

• 啟動

$ systemctl enable etcd.service && systemctl start etcd.service

• 檢視叢集健康狀態

$ etcdctl --endpoints=https://192.168.184.28:2379 --ca-file=/etc/etcd/ssl/ca.pem --cert-file=/etc/etcd/ssl/etcd.pem --key-file=/etc/etcd/ssl/etcd-key.pem member list

本作品採用《CC 協議》，轉載必須註明作者和本文連結