使用Rancher在Kubernetes上部署EMQ X叢集

emqx發表於2019-12-23
MQ

本文描述如何通過Rancher2.0部署kubernetes叢集，並將EMQ X部署到kubernetes叢集上

實驗環境：

  • 公有云環境：AWS EC2
  • 作業系統：ubuntu 16.04
  • Docker version：18.09.0

通過Rancher部署kubernetes叢集

Rancher的安裝以及部署kubernetes叢集的步驟推薦直接按照快速入門執行。

建立Rancher Api金鑰

EMQ X通過訪問kube-apiserver來實現自動叢集功能，在Rancher中，Rancher對kube-apiserver做了一層代理，在訪問kube-apiserver的時候必須提供用於向Rancher進行身份驗證的API金鑰。參考使用者手冊建立並儲存API Key。本實驗中建立的Access Key為： token-dksbl，Secret Key為： pshhhf5cp8d5v5x7bzjdm82qfrwgx7f2bzksnr748j42xmbvvklbdz，組合成的Token為： token-dksbl:pshhhf5cp8d5v5x7bzjdm82qfrwgx7f2bzksnr748j42xmbvvklbdz

下載並配置kubectl

  1. 下載並安裝kubectl
  2. 進入Rancher叢集頁面，點選Kubeconfig檔案。

WX20190803103351.png

將kubeconfig檔案儲存到 ~/.kube/config

WX20190803103449.png

執行 kubectl cluster-info驗證配置是否成功

$ kubectl cluster-infoKubernetes master is running at https://13.125.244.172/k8s/clusters/c-vvgjq
KubeDNS is running at https://13.125.244.172/k8s/clusters/c-vvgjq/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

訪問kube-apiserver

EMQ X通過訪問kube-apiserver來實現自動叢集，kube-apiserver的地址可以檢視 ~/.ssh/config檔案或者執行 kubectl cluster-info獲取，本實驗中kube-apiserver的地址為： https://13.125.244.172/k8s/clusters/c-vvgjq。

直接訪問kube-apiserver，可以看到會報錯需要認證。

$ curl -k https://13.125.244.172/k8s/clusters/c-vvgjq{"type":"error","status":"401","message":"must authenticate"}

在頭部加上Authorization認證則可以正常訪問

$ curl -k -H 'Authorization: Bearer token-dksbl:pshhhf5cp8d5v5x7bzjdm82qfrwgx7f2bzksnr748j42xmbvvklbdz' https://13.125.244.172/k8s/clusters/c-vvgjq

編輯emqx.yaml

在Kubernetes 上安裝 EMQ X 系列文章之二 ：EMQ X 自動叢集一文中分享了EMQ X部署kubernetes叢集的yaml檔案如下，在Rancher上部署EMQ X叢集的話需要稍加改動。

$cat emqx.yaml
apiVersion: v1
kind: Service
metadata:
  name: emqx
spec:
  ports:
  - port: 32333
    nodePort: 32333
    targetPort:  emqx-dashboard
    protocol: TCP
  selector:
    app: emqx
  type: NodePort
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: emqx
  labels:
        app: emqx
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: emqx
    spec:
      containers:
      - name: emqx
        image: emqx/emqx:latest
        ports:
        - name: emqx-dashboard
          containerPort: 18083
        env:
        - name: EMQX_CLUSTER__DISCOVERY
          value: k8s
        - name: EMQX_NAME
          value: emqx
        - name: EMQX_CLUSTER__K8S__APISERVER
          value: http://172.31.19.161:8080
        - name: EMQX_CLUSTER__K8S__NAMESPACE
          value: default
        - name: EMQX_CLUSTER__K8S__SERVICE_NAME
          value: emqx
        - name: EMQX_CLUSTER__K8S__ADDRESS_TYPE
          value: ip
        - name: EMQX_CLUSTER__K8S__APP_NAME
          value: emqx
        tty: true

EMQ X可以讀取 /var/run/secrets/kubernetes.io/serviceaccount/token檔案中的內容組合Authorization認證訪問kube-apiserver，所以只需要把Rancher的API Token通過Secret掛載到容器中就可以了。

Secret解決了密碼、token、金鑰等敏感資料的配置問題，而不需要把這些敏感資料暴露到映象或者Pod Spec中。Secret可以以Volume或者環境變數的方式使用。

Secret有三種型別：

  • Service Account ：用來訪問Kubernetes API，由Kubernetes自動建立，並且會自動掛載到Pod的 /run/secrets/kubernetes.io/serviceaccount目錄中；
  • Opaque ：base64編碼格式的Secret，用來儲存密碼、金鑰等；
  • kubernetes.io/dockerconfigjson ：用來儲存私有docker registry的認證資訊。

首先對API Token做base64編碼

$ echo -n token-dksbl:pshhhf5cp8d5v5x7bzjdm82qfrwgx7f2bzksnr748j42xmbvvklbdz | base64 -w 0dG9rZW4tZGtzYmw6cHNoaGhmNWNwOGQ1djV4N2J6amRtODJxZnJ3Z3g3ZjJiemtzbnI3NDhqNDJ4bWJ2dmtsYmR6

在yaml檔案中建立Secret

$vim emqx.yamlapiVersion: v1
kind: Secret
metadata:
  name: emqx-secret
type: Opaque
data:
  token: dG9rZW4tcGI2MjU6eDZ2eGJ0Y2NmdG1waGpseHR3NGNjdGN2d2txdzk5aDJzYmhxNHFtaDh4c2ZnbXd6dzJ0d2Rw
---
......

修改Deployment，將環境變數中的 EMQX_CLUSTER__K8S__APISERVER改為Rancher的Kube-apiserver的地址，增加volumeMounts

$vim emqx.yaml......
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: emqx
  labels:
        app: emqx
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: emqx
    spec:
      volumes:
      - name: emqx-secret
        secret:
           secretName: emqx-secret
      containers:
      - name: emqx
        image: emqx/emqx:latest
        ports:
        - name: emqx-dashboard
          containerPort: 18083
        - name: emqx-http
          containerPort: 8083
        - name: emqx-mqtt
          containerPort: 1883
        env:
        - name: EMQX_CLUSTER__DISCOVERY
          value: k8s
        - name: EMQX_NAME
          value: emqx
        - name: EMQX_CLUSTER__K8S__APISERVER
          value: https://13.125.244.172/k8s/clusters/c-vvgjq
        - name: EMQX_CLUSTER__K8S__NAMESPACE
          value: default
        - name: EMQX_CLUSTER__K8S__SERVICE_NAME
          value: emqx
        - name: EMQX_CLUSTER__K8S__ADDRESS_TYPE
          value: ip
        - name: EMQX_CLUSTER__K8S__APP_NAME
          value: emqx
        tty: true
        volumeMounts:
          - name: emqx-secret
            mountPath: "/var/run/secrets/kubernetes.io/serviceaccount"
            readOnly: true

部署EMQ X

檢視修改後的emqx.yaml

$cat emqx.yamlapiVersion: v1
kind: Secret
metadata:
  name: emqx-secret
type: Opaque
data:
  token: dG9rZW4tcGI2MjU6eDZ2eGJ0Y2NmdG1waGpseHR3NGNjdGN2d2txdzk5aDJzYmhxNHFtaDh4c2ZnbXd6dzJ0d2Rw
---
apiVersion: v1
kind: Service
metadata:
  name: emqx
spec:
  ports:
  - port: 32333
    nodePort: 32333
    targetPort:  emqx-dashboard
    protocol: TCP
  selector:
    app: emqx
  type: NodePort
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: emqx
  labels:
        app: emqx
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: emqx
    spec:
      volumes:
      - name: emqx-secret
        secret:
           secretName: emqx-secret
      containers:
      - name: emqx
        image: emqx/emqx:latest
        ports:
        - name: emqx-dashboard
          containerPort: 18083
        - name: emqx-http
          containerPort: 8083
        - name: emqx-mqtt
          containerPort: 1883
        env:
        - name: EMQX_CLUSTER__DISCOVERY
          value: k8s
        - name: EMQX_NAME
          value: emqx
        - name: EMQX_CLUSTER__K8S__APISERVER
          value: https://13.125.244.172/k8s/clusters/c-vvgjq
        - name: EMQX_CLUSTER__K8S__NAMESPACE
          value: default
        - name: EMQX_CLUSTER__K8S__SERVICE_NAME
          value: emqx
        - name: EMQX_CLUSTER__K8S__ADDRESS_TYPE
          value: ip
        - name: EMQX_CLUSTER__K8S__APP_NAME
          value: emqx
        tty: true
        volumeMounts:
          - name: emqx-secret
            mountPath: "/var/run/secrets/kubernetes.io/serviceaccount"
            readOnly: true

部署EMQ X

$ kubectl create -f emqx.yamlsecret/emqx-secret created
service/emqx created
deployment.extensions/emqx created

檢視狀態

$ kubectl get podsNAME                       READY   STATUS    RESTARTS   AGE
emqx-67b5fcf4d-gwzfn       1/1     Running   0          36s
emqx-67b5fcf4d-rb7m6       1/1     Running   0          36s

叢集成功

$ kubectl exec emqx-67b5fcf4d-gwzfn /opt/emqx/bin/emqx_ctl cluster statusCluster status: [{running_nodes,['emqx@10.42.1.24','emqx@10.42.2.19']}]

使用Rancher Dashboard部署EMQ X（可選）

刪除剛剛部署的EMQ X

$ kubectl delete -f emqx.yamlsecret "emqx-secret" deleted
service "emqx" deleted
deployment.extensions "emqx" deleted

進入Rancher叢集工作負載頁面，點選匯入YAML
WX20190803103410.png

在匯入頁面將emqx.yaml檔案的內容複製進去!
WX20190803103628.png

點選匯入，等待匯入成功。

來自 “ ITPUB部落格 ” ，連結：http://blog.itpub.net/69958136/viewspace-2669983/，如需轉載，請註明出處，否則將追究法律責任。

相關文章