kubernetes實踐之十：Kubernetes-dashboard+Heapster+InfluxDB+Grafana

百聯達發表於2018-04-01
UXGrafana
一：前言

1.kubernetes-dashboard 官方提供的使用者管理Kubernets叢集視覺化工具

2.cAdvisor是用於監控容器執行狀態的利器之一。在kubernetes系統中，cAdvisor已經被預設整合到了kubelet元件內，當kubelet服務啟動時，它會自動啟動cAdvisor服務，然後cAdvisor會實時採集所在節點的效能指標及節點上執行的容器的效能指標。cAdvisor主頁顯示了主機的實時執行狀態，包括CPU使用情況、記憶體使用情況、網路吞吐量及檔案系統使用情況等資訊。但是cAdvisor只提供了單機的容器資源佔用情況，而在大規模容器叢集中，需要對所有的Node和全部容器進行效能監控。這就需要一套工具來實現叢集效能資料的採集、儲存和展示。

3.Heapster 提供整個叢集的資源監控，並支援持久化資料儲存到InfluxDB或者其他的儲存後端。Heapster從kubelet提供的API採集節點和容器的資源佔用。另外，Heapster的 /metrics API提供了Prometheus格式的資料。

4.InfluxDB是一個開源分散式時序、事件和指標資料庫。

5.Grafana是一個開源儀表盤工具，它可用於Graphite、InfluxDB與 OpenTSDB 一起使用。最新的版本還可以用於其他的資料來源，比如Elasticsearch。（與heapster與Prometheus均可整合）

6.Zabbix是一個基於WEB介面的提供分散式系統監視以及網路監視功能的企業級的開源解決方案.多用於傳統（非微服務）分散式系統的監控。

7.Prometheus 容器時代的產物。整合採集、儲存（支援多種時序資料庫）、頁面展示與一體的監控工具。

監控方案：
a.Heapster+InfluxDB+Grafana
b.Prometheus

二：安裝
1.映象
k8s.gcr.io/kubernetes-dashboard-amd64：v1.8.2
gcr.io/google_containers/heapster-amd64：v1.4.2
gcr.io/google_containers/heapster-influxdb-amd64：v1.3.3 
gcr.io/google_containers/heapster-grafana-amd64：v4.4.3

2.yaml檔案


admin-user.yaml(不需修改)

點選(此處)摺疊或開啟

  1. apiVersion: v1
  2. kind: ServiceAccount
  3. metadata:
  4.   name: admin-user
  5.   namespace: kube-system
  6. ---
  7. apiVersion: rbac.authorization.k8s.io/v1
  8. kind: ClusterRoleBinding
  9. metadata:
  10.   name: admin-user
  11. roleRef:
  12.   apiGroup: rbac.authorization.k8s.io
  13.   kind: ClusterRole
  14.   name: cluster-admin
  15. subjects:
  16. - kind: ServiceAccount
  17.   name: admin-user
  18.   namespace: kube-system
dashboard.yaml(橙色標示是修改的地方。特別說明當前版本需要新增 - --heapster-host=http://heapster 配置，否則無法正常獲取metrics )

點選(此處)摺疊或開啟

  1. # Copyright 2017 The Kubernetes Authors.
  2. #
  3. # Licensed under the Apache License, Version 2.0 (the "License");
  4. # you may not use this file except in compliance with the License.
  5. # You may obtain a copy of the License at
  6. #
  7. # http://www.apache.org/licenses/LICENSE-2.0
  8. #
  9. # Unless required by applicable law or agreed to in writing, software
  10. # distributed under the License is distributed on an "AS IS" BASIS,
  11. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  12. # See the License for the specific language governing permissions and
  13. # limitations under the License.

  15. # Configuration to deploy release version of the Dashboard UI compatible with
  16. # Kubernetes 1.8.
  17. #
  18. # Example usage: kubectl create -f

  20. # ------------------- Dashboard Secret ------------------- #

  22. apiVersion: v1
  23. kind: Secret
  24. metadata:
  25.   labels:
  26.     k8s-app: kubernetes-dashboard
  27.   name: kubernetes-dashboard-certs
  28.   namespace: kube-system
  29. type: Opaque

  31. ---
  32. # ------------------- Dashboard Service Account ------------------- #

  34. apiVersion: v1
  35. kind: ServiceAccount
  36. metadata:
  37.   labels:
  38.     k8s-app: kubernetes-dashboard
  39.   name: kubernetes-dashboard
  40.   namespace: kube-system

  42. ---
  43. # ------------------- Dashboard Role & Role Binding ------------------- #

  45. kind: Role
  46. apiVersion: rbac.authorization.k8s.io/v1
  47. metadata:
  48.   name: kubernetes-dashboard-minimal
  49.   namespace: kube-system
  50. rules:
  51.   # Allow Dashboard to create 'kubernetes-dashboard-key-holder' secret.
  52. - apiGroups: [""]
  53.   resources: ["secrets"]
  54.   verbs: ["create"]
  55.   # Allow Dashboard to create 'kubernetes-dashboard-settings' config map.
  56. - apiGroups: [""]
  57.   resources: ["configmaps"]
  58.   verbs: ["create"]
  59.   # Allow Dashboard to get, update and delete Dashboard exclusive secrets.
  60. - apiGroups: [""]
  61.   resources: ["secrets"]
  62.   resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs"]
  63.   verbs: ["get", "update", "delete"]
  64.   # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
  65. - apiGroups: [""]
  66.   resources: ["configmaps"]
  67.   resourceNames: ["kubernetes-dashboard-settings"]
  68.   verbs: ["get", "update"]
  69.   # Allow Dashboard to get metrics from heapster.
  70. - apiGroups: [""]
  71.   resources: ["services"]
  72.   resourceNames: ["heapster"]
  73.   verbs: ["proxy"]
  74. - apiGroups: [""]
  75.   resources: ["services/proxy"]
  76.   resourceNames: ["heapster", "http:heapster:", "https:heapster:"]
  77.   verbs: ["get"]

  79. ---
  80. apiVersion: rbac.authorization.k8s.io/v1
  81. kind: RoleBinding
  82. metadata:
  83.   name: kubernetes-dashboard-minimal
  84.   namespace: kube-system
  85. roleRef:
  86.   apiGroup: rbac.authorization.k8s.io
  87.   kind: Role
  88.   name: kubernetes-dashboard-minimal
  89. subjects:
  90. - kind: ServiceAccount
  91.   name: kubernetes-dashboard
  92.   namespace: kube-system

  94. ---
  95. # ------------------- Dashboard Deployment ------------------- #

  97. kind: Deployment
  98. apiVersion: apps/v1beta2
  99. metadata:
  100.   labels:
  101.     k8s-app: kubernetes-dashboard
  102.   name: kubernetes-dashboard
  103.   namespace: kube-system
  104. spec:
  105.   replicas: 1
  106.   revisionHistoryLimit: 10
  107.   selector:
  108.     matchLabels:
  109.       k8s-app: kubernetes-dashboard
  110.   template:
  111.     metadata:
  112.       labels:
  113.         k8s-app: kubernetes-dashboard
  114.     spec:
  115.       containers:
  116.       - name: kubernetes-dashboard
  117.         image: k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.2
  118.         ports:
  119.         - containerPort: 8443
  120.           protocol: TCP
  121.         args:
  122.           - --auto-generate-certificates
  123.           # Uncomment the following line to manually specify Kubernetes API server Host
  124.           # If not specified, Dashboard will attempt to auto discover the API server and connect
  125.           # to it. Uncomment only if the default does not work.
  126.           # - --apiserver-host=http://my-address:port
  127.           - --heapster-host=http://heapster
  128.         volumeMounts:
  129.         - name: kubernetes-dashboard-certs
  130.           mountPath: /certs
  131.           # Create on-disk volume to store exec logs
  132.         - mountPath: /tmp
  133.           name: tmp-volume
  134.         livenessProbe:
  135.           httpGet:
  136.             scheme: HTTPS
  137.             path: /
  138.             port: 8443
  139.           initialDelaySeconds: 30
  140.           timeoutSeconds: 30
  141.       volumes:
  142.       - name: kubernetes-dashboard-certs
  143.         secret:
  144.           secretName: kubernetes-dashboard-certs
  145.       - name: tmp-volume
  146.         emptyDir: {}
  147.       serviceAccountName: kubernetes-dashboard
  148.       # Comment the following tolerations if Dashboard must not be deployed on master
  149.       tolerations:
  150.       - key: node-role.kubernetes.io/master
  151.         effect: NoSchedule

  153. ---
  154. # ------------------- Dashboard Service ------------------- #

  156. kind: Service
  157. apiVersion: v1
  158. metadata:
  159.   labels:
  160.     k8s-app: kubernetes-dashboard
  161.   name: kubernetes-dashboard
  162.   namespace: kube-system
  163. spec:
  164.   type: NodePort
  165.   ports:
  166.     - port: 443
  167.       targetPort: 8443
  168.       nodePort: 32666
  169.   selector:
  170.     k8s-app: kubernetes-dashboard
grafana.yaml(橙色標示為需要修改的地方)

點選(此處)摺疊或開啟

  1. apiVersion: extensions/v1beta1
  2. kind: Deployment
  3. metadata:
  4.   name: monitoring-grafana
  5.   namespace: kube-system
  6. spec:
  7.   replicas: 1
  8.   template:
  9.     metadata:
  10.       labels:
  11.         task: monitoring
  12.         k8s-app: grafana
  13.     spec:
  14.       containers:
  15.       - name: grafana
  16.         image: gcr.io/google_containers/heapster-grafana-amd64:v4.4.3
  17.         ports:
  18.         - containerPort: 3000
  19.           protocol: TCP
  20.         volumeMounts:
  21.         - mountPath: /etc/ssl/certs
  22.           name: ca-certificates
  23.           readOnly: true
  24.         - mountPath: /var
  25.           name: grafana-storage
  26.         env:
  27.         - name: INFLUXDB_HOST
  28.           value: monitoring-influxdb
  29.         - name: GF_SERVER_HTTP_PORT
  30.           value: "3000"
  31.           # The following env variables are required to make Grafana accessible via
  32.           # the kubernetes api-server proxy. On production clusters, we recommend
  33.           # removing these env variables, setup auth for grafana, and expose the grafana
  34.           # service using a LoadBalancer or a public IP.
  35.         - name: GF_AUTH_BASIC_ENABLED
  36.           value: "false"
  37.         - name: GF_AUTH_ANONYMOUS_ENABLED
  38.           value: "true"
  39.         - name: GF_AUTH_ANONYMOUS_ORG_ROLE
  40.           value: Admin
  41.         - name: GF_SERVER_ROOT_URL
  42.           # If you're only using the API Server proxy, set this value instead:
  43.           # value: /api/v1/namespaces/kube-system/services/monitoring-grafana/proxy
  44.           value: /
  45.       volumes:
  46.       - name: ca-certificates
  47.         hostPath:
  48.           path: /etc/ssl/certs
  49.       - name: grafana-storage
  50.         emptyDir: {}
  51. ---
  52. apiVersion: v1
  53. kind: Service
  54. metadata:
  55.   labels:
  56.     # For use as a Cluster add-on (https://github.com/kubernetes/kubernetes/tree/master/cluster/addons)
  57.     # If you are NOT using this as an addon, you should comment out this line.
  58.     kubernetes.io/cluster-service: 'true
heapster-rbac.yaml（不需要修改）

點選(此處)摺疊或開啟

  1. kind: ClusterRoleBinding
  2. apiVersion: rbac.authorization.k8s.io/v1beta1
  3. metadata:
  4.   name: heapster
  5. roleRef:
  6.   apiGroup: rbac.authorization.k8s.io
  7.   kind: ClusterRole
  8.   name: system:heapster
  9. subjects:
  10. - kind: ServiceAccount
  11.   name: heapster
  12.   namespace: kube-system
heapster.yaml（橙色標示為需要修改的地方）

點選(此處)摺疊或開啟

  1. apiVersion: v1
  2. kind: ServiceAccount
  3. metadata:
  4.   name: heapster
  5.   namespace: kube-system
  6. ---
  7. apiVersion: extensions/v1beta1
  8. kind: Deployment
  9. metadata:
  10.   name: heapster
  11.   namespace: kube-system
  12. spec:
  13.   replicas: 1
  14.   template:
  15.     metadata:
  16.       labels:
  17.         task: monitoring
  18.         k8s-app: heapster
  19.     spec:
  20.       serviceAccountName: heapster
  21.       containers:
  22.       - name: heapster
  23.         image: gcr.io/google_containers/heapster-amd64:v1.4.2
  24.         imagePullPolicy: IfNotPresent
  25.         command:
  26.         - /heapster
  27.         - --source=kubernetes:https://10.116.137.196:6443
  28.         - --sink=influxdb:http://monitoring-influxdb.kube-system.svc:8086
  29. ---
  30. apiVersion: v1
  31. kind: Service
  32. metadata:
  33.   labels:
  34.     task: monitoring
  35.     # For use as a Cluster add-on (https://github.com/kubernetes/kubernetes/tree/master/cluster/addons)
  36.     # If you are NOT using this as an addon, you should comment out this line.
  37.     kubernetes.io/cluster-service: 'true'
  38.     kubernetes.io/name: Heapster
  39.   name: heapster
  40.   namespace: kube-system
  41. spec:
  42.   ports:
  43.   - port: 80
  44.     targetPort: 8082
  45.   selector:
  46.     k8s-app: heapster
influxdb.yaml（橙色標示為需要修改的地方）

點選(此處)摺疊或開啟

  1. apiVersion: extensions/v1beta1
  2. kind: Deployment
  3. metadata:
  4.   name: monitoring-influxdb
  5.   namespace: kube-system
  6. spec:
  7.   replicas: 1
  8.   template:
  9.     metadata:
  10.       labels:
  11.         task: monitoring
  12.         k8s-app: influxdb
  13.     spec:
  14.       containers:
  15.       - name: influxdb
  16.         image: gcr.io/google_containers/heapster-influxdb-amd64:v1.3.3
  17.         volumeMounts:
  18.         - mountPath: /data
  19.           name: influxdb-storage
  20.       volumes:
  21.       - name: influxdb-storage
  22.         emptyDir: {}
  23. ---
  24. apiVersion: v1
  25. kind: Service
  26. metadata:
  27.   labels:
  28.     task: monitoring
  29.     # For use as a Cluster add-on (https://github.com/kubernetes/kubernetes/tree/master/cluster/addons)
  30.     # If you are NOT using this as an addon, you should comment out this line.
  31.     kubernetes.io/cluster-service: 'true'
  32.     kubernetes.io/name: monitoring-influxdb
  33.   name: monitoring-influxdb
  34.   namespace: kube-system
  35. spec:
  36.   ports:
  37.   - port: 8086
  38.     targetPort: 8086
  39.   selector:
  40.     k8s-app: influxdb
dashboard.yaml 檔案源於https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml

其他yaml.檔案源於
https://codeload.github.com/kubernetes/heapster/zip/v1.4.2  下載後解壓縮  位於目錄\deploy\kube-config\influxdb

3.建立相關資源
執行命令：
kubectl create -f admin-user.yaml -f dashboard.yaml -f grafana.yaml -f heapster-rbac.yaml -f heapster.yaml -f influxdb.yaml
檢視建立狀態：
kubectl get pods --all-namespaces

三：訪問
建議使用Firefox瀏覽器訪問https://nodeIp:32666



選擇使用令牌登入的方式，  生成令牌的命令如下（請儲存好生成的Token）：

點選(此處)摺疊或開啟

  1. kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}') | grep token
成功登入後的頁面：

來自 “ ITPUB部落格 ” ，連結：http://blog.itpub.net/28624388/viewspace-2152479/，如需轉載，請註明出處，否則將追究法律責任。

相關文章