Prometheus-operator新增basic auth

白羽翎發表於2024-07-19
Prometheus

問題：Prometheus存在未授權訪問，會導致/debug/pprof資訊洩露漏洞

image

解決方法：給prometheus新增basic auth

由於operator部署的prometheus是透過crd控制的，不可以透過修改控制器的方式修改pod的配置，所以修改statefulset無法生效，需要透過修改prometheus物件新增basic auth許可權。
相關部落格參考：https://blog.csdn.net/zfw_666666/article/details/126351312

具體步驟

1、對密碼透過 bcrypt 加密

# 這裡密碼設定為admin
[root@master01 ~]# htpasswd -nBC 10 "" | tr -d ':\n'
New password: 
Re-type new password: 
$2y$10$w5f0QZ5dM.NQ93mG95ZYv..BWlw5hqa1O0fRT.TXBJZbUngcJefRW

2、建立web-config.yaml

# 這裡使用者名稱也設定為admin
[root@master01 ~]# cat web.yaml 
basic_auth_users:
  admin: $2y$10$w5f0QZ5dM.NQ93mG95ZYv..BWlw5hqa1O0fRT.TXBJZbUngcJefRW

3、對web-config.yaml進行base64加密

[root@master01 ~]# cat web.yaml |base64 -w 0
YmFzaWNfYXV0aF91c2VyczoKICBhZG1pbjogJDJ5JDEwJHc1ZjBRWjVkTS5OUTkzbUc5NVpZdi4uQldsdzVocWExTzBmUlQuVFhCSlpiVW5nY0plZlJXCg==

4、建立secret

[root@master01 ~]# cat web-secret.yaml
apiVersion: v1
data:
  web.yaml: "YmFzaWNfYXV0aF91c2VyczoKICBhZG1pbjogJDJ5JDEwJHc1ZjBRWjVkTS5OUTkzbUc5NVpZdi4uQldsdzVocWExTzBmUlQuVFhCSlpiVW5nY0plZlJXCg=="
kind: Secret
metadata:
  name: prometheus-basic-auth
  namespace: monitoring
type: Opaque
[root@master01 ~]# kubectl apply -f web-secret.yaml
secret/prometheus-basic-auth created
[root@master01 ~]# kubectl get secret -n monitoring|grep prometheus-basic-auth
prometheus-basic-auth            Opaque   1      105m

5、修改prometheus物件

點選檢視程式碼 
[root@master01 ~]# kubectl get prometheus -A
NAMESPACE    NAME   VERSION   DESIRED   READY   RECONCILED   AVAILABLE   AGE
monitoring   k8s    2.48.1    2         2       True         True        19h
[root@master01 ~]# kubectl edit prometheus -n monitoring  k8s
apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  creationTimestamp: "2024-06-28T16:08:40Z"
  generation: 14
  labels:
    app.kubernetes.io/component: prometheus
    app.kubernetes.io/instance: k8s
    app.kubernetes.io/name: prometheus
    app.kubernetes.io/part-of: kube-prometheus
    app.kubernetes.io/version: 2.48.1
  name: k8s
  namespace: monitoring
  resourceVersion: "146719"
  uid: cf2abfde-4d20-4543-a108-393343f1539e
spec:
  alerting:
    alertmanagers:
    - apiVersion: v2
      name: alertmanager-main
      namespace: monitoring
      port: web
  # 下面開始一直到enableFeatures: []都需要新增，保活探針、就緒探針、啟動探針都需要新增認證
  # 且args中的引數以及探針中的具體port名要和原pod中的保持一致
  containers:
  - args:
    - --web.console.templates=/etc/prometheus/consoles
    - --web.console.libraries=/etc/prometheus/console_libraries
    - --config.file=/etc/prometheus/config_out/prometheus.env.yaml
    - --web.enable-lifecycle
    - --web.route-prefix=/
    - --storage.tsdb.retention.time=24h
    - --storage.tsdb.path=/prometheus
    # 需要修改的web.yaml會掛在容器內的這個位置
    - --web.config.file=/etc/prometheus/secrets/prometheus-basic-auth/web.yaml
    livenessProbe:
      failureThreshold: 6
      httpGet:
        httpHeaders:
        - name: Authorization
          # Basic之後的值為admin:admin透過base64加密得到的
          # echo -n "admin:admin" | base64
          value: Basic YWRtaW46YWRtaW4=
        path: /-/healthy
        port: web
        scheme: HTTP
      periodSeconds: 5
      successThreshold: 1
      timeoutSeconds: 3
    name: prometheus
    ports:
    - containerPort: 9090
      name: web
      protocol: TCP
    readinessProbe:
      failureThreshold: 3
      httpGet:
        httpHeaders:
        - name: Authorization
          value: Basic YWRtaW46YWRtaW4=
        path: /-/ready
        port: web
        scheme: HTTP
      periodSeconds: 5
      successThreshold: 1
      timeoutSeconds: 3
    startupProbe:
      failureThreshold: 60
      httpGet:
        httpHeaders:
        - name: Authorization
          value: Basic YWRtaW46YWRtaW4=
        path: /-/ready
        port: web
        scheme: HTTP
      periodSeconds: 15
      successThreshold: 1
      timeoutSeconds: 3
  enableFeatures: []
  evaluationInterval: 30s
  externalLabels: {}
  image: quay.io/prometheus/prometheus:v2.48.1
  nodeSelector:
    kubernetes.io/os: linux
  podMetadata:
    labels:
      app.kubernetes.io/component: prometheus
      app.kubernetes.io/instance: k8s
      app.kubernetes.io/name: prometheus
      app.kubernetes.io/part-of: kube-prometheus
      app.kubernetes.io/version: 2.48.1
  podMonitorNamespaceSelector: {}
  podMonitorSelector: {}
  portName: web
  probeNamespaceSelector: {}
  probeSelector: {}
  replicas: 2
  resources:
    requests:
      memory: 400Mi
  ruleNamespaceSelector: {}
  ruleSelector: {}
  scrapeConfigNamespaceSelector: {}
  scrapeConfigSelector: {}
  scrapeInterval: 30s
  # 此處新增之前apply的secret名
  # 容器執行成功後會將web.yaml掛載進/etc/prometheus/secrets/prometheus-basic-auth/目錄下
  secrets:
  - prometheus-basic-auth
  securityContext:
    fsGroup: 2000
    runAsNonRoot: true
    runAsUser: 1000
  serviceAccountName: prometheus-k8s
  serviceMonitorNamespaceSelector: {}
  serviceMonitorSelector: {}
  version: 2.48.1
status:
  availableReplicas: 2
  conditions:
  - lastTransitionTime: "2024-06-29T07:11:12Z"
    message: ""
    observedGeneration: 14
    reason: ""
    status: "True"
    type: Available
  - lastTransitionTime: "2024-06-29T07:11:12Z"
    message: ""
    observedGeneration: 14
    reason: ""
    status: "True"
    type: Reconciled
  paused: false
  replicas: 2
  shardStatuses:
    - availableReplicas: 2
    replicas: 2
    shardID: "0"
    unavailableReplicas: 0
    updatedReplicas: 2
  unavailableReplicas: 0
  updatedReplicas: 2

6、檢視pod並進入容器檢視secret掛載情況

[root@master01 ~]# kubectl get po -A|grep prometheus
monitoring    prometheus-adapter-d4fcc8bb9-4q6dv               1/1     Running            0                15h
monitoring    prometheus-adapter-d4fcc8bb9-q7zns               1/1     Running            0                15h
monitoring    prometheus-k8s-0                                 2/2     Running            0                126m
monitoring    prometheus-k8s-1                                 2/2     Running            0                126m
monitoring    prometheus-operator-94d855d67-n4lrh              2/2     Running            0                15h
[root@master01 ~]# kubectl exec -it -n monitoring    prometheus-k8s-1 -- /bin/sh
/prometheus $ ls
chunks_head     lock            queries.active  wal
/prometheus $ cd /etc/prometheus/secrets/prometheus-basic-auth/
/etc/prometheus/secrets/prometheus-basic-auth $ ls
web.yaml
/etc/prometheus/secrets/prometheus-basic-auth $ cat web.yaml 
basic_auth_users:
  admin: $2y$10$w5f0QZ5dM.NQ93mG95ZYv..BWlw5hqa1O0fRT.TXBJZbUngcJefRW
/etc/prometheus/secrets/prometheus-basic-auth $ exit

7、訪問prometheus的web ui（前提：開啟NodePort訪問）

[root@master01 ~]# kubectl get svc -A|grep prometheus-k8s
monitoring    prometheus-k8s                  NodePort    10.233.37.70    <none>        9090:32662/TCP,8080:30408/TCP   20h

瀏覽器輸入ip:32662/debug/pprof，提示需要使用者名稱和密碼
image

輸入使用者名稱和密碼後，登入成功！

image

相關文章