搭建https的es+kibana(7.9.1)

力王7314發表於2024-04-30
HTTP

背景：elasticsearch7需要開啟https才可以建立報警，因此就需要搭建https的elasticsearch

參考官方網站：https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-deploy-elasticsearch.html

第一步，建立crd

kubectl create -f https://download.elastic.co/downloads/eck/2.12.1/crds.yaml

第二步，安裝operator的RDBC

kubectl apply -f https://download.elastic.co/downloads/eck/2.12.1/operator.yaml

第三步，確保operator的pod成功啟動

[root@localhost ~]# kubectl -n elastic-system get pods 
NAME                 READY   STATUS    RESTARTS      AGE
elastic-operator-0   1/1     Running   6 (20h ago)   20h

第四步，建立es叢集

[root@localhost es]# cat es.yaml.bak 
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: eck-cluster
  namespace: elk
spec:
  version: 7.9.1
  http:
    tls:
      selfSignedCertificate:
        disabled: true
  nodeSets:
  - name: default
    count: 1
    config:
      node.master: true
      node.data: true
      node.ingest: true
      node.ml: false
      xpack.graph.enabled: false
      xpack.ml.enabled: false
      xpack.watcher.enabled: false
      xpack.monitoring.collection.enabled: true
      xpack.security.enabled: true
      xpack.security.http.ssl.enabled: true
      xpack.security.authc:
          anonymous:
            username: anonymous
            roles: superuser
            authz_exception: false
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 100Gi
        storageClassName: standard
    podTemplate:
      spec:
        initContainers:
        - name: sysctl
          securityContext:
            privileged: true
          command: ['sh', '-c', 'sysctl -w vm.max_map_count=262144']
        containers:
        - name: elasticsearch
          env:
          - name: ES_JAVA_OPTS
            value: -Xms512m -Xmx512m -Des.allow_insecure_settings=true
          - name: READINESS_PROBE_PROTOCOL
            value: https
          resources:
            requests:
              memory: 2Gi
            limits:
              memory: 4Gi

第五步，確保es正常啟動，我這裡只建立了一個節點，因此是yellow，啟動3個就是green了

[root@localhost es]# kubectl -n elk get es 
NAME          HEALTH   NODES   VERSION   PHASE   AGE
eck-cluster   yellow   1       7.9.1     Ready   61m

第六步，建立kibana，這裡要注意的是紅色字型的名字要對應kubectl -n elk get es顯示的名字

apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: eck-cluster
  namespace: elk
spec:
  version: 7.9.1
  count: 1
  config: 
    #elasticsearch.hosts: ["https://eck-cluster-es-http.elk.svc:9200"]
    elasticsearch.username: "elastic"
    elasticsearch.password: "xxxxx"
  elasticsearchRef:
    name: eck-cluster

第七步，確保es和kibana都正常啟動

[root@localhost es]# kubectl -n elk get pods 
NAME                              READY   STATUS    RESTARTS   AGE
eck-cluster-es-default-0          1/1     Running   0          44m
eck-cluster-kb-7d75dd6758-g2rcp   1/1     Running   0          57m

第八步，開啟埠轉發，訪問kibana UI

kubectl -n elk port-forward --address 0.0.0.0 svc/eck-cluster-kb-http 5601:5601

第九步，可以看到kibana的報警已經可以使用了

補充，如果需要filebeat連線es傳送日誌的話，也需要https協議，紅色字型是開啟https協議的內容具體配置如下：

filebeat.inputs:
- type: log
  processors:
  paths:
    - "/mnt/log/logstash.log"
  fields:
    app: xxx
    index: xxxx
    group: xxx
    namespaces: xxx
    clusterName: "${CLUSTER_NAME:}"
  #沒有新日誌採集後多長時間關閉檔案控制代碼，預設5分鐘，設定成1分鐘，加快檔案控制代碼關閉；
  close_inactive: 1m
  #傳輸了3h後荏沒有傳輸完成的話就強行關閉檔案控制代碼；
  close_timeout: 3h
  #這個配置項也應該配置上，預設值是0表示不清理，不清理的意思是採集過的檔案描述在registry檔案裡永不清理，在執行一段時間後，registry會變大，可能會帶來問題。
  clean_inactive: 72h
  #設定了clean_inactive後就需要設定ignore_older，且要保證ignore_older < clean_inactive
  ignore_older: 70h
  #multiline.pattern: ^20[0-9]{2}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}
  #multiline.negate: true
  #multiline.match: after
  # 和福林確認日誌只需要處理1毫秒內的多行輸出
  #multiline.timeout: 50ms
  # 只保留40行日誌
  #multiline.max_lines: 40
  exclude_files: ['debug']
  json.message_key: message
  json.keys_under_root: true
  json.overwrite_keys: true
  json.add_error_key: true
- type: log
  processors:
  paths:
    - "/mnt/log/requestlog.log"
  fields:
    app: xxx
    index: xxx
    group: xxx
    namespaces: xxx
    clusterName: "${CLUSTER_NAME:}"
  close_inactive: 1m
  close_timeout: 3h
  clean_inactive: 72h
  ignore_older: 70h
  exclude_files: ['debug']
  json.message_key: message
  json.keys_under_root: true
  json.overwrite_keys: true
  json.add_error_key: true
- type: log
  processors:
  paths:
    - "/mnt/log/gc.log"
  fields:
    app: xxxx
    index: gc
    group: xxx
    namespaces: xxxx
    clusterName: "${CLUSTER_NAME:}"
  close_inactive: 1m
  close_timeout: 3h
  clean_inactive: 72h
  ignore_older: 70h
  exclude_files: ['debug']
output.elasticsearch:
  enabled: true
  hosts: ["eck-cluster-es-http.elastic-system.svc:9200"]
  protocol: "https"
  username: "elastic-system-eck-cluster-kibana-user"
  password: "wYPuZ0719ix52U408M6prQXx"
  ssl:
    certificate_authorities: ["/usr/share/filebeat/ca.pem"]
    erification_mode: "certificate"
  index: "%{[fields.index]}-%{+yyyy.MM.dd}"
setup.ilm.enabled: false
setup.template.name: "%{[fields.index]}"
setup.template.pattern: "%{[fields.index]}-*"
setup.template.enabled: false
setup.template.overwrite: true

相關文章