ArgoWorkflow教程(八)---基於 LifecycleHook 實現流水線通知提醒

探索云原生發表於2024-10-29
原文網址 : https://www.cnblogs.com/KubeExplorer/p/18512868
GoHook

argoworkflow-8-workflow-notify.png

本篇介紹一下 ArgoWorkflow 中的 ExitHandler 和 LifecycleHook 功能，可以根據流水線每一步的不同狀態，執行不同操作，一般用於傳送通知。

1. 概述

本篇介紹一下 ArgoWorkflow 中的 ExitHandler 和 LifecycleHook 功能，可以根據流水線每一步的不同狀態，執行不同操作，一般用於傳送通知。

比如當某個步驟，或者某個 Workflow 執行失敗時，傳送郵件通知。

在 ArgoWorkflow 不同版本中中有兩種實現方式：

  • 1）v2.7 版本開始提供了 exit handler 功能，可以指定一個在流水線執行完成後執行的模板。同時這個模板中還可以使用 when 欄位來做條件配置，以實現比根據當前流水線執行結果來執行不同流程。
    • 已廢棄，v3.3 版本後不推薦使用
  • 2）v.3.3 版本新增 LifecycleHook，exit handler 功能則不推薦使用了，LifecycleHook 提供了更細粒度以及更多功能，exit handler 可以看做是一個簡單的 LifecycleHook。

2. ExitHandler

雖然官方已經不推薦使用該功能了，但是還是簡單介紹一下。

ArgoWorkflow 提供了 spec.onExit 欄位，可以指定一個 template，當 workflow 執行後(不論成功或者失敗)就會執行 onExit 指定的 template。

類似於 Tekton 中的 finally 欄位

同時這個 template 中可以使用 when 欄位來做條件配置。比如根據當前流水線執行結果來執行不同流程。

比如下面這個 Demo,完整 Workflow 內容如下：

# An exit handler is a template reference that executes at the end of the workflow
# irrespective of the success, failure, or error of the primary workflow. To specify
# an exit handler, reference the name of a template in 'spec.onExit'.
# Some common use cases of exit handlers are:
# - sending notifications of workflow status (e.g. e-mail/slack)
# - posting the pass/fail status to a webhook result (e.g. github build result)
# - cleaning up workflow artifacts
# - resubmitting or submitting another workflow
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: exit-handlers-
spec:
  entrypoint: intentional-fail
  onExit: exit-handler
  templates:
    # primary workflow template
    - name: intentional-fail
      container:
        image: alpine:latest
        command: [sh, -c]
        args: ["echo intentional failure; exit 1"]

    # exit handler related templates
    # After the completion of the entrypoint template, the status of the
    # workflow is made available in the global variable {{workflow.status}}.
    # {{workflow.status}} will be one of: Succeeded, Failed, Error
    - name: exit-handler
      steps:
        - - name: notify
            template: send-email
          - name: celebrate
            template: celebrate
            when: "{{workflow.status}} == Succeeded"
          - name: cry
            template: cry
            when: "{{workflow.status}} != Succeeded"
    - name: send-email
      container:
        image: alpine:latest
        command: [sh, -c]
        # Tip: {{workflow.failures}} is a JSON list. If you're using bash to read it, we recommend using jq to manipulate
        # it. For example:
        #
        # echo "{{workflow.failures}}" | jq -r '.[] | "Failed Step: \(.displayName)\tMessage: \(.message)"'
        #
        # Will print a list of all the failed steps and their messages. For more info look up the jq docs.
        # Note: jq is not installed by default on the "alpine:latest" image, however it can be installed with "apk add jq"
        args: ["echo send e-mail: {{workflow.name}} {{workflow.status}} {{workflow.duration}}. Failed steps {{workflow.failures}}"]
    - name: celebrate
      container:
        image: alpine:latest
        command: [sh, -c]
        args: ["echo hooray!"]
    - name: cry
      container:
        image: alpine:latest
        command: [sh, -c]
        args: ["echo boohoo!"]

首先是透過 spec.onExit 欄位配置了一個 template

spec:
  entrypoint: intentional-fail
  onExit: exit-handler

這個 template 內容如下：

    - name: exit-handler
      steps:
        - - name: notify
            template: send-email
          - name: celebrate
            template: celebrate
            when: "{{workflow.status}} == Succeeded"
          - name: cry
            template: cry
            when: "{{workflow.status}} != Succeeded"

內部包含 3 個步驟，每個步驟又是一個 template：

  • 1）傳送郵件，無論成功或者失敗
  • 2）若成功則執行 celebrate
  • 3）若失敗則執行 cry

該 Workflow 不論執行結果如何，都會傳送郵件，郵件內容包含了任務的執行資訊，若是執行成功則會額外列印執行成功，若是執行失敗則會列印執行失敗。

為了簡單，這裡所有操作都使用 echo 命令進行模擬

由於在主 template 中最後執行的是 exit 1 命令，因此會判斷為執行失敗，會傳送郵件並列印失敗資訊，Pod 列表如下：

[root@argo-1 lifecyclehook]# k get po
NAME                                              READY   STATUS      RESTARTS        AGE
exit-handlers-44ltf                               0/2     Error       0               2m45s
exit-handlers-44ltf-cry-1621717811                0/2     Completed   0               2m15s
exit-handlers-44ltf-send-email-2605424148         0/2     Completed   0               2m15s

各個 Pod 日誌

[root@argo-1 lifecyclehook]# k logs -f exit-handlers-44ltf-cry-1621717811
boohoo!
time="2024-05-25T11:34:39.472Z" level=info msg="sub-process exited" argo=true error="<nil>"
[root@argo-1 lifecyclehook]# k logs -f exit-handlers-44ltf-send-email-2605424148
send e-mail: exit-handlers-44ltf Failed 30.435347. Failed steps [{"displayName":"exit-handlers-44ltf","message":"Error (exit code 1)","templateName":"intentional-fail","phase":"Failed","podName":"exit-handlers-44ltf","finishedAt":"2024-05-25T11:34:16Z"}]
time="2024-05-25T11:34:44.424Z" level=info msg="sub-process exited" argo=true error="<nil>"
[root@argo-1 lifecyclehook]# k logs -f exit-handlers-44ltf
intentional failure
time="2024-05-25T11:34:15.856Z" level=info msg="sub-process exited" argo=true error="<nil>"
Error: exit status 1

至此，這個 exitHandler 功能就可以滿足我們基本的通知需求了，比如將結果以郵件發出，或者對接外部系統 Webhook，更加複雜的需求也可以實現。

不過存在一個問題，就是 exitHandler 是 Workflow 級別的，只能整個 Workflow 執行完成才會執行 exitHandler。

如果想要更細粒度的，比如 template 級別則做不到，v3.3 中提供的 LifecycleHook 則實現了更加細粒度的通知。

3. LifecycleHook

LifecycleHook 可以看做是一個比較靈活的 exit hander，官方描述如下：

Put differently, an exit handler is like a workflow-level LifecycleHook with an expression of workflow.status == "Succeeded" or workflow.status == "Failed" or workflow.status == "Error".

LifecycleHook 有兩種級別：

  • Workflow 級別
  • template 級別

Workflow 級別

Workflow 級別的 LifecycleHook 和 exitHandler 基本類似。

下面就是一個 Workflow 級別的 LifecycleHook Demo，完整 Workflow 內容如下：

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: lifecycle-hook-
spec:
  entrypoint: main
  hooks:
    exit: # Exit handler
      template: http
    running:
      expression: workflow.status == "Running"
      template: http
  templates:
    - name: main
      steps:
      - - name: step1
          template: heads
    
    - name: heads
      container:
        image: alpine:3.6
        command: [sh, -c]
        args: ["echo \"it was heads\""]
    
    - name: http
      http:
        # url: http://dummy.restapiexample.com/api/v1/employees
        url: "https://raw.githubusercontent.com/argoproj/argo-workflows/4e450e250168e6b4d51a126b784e90b11a0162bc/pkg/apis/workflow/v1alpha1/generated.swagger.json"

首先是配置 hook

spec:
  entrypoint: main
  hooks:
    exit: # Exit handler
      template: http
    running:
      expression: workflow.status == "Running"
      template: http

可以看到，原有的 onExit 被 hooks 欄位替代了，同時 hooks 欄位支援指定多個 hook，每個 hook 中可以透過 expression 設定不同的條件，只有滿足條件時才會執行。

這裡的 template 則是一個內建的 http 型別的 template

    - name: http
      http:
        # url: http://dummy.restapiexample.com/api/v1/employees
        url: "https://raw.githubusercontent.com/argoproj/argo-workflows/4e450e250168e6b4d51a126b784e90b11a0162bc/pkg/apis/workflow/v1alpha1/generated.swagger.json"

該 Workflow 的主 template 比較簡單，就是使用 echo 命令列印一句話，因此會執行成功，那麼 hooks 中的兩個 hooks 都會執行。

兩個 hook 對應的都是同一個 template，因此會執行兩遍。

template 級別

template 級別的 hooks 則是提供了更細粒度的配置，比如可能使用者比較關心 Workflow 中某一個步驟的狀態，可以單獨為該 template 設定 hook。

下面是一個template 級別的 hooks demo，Workflow 完整內容如下：

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: lifecycle-hook-tmpl-level-
spec:
  entrypoint: main
  templates:
    - name: main
      steps:
        - - name: step-1
            hooks:
              running: # Name of hook does not matter
                # Expr will not support `-` on variable name. Variable should wrap with `[]`
                expression: steps["step-1"].status == "Running"
                template: http
              success:
                expression: steps["step-1"].status == "Succeeded"
                template: http
            template: echo
        - - name: step2
            hooks:
              running:
                expression: steps.step2.status == "Running"
                template: http
              success:
                expression: steps.step2.status == "Succeeded"
                template: http
            template: echo

    - name: echo
      container:
        image: alpine:3.6
        command: [sh, -c]
        args: ["echo \"it was heads\""]

    - name: http
      http:
        # url: http://dummy.restapiexample.com/api/v1/employees
        url: "https://raw.githubusercontent.com/argoproj/argo-workflows/4e450e250168e6b4d51a126b784e90b11a0162bc/pkg/apis/workflow/v1alpha1/generated.swagger.json"

內容和 Workflow 級別的 Demo 差不多，只是 hooks 欄位的位置不同

spec:
  entrypoint: main
  templates:
    - name: main
      steps:
        - - name: step-1
            hooks:
              # ...
            template: echo
        - - name: step2
            hooks:
						  # ...
            template: echo

在 spec.templates 中我們分別為不同的步驟配置了 hooks，相比與 exiHandler 則更加靈活。

如何替代 exitHandler

LifecycleHook 可以完美替代 Exit Handler，就是把 Hook 命名為 exit，雖然 hook 的命名無無關緊要，但是如果是 exit 則是會特殊處理。

官方原文如下：

You must not name a LifecycleHook exit or it becomes an exit handler; otherwise the hook name has no relevance.

這個 exit 直接是寫死在程式碼裡的，具體如下：

const (
    ExitLifecycleEvent = "exit"
)

func (lchs LifecycleHooks) GetExitHook() *LifecycleHook {
    hook, ok := lchs[ExitLifecycleEvent]
    if ok {
       return &hook
    }
    return nil
}

func (lchs LifecycleHooks) HasExitHook() bool {
    return lchs.GetExitHook() != nil
}

那麼我們只需要將 LifecycleHook 命名為 exit 即可替代 exit handler，就像這樣：

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: lifecycle-hook-
spec:
  entrypoint: main
  hooks:
    exit: # if named exit, it'a an Exit handler
      template: http
  templates:
    - name: main
      steps:
      - - name: step1
          template: heads
    - name: http
      http:
        # url: http://dummy.restapiexample.com/api/v1/employees
        url: "https://raw.githubusercontent.com/argoproj/argo-workflows/4e450e250168e6b4d51a126b784e90b11a0162bc/pkg/apis/workflow/v1alpha1/generated.swagger.json"

4. 常見通知模板

通知一般支援 webhook、email、slack、微信通知等方式。

在 ArgoWorkflow 中則是準備對應的模板即可。

Webhook

這應該是最通用的一種方式，收到訊息後具體做什麼事情，可以靈活的在 webhook 服務調整。

對於 ArgoWorkflow 模板就是執行 curl 命令即可，因此只需要一個包含 curl 工具的容器

apiVersion: argoproj.io/v1alpha1
kind: ClusterWorkflowTemplate
metadata:
  name: step-notify-webhook
spec:
  templates:
    - name: webhook
      inputs:
        parameters:
          - name: POSITIONS # 指定什麼時候執行，多個以逗號隔開，例如：Pending,Running,Succeeded,Failed,Error
            value: "Succeeded,Failed,Error"
          - name: WEBHOOK_ENDPOINT
          - name: CURL_VERSION
            default: "8.4.0"

      container:
        image: curlimages/curl:{{inputs.parameters.CURL_VERSION}}
        command: [sh, -cx]
        args: [
          "curl -X POST  -H \"Content-type: application/json\" -d '{
          \"message\": \"{{workflow.name}} {{workflow.status}}\",
          \"workflow\": {
                \"name\": \"{{workflow.name}}\",
                \"namespace\": \"{{workflow.namespace}}\",
                \"uid\": \"{{workflow.uid}}\",
                \"creationTimestamp\": \"{{workflow.creationTimestamp}}\",
                \"status\": \"{{workflow.status}}\"
              }
        }'
        {{inputs.parameters.WEBHOOK_ENDPOINT}}"
        ]

Email

對於郵件方式，這裡簡單提供一個使用 Python 傳送郵件的 Demo。

# use golangcd-lint for lint
apiVersion: argoproj.io/v1alpha1
kind: ClusterWorkflowTemplate
metadata:
  name: step-notify-email
spec:
  templates:
    - name: email
      inputs:
        parameters:
          - name: POSITIONS # 指定什麼時候執行，多個以逗號隔開，例如：Pending,Running,Succeeded,Failed,Error
            value: "Succeeded,Failed,Error"
          - name: CREDENTIALS_SECRET
          - name: TO # 收件人郵箱
          - name: PYTHON_VERSION
            default: "3.8-alpine"
      script:
        image: docker.io/library/python:{{inputs.parameters.PYTHON_VERSION}}
        command: [ python ]
        env:
          - name: TO
            value: '{{inputs.parameters.TO}}'
          - name: HOST
            valueFrom:
              secretKeyRef:
                name: '{{inputs.parameters.CREDENTIALS_SECRET}}'
                key: host
          - name: PORT
            valueFrom:
              secretKeyRef:
                name: '{{inputs.parameters.CREDENTIALS_SECRET}}'
                key: port
          - name: FROM
            valueFrom:
              secretKeyRef:
                name: '{{inputs.parameters.CREDENTIALS_SECRET}}'
                key: from
          - name: USERNAME
            valueFrom:
              secretKeyRef:
                name: '{{inputs.parameters.CREDENTIALS_SECRET}}'
                key: username
          - name: PASSWORD
            valueFrom:
              secretKeyRef:
                name: '{{inputs.parameters.CREDENTIALS_SECRET}}'
                key: password
          - name: TLS
            valueFrom:
              secretKeyRef:
                name: '{{inputs.parameters.CREDENTIALS_SECRET}}'
                key: tls
        source: |
          import smtplib
          import ssl
          import os
          from email.header import Header
          from email.mime.text import MIMEText

          smtp_server = os.getenv('HOST')
          port = os.getenv('PORT')
          sender_email = os.getenv('FROM')
          receiver_emails = os.getenv('TO')
          user = os.getenv('USERNAME')
          password = os.getenv('PASSWORD')
          tls = os.getenv('TLS')

          # 郵件正文,文字格式
          # 構建郵件訊息
          workflow_info = f"""\
            "workflow": {{
              "name": "{{workflow.name}}",
              "namespace": "{{workflow.namespace}}",
              "uid": "{{workflow.uid}}",
              "creationTimestamp": "{{workflow.creationTimestamp}}",
              "status": "{{workflow.status}}"
            }}
          """
          msg = MIMEText(workflow_info, 'plain', 'utf-8')
          # 郵件頭資訊
          msg['From'] = Header(sender_email)  # 傳送者
          msg['To'] = Header(receiver_emails)  # 接收者
          subject = '{{workflow.name}} {{workflow.status}}'
          msg['Subject'] = Header(subject, 'utf-8')  # 郵件主題
          if tls == 'True':
            context = ssl.create_default_context()
            server = smtplib.SMTP_SSL(smtp_server, port, context=context)
          else:
            server = smtplib.SMTP(smtp_server, port)

          if password != '':
            server.login(user, password)

          for receiver in [item for item in receiver_emails.split(' ') if item]:
            server.sendmail(sender_email, receiver, msg.as_string())

            server.quit()

【ArgoWorkflow 系列】持續更新中，搜尋公眾號【探索雲原生】訂閱，閱讀更多文章。

5. 小結

本文主要分析了 Argo 中的通知觸發機制，包括舊版的 exitHandler 以及新版的 LifecycleHook，並提供了幾個簡單的通知模板。

最後則是推薦使用更加靈活的 LifecycleHook。

相關文章