Airflow 2正式出來了

Airflow 2.0.0, 2020-12-17
https://github.com/apache/airflow/releases/tag/2.0.0

1.新的寫dag指令碼方式。（使用python註釋）

@task
def extract():
return {“1001”: 301.27, “1002”: 433.21, “1003”: 502.22}

2.Fully specified REST API (AIP-32)

更全的 REST API 介面。提供外部呼叫。

3.Massive Scheduler performance improvements

As part of AIP-15 (Scheduler HA+performance) and other work Kamil did, we significantly improved the performance of the Airflow Scheduler. It now starts tasks much, MUCH quicker.
更快的啟動任務

4.Scheduler is now HA compatible (AIP-15)

排程高可用。（主備)

5.Task Groups (AIP-34)

可以在任務中建子任務(一個任務中包含多個子任務)，可以在web UI上鑽取操作。

6.Refreshed UI

自動重新整理介面任務狀態。不用手工老點介面重新整理了。

7.Smart Sensors for reduced load from sensors (AIP-17)

更智慧化的檢查器功能。(例如檢查其它dag中的任務)，在大量使用檢查器時，減小資源使用。

8.Simplified KubernetesExecutor

減化 KubernetesExecutor 執行器。

9.Airflow core and providers: Splitting Airflow into 60+ packages

將airflow 安裝包分成多個安裝包，可單獨下載

10.Configuration

Configuration in the form of the airflow.cfg file has been rationalized further in distinct sections, specifically around “core”. Additionally, a significant amount of configuration options have been deprecated or moved to individual component-specific configuration files, such as the pod-template-file for Kubernetes execution-related configuration.
將配置檔案分成多個獨立的模組檔案。