1. 背景：今天在叢集中搭建 Prometheus 監控，在使用 kubectl apply -f [檔名] 時，出現了報錯，我對叢集資源、名稱空間、許可權等進行一系列排查，甚至在沒部署任何服務新叢集部署該服務依舊顯示部署失敗。
第一次使用 kubectl apply -f 檔案命 顯示報錯。
#kubectl apply -f setup/
customresourcedefinition.apiextensions.k8s.io/alertmanagerconfigs.monitoring.coreos.com created
customresourcedefinition.apiextensions.k8s.io/podmonitors.monitoring.coreos.com created
customresourcedefinition.apiextensions.k8s.io/probes.monitoring.coreos.com created
customresourcedefinition.apiextensions.k8s.io/prometheusrules.monitoring.coreos.com created
customresourcedefinition.apiextensions.k8s.io/scrapeconfigs.monitoring.coreos.com created
customresourcedefinition.apiextensions.k8s.io/servicemonitors.monitoring.coreos.com created
namespace/monitoring created
Error from server (Invalid): error when creating "setup/0alertmanagerCustomResourceDefinition.yaml": CustomResourceDefinition.apiextensions.k8s.io "alertmanagers.monitoring.coreos.com" is invalid: metadata.annotations: Too long: must have at most 262144 bytes
Error from server (Invalid): error when creating "setup/0prometheusCustomResourceDefinition.yaml": CustomResourceDefinition.apiextensions.k8s.io "prometheuses.monitoring.coreos.com" is invalid: metadata.annotations: Too long: must have at most 262144 bytes
Error from server (Invalid): error when creating "setup/0prometheusagentCustomResourceDefinition.yaml": CustomResourceDefinition.apiextensions.k8s.io "prometheusagents.monitoring.coreos.com" is invalid: metadata.annotations: Too long: must have at most 262144 bytes
Error from server (Invalid): error when creating "setup/0thanosrulerCustomResourceDefinition.yaml": CustomResourceDefinition.apiextensions.k8s.io "thanosrulers.monitoring.coreos.com" is invalid: metadata.annotations: Too long: must have at most 262144 bytes
使用 kubectl delete -f setup/ --grace-period=0 --force 強制刪除建立yaml。
kubectl delete -f setup/ --grace-period=0 --force
Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
customresourcedefinition.apiextensions.k8s.io "alertmanagerconfigs.monitoring.coreos.com" force deleted
customresourcedefinition.apiextensions.k8s.io "podmonitors.monitoring.coreos.com" force deleted
customresourcedefinition.apiextensions.k8s.io "probes.monitoring.coreos.com" force deleted
customresourcedefinition.apiextensions.k8s.io "prometheusrules.monitoring.coreos.com" force deleted
customresourcedefinition.apiextensions.k8s.io "scrapeconfigs.monitoring.coreos.com" force deleted
customresourcedefinition.apiextensions.k8s.io "servicemonitors.monitoring.coreos.com" force deleted
namespace "monitoring" force deleted
Error from server (NotFound): error when deleting "setup/0alertmanagerCustomResourceDefinition.yaml": customresourcedefinitions.apiextensions.k8s.io "alertmanagers.monitoring.coreos.com" not found
Error from server (NotFound): error when deleting "setup/0prometheusCustomResourceDefinition.yaml": customresourcedefinitions.apiextensions.k8s.io "prometheuses.monitoring.coreos.com" not found
Error from server (NotFound): error when deleting "setup/0prometheusagentCustomResourceDefinition.yaml": customresourcedefinitions.apiextensions.k8s.io "prometheusagents.monitoring.coreos.com" not found
Error from server (NotFound): error when deleting "setup/0thanosrulerCustomResourceDefinition.yaml": customresourcedefinitions.apiextensions.k8s.io "thanosrulers.monitoring.coreos.com" not found
重新使用 kubectl create -f setup/ 顯示成功。
#kubectl create -f setup/
customresourcedefinition.apiextensions.k8s.io/alertmanagerconfigs.monitoring.coreos.com created
customresourcedefinition.apiextensions.k8s.io/alertmanagers.monitoring.coreos.com created
customresourcedefinition.apiextensions.k8s.io/podmonitors.monitoring.coreos.com created
customresourcedefinition.apiextensions.k8s.io/probes.monitoring.coreos.com created
customresourcedefinition.apiextensions.k8s.io/prometheuses.monitoring.coreos.com created
customresourcedefinition.apiextensions.k8s.io/prometheusagents.monitoring.coreos.com created
customresourcedefinition.apiextensions.k8s.io/prometheusrules.monitoring.coreos.com created
customresourcedefinition.apiextensions.k8s.io/scrapeconfigs.monitoring.coreos.com created
customresourcedefinition.apiextensions.k8s.io/servicemonitors.monitoring.coreos.com created
customresourcedefinition.apiextensions.k8s.io/thanosrulers.monitoring.coreos.com created
namespace/monitoring created
2. 經過查詢各種相關資訊，在知識星球有篇文章可以解釋create與apply在建立yaml有很大區別。
1. kubectl create 命令只是簡單地建立資源物件，它不會去檢查或者處理物件的註解資料，所以它不會因為註解資料過大而報錯。
2. kubectl apply 命令則會對資源物件進行更復雜的處理。在建立或者更新資源物件時，它會將整個資源物件的配置資料（包括註解資料）儲存在 kubectl.kubernetes.io/last-applied-configuration 註解中。這樣在以後的 apply 操作中，kubectl 就可以透過比較這個註解中的配置資料和當前的配置資料，來決定哪些欄位需要更新，哪些欄位不需要更新。由於這個原因，如果你的註解資料過大，超過了 Kubernetes 對註解資料大小的限制，那麼 kubectl apply 就會報錯。
總結，如果你的註解資料過大，你可以選擇使用 kubectl create 命令來建立資源物件，但是你需要注意，這樣做之後你就不能再使用 kubectl apply 命令來更新這個資源物件了，因為 apply 命令仍然會因為註解資料過大而報錯。你需要使用其他的命令，比如 kubectl edit 或者 kubectl patch 來更新資源物件。