最新 client-java 呼叫 k8s ApiServer

建立許可權繫結

sa-role.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-admin #賬號名
  namespace: kube-system
  
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
  name: my-cluster-admin
rules:
- apiGroups: ["*"]
  resources: ["*"]
  verbs: ["*"]
- nonResourceURLs: ["*"]
  verbs: ["*"]
  
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: clusterrolebinding-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: my-cluster-admin
subjects:
- kind: ServiceAccount
  name: my-admin
  namespace: kube-system

---
apiVersion: v1
kind: Secret
metadata:
  name: my-admin-token
  namespace: kube-system
  annotations:
    kubernetes.io/service-account.name: "my-admin"   
type: kubernetes.io/service-account-token

 kubectl apply -f .

獲取永久token

kubectl get secret my-admin-token -n kube-system -o jsonpath={".data.token"} | base64 -d

k8s apiserver的api檢視

生成k8s token

請求k8s-swagger檔案

curl -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6ImQ4Slh0Mk9lMzd0TXNlZW9sbGRRMUVfRWtYSHVnNnFwMG11TmhYR3dWM2cifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi10b2tlbiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJobC1hZG1pbiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6Ijc4MDk1MDgwLWE2MWYtNGQ0Ni05YTUyLTYxYzMxOTAyYzIyMCIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlLXN5c3RlbTpobC1hZG1pbiJ9.Gy8xiiLngFopG6EJVAUIsP7n9jQqsP6rWEg8q6LcqpaQ42FXYVX01o1wsqi6u5l3H5D4_dI-GjOU1ajc8Y_g4lZu-ClCxn360tsoJ6ZaCg7fuW4LIA2Mr1gT-rv7yLKhYplF6LDwEwsqlAh3nZopoWvMPtAKWfUQ0rI6q3CoNbpben7DAoJljmZRTa63QSjpnYH8hyZGfkgtXYhe6NC1wF0Q3FQJ5yWO1-oaDpkus3sjFa34OJmWx_VR8g-bAUlkrC5GFVMSEFytXGb1MlYSP3W0muel6-C7d-dWZBT7GV_kQrkgP8PYQC1i3weoA19t8JqT2CX1G1WmKo_F2DFktw" -k https://localhost:6443/openapi/v2 > k8s-swagger.json

獲取 swagger-ui 映象

docker pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/swaggerapi/swagger-ui:v4.15.5

建立serviceaccount

啟動 swagger-ui 映象

docker run -d -p 8080:8080 -e SWAGGER_JSON=/k8s-swagger.json -v /root/crd/k8s-swagger.json:/k8s-swagger.json swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/swaggerapi/swagger-ui:v4.15.5

訪問 swagger-ui

http://127.0.0.1:8080

生成k8s對應的CRD資源

參考資源https://kkgithub.com/kubernetes-client/java/blob/master/docs/generate-model-from-third-party-resources.md

docker pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/ghcr.io/kubernetes-client/java/crd-model-gen:v1.0.6

export LOCAL_MANIFEST_FILE=/root/prometheus/kube-prometheus-0.14.0/manifests/setup/0prometheusruleCustomResourceDefinition.yaml
docker run \
  --rm \
  -v "$LOCAL_MANIFEST_FILE":"$LOCAL_MANIFEST_FILE" \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v "$(pwd)":"$(pwd)" \
  -ti \
  --network host \
  swr.cn-north-4.myhuaweicloud.com/ddn-k8s/ghcr.io/kubernetes-client/java/crd-model-gen:v1.0.6 \
  /generate.sh \
  -u $LOCAL_MANIFEST_FILE \
  -n com.example.stable \
  -p com.example.stable \
  -o "$(pwd)"

線上生成方式參考https://blog.csdn.net/weixin_42340037/article/details/132496248

• fork kubernetes-client/java 倉庫到自己的github

https://github.com/kubernetes-client/java

• 點選Actions，在點選CRD Java Model Generate

• 執行run workflow；輸入必要的資料

Comma-separated paths to CRD yaml sources, can be either HTTP url or local file path.。k8s 自定義資源的 yaml檔案，kube-prometheus的自定義資源prometheusrule

https://github.com/prometheus-operator/kube-prometheus

https://raw.githubusercontent.com/prometheus-operator/kube-prometheus/refs/heads/main/manifests/setup/0prometheusruleCustomResourceDefinition.yaml

• 執行 Actions 後，下載生成為 CRD Java Model 的 zip 包

The package name of the generated java project. 可以隨便輸入報名

呼叫k8s的api

引入依賴

<dependency>
    <groupId>io.kubernetes</groupId>
    <artifactId>client-java</artifactId>
    <version>21.0.2</version>
</dependency>

編碼呼叫 k8s api的程式碼

package k8s_demo;

import java.io.FileInputStream;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import org.yaml.snakeyaml.Yaml;

import com.alibaba.fastjson2.JSON;
import com.crd.models.V1PrometheusRule;
import com.crd.models.V1PrometheusRuleSpec;
import com.crd.models.V1PrometheusRuleSpecGroups;
import com.crd.models.V1PrometheusRuleSpecRules;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;

import io.kubernetes.client.openapi.ApiClient;
import io.kubernetes.client.openapi.ApiException;
import io.kubernetes.client.openapi.Configuration;
import io.kubernetes.client.openapi.apis.CustomObjectsApi;
import io.kubernetes.client.openapi.apis.CustomObjectsApi.APIcreateClusterCustomObjectRequest;
import io.kubernetes.client.openapi.apis.CustomObjectsApi.APIcreateNamespacedCustomObjectRequest;
import io.kubernetes.client.openapi.apis.CustomObjectsApi.APIdeleteNamespacedCustomObjectRequest;
import io.kubernetes.client.openapi.apis.CustomObjectsApi.APIlistClusterCustomObjectRequest;
import io.kubernetes.client.openapi.apis.CustomObjectsApi.APIlistNamespacedCustomObjectRequest;
import io.kubernetes.client.openapi.models.V1ObjectMeta;
import io.kubernetes.client.util.ClientBuilder;
import io.kubernetes.client.util.credentials.AccessTokenAuthentication;

public class CRDdemo {

	public static void main(String[] args) {
		ApiClient client = new ClientBuilder().setBasePath("https://10.0.2.11:6443")
				.setVerifyingSsl(false)
				.setAuthentication(new AccessTokenAuthentication("eyJhbGciOiJSUzI1NiIsImtpZCI6ImQ4Slh0Mk9lMzd0TXNlZW9sbGRRMUVfRWtYSHVnNnFwMG11TmhYR3dWM2cifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJteS1hZG1pbi10b2tlbiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJteS1hZG1pbiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImMxYjgyNmU0LTM4YzctNGI1Yy05MzliLTU3MmExNzQ4ZjhjNiIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlLXN5c3RlbTpteS1hZG1pbiJ9.giP37iGgtXoBCPRj4YBu0ooovb8SbOXuMyAfQx5erzbh-s1AJZTsMVho-Hu2VlyeRcQ7AkLy44EUMdf__yy0XR44qXYRlN6-gG0yAMHTSt_mBbfbpt35uJ39jAnmFlS9SGgTfzAJdjoDzA6Vhq7_njab6Dkc9wmYuIAR4Q1fUEjIdkVb-558xlqegouUd4TNC855p6gwUoTLMZaNo1wGMHEa94HV37ECpGsQ2gSr4nEw29LQOHei96HfRuBdJa7lLhnuaqqKqE8tR9DuGVN5adtmC-AnSabRlkCgjM7KmB3b7BBndlRuG4ZcuARCCNvrbUM0N_Z43hL6PgEtSXFmxg"))
              // .setAuthentication(new AccessTokenAuthentication("eyJhbGciOiJSUzI1NiIsImtpZCI6ImQ4Slh0Mk9lMzd0TXNlZW9sbGRRMUVfRWtYSHVnNnFwMG11TmhYR3dWM2cifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi10b2tlbiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJobC1hZG1pbiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6Ijc4MDk1MDgwLWE2MWYtNGQ0Ni05YTUyLTYxYzMxOTAyYzIyMCIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlLXN5c3RlbTpobC1hZG1pbiJ9.Gy8xiiLngFopG6EJVAUIsP7n9jQqsP6rWEg8q6LcqpaQ42FXYVX01o1wsqi6u5l3H5D4_dI-GjOU1ajc8Y_g4lZu-ClCxn360tsoJ6ZaCg7fuW4LIA2Mr1gT-rv7yLKhYplF6LDwEwsqlAh3nZopoWvMPtAKWfUQ0rI6q3CoNbpben7DAoJljmZRTa63QSjpnYH8hyZGfkgtXYhe6NC1wF0Q3FQJ5yWO1-oaDpkus3sjFa34OJmWx_VR8g-bAUlkrC5GFVMSEFytXGb1MlYSP3W0muel6-C7d-dWZBT7GV_kQrkgP8PYQC1i3weoA19t8JqT2CX1G1WmKo_F2DFktw"))
				.build();
        Configuration.setDefaultApiClient(client);
        System.out.println(client);
        
        CustomObjectsApi apiInstance = new CustomObjectsApi(client);
        try {
        	String group = "monitoring.coreos.com"; // String | The custom resource's group name
            String version = "v1"; // String | The custom resource's version
            String plural = "prometheusrules"; // String | The custom resource's plural name. For TPRs this would be lowercase plural kind.
            Object body = null; // Object | The JSON schema of the Resource to create.
            String pretty = "true"; // String | If 'true', then the output is pretty printed.
            
            APIlistClusterCustomObjectRequest aPIlistClusterCustomObjectRequest = apiInstance.listClusterCustomObject(group, version, plural);
            Object result = aPIlistClusterCustomObjectRequest.execute();
            System.out.println(JSON.toJSON(result));
        	
            System.out.println("-----------------------------");
            APIlistNamespacedCustomObjectRequest aPIlistNamespacedCustomObjectRequest = apiInstance.listNamespacedCustomObject(group, version, pretty, plural);
            result = aPIlistNamespacedCustomObjectRequest.execute();
            System.out.println(result);
            System.out.println("-----------------------------");
        } catch (ApiException e) {
            System.err.println("Exception when calling CustomObjectsApi#listNamespacedCustomObject");
            System.err.println("Status code: " + e.getCode());
            System.err.println("Reason: " + e.getResponseBody());
            System.err.println("Response headers: " + e.getResponseHeaders());
            e.printStackTrace();
          }catch (Exception e) {
          	e.printStackTrace();
          }
	
        
        try {
        	String group = "monitoring.coreos.com"; // String | The custom resource's group name
            String version = "v1"; // String | The custom resource's version
            String plural = "prometheusrules"; // String | The custom resource's plural name. For TPRs this would be lowercase plural kind.
            String pretty = "true"; // String | If 'true', then the output is pretty printed.
            String namespace = "monitoring";
            
            
            V1PrometheusRule v1PrometheusRule = new V1PrometheusRule();
            v1PrometheusRule.setKind("PrometheusRule");
            v1PrometheusRule.setApiVersion("monitoring.coreos.com/v1");
            
            V1ObjectMeta v1ObjectMeta = new V1ObjectMeta();
            
            Map<String, String> labelsMap = new HashMap<String, String>();
            labelsMap.put("prometheus", "k8s");
            labelsMap.put("ole", "alert-rules");
            v1ObjectMeta.setLabels(labelsMap);
            v1ObjectMeta.setName("custom-rule2");
            v1ObjectMeta.setNamespace("monitoring");
            
            v1PrometheusRule.setMetadata(v1ObjectMeta);
            
            V1PrometheusRuleSpec v1PrometheusRuleSpec = new V1PrometheusRuleSpec();
            List<V1PrometheusRuleSpecGroups> groups = new ArrayList<V1PrometheusRuleSpecGroups>();
            V1PrometheusRuleSpecGroups v1PrometheusRuleSpecGroups = new V1PrometheusRuleSpecGroups();
            v1PrometheusRuleSpecGroups.setName("disk");
            groups.add(v1PrometheusRuleSpecGroups);
            
            v1PrometheusRuleSpec.setGroups(groups);
            
            List<V1PrometheusRuleSpecRules> rules = new ArrayList<V1PrometheusRuleSpecRules>();
            V1PrometheusRuleSpecRules v1PrometheusRuleSpecRules = new V1PrometheusRuleSpecRules();
            v1PrometheusRuleSpecRules.setAlert("diskFree");
            
            Map<String, String> annotationsMap = new HashMap<String, String>();
            annotationsMap.put("value", "{{$value}}");
            annotationsMap.put("summary", "{{ $labels.job }}  專案例項 {{ $labels.instance }} 磁碟使用率大於 80%");
            annotationsMap.put("description", "{{ $labels.instance }}  {{ $labels.mountpoint }}  磁碟使用率大於80%  (當前的值: {{ $value }}%),請及時處理");
            v1PrometheusRuleSpecRules.setAnnotations(annotationsMap);
            
            v1PrometheusRuleSpecRules.setExpr("(1-(node_filesystem_free_bytes{fstype=~\"ext4|xfs\",mountpoint!=\"/boot\"} / node_filesystem_size_bytes{fstype=~\"ext4|xfs\",mountpoint!=\"/boot\"}) )*100 > 80");
            v1PrometheusRuleSpecRules.setFor("1m");
            
            
            Map<String, String> labelsMap2 = new HashMap<String, String>();
            labelsMap2.put("level", "disaster");
            labelsMap2.put("severity", "warning");
            v1PrometheusRuleSpecRules.setLabels(labelsMap2);
            
            rules.add(v1PrometheusRuleSpecRules);
            v1PrometheusRuleSpecGroups.setRules(rules);
            
            v1PrometheusRule.setSpec(v1PrometheusRuleSpec);
            
            APIcreateNamespacedCustomObjectRequest aPIcreateNamespacedCustomObjectRequest = apiInstance.createNamespacedCustomObject(group, version, namespace, plural, v1PrometheusRule);
            aPIcreateNamespacedCustomObjectRequest.pretty(pretty);
            Object result = aPIcreateNamespacedCustomObjectRequest.execute();
            System.out.println(JSON.toJSON(result));
        	
        } catch (ApiException e) {
            System.err.println("Exception when calling CustomObjectsApi#createNamespacedCustomObject");
            System.err.println("Status code: " + e.getCode());
            System.err.println("Reason: " + e.getResponseBody());
            System.err.println("Response headers: " + e.getResponseHeaders());
            e.printStackTrace();
          }catch (Exception e) {
          	e.printStackTrace();
          }
        
        
        
        
        try {
        	String group = "monitoring.coreos.com"; // String | The custom resource's group name
            String version = "v1"; // String | The custom resource's version
            String plural = "prometheusrules"; // String | The custom resource's plural name. For TPRs this would be lowercase plural kind.
            String pretty = "true"; // String | If 'true', then the output is pretty printed.
            String namespace = "monitoring";
            String name = "";
         
            APIdeleteNamespacedCustomObjectRequest aPIdeleteNamespacedCustomObjectRequest = apiInstance.deleteNamespacedCustomObject(group, version, namespace, plural, name);
            Object result = aPIdeleteNamespacedCustomObjectRequest.execute();
            System.out.println(JSON.toJSON(result));
        	
        } catch (ApiException e) {
            System.err.println("Exception when calling CustomObjectsApi#aPIdeleteNamespacedCustomObjectRequest");
            System.err.println("Status code: " + e.getCode());
            System.err.println("Reason: " + e.getResponseBody());
            System.err.println("Response headers: " + e.getResponseHeaders());
            e.printStackTrace();
          }catch (Exception e) {
          	e.printStackTrace();
          }
        
        
	}
	
	/**
     * @param filename yaml檔名稱
     * @return 返回yaml檔案中對應的json資料
     */
    public static String yamlToJson(String filename) {
        String jsonData;
        try {
            // 建立Yaml物件
            Yaml yaml = new Yaml();
            // 開啟檔案輸入流
            FileInputStream input = new FileInputStream(filename);
            // 讀取整個檔案為一個Map物件,如果yaml檔案為列表，則資料型別為list
            Map<String, Object> data = yaml.load(input);
            // 建立ObjectMapper物件用於將資料轉換為JSON
            ObjectMapper mapper = new ObjectMapper();
            // 啟用格式化輸出
            mapper.enable(SerializationFeature.INDENT_OUTPUT);
            // 將資料轉換為JSON字串
            jsonData = mapper.writeValueAsString(data);
            // 返回讀取的資料
            return jsonData;
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }
}

https://kkgithub.com/kubernetes-client/java/blob/master/kubernetes/docs/CustomObjectsApi.md#createClusterCustomObject

程式碼中需要用到的變數值檢視

檢視k8s的自定義資源

kubectl get customresourcedefinitions.apiextensions.k8s.io

檢視自定義資源的group和version資訊

kubectl explain prometheusrules.monitoring.coreos.com

檢視自定義資源的plural資訊

kubectl describe customresourcedefinitions.apiextensions.k8s.io  prometheusrules.monitoring.coreos.com

刪除自定義資源的例項

kubectl delete prometheusrules.monitoring.coreos.com custom-rule -n monitoring