k8s 新版本部署 Ingress-nginx controller

AskaJohnny發表於2022-04-18

K8SNginxController

k8s 新版本部署 Ingress-nginx controller

本篇主要記錄一下 k8s 新版本 1.23.5 中如何搭建 ingress controller 以及裡面的注意項

新版本和老版本區別有不少，坑很多，只能從官網一點點擼

1.環境準備

作業系統：Centos7 （CentOS Linux release 7.9.2009）
Master 主節點： 1臺虛擬機器
Node 計算節點： 2臺虛擬機器
K8s version：v1.23.5 （選的較新的版本）
Docker version：20.10.14

環境和上一篇中 kubeadm 搭建 k8s 的一致

2.概述

下面從官網摘取的對於 Ingress 的介紹

2.1 什麼是 Ingress

Ingress 是對叢集中服務的外部訪問進行管理的 API 物件，典型的訪問方式是 HTTP。Ingress 可以提供負載均衡

Ingress 公開了從叢集外部到叢集內服務的 HTTP 和 HTTPS 路由。流量路由由 Ingress 資源上定義的規則控制。

下面是一個將所有流量都傳送到同一 Service 的簡單 Ingress 示例：

但是僅建立 Ingress 資源本身沒有任何效果，需要有對應的 Ingress 控制器，你可能需要部署 Ingress 控制器，例如 ingress-nginx。你可以從許多 Ingress 控制器中進行選擇。

2.2 什麼是 Ingress controller

為了讓 Ingress 資源工作，叢集必須有一個正在執行的 Ingress 控制器。k8s官網維護了 3 個Ingress控制器

目前支援和維護 AWS、 GCE 和 Nginx Ingress 控制器，本篇就拿 Ingress-nginx 作為控制器為例講解一下如何部署

還有一些社群等提供的控制器

3.部署 Ingress-nginx controller

本篇就拿最常用的 Ingress-nginx 作為 Ingress的控制器，實現k8s 把請求重定向到叢集內部（Cluster Ip）服務去

3.1 deploy.yaml 坑點

Ingress-nginx 官網 https://kubernetes.github.io/ingress-nginx/ 提到了 deploy.yaml 檔案

Ingress-nginx 新版本的 depoly.yaml 有些不同，需要拉取下面2個映象

k8s.gcr.io/ingress-nginx/controller:v1.1.2

k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1

多半是下載不到的，所以需要自己替換一下，可以去docker hub 上找到對應的映象檔案

總結坑點：

新版本中提供了 IngressClass ，需要在編寫 Ingress 的時候指定
Image 載入不到，需要手動去 docker hub 上找其他的並且修改 deploy.yaml檔案
把 ingress-nginx-controller 使用 hostNetwork: true 進行部署比 NodePort 減少一層轉發，但是需要指定選擇打了標籤的 node nodeSelector: app: ingress

3.2 deploy.yaml 樣例（我自己修改後的可以參考）

#GENERATED FOR K8S 1.20
apiVersion: v1
kind: Namespace
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  name: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.1.2
    helm.sh/chart: ingress-nginx-4.0.18
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.1.2
    helm.sh/chart: ingress-nginx-4.0.18
  name: ingress-nginx-admission
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.1.2
    helm.sh/chart: ingress-nginx-4.0.18
  name: ingress-nginx
  namespace: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - endpoints
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - networking.k8s.io
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - networking.k8s.io
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - networking.k8s.io
    resources:
      - ingressclasses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resourceNames:
      - ingress-controller-leader
    resources:
      - configmaps
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  annotations:
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.1.2
    helm.sh/chart: ingress-nginx-4.0.18
  name: ingress-nginx-admission
  namespace: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - secrets
    verbs:
      - get
      - create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.1.2
    helm.sh/chart: ingress-nginx-4.0.18
  name: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
      - namespaces
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - networking.k8s.io
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - networking.k8s.io
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - networking.k8s.io
    resources:
      - ingressclasses
    verbs:
      - get
      - list
      - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  annotations:
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.1.2
    helm.sh/chart: ingress-nginx-4.0.18
  name: ingress-nginx-admission
rules:
  - apiGroups:
      - admissionregistration.k8s.io
    resources:
      - validatingwebhookconfigurations
    verbs:
      - get
      - update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.1.2
    helm.sh/chart: ingress-nginx-4.0.18
  name: ingress-nginx
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx
subjects:
  - kind: ServiceAccount
    name: ingress-nginx
    namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  annotations:
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.1.2
    helm.sh/chart: ingress-nginx-4.0.18
  name: ingress-nginx-admission
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx-admission
subjects:
  - kind: ServiceAccount
    name: ingress-nginx-admission
    namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.1.2
    helm.sh/chart: ingress-nginx-4.0.18
  name: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-nginx
subjects:
  - kind: ServiceAccount
    name: ingress-nginx
    namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  annotations:
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.1.2
    helm.sh/chart: ingress-nginx-4.0.18
  name: ingress-nginx-admission
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-nginx-admission
subjects:
  - kind: ServiceAccount
    name: ingress-nginx-admission
    namespace: ingress-nginx
---
apiVersion: v1
data:
  allow-snippet-annotations: "true"
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.1.2
    helm.sh/chart: ingress-nginx-4.0.18
  name: ingress-nginx-controller
  namespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.1.2
    helm.sh/chart: ingress-nginx-4.0.18
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  externalTrafficPolicy: Local
  ipFamilies:
    - IPv4
  ipFamilyPolicy: SingleStack
  ports:
    - appProtocol: http
      name: http
      port: 80
      protocol: TCP
      targetPort: http
    - appProtocol: https
      name: https
      port: 443
      protocol: TCP
      targetPort: https
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.1.2
    helm.sh/chart: ingress-nginx-4.0.18
  name: ingress-nginx-controller-admission
  namespace: ingress-nginx
spec:
  ports:
    - appProtocol: https
      name: https-webhook
      port: 443
      targetPort: webhook
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.1.2
    helm.sh/chart: ingress-nginx-4.0.18
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  minReadySeconds: 0
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/component: controller
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/name: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/component: controller
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
    spec:
      hostNetwork: true #修改  ingress-nginx-controller 為 hostNetwork模式 
      nodeSelector: #選擇 node label 中有 app=ingress的節點進行部署
        app: ingress
      containers:
        - args:
            - /nginx-ingress-controller
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
            - --election-id=ingress-controller-leader
            - --controller-class=k8s.io/ingress-nginx
            - --ingress-class=nginx
            - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
            - --validating-webhook=:8443
            - --validating-webhook-certificate=/usr/local/certificates/cert
            - --validating-webhook-key=/usr/local/certificates/key
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: LD_PRELOAD
              value: /usr/local/lib/libmimalloc.so
          image: k8s.gcr.io/ingress-nginx/controller:v1.1.2 #修改映象地址
          imagePullPolicy: IfNotPresent
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown
          livenessProbe:
            failureThreshold: 5
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          name: controller
          ports:
            - containerPort: 80
              name: http
              protocol: TCP
            - containerPort: 443
              name: https
              protocol: TCP
            - containerPort: 8443
              name: webhook
              protocol: TCP
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          resources:
            requests:
              cpu: 100m
              memory: 90Mi
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              add:
                - NET_BIND_SERVICE
              drop:
                - ALL
            runAsUser: 101
          volumeMounts:
            - mountPath: /usr/local/certificates/
              name: webhook-cert
              readOnly: true
      dnsPolicy: ClusterFirst
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: ingress-nginx
      terminationGracePeriodSeconds: 300
      volumes:
        - name: webhook-cert
          secret:
            secretName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
  annotations:
    helm.sh/hook: pre-install,pre-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.1.2
    helm.sh/chart: ingress-nginx-4.0.18
  name: ingress-nginx-admission-create
  namespace: ingress-nginx
spec:
  template:
    metadata:
      labels:
        app.kubernetes.io/component: admission-webhook
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        app.kubernetes.io/version: 1.1.2
        helm.sh/chart: ingress-nginx-4.0.18
      name: ingress-nginx-admission-create
    spec:
      containers:
        - args:
            - create
            - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
            - --namespace=$(POD_NAMESPACE)
            - --secret-name=ingress-nginx-admission
          env:
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          image: k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1 #修改映象地址
          imagePullPolicy: IfNotPresent
          name: create
          securityContext:
            allowPrivilegeEscalation: false
      nodeSelector:
        kubernetes.io/os: linux
      restartPolicy: OnFailure
      securityContext:
        fsGroup: 2000
        runAsNonRoot: true
        runAsUser: 2000
      serviceAccountName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
  annotations:
    helm.sh/hook: post-install,post-upgrade
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.1.2
    helm.sh/chart: ingress-nginx-4.0.18
  name: ingress-nginx-admission-patch
  namespace: ingress-nginx
spec:
  template:
    metadata:
      labels:
        app.kubernetes.io/component: admission-webhook
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        app.kubernetes.io/version: 1.1.2
        helm.sh/chart: ingress-nginx-4.0.18
      name: ingress-nginx-admission-patch
    spec:
      containers:
        - args:
            - patch
            - --webhook-name=ingress-nginx-admission
            - --namespace=$(POD_NAMESPACE)
            - --patch-mutating=false
            - --secret-name=ingress-nginx-admission
            - --patch-failure-policy=Fail
          env:
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          image: k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1 #修改映象地址
          imagePullPolicy: IfNotPresent
          name: patch
          securityContext:
            allowPrivilegeEscalation: false
      nodeSelector:
        kubernetes.io/os: linux
      restartPolicy: OnFailure
      securityContext:
        fsGroup: 2000
        runAsNonRoot: true
        runAsUser: 2000
      serviceAccountName: ingress-nginx-admission
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.1.2
    helm.sh/chart: ingress-nginx-4.0.18
  name: nginx
spec:
  controller: k8s.io/ingress-nginx
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.1.2
    helm.sh/chart: ingress-nginx-4.0.18
  name: ingress-nginx-admission
webhooks:
  - admissionReviewVersions:
      - v1
    clientConfig:
      service:
        name: ingress-nginx-controller-admission
        namespace: ingress-nginx
        path: /networking/v1/ingresses
    failurePolicy: Fail
    matchPolicy: Equivalent
    name: validate.nginx.ingress.kubernetes.io
    rules:
      - apiGroups:
          - networking.k8s.io
        apiVersions:
          - v1
        operations:
          - CREATE
          - UPDATE
        resources:
          - ingresses
    sideEffects: None

4.部署 Ingress-nginx

4.1 準備工作

給 node2 節點打了app=ingress標籤，因為上面的ingress-nginx-controller 使用的是 hostNetwork 模式（只會放pod真實pod 的埠） + nodeSelector

kubectl label node kubec-node-2 app=ingress

4.2 部署 kubect apply -f

kubectl apply -f deploy.yaml #通過 kubectl apply 命令進行部署 ，前提是映象準備好，否則GG

4.3 檢視狀態

kubectl get all -n ingress-nginx #檢視 ingress-nginx namespace的 部署情況

kubectl logs -f ingress-nginx-controller-744f6d5bdf-tl6g4 -n ingress-nginx # 檢視 ingress-nginx-controller 的 日誌情況

4.4 測試訪問

直接訪問 kube-node-2的 ip 即可，因為 ingress-nginx-controller 預設是監聽 80埠，由於上面的 nodeSelector: #選擇 node label 中有 app=ingress的節點進行部署 ,而 kube-node-2 是被打了標籤的節點node

可以看到其實就是一個 nginx

5.部署一個 tomcat 測試 Ingress-nginx

通過部署一個tomcat ，測試Ingress-nginx的代理是否生效

5.1 編寫 deploy-tomcat.yaml

主要編寫

Deployment 部署tomcat:8.0-alpine，
Service 暴露 tomcat pod
一個 Ingress 資源它規定訪問 tomcat.demo.com 這個域名的所有請求 / 都轉發到 tomcat-demo Service 上

IngressClass 新版本提供的資源，用於在定義 Ingress資源的時候指定，在叢集中有多個 Ingress controller 的時候很有用處

apiVersion: apps/v1
kind: Deployment
metadata:
  name: tomcat-demo
spec:
  selector:
    matchLabels:
      app: tomcat-demo
  replicas: 1
  template:
    metadata:
      labels:
        app: tomcat-demo
    spec:
      containers:
      - name: tomcat-demo
        image: tomcat:8.0-alpine
        ports:
        - containerPort: 8080
---

apiVersion: v1
kind: Service
metadata:
  name: tomcat-demo
spec:
  selector:
    app: tomcat-demo #選擇 tomcat-demo pod 
  ports:
  - port: 80 #對外暴露 80 埠
    protocol: TCP
    targetPort: 8080 # tomcat 埠


---

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: tomcat-demo
spec:
  defaultBackend:
    service:
      name: default-http-backend #！！！ 指定 預設的backend服務 
      port: 
      	number: 80 
  ingressClassName: nginx #！！！重點 需要指定 哪個 IngressClass 可以看上面的 deploy.yaml 最後定義的
  rules:
  - host: tomcat.demo.com #所有的  tomcat.demo.com請求都轉發到  Service tomcat-demo
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: tomcat-demo
            port: 
              number: 80
              
---
#定義一個 default-http-backend 當沒有被Ingress規定的請求 負載給 它
apiVersion: apps/v1
kind: Deployment
metadata:
  name: default-http-backend
  labels:
    app: default-http-backend
spec:
  replicas: 1
  selector:
    matchLabels:
      app: default-http-backend
  template:
    metadata:
      labels:
        app: default-http-backend
    spec:
      terminationGracePeriodSeconds: 60
      containers:
      - name: default-http-backend
        # Any image is permissible as long as:
        # 1. It serves a 404 page at /
        # 2. It serves 200 on a /healthz endpoint
        image: registry.cn-hangzhou.aliyuncs.com/google_containers/defaultbackend:1.4
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 30
          timeoutSeconds: 5
        ports:
        - containerPort: 8080
        resources:
          limits:
            cpu: 10m
            memory: 20Mi
          requests:
            cpu: 10m
            memory: 20Mi
---
 
apiVersion: v1
kind: Service
metadata:
  name: default-http-backend
  labels:
    app: default-http-backend
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: default-http-backend

5.2 部署 tomcat + ingress + default-http-backend

部署上面的檔案即可

檢視部署情況可以看到都是Running 了

5.3 測試通過Ingress-nginx 能否訪問到tomcat

由於我們 ingress 資源配置的域名是 tomcat.demo.com ，所以我需要把它新增到宿主機的hosts 檔案中，如下

那麼按照預期，當我訪問 tomcat.demo.com 的時候其實就是訪問 192.168.56.22 上的 ingress-nginx-controller 這個nginx，那麼根據 ingress 的資源的設定它會把請求轉發到 Service tomcat-demo 上，從而訪問到 tomcat 介面

當我訪問 api.demo.com 的時候由於沒有對這個域名進行處理那麼會預設把請求轉發到 default-http-backend 上

訪問 tomcat.demo.com

訪問 api.demo.com

總結

至此在k8s 1.23.5上已經成功部署了 Ingress-nginx ，並且通過部署一個tomcat服務測試了 Ingress-nginx 已經代理成功了，由於不管是 k8s 新版本和 Ingress-nginx 新版本都有些變化所有很些坑，是我從官網不斷摸索的，下面總結一下坑點

image 映象需要自己從docker hub 上準備好，或者直接修改Ingress-nginx 的 deploy.yaml

k8s.gcr.io/ingress-nginx/controller:v1.1.2

k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1
重要！新版本中提供了 IngressClass ，需要在編寫 Ingress 的時候指定
把 ingress-nginx-controller 使用 hostNetwork: true 進行部署比 NodePort 減少一層轉發，但是需要指定選擇打了標籤的 node nodeSelector: app: ingress

通過 Ingress spec.defaultBackend 指定預設服務，並且需要保持在同一個 namespace

spec:
	defaultBackend:
    service:
      name: default-http-backend #！！！ 指定 預設的backend服務 
      port: 
      	number: 80

vscode k8s 外掛快捷生成Depoyment 的時候自定新增了資源限制，導致部署tomcat的時候總是啟動不起來..

導致tomcat pod 狀態 CrashLoopBackOff，從而導致通過 Ingress-nginx 無法訪問到tomcat
```
resources:
  limits:
    memory: "128Mi" #不夠部署tomcat 記憶體
    cpu: "500m"
```

歡迎大家訪問個人部落格 Johnny小屋

K8S 部署 ingress-nginx (二) 部署後端為 tomcat
2018-11-07
K8SNginx後端Tomcat
k8s使用ingress-nginx負載均衡
2019-04-18
K8SNginx負載
k8s endpoints controller分析
2021-11-14
K8SController
ingress-nginx-controller部署
2024-06-28
NginxController
k8s daemonset controller原始碼分析
2021-12-23
K8SController原始碼
k8s statefulset controller原始碼分析
2021-11-28
K8SController原始碼
k8s deployment controller原始碼分析
2021-10-10
K8SController原始碼
k8s dashboard部署
2019-01-03
K8S
rancher部署k8s
2024-11-01
K8S
k8s安裝nginx-ingress-controller
2022-03-07
K8SNginxController
通過helm部署EFK收集應用日誌，ingress-nginx日誌解析。
2021-07-01
應用日誌Nginx
K8S 生態週報| Knative 進入 CNCF 孵化，K8S ingress-nginx 解決多例項問題
2022-03-07
K8SNginx
Ingress-Nginx 安裝
2024-07-01
Nginx
K8s 部署 Prometheus + Grafana
2021-06-29
K8SPrometheusGrafana
k8s叢集部署
2021-02-08
K8S
centos 7 部署k8s
2024-08-21
CentOSK8S
k8s自定義controller設計與實現
2021-03-02
K8SController
k8s replicaset controller 分析（3）-expectations 機制分析
2021-10-27
K8SController
k8s自定義controller三部曲之三：編寫controller程式碼
2022-09-04
K8SController
來了，k8s！-----------------k8s叢集部署
2020-11-30
K8S
k8s 部署 Java 專案
2020-12-05
K8SJava
helm部署traefik到k8s
2020-10-09
K8S
在 k8s 中部署 Prometheus
2018-05-22
K8SPrometheus
K8S部署Metrics-Server
2024-06-12
K8SServer
Helm部署k8s應用
2024-07-23
K8S
微服務從程式碼到k8s部署應有盡有大結局（k8s部署）
2022-03-15
微服務K8S
k8s與監控--k8s部署grafana6.0
2019-02-28
K8SGrafana
Kubernetes Ingress-nginx使用
2020-12-03
Nginx
使用 helm 部署 k8s 資源
2019-11-04
K8S
K8s 部署 Gitlab CI Runner
2021-08-22
K8SGitlab
k8s環境部署及使用方式
2018-12-15
K8S
部署 Prometheus 和 Grafana 到 k8s
2020-12-05
PrometheusGrafanaK8S
kubeadm部署K8S叢集
2021-01-18
K8S
k8s二進位制部署
2020-11-22
K8S
k8s單master叢集部署
2022-06-20
K8SAST
Kubeadm叢集部署k8s
2022-03-08
K8S
Ansible部署K8s叢集
2022-02-28
K8S
K8s 部署應用必備
2020-09-29
K8S

k8s 新版本 部署 Ingress-nginx controller

k8s 新版本 部署 Ingress-nginx controller

1.環境準備

2.概述

2.1 什麼是 Ingress

2.2 什麼是 Ingress controller

3.部署 Ingress-nginx controller

3.1 deploy.yaml 坑點

3.2 deploy.yaml 樣例（我自己修改後的 可以參考）

4.部署 Ingress-nginx

4.1 準備工作

4.2 部署 kubect apply -f

4.3 檢視狀態

4.4 測試訪問

5.部署一個 tomcat 測試 Ingress-nginx

5.1 編寫 deploy-tomcat.yaml

5.2 部署 tomcat + ingress + default-http-backend

5.3 測試 通過Ingress-nginx 能否訪問到tomcat

總結

相關文章

k8s 新版本部署 Ingress-nginx controller

k8s 新版本部署 Ingress-nginx controller

3.2 deploy.yaml 樣例（我自己修改後的可以參考）

5.3 測試通過Ingress-nginx 能否訪問到tomcat