Cilium Gateway API 特性（轉載）

evescn發表於2024-07-16

Cilium Gateway API 特性（轉載）

一、環境資訊

主機	IP
ubuntu	10.0.0.234

軟體	版本
docker	26.1.4
helm	v3.15.0-rc.2
kind	0.18.0
kubernetes	1.23.4
ubuntu os	Ubuntu 22.04.6 LTS
kernel	5.15.0-106

二、`Cilium` `Gateway API` 流程圖

Cilium 現在提供完全一致的 Gateway API 實現。 Gateway API 是 Kubernetes 叢集中南北負載均衡和流量路由的新標準，並且是 Ingress API 的長期繼承者。 Gateway API 代表了流量管理的未來。

GatewaAPI 從設計之初就是為了解決 Ingress API 的侷限性

Gateway API 的建立是源於 Ingress API 存在一些侷限性：首先，它不提供使用者需要定義的高階負載均衡功能。它本身僅支援簡單的基於路徑的 HTTP 流量請求路由
其次，使用者管理變得不切實際：產品供應商透過註釋來解決 Ingress API 中功能缺乏的問題。但是註釋雖然非常強大，但最終會導致一個 Ingress 與另一個 Ingress 之間出現不一致
第三，由於 Ingress API 是單一 API 資源，因此它操作受限：不適合具有共享負載均衡基礎設施的多團隊叢集

在 Cilium 1.13 中，Cilium Gateway API 透過了所有 Gateway API 一致性測試 (v0.5.1)[https://isovalent.com/blog/post/cilium-release-113/]

參考官方連結

三、`Cilium` `Gateway API` 模式環境搭建

安裝須知

配置 Cilium 時，必須啟用 NodePort，使用 nodePort.enabled=true 或使用 kubeProxyReplacement 將 kube-proxy 替換為部分或嚴格。
配置 Cilium 時，必須使用 --enable-l7-proxy 標誌（預設已啟用）啟用 L7 代理。
必須預先安裝以下來自 Gateway API v0.5.1 的 CRD。(安裝步驟)(https://docs.cilium.io/en/v1.13/network/servicemesh/gateway-api/gateway-api/#prerequisites)
與 Ingress 類似，Gateway API 控制器會建立一個 LoadBalancer 型別的服務，因此您的環境需要支援該服務。

`kind` 配置檔案資訊

root@KinD:~# cat install.sh

#!/bin/bash
date
set -v

# 1.prep noCNI env
cat <<EOF | kind create cluster --name=cilium-gatewayapi-http --image=kindest/node:v1.23.4 --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
  # kind 預設使用 rancher cni，cni 我們需要自己建立
  disableDefaultCNI: true
  # 此處使用 cilium 代替 kube-proxy 功能
  kubeProxyMode: "none"
nodes:
  - role: control-plane
  - role: worker
  - role: worker

containerdConfigPatches:
- |-
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors."harbor.evescn.com"]
    endpoint = ["https://harbor.evescn.com"]
EOF

# 2.install necessary tools
for i in $(docker ps -a --format "table {{.Names}}" | grep cilium) 
do
    echo $i
    docker cp /usr/bin/ping $i:/usr/bin/ping
    docker exec -it $i bash -c "sed -i -e 's/jp.archive.ubuntu.com\|archive.ubuntu.com\|security.ubuntu.com/old-releases.ubuntu.com/g' /etc/apt/sources.list"
    docker exec -it $i bash -c "apt-get -y update >/dev/null && apt-get -y install net-tools tcpdump lrzsz bridge-utils >/dev/null 2>&1"
done

安裝 k8s 叢集和 cilium 服務

root@KinD:~# ./install.sh

Creating cluster "cilium-gatewayapi-http" ...
 ✓ Ensuring node image (kindest/node:v1.23.4) 🖼
 ✓ Preparing nodes 📦 📦 📦  
 ✓ Writing configuration 📜 
 ✓ Starting control-plane 🕹️ 
 ✓ Installing StorageClass 💾 
 ✓ Joining worker nodes 🚜 
Set kubectl context to "kind-cilium-gatewayapi-http"
You can now use your cluster with:

kubectl cluster-info --context kind-cilium-gatewayapi-http

Thanks for using kind! 😊

安裝 gateway api

root@KinD:~# kubectl apply -f https://gh.api.99988866.xyz/https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v0.5.1/config/crd/standard/gateway.networking.k8s.io_gatewayclasses.yaml
customresourcedefinition.apiextensions.k8s.io/gatewayclasses.gateway.networking.k8s.io created
root@KinD:~# kubectl apply -f https://gh.api.99988866.xyz/https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v0.5.1/config/crd/standard/gateway.networking.k8s.io_gateways.yaml
customresourcedefinition.apiextensions.k8s.io/gateways.gateway.networking.k8s.io created
root@KinD:~# kubectl apply -f https://gh.api.99988866.xyz/https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v0.5.1/config/crd/standard/gateway.networking.k8s.io_httproutes.yaml
customresourcedefinition.apiextensions.k8s.io/httproutes.gateway.networking.k8s.io created
root@KinD:~# kubectl apply -f https://gh.api.99988866.xyz/https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v0.5.1/config/crd/experimental/gateway.networking.k8s.io_referencegrants.yaml
customresourcedefinition.apiextensions.k8s.io/referencegrants.gateway.networking.k8s.io created

安裝 cilium 服務

#/bin/bash
set -v
controller_node_ip=`kubectl get node -o wide --no-headers | grep -E "control-plane|bpf1" | awk -F " " '{print $6}'`

helm repo add cilium https://helm.cilium.io > /dev/null 2>&1
helm repo update > /dev/null 2>&1

helm install cilium cilium/cilium \
  --set k8sServiceHost=$controller_node_ip \
  --set k8sServicePort=6443 \
  --version 1.13.0-rc5 \
  --namespace kube-system \
  --set debug.enabled=true \
  --set debug.verbose=datapath \
  --set monitorAggregation=none \
  --set ipam.mode=cluster-pool \
  --set cluster.name=cilium-gatewayapi-http \
  --set kubeProxyReplacement=strict \
  --set bpf.masquerade=true \
  --set gatewayAPI.enabled=true

--set 引數解釋

--set kubeProxyReplacement=strict
- 含義: 啟用 kube-proxy 替代功能，並以嚴格模式執行。
- 用途: Cilium 將完全替代 kube-proxy 實現服務負載均衡，提供更高效的流量轉發和網路策略管理。
--set bpf.masquerade
- 含義: 啟用 eBPF 功能。
- 用途: 使用 eBPF 實現資料路由，提供更高效和靈活的網路地址轉換功能。
--set gatewayAPI.enable=true:
- 啟用 Gateway API 支援，Cilium 將支援並管理 Gateway API 資源。

檢視安裝的服務

root@KinD:~# kubectl get pods -A
NAMESPACE            NAME                                                           READY   STATUS    RESTARTS   AGE
kube-system          cilium-55g65                                                   1/1     Running   0          103s
kube-system          cilium-bb4s7                                                   1/1     Running   0          103s
kube-system          cilium-operator-58959f76d6-46b62                               1/1     Running   0          103s
kube-system          cilium-operator-58959f76d6-92kwx                               1/1     Running   0          103s
kube-system          cilium-r5v9v                                                   1/1     Running   0          103s
kube-system          coredns-64897985d-l5rqq                                        1/1     Running   0          131m
kube-system          coredns-64897985d-qx7bl                                        1/1     Running   0          131m
kube-system          etcd-cilium-gatewayapi-http-control-plane                      1/1     Running   0          132m
kube-system          kube-apiserver-cilium-gatewayapi-http-control-plane            1/1     Running   0          132m
kube-system          kube-controller-manager-cilium-gatewayapi-http-control-plane   1/1     Running   0          132m
kube-system          kube-scheduler-cilium-gatewayapi-http-control-plane            1/1     Running   0          132m
local-path-storage   local-path-provisioner-5ddd94ff66-7t65r                        1/1     Running   0          131m

安裝 `metallb` 服務，提供 `LoadBanlencer` 功能

安裝 metallb 服務

root@KinD:~# kubectl apply -f https://gh.api.99988866.xyz/https://raw.githubusercontent.com/metallb/metallb/v0.13.10/config/manifests/metallb-native.yaml

namespace/metallb-system created
customresourcedefinition.apiextensions.k8s.io/addresspools.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bfdprofiles.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bgpadvertisements.metallb.io created
customresourcedefinition.apiextensions.k8s.io/bgppeers.metallb.io created
customresourcedefinition.apiextensions.k8s.io/communities.metallb.io created
customresourcedefinition.apiextensions.k8s.io/ipaddresspools.metallb.io created
customresourcedefinition.apiextensions.k8s.io/l2advertisements.metallb.io created
serviceaccount/controller created
serviceaccount/speaker created
role.rbac.authorization.k8s.io/controller created
role.rbac.authorization.k8s.io/pod-lister created
clusterrole.rbac.authorization.k8s.io/metallb-system:controller created
clusterrole.rbac.authorization.k8s.io/metallb-system:speaker created
rolebinding.rbac.authorization.k8s.io/controller created
rolebinding.rbac.authorization.k8s.io/pod-lister created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:controller created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:speaker created
configmap/metallb-excludel2 created
secret/webhook-server-cert created
service/webhook-service created
deployment.apps/controller created
daemonset.apps/speaker created
validatingwebhookconfiguration.admissionregistration.k8s.io/metallb-webhook-configuration created

建立 metalLb layer2 的 IPAddressPool

root@KinD:~# cat metallb-l2-ip-config.yaml
---
# metallb 分配給 loadbanlencer 的 ip 地址段定義
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: ippool
  namespace: metallb-system
spec:
  addresses:
  - 172.18.0.200-172.18.0.210

---
# 建立 L2Advertisement 進行 IPAddressPool 地址段
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: ippool
  namespace: metallb-system
spec:
  ipAddressPools:
  - ippool


root@KinD:~# kubectl apply -f metallb-l2-ip-config.yaml
ipaddresspool.metallb.io/ippool unchanged
l2advertisement.metallb.io/ippool created

root@KinD:~# kubectl get ipaddresspool -n metallb-system
NAME     AUTO ASSIGN   AVOID BUGGY IPS   ADDRESSES
ippool   true          false             ["172.18.0.200-172.18.0.210"]
root@KinD:~# kubectl get l2advertisement -n metallb-system
NAME     IPADDRESSPOOLS   IPADDRESSPOOL SELECTORS   INTERFACES
ippool   ["ippool"]

`k8s` 叢集安裝 `Pod` 和 `Service`

root@KinD:~# cat cni.yaml
---
apiVersion: apps/v1
kind: DaemonSet
#kind: Deployment
metadata:
  labels:
    app: cni
  name: cni
spec:
  #replicas: 1
  selector:
    matchLabels:
      app: cni
  template:
    metadata:
      labels:
        app: cni
    spec:
      containers:
      - image: harbor.dayuan1997.com/devops/nettool:0.9
        name: nettoolbox
        securityContext:
          privileged: true

---
apiVersion: v1
kind: Service
metadata:
  name: cni
spec:
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
  type: ClusterIP
  selector:
    app: cni

root@KinD:~# kubectl apply -f cni.yaml
daemonset.apps/cni created
service/serversvc created

檢視安裝服務資訊

root@KinD:~# kubectl get pods -o wide
NAME        READY   STATUS    RESTARTS   AGE   IP          NODE                             NOMINATED NODE   READINESS GATES
cni-9c8h5   1/1     Running   0          7s    10.0.2.2    cilium-gatewayapi-http-worker    <none>           <none>
cni-j9bzr   1/1     Running   0          7s    10.0.1.69   cilium-gatewayapi-http-worker2   <none>           <none>

root@KinD:~# kubectl get svc
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
cni          ClusterIP   10.96.145.109   <none>        80/TCP    20s
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP   134m

四、測試 `Gateway` `HTTP` 特性

配置 `Gateway` `HTTP`

root@KinD:~# cat gateway.yamnl
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  name: basic-gateway
spec:
  gatewayClassName: cilium
  listeners:
  - protocol: HTTP
    port: 80
    name: web-gateway
    allowedRoutes:
      namespaces:
        from: Same
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: http-app-cni
spec:
  parentRefs:
  - name: basic-gateway
    namespace: default
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: cni
      port: 80


root@KinD:~# kubectl apply -f gateway.yaml
gateway.gateway.networking.k8s.io/basic-gateway created
httproute.gateway.networking.k8s.io/http-app-cni created

root@KinD:~# kubectl get svc
NAME                           TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)        AGE
cilium-gateway-basic-gateway   LoadBalancer   10.96.30.37     172.18.0.200   80:30594/TCP   21s
cni                            ClusterIP      10.96.145.109   <none>         80/TCP         4m9s
kubernetes                     ClusterIP      10.96.0.1       <none>         443/TCP        138m

root@KinD:~# kubectl get gateway
NAME            CLASS    ADDRESS        READY   AGE
basic-gateway   cilium   172.18.0.200   True    37s

root@KinD:~# kubectl get HTTPRoute
NAME           HOSTNAMES   AGE
http-app-cni               54s

建立 Gateway 後 basic-gateway 在定義的地址池中自動獲取了一個 IP 地址，並且自動建立了一個 cilium-gateway-basic-gateway svc 服務，服務的 EXTERNAL-IP 為 basic-gateway 從地址池中自動獲取的 IP 地址

測試 `Gateway` `HTTP`

root@KinD:~# GATEWAY=$(kubectl get gateway basic-gateway -o jsonpath='{.status.addresses[0].value}')
root@KinD:~# curl -v http://"$GATEWAY"/
*   Trying 172.18.0.200:80...
* Connected to 172.18.0.200 (172.18.0.200) port 80 (#0)
> GET / HTTP/1.1
> Host: 172.18.0.200
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< server: envoy
< date: Tue, 16 Jul 2024 06:14:57 GMT
< content-type: text/html
< content-length: 46
< last-modified: Tue, 16 Jul 2024 06:09:16 GMT
< etag: "66960e8c-2e"
< accept-ranges: bytes
< x-envoy-upstream-service-time: 0
< 
PodName: cni-j9bzr | PodIP: eth0 10.0.1.69/32
* Connection #0 to host 172.18.0.200 left intact

在宿主機 172.18.0.1 主機上訪問業務正常訪問，使用 ip 地址為 172.18.0.200

五、測試 `Gateway` `HTTPS` 特性

建立 tls 證書，此處使用了一個測試證書

root@KinD:~# kubectl  create secret tls demo-cert --cert=tls.crt --key=tls.key
secret/demo-cert created

配置 `Gateway` `HTTPS`

root@KinD:~# cat gateway-https.yamnl
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  name: tls-gateway
spec:
  gatewayClassName: cilium
  listeners:
  - name: https
    protocol: HTTPS
    port: 443
    hostname: "cni.evescn.com"
    tls:
      certificateRefs:
      - kind: Secret
        name: test-cert
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: https-app-cni
spec:
  parentRefs:
  - name: tls-gateway
  hostnames:
  - "cni.evescn.com"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: cni
      port: 80


root@KinD:~# kubectl apply -f gateway-https.yaml
gateway.gateway.networking.k8s.io/tls-gateway created
httproute.gateway.networking.k8s.io/https-app-cni created

root@KinD:~#  kubectl get svc
NAME                           TYPE           CLUSTER-IP      EXTERNAL-IP    PORT(S)         AGE
cilium-gateway-basic-gateway   LoadBalancer   10.96.30.37     172.18.0.200   80:30594/TCP    7m21s
cilium-gateway-tls-gateway     LoadBalancer   10.96.5.61      172.18.0.201   443:30197/TCP   19s
cni                            ClusterIP      10.96.145.109   <none>         80/TCP          11m
kubernetes                     ClusterIP      10.96.0.1       <none>         443/TCP         145m

root@KinD:~# kubectl get gateway
NAME            CLASS    ADDRESS        READY   AGE
basic-gateway   cilium   172.18.0.200   True    7m47s
tls-gateway     cilium   172.18.0.201   True    45s

root@KinD:~# kubectl get HTTPRoute
NAME            HOSTNAMES                AGE
http-app-cni                             7m56s
https-app-cni   ["cni.evescn.com"]   54s

建立 gateway 後 tls-gateway 在定義的地址池中自動獲取了一個 IP 地址，並且自動建立了一個 cilium-gateway-tls-gateway svc 服務，服務的 EXTERNAL-IP 為 tls-gateway 從地址池中自動獲取的 IP 地址

測試 `Gateway` `HTTPS`

/etc/hosts 新增 dns 解析

root@KinD:~# cat /etc/hosts
......
172.18.0.201 cni.evescn.com

測試 Gateway HTTPS

root@KinD:~# curl --cacert /root/evescn.com.pem -v https://cni.evescn.com/
*   Trying 172.18.0.201:443...
* Connected to cni.evescn.com (172.18.0.201) port 443 (#0)
> GET / HTTP/1.1
> Host: cni.evescn.com
> User-Agent: curl/7.81.0
> Accept: */*
> 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< server: envoy
< date: Tue, 16 Jul 2024 06:25:27 GMT
< content-type: text/html
< content-length: 46
< last-modified: Tue, 16 Jul 2024 06:09:16 GMT
< etag: "66960e8c-2e"
< accept-ranges: bytes
< x-envoy-upstream-service-time: 0
< 
PodName: cni-j9bzr | PodIP: eth0 10.0.1.69/32
* Connection #0 to host cni.evescn.com left intact

在宿主機 172.18.0.1 主機上訪問業務正常訪問，使用 ip 地址為 172.18.0.201

六、轉載部落格

https://bbs.huaweicloud.com/blogs/418684

Cilium Ingress 特性（轉載）
2024-07-13
Cilium Socket LB 特性
2024-07-10
構建api gateway之負載均衡
2023-02-07
APIGateway負載
容器網路Cilium：DualStack雙棧特性分析
2023-12-15
Kubernetes Gateway API 介紹
2023-04-04
GatewayAPI
cilium 元件
2024-05-07
元件
C# 6.0新特性（轉載）
2016-01-15
C#
Spring Cloud Gateway限制API速率 - tanzu
2021-04-27
SpringCloudGatewayAPI
構建api gateway之健康檢查
2023-02-09
APIGateway
微服務實戰（二）：使用API Gateway
2017-09-28
微服務APIGateway
面向未來的閘道器： Kubernetes Gateway API 和 Envoy Gateway
2022-10-28
GatewayAPI
cilium Cluster Mesh
2024-05-30
Oracle 11g 新特性(轉載)
2007-07-13
Oracle
構建api gateway之動態外掛
2023-02-10
APIGateway
構建api gateway之 http路由實現
2023-01-30
APIGatewayHTTP路由
A gateway to automatically provide RESTful API for gRPC
2018-02-10
GatewayIDERESTAPIRPC
How to return plain text from AWS Lambda & API Gateway
2017-02-20
AIAPIGateway
JAVA8 新特性（二）轉載整理
2015-03-23
Java
使用 caddy 作為微服務的 API gateway
2019-02-16
微服務APIGateway
拆輪子：閘道器GOKU-API-Gateway
2018-07-15
GoAPIGateway
構建api gateway之如何給openresty打patch
2023-02-11
APIGatewayREST
[轉載] Java7中增加的新特性
2020-12-24
Java
轉載：Oracle10g新特性——審計
2009-04-09
Oracle
Spring Cloud Gateway之負載均衡
2019-02-05
SpringCloudGateway負載
在滴滴雲上搭建 API-Gateway Kong 實踐
2019-01-09
APIGateway
APISIX Ingress 對 Gateway API 的支援和應用
2022-12-29
APIGateway
Rainbond的 Gateway API 外掛製作實踐
2023-03-28
AIGatewayAPI
Kubernetes Gateway API 深入解讀和落地指南
2023-05-06
GatewayAPI
在 AWS Lambda 上寫 Go 語言搭配 API Gateway
2018-01-24
GoAPIGateway
ASP.NET Web API 特性
2015-07-24
ASP.NETWebAPI
Azure上七層負載均衡APP Gateway
2015-12-16
負載APPGateway
Cilium 首次整合國內雲服務，阿里雲 ENI 被納入新版本特性
2021-06-16
阿里
SpringCloud系列之API閘道器(Gateway)服務Zuul
2020-08-06
SpringGCCloudAPIGatewayZuul
SIA-GateWay之API閘道器安裝部署指南
2019-09-11
GatewayAPI
原始碼分析Gateway請求轉發
2021-06-24
原始碼Gateway
Yii2.0 RESTful API 基礎配置教程[轉載]
2019-02-16
RESTAPI
Java 8新特性：lambda表示式（四）轉載總結
2015-03-23
Java
Sun提供Java 1.5新特性原型實現下載 (轉)
2007-08-15
Java原型

Cilium Gateway API 特性（轉載）

Cilium Gateway API 特性（轉載）

一、環境資訊

二、Cilium Gateway API 流程圖

三、Cilium Gateway API 模式環境搭建

安裝須知

kind 配置檔案資訊

安裝 metallb 服務，提供 LoadBanlencer 功能

k8s 叢集安裝 Pod 和 Service

四、測試 Gateway HTTP 特性

配置 Gateway HTTP

測試 Gateway HTTP

五、測試 Gateway HTTPS 特性

建立 tls 證書，此處使用了一個測試證書

配置 Gateway HTTPS

測試 Gateway HTTPS

六、轉載部落格

相關文章

二、`Cilium` `Gateway API` 流程圖

三、`Cilium` `Gateway API` 模式環境搭建

`kind` 配置檔案資訊

安裝 `metallb` 服務，提供 `LoadBanlencer` 功能

`k8s` 叢集安裝 `Pod` 和 `Service`

四、測試 `Gateway` `HTTP` 特性

配置 `Gateway` `HTTP`

測試 `Gateway` `HTTP`

五、測試 `Gateway` `HTTPS` 特性

配置 `Gateway` `HTTPS`

測試 `Gateway` `HTTPS`