Istio中的流量配置

charlieroro發表於2020-09-18

Istio中的流量配置

Istio中的流量配置

Istio注入的容器

Istio的資料面會在pod中注入兩個容器：istio-init和istio-proxy。

Istio-init

istio-init會通過建立iptables規則來接管流量：

命令列引數 -p 15001表示出向流量被iptable重定向到Envoy的15001埠
命令列引數 -z 15006表示入向流量被iptable重定向到Envoy的15006埠
命令列引數 -u 1337引數用於排除使用者ID為1337，即Envoy自身的流量，以避免Iptable把Envoy發出的資料又重定向到Envoy，形成死迴圈。在istio-proxy容器中執行如下命令可以看到Envoy使用的使用者id為1337。
```
$ id
uid=1337(istio-proxy) gid=1337(istio-proxy) groups=1337(istio-proxy)
```

  istio-iptables
  -p
  15001
  -z
  15006
  -u
  1337
  -m
  REDIRECT
  -i
  *
  -x

  -b
  *
  -d
  15090,15021,15020
  --run-validation
  --skip-rule-apply

istio-proxy

istio-proxy容器中會執行兩個程式：pilot-agent和envoy。

$ ps -ef|cat
UID          PID    PPID  C STIME TTY          TIME CMD
istio-p+       1       0  0 Sep10 ?        00:03:39 /usr/local/bin/pilot-agent proxy sidecar --domain default.svc.cluster.local --serviceCluster sleep.default --proxyLogLevel=warning --proxyComponentLogLevel=misc:error --trust-domain=cluster.local --concurrency 2
istio-p+      27       1  0 Sep10 ?        00:14:30 /usr/local/bin/envoy -c etc/istio/proxy/envoy-rev0.json --restart-epoch 0 --drain-time-s 45 --parent-shutdown-time-s 60 --service-cluster sleep.default --service-node sidecar~10.80.3.109~sleep-856d589c9b-x6szk.default~default.svc.cluster.local --local-address-ip-version v4 --log-format-prefix-with-location 0 --log-format %Y-%m-%dT%T.%fZ.%l.envoy %n.%v -l warning --component-log-level misc:error --concurrency 2

Envoy架構

Envoy對入站/出站請求的處理過程如下，Envoy按照如下順序依次在各個過濾器中處理請求。

Pilot-agent生成的初始配置檔案

pilot-agent根據啟動引數和K8S API Server中的配置資訊生成Envoy的bootstrap檔案(/etc/istio/proxy/envoy-rev0.json)，並負責啟動Envoy程式(可以看到Envoy程式的父程式是pilot-agent)；envoy會通過xDS介面從istiod動態獲取配置檔案。envoy-rev0.json初始配置檔案結構如下：

node：給出了Envoy 例項的資訊

  "node": {
    "id": "sidecar~10.80.3.109~sleep-856d589c9b-x6szk.default~default.svc.cluster.local",
    "cluster": "sleep.default",
    "locality": {
    },
    "metadata": {"APP_CONTAINERS":"sleep,istio-proxy","CLUSTER_ID":"Kubernetes","EXCHANGE_KEYS":"NAME,NAMESPACE,INSTANCE_IPS,LABELS,OWNER,PLATFORM_METADATA,WORKLOAD_NAME,MESH_ID,SERVICE_ACCOUNT,CLUSTER_ID","INSTANCE_IPS":"10.80.3.109,fe80::40fb:daff:feed:e56c","INTERCEPTION_MODE":"REDIRECT","ISTIO_PROXY_SHA":"istio-proxy:f642a7fd07d0a99944a6e3529566e7985829839c","ISTIO_VERSION":"1.7.0","LABELS":{"app":"sleep","istio.io/rev":"default","pod-template-hash":"856d589c9b","security.istio.io/tlsMode":"istio","service.istio.io/canonical-name":"sleep","service.istio.io/canonical-revision":"latest"},"MESH_ID":"cluster.local","NAME":"sleep-856d589c9b-x6szk","NAMESPACE":"default","OWNER":"kubernetes://apis/apps/v1/namespaces/default/deployments/sleep","POD_PORTS":"[{\"name\":\"http-envoy-prom\",\"containerPort\":15090,\"protocol\":\"TCP\"}]","PROXY_CONFIG":{"binaryPath":"/usr/local/bin/envoy","concurrency":2,"configPath":"./etc/istio/proxy","controlPlaneAuthPolicy":"MUTUAL_TLS","discoveryAddress":"istiod.istio-system.svc:15012","drainDuration":"45s","envoyAccessLogService":{},"envoyMetricsService":{},"parentShutdownDuration":"60s","proxyAdminPort":15000,"proxyMetadata":{"DNS_AGENT":""},"serviceCluster":"sleep.default","statNameLength":189,"statusPort":15020,"terminationDrainDuration":"5s","tracing":{"zipkin":{"address":"zipkin.istio-system:9411"}}},"SDS":"true","SERVICE_ACCOUNT":"sleep","WORKLOAD_NAME":"sleep","k8s.v1.cni.cncf.io/networks":"istio-cni","sidecar.istio.io/interceptionMode":"REDIRECT","sidecar.istio.io/status":"{\"version\":\"8e6e902b765af607513b28d284940ee1421e9a0d07698741693b2663c7161c11\",\"initContainers\":[\"istio-validation\"],\"containers\":[\"istio-proxy\"],\"volumes\":[\"istio-envoy\",\"istio-data\",\"istio-podinfo\",\"istiod-ca-cert\"],\"imagePullSecrets\":null}","traffic.sidecar.istio.io/excludeInboundPorts":"15020","traffic.sidecar.istio.io/includeOutboundIPRanges":"*"}
  },

admin：給出了Envoy的日誌路徑和管理埠，例如可以通過curl -X POST localhost:15000/logging?level=trace設定日誌級別為trace。

  "admin": {
    "access_log_path": "/dev/null", /* 管理伺服器的訪問日誌路徑 */
    "profile_path": "/var/lib/istio/data/envoy.prof", /* 管理伺服器的CPU輸出路徑 */
    "address": { /* 管理伺服器監聽的TCP地址 */
      "socket_address": {
        "address": "127.0.0.1",
        "port_value": 15000
      }
    }
  },

dynamic_resources：配置動態資源，用於配置lds_config，cds_config和ads_config

Envoy通過xds-grpc cluster(參見static_resources)來獲得xDS服務的地址。

  "dynamic_resources": {
    "lds_config": { /* 通過一個LDS配置Listeners */
      "ads": {},
      "resource_api_version": "V3" /* LDS使用的API版本號 */
    },
    "cds_config": { /* 通過一個CDS配置Cluster */
      "ads": {},
      "resource_api_version": "V3"
    },
    "ads_config": { /* API配置資源，用於指定API型別和Envoy獲取xDS API的cluster */
      "api_type": "GRPC", /* 使用GRPC獲取xDS資訊 */
      "transport_api_version": "V3", /* xDS傳輸協議的版本號 */
      "grpc_services": [
        {
          "envoy_grpc": {
            "cluster_name": "xds-grpc" /* 動態獲取xDS配置的cluster */
          }
        }
      ]
    }
  },

static_resources：配置靜態資源，主要包括clusters和listeners兩種資源：

clusters：下面給出了幾個靜態配置的cluster。

    "clusters": [
      {
        "name": "prometheus_stats", /* 使用Prometheus暴露metrics，介面為127.0.0.1:15000/stats/prometheus */
        "type": "STATIC", /* 明確指定上游host的網路名(IP地址/埠等) */
        "connect_timeout": "0.250s",
        "lb_policy": "ROUND_ROBIN",
        "load_assignment": { /* 僅用於型別為STATIC, STRICT_DNS或LOGICAL_DNS，用於給非EDS的clyster內嵌與EDS等同的endpoint */
          "cluster_name": "prometheus_stats", 
          "endpoints": [{
            "lb_endpoints": [{ /* 負載均衡的後端 */
              "endpoint": {
                "address":{
                  "socket_address": {
                    "protocol": "TCP",
                    "address": "127.0.0.1",
                    "port_value": 15000
                  }
                }
              }
            }]
          }]
        }
      },
      {
        "name": "agent", /* 暴露健康檢查介面，可以使用curl http://127.0.0.1:15020/healthz/ready -v檢視 */
        "type": "STATIC",
        "connect_timeout": "0.250s",
        "lb_policy": "ROUND_ROBIN",
        "load_assignment": {
          "cluster_name": "prometheus_stats",
          "endpoints": [{
            "lb_endpoints": [{ /* 負載均衡的後端 */
              "endpoint": {
                "address":{
                  "socket_address": {
                    "protocol": "TCP",
                    "address": "127.0.0.1",
                    "port_value": 15020
                  }
                }
              }
            }]
          }]
        }
      },
      {
        "name": "sds-grpc", /* 配置SDS cluster */
        "type": "STATIC",
        "http2_protocol_options": {},
        "connect_timeout": "1s",
        "lb_policy": "ROUND_ROBIN",
        "load_assignment": {
          "cluster_name": "sds-grpc",
          "endpoints": [{
            "lb_endpoints": [{
              "endpoint": {
                "address":{
                  "pipe": {
                    "path": "./etc/istio/proxy/SDS" /* 進行SDS的UNIX socket路徑，用於在mTLS期間給istio-agent和proxy提供通訊 */
                  }
                }
              }
            }]
          }]
        }
      },
      {
        "name": "xds-grpc", /* 動態xDS使用的grpc伺服器配置 */
        "type": "STRICT_DNS",
        "respect_dns_ttl": true,
        "dns_lookup_family": "V4_ONLY",
        "connect_timeout": "1s",
        "lb_policy": "ROUND_ROBIN",
        "transport_socket": { /* 配置與上游連線的傳輸socket */
          "name": "envoy.transport_sockets.tls", /* 需要例項化的傳輸socket名稱 */
          "typed_config": {
            "@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext",
            "sni": "istiod.istio-system.svc", /* 建立TLS後端(即SDS伺服器)連線時要使用的SNI字串 */
            "common_tls_context": { /* 配置client和server使用的TLS上下文 */
              "alpn_protocols": [/* listener暴露的ALPN協議列表，如果為空，則不使用APPN */
                "h2"
              ],
              "tls_certificate_sds_secret_configs": [/*通過SDS API獲取TLS證書的配置 */
                {
                  "name": "default",
                  "sds_config": { /* 配置sds_config時將會從靜態資源載入secret */
                    "resource_api_version": "V3", /* xDS的API版本 */
                    "initial_fetch_timeout": "0s",
                    "api_config_source": { /* SDS API配置，如版本和SDS服務 */
                      "api_type": "GRPC",
                      "transport_api_version": "V3", /* xDS傳輸協議的API版本 */
                      "grpc_services": [
                        { /* SDS伺服器對應上面配置的sds-grpc cluster */
                          "envoy_grpc": { "cluster_name": "sds-grpc" }
                        }
                      ]
                    }
                  }
                }
              ],
              "validation_context": {
                "trusted_ca": {
                  "filename": "./var/run/secrets/istio/root-cert.pem" /* 本地檔案系統的資料來源。掛載當前名稱空間下的config istio-ca-root-cert，其中的CA證書與istio-system名稱空間下的istio-ca-secret中的CA證書相同，用於校驗對端istiod的證書 */
                },
                "match_subject_alt_names": [{"exact":"istiod.istio-system.svc"}] /* 驗證證書中的SAN，即來自istiod的證書 */
              }
            }
          }
        },
        "load_assignment": {
          "cluster_name": "xds-grpc", /* 可以看到xds-grpc的後端為istiod的15012埠 */
          "endpoints": [{
            "lb_endpoints": [{
              "endpoint": {
                "address":{
                  "socket_address": {"address": "istiod.istio-system.svc", "port_value": 15012}
                }
              }
            }]
          }]
        },
        "circuit_breakers": { /* 斷路器配置 */
          "thresholds": [
            {
              "priority": "DEFAULT",
              "max_connections": 100000,
              "max_pending_requests": 100000,
              "max_requests": 100000
            },
            {
              "priority": "HIGH",
              "max_connections": 100000,
              "max_pending_requests": 100000,
              "max_requests": 100000
            }
          ]
        },
        "upstream_connection_options": {
          "tcp_keepalive": {
            "keepalive_time": 300
          }
        },
        "max_requests_per_connection": 1,
        "http2_protocol_options": { }
      }
              ,
      {
        "name": "zipkin", /* 分散式鏈路跟蹤zipkin的cluster配置 */
        "type": "STRICT_DNS",
        "respect_dns_ttl": true,
        "dns_lookup_family": "V4_ONLY",
        "connect_timeout": "1s",
        "lb_policy": "ROUND_ROBIN",
        "load_assignment": {
          "cluster_name": "zipkin",
          "endpoints": [{
            "lb_endpoints": [{
              "endpoint": {
                "address":{
                  "socket_address": {"address": "zipkin.istio-system", "port_value": 9411}
                }
              }
            }]
          }]
        }
      }
    ],

上面使用type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext API介面來對傳輸socket進行配置，sni和common_tls_context 都屬於結構體UpstreamTlsContext中的成員變數。

可以使用istioctl pc cluster命令檢視靜態cluster資源，第一列對應上面的Cluster.name，其中sds-grpc用於提供SDS服務，SDS的原理可以參見官方文件。

# istioctl pc cluster sleep-856d589c9b-x6szk.default |grep STATIC
BlackHoleCluster                        -         -          -             STATIC
agent                                   -         -          -             STATIC
prometheus_stats                        -         -          -             STATIC
sds-grpc                                -         -          -             STATIC
sleep.default.svc.cluster.local         80        http       inbound       STATIC

listener，下面用到了Network filter中的HTTP connection manager

    "listeners":[
      {
        "address": { /* listener監聽的地址 */
          "socket_address": {
            "protocol": "TCP",
            "address": "0.0.0.0",
            "port_value": 15090
          }
        },
        "filter_chains": [
          {
            "filters": [
              {
                "name": "envoy.http_connection_manager",
                "typed_config": { /* 對擴充套件API的配置 */
                  "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager",
                  "codec_type": "AUTO", /* 由連線管理器判斷使用哪種編解碼器 */
                  "stat_prefix": "stats",
                  "route_config": { /* 連線管理器的靜態路由表 */
                    "virtual_hosts": [ /* 路由表使用的虛擬主機列表 */
                      {
                        "name": "backend", /* 路由表使用的虛擬主機 */
                        "domains": [ /* 匹配該虛擬主機的域列表 */
                          "*"
                        ],
                        "routes": [ /* 匹配入請求的路由列表，使用第一個匹配的路由 */
                          {
                            "match": { /* 將HTTP地址為/stats/prometheus的請求路由到cluster prometheus_stats */
                              "prefix": "/stats/prometheus" 
                            },
                            "route": {
                              "cluster": "prometheus_stats"
                            }
                          }
                        ]
                      }
                    ]
                  },
                  "http_filters": [{ /* 構成filter鏈的filter，用於處理請求。此處並沒有定義任何規則 */
                    "name": "envoy.router",
                    "typed_config": {
                      "@type": "type.googleapis.com/envoy.extensions.filters.http.router.v3.Router"
                    }
                  }]
                }
              }
            ]
          }
        ]
      },
      {
        "address": {
          "socket_address": {
            "protocol": "TCP",
            "address": "0.0.0.0",
            "port_value": 15021
          }
        },
        "filter_chains": [
          {
            "filters": [
              {
                "name": "envoy.http_connection_manager",
                "typed_config": {
                  "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager",
                  "codec_type": "AUTO",
                  "stat_prefix": "agent",
                  "route_config": { /* 靜態路由表配置 */
                    "virtual_hosts": [
                      {
                        "name": "backend",
                        "domains": [
                          "*"
                        ],
                        "routes": [
                          {
                            "match": { /* 將HTTP地址為/healthz/ready的請求路由到cluster agent */
                              "prefix": "/healthz/ready"
                            },
                            "route": {
                              "cluster": "agent"
                            }
                          }
                        ]
                      }
                    ]
                  },
                  "http_filters": [{
                    "name": "envoy.router",
                    "typed_config": {
                      "@type": "type.googleapis.com/envoy.extensions.filters.http.router.v3.Router"
                    }
                  }]
                }
              }
            ]
          }
        ]
      }
    ]
  }

tracing，對應上面static_resources裡定義的zipkin cluster。

  "tracing": {
    "http": {
      "name": "envoy.zipkin",
      "typed_config": {
        "@type": "type.googleapis.com/envoy.config.trace.v3.ZipkinConfig",
        "collector_cluster": "zipkin",
        "collector_endpoint": "/api/v2/spans",
        "collector_endpoint_version": "HTTP_JSON",
        "trace_id_128bit": true,
        "shared_span_context": false
      }
    }
  }

基本流程如下：

Envoy管理介面獲取的完整配置

可以在注入Envoy sidecar的pod中執行curl -X POST localhost:15000/config_dump來獲取完整的配置資訊。可以看到它主要包含BootstrapConfig，ClustersConfig，ListenersConfig，RoutesConfig，SecretsConfig這5部分。

Bootstrap：它與上面由Pilot-agent生成的envoy-rev0.json檔案中的內容相同，即提供給Envoy proxy的初始化配置，給出了xDS伺服器的地址等資訊。

Clusters：在Envoy中，Cluster是一個服務叢集，每個cluster包含一個或多個endpoint(可以將cluster近似看作是k8s中的service)。

從上圖可以看出，ClustersConfig包含兩種cluster配置：static_clusters和dynamic_active_clusters。前者中的cluster來自envoy-rev0.json中配置的靜態cluster資源，包含agent，prometheus_stats，sds-grpc，xds-grpc和zipkin；後者是通過xDS介面從istio的控制面獲取的動態配置資訊，dynamic_active_clusters主要分為如下四種型別：

BlackHoleCluster：

     "cluster": {
      "@type": "type.googleapis.com/envoy.config.cluster.v3.Cluster",
      "name": "BlackHoleCluster", /* cluster名稱 */
      "type": "STATIC",
      "connect_timeout": "10s",
      "filters": [ /* 出站連線的過濾器配置 */
       {
        "name": "istio.metadata_exchange",
        "typed_config": { /* 對擴充套件API的配置 */
         "@type": "type.googleapis.com/udpa.type.v1.TypedStruct",
         "type_url": "type.googleapis.com/envoy.tcp.metadataexchange.config.MetadataExchange",
         "value": {
          "protocol": "istio-peer-exchange"
         }
        }
       }
      ]
     },

BlackHoleCluster使用的API型別為type.googleapis.com/udpa.type.v1.TypedStruct，表示控制面缺少缺少該擴充套件的模式定義，client會使用type_url指定的API，將內容轉換為型別化的配置資源。

在上面可以看到Istio使用了協議istio-peer-exchange ，服務網格內部的兩個Envoy例項之間使用該協議來互動Node Metadata。NodeMetadata的資料結構如下：

type NodeMetadata struct {
	// ProxyConfig defines the proxy config specified for a proxy.
	// Note that this setting may be configured different for each proxy, due user overrides
	// or from different versions of proxies connecting. While Pilot has access to the meshConfig.defaultConfig,
	// this field should be preferred if it is present.
	ProxyConfig *NodeMetaProxyConfig `json:"PROXY_CONFIG,omitempty"`

	// IstioVersion specifies the Istio version associated with the proxy
	IstioVersion string `json:"ISTIO_VERSION,omitempty"`

	// Labels specifies the set of workload instance (ex: k8s pod) labels associated with this node.
	Labels map[string]string `json:"LABELS,omitempty"`

	// InstanceIPs is the set of IPs attached to this proxy
	InstanceIPs StringList `json:"INSTANCE_IPS,omitempty"`

	// Namespace is the namespace in which the workload instance is running.
	Namespace string `json:"NAMESPACE,omitempty"`

	// InterceptionMode is the name of the metadata variable that carries info about
	// traffic interception mode at the proxy
	InterceptionMode TrafficInterceptionMode `json:"INTERCEPTION_MODE,omitempty"`

	// ServiceAccount specifies the service account which is running the workload.
	ServiceAccount string `json:"SERVICE_ACCOUNT,omitempty"`

	// RouterMode indicates whether the proxy is functioning as a SNI-DNAT router
	// processing the AUTO_PASSTHROUGH gateway servers
	RouterMode string `json:"ROUTER_MODE,omitempty"`

	// MeshID specifies the mesh ID environment variable.
	MeshID string `json:"MESH_ID,omitempty"`

	// ClusterID defines the cluster the node belongs to.
	ClusterID string `json:"CLUSTER_ID,omitempty"`

	// Network defines the network the node belongs to. It is an optional metadata,
	// set at injection time. When set, the Endpoints returned to a note and not on same network
	// will be replaced with the gateway defined in the settings.
	Network string `json:"NETWORK,omitempty"`

	// RequestedNetworkView specifies the networks that the proxy wants to see
	RequestedNetworkView StringList `json:"REQUESTED_NETWORK_VIEW,omitempty"`

	// PodPorts defines the ports on a pod. This is used to lookup named ports.
	PodPorts PodPortList `json:"POD_PORTS,omitempty"`

	// TLSServerCertChain is the absolute path to server cert-chain file
	TLSServerCertChain string `json:"TLS_SERVER_CERT_CHAIN,omitempty"`
	// TLSServerKey is the absolute path to server private key file
	TLSServerKey string `json:"TLS_SERVER_KEY,omitempty"`
	// TLSServerRootCert is the absolute path to server root cert file
	TLSServerRootCert string `json:"TLS_SERVER_ROOT_CERT,omitempty"`
	// TLSClientCertChain is the absolute path to client cert-chain file
	TLSClientCertChain string `json:"TLS_CLIENT_CERT_CHAIN,omitempty"`
	// TLSClientKey is the absolute path to client private key file
	TLSClientKey string `json:"TLS_CLIENT_KEY,omitempty"`
	// TLSClientRootCert is the absolute path to client root cert file
	TLSClientRootCert string `json:"TLS_CLIENT_ROOT_CERT,omitempty"`

	CertBaseDir string `json:"BASE,omitempty"`

	// IdleTimeout specifies the idle timeout for the proxy, in duration format (10s).
	// If not set, no timeout is set.
	IdleTimeout string `json:"IDLE_TIMEOUT,omitempty"`

	// HTTP10 indicates the application behind the sidecar is making outbound http requests with HTTP/1.0
	// protocol. It will enable the "AcceptHttp_10" option on the http options for outbound HTTP listeners.
	// Alpha in 1.1, based on feedback may be turned into an API or change. Set to "1" to enable.
	HTTP10 string `json:"HTTP10,omitempty"`

	// Generator indicates the client wants to use a custom Generator plugin.
	Generator string `json:"GENERATOR,omitempty"`

	// DNSCapture indicates whether the workload has enabled dns capture
	DNSCapture string `json:"DNS_CAPTURE,omitempty"`

	// ProxyXDSViaAgent indicates that xds data is being proxied via the agent
	ProxyXDSViaAgent string `json:"PROXY_XDS_VIA_AGENT,omitempty"`

	// Contains a copy of the raw metadata. This is needed to lookup arbitrary values.
	// If a value is known ahead of time it should be added to the struct rather than reading from here,
	Raw map[string]interface{} `json:"-"`
}

Istio通過一些特定的TCP屬性來啟用TCP策略和控制(這些屬性由Envoy代理生成)，並通過Envoy的Node Metadata來獲取這些屬性。Envoy使用ALPN隧道和基於字首的協議來轉發Node Metadata到對端的Envoy。Istio定義了一個新的協議istio-peer-exchange，由網格中的客戶端和服務端的sidecar在TLS協商時進行宣告並確定優先順序。啟用istio代理的兩端會通過ALPN協商將協議解析為istio-peer-exchange(因此僅限於istio服務網格內的互動)，後續的TCP互動將會按照istio-peer-exchange的協議規則進行互動：

使用如下命令可以看到cluster BlackHoleCluster是沒有endpoint的。

# istioctl pc endpoint sleep-856d589c9b-x6szk.default --cluster BlackHoleCluster
ENDPOINT     STATUS     OUTLIER CHECK     CLUSTER

如下內容參考官方部落格：

對於外部服務，Istio提供了兩種管理方式：通過將global.outboundTrafficPolicy.mode 設定為REGISTRY_ONLY來block所有到外部服務的訪問；以及通過將global.outboundTrafficPolicy.mode設定為ALLOW_ANY來允許所有到外部服務的訪問。預設會允許所有到外部服務的訪問。

BlackHoleCluster ：當global.outboundTrafficPolicy.mode設定為REGISTRY_ONLY時，Envoy會建立一個虛擬的cluster BlackHoleCluster。該模式下，所有到外部服務的訪問都會被block(除非為每個服務新增service entries)。為了實現該功能，預設的outbound listener(監聽地址為 0.0.0.0:15001)使用原始目的地來設定TCP代理，BlackHoleCluster 作為一個靜態cluster。由於BlackHoleCluster 沒有任何endpoint，因此會丟棄所有到外部的流量。此外，Istio會為平臺服務的每個埠/協議組合建立唯一的listener，如果對同一埠的外部服務發起請求，則不會命中虛擬listener。這種情況會對route配置進行擴充套件，新增到BlackHoleCluster的路由。如果沒有匹配到其他路由，則Envoy代理會直接返回502 HTTP狀態碼(BlackHoleCluster可以看作是路由黑洞)。
       {
        "name": "block_all",
        "domains": [
         "*"
        ],
        "routes": [
         {
          "match": {
           "prefix": "/"
          },
          "direct_response": {
           "status": 502
          },
          "name": "block_all"
         }
        ],
        "include_request_attempt_count": true
       },

PassthroughCluster：可以看到PassthroughCluster也使用了istio-peer-exchange協議來處理TCP。

     "cluster": {
      "@type": "type.googleapis.com/envoy.config.cluster.v3.Cluster",
      "name": "PassthroughCluster",
      "type": "ORIGINAL_DST", /* 指定type為ORIGINAL_DST，這是一個特殊的cluster */
      "connect_timeout": "10s",
      "lb_policy": "CLUSTER_PROVIDED",
      "circuit_breakers": {
       "thresholds": [
        {
         "max_connections": 4294967295,
         "max_pending_requests": 4294967295,
         "max_requests": 4294967295,
         "max_retries": 4294967295
        }
       ]
      },
      "protocol_selection": "USE_DOWNSTREAM_PROTOCOL",
      "filters": [
       {
        "name": "istio.metadata_exchange", /* 配置使用ALPN istio-peer-exchange協議來交換Node Metadata */
        "typed_config": {
         "@type": "type.googleapis.com/udpa.type.v1.TypedStruct",
         "type_url": "type.googleapis.com/envoy.tcp.metadataexchange.config.MetadataExchange",
         "value": {
          "protocol": "istio-peer-exchange"
         }
        }
       }
      ]
     },

使用如下命令可以看到cluster PassthroughCluster是沒有endpoint的。

# istioctl pc endpoint sleep-856d589c9b-x6szk.default --cluster PassthroughCluster
ENDPOINT     STATUS     OUTLIER CHECK     CLUSTER

PassthroughCluster ：當global.outboundTrafficPolicy.mode設定為ALLOW_ANY時，Envoy會建立一個虛擬的cluster PassthroughCluster 。該模式下，會允許所有到外部服務的訪問。為了實現該功能，預設的outbound listener(監聽地址為 0.0.0.0:15001)使用SO_ORIGINAL_DST來配置TCP Proxy，PassthroughCluster作為一個靜態cluster。

PassthroughCluster cluster使用原始目的地負載均衡策略來配置Envoy傳送到原始目的地的流量。

與BlackHoleCluster類似，對於每個基於埠/協議的listener，都會新增虛擬路由，將PassthroughCluster作為為預設路由。
   {
    "name": "allow_any",
    "domains": [
     "*"
    ],
    "routes": [
     {
      "match": {
       "prefix": "/"
      },
      "route": {
       "cluster": "PassthroughCluster",
       "timeout": "0s",
       "max_grpc_timeout": "0s"
      },
      "name": "allow_any"
     }
    ],
    "include_request_attempt_count": true
   },

由於global.outboundTrafficPolicy.mode只能配置某一個值，因此BlackHoleCluster和PassthroughCluster的出現是互斥的，BlackHoleCluster和PassthroughCluster的路由僅存在istio服務網格內，即注入sidecar的pod中。

可以使用Prometheus metrics來監控到BlackHoleCluster和PassthroughCluster的訪問。

inbound cluster：處理入站請求的cluster，對於下面的sleep應用來說，其只有一個本地後端127.0.0.1:80，並通過load_assignment指定了cluster名稱和負載資訊。由於該監聽器上的流量不會出戰，因此下面並沒有配置過濾器。

 "cluster": {
  "@type": "type.googleapis.com/envoy.config.cluster.v3.Cluster",
  "name": "inbound|80|http|sleep.default.svc.cluster.local",
  "type": "STATIC",
  "connect_timeout": "10s",
  "circuit_breakers": {
   "thresholds": [
    {
     "max_connections": 4294967295,
     "max_pending_requests": 4294967295,
     "max_requests": 4294967295,
     "max_retries": 4294967295
    }
   ]
  },
  "load_assignment": { /* 設定入站的cluster的endpoint的負載均衡 */
   "cluster_name": "inbound|80|http|sleep.default.svc.cluster.local",
   "endpoints": [
    {
     "lb_endpoints": [
      {
       "endpoint": {
        "address": {
         "socket_address": {
          "address": "127.0.0.1",
          "port_value": 80
         }
        }
       }
      }
     ]
    }
   ]
  }
 },

也可以使用如下命令檢視inbound的cluster資訊：

# istioctl pc cluster sleep-856d589c9b-c6xsm.default --direction inbound
SERVICE FQDN                        PORT     SUBSET     DIRECTION     TYPE       DESTINATION RULE
sleep.default.svc.cluster.local     80       http       inbound       STATIC

outbound cluster：這類cluster為Envoy節點外的服務，配置如何連線上游。下面的EDS表示該cluster的endpoint來自EDS服務發現。下面給出的outbound cluster是istiod的15012埠上的服務。基本結構如下，transport_socket_matches僅在使用TLS才會出現，用於配置與TLS證書相關的資訊。

具體內容如下：

{
 "version_info": "2020-09-15T08:05:54Z/4",
 "cluster": {
  "@type": "type.googleapis.com/envoy.config.cluster.v3.Cluster",
  "name": "outbound|15012||istiod.istio-system.svc.cluster.local",
  "type": "EDS", 
  "eds_cluster_config": { /* EDS的配置 */
   "eds_config": {
    "ads": {},
    "resource_api_version": "V3"
   },
   "service_name": "outbound|15012||istiod.istio-system.svc.cluster.local" /* EDS的cluster的可替代名稱，無需與cluster名稱完全相同 */
  },
  "connect_timeout": "10s",
  "circuit_breakers": { /* 斷路器設定 */
   "thresholds": [
    {
     "max_connections": 4294967295,
     "max_pending_requests": 4294967295,
     "max_requests": 4294967295,
     "max_retries": 4294967295
    }
   ]
  },
  "filters": [ /* 設定Node Metadata互動使用的協議為istio-peer-exchange */
   {
    "name": "istio.metadata_exchange",
    "typed_config": {
     "@type": "type.googleapis.com/udpa.type.v1.TypedStruct",
     "type_url": "type.googleapis.com/envoy.tcp.metadataexchange.config.MetadataExchange",
     "value": {
      "protocol": "istio-peer-exchange"
     }
    }
   }
  ],
  "transport_socket_matches": [ /* 指定匹配的後端使用的帶TLS的傳輸socket */
   {
    "name": "tlsMode-istio", /* match的名稱 */
    "match": { /* 匹配後端的條件，注入istio sidecar的pod會打上標籤：security.istio.io/tlsMode=istio */
     "tlsMode": "istio"
    },
    "transport_socket": { /* 匹配cluster的後端使用的傳輸socket的配置 */
     "name": "envoy.transport_sockets.tls",
     "typed_config": {
      "@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext",
      "common_tls_context": { /* 配置client和server端使用的TLS上下文 */
       "alpn_protocols": [ /* 配置互動使用的ALPN協議集，供上游選擇 */
        "istio-peer-exchange",
        "istio"
       ],
       "tls_certificate_sds_secret_configs": [ /* 通過SDS API獲取TLS證書的配置 */
        {
         "name": "default",
         "sds_config": {
          "api_config_source": {
           "api_type": "GRPC",
           "grpc_services": [ /* SDS的cluster */
            {
             "envoy_grpc": {
              "cluster_name": "sds-grpc" /* 為上面靜態配置的cluster */
             }
            }
           ],
           "transport_api_version": "V3"
          },
          "initial_fetch_timeout": "0s",
          "resource_api_version": "V3"
         }
        }
       ],
       "combined_validation_context": { /* 包含一個CertificateValidationContext(即下面的default_validation_context)和SDS配置。當SDS服務返回動態的CertificateValidationContext時，動態和預設的CertificateValidationContext會合併為一個新的CertificateValidationContext來進行校驗  */
        "default_validation_context": { /* 配置如何認證對端istiod服務的證書 */
         "match_subject_alt_names": [ /* Envoy會按照如下配置來校驗證書中的SAN */
          {
           "exact": "spiffe://new-td/ns/istio-system/sa/istiod-service-account" /* 與Istio配置的serviceaccount授權策略相同 */
          }
         ]
        },
        "validation_context_sds_secret_config": { /* SDS配置，也是通過靜態的cluster sds-grpc提供SDS API服務 */
         "name": "ROOTCA",
         "sds_config": {
          "api_config_source": {
           "api_type": "GRPC",
           "grpc_services": [
            {
             "envoy_grpc": {
              "cluster_name": "sds-grpc"
             }
            }
           ],
           "transport_api_version": "V3"
          },
          "initial_fetch_timeout": "0s",
          "resource_api_version": "V3"
         }
        }
       }
      },
      "sni": "outbound_.15012_._.istiod.istio-system.svc.cluster.local" /* 建立TLS連線時使用的SNI字串，即TLS的server_name擴充套件欄位中的值 */
     }
    }
   },
   {
    "name": "tlsMode-disabled", /* 如果與沒有匹配到的後端(即istio服務網格外的後端)進行通訊時，則使用明文方式 */
    "match": {},
    "transport_socket": {
     "name": "envoy.transport_sockets.raw_buffer"
    }
   }
  ]
 },
 "last_updated": "2020-09-15T08:06:23.565Z"
},

Listeners：Envoy使用listener來接收並處理下游發來的請求。與cluster類似，listener也分為靜態和動態兩種配置。靜態配置來自Istio-agent生成的envoy-rev0.json檔案。動態配置為：

virtualOutbound Listener：Istio在注入sidecar時，會通過init容器來設定iptables規則，將所有出站的TCP流量攔截到本地的15001埠：

-A ISTIO_REDIRECT -p tcp -j REDIRECT --to-ports 15001

一個istio-agent配置中僅包含一個virtualOutbound listener，可以看到該listener並沒有配置transport_socket，它的下游流量就是來自本pod的業務容器，並不需要進行TLS校驗，直接將流量重定向到15001埠即可，然後轉發給和原始目的IP:Port匹配的listener。

    {
     "name": "virtualOutbound",
     "active_state": {
      "version_info": "2020-09-15T08:05:54Z/4",
      "listener": {
       "@type": "type.googleapis.com/envoy.config.listener.v3.Listener",
       "name": "virtualOutbound",
       "address": { /* 監聽器監聽的地址 */
        "socket_address": {
         "address": "0.0.0.0",
         "port_value": 15001
        }
       },
       "filter_chains": [ /* 應用到該監聽器的過濾器鏈 */
        {
         "filters": [ /* 與該監聽器建立連線時使用的過濾器，按順序處理各個過濾器。如果過濾器列表為空，則預設會關閉連線 */
          {
           "name": "istio.stats",
           "typed_config": {
            "@type": "type.googleapis.com/udpa.type.v1.TypedStruct",
            "type_url": "type.googleapis.com/envoy.extensions.filters.network.wasm.v3.Wasm",
            "value": {
             "config": { /* Wasm外掛配置 */
              "root_id": "stats_outbound", /* 一個VM中具有相同root_id的一組filters/services會共享相同的RootContext和Contexts，如果該欄位為空，所有該欄位為空的filters/services都會共享具有相同vm_id的Context(s) */
              "vm_config": { /* Wasm VM的配置 */
               "vm_id": "tcp_stats_outbound", /* 使用相同vm_id和code將使用相同的VM */
               "runtime": "envoy.wasm.runtime.null", /* Wasm執行時，v8或null */
               "code": {
                "local": {
                 "inline_string": "envoy.wasm.stats"
                }
               }
              },
              "configuration": {
               "@type": "type.googleapis.com/google.protobuf.StringValue",
               "value": "{\n  \"debug\": \"false\",\n  \"stat_prefix\": \"istio\"\n}\n"
              }
             }
            }
           }
          },
          {
           "name": "envoy.tcp_proxy", /* 處理TCP的過濾器 */
           "typed_config": {
            "@type": "type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy",
            "stat_prefix": "PassthroughCluster",
            "cluster": "PassthroughCluster", /* 連線的上游cluster */
            "access_log": [
             {
              "name": "envoy.file_access_log",
              "typed_config": { /* 配置日誌的輸出格式和路徑 */
               "@type": "type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog",
               "path": "/dev/stdout",
               "format": "[%START_TIME%] \"%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%\" %RESPONSE_CODE% %RESPONSE_FLAGS% \"%DYNAMIC_METADATA(istio.mixer:status)%\" \"%UPSTREAM_TRANSPORT_FAILURE_REASON%\" %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% \"%REQ(X-FORWARDED-FOR)%\" \"%REQ(USER-AGENT)%\" \"%REQ(X-REQUEST-ID)%\" \"%REQ(:AUTHORITY)%\" \"%UPSTREAM_HOST%\" %UPSTREAM_CLUSTER% %UPSTREAM_LOCAL_ADDRESS% %DOWNSTREAM_LOCAL_ADDRESS% %DOWNSTREAM_REMOTE_ADDRESS% %REQUESTED_SERVER_NAME% %ROUTE_NAME%\n"
              }
             }
            ]
           }
          }
         ],
         "name": "virtualOutbound-catchall-tcp"
        }
       ],
       "hidden_envoy_deprecated_use_original_dst": true,
       "traffic_direction": "OUTBOUND"
      },
      "last_updated": "2020-09-15T08:06:24.066Z"
     }
    },

上面envoy.tcp_proxy過濾器的cluster為PassthroughCluster，這是因為將global.outboundTrafficPolicy.mode設定為了ALLOW_ANY，預設可以訪問外部服務。如果global.outboundTrafficPolicy.mode設定為了REGISTRY_ONLY，則此處將變為cluster BlackHoleCluster，預設丟棄所有到外部服務的請求。

上面使用wasm(WebAssembly)來記錄遙測資訊，Envoy官方文件中目前缺少對wasm的描述，可以參考開原始碼的描述。從runtime欄位為null可以看到並沒有啟用。可以在安裝istio的時候使用如下引數來啟用基於Wasm的遙測。
$ istioctl install --set values.telemetry.v2.metadataExchange.wasmEnabled=true --set values.telemetry.v2.prometheus.wasmEnabled=true
啟用之後，與wasm有關的用於遙測的過濾器配置變為了如下內容，可以看到其runtime使用了envoy.wasm.runtime.v8。更多參見官方部落格。
          {
           "name": "istio.stats",
           "typed_config": {
            "@type": "type.googleapis.com/udpa.type.v1.TypedStruct",
            "type_url": "type.googleapis.com/envoy.extensions.filters.network.wasm.v3.Wasm",
            "value": {
             "config": {
              "root_id": "stats_outbound",
              "vm_config": { /* wasm虛擬機器配置 */
               "vm_id": "tcp_stats_outbound",
               "runtime": "envoy.wasm.runtime.v8", /* 使用的wasm runtime */
               "code": {
                "local": {
                 "filename": "/etc/istio/extensions/stats-filter.compiled.wasm" /* 編譯後的wasm外掛路徑 */
                }
               },
               "allow_precompiled": true
              },
              "configuration": {
               "@type": "type.googleapis.com/google.protobuf.StringValue",
               "value": "{\n  \"debug\": \"false\",\n  \"stat_prefix\": \"istio\"\n}\n"
              }
             }
            }
           }
          },
在istio-proxy容器的/etc/istio/extensions/目錄下可以看到wasm編譯的相關程式，包含用於交換Node Metadata的metadata-exchange-filter.wasm和用於遙測的stats-filter.wasm，帶compiled的wasm用於HTTP。
$ ls
metadata-exchange-filter.compiled.wasm  metadata-exchange-filter.wasm  stats-filter.compiled.wasm  stats-filter.wasm
Istio的filter處理示意圖如下：

VirtualInbound/Inbound Listener：與virtualOutbound listener類似，通過如下規則將所有入站的TCP流量重定向到15006埠

-A ISTIO_IN_REDIRECT -p tcp -j REDIRECT --to-ports 15006

下面是一個demo環境中的典型配置，可以看到對於每個監聽的地址，都配置了兩個過濾器：一個帶transport_socket，一個不帶transport_socket，分別處理使用TLS的連線和不使用TLS的連線。主要的入站監聽器為：

處理基於IPv4的帶TLS 的TCP連線
處理基於IPv4的不帶TLS 的TCP連線
處理基於IPv6的帶TLS 的TCP連線
處理基於IPv6的不帶TLS 的TCP連線
處理基於IPv4的帶TLS 的HTTP連線
處理基於IPv4的不帶TLS 的HTTP連線
處理基於IPv6的帶TLS 的HTTP連線
處理基於IPv6的不帶TLS 的HTTP連線
處理業務的帶TLS(不帶TLS)的連線

下面給出如下內容的inbound listener：

處理基於IPv4的帶TLS 的TCP連線
處理基於IPv4的不帶TLS 的TCP連線
處理業務的帶TLS的連線

    {
     "name": "virtualInbound",
     "active_state": {
      "version_info": "2020-09-15T08:05:54Z/4",
      "listener": {
       "@type": "type.googleapis.com/envoy.config.listener.v3.Listener",
       "name": "virtualInbound",
       "address": { /* 該listener繫結的地址和埠 */
        "socket_address": {
         "address": "0.0.0.0",
         "port_value": 15006
        }
       },
       "filter_chains": [
        /* 匹配所有IPV4地址，使用TLS且ALPN為istio-peer-exchange或istio的連線 */
        {
         "filter_chain_match": { /* 將連線匹配到該過濾器鏈時使用的標準 */
          "prefix_ranges": [ /* 當listener繫結到0.0.0.0/::時匹配的IP地址和字首長度，下面表示整個網路地址 */
           {
            "address_prefix": "0.0.0.0",
            "prefix_len": 0
           }
          ],
          "transport_protocol": "tls", /* 匹配的傳輸協議 */
          "application_protocols": [ /* 使用的ALPN */
           "istio-peer-exchange",
           "istio"
          ]
         },
         "filters": [
          {
           "name": "istio.metadata_exchange", /* 交換Node Metadata的配置 */
           "typed_config": {
            "@type": "type.googleapis.com/udpa.type.v1.TypedStruct",
            "type_url": "type.googleapis.com/envoy.tcp.metadataexchange.config.MetadataExchange",
            "value": {
             "protocol": "istio-peer-exchange"
            }
           }
          },
          {
           "name": "istio.stats", /* 使用wasm進行遙測的配置 */
           "typed_config": {
            "@type": "type.googleapis.com/udpa.type.v1.TypedStruct",
            "type_url": "type.googleapis.com/envoy.extensions.filters.network.wasm.v3.Wasm",
            "value": {
             "config": {
              "root_id": "stats_inbound",
              "vm_config": {
               "vm_id": "tcp_stats_inbound",
               "runtime": "envoy.wasm.runtime.null",
               "code": {
                "local": {
                 "inline_string": "envoy.wasm.stats"
                }
               }
              },
              "configuration": {
               "@type": "type.googleapis.com/google.protobuf.StringValue",
               "value": "{\n  \"debug\": \"false\",\n  \"stat_prefix\": \"istio\"\n}\n"
              }
             }
            }
           }
          },
          {
           "name": "envoy.tcp_proxy", /* 配置連線上游cluster InboundPassthroughClusterIpv4時的訪問日誌，InboundPassthroughClusterIpv4 cluster用於處理基於IPv4的HTTP */
           "typed_config": {
            "@type": "type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy",
            "stat_prefix": "InboundPassthroughClusterIpv4",
            "cluster": "InboundPassthroughClusterIpv4",
            "access_log": [
             {
              "name": "envoy.file_access_log",
              "typed_config": {
               "@type": "type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog",
               "path": "/dev/stdout",
               "format": "[%START_TIME%] \"%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%\" %RESPONSE_CODE% %RESPONSE_FLAGS% \"%DYNAMIC_METADATA(istio.mixer:status)%\" \"%UPSTREAM_TRANSPORT_FAILURE_REASON%\" %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% \"%REQ(X-FORWARDED-FOR)%\" \"%REQ(USER-AGENT)%\" \"%REQ(X-REQUEST-ID)%\" \"%REQ(:AUTHORITY)%\" \"%UPSTREAM_HOST%\" %UPSTREAM_CLUSTER% %UPSTREAM_LOCAL_ADDRESS% %DOWNSTREAM_LOCAL_ADDRESS% %DOWNSTREAM_REMOTE_ADDRESS% %REQUESTED_SERVER_NAME% %ROUTE_NAME%\n"
              }
             }
            ]
           }
          }
         ],
         "transport_socket": { /* 匹配TLS的傳輸socket */
          "name": "envoy.transport_sockets.tls",
          "typed_config": {
           "@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext",
           "common_tls_context": {
            "alpn_protocols": [ /* 監聽器使用的ALPN列表 */
             "istio-peer-exchange",
             "h2",
             "http/1.1"
            ],
            "tls_certificate_sds_secret_configs": [ /* 通過SDS API獲取證書的配置 */
             {
              "name": "default",
              "sds_config": {
               "api_config_source": {
                "api_type": "GRPC",
                "grpc_services": [
                 {
                  "envoy_grpc": {
                   "cluster_name": "sds-grpc"
                  }
                 }
                ],
                "transport_api_version": "V3"
               },
               "initial_fetch_timeout": "0s",
               "resource_api_version": "V3"
              }
             }
            ],
            "combined_validation_context": {
             "default_validation_context": { /* 對對端的證書的SAN進行認證 */
              "match_subject_alt_names": [ 
               {
                "prefix": "spiffe://new-td/"
               },
               {
                "prefix": "spiffe://old-td/"
               }
              ]
             },
             "validation_context_sds_secret_config": { /* 配置通過SDS API獲取證書 */
              "name": "ROOTCA",
              "sds_config": {
               "api_config_source": {
                "api_type": "GRPC",
                "grpc_services": [
                 {
                  "envoy_grpc": {
                   "cluster_name": "sds-grpc"
                  }
                 }
                ],
                "transport_api_version": "V3"
               },
               "initial_fetch_timeout": "0s",
               "resource_api_version": "V3"
              }
             }
            }
           },
           "require_client_certificate": true
          }
         },
         "name": "virtualInbound"
        },
        /* 與上面不同的是，此處匹配不帶TLS的連線 */
        {
         "filter_chain_match": {
          "prefix_ranges": [
           {
            "address_prefix": "0.0.0.0",
            "prefix_len": 0
           }
          ]
         },
         "filters": [
          {
           "name": "istio.metadata_exchange",
            ...
          },
          {
           "name": "istio.stats",
			...
          },
          {
           "name": "envoy.tcp_proxy",
            ...
          }
         ],
         "name": "virtualInbound"
        },
        ...
        /* 應用的監聽器，監聽埠為HTTP 80埠 */
        {
         "filter_chain_match": {
          "destination_port": 80, /* 匹配的請求的目的埠 */
          "application_protocols": [ /* 匹配的ALPN，僅在使用TLS時使用 */
           "istio",
           "istio-http/1.0",
           "istio-http/1.1",
           "istio-h2"
          ]
         },
         "filters": [
          {
           "name": "istio.metadata_exchange", /* 交換Node Metadata的配置 */
            ...
          },
          {
           "name": "envoy.http_connection_manager", /* HTTP連線管理過濾器 */
           "typed_config": {
            "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager",
            "stat_prefix": "inbound_0.0.0.0_80",
            "route_config": { /* 靜態路由表 */
             "name": "inbound|80|http|sleep.default.svc.cluster.local", /* 路由配置的名稱 */
             "virtual_hosts": [ /* 構成路由表的虛擬主機列表 */
              {
               "name": "inbound|http|80", /* 構成路由表的虛擬主機名 */
               "domains": [ /* 匹配到該虛擬主機的域列表 */
                "*"
               ],
               "routes": [ /* 對入站請求的路由，將路徑為"/"的HTTP請求路由到cluster inbound|80|http|sleep.default.svc.cluster.local*/
                {
                 "match": {
                  "prefix": "/"
                 },
                 "route": {
                  "cluster": "inbound|80|http|sleep.default.svc.cluster.local",
                  "timeout": "0s",
                  "max_grpc_timeout": "0s"
                 },
                 "decorator": {
                  "operation": "sleep.default.svc.cluster.local:80/*"
                 },
                 "name": "default" /* 路由的名稱 */
                }
               ]
              }
             ],
             "validate_clusters": false
            },
            "http_filters": [ /* HTTP連線過濾器鏈 */
             {
              "name": "istio.metadata_exchange", /* 基於HTTP的Metadata的交換配置 */
			   ...
             },
             {
              "name": "istio_authn", /* istio的mTLS的預設值 */
              "typed_config": {
               "@type": "type.googleapis.com/istio.envoy.config.filter.http.authn.v2alpha1.FilterConfig",
               "policy": {
                "peers": [
                 {
                  "mtls": {
                   "mode": "PERMISSIVE"
                  }
                 }
                ]
               },
               "skip_validate_trust_domain": true
              }
             },
             {
              "name": "envoy.filters.http.cors",
              "typed_config": {
               "@type": "type.googleapis.com/envoy.extensions.filters.http.cors.v3.Cors"
              }
             },
             {
              "name": "envoy.fault",
              "typed_config": {
               "@type": "type.googleapis.com/envoy.extensions.filters.http.fault.v3.HTTPFault"
              }
             },
             {
              "name": "istio.stats", /* 基於HTTP的遙測配置 */
               ...
             },
             {
              "name": "envoy.router",
              "typed_config": {
               "@type": "type.googleapis.com/envoy.extensions.filters.http.router.v3.Router"
              }
             }
            ],
            "tracing": {
             "client_sampling": {
              "value": 100
             },
             "random_sampling": {
              "value": 1
             },
             "overall_sampling": {
              "value": 100
             }
            },
            "server_name": "istio-envoy", /* 設定訪問日誌格式 */
            "access_log": [
             {
              "name": "envoy.file_access_log",
               ...
             }
            ],
            "use_remote_address": false,
            "generate_request_id": true,
            "forward_client_cert_details": "APPEND_FORWARD",
            "set_current_client_cert_details": {
             "subject": true,
             "dns": true,
             "uri": true
            },
            "upgrade_configs": [
             {
              "upgrade_type": "websocket"
             }
            ],
            "stream_idle_timeout": "0s",
            "normalize_path": true
           }
          }
         ],
         "transport_socket": { /* TLS傳輸socket配置 */
          "name": "envoy.transport_sockets.tls",
           ...
         },
         "name": "0.0.0.0_80"
        },

Outbound listener: 下面是到Prometheus服務9092埠的outbound listener。10.84.30.227為Prometheus的k8s service地址，指定了後端的cluster outbound|9092||prometheus-k8s.openshift-monitoring.svc.cluster.local。route_config_name欄位指定了該listener使用的route prometheus-k8s.openshift-monitoring.svc.cluster.local:9092。

    {
     "name": "10.84.30.227_9092",
     "active_state": {
      "version_info": "2020-09-15T08:05:54Z/4",
      "listener": {
       "@type": "type.googleapis.com/envoy.config.listener.v3.Listener",
       "name": "10.84.30.227_9092",
       "address": {
        "socket_address": {
         "address": "10.84.30.227",
         "port_value": 9092
        }
       },
       "filter_chains": [
        {
         "filters": [
          {
           "name": "istio.stats",
           "typed_config": {
            "@type": "type.googleapis.com/udpa.type.v1.TypedStruct",
            "type_url": "type.googleapis.com/envoy.extensions.filters.network.wasm.v3.Wasm",
            ...
           }
          },
          {
           "name": "envoy.tcp_proxy",/* TCP過濾器設定，設定連線到對應cluster的日誌格式 */
           "typed_config": {
            "@type": "type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy",
            "stat_prefix": "outbound|9092||prometheus-k8s.openshift-monitoring.svc.cluster.local",
            "cluster": "outbound|9092||prometheus-k8s.openshift-monitoring.svc.cluster.local",
            "access_log": [
             ...
            ]
           }
          }
         ]
        },
        {
         "filter_chain_match": {
          "application_protocols": [
           "http/1.0",
           "http/1.1",
           "h2c"
          ]
         },
         "filters": [
          {
           "name": "envoy.http_connection_manager", /* 配置HTTP連線 */
           "typed_config": {
            "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager",
            "stat_prefix": "outbound_10.84.30.227_9092",
            "rds": { /* RDS介面配置 */
             "config_source": {
              "ads": {},
              "resource_api_version": "V3"
             },
             "route_config_name": "prometheus-k8s.openshift-monitoring.svc.cluster.local:9092" /* 指定路由配置 */
            },
            "http_filters": [
             {
              "name": "istio.metadata_exchange",
              "typed_config": {
               "@type": "type.googleapis.com/udpa.type.v1.TypedStruct",
               "type_url": "type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm",
               ...
              }
             },
             {
              "name": "istio.alpn",
              "typed_config": {
               "@type": "type.googleapis.com/istio.envoy.config.filter.http.alpn.v2alpha1.FilterConfig",
               "alpn_override": [
                {
                 "alpn_override": [
                  "istio-http/1.0",
                  "istio"
                 ]
                },
                {
                 "upstream_protocol": "HTTP11",
                 "alpn_override": [
                  "istio-http/1.1",
                  "istio"
                 ]
                },
                {
                 "upstream_protocol": "HTTP2",
                 "alpn_override": [
                  "istio-h2",
                  "istio"
                 ]
                }
               ]
              }
             },
             {
              "name": "envoy.filters.http.cors",
              "typed_config": {
               "@type": "type.googleapis.com/envoy.extensions.filters.http.cors.v3.Cors"
              }
             },
             {
              "name": "envoy.fault",
              "typed_config": {
               "@type": "type.googleapis.com/envoy.extensions.filters.http.fault.v3.HTTPFault"
              }
             },
             {
              "name": "istio.stats",
              "typed_config": {
               "@type": "type.googleapis.com/udpa.type.v1.TypedStruct",
               "type_url": "type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm",
               ...
              }
             },
             {
              "name": "envoy.router",
              "typed_config": {
               "@type": "type.googleapis.com/envoy.extensions.filters.http.router.v3.Router"
              }
             }
            ],
            "tracing": {
             ...
            },
            "access_log": [
             {
              "name": "envoy.file_access_log",
              ...
              }
             }
            ],
            "use_remote_address": false,
            "generate_request_id": true,
            "upgrade_configs": [
             {
              "upgrade_type": "websocket"
             }
            ],
            "stream_idle_timeout": "0s",
            "normalize_path": true
           }
          }
         ]
        }
       ],
       "deprecated_v1": {
        "bind_to_port": false
       },
       "listener_filters": [
        {
         "name": "envoy.listener.tls_inspector",
         "typed_config": {
          "@type": "type.googleapis.com/envoy.extensions.filters.listener.tls_inspector.v3.TlsInspector"
         }
        },
        {
         "name": "envoy.listener.http_inspector",
         "typed_config": {
          "@type": "type.googleapis.com/envoy.extensions.filters.listener.http_inspector.v3.HttpInspector"
         }
        }
       ],
       "listener_filters_timeout": "5s",
       "traffic_direction": "OUTBOUND",
       "continue_on_listener_filters_timeout": true
      },
      "last_updated": "2020-09-15T08:06:23.989Z"
     }
    },

從上面的配置可以看出，路由配置位於HttpConnectionManager型別中，因此如果某個listener沒有用到HTTP，則不會有對應的route。如下面的istiod15012埠上的服務，提供了基於gRPC協議的XDP和CA的服務(使用TLS)。

{
 "name": "10.84.251.157_15012",
 "active_state": {
  "version_info": "2020-09-16T07:48:42Z/22",
  "listener": {
   "@type": "type.googleapis.com/envoy.config.listener.v3.Listener",
   "name": "10.84.251.157_15012",
   "address": {
    "socket_address": {
     "address": "10.84.251.157",
     "port_value": 15012
    }
   },
   "filter_chains": [
    {
     "filters": [
      {
       "name": "istio.stats",
       ...
      },
      {
       "name": "envoy.tcp_proxy",
       "typed_config": {
        "@type": "type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy",
        "stat_prefix": "outbound|15012||istiod.istio-system.svc.cluster.local",
        "cluster": "outbound|15012||istiod.istio-system.svc.cluster.local",
        "access_log": [
         ...
        ]
       }
      }
     ]
    }
   ],
   "deprecated_v1": {
    "bind_to_port": false
   },
   "traffic_direction": "OUTBOUND"
  },
  "last_updated": "2020-09-16T07:49:34.134Z"
 }
},

Route：Istio的route也分為靜態配置和動態配置。靜態路由配置與靜態監聽器，以及inbound 動態監聽器中設定的靜態路由配置(envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager中的route_config)有關。

下面看一個與Prometheus 9092埠提供的服務有關的動態路由，路由配置名稱route_config.name與上面Prometheus outbound監聽器route_config_name欄位指定的值是相同的。

    {
     "version_info": "2020-09-16T07:48:42Z/22",
     "route_config": {
      "@type": "type.googleapis.com/envoy.config.route.v3.RouteConfiguration",
      "name": "prometheus-k8s.openshift-monitoring.svc.cluster.local:9092",
      "virtual_hosts": [
       {
        "name": "prometheus-k8s.openshift-monitoring.svc.cluster.local:9092",
        "domains": [
         "prometheus-k8s.openshift-monitoring.svc.cluster.local",
         "prometheus-k8s.openshift-monitoring.svc.cluster.local:9092",
         "prometheus-k8s.openshift-monitoring",
         "prometheus-k8s.openshift-monitoring:9092",
         "prometheus-k8s.openshift-monitoring.svc.cluster",
         "prometheus-k8s.openshift-monitoring.svc.cluster:9092",
         "prometheus-k8s.openshift-monitoring.svc",
         "prometheus-k8s.openshift-monitoring.svc:9092",
         "10.84.30.227",
         "10.84.30.227:9092"
        ],
        "routes": [
         {
          "match": {
           "prefix": "/"
          },
          "route": { /* 路由到的後端cluster */
           "cluster": "outbound|9092||prometheus-k8s.openshift-monitoring.svc.cluster.local",
           "timeout": "0s",
           "retry_policy": {
            "retry_on": "connect-failure,refused-stream,unavailable,cancelled,retriable-status-codes",
            "num_retries": 2,
            "retry_host_predicate": [
             {
              "name": "envoy.retry_host_predicates.previous_hosts"
             }
            ],
            "host_selection_retry_max_attempts": "5",
            "retriable_status_codes": [
             503
            ]
           },
           "max_grpc_timeout": "0s"
          },
          "decorator": {
           "operation": "prometheus-k8s.openshift-monitoring.svc.cluster.local:9092/*"
          },
          "name": "default"
         }
        ],
        "include_request_attempt_count": true
       }
      ],
      "validate_clusters": false
     },
     "last_updated": "2020-09-16T07:49:52.551Z"
    },

整個訪問流程簡單可以視為：

Inbound請求：

+----------+    +-----------------------+    +-----------------+    +----------+
| iptables +--->+ virtualInbound:15006  +--->+ Inbound Cluster +--->+ endpoint |
+----------+    +-----------------------+    +-----------------+    +----------+

出站請求：

+----------+    +-------------------------+    +-------------------+
| iptables +--->+ virtualOutbound：105001 +--->+ Outbound Listener +--+
+----------+    +-------------------------+    +-------------------+  |
                                                                      |
                                                                      |
      +---------------------------------------------------------------+
      |
      |     +-------+    +-----------------+    +----------+
      +---->+ route +--->+Outbound Cluster +--->+ endpoint |
            +-------+    +-----------------+    +----------+

更多內容可以參考Envoy的官方文件

下面是基於Istio官方BookInfo的一個訪問流程圖，可以幫助理解整個流程。

參考

Istio Ingress流量入口配置原理
2018-09-02
Istio的流量管理(概念)(istio 系列二)
2020-05-11
Istio中的服務和流量的抽象模型
2018-12-18
抽象模型
Istio的流量管理(實操二)(istio 系列四)
2020-05-25
Istio的流量管理(實操一)(istio 系列三)
2020-05-14
Istio Ingress Gateway中的Envoy配置解析
2018-11-05
Gateway
Istio的流量管理(實操三)
2020-08-28
Istio 流量劫持過程
2020-05-29
Istio 的配置分析
2020-09-07
理解 Istio Service Mesh 中 Envoy 代理 Sidecar 注入及流量劫持
2018-09-11
IDE
Istio 流量治理功能原理與實戰
2018-11-26
Istio流量治理原理之負載均衡
2019-07-08
負載
Istio技術與實踐05：如何用istio實現流量管理
2018-09-04
idou老師教你學Istio 19 : Istio 流量治理功能原理與實戰
2019-02-15
Istio流量管理實現機制深度解析
2018-10-08
淺談 istio 配置下發（下篇，istio 處理來自 k8s 的配置）
2024-10-27
K8S
idou老師教你學Istio06: 如何用istio實現流量遷移
2019-03-01
kubernetes實踐之七十：Istio之流量管理(上)
2018-08-14
基於istio實現多叢集流量治理
2024-04-11
使用Istio服務網格實現流量映象
2018-12-31
最小輕量級的Istio來了，僅使用流量治理能力
2018-09-11
在 Istio 中實現 Redis 叢集的資料分片、讀寫分離和流量映象
2020-10-22
Redis
idou教你學Istio10 : 如何用Istio實現K8S Egress流量管理
2019-03-01
K8S
idou老師教你學Istio: 如何用Istio實現K8S Egress流量管理
2018-11-22
K8S
kubernetes實踐之七十一：Istio之流量管理(下)
2018-08-14
idou老師教你學Istio 09: 如何用Istio實現K8S Ingress流量管理
2018-11-08
K8S
為新服務快速配置Istio的模板
2019-08-27
Istio實踐（2）-流量控制及服務間呼叫
2022-04-12
暴露 Istio Service Mesh 中的 Gateway
2019-03-31
Gateway
在生產中執行kubernetes上的Istio
2019-01-21
使用 Istio CNI 支援強安全 TKE Stack 叢集的服務網格流量捕獲
2022-03-09
在阿里雲容器服務上基於Istio實現東西向流量管理
2018-09-14
阿里
Istio的運維-診斷工具(istio 系列五)
2020-06-09
運維
Istio 中實現客戶端源 IP 的保持
2022-06-08
客戶端
kubernetes實踐之七十三：Istio之配置請求路由
2018-08-22
路由
在 Istio 中除錯 503 錯誤
2019-03-31
除錯
Istio
2024-03-25
Istio安全-認證(istio 系列七)
2020-07-15

Istio中的流量配置