Fedora 31 k8s kubernetes kubeasz 防火牆 firewalld 導致 harbor pod 容器 例項 網路不通 connect: connection refused
防火牆開啟狀態，harbor pod出錯
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning Unhealthy 19m (x319 over 7h28m) kubelet, 10.51.72.167 Readiness probe failed: Get http://172.20.0.175:8080/api/v2.0/ping: dial tcp 172.20.0.175:8080: connect: connection refused
Warning BackOff 4m4s (x1728 over 7h27m) kubelet, 10.51.72.167 Back-off restarting failed container
關閉防火牆
sudo systemctl stop firewalld等待一段時間，harbor恢復正常
$ kubectl get pods -n harbor -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
harbor-harbor-chartmuseum-556767f8dd-ccjfx 1/1 Running 51 59d 172.20.0.170 10.51.72.167 <none> <none>
harbor-harbor-clair-7d796cbd8b-mh26g 2/2 Running 1731 58d 172.20.0.178 10.51.72.167 <none> <none>
harbor-harbor-core-7d5d7588bb-f6vh5 1/1 Running 1452 59d 172.20.0.175 10.51.72.167 <none> <none>
harbor-harbor-database-0 1/1 Running 51 59d 172.20.0.172 10.51.72.167 <none> <none>
harbor-harbor-jobservice-57c4478bb9-4qv9t 1/1 Running 1196 59d 172.20.0.174 10.51.72.167 <none> <none>
harbor-harbor-notary-server-c45bb4b7b-khpq5 1/1 Running 1601 126d 172.20.0.177 10.51.72.167 <none> <none>
harbor-harbor-notary-signer-5b5576db77-s7bjs 1/1 Running 1600 126d 172.20.0.167 10.51.72.167 <none> <none>
harbor-harbor-portal-b8c64dcf6-qk8h8 1/1 Running 51 59d 172.20.0.181 10.51.72.167 <none> <none>
harbor-harbor-redis-0 1/1 Running 51 59d 172.20.0.182 10.51.72.167 <none> <none>
harbor-harbor-registry-6fc4b895cf-k2cqf 2/2 Running 102 59d 172.20.0.176 10.51.72.167 <none> <none>
harbor-harbor-trivy-0 1/1 Running 51 59d 172.20.0.169 10.51.72.167 <none> <none>
分析：
禁用防火牆，重啟
sudo systemctl disable firewalld
rebootharbor故障
[yeqiang@harbor ~]$ kubectl get pods -n harbor -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
harbor-harbor-chartmuseum-556767f8dd-ccjfx 1/1 Running 52 59d 172.20.0.201 10.51.72.167 <none> <none>
harbor-harbor-clair-7d796cbd8b-mh26g 1/2 CrashLoopBackOff 1738 58d 172.20.0.187 10.51.72.167 <none> <none>
harbor-harbor-core-7d5d7588bb-f6vh5 0/1 CrashLoopBackOff 1457 59d 172.20.0.188 10.51.72.167 <none> <none>
harbor-harbor-database-0 1/1 Running 52 59d 172.20.0.196 10.51.72.167 <none> <none>
harbor-harbor-jobservice-57c4478bb9-4qv9t 0/1 Running 1200 59d 172.20.0.191 10.51.72.167 <none> <none>
harbor-harbor-notary-server-c45bb4b7b-khpq5 0/1 CrashLoopBackOff 1607 126d 172.20.0.186 10.51.72.167 <none> <none>
harbor-harbor-notary-signer-5b5576db77-s7bjs 0/1 CrashLoopBackOff 1606 126d 172.20.0.192 10.51.72.167 <none> <none>
harbor-harbor-portal-b8c64dcf6-qk8h8 1/1 Running 52 59d 172.20.0.190 10.51.72.167 <none> <none>
harbor-harbor-redis-0 1/1 Running 52 59d 172.20.0.185 10.51.72.167 <none> <none>
harbor-harbor-registry-6fc4b895cf-k2cqf 2/2 Running 104 59d 172.20.0.202 10.51.72.167 <none> <none>
harbor-harbor-trivy-0 1/1 Running 52 59d 172.20.0.197 10.51.72.167 <none> <none>
檢視故障
[yeqiang@harbor ~]$ kubectl describe pod -n harbor harbor-harbor-core-7d5d7588bb-f6vh5
Name: harbor-harbor-core-7d5d7588bb-f6vh5
Namespace: harbor
Priority: 0
Node: 10.51.72.167/10.51.72.167
Start Time: Tue, 27 Oct 2020 08:53:03 +0800
Labels: app=harbor
component=core
pod-template-hash=7d5d7588bb
release=harbor
Annotations: checksum/configmap: 3e352575d9b0f3eafcd9910f11194507b7605186b387f6a586fe0378e357e944
checksum/secret: 1d810e908cbf53045d899e09f3610777d0875f4cbf737c8457a4c426bc4c96a4
checksum/secret-jobservice: a9746a4cd6c2a3e3fb4a4d4c19beaf93a2652c07692641e8a5a4d90bfdf1ace0
Status: Running
IP: 172.20.0.188
IPs:
IP: 172.20.0.188
Controlled By: ReplicaSet/harbor-harbor-core-7d5d7588bb
Containers:
core:
Container ID: docker://d015706c507ac823bb936bdcd58bf2529c1f015c200d78e087f48d6570322a89
Image: r6w9c7qa.mirror.aliyuncs.com/goharbor/harbor-core:v2.0.0
Image ID: docker-pullable://r6w9c7qa.mirror.aliyuncs.com/goharbor/harbor-core@sha256:75b5900d2335c87ca34ee80e64e2a6b56cbe218aceaef8614efc31594ad84d38
Port: 8080/TCP
Host Port: 0/TCP
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 1
Started: Fri, 25 Dec 2020 15:57:32 +0800
Finished: Fri, 25 Dec 2020 15:58:34 +0800
Ready: False
Restart Count: 1457
Liveness: http-get http://:8080/api/v2.0/ping delay=300s timeout=1s period=10s #success=1 #failure=3
Readiness: http-get http://:8080/api/v2.0/ping delay=20s timeout=1s period=10s #success=1 #failure=3
Environment Variables from:
harbor-harbor-core ConfigMap Optional: false
harbor-harbor-core Secret Optional: false
Environment:
CORE_SECRET: <set to the key 'secret' in secret 'harbor-harbor-core'> Optional: false
JOBSERVICE_SECRET: <set to the key 'JOBSERVICE_SECRET' in secret 'harbor-harbor-jobservice'> Optional: false
Mounts:
/etc/core/app.conf from config (rw,path="app.conf")
/etc/core/ca from ca-download (rw)
/etc/core/key from secret-key (rw,path="key")
/etc/core/private_key.pem from token-service-private-key (rw,path="tls.key")
/etc/core/token from psc (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-lqv24 (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: harbor-harbor-core
Optional: false
secret-key:
Type: Secret (a volume populated by a Secret)
SecretName: harbor-harbor-core
Optional: false
token-service-private-key:
Type: Secret (a volume populated by a Secret)
SecretName: harbor-harbor-core
Optional: false
ca-download:
Type: Secret (a volume populated by a Secret)
SecretName: hknaruto.com
Optional: false
psc:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
default-token-lqv24:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-lqv24
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning Unhealthy 35m (x319 over 7h44m) kubelet, 10.51.72.167 Readiness probe failed: Get http://172.20.0.175:8080/api/v2.0/ping: dial tcp 172.20.0.175:8080: connect: connection refused
Warning BackOff 20m (x1728 over 7h44m) kubelet, 10.51.72.167 Back-off restarting failed container
Warning FailedCreatePodSandBox 9m43s kubelet, 10.51.72.167 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "8d4a1398862a5a2af909879f58e14fcd96702a53e032b79cf0bc37561285511e" network for pod "harbor-harbor-core-7d5d7588bb-f6vh5": networkPlugin cni failed to set up pod "harbor-harbor-core-7d5d7588bb-f6vh5_harbor" network: open /run/flannel/subnet.env: no such file or directory
Warning FailedCreatePodSandBox 9m43s kubelet, 10.51.72.167 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "f1d1b91d8b9142b7b3da3a7406b07c716f0ef75902243951f518a892ee26a803" network for pod "harbor-harbor-core-7d5d7588bb-f6vh5": networkPlugin cni failed to set up pod "harbor-harbor-core-7d5d7588bb-f6vh5_harbor" network: open /run/flannel/subnet.env: no such file or directory
Normal SandboxChanged 9m41s (x4 over 9m44s) kubelet, 10.51.72.167 Pod sandbox changed, it will be killed and re-created.
Warning FailedCreatePodSandBox 9m41s kubelet, 10.51.72.167 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "3933a5273489a38d8832a9813972088ae18f570a68a5a4cfd106c844314252e0" network for pod "harbor-harbor-core-7d5d7588bb-f6vh5": networkPlugin cni failed to set up pod "harbor-harbor-core-7d5d7588bb-f6vh5_harbor" network: open /run/flannel/subnet.env: no such file or directory
Warning BackOff 8m31s (x2 over 8m37s) kubelet, 10.51.72.167 Back-off restarting failed container
Normal Pulled 8m19s (x2 over 9m40s) kubelet, 10.51.72.167 Container image "r6w9c7qa.mirror.aliyuncs.com/goharbor/harbor-core:v2.0.0" already present on machine
Normal Created 8m19s (x2 over 9m40s) kubelet, 10.51.72.167 Created container core
Normal Started 8m19s (x2 over 9m40s) kubelet, 10.51.72.167 Started container core
Warning Unhealthy 4m37s (x16 over 9m17s) kubelet, 10.51.72.167 Readiness probe failed: Get http://172.20.0.188:8080/api/v2.0/ping: dial tcp 172.20.0.188:8080: connect: connection refused
當前iptables狀態
[yeqiang@harbor startup_firewalld_off]$ sudo iptables -L -n
Chain INPUT (policy ACCEPT)
target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:22
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
Chain FORWARD (policy ACCEPT)
target prot opt source destination
KUBE-FORWARD all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules */
DOCKER-USER all -- 0.0.0.0/0 0.0.0.0/0
DOCKER-ISOLATION-STAGE-1 all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
DOCKER all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
ACCEPT all -- 172.20.0.0/16 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 172.20.0.0/16
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
Chain DOCKER (1 references)
target prot opt source destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target prot opt source destination
DOCKER-ISOLATION-STAGE-2 all -- 0.0.0.0/0 0.0.0.0/0
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-USER (1 references)
target prot opt source destination
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-FIREWALL (2 references)
target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes firewall for dropping marked packets */ mark match 0x8000/0x8000
Chain KUBE-FORWARD (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules */ mark match 0x4000/0x4000
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding conntrack pod source rule */ ctstate RELATED,ESTABLISHED
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding conntrack pod destination rule */ ctstate RELATED,ESTABLISHED
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination [yeqiang@harbor startup_firewalld_off]$ sudo iptables -L -n -t nat
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
DOCKER all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
DOCKER all -- 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
KUBE-POSTROUTING all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes postrouting rules */
MASQUERADE all -- 172.17.0.0/16 0.0.0.0/0
RETURN all -- 172.20.0.0/16 172.20.0.0/16
MASQUERADE all -- 172.20.0.0/16 !224.0.0.0/4 random-fully
RETURN all -- !172.20.0.0/16 172.20.0.0/24
MASQUERADE all -- !172.20.0.0/16 172.20.0.0/16 random-fully
Chain DOCKER (2 references)
target prot opt source destination
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-FIREWALL (0 references)
target prot opt source destination
KUBE-MARK-DROP all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
Chain KUBE-LOAD-BALANCER (0 references)
target prot opt source destination
KUBE-MARK-MASQ all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-MARK-DROP (1 references)
target prot opt source destination
Chain KUBE-MARK-MASQ (3 references)
target prot opt source destination
MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK or 0x4000
Chain KUBE-NODE-PORT (1 references)
target prot opt source destination
KUBE-MARK-MASQ tcp -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes nodeport TCP port for masquerade purpose */ match-set KUBE-NODE-PORT-TCP dst
Chain KUBE-POSTROUTING (1 references)
target prot opt source destination
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service traffic requiring SNAT */ mark match 0x4000/0x4000 random-fully
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes endpoints dst ip:port, source ip for solving hairpin purpose */ match-set KUBE-LOOP-BACK dst,dst,src
Chain KUBE-SERVICES (2 references)
target prot opt source destination
KUBE-MARK-MASQ all -- !172.20.0.0/16 0.0.0.0/0 /* Kubernetes service cluster ip + port for masquerade purpose */ match-set KUBE-CLUSTER-IP dst,dst
KUBE-NODE-PORT all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-CLUSTER-IP dst,dst
[yeqiang@harbor startup_firewalld_off]$ sudo iptables -L -n -t mangle
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
[yeqiang@harbor startup_firewalld_off]$ sudo iptables -L -n -t raw
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination [yeqiang@harbor startup_firewalld_off]$ sudo iptables -L -n -t security
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination 啟用防火牆，重啟
[yeqiang@harbor startup_firewalld_off]$ sudo systemctl enable firewalld
Created symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service → /usr/lib/systemd/system/firewalld.service.
Created symlink /etc/systemd/system/multi-user.target.wants/firewalld.service → /usr/lib/systemd/system/firewalld.service.
[yeqiang@harbor startup_firewalld_off]$ reboot
檢查harbor狀態
[yeqiang@harbor ~]$ kubectl get pods -n harbor -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
harbor-harbor-chartmuseum-556767f8dd-ccjfx 1/1 Running 53 59d 172.20.0.222 10.51.72.167 <none> <none>
harbor-harbor-clair-7d796cbd8b-mh26g 1/2 CrashLoopBackOff 1742 58d 172.20.0.221 10.51.72.167 <none> <none>
harbor-harbor-core-7d5d7588bb-f6vh5 0/1 Error 1459 59d 172.20.0.208 10.51.72.167 <none> <none>
harbor-harbor-database-0 1/1 Running 53 59d 172.20.0.205 10.51.72.167 <none> <none>
harbor-harbor-jobservice-57c4478bb9-4qv9t 0/1 Running 1202 59d 172.20.0.210 10.51.72.167 <none> <none>
harbor-harbor-notary-server-c45bb4b7b-khpq5 1/1 Running 1609 126d 172.20.0.214 10.51.72.167 <none> <none>
harbor-harbor-notary-signer-5b5576db77-s7bjs 1/1 Running 1608 126d 172.20.0.212 10.51.72.167 <none> <none>
harbor-harbor-portal-b8c64dcf6-qk8h8 1/1 Running 53 59d 172.20.0.215 10.51.72.167 <none> <none>
harbor-harbor-redis-0 1/1 Running 53 59d 172.20.0.219 10.51.72.167 <none> <none>
harbor-harbor-registry-6fc4b895cf-k2cqf 2/2 Running 106 59d 172.20.0.218 10.51.72.167 <none> <none>
harbor-harbor-trivy-0 1/1 Running 53 59d 172.20.0.211 10.51.72.167 <none> <none>
harbor-core錯誤
[yeqiang@harbor ~]$ kubectl describe pod -n harbor harbor-harbor-core-7d5d7588bb-f6vh5
Name: harbor-harbor-core-7d5d7588bb-f6vh5
Namespace: harbor
Priority: 0
Node: 10.51.72.167/10.51.72.167
Start Time: Tue, 27 Oct 2020 08:53:03 +0800
Labels: app=harbor
component=core
pod-template-hash=7d5d7588bb
release=harbor
Annotations: checksum/configmap: 3e352575d9b0f3eafcd9910f11194507b7605186b387f6a586fe0378e357e944
checksum/secret: 1d810e908cbf53045d899e09f3610777d0875f4cbf737c8457a4c426bc4c96a4
checksum/secret-jobservice: a9746a4cd6c2a3e3fb4a4d4c19beaf93a2652c07692641e8a5a4d90bfdf1ace0
Status: Running
IP: 172.20.0.208
IPs:
IP: 172.20.0.208
Controlled By: ReplicaSet/harbor-harbor-core-7d5d7588bb
Containers:
core:
Container ID: docker://8057d34c9f708d2000e4cfc4449e4be49a7a73417aba76750145d6cc24149260
Image: r6w9c7qa.mirror.aliyuncs.com/goharbor/harbor-core:v2.0.0
Image ID: docker-pullable://r6w9c7qa.mirror.aliyuncs.com/goharbor/harbor-core@sha256:75b5900d2335c87ca34ee80e64e2a6b56cbe218aceaef8614efc31594ad84d38
Port: 8080/TCP
Host Port: 0/TCP
State: Running
Started: Fri, 25 Dec 2020 16:06:15 +0800
Last State: Terminated
Reason: Error
Exit Code: 1
Started: Fri, 25 Dec 2020 16:04:52 +0800
Finished: Fri, 25 Dec 2020 16:05:54 +0800
Ready: False
Restart Count: 1460
Liveness: http-get http://:8080/api/v2.0/ping delay=300s timeout=1s period=10s #success=1 #failure=3
Readiness: http-get http://:8080/api/v2.0/ping delay=20s timeout=1s period=10s #success=1 #failure=3
Environment Variables from:
harbor-harbor-core ConfigMap Optional: false
harbor-harbor-core Secret Optional: false
Environment:
CORE_SECRET: <set to the key 'secret' in secret 'harbor-harbor-core'> Optional: false
JOBSERVICE_SECRET: <set to the key 'JOBSERVICE_SECRET' in secret 'harbor-harbor-jobservice'> Optional: false
Mounts:
/etc/core/app.conf from config (rw,path="app.conf")
/etc/core/ca from ca-download (rw)
/etc/core/key from secret-key (rw,path="key")
/etc/core/private_key.pem from token-service-private-key (rw,path="tls.key")
/etc/core/token from psc (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-lqv24 (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: harbor-harbor-core
Optional: false
secret-key:
Type: Secret (a volume populated by a Secret)
SecretName: harbor-harbor-core
Optional: false
token-service-private-key:
Type: Secret (a volume populated by a Secret)
SecretName: harbor-harbor-core
Optional: false
ca-download:
Type: Secret (a volume populated by a Secret)
SecretName: hknaruto.com
Optional: false
psc:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
default-token-lqv24:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-lqv24
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning Unhealthy 41m (x319 over 7h50m) kubelet, 10.51.72.167 Readiness probe failed: Get http://172.20.0.175:8080/api/v2.0/ping: dial tcp 172.20.0.175:8080: connect: connection refused
Warning BackOff 26m (x1728 over 7h50m) kubelet, 10.51.72.167 Back-off restarting failed container
Warning FailedCreatePodSandBox 15m kubelet, 10.51.72.167 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "8d4a1398862a5a2af909879f58e14fcd96702a53e032b79cf0bc37561285511e" network for pod "harbor-harbor-core-7d5d7588bb-f6vh5": networkPlugin cni failed to set up pod "harbor-harbor-core-7d5d7588bb-f6vh5_harbor" network: open /run/flannel/subnet.env: no such file or directory
Warning FailedCreatePodSandBox 15m kubelet, 10.51.72.167 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "f1d1b91d8b9142b7b3da3a7406b07c716f0ef75902243951f518a892ee26a803" network for pod "harbor-harbor-core-7d5d7588bb-f6vh5": networkPlugin cni failed to set up pod "harbor-harbor-core-7d5d7588bb-f6vh5_harbor" network: open /run/flannel/subnet.env: no such file or directory
Normal SandboxChanged 15m (x4 over 15m) kubelet, 10.51.72.167 Pod sandbox changed, it will be killed and re-created.
Warning FailedCreatePodSandBox 15m kubelet, 10.51.72.167 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "3933a5273489a38d8832a9813972088ae18f570a68a5a4cfd106c844314252e0" network for pod "harbor-harbor-core-7d5d7588bb-f6vh5": networkPlugin cni failed to set up pod "harbor-harbor-core-7d5d7588bb-f6vh5_harbor" network: open /run/flannel/subnet.env: no such file or directory
Normal Pulled 14m (x2 over 15m) kubelet, 10.51.72.167 Container image "r6w9c7qa.mirror.aliyuncs.com/goharbor/harbor-core:v2.0.0" already present on machine
Normal Created 14m (x2 over 15m) kubelet, 10.51.72.167 Created container core
Normal Started 14m (x2 over 15m) kubelet, 10.51.72.167 Started container core
Warning Unhealthy 10m (x16 over 15m) kubelet, 10.51.72.167 Readiness probe failed: Get http://172.20.0.188:8080/api/v2.0/ping: dial tcp 172.20.0.188:8080: connect: connection refused
Warning BackOff 5m33s (x27 over 14m) kubelet, 10.51.72.167 Back-off restarting failed container
Warning FailedCreatePodSandBox 90s kubelet, 10.51.72.167 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "1729d36f31a0d7a96f92e457e34ec2da90162d3db0835c5b0719f06867da7a9e" network for pod "harbor-harbor-core-7d5d7588bb-f6vh5": networkPlugin cni failed to set up pod "harbor-harbor-core-7d5d7588bb-f6vh5_harbor" network: open /run/flannel/subnet.env: no such file or directory
Warning FailedCreatePodSandBox 89s kubelet, 10.51.72.167 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "bb5669a2e842d59dccb9e71647a37365a09459980834a9f356dad0c1561608ea" network for pod "harbor-harbor-core-7d5d7588bb-f6vh5": networkPlugin cni failed to set up pod "harbor-harbor-core-7d5d7588bb-f6vh5_harbor" network: open /run/flannel/subnet.env: no such file or directory
Normal SandboxChanged 88s (x4 over 91s) kubelet, 10.51.72.167 Pod sandbox changed, it will be killed and re-created.
Warning FailedCreatePodSandBox 88s kubelet, 10.51.72.167 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "df5c38f5ba29f3601cc5fec1df73803ebbbbe53905bed96d75119dd2828af45a" network for pod "harbor-harbor-core-7d5d7588bb-f6vh5": networkPlugin cni failed to set up pod "harbor-harbor-core-7d5d7588bb-f6vh5_harbor" network: open /run/flannel/subnet.env: no such file or directory
Warning Unhealthy 27s (x4 over 57s) kubelet, 10.51.72.167 Readiness probe failed: Get http://172.20.0.208:8080/api/v2.0/ping: dial tcp 172.20.0.208:8080: connect: connection refused
Warning BackOff 15s (x2 over 23s) kubelet, 10.51.72.167 Back-off restarting failed container
Normal Pulled 3s (x2 over 87s) kubelet, 10.51.72.167 Container image "r6w9c7qa.mirror.aliyuncs.com/goharbor/harbor-core:v2.0.0" already present on machine
Normal Created 3s (x2 over 86s) kubelet, 10.51.72.167 Created container core
Normal Started 3s (x2 over 86s) kubelet, 10.51.72.167 Started container core
iptables狀態
[yeqiang@harbor startup_firewalld_on]$ sudo iptables -L -n
Chain INPUT (policy ACCEPT)
target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED,DNAT
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
INPUT_direct all -- 0.0.0.0/0 0.0.0.0/0
INPUT_ZONES all -- 0.0.0.0/0 0.0.0.0/0
DROP all -- 0.0.0.0/0 0.0.0.0/0 ctstate INVALID
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
Chain FORWARD (policy ACCEPT)
target prot opt source destination
KUBE-FORWARD all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules */
DOCKER-USER all -- 0.0.0.0/0 0.0.0.0/0
DOCKER-ISOLATION-STAGE-1 all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
DOCKER all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED,DNAT
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
FORWARD_direct all -- 0.0.0.0/0 0.0.0.0/0
FORWARD_IN_ZONES all -- 0.0.0.0/0 0.0.0.0/0
FORWARD_OUT_ZONES all -- 0.0.0.0/0 0.0.0.0/0
DROP all -- 0.0.0.0/0 0.0.0.0/0 ctstate INVALID
REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
ACCEPT all -- 172.20.0.0/16 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 172.20.0.0/16
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0
Chain DOCKER (1 references)
target prot opt source destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target prot opt source destination
DOCKER-ISOLATION-STAGE-2 all -- 0.0.0.0/0 0.0.0.0/0
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-USER (1 references)
target prot opt source destination
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain FORWARD_IN_ZONES (1 references)
target prot opt source destination
FWDI_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
FWDI_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
Chain FORWARD_OUT_ZONES (1 references)
target prot opt source destination
FWDO_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
FWDO_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
Chain FORWARD_direct (1 references)
target prot opt source destination
Chain FWDI_FedoraWorkstation (2 references)
target prot opt source destination
FWDI_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0.0.0/0
FWDI_FedoraWorkstation_log all -- 0.0.0.0/0 0.0.0.0/0
FWDI_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0.0.0/0
FWDI_FedoraWorkstation_allow all -- 0.0.0.0/0 0.0.0.0/0
FWDI_FedoraWorkstation_post all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0
Chain FWDI_FedoraWorkstation_allow (1 references)
target prot opt source destination
Chain FWDI_FedoraWorkstation_deny (1 references)
target prot opt source destination
Chain FWDI_FedoraWorkstation_log (1 references)
target prot opt source destination
Chain FWDI_FedoraWorkstation_post (1 references)
target prot opt source destination
Chain FWDI_FedoraWorkstation_pre (1 references)
target prot opt source destination
Chain FWDO_FedoraWorkstation (2 references)
target prot opt source destination
FWDO_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0.0.0/0
FWDO_FedoraWorkstation_log all -- 0.0.0.0/0 0.0.0.0/0
FWDO_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0.0.0/0
FWDO_FedoraWorkstation_allow all -- 0.0.0.0/0 0.0.0.0/0
FWDO_FedoraWorkstation_post all -- 0.0.0.0/0 0.0.0.0/0
Chain FWDO_FedoraWorkstation_allow (1 references)
target prot opt source destination
Chain FWDO_FedoraWorkstation_deny (1 references)
target prot opt source destination
Chain FWDO_FedoraWorkstation_log (1 references)
target prot opt source destination
Chain FWDO_FedoraWorkstation_post (1 references)
target prot opt source destination
Chain FWDO_FedoraWorkstation_pre (1 references)
target prot opt source destination
Chain INPUT_ZONES (1 references)
target prot opt source destination
IN_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
IN_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
Chain INPUT_direct (1 references)
target prot opt source destination
Chain IN_FedoraWorkstation (2 references)
target prot opt source destination
IN_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0.0.0/0
IN_FedoraWorkstation_log all -- 0.0.0.0/0 0.0.0.0/0
IN_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0.0.0/0
IN_FedoraWorkstation_allow all -- 0.0.0.0/0 0.0.0.0/0
IN_FedoraWorkstation_post all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0
Chain IN_FedoraWorkstation_allow (1 references)
target prot opt source destination
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:111 ctstate NEW,UNTRACKED
Chain IN_FedoraWorkstation_deny (1 references)
target prot opt source destination
Chain IN_FedoraWorkstation_log (1 references)
target prot opt source destination
Chain IN_FedoraWorkstation_post (1 references)
target prot opt source destination
Chain IN_FedoraWorkstation_pre (1 references)
target prot opt source destination
Chain KUBE-FIREWALL (2 references)
target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes firewall for dropping marked packets */ mark match 0x8000/0x8000
Chain KUBE-FORWARD (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules */ mark match 0x4000/0x4000
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding conntrack pod source rule */ ctstate RELATED,ESTABLISHED
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding conntrack pod destination rule */ ctstate RELATED,ESTABLISHED
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
Chain OUTPUT_direct (1 references)
target prot opt source destination
[yeqiang@harbor startup_firewalld_on]$ sudo iptables -L -n -t nat
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
PREROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0
PREROUTING_ZONES all -- 0.0.0.0/0 0.0.0.0/0
DOCKER all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0
DOCKER all -- 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
KUBE-POSTROUTING all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes postrouting rules */
MASQUERADE all -- 172.17.0.0/16 0.0.0.0/0
POSTROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0
POSTROUTING_ZONES all -- 0.0.0.0/0 0.0.0.0/0
RETURN all -- 172.20.0.0/16 172.20.0.0/16
MASQUERADE all -- 172.20.0.0/16 !224.0.0.0/4 random-fully
RETURN all -- !172.20.0.0/16 172.20.0.0/24
MASQUERADE all -- !172.20.0.0/16 172.20.0.0/16 random-fully
Chain DOCKER (2 references)
target prot opt source destination
RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-FIREWALL (0 references)
target prot opt source destination
KUBE-MARK-DROP all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
Chain KUBE-LOAD-BALANCER (0 references)
target prot opt source destination
KUBE-MARK-MASQ all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-MARK-DROP (1 references)
target prot opt source destination
Chain KUBE-MARK-MASQ (3 references)
target prot opt source destination
MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK or 0x4000
Chain KUBE-NODE-PORT (1 references)
target prot opt source destination
KUBE-MARK-MASQ tcp -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes nodeport TCP port for masquerade purpose */ match-set KUBE-NODE-PORT-TCP dst
Chain KUBE-POSTROUTING (1 references)
target prot opt source destination
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service traffic requiring SNAT */ mark match 0x4000/0x4000 random-fully
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes endpoints dst ip:port, source ip for solving hairpin purpose */ match-set KUBE-LOOP-BACK dst,dst,src
Chain KUBE-SERVICES (2 references)
target prot opt source destination
KUBE-MARK-MASQ all -- !172.20.0.0/16 0.0.0.0/0 /* Kubernetes service cluster ip + port for masquerade purpose */ match-set KUBE-CLUSTER-IP dst,dst
KUBE-NODE-PORT all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-CLUSTER-IP dst,dst
Chain OUTPUT_direct (1 references)
target prot opt source destination
Chain POSTROUTING_ZONES (1 references)
target prot opt source destination
POST_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
POST_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
Chain POSTROUTING_direct (1 references)
target prot opt source destination
Chain POST_FedoraWorkstation (2 references)
target prot opt source destination
POST_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0.0.0/0
POST_FedoraWorkstation_log all -- 0.0.0.0/0 0.0.0.0/0
POST_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0.0.0/0
POST_FedoraWorkstation_allow all -- 0.0.0.0/0 0.0.0.0/0
POST_FedoraWorkstation_post all -- 0.0.0.0/0 0.0.0.0/0
Chain POST_FedoraWorkstation_allow (1 references)
target prot opt source destination
Chain POST_FedoraWorkstation_deny (1 references)
target prot opt source destination
Chain POST_FedoraWorkstation_log (1 references)
target prot opt source destination
Chain POST_FedoraWorkstation_post (1 references)
target prot opt source destination
Chain POST_FedoraWorkstation_pre (1 references)
target prot opt source destination
Chain PREROUTING_ZONES (1 references)
target prot opt source destination
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
Chain PREROUTING_direct (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation (2 references)
target prot opt source destination
PRE_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_log all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_allow all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_post all -- 0.0.0.0/0 0.0.0.0/0
Chain PRE_FedoraWorkstation_allow (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_deny (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_log (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_post (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_pre (1 references)
target prot opt source destination
[yeqiang@harbor startup_firewalld_on]$ sudo iptables -L -n -t mangle
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
PREROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0
PREROUTING_ZONES all -- 0.0.0.0/0 0.0.0.0/0
Chain INPUT (policy ACCEPT)
target prot opt source destination
INPUT_direct all -- 0.0.0.0/0 0.0.0.0/0
Chain FORWARD (policy ACCEPT)
target prot opt source destination
FORWARD_direct all -- 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
POSTROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0
Chain FORWARD_direct (1 references)
target prot opt source destination
Chain INPUT_direct (1 references)
target prot opt source destination
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
Chain OUTPUT_direct (1 references)
target prot opt source destination
Chain POSTROUTING_direct (1 references)
target prot opt source destination
Chain PREROUTING_ZONES (1 references)
target prot opt source destination
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
Chain PREROUTING_direct (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation (2 references)
target prot opt source destination
PRE_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_log all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_allow all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_post all -- 0.0.0.0/0 0.0.0.0/0
Chain PRE_FedoraWorkstation_allow (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_deny (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_log (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_post (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_pre (1 references)
target prot opt source destination
[yeqiang@harbor startup_firewalld_on]$ sudo iptables -L -n -t raw
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
PREROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0
PREROUTING_ZONES all -- 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT_direct (1 references)
target prot opt source destination
Chain PREROUTING_ZONES (1 references)
target prot opt source destination
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
Chain PREROUTING_direct (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation (2 references)
target prot opt source destination
PRE_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_log all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_allow all -- 0.0.0.0/0 0.0.0.0/0
PRE_FedoraWorkstation_post all -- 0.0.0.0/0 0.0.0.0/0
Chain PRE_FedoraWorkstation_allow (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_deny (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_log (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_post (1 references)
target prot opt source destination
Chain PRE_FedoraWorkstation_pre (1 references)
target prot opt source destination
[yeqiang@harbor startup_firewalld_on]$ sudo iptables -L -n -t security
Chain INPUT (policy ACCEPT)
target prot opt source destination
INPUT_direct all -- 0.0.0.0/0 0.0.0.0/0
Chain FORWARD (policy ACCEPT)
target prot opt source destination
FORWARD_direct all -- 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0
Chain FORWARD_direct (1 references)
target prot opt source destination
Chain INPUT_direct (1 references)
target prot opt source destination
Chain OUTPUT_direct (1 references)
target prot opt source destination
此時，關閉防火牆
[yeqiang@harbor startup_firewalld_on]$ sudo systemctl stop firewalld等待一段時間，故障小時
[yeqiang@harbor ~]$ kubectl get pod -n harbor -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
harbor-harbor-chartmuseum-556767f8dd-ccjfx 1/1 Running 53 59d 172.20.0.222 10.51.72.167 <none> <none>
harbor-harbor-clair-7d796cbd8b-mh26g 1/2 Running 1747 58d 172.20.0.221 10.51.72.167 <none> <none>
harbor-harbor-core-7d5d7588bb-f6vh5 1/1 Running 1464 59d 172.20.0.208 10.51.72.167 <none> <none>
harbor-harbor-database-0 1/1 Running 53 59d 172.20.0.205 10.51.72.167 <none> <none>
harbor-harbor-jobservice-57c4478bb9-4qv9t 1/1 Running 1206 59d 172.20.0.210 10.51.72.167 <none> <none>
harbor-harbor-notary-server-c45bb4b7b-khpq5 1/1 Running 1613 126d 172.20.0.214 10.51.72.167 <none> <none>
harbor-harbor-notary-signer-5b5576db77-s7bjs 1/1 Running 1612 126d 172.20.0.212 10.51.72.167 <none> <none>
harbor-harbor-portal-b8c64dcf6-qk8h8 1/1 Running 53 59d 172.20.0.215 10.51.72.167 <none> <none>
harbor-harbor-redis-0 1/1 Running 53 59d 172.20.0.219 10.51.72.167 <none> <none>
harbor-harbor-registry-6fc4b895cf-k2cqf 2/2 Running 106 59d 172.20.0.218 10.51.72.167 <none> <none>
harbor-harbor-trivy-0 1/1 Running 53 59d 172.20.0.211 10.51.72.167 <none> <none>
此時iptables狀態
[yeqiang@harbor startup_firewalld_on2off]$ sudo iptables -L -n
[sudo] password for yeqiang:
Chain INPUT (policy ACCEPT)
target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
Chain FORWARD (policy ACCEPT)
target prot opt source destination
KUBE-FORWARD all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules */
ACCEPT all -- 172.20.0.0/16 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 172.20.0.0/16
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-FIREWALL (2 references)
target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes firewall for dropping marked packets */ mark match 0x8000/0x8000
Chain KUBE-FORWARD (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules */ mark match 0x4000/0x4000
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding conntrack pod source rule */ ctstate RELATED,ESTABLISHED
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding conntrack pod destination rule */ ctstate RELATED,ESTABLISHED
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
[yeqiang@harbor startup_firewalld_on2off]$ sudo iptables -L -n
Chain INPUT (policy ACCEPT)
target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
Chain FORWARD (policy ACCEPT)
target prot opt source destination
KUBE-FORWARD all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules */
ACCEPT all -- 172.20.0.0/16 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 172.20.0.0/16
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-FIREWALL (2 references)
target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes firewall for dropping marked packets */ mark match 0x8000/0x8000
Chain KUBE-FORWARD (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules */ mark match 0x4000/0x4000
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding conntrack pod source rule */ ctstate RELATED,ESTABLISHED
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding conntrack pod destination rule */ ctstate RELATED,ESTABLISHED
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
[yeqiang@harbor startup_firewalld_on2off]$ sudo iptables -L -n -t nat
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
KUBE-POSTROUTING all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes postrouting rules */
RETURN all -- 172.20.0.0/16 172.20.0.0/16
MASQUERADE all -- 172.20.0.0/16 !224.0.0.0/4 random-fully
RETURN all -- !172.20.0.0/16 172.20.0.0/24
MASQUERADE all -- !172.20.0.0/16 172.20.0.0/16 random-fully
Chain KUBE-FIREWALL (0 references)
target prot opt source destination
KUBE-MARK-DROP all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
Chain KUBE-LOAD-BALANCER (0 references)
target prot opt source destination
KUBE-MARK-MASQ all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-MARK-DROP (1 references)
target prot opt source destination
Chain KUBE-MARK-MASQ (3 references)
target prot opt source destination
MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK or 0x4000
Chain KUBE-NODE-PORT (1 references)
target prot opt source destination
KUBE-MARK-MASQ tcp -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes nodeport TCP port for masquerade purpose */ match-set KUBE-NODE-PORT-TCP dst
Chain KUBE-POSTROUTING (1 references)
target prot opt source destination
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service traffic requiring SNAT */ mark match 0x4000/0x4000 random-fully
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes endpoints dst ip:port, source ip for solving hairpin purpose */ match-set KUBE-LOOP-BACK dst,dst,src
Chain KUBE-SERVICES (2 references)
target prot opt source destination
KUBE-MARK-MASQ all -- !172.20.0.0/16 0.0.0.0/0 /* Kubernetes service cluster ip + port for masquerade purpose */ match-set KUBE-CLUSTER-IP dst,dst
KUBE-NODE-PORT all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-CLUSTER-IP dst,dst
[yeqiang@harbor startup_firewalld_on2off]$ sudo iptables -L -n -t mangle
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination [yeqiang@harbor startup_firewalld_on2off]$ sudo iptables -L -n -t raw
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination [yeqiang@harbor startup_firewalld_on2off]$ sudo iptables -L -n -t security
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination 對比防火牆開啟到關閉差異
[yeqiang@harbor iptables]$ diff -y -r startup_firewalld_on startup_firewalld_on2off/
diff -y -r startup_firewalld_on/filter.txt startup_firewalld_on2off/filter.txt
Chain INPUT (policy ACCEPT) Chain INPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0 KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
INPUT_direct all -- 0.0.0.0/0 0.0.0.0/0 <
INPUT_ZONES all -- 0.0.0.0/0 0.0.0.0/0 <
DROP all -- 0.0.0.0/0 0.0.0.0/0 <
REJECT all -- 0.0.0.0/0 0.0.0.0/0 <
Chain FORWARD (policy ACCEPT) Chain FORWARD (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-FORWARD all -- 0.0.0.0/0 0.0.0.0/0 KUBE-FORWARD all -- 0.0.0.0/0 0.0.0.0/0
DOCKER-USER all -- 0.0.0.0/0 0.0.0.0/0 <
DOCKER-ISOLATION-STAGE-1 all -- 0.0.0.0/0 0.0.0 <
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
DOCKER all -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
FORWARD_direct all -- 0.0.0.0/0 0.0.0.0/0 <
FORWARD_IN_ZONES all -- 0.0.0.0/0 0.0.0.0/0 <
FORWARD_OUT_ZONES all -- 0.0.0.0/0 0.0.0.0/0 <
DROP all -- 0.0.0.0/0 0.0.0.0/0 <
REJECT all -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT all -- 172.20.0.0/16 0.0.0.0/0 ACCEPT all -- 172.20.0.0/16 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 172.20.0.0/16 ACCEPT all -- 0.0.0.0/0 172.20.0.0/16
Chain OUTPUT (policy ACCEPT) Chain OUTPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0 KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain DOCKER (1 references) <
target prot opt source destination <
<
Chain DOCKER-ISOLATION-STAGE-1 (1 references) <
target prot opt source destination <
DOCKER-ISOLATION-STAGE-2 all -- 0.0.0.0/0 0.0.0 <
RETURN all -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain DOCKER-ISOLATION-STAGE-2 (1 references) <
target prot opt source destination <
DROP all -- 0.0.0.0/0 0.0.0.0/0 <
RETURN all -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain DOCKER-USER (1 references) <
target prot opt source destination <
RETURN all -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain FORWARD_IN_ZONES (1 references) <
target prot opt source destination <
FWDI_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0 <
FWDI_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0 <
<
Chain FORWARD_OUT_ZONES (1 references) <
target prot opt source destination <
FWDO_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0 <
FWDO_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0 <
<
Chain FORWARD_direct (1 references) <
target prot opt source destination <
<
Chain FWDI_FedoraWorkstation (2 references) <
target prot opt source destination <
FWDI_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0 <
FWDI_FedoraWorkstation_log all -- 0.0.0.0/0 0.0 <
FWDI_FedoraWorkstation_deny all -- 0.0.0.0/0 0. <
FWDI_FedoraWorkstation_allow all -- 0.0.0.0/0 0 <
FWDI_FedoraWorkstation_post all -- 0.0.0.0/0 0. <
ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain FWDI_FedoraWorkstation_allow (1 references) <
target prot opt source destination <
<
Chain FWDI_FedoraWorkstation_deny (1 references) <
target prot opt source destination <
<
Chain FWDI_FedoraWorkstation_log (1 references) <
target prot opt source destination <
<
Chain FWDI_FedoraWorkstation_post (1 references) <
target prot opt source destination <
<
Chain FWDI_FedoraWorkstation_pre (1 references) <
target prot opt source destination <
<
Chain FWDO_FedoraWorkstation (2 references) <
target prot opt source destination <
FWDO_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0 <
FWDO_FedoraWorkstation_log all -- 0.0.0.0/0 0.0 <
FWDO_FedoraWorkstation_deny all -- 0.0.0.0/0 0. <
FWDO_FedoraWorkstation_allow all -- 0.0.0.0/0 0 <
FWDO_FedoraWorkstation_post all -- 0.0.0.0/0 0. <
<
Chain FWDO_FedoraWorkstation_allow (1 references) <
target prot opt source destination <
<
Chain FWDO_FedoraWorkstation_deny (1 references) <
target prot opt source destination <
<
Chain FWDO_FedoraWorkstation_log (1 references) <
target prot opt source destination <
<
Chain FWDO_FedoraWorkstation_post (1 references) <
target prot opt source destination <
<
Chain FWDO_FedoraWorkstation_pre (1 references) <
target prot opt source destination <
<
Chain INPUT_ZONES (1 references) <
target prot opt source destination <
IN_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 <
IN_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain INPUT_direct (1 references) <
target prot opt source destination <
<
Chain IN_FedoraWorkstation (2 references) <
target prot opt source destination <
IN_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0.0 <
IN_FedoraWorkstation_log all -- 0.0.0.0/0 0.0.0 <
IN_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0. <
IN_FedoraWorkstation_allow all -- 0.0.0.0/0 0.0 <
IN_FedoraWorkstation_post all -- 0.0.0.0/0 0.0. <
ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain IN_FedoraWorkstation_allow (1 references) <
target prot opt source destination <
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain IN_FedoraWorkstation_deny (1 references) <
target prot opt source destination <
<
Chain IN_FedoraWorkstation_log (1 references) <
target prot opt source destination <
<
Chain IN_FedoraWorkstation_post (1 references) <
target prot opt source destination <
<
Chain IN_FedoraWorkstation_pre (1 references) <
target prot opt source destination <
Chain KUBE-FIREWALL (2 references) Chain KUBE-FIREWALL (2 references)
target prot opt source destination target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0 DROP all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-FORWARD (1 references) Chain KUBE-FORWARD (1 references)
target prot opt source destination target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-KUBELET-CANARY (0 references) Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination <
<
Chain OUTPUT_direct (1 references) <
target prot opt source destination target prot opt source destination
diff -y -r startup_firewalld_on/mangle.txt startup_firewalld_on2off/mangle.txt
Chain PREROUTING (policy ACCEPT) Chain PREROUTING (policy ACCEPT)
target prot opt source destination target prot opt source destination
PREROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0 <
PREROUTING_ZONES all -- 0.0.0.0/0 0.0.0.0/0 <
Chain INPUT (policy ACCEPT) Chain INPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
INPUT_direct all -- 0.0.0.0/0 0.0.0.0/0 <
Chain FORWARD (policy ACCEPT) Chain FORWARD (policy ACCEPT)
target prot opt source destination target prot opt source destination
FORWARD_direct all -- 0.0.0.0/0 0.0.0.0/0 <
Chain OUTPUT (policy ACCEPT) Chain OUTPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0 <
Chain POSTROUTING (policy ACCEPT) Chain POSTROUTING (policy ACCEPT)
target prot opt source destination target prot opt source destination
POSTROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain FORWARD_direct (1 references) <
target prot opt source destination <
<
Chain INPUT_direct (1 references) <
target prot opt source destination <
Chain KUBE-KUBELET-CANARY (0 references) Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination <
<
Chain OUTPUT_direct (1 references) <
target prot opt source destination <
<
Chain POSTROUTING_direct (1 references) <
target prot opt source destination <
<
Chain PREROUTING_ZONES (1 references) <
target prot opt source destination <
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/ <
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/ <
<
Chain PREROUTING_direct (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation (2 references) <
target prot opt source destination <
PRE_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0. <
PRE_FedoraWorkstation_log all -- 0.0.0.0/0 0.0. <
PRE_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0 <
PRE_FedoraWorkstation_allow all -- 0.0.0.0/0 0. <
PRE_FedoraWorkstation_post all -- 0.0.0.0/0 0.0 <
<
Chain PRE_FedoraWorkstation_allow (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_deny (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_log (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_post (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_pre (1 references) <
target prot opt source destination target prot opt source destination
diff -y -r startup_firewalld_on/nat.txt startup_firewalld_on2off/nat.txt
Chain PREROUTING (policy ACCEPT) Chain PREROUTING (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0
PREROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0 <
PREROUTING_ZONES all -- 0.0.0.0/0 0.0.0.0/0 <
DOCKER all -- 0.0.0.0/0 0.0.0.0/0 <
Chain INPUT (policy ACCEPT) Chain INPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
Chain OUTPUT (policy ACCEPT) Chain OUTPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0
OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0 <
DOCKER all -- 0.0.0.0/0 !127.0.0.0/8 <
Chain POSTROUTING (policy ACCEPT) Chain POSTROUTING (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-POSTROUTING all -- 0.0.0.0/0 0.0.0.0/0 KUBE-POSTROUTING all -- 0.0.0.0/0 0.0.0.0/0
MASQUERADE all -- 172.17.0.0/16 0.0.0.0/0 <
POSTROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0 <
POSTROUTING_ZONES all -- 0.0.0.0/0 0.0.0.0/0 <
RETURN all -- 172.20.0.0/16 172.20.0.0/16 RETURN all -- 172.20.0.0/16 172.20.0.0/16
MASQUERADE all -- 172.20.0.0/16 !224.0.0.0/4 MASQUERADE all -- 172.20.0.0/16 !224.0.0.0/4
RETURN all -- !172.20.0.0/16 172.20.0.0/24 RETURN all -- !172.20.0.0/16 172.20.0.0/24
MASQUERADE all -- !172.20.0.0/16 172.20.0.0/16 MASQUERADE all -- !172.20.0.0/16 172.20.0.0/16
Chain DOCKER (2 references) <
target prot opt source destination <
RETURN all -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain KUBE-FIREWALL (0 references) Chain KUBE-FIREWALL (0 references)
target prot opt source destination target prot opt source destination
KUBE-MARK-DROP all -- 0.0.0.0/0 0.0.0.0/0 KUBE-MARK-DROP all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-KUBELET-CANARY (0 references) Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination target prot opt source destination
Chain KUBE-LOAD-BALANCER (0 references) Chain KUBE-LOAD-BALANCER (0 references)
target prot opt source destination target prot opt source destination
KUBE-MARK-MASQ all -- 0.0.0.0/0 0.0.0.0/0 KUBE-MARK-MASQ all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-MARK-DROP (1 references) Chain KUBE-MARK-DROP (1 references)
target prot opt source destination target prot opt source destination
Chain KUBE-MARK-MASQ (3 references) Chain KUBE-MARK-MASQ (3 references)
target prot opt source destination target prot opt source destination
MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-NODE-PORT (1 references) Chain KUBE-NODE-PORT (1 references)
target prot opt source destination target prot opt source destination
KUBE-MARK-MASQ tcp -- 0.0.0.0/0 0.0.0.0/0 KUBE-MARK-MASQ tcp -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-POSTROUTING (1 references) Chain KUBE-POSTROUTING (1 references)
target prot opt source destination target prot opt source destination
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-SERVICES (2 references) Chain KUBE-SERVICES (2 references)
target prot opt source destination target prot opt source destination
KUBE-MARK-MASQ all -- !172.20.0.0/16 0.0.0.0/0 KUBE-MARK-MASQ all -- !172.20.0.0/16 0.0.0.0/0
KUBE-NODE-PORT all -- 0.0.0.0/0 0.0.0.0/0 KUBE-NODE-PORT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
<
Chain OUTPUT_direct (1 references) <
target prot opt source destination <
<
Chain POSTROUTING_ZONES (1 references) <
target prot opt source destination <
POST_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0 <
POST_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0 <
<
Chain POSTROUTING_direct (1 references) <
target prot opt source destination <
<
Chain POST_FedoraWorkstation (2 references) <
target prot opt source destination <
POST_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0 <
POST_FedoraWorkstation_log all -- 0.0.0.0/0 0.0 <
POST_FedoraWorkstation_deny all -- 0.0.0.0/0 0. <
POST_FedoraWorkstation_allow all -- 0.0.0.0/0 0 <
POST_FedoraWorkstation_post all -- 0.0.0.0/0 0. <
<
Chain POST_FedoraWorkstation_allow (1 references) <
target prot opt source destination <
<
Chain POST_FedoraWorkstation_deny (1 references) <
target prot opt source destination <
<
Chain POST_FedoraWorkstation_log (1 references) <
target prot opt source destination <
<
Chain POST_FedoraWorkstation_post (1 references) <
target prot opt source destination <
<
Chain POST_FedoraWorkstation_pre (1 references) <
target prot opt source destination <
<
Chain PREROUTING_ZONES (1 references) <
target prot opt source destination <
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/ <
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/ <
<
Chain PREROUTING_direct (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation (2 references) <
target prot opt source destination <
PRE_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0. <
PRE_FedoraWorkstation_log all -- 0.0.0.0/0 0.0. <
PRE_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0 <
PRE_FedoraWorkstation_allow all -- 0.0.0.0/0 0. <
PRE_FedoraWorkstation_post all -- 0.0.0.0/0 0.0 <
<
Chain PRE_FedoraWorkstation_allow (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_deny (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_log (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_post (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_pre (1 references) <
target prot opt source destination <
diff -y -r startup_firewalld_on/raw.txt startup_firewalld_on2off/raw.txt
Chain PREROUTING (policy ACCEPT) Chain PREROUTING (policy ACCEPT)
target prot opt source destination target prot opt source destination
PREROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0 <
PREROUTING_ZONES all -- 0.0.0.0/0 0.0.0.0/0 <
Chain OUTPUT (policy ACCEPT) Chain OUTPUT (policy ACCEPT)
target prot opt source destination <
OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain OUTPUT_direct (1 references) <
target prot opt source destination <
<
Chain PREROUTING_ZONES (1 references) <
target prot opt source destination <
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/ <
PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/ <
<
Chain PREROUTING_direct (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation (2 references) <
target prot opt source destination <
PRE_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0. <
PRE_FedoraWorkstation_log all -- 0.0.0.0/0 0.0. <
PRE_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0 <
PRE_FedoraWorkstation_allow all -- 0.0.0.0/0 0. <
PRE_FedoraWorkstation_post all -- 0.0.0.0/0 0.0 <
<
Chain PRE_FedoraWorkstation_allow (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_deny (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_log (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_post (1 references) <
target prot opt source destination <
<
Chain PRE_FedoraWorkstation_pre (1 references) <
target prot opt source destination target prot opt source destination
diff -y -r startup_firewalld_on/security.txt startup_firewalld_on2off/security.txt
Chain INPUT (policy ACCEPT) Chain INPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
INPUT_direct all -- 0.0.0.0/0 0.0.0.0/0 <
Chain FORWARD (policy ACCEPT) Chain FORWARD (policy ACCEPT)
target prot opt source destination target prot opt source destination
FORWARD_direct all -- 0.0.0.0/0 0.0.0.0/0 <
Chain OUTPUT (policy ACCEPT) Chain OUTPUT (policy ACCEPT)
target prot opt source destination <
OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain FORWARD_direct (1 references) <
target prot opt source destination <
<
Chain INPUT_direct (1 references) <
target prot opt source destination <
<
Chain OUTPUT_direct (1 references) <
target prot opt source destination target prot opt source destination
對比關閉防火牆狀態與開啟防火牆後再關閉
[yeqiang@harbor iptables]$ diff -y -r startup_firewalld_off startup_firewalld_on2off/
diff -y -r startup_firewalld_off/filter.txt startup_firewalld_on2off/filter.txt
Chain INPUT (policy ACCEPT) Chain INPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0 KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 <
REJECT all -- 0.0.0.0/0 0.0.0.0/0 <
Chain FORWARD (policy ACCEPT) Chain FORWARD (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-FORWARD all -- 0.0.0.0/0 0.0.0.0/0 KUBE-FORWARD all -- 0.0.0.0/0 0.0.0.0/0
DOCKER-USER all -- 0.0.0.0/0 0.0.0.0/0 <
DOCKER-ISOLATION-STAGE-1 all -- 0.0.0.0/0 0.0.0 <
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
DOCKER all -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 <
REJECT all -- 0.0.0.0/0 0.0.0.0/0 <
ACCEPT all -- 172.20.0.0/16 0.0.0.0/0 ACCEPT all -- 172.20.0.0/16 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 172.20.0.0/16 ACCEPT all -- 0.0.0.0/0 172.20.0.0/16
Chain OUTPUT (policy ACCEPT) Chain OUTPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0 KUBE-FIREWALL all -- 0.0.0.0/0 0.0.0.0/0
<
Chain DOCKER (1 references) <
target prot opt source destination <
<
Chain DOCKER-ISOLATION-STAGE-1 (1 references) <
target prot opt source destination <
DOCKER-ISOLATION-STAGE-2 all -- 0.0.0.0/0 0.0.0 <
RETURN all -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain DOCKER-ISOLATION-STAGE-2 (1 references) <
target prot opt source destination <
DROP all -- 0.0.0.0/0 0.0.0.0/0 <
RETURN all -- 0.0.0.0/0 0.0.0.0/0 <
<
Chain DOCKER-USER (1 references) <
target prot opt source destination <
RETURN all -- 0.0.0.0/0 0.0.0.0/0 <
Chain KUBE-FIREWALL (2 references) Chain KUBE-FIREWALL (2 references)
target prot opt source destination target prot opt source destination
DROP all -- 0.0.0.0/0 0.0.0.0/0 DROP all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-FORWARD (1 references) Chain KUBE-FORWARD (1 references)
target prot opt source destination target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-KUBELET-CANARY (0 references) Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination target prot opt source destination
diff -y -r startup_firewalld_off/mangle.txt startup_firewalld_on2off/mangle.txt
Chain PREROUTING (policy ACCEPT) Chain PREROUTING (policy ACCEPT)
target prot opt source destination target prot opt source destination
Chain INPUT (policy ACCEPT) Chain INPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
Chain FORWARD (policy ACCEPT) Chain FORWARD (policy ACCEPT)
target prot opt source destination target prot opt source destination
Chain OUTPUT (policy ACCEPT) Chain OUTPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
Chain POSTROUTING (policy ACCEPT) Chain POSTROUTING (policy ACCEPT)
target prot opt source destination target prot opt source destination
Chain KUBE-KUBELET-CANARY (0 references) Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination target prot opt source destination
diff -y -r startup_firewalld_off/nat.txt startup_firewalld_on2off/nat.txt
Chain PREROUTING (policy ACCEPT) Chain PREROUTING (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0
DOCKER all -- 0.0.0.0/0 0.0.0.0/0 <
Chain INPUT (policy ACCEPT) Chain INPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
Chain OUTPUT (policy ACCEPT) Chain OUTPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0
DOCKER all -- 0.0.0.0/0 !127.0.0.0/8 <
Chain POSTROUTING (policy ACCEPT) Chain POSTROUTING (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-POSTROUTING all -- 0.0.0.0/0 0.0.0.0/0 KUBE-POSTROUTING all -- 0.0.0.0/0 0.0.0.0/0
MASQUERADE all -- 172.17.0.0/16 0.0.0.0/0 <
RETURN all -- 172.20.0.0/16 172.20.0.0/16 RETURN all -- 172.20.0.0/16 172.20.0.0/16
MASQUERADE all -- 172.20.0.0/16 !224.0.0.0/4 MASQUERADE all -- 172.20.0.0/16 !224.0.0.0/4
RETURN all -- !172.20.0.0/16 172.20.0.0/24 RETURN all -- !172.20.0.0/16 172.20.0.0/24
MASQUERADE all -- !172.20.0.0/16 172.20.0.0/16 MASQUERADE all -- !172.20.0.0/16 172.20.0.0/16
<
Chain DOCKER (2 references) <
target prot opt source destination <
RETURN all -- 0.0.0.0/0 0.0.0.0/0 <
Chain KUBE-FIREWALL (0 references) Chain KUBE-FIREWALL (0 references)
target prot opt source destination target prot opt source destination
KUBE-MARK-DROP all -- 0.0.0.0/0 0.0.0.0/0 KUBE-MARK-DROP all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-KUBELET-CANARY (0 references) Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination target prot opt source destination
Chain KUBE-LOAD-BALANCER (0 references) Chain KUBE-LOAD-BALANCER (0 references)
target prot opt source destination target prot opt source destination
KUBE-MARK-MASQ all -- 0.0.0.0/0 0.0.0.0/0 KUBE-MARK-MASQ all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-MARK-DROP (1 references) Chain KUBE-MARK-DROP (1 references)
target prot opt source destination target prot opt source destination
Chain KUBE-MARK-MASQ (3 references) Chain KUBE-MARK-MASQ (3 references)
target prot opt source destination target prot opt source destination
MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-NODE-PORT (1 references) Chain KUBE-NODE-PORT (1 references)
target prot opt source destination target prot opt source destination
KUBE-MARK-MASQ tcp -- 0.0.0.0/0 0.0.0.0/0 KUBE-MARK-MASQ tcp -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-POSTROUTING (1 references) Chain KUBE-POSTROUTING (1 references)
target prot opt source destination target prot opt source destination
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-SERVICES (2 references) Chain KUBE-SERVICES (2 references)
target prot opt source destination target prot opt source destination
KUBE-MARK-MASQ all -- !172.20.0.0/16 0.0.0.0/0 KUBE-MARK-MASQ all -- !172.20.0.0/16 0.0.0.0/0
KUBE-NODE-PORT all -- 0.0.0.0/0 0.0.0.0/0 KUBE-NODE-PORT all -- 0.0.0.0/0 0.0.0.0/0
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
diff -y -r startup_firewalld_off/raw.txt startup_firewalld_on2off/raw.txt
Chain PREROUTING (policy ACCEPT) Chain PREROUTING (policy ACCEPT)
target prot opt source destination target prot opt source destination
Chain OUTPUT (policy ACCEPT) Chain OUTPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
diff -y -r startup_firewalld_off/security.txt startup_firewalld_on2off/security.txt
Chain INPUT (policy ACCEPT) Chain INPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
Chain FORWARD (policy ACCEPT) Chain FORWARD (policy ACCEPT)
target prot opt source destination target prot opt source destination
Chain OUTPUT (policy ACCEPT) Chain OUTPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
可以看到，開機，防火牆關閉狀態下，nat Chain PREROUTING, Chain OUTPUT 多了一行DOCKER鏈
Chain PREROUTING (policy ACCEPT) Chain PREROUTING (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0
DOCKER all -- 0.0.0.0/0 0.0.0.0/0 <
Chain OUTPUT (policy ACCEPT) Chain OUTPUT (policy ACCEPT)
target prot opt source destination target prot opt source destination
KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0
DOCKER all -- 0.0.0.0/0 !127.0.0.0/8 <
列印出規則編號
[yeqiang@harbor iptables]$ sudo iptables -L -n -t nat --line-number
Chain PREROUTING (policy ACCEPT)
num target prot opt source destination
1 KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
2 PREROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0
3 PREROUTING_ZONES all -- 0.0.0.0/0 0.0.0.0/0
4 DOCKER all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
Chain INPUT (policy ACCEPT)
num target prot opt source destination
Chain OUTPUT (policy ACCEPT)
num target prot opt source destination
1 KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service portals */
2 OUTPUT_direct all -- 0.0.0.0/0 0.0.0.0/0
3 DOCKER all -- 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL
Chain POSTROUTING (policy ACCEPT)
num target prot opt source destination
1 KUBE-POSTROUTING all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes postrouting rules */
2 MASQUERADE all -- 172.17.0.0/16 0.0.0.0/0
3 POSTROUTING_direct all -- 0.0.0.0/0 0.0.0.0/0
4 POSTROUTING_ZONES all -- 0.0.0.0/0 0.0.0.0/0
5 RETURN all -- 172.20.0.0/16 172.20.0.0/16
6 MASQUERADE all -- 172.20.0.0/16 !224.0.0.0/4 random-fully
7 RETURN all -- !172.20.0.0/16 172.20.0.0/24
8 MASQUERADE all -- !172.20.0.0/16 172.20.0.0/16 random-fully
Chain DOCKER (2 references)
num target prot opt source destination
1 RETURN all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-FIREWALL (0 references)
num target prot opt source destination
1 KUBE-MARK-DROP all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-KUBELET-CANARY (0 references)
num target prot opt source destination
Chain KUBE-LOAD-BALANCER (0 references)
num target prot opt source destination
1 KUBE-MARK-MASQ all -- 0.0.0.0/0 0.0.0.0/0
Chain KUBE-MARK-DROP (1 references)
num target prot opt source destination
Chain KUBE-MARK-MASQ (3 references)
num target prot opt source destination
1 MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK or 0x4000
Chain KUBE-NODE-PORT (1 references)
num target prot opt source destination
1 KUBE-MARK-MASQ tcp -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes nodeport TCP port for masquerade purpose */ match-set KUBE-NODE-PORT-TCP dst
Chain KUBE-POSTROUTING (1 references)
num target prot opt source destination
1 MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service traffic requiring SNAT */ mark match 0x4000/0x4000 random-fully
2 MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 /* Kubernetes endpoints dst ip:port, source ip for solving hairpin purpose */ match-set KUBE-LOOP-BACK dst,dst,src
Chain KUBE-SERVICES (2 references)
num target prot opt source destination
1 KUBE-MARK-MASQ all -- !172.20.0.0/16 0.0.0.0/0 /* Kubernetes service cluster ip + port for masquerade purpose */ match-set KUBE-CLUSTER-IP dst,dst
2 KUBE-NODE-PORT all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
3 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-CLUSTER-IP dst,dst
Chain OUTPUT_direct (1 references)
num target prot opt source destination
Chain POSTROUTING_ZONES (1 references)
num target prot opt source destination
1 POST_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
2 POST_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
Chain POSTROUTING_direct (1 references)
num target prot opt source destination
Chain POST_FedoraWorkstation (2 references)
num target prot opt source destination
1 POST_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0.0.0/0
2 POST_FedoraWorkstation_log all -- 0.0.0.0/0 0.0.0.0/0
3 POST_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0.0.0/0
4 POST_FedoraWorkstation_allow all -- 0.0.0.0/0 0.0.0.0/0
5 POST_FedoraWorkstation_post all -- 0.0.0.0/0 0.0.0.0/0
Chain POST_FedoraWorkstation_allow (1 references)
num target prot opt source destination
Chain POST_FedoraWorkstation_deny (1 references)
num target prot opt source destination
Chain POST_FedoraWorkstation_log (1 references)
num target prot opt source destination
Chain POST_FedoraWorkstation_post (1 references)
num target prot opt source destination
Chain POST_FedoraWorkstation_pre (1 references)
num target prot opt source destination
Chain PREROUTING_ZONES (1 references)
num target prot opt source destination
1 PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
2 PRE_FedoraWorkstation all -- 0.0.0.0/0 0.0.0.0/0 [goto]
Chain PREROUTING_direct (1 references)
num target prot opt source destination
Chain PRE_FedoraWorkstation (2 references)
num target prot opt source destination
1 PRE_FedoraWorkstation_pre all -- 0.0.0.0/0 0.0.0.0/0
2 PRE_FedoraWorkstation_log all -- 0.0.0.0/0 0.0.0.0/0
3 PRE_FedoraWorkstation_deny all -- 0.0.0.0/0 0.0.0.0/0
4 PRE_FedoraWorkstation_allow all -- 0.0.0.0/0 0.0.0.0/0
5 PRE_FedoraWorkstation_post all -- 0.0.0.0/0 0.0.0.0/0
Chain PRE_FedoraWorkstation_allow (1 references)
num target prot opt source destination
Chain PRE_FedoraWorkstation_deny (1 references)
num target prot opt source destination
Chain PRE_FedoraWorkstation_log (1 references)
num target prot opt source destination
Chain PRE_FedoraWorkstation_post (1 references)
num target prot opt source destination
Chain PRE_FedoraWorkstation_pre (1 references)
num target prot opt source destination
手動刪除掉這兩條規則
[yeqiang@harbor iptables]$ sudo iptables -t nat -D PREROUTING 4
[yeqiang@harbor iptables]$ sudo iptables -t nat -D OUTPUT 3
還是無效，只能手動停止防火牆服務。沒有頭緒。。。
相關文章
- AS編譯不通過,提示Error:Connection refused: connect編譯Error
- Gradle 編譯不通過 connection refused connectGradle編譯
- Kubernetes node的防火牆問題導致pod ip無法訪問防火牆
- 容器網路防火牆狀態異常導致丟包排查記錄防火牆
- Error:Can't connect to SOCKS proxy:Connection refused (Connection refused)Error
- LINUX 防火牆 firewalldLinux防火牆
- 伺服器由於防火牆問題導致埠不通解決方法伺服器防火牆
- 防火牆基礎Firewalld命令配置防火牆
- firewalld管理防火牆常用命令防火牆
- centos7預設防火牆firewalldCentOS防火牆
- 使用 FirewallD 構建動態防火牆防火牆
- Centos 7 systemctl和防火牆firewalld命令CentOS防火牆
- ssh: connect to host localhost port 22: Connection refusedlocalhost
- Centos 7防火牆firewalld開放80埠CentOS防火牆
- 私有網路介面丟失導致例項崩潰
- windows 防火牆 導致 tnsping 不能成功Windows防火牆
- ASM例項出現ORA-4031錯誤導致例項崩潰ASM
- 防火牆 | 網路協議防火牆協議
- 安全網路防火牆的十二個注意事項(轉)防火牆
- 案例分享-full gc導致k8s pod重啟GCK8S
- Linux系統iptables與Firewalld防火牆區別？Linux防火牆
- Linux 防火牆配置（iptables和firewalld）詳細教程。Linux防火牆
- Fedora Core 6 下載量大 導致網路擁塞(轉)
- 坑：ssh: connect to host github.com port 22: Connection refusedGithub
- java.net.ConnectException: Connection refused (Connection refused)JavaException
- 網路安全——防火牆詳解防火牆
- kubernetes/k8s CNI分析-容器網路介面分析K8S
- kubernetes應用flannel失敗 Connection refused.
- Linux 防火牆：關於 iptables 和 firewalld 的那些事Linux防火牆
- 如何在 Linux 系統中配置 firewalld 防火牆策略Linux防火牆
- mysql connection refusedMySql
- CentOS 防火牆配置與REJECT導致沒有生效問題CentOS防火牆
- PLSQL連線不了資料庫（防火牆，埠原因導致）SQL資料庫防火牆
- 網路防火牆的配置與管理防火牆
- CentOS7使用firewalld開啟關閉防火牆與埠CentOS防火牆
- Centos7預設防火牆之firewalld講解及配置CentOS防火牆
- 異常解決——GitLab ： ssh: connect to host port 22: Connection refusedGitlab
- telnet localhost 44444 telnet: connect to address ::1: Connection refusedlocalhost