環境:
11.2.0.3 ADG (db11g\db11gadg\db11gcas)
在自己先前克隆後的環境互相tnsping報錯。
tnsping 本機ok,tnsping其他機器均報錯:
[oracle@db11g ~]$ tnsping jingyu
TNS Ping Utility for Linux: Version 11.2.0.3.0 - Production on 13-MAY-2023 08:09:11
Copyright (c) 1997, 2011, Oracle. All rights reserved.
Used parameter files:
/u01/app/oracle/product/11.2.0/dbhome_1/network/admin/sqlnet.ora
Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.1.18)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = jingyu)))
OK (0 msec)
[oracle@db11g ~]$ tnsping jyadg
TNS Ping Utility for Linux: Version 11.2.0.3.0 - Production on 13-MAY-2023 08:09:14
Copyright (c) 1997, 2011, Oracle. All rights reserved.
Used parameter files:
/u01/app/oracle/product/11.2.0/dbhome_1/network/admin/sqlnet.ora
Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.1.19)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = jyadg)))
TNS-12543: TNS:destination host unreachable
[oracle@db11g ~]$ tnsping jycas
TNS Ping Utility for Linux: Version 11.2.0.3.0 - Production on 13-MAY-2023 08:09:17
Copyright (c) 1997, 2011, Oracle. All rights reserved.
Used parameter files:
/u01/app/oracle/product/11.2.0/dbhome_1/network/admin/sqlnet.ora
Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.1.20)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = jycas)))
TNS-12543: TNS:destination host unreachable
1.檢查防火牆和SElinux狀態
第一反應就是防火牆或SELinux導致,初步檢查發現:防火牆未執行,SELinux也禁用掉。[root@db11g ~]# service iptables status
Redirecting to /bin/systemctl status iptables.service
Unit iptables.service could not be found.
[root@db11g ~]# getenforce
Permissive
[root@db11g ~]# cat /etc/sysconfig/selinux
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
配置檔案tnsnames.ora同步到3個環境,保持一致:
# tnsnames.ora Network Configuration File: /u01/app/oracle/product/11.2.0/dbhome_1/network/admin/tnsnames.ora
# Generated by Oracle configuration tools.
JINGYU =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.1.18)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = jingyu)
)
)
JYADG =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.1.19)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = jyadg)
)
)
JYCAS =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.1.20)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = jycas)
)
)
2.確認克隆的網路卡是否正常
主要懷疑克隆是否MAC地址之類做了變化,期望是不一樣的MAC,實際也是,在宿主機看arp記錄的和目前的都匹配:
[oracle@bogon ~]$ arp -a|grep 1.18
bogon (192.168.1.18) at 52:54:00:6f:92:54 [ether] on br0
[oracle@bogon ~]$ arp -a|grep 1.19
localhost (192.168.1.19) at 52:54:00:d0:49:1b [ether] on br0
[oracle@bogon ~]$ arp -a|grep 1.20
localhost (192.168.1.20) at 52:54:00:71:f1:9a [ether] on br0
可現在就是網路IP可以ping通,但是tnsping說啥不通:
[oracle@bogon ~]$ ping 192.168.1.18
PING 192.168.1.18 (192.168.1.18) 56(84) bytes of data.
64 bytes from 192.168.1.18: icmp_seq=1 ttl=64 time=0.525 ms
64 bytes from 192.168.1.18: icmp_seq=2 ttl=64 time=0.312 ms
^C
--- 192.168.1.18 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1024ms
rtt min/avg/max/mdev = 0.312/0.418/0.525/0.108 ms
[oracle@bogon ~]$ tnsping 192.168.1.18
TNS Ping Utility for Linux: Version 19.0.0.0.0 - Production on 13-MAY-2023 08:39:50
Copyright (c) 1997, 2022, Oracle. All rights reserved.
Used parameter files:
/u01/app/oracle/product/19.3.0/db_1/network/admin/sqlnet.ora
Used HOSTNAME adapter to resolve the alias
Attempting to contact (DESCRIPTION=(CONNECT_DATA=(SERVICE_NAME=))(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.1.18)(PORT=1521)))
TNS-12543: TNS:destination host unreachable
3.網路層面排查1521埠
以db11g環境為例說明:
[oracle@db11g ~]$ netstat -ano|grep 1521
tcp6 0 0 :::1521 :::* LISTEN off (0.00/0/0)
unix 2 [ ACC ] STREAM LISTENING 26303 /var/tmp/.oracle/sEXTPROC1521
使用telnet工具定位:
[root@bogon ~]# telnet 192.168.1.18 1521
Trying 192.168.1.18...
telnet: connect to address 192.168.1.18: No route to host
而正常的環境(其他可以tnsping的正常虛機)應該類似這樣:
[root@bogon ~]# telnet 192.168.1.11 1521
Trying 192.168.1.11...
Connected to 192.168.1.11.
Escape character is '^]'.
看來的確是網路問題了。
說沒有路由可達主機,直接traceroute對比看下有何區別:
# 正常的虛機:
[root@bogon ~]# traceroute 192.168.1.11
traceroute to 192.168.1.11 (192.168.1.11), 30 hops max, 60 byte packets
1 bogon (192.168.1.11) 0.556 ms 0.507 ms 0.486 ms
# 不正常的虛機:
[root@bogon ~]# traceroute 192.168.1.18
traceroute to 192.168.1.18 (192.168.1.18), 30 hops max, 60 byte packets
1 bogon (192.168.1.18) 0.516 ms !X 0.450 ms !X 0.404 ms !X
[root@bogon ~]# traceroute 192.168.1.19
traceroute to 192.168.1.19 (192.168.1.19), 30 hops max, 60 byte packets
1 bogon (192.168.1.19) 0.546 ms !X 0.491 ms !X 0.455 ms !X
[root@bogon ~]# traceroute 192.168.1.20
traceroute to 192.168.1.20 (192.168.1.20), 30 hops max, 60 byte packets
1 localhost (192.168.1.20) 0.730 ms !X 0.680 ms !X 0.648 ms !X
又回到防火牆。。之前檢查命令搞錯了,來吧,趕緊正常關閉防火牆:
[root@db11g ~]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
Active: active (running) since Sat 2023-05-13 08:38:13 CST; 23min ago
Docs: man:firewalld(1)
Main PID: 850 (firewalld)
CGroup: /system.slice/firewalld.service
└─850 /usr/bin/python2 -Es /usr/sbin/firewalld --nofork --nopid
May 13 08:38:12 db11g systemd[1]: Starting firewalld - dynamic firewall dae.....
May 13 08:38:13 db11g systemd[1]: Started firewalld - dynamic firewall daemon.
May 13 08:38:13 db11g firewalld[850]: WARNING: AllowZoneDrifting is enabled...w.
Hint: Some lines were ellipsized, use -l to show in full.
[root@db11g ~]# systemctl stop firewalld
一切恢復正常:
[root@bogon ~]# traceroute 192.168.1.18
traceroute to 192.168.1.18 (192.168.1.18), 30 hops max, 60 byte packets
1 bogon (192.168.1.18) 0.559 ms 0.497 ms 0.445 ms
[root@bogon ~]# telnet 192.168.1.18 1521
Trying 192.168.1.18...
Connected to 192.168.1.18.
Escape character is '^]'.
[oracle@bogon ~]$ tnsping 192.168.1.18
TNS Ping Utility for Linux: Version 19.0.0.0.0 - Production on 13-MAY-2023 08:59:56
Copyright (c) 1997, 2022, Oracle. All rights reserved.
Used parameter files:
/u01/app/oracle/product/19.3.0/db_1/network/admin/sqlnet.ora
Used HOSTNAME adapter to resolve the alias
Attempting to contact (DESCRIPTION=(CONNECT_DATA=(SERVICE_NAME=))(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.1.18)(PORT=1521)))
OK (0 msec)
克隆的環境全部都關閉下防火牆,並禁用:
systemctl status firewalld
systemctl stop firewalld
systemctl disable firewalld
一切正常!
這個根因找到後著實有點丟人了,折騰半天居然就是最開始懷疑的防火牆問題,但卻因為命令搞錯誤判了。
記得早些時候這個Linux6檢視防火牆的命令在Linux7中執行,重定向是可以變成正確命令的。
再就是應該多查下,如果是類似下面的輸出才是沒有防火牆執行:
[root@db11g ~]# /sbin/iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
不過troubleshooting的有意思之處也就在於此,你總不能每次都可以快速解決問題,哪怕是相對簡單的問題,也可能因為一個疏忽而走不少彎路,但這個過程是有意思的!