ORA-07445 [kslgetl()+80] ORA-00108主機名配置不一致引起的報錯

環境：
Oracle 10.2.0.4, Red Hat Linux4.5 32bit

3.8婦女節這天，一臺DB server因硬體故障掛掉。固透過備份恢復後的DB到新server上，恢復的DB開啟沒用多久硬碟就滿了，檢查DB bdump目錄寫了大量trc檔案，且open後alert.log不斷的報錯：
Fri Mar  9 04:19:59 2018
dispatcher 'D000' encountered error getting listening address
Fri Mar  9 04:19:59 2018
Errors in file /u01/product/admin/umlaut/bdump/umlaut_ora_20942.trc:
ORA-07445: exception encountered: core dump [kslgetl()+80] [SIGSEGV] [Address not mapped to object] [0x130] [] []
ORA-00108: failed to set up dispatcher to accept connection asynchronously
Fri Mar  9 04:20:02 2018
found dead dispatcher 'D000', pid = (13, 219)
Fri Mar  9 04:20:02 2018
dispatcher 'D000' encountered error getting listening address
Fri Mar  9 04:20:02 2018
Errors in file /u01/product/admin/umlaut/bdump/umlaut_ora_20944.trc:
ORA-07445: exception encountered: core dump [kslgetl()+80] [SIGSEGV] [Address not mapped to object] [0x130] [] []
ORA-00108: failed to set up dispatcher to accept connection asynchronously
....

trc檔案內容：
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
ORACLE_HOME = /u01/product/oracle
System name:    Linux
Node name:      dthistory02
Release:        2.6.9-55.ELsmp
Version:        #1 SMP Fri Apr 20 17:03:35 EDT 2007
Machine:        i686
Instance name: umlaut
Redo thread mounted by this instance: 1
Oracle process number: 13
Unix process pid: 18998, image: oracle@dthistory02 (D000)
Warning: keltnfy call to ldmInit failed with error 46
*** 2018-03-09 03:42:51.581
network error encountered getting listening address:
  NS Primary Error: TNS-12533: TNS:illegal ADDRESS parameters
  NS Secondary Error: TNS-12560: TNS:protocol adapter error
  NT Generic Error: TNS-00503: Illegal ADDRESS parameters
OPIRIP: Uncaught error 108. Error stack:
ORA-00108: failed to set up dispatcher to accept connection asynchronously
Exception signal: 11 (SIGSEGV), code: 1 (Address not mapped to object), addr: 0x130, PC: [0x831f388, kslgetl()+80]
Registers:
%eax: 0x00000000 %ebx: 0x00000007 %ecx: 0x00001768
%edx: 0x00003980 %edi: 0x20098a44 %esi: 0x00000000
%esp: 0xbffff188 %ebp: 0xbffff194 %eip: 0x0831f388
%efl: 0x00010246
  kslgetl()+57 (0x831f371) jnz 0x831f5a2
  kslgetl()+63 (0x831f377) mov 0xce77488,%esi
  kslgetl()+69 (0x831f37d) mov 0xc(%edi),%ebx
  kslgetl()+72 (0x831f380) movzw 0xc1e0d20(,%ebx,2),%edx
> kslgetl()+80 (0x831f388) test %edx,0x130(%esi)
  kslgetl()+86 (0x831f38e) lea 0x9c(%esi),%eax
  kslgetl()+92 (0x831f394) jnz 0x831f58c
  kslgetl()+98 (0x831f39a) test %ebx,%ebx
  kslgetl()+100 (0x831f39c) jl 0x831f4e7
*** 2018-03-09 03:42:51.583
ksedmp: internal or fatal error
ORA-07445: exception encountered: core dump [kslgetl()+80] [SIGSEGV] [Address not mapped to object] [0x130] [] []
ORA-00108: failed to set up dispatcher to accept connection asynchronously
Current SQL information unavailable - no session.

查到MOS文章: ORA-07445: [kslgetl()+80] Followed by ORA-108: failed to set up dispatcher to accept connection asynchronously(ID 1298804.1)和報錯基本一致，發現此問題主要和hostname及hosts設定相關。

檢查配置：
# hostname 
dthistory02
# cat /etc/hosts
10.10.10.191   dt     history02  localhost
果然 hostname和hosts中不一致，修改/etc/hosts後再無報錯(不需要重啟DB)，至此問題解決。

附：文件 ID 1298804.1
ORA-07445: [kslgetl()+80] Followed by ORA-108: failed to set up dispatcher to accept connection asynchronously

APPLIES TO:

Oracle Net Services - Version 11.1.0.7 and later
Information in this document applies to any platform.

SYMPTOMS

The following errors are seen in the trace file written by an ORA-7445 [kslgetl]:

network error encountered getting listening address:
NS Primary Error: TNS-12533: TNS:illegal ADDRESS parameters
NS Secondary Error: TNS-12560: TNS:protocol adapter error
NT Generic Error: TNS-00503: Illegal ADDRESS parameters
OPIRIP: Uncaught error 108. Error stack:
ORA-00108: failed to set up dispatcher to accept connection asynchronously
Exception signal: 11 (SIGSEGV), code: 1 (Address not mapped to object), addr: 0x130, PC: [0x82f09dc, kslgetl()+80]

The trace file indicates that there is no session:

Current SQL information unavailable - no session.

The Call Stack Trace in the ORA-7445 trace file contains a function list similar to:

 kslgetl <- PGOSF57_ksfglt
<- kghfre <- kmnsbf <- nsbfr <- nsiofrrg <- nsiocancel
<- nsopen_shutitdown <- nsclose <- nsgblclose <- nsgblTRMHelper <- nsgblRealTerm
<- nlstdstp <- npinlt <- ksuabt <- opidrv <- sou2o
<- opimai_real <- main <- libc_start_main

CHANGES

None.

CAUSE

The trace file first reports: Warning: keltnfy call to ldmInit failed with error 46 

The ORA-7445 is not the starting point here. This exception is just a spin-off from ORA-108 and it is possible that different internal errors may be seen, such as ORA-600 [504], depending on what is happening when the ORA-108 is encountered.

The cause for the ORA-108 is related to the inital message at the beginning of the trace file: "keltnfy call to ldmInit failed with error 46" and this is followed by: "network error encountered getting listening address:" 

The error code (here: 46) is the key for solving the issue.
This warning says that ldmInit() returned error 46 which is LDMERR_HOST_NOT_FOUND (host not found).

This error is returned if the OS call gethostbyname() fails with an error. So these appears to be a network specific issue.

SOLUTION

1) Check permission on /etc/hosts

$ ls -l /etc/hosts -rw-r--r-- 2 root root 194 Oct 17 2006 /etc/hosts


Check if /etc/hosts file is correctly configured


( all of this on one line ).


2) Check the hostname:

$ hostname
$ ping `hostname`

Make sure you are able to ping the hostname


3) Check if /etc/nodename is correctly configured
If you have DNS setup, ping is not a tool to diagnose DNS problem. A better tool to use is nslookup, dnsquery, or dig.


$ nslookup $ nslookup $ nslookup

The forward and reverse lookup should succeed and return consistent address/info.


4) Check nsswitch.conf


$ more nsswitch.confhosts: files dnsMake sure host lookup is also done through the /etc/hosts file and not just dns. It is recommended that FILES come first before DNS.


Also, check the resolv.conf. This makes sure that the DNS is working properly.