ORA-27302: failure occurred at: sskgxprcv1
客戶的有臺機器報錯
Thu Jun 2 06:02:39 2011
Errors in file /home/oracle/admin/orcl/udump/orcl2_ora_21099.trc:
ORA-01013: user requested cancel of current operation
Thu Jun 2 06:02:40 2011
Errors in file /home/oracle/admin/orcl/udump/orcl2_ora_21099.trc:
ORA-27509: IPC error receiving a message
ORA-27300: OS system dependent operation:recvmsg failed with status: 88
ORA-27301: OS failure message: Socket operation on non-socket
ORA-27302: failure occurred at: sskgxprcv1
ORA-01013: user requested cancel of current operation
Thu Jun 2 08:22:11 2011
Detected change in CPU count to 16
Thu Jun 2 08:22:11 2011
Resource Manager cpu_count => Low Threshold (20) : High Threshold (24)
Thu Jun 2 08:22:49 2011
ospid 20693: network interface with IP address 192.168.0.2 is now running
Thu Jun 2 08:23:34 2011
IPC Send timeout detected. Receiver ospid 20730
Thu Jun 2 08:23:34 2011
Errors in file /home/oracle/admin/orcl/bdump/orcl2_mmon_20730.trc:
Thu Jun 2 08:23:35 2011
Errors in file /home/oracle/admin/orcl/bdump/orcl2_mmon_20730.trc:
ORA-27509: IPC error receiving a message
ORA-27300: OS system dependent operation:recvmsg failed with status: 88
ORA-27301: OS failure message: Socket operation on non-socket
ORA-27302: failure occurred at: sskgxprcv1
Thu Jun 2 08:23:58 2011
IPC Send timeout detected. Receiver ospid 21099
Thu Jun 2 08:23:58 2011
Errors in file /home/oracle/admin/orcl/udump/orcl2_ora_21099.trc:
ORA-27505: IPC error destroying a port
ORA-27300: OS system dependent operation:close failed with status: 9
ORA-27301: OS failure message: Bad file descriptor
ORA-27302: failure occurred at: skgxpdelpt1
ORA-01013: user requested cancel of current operation
Thu Jun 2 08:42:46 2011
Thread 2 advanced to log sequence 13195 (LGWR switch)
Current log# 3 seq# 13195 mem# 0: /oradata/orcl/redo031.log
Current log# 3 seq# 13195 mem# 1: /oradata/orcl/redo032.log
Thu Jun 2 09:05:14 2011
Restarting dead background process MMON
MMON started with pid=19, OS id=8779
Thu Jun 2 09:05:50 2011
ospid 20693: network interface with IP address 192.168.0.2 is now running
針對ORA-27302: failure occurred at: skgxpdelpt1，查了下MOS，沒任何線索
一般這個錯和OS有關，讓客戶檢查了下OS日誌，發現一直報告file-max使用完了
Jun 2 05:57:06 orcljsapp kernel: VFS: file-max li0 reached
Jun 2 05:57:06 orcljsapp kernel: VFS: file-max limit 150000 reached
Jun 2 05:57:06 orcljsapp last message repeated 106 times
Jun 2 05:57:06 orcljsapp kernel: VFS: file-max li0 reached
Jun 2 05:57:06 orcljsapp kernel: VFS: file-max limit 150000 reached
Jun 2 05:57:06 orcljsapp last message repeated 106 times
Jun 2 05:57:06 orcljsapp kernel: VFS: file-max li0 reached
Jun 2 05:57:06 orcljsapp kernel: VFS: file-max limit 150000 reached
150000的控制程式碼數還是很高了，首先想到了一個10203的問題 File handles not released after upgrade to 10.2.0.3 CRS Bundle#2 or 10.2.0.4
可是該問題是HPUX下的，客戶機器為LINUX
讓客戶先確認下那個使用者開啟的控制程式碼多
[root@orcljsapp ~]# lsof -u jmssssftp|wc -l
125295
[root@orcljsapp ~]# lsof -u oracle|wc -l
4455
[root@orcljsapp ~]# lsof -u lpost|wc -l
31293
[root@orcljsapp ~]# lsof -u mqm|wc -l
6829
[root@orcljsapp ~]# lsof -u webftp|wc -l
0
[root@orcljsapp ~]# lsof -u orclwebdz|wc -l
0
[root@orcljsapp ~]# lsof -u monitor|wc -l
623
[root@orcljsapp ~]# lsof -u zhiban|wc -l
0
[root@orcljsapp ~]# lsof -u startting|wc -l
0
[root@orcljsapp ~]# lsof -u tuxedo|wc -l
0
[root@orcljsapp ~]# lsof -u zbping|wc -l
0
jmssssftp開啟了12W，恐怖啊
[root@orcljsapp print]# lsof -u jmssssftp|more
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
sftp-serv 301 jmssssftp cwd DIR 8,3 933888 3473820 /home/lpost/print
sftp-serv 301 jmssssftp rtd DIR 104,2 4096 2 /
sftp-serv 301 jmssssftp txt REG 104,2 36424 475351 /usr/libexec/openssh/sftp-server
sftp-serv 301 jmssssftp mem REG 104,2 105320 1197091 /lib64/ld-2.3.4.so
sftp-serv 301 jmssssftp mem REG 104,2 1230232 1197099 /lib64/libcrypto.so.0.9.7a
sftp-serv 301 jmssssftp mem REG 104,2 17431 1197097 /lib64/libutil-2.3.4.so
sftp-serv 301 jmssssftp mem REG 104,2 79336 753665 /usr/lib64/libz.so.1.2.1.2
sftp-serv 301 jmssssftp mem REG 104,2 107451 1197101 /lib64/libnsl-2.3.4.so
sftp-serv 301 jmssssftp mem REG 104,2 30134 1197068 /lib64/libcrypt-2.3.4.so
sftp-serv 301 jmssssftp mem REG 104,2 91548 1196034 /lib64/libresolv-2.3.4.so
sftp-serv 301 jmssssftp mem REG 104,2 62520 1196084 /lib64/libselinux.so.1
sftp-serv 301 jmssssftp mem REG 104,2 93832 760150 /usr/lib64/libgssapi_krb5.so.2.2
sftp-serv 301 jmssssftp mem REG 104,2 464040 760149 /usr/lib64/libkrb5.so.3.2
sftp-serv 301 jmssssftp mem REG 104,2 145456 388914 /usr/lib64/libk5crypto.so.3.0
sftp-serv 301 jmssssftp mem REG 104,2 10384 1196123 /lib64/libcom_err.so.2.1
sftp-serv 301 jmssssftp mem REG 104,2 1499873 1197092 /lib64/tls/libc-2.3.4.so
sftp-serv 301 jmssssftp mem REG 104,2 18039 1197094 /lib64/libdl-2.3.4.so
sftp-serv 301 jmssssftp 0u unix 0x0000010476e09640 185609988 socket
sftp-serv 301 jmssssftp 1u unix 0x0000010476e09640 185609988 socket
sftp-serv 301 jmssssftp 2u unix 0x0000010476e09c40 185609990 socket
sftp-serv 301 jmssssftp 3u unix 0x0000010476e09640 185609988 socket
sftp-serv 301 jmssssftp 4u unix 0x0000010476e09640 185609988 socket
sshd 302 jmssssftp cwd DIR 104,2 4096 2 /
sshd 302 jmssssftp rtd DIR 104,2 4096 2 /
sshd 302 jmssssftp txt REG 104,2 351288 386292 /usr/sbin/sshd
sshd 302 jmssssftp mem REG 104,2 105320 1197091 /lib64/ld-2.3.4.so
sshd 302 jmssssftp mem REG 104,2 35176 382450 /usr/lib64/libwrap.so.0.7.6
sshd 302 jmssssftp mem REG 104,2 38400 1197105 /lib64/libpam.so.0.77
sshd 302 jmssssftp mem REG 104,2 18039 1197094 /lib64/libdl-2.3.4.so
sshd 302 jmssssftp mem REG 104,2 1230232 1197099 /lib64/libcrypto.so.0.9.7a
sshd 302 jmssssftp mem REG 104,2 17431 1197097 /lib64/libutil-2.3.4.so
sshd 302 jmssssftp mem REG 104,2 79336 753665 /usr/lib64/libz.so.1.2.1.2
sshd 302 jmssssftp mem REG 104,2 107451 1197101 /lib64/libnsl-2.3.4.so
sshd 302 jmssssftp mem REG 104,2 30134 1197068 /lib64/libcrypt-2.3.4.so
sshd 302 jmssssftp mem REG 104,2 91548 1196034 /lib64/libresolv
估計是FTP有什麼問題
Thu Jun 2 06:02:39 2011
Errors in file /home/oracle/admin/orcl/udump/orcl2_ora_21099.trc:
ORA-01013: user requested cancel of current operation
Thu Jun 2 06:02:40 2011
Errors in file /home/oracle/admin/orcl/udump/orcl2_ora_21099.trc:
ORA-27509: IPC error receiving a message
ORA-27300: OS system dependent operation:recvmsg failed with status: 88
ORA-27301: OS failure message: Socket operation on non-socket
ORA-27302: failure occurred at: sskgxprcv1
ORA-01013: user requested cancel of current operation
Thu Jun 2 08:22:11 2011
Detected change in CPU count to 16
Thu Jun 2 08:22:11 2011
Resource Manager cpu_count => Low Threshold (20) : High Threshold (24)
Thu Jun 2 08:22:49 2011
ospid 20693: network interface with IP address 192.168.0.2 is now running
Thu Jun 2 08:23:34 2011
IPC Send timeout detected. Receiver ospid 20730
Thu Jun 2 08:23:34 2011
Errors in file /home/oracle/admin/orcl/bdump/orcl2_mmon_20730.trc:
Thu Jun 2 08:23:35 2011
Errors in file /home/oracle/admin/orcl/bdump/orcl2_mmon_20730.trc:
ORA-27509: IPC error receiving a message
ORA-27300: OS system dependent operation:recvmsg failed with status: 88
ORA-27301: OS failure message: Socket operation on non-socket
ORA-27302: failure occurred at: sskgxprcv1
Thu Jun 2 08:23:58 2011
IPC Send timeout detected. Receiver ospid 21099
Thu Jun 2 08:23:58 2011
Errors in file /home/oracle/admin/orcl/udump/orcl2_ora_21099.trc:
ORA-27505: IPC error destroying a port
ORA-27300: OS system dependent operation:close failed with status: 9
ORA-27301: OS failure message: Bad file descriptor
ORA-27302: failure occurred at: skgxpdelpt1
ORA-01013: user requested cancel of current operation
Thu Jun 2 08:42:46 2011
Thread 2 advanced to log sequence 13195 (LGWR switch)
Current log# 3 seq# 13195 mem# 0: /oradata/orcl/redo031.log
Current log# 3 seq# 13195 mem# 1: /oradata/orcl/redo032.log
Thu Jun 2 09:05:14 2011
Restarting dead background process MMON
MMON started with pid=19, OS id=8779
Thu Jun 2 09:05:50 2011
ospid 20693: network interface with IP address 192.168.0.2 is now running
針對ORA-27302: failure occurred at: skgxpdelpt1，查了下MOS，沒任何線索
一般這個錯和OS有關，讓客戶檢查了下OS日誌，發現一直報告file-max使用完了
Jun 2 05:57:06 orcljsapp kernel: VFS: file-max li0 reached
Jun 2 05:57:06 orcljsapp kernel: VFS: file-max limit 150000 reached
Jun 2 05:57:06 orcljsapp last message repeated 106 times
Jun 2 05:57:06 orcljsapp kernel: VFS: file-max li0 reached
Jun 2 05:57:06 orcljsapp kernel: VFS: file-max limit 150000 reached
Jun 2 05:57:06 orcljsapp last message repeated 106 times
Jun 2 05:57:06 orcljsapp kernel: VFS: file-max li0 reached
Jun 2 05:57:06 orcljsapp kernel: VFS: file-max limit 150000 reached
150000的控制程式碼數還是很高了，首先想到了一個10203的問題 File handles not released after upgrade to 10.2.0.3 CRS Bundle#2 or 10.2.0.4
可是該問題是HPUX下的，客戶機器為LINUX
讓客戶先確認下那個使用者開啟的控制程式碼多
[root@orcljsapp ~]# lsof -u jmssssftp|wc -l
125295
[root@orcljsapp ~]# lsof -u oracle|wc -l
4455
[root@orcljsapp ~]# lsof -u lpost|wc -l
31293
[root@orcljsapp ~]# lsof -u mqm|wc -l
6829
[root@orcljsapp ~]# lsof -u webftp|wc -l
0
[root@orcljsapp ~]# lsof -u orclwebdz|wc -l
0
[root@orcljsapp ~]# lsof -u monitor|wc -l
623
[root@orcljsapp ~]# lsof -u zhiban|wc -l
0
[root@orcljsapp ~]# lsof -u startting|wc -l
0
[root@orcljsapp ~]# lsof -u tuxedo|wc -l
0
[root@orcljsapp ~]# lsof -u zbping|wc -l
0
jmssssftp開啟了12W，恐怖啊
[root@orcljsapp print]# lsof -u jmssssftp|more
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
sftp-serv 301 jmssssftp cwd DIR 8,3 933888 3473820 /home/lpost/print
sftp-serv 301 jmssssftp rtd DIR 104,2 4096 2 /
sftp-serv 301 jmssssftp txt REG 104,2 36424 475351 /usr/libexec/openssh/sftp-server
sftp-serv 301 jmssssftp mem REG 104,2 105320 1197091 /lib64/ld-2.3.4.so
sftp-serv 301 jmssssftp mem REG 104,2 1230232 1197099 /lib64/libcrypto.so.0.9.7a
sftp-serv 301 jmssssftp mem REG 104,2 17431 1197097 /lib64/libutil-2.3.4.so
sftp-serv 301 jmssssftp mem REG 104,2 79336 753665 /usr/lib64/libz.so.1.2.1.2
sftp-serv 301 jmssssftp mem REG 104,2 107451 1197101 /lib64/libnsl-2.3.4.so
sftp-serv 301 jmssssftp mem REG 104,2 30134 1197068 /lib64/libcrypt-2.3.4.so
sftp-serv 301 jmssssftp mem REG 104,2 91548 1196034 /lib64/libresolv-2.3.4.so
sftp-serv 301 jmssssftp mem REG 104,2 62520 1196084 /lib64/libselinux.so.1
sftp-serv 301 jmssssftp mem REG 104,2 93832 760150 /usr/lib64/libgssapi_krb5.so.2.2
sftp-serv 301 jmssssftp mem REG 104,2 464040 760149 /usr/lib64/libkrb5.so.3.2
sftp-serv 301 jmssssftp mem REG 104,2 145456 388914 /usr/lib64/libk5crypto.so.3.0
sftp-serv 301 jmssssftp mem REG 104,2 10384 1196123 /lib64/libcom_err.so.2.1
sftp-serv 301 jmssssftp mem REG 104,2 1499873 1197092 /lib64/tls/libc-2.3.4.so
sftp-serv 301 jmssssftp mem REG 104,2 18039 1197094 /lib64/libdl-2.3.4.so
sftp-serv 301 jmssssftp 0u unix 0x0000010476e09640 185609988 socket
sftp-serv 301 jmssssftp 1u unix 0x0000010476e09640 185609988 socket
sftp-serv 301 jmssssftp 2u unix 0x0000010476e09c40 185609990 socket
sftp-serv 301 jmssssftp 3u unix 0x0000010476e09640 185609988 socket
sftp-serv 301 jmssssftp 4u unix 0x0000010476e09640 185609988 socket
sshd 302 jmssssftp cwd DIR 104,2 4096 2 /
sshd 302 jmssssftp rtd DIR 104,2 4096 2 /
sshd 302 jmssssftp txt REG 104,2 351288 386292 /usr/sbin/sshd
sshd 302 jmssssftp mem REG 104,2 105320 1197091 /lib64/ld-2.3.4.so
sshd 302 jmssssftp mem REG 104,2 35176 382450 /usr/lib64/libwrap.so.0.7.6
sshd 302 jmssssftp mem REG 104,2 38400 1197105 /lib64/libpam.so.0.77
sshd 302 jmssssftp mem REG 104,2 18039 1197094 /lib64/libdl-2.3.4.so
sshd 302 jmssssftp mem REG 104,2 1230232 1197099 /lib64/libcrypto.so.0.9.7a
sshd 302 jmssssftp mem REG 104,2 17431 1197097 /lib64/libutil-2.3.4.so
sshd 302 jmssssftp mem REG 104,2 79336 753665 /usr/lib64/libz.so.1.2.1.2
sshd 302 jmssssftp mem REG 104,2 107451 1197101 /lib64/libnsl-2.3.4.so
sshd 302 jmssssftp mem REG 104,2 30134 1197068 /lib64/libcrypt-2.3.4.so
sshd 302 jmssssftp mem REG 104,2 91548 1196034 /lib64/libresolv
估計是FTP有什麼問題
來自 “ ITPUB部落格 ” ，連結：http://blog.itpub.net/8242091/viewspace-696933/，如需轉載，請註明出處，否則將追究法律責任。
相關文章
- ORA-27302 “failure occurred at skgxpvfymmtu” in RACAI
- ORA-27302: failure occurred at: skgpwinit6AI
- 例項重啟 - ORA-27302: failure occurred at: sskgxpsnd1AI
- oracle alert日誌檔案報錯ORA-27302: failure occurred at: skgpspawn3OracleAI
- ORA-27154,ORA-27300,ORA-27301,ORA-27302: failure occurred at: sskgpcreatesAI
- EBS R12 - Exception occurred: java.sql.SQLException: OAUTH marshaling failureExceptionJavaSQLOAuthAI
- 練習使用list failure，advise failure; repair failure;AI
- Failure is not fatal, but failure to change might be.AI
- design for failureAI
- An out of memory error has occurred.Error
- Assertion failure 錯誤AI
- Ext success failure 呼叫AI
- DRIVER_POWER_STATE_FAILUREAI
- Vagrant：NS_ERROR_FAILUREErrorAI
- Assertion failure in -[UICollectionViewData validateLayoutInRect:]AIUIView
- An internal error has occurred. Java heap spaceErrorJava
- An error occurred during bosboot verification processingErrorboot
- MyEclipse提示Errors occurred during the buildEclipseErrorUI
- backup: 0511-432 A write error occurredError
- ORA-32700: error occurred in DIAG Group ServiceError
- How do you find that an operation mode switch occurred?
- Error: Debug Failure at typeToStringErrorAI
- Preferred Read Failure GroupsAI
- [Linux] kernel: page allocation failureLinuxAI
- [virtualbox] temporary failure in name resolutionAI
- ORA-29516: Aurora assertion failure: Assertion failure at joez.c:3311AI
- 使用CocoaPods出錯，[!] Oh no, an error occurred.Error
- ASM Failure Group的一點理解ASMAI
- datafilecopy header validation failure problemHeaderAI
- Failure at scls_scr_create with code 1AI
- ORA-29702: error occurred in Cluster Group Service operationError
- an error has occurred. - no valid devices were found...Errordev
- An error has occurred when creating this preference page.Error
- Oracle ASM (10) - ASM中優先讀failure group配置ASM_PREFERRED_READ_FAILURE_GROUPSOracleASMAI
- Oracle ASM (11) - ASM中優先讀failure group配置ASM_PREFERRED_READ_FAILURE_GROUPSOracleASMAI
- mysql 報錯 Communications link failureMySqlAI
- Assertion failure in -[UITableView -configureCellForDisplay:forIndexPath:]AIUIViewIndex
- Failure at final check of Oracle CRS stack.AIOracle