ORA-15055: unable to connect to ASM instance 造成 RAC的db不能啟動
$ crsctl status res ora.racdb.db
NAME=ora.racdb.db
TYPE=ora.database.type
TARGET=ONLINE , ONLINE
STATE=OFFLINE, OFFLINE
啟動crs，發現db服務沒起來，如是嘗試單獨啟動
$ crs_start ora.racdb.db
Attempting to start `ora.racdb.db` on member `rac01`
Attempting to start `ora.racdb.db` on member `rac02`
ORA-00205: error in identifying control file, check alert log for more info
Start of `ora.racdb.db` on member `rac01` failed.
Attempting to stop `ora.racdb.db` on member `rac01`
ORA-00205: error in identifying control file, check alert log for more info
ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
Linux-x86_64 Error: 2: No such file or directory
Process ID: 0
Session ID: 0 Serial number: 0
Start of `ora.racdb.db` on member `rac02` failed.
Attempting to stop `ora.racdb.db` on member `rac02`
Stop of `ora.racdb.db` on member `rac01` succeeded.
CRS-2632: There are no more servers to try to place resource 'ora.racdb.db' on that would satisfy its placement policy
ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
Linux-x86_64 Error: 2: No such file or directory
Process ID: 0
Session ID: 0 Serial number: 0
Stop of `ora.racdb.db` on member `rac02` succeeded.
CRS-0223: Resource 'ora.racdb.db 1 1' has placement error.
CRS-0215: Could not start resource 'ora.racdb.db 2 1'.
還是不能啟動。
檢查oracle警告日誌
# tail -200 /app/oracle/product/oracle/diag/rdbms/racdb/racdb1/trace/alert_racdb1.log
lmon registered with NM - instance number 1 (internal mem no 0)
***********************************************************************
Fatal NI connect error 12547, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=beq)(PROGRAM=/app/grid/11.2.0/bin/oracle)(ARGV0=oracle+ASM1_asmb_racdb1)(ENVS='ORACLE_HOME=/app/grid/11.2.0,ORACLE_SID=+ASM1')(ARGS='(DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))'))(enable=setuser)(CONNECT_DATA=(CID=(PROGRAM=oracle@rac01)(HOST=rac01)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 11.2.0.1.0 - Production
Oracle Bequeath NT Protocol Adapter for Linux: Version 11.2.0.1.0 - Production
Time: 08-OCT-2013 14:38:07
Tracing not turned on.
Tns error struct:
ns main err code: 12547
TNS-12547: TNS:lost contact
ns secondary err code: 12560
nt main err code: 517
TNS-00517: Lost contact
nt secondary err code: 32
nt OS err code: 0
ERROR: Failed to connect with connect string: (DESCRIPTION=(ADDRESS=(PROTOCOL=beq)(PROGRAM=/app/grid/11.2.0/bin/oracle)(ARGV0=oracle+ASM1_asmb_racdb1)(ENVS='ORACLE_HOME=/app/grid/11.2.0,ORACLE_SID=+ASM1')(ARGS='(DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))'))(enable=setuser))
Errors in file /app/oracle/product/oracle/diag/rdbms/racdb/racdb1/trace/racdb1_asmb_7238.trc:
ORA-15055: unable to connect to ASM instance
ORA-12547: TNS:lost contact
Reconfiguration started (old inc 0, new inc 4)
List of instances:
1 2 (myinst: 1)
Global Resource Directory frozen
* allocate domain 0, invalid = TRUE
Communication channels reestablished
Master broadcasted resource hash value bitmaps
Non-local Process blocks cleaned out
LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
Set master node info
Submitted all remote-enqueue requests
Dwn-cvts replayed, VALBLKs dubious
All grantable enqueues granted
Post SMON to start 1st pass IR
Submitted all GCS remote-cache requests
Post SMON to start 1st pass IR
Fix write in gcs resources
Reconfiguration complete
Tue Oct 08 14:38:08 2013
LCK0 started with pid=23, OS id=7254
Starting background process RSMN
Tue Oct 08 14:38:08 2013
RSMN started with pid=28, OS id=7257
ORACLE_BASE not set in environment. It is recommended
that ORACLE_BASE be set in the environment
Reusing ORACLE_BASE from an earlier startup = /app/grid/crs
Tue Oct 08 14:38:09 2013
ALTER DATABASE MOUNT
This instance was first to mount
Starting background process ASMB
Tue Oct 08 14:38:09 2013
ASMB started with pid=30, OS id=7267
***********************************************************************
Fatal NI connect error 12547, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=beq)(PROGRAM=/app/grid/11.2.0/bin/oracle)(ARGV0=oracle+ASM1_asmb_racdb1)(ENVS='ORACLE_HOME=/app/grid/11.2.0,ORACLE_SID=+ASM1')(ARGS='(DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))'))(enable=setuser)(CONNECT_DATA=(CID=(PROGRAM=oracle@rac01)(HOST=rac01)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 11.2.0.1.0 - Production
Oracle Bequeath NT Protocol Adapter for Linux: Version 11.2.0.1.0 - Production
Time: 08-OCT-2013 14:38:09
Tracing not turned on.
Tns error struct:
ns main err code: 12547
TNS-12547: TNS:lost contact
ns secondary err code: 12560
nt main err code: 517
TNS-00517: Lost contact
nt secondary err code: 32
nt OS err code: 0
ERROR: Failed to connect with connect string: (DESCRIPTION=(ADDRESS=(PROTOCOL=beq)(PROGRAM=/app/grid/11.2.0/bin/oracle)(ARGV0=oracle+ASM1_asmb_racdb1)(ENVS='ORACLE_HOME=/app/grid/11.2.0,ORACLE_SID=+ASM1')(ARGS='(DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))'))(enable=setuser))
Starting background process ASMB
Tue Oct 08 14:38:09 2013
ASMB started with pid=31, OS id=7280
***********************************************************************
Fatal NI connect error 12547, connecting to:
(DESCRIPTION=(ADDRESS=(PROTOCOL=beq)(PROGRAM=/app/grid/11.2.0/bin/oracle)(ARGV0=oracle+ASM1_asmb_racdb1)(ENVS='ORACLE_HOME=/app/grid/11.2.0,ORACLE_SID=+ASM1')(ARGS='(DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))'))(enable=setuser)(CONNECT_DATA=(CID=(PROGRAM=oracle@rac01)(HOST=rac01)(USER=oracle))))
VERSION INFORMATION:
TNS for Linux: Version 11.2.0.1.0 - Production
Oracle Bequeath NT Protocol Adapter for Linux: Version 11.2.0.1.0 - Production
Time: 08-OCT-2013 14:38:09
Tracing not turned on.
Tns error struct:
ns main err code: 12547
TNS-12547: TNS:lost contact
ns secondary err code: 12560
nt main err code: 517
TNS-00517: Lost contact
nt secondary err code: 32
nt OS err code: 0
ERROR: Failed to connect with connect string: (DESCRIPTION=(ADDRESS=(PROTOCOL=beq)(PROGRAM=/app/grid/11.2.0/bin/oracle)(ARGV0=oracle+ASM1_asmb_racdb1)(ENVS='ORACLE_HOME=/app/grid/11.2.0,ORACLE_SID=+ASM1')(ARGS='(DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))'))(enable=setuser))
ORA-00210: cannot open the specified control file
ORA-00202: control file: '+DATA/racdb/control02.ctl'
ORA-17503: ksfdopn:2 Failed to open file +DATA/racdb/control02.ctl
ORA-15001: diskgroup "DATA" does not exist or is not mounted
ORA-15055: unable to connect to ASM instance
ORA-12547: TNS:lost contact
ORA-00210: cannot open the specified control file
ORA-00202: control file: '+DATA/racdb/control01.ctl'
ORA-17503: ksfdopn:2 Failed to open file +DATA/racdb/control01.ctl
ORA-15001: diskgroup "DATA" does not exist or is not mounted
ORA-15055: unable to connect to ASM instance
ORA-12547: TNS:lost contact
Starting background process ASMB
ORA-205 signalled during: ALTER DATABASE MOUNT...
Tue Oct 08 14:38:09 2013
ASMB started with pid=30, OS id=7290
Tue Oct 08 14:38:09 2013
Shutting down instance (abort)
License high water mark = 1
USER (ospid: 7294): terminating the instance
Instance terminated by USER, pid = 7294
Tue Oct 08 14:38:11 2013
Instance shutdown complete
發現 asm 報錯，不能連線。如是，找到asm的日誌
# tail -f /app/grid/crs/diag/asm/+asm/+ASM1/trace/alert_+ASM1.log
...
Tue Oct 08 14:57:17 2013
Errors in file /app/grid/crs/diag/asm/+asm/+ASM1/trace/+ASM1_ora_8661.trc:
ORA-27070: async read/write failed
WARNING: IO Failed. group:0 disk(number.incarnation):0.0xe96ba85e disk_path:/dev/random
AU:0 disk_offset(bytes):0 io_size:4096 operation:Read type:asynchronous
result:I/O error process_id:8661
Errors in file /app/grid/crs/diag/asm/+asm/+ASM1/trace/+ASM1_ora_8661.trc:
ORA-27070: async read/write failed
WARNING: IO Failed. group:0 disk(number.incarnation):0.0xe96ba85e disk_path:/dev/random
AU:0 disk_offset(bytes):0 io_size:4096 operation:Read type:asynchronous
result:I/O error process_id:8661
根據日誌資訊找到報錯的trace檔案。
# tail -100 /app/grid/crs/diag/asm/+asm/+ASM1/trace/+ASM1_ora_8661.trc
...
*** 2013-10-08 14:57:17.778
ORA-27070: async read/write failed
WARNING: IO Failed. group:0 disk(number.incarnation):0.0xe96ba85e disk_path:/dev/random
AU:0 disk_offset(bytes):0 io_size:4096 operation:Read type:asynchronous
result:I/O error process_id:8661
subsys:System iop:0x7fd245f3f108 bufp:0x7fd246078600 osderr:0x0 osderr1:0x0
*** 2013-10-08 14:57:27.144
ORA-27070: async read/write failed
WARNING: IO Failed. group:0 disk(number.incarnation):0.0xe96ba85e disk_path:/dev/random
AU:0 disk_offset(bytes):0 io_size:4096 operation:Read type:asynchronous
result:I/O error process_id:8661
subsys:System iop:0x7fd245f3efe0 bufp:0x7fd246078600 osderr:0x0 osderr1:0x0
貌似是 disk_path:/dev/random 的 IO錯誤。奇怪，怎麼多了這個磁碟，於是切換到grid使用者
SQL> select name,path from v$asm_disk;
NAME PATH
------------------------------ --------------------------------------------------
/dev/random
FRA_0000 /dev/asm-diskd
CRS_0000 /dev/asm-diskb
DATA_0000 /dev/asm-diskc
發現asm disk磁碟裡面多了一條記錄“/dev/random”。
SQL> select name,state,total_mb from v$asm_diskgroup;
NAME STATE TOTAL_MB
------------------------------ ----------- ----------
FRA MOUNTED 51199
CRS MOUNTED 51199
DATA MOUNTED 51199
SQL> select group_number, name, path, total_mb, free_mb, total_mb-free_mb used_mb from v$asm_disk_stat;
GROUP_NUMBER NAME PATH TOTAL_MB FREE_MB USED_MB
------------ ------------------------------ -------------------------------------------------- ---------- ---------- ----------
3 FRA_0000 /dev/asm-diskd 51199 51097 102
1 CRS_0000 /dev/asm-diskb 51199 50803 396
2 DATA_0000 /dev/asm-diskc 51199 45917 5282
經過朋友指點
$ sqlplus / as sysasm
SQL> show parameter disk
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
asm_diskgroups string DATA, FRA
asm_diskstring string /dev
修改第二個引數，把範圍進一步精確。
SQL> alter system set asm_diskstring='/dev/asm-disk*' scope=both sid='+ASM1';
SQL> alter system set asm_diskstring='/dev/asm-disk*' scope=both sid='+ASM2';
修改後，重啟crs服務。分別在2個節點上執行如下：
# /app/grid/11.2.0/bin/crsctl stop crs
# /app/grid/11.2.0/bin/crsctl start crs
自此，不再報關於“/dev/random”磁碟IO錯誤。
來自 “ ITPUB部落格 ” ，連結：http://blog.itpub.net/9399028/viewspace-773919/，如需轉載，請註明出處，否則將追究法律責任。
相關文章
- ORA-15055 Unable to Connect to ASM When Starting an InstanceASM
- ORA-15055 ORA-12549 Unable To Connect To ASM Instance問題ASM
- 【RAC】ORA-15055: unable to connect to ASM instanceORA-12547: TNS:lost contactASM
- Unable To Create Database Using ASM ORA-15055DatabaseASM
- ORA-15055: unable to connect to ASM instanceORA-12547: TNS:lost contact(轉載)ASM
- ASM例項啟動報錯：ORA-29701 :unable to connect to Cluster ManagerASM
- ORA-15055 ASM Diskgroup Status Showing Dismounted From RDBMS InstanceASM
- Linux中兩塊device的minor number相同而造成RAC不能啟動的問題Linuxdev
- 私有IP丟失造成Oracle 12C RAC叢集節點不能啟動Oracle
- Convert a Single-Instance to RAC with ASMASM
- [RAC] Oracle11g RAC禁止DB自動啟動Oracle
- 什麼原因造成資料庫不能自動啟動！資料庫
- 11gR2 RAC ASM 啟動ASM
- RAC DG 物理standby ASM無法啟動ASM
- 11gR2 RAC ASM啟動過程ASM
- 【RAC】啟動/關閉CRS, OHAS, ASM & RDBMS 的步驟ASM
- 管理ASM 檢索ocr資訊 禁用RAC自動啟動ASM
- 【RAC】Diskgroup shows offline after restart even it is mounted in ASM instanceRESTASM
- Unable to connect to SQL Server ExpressSQLServerExpress
- ORA-29701--10g 單機asm不能啟動ASM
- oracle 10g rac資料庫不能自動啟動Oracle 10g資料庫
- RAC節點啟動失敗--ASM無法連線ASM
- 虛擬機器搭建rac ASM盤啟動失敗虛擬機ASM
- 11.2.0.4 RAC CSSD服務無法啟動故障 unable to set priority to 4CSS
- ASM Spfile Is Not Used when ASM Instance StartsASM
- 【ASM】RAC19C因引數設定不當，asm無法啟動ASM
- oracle instance and listener自動啟動Oracle
- Unable to connect to any hosts due to exceptionException
- Oracle RAC 一個節點不能自動啟動 怪問題Oracle
- 【ASM】Oracle RAC css啟動報錯"Duplicate voting file found"ASMOracleCSS
- oracle11g RAC 啟動時無法識別ASMOracleASM
- ORA-01078 ORA-29701 11g單機asm系統啟動後不能啟動asmASM
- RAC建立ASM單個磁碟不能操作2TASM
- terminating the instance due to error481導致ASM無法啟動故障ErrorASM
- oracle 11gR2 asm例項 不能啟動處理方法OracleASM
- Oracle10g RAC 怎麼Listener. CRS , Instance ..... 都是自動開啟 ？Oracle
- SVN-Unable to connect to a repository at URL
- Unable to Connect to Database with Oracle Client Software for WindowsDatabaseOracleclientWindows