Oracle RAC root.sh 報錯 Timed out waiting for the CRS stack to start 解決方法
一．問題描述
在Oracle Linux 6.1 上安裝11.2.0.1的RAC，在第二個節點執行root.sh時，報time out，如下：
[root@rac2 ~]# /u01/app/11.2.0/grid/root.sh
Running Oracle 11g root.sh script...
The following environment variables are setas:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/11.2.0/grid
Enter the full pathname of the local bindirectory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Entries will be added to the /etc/oratabfile as needed by
Database Configuration Assistant when adatabase is created
Finished running generic part of root.shscript.
Now product-specific root actions will beperformed.
2012-06-27 14:46:35: Parsing the host name
2012-06-27 14:46:35: Checking for superuser privileges
2012-06-27 14:46:35: User has super userprivileges
Using configuration parameter file:/u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp'root'..
Operation successful.
Adding daemon to inittab
CRS-4123: Oracle High Availability Serviceshas been started.
ohasd is starting
ADVM/ACFS is not supported onoraclelinux-release-6Server-1.0.2.x86_64
CRS-4402: The CSS daemon was started inexclusive mode but found an active CSS daemon on node rac1, number 1, and isterminating
An active cluster was found duringexclusive startup, restarting to join the cluster
CRS-2672: Attempting to start 'ora.mdnsd'on 'rac2'
CRS-2676: Start of 'ora.mdnsd' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.gipcd'on 'rac2'
CRS-2676: Start of 'ora.gipcd' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.gpnpd'on 'rac2'
CRS-2676: Start of 'ora.gpnpd' on 'rac2'succeeded
CRS-2672: Attempting to start'ora.cssdmonitor' on 'rac2'
CRS-2676: Start of 'ora.cssdmonitor' on'rac2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on'rac2'
CRS-2672: Attempting to start 'ora.diskmon'on 'rac2'
CRS-2676: Start of 'ora.diskmon' on 'rac2'succeeded
CRS-2676: Start of 'ora.cssd' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.ctssd'on 'rac2'
CRS-2676: Start of 'ora.ctssd' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.asm' on'rac2'
CRS-2676: Start of 'ora.asm' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.crsd' on'rac2'
CRS-2676: Start of 'ora.crsd' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.evmd' on'rac2'
CRS-2676: Start of 'ora.evmd' on 'rac2'succeeded
Timed outwaiting for the CRS stack to start.
檢視相關的狀態：
[oracle@rac1 bin]$ ./crsctl check cluster-all
**************************************************************
rac1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Servicesis online
CRS-4533: EventManager is online
[oracle@rac2 bin]$ ./crsctl check cluster -all
**************************************************************
rac2:
CRS-4535: Cannot communicate with ClusterReady Services
CRS-4529: Cluster Synchronization Servicesis online
CRS-4533: EventManager is online
[oracle@rac1 bin]$ ./crs_stat -t -v
Name Type R/RA F/FT Target State Host
----------------------------------------------------------------------
ora.DATA.dg ora....up.type 0/5 0/ ONLINE ONLINE rac1
ora....N1.lsnr ora....er.type 0/5 0/0 ONLINE ONLINE rac1
ora.asm ora.asm.type 0/5 0/ ONLINE ONLINE rac1
ora.eons ora.eons.type 0/3 0/ ONLINE ONLINE rac1
ora.gsd ora.gsd.type 0/5 0/ OFFLINE OFFLINE
ora....network ora....rk.type 0/5 0/ ONLINE ONLINE rac1
ora.oc4j ora.oc4j.type 0/5 0/0 OFFLINE OFFLINE
ora.ons ora.ons.type 0/3 0/ ONLINE ONLINE rac1
ora....SM1.asm application 0/5 0/0 ONLINE ONLINE rac1
ora.rac1.gsd application 0/5 0/0 OFFLINE OFFLINE
ora.rac1.ons application 0/3 0/0 ONLINE ONLINE rac1
ora.rac1.vip ora....t1.type 0/0 0/0 ONLINE ONLINE rac1
ora.scan1.vip ora....ip.type 0/0 0/0 ONLINE ONLINE rac1
[oracle@rac2 bin]$ ./crs_stat -t -v
CRS-0184: Cannot communicate with the CRSdaemon.
[oracle@rac2 bin]$
在節點2上的命令沒有成功執行的。
二．MOS 上的說明
Root.Sh Failing with 'Prom_rpc: Clsc SendFailure..Ret Code 6' [ID 745215.1]
2.1 Symptoms
During CRS install root.sh fails on thelast node with follow message:
Waiting for the Oracle CRSD and EVMD tostart
Waiting for the Oracle CRSD and EVMD to start
Waiting for the Oracle CRSD and EVMD to start
Waiting for the Oracle CRSD and EVMD to start
Waiting for the Oracle CRSD and EVMD to start
Timed out waiting for the CRS stack to start.
The crsd.log on the first node shows:
2008-10-21 21:04:55.087: [OCRMSG][1325496672]prom_rpc: CLSC send failure..ret
code 6
2008-10-21 21:04:55.087: [ OCRMSG][1325496672]prom_rpc: possible OCRretry
scenario
2008-10-21 21:04:55.087: [ OCRSRV][1325496672]proas_forward_request:PROM_TIM
E_OUT or Master Fail
2008-10-21 21:04:55.296: [ COMMCRS][2540957248]clscsendx: (0xc43bd0)Connection
not active
2008-10-21 21:04:55.296: [OCRMSG][2540957248]prom_rpc: CLSC send failure..ret
code 6
2008-10-21 21:04:55.296: [ OCRMSG][2540957248]prom_rpc: possible OCRretry
scenario
2008-10-21 21:04:55.296: [ OCRCLI][2540957248]proac_open_key:[SYSTEM.crs.debug.
ist4-db1-3-sfm.COMMNS]: Writer failed. Retval [203]
The ocssd.log on the first node shows:
2008-10-21 20:33:01.626: [OCRAPI][2540958848]procr_open: Node Failure.
Attempting retry #0
2008-10-21 20:33:02.628: [ OCRAPI][2540958848]procr_open: Node Failure.
Attempting retry #1
2008-10-21 20:33:03.631: [ OCRAPI][2540958848]procr_open: Node Failure.
Attempting retry #2
The ocssd.log on the last node shows:
[ CSSD]2008-10-22 04:46:42.457 [1241577824]>TRACE: clssnmRcfgMgrThread:
lastleader(1) unique(1224650474)
[ CSSD]2008-10-22 04:46:43.219 [1168148832] >TRACE:clssnmSendVoteInfo:
node(1) syncSeqNo(4)
[ CSSD]2008-10-22 04:46:56.338 [2537955008] >ERROR: clssgmStartNMMon:
timed out waiting on nested NM reconfig. Self-sacrificing to kick othersawake.
[ CSSD]2008-10-22 04:46:56.338 [2537955008] >ERROR: StartCMMon():
clssnmNMDetach failed - 2
[ CSSD]2008-10-22 04:46:56.338 [2537955008] >TRACE: clssscctx: dump of
0x0x5d2360, len 3792
2.2 Cause
This is due to a failure of communicationbetween crsd.bin on nodes.
2.3 Solution
Check the network for the following items:
- Check to see that there is No firewall between the nodes
- Make sure that the MTU size is same.
- If MTU is larger than 1500, then the switch must be able to support largerMTU size.
- Make sure that you have disabled SELINUX
- Make sure that NICs are using full duplex and not auto negotiate.
- Misconfiguration on the switches will also cause this issue.
三．解決方法
在MOS的文件裡提示的原因和防火牆，時間，SELINUX，網路卡型別有關，基本可以確定就是和網路卡相關的原因導致這類問題，我的的原因是是2個節點的網路卡名稱不一致，所以修改網路卡名一致後，嘗試重新執行一下root.sh 命令。
即修改之前：rac1 是eth0和eth1，節點2是：eth5和eth6. 怎麼修改網路卡名，這個google一下，這裡不做說明。
解除安裝之前的操作，命令如下：
/u01/app/11.2.0/grid/crs/install/rootcrs.pl-deconfig -verbose -force
注意這裡，Oracle11g與10g中命令的區別。
--解除安裝：
[root@rac2 ~]#/u01/app/11.2.0/grid/crs/install/rootcrs.pl -deconfig -verbose -force
2012-06-27 15:12:30: Parsing the host name
2012-06-27 15:12:30: Checking for superuser privileges
2012-06-27 15:12:30: User has super userprivileges
Using configuration parameter file:/u01/app/11.2.0/grid/crs/install/crsconfig_params
PRCR-1035 : Failed to look up CRS resourceora.cluster_vip.type for 1
PRCR-1068 : Failed to query resources
Cannot communicate with crsd
PRCR-1070 : Failed to check if resourceora.gsd is registered
Cannot communicate with crsd
PRCR-1070 : Failed to check if resourceora.ons is registered
Cannot communicate with crsd
PRCR-1070 : Failed to check if resourceora.eons is registered
Cannot communicate with crsd
ADVM/ACFS is not supported onoraclelinux-release-6Server-1.0.2.x86_64
ACFS-9201: Not Supported
CRS-2791: Starting shutdown of Oracle HighAvailability Services-managed resources on 'rac2'
CRS-2673: Attempting to stop 'ora.mdnsd' on'rac2'
CRS-2673: Attempting to stop 'ora.gpnpd' on'rac2'
CRS-2673: Attempting to stop'ora.cssdmonitor' on 'rac2'
CRS-2673: Attempting to stop 'ora.ctssd' on'rac2'
CRS-2673: Attempting to stop 'ora.evmd' on'rac2'
CRS-2673: Attempting to stop 'ora.asm' on'rac2'
CRS-2677: Stop of 'ora.cssdmonitor' on 'rac2'succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'rac2'succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'rac2'succeeded
CRS-2677: Stop of 'ora.evmd' on 'rac2'succeeded
CRS-2677: Stop of 'ora.ctssd' on 'rac2'succeeded
CRS-2677: Stop of 'ora.asm' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on'rac2'
CRS-2677: Stop of 'ora.cssd' on 'rac2'succeeded
CRS-2673: Attempting to stop 'ora.diskmon'on 'rac2'
CRS-2673: Attempting to stop 'ora.gipcd' on'rac2'
CRS-2677: Stop of 'ora.gipcd' on 'rac2'succeeded
CRS-2677: Stop of 'ora.diskmon' on 'rac2'succeeded
CRS-2793: Shutdown of Oracle HighAvailability Services-managed resources on 'rac2' has completed
CRS-4133: Oracle High Availability Serviceshas been stopped.
error: package cvuqdisk is not installed
Successfully deconfigured Oracleclusterware stack on this node
--重新執行root.sh，這次成功。
[root@rac2 ~]# /u01/app/11.2.0/grid/root.sh
Running Oracle 11g root.sh script...
The following environment variables are setas:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/11.2.0/grid
Enter the full pathname of the local bindirectory: [/usr/local/bin]:
The file "dbhome" already existsin /usr/local/bin. Overwrite it? (y/n)
[n]:
The file "oraenv" already existsin /usr/local/bin. Overwrite it? (y/n)
[n]:
The file "coraenv" already existsin /usr/local/bin. Overwrite it? (y/n)
[n]:
Entries will be added to the /etc/oratabfile as needed by
Database Configuration Assistant when adatabase is created
Finished running generic part of root.shscript.
Now product-specific root actions will beperformed.
2012-06-27 16:21:25: Parsing the host name
2012-06-27 16:21:25: Checking for superuser privileges
2012-06-27 16:21:25: User has super userprivileges
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp'root'..
Operation successful.
Adding daemon to inittab
CRS-4123: Oracle High Availability Serviceshas been started.
ohasd is starting
ADVM/ACFS is not supported onoraclelinux-release-6Server-1.0.2.x86_64
CRS-4402: The CSS daemon was started inexclusive mode but found an active CSS daemon on node rac1, number 1, and isterminating
An active cluster was found duringexclusive startup, restarting to join the cluster
CRS-2672: Attempting to start 'ora.mdnsd'on 'rac2'
CRS-2676: Start of 'ora.mdnsd' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.gipcd'on 'rac2'
CRS-2676: Start of 'ora.gipcd' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.gpnpd'on 'rac2'
CRS-2676: Start of 'ora.gpnpd' on 'rac2'succeeded
CRS-2672: Attempting to start'ora.cssdmonitor' on 'rac2'
CRS-2676: Start of 'ora.cssdmonitor' on'rac2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on'rac2'
CRS-2672: Attempting to start 'ora.diskmon'on 'rac2'
CRS-2676: Start of 'ora.diskmon' on 'rac2'succeeded
CRS-2676: Start of 'ora.cssd' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.ctssd'on 'rac2'
CRS-2676: Start of 'ora.ctssd' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.asm' on'rac2'
CRS-2676: Start of 'ora.asm' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.crsd' on'rac2'
CRS-2676: Start of 'ora.crsd' on 'rac2'succeeded
CRS-2672: Attempting to start 'ora.evmd' on'rac2'
CRS-2676: Start of 'ora.evmd' on 'rac2'succeeded
rac2 2012/06/27 16:25:16 /u01/app/11.2.0/grid/cdata/rac2/backup_20120627_162516.olr
Preparing packages for installation...
cvuqdisk-1.0.7-1
Configure Oracle Grid Infrastructure for aCluster ... succeeded
Updating inventory properties forclusterware
Starting Oracle Universal Installer...
Checking swap space: must be greater than500 MB. Actual 999 MB Passed
The inventory pointer is located at/etc/oraInst.loc
The inventory is located at/u01/app/oraInventory
[root@rac2 ~]#
驗證：
[oracle@rac2 bin]$ ./crsctl check cluster-all
**************************************************************
rac1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Servicesis online
CRS-4533: Event Manager is online
**************************************************************
rac2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Servicesis online
CRS-4533: Event Manager is online
**************************************************************
[oracle@rac2 bin]$ ./crs_stat -t -v
Name Type R/RA F/FT Target State Host
----------------------------------------------------------------------
ora.DATA.dg ora....up.type 0/5 0/ ONLINE ONLINE rac1
ora....N1.lsnr ora....er.type 0/5 0/0 ONLINE ONLINE rac1
ora.asm ora.asm.type 0/5 0/ ONLINE ONLINE rac1
ora.eons ora.eons.type 0/3 0/ ONLINE ONLINE rac1
ora.gsd ora.gsd.type 0/5 0/ OFFLINE OFFLINE
ora....network ora....rk.type 0/5 0/ ONLINE ONLINE rac1
ora.oc4j ora.oc4j.type 0/5 0/0 OFFLINE OFFLINE
ora.ons ora.ons.type 0/3 0/ ONLINE ONLINE rac1
ora....SM1.asm application 0/5 0/0 ONLINE ONLINE rac1
ora.rac1.gsd application 0/5 0/0 OFFLINE OFFLINE
ora.rac1.ons application 0/3 0/0 ONLINE ONLINE rac1
ora.rac1.vip ora....t1.type 0/0 0/0 ONLINE ONLINE rac1
ora....SM2.asm application 0/5 0/0 ONLINE ONLINE rac2
ora.rac2.gsd application 0/5 0/0 OFFLINE OFFLINE
ora.rac2.ons application 0/3 0/0 ONLINE ONLINE rac2
ora.rac2.vip ora....t1.type 0/0 0/0 ONLINE ONLINE rac2
ora.scan1.vip ora....ip.type 0/0 0/0 ONLINE ONLINE rac1
[oracle@rac2 bin]$
Root.sh 執行成功。
來自 “ ITPUB部落格 ” ，連結：http://blog.itpub.net/29096438/viewspace-1731703/，如需轉載，請註明出處，否則將追究法律責任。
相關文章
- oracle11gR2 Timed out waiting for the CRS stack to startOracleAI
- grid安裝執行root.sh時Timed out waiting for the CRS stack to start - 解除安裝gridAI
- 報錯（已解決）Command timed out after no timeout
- [排錯]安裝Oracle 10g RAC報Failure at final check of Oracle CRS stack 10錯誤Oracle 10gAI
- 10g rac安裝crs,執行root.sh報錯
- RHEL 7 安裝oracle rac 11.2.0.4執行root.sh報錯ohasd failed to startOracleAI
- 安裝Oracle cluster報錯 Failure at final check of Oracle CRS Stack 10OracleAI
- Root.sh failed at Failure at final check of Oracle CRS stack 10 問題AIOracle
- Oracle 11gR2 RAC ohasd failed to start 解決方法OracleAI
- RAC: SRVCTL and VIPCA 命令報錯解決方法PCA
- 安裝RAC 執行root.sh指令碼報錯，解決辦法指令碼
- Nginx 報錯 504 Gateway Time-out 的解決方法NginxGateway
- pip安裝selenium報錯：Read timed out
- java.net.SocketTimeoutException: Read timed out異常解決方法JavaException
- root.sh最後出現"failiure at final check of oracle CRS stack 10"的問題AIOracle
- rac中 crsctl start/stop crs and crsctl start/stop cluster 區別
- Oracle Enterprise Linux 5.6下安裝Oracle 10g RAC執行root.sh報錯問題解決LinuxOracle 10g
- Failed to connect to ESP8266: Timed out waiting for packet headerAIHeader
- oracle10g rac(rhel4)__CRS-0215_srvctl start出錯一則Oracle
- Oracle RAC啟動CRS報錯：登陸許可權問題Oracle
- [Clickhouse] Clickhouse 報SQLException : Read timed outSQLException
- Detailed Item Cost Report (XML) timed out waiting for the Output Post-processor to finishAIXML
- RAC解決單節點報 CRS-4047: No Oracle Clusterware components configured.Oracle
- Failure at final check of Oracle CRS stack.AIOracle
- 【解決】io.lettuce.core.RedisCommandTimeoutException: Command timed outRedisException
- 解決 connect to host github.com port 22 operation timed outGithub
- 【rac故障】root.sh報錯Unable to get VIP info for new node
- shut down and start crs for Oracle10GOracle
- 啟動Amoeba報The stack size specified is too small解決方法
- java.io.IOException: Timed out waiting 20000ms for a quorum of nodes to respondJavaExceptionAI
- oracle 11.2.0.3 grid ons 程式 checked timed outOracle
- 【Oracle】 inbound connection timed out (ORA-3136)Oracle
- AIX5.3安裝 ORACLE 10.2 RAC 執行root.sh出現lsdb: Cannot allocate memory of size 0 錯誤的解決方法AIOracle
- hive使用報錯解決方法Hive
- 手動清除Oracle 10g RAC CRS的方法Oracle 10g
- open-falcon ---安裝Dashboard時候報錯"SSLError: The read operation timed out"Error
- adstrtal.sh報超時錯誤 ERROR : Timed out( 100000 ): Interrupted ExceptionErrorException
- io.lettuce.core.RedisCommandTimeoutException: Command timed out 解決辦法RedisException