CRS-215 Srvctl unable to start ASM, Listener, RDBMS Resources
In this Document
Applies to:
Oracle Server - Enterprise Edition - Version: 10.2.0.1 to 10.2.0.4Information in this document applies to any platform.
Oracle Server Enterprise Edition - Version: 10.2.0.1 to 10.2.0.4
Symptoms
After patching CRS, RDBMS & ASM ORACLE_HOME, unable to start/stop ASM, Database & Listener resources via srvctl:node68:/u01>srvctl start asm -n node68
PRKS-1009 : Failed to start ASM instance "+ASM4" on node "node68", [PRKS-1009
: Failed to start ASM instance "+ASM4" on node "node68", [CRS-1028: Dependency analysis failed because of:
CRS-0223: Resource 'ora.node68.ASM4.asm' has placement error.]]
[PRKS-1009 : Failed to start ASM instance "+ASM4" on node "node68", [CRS-1028: Dependency analysis failed because of:
CRS-0223: Resource 'ora.node68.ASM4.asm' has placement error.]]
node68:/u01>srvctl stop asm -n node68
Attempting to stop `ora.node68.ASM4.asm` on member `node68`
`ora.node68.ASM4.asm` on member `node68` has experienced an unrecoverable failure.
Human intervention required to resume its availability.
CRS-0216: Could not stop resource 'ora.node68.ASM4.asm'.
node68:/u01>crs_stat -t | grep 68
PRKS-1009 : Failed to start ASM instance "+ASM4" on node "node68", [PRKS-1009
: Failed to start ASM instance "+ASM4" on node "node68", [CRS-1028: Dependency analysis failed because of:
CRS-0223: Resource 'ora.node68.ASM4.asm' has placement error.]]
[PRKS-1009 : Failed to start ASM instance "+ASM4" on node "node68", [CRS-1028: Dependency analysis failed because of:
CRS-0223: Resource 'ora.node68.ASM4.asm' has placement error.]]
node68:/u01>srvctl stop asm -n node68
Attempting to stop `ora.node68.ASM4.asm` on member `node68`
`ora.node68.ASM4.asm` on member `node68` has experienced an unrecoverable failure.
Human intervention required to resume its availability.
CRS-0216: Could not stop resource 'ora.node68.ASM4.asm'.
node68:/u01>crs_stat -t | grep 68
ora....TA4.srv application ONLINE UNKNOWN node68
ora....SM4.asm application ONLINE UNKNOWN node68
ora....68.lsnr application ONLINE UNKNOWN node68
ora....d68.gsd application ONLINE ONLINE node68
ora....d68.ons application ONLINE ONLINE node68
ora....d68.vip application ONLINE ONLINE node68
Starting ASM instance via sqlplus is fine. Starting listener via lsnrctl is fine too.
Changes
CRS, RDBMS & ASM is patched with the latest bundle patch.Cause
While the resource is in an UNKNOWN state, one can not start resource via srvctl command. Need to force resource OFFLINE with command:% $ORA_CRS_HOME/bin/crs_stop -f
E.g.
$ ./crs_stop -f ora.node68ASM4.asm
Attempting to stop `ora.node68.ASM4.asm` on member `node68`
Stop of `ora.node68.ASM4.asm` on member `node68` succeeded
Attempting to stop `ora.node68.ASM4.asm` on member `node68`
Stop of `ora.node68.ASM4.asm` on member `node68` succeeded
Please note the preferred method for management of a resource is through srvctl, any issues with the usage of crs_start & crs_stop should be done under the guidance of Oracle Support.
After this, the real error shows up:
node68:/u01>srvctl start asm -n node68
[PRKS-1009 : Failed to start ASM instance "ASM4" on node "node68",
[node68:ora.node68.ASM4.asm:/u01/oracle/product/10.2.0/asm/bin/racgwrap: %ORACLE_HOME%/bin/racgmain:
not found
CRS-0215: Could not start resource 'ora.node68.ASM4.asm'.]]
[PRKS-1009 : Failed to start ASM instance "ASM4" on node "node68",
[node68:ora.node68.ASM4.asm:/u01/oracle/product/10.2.0/asm/bin/racgwrap: %ORACLE_HOME%/bin/racgmain:
not found
CRS-0215: Could not start resource 'ora.node68.ASM4.asm'.]]
$ORACLE_HOME/bin/racgwrap does not have ORACLE_HOME variable correctly defined.
The cause is actually that scripts like racgwrap weren't modified when the CRS bundle was applied against the ASM/database home.
In most cases this will be a result of not having run the pre & post scripts as per patch README.
Example of Patch# 7715304
% cd
% custom/server/7715304/custom/scripts/prepatch.sh -dbhome
% opatch napply custom/server/ -local -oh -id 7715304
% custom/server/7715304/custom/scripts/postpatch.sh -dbhome
The last step parses all scripts that have been included in the patch, replacing installer variables like %ORACLE_HOME% with the actual path to the ORACLE_HOME. If steps 1 & 3 are omitted any patched scripts may have incorrect values for such variables and subsequently scripts like 'racgwrap' fail causing the status of the associated resource to become 'UNKNOWN'.
% cd
% custom/server/7715304/custom/scripts/prepatch.sh -dbhome
% opatch napply custom/server/ -local -oh -id 7715304
% custom/server/7715304/custom/scripts/postpatch.sh -dbhome
The last step parses all scripts that have been included in the patch, replacing installer variables like %ORACLE_HOME% with the actual path to the ORACLE_HOME. If steps 1 & 3 are omitted any patched scripts may have incorrect values for such variables and subsequently scripts like 'racgwrap' fail causing the status of the associated resource to become 'UNKNOWN'.
Solution
Given that the root cause to this is actually the result of the patch being applied incorrectly you should re-apply the patch following the instructions but first rolling back the patch.Steps..
1. shutdown any instances running out of this RDBMS home, you can do this with sqlplus for each instance. You may also need to place the resource in an OFFLINE state instead of UNKNOWN, can do this with crs_stop -f
E.g.
% cd $ORA_CRS_HOME/bin
% ./crs_stop -f ora.node4.ASM4.asm
This will ensure the state of the resource changes from UNKNOWN status to OFFLINE status.
% cd $ORA_CRS_HOME/bin
% ./crs_stop -f ora.node4.ASM4.asm
This will ensure the state of the resource changes from UNKNOWN status to OFFLINE status.
2. rollback patch 7715304 (as the oracle user) as per README:
E.g.
% opatch rollback -id 7715304 -local -oh
% opatch rollback -id 7715304 -local -oh
3. Re-apply the patch against the RDBMS home as the oracle user:
% cd
% custom/server/7715304/custom/scripts/prepatch.sh -dbhome
% opatch napply custom/server/ -local -oh -id 7715304
% custom/server/7715304/custom/scripts/postpatch.sh -dbhome
% custom/server/7715304/custom/scripts/prepatch.sh -dbhome
% opatch napply custom/server/ -local -oh
% custom/server/7715304/custom/scripts/postpatch.sh -dbhome
4. when done check the script. 'racgwrap' if it now contains the correct path for ORACLE_HOME by running the following command
grep 'ORACLE_HOME=' $ORACLE_HOME/bin/racgwrap
Above needs to be repeated on each node.
來自 “ ITPUB部落格 ” ,連結:http://blog.itpub.net/7199859/viewspace-663441/,如需轉載,請註明出處,否則將追究法律責任。
相關文章
- stop/start The CRS, OHAS, ASM, RDBMS & ACFS Services on RAC 11.2_1355977.1ASM
- 4.5.1.1 srvctl add asmASM
- Unable to start HTTP server after restoreHTTPServerREST
- oracle-rac--srvctl add scan_listenerOracle
- Unable To Start ASM (ORA-838、4031) On 11.2.0.3 If OS CPUs # > 64-1416083.1ASM
- Unable to preventDefault inside passive event listenerIDE
- Troubleshooting when srvctl can't start RAC instance, but sqlplus can start it [ID 844272.1]SQL
- android studio Error:Unable to start the daemon process【轉】AndroidError
- Start of resource “ora.asm -init” failed -ASMAI
- CRS: Resource in UNKNOWN state and srvctl Cannot Start/Stop Resource_845709.1
- oracle asm Unable to mount ASMlib driver filesystemOracleASM
- 【RAC】啟動/關閉CRS, OHAS, ASM & RDBMS 的步驟ASM
- Change AUTO_START in ASM Resource Oracle ClusterwareASMOracle
- springcloud和Eureka整合時報錯:Unable to start embedded TomcatSpringGCCloudTomcat
- the application was unable to start correctly 0xc000007bAPP
- ORA-12500: TNS:listener failed to start a dedicated server processAIServer
- ORA-12500 TNS : listener failed to start a dedicated server processAIServer
- 解決滾動警告:Unable to preventDefault inside passive event listenerIDE
- Tomcat啟動錯誤一例org.apache.catalina.core.StandardContext resources Start Error starting static ResourcesTomcatApacheContextError
- Unable To Create Database Using ASM ORA-15055DatabaseASM
- 啟動Tomcat 失敗(Unable to start cluster)及解決方法Tomcat
- listener start fail TNS-12555 利用trace找到問題AI
- ORA-15055 ASM Diskgroup Status Showing Dismounted From RDBMS InstanceASM
- Connecting to ASM through the TNS Listener from a Client DesktopASMclient
- Unable To Recreate DG Fast Start Failover Configuration With DGMGRL-454418.1ASTAI
- Can't debug c++ project because unable to static library start program *.libC++Project
- oracle10g rac(rhel4)__CRS-0215_srvctl start出錯一則Oracle
- ORA-15055 Unable to Connect to ASM When Starting an InstanceASM
- 建立asm磁碟 報錯 oracleasm-write-label: Unable to clear deviceASMOracledev
- CLSRSC-184 CLSRSC-258: Failed to configure and start ASMAIASM
- ORA-15183 Unable to Create Database on Server using 11.2 ASMDatabaseServerASM
- Unable To Open Database After ASM Upgrade From Release 11.1 To Release 11.2DatabaseASM
- angularjs resourcesAngularJS
- Overview of Unix ResourcesView
- DBCA建立ASM報錯ORA-27125: unable to create shared memory segmentASM
- ASM例項啟動報錯:ORA-29701 :unable to connect to Cluster ManagerASM
- RDBMS VS XML VS NoSQLXMLSQL
- Oracle RDBMS History Over the YearsOracle