PRKP-1001 CRS-1028 CRS-0223: Resource 'ora.RACDB.RACDB1.inst' has placement error解決辦法

lhyvsxman發表於2010-05-24

Correcting our RAC problem

We have seen the CRS commands and syntax, now lets look at the practical aspect of it. I was having some trouble while running my RAC on RHEL4 (I think the problem primarily arose because I supplied my virtual machines with inadequate memory. When I noticed that I was getting some alerts on my SoRAC tool, I looked at the status of my RAC.

oracle@vmora01rh4 ~]$ cd /u01/app/oracle/product/10.2.0/crs/bin
[oracle@vmora01rh4 bin]$ crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora.fokerac.db application    ONLINE    ONLINE    vmora02rh4
ora....c1.inst application    ONLINE    ONLINE    vmora01rh4
ora....c2.inst application    OFFLINE   UNKNOWN   vmora02rh4
ora....serv.cs application    ONLINE    ONLINE    vmora02rh4
ora....ac1.srv application    ONLINE    ONLINE    vmora01rh4
ora....ac2.srv application    ONLINE    OFFLINE
ora....SM1.asm application    ONLINE    ONLINE    vmora01rh4
ora....H4.lsnr application    ONLINE    ONLINE    vmora01rh4
ora....rh4.gsd application    ONLINE    UNKNOWN   vmora01rh4
ora....rh4.ons application    ONLINE    UNKNOWN   vmora01rh4
ora....rh4.vip application    ONLINE    ONLINE    vmora01rh4
ora....SM2.asm application    OFFLINE   UNKNOWN   vmora02rh4
ora....H4.lsnr application    OFFLINE   UNKNOWN   vmora02rh4
ora....rh4.gsd application    ONLINE    UNKNOWN   vmora02rh4
ora....rh4.ons application    OFFLINE   UNKNOWN   vmora02rh4
ora....rh4.vip application    ONLINE    ONLINE    vmora02rh4

As you can see above, some of the applications are UNKNOWN or OFFLINE, either of which is not good for my RAC.

The crs_stat command gives you the names of the applications, which you might need to shut down some applications manually, in order to shut the whole cluster down and restart it.

[oracle@vmora01rh4 bin]$ crs_stat
NAME=ora.fokerac.db
TYPE=application
TARGET=ONLINE
STATE=ONLINE on vmora02rh4
NAME=ora.fokerac.fokerac1.inst
TYPE=application
TARGET=ONLINE
STATE=ONLINE on vmora01rh4
NAME=ora.fokerac.fokerac2.inst
TYPE=application
TARGET=OFFLINE
STATE=OFFLINE
NAME=ora.fokerac.fokeserv.cs
TYPE=application
TARGET=ONLINE
STATE=ONLINE on vmora02rh4
NAME=ora.fokerac.fokeserv.fokerac1.srv
TYPE=application
TARGET=ONLINE
STATE=ONLINE on vmora01rh4
NAME=ora.fokerac.fokeserv.fokerac2.srv
TYPE=application
TARGET=ONLINE
STATE=OFFLINE
NAME=ora.vmora01rh4.ASM1.asm
TYPE=application
TARGET=ONLINE
STATE=ONLINE on vmora01rh4
NAME=ora.vmora01rh4.LISTENER_VMORA01RH4.lsnr
TYPE=application
TARGET=ONLINE
STATE=ONLINE on vmora01rh4
NAME=ora.vmora01rh4.gsd
TYPE=application
TARGET=ONLINE
STATE=UNKNOWN on vmora01rh4
NAME=ora.vmora01rh4.ons
TYPE=application
TARGET=ONLINE
STATE=UNKNOWN on vmora01rh4
NAME=ora.vmora01rh4.vip
TYPE=application
TARGET=ONLINE
STATE=ONLINE on vmora01rh4
NAME=ora.vmora02rh4.ASM2.asm
TYPE=application
TARGET=OFFLINE
STATE=UNKNOWN on vmora02rh4
NAME=ora.vmora02rh4.LISTENER_VMORA02RH4.lsnr
TYPE=application
TARGET=OFFLINE
STATE=UNKNOWN on vmora02rh4
NAME=ora.vmora02rh4.gsd
TYPE=application
TARGET=ONLINE
STATE=UNKNOWN on vmora02rh4
NAME=ora.vmora02rh4.ons
TYPE=application
TARGET=OFFLINE
STATE=UNKNOWN on vmora02rh4
NAME=ora.vmora02rh4.vip
TYPE=application
TARGET=ONLINE
STATE=ONLINE on vmora02rh4

I could have also attempted to stop them all using crs_stop –all, but it normally throws enough errors to force you do it manually one by one.

[oracle@vmora01rh4 bin]$ crs_stop -all
Attempting to stop `ora.vmora01rh4.ons` on member `vmora01rh4`
Attempting to stop `ora.vmora02rh4.ons` on member `vmora02rh4`
`ora.vmora02rh4.ons` on member `vmora02rh4` has experienced an unrecoverable failure.
Human intervention required to resume its availability.
Stop of `ora.vmora01rh4.ons` on member `vmora01rh4` succeeded.
Attempting to stop `ora.vmora01rh4.ASM1.asm` on member `vmora01rh4`
Attempting to stop `ora.fokerac.fokerac2.inst` on member `vmora02rh4`
`ora.fokerac.fokerac2.inst` on member `vmora02rh4` has experienced an unrecoverable failure.
Human intervention required to resume its availability.
Attempting to stop `ora.vmora02rh4.ASM2.asm` on member `vmora02rh4`
`ora.vmora02rh4.ASM2.asm` on member `vmora02rh4` has experienced an unrecoverable failure.
Human intervention required to resume its availability.
Attempting to stop `ora.vmora02rh4.LISTENER_VMORA02RH4.lsnr` on member `vmora02rh4`
`ora.vmora02rh4.LISTENER_VMORA02RH4.lsnr` on member `vmora02rh4` has experienced an unrecoverable failure.
Human intervention required to resume its availability.
Attempting to stop `ora.fokerac.fokerac2.inst` on member `vmora02rh4`
`ora.fokerac.fokerac2.inst` on member `vmora02rh4` has experienced an unrecoverable failure.
Human intervention required to resume its availability.
Attempting to stop `ora.vmora02rh4.ASM2.asm` on member `vmora02rh4`
`ora.vmora02rh4.ASM2.asm` on member `vmora02rh4` has experienced an unrecoverable failure.
Human intervention required to resume its availability.
Attempting to stop `ora.vmora02rh4.vip` on member `vmora02rh4`
Stop of `ora.vmora02rh4.vip` on member `vmora02rh4` succeeded.
Stop of `ora.vmora01rh4.ASM1.asm` on member `vmora01rh4` succeeded.
Attempting to stop `ora.vmora01rh4.LISTENER_VMORA01RH4.lsnr` on member `vmora01rh4`
Stop of `ora.vmora01rh4.LISTENER_VMORA01RH4.lsnr` on member `vmora01rh4` succeeded.
Attempting to stop `ora.vmora01rh4.vip` on member `vmora01rh4`
Stop of `ora.vmora01rh4.vip` on member `vmora01rh4` succeeded.
CRS-0216: Could not stop resource 'ora.vmora02rh4.ASM2.asm'.
CRS-0216: Could not stop resource 'ora.vmora02rh4.ons'.
CRS-0216: Could not stop resource 'ora.vmora02rh4.vip'.

For the very same reason we will go ahead and do it our way. Therefore, we need to stop our instances first.

[oracle@vmora01rh4 bin]$ srvctl stop instance -d fokerac -i fokerac1
[oracle@vmora01rh4 bin]$ srvctl stop instance -d fokerac -i fokerac2

Check our status

[oracle@vmora01rh4 bin]$ crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora.fokerac.db application    OFFLINE   OFFLINE
ora....c1.inst application    OFFLINE   OFFLINE
ora....c2.inst application    OFFLINE   OFFLINE
ora....serv.cs application    ONLINE    UNKNOWN   vmora02rh4
ora....ac1.srv application    OFFLINE   OFFLINE
ora....ac2.srv application    OFFLINE   OFFLINE
ora....SM1.asm application    OFFLINE   OFFLINE
ora....H4.lsnr application    OFFLINE   OFFLINE
ora....rh4.gsd application    ONLINE    UNKNOWN   vmora01rh4
ora....rh4.ons application    OFFLINE   OFFLINE
ora....rh4.vip application    OFFLINE   OFFLINE
ora....SM2.asm application    OFFLINE   UNKNOWN   vmora02rh4
ora....H4.lsnr application    OFFLINE   UNKNOWN   vmora02rh4
ora....rh4.gsd application    ONLINE    UNKNOWN   vmora02rh4
ora....rh4.ons application    OFFLINE   UNKNOWN   vmora02rh4
ora....rh4.vip application    OFFLINE   OFFLINE

Stop the service

[oracle@vmora01rh4 bin]$ srvctl stop service -d fokerac -s fokeserv

Check status again

[oracle@vmora01rh4 bin]$ crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora.fokerac.db application    OFFLINE   OFFLINE
ora....c1.inst application    OFFLINE   OFFLINE
ora....c2.inst application    OFFLINE   OFFLINE
ora....serv.cs application    OFFLINE   OFFLINE
ora....ac1.srv application    OFFLINE   OFFLINE
ora....ac2.srv application    OFFLINE   OFFLINE
ora....SM1.asm application    OFFLINE   OFFLINE
ora....H4.lsnr application    OFFLINE   OFFLINE
ora....rh4.gsd application    ONLINE    UNKNOWN   vmora01rh4
ora....rh4.ons application    OFFLINE   OFFLINE
ora....rh4.vip application    OFFLINE   OFFLINE
ora....SM2.asm application    OFFLINE   UNKNOWN   vmora02rh4
ora....H4.lsnr application    OFFLINE   UNKNOWN   vmora02rh4
ora....rh4.gsd application    ONLINE    UNKNOWN   vmora02rh4
ora....rh4.ons application    OFFLINE   UNKNOWN   vmora02rh4
ora....rh4.vip application    OFFLINE   OFFLINE

OK, so we need to stop those applications now.

[oracle@vmora01rh4 bin]$ crs_stop ora.vmora01rh4.gsd
Attempting to stop `ora.vmora01rh4.gsd` on member `vmora01rh4`
Stop of `ora.vmora01rh4.gsd` on member `vmora01rh4` succeeded.
[oracle@vmora01rh4 bin]$ crs_stop ora.vmora02rh4.ASM2.asm
Attempting to stop `ora.vmora02rh4.ASM2.asm` on member `vmora02rh4`
Stop of `ora.vmora02rh4.ASM2.asm` on member `vmora02rh4` succeeded.
[oracle@vmora01rh4 bin]$ crs_stop ora.vmora02rh4.LISTENER_VMORA02RH4.lsnr
Attempting to stop `ora.vmora02rh4.LISTENER_VMORA02RH4.lsnr` on member `vmora02rh4`
Stop of `ora.vmora02rh4.LISTENER_VMORA02RH4.lsnr` on member `vmora02rh4` succeeded.
[oracle@vmora01rh4 bin]$ crs_stop ora.vmora02rh4.gsd
Attempting to stop `ora.vmora02rh4.gsd` on member `vmora02rh4`
Stop of `ora.vmora02rh4.gsd` on member `vmora02rh4` succeeded.
[oracle@vmora01rh4 bin]$ crs_stop ora.vmora02rh4.ons
Attempting to stop `ora.vmora02rh4.ons` on member `vmora02rh4`
Stop of `ora.vmora02rh4.ons` on member `vmora02rh4` succeeded.

Check status

[oracle@vmora01rh4 bin]$ crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora.fokerac.db application    OFFLINE   OFFLINE
ora....c1.inst application    OFFLINE   OFFLINE
ora....c2.inst application    OFFLINE   OFFLINE
ora....serv.cs application    OFFLINE   OFFLINE
ora....ac1.srv application    OFFLINE   OFFLINE
ora....ac2.srv application    OFFLINE   OFFLINE
ora....SM1.asm application    OFFLINE   OFFLINE
ora....H4.lsnr application    OFFLINE   OFFLINE
ora....rh4.gsd application    OFFLINE   OFFLINE
ora....rh4.ons application    OFFLINE   OFFLINE
ora....rh4.vip application    OFFLINE   OFFLINE
ora....SM2.asm application    OFFLINE   OFFLINE
ora....H4.lsnr application    OFFLINE   OFFLINE
ora....rh4.gsd application    OFFLINE   OFFLINE
ora....rh4.ons application    OFFLINE   OFFLINE
ora....rh4.vip application    OFFLINE   OFFLINE

OK all set , now lets bring them all online.

[oracle@vmora01rh4 bin]$ crs_start -all
Attempting to start `ora.vmora02rh4.vip` on member `vmora02rh4`
Attempting to start `ora.vmora01rh4.vip` on member `vmora01rh4`
Start of `ora.vmora02rh4.vip` on member `vmora02rh4` succeeded.
Start of `ora.vmora01rh4.vip` on member `vmora01rh4` succeeded.
Attempting to start `ora.vmora01rh4.ASM1.asm` on member `vmora01rh4`
Attempting to start `ora.vmora02rh4.ASM2.asm` on member `vmora02rh4`
Start of `ora.vmora02rh4.ASM2.asm` on member `vmora02rh4` succeeded.
Attempting to start `ora.fokerac.fokerac2.inst` on member `vmora02rh4`
Start of `ora.vmora01rh4.ASM1.asm` on member `vmora01rh4` succeeded.
Attempting to start `ora.fokerac.fokerac1.inst` on member `vmora01rh4`
Start of `ora.fokerac.fokerac2.inst` on member `vmora02rh4` succeeded.
Attempting to start `ora.vmora02rh4.LISTENER_VMORA02RH4.lsnr` on member `vmora02rh4`
Start of `ora.fokerac.fokerac1.inst` on member `vmora01rh4` succeeded.
Attempting to start `ora.vmora01rh4.LISTENER_VMORA01RH4.lsnr` on member `vmora01rh4`
Start of `ora.vmora02rh4.LISTENER_VMORA02RH4.lsnr` on member `vmora02rh4` succeeded.
Start of `ora.vmora01rh4.LISTENER_VMORA01RH4.lsnr` on member `vmora01rh4` succeeded.
CRS-1002: Resource 'ora.vmora02rh4.ons' is already running on member 'vmora02rh4'
CRS-1002: Resource 'ora.vmora01rh4.ons' is already running on member 'vmora01rh4'
Attempting to start `ora.fokerac.fokeserv.fokerac1.srv` on member `vmora01rh4`
Attempting to start `ora.vmora01rh4.gsd` on member `vmora01rh4`
Attempting to start `ora.fokerac.db` on member `vmora01rh4`
Attempting to start `ora.fokerac.fokeserv.fokerac2.srv` on member `vmora02rh4`
Attempting to start `ora.fokerac.fokeserv.cs` on member `vmora02rh4`
Attempting to start `ora.vmora02rh4.gsd` on member `vmora02rh4`
Start of `ora.fokerac.fokeserv.fokerac2.srv` on member `vmora02rh4` succeeded.
Start of `ora.fokerac.fokeserv.cs` on member `vmora02rh4` succeeded.
Start of `ora.fokerac.db` on member `vmora01rh4` succeeded.
Start of `ora.vmora02rh4.gsd` on member `vmora02rh4` succeeded.
Start of `ora.vmora01rh4.gsd` on member `vmora01rh4` succeeded.
Start of `ora.fokerac.fokeserv.fokerac1.srv` on member `vmora01rh4` succeeded.
*CRS-0223: Resource 'ora.vmora01rh4.ons' has placement error.
CRS-0223: Resource 'ora.vmora02rh4.ons' has placement error.

*Don’t bother about those errors, as they just did not report back to us in the sequence they were started by the clusterware.

[oracle@vmora01rh4 bin]$ crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora.fokerac.db application    ONLINE    ONLINE    vmora01rh4
ora....c1.inst application    ONLINE    ONLINE    vmora01rh4
ora....c2.inst application    ONLINE    ONLINE    vmora02rh4
ora....serv.cs application    ONLINE    ONLINE    vmora02rh4
ora....ac1.srv application    ONLINE    ONLINE    vmora01rh4
ora....ac2.srv application    ONLINE    ONLINE    vmora02rh4
ora....SM1.asm application    ONLINE    ONLINE    vmora01rh4
ora....H4.lsnr application    ONLINE    ONLINE    vmora01rh4
ora....rh4.gsd application    ONLINE    ONLINE    vmora01rh4
ora....rh4.ons application    ONLINE    ONLINE    vmora01rh4
ora....rh4.vip application    ONLINE    ONLINE    vmora01rh4
ora....SM2.asm application    ONLINE    ONLINE    vmora02rh4
ora....H4.lsnr application    ONLINE    ONLINE    vmora02rh4
ora....rh4.gsd application    ONLINE    ONLINE    vmora02rh4
ora....rh4.ons application    ONLINE    ONLINE    vmora02rh4
ora....rh4.vip application    ONLINE    ONLINE    vmora02rh4
[oracle@vmora01rh4 bin]$

Conclusion:

The power and strength of a command or any other tool lies not in its intelligence, it lies in our ability to do some effective troubleshooting with them. This article has discussed how the CRS commands can be used to fix/troubleshoot the quirkiness of our RAC.

[@more@]

來自 “ ITPUB部落格 ” ,連結:http://blog.itpub.net/12180666/viewspace-1033872/,如需轉載,請註明出處,否則將追究法律責任。

相關文章