RAC刪除節點失敗重啟大法解決

近期需要測試一下RAC增加、刪除節點操作。

在LINUX5.7+ORACLE10.2.0.5 RAC上，先是為兩節點RAC增加一個節點，然後再刪除節點2.

在刪除節點2時遇到無法刪除的錯誤，如下資訊，

[root@rac1 install]# ./rootdeletenode.sh rac2,2
CRS-0210: Could not find resource 'ora.rac2.ons'.
CRS-0210: Could not find resource 'ora.rac2.vip'.
CRS-0210: Could not find resource 'ora.rac2.gsd'.
CRS-0210: Could not find resource ora.rac2.vip.
CRS nodeapps are deleted successfully
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
CSS indicates failure 17 acknowledging the deletion.
Attempting to roll back the deletion.
Rollback of node deletion successful.
Node deletion failed but configuration is intact.
Failed in deleting nodes 'rac2,2'

此時節點2的CSS資訊：
[root@rac2 crs]# ps -ef|grep css
root      3348     1  0 18:54 ?        00:00:24 /bin/sh /etc/init.d/init.cssd fatal
root      4356  3348  0 18:54 ?        00:00:00 /bin/sh /etc/init.d/init.cssd oprocd
root      4371  3348  0 18:54 ?        00:00:00 /bin/sh /etc/init.d/init.cssd oclsomon
root      4401  3348  0 18:54 ?        00:00:00 /bin/sh /etc/init.d/init.cssd daemon
root      4902  4371  0 18:54 ?        00:00:00 /sbin/runuser -l oracle -c /bin/sh -c 'cd /u01/app/oracle/product/crs/log/rac2/cssd/oclsomon; ulimit -c unlimited; /u01/app/oracle/product/crs/bin/oclsomon  || exit $?'
oracle    4903  4902  0 18:54 ?        00:00:00 /bin/sh -c cd /u01/app/oracle/product/crs/log/rac2/cssd/oclsomon; ulimit -c unlimited; /u01/app/oracle/product/crs/bin/oclsomon  || exit $?
oracle    4975  4401  0 18:54 ?        00:00:36 /u01/app/oracle/product/crs/bin/ocssd.bin
root     29835  5821  0 22:39 pts/3    00:00:00 grep css

解決辦法：

重啟節點2，此時將要刪除的節點關閉或者重啟，然後重新刪除即可：

[root@rac1 install]# ./rootdeletenode.sh rac2,2
CRS-0210: Could not find resource 'ora.rac2.ons'.
CRS-0210: Could not find resource 'ora.rac2.vip'.
CRS-0210: Could not find resource 'ora.rac2.gsd'.
CRS-0210: Could not find resource ora.rac2.vip.
CRS nodeapps are deleted successfully
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
Successfully deleted 14 values from OCR.
Key SYSTEM.css.interfaces.noderac2 marked for deletion is not there. Ignoring.
Successfully deleted 5 keys from OCR.
Node deletion operation successful.
'rac2,2' deleted successfully