oracle9207 rac 升級到9208異常問題

今天晚上在貴州電信升級一套rac，發生異常，資料庫升級到9208以後無法以cluster的方式啟動，在metalink查了好多文件，都沒有能夠匹配的，見了鬼了，最終經過5個小時折騰終於找到了答案。[@more@]

環境：oracle9207 rac 升級到oracle9208 rac

問題現象：升級完成後兩個節點的資料庫無法同時啟動，只能啟動一個，無論先啟動哪個節點，另外一個節點都無法啟動，在mount階段hang

報錯：

node1：Sat Jan 9 03:13:27 2010

starting up 1 shared server(s) ...

starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...

ARCH: STARTING ARCH PROCESSES

ARC0 started with pid=19, OS id=16038

ARC0: Archival started

ARC1 started with pid=20, OS id=16040

Sat Jan 9 03:13:28 2010

ARCH: STARTING ARCH PROCESSES COMPLETE

Sat Jan 9 03:13:28 2010

ARC1: Archival started

Sat Jan 9 03:13:28 2010

ARC0: Becoming the 'no FAL' ARCH

ARC0: Becoming the 'no FAL' ARCHARC0: Thread not mounted

Sat Jan 9 03:13:28 2010

lmon registered with NM - instance id 1 (internal mem no 0)

tossing admin msg

tossing bad msg pid 16008

tossing admin msg

tossing bad msg pid 16008

tossing admin msg

node2：Sat Jan 9 03:14:40 2010

starting up 1 shared server(s) ...

starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...

ARCH: STARTING ARCH PROCESSES

ARC0 started with pid=19, OS id=11307

ARC0: Archival started

ARC1 started with pid=20, OS id=11309

Sat Jan 9 03:14:41 2010

ARC1: Archival started

Sat Jan 9 03:14:41 2010

ARCH: STARTING ARCH PROCESSES COMPLETE

ARC1: Becoming the 'no FAL' ARCH

ARC1: Becoming the 'no FAL' ARCHARC1: Thread not mounted

Sat Jan 9 03:14:41 2010

ARC0: Becoming the heartbeat ARCH

ARC0: Becoming the heartbeat ARCHARC0: Thread not mounted

Sat Jan 9 03:14:41 2010

ALTER DATABASE MOUNT

Sat Jan 9 03:14:41 2010

lmon registered with NM - instance id 2 (internal mem no 1)

Sat Jan 9 03:15:41 2010

ARC0: Thread not mounted

Sat Jan 9 03:16:37 2010

LMON: terminating instance due to error 472

Sat Jan 9 03:16:39 2010

System state dump is made for local instance

Sat Jan 9 03:16:39 2010

Trace dumping is performing id=[cdmp_20100109031639]

Sat Jan 9 03:16:42 2010

Instance terminated by LMON, pid = 11277

主要是tossing admin msg

tossing bad msg pid 16008 和ARC1: Thread not mounted

處理過程：

主要問題是集中在$ORACLE_HOME/lib/libskgxp*檔案有問題

兩個節點執行ls -l $ORACLE_HOME/lib/libskgxp*

發現兩個節點的結果不一致：

原來RAC配置後，libskgxp9.so應該是 libskgxpu.so的複製，而節點1，2的這兩個檔案並不一致，因此分別在兩個節點執行以下步驟：

cd $ORACLE_HOME/lib

cp libskgxpu.so libskgxp9.so

然後重啟資料庫：srvctl start database –d ods

問題解決。

^_^