安裝Oracle11.2 RAC for Solaris10 sparc64（四）

簡單記錄一下Solaris 10上Oracle 11.2 RAC環境的安裝過程。

這一篇描述安裝碰到的問題。

安裝Oracle11.2 RAC for Solaris10 sparc64（一）：http://yangtingkun.itpub.net/post/468/515361

安裝Oracle11.2 RAC for Solaris10 sparc64（二）：http://yangtingkun.itpub.net/post/468/515415

安裝Oracle11.2 RAC for Solaris10 sparc64（三）：http://yangtingkun.itpub.net/post/468/515473

第一個問題是沒有配置private網路卡導致的。

由於對11.2的安裝不是很熟悉，看了半天的文件，似乎Oracle可以自動設定vip、private ip，因此在第一次安裝過程中，就可以處理這塊。

安裝到最後執行root.sh時報錯：

# /data/grid/root.sh
Running Oracle 11g root.sh script...

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /data/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...

Creating /var/opt/oracle/oratab file...
Entries will be added to the /var/opt/oracle/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
2010-12-10 17:35:55: Parsing the host name
2010-12-10 17:35:55: Checking for super user privileges
2010-12-10 17:35:55: User has super user privileges
Using configuration parameter file: /data/grid/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
  root wallet
  root wallet cert
  root cert export
  peer wallet
  profile reader wallet
  pa wallet
  peer wallet keys
  pa wallet keys
  peer cert request
  pa cert request
  peer cert
  pa cert
  peer root cert TP
  profile reader root cert TP
  pa root cert TP
  peer pa cert TP
  pa peer cert TP
  profile reader pa cert TP
  profile reader peer cert TP
  peer user cert
  pa user cert
Adding daemon to inittab
CRS-4123: Oracle High Availability Services has been started.
ohasd is starting
CRS-2672: 嘗試啟動 'ora.gipcd' (在 'racnode1' 上)
CRS-2672: 嘗試啟動 'ora.mdnsd' (在 'racnode1' 上)
CRS-2676: 成功啟動 'ora.gipcd' (在 'racnode1' 上)
CRS-2676: 成功啟動 'ora.mdnsd' (在 'racnode1' 上)
CRS-2672: 嘗試啟動 'ora.gpnpd' (在 'racnode1' 上)
CRS-2676: 成功啟動 'ora.gpnpd' (在 'racnode1' 上)
CRS-2672: 嘗試啟動 'ora.cssdmonitor' (在 'racnode1' 上)
CRS-2676: 成功啟動 'ora.cssdmonitor' (在 'racnode1' 上)
CRS-2672: 嘗試啟動 'ora.cssd' (在 'racnode1' 上)
CRS-2672: 嘗試啟動 'ora.diskmon' (在 'racnode1' 上)
CRS-2676: 成功啟動 'ora.diskmon' (在 'racnode1' 上)
CRS-2676: 成功啟動 'ora.cssd' (在 'racnode1' 上)
CRS-2672: 嘗試啟動 'ora.ctssd' (在 'racnode1' 上)
Start action for daemon aborted
CRS-2674: 未能啟動 'ora.ctssd' (在 'racnode1' 上)
CRS-2679: 嘗試清除 'ora.ctssd' (在 'racnode1' 上)
CRS-2681: 成功清除 'ora.ctssd' (在 'racnode1' 上)
CRS-4000: Command Start failed, or completed with errors.
Command return code of 1 (256) from command: /data/grid/bin/crsctl start resource ora.ctssd -init
Start of resource "ora.ctssd -init" failed
Clusterware exclusive mode start of resource ora.ctssd failed
CRS-2500: 由於資源 'ora.crsd' 未執行, 無法停止該資源
CRS-4000: Command Stop failed, or completed with errors.
Command return code of 1 (256) from command: /data/grid/bin/crsctl stop resource ora.crsd -init
Stop of resource "ora.crsd -init" failed
Failed to stop CRSD
CRS-2500: 由於資源 'ora.asm' 未執行, 無法停止該資源
CRS-4000: Command Stop failed, or completed with errors.
Command return code of 1 (256) from command: /data/grid/bin/crsctl stop resource ora.asm -init
Stop of resource "ora.asm -init" failed
Failed to stop ASM
CRS-2673: 嘗試停止 'ora.cssdmonitor' (在 'racnode1' 上)
CRS-2677: 成功停止 'ora.cssdmonitor' (在 'racnode1' 上)
CRS-2673: 嘗試停止 'ora.cssd' (在 'racnode1' 上)
CRS-2677: 成功停止 'ora.cssd' (在 'racnode1' 上)
CRS-2673: 嘗試停止 'ora.gpnpd' (在 'racnode1' 上)
CRS-2677: 成功停止 'ora.gpnpd' (在 'racnode1' 上)
CRS-2673: 嘗試停止 'ora.gipcd' (在 'racnode1' 上)
CRS-2677: 成功停止 'ora.gipcd' (在 'racnode1' 上)
CRS-2673: 嘗試停止 'ora.mdnsd' (在 'racnode1' 上)
CRS-2677: 成功停止 'ora.mdnsd' (在 'racnode1' 上)
Initial cluster configuration failed.  See /data/grid/cfgtoollogs/crsconfig/rootcrs_racnode1.log for details

檢查對應的trace檔案：

2010-12-10 17:38:43: CRS-2676: 成功啟動 'ora.cssd' (在 'racnode1' 上)
2010-12-10 17:38:43: Querying for existing CSS voting disks
2010-12-10 17:38:43: Performing initial configuration for cluster
2010-12-10 17:48:50: Command return code of 1 (256) from command: /data/grid/bin/crsctl start resource ora.ctssd -init
2010-12-10 17:48:50: Start of resource "ora.ctssd -init" failed
2010-12-10 17:48:50: Clusterware exclusive mode start of resource ora.ctssd failed
2010-12-10 17:48:50: Exiting exclusive mode
2010-12-10 17:48:50: Command return code of 1 (256) from command: /data/grid/bin/crsctl stop resource ora.crsd -init
2010-12-10 17:48:50: Stop of resource "ora.crsd -init" failed
2010-12-10 17:48:50: Failed to stop CRSD
2010-12-10 17:48:51: Command return code of 1 (256) from command: /data/grid/bin/crsctl stop resource ora.asm -init
2010-12-10 17:48:51: Stop of resource "ora.asm -init" failed
2010-12-10 17:48:51: Failed to stop ASM
2010-12-10 17:49:03: Initial cluster configuration failed.  See /data/grid/cfgtoollogs/crsconfig/rootcrs_racnode1.log for details

檔案的最後包含下列的資訊。

雖然Oracle可以自動配置private ip，但是由於Private ip所使用的網路卡根本沒有載入，Oracle顯然不會自動進行這個操作，導致安裝報錯。

節點1上啟用第二網路卡：

bash-3.00# ifconfig ce1 plumb
bash-3.00# ifconfig ce1 10.0.0.1 netmask 255.255.255.0 broadcast 10.0.0.255 up
# vi /etc/hostname.ce1
racnode1-priv

在節點2上啟用第二網路卡：

# ifconfig ce1 plumb
# ifconfig ce1 10.0.0.2 netmask 255.255.255.0 broadcast 10.0.0.255 up

# vi /etc/hostname.ce1
racnode2-priv

並將第二網路卡的配置資訊新增到/etc/hosts檔案中：

10.0.0.1        racnode1-priv
10.0.0.2        racnode2-priv

刪除兩個節點上的ocr配置：

# rm /var/opt/oracle/ocr.loc

刪除/data/grid目錄下所有物件，將/data/grid目錄修改為oracle所有：

# rm -rf /data/grid
# chown -R oracle:oinstall /data

刪除產品資訊目錄：

$ rm -rf /data/oraInventory

清除兩個節點上啟動的cluster相關程式：

# ps -ef|grep grid
    root 11584 11567   0 09:41:14 pts/1       0:00 grep grid
    root  4027     1   0  12月 10 ?          13:02 /data/grid/bin/ohasd.bin reboot
# kill -9 4027

清除兩個節點上/var/opt/oracle目錄下的配置資訊：

# rm -rf /var/opt/oracle/*

重新啟動cluster安裝程式，問題消失。