簡單描述一下，在Oracle 10203 for Solaris sparc的RAC雙節點環境中，新增一個節點的過程。共享儲存已經在第三個節點上配置完成，這裡主要介紹作業系統上和Oracle上的配置。

簡單描述新增節點的軟體安裝過程中碰到的問題。

Oracle10203RAC環境新增新節點（一）：http://yangtingkun.itpub.net/post/468/493738

Oracle10203RAC環境新增新節點（二）：http://yangtingkun.itpub.net/post/468/493764

Oracle10203RAC環境新增新節點（三）：http://yangtingkun.itpub.net/post/468/493811

Oracle10203RAC環境新增新節點（四）：http://yangtingkun.itpub.net/post/468/493848

第一個錯誤是在執行ADDNODE.SH工具時碰到的，在節點3的安裝過程中出現了空間不足的錯誤：

INFO: ld.so.1: lsnodes: fatal: libskgxn2.so: open failed: No such file or directory
INFO: Vendor clusterware is not detected.
INFO: Assigning node number '3' to node 'racnode3'
INFO: Setting variable 'CLUSTER_NEW_HOST_NAMES' to 'racnode3,'. Received the value from a code block.
INFO: Setting variable 'CLUSTER_NEW_NODE_NUMBERS' to '3,'. Received the value from a code block.
INFO: Initializing OUI access setup
INFO: Performing operation for OUI access setup
INFO:
*** Cluster Node Addition Summary Page***
m_asNodeArray:racnode2,racnode3,racnode1
m_sLocalNode:racnode2
sNativeVolName:/data1/
m_asNodeArray:racnode2,racnode3,racnode1
m_sLocalNode:racnode2
SEVERE: oracle.sysman.oii.oiip.oiipg.OiipgRemoteOpsException: Error in checking free space on path '/data1' on node 'racnode3' .
        at oracle.sysman.oii.oiip.oiipg.OiipgClusterOps.throwRemoteOpsException(OiipgClusterOps.java:2456)
        at oracle.sysman.oii.oiip.oiipg.OiipgClusterOps.getFreeSpaceOnNode(OiipgClusterOps.java:2236)
        at oracle.sysman.oii.oiif.oiifp.OiifpAddNodeSummaryPanel.addAddNodeSpaceInfos(OiifpAddNodeSummaryPanel.java:439)
        at oracle.sysman.oii.oiif.oiifp.OiifpAddNodeSummaryPanel.populateTree(OiifpAddNodeSummaryPanel.java:246)
        at oracle.sysman.oii.oiif.oiifp.OiifpAddNodeSummaryPanel.init(OiifpAddNodeSummaryPanel.java:227)
        at oracle.sysman.oii.oiif.oiifp.OiifpAddNodeSummaryPanel.createPanelElements(OiifpAddNodeSummaryPanel.java:148)
        at oracle.sysman.oii.oiif.oiifm.OiifmGraphicPageHandler.createWizardPanel(OiifmGraphicPageHandler.java:583)
        at oracle.sysman.oii.oiif.oiifm.OiifmGraphicPageHandler.showWizardPage(OiifmGraphicPageHandler.java:241)
        at oracle.sysman.oii.oiif.oiifm.OiifmGraphicPageHandler.processWizardPage(OiifmGraphicPageHandler.java:193)
        at oracle.sysman.oii.oiif.oiifm.OiifmGraphicSummaryPageHandler.processWizardPage(OiifmGraphicSummaryPageHandler.java:90)
        at oracle.sysman.oii.oiif.oiifm.OiifmGraphicInterfaceManager.doWizardPageOperation(OiifmGraphicInterfaceManager.java:352)
        at oracle.sysman.oii.oiif.oiifb.OiifbWizChainDlgElem.doOperation(OiifbWizChainDlgElem.java:690)
        at oracle.sysman.oii.oiif.oiifw.OiifwAddNodeSummaryWCDE.doOperation(OiifwAddNodeSummaryWCDE.java:176)
        at oracle.sysman.oii.oiif.oiifb.OiifbCondIterator.iterate(OiifbCondIterator.java:171)
        at oracle.sysman.oii.oiic.OiicPullSession.doOperation(OiicPullSession.java:1273)
        at oracle.sysman.oii.oiic.OiicSessionWrapper.doOperation(OiicSessionWrapper.java:289)
        at oracle.sysman.oii.oiic.OiicInstaller.run(OiicInstaller.java:546)
        at oracle.sysman.oii.oiic.OiicInstaller.runInstaller(OiicInstaller.java:929)
        at oracle.sysman.oii.oiic.OiicInstaller.main(OiicInstaller.java:866)
Caused by: oracle.ops.mgmt.cluster.ClusterException: PRKC-1031 : Error checking free space for /data1 on racnode3
df: (/data1    ) ????????????????????????????????
        at oracle.ops.mgmt.cluster.ClusterCmd.getFreeSpace(ClusterCmd.java:1697)
        at oracle.sysman.oii.oiip.oiipg.OiipgClusterOps.getFreeSpaceOnNode(OiipgClusterOps.java:2218)
        ... 17 more

解決了空間問題後，圖形安裝工具自動解除了錯誤。

第二個問題是在執行root.sh指令碼的過程中，安裝結束後，需要在新節點上執行root.sh，不過執行出現了下面的錯誤：

# . /data/oracle/product/10.2/crs/root.sh
WARNING: directory '/data/oracle/product/10.2' is not owned by root
WARNING: directory '/data/oracle/product' is not owned by root
WARNING: directory '/data/oracle' is not owned by root
WARNING: directory '/data' is not owned by root
Checking to see if Oracle CRS stack is already configured
Oracle CRS stack is already configured and will be running under init(1M)

檢查後發現，是由於手工複製Oracle目錄的時候，將/var/opt/oracle目錄也複製過來，導致Oracle檢查ocr.loc發現已經配置過CRS了，刪除ocr.loc後，再次執行root.sh指令碼，恢復正常。

# . /data/oracle/product/10.2/crs/root.sh
WARNING: directory '/data/oracle/product/10.2' is not owned by root
WARNING: directory '/data/oracle/product' is not owned by root
WARNING: directory '/data/oracle' is not owned by root
WARNING: directory '/data' is not owned by root
Checking to see if Oracle CRS stack is already configured
OCR LOCATIONS = /dev/rac/ocr
Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
WARNING: directory '/data/oracle/product/10.2' is not owned by root
WARNING: directory '/data/oracle/product' is not owned by root
WARNING: directory '/data/oracle' is not owned by root
WARNING: directory '/data' is not owned by root
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node :
node 1: racnode1 racnode1-priv racnode1
node 2: racnode2 racnode2-priv racnode2
clscfg: Arguments check out successfully.

NO KEYS WERE WRITTEN. Supply -force parameter to override.
-force is destructive and will destroy any previous cluster
configuration.
Oracle Cluster Registry for cluster has already been initialized
Startup will be queued to init within 30 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
        racnode1
        racnode2
        racnode3
CSS is active on all nodes.
Waiting for the Oracle CRSD and EVMD to start
Oracle CRS stack installed and running under init(1M)
Running vipca(silent) for configuring nodeapps
IP 地址 "racnode1-vip" 已被使用。請輸入一個未使用的 IP 地址。

不過這裡又出現了第三個錯誤。由於以前存在的例項現在處於啟動狀態，VIP已經成功分配，所以這裡出現了這個問題，下面用oracle使用者切換為root許可權，啟動vipca進行配置：

$ su root
Password:
# ./vipca

成功配置後，問題解決。

隨後碰到的錯誤仍然和COPY目錄有關，由於手工將/data/oracle/oraInventroy目錄複製到了目標伺服器上，因此無論是cluster軟體還是database軟體安裝過程的attach操作都會報錯：

INFO: Invoking OUI on cluster nodes racnode3
INFO: /data1/data/oracle/product/10.2/crs/oui/bin/../bin/runInstaller -paramFile /data1/data/oracle/product/10.2/crs/oui/bin/../clu
sterparam.ini -silent -ignoreSysPrereqs -attachHome -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs ORACLE_HOME_NAME=Or
aCrs10g_home1 CLUSTER_NODES=racnode1,racnode2,racnode3 CRS=true "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode3
-remoteInvocation -invokingNodeName racnode2 -logFilePath "/data/oracle/oraInventory/logs" -timestamp 2009-01-30_01-01-56AM
SEVERE: oracle.sysman.oii.oiip.oiipg.OiipgRemoteOpsException: Error occured while trying to run Unix command /data1/data/oracle/prod
uct/10.2/crs/oui/bin/../bin/runInstaller -paramFile /data1/data/oracle/product/10.2/crs/oui/bin/../clusterparam.ini -silent -ignor
eSysPrereqs -attachHome -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs ORACLE_HOME_NAME=OraCrs10g_home1 CLUSTER_NODES=r
acnode1,racnode2,racnode3 CRS=true "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode3 -remoteInvocation -invokingNo
deName racnode2 -logFilePath "/data/oracle/oraInventory/logs" -timestamp 2009-01-30_01-01-56AM on nodes racnode3. [/usr/bin/sh: /dat
a1/data/oracle/product/10.2/crs/oui/bin/../bin/runInstaller: ?????? :failed]
        .
        .
        .
        at oracle.sysman.oii.oiic.OiicInstaller.main(OiicInstaller.java:866)
Caused by: oracle.ops.mgmt.cluster.ClusterException: /usr/bin/sh: /data1/data/oracle/product/10.2/crs/oui/bin/../bin/runInstaller: ?
????? :failed
        at oracle.ops.mgmt.cluster.ClusterCmd.runCmd(ClusterCmd.java:1959)
        at oracle.sysman.oii.oiip.oiipg.OiipgClusterRunCmd.runCmdOnUnix(OiipgClusterRunCmd.java:256)
        ... 13 more

SEVERE: Remote 'AttachHome' failed on nodes: 'racnode3'. Refer to '/data/oracle/oraInventory/logs/addNodeActions2009-01-30_01-01-56A
M.log' for details.
You can manually re-run the following command on the failed nodes after the installation:
/data/oracle/product/10.2/crs/oui/bin/runInstaller -attachHome -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs ORACLE_H
OME_NAME=OraCrs10g_home1 CLUSTER_NODES=racnode1,racnode2,racnode3 CRS=true "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NOD
E=.

DATABASE軟體的安裝過程碰到同樣的問題，如果手工執行命令錯誤如下：

$ runInstaller -attachHome -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs ORACLE_HOME_NAME=OraCrs10g_home1 CLUSTER_NODES=racnode1,racnode2,racnode3 CRS=true "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode3
正在啟動 Oracle Universal Installer...

正在檢查安裝程式要求...

檢查作業系統版本: 必須是5.8, 5.9 or 5.10。實際為 5.10
透過

檢查臨時空間: 必須大於 250 MB。實際為 4149 MB 透過
檢查交換空間: 必須大於 500 MB。實際為 4289 MB 透過

所有安裝程式要求均已滿足。

OUI-10197: 無法在 /data/oracle/product/10.2/crs 建立新的 Oracle 主目錄。此位置已存在 Oracle 主目錄。請選擇其它位置。
bash-3.00$ pwd
/data/oracle/product/10.2/crs/oui/bin
bash-3.00$ cd ../../../database/oui/bin/
bash-3.00$ runInstaller -attachHome -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/database ORACLE_HOME_NAME=OraDb10g_home1 CLUSTER_NODES=racnode1,racnode2,racnode3 "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode3
正在啟動 Oracle Universal Installer...

在 oraparam.ini 中未找到先決條件檢查, 不執行系統先決條件檢查。
The inventory pointer is located at /var/opt/oracle/oraInst.loc
The inventory is located at /data/oracle/oraInventory
OUI-10197: 無法在 /data/oracle/product/10.2/database 建立新的 Oracle 主目錄。此位置已存在 Oracle 主目錄。請選擇其它位置。
'AttachHome' 失敗。

這是由於複製的oraInvertory目錄已經包含了CLUSTER和ORACLE軟體資訊，因此新增會失敗，這裡簡單的忽略這個錯誤即可。

隨後出現了UPDATENODELIST的錯誤，情況類似：

INFO: Invoking OUI on cluster nodes racnode1
INFO: /data1/data/oracle/product/10.2/crs/oui/bin/../bin/runInstaller -paramFile /data1/data/oracle/product/10.2/crs/oui/bin/../clu
sterparam.ini -silent -ignoreSysPrereqs -updateNodeList -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs CLUSTER_NODES=r
acnode1,racnode2,racnode3 CRS=true "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode1 -remoteInvocation -invokingNo
deName racnode2 -logFilePath "/data/oracle/oraInventory/logs" -timestamp 2009-01-30_01-01-56AM
SEVERE: oracle.sysman.oii.oiip.oiipg.OiipgRemoteOpsException: Error occured while trying to run Unix command /data1/data/oracle/prod
uct/10.2/crs/oui/bin/../bin/runInstaller -paramFile /data1/data/oracle/product/10.2/crs/oui/bin/../clusterparam.ini -silent -ignor
eSysPrereqs -updateNodeList -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs CLUSTER_NODES=racnode1,racnode2,racnode3 CRS
=true "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode1 -remoteInvocation -invokingNodeName racnode2 -logFilePath
"/data/oracle/oraInventory/logs" -timestamp 2009-01-30_01-01-56AM on nodes racnode1. [Starting Oracle Universal Installer...Checking
installer requirements...Checking operating system version: must be 5.6, 5.7, 5.8 or 5.9.    Actual 5.8
          PassedChecking Temp space: must be greater than 250 MB.   Actual 8591 MB    PassedChecking swap space: must be greater tha
n 500 MB.   Actual 11377 MB    PassedAll installer requirements met.Checking operating system patches: 108652-66,108921-16,108940-53
,108773-18,111310-01,109147-24,111308-03,111111-03,112396-02,110386-03,111023-02,108987-13,108528-21,108989-02,108993-18Checking for
patch 108652-66; found 108652-98 obsoleted Checking for patch 108921-16; found Checking for patch 108940-53; found Checking for pat
ch 108773-18; found Checking for patch 111310-01; found Checking for patch 109147-24; found Checking for patch 111308-03; found Chec
king for patch 111111-03; found Checking for patch 112396-02; found Checking for patch 110386-03; found Checking for patch 111023-02
; found Checking for patch 108987-13; found Checking for patch 108528-21; found Checking for patch 108989-02; found Checking for pat
ch 108993-18; found     Passed'UpdateNodeList' failed. :failed]
        .
        .
        .
        at oracle.sysman.oii.oiic.OiicInstaller.main(OiicInstaller.java:866)
Caused by: oracle.ops.mgmt.cluster.ClusterException: /usr/bin/sh: /data1/data/oracle/product/10.2/crs/oui/bin/../bin/runInstaller: ?
????? :failed
        at oracle.ops.mgmt.cluster.ClusterCmd.runCmd(ClusterCmd.java:1959)
        at oracle.sysman.oii.oiip.oiipg.OiipgClusterRunCmd.runCmdOnUnix(OiipgClusterRunCmd.java:256)
        ... 13 more

SEVERE: Remote 'UpdateNodeList' failed on nodes: 'racnode1,racnode3'. Refer to '/data/oracle/oraInventory/logs/addNodeActions2009-01
-30_01-01-56AM.log' for details.
You can manually re-run the following command on the failed nodes after the installation:
/data/oracle/product/10.2/crs/oui/bin/runInstaller -updateNodeList -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs CLUS
TER_NODES=racnode1,racnode2,racnode3 CRS=true "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=o be run>.
INFO: User Selected: Yes/OK

無論是資料庫還是CLUSTER的UPDATENODELIST操作也都失敗了，不過隨後無論是在racnode1上，還是在racnode3上指向的updatenodelist操作都成功了：

$ runInstaller -updateNodeList -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs CLUSTER_NODES=racnode1,racnode2,racnode3 CRS=true "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode1
Starting Oracle Universal Installer...

Checking installer requirements...

Checking operating system version: must be 5.8, 5.9 or 5.10. Actual 5.8
Passed

Checking Temp space: must be greater than 150 MB. Actual 8639 MB Passed
Checking monitor: must be configured to display at least 256 colors. Actual 16777216 Passed

All installer requirements met.

'UpdateNodeList' was successful.

下面是racnode3上分別執行cluster和database的updatenodelist操作：

$ runInstaller -updateNodeList -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/crs CLUSTER_NODES=racnode1,racnode2,racnode3 CRS=true "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode3
正在啟動 Oracle Universal Installer...

正在檢查安裝程式要求...

檢查作業系統版本: 必須是5.8, 5.9 or 5.10。實際為 5.10
透過

檢查臨時空間: 必須大於 250 MB。   實際為 4149 MB    透過
檢查交換空間: 必須大於 500 MB。   實際為 4289 MB    透過
檢查監視器: 監視器配置至少必須顯示 256 種顏色。    實際為 16777216    透過

所有安裝程式要求均已滿足。

'UpdateNodeList' 成功。
$ /runInstaller -updateNodeList -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/database CLUSTER_NODES=racnode1,racnode2,racnode3 CRS=false "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode3
bash: /runInstaller: 無此檔案或目錄
bash-3.00$ runInstaller -updateNodeList -noClusterEnabled ORACLE_HOME=/data/oracle/product/10.2/database CLUSTER_NODES=racnode1,racnode2,racnode3 CRS=false "INVENTORY_LOCATION=/data/oracle/oraInventory" LOCAL_NODE=racnode3
正在啟動 Oracle Universal Installer...

在 oraparam.ini 中未找到先決條件檢查, 不執行系統先決條件檢查。
The inventory pointer is located at /var/opt/oracle/oraInst.loc
The inventory is located at /data/oracle/oraInventory
'UpdateNodeList' 成功。

透過手工執行，UPDATENODELIST操作成功執行。

Oracle10203RAC環境新增新節點（五）

相關文章