Solaris10下安裝Oracle1106RAC環境（二）

安裝一套RAC11G的測試環境，作業系統為Solaris 10，共享磁碟管理使用Volumn Cluster Manager，Oracle版本為1106。

這篇描述Oracle Clusterware的安裝。

Solaris10下安裝Oracle1106RAC環境（一）：http://yangtingkun.itpub.net/post/468/467067

 

 

首先利用Oracle Cluster包中自帶的檢測工具，檢查是否所有的配置準備工作已經完成：

bash-3.00$ ./runcluvfy.sh stage -pre crsinst -n newtrade1,newtrade2 -verbose

Performing pre-checks for cluster services setup

Checking node reachability...

Check: Node reachability from node "newtrade1"
  Destination Node                      Reachable?             
  ------------------------------------  ------------------------
  newtrade2                             yes                    
  newtrade1                             yes                    
Result: Node reachability check passed from node "newtrade1".

Checking user equivalence...

Check: User equivalence for user "oracle"
  Node Name                             Comment                
  ------------------------------------  ------------------------
  newtrade2                             passed                 
  newtrade1                             passed                 
Result: User equivalence check passed for user "oracle".

Checking administrative privileges...

Check: Existence of user "oracle"
  Node Name     User Exists               Comment                
  ------------  ------------------------  ------------------------
  newtrade2     yes                       passed                 
  newtrade1     yes                       passed                 
Result: User existence check passed for "oracle".

Check: Existence of group "oinstall"
  Node Name     Status                    Group ID               
  ------------  ------------------------  ------------------------
  newtrade2     exists                    100                    
  newtrade1     exists                    100                    
Result: Group existence check passed for "oinstall".

Check: Membership of user "oracle" in group "oinstall" [as Primary]
  Node Name         User Exists   Group Exists  User in Group  Primary       Comment    
  ----------------  ------------  ------------  ------------  ------------  ------------
  newtrade2         yes           yes           yes           yes           passed     
  newtrade1         yes           yes           yes           yes           passed     
Result: Membership check for user "oracle" in group "oinstall" [as Primary] passed.

Administrative privileges check passed.

Checking node connectivity...

Interface information for node "newtrade2"
  Interface Name    IP Address    Subnet        Subnet Gateway  Default Gateway  Hardware Address
  ----------------  ------------  ------------  ------------  ------------  ------------
  ce0               172.0.2.55    172.0.2.0     172.0.2.55    UNKNOWN       00:14:4f:47:8d:12
  ce1               10.0.1.4      10.0.1.0      10.0.1.4      UNKNOWN       00:14:4f:47:8d:12

Interface information for node "newtrade1"
  Interface Name    IP Address    Subnet        Subnet Gateway  Default Gateway  Hardware Address
  ----------------  ------------  ------------  ------------  ------------  ------------
  ce0               172.0.2.54    172.0.2.0     172.0.2.54    UNKNOWN       00:14:4f:ad:47:cc
  ce1               10.0.1.3      10.0.1.0      10.0.1.3      UNKNOWN       00:14:4f:ad:47:cc

Check: Node connectivity of subnet "172.0.2.0"
  Source                          Destination                     Connected?     
  ------------------------------  ------------------------------  ----------------
  newtrade2:ce0                   newtrade1:ce0                   yes            
Result: Node connectivity check passed for subnet "172.0.2.0" with node(s) newtrade2,newtrade1.

Check: Node connectivity of subnet "10.0.1.0"
  Source                          Destination                     Connected?     
  ------------------------------  ------------------------------  ----------------
  newtrade2:ce1                   newtrade1:ce1                   yes            
Result: Node connectivity check passed for subnet "10.0.1.0" with node(s) newtrade2,newtrade1.

Interfaces found on subnet "172.0.2.0" that are likely candidates for VIP:
newtrade2 ce0:172.0.2.55
newtrade1 ce0:172.0.2.54

Interfaces found on subnet "10.0.1.0" that are likely candidates for VIP:
newtrade2 ce1:10.0.1.4
newtrade1 ce1:10.0.1.3

WARNING:
Could not find a suitable set of interfaces for the private interconnect.

Result: Node connectivity check passed.

Checking system requirements for 'crs'...

Check: Total memory
  Node Name     Available                 Required                  Comment  
  ------------  ------------------------  ------------------------  ----------
  newtrade2     32GB (33554432KB)         1GB (1048576KB)           passed   
  newtrade1     32GB (33554432KB)         1GB (1048576KB)           passed   
Result: Total memory check passed.

Check: Free disk space in "/tmp" dir
  Node Name     Available                 Required                  Comment  
  ------------  ------------------------  ------------------------  ----------
  newtrade2     57.62GB (60423736KB)      400MB (409600KB)          passed   
  newtrade1     55.95GB (58670696KB)      400MB (409600KB)          passed   
Result: Free disk space check passed.

Check: Swap space
  Node Name     Available                 Required                  Comment  
  ------------  ------------------------  ------------------------  ----------
  newtrade2     31.26GB (32776888KB)      1.5GB (1572864KB)         passed   
  newtrade1     30.28GB (31749112KB)      1.5GB (1572864KB)         passed   
Result: Swap space check passed.

Check: System architecture
  Node Name     Available                 Required                  Comment  
  ------------  ------------------------  ------------------------  ----------
  newtrade2     64-bit sparcv9 kernel modules  64-bit                    passed   
  newtrade1     64-bit sparcv9 kernel modules  64-bit                    passed   
Result: System architecture check passed.

Check: Kernel version
  Node Name     Available                 Required                  Comment  
  ------------  ------------------------  ------------------------  ----------
  newtrade2     10,REV=2006.11.14.15.02   10,REV=2005.01.22.19.14   passed   
  newtrade1     10,REV=2007.08.16.13.29   10,REV=2005.01.22.19.14   passed   
Result: Kernel version check passed.

Check: Package existence for "SUNWarc"
  Node Name                       Status                          Comment        
  ------------------------------  ------------------------------  ----------------
  newtrade2                       SUNWarc:11.10.0                 passed         
  newtrade1                       SUNWarc:11.10.0                 passed         
Result: Package existence check passed for "SUNWarc".

Check: Package existence for "SUNWbtool"
  Node Name                       Status                          Comment        
  ------------------------------  ------------------------------  ----------------
  newtrade2                       SUNWbtool:11.10.0               passed         
  newtrade1                       SUNWbtool:11.10.0               passed         
Result: Package existence check passed for "SUNWbtool".

Check: Package existence for "SUNWhea"
  Node Name                       Status                          Comment        
  ------------------------------  ------------------------------  ----------------
  newtrade2                       SUNWhea:11.10.0                 passed         
  newtrade1                       SUNWhea:11.10.0                 passed         
Result: Package existence check passed for "SUNWhea".

Check: Package existence for "SUNWlibm"
  Node Name                       Status                          Comment        
  ------------------------------  ------------------------------  ----------------
  newtrade2                       SUNWlibm:5.10                   passed         
  newtrade1                       SUNWlibm:5.10                   passed         
Result: Package existence check passed for "SUNWlibm".

Check: Package existence for "SUNWlibms"
  Node Name                       Status                          Comment        
  ------------------------------  ------------------------------  ----------------
  newtrade2                       SUNWlibms:5.10                  passed         
  newtrade1                       SUNWlibms:5.10                  passed         
Result: Package existence check passed for "SUNWlibms".

Check: Package existence for "SUNWsprot"
  Node Name                       Status                          Comment        
  ------------------------------  ------------------------------  ----------------
  newtrade2                       SUNWsprot:5.10                  passed         
  newtrade1                       SUNWsprot:5.10                  passed         
Result: Package existence check passed for "SUNWsprot".

Check: Package existence for "SUNWtoo"
  Node Name                       Status                          Comment        
  ------------------------------  ------------------------------  ----------------
  newtrade2                       SUNWtoo:11.10.0                 passed         
  newtrade1                       SUNWtoo:11.10.0                 passed         
Result: Package existence check passed for "SUNWtoo".

Check: Package existence for "SUNWi1of"
  Node Name                       Status                          Comment        
  ------------------------------  ------------------------------  ----------------
  newtrade2                       SUNWi1of:6.6.2.7400             passed         
  newtrade1                       SUNWi1of:6.6.2.7400             passed         
Result: Package existence check passed for "SUNWi1of".

Check: Package existence for "SUNWi1cs"
  Node Name                       Status                          Comment        
  ------------------------------  ------------------------------  ----------------
  newtrade2                       ERROR: information for "SUNWi1cs" was not found  passed         
  newtrade1                       SUNWi1cs:2.0                    passed         
Result: Package existence check passed for "SUNWi1cs".

Check: Package existence for "SUNWi15cs"
  Node Name                       Status                          Comment        
  ------------------------------  ------------------------------  ----------------
  newtrade2                       ERROR: information for "SUNWi15cs" was not found  passed         
  newtrade1                       SUNWi15cs:2.0                   passed         
Result: Package existence check passed for "SUNWi15cs".

Check: Package existence for "SUNWxwfnt"
  Node Name                       Status                          Comment        
  ------------------------------  ------------------------------  ----------------
  newtrade2                       SUNWxwfnt:6.6.2.7400            passed         
  newtrade1                       SUNWxwfnt:6.6.2.7400            passed         
Result: Package existence check passed for "SUNWxwfnt".

Check: Package existence for "SUNWlibC"
  Node Name                       Status                          Comment        
  ------------------------------  ------------------------------  ----------------
  newtrade2                       SUNWlibC:5.10                   passed         
  newtrade1                       SUNWlibC:5.10                   passed         
Result: Package existence check passed for "SUNWlibC".

Check: Group existence for "dba"
  Node Name     Status                    Comment                
  ------------  ------------------------  ------------------------
  newtrade2     exists                    passed                 
  newtrade1     exists                    passed                 
Result: Group existence check passed for "dba".

Check: Group existence for "oinstall"
  Node Name     Status                    Comment                
  ------------  ------------------------  ------------------------
  newtrade2     exists                    passed                 
  newtrade1     exists                    passed                 
Result: Group existence check passed for "oinstall".

Check: User existence for "nobody"
  Node Name     Status                    Comment                
  ------------  ------------------------  ------------------------
  newtrade2     exists                    passed                 
  newtrade1     exists                    passed                 
Result: User existence check passed for "nobody".

System requirement passed for 'crs'

Pre-check for cluster services setup was successful.

可以看到，除了節點2上個別系統包沒有完全安裝之外，所有的檢查已經順利透過。由於安裝是在節點1上進行，所以這個問題可以忽略。

利用VOLUMN CLUSTER MANAGER劃分共享儲存，建立CLUSTER安裝所需的VOTE和OCR裸裝置：

root@newtrade1 # vxassist -g datavg make ocr 300m user=oracle group=oinstall mode=660
root@newtrade1 # vxassist -g datavg make vot 300m user=oracle group=oinstall mode=660

下面就可以安裝CLUSTER：

root@newtrade1 # xhost +
access control disabled, clients can connect from any host
root@newtrade1 # su - oracle
Sun Microsystems Inc.   SunOS 5.10      Generic January 2005
$ bash    
bash-3.00$ cd /data/clusterware/
bash-3.00$ ./runInstaller

需要注意幾點，在安裝CLUSTER的時候Oracle預設選擇路徑/data/oracle/product/11.1/database，這裡將其改為/data/oracle/product/11.1/crs。在新增節點後，Oracle認為172.0.2.0和10.0.1.0兩個網段都是PUBLIC的候選，這裡需要手工編輯將其改為PRIVATE。

由於是測試環境，選擇外部冗餘，在OCR處輸入前面建立的：/dev/vx/rdsk/datavg/ocr。

在VOT磁碟處同樣選擇外部冗餘，輸入：/dev/vx/rdsk/datavg/vot。

最後安裝成功後，需要分別在節點1、節點2執行兩個shell指令碼：

root@newtrade1 # . /data/oracle/oraInventory/orainstRoot.sh
Changing permissions of /data/oracle/oraInventory to 770.
Changing groupname of /data/oracle/oraInventory to oinstall.
The execution of the script. is complete

節點1上執行第二個指令碼：

root@newtrade1 # . /data/oracle/product/11.1/crs/root.sh
WARNING: directory '/data/oracle/product/11.1' is not owned by root
WARNING: directory '/data/oracle/product' is not owned by root
WARNING: directory '/data/oracle' is not owned by root
WARNING: directory '/data' is not owned by root
Checking to see if Oracle CRS stack is already configured

Setting the permissions on OCR backup directory
Setting up Network socket directories
Oracle Cluster Registry configuration upgraded successfully
The directory '/data/oracle/product/11.1' is not owned by root. Changing owner to root
The directory '/data/oracle/product' is not owned by root. Changing owner to root
The directory '/data/oracle' is not owned by root. Changing owner to root
The directory '/data' is not owned by root. Changing owner to root
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node :
node 1: newtrade1 newtrade1-priv newtrade1
node 2: newtrade2 newtrade2-priv newtrade2
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Now formatting voting device: /dev/vx/rdsk/datavg/vot
Format of 1 voting devices complete.
Startup will be queued to init within 30 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
Cluster Synchronization Services is active on these nodes.
        newtrade1
Cluster Synchronization Services is inactive on these nodes.
        newtrade2
Local node checking complete. Run root.sh on remaining nodes to start CRS daemons.

節點2上執行第二個SQL：

root@newtrade2 # . /data/oracle/product/11.1/crs/root.sh
WARNING: directory '/data/oracle/product/11.1' is not owned by root
WARNING: directory '/data/oracle/product' is not owned by root
WARNING: directory '/data/oracle' is not owned by root
WARNING: directory '/data' is not owned by root
Checking to see if Oracle CRS stack is already configured

Setting the permissions on OCR backup directory
Setting up Network socket directories
Oracle Cluster Registry configuration upgraded successfully
The directory '/data/oracle/product/11.1' is not owned by root. Changing owner to root
The directory '/data/oracle/product' is not owned by root. Changing owner to root
The directory '/data/oracle' is not owned by root. Changing owner to root
The directory '/data' is not owned by root. Changing owner to root
clscfg: EXISTING configuration version 4 detected.
clscfg: version 4 is 11 Release 1.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node :
node 1: newtrade1 newtrade1-priv newtrade1
node 2: newtrade2 newtrade2-priv newtrade2
clscfg: Arguments check out successfully.

NO KEYS WERE WRITTEN. Supply -force parameter to override.
-force is destructive and will destroy any previous cluster
configuration.
Oracle Cluster Registry for cluster has already been initialized
Startup will be queued to init within 30 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
Cluster Synchronization Services is active on these nodes.
        newtrade1
        newtrade2
Cluster Synchronization Services is active on all the nodes.
Waiting for the Oracle CRSD and EVMD to start
Oracle CRS stack installed and running under init(1M)
Running vipca(silent) for configuring nodeapps

Creating VIP application resource on (2) nodes...
Creating GSD application resource on (2) nodes...
Creating ONS application resource on (2) nodes...
Starting VIP application resource on (2) nodes...
Starting GSD application resource on (2) nodes...
Starting ONS application resource on (2) nodes...

Done.

最後Oracle會自動設定3個和CLUSTER相關的工具，至此CLUSTER安裝完成。