[RAC] HP Unix Install Oracle11g(RAC)

1.Hardware check.
  (1)machinfo | grep -i memory   # memory>2G
  (2)swapinfo -a
  (3)bdf                         #Disk Space Require

2.Software Check.
   uname -a                    
     HP-UX hostname B.11.31 U ia64 hostid unlimited-user license
     swlist | grep -i qpk
     B.11.31.1609              # hp os 11.31 patch需要0809以上的版本.

3.Patch Check.
 swlist -l product | grep -i PHCO_43503              #已經安裝
 swlist -l product | grep -i PHCO_41479
 swlist -l product | grep -i PHKL_38038
 swlist -l product | grep -i PHKL_38938
 swlist -l product | grep -i PHKL_40941
 swlist -l product | grep -i PHSS_36354
 swlist -l product | grep -i PHSS_37042              #已經安裝
 swlist -l product | grep -i PHSS_37959
 swlist -l product | grep -i PHSS_39094
 swlist -l product | grep -i PHSS_39100
 swlist -l product | grep -i PHSS_39102
 swlist -l product | grep -i PHSS_38141
 swlist -l product | grep -i PHSS_39824
 swlist -l product | grep -i PHKL_39773               #for VERITAS File System
 
4.group和user
groupadd -g 1000 oinstall
groupadd -g 1001 asmadmin
groupadd -g 1002 asmdba
groupadd -g 1003 asmoper
groupadd -g 1004 dba
groupadd -g 1005 oper
useradd -u 1100 -g oinstall -G dba,asmdba -d /oracle/testdb/oracle oracle
useradd -u 1101 -g oinstall -G dba,asmadmin,asmdba -d /oracle/testdb/oragrid oragrid
passwd oracle    #oracle/oracle
passwd oragrid   #oragrid/oragrid

5.directory create
mkdir -p /oracle/testdb/oracle
mkdir -p /oracle/testdb/oragrid
chown -R oragrid:oinstall oragrid
chmod -R 775 oragrid
chown -R oracle:oinstall oracle
chmod -R 775 oracle


6.檢查符合的連線是否存在，如果不存在則需要建立
cd /usr/lib
ln -s libX11.3 libX11.sl
ln -s libXIE.2 libXIE.sl
ln -s libXext.3 libXext.sl
ln -s libXhp11.3 libXhp11.sl
ln -s libXi.3 libXi.sl
ln -s libXm.4 libXm.sl
ln -s libXp.2 libXp.sl
ln -s libXt.3 libXt.sl
ln -s libXtst.2 libXtst.sl


7.OS kernel Check.
                     Value             Oracle recommend
------------------------------------------------------------------
 ksi_alloc_max       56000             32768
 executable_stack    0                 0
 max_thread_proc     10000             1024
 maxdsiz             2147483648 (2G)   1073741824 (1G)  
 maxdsiz_64bit       4294967296 (4G)   2147483648 (2G)
 maxssiz             268435456 (256M)  134217728  (128M)
 maxssiz_64bit       1073741824 (1G)   1073741824 (1G)
 maxuprc             9000 (nproc*)/10  3686                      #設定nproc=10000
 msgmni              7000              4096
 msgtql              7000              4096
 ncsize              59072             35840
 maxfiles_lim        65536             32767
 maxfiles            16384             1024
 nflocks             7000              4096
 ninode              58048             34816
 nkthread            52516             7184
 nproc               10000             4096
 semmni              7000              4096
 semmns              14000             8192
 semmnu              9996 (nproc-4)    4092
 semvmx              32767             32767
 shmmax              17179869184 (16G) 1073741824 (1G)
 shmmni              4096              4096
 shmseg              512               512
 tcp_largest_anon_port/udp_largest_anon_port  kctune檢視沒有這個引數,如果需要設定，oracle recommend設定為65500
 
 如果要檢視的話，需要 kctune nproc 檢視設定的值,kctune 加上os 引數檢視.
 修改例子如下:
  kctune max_thread_proc=10000
     ==> Update the automatic 'backup' configuration first? y
       * The automatic 'backup' configuration has been updated.
       * Future operations will update the backup without prompting.
       * The requested changes have been applied to the currently
         running configuration.
  Tunable                    Value  Expression  Changes
  max_thread_proc  (before)   1200  1200        Immed
                   (now)     10000  10000
  kctune max_thread_proc
  Tunable          Value  Expression  Changes
  max_thread_proc  10000  10000       Immed
  修改後需要重啟生效

 
8.filesystem相關建立(2個node一起操作)
node1:
pvcreate /dev/rdsk/c2t0d0
vgcreate -s 32 vgora /dev/dsk/c2t0d0
lvcreate -L 40960M -n lvoragrid /dev/vgora
lvcreate -L 40960M -n lvoracle /dev/vgora
newfs -F vxfs /dev/vgora/rlvoracle
newfs -F vxfs /dev/vgora/rlvoragrid
mount /dev/vgora/lvoracle /oracle/testdb/oracle
mount /dev/vgora/lvoragrid /oracle/testdb/oragrid
在/etc/fstab中加上如下內容，讓filesystem自動掛載
/dev/vgora/lvoracle   /oracle/testdb/oracle     vxfs delaylog 0 2
/dev/vgora/lvoragrid  /oracle/testdb/oragrid    vxfs delaylog 0 2

node2:
pvcreate -f /dev/rdsk/c2t0d1
vgcreate -s 32 vgora /dev/dsk/c2t0d1
lvcreate -L 40960M -n lvoragrid /dev/vgora
lvcreate -L 40960M -n lvoracle /dev/vgora
newfs -F vxfs /dev/vgora/rlvoracle
newfs -F vxfs /dev/vgora/rlvoragrid
mount /dev/vgora/lvoracle /oracle/testdb/oracle
mount /dev/vgora/lvoragrid /oracle/testdb/oragrid
在/etc/fstab中加上如下內容，讓filesystem自動掛載
/dev/vgora/lvoracle   /oracle/testdb/oracle     vxfs delaylog 0 2
/dev/vgora/lvoragrid  /oracle/testdb/oragrid    vxfs delaylog 0 2

9.user環境變數
oracle
export ORACLE_HOME=/oracle/testdb/oracle/DB
export ORACLE_BASE=/oracle/testdb/oracle
export ORACLE_SID=testdb1              #node2  寫 export ORACLE_SID=testdb2  
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$ORACLE_HOME/lib32:/usr/ccs/lib
export PATH=$PATH:$ORACLE_HOME/bin:/bin:/usr/bin:/usr/ccs/bin:/opt/networker/bin:/usr/sbin:$ORACLE_HOME/OPatch:.:

oragrid:
export ORACLE_BASE=/oracle/testdb/oragrid
export ORACLE_HOME=/oracle/testdb/oragrid/grid
export ORACLE_SID=+ASM1                 #node2  寫 export ORACLE_SID=+ASM2  
export PATH=$ORACLE_HOME/bin:$PATH


10./etc/hosts配置
node1和node2上都要配置
xxx.xxx.xx.xxx   node1
xxx.xxx.xx.xxx   node2
xxx.xxx.xx.xxx   node1-vip
xxx.xxx.xx.xxx   node2-vip
100.100.100.xxx  node1-priv
100.100.100.xxx  node2-priv
xxx.xxx.xx.xxx   node-scan

新增網路
ifconfig lan900:1 xxx.xxx.xx.xxx  netmask 255.255.255.0

11.grid安裝
grid安裝彈出ui介面後下一步下一步安裝即可。

遇到的問題總結
(1)之前安裝了serviceguard，在cluster node information不能新增node的資訊。需要刪除如下patch
swremove PHSS_43855
swremove PHSS_44469
swremove T1905CA
swremove T1907BA

(2)cluster node information介面ssh不了，提示INS-06001錯誤。解決方法如下：
INS-06001
existence check failed for /usr/local/bin/bash on node:xxxxx
[xxxxxxx#] which bash
/usr/local/bin/bash
[xxxxxxx#] mv bash bash_20170623
[xxxxxxx#] ln -s /usr/bin/ksh /usr/local/bin/bash
[xxxxxxx#] ls -l
lrwxr-xr-x   1 root       sys             12 Jun 23 14:12 bash -> /usr/bin/ksh

(3)新增網路卡介面提示Installer has detected that network interface.
中間做過的操作,一將/tmp目錄下oragrid屬組的檔案全部清除.二重啟了server。
然後就好了.不知道做了操作之後有效的。

(4)建立OCR ASM的時候，需要對disk賦予相應的許可權。賦予oragrid:oinstall和660許可權。
chown oragrid:oinstall /dev/rdsk/c2t1d3
chmod 660 /dev/rdsk/c2t1d3
or
chown oragrid:oinstall /dev/rdisk/disk31
chmod 660 /dev/rdisk/disk31

(5)建立OCR ASM的時候的時候,發現只能識別一部分的disk。
解決方法dd
dd if=/dev/zero of=/dev/rdsk/c2t0d3 bs=1024k count=1000  
後正常識別.

(6)oragrid用下
oragrid home目錄和ORACLE_BASE和ORACLE_HOME不能在相同的目錄下

(7)執行root.sh的時候一直報如下錯誤.
ORA-29783: GPnP attribute SET failed with error [CLSGPNP_RD_ERR]
先deconfig操作.
/oracle/testdb/oragrid/11203/grid/crs/install/rootcrs.pl -verbose -deconfig -force
dd清空disk裡面的內容
dd if=/dev/zero of=/dev/rdisk/disk29 bs=1024k count=1000
dd if=/dev/zero of=/dev/rdisk/disk30 bs=1024k count=1000
dd if=/dev/zero of=/dev/rdisk/disk31 bs=1024k count=1000
再執行root.sh
/oracle/testdb/oragrid/11203/grid/root.sh

(8)第二個節點執行的時候也報錯. CRS-4402
發現兩邊的private network設定的網路有問題.
重新配置後.執行如下兩個指令碼正常.
/oracle/testdb/oragrid/11203/grid/crs/install/rootcrs.pl -verbose -deconfig -force
/oracle/testdb/oragrid/11203/grid/root.sh

12.asmca create data/fra diskgroup
oragrid使用者下執行asmca調出圖形視窗操作就可以了

13.database software安裝.
oracle使用者下執行./runInstaller下一步下一步操作。

14.接下來就是dbca了.
oracle使用者下輸入dbca調出ui介面下一步下一步操作.