Oracle RAC搭建(二)

 1.配置雙機互信
1)在配置SSH之前，先在兩個節點上生成RSA和DSA密碼
在節點rac1上：
[root@rac1 ~]# su - oracle
[oracle@rac1 ~]$ mkdir .ssh
[oracle@rac1 ~]$ chmod 700 ~/.ssh
建立一個RSA型別的公鑰和私鑰，如下：
[oracle@rac1 ~]$ /usr/bin/ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_rsa.
Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
dc:19:3c:60:ad:a7:32:12:70:74:b9:c0:5e:1d:a0:90
透過如上命令將會建立公鑰檔案/home/oracle/.ssh/id_dsa.pub和私鑰檔案/home/oracle/.ssh/id_dsa，如下：
[oracle@rac1 .ssh]$ ll
total 8
-rw------- 1 oracle oinstall 1675 May 12 14:35 id_rsa
-rw-r--r-- 1 oracle oinstall  393 May 12 14:35 id_rsa.pub
建立一個DSA型別的公鑰和私鑰，如下：
[oracle@rac1 .ssh]$ /usr/bin/ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_dsa.
Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.
The key fingerprint is:
4d:69:16:a4:23:6a:3d:bf:07:3c:e3:d0:15:ad:3f:8d
透過如上命令將會建立公鑰檔案/home/oracle/.ssh/id_rsa.pub和私鑰檔案/home/oracle/.ssh/id_rsa，如下：
[oracle@rac1 .ssh]$ ll
total 16
-rw------- 1 oracle oinstall  668 May 12 14:45 id_dsa
-rw-r--r-- 1 oracle oinstall  601 May 12 14:45 id_dsa.pub
-rw------- 1 oracle oinstall 1675 May 12 14:35 id_rsa
-rw-r--r-- 1 oracle oinstall  393 May 12 14:35 id_rsa.pub
2)授權金鑰檔案中新增金鑰
節點rac1上操作：
[oracle@rac1 .ssh]$ cat id_rsa.pub >>authorized_keys
[oracle@rac1 .ssh]$ cat id_dsa.pub >>authorized_keys
[oracle@rac1 .ssh]$ ls
authorized_keys  id_dsa  id_dsa.pub  id_rsa  id_rsa.pub   
節點rac2上操作：       
建立公鑰和私鑰，如下：
[oracle@rac2 .ssh]$ ll
total 16
-rw------- 1 oracle oinstall  668 May 12 14:50 id_dsa
-rw-r--r-- 1 oracle oinstall  601 May 12 14:50 id_dsa.pub
-rw------- 1 oracle oinstall 1675 May 12 14:49 id_rsa
-rw-r--r-- 1 oracle oinstall  393 May 12 14:49 id_rsa.pub
[oracle@rac2 .ssh]$ ls
authorized_keys  id_dsa  id_dsa.pub  id_rsa  id_rsa.pub
傳輸公鑰和私鑰至另一節點的authorized_keys中,如下：
[oracle@rac1 .ssh]$ ssh rac2 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
The authenticity of host 'rac2 (192.168.10.2)' can't be established.
RSA key fingerprint is 24:25:0c:03:51:1c:4b:60:24:dd:7d:04:85:94:b9:66.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'rac2,192.168.10.2' (RSA) to the list of known hosts.
password:
[oracle@rac1 .ssh]$ ssh rac2 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
password:

[oracle@rac1 .ssh]$ scp ~/.ssh/authorized_keys rac2:~/.ssh/authorized_keys
password:
authorized_keys                               100% 1988     1.9KB/s   00:00
3)驗證雙機互信            #確保有日期返回，並且不需要輸入密碼
[oracle@rac1 .ssh]$ ssh rac1 date
Wed May 14 07:45:12 CST 2014
[oracle@rac1 .ssh]$ ssh rac2 date
Wed May 14 07:45:17 CST 2014
[oracle@rac1 .ssh]$ ssh rac1-priv date
Wed May 14 07:44:31 CST 2014
[oracle@rac1 .ssh]$ ssh rac2-priv date
Wed May 14 07:44:33 CST 2014
[oracle@rac2 .ssh]$ ssh rac1 date
Wed May 14 07:45:40 CST 2014
[oracle@rac2 .ssh]$ ssh rac2 date
Wed May 14 07:45:41 CST 2014
[oracle@rac2 .ssh]$ ssh rac1-priv date
Wed May 14 07:46:14 CST 2014
[oracle@rac2 .ssh]$ ssh rac2-priv date
Wed May 14 07:46:15 CST 2014
[oracle@rac1 .ssh]$ exec /usr/bin/ssh-agent $SHELL
[oracle@rac1 .ssh]$ exec /usr/bin/ssh-add                   //檢視是否有資訊彈出
Identity added: /home/oracle/.ssh/id_rsa (/home/oracle/.ssh/id_rsa)
Identity added: /home/oracle/.ssh/id_dsa (/home/oracle/.ssh/id_dsa)
2.檢視兩節點的網路配置是否能夠相互ping通，如下：

[root@rac1 ~]# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0C:29:40:6C:B8
          inet addr:192.168.10.1  Bcast:192.168.10.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fe40:6cb8/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:667 errors:0 dropped:0 overruns:0 frame:0
          TX packets:882 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:92478 (90.3 KiB)  TX bytes:123456 (120.5 KiB)
          Interrupt:75 Base address:0x2424

eth1      Link encap:Ethernet  HWaddr 00:0C:29:40:6C:C2
          inet addr:10.0.0.1  Bcast:10.0.0.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fe40:6cc2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:401 errors:0 dropped:0 overruns:0 frame:0
          TX packets:63 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:44958 (43.9 KiB)  TX bytes:8950 (8.7 KiB)
          Interrupt:67 Base address:0x24a4

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:1389 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1389 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:2969540 (2.8 MiB)  TX bytes:2969540 (2.8 MiB)
[root@rac2 ~]# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0C:29:D8:29:91
          inet addr:192.168.10.2  Bcast:192.168.10.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fed8:2991/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:538 errors:0 dropped:0 overruns:0 frame:0
          TX packets:757 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:90042 (87.9 KiB)  TX bytes:108986 (106.4 KiB)
          Interrupt:75 Base address:0x2424

eth1      Link encap:Ethernet  HWaddr 00:0C:29:D8:29:9B
          inet addr:10.0.0.2  Bcast:10.0.0.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fed8:299b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:688 errors:0 dropped:0 overruns:0 frame:0
          TX packets:70 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:78348 (76.5 KiB)  TX bytes:9618 (9.3 KiB)
          Interrupt:67 Base address:0x24a4

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:1527 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1527 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:3073680 (2.9 MiB)  TX bytes:3073680 (2.9 MiB)
驗證連通性，如下：
[root@rac1 ~]# ping -c 3 192.168.10.2
PING 192.168.10.2 (192.168.10.2) 56(84) bytes of data.
64 bytes from 192.168.10.2: icmp_seq=1 ttl=64 time=16.3 ms
64 bytes from 192.168.10.2: icmp_seq=2 ttl=64 time=0.231 ms
64 bytes from 192.168.10.2: icmp_seq=3 ttl=64 time=0.140 ms

--- 192.168.10.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2011ms
rtt min/avg/max/mdev = 0.140/5.563/16.319/7.605 ms
[root@rac1 ~]# ping -c 3 10.0.0.2
PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data.
64 bytes from 10.0.0.2: icmp_seq=1 ttl=64 time=0.738 ms
64 bytes from 10.0.0.2: icmp_seq=2 ttl=64 time=0.122 ms
64 bytes from 10.0.0.2: icmp_seq=3 ttl=64 time=0.135 ms

--- 10.0.0.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2024ms
rtt min/avg/max/mdev = 0.122/0.331/0.738/0.288 ms
[root@rac2 ~]# ping -c 3 192.168.10.1
PING 192.168.10.1 (192.168.10.1) 56(84) bytes of data.
64 bytes from 192.168.10.1: icmp_seq=1 ttl=64 time=0.197 ms
64 bytes from 192.168.10.1: icmp_seq=2 ttl=64 time=0.134 ms
64 bytes from 192.168.10.1: icmp_seq=3 ttl=64 time=0.229 ms

--- 192.168.10.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2028ms
rtt min/avg/max/mdev = 0.134/0.186/0.229/0.042 ms
[root@rac2 ~]# ping -c 3 10.0.0.1
PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data.
64 bytes from 10.0.0.1: icmp_seq=1 ttl=64 time=0.209 ms
64 bytes from 10.0.0.1: icmp_seq=2 ttl=64 time=0.222 ms
64 bytes from 10.0.0.1: icmp_seq=3 ttl=64 time=0.231 ms

--- 10.0.0.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2025ms
rtt min/avg/max/mdev = 0.209/0.220/0.231/0.019 ms
批註：共有IP能夠相互ping通說明資料庫能夠透過兩節點對外提供服務，私有IP能夠ping通，說明兩節點能夠相互通訊。
3.配置時間同步
在rac1配置時間同步伺服器：
[root@rac1 ~]# vi /etc/ntp.conf
新增或修改內容如下：
server 127.127.1.0
fudge 127.127.1.0 stratum 11
broadcastdelay 0.008
driftfile /var/lib/ntp/drift
啟動ntp服務：
[root@rac1 ~]# /etc/init.d/ntpd start
Starting ntpd:                                             [  OK  ]
設定ntp隨系統啟動而啟動，如下：
[root@rac1 ~]# chkconfig --level 345 ntpd on
[root@rac1 ~]# chkconfig --list|grep ntp
ntpd            0:off   1:off   2:off   3:on    4:on    5:on    6:off
在節點rac2上的配置：
檢視rac2節點上ntp服務狀態：
[root@rac2 ~]# chkconfig --list |grep ntp
ntpd            0:off   1:off   2:off   3:off   4:off   5:off   6:off
設定執行計劃，每30秒進行一次時間同步，如下：
[root@rac2 ~]# vi /etc/crontab          #已其中一個節點為時間伺服器，這樣兩節點時間就一致了
#新增執行計劃如下：
*/2 * * * * root /usr/sbin/ntpdate 192.168.10.1
在兩節點上進行時間同步測試，如下：
[root@rac1 ~]# ntpdate rac2
14 May 07:50:59 ntpdate[4153]: the NTP socket is in use, exiting
[root@rac2 ~]# ntpdate rac1
14 May 07:51:07 ntpdate[4128]: no server suitable for synchronization found
4.安裝和配置Oracle Clusterware
下載連結：
1)準備安裝檔案
[root@rac1 mnt]# ll
total 2
dr-xr-xr-x 1 root root 2048 Oct 17  2010 clusterware
[root@rac1 mnt]# cp -R clusterware/ /u01/app/
[root@rac1 app]# chown -R oracle:oinstall clusterware/
[root@rac1 app]# ll
total 8
dr-xr-xr-x 9 oracle oinstall 4096 May 12 15:40 clusterware
drwxr-xr-x 3 oracle oinstall 4096 May 12 09:37 oracle
2)圖形介面安裝clusterware
[root@rac1 ~]# xhost +
access control disabled, clients can connect from any host
[root@rac1 ~]# su - oracle
[oracle@rac1 ~]$ export LANG=en_US
[oracle@rac1 ~]$ cd /u01/app/clusterware/
[oracle@rac1 clusterware]$ ls
cluvfy  doc  install  response  rpm  runInstaller  stage  upgrade  welcome.html
[oracle@rac1 clusterware]$ ./runInstaller


[oracle@rac1 oracle]$ mkdir crs
[oracle@rac1 oracle]$ ll
total 12
drwxr-xr-x 2 oracle oinstall 4096 May 12 16:02 crs
drwxr-xr-x 2 oracle oinstall 4096 May 12 09:37 db_1
drwxrwxr-x 5 oracle oinstall 4096 May 12 16:05 oraInventory
 





[root@rac1 oracle]$/u01/app/oracle/oralnventory/orainstRoot.ssh
[root@rac2 oracle]$/u01/app/oracle/oralnventory/orainstRoot.ssh
[root@rac1 oracle]$/u01/app/oracle/crs/root.sh
[root@rac2 oracle]$/u01/app/oracle/crs/root.sh
在節點rac2上執行/u01/app/oracle/crs/root.sh出現如下錯誤：

解決辦法：
用oracle使用者編輯$CRS_HOME/bin下的vipca，找到如下欄位：
117        #Remove this workaround when the bug 3937317 is fixed
118        arch=`uname -m`
119        if [ "$arch" = "i686" -o "$arch" = "ia64" ]
120        then
121             LD_ASSUME_KERNEL=2.4.19
122             export LD_ASSUME_KERNEL
123        fi
在fi 後新增一行：unset LD_ASSUME_KERNEL
同樣用oracle使用者編輯$CRS_HOME/bin下的srvctl，找到如下欄位：
165 #Remove this workaround when the bug 3937317 is fixed
166 LD_ASSUME_KERNEL=2.4.19
167 export LD_ASSUME_KERNEL
在其後新增一行：unset LD_ASSUME_KERNEL
兩個檔案修改後用scp複製到第二個節點：
[oracle@rac02 ~]$ scp $CRS_HOME/bin/srvctl rac01:/u01/app/oracle/crs/bin/
srvctl                                        100% 5501     5.4KB/s   00:00
[oracle@rac02 ~]$ scp /u01/app/oracle/crs/bin/vipca rac01:/u01/app/oracle/crs/bin/
vipca                                         100% 5003     4.9KB/s   00:00

OUI-25031錯誤處理：手動配置VIPCA
手動修改vipca設定（兩節點都要執行）
[root@rac1 ~]# cd /u01/app/oracle/crs/bin/
[root@rac1 bin]# ./oifcfg setif -global eth0/192.168.10.0:public
PRIF-50: duplicate interface is given in the input
[root@rac1 bin]# ./oifcfg setif -global eth1/10.0.0.0:cluster_interconnect
PRIF-50: duplicate interface is given in the input
[root@rac1 bin]# ./oifcfg getif
eth0  192.168.10.0  global  public
eth1  10.0.0.0  global  cluster_interconnect
[root@rac1 bin]# vi vipca
    117        #Remove this workaround when the bug 3937317 is fixed
    118        arch=`uname -m`
    119        if [ "$arch" = "i686" -o "$arch" = "ia64" ]
    120        then
    121             LD_ASSUME_KERNEL=2.4.19
    122             export LD_ASSUME_KERNEL
    123             unset LD_ASSUME_KERNEL=2.4.19
    124        fi
    125        unset LD_ASSUME_KERNEL
    126        #End workaround

新開一個終端，進入CRS_HOME/bin，再次執行./vipca

開始配置VIP，如下：



VIP配置完成之後，再次執行檢查，點選Retry


在oracle使用者下執行crs_stat -t
[oracle@rac2 ~]$ crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora.rac1.gsd   application    ONLINE    ONLINE    rac1
ora.rac1.ons   application    ONLINE    ONLINE    rac1
ora.rac1.vip   application    ONLINE    ONLINE    rac1
ora.rac2.gsd   application    ONLINE    ONLINE    rac2
ora.rac2.ons   application    ONLINE    ONLINE    rac2
ora.rac2.vip   application    ONLINE    ONLINE    rac2