oracle 11g R2安裝RAC

longqidong發表於2012-08-03
嘗試安裝了oracle 11g R2 RAC,和10g過程差不多。參考一個網友的文件安裝的。

主要是前期的準備工作要做好 ,特別共享磁碟的問題

我是在虛擬機器上安裝的,記錄一下:
 vmware 7.0。 
1.在cmd命令列下,使用命令建立3個共享磁碟,分別是5G、5G、2G。
vmware-vdiskmanager.exe -c -s 5G -a lsilogic -t 2 data1.vmdk
。。。
2.在虛擬機器配置中新增磁碟,選擇已存在的磁碟,注意,在高階設定中,scsi:1.0這種選項不能重複,否則報錯。修改配置檔案,加上如下內容:
scsci1.sharedBus = "virtual"      # /dev/sdb     scsi1:0    DATA 5G
scsci2.sharedBus = "virtual"       # /dev/sdc    scsi2:0    CRS  2G
scsci3.sharedBus = "virtual"       # /dev/sdd    scsi3:0     RECOVERY 5G
disk.locking = "false"

diskLib.dataCacheMaxSize = "0" 
diskLib.dataCacheMaxReadAheadSize = "0" 
diskLib.DataCacheMinReadAheadSize = "0" 
diskLib.dataCachePageSize = "4096" 
diskLib.maxUnsyncedWrites = "0"        剛開始沒加上面這5個引數,分割槽在另外一臺機器識別不了,加上之後,可以了。

3.在一個節點分割槽,在另外一個節點能正常是被分割槽就可以了。
 
使用ASM做磁碟組,安裝asmna 三個rpm包,一個要和核心對應,否則config的時候通不過。

我安裝過程中,配置vip的時候,開始時是rac1vip,rac2vip。然後報錯:vip無效,然後改名為 rac1_vip,rac2_vip.透過了。

附上我參考的文章:(僅供參考)


軟體環境:
redhat5.4  64bit
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0(64位)
1、伺服器本地磁碟分割槽:
Disk /dev/sda: 300.0 GB, 300000000000 bytes
255 heads, 63 sectors/track, 36472 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          25      200781   83  Linux
/dev/sda2              26       36472   292760527+  8e  Linux LVM


使用了LVM 分割槽,後使用卷。
關於LVM 資料如下:
http://hi.baidu.com/dongfangmn/blog/item/23f7ccd813c9213831fa1c67.html

2、IP規劃

[root@rac2 app]# cat /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1        localhost.localdomain localhost
#::1        localhost6.localdomain6 localhost6


#public ip
192.168.18.101 rac1
192.168.18.103 rac2

#priv ip
192.168.0.101     rac1-private
192.168.0.103     rac2-private

#vip ip
192.168.18.121      rac1-vip
192.168.18.123      rac2-vip

#scan ip
192.168.18.100      rac-scan

(注意:All host names must conform. to the RFC 952 standard,
which permits alphanumeric characters, Host name using underscores(“_”) 
are not allowed.HOSTS 檔案中不支援“_” 字元)

3、使用者/組

/usr/sbin/groupadd -g 501 oinstall
/usr/sbin/groupadd -g 502 dba
/usr/sbin/groupadd -g 503 oper
/usr/sbin/groupadd -g 504 asmadmin
/usr/sbin/groupadd -g 505 asmoper
/usr/sbin/groupadd -g 506 asmdba
/usr/sbin/useradd -g oinstall -G dba,asmdba,oper oracle
/usr/sbin/useradd -g oinstall -G asmadmin,asmdba,asmoper,oper,dba grid

[root@ora1 ~]# id oracle
uid=501(oracle) gid=501(oinstall) groups=501(oinstall),502(dba),503(oper),506(asmdba)
[root@ora1 ~]# id grid
uid=502(grid) gid=501(oinstall) groups=501(oinstall),502(dba),503(oper),504(asmadmin),505(asmoper),506(asmdba)


mkdir -p /opt/app/oraInventory
chown -R grid:oinstall /opt/app/oraInventory
chmod -R 775 /opt/app/oraInventory

mkdir -p /opt/app/grid
mkdir -p /opt/app/oracle
chown -R grid:oinstall /opt/app/grid
chown -R oracle:oinstall /opt/app/oracle
chmod -R 775 /opt/app/oracle
chmod -R 775 /opt/app/grid

passwd grid
passwd oracle

4、修改系統引數:
vi /etc/security/limits.conf

#ORACLE SETTING
grid                 soft    nproc   2047
grid                 hard    nproc   16384
grid                 soft    nofile  1024
grid                 hard    nofile  65536
oracle               soft    nproc   2047
oracle               hard    nproc   16384
oracle               soft    nofile  1024
oracle               hard    nofile  65536


vi /etc/pam.d/login

#ORACLE SETTING
session    required     pam_limits.so

# vi /etc/sysctl.conf

#ORACLE SETTING
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 536870912
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586


5、gird時間同步所需要的設定(11gR2新增檢查項)

#Network Time Protocol Setting
/sbin/service ntpd stop
chkconfig ntpd off

mv /etc/ntp.conf  /etc/ntp.conf.org
#rm /var/run/ntpd.pid



選擇是開啟還是關閉SELINUX的工作模式(修改這一項後最好重啟一下作業系統)

[root@oracle ~]# vi /etc/selinux/config

# 設定SELINUX為disabled

      SELINUX=disabled

6、作業系統版本:

[root@rac1 ~]# lsb_release -a
LSB Version:    :core-3.1-amd64:core-3.1-ia32:core-3.1-noarch:graphics-3.1-amd64:graphics-3.1-ia32:graphics-3.1-noarch
Distributor ID:    CentOS
Description:    CentOS release 5.5 (Final)
Release:    5.5
Codename:    Final

[root@rac1 ~]# uname -a

Linux solr03 2.6.18-194.11.4.el5 #1 SMP Tue Sep 21 05:04:09 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux
[root@rac1 ~]#

7、修改系統的發行版本

[root@oracle ~]# vi /etc/RedHat-release

將CentOS release 5 (Final) 修改成
#CentOS release 5 (Final)
Red Hat Enterprise Linux AS release 5 (Nahant Update 5)

(因為oracle公司沒推出CentOS版本的oracle)
8、修改gird、oracle使用者的.bash_profile檔案:

#grid 使用者配置檔案 ORACLE_HOSTNAME請自行設定
TMP=/tmp; export TMP
TMPDIR=$TMP; export TMPDIR

ORACLE_SID=+ASM1; export ORACLE_SID
ORACLE_BASE=/opt/oracle; export ORACLE_BASE 

ORACLE_HOME=/opt/oracle/product/11.2.0; export ORACLE_HOME
NLS_DATE_FORMAT="yyyy-mm-dd HH24:MI:SS"; export NLS_DATE_FORMAT
THREADS_FLAG=native; export THREADS_FLAG

PATH=$ORACLE_HOME/bin:$PATH; export PATH
THREADS_FLAG=native; export THREADS_FLAG

PATH=$ORACLE_HOME/bin:$PATH; export PATH

if [ $USER = "oracle" ] || [ $USER = "grid" ]; then
        if [ $SHELL = "/bin/ksh" ]; then
            ulimit -p 16384
              ulimit -n 65536
  else
   ulimit -u 16384 -n 65536
      fi
    umask 022
fi
stty erase ^h #刪除鍵可用處理


#oracle使用者配置檔案 ORACLE_HOSTNAME請自行設定
# Oracle Settings oracle
TMP=/tmp; export TMP
TMPDIR=$TMP; export TMPDIR

ORACLE_BASE=/opt/oracle; export ORACLE_BASE
ORACLE_HOME=$ORACLE_BASE/product/11.2.0; export ORACLE_HOME
ORACLE_SID=racdb1; export ORACLE_SID
ORACLE_TERM=xterm; export ORACLE_TERM

PATH=/usr/sbin:$PATH; export PATH
PATH=$ORACLE_HOME/bin:$PATH; export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH
CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH
NLS_DATE_FORMAT="yyyy-mm-dd HH24:MI:SS"; export NLS_DATE_FORMAT
NLS_LANG=AMERICAN_AMERICA.ZHS16GBK;export NLS_LANG

if [ $USER = "oracle" ] || [ $USER = "grid" ]; then
        if [ $SHELL = "/bin/ksh" ]; then
         ulimit -p 16384
              ulimit -n 65536
        else
              ulimit -u 16384 -n 65536
        fi
        umask 022
fi
stty erase ^h #刪除鍵可用處理

8、配置信任關係

設定SSH,
1).在主節點RAC1上以grid,oracle使用者身份生成使用者的公匙和私匙

# su - oracle
$ mkdir ~/.ssh
$ ssh-keygen -t rsa
$ ssh-keygen -t dsa

2).在副節點RAC2上執行相同的操作,確保通訊無阻
# ping rac1-eth0
# ping rac1-eth1
# su - oracle
$ mkdir ~/.ssh
$ ssh-keygen -t rsa
$ ssh-keygen -t dsa

3).在主節點RAC1上oracle使用者執行以下操作

$ touch ~/.ssh/authorized_keys 

$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

#將RAC2的加入
$ ssh rac2 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
$ ssh rac2 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
$ scp ~/.ssh/authorized_keys rac2:~/.ssh/authorized_keys



4).主節點RAC1上執行檢驗操作
$ ssh rac1 date
$ ssh rac2 date
$ ssh rac1-private date
$ ssh rac2-private date



5).在副節點RAC2上執行檢驗操作
$ ssh rac1 date
$ ssh rac2 date

$ ssh rac1-private date
$ ssh rac2-private date

-------------------------------------------------------------------------------------
9、共享磁碟規劃
   
+CRS        三個2G的盤
+DGDATA     四個1T的盤
+DGRECOVERY 兩個1T的盤

硬碟分割槽

[root@rac1 init.d]# fdisk /dev/sdc

Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.
 
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
 
Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-130, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-130, default 130):
Using default value 130
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
在用fdisk -l 就會看到新的分割槽sdb1,sdc1,sdd1,sde1,sdf1
10、每個節點安裝ASM

oracleasmlib-2.0.4-1.el5.x86_64.rpm
oracleasm-2.6.18-194.el5-2.0.5-1.el5.x86_64.rpm
oracleasm-support-2.1.7-1.el5.x86_64.rpm
 

 檢視對應的核心,下載對應的支援包
uname -r" command on your machine to determine your kernel version.  

The corresponding package has the name oracleasm-

 
下載地址:

 


[root@rac1 oracle]# rpm -ivh oracleasm-support-2.1.7-1.el5.x86_64.rpm
warning: oracleasm-support-2.1.7-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID b38a8516
Preparing...                ########################################### [100%]
   1:oracleasm-support      ########################################### [100%]

[root@rac1 oracle]# rpm -ivh oracleasmlib-2.0.4-1.el5.x86_64.rpm 
warning: oracleasmlib-2.0.4-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
error: Failed dependencies:
    oracleasm >= 1.0.4 is needed by oracleasmlib-2.0.4-1.el5.x86_64

[root@rac1 oracle]# rpm -ivh oracleasm-2.6.18-194.11.4.el5-2.0.5-1.el5.x86_64.rpm 
warning: oracleasm-2.6.18-194.11.4.el5-2.0.5-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing...                ########################################### [100%]
   1:oracleasm-2.6.18-194.11########################################### [100%]



11、每個節點配置ASM
 
 [root@rac2 init.d]# ./oracleasm configure
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface [grid]: 
Default group to own the driver interface [asmadmin]: 
Start Oracle ASM library driver on boot (y/n) [y]: 
Scan for Oracle ASM disks on boot (y/n) [y]: 
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver:                     [  OK  ]
Scanning the system for Oracle ASMLib disks:               [  OK  ]

 建立asm盤
   
+CRS        三個2G的盤
+DGDATA     四個1T的盤
+DGRECOVERY 兩個1T的盤

(以下各分割槽大小和實際大小不同,以下內容只供借鑑)
CRS1 /dev/sdk: 2147 MB
CRS2 /dev/sdl: 2147 MB
CRS3 /dev/sdm: 2147 MB

DGDATA1 /dev/sdn: 1099.5 GB
DGDATA2 /dev/sdo: 1099.5 GB
DGDATA3 /dev/sdp: 1099.5 GB
DGDATA4 /dev/sdq: 1099.5 GB

DGRECOVERY1 /dev/sdr: 1099.5 GB
DGRECOVERY2 /dev/sds: 1091.3 GB


[root@ora2 asm]# /etc/init.d/oracleasm createdisk CRS1 /dev/sdb1
Marking disk "CRS1" as an ASM disk: [  OK  ]
[root@ora2 asm]# /etc/init.d/oracleasm createdisk CRS2 /dev/sdc1
Marking disk "CRS2" as an ASM disk: [  OK  ]
[root@ora2 asm]# /etc/init.d/oracleasm createdisk CRS3 /dev/sdd1
Marking disk "CRS3" as an ASM disk: [  OK  ]
[root@ora2 asm]# /etc/init.d/oracleasm createdisk DATA1 /dev/sde1
Marking disk "DATA1" as an ASM disk: [FAILED]
[root@ora2 asm]# /etc/init.d/oracleasm createdisk DATA2 /dev/sdf1
Marking disk "DATA1" as an ASM disk: [  OK  ]
[root@ora2 asm]# /etc/init.d/oracleasm createdisk DATA3 /dev/sdg1
Marking disk "DATA2" as an ASM disk: [  OK  ]
[root@ora2 asm]# /etc/init.d/oracleasm createdisk DATA4 /dev/sdh1
Marking disk "DATA3" as an ASM disk: [  OK  ]
[root@ora2 asm]# /etc/init.d/oracleasm createdisk RECOVERY1 /dev/sdi1
Marking disk "REC1" as an ASM disk: [  OK  ]
[root@ora2 asm]# /etc/init.d/oracleasm createdisk RECOVERY2 /dev/sdj1
Marking disk "REC2" as an ASM disk: [  OK  ]

[root@rac2 ~]# /etc/init.d/oracleasm scandisks
Scanning the system for Oracle ASMLib disks:               [  OK  ]

[root@rac2 ~]# /etc/init.d/oracleasm listdisks
CRS1
CRS2
CRS3
DATA1
DATA2
DATA3
DATE4
RECOVERY1
RECOVERY2
12、安裝cvuqdisk包並驗證

在兩個 Oracle RAC 節點上安裝作業系統程式包 cvuqdisk。如果沒有 cvuqdisk,
叢集驗證實用程式就無法發現共享磁碟,當執行(手動執行或在 Oracle Grid Infrastructure 
安裝結束時自動執行)叢集驗證實用程式時,您會收到這樣的錯誤訊息:“Package cvuqdisk not installed”。 

使用適用於您的硬體體系結構(例如,x86_64 或 i386)的 cvuqdisk RPM。
cvuqdisk RPM 包含在 Oracle Grid Infrastructure 安裝介質上的 rpm 目錄中。


安裝cvuqdisk包

如果不安裝cvuqdisk包, Cluster Verification Utility不能發現共享盤,
當你執行Cluster Verification程式時你將收到"Package cvuqdisk not installed"

檢查是否已經安裝cvuqdisk包

# rpm -qi cvuqdisk

如果已經安裝,則檢視版本

# rpm -e cvuqdisk


在你儲存cvuqdisk的目錄使用下列命令安裝cvuqdisk包如:

# rpm -iv cvuqdisk-1.0.7-1.rpm

[root@rac1 rpm]# pwd
/home/oracle/grid/grid/rpm
[root@rac1 rpm]# ll
-rw-rw-r-- 1 root root 8173 Jul 15  2009 cvuqdisk-1.0.7-1.rpm
[root@rac1 rpm]# rpm -qi cvuqdisk-1.0.7-1.rpm 
package cvuqdisk-1.0.7-1.rpm is not installed
[root@rac1 rpm]# rpm -ivh cvuqdisk-1.0.7-1.rpm 
Preparing...                ########################################### [100%]
Using default group oinstall to install package
   1:cvuqdisk               ########################################### [100%]




設定環境變數 CVUQDISK_GRP,使其指向作為 cvuqdisk 的所有者所在的組(本文為 oinstall):
export CVUQDISK_GRP=oinstall
使用 CVU 驗證是否滿足 Oracle 叢集件要求
記住要作為 grid 使用者在將要執行 Oracle 安裝的節點 (racnode1) 上執行。此外,
必須為 grid 使用者配置透過使用者等效性實現的 SSH 連通性。

檢查結點、使用者
在grid軟體安裝目錄裡執行以下命令:
./runcluvfy.sh stage -pre crsinst -n rac1,rac2 -fixup -verbose
 
[grid@rac1 grid]$ ./runcluvfy.sh stage -pre crsinst -n rac1,rac2 -fixup -verbose
Performing pre-checks for cluster services setup
Checking node reachability...
 


檢查過程省略...
 
使用 CVU 驗證硬體和作業系統設定
./runcluvfy.sh stage -post hwos -n rac1,rac2 -verbose
[grid@rac1 grid]$ ./runcluvfy.sh stage -post hwos -n rac1,rac2 -verbose
Performing post-checks for hardware and operating system setup
Checking node reachability...
 
檢查過程省略...
 



13、檢查安裝要求的支援包

binutils-2.17.50.0.6
compat-libstdc++-33-3.2.3
compat-libstdc++-33-3.2.3 (32 bit)
elfutils-libelf-0.125
elfutils-libelf-devel-0.125
gcc-4.1.2
gcc-c++-4.1.2
glibc-2.5-24
glibc-2.5-24 (32 bit)
glibc-common-2.5
glibc-devel-2.5
glibc-devel-2.5 (32 bit)
glibc-headers-2.5
ksh-20060214
libaio-0.3.106
libaio-0.3.106 (32 bit)
libaio-devel-0.3.106
libaio-devel-0.3.106 (32 bit)
libgcc-4.1.2
libgcc-4.1.2 (32 bit)
libstdc++-4.1.2
libstdc++-4.1.2 (32 bit)
libstdc++-devel 4.1.2
make-3.81
numactl-devel-0.9.8.x86_64
sysstat-7.0.2


Are the following packages installed (assumes CentOS-5 and above distribution)
    command:
    rpm -q binutils \
    compat-libstdc++ \
    elfutils-libelf \
    elfutils-libelf-devel \
    glibc \
    glibc-common \
    glibc-devel \
    glibc-headers \
    gcc \
    gcc-c++ \
    libaio-devel \
    libaio \
    libgcc \
    libstdc++ \
    libstdc++ \
    make \
    sysstat \
    unixODBC \
    unixODBC-devel



    results: binutils-2.17.50.0.6-14.el5
    package compat-libstdc++ is not installed
    elfutils-libelf-0.137-3.el5
    package elfutils-libelf-devel is not installed
    glibc-2.5-49
    glibc-common-2.5-49
    glibc-devel-2.5-49
    glibc-headers-2.5-49
    gcc-4.1.2-48.el5
    package gcc-c++ is not installed
    package libaio-devel is not installed
    libaio-0.3.106-5
    libgcc-4.1.2-48.el5
    libstdc++-4.1.2-48.el5
    libstdc++-4.1.2-48.el5
    make-3.81-3.el5
    package sysstat is not installed
    package unixODBC is not installed
    package unixODBC-devel is not installed
    notes : you will probably get quite a few packages that are not installed with a 
    typical installation. You can investigate missing packages with the following command 
    making sure you add/subtract from this list if necessary.
    yum list available "compat*" \
    "elfutils*" \
    "gcc*" \
    "libaio*" \
    "sysstat*" \
    "unixODBC*"
    action : Our base installation left us needing to issue the following command to install missing packages.
14、安裝GRID RAC 軟體

su - grid
./runInstaller

scan配置:

cluster name(叢集名稱): rac
scanname:  rac-scan
scanport:  1521

sys
asmsnp  password: sysPass  dragonZCLLGM1

 

/opt/oraInventory/orainstRoot.sh

[root@rac1 soft]# /opt/app/oraInventory/orainstRoot.sh
Changing permissions of /opt/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /opt/oraInventory to oinstall.
The execution of the script. is complete.

[root@rac2 soft]# /opt/oraInventory/orainstRoot.sh
Changing permissions of /opt/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /opt/oraInventory to oinstall.
The execution of the script. is complete.

 

 

/opt/grid/product/11.2.0/root.sh

[root@rac1 soft]# /opt/oraInventory/orainstRoot.sh
Changing permissions of /opt/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /opt/oraInventory to oinstall.
The execution of the script. is complete.
[root@rac1 soft]# /opt/grid/product/11.2.0/root.sh
Running Oracle 11g root.sh script...

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /opt/grid/product/11.2.0

Enter the full pathname of the local bin directory: [/usr/local/bin]:
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.

2010-07-28 16:19:03: Parsing the host name
2010-07-28 16:19:03: Checking for super user privileges
2010-07-28 16:19:03: User has super user privileges
Using configuration parameter file: /opt/grid/product/11.2.0/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
  root wallet
  root wallet cert
  root cert export
  peer wallet
  profile reader wallet
  pa wallet
  peer wallet keys
  pa wallet keys
  peer cert request
  pa cert request
  peer cert
  pa cert
  peer root cert TP
  profile reader root cert TP
  pa root cert TP
  peer pa cert TP
  pa peer cert TP
  profile reader pa cert TP
  profile reader peer cert TP
  peer user cert
  pa user cert
Adding daemon to inittab
CRS-4123: Oracle High Availability Services has been started.
ohasd is starting
CRS-4123: Oracle High Availability Services has been started.
ohasd is starting
CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'
CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded
CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'
CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'
CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded

ASM created and started successfully.

DiskGroup CRS created successfully.

clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-2672: Attempting to start 'ora.crsd' on 'rac1'
CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded
CRS-4256: Updating the profile
Successful addition of voting disk a81aaf52b2b74ff5bf7a773e7966ea7c.
Successfully replaced voting disk group with +CRS.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   a81aaf52b2b74ff5bf7a773e7966ea7c (ORCL:CRS1) [CRS]
Located 1 voting disk(s).
CRS-2673: Attempting to stop 'ora.crsd' on 'rac1'
CRS-2677: Stop of 'ora.crsd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1'
CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'rac1'
CRS-2677: Stop of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac1'
CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1'
CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1'
CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1'
CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'
CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'
CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'
CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'
CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac1'
CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rac1'
CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'rac1'
CRS-2676: Start of 'ora.evmd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac1'
CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.CRS.dg' on 'rac1'
CRS-2676: Start of 'ora.CRS.dg' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.registry.acfs' on 'rac1'
CRS-2676: Start of 'ora.registry.acfs' on 'rac1' succeeded

rac1     2010/07/28 16:31:27     /opt/grid/product/11.2.0/cdata/rac1/backup_20100728_163127.olr
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
Updating inventory properties for clusterware
Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB.   Actual 971 MB    Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /opt/oraInventory
'UpdateNodeList' was successful.

 
---------------------- 檢測 -----------------------------------------------

[grid@rac2 ~]$ srvctl enable oc4j
PRKO-2116 : OC4J is already enabled
[grid@rac2 ~]$ srvctl start oc4j
[grid@rac2 ~]$ srvctl enable nodeapps
PRKO-2415 : VIP is already enabled on node(s): rac1,rac2
PRKO-2416 : Network resource is already enabled.
PRKO-2417 : ONS is already enabled on node(s): rac1,rac2
PRKO-2418 : eONS is already enabled on node(s): rac1,rac2

[grid@rac2 ~]$ crs_stat -t
Name           Type           Target    State     Host       
------------------------------------------------------------
ora.CRS.dg     ora....up.type ONLINE    ONLINE    rac1       
ora....ER.lsnr ora....er.type ONLINE    ONLINE    rac1       
ora....N1.lsnr ora....er.type ONLINE    ONLINE    rac1       
ora.asm        ora.asm.type   ONLINE    ONLINE    rac1       
ora.eons       ora.eons.type  ONLINE    ONLINE    rac1       
ora.gsd        ora.gsd.type   ONLINE    ONLINE    rac1       
ora....network ora....rk.type ONLINE    ONLINE    rac1       
ora.oc4j       ora.oc4j.type  ONLINE    ONLINE    rac2       
ora.ons        ora.ons.type   ONLINE    ONLINE    rac1       
ora....SM1.asm application    ONLINE    ONLINE    rac1       
ora....C1.lsnr application    ONLINE    ONLINE    rac1       
ora.rac1.gsd   application    ONLINE    ONLINE    rac1       
ora.rac1.ons   application    ONLINE    ONLINE    rac1       
ora.rac1.vip   ora....t1.type ONLINE    ONLINE    rac1       
ora....SM2.asm application    ONLINE    ONLINE    rac2       
ora....C2.lsnr application    ONLINE    ONLINE    rac2       
ora.rac2.gsd   application    ONLINE    ONLINE    rac2       
ora.rac2.ons   application    ONLINE    ONLINE    rac2       
ora.rac2.vip   ora....t1.type ONLINE    ONLINE    rac2       
ora....ry.acfs ora....fs.type ONLINE    ONLINE    rac1       
ora.scan1.vip  ora....ip.type ONLINE    ONLINE    rac1 


6、
su - grid
./runInstaller

scan配置:
cluster scan: sanclusters
scanname:  racscan
scanport:  1521


/opt/oraInventory/orainstRoot.sh

[root@rac1 soft]# /opt/oraInventory/orainstRoot.sh
Changing permissions of /opt/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /opt/oraInventory to oinstall.
The execution of the script. is complete.

[root@rac2 soft]# /opt/oraInventory/orainstRoot.sh
Changing permissions of /opt/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /opt/oraInventory to oinstall.
The execution of the script. is complete.

  

/opt/grid/product/11.2.0/root.sh

[root@rac1 soft]# /opt/oraInventory/orainstRoot.sh
Changing permissions of /opt/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /opt/oraInventory to oinstall.
The execution of the script. is complete.
[root@rac1 soft]# /opt/grid/product/11.2.0/root.sh
Running Oracle 11g root.sh script...

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /opt/grid/product/11.2.0

Enter the full pathname of the local bin directory: [/usr/local/bin]:
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...
 

Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
2010-07-28 16:19:03: Parsing the host name
2010-07-28 16:19:03: Checking for super user privileges
2010-07-28 16:19:03: User has super user privileges
Using configuration parameter file: /opt/grid/product/11.2.0/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
  root wallet
  root wallet cert
  root cert export
  peer wallet
  profile reader wallet
  pa wallet
  peer wallet keys
  pa wallet keys
  peer cert request
  pa cert request
  peer cert
  pa cert
  peer root cert TP
  profile reader root cert TP
  pa root cert TP
  peer pa cert TP
  pa peer cert TP
  profile reader pa cert TP
  profile reader peer cert TP
  peer user cert
  pa user cert
Adding daemon to inittab
CRS-4123: Oracle High Availability Services has been started.
ohasd is starting
CRS-4123: Oracle High Availability Services has been started.
ohasd is starting
CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'
CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded
CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'
CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'
CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded
ASM created and started successfully.
DiskGroup CRS created successfully.
clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-2672: Attempting to start 'ora.crsd' on 'rac1'
CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded
CRS-4256: Updating the profile
Successful addition of voting disk a81aaf52b2b74ff5bf7a773e7966ea7c.
Successfully replaced voting disk group with +CRS.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   a81aaf52b2b74ff5bf7a773e7966ea7c (ORCL:CRS1) [CRS]
Located 1 voting disk(s).
CRS-2673: Attempting to stop 'ora.crsd' on 'rac1'
CRS-2677: Stop of 'ora.crsd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1'
CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'rac1'
CRS-2677: Stop of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac1'
CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1'
CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1'
CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1'
CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'
CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'
CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'
CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'
CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac1'
CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rac1'
CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'rac1'
CRS-2676: Start of 'ora.evmd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac1'
CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.CRS.dg' on 'rac1'
CRS-2676: Start of 'ora.CRS.dg' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.registry.acfs' on 'rac1'
CRS-2676: Start of 'ora.registry.acfs' on 'rac1' succeeded
rac1     2010/07/28 16:31:27     /opt/grid/product/11.2.0/cdata/rac1/backup_20100728_163127.olr
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
Updating inventory properties for clusterware
Starting Oracle Universal Installer...
Checking swap space: must be greater than 500 MB.   Actual 971 MB    Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /opt/oraInventory
'UpdateNodeList' was successful.
 
[root@rac2 soft]# /opt/grid/product/11.2.0/root.sh
Running Oracle 11g root.sh script...
The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /opt/grid/product/11.2.0
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file "dbhome" already exists in /usr/local/bin.  Overwrite it? (y/n)
[n]: y
   Copying dbhome to /usr/local/bin ...
The file "oraenv" already exists in /usr/local/bin.  Overwrite it? (y/n)
[n]: y
   Copying oraenv to /usr/local/bin ...
The file "coraenv" already exists in /usr/local/bin.  Overwrite it? (y/n)
[n]: y
   Copying coraenv to /usr/local/bin ...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
2010-08-02 14:32:28: Parsing the host name
2010-08-02 14:32:28: Checking for super user privileges
2010-08-02 14:32:28: User has super user privileges
Using configuration parameter file: /opt/grid/product/11.2.0/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Adding daemon to inittab
CRS-4123: Oracle High Availability Services has been started.
ohasd is starting
CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node rac1, number 1, and is terminating
An active cluster was found during exclusive startup, restarting to join the cluster
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac2'
CRS-2676: Start of 'ora.mdnsd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'rac2'
CRS-2676: Start of 'ora.gipcd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac2'
CRS-2676: Start of 'ora.gpnpd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac2'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac2'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac2'
CRS-2676: Start of 'ora.diskmon' on 'rac2' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'rac2'
CRS-2676: Start of 'ora.ctssd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.drivers.acfs' on 'rac2'
CRS-2676: Start of 'ora.drivers.acfs' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac2'
CRS-2676: Start of 'ora.asm' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rac2'
CRS-2676: Start of 'ora.crsd' on 'rac2' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'rac2'
CRS-2676: Start of 'ora.evmd' on 'rac2' succeeded
rac2     2010/08/02 14:37:51     /opt/grid/product/11.2.0/cdata/rac2/backup_20100802_143751.olr
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
Updating inventory properties for clusterware
Starting Oracle Universal Installer...
Checking swap space: must be greater than 500 MB.   Actual 1202 MB    Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /opt/oraInventory
'UpdateNodeList' was successful.


Oc4j,gsd服務預設是disable的,如果要啟動,啟用即可

[grid@rac2 ~]$ srvctl enable oc4j
PRKO-2116 : OC4J is already enabled
[grid@rac2 ~]$ srvctl start oc4j
[grid@rac2 ~]$ srvctl enable nodeapps
PRKO-2415 : VIP is already enabled on node(s): rac1,rac2
PRKO-2416 : Network resource is already enabled.
PRKO-2417 : ONS is already enabled on node(s): rac1,rac2
PRKO-2418 : eONS is already enabled on node(s): rac1,rac2

[grid@rac2 ~]$ crs_stat -t
Name           Type           Target    State     Host       
------------------------------------------------------------
ora.CRS.dg     ora....up.type ONLINE    ONLINE    rac1       
ora....ER.lsnr ora....er.type ONLINE    ONLINE    rac1       
ora....N1.lsnr ora....er.type ONLINE    ONLINE    rac1       
ora.asm        ora.asm.type   ONLINE    ONLINE    rac1       
ora.eons       ora.eons.type  ONLINE    ONLINE    rac1       
ora.gsd        ora.gsd.type   ONLINE    ONLINE    rac1       
ora....network ora....rk.type ONLINE    ONLINE    rac1       
ora.oc4j       ora.oc4j.type  ONLINE    ONLINE    rac2       
ora.ons        ora.ons.type   ONLINE    ONLINE    rac1       
ora....SM1.asm application    ONLINE    ONLINE    rac1       
ora....C1.lsnr application    ONLINE    ONLINE    rac1       
ora.rac1.gsd   application    ONLINE    ONLINE    rac1       
ora.rac1.ons   application    ONLINE    ONLINE    rac1       
ora.rac1.vip   ora....t1.type ONLINE    ONLINE    rac1       
ora....SM2.asm application    ONLINE    ONLINE    rac2       
ora....C2.lsnr application    ONLINE    ONLINE    rac2       
ora.rac2.gsd   application    ONLINE    ONLINE    rac2       
ora.rac2.ons   application    ONLINE    ONLINE    rac2       
ora.rac2.vip   ora....t1.type ONLINE    ONLINE    rac2       
ora....ry.acfs ora....fs.type ONLINE    ONLINE    rac1       
ora.scan1.vip  ora....ip.type ONLINE    ONLINE    rac1 

 
資源組狀態:
crsctl status resource -t
[grid@rac2 ~]$ crsctl status resource -t

NAME           TARGET  STATE        SERVER                   STATE_DETAILS       Local Resources
ora.CRS.dg
               ONLINE  ONLINE       rac1                                        
               ONLINE  ONLINE       rac2                                        
ora.LISTENER.lsnr
               ONLINE  ONLINE       rac1                                        
               ONLINE  ONLINE       rac2                                        
ora.asm
               ONLINE  ONLINE       rac1                     Started            
               ONLINE  ONLINE       rac2                     Started            
ora.eons
               ONLINE  ONLINE       rac1                                        
               ONLINE  ONLINE       rac2                                        
ora.gsd
               ONLINE  ONLINE       rac1                                        
               ONLINE  ONLINE       rac2                                        
ora.net1.network
               ONLINE  ONLINE       rac1                                        
               ONLINE  ONLINE       rac2                                        
ora.ons
               ONLINE  ONLINE       rac1                                        
               ONLINE  ONLINE       rac2                                        
ora.registry.acfs
               ONLINE  ONLINE       rac1                                        
               ONLINE  ONLINE       rac2     
                                   
Cluster Resources

ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       rac1                                        
ora.oc4j
      1        ONLINE  ONLINE       rac2                                        
ora.rac1.vip
      1        ONLINE  ONLINE       rac1                                        
ora.rac2.vip
      1        ONLINE  ONLINE       rac2                                        
ora.scan1.vip
      1        ONLINE  ONLINE       rac1                                    
  
檢查叢集節點:
olsnodes -l
檢查 Oracle 叢集登錄檔 (OCR):
ocrcheck
[grid@rac1 ~]$  ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2404
         Available space (kbytes) :     259716
         ID                       : 1095774749
         Device/File Name         :       +CRS
                                    Device/File integrity check succeeded
                                    Device/File not configured
                                    Device/File not configured
                                    Device/File not configured
                                    Device/File not configured
         Cluster registry integrity check succeeded
         Logical corruption check bypassed due to non-privileged user
  
  
[grid@rac2 ~]$ ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2404
         Available space (kbytes) :     259716
         ID                       : 1095774749
         Device/File Name         :       +CRS
                                    Device/File integrity check failed
                                    Device/File not configured
                                    Device/File not configured
                                    Device/File not configured
                                    Device/File not configured
         Cluster registry integrity check failed
         Logical corruption check bypassed due to insufficient quorum
檢查表決磁碟:
crsctl query css votedisk
[grid@rac1 ~]$ crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   589c0071c9304fecbff651e64eeb281a (ORCL:CRS1) [CRS]
Located 1 voting disk(s).

Oc4j,gsd服務預設是disable的,如果要啟動,啟用即可


重啟grid cluster

為資料和快速恢復區建立 ASM 磁碟組:
asmca
安裝ORACLE 11G 
su - oracle
./runInstaller
[root@rac2 ~]# /opt/oracle/product/11.2.0/root.sh
Running Oracle 11g root.sh script...
The following environment variables are set as:
    ORACLE_OWNER= oracle 

    ORACLE_HOME=  /opt/oracle/product/11.2.0
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file "dbhome" already exists in /usr/local/bin.  Overwrite it? (y/n)
[n]: y
   Copying dbhome to /usr/local/bin ...
The file "oraenv" already exists in /usr/local/bin.  Overwrite it? (y/n)
[n]: y
   Copying oraenv to /usr/local/bin ...
The file "coraenv" already exists in /usr/local/bin.  Overwrite it? (y/n)
[n]: y
   Copying coraenv to /usr/local/bin ...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
Finished product-specific root actions.


[root@rac2 ~]# crs_stat -t


TAG:


    Creating /etc/oratab file...
    Entries will be added to the /etc/oratab file as needed by
    Database Configuration Assistant when a database is created
    Finished running generic part of root.sh script.
    Now product-specific root actions will be performed.
    2010-07-28 16:19:03: Parsing the host name
    2010-07-28 16:19:03: Checking for super user privileges
    2010-07-28 16:19:03: User has super user privileges
    Using configuration parameter file: /opt/grid/product/11.2.0/crs/install/crsconfig_params
    Creating trace directory
    LOCAL ADD MODE
    Creating OCR keys for user 'root', privgrp 'root'..
    Operation successful.
      root wallet
      root wallet cert
      root cert export
      peer wallet
      profile reader wallet
      pa wallet
      peer wallet keys
      pa wallet keys
      peer cert request
      pa cert request
      peer cert
      pa cert
      peer root cert TP
      profile reader root cert TP
      pa root cert TP
      peer pa cert TP
      pa peer cert TP
      profile reader pa cert TP
      profile reader peer cert TP
      peer user cert
      pa user cert
    Adding daemon to inittab
    CRS-4123: Oracle High Availability Services has been started.
    ohasd is starting
    CRS-4123: Oracle High Availability Services has been started.
    ohasd is starting
    CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'
    CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'
    CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded
    CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'
    CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
    CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
    CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
    CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
    CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'
    CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded
    ASM created and started successfully.
    DiskGroup CRS created successfully.
    clscfg: -install mode specified
    Successfully accumulated necessary OCR keys.
    Creating OCR keys for user 'root', privgrp 'root'..
    Operation successful.
    CRS-2672: Attempting to start 'ora.crsd' on 'rac1'
    CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded
    CRS-4256: Updating the profile
    Successful addition of voting disk a81aaf52b2b74ff5bf7a773e7966ea7c.
    Successfully replaced voting disk group with +CRS.
    CRS-4256: Updating the profile
    CRS-4266: Voting file(s) successfully replaced
    ##  STATE    File Universal Id                File Name Disk group
    --  -----    -----------------                --------- ---------
    1. ONLINE   a81aaf52b2b74ff5bf7a773e7966ea7c (ORCL:CRS1) [CRS]
    Located 1 voting disk(s).
    CRS-2673: Attempting to stop 'ora.crsd' on 'rac1'
    CRS-2677: Stop of 'ora.crsd' on 'rac1' succeeded
    CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
    CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
    CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1'
    CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded
    CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'rac1'
    CRS-2677: Stop of 'ora.cssdmonitor' on 'rac1' succeeded
    CRS-2673: Attempting to stop 'ora.cssd' on 'rac1'
    CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded
    CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1'
    CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded
    CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1'
    CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded
    CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1'
    CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'
    CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'
    CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'
    CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
    CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
    CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
    CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
    CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'
    CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.asm' on 'rac1'
    CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.crsd' on 'rac1'
    CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.evmd' on 'rac1'
    CRS-2676: Start of 'ora.evmd' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.asm' on 'rac1'
    CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.CRS.dg' on 'rac1'
    CRS-2676: Start of 'ora.CRS.dg' on 'rac1' succeeded
    CRS-2672: Attempting to start 'ora.registry.acfs' on 'rac1'
    CRS-2676: Start of 'ora.registry.acfs' on 'rac1' succeeded
    rac1     2010/07/28 16:31:27     /opt/grid/product/11.2.0/cdata/rac1/backup_20100728_163127.olr
    Configure Oracle Grid Infrastructure for a Cluster ... succeeded
    Updating inventory properties for clusterware
    Starting Oracle Universal Installer...
    Checking swap space: must be greater than 500 MB.   Actual 971 MB    Passed
    The inventory pointer is located at /etc/oraInst.loc
    The inventory is located at /opt/oraInventory
    'UpdateNodeList' was successful.

    [root@rac2 soft]# /opt/grid/product/11.2.0/root.sh
    Running Oracle 11g root.sh script...
    The following environment variables are set as:
        ORACLE_OWNER= grid
        ORACLE_HOME=  /opt/grid/product/11.2.0
    Enter the full pathname of the local bin directory: [/usr/local/bin]:
    The file "dbhome" already exists in /usr/local/bin.  Overwrite it? (y/n)
    [n]: y
       Copying dbhome to /usr/local/bin ...
    The file "oraenv" already exists in /usr/local/bin.  Overwrite it? (y/n)
    [n]: y
       Copying oraenv to /usr/local/bin ...
    The file "coraenv" already exists in /usr/local/bin.  Overwrite it? (y/n)
    [n]: y
       Copying coraenv to /usr/local/bin ...
    Entries will be added to the /etc/oratab file as needed by
    Database Configuration Assistant when a database is created
    Finished running generic part of root.sh script.
    Now product-specific root actions will be performed.
    2010-08-02 14:32:28: Parsing the host name
    2010-08-02 14:32:28: Checking for super user privileges
    2010-08-02 14:32:28: User has super user privileges
    Using configuration parameter file: /opt/grid/product/11.2.0/crs/install/crsconfig_params
    Creating trace directory
    LOCAL ADD MODE
    Creating OCR keys for user 'root', privgrp 'root'..
    Operation successful.
    Adding daemon to inittab
    CRS-4123: Oracle High Availability Services has been started.
    ohasd is starting
    CRS-4402: The CSS daemon was started in exclusive mode but found an active 
    CSS daemon on node rac1, number 1, and is terminating
    An active cluster was found during exclusive startup, restarting to join the cluster
    CRS-2672: Attempting to start 'ora.mdnsd' on 'rac2'
    CRS-2676: Start of 'ora.mdnsd' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.gipcd' on 'rac2'
    CRS-2676: Start of 'ora.gipcd' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.gpnpd' on 'rac2'
    CRS-2676: Start of 'ora.gpnpd' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac2'
    CRS-2676: Start of 'ora.cssdmonitor' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.cssd' on 'rac2'
    CRS-2672: Attempting to start 'ora.diskmon' on 'rac2'
    CRS-2676: Start of 'ora.diskmon' on 'rac2' succeeded
    CRS-2676: Start of 'ora.cssd' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.ctssd' on 'rac2'
    CRS-2676: Start of 'ora.ctssd' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.drivers.acfs' on 'rac2'
    CRS-2676: Start of 'ora.drivers.acfs' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.asm' on 'rac2'
    CRS-2676: Start of 'ora.asm' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.crsd' on 'rac2'
    CRS-2676: Start of 'ora.crsd' on 'rac2' succeeded
    CRS-2672: Attempting to start 'ora.evmd' on 'rac2'
    CRS-2676: Start of 'ora.evmd' on 'rac2' succeeded
    rac2     2010/08/02 14:37:51     /opt/grid/product/11.2.0/cdata/rac2/backup_20100802_143751.olr
    Configure Oracle Grid Infrastructure for a Cluster ... succeeded
    Updating inventory properties for clusterware
    Starting Oracle Universal Installer...
    Checking swap space: must be greater than 500 MB.   Actual 1202 MB    Passed
    The inventory pointer is located at /etc/oraInst.loc
    The inventory is located at /opt/oraInventory
    'UpdateNodeList' was successful.




    --檢視歸檔丟失的情況:

    SELECT THREAD#,LOW_SEQUENCE#,HIGH_SEQUENCE# from v$archive_gap;


    --啟動閃回

    alter system set db_recovery_file_dest_size=2g scope=both;

    alter system set db_recovery_file_dest='+DG_RECOVERY' scope=both;

    alter database flashback ON;

    SQL> show parameter recovery   

    NAME                                 TYPE        VALUE
    ------------------------------------ ----------- ------------------------------
    db_recovery_file_dest                string      +DG_RECOVERY
    db_recovery_file_dest_size           big integer 2G
    recovery_parallelism                 integer     0


---------------------------------------------------------------------------------------
    tnsnames:
      
      RACDB1 =
      (DESCRIPTION =
        (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.10.21)(PORT = 1521))
        (CONNECT_DATA =
          (SERVER = DEDICATED)
          (SERVICE_NAME = racdb)
          (INSTANCE_NAME = racdb1)
        )
      )

    RACDB =
      (DESCRIPTION =
        (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.10.21)(PORT = 1521))
        (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.10.22)(PORT = 1521))
        (LOAD_BALANCE = yes)
        (CONNECT_DATA =
          (SERVER = DEDICATED)
          (SERVICE_NAME = racdb)
          (FAILOVER_MODE =
            (TYPE = SELECT)
            (METHOD = BASIC)
            (RETRIES = 180)
            (DELAY = 5)
          )
        )
      )

    LISTENERS_RACDB =
      (ADDRESS_LIST =
        (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.10.21)(PORT = 1521))
        (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.10.22)(PORT = 1521))
      )
      
    LISTENERS_RACDB1 =
      (ADDRESS_LIST =
        (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.10.21)(PORT = 1521))
      )

    LISTENERS_RACDB2 =
      (ADDRESS_LIST =
        (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.10.22)(PORT = 1521))
      )
      
      
    EXTPROC_CONNECTION_DATA =
      (DESCRIPTION =
        (ADDRESS_LIST =
          (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC0))
        )
        (CONNECT_DATA =
          (SID = PLSExtProc)
          (PRESENTATION = RO)
        )
      )

    RACDB2 =
      (DESCRIPTION =
        (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.10.22)(PORT = 1521))
        (CONNECT_DATA =
          (SERVER = DEDICATED)
          (SERVICE_NAME = racdb)
          (INSTANCE_NAME = racdb2)
        )
      )
      
      
    alter system set LOCAL_LISTENER = 'LISTENERS_RACDB1' scope=both sid='racdb1';
    alter system set REMOTE_LISTENER = 'LISTENERS_RACDB' scope=both sid='racdb1';

    alter system set LOCAL_LISTENER = 'LISTENERS_RACDB2' scope=both sid='racdb2';
    alter system set REMOTE_LISTENER = 'LISTENERS_RACDB' scope=both sid='racdb2';

    ############################################
    10、配置啟動歸檔及閃回

    --先配歸檔才可以啟動閃回

    alter system set cluster_database=false scope=spfile

    #alter system set db_recovery_file_dest='+recovery' scope=both;

    alter system switch logfile;

    startup mount

    alter database archivelog;

    alter system set cluster_database=true scope=spfile

    shutdown immediate;

    startup


    [root@rac2 ~]# crs_stat -t
    Name           Type           Target    State     Host        
    ------------------------------------------------------------
    ora....SM1.asm application    ONLINE    ONLINE    rac1        
    ora....C1.lsnr application    ONLINE    ONLINE    rac1        
    ora.rac1.gsd   application    ONLINE    ONLINE    rac1        
    ora.rac1.ons   application    ONLINE    ONLINE    rac1        
    ora.rac1.vip   application    ONLINE    ONLINE    rac1        
    ora....SM2.asm application    ONLINE    ONLINE    rac2        
    ora....C2.lsnr application    ONLINE    ONLINE    rac2        
    ora.rac2.gsd   application    ONLINE    ONLINE    rac2        
    ora.rac2.ons   application    ONLINE    ONLINE    rac2        
    ora.rac2.vip   application    ONLINE    ONLINE    rac2        
    ora.racdb.db   application    ONLINE    ONLINE    rac1        
    ora....b1.inst application    ONLINE    ONLINE    rac1        
    ora....b2.inst application    ONLINE    ONLINE    rac2        
    ora....db_s.cs application    ONLINE    ONLINE    rac1        
    ora....db1.srv application    ONLINE    ONLINE    rac1        
    ora....db2.srv application    ONLINE    ONLINE    rac2        

    與之前不同service名後,則資料庫會自己變為新的service名字,如下:

    NAME                                 TYPE        VALUE
    ------------------------------------ ----------- ------------------------------
    service_names                        string      racdb_s


    7、crs_stat -t

    [oracle@rac2 bin]$ crs_stat -t            
    Name           Type           Target    State     Host        
    ------------------------------------------------------------
    ora.rac1.gsd   application    ONLINE    ONLINE    rac1        
    ora.rac1.ons   application    ONLINE    ONLINE    rac1        
    ora.rac1.vip   application    ONLINE    ONLINE    rac1        
    ora.rac2.gsd   application    ONLINE    ONLINE    rac2        
    ora.rac2.ons   application    ONLINE    ONLINE    rac2        
    ora.rac2.vip   application    ONLINE    ONLINE    rac2        


    ALTER DISKGROUP RACDISK DROP DISK RACDISK_0001 REBALANCE POWER 11;


    8、rac status

    [oracle@rac2 bin]$ crs_stat -t
    Name           Type           Target    State     Host        
    ------------------------------------------------------------
    ora....SM1.asm application    ONLINE    ONLINE    rac1        
    ora....C1.lsnr application    ONLINE    ONLINE    rac1        
    ora.rac1.gsd   application    ONLINE    ONLINE    rac1        
    ora.rac1.ons   application    ONLINE    ONLINE    rac1        
    ora.rac1.vip   application    ONLINE    ONLINE    rac1        
    ora....SM2.asm application    ONLINE    ONLINE    rac2        
    ora....C2.lsnr application    ONLINE    ONLINE    rac2        
    ora.rac2.gsd   application    ONLINE    ONLINE    rac2        
    ora.rac2.ons   application    ONLINE    ONLINE    rac2        
    ora.rac2.vip   application    ONLINE    ONLINE    rac2        
    ora.racdb.db   application    ONLINE    ONLINE    rac2        
    ora....acdb.cs application    ONLINE    ONLINE    rac1        
    ora....db1.srv application    ONLINE    ONLINE    rac1        
    ora....db2.srv application    ONLINE    ONLINE    rac2        
    ora....b1.inst application    ONLINE    ONLINE    rac1        
    ora....b2.inst application    ONLINE    ONLINE    rac2 
9、listener.ora與tnsnames.ora

使用 netca 或是netmgr 設定好監聽和tnsnames.ora

cat /opt/app/grid/network/admin/listener.ora
LISTENER=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER))))       # line added by Agent 

LISTENER_SCAN1=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN1))))        # line added by Agent
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER_SCAN1=ON        # line added by Agent
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER=ON        # line added by Agent



---------------------------------------------
cat /opt/app/oracle/product/11.2.0/network/admin/listener.ora
# listener.ora Network Configuration File: /opt/app/oracle/product/11.2.0/network/admin/listener.ora
# Generated by Oracle configuration tools.

LISTENER =
  (DESCRIPTION_LIST =
    (DESCRIPTION =
      (ADDRESS = (PROTOCOL = TCP)(HOST = rac2-vip)(PORT = 1521))
    )
    (DESCRIPTION =
      (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.18.103)(PORT = 1521))
    )
  )

ADR_BASE_LISTENER = /opt/app/oracle


----------------------------------------------------
# tnsnames.ora Network Configuration File: /opt/app/oracle/product/11.2.0/network/admin/tnsnames.ora
# Generated by Oracle configuration tools.

RACDB1 =
  (DESCRIPTION =
    (ADDRESS_LIST =
      (ADDRESS = (PROTOCOL = TCP)(HOST = rac1-vip)(PORT = 1521))
    )
    (CONNECT_DATA =
      (SERVICE_NAME = racdb1)
    )
  )

RACDB =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = rac-scan)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = racdb)
    )
  )

RACDB2 =
  (DESCRIPTION =
    (ADDRESS_LIST =
      (ADDRESS = (PROTOCOL = TCP)(HOST = rac2-vip)(PORT = 1521))



***************************************************************************

配置建立資料庫
    su - oracle
    ./dbca


來自 “ ITPUB部落格 ” ,連結:http://blog.itpub.net/22418990/viewspace-739748/,如需轉載,請註明出處,否則將追究法律責任。

相關文章