京東雲環境搭建oracle rac詳細部署梳理(可信的結果輸出)

caohongfeng666發表於2018-11-14

一、環境準備

雲主機數量:2臺
雲硬碟資料:1塊(透過分割槽,邏輯上產生多個盤。正式環境要用獨立的盤)
作業系統環境:centos 6.5
資料庫版本: oracle 11.2.0.4

1.叢集的IP規劃

#pub
10.10.10.101 oracle-rac1
10.10.10.102 oracle-rac2
#vip
10.10.10.103 oracle-rac1-vip
10.10.10.104 oracle-rac2-vip 
#priv
192.168.100.101 oracle-rac1-priv 
192.168.100.102 oracle-rac2-priv 
#scan-ip
10.10.10.105 scan-ip

2.在各個節點安裝n2n

(1). 安裝n2n
cd /usr/src
svn co 
cd n2n/n2n_v2
make
make PREFIX=/opt/n2n install


(2).配置和啟動n2n服務
n2n服務端啟動supernode
/opt/n2n/sbin/supernode -l 65530

寫入開機自啟動
echo "/opt/n2n/sbin/supernode -l 65530" >> /etc/rc.local

兩個節點執行客戶端,並寫入到/etc/rc.local
節點1
/opt/n2n/sbin/edge -d edge0 -a 10.10.10.101 -s 255.255.255.0 -c dtstack -k dtstack -l 10.0.0.32:65530 -E -r
/opt/n2n/sbin/edge -d edge1 -a 192.168.100.101 -s 255.255.255.0 -c dtstack -k dtstack -l 10.0.0.32:65530 -E -r


節點2
/opt/n2n/sbin/edge -d edge0 -a 10.10.10.102 -s 255.255.255.0 -c dtstack -k dtstack -l 10.0.0.32:65530 -E -r
/opt/n2n/sbin/edge -d edge1 -a 192.168.100.102 -s 255.255.255.0 -c dtstack -k dtstack -l 10.0.0.32:65530 -E -r


3. 使用者和目錄配置
(1). 建立grid、oracle使用者

/usr/sbin/groupadd -g 501 oinstall
/usr/sbin/groupadd -g 502 asmadmin
/usr/sbin/groupadd -g 503 dba
/usr/sbin/groupadd -g 504 oper
/usr/sbin/groupadd -g 505 asmdba
/usr/sbin/groupadd -g 506 asmoper
/usr/sbin/useradd -u 501 -g oinstall -G asmadmin,asmdba,asmoper,oper,dba grid
/usr/sbin/useradd -u 502 -g oinstall -G dba,asmdba,oper oracle


(2). 配置對等性

節點1:
su - grid (oracle)
mkdir .ssh
chmod -R 700 .ssh
ssh-keygen -t rsa
ssh-keygen -t dsa

節點2:
su - grid 
mkdir .ssh
chmod -R 700 .ssh
ssh-keygen -t rsa
ssh-keygen -t dsa

節點1:
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys 
cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
ssh oracle-rac2 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys 
ssh oracle-rac2 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
scp ~/.ssh/authorized_keys oracle-rac2:~/.ssh/authorized_keys


驗證
ssh oracle-rac1 date
ssh oracle-rac2 date
ssh oracle-rac1-priv date 
ssh oracle-rac2-priv date

(3). 建立目錄
mkdir -p /u01/app/
mkdir -p /u01/app/grid_base
mkdir -p /u01/app/grid_home
chown -R grid:oinstall /u01/app/grid_base
chown -R grid:oinstall /u01/app/grid_home
chown -R oracle:oinstall /u01/app
chmod -R 775 /u01/app/

(4). 配置環境變數
#Grid使用者:
vi ~/.bash_profile
umask 022
export ORACLE_BASE=/u01/app/grid_base
export ORACLE_HOME=/u01/app/grid_home
export ORACLE_SID=+ASM1 #第二個節點+ASM2
export PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$PATH

#Oracle使用者:
umask 022
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1
export ORA_GRID_HOME=/u01/app/grid_home
export ORACLE_UNQNAME=orcl 
export ORACLE_SID=orcl1 #第二個節點orcl2
export PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ORACLE_HOME/
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK
export NLS_DATE_FORMAT="YYYY-MM-DD HH24:MI:SS"
export TNS_ADMIN=$ORACLE_HOME/network/admin
export

4. 自建swap
root使用者下
(1).以檔案系統增加(以下是以增加2048M的檔案系統為例)
mkdir -p /home/oracle/myswaps
dd if=/dev/zero of=/home/oracle/myswaps/swapfile1 bs=1M count=2048
  注:of後面的路徑一定不能是/dev下,否則在啟用swap檔案時報引數無效

(2).建立swap檔案
  mkswap /home/oracle/myswaps/swapfile1

(3).啟用swap檔案
  swapon /home/oracle/myswaps/swapfile1

(4).此時用free -m就可以看到swap空間已經增加了,但是重啟後就不會存在了

(5).新增重啟後還會生效,vi /etc/fstab新增如下內容:
  echo "/home/oracle/myswaps/swapfile1 swap swap defaults 0 0" >>/etc/fstab

5. 安裝oracle rac的依賴包安裝

5.1 檢查缺少的rpm包
rpm -q binutils \
compat-libstdc++-33 \
elfutils-libelf \
elfutils-libelf-devel \
gcc \
gcc-c++ \
glibc \
glibc-common \
glibc-devel \
glibc-headers \
ksh \
libaio \
libaio-devel \
libgcc \
libstdc++ \
libstdc++-devel \
libXp \
make \
numactl-devel \
sysstat \
unixODBC \
unixODBC-devel \
compat-libcap1.x86_64 \
libcap.x86_64|grep not


package compat-libstdc++-33 is not installed
package elfutils-libelf-devel is not installed
package ksh is not installed
package libaio-devel is not installed
package libXp is not installed
package numactl-devel is not installed
package sysstat is not installed
package unixODBC is not installed
package unixODBC-devel is not installed
package compat-libcap1.x86_64 is not installed

處理缺少的基礎包:
yum install -y compat-libstdc++-33
yum install -y elfutils-libelf-devel
yum install -y ksh
yum install -y libaio-devel
yum install -y libXp
yum install -y numactl-devel
yum install -y sysstat
yum install -y unixODBC*
yum install -y compat-libcap1.x86_64

 

5.2 2個rpm包的特殊處理 pdksh 和 cvuqdisk ,一個需要單獨下載,一個需要在oracle安裝介質中找

(1). pdksh
ksh無用,需要安裝pdksh
rpm -ivh pdksh-5.2.14-21.x86_64.rpm

pdksh conflicts with ksh-20120801-37.el6_9.x86_64
rpm -e ksh-20120801-37.el6_9.x86_64

(2). cvuqdisk在安裝介質中可以找到

rpm -ivh cvuqdisk-1.0.9-1.rpm

依賴於:yum install -y smartmontools


5.3 安裝asmlib的相關包,oracleasmlib、oracleasm-support、Kmod-oracleasm

rpm -ivh kmod-oracleasm-2.0.6.rh1-3.el6.x86_64.rpm
rpm -ivh oracleasm-support-2.1.8-1.el6.x86_64.rpm 
rpm -ivh oracleasmlib-2.0.4-1.el6.x86_64.rpm

 

6. 系統引數設定

(1). 修改核心引數 
vi /etc/sysctl.conf

echo "fs.aio-max-nr = 1048576" >> /etc/sysctl.conf
echo "fs.file-max = 6815744" >> /etc/sysctl.conf
echo "kernel.shmall =7864320" >> /etc/sysctl.conf 
echo "kernel.shmmax = 52451655680" >> /etc/sysctl.conf
echo "kernel.shmmni = 4096" >> /etc/sysctl.conf
echo "kernel.sem = 250 32000 100 128" >> /etc/sysctl.conf
echo "net.ipv4.ip_local_port_range = 9000 65500" >> /etc/sysctl.conf
echo "net.core.rmem_default = 262144" >> /etc/sysctl.conf
echo "net.core.rmem_max = 4194304" >> /etc/sysctl.conf
echo "net.core.wmem_default = 262144" >> /etc/sysctl.conf
echo "net.core.wmem_max = 1048576" >> /etc/sysctl.conf

sysctl -p 

(2).修改使用者限制
#vi /etc/security/limits.conf

echo "oracle soft nofile 4096" >> /etc/security/limits.conf
echo "oracle hard nofile 65536" >> /etc/security/limits.conf
echo "oracle soft nproc 2047" >> /etc/security/limits.conf
echo "oracle hard nproc 16384" >> /etc/security/limits.conf
echo "oracle soft stack 10240" >> /etc/security/limits.conf
echo "grid soft nofile 4096" >> /etc/security/limits.conf
echo "grid hard nofile 65536" >> /etc/security/limits.conf
echo "grid soft nproc 2047" >> /etc/security/limits.conf
echo "grid hard nproc 16384" >> /etc/security/limits.conf
echo "grid soft stack 10240" >> /etc/security/limits.conf
echo "* soft memlock 18874368" >> /etc/security/limits.conf
echo "* hard memlock 18874368" >> /etc/security/limits.conf

 

(3). 修改/etc/pam.d/login,加入認證模組
echo "session required  " >> /etc/pam.d/login

(4). disable iptables 
chkconfig --list iptables
chkconfig iptables off
chkconfig --list iptables
service iptables stop
service network restart

(5). disable selinux
#### echo "SELINUX=disabled" >>/etc/selinux/config

(6). disable ntp server
/sbin/service ntpd stop
chkconfig ntpd off
mv /etc/ntp.conf /etc/ntp.conf.bk
rm /var/run/ntpd.pid

------------------------------------
6. 配置asmlib

(1).分割槽規劃(6個分割槽)

parted /dev/vdb mklabel gpt
parted /dev/vdb mkpart primary 0 10000 
parted /dev/vdb mkpart primary 10000 20000
parted /dev/vdb mkpart primary 20000 30000
parted /dev/vdb mkpart primary 30000 40000
parted /dev/vdb mkpart primary 40000 50000
parted /dev/vdb mkpart primary 50000 60000
parted /dev/vdb p


(2). 配置asmlib
/usr/sbin/oracleasm configure -i #兩個節點
/etc/init.d/oracleasm enable #兩個節點
/etc/init.d/oracleasm start #兩個節點

#本次建立的磁碟
/etc/init.d/oracleasm createdisk VOLCRS01 /dev/vdb1
/etc/init.d/oracleasm createdisk VOLCRS02 /dev/vdb2
/etc/init.d/oracleasm createdisk VOLCRS03 /dev/vdb3

/etc/init.d/oracleasm createdisk VOLDATA01 /dev/vdb4
/etc/init.d/oracleasm createdisk VOLDATA02 /dev/vdb5
/etc/init.d/oracleasm createdisk VOLDATA03 /dev/vdb6

/usr/sbin/oracleasm scandisks #一個節點建立磁碟後,另一個節點掃描磁碟即可
/usr/sbin/oracleasm listdisks #

------------------------------------------------------------------------------------------------
************************************************************************************************
------------------------------------------------------------------------------------------------
二、grid安裝

1. 環境核驗

/home/grid/grid/runcluvfy.sh stage -pre crsinst -n oracle-rac1,oracle-rac2 -verbose 


2. 準備響應檔案

修改參考:
ORACLE_HOSTNAME=oracle-rac1
INVENTORY_LOCATION=/u01/app/oraInventory
oracle.install.option=CRS_CONFIG
ORACLE_BASE=/u01/app/grid_base
ORACLE_HOME=/u01/app/grid_home
oracle.install.asm.OSDBA=asmdba
oracle.install.asm.OSOPER=asmoper
oracle.install.asm.OSASM=asmadmin
oracle.install.crs.config.gpnp.scanPort=1521
oracle.install.crs.config.clusterName=rac-cluster
oracle.install.crs.config.clusterNodes=oracle-rac1:oracle-rac1-vip,oracle-rac2:oracle-rac2-vip
oracle.install.crs.config.privateInterconnects=edge1:192.168.100.0:2,edge0:10.10.10.0:1 //1代表public,2代表private,3代表在群集中不使用該網路卡
oracle.install.crs.config.storageOption=ASM_STORAGE
oracle.install.asm.SYSASMPassword=Grid123
=OCR
oracle.install.asm.diskGroup.redundancy=NORMAL
oracle.install.asm.diskGroup.AUSize=1
oracle.install.asm.diskGroup.disks=/dev/oracleasm/disks/VOLCRS01,/dev/oracleasm/disks/VOLCRS02,/dev/oracleasm/disks/VOLCRS03
oracle.install.asm.diskGroup.diskDiscoveryString=/dev/oracleasm/disks
oracle.install.asm.monitorPassword=Grid123

3. 安裝grid

./runInstaller -silent -responseFile /home/grid/grid/response/grid_install_jdtest.rsp -ignoreSysPrereqs -ignorePrereq

等待一段時間後,會提示在兩個節點分別用root使用者執行:
/u01/grid/oraInventory/orainstRoot.sh
/u01/crs/11.2.0/root.sh

 

4. grid安裝完成,檢視叢集資源的輸出如下:
[grid@oracle-rac1 response]$ crs_stat -t
Name Type Target State Host 
------------------------------------------------------------
ora....ER.lsnr ora....er.type ONLINE ONLINE oracle-rac1 
ora....N1.lsnr ora....er.type ONLINE ONLINE oracle-rac1 
ora.OCR.dg ora....up.type ONLINE ONLINE oracle-rac1 
ora.asm ora.asm.type ONLINE ONLINE oracle-rac1 
ora.cvu ora.cvu.type ONLINE ONLINE oracle-rac1 
ora.gsd ora.gsd.type OFFLINE OFFLINE 
ora....network ora....rk.type ONLINE ONLINE oracle-rac1 
ora.oc4j ora.oc4j.type ONLINE ONLINE oracle-rac1 
ora.ons ora.ons.type ONLINE ONLINE oracle-rac1 
ora....SM1.asm application ONLINE ONLINE oracle-rac1 
ora....C1.lsnr application ONLINE ONLINE oracle-rac1 
ora....ac1.gsd application OFFLINE OFFLINE 
ora....ac1.ons application ONLINE ONLINE oracle-rac1 
ora....ac1.vip ora....t1.type ONLINE ONLINE oracle-rac1 
ora....SM2.asm application ONLINE ONLINE oracle-rac2 
ora....C2.lsnr application ONLINE ONLINE oracle-rac2 
ora....ac2.gsd application OFFLINE OFFLINE 
ora....ac2.ons application ONLINE ONLINE oracle-rac2 
ora....ac2.vip ora....t1.type ONLINE ONLINE oracle-rac2 
ora.scan1.vip ora....ip.type ONLINE ONLINE oracle-rac1 
[grid@oracle-rac1 response]$ crsctl check cluster -all
**************************************************************
oracle-rac1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
oracle-rac2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************

5. 建立asmdisk group
#su – grid

使用命令列
asmca -silent -createDiskGroup -sysAsmPassword Grid123 -diskString '/dev/oracleasm/disks/*' -diskGroupName DATA -diskList '/dev/oracleasm/disks/VOLDATA01' -redundancy EXTERNAL -compatible.asm 11.2 -compatible.rdbms 11.2
asmca -silent -addDisk -sysAsmPassword Grid123 -diskGroupName DATA -diskList '/dev/oracleasm/disks/VOLDATA02','/dev/oracleasm/disks/VOLDATA03'


檢視資訊:
select name,path,STATE,GROUP_NUMBER 
from v$asm_disk;

select name,state,type,total_mb,free_mb 
from v$asm_diskgroup;

-------------------------------------------------------------------------------------------------
*************************************************************************************************
-------------------------------------------------------------------------------------------------

三、安裝oracle資料庫軟體:

1.安裝前驗證:

/home/grid/grid/runcluvfy.sh stage -pre dbinst -n oracle-rac1,oracle-rac2 -verbose


2. 編譯響應檔案
/home/oracle/database/response/db_install_jdtest.rsp

修改參考:
oracle.install.option=INSTALL_DB_SWONLY
ORACLE_HOSTNAME=oracle-rac1
UNIX_GROUP_NAME=oinstall
INVENTORY_LOCATION=/u01/app/oraInventory
SELECTED_LANGUAGES=en
ORACLE_HOME=/u01/app/oracle/product/11.2.0/db_1
ORACLE_BASE=/u01/app/oracle
oracle.install.db.InstallEdition=EE
oracle.install.db.EEOptionsSelection=false
oracle.install.db.optionalComponents=oracle.rdbms.partitioning:11.2.0.4.0,oracle.oraolap:11.2.0.4.0, :11.2.0.4.0,oracle.rdbms.dv:11.2.0.4.0,oracle.rdbms.lbac:11.2.0.4.0,oracle.rdbms.rat:11.2.0.4.0
oracle.install.db.DBA_GROUP=dba
oracle.install.db.OPER_GROUP=oinstall
oracle.install.db.CLUSTER_NODES=rac1,rac2
DECLINE_SECURITY_UPDATES=true

3. 執行靜默安裝oracle軟體
./runInstaller -silent -ignoreSysPrereqs -ignorePrereq -responseFile /home/oracle/database/response/db_install_jdtest.rsp

根據提示執行下面的指令碼:
/u01/app/oracle/product/11.2.0/db_1/root.sh

-------------------------------------------------------------------------------------------------
*************************************************************************************************
-------------------------------------------------------------------------------------------------
四、安裝DB

1. 準備響應檔案
/u01/soft/oracle/database/response/dbca_jdtest.rsp

修改參考:
GDBNAME = "orcl"
SID = "orcl"
NODELIST=oracle-rac1,oracle-rac2
SYSPASSWORD = "Oracle123"
SYSTEMPASSWORD = "Oracle123"
STORAGETYPE=ASM
DISKGROUPNAME=DATA
RECOVERYGROUPNAME=DATA
CHARACTERSET = "ZHS16GBK"

2. 靜默建立資料庫
dbca -silent -responseFile /home/oracle/database/response/dbca_jdtest.rsp


--------------------------------------
至此全部安裝完成,確認結果的輸出如下:

[grid@oracle-rac1 ~]$ crs_stat -t
Name Type Target State Host 
------------------------------------------------------------
ora.DATA.dg ora....up.type ONLINE ONLINE oracle-rac1 
ora....ER.lsnr ora....er.type ONLINE ONLINE oracle-rac1 
ora....N1.lsnr ora....er.type ONLINE ONLINE oracle-rac1 
ora.OCR.dg ora....up.type ONLINE ONLINE oracle-rac1 
ora.asm ora.asm.type ONLINE ONLINE oracle-rac1 
ora.cvu ora.cvu.type ONLINE ONLINE oracle-rac1 
ora.gsd ora.gsd.type OFFLINE OFFLINE 
ora....network ora....rk.type ONLINE ONLINE oracle-rac1 
ora.oc4j ora.oc4j.type ONLINE ONLINE oracle-rac1 
ora.ons ora.ons.type ONLINE ONLINE oracle-rac1 
ora....SM1.asm application ONLINE ONLINE oracle-rac1 
ora....C1.lsnr application ONLINE ONLINE oracle-rac1 
ora....ac1.gsd application OFFLINE OFFLINE 
ora....ac1.ons application ONLINE ONLINE oracle-rac1 
ora....ac1.vip ora....t1.type ONLINE ONLINE oracle-rac1 
ora....SM2.asm application ONLINE ONLINE oracle-rac2 
ora....C2.lsnr application ONLINE ONLINE oracle-rac2 
ora....ac2.gsd application OFFLINE OFFLINE 
ora....ac2.ons application ONLINE ONLINE oracle-rac2 
ora....ac2.vip ora....t1.type ONLINE ONLINE oracle-rac2 
ora.orcl.db ora....se.type ONLINE ONLINE oracle-rac1 
ora.scan1.vip ora....ip.type ONLINE ONLINE oracle-rac1

[grid@oracle-rac1 ~]$ crsctl status res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS 
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
ONLINE ONLINE oracle-rac1 
ONLINE ONLINE oracle-rac2 
ora.LISTENER.lsnr
ONLINE ONLINE oracle-rac1 
ONLINE ONLINE oracle-rac2 
ora.OCR.dg
ONLINE ONLINE oracle-rac1 
ONLINE ONLINE oracle-rac2 
ora.asm
ONLINE ONLINE oracle-rac1 Started 
ONLINE ONLINE oracle-rac2 Started 
ora.gsd
OFFLINE OFFLINE oracle-rac1 
OFFLINE OFFLINE oracle-rac2 
ora.net1.network
ONLINE ONLINE oracle-rac1 
ONLINE ONLINE oracle-rac2 
ora.ons
ONLINE ONLINE oracle-rac1 
ONLINE ONLINE oracle-rac2 
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE oracle-rac1 
ora.cvu
1 ONLINE ONLINE oracle-rac1 
ora.oc4j
1 ONLINE ONLINE oracle-rac1 
ora.oracle-rac1.vip
1 ONLINE ONLINE oracle-rac1 
ora.oracle-rac2.vip
1 ONLINE ONLINE oracle-rac2 
ora.orcl.db
1 ONLINE ONLINE oracle-rac1 Open 
2 ONLINE ONLINE oracle-rac2 Open 
ora.scan1.vip
1 ONLINE ONLINE oracle-rac1


來自 “ ITPUB部落格 ” ,連結:http://blog.itpub.net/25380026/viewspace-2220036/,如需轉載,請註明出處,否則將追究法律責任。

相關文章