Oracle RAC安裝前的磁碟分割槽及裸裝置配置

在Oracle RAC安裝中，前期配置是安裝是否成功的關鍵。看過了不少文件，但是幾乎都忽略了前期分割槽和裸裝置的配置過程，因此簡單的做了總結性記錄。

安裝環境為Oracle 10g for Linux x86

對獨立儲存（即，磁碟陣列），當儲存工程師建立完raid group ,並在此group上分幾個Lun，最終透過多路徑封包後，輪到DBA上場動手了。Lun實際上從Linux OS的fdisk -l命令檢視時，相當於一個磁碟。

1、對磁碟做分割槽

在這裡使用fdisk命令，但是在對磁碟進行分割槽的時候需要注意，根據ORACLE的官方文件要求，需要從磁碟頭開始1MB的偏移量使用磁碟，ORACLE官方文件裡的說明如下：

[@more@]

Once the LUNs have been presented from the SAN to ALL servers in the cluster, partition the LUNs from one node only, run fdisk to create a single whole-disk partition with exactly 1 MB offset on each LUN to be used as ASM Disk.

Tip: From the fdisk prompt, type "u" to switch the display unit from cylinder to sector. Then create a single

primary partition starting on sector 2048 (1MB offset assuming sectors of 512 bytes per unit). See below

example for /dev/sda:

fdisk /dev/sda

Command (m for help): u

Changing display/entry units to sectors

Command (m for help): n

Command action

e extended

p primary partition (1-4)

p

Partition number (1-4): 1

First sector (61-1048575, default 61): 2048

Last sector or +size or +sizeM or +sizeK (2048-1048575, default 1048575):

Using default value 1048575

Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read partition table.

Syncing disks.

假設透過以上步驟對/dev/sdb進行分割槽後最終結果如下：

[root@ dbserver 1 ~]# fdisk -l /dev/sdb

Disk /dev/sdb: 2035.7 GB, 146727239680 bytes

255 heads, 63 sectors/track, 17838 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

/dev/sdb1 1 25 200781 83 Linux

/dev/sdb2 26 50 200812+ 83 Linux

/dev/sdb3 51 17838 142882110 5 Extended

/dev/sdb5 51 63 104391 83 Linux

/dev/sdb6 64 76 104391 83 Linux

/dev/sdb7 77 89 104391 83 Linux

/dev/sdb8 90 5561 43953808+ 83 Linux

/dev/sdb9 5562 11033 43953808+ 83 Linux

/dev/sdb10 11034 17838 54661131 83 Linux

[root@ dbserver 1 ~]#

分割槽命令只在一個節點上執行，另外一個節點上執行分割槽同步命令

#partprobe

2、將分割槽繫結到RAW裝置

在Oracle 10g由於我們把oracle的OCR(oracle叢集登錄檔)、 表決磁碟（voting disk）安裝在裸裝置上，對於裸裝置要使用它，需要編/etc/sysconfig/rawdevices檔案，將裸裝置新增到系統中，然後再為裸裝置設定屬組資訊以及許可權資訊。

在Oracle 11g 將OCR放到ASM上，也不必為表決磁碟分割槽空間。

把下面的內容新增到/etc/sysconfig/rawdevices檔案中

#Vi /etc/sysconfig/rawdevices

#OCR

/dev/raw/raw1 /dev/sdb1

/dev/raw/raw2 /dev/sdb2

#Voting Disk

/dev/raw/raw5 /dev/sdb5

/dev/raw/raw6 /dev/sdb6

/dev/raw/raw7 /dev/sdb7

注意：在2.6核心的Red Hat Linux上，當系統重啟後，裸裝置的屬組資訊將被還原為系統預設值（root：disk）。為了使得裸裝置屬組資訊可以被保留，你要在/etc/rc.d/rc.local中強制更新裸裝置的屬組資訊，即在檔案中加入下述內容：

#vi /etc/rc.d/rc.local

chown root:dba /dev/raw/raw1

chown root:dba /dev/raw/raw2

chmod 660 /dev/raw/raw1

chmod 660 /dev/raw/raw2

chown oracle:dba /dev/raw/raw5

chown oracle:dba /dev/raw/raw6

chown oracle:dba /dev/raw/raw7

chmod 644 /dev/raw/raw5

chmod 644 /dev/raw/raw6

chmod 644 /dev/raw/raw7

以上配置完畢後，部分資料要求務必重新啟動兩臺伺服器，在官方文件上對此也有如下的操作說明，是否不必重啟,這個具體沒有嘗試過。

# /sbin/service rawdevices restart

3、 ASMlib安裝與配置

每個節點上進行ASMlib的配置使用 ASMLib 之前，必須執行配置指令碼來準備驅動程式。 在每個叢集主機上執行以下命令並回應提示，如下例所示。

配置ASMlib

# /etc/init.d/oracleasm configure

這將配置 Oracle ASM 庫驅動程式的啟動時屬性。以下問題將確定在啟動時是否載入驅動程式以及它將擁有的許可權。當前值將顯示在方括號（“[]”）中。按 而不鍵入回應將保留該當前值。按 Ctrl-C 將終止。

Default user to own the driver interface []:oracle

Default group to own the driver interface []:dba

Start Oracle ASM library driver on boot (y/n) [n]:y

Fix permissions of Oracle ASM disks on boot (y/n) [y]:y

Writing Oracle ASM library driver configuration [ OK ]

Creating /dev/oracleasm mount point [ OK ]

Loading module "oracleasm" [ OK ]

Mounting ASMlib driver filesystem [ OK ]

Scanning system for ASM disks [ OK ]

啟用 ASMLib 驅動程式。

# /etc/init.d/oracleasm enable

Writing Oracle ASM library driver configuration [ OK ]

Scanning system for ASM disks [ OK ]

以上需要在所有節點上進行配置

下面我們來建立ASM磁碟（以下部分在一個節點上建立在另一個節點上掃描發現就可以了。

/etc/init.d/oracleasm createdisk DISK_NAME device_name

提示： 以大寫字母輸入 DISK_NAME。當前版本中有一個錯誤，即如果使用小寫字母，ASM 例項將無法識別磁碟。

# /etc/init.d/oracleasm createdisk VOL1 /dev/sd8

Marking disk "/dev/sdb8" as an ASM disk [ OK ]

# /etc/init.d/oracleasm createdisk VOL2 /dev/sd9

Marking disk "/dev/sdb9" as an ASM disk [ OK ]

.

以下示例演示瞭如何列出標記為由 ASMLib 使用的所有磁碟。

# /etc/init.d/oracleasm listdisks

VOL1

VOL2.

注意在所有叢集其他節點主機上，執行下列命令進行掃描所配置的 ASMLib 磁碟：

#/etc/init.d/oracleasm scandisks

以上操作步驟後的下一步就是Clusterwar的安裝，後續步驟一般的文件寫的也比較全面，也有圖例，在此不作介紹。

最後作為備忘，將官方的相關內容也要貼上過來了，認真閱讀肯定會有收穫。

對此Oracle官方文件的說明如下：

3.4.5 Binding Partitions to Raw Devices for Oracle Clusterware Files

Red Hat

1. To determine what raw devices are already bound to other devices, enter the following command on every node:
   # /usr/bin/raw -qa
   
   Raw devices have device names in the form /dev/raw/rawn, where n is a number that identifies the raw device.
   For each device that you want to use, identify a raw device name that is unused on all nodes.

2. Open the /etc/sysconfig/rawdevices file in any text editor and add a line similar to the following for each partition that you created:
   /dev/raw/raw1 /dev/sdb1
   
   Specify an unused raw device for each partition.

3. For the raw device that you created for the Oracle Cluster Registry (OCR), enter commands similar to the following to set the owner, group, and permissions on the device file:
   # chown root:oinstall /dev/raw/rawn
   # chmod 640 /dev/raw/rawn
   
   By making the oinstall group the owner of the OCR, this permits the OCR to be read by multiple Oracle homes, including those with different OSDBA groups.
4. To bind the partitions to the raw devices, enter the following command:
   # /sbin/service rawdevices restart
   
   The system automatically binds the devices listed in the rawdevices file when it restarts.

5. Repeat step 2 through step on each node in the cluster.

4.12 Raw Devices on Oracle Enterprise Linux and Red Hat Enterprise Linux

Verify that an appropriate raw devices utility (util-linux) rpm is installed for the update of the operating systems. For example, on Oracle Enterprise Linux 4 and Red Hat Enterprise Linux 4 (update 5), util-linux-2.12a-16.EL4.23.x86_64 or later rpm should be installed. On Oracle Enterprise Linux 5 and Red Hat Enterprise Linux 5, util-linux-2.13-0.44.EL5.x86_64 or later rpm should be installed.

When you restart an Oracle Enterprise Linux 4, Oracle Enterprise Linux 5, Red Hat Enterprise Linux 4, or Red Hat Enterprise Linux 5 system, raw devices revert to their original owners and permissions by default. If you are using raw devices with this operating system for your Oracle files, for example, for ASM storage or Oracle Clusterware files, you must override this default behavior. To do this, add an entry to the /etc/rc.d/rc.local file for each raw device containing the chmod and chown commands required to reset them to the required values.

As an example, here are sample entries in a /etc/rc.d/rc.local file that control the restart behavior of raw devices for two ASM disk files (/dev/raw/raw6 and /dev/raw/raw7), two Oracle Cluster Registry files (/dev/raw/raw1 and /dev/raw/raw2), and three Oracle Clusterware voting disks (/dev/raw/raw3, /dev/raw/raw4, and /dev/raw/raw5):

# ASM
chown oracle:dba /dev/raw/raw6
chown oracle:dba /dev/raw/raw7
chmod 660 /dev/raw/raw6
chmod 660 /dev/raw/raw7
# OCR
chown root:oinstall /dev/raw/raw1
chown root:oinstall /dev/raw/raw2
chmod 660 /dev/raw/raw1
chmod 660 /dev/raw/raw2
# Voting Disks
chown oracle:oinstall /dev/raw/raw3
chown oracle:oinstall /dev/raw/raw4
chown oracle:oinstall /dev/raw/raw5
chmod 644 /dev/raw/raw3
chmod 644 /dev/raw/raw4
chmod 644 /dev/raw/raw5

對於硬碟資料儲存結構如下說明有助於以上文件的理解：

Sector：扇區（sector）是碟片的最小的可定址單元，一個扇區可容納512位元組，也叫做磁碟塊（disk block）。扇區會被集合成簇（cluster），簇也叫資料塊，作業系統通常以資料塊為單位對硬碟進行讀寫。

Track：磁軌，多個扇區端對端地連成一個圓圈構成一個磁軌，容量隨半徑的增加而增大。

Cylinder：磁柱，也叫柱面。構成硬碟的每個碟片都被劃分為數目相等的磁軌，並從外緣的“0”開始編號，具有相同編號的磁軌形成一個圓柱，稱之為磁碟的柱面。因此硬碟的柱面數和一個碟片上的磁軌數是相同的。處在外面的磁軌比處在裡面的磁軌擁有更多的扇區，能容納更多的資料。