PURPOSE
------------

The purpose of this document is to help a Real Application Clusters database administrator configure raw devices for their RAC environment on Linux.

SCOPE & APPLICATION
-----------------------------

For DBA's and Support Analysts setting up RAC on Linux with raw devices.

Configuring Raw Devices for Real Application Clusters on Linux
-----------------------------------------------------------------

If you choose to use raw devices for database storage, Real Application Clusters requires that all each instance be able to access a set of unformatted devices on a shared disk subsystem. These shared disks are also referred to as raw devices. 

The Oracle instances in Real Application Clusters write data onto the raw devices to update the control file, server parameter file, each datafile, and each redo log file. All instances in the cluster share these files.

The Oracle instances in the RAC configuration write information to raw devices defined for:

• The control file
• The spfile.ora
• Each datafile
• Each ONLINE redo log file
• Server Manager (SRVM) configuration information
• Quorom File for the ORACM (Oracle Cluster Manager)

It is therefore necessary to define raw devices for each of these categories of files. The Oracle Database Configuration Assistant (DBCA) will create a seed database expecting the following configuration (replace db_name with the actual name of your database):

Note: The logical volumes should be bigger than the size of the oracle datafiles, in the following table, Datafile Size indicates the size used for the Oracle datafiles. In this article, we created the logical volumes or disk partitions 1Mb bigger (indicated by the sample filenames).The sizes indicated here are according to what dbca uses as defaults in V9.2+ except for the redo logfiles, the default sizes are mostly the same for V9.0.1. You are strongly encouraged to adjust the sizes to your needs. After the database creation, you can allow them to auto extent to the size you need.

The Oracle Database Configuration Assistant (DBCA) will create a seed database expecting the following configuration:-

Note: Automatic Undo Management requires an undo tablespace per instance therefore you would require a minimum of 2 tablespaces as described above. By following the naming convention described in the table above, raw partitions are identified with the database and the raw volume type (the data contained in the raw volume). Raw volume size is also identified using this method.

Note: In the sample names listed in the table, the string db_name should be replaced with the actual database name, thread is the thread number of the instance, and lognumber is the log number within a thread.

Note: The ORACM raw device (quorum disk) is used by the cluster software for checking the status of the different nodes in the cluster.  Please do not confuse this with the srvctl configuration file.

After creating the necessary partitions or logical volumes, , you have to bind the raw devices.  Please check where your raw devices are located, look for /dev/raw1 or /dev/raw/raw1.These bindings have to be recreated at system reboot, so put the commands in your startup scripts (startoracle_root.sh) .  This must be executed as root on all nodes.

Note: Make sure you have sufficient raw devices. If not, create additional devices as root using
(xx represents a number):

$ mknod rawxx c  162 xx

The bind commands look like:

/usr/sbin/raw /dev/raw1 /dev/oracle/db_name_raw_system_411m
/usr/sbin/raw /dev/raw2 /dev/oracle/db_name_raw_users_120m
/usr/sbin/raw /dev/raw3 /dev/oracle/db_name_raw_temp_41m
/usr/sbin/raw /dev/raw4 /dev/oracle/db_name_raw_undotbs1_201m
/usr/sbin/raw /dev/raw5 /dev/oracle/db_name_raw_undotbs2_201m
/usr/sbin/raw /dev/raw6 /dev/oracle/db_name_raw_indx_26m
/usr/sbin/raw /dev/raw7 /dev/oracle/db_name_raw_controlfile1_110m
/usr/sbin/raw /dev/raw8 /dev/oracle/db_name_raw_controlfile2_110m
/usr/sbin/raw /dev/raw9 /dev/oracle/db_name_raw_rdo_1_1_121m
/usr/sbin/raw /dev/raw10 /dev/oracle/db_name_raw_rdo_1_2_121m
/usr/sbin/raw /dev/raw11 /dev/oracle/db_name_raw_rdo_2_1_121m
/usr/sbin/raw /dev/raw12 /dev/oracle/db_name_raw_rdo_2_2_121m
/usr/sbin/raw /dev/raw13 /dev/oracle/db_name_raw_spfile_5m
/usr/sbin/raw /dev/raw14 /dev/oracle/quorum_raw_20m
/usr/sbin/raw /dev/raw15 /dev/oracle/srvmconfig_raw_100m
/usr/sbin/raw /dev/raw16 /dev/oracle/db_name_raw_tools_11m
/usr/sbin/raw /dev/raw17 /dev/oracle/db_name_raw_cwmlite_21m
/usr/sbin/raw /dev/raw18 /dev/oracle/db_name_raw_example_141m
/usr/sbin/raw /dev/raw19 /dev/oracle/db_name_raw_oemrepo_21m
/usr/sbin/raw /dev/raw20 /dev/oracle/db_name_raw_drsys_21m
/usr/sbin/raw /dev/raw21 /dev/oracle/db_name_raw_odm_21m
/usr/sbin/raw /dev/raw22 /dev/oracle/db_name_raw_tools_41m
/usr/sbin/raw /dev/raw23 /dev/oracle/db_name_raw_controlfile3_110m

/bin/chmod 600 /dev/raw1
/bin/chmod 600 /dev/raw2
/bin/chmod 600 /dev/raw3
/bin/chmod 600 /dev/raw4
/bin/chmod 600 /dev/raw5
/bin/chmod 600 /dev/raw6
/bin/chmod 600 /dev/raw7
/bin/chmod 600 /dev/raw8
/bin/chmod 600 /dev/raw9
/bin/chmod 600 /dev/raw10
/bin/chmod 600 /dev/raw11
/bin/chmod 600 /dev/raw12
/bin/chmod 600 /dev/raw13
/bin/chmod 600 /dev/raw14
/bin/chmod 600 /dev/raw15
/bin/chmod 600 /dev/raw16
/bin/chmod 600 /dev/raw17
/bin/chmod 600 /dev/raw18
/bin/chmod 600 /dev/raw19
/bin/chmod 600 /dev/raw20
/bin/chmod 600 /dev/raw21
/bin/chmod 600 /dev/raw22
/bin/chmod 600 /dev/raw23

/bin/chown oracle.dba /dev/raw1
/bin/chown oracle.dba /dev/raw2
/bin/chown oracle.dba /dev/raw3
/bin/chown oracle.dba /dev/raw4
/bin/chown oracle.dba /dev/raw5
/bin/chown oracle.dba /dev/raw6
/bin/chown oracle.dba /dev/raw7
/bin/chown oracle.dba /dev/raw8
/bin/chown oracle.dba /dev/raw9
/bin/chown oracle.dba /dev/raw10
/bin/chown oracle.dba /dev/raw11
/bin/chown oracle.dba /dev/raw12
/bin/chown oracle.dba /dev/raw13
/bin/chown oracle.dba /dev/raw14
/bin/chown oracle.dba /dev/raw15
/bin/chown oracle.dba /dev/raw16
/bin/chown oracle.dba /dev/raw17
/bin/chown oracle.dba /dev/raw18
/bin/chown oracle.dba /dev/raw19
/bin/chown oracle.dba /dev/raw20
/bin/chown oracle.dba /dev/raw21
/bin/chown oracle.dba /dev/raw22
/bin/chown oracle.dba /dev/raw23
 

$ raw -qa
/dev/raw1:  bound to major 3, minor 7
/dev/raw2:  bound to major 3, minor 8
...

Don't forget to substitute the string db_name with your actual database name. Do this as oracle user on all nodes. This does however increase the management work when you decide to modify the configuration.

$ mkdir /oracle/oradata
$ mkdir /oracle/oradata/db_name
ln -s /dev/raw1 /oracle/oradata/db_name/db_name_raw_system_411m
ln -s /dev/raw2 /oracle/oradata/db_name/db_name_raw_users_120m
ln -s /dev/raw3 /oracle/oradata/db_name/db_name_raw_temp_41m
ln -s /dev/raw4 /oracle/oradata/db_name/db_name_raw_undotbs1_201m
ln -s /dev/raw5 /oracle/oradata/db_name/db_name_raw_undotbs2_201m
ln -s /dev/raw6 /oracle/oradata/db_name/db_name_raw_indx_26m
ln -s /dev/raw7 /oracle/oradata/db_name/db_name_raw_controlfile1_110m
ln -s /dev/raw8 /oracle/oradata/db_name/db_name_raw_controlfile2_110m
ln -s /dev/raw9 /oracle/oradata/db_name/db_name_raw_rdo_1_1_121m
ln -s /dev/raw10 /oracle/oradata/db_name/db_name_raw_rdo_1_2_121m
ln -s /dev/raw11 /oracle/oradata/db_name/db_name_raw_rdo_2_1_121m
ln -s /dev/raw12 /oracle/oradata/db_name/db_name_raw_rdo_2_2_121m
ln -s /dev/raw13 /oracle/oradata/db_name/db_name_raw_spfile_5m
ln -s /dev/raw/raw14 /oracle/oradata/db_name/db_name_raw_quorum_20m
ln -s /dev/raw/raw15/oracle/oradata/db_name/db_name_raw_srvmconfig_100m
ln -s /dev/raw16 /oracle/oradata/db_name/db_name_raw_tools_11m
ln -s /dev/raw17 /oracle/oradata/db_name/db_name_raw_cwmlite_21m
ln -s /dev/raw18 /oracle/oradata/db_name/db_name_raw_example_141m
ln -s /dev/raw19 /oracle/oradata/db_name/db_name_raw_oemrepo_21m
ln -s /dev/raw20 /oracle/oradata/db_name/db_name_raw_drsys_21m
ln -s /dev/raw21 /oracle/oradata/db_name/db_name_raw_odm_21m
ln -s /dev/raw22 /oracle/oradata/db_name/db_name_raw_xdb_41m
ln -s /dev/raw23 /oracle/oradata/db_name/db_name_raw_controlfile3_110m