OCR / Vote disk Maintenance Operations: (ADD/REMOVE/REPLACE/MOVE) [ID 428681.1]

In this Document

Applies to:

Goal

The goal of this note is to provide steps to add, remove, replace or move an Oracle Cluster Repository (OCR) or voting disk in Oracle Clusterware 10gR2, 11gR1 and 11gR2 environment. It will also provide steps to move OCR / voting and ASM devices from raw device to block device.

This article is intended for DBA and Support Engineers who need to modify, or move OCR and voting disks files, customers who have an existing clustered environment deployed on a storage array and might want to migrate to a new storage array with minimal downtime.

Typically, one would simply cp or dd the files once the new storage has been presented to the hosts. In this case, it is a little more difficult because:

1. The Oracle Clusterware has the OCR and voting disks open and is actively using them. (Both primary and mirrors)
2. There is an API provided for this function (ocrconfig and crsctl), which is the appropriate interface than typical cp and/or dd commands.

It is highly recommended to take a backup of the voting disk, and OCR device before making any changes.

Oracle Cluster Registry (OCR) and Voting Disk Additional clarifications

For Voting disks (never use even number of voting disks):
External redundancy means 1 voting disk
Normal redundancy means 3 voting disks
High redundancy means 5 voting disks

For OCR: 10.2 and 11.1, maximum 2 OCR devices: OCR and OCRMIRROR
11.2+, upto 5 OCR devices can be added.

Fix

Prepare the disks

For OCR or voting disk addition or replacement, new disks need to be prepared. Please refer to Clusteware/Gird Infrastructure installation guide for different platform. for the disk requirement and preparation.

1. Size

For 10.1:
OCR device minimum size (each): 100M
Voting disk minimum size (each): 20M

For 10.2:
OCR device minimum size (each): 256M
Voting disk minimum size (each): 256M

For 11.1:
OCR device minimum size (each): 280M
Voting disk minimum size (each): 280M

For 11.2:
OCR device minimum size (each): 300M
Voting disk minimum size (each): 300M

2. For raw or block device (pre 11.2)

Please refer to Clusterware installation guide on different platform. for more details.
On windows platform. the new raw device link is created via $CRS_HOME\bin\GUIOracleOBJManager.exe, for example:
\\.\VOTEDSK2
\\.\OCR2

3. For ASM disks (11.2+)

On Windows platform, please refer to Document 331796.1 How to setup ASM on Windows
On Linux platform, please refer to Document 580153.1 How To Setup ASM on Linux Using ASMLIB Disks, Raw Devices or Block Devices?
For other platform, please refer to Clusterware/Gird Infrastructure installation guide.

4. For cluster file system

If OCR is on cluster file system, the new OCR or OCRMIRROR file must be touched before add/replace command can be issued. Otherwise PROT-21: Invalid parameter (10.2/11.) or PROT-30 The Oracle Cluster Registry location to be added is not accessible (for 11.2) will occur.

It is not required to pre-touch voting disk file on cluster file system.

After delete command is issued, the ocr/voting files on the cluster file system require to be removed manually.

5. Permissions

For OCR device:
chown root:oinstall
chmod 640

For Voting device:
chown :oinstall
chmod 644

For ASM disks used for OCR/Voting disk:
chown griduser:asmadmin
chmod 660

ADD/REMOVE/REPLACE/MOVE OCR Device

Make sure there is a recent copy of the OCR file before making any changes:

ocrconfig -showbackup

If there is not a recent backup copy of the OCR file, an export can be taken for the current OCR file. Use the following command to generate an export of the online OCR file:

In 10.2

# ocrconfig -export -s online

In 11.1 and 11.2

# ocrconfig -manualbackup
node1 2008/08/06 06:11:58 /crs/cdata/crs/backup_20080807_003158.ocr

To recover using this file, the following command can be used:

From 11.2+, please also refer How to restore ASM based OCR after complete loss of the CRS diskgroup on Linux/Unix systems Document 1062983.1

To see whether OCR is healthy, run an ocrcheck, which should return with like below.

# ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) : 497928
Used space (kbytes) : 312
Available space (kbytes) : 497616
ID : 576761409
Device/File Name : /dev/raw/raw1
Device/File integrity check succeeded
Device/File Name : /dev/raw/raw2
Device/File integrity check succeeded

Cluster registry integrity check succeeded

For 11.1+, ocrcheck as root user should also show:
Logical corruption check succeeded

1. To add an OCRMIRROR device when only OCR device is defined:

To add an OCR mirror device, provide the full path including file name.
10.2 and 11.1:

# ocrconfig -replace ocrmirror
eg:
# ocrconfig -replace ocrmirror /dev/raw/raw2
# ocrconfig -replace ocrmirror /dev/sdc1
# ocrconfig -replace ocrmirror /cluster_fs/ocrdisk.dat
> ocrconfig -replace ocrmirror \\.\OCRMIRROR2  - for Windows

11.2+: From 11.2 onwards, upto 4 ocrmirrors can be added

# ocrconfig -add
eg:
# ocrconfig -add +OCRVOTE2
# ocrconfig -add /cluster_fs/ocrdisk.dat

2. To remove an OCR device

To remove an OCR device:
10.2 and 11.1:

# ocrconfig -replace ocr

11.2+:

# ocrconfig -delete
eg:
# ocrconfig -delete +OCRVOTE1

* Once an OCR device is removed, ocrmirror device automatically changes to be OCR device.
* It is not allowed to remove OCR device if only 1 OCR device is defined, the command will return PROT-16.

To remove an OCR mirror device:
10.2 and 11.1:

# ocrconfig -replace ocrmirror

11.2+:

# ocrconfig -delete
eg:
# ocrconfig -delete +OCRVOTE2

After removal, the old OCR/OCRMIRROR can be deleted if they are on cluster filesystem.

3. To replace or move the location of an OCR device

10.2 and 11.1:
To replace the OCR device with , provide the full path including file name.

# ocrconfig -replace ocr
eg:
# ocrconfig -replace ocr /dev/sdd1
$ ocrconfig -replace ocr \\.\OCR2 - for Windows

To replace the OCR mirror device with , provide the full path including file name.

# ocrconfig -replace ocrmirror
eg:
# ocrconfig -replace ocrmirrow /dev/raw/raw4
# ocrconfig -replace ocrmirrow \\.\OCRMIRROR2  - for Windows

11.2:
The command is same for replace either OCR or OCRMIRRORs (at least 2 OCR exist for replace command to work):

ADD/DELETE/MOVE Voting Disk

To take a backup of voting disk:

$ dd if=voting_disk_name f=backup_file_name

For Windows:

ocopy \\.\votedsk1 o:\backup\votedsk1.bak

For 10gR2 release

Shutdown the Oracle Clusterware (crsctl stop crs as root) on all nodes before making any modification to the voting disk. Determine the current voting disk location using:
crsctl query css votedisk

1. To add a Voting Disk, provide the full path including file name:

# crsctl add css votedisk -force
eg:
# crsctl add css votedisk /dev/raw/raw1 -force
# crsctl add css votedisk /cluster_fs/votedisk.dat -force
> crsctl add css votedisk \\.\VOTEDSK2 -force   - for windows

2. To delete a Voting Disk, provide the full path including file name:

# crsctl delete css votedisk -force
eg:
# crsctl delete css votedisk /dev/raw/raw1 -force
# crsctl delete css votedisk /cluster_fs/votedisk.dat -force
> crsctl delete css votedisk \\.\VOTEDSK1 -force   - for windows

3. To move a Voting Disk, provide the full path including file name, add a device first before deleting the old one:

# crsctl add css votedisk -force
# crsctl delete css votedisk -force
eg:
# crsctl add css votedisk /dev/raw/raw4 -force
# crsctl delete css votedisk /dev/raw/raw1 -force

After modifying the voting disk, start the Oracle Clusterware stack on all nodes

# crsctl start crs

Verify the voting disk location using

# crsctl query css votedisk

For 11gR1 release

Starting with 11.1.0.6, the below commands can be performed online (CRS is up and running).

1. To add a Voting Disk, provide the full path including file name:

2. To delete a Voting Disk, provide the full path including file name:

3. To move a Voting Disk, provide the full path including file name:

Verify the voting disk location:

For 11gR2 release

From 11.2, votedisk can be stored on either ASM diskgroup or cluster file systems. The following commands can only be executed when Grid Infrastructure is running. As grid user:

1. To add a Voting Disk
a. When votedisk is on cluster file system:

b. When votedisk is on ASM diskgroup, no add option available.
The number of votedisk is determined by the diskgroup redundancy. If more copy of votedisk is desired, one can move votedisk to a diskgroup with higher redundancy. See step 4.
If a votedisk is removed from a normal or high redundancy diskgroup for abnormal reason, it can be added back using:

2. To delete a Voting Disk
a. When votedisk is on cluster file system:

b. When votedisk is on ASM, no delete option available, one can only replace the existing votedisk group with another ASM diskgroup

3. To move a Voting Disk on cluster file system

4. To move voting disk on ASM from one diskgroup to another diskgroup due to redundancy change or disk location change

Example here is moving from external redundancy +OCRVOTE diskgroup to normal redundancy +CRS diskgroup

5. To move voting disk between ASM diskgroup and cluster file system
a. Move from ASM diskgroup to cluster file system:

b. Move from cluster file system to ASM diskgroup

6. To verify:

References