In this Document

Applies to:

Symptoms

A new LUN was created and made visible to two nodes. Then as root user a partition was created with fdisk, then made visible to the kernel by partprobe/partx on both nodes. Then multipath was configured the same on both nodes. Once that was done, the following commands were run.

On Node 1:
$ sudo /etc/init.d/oracleasm createdisk ASMFS02 /dev/mapper/
$ /etc/init.d/oracleasm listdisks

Then on Node 2:

$ sudo /etc/init.d/oracleasm scandisks
$ /etc/init.d/oracleasm listdisks

The listdisks on both nodes showed that the ASM disk ASMFS02 was created and visible.

Also, on both nodes, the command "/etc/init.d/oracleasm querydisk ASMFS02" gave the message:

'Disk "ASMFS02" is a valid ASM disk'

In spite of all that, in SQLPLU, querying the view GV$ASM_DISK shows that HEADER_STATUS=PROVISIONED on INST_ID=1, but HEADER_STATUS=UNKNOWN on INST_ID=2. And, when the following command was executed from instance +ASM1:

ALTER DISKGROUP ASMFS ADD DISK 'ORCL:ASMFS02' SIZE 81917 M REBALANCE POWER 8;

It failed with the following error messages:

ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15075: disk(s) are not visible cluster-wide

Cause

There is issue in the multipath configuration.
As you can see that on node 1 for asmfs02p1we are seeing major number as 253 and node 2 its 66.
This shows that on node 1 we are using multipath devices but on node 2 its not used.

[oracle@xyz-db1 ~]$ ls -l /dev/oracleasm/disks/
total 0
brw-rw---- 1 oracle dba 253, 14 Aug 19  2011 ASMDATA01
brw-rw---- 1 oracle dba 253, 13 Aug 19  2011 ASMDATA02
brw-rw---- 1 oracle dba 253, 20 Dec 13 14:32 ASMDATA03
brw-rw---- 1 oracle dba 253, 18 Dec  3  2011 ASMDATA04
brw-rw---- 1 oracle dba 253, 19 Dec  3  2011 ASMDATA05
brw-rw---- 1 oracle dba 253, 24 May 14 11:23 ASMDATA06
brw-rw---- 1 oracle dba 253, 23 May 14 11:23 ASMDATA07
brw-rw---- 1 oracle dba 253, 11 Aug 19  2011 ASMFLASH01
brw-rw---- 1 oracle dba 253, 12 Aug 19  2011 ASMFS01
brw-rw---- 1 oracle dba 253, 26 Jun  5 16:52 ASMFS02<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<253

Node2

[oracle@xyz-db2 ~]$ ls -l /dev/oracleasm/disks/
total 0
brw-rw---- 1 oracle dba 253, 14 Aug 19  2011 ASMDATA01
brw-rw---- 1 oracle dba 253, 13 Aug 19  2011 ASMDATA02
brw-rw---- 1 oracle dba 253, 20 Dec 13 14:33 ASMDATA03
brw-rw---- 1 oracle dba 253, 18 Dec  3  2011 ASMDATA04
brw-rw---- 1 oracle dba 253, 19 Dec  3  2011 ASMDATA05
brw-rw---- 1 oracle dba 253, 23 May 14 11:24 ASMDATA06
brw-rw---- 1 oracle dba 253, 24 May 14 11:24 ASMDATA07
brw-rw---- 1 oracle dba 253, 11 Aug 19  2011 ASMFLASH01
brw-rw---- 1 oracle dba 253, 12 Aug 19  2011 ASMFS01
brw-rw---- 1 oracle dba  66, 65 Jun  5 16:53 ASMFS02<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<66

Solution

Sysadmin re-created the ASM disk, it has correct major numbers, and the disks were added to the disk group and rebalanced successfully.