In this Document

Applies to:

Symptoms

While mounting the diskgroup ,it was failing with below errors, 

Fri Aug 01 13:56:48 2014
SQL> ALTER DISKGROUP ALL MOUNT
NOTE: Diskgroups listed in ASM_DISKGROUPS are
  data
NOTE: cache registered group DATA number=1 incarn=0x228ac17d
NOTE: cache began mount (first) of group DATA number=1 incarn=0x228ac17d
ERROR: no read quorum in group: required 2, found 0 disks
NOTE: cache dismounting (clean) group 1/0x228AC17D (DATA)
NOTE: messaging CKPT to quiesce pins Unix process pid: 9148, image: oracle@frscforpo10 (TNS V1-V3)
NOTE: dbwr not being msg'd to dismount
NOTE: lgwr not being msg'd to dismount
NOTE: cache dismounted group 1/0x228AC17D (DATA)
NOTE: cache ending mount (fail) of group DATA number=1 incarn=0x228ac17d
NOTE: cache deleting context for group DATA 1/0x228ac17d
Fri Aug 01 13:56:48 2014
NOTE: No asm libraries found in the system
ASM Health Checker found 1 new failures
GMON dismounting group 1 at 2 for pid 17, osid 9148
ERROR: diskgroup DATA was not mounted
ORA-15032: not all alterations performed
ORA-15017: diskgroup "DATA" cannot be mounted
ORA-15063: ASM discovered an insufficient number of disks for diskgroup "DATA"
ERROR: ALTER DISKGROUP ALL MOUNT

Changes

 System administrattor updated udev rules

Cause

Strace on kfod was failing for asmdata*p1 devices due to permission issue.

19792 access("/dev/mapper/asmdata4p1", R_OK|W_OK) = -1 EACCES (Permission denied)
19792 access("/dev/mapper/asm_prueba_accesop1", R_OK|W_OK) = -1 EACCES (Permission denied)
19792 access("/dev/mapper/asmdata5p1", R_OK|W_OK) = -1 EACCES (Permission denied)
19792 access("/dev/mapper/asmdata2p1", R_OK|W_OK) = -1 EACCES (Permission denied)
19792 access("/dev/mapper/asmdata1p1", R_OK|W_OK) = -1 EACCES (Permission denied)
19792 access("/dev/mapper/asmdata3p1", R_OK|W_OK) = -1 EACCES (Permission denied)

Wrongly device was mapped at udev rules level
 
oracle@frscforpo10/tmp:+ASM>ls -ltr /dev/mapper/asm*
lrwxrwxrwx 1 root root 8 Aug 3 00:17 /dev/mapper/asmdata1p1 -> ../dm-28 < lrwxrwxrwx 1 root root 8 Aug 3 00:18 /dev/mapper/asmdata2p1 -> ../dm-29 < lrwxrwxrwx 1 root root 8 Aug 3 00:18 /dev/mapper/asmdata3p1 -> ../dm-27 < lrwxrwxrwx 1 root root 8 Aug 3 00:18 /dev/mapper/asmdata4p1 -> ../dm-32 < lrwxrwxrwx 1 root root 8 Aug 3 00:18 /dev/mapper/asmdata5p1 -> ../dm-30 < oracle@frscforpo10/tmp:+ASM>
oracle@frscforpo10/tmp:+ASM>ls -ltr /dev/dm*
brw-rw---- 1 root disk 253, 28 Aug 3 00:17 /dev/dm-28
brw-rw---- 1 root disk 253, 29 Aug 3 00:18 /dev/dm-29
brw-rw---- 1 root disk 253, 27 Aug 3 00:18 /dev/dm-27
brw-rw---- 1 root disk 253, 32 Aug 3 00:18 /dev/dm-32
brw-rw---- 1 root disk 253, 30 Aug 3 00:18 /dev/dm-30
 

Solution

Action plan ::----------
=====================

+ Change ownership of related dm* devices at OS level.

/dev/mapper/asmdata5p1 -> ../dm-30
/dev/mapper/asmdata4p1 -> ../dm-32
/dev/mapper/asmdata3p1 -> ../dm-27
/dev/mapper/asmdata2p1 -> ../dm-29
/dev/mapper/asmdata1p1 -> ../dm-28

brw-rw---- 1 root disk 253, 30 Aug 3 00:18 /dev/dm-30
brw-rw---- 1 root disk 253, 32 Aug 3 00:18 /dev/dm-32
brw-rw---- 1 root disk 253, 29 Aug 3 00:18 /dev/dm-29
brw-rw---- 1 root disk 253, 27 Aug 3 00:18 /dev/dm-27
brw-rw---- 1 root disk 253, 28 Aug 3 00:17 /dev/dm-28

As root OS user ,

chown oracle:oinstall /dev/dm-30
chown oracle:oinstall /dev/dm-32
chown oracle:oinstall /dev/dm-29
chown oracle:oinstall /dev/dm-27
chown oracle:oinstall /dev/dm-28

+ Then make sure udev rules defined proper way at OS level ,so that these ownership does not change at OS level for asmdata*p1 .

In /etc/udev/rules.d/99-oracle-asmdevices.rules

ENV{DM_UUID}=="mpath-360050768028082a7100000000000000a"
to
ENV{DM_UUID}=="part1-mpath-360050768028082a7100000000000000a"

reload rules:
/sbin/udevadm control --reload-rules
/sbin/udevadm trigger --type=devices --action=change

+ Then validate and mount this diskgroup.

This is not a ASM issue ,rather OS level configuration issue .