修復ASM磁碟頭（二）
上一篇文章（修復ASM磁碟頭（一）：http://blog.itpub.net/23135684/viewspace-1260502/）討論了透過ASMCMD中的md_backup，md_restore命令是否能夠修復ASM磁碟頭，答案是否定的。這篇文章討論透過Grid下的kfed工具對磁碟頭進行修復。
一.檢查實驗環境。
# pwd
/u01/app/11.2.0/grid/bin
# ./crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA01.dg
ONLINE ONLINE localhost
ora.FRA01.dg
ONLINE ONLINE localhost
ora.LISTENER.lsnr
ONLINE ONLINE localhost
ora.asm
ONLINE ONLINE localhost Started
ora.ons
OFFLINE OFFLINE localhost
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
1 ONLINE ONLINE localhost
ora.diskmon
1 OFFLINE OFFLINE
ora.evmd
1 ONLINE ONLINE localhost
ora.orcl.db
1 ONLINE ONLINE localhost Open
相關服務都是ONLINE狀態。
# lspv
hdisk0 00cc1ad4ef095bf0 rootvg active
hdisk1 00cc1ad46aff307f None
hdisk2 none None
hdisk3 none None
hdisk4 none None
# lsdev -Cc disk
hdisk0 Available 00-08-00 SAS Disk Drive
hdisk1 Available 00-08-00 SAS Disk Drive
hdisk2 Available 04-00-01 MPIO DS4700 Disk
hdisk3 Available 04-00-01 MPIO DS4700 Disk
hdisk4 Available 04-00-01 MPIO DS4700 Disk
伺服器一共有5張磁碟，其中hdisk2，hdisk3和hdisk4是儲存盤。
# su - grid
$ asmcmd -p
ASMCMD [+] > lsdg
State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 4096 1048576 51200 49637 0 49637 0 N DATA01/
MOUNTED EXTERN N 512 4096 1048576 51200 50980 0 50980 0 N FRA01/
ASMCMD [+] > lsdsk -G data01
Path
/dev/rhdisk3
ASMCMD [+] > lsdsk -G fra01
Path
/dev/rhdisk4
hdisk3屬於DATA01磁碟組，hdisk4屬於FRA01磁碟組。
# su - oracle
$ sql
SQL*Plus: Release 11.2.0.3.0 Production on Thu Sep 18 14:28:59 2014
Copyright (c) 1982, 2011, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, Automatic Storage Management, OLAP, Data Mining
and Real Application Testing options
SQL> set linesize 200
SQL> col name format a50
SQL> select name ,status from v$datafile;
NAME STATUS
-------------------------------------------------- -------
+DATA01/orcl/datafile/system.256.858607953 SYSTEM
+DATA01/orcl/datafile/sysaux.257.858607953 ONLINE
+DATA01/orcl/datafile/undotbs1.258.858607953 ONLINE
+DATA01/orcl/datafile/users.259.858607953 ONLINE
SQL> select name,open_mode from v$database;
NAME OPEN_MODE
-------------------------------------------------- --------------------
ORCL READ WRITE
SQL> select status from v$instance;
STATUS
------------
OPEN
資料庫狀態是OPEN的。
二.為ASM磁碟分配PVID。
1).停止所有CRS服務：
# pwd
/u01/app/11.2.0/grid/bin
# ./crsctl stop has
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'localhost'
CRS-2673: Attempting to stop 'ora.orcl.db' on 'localhost'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'localhost'
CRS-2677: Stop of 'ora.orcl.db' on 'localhost' succeeded
CRS-2673: Attempting to stop 'ora.DATA01.dg' on 'localhost'
CRS-2673: Attempting to stop 'ora.FRA01.dg' on 'localhost'
CRS-2677: Stop of 'ora.DATA01.dg' on 'localhost' succeeded
CRS-2677: Stop of 'ora.FRA01.dg' on 'localhost' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'localhost'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'localhost' succeeded
CRS-2677: Stop of 'ora.asm' on 'localhost' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'localhost'
CRS-2677: Stop of 'ora.cssd' on 'localhost' succeeded
CRS-2673: Attempting to stop 'ora.evmd' on 'localhost'
CRS-2677: Stop of 'ora.evmd' on 'localhost' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'localhost' has completed
CRS-4133: Oracle High Availability Services has been stopped.
# ps -ef | grep d.bin
root 6815998 3211342 0 14:30:33 pts/0 0:00 grep d.bin
# ps -ef | grep ora_
root 3670204 3211342 0 14:30:36 pts/0 0:00 grep ora_
2).為所有ASM磁碟分配PVID：
# lspv
hdisk0 00cc1ad4ef095bf0 rootvg active
hdisk1 00cc1ad46aff307f None
hdisk2 none None
hdisk3 none None
hdisk4 none None
# chdev -l hdisk2 -a pv=yes
hdisk2 changed
# chdev -l hdisk3 -a pv=yes
hdisk3 changed
# chdev -l hdisk4 -a pv=yes
hdisk4 changed
# lspv
hdisk0 00cc1ad4ef095bf0 rootvg active
hdisk1 00cc1ad46aff307f None
hdisk2 00cc1ad48a3f2543 None
hdisk3 00cc1ad48a3f5a17 None
hdisk4 00cc1ad48a43b989 None
3).重新啟動所有CRS服務：
# pwd
/u01/app/11.2.0/grid/bin
# ./crsctl start has
CRS-4123: Oracle High Availability Services has been started.
經過一段時間後，ASM例項能正常的啟動，但是所有的磁碟組都無法ONLINE：
# ./crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA01.dg
ONLINE OFFLINE localhost
ora.FRA01.dg
ONLINE OFFLINE localhost
ora.LISTENER.lsnr
ONLINE ONLINE localhost
ora.asm
ONLINE ONLINE localhost Started
ora.ons
OFFLINE OFFLINE localhost
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
1 ONLINE ONLINE localhost
ora.diskmon
1 OFFLINE OFFLINE
ora.evmd
1 ONLINE ONLINE localhost
ora.orcl.db
1 ONLINE OFFLINE Instance Shutdown
4).檢視ASM例項告警日誌：
# su - grid
$ cd $ORACLE_HOME/log/diag/asm/+asm/+ASM/trace
$ ls *.log
alert_+ASM.log
$ tail -200 alert*.log
......
ERROR: diskgroup DATA01 was not mounted
ORA-15032: not all alterations performed
ORA-15017: diskgroup "DATA01" cannot be mounted
ORA-15063: ASM discovered an insufficient number of disks for diskgroup "DATA01"
ERROR: ALTER DISKGROUP DATA01 MOUNT /* asm agent *//* {0:0:2} */
NOTE: cache registered group FRA01 number=1 incarn=0x408082f3
NOTE: cache began mount (first) of group FRA01 number=1 incarn=0x408082f3
Thu Sep 18 14:37:41 2014
NOTE: No asm libraries found in the system
ASM Health Checker found 1 new failures
ERROR: no read quorum in group: required 2, found 0 disks
NOTE: cache dismounting (clean) group 1/0x408082F3 (FRA01)
NOTE: messaging CKPT to quiesce pins Unix process pid: 13238424, image: oracle@localhost (TNS V1-V3)
NOTE: dbwr not being msg'd to dismount
NOTE: lgwr not being msg'd to dismount
NOTE: cache dismounted group 1/0x408082F3 (FRA01)
NOTE: cache ending mount (fail) of group FRA01 number=1 incarn=0x408082f3
NOTE: cache deleting context for group FRA01 1/0x408082f3
GMON dismounting group 1 at 4 for pid 18, osid 13238424
ERROR: diskgroup FRA01 was not mounted
ORA-15032: not all alterations performed
ORA-15017: diskgroup "FRA01" cannot be mounted
ORA-15063: ASM discovered an insufficient number of disks for diskgroup "FRA01"
ERROR: ALTER DISKGROUP FRA01 MOUNT /* asm agent *//* {0:0:2} */
......
ASM Health Checker found 1 new failures
ASM Health Checker found 1 new failures
hdisk3和hdisk4 ASM磁碟頭已經被破壞。
三.使用kfed工具修復ASM磁碟頭。
ASM disk header是ASM disk的第一個塊，即AU#0 BLOCK#0，disk header中記錄了ASM disk的重要資訊，比如ASM disk名字，diskgroup名字，failure group名字，AU size等等。
1).使用kfed工具檢視破壞後磁碟頭內容：
# su - grid
$ which kfed
/u01/app/11.2.0/grid/bin/kfed
$ kfed read /dev/rhdisk3
kfbh.endian: 201 ; 0x000: 0xc9
kfbh.hard: 194 ; 0x001: 0xc2
kfbh.type: 212 ; 0x002: *** Unknown Enum *** <<<< 表示是否是磁碟頭資訊，這裡的狀態是被破壞的狀態。
kfbh.datfmt: 193 ; 0x003: 0xc1
kfbh.block.blk: 0 ; 0x004: blk=0
kfbh.block.obj: 0 ; 0x008: file=0
kfbh.check: 0 ; 0x00c: 0x00000000
kfbh.fcn.base: 0 ; 0x010: 0x00000000
kfbh.fcn.wrap: 0 ; 0x014: 0x00000000
kfbh.spare1: 0 ; 0x018: 0x00000000
kfbh.spare2: 0 ; 0x01c: 0x00000000
110285600 C9C2D4C1 00000000 00000000 00000000 [................]
110285610 00000000 00000000 00000000 00000000 [................]
Repeat 6 times
110285680 00CC1AD4 8A3F5A17 00000000 00000000 [.....?Z.........]
110285690 00000000 00000000 00000000 00000000 [................]
Repeat 246 times
KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][212]
$ kfed read /dev/rhdisk4
kfbh.endian: 201 ; 0x000: 0xc9
kfbh.hard: 194 ; 0x001: 0xc2
kfbh.type: 212 ; 0x002: *** Unknown Enum ***
kfbh.datfmt: 193 ; 0x003: 0xc1
kfbh.block.blk: 0 ; 0x004: blk=0
kfbh.block.obj: 0 ; 0x008: file=0
kfbh.check: 0 ; 0x00c: 0x00000000
kfbh.fcn.base: 0 ; 0x010: 0x00000000
kfbh.fcn.wrap: 0 ; 0x014: 0x00000000
kfbh.spare1: 0 ; 0x018: 0x00000000
kfbh.spare2: 0 ; 0x01c: 0x00000000
110285600 C9C2D4C1 00000000 00000000 00000000 [................]
110285610 00000000 00000000 00000000 00000000 [................]
Repeat 6 times
110285680 00CC1AD4 8A43B989 00000000 00000000 [.....C..........]
110285690 00000000 00000000 00000000 00000000 [................]
Repeat 246 times
KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][212]
hdisk3和hdisk4磁碟頭資訊的條目都為0，且有明顯的報錯，說明磁碟頭已經被破壞。
2).檢視磁碟頭備份資訊：
在11.1.0.7之前ASM disk header沒有自動備份，需要手動執行kfed read或者dd來備份，從11.1.0.7開始ASM disk header會自動備份到AU#1的倒數第二個block。對於AU size是1MB的DISKGROUP，每個AU包括block數量=1024KB/4KB=256個，因此備份資訊位於AU#1的第254號block，執行下面的命令檢視ASM disk header備份資訊，備份資訊與ASM disk header資訊完全一致：
$ kfed read /dev/rhdisk3 AUNUM=1 BLKNUM=254 ausz=1048576 | more
kfbh.endian: 0 ; 0x000: 0x00
kfbh.hard: 130 ; 0x001: 0x82
kfbh.type: 1 ; 0x002: KFBTYP_DISKHEAD <<<< 表示是磁碟頭資訊。
kfbh.datfmt: 1 ; 0x003: 0x01
kfbh.block.blk: 254 ; 0x004: blk=254
kfbh.block.obj: 2147483648 ; 0x008: disk=0
kfbh.check: 3830034039 ; 0x00c: 0xe449ae77
kfbh.fcn.base: 0 ; 0x010: 0x00000000
kfbh.fcn.wrap: 0 ; 0x014: 0x00000000
kfbh.spare1: 0 ; 0x018: 0x00000000
kfbh.spare2: 0 ; 0x01c: 0x00000000
kfdhdb.driver.provstr: ORCLDISK ; 0x000: length=8
kfdhdb.driver.reserved[0]: 0 ; 0x008: 0x00000000
kfdhdb.driver.reserved[1]: 0 ; 0x00c: 0x00000000
kfdhdb.driver.reserved[2]: 0 ; 0x010: 0x00000000
kfdhdb.driver.reserved[3]: 0 ; 0x014: 0x00000000
kfdhdb.driver.reserved[4]: 0 ; 0x018: 0x00000000
kfdhdb.driver.reserved[5]: 0 ; 0x01c: 0x00000000
kfdhdb.compat: 186646528 ; 0x020: 0x0b200000
kfdhdb.dsknum: 0 ; 0x024: 0x0000
kfdhdb.grptyp: 1 ; 0x026: KFDGTP_EXTERNAL
kfdhdb.hdrsts: 3 ; 0x027: KFDHDR_MEMBER
kfdhdb.dskname: DATA01_0000 ; 0x028: length=11 <<<< disk名稱
kfdhdb.grpname: DATA01 ; 0x048: length=6 <<<< ASM磁碟組名稱呢個
kfdhdb.fgname: DATA01_0000 ; 0x068: length=11 <<<< Failgroup名稱
kfdhdb.capname: ; 0x088: length=0
kfdhdb.crestmp.hi: 33007182 ; 0x0a8: HOUR=0xe DAYS=0x12 MNTH=0x9 YEAR=0x7de
kfdhdb.crestmp.lo: 417121280 ; 0x0ac: USEC=0x0 MSEC=0x331 SECS=0xd MINS=0x6
kfdhdb.mntstmp.hi: 33007182 ; 0x0b0: HOUR=0xe DAYS=0x12 MNTH=0x9 YEAR=0x7de
kfdhdb.mntstmp.lo: 427397120 ; 0x0b4: USEC=0x0 MSEC=0x264 SECS=0x17 MINS=0x6
kfdhdb.secsize: 512 ; 0x0b8: 0x0200 <<<< 扇區 size= 512 byte
kfdhdb.blksize: 4096 ; 0x0ba: 0x1000 <<<< 塊大小 size = 4KB
kfdhdb.ausize: 1048576 ; 0x0bc: 0x00100000 <<<< AU size =1 MB
kfdhdb.mfact: 113792 ; 0x0c0: 0x0001bc80
kfdhdb.dsksize: 51200 ; 0x0c4: 0x0000c800
kfdhdb.pmcnt: 2 ; 0x0c8: 0x00000002
kfdhdb.fstlocn: 1 ; 0x0cc: 0x00000001
kfdhdb.altlocn: 2 ; 0x0d0: 0x00000002
kfdhdb.f1b1locn: 2 ; 0x0d4: 0x00000002
kfdhdb.redomirrors[0]: 0 ; 0x0d8: 0x0000
kfdhdb.redomirrors[1]: 0 ; 0x0da: 0x0000
kfdhdb.redomirrors[2]: 0 ; 0x0dc: 0x0000
kfdhdb.redomirrors[3]: 0 ; 0x0de: 0x0000
kfdhdb.dbcompat: 168820736 ; 0x0e0: 0x0a100000
kfdhdb.grpstmp.hi: 33007182 ; 0x0e4: HOUR=0xe DAYS=0x12 MNTH=0x9 YEAR=0x7de
......
$ kfed read /dev/rhdisk4 AUNUM=1 BLKNUM=254 ausz=1048576 | more
kfbh.endian: 0 ; 0x000: 0x00
kfbh.hard: 130 ; 0x001: 0x82
kfbh.type: 1 ; 0x002: KFBTYP_DISKHEAD <<<< 表示是磁碟頭資訊。
kfbh.datfmt: 1 ; 0x003: 0x01
kfbh.block.blk: 254 ; 0x004: blk=254
kfbh.block.obj: 2147483648 ; 0x008: disk=0
kfbh.check: 3839790854 ; 0x00c: 0xe4de8f06
kfbh.fcn.base: 0 ; 0x010: 0x00000000
kfbh.fcn.wrap: 0 ; 0x014: 0x00000000
kfbh.spare1: 0 ; 0x018: 0x00000000
kfbh.spare2: 0 ; 0x01c: 0x00000000
kfdhdb.driver.provstr: ORCLDISK ; 0x000: length=8
kfdhdb.driver.reserved[0]: 0 ; 0x008: 0x00000000
kfdhdb.driver.reserved[1]: 0 ; 0x00c: 0x00000000
kfdhdb.driver.reserved[2]: 0 ; 0x010: 0x00000000
kfdhdb.driver.reserved[3]: 0 ; 0x014: 0x00000000
kfdhdb.driver.reserved[4]: 0 ; 0x018: 0x00000000
kfdhdb.driver.reserved[5]: 0 ; 0x01c: 0x00000000
kfdhdb.compat: 186646528 ; 0x020: 0x0b200000
kfdhdb.dsknum: 0 ; 0x024: 0x0000
kfdhdb.grptyp: 1 ; 0x026: KFDGTP_EXTERNAL
kfdhdb.hdrsts: 3 ; 0x027: KFDHDR_MEMBER
kfdhdb.dskname: FRA01_0000 ; 0x028: length=10
kfdhdb.grpname: FRA01 ; 0x048: length=5
kfdhdb.fgname: FRA01_0000 ; 0x068: length=10
kfdhdb.capname: ; 0x088: length=0
kfdhdb.crestmp.hi: 33007182 ; 0x0a8: HOUR=0xe DAYS=0x12 MNTH=0x9 YEAR=0x7de
kfdhdb.crestmp.lo: 438000640 ; 0x0ac: USEC=0x0 MSEC=0x2d7 SECS=0x21 MINS=0x6
kfdhdb.mntstmp.hi: 33007182 ; 0x0b0: HOUR=0xe DAYS=0x12 MNTH=0x9 YEAR=0x7de
kfdhdb.mntstmp.lo: 449592320 ; 0x0b4: USEC=0x0 MSEC=0x30f SECS=0x2c MINS=0x6
kfdhdb.secsize: 512 ; 0x0b8: 0x0200
kfdhdb.blksize: 4096 ; 0x0ba: 0x1000
kfdhdb.ausize: 1048576 ; 0x0bc: 0x00100000
kfdhdb.mfact: 113792 ; 0x0c0: 0x0001bc80
kfdhdb.dsksize: 51200 ; 0x0c4: 0x0000c800
kfdhdb.pmcnt: 2 ; 0x0c8: 0x00000002
kfdhdb.fstlocn: 1 ; 0x0cc: 0x00000001
kfdhdb.altlocn: 2 ; 0x0d0: 0x00000002
kfdhdb.f1b1locn: 2 ; 0x0d4: 0x00000002
kfdhdb.redomirrors[0]: 0 ; 0x0d8: 0x0000
kfdhdb.redomirrors[1]: 0 ; 0x0da: 0x0000
kfdhdb.redomirrors[2]: 0 ; 0x0dc: 0x0000
kfdhdb.redomirrors[3]: 0 ; 0x0de: 0x0000
kfdhdb.dbcompat: 168820736 ; 0x0e0: 0x0a100000
kfdhdb.grpstmp.hi: 33007182 ; 0x0e4: HOUR=0xe DAYS=0x12 MNTH=0x9 YEAR=0x7de
3).使用kfed repair命令修復ASM磁碟頭：
恢復ASM disk header的方法是執行kfed repair命令，利用備份資訊恢復ASM disk header：
$ kfed repair /dev/rhdisk3 aus=1048576
$ kfed repair /dev/rhdisk4 aus=1048576
4).檢視恢復後的ASM磁碟頭資訊：
$ kfed read /dev/rhdisk3
kfbh.endian: 0 ; 0x000: 0x00
kfbh.hard: 130 ; 0x001: 0x82
kfbh.type: 1 ; 0x002: KFBTYP_DISKHEAD <<<< 表示是磁碟頭資訊。
kfbh.datfmt: 1 ; 0x003: 0x01
kfbh.block.blk: 0 ; 0x004: blk=0
kfbh.block.obj: 2147483648 ; 0x008: disk=0
kfbh.check: 3830034057 ; 0x00c: 0xe449ae89
kfbh.fcn.base: 0 ; 0x010: 0x00000000
kfbh.fcn.wrap: 0 ; 0x014: 0x00000000
kfbh.spare1: 0 ; 0x018: 0x00000000
kfbh.spare2: 0 ; 0x01c: 0x00000000
kfdhdb.driver.provstr: ORCLDISK ; 0x000: length=8
kfdhdb.driver.reserved[0]: 0 ; 0x008: 0x00000000
kfdhdb.driver.reserved[1]: 0 ; 0x00c: 0x00000000
kfdhdb.driver.reserved[2]: 0 ; 0x010: 0x00000000
kfdhdb.driver.reserved[3]: 0 ; 0x014: 0x00000000
kfdhdb.driver.reserved[4]: 0 ; 0x018: 0x00000000
kfdhdb.driver.reserved[5]: 0 ; 0x01c: 0x00000000
kfdhdb.compat: 186646528 ; 0x020: 0x0b200000
kfdhdb.dsknum: 0 ; 0x024: 0x0000
kfdhdb.grptyp: 1 ; 0x026: KFDGTP_EXTERNAL
kfdhdb.hdrsts: 3 ; 0x027: KFDHDR_MEMBER
kfdhdb.dskname: DATA01_0000 ; 0x028: length=11 <<<< disk名稱
kfdhdb.grpname: DATA01 ; 0x048: length=6 <<<< group名稱
kfdhdb.fgname: DATA01_0000 ; 0x068: length=11 <<<< failure group名稱
kfdhdb.capname: ; 0x088: length=0
kfdhdb.crestmp.hi: 33007182 ; 0x0a8: HOUR=0xe DAYS=0x12 MNTH=0x9 YEAR=0x7de
kfdhdb.crestmp.lo: 417121280 ; 0x0ac: USEC=0x0 MSEC=0x331 SECS=0xd MINS=0x6
kfdhdb.mntstmp.hi: 33007182 ; 0x0b0: HOUR=0xe DAYS=0x12 MNTH=0x9 YEAR=0x7de
kfdhdb.mntstmp.lo: 427397120 ; 0x0b4: USEC=0x0 MSEC=0x264 SECS=0x17 MINS=0x6
kfdhdb.secsize: 512 ; 0x0b8: 0x0200
kfdhdb.blksize: 4096 ; 0x0ba: 0x1000 <<<< ASM block size = 4KB
kfdhdb.ausize: 1048576 ; 0x0bc: 0x00100000 <<<< ASM AU size =1 MB
kfdhdb.mfact: 113792 ; 0x0c0: 0x0001bc80
kfdhdb.dsksize: 51200 ; 0x0c4: 0x0000c800
kfdhdb.pmcnt: 2 ; 0x0c8: 0x00000002
kfdhdb.fstlocn: 1 ; 0x0cc: 0x00000001
kfdhdb.altlocn: 2 ; 0x0d0: 0x00000002
kfdhdb.f1b1locn: 2 ; 0x0d4: 0x00000002
kfdhdb.redomirrors[0]: 0 ; 0x0d8: 0x0000
kfdhdb.redomirrors[1]: 0 ; 0x0da: 0x0000
kfdhdb.redomirrors[2]: 0 ; 0x0dc: 0x0000
kfdhdb.redomirrors[3]: 0 ; 0x0de: 0x0000
kfdhdb.dbcompat: 168820736 ; 0x0e0: 0x0a100000
kfdhdb.grpstmp.hi: 33007182 ; 0x0e4: HOUR=0xe DAYS=0x12 MNTH=0x9 YEAR=0x7de
kfdhdb.grpstmp.lo: 416513024 ; 0x0e8: USEC=0x0 MSEC=0xdf SECS=0xd MINS=0x6
......
$ kfed read /dev/rhdisk4
kfbh.endian: 0 ; 0x000: 0x00
kfbh.hard: 130 ; 0x001: 0x82
kfbh.type: 1 ; 0x002: KFBTYP_DISKHEAD
kfbh.datfmt: 1 ; 0x003: 0x01
kfbh.block.blk: 0 ; 0x004: blk=0
kfbh.block.obj: 2147483648 ; 0x008: disk=0
kfbh.check: 3839791096 ; 0x00c: 0xe4de8ff8
kfbh.fcn.base: 0 ; 0x010: 0x00000000
kfbh.fcn.wrap: 0 ; 0x014: 0x00000000
kfbh.spare1: 0 ; 0x018: 0x00000000
kfbh.spare2: 0 ; 0x01c: 0x00000000
kfdhdb.driver.provstr: ORCLDISK ; 0x000: length=8
kfdhdb.driver.reserved[0]: 0 ; 0x008: 0x00000000
kfdhdb.driver.reserved[1]: 0 ; 0x00c: 0x00000000
kfdhdb.driver.reserved[2]: 0 ; 0x010: 0x00000000
kfdhdb.driver.reserved[3]: 0 ; 0x014: 0x00000000
kfdhdb.driver.reserved[4]: 0 ; 0x018: 0x00000000
kfdhdb.driver.reserved[5]: 0 ; 0x01c: 0x00000000
kfdhdb.compat: 186646528 ; 0x020: 0x0b200000
kfdhdb.dsknum: 0 ; 0x024: 0x0000
kfdhdb.grptyp: 1 ; 0x026: KFDGTP_EXTERNAL
kfdhdb.hdrsts: 3 ; 0x027: KFDHDR_MEMBER
kfdhdb.dskname: FRA01_0000 ; 0x028: length=10
kfdhdb.grpname: FRA01 ; 0x048: length=5
kfdhdb.fgname: FRA01_0000 ; 0x068: length=10
kfdhdb.capname: ; 0x088: length=0
kfdhdb.crestmp.hi: 33007182 ; 0x0a8: HOUR=0xe DAYS=0x12 MNTH=0x9 YEAR=0x7de
kfdhdb.crestmp.lo: 438000640 ; 0x0ac: USEC=0x0 MSEC=0x2d7 SECS=0x21 MINS=0x6
kfdhdb.mntstmp.hi: 33007182 ; 0x0b0: HOUR=0xe DAYS=0x12 MNTH=0x9 YEAR=0x7de
kfdhdb.mntstmp.lo: 449592320 ; 0x0b4: USEC=0x0 MSEC=0x30f SECS=0x2c MINS=0x6
kfdhdb.secsize: 512 ; 0x0b8: 0x0200
kfdhdb.blksize: 4096 ; 0x0ba: 0x1000
kfdhdb.ausize: 1048576 ; 0x0bc: 0x00100000
kfdhdb.mfact: 113792 ; 0x0c0: 0x0001bc80
kfdhdb.dsksize: 51200 ; 0x0c4: 0x0000c800
kfdhdb.pmcnt: 2 ; 0x0c8: 0x00000002
kfdhdb.fstlocn: 1 ; 0x0cc: 0x00000001
kfdhdb.altlocn: 2 ; 0x0d0: 0x00000002
kfdhdb.f1b1locn: 2 ; 0x0d4: 0x00000002
kfdhdb.redomirrors[0]: 0 ; 0x0d8: 0x0000
kfdhdb.redomirrors[1]: 0 ; 0x0da: 0x0000
kfdhdb.redomirrors[2]: 0 ; 0x0dc: 0x0000
kfdhdb.redomirrors[3]: 0 ; 0x0de: 0x0000
kfdhdb.dbcompat: 168820736 ; 0x0e0: 0x0a100000
kfdhdb.grpstmp.hi: 33007182 ; 0x0e4: HOUR=0xe DAYS=0x12 MNTH=0x9 YEAR=0x7de
kfdhdb.grpstmp.lo: 437550080 ; 0x0e8: USEC=0x0 MSEC=0x11f SECS=0x21 MINS=0x6
......
可以看到，恢復後的ASM磁碟頭可以看到ASM磁碟組相關的資訊，注意和損壞的進行對比，損壞的ASM磁碟頭看不到這些資訊，在最後有明顯的報錯。
四.重新MOUNT磁碟組，開啟資料庫。
1).檢查當前CRS服務狀態：
/u01/app/11.2.0/grid/bin
# ./crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA01.dg
ONLINE OFFLINE localhost
ora.FRA01.dg
ONLINE OFFLINE localhost
ora.LISTENER.lsnr
ONLINE ONLINE localhost
ora.asm
ONLINE ONLINE localhost Started
ora.ons
OFFLINE OFFLINE localhost
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
1 ONLINE ONLINE localhost
ora.diskmon
1 OFFLINE OFFLINE
ora.evmd
1 ONLINE ONLINE localhost
ora.orcl.db
1 ONLINE OFFLINE Instance Shutdown
2).載入ASM磁碟組：
# su - grid
$ sqlplus / as sysasm
SQL*Plus: Release 11.2.0.3.0 Production on Thu Sep 18 14:53:19 2014
Copyright (c) 1982, 2011, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Automatic Storage Management option
SQL> alter diskgroup data01 mount;
Diskgroup altered.
SQL> alter diskgroup fra01 mount;
Diskgroup altered.
3).檢查CRS服務狀態：
# ./crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA01.dg
ONLINE ONLINE localhost
ora.FRA01.dg
ONLINE ONLINE localhost
ora.LISTENER.lsnr
ONLINE ONLINE localhost
ora.asm
ONLINE ONLINE localhost Started
ora.ons
OFFLINE OFFLINE localhost
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
1 ONLINE ONLINE localhost
ora.diskmon
1 OFFLINE OFFLINE
ora.evmd
1 ONLINE ONLINE localhost
ora.orcl.db
1 ONLINE ONLINE localhost Open
4).檢查資料庫狀態：
# su - oracle
$ sql
SQL*Plus: Release 11.2.0.3.0 Production on Thu Sep 18 14:54:17 2014
Copyright (c) 1982, 2011, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, Automatic Storage Management, OLAP, Data Mining
and Real Application Testing options
SQL> select status from v$instance;
STATUS
------------
OPEN
SQL> select name,open_mode from v$database;
NAME OPEN_MODE
--------- --------------------
ORCL READ WRITE
參考文章：
《ASM常見錯誤 ORA-15063: ASM discovered an insufficient number of disks for diskgroup》：https://blogs.oracle.com/Database4CN/entry/asm%E5%B8%B8%E8%A7%81%E9%94%99%E8%AF%AF_ora_15063_asm_discovered
相關文章：
《瞭解和使用kfed維護ASM後設資料》：http://blog.itpub.net/23135684/viewspace-1079197/
注意：在使用kfed工具修復ASM磁碟頭之前，最好將磁碟上的PVID清除掉。
--end--
來自 “ ITPUB部落格 ” ，連結：http://blog.itpub.net/23135684/viewspace-1273077/，如需轉載，請註明出處，否則將追究法律責任。
相關文章
- 修復ASM磁碟頭（一）ASM
- 【ASM】ASM磁碟頭被重寫，如何修復ASM
- ASM磁碟頭資訊損壞和修復（kfed/dd）ASM
- ASM磁碟頭ASM
- ASM磁碟組丟失member kfed修復ASM
- asm磁碟組中的磁碟頭資訊丟失恢復實驗ASM
- ASM磁碟頭比較ASM
- 揭祕ASM磁碟頭資訊ASM
- ASM 磁碟頭資訊備份ASM
- Asm diskgroup 的修復ASM
- 磁碟修復工具TestDisk
- ASM 磁碟頭資訊備份與恢復--有空研究一下ASM
- ASM磁碟故障診斷（二）ASM
- ASM磁碟頭資訊備份,轉載ASM
- Oracle 10g rac+asm 磁碟頭備份與恢復實驗Oracle 10gASM
- 「磁碟工具」不能修復磁碟時，怎麼做?
- 【LINUX】Oracle資料庫 linux磁碟頭資料損壞修復LinuxOracle資料庫
- Oracle asm磁碟損壞異常恢復OracleASM
- DiskGenius(磁碟修復工具)官方版
- linux下修復磁碟損壞Linux
- 用Mac磁碟工具進行磁碟自檢與修復Mac
- 電腦開機修復磁碟半天 win10一直卡在磁碟修復怎麼辦Win10
- 一次asm磁碟頭部資訊丟失故障ASM
- RAC 11G ASM磁碟損壞恢復ASM
- win10一直卡在磁碟修復怎麼修復_win10一直卡在磁碟修復開不了機如何處理Win10
- 【ASM】如何建立ASM磁碟ASM
- ASM之建立ASM磁碟ASM
- 從ASM磁碟中還原出檔案（二）ASM
- 恢復表空間到不同的ASM磁碟組ASM
- 某殼分析+修復（二）
- ASM 增加磁碟ASM
- 蘋果mac怎麼修復磁碟許可權蘋果Mac
- Oracle asm磁碟中新加磁碟OracleASM
- 存貯在asm中的ocr/votedisk/asm磁碟組損害的恢復ASM
- ASM之OCR所在磁碟組損壞後的恢復ASM
- asm 磁碟組 增刪磁碟組ASM
- 如何掃描和修復 Linux 磁碟錯誤Linux
- ASM磁碟組限制ASM