RAC ORA-15096: lost disk write detected錯誤解決方法

新部署機器因為意外斷電導致bug
REASON:
NOTE: GMON heartbeating for grp 1
GMON querying group 1 at 5 for pid 23, osid 2872
NOTE: cache opening disk 0 of grp 1: FRAVOL label:FRAVOL
NOTE: F1X0 found on disk 0 au 2 fcn 0.0
NOTE: cache mounting (first) external redundancy group 1/0x9B8856B5 (ORAARCH)
Sun May 25 13:23:18 2014
* allocate domain 1, invalid = TRUE 
Sun May 25 13:23:18 2014
NOTE: attached to recovery domain 1
NOTE: cache recovered group 1 to fcn 0.1567
NOTE: redo buffer size is 256 blocks (1053184 bytes)
Sun May 25 13:23:19 2014
NOTE: LGWR attempting to mount thread 1 for diskgroup 1 (ORAARCH)
NOTE: LGWR found thread 1 closed at ABA 10.173
NOTE: LGWR mounted thread 1 for diskgroup 1 (ORAARCH)
NOTE: LGWR opening thread 1 at fcn 0.1567 ABA 11.174
NOTE: cache mounting group 1/0x9B8856B5 (ORAARCH) succeeded
NOTE: cache ending mount (success) of group ORAARCH number=1 incarn=0x9b8856b5
NOTE: GMON heartbeating for grp 2
GMON querying group 2 at 7 for pid 23, osid 2872
NOTE: cache opening disk 0 of grp 2: CRSVOL label:CRSVOL
NOTE: F1X0 found on disk 0 au 2 fcn 0.0
NOTE: cache mounting (first) external redundancy group 2/0x9B9856B6 (ORACRS)
* allocate domain 2, invalid = TRUE 
NOTE: attached to recovery domain 2
NOTE: starting recovery of thread=1 ckpt=10.62 group=2 (ORACRS)
NOTE: starting recovery of thread=2 ckpt=7.10 group=2 (ORACRS)
NOTE: BWR validation signaled ORA-15096
Errors in file /app/grid/grid_base/diag/asm/+asm/+ASM2/trace/+ASM2_ora_2872.trc:
ORA-15096: lost disk write detected
NOTE: crash recovery signalled OER-15096
ERROR: ORA-15096 signalled during mount of diskgroup ORACRS
NOTE: cache dismounting (clean) group 2/0x9B9856B6 (ORACRS) 
NOTE: messaging CKPT to quiesce pins Unix process pid: 2872, image: oracle@his1 (TNS V1-V3)
NOTE: lgwr not being msg'd to dismount
freeing rdom 2
NOTE: detached from domain 2
NOTE: cache dismounted group 2/0x9B9856B6 (ORACRS) 
NOTE: cache ending mount (fail) of group ORACRS number=2 incarn=0x9b9856b6
NOTE: cache deleting context for group ORACRS 2/0x9b9856b6
GMON dismounting group 2 at 8 for pid 23, osid 2872
NOTE: Disk  in mode 0x8 marked for de-assignment
ERROR: diskgroup ORACRS was not mounted
NOTE: GMON heartbeating for grp 3
GMON querying group 3 at 10 for pid 23, osid 2872
NOTE: cache opening disk 0 of grp 3: DATVOL label:DATVOL
NOTE: F1X0 found on disk 0 au 2 fcn 0.0
NOTE: cache mounting (first) external redundancy group 3/0x9BA856B7 (ORADATA)
* allocate domain 3, invalid = TRUE 
NOTE: attached to recovery domain 3
NOTE: cache recovered group 3 to fcn 0.5984
NOTE: redo buffer size is 256 blocks (1053184 bytes)
NOTE: LGWR attempting to mount thread 1 for diskgroup 3 (ORADATA)
NOTE: LGWR found thread 1 closed at ABA 12.588
NOTE: LGWR mounted thread 1 for diskgroup 3 (ORADATA)
NOTE: LGWR opening thread 1 at fcn 0.5984 ABA 13.589
NOTE: cache mounting group 3/0x9BA856B7 (ORADATA) succeeded
NOTE: cache ending mount (success) of group ORADATA number=3 incarn=0x9ba856b7
GMON querying group 1 at 11 for pid 18, osid 2862
Sun May 25 13:23:20 2014
NOTE: Instance updated compatible.asm to 11.2.0.0.0 for grp 1
SUCCESS: diskgroup ORAARCH was mounted
GMON querying group 3 at 12 for pid 18, osid 2862
NOTE: Instance updated compatible.asm to 11.2.0.0.0 for grp 3
SUCCESS: diskgroup ORADATA was mounted
WARNING: Disk Group ORACRS containing spfile for this instance is not mounted
WARNING: Disk Group ORACRS containing configured OCR is not mounted
WARNING: Disk Group ORACRS containing voting files is not mounted
ORA-15032: not all alterations performed
ORA-15096: lost disk write detected
ERROR: ALTER DISKGROUP ALL MOUNT /* asm agent call crs *//* {0:0:2} */
SQL> ALTER DISKGROUP ALL ENABLE VOLUME ALL /* asm agent *//* {0:0:2} */ 
SUCCESS: ALTER DISKGROUP ALL ENABLE VOLUME ALL /* asm agent *//* {0:0:2} */
Sun May 25 13:23:22 2014
WARNING: failed to online diskgroup resource ora.ORAARCH.dg (unable to communicate with CRSD/OHASD)
WARNING: failed to online diskgroup resource ora.ORADATA.dg (unable to communicate with CRSD/OHASD)
Sun May 25 13:24:01 2014