測試環境的一套兩節點RAC，一個節點出現故障，啟動不了例項

檢視節點1的grid日誌，發現找不到表決盤
tail -100 /opt/ora11grid/log/vgerndpud853/alertvgerndpud853.log
..
[cssd(3824)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /opt/ora11grid/log/vgerndpud853/cssd/ocssd.log
2016-10-17 07:56:11.768
[cssd(3824)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /opt/ora11grid/log/vgerndpud853/cssd/ocssd.log
2016-10-17 07:56:26.781
[cssd(3824)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /opt/ora11grid/log/vgerndpud853/cssd/ocssd.log
2016-10-17 07:56:41.794
[cssd(3824)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /opt/ora11grid/log/vgerndpud853/cssd/ocssd.log
2016-10-17 07:56:55.095
[/opt/ora11grid/bin/cssdagent(3812)]CRS-5818:Aborted command 'start' for resource 'ora.cssd'. Details at (:CRSAGF00113:) {0:34:3} in /opt/ora11grid/log/vgerndpud853/agent/ohasd/oracssdagent_root/oracssdagent_root.log.
2016-10-17 07:56:55.096
[cssd(3824)]CRS-1656:The CSS daemon is terminating due to a fatal error; Details at (:CSSSC00012:) in /opt/ora11grid/log/vgerndpud853/cssd/ocssd.log

檢視grid日誌中提到的詳細日誌，進一步驗證了表決盤丟失
tail -100 /opt/ora11grid/log/vgerndpud853/cssd/ocssd.log
..
222499 2016-10-17 08:06:58.959: [   SKGFD][257890048]Discovery with str:/voting/vot1/vot1.data,/voting/vot2/vot2.data,/voting/vot3/vot3.data:
222500
222501 2016-10-17 08:06:58.959: [   SKGFD][257890048]UFS discovery with :/voting/vot1/vot1.data:
222502
222503 2016-10-17 08:06:58.959: [   SKGFD][257890048]OSS discovery with :/voting/vot1/vot1.data:
222504
222505 2016-10-17 08:06:58.959: [   SKGFD][257890048]Discovery advancing to nxt string :/voting/vot2/vot2.data:
222506
222507 2016-10-17 08:06:58.959: [   SKGFD][257890048]UFS discovery with :/voting/vot2/vot2.data:
222508
222509 2016-10-17 08:06:58.959: [   SKGFD][257890048]OSS discovery with :/voting/vot2/vot2.data:
222510
222511 2016-10-17 08:06:58.959: [   SKGFD][257890048]Discovery advancing to nxt string :/voting/vot3/vot3.data:
222512
222513 2016-10-17 08:06:58.959: [   SKGFD][257890048]UFS discovery with :/voting/vot3/vot3.data:
222514
222515 2016-10-17 08:06:58.959: [   SKGFD][257890048]OSS discovery with :/voting/vot3/vot3.data:
222516
222517 2016-10-17 08:06:58.959: [    CSSD][257890048]clssnmvDiskVerify: Successful discovery of 0 disks
222518 2016-10-17 08:06:58.959: [    CSSD][257890048]clssnmCompleteInitVFDiscovery: Completing initial voting file discovery

登陸節點2
檢視叢集狀態，發現節點2的狀態正常
crs_stat -t -v

檢視錶決盤和crs盤的掛載情況，正常
vgerndpud852: /opt/ora11grid/bin # mount | grep vot
/dev/vx/dsk/dg_orabe/v_vot2 on /voting/vot2 type vxfs (rw,mntlock=VCS,cluster,crw,delaylog,largefiles,ioerror=mdisable)
/dev/vx/dsk/dg_orabe/v_vot1 on /voting/vot1 type vxfs (rw,mntlock=VCS,cluster,crw,delaylog,largefiles,ioerror=mdisable)
/dev/vx/dsk/dg_orabe/v_vot3 on /voting/vot3 type vxfs (rw,mntlock=VCS,cluster,crw,delaylog,largefiles,ioerror=mdisable)

vgerndpud852: /opt/ora11grid/bin # mount | grep ocr
/dev/vx/dsk/dg_orabe/v_ocr2 on /ocr/ocr2 type vxfs (rw,mntlock=VCS,cluster,crw,delaylog,largefiles,ioerror=mdisable)
/dev/vx/dsk/dg_orabe/v_ocr3 on /ocr/ocr3 type vxfs (rw,mntlock=VCS,cluster,crw,delaylog,largefiles,ioerror=mdisable)
/dev/vx/dsk/dg_orabe/v_ocr1 on /ocr/ocr1 type vxfs (rw,mntlock=VCS,cluster,crw,delaylog,largefiles,ioerror=mdisable)

在節點1，檢視錶決盤和crs盤的掛載情況，發現盤沒有掛載上
/opt/ora11grid[FRWK]:mount | grep vot
/opt/ora11grid[FRWK]:mount | grep ocr

重新掛載盤，重啟叢集，RAC恢復正常

Oracle 11g RAC表決盤和OCR盤掛載失敗引發的節點故障

相關文章