儲存裝置許可權不對導致crs啟動出錯
今天在3節點測試庫進行了一下壓力測試，導致一個節點hang住。
另外一個節點也出現I/O error：
重啟後發現節點crs起不來：
1 檢查crs是否啟動
[root@rac3 log]# ps -ef | grep -i crs
root 28137 1 0 11:17 ? 00:00:00 /bin/sh /etc/init.d/init.crsd run
root 31789 30006 0 11:29 pts/1 00:00:00 grep -i crs
[root@rac3 log]#
[root@rac3 log]#
[root@rac3 log]# crsctl check crs
Failure 1 contacting CSS daemon
Cannot communicate with CRS
Cannot communicate with EVM
[root@rac3 log]#
[root@rac3 log]# crsctl check crs
Failure 1 contacting CSS daemon
Cannot communicate with CRS
Cannot communicate with EVM
[root@rac3 log]# ps -ef |egrep "crsd.bin|ocssd.bin|evmd.bin|oprocd"
root 31938 30006 0 11:30 pts/1 00:00:00 egrep crsd.bin|ocssd.bin|evmd.bin|oprocd
[root@rac3 log]#
[root@rac3 log]#
2 檢查OCR and voting disk裝置是否正確配置；
檢查OCR裝置的配置檔案/etc/oracle/ocr.loc：
[root@rac3 log]# crsctl check crs
Failure 1 contacting CSS daemon
Cannot communicate with CRS
Cannot communicate with EVM
[root@rac3 log]# crsctl query css votedisk
0. 0 /dev/raw/raw3
1. 0 /dev/raw/raw4
2. 0 /dev/raw/raw5
located 3 votedisk(s).
[root@rac3 log]# ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) : 106564
Used space (kbytes) : 5444
Available space (kbytes) : 101120
ID : 1393012097
Device/File Name : /dev/raw/raw1
Device/File integrity check succeeded
Device/File Name : /dev/raw/raw2
Device/File integrity check succeeded
Cluster registry integrity check succeeded
[root@rac3 log]# crsctl stop crs
Stopping resources. This could take several minutes.
Error while stopping resources. Possible cause: CRSD is down.
[root@rac3 log]# crsctl start crs
Attempting to start CRS stack
The CRS stack will be started shortly
[root@rac3 log]#
[root@rac3 log]# ps -ef |grep d.bin
root 607 30006 0 11:39 pts/1 00:00:00 grep d.bin
[root@rac3 log]# ps -ef |grep d.bin
root 609 30006 0 11:39 pts/1 00:00:00 grep d.bin
[root@rac3 log]# ps -ef |grep d.bin
root 618 30006 0 11:39 pts/1 00:00:00 grep d.bin
[root@rac3 log]# ps -ef |grep d.bin
root 620 30006 0 11:39 pts/1 00:00:00 grep d.bin
[root@rac3 log]# ps -ef |grep d.bin
root 896 30006 0 11:41 pts/1 00:00:00 grep d.bin
[root@rac3 mapper]# cat /etc/oracle/ocr.loc
ocrconfig_loc=/dev/raw/raw1
ocrmirrorconfig_loc=/dev/raw/raw2
local_only=FALSE
3 檢查 crsd.log，發現沒有任何日誌：
[oracle@rac3 rac3]$ cd crsd/
[oracle@rac3 crsd]$ ls -al
total 76
drwxr-x--- 2 root oinstall 4096 May 3 19:03 .
drwxr-xr-t 8 root oinstall 4096 May 3 19:03 ..
-rw-r--r-- 1 root root 53171 May 11 10:38 crsd.log
[oracle@rac3 crsd]$ tail -n 50 crsd.log
2011-05-11 10:15:09.400: [ CRSRES][1484962144]0startRunnable: setting CLI values
2011-05-11 10:16:29.767: [ OCRUTL][1252067680]u_freem: mem passed is null
2011-05-11 10:32:25.140: [ CRSEVT][1487063392]0CAAMonitorHandler :: 0:Could not join /home/oracle/oracle/product/10.2.0/crs/bin/racgwrap(check)
category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
2011-05-11 10:32:25.140: [ CRSEVT][1487063392]0CAAMonitorHandler :: 0:Action Script. /home/oracle/oracle/product/10.2.0/crs/bin/racgwrap(check) timed out for ora.rac3.vip!
(timeout=60)
2011-05-11 10:32:25.140: [ CRSAPP][1487063392]0CheckResource error for ora.rac3.vip error code = -2
2011-05-11 10:38:14.731: [ CRSEVT][1499654496]0CAAMonitorHandler :: 0:Could not join /home/oracle/oracle/product/10.2.0/crs/bin/racgwrap(check)
category: 1234, operation: scls_process_join, loc: childcrash, OS error: 0, other: Abnormal termination of the child
2011-05-11 10:38:25.335: [ CRSEVT][1499654496]0CAAMonitorHandler :: 0:Action Script. /home/oracle/oracle/product/10.2.0/crs/bin/racgwrap(check) timed out for ora.rac3.vip!
(timeout=60)
2011-05-11 10:38:25.335: [ CRSAPP][1499654496]0CheckResource error for ora.rac3.vip error code = -2
2011-05-11 10:38:33.140: [ CRSRES][1503856992]0In stateChanged, ora.rac.rac3.inst target is ONLINE
2011-05-11 10:38:33.141: [ CRSRES][1503856992]0ora.rac.rac3.inst on rac3 went OFFLINE unexpectedly
2011-05-11 10:38:33.141: [ CRSRES][1503856992]0StopResource: setting CLI values
2011-05-11 10:38:38.143: [ CRSRES][1503856992]0Attempting to stop `ora.rac.rac3.inst` on member `rac3`
4 注意到/etc/init.d/init.cssd startcheck ，於是檢查/tmp
[root@rac3 log]# ps -ef
root 28128 1 0 11:17 ? 00:00:00 /usr/bin/gdm-binary -nodaemon
root 28132 1 0 11:17 ? 00:00:00 /bin/sh /etc/init.d/init.evmd run
root 28135 1 0 11:17 ? 00:00:00 /bin/sh /etc/init.d/init.cssd fatal
root 28137 1 0 11:17 ? 00:00:00 /bin/sh /etc/init.d/init.crsd run
root 28678 28132 0 11:17 ? 00:00:00 /bin/sh /etc/init.d/init.cssd startcheck
root 28982 28135 0 11:17 ? 00:00:00 /bin/sh /etc/init.d/init.cssd startcheck
root 29026 28128 0 11:17 ? 00:00:00 /usr/bin/gdm-binary -nodaemon
root 29068 29026 0 11:17 ? 00:00:05 /usr/X11R6/bin/X :0 -audit 0 -auth /var/gdm/:0.Xauth -nolisten tcp vt7
root 29201 28137 0 11:17 ? 00:00:00 /bin/sh /etc/init.d/init.cssd startcheck
[root@rac3 mapper]# cd /tmp
[root@rac3 tmp]# ls -altr
-rw-r--r-- 1 oracle oinstall 148 May 11 12:34 crsctl.29201
-rw-r--r-- 1 oracle oinstall 148 May 11 12:34 crsctl.28982
-rw-r--r-- 1 oracle oinstall 148 May 11 12:34 crsctl.28678
[root@rac3 tmp]# cat crsctl.28678
OCR initialization failed accessing OCR device: PROC-26: Error while accessing the physical storage Operating System error [Permission denied] [13]
[root@rac3 tmp]#
[root@rac3 tmp]#
5 通過上述錯誤，應該是許可權問題：
[root@rac3 tmp]# cd /dev/raw
[root@rac3 raw]# ls -al
total 0
drwxr-xr-x 2 root root 180 May 11 11:17 .
drwxr-xr-x 11 root root 7620 May 11 11:17 ..
crw-rw---- 1 root disk 162, 1 May 11 11:17 raw1
crw-rw---- 1 root disk 162, 2 May 11 11:17 raw2
crw-rw---- 1 root disk 162, 3 May 11 11:17 raw3
crw-rw---- 1 root disk 162, 4 May 11 11:17 raw4
crw-rw---- 1 root disk 162, 5 May 11 11:17 raw5
crw-rw---- 1 root disk 162, 6 May 11 11:17 raw6
crw-rw---- 1 root disk 162, 7 May 11 11:17 raw7
[root@rac3 raw]# chown -R oracle:dba /dev/raw
[root@rac3 raw]# chmod -R 777 /dev/raw
[root@rac3 raw]#
6 檢查許可權情況，啟動crs：
[root@rac3 raw]# ls -al
total 0
drwxrwxrwx 2 oracle dba 180 May 11 11:17 .
drwxr-xr-x 11 root root 7620 May 11 11:17 ..
crwxrwxrwx 1 oracle dba 162, 1 May 11 11:17 raw1
crwxrwxrwx 1 oracle dba 162, 2 May 11 11:17 raw2
crwxrwxrwx 1 oracle dba 162, 3 May 11 11:17 raw3
crwxrwxrwx 1 oracle dba 162, 4 May 11 11:17 raw4
crwxrwxrwx 1 oracle dba 162, 5 May 11 11:17 raw5
crwxrwxrwx 1 oracle dba 162, 6 May 11 11:17 raw6
crwxrwxrwx 1 oracle dba 162, 7 May 11 11:17 raw7
[root@rac3 raw]#
[root@rac3 raw]# crsctl stop crs
Stopping resources. This could take several minutes.
Successfully stopped CRS resources.
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.
[root@rac3 raw]# crsctl start crs
Attempting to start CRS stack
The CRS stack will be started shortly
[root@rac3 raw]# crsctl check crs
CSS appears healthy
CRS appears healthy
EVM appears healthy
[root@rac3 raw]# crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.rac.db application ONLINE ONLINE rac2
ora....c1.inst application ONLINE ONLINE rac1
ora....c2.inst application ONLINE ONLINE rac2
ora....c3.inst application ONLINE ONLINE rac3
ora....SM1.asm application ONLINE ONLINE rac1
ora....C1.lsnr application ONLINE ONLINE rac1
ora.rac1.gsd application ONLINE ONLINE rac1
ora.rac1.ons application ONLINE ONLINE rac1
ora.rac1.vip application ONLINE ONLINE rac1
ora....SM2.asm application ONLINE ONLINE rac2
ora....C2.lsnr application ONLINE ONLINE rac2
ora.rac2.gsd application ONLINE ONLINE rac2
ora.rac2.ons application ONLINE ONLINE rac2
ora.rac2.vip application ONLINE ONLINE rac2
ora....SM3.asm application ONLINE ONLINE rac3
ora....C3.lsnr application ONLINE ONLINE rac3
ora.rac3.gsd application ONLINE ONLINE rac3
ora.rac3.ons application ONLINE ONLINE rac3
ora.rac3.vip application ONLINE ONLINE rac3
[root@rac3 raw]# ps -ef |grep d.bin
oracle 11568 11563 0 12:38 ? 00:00:00 /home/oracle/oracle/product/10.2.0/crs/bin/evmd.bin
root 11773 10601 0 12:38 ? 00:00:02 /home/oracle/oracle/product/10.2.0/crs/bin/crsd.bin reboot
root 12310 11853 0 12:38 ? 00:00:00 /home/oracle/oracle/product/10.2.0/crs/bin/oprocd.bin run -t 1000 -m 10000 -hsi 5:10:50:75:90 -f
oracle 12454 11890 0 12:38 ? 00:00:01 /home/oracle/oracle/product/10.2.0/crs/bin/ocssd.bin
root 22908 30006 0 13:03 pts/1 00:00:00 grep d.bin
來自 “ ITPUB部落格 ” ，連結：http://blog.itpub.net/758322/viewspace-695008/，如需轉載，請註明出處，否則將追究法律責任。
相關文章
- Oracle RAC啟動CRS報錯：登陸許可權問題Oracle
- Oracle 11.2.0.4 RAC $ORACLE_HOME/bin/oracle許可權不對導致無法啟動例項Oracle
- RAC oracle 許可權更改導致 實力啟動失敗Oracle
- Vim儲存時許可權不足
- 儲存過程與許可權儲存過程
- 自動掛載裸裝置及許可權
- /tmp檔案系統無許可權導致監聽listener啟動失敗
- 儲存過程與許可權（二）儲存過程
- 安裝clusterware報告對裸裝置沒有操作許可權
- Grant許可權導致執行計劃失效
- mysql儲存過程的許可權 definerMySql儲存過程
- oracle的儲存許可權的檢視Oracle
- 資料庫許可權-儲存過程資料庫儲存過程
- HP-UX+11.2.0.3RAC因裸裝置許可權不一致導致RAC一個節點報錯的解決UX
- 記一次 Laravel日誌許可權許可權問題（定時器導致）Laravel定時器
- 讓使用者擁有儲存過程的除錯許可權儲存過程除錯
- MySQL儲存過程的許可權問題MySql儲存過程
- uid不一致導致tomcat對目錄沒有寫許可權UITomcat
- 11gRAC許可權問題導致的叢集及資料庫無法啟動資料庫
- jboss + jdk5.0 + my sql 安裝後報錯"對不起，沒有許可權操作"JDKSQL
- 動態SQL 無許可權錯誤SQL
- win10 儲存excel報錯 許可權問題怎麼解決Win10Excel
- 使用者有connect,resource,dba角色許可權後回收dba許可權導致無UNLIMITED TABLESPACE許可權造成業務中斷MIT
- oracle儲存過程許可權繼承小結Oracle儲存過程繼承
- 儲存過程許可權__Authid Current_User儲存過程
- ORACLE中儲存過程的許可權問題Oracle儲存過程
- 【許可權】儲存過程執行時，報ORA-01031許可權不足儲存過程
- /dev/bpf裝置缺失導致RAC安裝時HAIP啟動失敗devAI
- iOS裝置識別符號及許可權合集iOS符號
- Juniper裝置系統安全許可權管理評估
- 11g啟動sqlplus報錯沒有許可權SQL
- 11gR2私有ip修改錯誤導致crs無法啟動解決方法
- 檢視、儲存過程以及許可權控制練習儲存過程
- 儲存過程,角色相關的呼叫者許可權和定義者許可權問題儲存過程
- 超過40個Windows裝置驅動程式包含提升許可權的漏洞Windows
- iOS使用者注意了：日期設定錯誤可導致裝置“變磚”iOS
- EXECUTE IMMEDIATE 儲存過程中 許可權不足及EXECUTE IMMEDIATE的除錯避坑儲存過程除錯
- 【MySql】許可權不足導致的無法連線到資料庫以及許可權的授予和撤銷MySql資料庫