分析結論

1,oracle 11.2.0.4,redhat 6.5,gi單節點ASM架構下，CSSD程式可能會在OS REBOOT後無法自動重啟

2，原因在於其資源ORA.CSSD的指項 AUTO_START不為ALWAYS

3，透過如下命令修改上述功能
crsctl modify res ora.cssd -attr "AUTO_START=always"

4，檢視某資源的配置資訊
[grid@zxy ~]$ crsctl stat res ora.cssd -p

5，

分析明細

1，作業系統版本
[root@zxy ~]# more /etc/redhat-release
Red Hat Enterprise Linux Server release 6.5 (Santiago)

2，重啟作業系統後發現CSSD程式沒有自動重啟
[grid@zxy ~]$ ps -ef|grep d.bin
grid 1715 1 1 00:20 ? 00:00:01 /u01/app/11.2.0.4/grid/bin/ohasd.bin reboot
grid 2008 1 0 00:21 ? 00:00:00 /u01/app/11.2.0.4/grid/bin/oraagent.bin
grid 2023 1 0 00:21 ? 00:00:00 /u01/app/11.2.0.4/grid/bin/evmd.bin
grid 2057 2023 0 00:21 ? 00:00:00 /u01/app/11.2.0.4/grid/bin/evmlogger.bin -o /u01/app/11.2.0.4/grid/evm/log/evmlogger.info -l /u01/app/11.2.0.4/grid/evm/log/evmlogger.log
grid 2060 1 0 00:21 ? 00:00:00 /u01/app/11.2.0.4/grid/bin/tnslsnr LISTENER -inherit
grid 2082 1979 0 00:22 pts/0 00:00:00 grep d.bin

3，由於CSSD程式未啟動，ASM例項也無法啟動
[grid@zxy ~]$ sqlplus '/as sysasm'

SQL*Plus: Release 11.2.0.4.0 Production on Sat Jun 3 00:22:02 2017

Copyright (c) 1982, 2013, Oracle. All rights reserved.

Connected to an idle instance.

SQL> startup
ORA-01078: failure in processing system parameters
ORA-29701: unable to connect to Cluster Synchronization Service
SQL> exit

4，只能手工使用SRVCTL啟動ASM例項
[grid@zxy ~]$ srvctl start asm
[grid@zxy ~]$

5，此時CSSD程式已啟動
[grid@zxy ~]$ ps -ef|grep d.bin
grid 1715 1 0 00:20 ? 00:00:02 /u01/app/11.2.0.4/grid/bin/ohasd.bin reboot
grid 2008 1 0 00:21 ? 00:00:00 /u01/app/11.2.0.4/grid/bin/oraagent.bin
grid 2023 1 0 00:21 ? 00:00:00 /u01/app/11.2.0.4/grid/bin/evmd.bin
grid 2057 2023 0 00:21 ? 00:00:00 /u01/app/11.2.0.4/grid/bin/evmlogger.bin -o /u01/app/11.2.0.4/grid/evm/log/evmlogger.info -l /u01/app/11.2.0.4/grid/evm/log/evmlogger.log
grid 2060 1 0 00:21 ? 00:00:00 /u01/app/11.2.0.4/grid/bin/tnslsnr LISTENER -inherit
grid 2230 1 0 00:23 ? 00:00:00 /u01/app/11.2.0.4/grid/bin/cssdagent
grid 2254 1 0 00:23 ? 00:00:00 /u01/app/11.2.0.4/grid/bin/ocssd.bin
grid 2337 1979 0 00:24 pts/0 00:00:00 grep d.bin

6，準備透過CRSCTL命令分析ORA.CSSD資源的相關資訊
[grid@zxy ~]$ crsctl status resource -help
Usage:
crsctl status resource [<resName>[...]|-w <filter>] [<-p|-v> [-e]] | [[-f|-l|-g]] | [[-k <cid>|-n <server>] [-d <did>]] | [-s -k <cid> [-d <did>]]
Check status of designated resources

crsctl status resource [<resName>[...]|-w <filter>] -t
Print status of resources in tabular format

crsctl status resource [<resName>[...]] -dependency [-stop | -pullup]
Print resource dependencies
where
resName [...] One or more blank-separated resource names
-w Resource filter (e.g., "TYPE = ora.database.type")
-p Print static configuration
-v Print runtime configuration
-e Evaluate a resource instance's special values
-f Print full configuration
-l Print all cardinal and degree members
-g Check if resources are registered
-k Cardinality ID
-d Degree ID
-n Server name
-s Get target servers for relocation
-t Tabular display
-dependency Display resource dependencies, default is start dependencies
-stop Display resource stop dependencies
-pullup Display resource pullup dependencies

[grid@zxy ~]$

7，據上，輸出ORA.CSSD資源的詳細資訊，主要檢視AUTO_START選項（用於是否自動重啟）
[grid@zxy ~]$ crsctl stat res ora.cssd -p
NAME=ora.cssd
TYPE=ora.cssd.type
ACL=owner:grid:rwx,pgrp:oinstall:rwx,other::r--
ACTION_FAILURE_TEMPLATE=
ACTION_SCRIPT=
ACTIVE_PLACEMENT=0
AGENT_FILENAME=%CRS_HOME%/bin/cssdagent%CRS_EXE_SUFFIX%
AGENT_HB_INTERVAL=0
AGENT_HB_MISCOUNT=10
AUTO_START=never ---可見自動重啟屬性為NEVER
CARDINALITY=1
CHECK_ARGS=
CHECK_COMMAND=
CHECK_INTERVAL=30
CLEAN_ARGS=abort
CLEAN_COMMAND=
CSSD_MODE=
CSSD_PATH=%CRS_HOME%/bin/ocssd%CRS_EXE_SUFFIX%
CSS_USER=grid
DAEMON_LOGGING_LEVELS=CSSD=2,GIPCNM=2,GIPCGM=2,GIPCCM=2,CLSF=0,SKGFD=0,GPNP=1,OLR=0
DAEMON_TRACING_LEVELS=CSSD=0,GIPCNM=0,GIPCGM=0,GIPCCM=0,CLSF=0,SKGFD=0,GPNP=0,OLR=0
DEFAULT_TEMPLATE=
DEGREE=1
DESCRIPTION="Resource type for CSSD"
DETACHED=true
ENABLED=1
ENV_OPTS=
FAILOVER_DELAY=0
FAILURE_INTERVAL=3
FAILURE_THRESHOLD=5
HOSTING_MEMBERS=
INTERVAL_FS=3
LOAD=1
LOGGING_LEVEL=1
NOT_RESTARTING_TEMPLATE=
OFFLINE_CHECK_INTERVAL=0
OMON_INITRATE=1000
OMON_POLLRATE=500
ORA_OPROCD_MODE=
ORA_VERSION=11.2.0.4.0
PID_FILE=
PLACEMENT=balanced
PROCD_TIMEOUT=1000
PROCESS_TO_MONITOR=
PROFILE_CHANGE_TEMPLATE=
REBOOT_OPTS=
RESTART_ATTEMPTS=5
SCRIPT_TIMEOUT=600
SERVER_POOLS=
START_ARGS=
START_COMMAND=
START_DEPENDENCIES=weak(concurrent:ora.diskmon)
START_TIMEOUT=600
STATE_CHANGE_TEMPLATE=
STOP_ARGS=
STOP_COMMAND=
STOP_DEPENDENCIES=hard(shutdown:ora.diskmon)
STOP_TIMEOUT=900
TIMEOUT_FS=0
UNRESPONSIVE_TIMEOUT=180
UPTIME_THRESHOLD=1m
USR_ORA_ENV=
VMON_INITLIMIT=16
VMON_INITRATE=500
VMON_POLLRATE=500

[grid@zxy ~]$

8，順便學習下CRSCTL相關其它命令，輸出所有資源的狀態執行資訊
[grid@zxy bin]$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASM_OCRVOTE.dg
ONLINE ONLINE zxy
ora.DG_MOUNT.dg
OFFLINE OFFLINE zxy
ora.LISTENER.lsnr
ONLINE ONLINE zxy
ora.asm
ONLINE ONLINE zxy Started
ora.ons
OFFLINE OFFLINE zxy
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
1 ONLINE ONLINE zxy
ora.diskmon
1 OFFLINE OFFLINE
ora.evmd
1 ONLINE ONLINE zxy
ora.god.db
1 OFFLINE OFFLINE Instance Shutdown
[grid@zxy bin]$

9，顯示某個資源的啟動的依賴關係
[grid@zxy bin]$ crsctl stat res ora.cssd -dependency
================================================================================
Resource Start Dependencies
================================================================================
------------------------------------ora.cssd------------------------------------
ora.cssd(ora.cssd.type)->
| ora.diskmon(ora.diskmon.type)[weak:concurrent]
| | ora.cssd(ora.cssd.type)[weak:concurrent,pullup:always]
--------------------------------------------------------------------------------
[grid@zxy bin]$

10，顯示某個資源關閉的依賴關係
[grid@zxy bin]$ crsctl stat res ora.cssd -dependency -stop
================================================================================
Resource Stop Dependencies
================================================================================
------------------------------------ora.cssd------------------------------------
ora.cssd(ora.cssd.type)->
| ora.asm(ora.asm.type)[hard]
| | ora.ASM_OCRVOTE.dg(ora.diskgroup.type)[hard:intermediate]
| | ora.DG_MOUNT.dg(ora.diskgroup.type)[hard:intermediate]
| | | ora.god.db(ora.database.type)[hard:shutdown]
| | ora.god.db(ora.database.type)[hard:intermediate]
--------------------------------------------------------------------------------
[grid@zxy bin]$

11，修改上述ORA.CSSD資源的選項AUTO_START
[grid@zxy bin]$ crsctl -help
Usage: crsctl add - add a resource, type or other entity
crsctl check - check a service, resource or other entity
crsctl config - output autostart configuration
crsctl debug - obtain or modify debug state
crsctl delete - delete a resource, type or other entity
crsctl disable - disable autostart
crsctl discover - discover DHCP server
crsctl enable - enable autostart
crsctl get - get an entity value
crsctl getperm - get entity permissions
crsctl lsmodules - list debug modules
crsctl modify - modify a resource, type or other entity
crsctl query - query service state
crsctl pin - pin the nodes in the node list
crsctl relocate - relocate a resource, server or other entity
crsctl replace - replaces the location of voting files
crsctl release - release a DHCP lease
crsctl request - request a DHCP lease
crsctl setperm - set entity permissions
crsctl set - set an entity value
crsctl start - start a resource, server or other entity
crsctl status - get status of a resource or other entity
crsctl stop - stop a resource, server or other entity
crsctl unpin - unpin the nodes in the node list
crsctl unset - unset an entity value, restoring its default
[grid@zxy bin]$

[grid@zxy bin]$ crsctl modify -help
Usage:
crsctl modify {resource|type|serverpool} <name> <options>
where
name Name of the CRS entity
options Options to be passed to the modify command

See individual CRS entity help for more details

[grid@zxy bin]$ crsctl modify res -help
Usage:
crsctl modify resource <resName> -attr "<specification>[,...]" [-f] [-delete] [-i]
<specification>: {<attrName>=<value> | <attrName>@<scope>=<value>}
<scope>: {@SERVERNAME(<server>)[@DEGREEID(<did>)] |
@CARDINALITYID(<cid>)[@DEGREEID(<did>)] }
where
resName Modify named resource
attrName Attribute name
value Attribute value
server Server name
cid Resource cardinality ID
did Resource degree ID
-f Force option
-delete Delete named attribute
-i Fail if request cannot be processed immediately

[grid@zxy bin]$

由提示可知選項共計3個可選值，選擇ALWYAS
[grid@zxy bin]$ crsctl modify res ora.cssd -attr "AUTO_START=enable"
CRS-2671: Error processing attribute 'AUTO_START': Value 'enable' is not any of the following 'restore', 'always', 'never'
CRS-4000: Command Modify failed, or completed with errors.

[grid@zxy bin]$ crsctl modify res ora.cssd -attr "AUTO_START=always"
[grid@zxy bin]$

11，關閉GI程式
[root@zxy ~]# !348
/u01/app/11.2.0.4/grid/bin/crsctl stop has
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'zxy'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'zxy'
CRS-2673: Attempting to stop 'ora.ASM_OCRVOTE.dg' on 'zxy'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'zxy' succeeded
CRS-2677: Stop of 'ora.ASM_OCRVOTE.dg' on 'zxy' succeeded
CRS-2679: Attempting to clean 'ora.ASM_OCRVOTE.dg' on 'zxy'
CRS-2681: Clean of 'ora.ASM_OCRVOTE.dg' on 'zxy' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'zxy'
CRS-2677: Stop of 'ora.asm' on 'zxy' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'zxy'
CRS-2677: Stop of 'ora.cssd' on 'zxy' succeeded
CRS-2673: Attempting to stop 'ora.evmd' on 'zxy'
CRS-2677: Stop of 'ora.evmd' on 'zxy' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'zxy' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@zxy ~]#

[root@zxy ~]# ps -ef|grep d.bin
root 2638 1864 0 00:59 pts/0 00:00:00 grep d.bin
[root@zxy ~]# ps -ef|grep pmon
root 2640 1864 0 00:59 pts/0 00:00:00 grep pmon

12，重啟作業系統
[root@zxy ~]# reboot

Broadcast message from root@zxy
(/dev/pts/0) at 1:00 ...

The system is going down for reboot NOW!

13，重啟作業系統後CSSSD程式自動重啟
[root@zxy ~]# ps -ef|grep d.bin
grid 1660 1 1 01:00 ? 00:00:01 /u01/app/11.2.0.4/grid/bin/ohasd.bin reboot
grid 1901 1 0 01:02 ? 00:00:00 /u01/app/11.2.0.4/grid/bin/cssdagent
grid 1905 1 0 01:02 ? 00:00:00 /u01/app/11.2.0.4/grid/bin/oraagent.bin
grid 1935 1 0 01:02 ? 00:00:00 /u01/app/11.2.0.4/grid/bin/evmd.bin
grid 1937 1 0 01:02 ? 00:00:00 /u01/app/11.2.0.4/grid/bin/tnslsnr LISTENER -inherit
grid 1947 1 0 01:02 ? 00:00:00 /u01/app/11.2.0.4/grid/bin/ocssd.bin
grid 1987 1935 0 01:02 ? 00:00:00 /u01/app/11.2.0.4/grid/bin/evmlogger.bin -o /u01/app/11.2.0.4/grid/evm/log/evmlogger.info -l /u01/app/11.2.0.4/grid/evm/log/evmlogger.log
root 2043 1832 0 01:02 pts/0 00:00:00 grep d.bin
[root@zxy ~]#

[root@zxy ~]# ps -ef|grep css
grid 1901 1 0 01:02 ? 00:00:00 /u01/app/11.2.0.4/grid/bin/cssdagent
grid 1947 1 0 01:02 ? 00:00:00 /u01/app/11.2.0.4/grid/bin/ocssd.bin
root 2073 1832 0 01:03 pts/0 00:00:00 grep css

[root@zxy ~]# ps -ef|grep pmon
grid 2009 1 0 01:02 ? 00:00:00 asm_pmon_+ASM
root 2071 1832 0 01:02 pts/0 00:00:00 grep pmon

redhat 6.5 oracle 11.2.0.4 gi oracle asm在作業系統重啟cssd程式無法自動重啟簡析

相關文章