啟動ASM例項出現ORA-29701錯誤

啟動ASM例項：

p55a@/home/oracle> export ORACLE_SID=+ASM
p55a@/home/oracle> sqlplus / as sysdba

SQL*Plus: Release 10.2.0.5.0 - Production on Thu Jun 23 22:21:20 2011

Copyright (c) 1982, 2010, Oracle. All Rights Reserved.

Connected to an idle instance.

SQL> startup
ORA-29701: unable to connect to Cluster Manager
SQL> exit
Disconnected

檢查發現當主機存在多個ORACLE_HOME時，CLUSTER服務連線的ORACLE_HOME和啟動ASM例項的ORACLE_HOME不一致，導致了這個錯誤，metalink文件ID 459775.1描述了這個問題。

檢查當前環節的設定：

p55a@/home/oracle> env | grep HOME
ORACLE_CRS_HOME=/u01/crs
HOME=/home/oracle
ORACLE_HOME=/u01/db10g
p55a@/home/oracle> ps -ef|grep css
oracle 184536 299068 0 22:31:43 pts/1 0:00 grep css
root 237570 1 0 22:00:09 - 0:00 /bin/sh /etc/init.cssd run

這裡設定了ORACLE_CRS_HOME，但是當前資料庫並非是一個RAC環境，而是單機下啟動的ASM例項，顯然這個設定ORACLE_CRS_HOME存在問題。

p55a@/home/oracle> cd /u01/crs
ksh: /u01/crs: not found.

而ORACLE_CRS_HOME對應的目錄根本就不存在，正是這個設定導致了問題的產生。

解決問題並不複雜，首先註釋或刪除環境變數中ORACLE_CRS_HOME的設定：

p55a@/home/oracle> vi .profile
export ORACLE_BASE=/u01
#export ORACLE_CRS=$ORACLE_BASE/crs
#export ORACLE_CRS_HOME=$ORACLE_BASE/crs
export ORACLE_HOME=$ORACLE_BASE/db10g
export PATH=$ORACLE_HOME/bin:$ORACLE_CRS/bin:$PATH
export ORACLE_SID=cnddr
PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:$HOME/bin:/usr/bin/X11:/sbin:.:$ORACLE_HOME/bin:$ORACLE_CRS_HOME/bin:$PATH
export PATH

切換到root，kill當前的css程式：

p55a@/home/oracle> su -
root's Password:
# ps -ef|grep css
root 237570 1 0 22:00:09 - 0:00 /bin/sh /etc/init.cssd run
root 270456 328084 0 23:32:53 pts/1 0:00 grep css
# kill -9 237570

然後利用重置ORACLE_HOME：

# cd /u01/db10g/bin
# ./localconfig reset $ORACLE_HOME
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'system'..
Operation successful.
Configuration for local CSS has been initialized

Adding to inittab
Startup will be queued to init within 30 seconds.
Checking the status of new Oracle init process...
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
p55a
CSS is active on all nodes.
Oracle CSS service is installed and running under init(1M)

啟動css程式：

# /etc/init.cssd start
Startup will be queued to init within 30 seconds.

下面檢查資料庫的狀態：

# su - oracle
p55a@/home/oracle> ps -ef | grep ora
oracle 184370 332060 0 22:35:53 pts/1 0:00 -ksh
oracle 217308 1 0 23:29:35 - 0:00 ora_dbw1_cnddr
oracle 241672 1 0 23:40:43 - 0:00 /var/www/nagios/bin/nrpe -c /var/www/nagios/etc/nrpe.cfg -d
oracle 250100 1 0 23:29:34 - 0:02 ora_mman_cnddr
oracle 253992 1 0 23:29:35 - 0:00 ora_lgwr_cnddr
oracle 258138 1 0 23:29:35 - 0:00 ora_smon_cnddr
oracle 266296 1 0 23:29:35 - 0:00 ora_ckpt_cnddr
oracle 274462 1 0 23:29:35 - 0:00 ora_d000_cnddr
oracle 278572 1 0 23:29:35 - 0:00 ora_mmon_cnddr
oracle 282678 1 0 22:03:10 pts/0 0:00 /u01/db10g/bin/tnslsnr LISTENER -inherit
oracle 290942 1 0 23:29:35 - 0:00 ora_reco_cnddr
oracle 299022 352718 0 23:42:30 pts/1 0:00 grep ora
oracle 180614 1 0 23:29:35 - 0:00 ora_cjq0_cnddr
oracle 262420 287004 0 22:00:24 pts/0 0:00 -ksh
oracle 295422 1 0 23:29:34 - 0:00 ora_psp0_cnddr
oracle 303432 1 0 23:29:35 - 0:00 ora_dbw0_cnddr
oracle 307642 1 0 23:29:35 - 0:00 ora_s000_cnddr
oracle 311626 1 0 23:29:34 - 0:00 ora_pmon_cnddr
oracle 315648 299318 0 23:27:22 pts/0 0:00 -ksh
oracle 319844 352718 0 23:42:30 pts/1 0:00 ps -ef
oracle 323920 1 0 23:29:35 - 0:00 ora_mmnl_cnddr
oracle 352718 328084 0 23:42:14 pts/1 0:00 -ksh
p55a@/home/oracle> ps -ef|grep ASM
oracle 299026 352718 0 23:42:46 pts/1 0:00 grep ASM
p55a@/home/oracle> ps -ef|grep asm
oracle 299028 352718 0 23:42:53 pts/1 0:00 grep asm
p55a@/home/oracle> sqlplus / as sysdba

SQL*Plus: Release 10.2.0.5.0 - Production on Thu Jun 23 23:43:28 2011

Copyright (c) 1982, 2010, Oracle. All Rights Reserved.

Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> set pages 100 lines 120
SQL> select instance_number, instance_name, status from v$instance;

INSTANCE_NUMBER INSTANCE_NAME    STATUS
--------------- ---------------- ---------
              1 cnddr            STARTED

SQL> select * from v$database;
select * from v$database
*
ERROR at line 1:
ORA-01507: database not mounted

SQL> select * from v$asm_diskgroup;

no rows selected

SQL> select * from v$asm_disk;

no rows selected

SQL> exit
Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

雖然資料庫已經啟動，但是隻到了NOMOUNT狀態，ASM例項沒有啟動，因此資料庫無法MOUNT：

p55a@/home/oracle> env|grep SID
ORACLE_SID=cnddr
p55a@/home/oracle> export ORACLE_SID=+ASM
p55a@/home/oracle> sqlplus / as sysdba

SQL*Plus: Release 10.2.0.5.0 - Production on Thu Jun 23 23:45:06 2011

Copyright (c) 1982, 2010, Oracle. All Rights Reserved.

Connected to an idle instance.

SQL> startup
ASM instance started

Total System Global Area 130023424 bytes
Fixed Size 2094648 bytes
Variable Size 102762952 bytes
ASM Cache 25165824 bytes
ASM diskgroups mounted
SQL> exit
Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
p55a@/home/oracle> export ORACLE_SID=cnddr
p55a@/home/oracle> sqlplus / as sysdba

SQL*Plus: Release 10.2.0.5.0 - Production on Thu Jun 23 23:45:35 2011

Copyright (c) 1982, 2010, Oracle. All Rights Reserved.

Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> shutdown immediate
ORA-01507: database not mounted

ORACLE instance shut down.
SQL> startup
ORACLE instance started.

Total System Global Area 1.0737E+10 bytes
Fixed Size 2114208 bytes
Variable Size 3489664352 bytes
Database Buffers 7230980096 bytes
Redo Buffers 14659584 bytes
Database mounted.
Database opened.
SQL> alter database start logical standby apply;

Database altered.

手工啟動ASM例項，並啟動資料庫，問題解決。