RAC 例項不能啟動 ORA-1589 signalled during ALTER DATABASE OPEN

今天啟動RAC 做實驗，發現RAC 例項怎麼啟動不了。

[@more@]

[oracle@rac2 bin]$ crs_stat -t

Name Type Target State Host

------------------------------------------------------------

ora.orcl.db application ONLINE OFFLINE

ora....oltp.cs application ONLINE OFFLINE

ora....cl1.srv application ONLINE OFFLINE

ora....cl2.srv application ONLINE OFFLINE

ora....l1.inst application ONLINE OFFLINE

ora....l2.inst application ONLINE OFFLINE

ora....SM1.asm application ONLINE ONLINE rac1

ora....C1.lsnr application ONLINE ONLINE rac1

ora.rac1.gsd application ONLINE ONLINE rac1

ora.rac1.ons application ONLINE ONLINE rac1

ora.rac1.vip application ONLINE ONLINE rac1

ora....SM2.asm application ONLINE ONLINE rac2

ora....C2.lsnr application ONLINE ONLINE rac2

ora.rac2.gsd application ONLINE ONLINE rac2

ora.rac2.ons application ONLINE ONLINE rac2

ora.rac2.vip application ONLINE ONLINE rac2

看了一下ASM 的磁碟組，都掛載成功。

[oracle@rac2 bin]$ export ORACLE_SID=+ASM2

[oracle@rac2 bin]$ sqlplus / as sysdba;

SQL*Plus: Release 10.2.0.1.0 - Production on Tue Sep 28 04:33:24 2010

Copyright (c) 1982, 2005, Oracle. All rights reserved.

Connected to:

Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production

With the Partitioning, Real Application Clusters, OLAP and Data Mining options

SQL> select name,state from v$asm_diskgroup;

NAME STATE

------------------------------ -----------

DATA MOUNTED

FLASH_RECOVERY_AREA MOUNTED

SQL>

之前遇到過這種情況情況，所以第一時間就檢視了這個ASM 磁碟組的資訊。

ASM 沒有問題，只能檢視資料庫的alert log了。

[oracle@rac1 bdump]$ pwd

/u01/app/oracle/admin/orcl/bdump

[oracle@rac1 bdump]$ ls

alert_orcl1.log orcl1_diag_22153.trc orcl1_lms0_22169.trc

cdmp_20100917173112 orcl1_lgwr_22186.trc orcl1_smon_22190.trc

orcl1_arc0_24180.trc orcl1_lmd0_22167.trc

orcl1_arc1_24182.trc orcl1_lmon_22165.trc

[oracle@rac1 bdump]$ tail -100 alert_orcl1.log

Tue Sep 28 04:05:20 2010

starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...

MMON started with pid=15, OS id=5650

Tue Sep 28 04:05:20 2010

starting up 1 shared server(s) ...

Tue Sep 28 04:05:20 2010

lmon registered with NM - instance id 1 (internal mem no 0)

Tue Sep 28 04:05:21 2010

Reconfiguration started (old inc 0, new inc 2)

List of nodes:

0

Global Resource Directory frozen

* allocate domain 0, invalid = TRUE

Communication channels reestablished

Master broadcasted resource hash value bitmaps

Non-local Process blocks cleaned out

Tue Sep 28 04:05:21 2010

LMS 0: 0 GCS shadows cancelled, 0 closed

Set master node info

Submitted all remote-enqueue requests

Dwn-cvts replayed, VALBLKs dubious

All grantable enqueues granted

Post SMON to start 1st pass IR

Tue Sep 28 04:05:21 2010

LMS 0: 0 GCS shadows traversed, 0 replayed

Tue Sep 28 04:05:21 2010

Submitted all GCS remote-cache requests

Post SMON to start 1st pass IR

Fix write in gcs resources

Reconfiguration complete

LCK0 started with pid=19, OS id=5683

Tue Sep 28 04:05:22 2010

ALTER DATABASE MOUNT

Tue Sep 28 04:05:22 2010

This instance was first to mount

Tue Sep 28 04:05:22 2010

Starting background process ASMB

ASMB started with pid=21, OS id=5706

Starting background process RBAL

RBAL started with pid=22, OS id=5710

Loaded ASM Library - Generic Linux, version 2.0.4 (KABI_V2) library for asmlib interface

Tue Sep 28 04:05:26 2010

SUCCESS: diskgroup DATA was mounted

SUCCESS: diskgroup FLASH_RECOVERY_AREA was mounted

Tue Sep 28 04:05:34 2010

Setting recovery target incarnation to 3

Tue Sep 28 04:05:35 2010

Successful mount of redo thread 1, with mount id 1258660482

Tue Sep 28 04:05:35 2010

Database mounted in Shared Mode (CLUSTER_DATABASE=TRUE)

Completed: ALTER DATABASE MOUNT

Tue Sep 28 04:05:35 2010

ALTER DATABASE OPEN

This instance was first to open

ORA-1589 signalled during: ALTER DATABASE OPEN...

Tue Sep 28 04:05:39 2010

Shutting down instance (abort)

License high water mark = 1

Instance terminated by USER, pid = 6080

從日誌裡我們發現了一個很重要的資訊：

ALTER DATABASE OPEN

This instance was first to open

ORA-1589 signalled during: ALTER DATABASE OPEN...

Tue Sep 28 04:05:39 2010

Shutting down instance (abort)

檢視了一下錯誤程式碼：

ORA-01589:
must use RESETLOGS or NORESETLOGS option for database open

Cause:
Either incomplete or backup control file recovery has been performed. After these types of recovery you must specify either the RESETLOGS option or the NORESETLOGS option to open your database.

Action:
Specify the appropriate option.

提示我們必須使用resetlogs 模式開啟資料。 開到這才明白，昨天做RMAN 恢復後直接把資料庫關了，沒有用resetlogs開啟。 杯具中..

手工啟動資料庫至mount 狀態，在用resetlogs 開啟：

[oracle@rac1 bdump]$ export ORACLE_SID=orcl1

[oracle@rac1 bdump]$ sqlplus /nolog

SQL*Plus: Release 10.2.0.1.0 - Production on Tue Sep 28 04:17:47 2010

Copyright (c) 1982, 2005, Oracle. All rights reserved.

SQL> conn / as sysdba;

Connected to an idle instance.

SQL> startup mount;

ORACLE instance started.

Total System Global Area 180355072 bytes

Fixed Size 1218388 bytes

Variable Size 109054124 bytes

Database Buffers 67108864 bytes

Redo Buffers 2973696 bytes

Database mounted.

SQL> alter database open resetlogs;

Database altered.

SQL>

在另一個節點，直接正常啟動就可以了。

[oracle@rac2 bin]$ export ORACLE_SID=orcl2

[oracle@rac2 bin]$ sqlplus /nolog

SQL*Plus: Release 10.2.0.1.0 - Production on Tue Sep 28 04:22:50 2010

Copyright (c) 1982, 2005, Oracle. All rights reserved.

SQL> conn / as sysdba

Connected to an idle instance.

SQL> startup mount;

ORACLE instance started.

Total System Global Area 180355072 bytes

Fixed Size 1218388 bytes

Variable Size 88082604 bytes

Database Buffers 88080384 bytes

Redo Buffers 2973696 bytes

Database mounted.

SQL> alter database open;

Database altered.

SQL>

檢視一下RAC 程式的狀態：

[oracle@rac2 bin]$ crs_stat -t

Name Type Target State Host

------------------------------------------------------------

ora.orcl.db application ONLINE ONLINE rac2

ora....oltp.cs application ONLINE ONLINE rac2

ora....cl1.srv application ONLINE ONLINE rac1

ora....cl2.srv application ONLINE ONLINE rac2

ora....l1.inst application ONLINE ONLINE rac1

ora....l2.inst application ONLINE ONLINE rac2

ora....SM1.asm application ONLINE ONLINE rac1

ora....C1.lsnr application ONLINE ONLINE rac1

ora.rac1.gsd application ONLINE ONLINE rac1

ora.rac1.ons application ONLINE ONLINE rac1

ora.rac1.vip application ONLINE ONLINE rac1

ora....SM2.asm application ONLINE ONLINE rac2

ora....C2.lsnr application ONLINE ONLINE rac2

ora.rac2.gsd application ONLINE ONLINE rac2

ora.rac2.ons application ONLINE ONLINE rac2

ora.rac2.vip application ONLINE ONLINE rac2

一切正常，從這個小錯誤中明白，遇到錯誤，要養成看alert log 的習慣。 不管是單例項還是RAC 例項，看log 永遠是第一條。 還有就是要心細，昨天偷了懶，為了補這個懶，花的時間就比昨天多了很多。 杯具啊.

本文來自CSDN部落格，轉載請標明出處：http://blog.csdn.net/tianlesoftware/archive/2010/09/28/5912227.aspx