HP-UX+11.2.0.3RAC因裸裝置許可權不一致導致RAC一個節點報錯的解決

還不算暈發表於2016-01-11
UX
故障情況：
HP-UX+11.2.0.3RAC；未使用ASM使用了裸裝置；--這裡就不討論11gR2使用裸裝置的坑了，歷史問題了。。
在儲存劃盤過來後，A節點的裸裝置許可權不正確，B節點許可權正確；
然後，將裸裝置做為資料檔案加入到表空間後，很快，遇到日誌切換後開始報錯(因為檢查點)；
此時，已經排查到磁碟許可權問題並修改為正確的許可權；但是A節點ALERT日誌中仍然在報錯。
報錯資訊：
Mon Jan 11 09:41:10 2016
Thread 1 advanced to log sequence 169313 (LGWR switch)
  Current log# 1 seq# 169313 mem# 0: /dev/vgpAAAdb11/rrredo_11_2048
Mon Jan 11 09:41:20 2016
Archived Log entry 240885 added for thread 1 sequence 169312 ID 0x360aa448 dest 1:
Mon Jan 11 09:42:08 2016
Errors in file /app/oracle/diag/rdbms/p1AAAdb/P1AAADB1/trace/P1AAADB1_m002_8386.trc
ORA-01157: cannot identify/lock data file 743 - see DBWR trace file
ORA-01110: data file 743: '/dev/vgpAAAdb17/rLV_AAA_L_D01'
Errors in file /app/oracle/diag/rdbms/p1AAAdb/P1AAADB1/trace/P1AAADB1_m002_8386.trc:
ORA-01157: cannot identify/lock data file 744 - see DBWR trace file
ORA-01110: data file 744: '/dev/vgpAAAdb18/rLV_AAA_M_D01'
Mon Jan 11 09:42:28 2016
Thread 1 advanced to log sequence 169314 (LGWR switch)
  Current log# 2 seq# 169314 mem# 0: /dev/vgpAAAdb11/rrredo_12_2048
Mon Jan 11 09:42:46 2016

解決方法：
此時需要在問題節點上將報錯的資料檔案OFFLINE，然後再對資料檔案進行RECOVER；然後重新ONLINE資料檔案即可；
未報錯的節點上可以不操作；
---------
解決過程：
SQL> select file_name,status from dba_data_files where file_id=743;
select file_name,status from dba_data_files where file_id=743
                             *
ERROR at line 1:
ORA-01157: cannot identify/lock data file 743 - see DBWR trace file
ORA-01110: data file 743: '/dev/vgpAAAdb17/rLV_AAA_L_D01'
----offline操作
SQL> alter database datafile '/dev/vgpAAAdb17/rLV_AAA_L_D01' offline;
Database altered.
SQL> alter database datafile '/dev/vgpAAAdb18/rLV_AAA_M_D01' offline;
Database altered.
----online操作
SQL> alter database datafile '/dev/vgpAAAdb18/rLV_AAA_M_D01' online;
alter database datafile '/dev/vgpAAAdb18/rLV_AAA_M_D01' online
*
ERROR at line 1:
ORA-01113: file 744 needs media recovery
ORA-01110: data file 744: '/dev/vgpAAAdb18/rLV_AAA_M_D01'
----recover資料檔案操作
SQL> recover datafile '/dev/vgpAAAdb18/rLV_AAA_M_D01';
Media recovery complete.
SQL> alter database datafile '/dev/vgpAAAdb18/rLV_AAA_M_D01' online;
Database altered.

SQL> alter database datafile  '/dev/vgpAAAdb17/rLV_AAA_L_D01' online;
alter database datafile  '/dev/vgpAAAdb17/rLV_AAA_L_D01' online
*
ERROR at line 1:
ORA-01113: file 743 needs media recovery
ORA-01110: data file 743: '/dev/vgpAAAdb17/rLV_AAA_L_D01'

SQL> recover datafile '/dev/vgpAAAdb17/rLV_AAA_L_D01';
Media recovery complete.
SQL> alter database datafile  '/dev/vgpAAAdb17/rLV_AAA_L_D01' online;
Database altered.
----進行檢查點及日誌切換操作，均可以正常進行。--稍後業務部門對錶等資料查詢驗證均正常。
SQL> alter system checkpoint;
System altered.
SQL> alter system switch logfile;
System altered.
SQL> select file_name,status from dba_data_files where file_id=743;
FILE_NAME
--------------------------------------------------------------------------------
STATUS
---------
/dev/vgpAAAdb17/rLV_AAA_L_D01
AVAILABLE
SQL> select file_name,status from dba_data_files where file_id=744;
FILE_NAME
--------------------------------------------------------------------------------
STATUS
---------
/dev/vgpAAAdb18/rLV_AAA_M_D01
AVAILABLE
------------------
對應的ALERT日誌：
Mon Jan 11 10:01:56 2016
Recovery of Online Redo Log: Thread 2 Group 15 Seq 71551 Reading mem 0
  Mem# 0: /dev/vgpAAAdb11/rrredo_27_2048
Mon Jan 11 10:02:08 2016
Recovery of Online Redo Log: Thread 1 Group 6 Seq 169318 Reading mem 0
  Mem# 0: /dev/vgpAAAdb11/rrredo_16_2048
Mon Jan 11 10:02:09 2016
Errors in file /app/oracle/diag/rdbms/p1AAAdb/P1AAADB1/trace/P1AAADB1_m005_11643.trc:
ORA-01135: file 743 accessed for DML/query is offline
ORA-01110: data file 743: '/dev/vgpAAAdb17/rLV_AAA_L_D01'
Errors in file /app/oracle/diag/rdbms/p1AAAdb/P1AAADB1/trace/P1AAADB1_m005_11643.trc:
ORA-01135: file 744 accessed for DML/query is offline
ORA-01110: data file 744: '/dev/vgpAAAdb18/rLV_AAA_M_D01'
Mon Jan 11 10:02:22 2016
Recovery of Online Redo Log: Thread 1 Group 7 Seq 169319 Reading mem 0
  Mem# 0: /dev/vgpAAAdb11/rrredo_17_2048
Media Recovery Complete (P1AAADB1)
Completed: ALTER DATABASE RECOVER  datafile '/dev/vgpAAAdb18/rLV_AAA_M_D01'  
Mon Jan 11 10:02:37 2016
alter database datafile '/dev/vgpAAAdb18/rLV_AAA_M_D01' online
Completed: alter database datafile '/dev/vgpAAAdb18/rLV_AAA_M_D01' online
Mon Jan 11 10:03:00 2016
alter database datafile  '/dev/vgpAAAdb17/rLV_AAA_L_D01' online
ORA-1113 signalled during: alter database datafile  '/dev/vgpAAAdb17/rLV_AAA_L_D01' online...
Mon Jan 11 10:03:14 2016
ALTER DATABASE RECOVER  datafile '/dev/vgpAAAdb17/rLV_AAA_L_D01'  
Media Recovery Start
Serial Media Recovery started
Recovery of Online Redo Log: Thread 1 Group 4 Seq 169316 Reading mem 0
  Mem# 0: /dev/vgpAAAdb11/rrredo_14_2048
Recovery of Online Redo Log: Thread 2 Group 14 Seq 71550 Reading mem 0
  Mem# 0: /dev/vgpAAAdb11/rrredo_26_2048
Recovery of Online Redo Log: Thread 1 Group 5 Seq 169317 Reading mem 0
  Mem# 0: /dev/vgpAAAdb11/rrredo_15_2048
Recovery of Online Redo Log: Thread 2 Group 15 Seq 71551 Reading mem 0
  Mem# 0: /dev/vgpAAAdb11/rrredo_27_2048
Mon Jan 11 10:03:32 2016
Recovery of Online Redo Log: Thread 1 Group 6 Seq 169318 Reading mem 0
  Mem# 0: /dev/vgpAAAdb11/rrredo_16_2048
Mon Jan 11 10:03:45 2016
Recovery of Online Redo Log: Thread 1 Group 7 Seq 169319 Reading mem 0
  Mem# 0: /dev/vgpAAAdb11/rrredo_17_2048
Media Recovery Complete (P1AAADB1)
Completed: ALTER DATABASE RECOVER  datafile '/dev/vgpAAAdb17/rLV_AAA_L_D01'  
Mon Jan 11 10:03:58 2016
alter database datafile  '/dev/vgpAAAdb17/rLV_AAA_L_D01' online
Completed: alter database datafile  '/dev/vgpAAAdb17/rLV_AAA_L_D01' online
Mon Jan 11 10:04:42 2016
Thread 1 advanced to log sequence 169320 (LGWR switch)
  Current log# 8 seq# 169320 mem# 0: /dev/vgpAAAdb11/rrredo_18_2048




相關文章