基於oracle 10.2.0.1 rac學習lmon程式系列六之增補一

wisdomone1發表於2015-11-17

背景

   本文是文章:基於oracle 10.2.0.1 rac學習lmon程式系列六 :http://blog.itpub.net/9240380/viewspace-1839875/

的 延續,我們繼續深入理解lmon程式的一些原理及含義,好我們進入正題。




結論

1,如果rac中某個節點的lmon出現故障,並且其它節點關閉例項,會產生IPT TIMEPUT
  而且如果IPC TIMEOUT一直不解決,準確來說,就是某個節點的LMON不恢復正常,某個節點的任何操作會HANG住
2,某個節點lmon出現故障時,剛開始某個節點的告警日誌及LMON TRC檔案不會出現任何問題
  但與此同時例項關閉哪個RAC節點的告警日誌及LMON TRC會有相應資訊  
  
  ---告警日誌
  Tue Nov 17 03:54:45 2015
  SHUTDOWN: waiting for detached processes to terminate.


  ----LMON TRC
  *** 2015-11-17 03:49:40.116
Begin DRM(13)
*** 2015-11-17 03:49:41.148
Control file closed, terminating IMR --剛開始發現的資訊,為何控制檔案關閉,終止IMR,這個IMR是什麼呢?


  而且關閉RAC節點的SHUTDOWN IMMEDIATE
  會在DISMOUNT這兒停一會兒


  過一會兒,在出現LMON故障的節點及關閉例項的節點的告警日誌及LMON TRC會出現ipc timeout以及oradebug ipc dump,
  
  Tue Nov 17 03:54:34 2015
IPC Send timeout detected. Receiver ospid 17921
Tue Nov 17 03:54:35 2015
Errors in file /u01/app/oracle/admin/jingfa/bdump/jingfa1_lmon_17921.trc:
Tue Nov 17 03:54:37 2015
Trace dumping is performing id=[cdmp_20151117035457]


  續上,並且在關閉RAC例項的節點之LMON TRC會記錄IPC 通訊相關的詳細日誌
KSTDUMP: In-memory trace dump
TIME:SEQ#        ORAPID   SID EVENT  OP DATA
========================================================================
B2208504:001AEDB1     5   167 10005   2 KSL WAIT END [rdbms ipc message] 100/0x64 0/0x0 0/0x0 time=976354 ---從這些資訊可以看出是RAC 2個節點的IPC通訊出現問題,rdbms ipc message,cgs wait for ipc msg
B2208532:001AEDB2     5   167 10005   1 KSL WAIT BEG [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0
B220853A:001AEDB3     5   167 10005   2 KSL WAIT END [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0 time=8 --並且這裡把程式號,會話號,操作型別相關重要資訊也列在這兒
B220853D:001AEDB4     5   167 10005   1 KSL WAIT BEG [rdbms ipc message] 100/0x64 0/0x0 0/0x0
B22F6DD3:001AEE0F     5   167 10005   2 KSL WAIT END [rdbms ipc message] 100/0x64 0/0x0 0/0x0 time=977044
B22F6DF6:001AEE10     5   167 10005   1 KSL WAIT BEG [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0
B22F6DFC:001AEE11     5   167 10005   2 KSL WAIT END [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0 time=5
B22F6DFE:001AEE12     5   167 10005   1 KSL WAIT BEG [rdbms ipc message] 100/0x64 0/0x0 0/0x0
B23E5C18:001AEE71     5   167 10005   2 KSL WAIT END [rdbms ipc message] 100/0x64 0/0x0 0/0x0 time=978455
B23E5C42:001AEE72     5   167 10005   1 KSL WAIT BEG [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0
B23E5C49:001AEE73     5   167 10005   2 KSL WAIT END [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0 time=8
B23E5C4C:001AEE74     5   167 10005   1 KSL WAIT BEG [rdbms ipc message] 100/0x64 0/0x0 0/0x0




  此時如果在出現故障的LMON節點進行查詢,會HANG住,我估計任何的操作也會HANG住,因為資料庫正常不正常,沒有完成重配操作




  然後我在出現故障的節點恢復LMON
  從告警日誌可見開始重配動作以及例項恢復
  Tue Nov 17 04:10:40 2015
Reconfiguration started (old inc 24, new inc 26)
List of nodes:
 0
 Global Resource Directory frozen
 * dead instance detected - domain 0 invalid = TRUE 
 Communication channels reestablished
 Master broadcasted resource hash value bitmaps
 Non-local Process blocks cleaned out
Tue Nov 17 04:10:40 2015
 LMS 0: 0 GCS shadows cancelled, 0 closed
 Set master node info 
 Submitted all remote-enqueue requests
 Dwn-cvts replayed, VALBLKs dubious
 All grantable enqueues granted
 Post SMON to start 1st pass IR
Tue Nov 17 04:10:40 2015
 LMS 0: 3470 GCS shadows traversed, 0 replayed
Tue Nov 17 04:10:40 2015
 Submitted all GCS remote-cache requests
 Post SMON to start 1st pass IR
 Fix write in gcs resources
Reconfiguration complete


Tue Nov 17 04:10:40 2015  
Instance recovery: looking for dead threads  ---可見重配完成後會進行例項恢復,即恢復NODE2節點2的相關中斷事務或資源等資訊
Instance recovery: lock domain invalid but no dead threads


3,只要資料庫不重啟,LMON程式的相關操作會記錄同一個LMON TRC檔案中,並且每次新的操作會產生一條日期相關的記錄,也就是說分析時,只要分析與日期區配的相關資料即可
這是極為重要思路之一
[oracle@jingfa1 bdump]$ more jingfa1_lmon_17921.trc|grep "*** 2015-11-17 03"
*** 2015-11-17 03:48:57.367
*** 2015-11-17 03:54:35.697
*** 2015-11-17 03:54:35.697








測試


---oracle version
SQL> select * from v$version where rownum=1;


BANNER
----------------------------------------------------------------
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bi




---node1,hang lmon
oradebug suspend lmon


--同時在node2關閉節點的資料庫例項,同時監控2個節點的告警日誌及LMON的TRC檔案


剛開始發現節點2關閉到database dismounted就停掉了一會兒,此時節點1的告警及LMON的TRC檔案無任何資訊
SQL> shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.


但是節點2有資訊
---node2告警日誌
發現出現IPC TIMEOUT的資訊
Tue Nov 17 03:54:45 2015
SHUTDOWN: waiting for detached processes to terminate.
Tue Nov 17 03:54:54 2015
IPC Send timeout detected.Sender: ospid 25241
Receiver: inst 1 binc 432043878 ospid 17921
Communications reconfiguration: instance_number 1
Tue Nov 17 03:54:57 2015
Trace dumping is performing id=[cdmp_20151117035457]
Tue Nov 17 03:54:58 2015
freeing rdom 0


---node2 lmon trc
*** 2015-11-17 03:49:40.116
Begin DRM(13)
*** 2015-11-17 03:49:41.148
Control file closed, terminating IMR --剛開始發現的資訊,為何控制檔案關閉,終止IMR,這個IMR是什麼呢?




---過了一會兒即告警日誌出現IPC TIMEOUT對應出現的如下資訊,我看和ORADEBUG IPC DUMP類似
*** 2015-11-17 03:54:54.470
*** 2015-11-17 03:54:54.470
IPC Send timeout detected.Sender: ospid 25241
Receiver: inst 1 binc 432043878 ospid 17921
SKGXPCTX: 0x0xf1d5618 ctx
WAIT HISTORY
Time(msec)       Wait Type       Return Code
----------       ---------       ------------
1004             NORMAL          TIMEDOUT
0                NORMAL          TIMEDOUT
974              NORMAL          TIMEDOUT
27               NORMAL          TIMEDOUT
0                NORMAL          TIMEDOUT
1001             NORMAL          TIMEDOUT
0                NORMAL          TIMEDOUT
1002             NORMAL          TIMEDOUT
0                NORMAL          TIMEDOUT
1002             NORMAL          TIMEDOUT
0                NORMAL          TIMEDOUT
1004             NORMAL          TIMEDOUT
0                NORMAL          TIMEDOUT
1029             NORMAL          TIMEDOUT
0                NORMAL          TIMEDOUT
1004             NORMAL          TIMEDOUT
wait delta 0 sec (0 msec) ctx ts 0x56340f last ts 0x563413
user cpu time since last wait 0 sec 0 ticks
system cpu time since last wait 0 sec 0 ticks
locked 1
blocked 2
timed wait receives 0
admno 0x63c82721 admport:
SSKGXPT 0xf1d5af4 flags         info for network 0
        socket no 9     IP 10.10.10.9   UDP 54219
        sflags SSKGXPT_WRITESSKGXPT_UP
        info for network 1
        socket no 0     IP 0.0.0.0      UDP 0
        sflags SSKGXPT_DOWN
        active 0        actcnt 1 
context timestamp 0x56340f
buffers queued on port 0x2af02751b148
    sconno     accono   ertt  state   seq#   sent  async   sync rtrans   acks
0x23c75d22 0x2da31a1b     32      3 47210280550785    390    397      0   2418 47210280517960
slot 0 rqh=0x2af027a16100
seq=33152 len=472 accno=0x2da31a1b start TS=0x515deb rt TS=0x563767 X CNT=303
slot 1 rqh=0x2af027a16d00
seq=33145 len=472 accno=0x2da31a1b start TS=0x515a5a rt TS=0x5637e7 X CNT=304
slot 2 rqh=0x2af027a14900
seq=33146 len=472 accno=0x2da31a1b start TS=0x515a5a rt TS=0x5637e7 X CNT=304
slot 3 rqh=0x2af027a14500
seq=33147 len=472 accno=0x2da31a1b start TS=0x515b89 rt TS=0x563527 X CNT=303
slot 4 rqh=0x2af027a10520
seq=33148 len=472 accno=0x2da31a1b start TS=0x515b8a rt TS=0x563527 X CNT=303
slot 5 rqh=0x2af027a16500
seq=33149 len=472 accno=0x2da31a1b start TS=0x515cbd rt TS=0x563667 X CNT=303
slot 6 rqh=0x2af027a14100
seq=33150 len=472 accno=0x2da31a1b start TS=0x515cbd rt TS=0x563667 X CNT=303
slot 7 rqh=0x2af027a18500
seq=33151 len=472 accno=0x2da31a1b start TS=0x515deb rt TS=0x563767 X CNT=303
0x23c75d23 0x24cf7936     64      3 47210280550395      0      0      0      0 47210280517632
0x23c75d24 0x1bfbd842     64      3 47210280550395      0      0      0      0 47210280517632
       ach     accono     sconno      admno  state   seq#    rcv rtrans   acks
0x0f259e20 0x4daa7926 0x524cf969 0x3f03d9f5 47210280517672  33148    385      0    321 
Submitting synchronized dump request [268435460]
kjxgfipccb: msg 0x0x2af027af16c8, mbo 0x0x2af027af16c0, type 22, ack 0, ref 0, stat 32
------ Dumping SKGXP context ------
SKGXPCTX: 0x0xf1d5618 ctx
WAIT HISTORY
Time(msec)       Wait Type       Return Code
----------       ---------       ------------
1004             NORMAL          TIMEDOUT
0                NORMAL          TIMEDOUT
974              NORMAL          TIMEDOUT
27               NORMAL          TIMEDOUT
0                NORMAL          TIMEDOUT
1001             NORMAL          TIMEDOUT
0                NORMAL          TIMEDOUT
1002             NORMAL          TIMEDOUT
0                NORMAL          TIMEDOUT
1002             NORMAL          TIMEDOUT
0                NORMAL          TIMEDOUT
1004             NORMAL          TIMEDOUT
0                NORMAL          TIMEDOUT
1029             NORMAL          TIMEDOUT
0                NORMAL          TIMEDOUT
1004             NORMAL          TIMEDOUT
wait delta 2 sec (2978 msec) ctx ts 0x56340f last ts 0x563413
user cpu time since last wait 0 sec 0 ticks
system cpu time since last wait 0 sec 0 ticks
locked 1
blocked 2
timed wait receives 0
admno 0x63c82721 admport:
SSKGXPT 0xf1d5af4 flags         info for network 0
        socket no 9     IP 10.10.10.9   UDP 54219
        sflags SSKGXPT_WRITESSKGXPT_UP
        info for network 1
        socket no 0     IP 0.0.0.0      UDP 0
        sflags SSKGXPT_DOWN
        active 0        actcnt 1 
context timestamp 0x56340f
buffers queued on port 0x2af02751b148
    sconno     accono   ertt  state   seq#   sent  async   sync rtrans   acks
0x23c75d23 0x24cf7936     64      3 47210280550395      0      0      0      0 47210280517632
0x23c75d24 0x1bfbd842     64      3 47210280550395      0      0      0      0 47210280517632
       ach     accono     sconno      admno  state   seq#    rcv rtrans   acks
0x0f259e20 0x4daa7926 0x524cf969 0x3f03d9f5 47210280517672  33148    385      0    321 
------ Dumping KST traces ------
*** 2015-11-17 03:54:57.448
KSTDUMP: In-memory trace dump
TIME:SEQ#        ORAPID   SID EVENT  OP DATA
========================================================================
B2208504:001AEDB1     5   167 10005   2 KSL WAIT END [rdbms ipc message] 100/0x64 0/0x0 0/0x0 time=976354 ---從這些資訊可以看出是RAC 2個節點的IPC通訊出現問題,rdbms ipc message,cgs wait for ipc msg
B2208532:001AEDB2     5   167 10005   1 KSL WAIT BEG [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0
B220853A:001AEDB3     5   167 10005   2 KSL WAIT END [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0 time=8 --並且這裡把程式號,會話號,操作型別相關重要資訊也列在這兒
B220853D:001AEDB4     5   167 10005   1 KSL WAIT BEG [rdbms ipc message] 100/0x64 0/0x0 0/0x0
B22F6DD3:001AEE0F     5   167 10005   2 KSL WAIT END [rdbms ipc message] 100/0x64 0/0x0 0/0x0 time=977044
B22F6DF6:001AEE10     5   167 10005   1 KSL WAIT BEG [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0
B22F6DFC:001AEE11     5   167 10005   2 KSL WAIT END [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0 time=5
B22F6DFE:001AEE12     5   167 10005   1 KSL WAIT BEG [rdbms ipc message] 100/0x64 0/0x0 0/0x0
B23E5C18:001AEE71     5   167 10005   2 KSL WAIT END [rdbms ipc message] 100/0x64 0/0x0 0/0x0 time=978455
B23E5C42:001AEE72     5   167 10005   1 KSL WAIT BEG [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0
B23E5C49:001AEE73     5   167 10005   2 KSL WAIT END [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0 time=8
B23E5C4C:001AEE74     5   167 10005   1 KSL WAIT BEG [rdbms ipc message] 100/0x64 0/0x0 0/0x0
B24D49D0:001AEED1     5   167 10005   2 KSL WAIT END [rdbms ipc message] 100/0x64 0/0x0 0/0x0 time=978257
B24D4AE2:001AEED2     5   167 10005   1 KSL WAIT BEG [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0
B24D4AEF:001AEED3     5   167 10005   2 KSL WAIT END [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0 time=14
B24D4AF3:001AEED4     5   167 10005   1 KSL WAIT BEG [rdbms ipc message] 100/0x64 0/0x0 0/0x0
B25C3D83:001AEF33     5   167 10005   2 KSL WAIT END [rdbms ipc message] 100/0x64 0/0x0 0/0x0 time=979595
B25C3DCA:001AEF34     5   167 10005   1 KSL WAIT BEG [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0
B25C3DDE:001AEF35     5   167 10005   2 KSL WAIT END [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0 time=22
B25C3DE3:001AEF36     5   167 10005   1 KSL WAIT BEG [rdbms ipc message] 100/0x64 0/0x0 0/0x0
B26B3470:001AEF95     5   167 10005   2 KSL WAIT END [rdbms ipc message] 100/0x64 0/0x0 0/0x0 time=980587
B26B352D:001AEF96     5   167 10005   1 KSL WAIT BEG [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0
B26B35A5:001AEF97     5   167 10005   2 KSL WAIT END [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0 time=74
B26B35AB:001AEF98     5   167 10005   1 KSL WAIT BEG [rdbms ipc message] 100/0x64 0/0x0 0/0x0
B27A1DF3:001AEFF5     5   167 10005   2 KSL WAIT END [rdbms ipc message] 100/0x64 0/0x0 0/0x0 time=976916
B27A1F12:001AEFF6     5   167 10005   1 KSL WAIT BEG [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0
B27A1F58:001AEFF7     5   167 10005   2 KSL WAIT END [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0 time=94
B27A1F5F:001AEFF8     5   167 10005   1 KSL WAIT BEG [rdbms ipc message] 100/0x64 0/0x0 0/0x0
B28910B9:001AF059     5   167 10005   2 KSL WAIT END [rdbms ipc message] 100/0x64 0/0x0 0/0x0 time=979286
B289111D:001AF05A     5   167 10005   1 KSL WAIT BEG [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0
B2891124:001AF05B     5   167 10005   2 KSL WAIT END [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0 time=8
B2891127:001AF05C     5   167 10005   1 KSL WAIT BEG [rdbms ipc message] 100/0x64 0/0x0 0/0x0
B2980FEE:001AF0BB     5   167 10005   2 KSL WAIT END [rdbms ipc message] 100/0x64 0/0x0 0/0x0 time=982666
B29810C3:001AF0BC     5   167 10005   1 KSL WAIT BEG [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0
B29810CC:001AF0BD     5   167 10005   2 KSL WAIT END [CGS wait for IPC msg] 0/0x0 0/0x0 0/0x0 time=10
B29810D0:001AF0BE     5   167 10005   1 KSL WAIT BEG [rdbms ipc message] 100/0x64 0/0x0 0/0x0
B2A705C0:001AF11D     5   167 10005   2 KSL WAIT END [rdbms ipc message] 100/0x64 0/0x0 0/0x0 time=980205




--過了一會兒,發現在節點1也出現資訊
--node 1告警日誌
出現ipc timeout,而且ipc timeout也會出現多次,不過每次相關的程式不同
Tue Nov 17 03:54:34 2015
IPC Send timeout detected. Receiver ospid 17921
Tue Nov 17 03:54:35 2015
Errors in file /u01/app/oracle/admin/jingfa/bdump/jingfa1_lmon_17921.trc:
Tue Nov 17 03:54:37 2015
Trace dumping is performing id=[cdmp_20151117035457]
Tue Nov 17 04:00:15 2015
IPC Send timeout detected.Sender: ospid 17925
Receiver: inst 2 binc 432109341 ospid 25245
Tue Nov 17 04:00:15 2015
IPC Send timeout to 1.1 inc 24 for msg type 73 from opid 7
Tue Nov 17 04:04:18 2015
IPC Send timeout detected.Sender: ospid 931
Receiver: inst 2 binc 432109335 ospid 25243
Tue Nov 17 04:04:18 2015
IPC Send timeout to 1.0 inc 24 for msg type 12 from opid 18




---node1 lmon trc,這種方法可以分析lmon trc,也就是lmon trc會一直記錄資料庫LMON程式的相關操作活動,每次產生新的活動,會對應一個新的操作日期,可以基於這個日期進行分析,
而不要無針對性和目的性進行分析,這是極為重要的
[oracle@jingfa1 bdump]$ more jingfa1_lmon_17921.trc|grep "*** 2015-11-17 03"
*** 2015-11-17 03:48:57.367
*** 2015-11-17 03:54:35.697
*** 2015-11-17 03:54:35.697


---node1的lmon trc
*** 2015-11-17 03:54:35.697
Received ORADEBUG command 'dump errorstack 1' from process Unix process pid: 17917, image:
*** 2015-11-17 03:54:35.697
ksedmp: internal or fatal error
----- Call Stack Trace -----
calling              call     entry                argument values in hex
location             type     point                (? means dubious value)
-------------------- -------- -------------------- ----------------------------
內容略


WAIT HISTORY
Time(msec)       Wait Type       Return Code
----------       ---------       ------------
0                NORMAL          TIMEDOUT
101              NORMAL          TIMEDOUT
0                NORMAL          TIMEDOUT
100              NORMAL          TIMEDOUT
0                NORMAL          TIMEDOUT
100              NORMAL          TIMEDOUT
0                NORMAL          TIMEDOUT
101              NORMAL          TIMEDOUT
0                NORMAL          TIMEDOUT
100              NORMAL          TIMEDOUT
0                NORMAL          TIMEDOUT
99               NORMAL          TIMEDOUT
2                NORMAL          TIMEDOUT
0                NORMAL          TIMEDOUT
100              NORMAL          TIMEDOUT
0                NORMAL          TIMEDOUT
wait delta 339 sec (339493 msec) ctx ts 0x5add78 last ts 0x5add78
user cpu time since last wait 0 sec 0 ticks
system cpu time since last wait 0 sec 0 ticks
locked 1
blocked 4
timed wait receives 0
admno 0x3f03d9f5 admport:




timed wait receives 0
admno 0x3f03d9f5 admport:
SSKGXPT 0xcf74af4 flags         info for network 0
        socket no 9     IP 10.10.10.8   UDP 44498
        sflags SSKGXPT_UP
        info for network 1
        socket no 0     IP 0.0.0.0      UDP 0
        sflags SSKGXPT_DOWN
        active 0        actcnt 1
context timestamp 0x5add78
buffers queued on port 0x2b13927fb148
    sconno     accono   ertt  state   seq#   sent  async   sync rtrans   acks
0x524cf969 0x4daa7926     32      3 47360604406140    385    385      0      0 47360604373313
0x524cf96a 0x44d6d837     64      3 47360604405755      0      0      0      0 47360604372992
0x524cf96b 0x3c033748     64      3 47360604405755      0      0      0      0 47360604372992
       ach     accono     sconno      admno  state   seq#    rcv rtrans   acks
0x0cff90c0 0x2da31a1b 0x23c75d22 0x63c82721 47360604373032  33145    485      0    394




而且其後在節點1執行查詢也會HANG住,即lmon程式出現故障,可能會導致資料庫DML及查詢HANG住
SQL> select * from v$version where rownum=1;


我馬上恢復節點1的lmon,節點1馬上進行重配操作,可見LMON程式極為重要,可見如果LMON出現故障,當前例項無法獲取到另一個故障例項的相關資源配置情況,RAC無法完成重配操作,資料庫就會處於不一致的情況
進而引發RAC HANG或其它嚴重的問題
Tue Nov 17 04:10:40 2015
Reconfiguration started (old inc 24, new inc 26)
List of nodes:
 0
 Global Resource Directory frozen
 * dead instance detected - domain 0 invalid = TRUE 
 Communication channels reestablished
 Master broadcasted resource hash value bitmaps
 Non-local Process blocks cleaned out
Tue Nov 17 04:10:40 2015
 LMS 0: 0 GCS shadows cancelled, 0 closed
 Set master node info 
 Submitted all remote-enqueue requests
 Dwn-cvts replayed, VALBLKs dubious
 All grantable enqueues granted
 Post SMON to start 1st pass IR
Tue Nov 17 04:10:40 2015
 LMS 0: 3470 GCS shadows traversed, 0 replayed
Tue Nov 17 04:10:40 2015
 Submitted all GCS remote-cache requests
 Post SMON to start 1st pass IR
 Fix write in gcs resources
Reconfiguration complete


Tue Nov 17 04:10:40 2015  
Instance recovery: looking for dead threads  ---可見重配完成後會進行例項恢復,即恢復NODE2節點2的相關中斷事務或資源等資訊
Instance recovery: lock domain invalid but no dead threads


---節點1的SQL查詢也恢復正常
SQL> select * from v$version where rownum=1;


BANNER
----------------------------------------------------------------
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bi


來自 “ ITPUB部落格 ” ,連結:http://blog.itpub.net/9240380/viewspace-1839890/,如需轉載,請註明出處,否則將追究法律責任。

相關文章