RAC的VIP切換測試

Oracle 10g開始推出VIP，能夠幫助資料庫或者程式能夠快速地進行failover。
本文主要測試RAC如何進行VIP的failover。

[@more@]

1. 原來的資源狀況

[root@dwdb01 suk]# crs_stat
HA Resource Target State
----------- ------ -----
ora.dwdb01.ASM1.asm ONLINE ONLINE on dwdb01
ora.dwdb01.LISTENER_DWDB01.lsnr ONLINE ONLINE on dwdb01
ora.dwdb01.gsd ONLINE ONLINE on dwdb01
ora.dwdb01.ons ONLINE ONLINE on dwdb01
ora.dwdb01.vip ONLINE ONLINE on dwdb01
ora.dwdb02.ASM2.asm ONLINE ONLINE on dwdb02
ora.dwdb02.LISTENER_DWDB02.lsnr ONLINE ONLINE on dwdb02
ora.dwdb02.gsd ONLINE ONLINE on dwdb02
ora.dwdb02.ons ONLINE ONLINE on dwdb02
ora.dwdb02.vip ONLINE ONLINE on dwdb02
ora.dwdb03.ASM3.asm ONLINE ONLINE on dwdb03
ora.dwdb03.LISTENER_DWDB03.lsnr ONLINE ONLINE on dwdb03
ora.dwdb03.gsd ONLINE ONLINE on dwdb03
ora.dwdb03.ons ONLINE ONLINE on dwdb03
ora.dwdb03.vip ONLINE ONLINE on dwdb03
ora.dwdb04.ASM4.asm ONLINE ONLINE on dwdb04
ora.dwdb04.LISTENER_DWDB04.lsnr ONLINE ONLINE on dwdb04
ora.dwdb04.gsd ONLINE ONLINE on dwdb04
ora.dwdb04.ons ONLINE ONLINE on dwdb04
ora.dwdb04.vip ONLINE ONLINE on dwdb04
ora.dwrac.db ONLINE ONLINE on dwdb02
ora.dwrac.dw.cs ONLINE ONLINE on dwdb04
ora.dwrac.dw.dwrac1.srv ONLINE ONLINE on dwdb01
ora.dwrac.dw.dwrac2.srv ONLINE ONLINE on dwdb02
ora.dwrac.dw.dwrac3.srv ONLINE ONLINE on dwdb03
ora.dwrac.dw.dwrac4.srv ONLINE ONLINE on dwdb04
ora.dwrac.dwrac1.inst ONLINE ONLINE on dwdb01
ora.dwrac.dwrac2.inst ONLINE ONLINE on dwdb02
ora.dwrac.dwrac3.inst ONLINE ONLINE on dwdb03
ora.dwrac.dwrac4.inst ONLINE ONLINE on dwdb04
------------------------
7 STATE=ONLINE on dwdb01
8 STATE=ONLINE on dwdb02
7 STATE=ONLINE on dwdb03
8 STATE=ONLINE on dwdb04

可以看到，所有資源都是正常online的，且VIP分佈執行在各自的節點中。

2. 把其中一個節點(dwdb04)的crs關掉，模擬故障

關閉例項不會導致VIP切換，VIP屬於nodeapp資源，因此，需要關閉crs來模擬節點或者crs故障才能觀察VIP切換。

[root@dwdb04 ~]# /oracle/product/10.2.0/crs/bin/crsctl stop crs
Stopping resources. This could take several minutes.
Successfully stopped CRS resources.
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.

3. 觀察crs_stat的變化

1) 停止crs後立刻在其他節點檢視crs_stat：
[root@dwdb01 suk]# crs_stat
HA Resource Target State
----------- ------ -----
ora.dwdb01.ASM1.asm ONLINE ONLINE on dwdb01
ora.dwdb01.LISTENER_DWDB01.lsnr ONLINE ONLINE on dwdb01
ora.dwdb01.gsd ONLINE ONLINE on dwdb01
ora.dwdb01.ons ONLINE ONLINE on dwdb01
ora.dwdb01.vip ONLINE ONLINE on dwdb01
ora.dwdb02.ASM2.asm ONLINE ONLINE on dwdb02
ora.dwdb02.LISTENER_DWDB02.lsnr ONLINE ONLINE on dwdb02
ora.dwdb02.gsd ONLINE ONLINE on dwdb02
ora.dwdb02.ons ONLINE ONLINE on dwdb02
ora.dwdb02.vip ONLINE ONLINE on dwdb02
ora.dwdb03.ASM3.asm ONLINE ONLINE on dwdb03
ora.dwdb03.LISTENER_DWDB03.lsnr ONLINE ONLINE on dwdb03
ora.dwdb03.gsd ONLINE ONLINE on dwdb03
ora.dwdb03.ons ONLINE ONLINE on dwdb03
ora.dwdb03.vip ONLINE ONLINE on dwdb03
ora.dwdb04.ASM4.asm ONLINE OFFLINE
ora.dwdb04.LISTENER_DWDB04.lsnr ONLINE OFFLINE
ora.dwdb04.gsd ONLINE OFFLINE
ora.dwdb04.ons ONLINE OFFLINE
ora.dwdb04.vip ONLINE OFFLINE
ora.dwrac.db ONLINE ONLINE on dwdb02
ora.dwrac.dw.cs ONLINE OFFLINE
ora.dwrac.dw.dwrac1.srv ONLINE ONLINE on dwdb01
ora.dwrac.dw.dwrac2.srv ONLINE ONLINE on dwdb02
ora.dwrac.dw.dwrac3.srv ONLINE ONLINE on dwdb03
ora.dwrac.dw.dwrac4.srv ONLINE OFFLINE
ora.dwrac.dwrac1.inst ONLINE ONLINE on dwdb01
ora.dwrac.dwrac2.inst ONLINE ONLINE on dwdb02
ora.dwrac.dwrac3.inst ONLINE ONLINE on dwdb03
ora.dwrac.dwrac4.inst ONLINE OFFLINE
------------------------
8 STATE=OFFLINE
7 STATE=ONLINE on dwdb01
8 STATE=ONLINE on dwdb02
7 STATE=ONLINE on dwdb03

可以看到，dwdb04的VIP資源此時處於offline狀態。

2) 隔一會再檢視crs_stat
[root@dwdb01 suk]# crs_stat
HA Resource Target State
----------- ------ -----
ora.dwdb01.ASM1.asm ONLINE ONLINE on dwdb01
ora.dwdb01.LISTENER_DWDB01.lsnr ONLINE ONLINE on dwdb01
ora.dwdb01.gsd ONLINE ONLINE on dwdb01
ora.dwdb01.ons ONLINE ONLINE on dwdb01
ora.dwdb01.vip ONLINE ONLINE on dwdb01
ora.dwdb02.ASM2.asm ONLINE ONLINE on dwdb02
ora.dwdb02.LISTENER_DWDB02.lsnr ONLINE ONLINE on dwdb02
ora.dwdb02.gsd ONLINE ONLINE on dwdb02
ora.dwdb02.ons ONLINE ONLINE on dwdb02
ora.dwdb02.vip ONLINE ONLINE on dwdb02
ora.dwdb03.ASM3.asm ONLINE ONLINE on dwdb03
ora.dwdb03.LISTENER_DWDB03.lsnr ONLINE ONLINE on dwdb03
ora.dwdb03.gsd ONLINE ONLINE on dwdb03
ora.dwdb03.ons ONLINE ONLINE on dwdb03
ora.dwdb03.vip ONLINE ONLINE on dwdb03
ora.dwdb04.ASM4.asm ONLINE OFFLINE
ora.dwdb04.LISTENER_DWDB04.lsnr ONLINE OFFLINE
ora.dwdb04.gsd ONLINE OFFLINE
ora.dwdb04.ons ONLINE OFFLINE
ora.dwdb04.vip ONLINE ONLINE on dwdb01
ora.dwrac.db ONLINE ONLINE on dwdb02
ora.dwrac.dw.cs ONLINE ONLINE on dwdb03
ora.dwrac.dw.dwrac1.srv ONLINE ONLINE on dwdb01
ora.dwrac.dw.dwrac2.srv ONLINE ONLINE on dwdb02
ora.dwrac.dw.dwrac3.srv ONLINE ONLINE on dwdb03
ora.dwrac.dw.dwrac4.srv ONLINE OFFLINE
ora.dwrac.dwrac1.inst ONLINE ONLINE on dwdb01
ora.dwrac.dwrac2.inst ONLINE ONLINE on dwdb02
ora.dwrac.dwrac3.inst ONLINE ONLINE on dwdb03
ora.dwrac.dwrac4.inst ONLINE OFFLINE
------------------------
6 STATE=OFFLINE
8 STATE=ONLINE on dwdb01
8 STATE=ONLINE on dwdb02
8 STATE=ONLINE on dwdb03

可以看到，dwdb04的VIP已經切換到節點dwdb01了，說明VIP的failover已經成功發生了。

4. 重啟啟動dwdb04的crs，模擬故障節點恢復

[root@dwdb04 ~]# /oracle/product/10.2.0/crs/bin/crsctl start crs
Attempting to start CRS stack
The CRS stack will be started shortly
[root@dwdb04 ~]#

[root@dwdb01 suk]# crs_stat
HA Resource Target State
----------- ------ -----
ora.dwdb01.ASM1.asm ONLINE ONLINE on dwdb01
ora.dwdb01.LISTENER_DWDB01.lsnr ONLINE ONLINE on dwdb01
ora.dwdb01.gsd ONLINE ONLINE on dwdb01
ora.dwdb01.ons ONLINE ONLINE on dwdb01
ora.dwdb01.vip ONLINE ONLINE on dwdb01
ora.dwdb02.ASM2.asm ONLINE ONLINE on dwdb02
ora.dwdb02.LISTENER_DWDB02.lsnr ONLINE ONLINE on dwdb02
ora.dwdb02.gsd ONLINE ONLINE on dwdb02
ora.dwdb02.ons ONLINE ONLINE on dwdb02
ora.dwdb02.vip ONLINE ONLINE on dwdb02
ora.dwdb03.ASM3.asm ONLINE ONLINE on dwdb03
ora.dwdb03.LISTENER_DWDB03.lsnr ONLINE ONLINE on dwdb03
ora.dwdb03.gsd ONLINE ONLINE on dwdb03
ora.dwdb03.ons ONLINE ONLINE on dwdb03
ora.dwdb03.vip ONLINE ONLINE on dwdb03
ora.dwdb04.ASM4.asm ONLINE ONLINE on dwdb04
ora.dwdb04.LISTENER_DWDB04.lsnr ONLINE ONLINE on dwdb04
ora.dwdb04.gsd ONLINE ONLINE on dwdb04
ora.dwdb04.ons ONLINE ONLINE on dwdb04
ora.dwdb04.vip ONLINE ONLINE on dwdb04
ora.dwrac.db ONLINE ONLINE on dwdb02
ora.dwrac.dw.cs ONLINE ONLINE on dwdb03
ora.dwrac.dw.dwrac1.srv ONLINE ONLINE on dwdb01
ora.dwrac.dw.dwrac2.srv ONLINE ONLINE on dwdb02
ora.dwrac.dw.dwrac3.srv ONLINE ONLINE on dwdb03
ora.dwrac.dw.dwrac4.srv ONLINE ONLINE on dwdb04
ora.dwrac.dwrac1.inst ONLINE ONLINE on dwdb01
ora.dwrac.dwrac2.inst ONLINE ONLINE on dwdb02
ora.dwrac.dwrac3.inst ONLINE ONLINE on dwdb03
ora.dwrac.dwrac4.inst ONLINE ONLINE on dwdb04
------------------------
7 STATE=ONLINE on dwdb01
8 STATE=ONLINE on dwdb02
8 STATE=ONLINE on dwdb03
7 STATE=ONLINE on dwdb04


可以看到，當故障節點恢復了，dwdb04的VIP資源又從dwdb01切換回來了。

5. 模擬看看多個節點發生故障的情況

[root@dwdb02 suk]# /oracle/product/10.2.0/crs/bin/crsctl stop crs
Stopping resources. This could take several minutes.
Successfully stopped CRS resources.
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.

[root@dwdb03 orion]# /oracle/product/10.2.0/crs/bin/crsctl stop crs
Stopping resources. This could take several minutes.
Successfully stopped CRS resources.
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.

[root@dwdb04 ~]# /oracle/product/10.2.0/crs/bin/crsctl stop crs
Stopping resources. This could take several minutes.
Successfully stopped CRS resources.
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.

[root@dwdb01 suk]# crs_stat
HA Resource Target State
----------- ------ -----
ora.dwdb01.ASM1.asm ONLINE ONLINE on dwdb01
ora.dwdb01.LISTENER_DWDB01.lsnr ONLINE ONLINE on dwdb01
ora.dwdb01.gsd ONLINE ONLINE on dwdb01
ora.dwdb01.ons ONLINE ONLINE on dwdb01
ora.dwdb01.vip ONLINE ONLINE on dwdb01
ora.dwdb02.ASM2.asm ONLINE OFFLINE
ora.dwdb02.LISTENER_DWDB02.lsnr ONLINE OFFLINE
ora.dwdb02.gsd ONLINE OFFLINE
ora.dwdb02.ons ONLINE OFFLINE
ora.dwdb02.vip ONLINE ONLINE on dwdb01
ora.dwdb03.ASM3.asm ONLINE OFFLINE
ora.dwdb03.LISTENER_DWDB03.lsnr ONLINE OFFLINE
ora.dwdb03.gsd ONLINE OFFLINE
ora.dwdb03.ons ONLINE OFFLINE
ora.dwdb03.vip ONLINE ONLINE on dwdb01
ora.dwdb04.ASM4.asm ONLINE OFFLINE
ora.dwdb04.LISTENER_DWDB04.lsnr ONLINE OFFLINE
ora.dwdb04.gsd ONLINE OFFLINE
ora.dwdb04.ons ONLINE OFFLINE
ora.dwdb04.vip ONLINE ONLINE on dwdb01
ora.dwrac.db ONLINE ONLINE on dwdb01
ora.dwrac.dw.cs ONLINE ONLINE on dwdb01
ora.dwrac.dw.dwrac1.srv ONLINE ONLINE on dwdb01
ora.dwrac.dw.dwrac2.srv ONLINE OFFLINE
ora.dwrac.dw.dwrac3.srv ONLINE OFFLINE
ora.dwrac.dw.dwrac4.srv ONLINE OFFLINE
ora.dwrac.dwrac1.inst ONLINE ONLINE on dwdb01
ora.dwrac.dwrac2.inst ONLINE OFFLINE
ora.dwrac.dwrac3.inst ONLINE OFFLINE
ora.dwrac.dwrac4.inst ONLINE OFFLINE
------------------------
18 STATE=OFFLINE
12 STATE=ONLINE on dwdb01

可見，所有故障節點的VIP都切換到正常的節點(dwdb01)上。

簡單總結：
1. 一個節點或者crs發生故障時，VIP資源會自動切換到其他節點上。切換時間在1分鐘以內。
2. 只要不是全部的節點都發生故障，VIP都可以failover到正常的節點上。VIP一般優先切換到master節點上
2. 當故障節點恢復時，VIP又會自動恢復到原來的節點上