Oracle Real Application Testing之DB Replay實踐案例
RAT 作用是在系統改變後,在測試環境上進行了全面評估.RAT有兩個重要的元件:Database Replay、SQL Performance Analyzer
Database Replay | SQL Performance Analyzer | |
---|---|---|
What is ......? | Replays real database workload on test system | Predicts SQL performance deviations before end-users can be impacted |
What is the Purpose of .....? | Assess impact of change on workload throughput | Assess impact of change on SQL response time |
How does ..... work? |
Database Replay Workflow:
|
SQL Performance Analyzer Workflow:
|
When should I use .....? |
Comprehensive testing of all sub-systems of the database server using real production workload, including:
-Database and operating system upgrades |
Unit testing of SQL with the goal to identify the set of SQL statements with improved/regressed performance, including:
- Database upgrade |
二.使用DB Replay環境需要
1.可以使用DB Replay的資料版本及PATCH包的需要
參考文件:Using Real Application Testing Functionality in Earlier Releases (文件 ID 560977.1)
在11g 之前安裝Real Application testing,Opatch 工具應該被使用,如果你想要使用DB replay或SQL Performance Analyzer 僅打上對應的補丁即可;如果想同時使用DB Replay或SPA就需要把對應的被丁都要打上;
Database Replay
升級資料庫版本到11g或11g版本以上,Database Replay 的capture功能在以下表所列出的早期版本中已生效;
Note: The replay of the captured workload can only be done on Oracle Database 11g and higher.
注:Replay功能僅僅可以在11G及以後版本可用;如果是12.1.0.1或更高版本則都不需要任何強制性的PATCH
Table 1: Database Replay Availability Information for All Platforms except WindowsNote:
- These are mandatory patches.
- Table may not show all possible combinations.
-
Any new Fixes, patches or merge patches for Real Application Testing
will be provided as per Oracle’s Error Correction Support Policy documented in
Database, FMW, EM Grid Control, and OCS Software Error Correction Support Policy
Document 209768.1.
Source DB Upgrade from release | Destination DB Upgrade to any release | What patch you need to apply? | Download Information | Comments |
---|---|---|---|---|
9.2.0.8.0 | >=11.1.0.6.0 |
9.2.0.8.0 + one-off patch 9373986 For AIX platform one-off patch 13370576 (it includes 9373986) |
One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded from MOS |
9.2.0.8.0 | >=11.1.0.7.0 |
9.2.0.8.0 + one-off patch 9373986 + For AIX platform one-off patch 13370576 (it includes 9373986) one-off patch 8712466 on top of 11.1.0.7.0 |
One-off patch can be downloaded or requested from MOS | 8712466 is a merge patch on top of 11.1.0.7.0 |
9.2.0.8.0 | >=11.2.0.1.0 |
9.2.0.8.0 + one-off patch 9373986 For AIX platform one-off patch 13370576 ( it includes 9373986) |
One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded from MOS |
9.2.0.8.0 | >=11.2.0.2.0 |
9.2.0.8.0 + one-off patch 9373986 |
One-off patch can be downloaded from MOS |
One-off patch can be downloaded from MOS
|
9.2.0.8.0 | >=11.2.0.3.0 |
9.2.0.8.0 + one-off patch 9373986 For AIX platform one-off patch 13370576 ( it includes 9373986) and 11.2.0.4.0 + patch 17411249 |
One-off patch can be downloaded or requested from MOS |
One-off patch can be downloaded or requested from MOS
|
9.2.0.8.0 | >=11.2.0.4.0 |
9.2.0.8.0 + one-off patch 9373986 |
One-off patch can be downloaded or requested from MOS |
One-off patch can be downloaded or requested from MOS |
10.2.0.2.0 | >=11.1.0.6.0 | 10.2.0.2.0 + one-off patch 9373986 | One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded or requested from MOS |
10.2.0.2.0 | >=11.1.0.7.0 |
10.2.0.2.0 + one-off patch 9373986 and one-off patch 8712466 on top of 11.1.0.7.0 |
One-off patch can be downloaded or requested from MOS | 8712466 is a merge patch on top of 11.1.0.7.0 |
10.2.0.2.0 | >=11.2.0.1.0 | 10.2.0.2.0 + one-off patch 9373986 | One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded or requested from MOS |
10.2.0.2.0 | >=11.2.0.2.0 |
10.2.0.2.0 + one-off patch 9373986 |
One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded or requested from MOS |
10.2.0.2.0 | >=11.2.0.3.0 |
10.2.0.2.0 + one-off patch 9373986 |
One-off patch can be downloaded or requested from MOS |
One-off patch can be downloaded or requested from MOS |
10.2.0.2.0 | >=11.2.0.4.0 |
10.2.0.2.0 + one-off patch 9373986 |
One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded or requested from MOS |
10.2.0.3.0 | >=11.1.0.6.0 | 10.2.0.3.0 + one-off patch 9373986 | One-off patch can be downloaded from MOS | One-off patch can be downloaded from MOS |
10.2.0.3.0 | >=11.1.0.7.0 | 10.2.0.3.0 + one-off patch 9373986+ one-off patch 8712466 on top of 11.1.0.7.0 | One-off patch can be downloaded from MOS | 8712466 is a merge patch on top of 11.1.0.7.0 |
10.2.0.3.0 | >=11.2.0.1.0 | 10.2.0.3.0 + one-off patch 9373986 | One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded or requested from MOS |
10.2.0.3.0 | >=11.2.0.2.0 |
10.2.0.3.0 + one-off patch 9373986 |
One-off patch can be downloaded from MOS | One-off patch can be downloaded from MOS |
10.2.0.3.0 | >=11.2.0.3.0 |
10.2.0.3.0 + one-off patch 9373986 |
One-off patch can be downloaded or requested from MOS |
One-off patch can be downloaded or requested from MOS
|
10.2.0.3.0 | >=11.2.0.4.0 |
10.2.0.3.0 + one-off patch 9373986 |
One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded or requested from MOS |
10.2.0.4.0 | >= 11.1.0.6.0 | 10.2.0.4.0 Patchset + one-off patch 10239989 | Patchset can be downloaded from MOS |
Functionality already exists in Patchset,
download from MOS Enable workload capture by following instructions below |
10.2.0.4.0 | >=11.1.0.7.0 | 10.2.0.4.0 patchset+ one-off patch 10239989 + one-off patch 8712466 on top of 11.1.0.7.0 | Patchset can be downloaded from MOS |
Functionality already exists in Patchset, download from Metalink
8712466 is a merge patch on top of 11.1.0.7.0 |
10.2.0.4.0 | >=11.2.0.1.0 | 10.2.0.4.0 Patchset + one-off patch 10239989 | Patchset can be downloaded from MOS |
Functionality already exists in Patchset,
download from MOS Enable workload capture by following instructions below |
10.2.0.4.0 | >=11.2.0.2.0 |
10.2.0.4.0 Patchset + one-off patch 10239989 and 11.2.0.2.0 + patch 13947480 |
Patchset can be downloaded from MOS |
Functionality already exists in Patchset,
download from MOS Enable workload capture by following instructions below |
10.2.0.4.0 | >=11.2.0.3.0 |
10.2.0.4.0 Patchset + one-off patch 10239989 and 11.2.0.3.0 + patch 17411249 |
Patchset can be downloaded from MOS |
Functionality already exists in Patchset, download from MOS |
10.2.0.4.0 | >=11.2.0.4.0 |
10.2.0.4.0 Patchset + one-off patch 1023998 and 11.2.0.4.0 + patch 17411249 |
Patchset can be downloaded from MOS |
Functionality already exists in Patchset, download from MOS |
10.2.0.5.0 | >=11.2.0.1.0 |
10.2.0.5.0 Patchset + one-off patch 9373986 |
One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded or requested from MOS |
10.2.0.5.0 | >=11.2.0.2.0 |
10.2.0.5.0 Patchset + one-off patch 9373986 and 11.2.0.2.0 + patch 13947480 |
One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded or requested from MOS |
10.2.0.5.0 | >=11.2.0.3.0 |
10.2.0.5.0 Patchset + one-off patch 9373986
|
One-off patch can be downloaded or requested from MOS |
One-off patch can be downloaded or requested from MOS
|
10.2.0.5.0 | >=11.2.0.4.0 |
10.2.0.5.0 Patchset + one-off patch 9373986 11.2.0.4.0 + patch 17411249 |
One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded or requested from MOS |
11.1.0.6.0 | >=11.2.0.1.0 | 11.1.0.6.0 + one-off patch 8712466 + 9373986 | One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded or requested from MOS |
11.1.0.6.0 | >=11.2.0.2.0 |
11.1.0.6.0 + one-off patch 8712466 + 9373986 and 11.2.0.2.0 + patch 13947480 |
One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded or requested from MOS |
11.1.0.6.0 | >=11.2.0.3.0 |
11.1.0.6.0 + one-off patch 8712466 + 9373986 |
One-off patch can be downloaded or requested from MOS |
One-off patch can be downloaded or requested from MOS
|
11.1.0.6.0 | >=11.2.0.4.0 |
11.1.0.6.0 + one-off patch 8712466 + 9373986 |
One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded or requested from MOS |
11.1.0.7.0 | >=11.2.0.1.0 | 11.1.0.7.0 + one-off patch 8712466 + 9373986 | One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded or requested from MOS |
11.1.0.7.0 | >=11.2.0.2.0 |
11.1.0.7.0 + one-off patch 8712466 + 9373986 and 11.2.0.2.0 + patch 13947480 |
One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded or requested from MOS |
11.1.0.7.0 | >=11.2.0.3.0 |
11.1.0.7.0 + one-off patch 8712466 + 9373986 |
One-off patch can be downloaded or requested from MOS |
One-off patch can be downloaded or requested from MOS
|
11.1.0.7.0 | >=11.2.0.4.0 |
11.1.0.7.0 + one-off patch 8712466 + 9373986 |
One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded or requested from MOS |
11.2.0.1.0 | >=11.2.0.2.0 |
11.2.0.1.0 + one-off patch 9373986 11.2.0.2.0 + patch 13947480 |
One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded or requested from MOS |
11.2.0.1.0 | >=11.2.0.3.0 |
11.2.0.1.0 + one-off patch 9373986 11.2.0.3.0 + patch 17411249 |
One-off patch can be downloaded or requested from MOS |
One-off patch can be downloaded or requested from MOS
|
11.2.0.1.0 | >=11.2.0.4.0 |
11.2.0.1.0 + one-off patch 9373986 11.2.0.4.0 + patch 17411249 |
One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded or requested from MOS |
11.2.0.2.0 | >= 11.2.0.2.0 |
Capture side No capture side patches are needed for 11.2.0.2.0 and Replay Side 11.2.0.2.0 + patch 13947480 |
One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded or requested from MOS |
11.2.0.2.0 | >=11.2.0.3.0 |
Capture side No capture side patches are needed for 11.2.0.2.0 and Replay Side 11.2.0.3.0 + patch 17411249 |
One-off patch can be downloaded or requested from MOS |
One-off patch can be downloaded or requested from MOS
|
11.2.0.2.0 | >=11.2.0.4.0 |
Capture side No capture side patches are needed for 11.2.0.2.0 and Replay Side 11.2.0.4.0 + patch 17411249 |
One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded or requested from MOS |
11.2.0.3.0 | >=11.2.0.3.0 |
Capture side No capture side patch are needed for 11.2.0.3.0 and Replay side 11.2.0.3.0 + patch 17411249 |
One-off patch can be downloaded or requested from MOS |
One-off patch can be downloaded or requested from MOS
|
11.2.0.3.0 | >=11.2.0.4.0 |
Capture side No capture side patch are needed for 11.2.0.3.0 and Replay side 11.2.0.4.0 + patch 17411249 |
One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded or requested from MOS |
11.2.0.4.0 | >=11.2.0.4.0 |
Capture side No capture side patch is needed for 11.2.0.4.0 and Replay side 11.2.0.4.0 + 17411249 |
One-off patch can be downloaded or requested from MOS | One-off patch can be downloaded or requested from MOS |
Note:
- These are mandatory patches.
- Table may not show all possible combinations.
- Any new Fixes, patches or merge patches for Real Application Testing will be provided as per Oracle’s Error Correction Support Policy documented in Database, FMW, EM Grid Control, and OCS Software Error Correction Support Policy Document 209768.1.
Source DB Upgrade from release | Destination DB Upgrade to any release | What patch you need to apply? | Download Information | Comments |
---|---|---|---|---|
9.2.0.8.0 For Windows 32-bit |
>=11.1.0.6.0 including 11.2.X |
9.2.0.8.0 + patch bundle 7047008 | Patch bundle can be downloaded from MOS | Patch Available as part of Bundle 18. |
9.2.0.8.0 For Windows Itanium 64-bit |
>=11.1.0.6.0 including 11.2.X |
9.2.0.8.0 + patch bundle 7047015 | Patch bundle can be downloaded from MOS | Patch Available as part of Bundle 18. |
10.2.0.3.0 For Windows 32-bit |
>=11.1.0.6.0 including 11.2.X |
10.2.0.3.0 + patch bundle 6998002 | Patch bundle can be downloaded from MOS | Patch Available as part of Bundle 23. |
10.2.0.3.0 For Windows Itanium 64-bit |
>=11.1.0.6.0 including 11.2.X |
10.2.0.3.0 + patch bundle 6998003 | Patch bundle can be downloaded from MOS | Patch Available as part of Bundle 23. |
10.2.0.3.0 For Windows 64-bit AMD64 AND Intel EM64T XP AND 2003 |
>=11.1.0.6.0 including 11.2.X |
10.2.0.3.0 + patch bundle 6998004 | Patch bundle can be downloaded from MOS | Patch Available as part of Bundle 23. |
10.2.0.4.0 (Windows 32-bit, Windows Itanium Windows 64-bit AMD64 AND Intel EM64T XP AND 2003) |
>= 11.1.0.6.0 including 11.2.X |
10.2.0.4.0 Patchset + one-off patch 8542772 | Patchset can be downloaded from MOS: 6810189 |
Functionality already exists in Patchset, download from MOS.
Enable workload capture by following instructions |
10.2.0.5.0 ( all windows platform ) |
>=11.1.0.6.0 including 11.2.X |
No mandatory patches needed. Functionality already exists in Patchset. |
Patchset can be downloaded from MOS | Functionality already exists in Patchset, download from MOS. |
本案例我們的版本源及目標端均為11.2.0.3,因此我們只需要在目標端打PATCH 17411249即可
檢查目標端是否打PATCH
opatch lsinventory |grep 17411249
2.開啟Workload Capture功能(11G版本是開啟的,不需要單獨設定PRE_11G_ENABLE_CAPTURE=TURE)
Enabling Workload Capture (Only Required for 10.2.0.4.0)
By default, the workload capture is disabled on the pre-11g database releases. Database workload capture functionality is enabled on the system by specifying the PRE_11G_ENABLE_CAPTURE initialization parameter. In order to enable workload capture on 10.2.0.4 run the wrrenbl.sql script at the SQL prompt as SYS or SYSTEM as follows:
The wrrenbl.sql script calls the ALTER SYSTEM SQL statement to set the
PRE_11G_ENABLE_CAPTURE initialization parameter to TRUE.
This step is NOT
required for version lower then 10.2.0.4.0
Please check the OTN documentation
mentioned above for more details.
3.檢查元件
透過下述方法確認Real Application Testing元件已經安裝:
(1): 檢查inventory:
$ORACLE_HOME/OPatch/opatch lsinventory -details|grep "Oracle Real Application Testing"
(2): 使用aix archiver (ar)命令:
cd $ORACLE_HOME/rdbms/lib
$ ar -X64 -t libknlopt.a | grep -c kecwr.o
1
ar命令返回0表示RAT被禁用,返回> 0表示啟用
(3): 檢查資料字典:
select value from v$option where parameter = 'Real Application Testing';
(4).檢查Package:
select object_name, object_type, status from dba_objects where object_name like '%DBMS_WORKLOAD_%';
4.Workload Catpure的限制
The following types of client requests are not supported.
- Direct path load of data from external files using utilities such as SQL*Loader
- Non-PL/SQL based Advanced Queuing (AQ)
- Flashback queries
- Oracle Call Interface (OCI) based object navigations
- Non SQL-based object access
- Distributed transactions (any distributed transactions that are captured will be replayed as local transactions)
- Oracle Streams/Advanced Replication workload is not supported prior to 11.2.
- Database session migration
- Database Resident Connection Pooling ( DRCP )
- XA transactions
- Workloads having Object Out Bind
三、實施DB Replay
引數文件:Using Workload Capture and Replay (文件 ID 445116.1)
3.1 實施Database Replay,要對生產環境主機、資料庫,以及測試環境主機、資料庫做必要的準備:
生產環境c4oyy3a/c4oyy3b主機、資料庫準備:
1. 安裝主機效能監控軟體,如OSW或NMON,用於Database Replay對負載捕獲階段和負載重演階段的主機負載指標做對比;
2. 在2臺RAC資料庫主機上劃分500GB(預估可滿足3小時抓取要求?)的檔案系統,用於儲存負載記錄檔案、AWR的DUMP檔案等,該檔案系統必須配置為共享的檔案系統,例如CFS或NFS;(NAS儲存提供幾十TB的空間?);
3. 生產環境負載捕捉的起始時間是Database Replay的基準時間,必需準確記錄這一時間點,在這一時間點開始BCV同步保留一份生產環境資料庫的映象(該映象在Database Replay測試期間要一直保留);初步定這個時間是c4oyy3a/c4oyy3b的5月12:30
4. 如果無法保留BCV映象,就必須保留一份老的全庫備份集和相關歸檔,這個全庫備份集必需是生產環境負載捕捉的起始時間點之前完成的全庫備份集,確保將來可以根據這個備份集和歸檔把資料庫恢復到生產環境負載捕捉的起始時間點;
5. 負載捕捉多長時間支援兩種模式,一種是透過引數duration設定時間長度,到時間後自動停止捕捉;另一種是設定duration引數為NULL或者忽略該引數,在捕捉啟動後的任意時間點透過手工停止;
測試環境toyy3a_cs/toyy3b_cs主機、資料庫準備:
1. 安裝主機效能監控軟體,如OSW或NMON,用於Database Replay對負載捕獲階段和負載重演階段的主機負載指標做對比;
2. 在測試環境的4臺RAC資料庫主機上劃分500GB的共享檔案系統,用於存放從生產環境傳輸過來的負載記錄檔案、AWR的DUMP檔案,以及預處理檔案、重演時的日誌檔案等;該檔案系統必須配置為共享的檔案系統,例如CFS或NFS;
3. 在測試環境的4臺RAC資料庫主機上劃分500GB的共享檔案系統,做為資料庫的Flash Recovery Area,用於啟用Flashback Database功能,在每次負載重演開始時(假設會多次重演)可以把資料庫回退到一致的初始狀態;該檔案系統必須配置為共享的檔案系統,例如CFS或NFS; 如果採用BCV 反向複製回退資料庫,則此處要求的Flash Recovery Area檔案系統可不配置;
4. 測試環境的4臺RAC資料庫主機在每次負載重演開始時都會調整時間為生產環境負載捕捉的起始時間,確保這樣的時間調整不會影響其他系統;
5. 應用軟體開發商檢查、整理資料庫的外部訪問,例如DB LINK、外部表、URL、目錄物件等等,以及應用訪問的連線串等,避免測試環境誤連誤改其他生產系統;把所有DB LINK全部DROP掉、或者不配置tnsnames.ora;ezconnect host1:1521/orcl
6. 資料庫的AWR保留週期要滿足負載捕捉視窗的要求,避免捕捉期間的AWR資料在匯出之前就被PURGE掉;該項準備已經完成
3.2 DB Replay 資料捕獲
任一時刻,每套資料庫只能啟動一個負載捕獲作業(RAC環境也只能在一個節點上開啟)。為了確保捕獲的負載準確、有效,要考慮如下因素:
1.設定捕獲過濾器。預設情況下,所有使用者的會話產生的負載都會被捕捉。使用包含過濾器或排除過濾器(inclusion filters or exclusion filters)用於限定針對哪些使用者、應用、模組等等的負載進行捕獲;例如,下列演示了新增、刪除一個過濾器:
SQL> conn /as sysdba;
SQL> BEGIN
DBMS_WORKLOAD_CAPTURE.ADD_FILTER (
fname => 'user_ichan',
fattribute => 'USER',
fvalue => 'ICHAN');
END;
/
SQL> BEGIN
DBMS_WORKLOAD_CAPTURE.DELETE_FILTER (fname => 'user_ichan');
END;
/
2.建立、配置捕獲目錄。確保該目錄是空的,且空間足夠用於存放負載記錄檔案,對於RAC資料庫,建議該目錄建立於叢集共享檔案系統;例如:
$ mkdir –p /dbreplay
SQL> conn /as sysdba;
drop directory db_replay;
SQL> create directory db_replay as '/dbreplay/capture';
SQL> grant read,write on directory db_replay to public;
3.官方建議在捕獲之前把資料庫重啟到RESTRICT模式(這一過程雖然不是強制的,但這樣做可以保證捕獲的事務完整性及重演時的準確性),然後開始負載捕獲,預設情況下,一旦負載捕獲開始,資料庫會自動從RESTRICTED模式轉換為UNRESTRICTED模式,業務使用者可以正常連線。由於申請停機時間視窗困難,此次的負載捕獲將省略這一過程;
4.負載捕捉多長時間支援兩種模式,一種是透過引數duration設定時間長度,到時間後自動停止捕捉;另一種是設定duration引數為NULL或者忽略該引數,在捕捉啟動後的任意時間點透過手工停止;例如,下列演示了手工啟動、手工停止:
需要要注意的地方:
o The directory name should always be upper case.
o It is possible to start the capture without restricted mode using the argument no_restart_mode=TRUE but this is not advisable as it relies on transactional integrity
--開啟捕獲
SQL> conn /as sysdba;
SQL> BEGIN
DBMS_WORKLOAD_CAPTURE.START_CAPTURE (name => 'db_replay_peak',
dir => 'DB_REPLAY',
duration => null);
END;
/
--結束捕獲
SQL> BEGIN
DBMS_WORKLOAD_CAPTURE.FINISH_CAPTURE ();
END;
/
--刪除捕獲(如有多餘的捕獲資訊)
BEGIN
DBMS_WORKLOAD_CAPTURE.DELETE_CAPTURE_INFO(15);
END;
/
5.匯出生產環境資料庫的AWR資料,用於對負載捕捉期間和負載重演期間的資料庫效能做詳細比對。例如,下列演示了匯出某個負載捕捉作業ID對應的AWR資料:
SQL> conn /as sysdba;
SQL> select id,AWR_BEGIN_SNAP,AWR_END_SNAP from dba_workload_captures;
SQL> BEGIN
DBMS_WORKLOAD_CAPTURE.EXPORT_AWR (capture_id => 14);
END;
/
6.監控捕捉
負載捕捉的過程中,可以透過檢視DBA_WORKLOAD_CAPTURES和DBA_WORKLOAD_FILTERS進行監控,還可以生成負載捕捉報告,對捕捉總體情況進行概覽。例如:
SQL> conn /as sysdba;
SQL> select id,name,status,start_time,end_time,connects,user_calls,dir_path from dba_workload_captures where id = (select max(id) from dba_workload_captures) ;
SQL> set pagesize 0 long 30000000 longchunksize 1000
SQL> select dbms_workload_capture.report(1,'TEXT') from dual;
3.3 Database Replay (重放)
1.預處理
一旦負載捕獲完成,且測試環境搭建完以後,就可以開始預處理負載記錄檔案。預處理是建立後設資料用於負載重演的必要前提步驟。
預處理是一個耗資源、耗時間的過程,需要從生產環境把所有負載記錄檔案傳輸到測試環境的一個單一的目錄中進行集中預處理。對於測試環境的RAC資料庫,選擇其中一個例項做預處理。例如,下列演示了匯出某個負載捕獲作業ID對應的AWR資料:
$ mkdir –p /cluster_file_system/db_replay
SQL> conn /as sysdba;
SQL> create directory db_replay as '/cluster_file_system/db_replay';
SQL> grant read,write on directory db_replay to public;
SQL> BEGIN
DBMS_WORKLOAD_REPLAY.PROCESS_CAPTURE (capture_dir => 'DB_REPLAY');
END;
/
注:在11.2.0.3時會遇到一個,該BUG會導致system表空間快速增加,且DB replay處理完了也不釋放;
參考文件:Pre-Processing Using DBMS_WORKLOAD_REPLAY Uses a Lot of System Tablespace Space (文件 ID 1497607.1)
SQL> select sum(bytes/1024/1024) from dba_segments where segment_name='WRR$_REPLAY_DEPENDENCIES_TMP';
SUM(BYTES/1024/1024)
--------------------
3443
Solution
1. is fixed in 12.1.
2. The Workaround is to drop all these temporary tables after preprocessing.
The selects from dba_segments before and after are simply to help show the space that has been freed.
select sum(blocks) from dba_segments where segment_name like 'WRR$%';
drop table sys.WRR$_REPLAY_COMMITS_TMP cascade constraints purge;
drop table sys.WRR$_REPLAY_CONN_DATA_TMP cascade constraints purge;
drop table sys.WRR$_REPLAY_DATA_TMP cascade constraints purge;
drop table sys.WRR$_REPLAY_DEPENDENCIES_TMP cascade constraints purge;
drop table sys.WRR$_REPLAY_DEP_GRAPH_TMP cascade constraints purge;
drop table sys.WRR$_REPLAY_LOGIN_QUEUE_TMP cascade constraints purge;
drop table sys.WRR$_REPLAY_REFERENCES_TMP cascade constraints purge;
drop table sys.WRR$_REPLAY_SCN_ORDER_TMP cascade constraints purge;
drop table sys.WRR$_REPLAY_SEQ_DATA_TMP cascade constraints purge;
select sum(blocks) from dba_segments where segment_name like 'WRR$%';
2.重演負載
預處理完成後,重演負載可參考如下步驟執行:
(1).初始化重演資料,該步驟是將預處理獲取的後設資料裝載入庫的過程。例如:
SQL> conn /as sysdba;
SQL> BEGIN
DBMS_WORKLOAD_REPLAY.INITIALIZE_REPLAY (replay_name => 'db_replay_102',
replay_dir => 'DB_REPLAY');
END;
/
2。連線串重新對映,該步驟是把生產環境的連線串影射為測試環境的連線串。例如:
SQL> conn /as sysdba;
SQL> BEGIN
DBMS_WORKLOAD_REPLAY.REMAP_CONNECTION (connection_id => 101,
replay_connection => 'dlsun244:3434/bjava21');
END;
/
例如:
col CAPTURE_CONN for a50
col REPLAY_CONN for a50
set lines 200
select * from DBA_WORKLOAD_CONNECTION_MAP where rownum<2;
REPLAY_ID CONN_ID CAPTURE_CONN REPLAY_CONN
---------- ---------- -------------------------------------------------- --------------------------------------------------
1 97 (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(Host=oyy3a.or
a.boss)(Port=1521))(CONNECT_DATA=(SERVICE_NAME=ora
yy3)(INSTANCE_NAME=oyy3a)(server=dedicated)(FAILOV
ER_MODE=(BACKUP=oyy3b)(TYPE=NONE))(CID=(PROGRAM=ci
csas@c4m3a)(HOST=c4m3a)(USER=cics))))
1 98 (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(Host=oyy3a.or
a.boss)(Port=1521))(CONNECT_DATA=(SERVICE_NAME=ora
yy3)(INSTANCE_NAME=oyy3a)(server=dedicated)(FAILOV
ER_MODE=(BACKUP=oyy3b)(TYPE=NONE))(CID=(PROGRAM=ci
csas@c4m3b)(HOST=c4m3b)(USER=cics))))
透過如下方式,把字串連線資訊,放到根據conn_id,找到capture_conn串,針對性的指定相應的測試環境例項(如捕獲例項是A,同樣建議設定REPLAY環境也是測試機的A節點)
BEGIN
DBMS_WORKLOAD_REPLAY.REMAP_CONNECTION (connection_id => 97,
replay_connection => '(description_list=(load_balance=off)(failover=on)
(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(Host=10.19.243.23)(Port=1521)))
(CONNECT_DATA=(SERVICE_NAME=orayy3)(INSTANCE_NAME=oyy3a)(server=dedicated)
(FAILOVER_MODE=(BACKUP=toyy3b_cs)(TYPE=SESSION))))
(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(Host=10.19.243.25)(Port=1521)))
(CONNECT_DATA=(SERVICE_NAME=orayy3)(INSTANCE_NAME=oyy3b)(server=dedicated)
(FAILOVER_MODE=(BACKUP=toyy3a_cs)(TYPE=SESSION)))))');
END;
/
(3) 設定重演選項,該步驟是透過設定重演引數(參考前面的“重演條件”章節對於這些引數的說明)為重演做準備。例如:
SQL> conn /as sysdba;
--使用該種方式,是按照捕獲時候的順序來執行的;也是預設的
execute DBMS_WORKLOAD_REPLAY.PREPARE_REPLAY(SYNCHRONIZATION => 'OBJECT_ID');
--The most common changes occur in the sync settings where FALSE/'OFF' or 'SCN' may provide better results.
--檢視replay狀態(只有在PAREPARE時,才可以進行REPLAY)
SQL> select name,status from dba_workload_replays;
SQL> select name,status from dba_workload_replays;
NAME STATUS
-------------------- --------------------
test_capture_1 PREPARE
select name,STATUS,CAPTURE_ID,PREPARE_TIME,START_TIME,NUM_CLIENTS THINK_TIME_SCALE,SYNCHRONIZATION,PREPROCESSING_ID from WRR$_REPLAYS;
注:
在prepare replay時,11.2.0.3環境如果沒有打相應的PATCH,很可能會遇到BUG。正常情況下捕獲5個小時,PREPARE建立也就是3-5分鐘的左右,BUG現象執行時間需要10多個小時:
解決方案:需要建立如下兩個索引來
create index wrr$_replay_dep_calls ON wrr$_replay_dep_graph(file_id,call_ctr, sync_point,file_id_dep,call_ctr_dep);
create index wrr$_replay_dep_commits ON wrr$_replay_dep_graph(file_id,call_ctr_dep,file_id_dep,call_ctr,sync_point);
SQL> exec DBMS_WORKLOAD_REPLAY.prepare_replay (synchronization => FALSE);
PL/SQL procedure successfully completed.
(4) 定義過濾器和過濾集,該步驟是使用包含過濾器或排除過濾器(inclusion filters or exclusion filters)用於限定針對哪些使用者、應用、模組等等的負載進行捕獲。例如,下例演示了新增、刪除過濾器,以及定義過濾集,啟用過濾集:
SQL> conn /as sysdba;
SQL> BEGIN
DBMS_WORKLOAD_REPLAY.ADD_FILTER (
fname => 'user_ichan',
fattribute => 'USER',
fvalue => 'ICHAN');
END;
/
BEGIN
DBMS_WORKLOAD_REPLAY.DELETE_FILTER (fname => 'user_ichan');
END;
/
BEGIN
DBMS_WORKLOAD_REPLAY.CREATE_FILTER_SET (
replay_dir => 'DB_REPLAY',
filter_set => 'replayfilters',
default_action => 'INCLUDE');
END;
/
BEGIN
DBMS_WORKLOAD_REPLAY.USE_FILTER_SET (filter_set => 'replayfilters');
END;
/
(5)評估需要多少客戶端來滿足REPLAY條件(在新的測試環境下)
注:為了更準備的模擬測試環境,需要單獨準備一個裝有oracle client機器來提供所有WRC的連線;本案例評估需要97個客戶端
wrc mode=calibrate replaydir=/dbreplay/capture
Report for Workload in: /dbreplay/capture
-----------------------
Recommendation:
Consider using at least 97 clients divided among 25 CPU(s)
You will need at least 135 MB of memory per client process.
If your machine(s) cannot match that number, consider using more clients.
Workload Characteristics:
- max concurrency: 3449 sessions
- total number of sessions: 286891
Assumptions:
- 1 client process per 50 concurrent sessions
- 4 client process per CPU
- 256 KB of memory cache per concurrent session
- think time scale = 100
- connect time scale = 100
- synchronization = TRUE
(6)開啟客戶端連線
由於本案例是RAC環境,本次wrc需要透過SCAN TNS連線到測試庫的SCAN 監聽,再由SCAN監聽來根據DBA_REPLAY_CONNECTION_MAP檢視來決定客戶端資訊要連線到哪個例項;
6.1 在測試庫環境下配置SCAN 監聽,確保SCAN監聽可用:
參考文件:11.2 Scan and Node TNS Listener Setup Examples (文件 ID 1070607.1)
6.2 測試庫客戶端
配置TNS:
toyy3_scan =(DESCRIPTION =(ADDRESS_LIST =(ADDRESS =(PROTOCOL= TCP)(HOST= SCAN-IP)(PORT=1521)))(CONNECT_DATA =(SERVICE_NAME = orayy3)))
6.3 開啟客戶端連線
注:修改作業系統時間為捕獲時間
Before we can start the replay, we need to calibrate and start a replay client using the "wrc" utility. The calibration step tells us the number of replay clients and hosts necessary to faithfully replay the workload.
如果測試是RAC環境需要使用SCAN TNS連線,來根據dba_replay_connection_map檢視裡的會話資訊來連線資料庫例項
$nohup wrc system/system#123@toyy3_scan mode=replay replaydir=/dbreplay/capture &
$nohup wrc system/system#123@toyy3_scan mode=replay replaydir=/dbreplay/capture &
(7).啟動、暫停、繼續、取消重演,當完成負載記錄檔案預處理、重演初始化、設定重演選項、啟動重演客戶端以後,就可以開始負載重演。例如,下例演示了啟動、暫停、繼續、取消重演:
SQL> conn /as sysdba;
SQL> BEGIN
DBMS_WORKLOAD_REPLAY.START_REPLAY ();
END;
/
BEGIN
DBMS_WORKLOAD_REPLAY.PAUSE_REPLAY ();
END;
/
BEGIN
DBMS_WORKLOAD_REPLAY.RESUME_REPLAY ();
END;
/
BEGIN
DBMS_WORKLOAD_REPLAY.CANCEL_REPLAY ();
END;
/
(8)診斷DB REPLAY慢的指令碼=============
SCRIPT
Query to Obtain Information on Blocking Sessions
--------------------------------------
-- Query 1 --
--------------------------------------
connect sys/<password> as sysdba
col inst_id format 99999
col sid format 99999
col spid format a6
col blocking_session_status format a6 heading 'BS'
col blocking_instance format 99 heading 'BI'
col blocking_session format 99999 heading 'BLKSID'
col session_type format a11
col event format a31
col file_name format a21
col file_id format 9999999999999999999
col call_counter format 9999999
col wait_for_scn format 99999999999999 heading 'WAITING FOR'
col wfscn format 99999999999999 heading 'WFSCN'
col commit_wait_scn format 99999999999999 heading 'CWSCN'
col post_commit_scn format 99999999999999 heading 'PCSCN'
col clock format 99999999999999999999 heading 'CLOCK'
col next_ticker format 999999999999999999999 heading 'NEXT TICKER'
select wrt.inst_id, wrt.sid, wrt.serial#, wrt.spid,
s.BLOCKING_SESSION_STATUS, s.BLOCKING_INSTANCE,
s.blocking_session,
wrt.session_type, wrt.event,
wrt.file_name, wrt.file_id, wrt.call_counter,
wrt.wait_for_scn,
greatest(wrt.dependent_scn, wrt.statement_scn) as wfscn,
wrt.commit_wait_scn, wrt.post_commit_scn,
wrt.clock, wrt.next_ticker
from gv$workload_replay_thread wrt, gv$session s
where wrt.sid = s.sid
and wrt.serial# = s.serial#
order by inst_id, sid
;set linesize 500 pagesize 200
Query to Monitor the Approximate Progress of the Replay
--------------------------------------
-- Query 2 --
--------------------------------------
set echo off
connect sys/<password> as sysdba
set serveroutput on
DECLARE
my_next_ticker NUMBER;
clock NUMBER;
wait_for_scn NUMBER;
counts NUMBER;
replay_id NUMBER;
thr_failure NUMBER;
start_time DATE;
num_tickers NUMBER;
min_scn NUMBER;
max_scn NUMBER;
done NUMBER;
total_time INTERVAL DAY TO SECOND;
CURSOR get_next_ticker(my_next_ticker NUMBER) IS
SELECT spid, event, inst_id, wrc_id, client_pid
FROM gv$workload_replay_thread
WHERE file_id = my_next_ticker;
BEGIN
dbms_output.put_line('********************************');
dbms_output.put_line('* Replay Status Report *');
dbms_output.put_line('********************************');
-----------------------------------------
-- Make sure that a replay is in progress
-----------------------------------------
SELECT count(*) INTO counts
FROM dba_workload_replays
WHERE status='IN PROGRESS';
if (counts = 0) then
dbms_output.put_line('No replay in progress!');
return;
end if;
-------------------
-- Get replay state
-------------------
SELECT id,start_time INTO replay_id, start_time
FROM dba_workload_replays
WHERE status='IN PROGRESS';
SELECT count(*) INTO counts
FROM gv$workload_replay_thread
WHERE session_type = 'REPLAY';
SELECT min(wait_for_scn), max(next_ticker), max(clock)
INTO wait_for_scn, my_next_ticker, clock
FROM v$workload_replay_thread
WHERE wait_for_scn <> 0
AND session_type = 'REPLAY';
dbms_output.put_line('Replay has been running for: ' ||
to_char(systimestamp - start_time));
dbms_output.put_line('Current clock is: ' || clock);
dbms_output.put_line('Replay is waiting on clock: ' ||
wait_for_scn);
dbms_output.put_line(counts || ' threads are currently being
replayed.');
----------------------------------------
-- Find info about the next clock ticker
----------------------------------------
num_tickers := 0;
for rec in get_next_ticker(my_next_ticker) loop
-- We only want the next clock ticker
num_tickers := num_tickers + 1;
exit when num_tickers > 1;
dbms_output.put_line('Next ticker is process ' || rec.spid ||
' (' || rec.wrc_id || ',' || rec.client_pid ||
') in instance ' || rec.inst_id ||
' and is waiting on ');
dbms_output.put_line(' ' || rec.event);
end loop;
---------------------------------------------------------------------------------------
-- Compute the replay progression and estimate the time left
-- Note: This is an estimated time only, not an absolute value as it is based on SCN.
---------------------------------------------------------------------------------------
SELECT min(post_commit_scn), max(post_commit_scn)
INTO min_scn,max_scn
FROM wrr$_replay_scn_order;
done := (clock - min_scn) / (max_scn - min_scn);
total_time := (systimestamp - start_time) / done;
dbms_output.put_line('Estimated progression in replay: ' ||
to_char(100*done, '00') || '% done.');
dbms_output.put_line('Estimated time before completion: ' ||
((1 - done) * total_time));
dbms_output.put_line('Estimated total time for replay: ' ||
total_time);
dbms_output.put_line('Estimated final time for replay: ' ||
to_char(start_time + total_time,
'DD-MON-YY HH24:MI:SS'));
END;
/
Query to Provide Summary of Wait Events
--------------------------------------------------------------
-- Query 3 --
---------------------------------------------------------------
column event format a40
select event, count(*), min(wait_for_scn)
from gv$workload_replay_thread
where session_type = 'REPLAY'
group by event;
(9).匯出測試環境資料庫的AWR資料,用於對負載捕捉期間和負載重演期間的資料庫效能做詳細比對。例如,下列演示了匯出某個負載重演作業ID對應的AWR資料:
SQL> conn /as sysdba;
SQL> BEGIN
DBMS_WORKLOAD_REPLAY.EXPORT_AWR (replay_id => 1);
END;
/
(10).匯入負載捕獲期間的AWR資料,在測試環境資料庫中利用dbms_workload_repository.awr_diff_report_text或dbms_workload_repository.awr_diff_report_html對負載捕捉期間和負載重演期間的資料庫效能做詳細比對。例如:
SQL> conn /as sysdba;
SQL> select dbms_workload_capture.import_awr(capture_id => 1, staging_schema => 'AWRRPT') from dual;
SQL> -- get the capture data details
SQL> select id, awr_begin_snap, awr_end_snap from dba_workload_captures;
SQL> -- get the replay data details
SQL> select id, awr_begin_snap, awr_end_snap from dba_workload_replays;
(11) 監控重演
在重演中,任何錯誤,或者出現在生產環境和測試環境之間的資料不符,都會被記錄為分歧呼叫。獲取這些分歧呼叫的資訊,包括:SQL ID、SQL文字、繫結變數取值,都可以透過檢視DBA_WORKLOAD_REPLAY_DIVERGENCE,以及GET_DIVERGENT_STATEMENT獲取。例如:
--方法一
SQL> conn /as sysdba;
SQL> SELECT REPLAY_ID,STREAM_ID,CALL_COUNTER FROM DBA_WORKLOAD_REPLAY_DIVERGENCE;
--方法二
Sqlplus / as sysdba
set long 30000000 longchunksize 1000
DECLARE
r CLOB;
ls_stream_id NUMBER;
ls_call_counter NUMBER;
ls_sql_cd VARCHAR2(20);
ls_sql_err VARCHAR2(512);
CURSOR c IS
SELECT stream_id,call_counter
FROM DBA_WORKLOAD_REPLAY_DIVERGENCE
WHERE replay_id = <Replay ID>;
BEGIN
OPEN c;
LOOP
FETCH c INTO ls_stream_id, ls_call_counter;
EXIT when c%notfound;
DBMS_OUTPUT.PUT_LINE (ls_stream_id||''||ls_call_counter);
r:=DBMS_WORKLOAD_REPLAY.GET_DIVERGING_STATEMENT(replay_id => <Replay ID>,
stream_id => ls_stream_id, call_counter => ls_call_counter);
DBMS_OUTPUT.PUT_LINE (r);
END LOOP;
END;
/
同時,還可以生成負載重演報告,對重演總體情況進行概覽。例如:
SQL> conn /as sysdba;
SQL> select id,name,status,start_time,end_time,num_clients,user_calls,dir_path from dba_workload_replays where id = (select max(id) from dba_workload_replays) ;
SQL> set pagesize 0 long 30000000 longchunksize 2000
SQL> select dbms_workload_replay.report(replay_id => 1,format => 'TEXT') from dual;
此外,還可以透過如下檢視獲取負載捕捉和負載重演期間的更多資訊:
1. DBA_WORKLOAD_CAPTURES列出所有捕獲到的負載;
2. DBA_WORKLOAD_FILTERS列出所有捕獲的負載定義的過濾器;
3. DBA_WORKLOAD_REPLAYS列出所有已經重演過的負載; view lists all the workload replays that have been replayed in the current database.
4. DBA_WORKLOAD_REPLAY_DIVERGENCE列出所有分歧呼叫,包括replay identifier, stream identifier, 以及call counter.
5. DBA_WORKLOAD_REPLAY_FILTER_SET列出所有重演的負載定義的過濾器;
6. DBA_WORKLOAD_CONNECTION_MAP列出所有重演使用的連線串的對映資訊;
7. V$WORKLOAD_REPLAY_THREAD列出當前所有重演客戶端的會話資訊;
來自 “ ITPUB部落格 ” ,連結:http://blog.itpub.net/29320885/viewspace-2129364/,如需轉載,請註明出處,否則將追究法律責任。
相關文章
- RAT(Oracle Real Application Testing)時ORA 600報錯處理OracleAPP
- 【RAT】Oracle Real Application Testing(真用應用測試)介紹OracleAPP
- Oracle Real Application Clusters GFSOracleAPP
- Using Real Application Testing Functionality in Earlier Releases_560977.1APPFunction
- Deinstallation Tool for Oracle Clusterware and Oracle Real Application ClustersOracleAPP
- Split Brain in Oracle Clusterware and Real Application ClusterAIOracleAPP
- oracle real cluster application install on windows with VMweaverOracleAPPWindows
- Real Application Test(RAT)APP
- Laravel 之 Application---實踐篇LaravelAPP
- How a Database Is Mounted with Real Application Clusters (294)DatabaseAPP
- CRS and 10g Real Application ClustersAPP
- oracle real application clusters and administration and deployment guide 7-13OracleAPPGUIIDE
- Oracle Database ReplayOracleDatabase
- 輕量級流量複製goreplay實踐Go
- Oracle 11g Database Replay 實驗OracleDatabase
- ArgoCD實踐之基於配置清單建立ApplicationGoAPP
- oracle rat database replayOracleDatabase
- ORA-00439: feature not enabled: Real Application ClustersAPP
- INE - Advanced Web Application Penetration Testing (eWPTx)WebAPP
- zt_oracle10g rac real application cluster+asm修改spfile到新路徑OracleAPPASM
- 10g Hp-ux Oracle Real Application Clusters 安裝和配置指南(1)UXOracleAPP
- Oracle replayc測試Oracle
- Read Consistency with Real Application Clusters (314)APP
- DB2 獲取當前DB連線的application id, application handleDB2APP
- 效能改進之專案例會匯入實踐
- Go-Testing golang 測試案例Golang
- zt_oracle10g rac real application cluster ocr and voting disk備份恢復OracleAPP
- INE - Web Application Penetration Testing Professional (萌新級)WebAPP
- Flex Box 案例實踐Flex
- Oracle Performance Testing PrincipleOracleORM
- Netbackup+oracle db 異機恢復案例Oracle
- Django websocket之web端實時檢視日誌實踐案例DjangoWeb
- Oracle Real Time SQL MonitorOracleSQL
- 對於診斷Oracle Clusterware CRS或GI和Real Application Cluster RAC問題的資料收集OracleAPP
- Oracle Real Time SQL MonitoringOracleSQL
- Oracle實踐Oracle
- 分割槽表索引實踐案例索引
- Configuring Raw Devices for Real Application Clusters on Linux_246205.1devAPPLinux