Oracle Real Application Testing之DB Replay實踐案例

靜以致遠√團團發表於2016-11-30
一、Oracle Real Application testing介紹
RAT 作用是在系統改變後,在測試環境上進行了全面評估.RAT有兩個重要的元件:Database Replay、SQL Performance Analyzer 

Database Replay SQL Performance Analyzer
What is ......? Replays real database workload on test system Predicts SQL performance deviations before end-users can be impacted
What is the Purpose of .....? Assess impact of change on workload throughput Assess impact of change on SQL response time
How does ..... work?

Database Replay Workflow:

  1. Workload Capture
  2. Workload Processing
  3. Workload Replay – including DML and SQL queries
  4. Analysis and Reporting

SQL Performance Analyzer Workflow:

  1. Capture the SQL workload that you want to analyze with SPA
  2. Measure the performance of the workload before a change by executing SPA on the SQL tuning set
  3. Make the change, such as database upgrade or gathering optimizer statistics
  4. Measure the performance of the workload after the change by executing SPA on The SQL Tuning Set again
  5. Compare performance of the two executions of the SQL tuning set to identify the SQL statements that have regressed, improved, or were unchanged
When should I use .....?

Comprehensive testing of all sub-systems of the database server using real production workload, including:

-Database and operating system upgrades
- Configuration changes, such as conversion of a database from a single instance to an Oracle Real   Application Clusters (Oracle RAC) environment     
- Storage, network, and interconnect changes
- Operating system and hardware migrations

Unit testing of SQL with the goal to identify the set of SQL statements with improved/regressed performance, including:

- Database upgrade
- Configuration changes to the operating system or  hardware
- Schema changes
- Changes to database initialization parameters
- Refreshing optimizer statistics
- SQL tuning actions

Database Replay Workflow 




二.使用DB Replay環境需要
1.可以使用DB Replay的資料版本及PATCH包的需要
參考文件:Using Real Application Testing Functionality in Earlier Releases (文件 ID 560977.1)
在11g 之前安裝Real Application testing,Opatch 工具應該被使用,如果你想要使用DB replay或SQL Performance Analyzer 僅打上對應的補丁即可;如果想同時使用DB Replay或SPA就需要把對應的被丁都要打上;
Database Replay
升級資料庫版本到11g或11g版本以上,Database Replay 的capture功能在以下表所列出的早期版本中已生效;
Note: The replay of the captured workload can only be done on Oracle Database 11g and higher. 
注:Replay功能僅僅可以在11G及以後版本可用;如果是12.1.0.1或更高版本則都不需要任何強制性的PATCH
Table 1: Database Replay Availability Information for All Platforms except WindowsNote:
  • These are mandatory patches.
  • Table may not show all possible combinations.
  • Any new Fixes, patches or merge patches for Real Application Testing will be provided as per Oracle’s Error Correction Support Policy documented in Database, FMW, EM Grid Control, and OCS Software Error Correction Support Policy Document 209768.1.

Source DB Upgrade from release Destination DB Upgrade to any release What patch you need to apply? Download Information Comments
9.2.0.8.0 >=11.1.0.6.0 9.2.0.8.0 + one-off patch 9373986

For AIX platform one-off patch 13370576 (it includes 9373986)
One-off patch can be downloaded or requested from MOS One-off patch can be downloaded from MOS
9.2.0.8.0 >=11.1.0.7.0 9.2.0.8.0 + one-off patch 9373986 +

For AIX platform one-off patch 13370576 (it includes 9373986)


one-off patch
8712466 on top of 11.1.0.7.0
One-off patch can be downloaded or requested from MOS 8712466 is a merge patch on top of 11.1.0.7.0
9.2.0.8.0 >=11.2.0.1.0 9.2.0.8.0 + one-off patch 9373986

For AIX platform one-off patch 13370576 ( it includes 9373986)
One-off patch can be downloaded or requested from MOS One-off patch can be downloaded from MOS
9.2.0.8.0 >=11.2.0.2.0

9.2.0.8.0 + one-off patch 9373986

For AIX platform one-off patch 13370576 (it includes 9373986)


and 11.2.0.2.0 + patch 13947480

One-off patch can be downloaded from MOS

One-off patch can be downloaded from MOS

 

9.2.0.8.0 >=11.2.0.3.0 9.2.0.8.0 + one-off patch 9373986

For AIX platform one-off patch 13370576 ( it includes 9373986)

and 11.2.0.4.0 + patch 17411249
One-off patch can be downloaded or requested from MOS

One-off patch can be downloaded or requested from MOS

 

9.2.0.8.0 >=11.2.0.4.0

9.2.0.8.0 + one-off patch 9373986

For AIX platform one-off patch 13370576 ( it includes 9373986)

and 11.2.0.4.0 + patch 17411249

One-off patch can be downloaded or requested from MOS

One-off patch can be downloaded or requested from MOS

10.2.0.2.0 >=11.1.0.6.0 10.2.0.2.0 + one-off patch 9373986 One-off patch can be downloaded or requested from MOS One-off patch can be downloaded or requested from MOS
10.2.0.2.0 >=11.1.0.7.0

10.2.0.2.0 + one-off patch 9373986 and

one-off patch 8712466 on top of 11.1.0.7.0

One-off patch can be downloaded or requested from MOS 8712466 is a merge patch on top of 11.1.0.7.0
10.2.0.2.0 >=11.2.0.1.0 10.2.0.2.0 + one-off patch 9373986 One-off patch can be downloaded or requested from MOS One-off patch can be downloaded or requested from MOS
10.2.0.2.0 >=11.2.0.2.0

10.2.0.2.0 + one-off patch 9373986

and 11.2.0.2.0 + patch 13947480

One-off patch can be downloaded or requested from MOS One-off patch can be downloaded or requested from MOS
10.2.0.2.0 >=11.2.0.3.0

10.2.0.2.0 + one-off patch 9373986

and 11.2.0.3.0 + patch 17411249

One-off patch can be downloaded or requested from MOS

One-off patch can be downloaded or requested from MOS

10.2.0.2.0  >=11.2.0.4.0

10.2.0.2.0 + one-off patch 9373986

and 11.2.0.4.0 + patch 17411249

One-off patch can be downloaded or requested from MOS One-off patch can be downloaded or requested from MOS
10.2.0.3.0 >=11.1.0.6.0 10.2.0.3.0 + one-off patch 9373986 One-off patch can be downloaded from MOS One-off patch can be downloaded from MOS
10.2.0.3.0 >=11.1.0.7.0 10.2.0.3.0 + one-off patch 9373986+ one-off patch 8712466 on top of 11.1.0.7.0 One-off patch can be downloaded from MOS 8712466 is a merge patch on top of 11.1.0.7.0
10.2.0.3.0 >=11.2.0.1.0 10.2.0.3.0 + one-off patch 9373986 One-off patch can be downloaded or requested from MOS One-off patch can be downloaded or requested from MOS
10.2.0.3.0 >=11.2.0.2.0

10.2.0.3.0 + one-off patch 9373986

and 11.2.0.2.0 + patch 13947480

One-off patch can be downloaded from MOS One-off patch can be downloaded from MOS
10.2.0.3.0 >=11.2.0.3.0

10.2.0.3.0 + one-off patch 9373986

and 11.2.0.3.0 + patch 17411249

One-off patch can be downloaded or requested from MOS

One-off patch can be downloaded or requested from MOS

 

10.2.0.3.0 >=11.2.0.4.0

10.2.0.3.0 + one-off patch 9373986

and 11.2.0.4.0 + 17411249

One-off patch can be downloaded or requested from MOS One-off patch can be downloaded or requested from MOS
10.2.0.4.0 >= 11.1.0.6.0 10.2.0.4.0 Patchset + one-off patch 10239989 Patchset can be downloaded from MOS Functionality already exists in Patchset, download from MOS
Enable workload capture by following instructions below
10.2.0.4.0 >=11.1.0.7.0 10.2.0.4.0 patchset+ one-off patch 10239989 + one-off patch 8712466 on top of 11.1.0.7.0 Patchset can be downloaded from MOS

Functionality already exists in Patchset, download from Metalink


Enable workload capture by following instructions below

8712466 is a merge patch on top of 11.1.0.7.0

10.2.0.4.0 >=11.2.0.1.0 10.2.0.4.0 Patchset + one-off patch 10239989 Patchset can be downloaded from MOS Functionality already exists in Patchset, download from MOS


Enable workload capture by following instructions below
10.2.0.4.0 >=11.2.0.2.0

10.2.0.4.0 Patchset + one-off patch 10239989

and 11.2.0.2.0 + patch 13947480

Patchset can be downloaded from MOS Functionality already exists in Patchset, download from MOS

Enable workload capture by following instructions below
10.2.0.4.0 >=11.2.0.3.0

10.2.0.4.0 Patchset + one-off patch 10239989

and

11.2.0.3.0 + patch 17411249

Patchset can be downloaded from MOS

Functionality already exists in Patchset, download from MOS

Enable workload capture by following instructions below

10.2.0.4.0 >=11.2.0.4.0

10.2.0.4.0 Patchset + one-off patch 1023998

and

11.2.0.4.0 + patch 17411249

Patchset can be downloaded from MOS

Functionality already exists in Patchset, download from MOS

Enable workload capture by following instructions below

10.2.0.5.0 >=11.2.0.1.0

10.2.0.5.0 Patchset + one-off patch 9373986

One-off patch can be downloaded or requested from MOS One-off patch can be downloaded or requested from MOS
10.2.0.5.0 >=11.2.0.2.0 10.2.0.5.0 Patchset + one-off patch 9373986
and
11.2.0.2.0 + patch 13947480
One-off patch can be downloaded or requested from MOS One-off patch can be downloaded or requested from MOS
10.2.0.5.0 >=11.2.0.3.0

10.2.0.5.0 Patchset + one-off patch 9373986


and
11.2.0.3.0 + patch 17411249

One-off patch can be downloaded or requested from MOS

One-off patch can be downloaded or requested from MOS

 

10.2.0.5.0 >=11.2.0.4.0

10.2.0.5.0 Patchset + one-off patch 9373986
and

11.2.0.4.0 + patch 17411249

One-off patch can be downloaded or requested from MOS One-off patch can be downloaded or requested from MOS
11.1.0.6.0 >=11.2.0.1.0 11.1.0.6.0  + one-off patch 8712466 + 9373986 One-off patch can be downloaded or requested from MOS One-off patch can be downloaded or requested from MOS
11.1.0.6.0 >=11.2.0.2.0 11.1.0.6.0 + one-off patch 8712466 + 9373986

and

11.2.0.2.0 + patch 13947480
One-off patch can be downloaded or requested from MOS One-off patch can be downloaded or requested from MOS
11.1.0.6.0 >=11.2.0.3.0

11.1.0.6.0 + one-off patch 8712466 + 9373986

and

11.2.0.3.0 + patch  17411249

One-off patch can be downloaded or requested from MOS

One-off patch can be downloaded or requested from MOS

 

11.1.0.6.0 >=11.2.0.4.0

11.1.0.6.0 + one-off patch 8712466 + 9373986

and

11.2.0.4.0 + patch 17411249

One-off patch can be downloaded or requested from MOS One-off patch can be downloaded or requested from MOS
11.1.0.7.0 >=11.2.0.1.0 11.1.0.7.0 + one-off patch 8712466 + 9373986 One-off patch can be downloaded or requested from MOS One-off patch can be downloaded or requested from MOS
11.1.0.7.0 >=11.2.0.2.0 11.1.0.7.0 + one-off patch 8712466 + 9373986

and

11.2.0.2.0 + patch 13947480
One-off patch can be downloaded or requested from MOS One-off patch can be downloaded or requested from MOS
11.1.0.7.0 >=11.2.0.3.0

11.1.0.7.0 + one-off patch 8712466 + 9373986

and

11.2.0.3.0 + patch 17411249

One-off patch can be downloaded or requested from MOS

One-off patch can be downloaded or requested from MOS

 

11.1.0.7.0 >=11.2.0.4.0

11.1.0.7.0 + one-off patch 8712466 + 9373986

and

11.2.0.4.0 + patch 17411249

One-off patch can be downloaded or requested from MOS One-off patch can be downloaded or requested from MOS
11.2.0.1.0 >=11.2.0.2.0

11.2.0.1.0 + one-off patch 9373986

and

11.2.0.2.0 +  patch 13947480

One-off patch can be downloaded or requested from MOS One-off patch can be downloaded or requested from MOS
11.2.0.1.0 >=11.2.0.3.0

11.2.0.1.0 + one-off patch 9373986

and

11.2.0.3.0 + patch 17411249

One-off patch can be downloaded or requested from MOS

One-off patch can be downloaded or requested from MOS

 

11.2.0.1.0 >=11.2.0.4.0

11.2.0.1.0 + one-off patch 9373986

and

11.2.0.4.0 + patch 17411249

One-off patch can be downloaded or requested from MOS One-off patch can be downloaded or requested from MOS
11.2.0.2.0 >= 11.2.0.2.0

Capture side

No capture side patches are needed for 11.2.0.2.0

and

Replay Side

11.2.0.2.0 + patch 13947480

One-off patch can be downloaded or requested from MOS One-off patch can be downloaded or requested from MOS
11.2.0.2.0 >=11.2.0.3.0

Capture side

No capture side patches are needed for 11.2.0.2.0

and

Replay Side

11.2.0.3.0 + patch 17411249

One-off patch can be downloaded or requested from MOS

One-off patch can be downloaded or requested from MOS

 

11.2.0.2.0 >=11.2.0.4.0

Capture side

No capture side patches are needed for 11.2.0.2.0

and

Replay Side

11.2.0.4.0 + patch 17411249
One-off patch can be downloaded or requested from MOS One-off patch can be downloaded or requested from MOS
11.2.0.3.0 >=11.2.0.3.0

Capture side

No capture side patch are needed for 11.2.0.3.0

and

Replay side

11.2.0.3.0 + patch 17411249

One-off patch can be downloaded or requested from MOS

One-off patch can be downloaded or requested from MOS

 

11.2.0.3.0 >=11.2.0.4.0

Capture side

No capture side patch are needed for 11.2.0.3.0

and

Replay side

11.2.0.4.0 + patch 17411249

One-off patch can be downloaded or requested from MOS One-off patch can be downloaded or requested from MOS
11.2.0.4.0 >=11.2.0.4.0

Capture side

No capture side patch is needed for 11.2.0.4.0

and

Replay side

11.2.0.4.0 + 17411249

One-off patch can be downloaded or requested from MOS One-off patch can be downloaded or requested from MOS
Table 2: Database Replay Availability Information Windows Platform

Note:

  • These are mandatory patches.
  • Table may not show all possible combinations.
  • Any new Fixes, patches or merge patches for Real Application Testing will be provided as per Oracle’s Error Correction Support Policy documented in Database, FMW, EM Grid Control, and OCS Software Error Correction Support Policy Document 209768.1.
Source DB Upgrade from release Destination DB Upgrade to any release What patch you need to apply? Download Information Comments
9.2.0.8.0 For Windows 32-bit

>=11.1.0.6.0

including 11.2.X

9.2.0.8.0 + patch bundle 7047008 Patch bundle can be downloaded from MOS Patch Available as part of Bundle 18.
9.2.0.8.0 For Windows Itanium 64-bit

>=11.1.0.6.0

including 11.2.X

9.2.0.8.0 + patch bundle 7047015 Patch bundle can be downloaded from MOS Patch Available as part of Bundle 18.
10.2.0.3.0 For Windows 32-bit

>=11.1.0.6.0

including 11.2.X

10.2.0.3.0 + patch bundle 6998002 Patch bundle can be downloaded from MOS Patch Available as part of Bundle 23.
10.2.0.3.0 For Windows Itanium 64-bit

>=11.1.0.6.0

including 11.2.X

10.2.0.3.0 + patch bundle 6998003 Patch bundle can be downloaded from MOS Patch Available as part of Bundle 23.
10.2.0.3.0
For Windows 64-bit AMD64 AND Intel EM64T XP AND 2003

>=11.1.0.6.0

including 11.2.X

10.2.0.3.0 + patch bundle 6998004 Patch bundle can be downloaded from MOS Patch Available as part of Bundle 23.
10.2.0.4.0
(Windows 32-bit, Windows Itanium Windows 64-bit AMD64 AND Intel EM64T XP AND 2003)

>= 11.1.0.6.0

including 11.2.X

10.2.0.4.0 Patchset + one-off patch 8542772 Patchset can be downloaded from MOS: 6810189

Functionality already exists in Patchset, download from MOS.


One-off patch 8542772 needs to be requested.

Enable workload capture by following instructions

10.2.0.5.0 ( all windows platform )

>=11.1.0.6.0

including 11.2.X

No mandatory patches needed.

Functionality already exists in Patchset.
Patchset can be downloaded from MOS Functionality already exists in Patchset, download from MOS.

本案例我們的版本源及目標端均為11.2.0.3,因此我們只需要在目標端打PATCH  17411249即可
檢查目標端是否打PATCH 
opatch lsinventory |grep 17411249

2.開啟Workload Capture功能(11G版本是開啟的,不需要單獨設定PRE_11G_ENABLE_CAPTURE=TURE)
Enabling Workload Capture (Only Required for 10.2.0.4.0)

By default, the workload capture is disabled on the pre-11g database releases. Database workload capture functionality is enabled on the system by specifying the PRE_11G_ENABLE_CAPTURE initialization parameter. In order to enable workload capture on 10.2.0.4  run the wrrenbl.sql script at the SQL prompt as SYS or SYSTEM as follows:

@$ORACLE_HOME/rdbms/admin/wrrenbl.sql

The wrrenbl.sql script calls the ALTER SYSTEM SQL statement to set the PRE_11G_ENABLE_CAPTURE initialization parameter to TRUE.
This step is NOT required for version lower then 10.2.0.4.0
Please check the OTN documentation mentioned above for more details.


3.檢查元件
通過下述方法確認Real Application Testing元件已經安裝:
(1): 檢查inventory:
$ORACLE_HOME/OPatch/opatch lsinventory -details|grep "Oracle Real Application Testing"
(2): 使用aix archiver (ar)命令:
cd $ORACLE_HOME/rdbms/lib 
$ ar -X64 -t libknlopt.a | grep -c kecwr.o 
1
ar命令返回0表示RAT被禁用,返回> 0表示啟用
(3): 檢查資料字典:
select value from v$option where parameter = 'Real Application Testing';
(4).檢查Package:
select object_name, object_type, status from dba_objects where object_name like '%DBMS_WORKLOAD_%';

4.Workload Catpure的限制

The following types of client requests are not supported.

  • Direct path load of data from external files using utilities such as SQL*Loader
  • Non-PL/SQL based Advanced Queuing (AQ)
  • Flashback queries
  • Oracle Call Interface (OCI) based object navigations
  • Non SQL-based object access
  • Distributed transactions (any distributed transactions that are captured will be replayed as local transactions)
  • Oracle Streams/Advanced Replication workload is not supported prior to 11.2.
  • Database session migration
  • Database Resident Connection Pooling ( DRCP )
  • XA transactions
  • Workloads having Object Out Bind
注:不支援的型別將可能在Replay時失敗,因此最好的方式是我們在捕獲時過濾掉這此不滿足條件的客戶端請求,預設情況下,捕獲的時候全部捕獲,但在Replay的時候不支援的型別將不重放;

三、實施DB Replay 
引數文件:Using Workload Capture and Replay (文件 ID 445116.1)
3.1 實施Database Replay,要對生產環境主機、資料庫,以及測試環境主機、資料庫做必要的準備:
生產環境c4oyy3a/c4oyy3b主機、資料庫準備:
1. 安裝主機效能監控軟體,如OSW或NMON,用於Database Replay對負載捕獲階段和負載重演階段的主機負載指標做對比;
2. 在2臺RAC資料庫主機上劃分500GB(預估可滿足3小時抓取要求?)的檔案系統,用於儲存負載記錄檔案、AWR的DUMP檔案等,該檔案系統必須配置為共享的檔案系統,例如CFS或NFS;(NAS儲存提供幾十TB的空間?);
3. 生產環境負載捕捉的起始時間是Database Replay的基準時間,必需準確記錄這一時間點,在這一時間點開始BCV同步保留一份生產環境資料庫的映象(該映象在Database Replay測試期間要一直保留);初步定這個時間是c4oyy3a/c4oyy3b的5月12:30
4. 如果無法保留BCV映象,就必須保留一份老的全庫備份集和相關歸檔,這個全庫備份集必需是生產環境負載捕捉的起始時間點之前完成的全庫備份集,確保將來可以根據這個備份集和歸檔把資料庫恢復到生產環境負載捕捉的起始時間點;
5. 負載捕捉多長時間支援兩種模式,一種是通過引數duration設定時間長度,到時間後自動停止捕捉;另一種是設定duration引數為NULL或者忽略該引數,在捕捉啟動後的任意時間點通過手工停止;

測試環境toyy3a_cs/toyy3b_cs主機、資料庫準備:
1. 安裝主機效能監控軟體,如OSW或NMON,用於Database Replay對負載捕獲階段和負載重演階段的主機負載指標做對比;
2. 在測試環境的4臺RAC資料庫主機上劃分500GB的共享檔案系統,用於存放從生產環境傳輸過來的負載記錄檔案、AWR的DUMP檔案,以及預處理檔案、重演時的日誌檔案等;該檔案系統必須配置為共享的檔案系統,例如CFS或NFS;
3. 在測試環境的4臺RAC資料庫主機上劃分500GB的共享檔案系統,做為資料庫的Flash Recovery Area,用於啟用Flashback Database功能,在每次負載重演開始時(假設會多次重演)可以把資料庫回退到一致的初始狀態;該檔案系統必須配置為共享的檔案系統,例如CFS或NFS;   如果採用BCV 反向複製回退資料庫,則此處要求的Flash Recovery Area檔案系統可不配置;
4. 測試環境的4臺RAC資料庫主機在每次負載重演開始時都會調整時間為生產環境負載捕捉的起始時間,確保這樣的時間調整不會影響其他系統;
5. 應用軟體開發商檢查、整理資料庫的外部訪問,例如DB LINK、外部表、URL、目錄物件等等,以及應用訪問的連線串等,避免測試環境誤連誤改其他生產系統;把所有DB LINK全部DROP掉、或者不配置tnsnames.ora;ezconnect host1:1521/orcl
6. 資料庫的AWR保留週期要滿足負載捕捉視窗的要求,避免捕捉期間的AWR資料在匯出之前就被PURGE掉;該項準備已經完成

3.2 DB Replay 資料捕獲

任一時刻,每套資料庫只能啟動一個負載捕獲作業(RAC環境也只能在一個節點上開啟)。為了確保捕獲的負載準確、有效,要考慮如下因素:
1.設定捕獲過濾器。預設情況下,所有使用者的會話產生的負載都會被捕捉。使用包含過濾器或排除過濾器(inclusion filters or exclusion filters)用於限定針對哪些使用者、應用、模組等等的負載進行捕獲;例如,下列演示了新增、刪除一個過濾器:
SQL> conn /as sysdba;
SQL> BEGIN
  DBMS_WORKLOAD_CAPTURE.ADD_FILTER (
                           fname => 'user_ichan',
                           fattribute => 'USER',
                           fvalue => 'ICHAN');
END;
/


SQL> BEGIN
  DBMS_WORKLOAD_CAPTURE.DELETE_FILTER (fname => 'user_ichan');
END;
/




2.建立、配置捕獲目錄。確保該目錄是空的,且空間足夠用於存放負載記錄檔案,對於RAC資料庫,建議該目錄建立於叢集共享檔案系統;例如:
$ mkdir –p /dbreplay
SQL> conn /as sysdba;
drop directory db_replay;
SQL> create directory db_replay as '/dbreplay/capture';
SQL> grant read,write on directory db_replay to public;


3.官方建議在捕獲之前把資料庫重啟到RESTRICT模式(這一過程雖然不是強制的,但這樣做可以保證捕獲的事務完整性及重演時的準確性),然後開始負載捕獲,預設情況下,一旦負載捕獲開始,資料庫會自動從RESTRICTED模式轉換為UNRESTRICTED模式,業務使用者可以正常連線。由於申請停機時間視窗困難,此次的負載捕獲將省略這一過程;
4.負載捕捉多長時間支援兩種模式,一種是通過引數duration設定時間長度,到時間後自動停止捕捉;另一種是設定duration引數為NULL或者忽略該引數,在捕捉啟動後的任意時間點通過手工停止;例如,下列演示了手工啟動、手工停止:
需要要注意的地方:
o The directory name should always be upper case. 
o It is possible to start the capture without restricted mode using the argument no_restart_mode=TRUE but this is not advisable as it relies on transactional integrity
--開啟捕獲
SQL> conn /as sysdba;

SQL> BEGIN
  DBMS_WORKLOAD_CAPTURE.START_CAPTURE (name => 'db_replay_peak', 
                           dir => 'DB_REPLAY',
                           duration => null);
END;
/

--結束捕獲
SQL> BEGIN
  DBMS_WORKLOAD_CAPTURE.FINISH_CAPTURE (); 
END;
/

--刪除捕獲(如有多餘的捕獲資訊)
BEGIN
DBMS_WORKLOAD_CAPTURE.DELETE_CAPTURE_INFO(15);
END;
/
5.匯出生產環境資料庫的AWR資料,用於對負載捕捉期間和負載重演期間的資料庫效能做詳細比對。例如,下列演示了匯出某個負載捕捉作業ID對應的AWR資料:
SQL> conn /as sysdba;
SQL> select id,AWR_BEGIN_SNAP,AWR_END_SNAP from dba_workload_captures;
SQL> BEGIN
  DBMS_WORKLOAD_CAPTURE.EXPORT_AWR (capture_id => 14);
END;
/
6.監控捕捉
負載捕捉的過程中,可以通過檢視DBA_WORKLOAD_CAPTURES和DBA_WORKLOAD_FILTERS進行監控,還可以生成負載捕捉報告,對捕捉總體情況進行概覽。例如:
SQL> conn /as sysdba;
SQL> select id,name,status,start_time,end_time,connects,user_calls,dir_path from dba_workload_captures where id = (select max(id) from dba_workload_captures) ;

SQL> set pagesize 0 long 30000000 longchunksize 1000
SQL> select dbms_workload_capture.report(1,'TEXT') from dual;

3.3 Database Replay (重放)

1.預處理
一旦負載捕獲完成,且測試環境搭建完以後,就可以開始預處理負載記錄檔案。預處理是建立後設資料用於負載重演的必要前提步驟。
預處理是一個耗資源、耗時間的過程,需要從生產環境把所有負載記錄檔案傳輸到測試環境的一個單一的目錄中進行集中預處理。對於測試環境的RAC資料庫,選擇其中一個例項做預處理。例如,下列演示了匯出某個負載捕獲作業ID對應的AWR資料:
$ mkdir –p /cluster_file_system/db_replay
SQL> conn /as sysdba;
SQL> create directory db_replay as '/cluster_file_system/db_replay';
SQL> grant read,write on directory db_replay to public;
SQL> BEGIN
  DBMS_WORKLOAD_REPLAY.PROCESS_CAPTURE (capture_dir => 'DB_REPLAY');
END;
/

注:在11.2.0.3時會遇到一個Bug 14500154,該BUG會導致system表空間快速增加,且DB replay處理完了也不釋放;
參考文件:Pre-Processing Using DBMS_WORKLOAD_REPLAY Uses a Lot of System Tablespace Space (文件 ID 1497607.1)
SQL> select sum(bytes/1024/1024) from dba_segments where segment_name='WRR$_REPLAY_DEPENDENCIES_TMP';
SUM(BYTES/1024/1024)
--------------------
                3443

Solution

1. Bug 14500154 is fixed in 12.1.

2. The Workaround is to drop all these temporary tables after preprocessing.
The selects from dba_segments before and after are simply to help show the space that has been freed.

select sum(blocks) from dba_segments where segment_name like 'WRR$%';

drop table sys.WRR$_REPLAY_COMMITS_TMP cascade constraints purge;
drop table sys.WRR$_REPLAY_CONN_DATA_TMP cascade constraints purge;
drop table sys.WRR$_REPLAY_DATA_TMP cascade constraints purge;
drop table sys.WRR$_REPLAY_DEPENDENCIES_TMP cascade constraints purge;
drop table sys.WRR$_REPLAY_DEP_GRAPH_TMP cascade constraints purge;
drop table sys.WRR$_REPLAY_LOGIN_QUEUE_TMP cascade constraints purge;
drop table sys.WRR$_REPLAY_REFERENCES_TMP cascade constraints purge;
drop table sys.WRR$_REPLAY_SCN_ORDER_TMP cascade constraints purge;
drop table sys.WRR$_REPLAY_SEQ_DATA_TMP cascade constraints purge;

select sum(blocks) from dba_segments where segment_name like 'WRR$%';



2.重演負載
預處理完成後,重演負載可參考如下步驟執行:
(1).初始化重演資料,該步驟是將預處理獲取的後設資料裝載入庫的過程。例如:
SQL> conn /as sysdba;
SQL> BEGIN
  DBMS_WORKLOAD_REPLAY.INITIALIZE_REPLAY (replay_name => 'db_replay_102',
                           replay_dir => 'DB_REPLAY');
END;
/
2。連線串重新對映,該步驟是把生產環境的連線串影射為測試環境的連線串。例如:
SQL> conn /as sysdba;
SQL> BEGIN
  DBMS_WORKLOAD_REPLAY.REMAP_CONNECTION (connection_id => 101,
                           replay_connection => 'dlsun244:3434/bjava21');
END;
/
例如:
col CAPTURE_CONN for a50
col REPLAY_CONN for a50
set lines 200
select * from DBA_WORKLOAD_CONNECTION_MAP where rownum<2;


 REPLAY_ID    CONN_ID CAPTURE_CONN                                       REPLAY_CONN                                                                                    
                                
---------- ---------- -------------------------------------------------- --------------------------------------------------                                             
                                
         1         97 (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(Host=oyy3a.or                                                                                                
                                
                      a.boss)(Port=1521))(CONNECT_DATA=(SERVICE_NAME=ora                                                                                                
                                
                      yy3)(INSTANCE_NAME=oyy3a)(server=dedicated)(FAILOV                                                                                                
                                
                      ER_MODE=(BACKUP=oyy3b)(TYPE=NONE))(CID=(PROGRAM=ci                                                                                                
                                
                      csas@c4m3a)(HOST=c4m3a)(USER=cics))))                                                                                                             
                                
                                                                                                                                                                        
                                
         1         98 (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(Host=oyy3a.or                                                                                                
                                
                      a.boss)(Port=1521))(CONNECT_DATA=(SERVICE_NAME=ora                                                                                                
                                
                      yy3)(INSTANCE_NAME=oyy3a)(server=dedicated)(FAILOV                                                                                                
                                
                      ER_MODE=(BACKUP=oyy3b)(TYPE=NONE))(CID=(PROGRAM=ci                                                                                                
                                
                      csas@c4m3b)(HOST=c4m3b)(USER=cics))))  
通過如下方式,把字串連線資訊,放到根據conn_id,找到capture_conn串,針對性的指定相應的測試環境例項(如捕獲例項是A,同樣建議設定REPLAY環境也是測試機的A節點)                 
BEGIN
  DBMS_WORKLOAD_REPLAY.REMAP_CONNECTION (connection_id => 97,
                           replay_connection => '(description_list=(load_balance=off)(failover=on)                                             
            (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(Host=10.19.243.23)(Port=1521)))    
                         (CONNECT_DATA=(SERVICE_NAME=orayy3)(INSTANCE_NAME=oyy3a)(server=dedicated)
                                       (FAILOVER_MODE=(BACKUP=toyy3b_cs)(TYPE=SESSION))))              
            (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(Host=10.19.243.25)(Port=1521)))    
                         (CONNECT_DATA=(SERVICE_NAME=orayy3)(INSTANCE_NAME=oyy3b)(server=dedicated)
                                       (FAILOVER_MODE=(BACKUP=toyy3a_cs)(TYPE=SESSION)))))');
END;
/



(3) 設定重演選項,該步驟是通過設定重演引數(參考前面的“重演條件”章節對於這些引數的說明)為重演做準備。例如:
SQL> conn /as sysdba;
--使用該種方式,是按照捕獲時候的順序來執行的;也是預設的
execute DBMS_WORKLOAD_REPLAY.PREPARE_REPLAY(SYNCHRONIZATION => 'OBJECT_ID'); 
--The most common changes occur in the sync settings where FALSE/'OFF' or 'SCN' may provide better results.
--檢視replay狀態(只有在PAREPARE時,才可以進行REPLAY)
SQL> select name,status from  dba_workload_replays;
SQL> select name,status from  dba_workload_replays;


NAME                 STATUS
-------------------- --------------------
test_capture_1       PREPARE

select name,STATUS,CAPTURE_ID,PREPARE_TIME,START_TIME,NUM_CLIENTS THINK_TIME_SCALE,SYNCHRONIZATION,PREPROCESSING_ID from WRR$_REPLAYS;

注:
在prepare replay時,11.2.0.3環境如果沒有打相應的PATCH,很可能會遇到BUG。正常情況下捕獲5個小時,PREPARE建立也就是3-5分鐘的左右,BUG現象執行時間需要10多個小時:
解決方案:需要建立如下兩個索引來
create index wrr$_replay_dep_calls ON wrr$_replay_dep_graph(file_id,call_ctr, sync_point,file_id_dep,call_ctr_dep); 
create index wrr$_replay_dep_commits ON wrr$_replay_dep_graph(file_id,call_ctr_dep,file_id_dep,call_ctr,sync_point); 
SQL> exec DBMS_WORKLOAD_REPLAY.prepare_replay (synchronization => FALSE);
PL/SQL procedure successfully completed.

(4) 定義過濾器和過濾集,該步驟是使用包含過濾器或排除過濾器(inclusion filters or exclusion filters)用於限定針對哪些使用者、應用、模組等等的負載進行捕獲。例如,下例演示了新增、刪除過濾器,以及定義過濾集,啟用過濾集:
SQL> conn /as sysdba;
SQL> BEGIN
  DBMS_WORKLOAD_REPLAY.ADD_FILTER (
                           fname => 'user_ichan',
                           fattribute => 'USER',
                           fvalue => 'ICHAN');
END;
/


BEGIN
  DBMS_WORKLOAD_REPLAY.DELETE_FILTER (fname => 'user_ichan');
END;
/


BEGIN
  DBMS_WORKLOAD_REPLAY.CREATE_FILTER_SET (
                           replay_dir => 'DB_REPLAY',
                           filter_set => 'replayfilters',
                           default_action => 'INCLUDE');
END;
/


BEGIN
  DBMS_WORKLOAD_REPLAY.USE_FILTER_SET (filter_set => 'replayfilters');
END;
/
(5)評估需要多少客戶端來滿足REPLAY條件(在新的測試環境下)
注:為了更準備的模擬測試環境,需要單獨準備一個裝有oracle client機器來提供所有WRC的連線;本案例評估需要97個客戶端
wrc mode=calibrate replaydir=/dbreplay/capture
Report for Workload in: /dbreplay/capture
-----------------------
Recommendation:
Consider using at least 97 clients divided among 25 CPU(s)  
You will need at least 135 MB of memory per client process.
If your machine(s) cannot match that number, consider using more clients.
Workload Characteristics:
- max concurrency: 3449 sessions
- total number of sessions: 286891
Assumptions:
- 1 client process per 50 concurrent sessions
- 4 client process per CPU
- 256 KB of memory cache per concurrent session
- think time scale = 100
- connect time scale = 100
- synchronization = TRUE

(6)開啟客戶端連線
   由於本案例是RAC環境,本次wrc需要通過SCAN TNS連線到測試庫的SCAN 監聽,再由SCAN監聽來根據DBA_REPLAY_CONNECTION_MAP檢視來決定客戶端資訊要連線到哪個例項;
 6.1 在測試庫環境下配置SCAN 監聽,確保SCAN監聽可用:
 參考文件11.2 Scan and Node TNS Listener Setup Examples (文件 ID 1070607.1)
 6.2 測試庫客戶端
 配置TNS:
  toyy3_scan =(DESCRIPTION =(ADDRESS_LIST =(ADDRESS =(PROTOCOL= TCP)(HOST= SCAN-IP)(PORT=1521)))(CONNECT_DATA =(SERVICE_NAME = orayy3)))  
 
 6.3 開啟客戶端連線
注:修改作業系統時間為捕獲時間
Before we can start the replay, we need to calibrate and start a replay client using the "wrc" utility. The calibration step tells us the number of replay clients and hosts necessary to faithfully replay the workload.
如果測試是RAC環境需要使用SCAN TNS連線,來根據dba_replay_connection_map檢視裡的會話資訊來連線資料庫例項
$nohup  wrc system/system#123@toyy3_scan mode=replay replaydir=/dbreplay/capture &
$nohup  wrc system/system#123@toyy3_scan mode=replay replaydir=/dbreplay/capture &



(7).啟動、暫停、繼續、取消重演,當完成負載記錄檔案預處理、重演初始化、設定重演選項、啟動重演客戶端以後,就可以開始負載重演。例如,下例演示了啟動、暫停、繼續、取消重演:
SQL> conn /as sysdba;
SQL> BEGIN
  DBMS_WORKLOAD_REPLAY.START_REPLAY ();
END;
/


BEGIN
  DBMS_WORKLOAD_REPLAY.PAUSE_REPLAY ();
END;
/


BEGIN
  DBMS_WORKLOAD_REPLAY.RESUME_REPLAY ();
END;
/


BEGIN
  DBMS_WORKLOAD_REPLAY.CANCEL_REPLAY ();
END;
/

(8)診斷DB REPLAY慢的指令碼=============
SCRIPT


Query to Obtain Information on Blocking Sessions 


--------------------------------------
--              Query 1             --
--------------------------------------


connect sys/ as sysdba 


col inst_id format 99999 
col sid format 99999 
col spid format a6 
col blocking_session_status format a6 heading 'BS' 
col blocking_instance format 99 heading 'BI' 
col blocking_session format 99999 heading 'BLKSID' 
col session_type format a11 
col event format a31 
col file_name format a21 
col file_id format 9999999999999999999 
col call_counter format 9999999 
col wait_for_scn format 99999999999999 heading 'WAITING FOR' 
col wfscn format 99999999999999 heading 'WFSCN' 
col commit_wait_scn format 99999999999999 heading 'CWSCN' 
col post_commit_scn format 99999999999999 heading 'PCSCN' 
col clock format 99999999999999999999 heading 'CLOCK' 
col next_ticker format 999999999999999999999 heading 'NEXT TICKER' 


select wrt.inst_id, wrt.sid, wrt.serial#, wrt.spid, 
s.BLOCKING_SESSION_STATUS, s.BLOCKING_INSTANCE, 
s.blocking_session, 
wrt.session_type, wrt.event, 
wrt.file_name, wrt.file_id, wrt.call_counter, 
wrt.wait_for_scn, 
greatest(wrt.dependent_scn, wrt.statement_scn) as wfscn, 
wrt.commit_wait_scn, wrt.post_commit_scn, 
wrt.clock, wrt.next_ticker 
from gv$workload_replay_thread wrt, gv$session s 
where wrt.sid = s.sid 
and wrt.serial# = s.serial# 
order by inst_id, sid 
;set linesize 500 pagesize 200 




 


Query to Monitor the Approximate Progress of the Replay


--------------------------------------
--              Query 2             --
--------------------------------------


set echo off 


connect sys/ as sysdba 


set serveroutput on 


DECLARE 
my_next_ticker NUMBER; 
clock NUMBER; 
wait_for_scn NUMBER; 
counts NUMBER; 
replay_id NUMBER; 
thr_failure NUMBER; 
start_time DATE; 
num_tickers NUMBER; 
min_scn NUMBER; 
max_scn NUMBER; 
done NUMBER; 
total_time INTERVAL DAY TO SECOND; 


CURSOR get_next_ticker(my_next_ticker NUMBER) IS 
SELECT spid, event, inst_id, wrc_id, client_pid 
FROM gv$workload_replay_thread 
WHERE file_id = my_next_ticker; 


BEGIN 
dbms_output.put_line('********************************'); 
dbms_output.put_line('* Replay Status Report *'); 
dbms_output.put_line('********************************'); 


----------------------------------------- 
-- Make sure that a replay is in progress 
----------------------------------------- 
SELECT count(*) INTO counts 
FROM dba_workload_replays 
WHERE status='IN PROGRESS'; 


if (counts = 0) then 
dbms_output.put_line('No replay in progress!'); 
return; 
end if; 


------------------- 
-- Get replay state 
------------------- 
SELECT id,start_time INTO replay_id, start_time 
FROM dba_workload_replays 
WHERE status='IN PROGRESS'; 


SELECT count(*) INTO counts 
FROM gv$workload_replay_thread 
WHERE session_type = 'REPLAY'; 


SELECT min(wait_for_scn), max(next_ticker), max(clock) 
INTO wait_for_scn, my_next_ticker, clock 
FROM v$workload_replay_thread 
WHERE wait_for_scn <> 0 
AND session_type = 'REPLAY'; 


dbms_output.put_line('Replay has been running for: ' || 
to_char(systimestamp - start_time)); 
dbms_output.put_line('Current clock is: ' || clock); 
dbms_output.put_line('Replay is waiting on clock: ' || 
wait_for_scn); 
dbms_output.put_line(counts || ' threads are currently being 
replayed.'); 


---------------------------------------- 
-- Find info about the next clock ticker 
---------------------------------------- 
num_tickers := 0; 
for rec in get_next_ticker(my_next_ticker) loop 
-- We only want the next clock ticker 
num_tickers := num_tickers + 1; 
exit when num_tickers > 1; 


dbms_output.put_line('Next ticker is process ' || rec.spid || 
' (' || rec.wrc_id || ',' || rec.client_pid || 
') in instance ' || rec.inst_id || 
' and is waiting on '); 
dbms_output.put_line(' ' || rec.event); 


end loop; 


--------------------------------------------------------------------------------------- 
-- Compute the replay progression and estimate the time left 
-- Note: This is an estimated time only, not an absolute value as it is based on SCN.
--------------------------------------------------------------------------------------- 
SELECT min(post_commit_scn), max(post_commit_scn) 
INTO min_scn,max_scn 
FROM wrr$_replay_scn_order; 
done := (clock - min_scn) / (max_scn - min_scn); 
total_time := (systimestamp - start_time) / done; 
dbms_output.put_line('Estimated progression in replay: ' || 
to_char(100*done, '00') || '% done.'); 
dbms_output.put_line('Estimated time before completion: ' || 
((1 - done) * total_time)); 
dbms_output.put_line('Estimated total time for replay: ' || 
total_time); 
dbms_output.put_line('Estimated final time for replay: ' || 
to_char(start_time + total_time, 
'DD-MON-YY HH24:MI:SS')); 
END; 
/


 


Query to Provide Summary of Wait Events


--------------------------------------------------------------
--                           Query 3                        --
--------------------------------------------------------------- 


column event format a40 
select event, count(*), min(wait_for_scn) 
from gv$workload_replay_thread 
where session_type = 'REPLAY' 
group by event;



(9).匯出測試環境資料庫的AWR資料,用於對負載捕捉期間和負載重演期間的資料庫效能做詳細比對。例如,下列演示了匯出某個負載重演作業ID對應的AWR資料:
SQL> conn /as sysdba;
SQL> BEGIN
  DBMS_WORKLOAD_REPLAY.EXPORT_AWR (replay_id => 1);
END;
/


(10).匯入負載捕獲期間的AWR資料,在測試環境資料庫中利用dbms_workload_repository.awr_diff_report_text或dbms_workload_repository.awr_diff_report_html對負載捕捉期間和負載重演期間的資料庫效能做詳細比對。例如:
SQL> conn /as sysdba;
SQL> select dbms_workload_capture.import_awr(capture_id => 1, staging_schema => 'AWRRPT') from dual; 
SQL>  -- get the capture data details 
SQL>  select id, awr_begin_snap, awr_end_snap from dba_workload_captures; 

SQL>  -- get the replay data details 
SQL>  select id, awr_begin_snap, awr_end_snap from dba_workload_replays;

(11) 監控重演
在重演中,任何錯誤,或者出現在生產環境和測試環境之間的資料不符,都會被記錄為分歧呼叫。獲取這些分歧呼叫的資訊,包括:SQL ID、SQL文字、繫結變數取值,都可以通過檢視DBA_WORKLOAD_REPLAY_DIVERGENCE,以及GET_DIVERGENT_STATEMENT獲取。例如:
--方法一
SQL> conn /as sysdba;
SQL> SELECT REPLAY_ID,STREAM_ID,CALL_COUNTER FROM DBA_WORKLOAD_REPLAY_DIVERGENCE;
--方法二
Sqlplus / as sysdba
set long 30000000 longchunksize 1000

DECLARE
r CLOB;
ls_stream_id NUMBER;
ls_call_counter NUMBER;
ls_sql_cd VARCHAR2(20);
ls_sql_err VARCHAR2(512);
CURSOR c IS
SELECT stream_id,call_counter
FROM DBA_WORKLOAD_REPLAY_DIVERGENCE
WHERE replay_id = ;
BEGIN
OPEN c;
LOOP
FETCH c INTO ls_stream_id, ls_call_counter;
EXIT when c%notfound;
DBMS_OUTPUT.PUT_LINE (ls_stream_id||''||ls_call_counter);
r:=DBMS_WORKLOAD_REPLAY.GET_DIVERGING_STATEMENT(replay_id => ,
stream_id => ls_stream_id, call_counter => ls_call_counter);
DBMS_OUTPUT.PUT_LINE (r);
END LOOP;
END;
/

同時,還可以生成負載重演報告,對重演總體情況進行概覽。例如:
SQL> conn /as sysdba;
SQL> select id,name,status,start_time,end_time,num_clients,user_calls,dir_path from dba_workload_replays where id = (select max(id) from dba_workload_replays) ;

SQL> set pagesize 0 long 30000000 longchunksize 2000 
SQL> select dbms_workload_replay.report(replay_id => 1,format => 'TEXT') from dual;

此外,還可以通過如下檢視獲取負載捕捉和負載重演期間的更多資訊:
1. DBA_WORKLOAD_CAPTURES列出所有捕獲到的負載;
2. DBA_WORKLOAD_FILTERS列出所有捕獲的負載定義的過濾器;
3. DBA_WORKLOAD_REPLAYS列出所有已經重演過的負載; view lists all the workload replays that have been replayed in the current database.
4. DBA_WORKLOAD_REPLAY_DIVERGENCE列出所有分歧呼叫,包括replay identifier, stream identifier, 以及call counter.
5. DBA_WORKLOAD_REPLAY_FILTER_SET列出所有重演的負載定義的過濾器;
6. DBA_WORKLOAD_CONNECTION_MAP列出所有重演使用的連線串的對映資訊;
7. V$WORKLOAD_REPLAY_THREAD列出當前所有重演客戶端的會話資訊;











來自 “ ITPUB部落格 ” ,連結:http://blog.itpub.net/29446986/viewspace-1650269/,如需轉載,請註明出處,否則將追究法律責任。

相關文章