In this Document

Applies to:

Purpose

The purpose of this note is to list some of the common errors and its reasons that a user may get during Database Capture and Replay process. 

Troubleshooting Steps

ORA-00439  "feature not enabled: %s"

Solution

Partitioning is not required for RAT. Please see Bug 9528307. The following workaround is available:

Or, if the session in which the processing is done cannot be targeted (because of EM for instance), the event can be set system-wide for the duration of the processing (and turned off afterward).

alert.log may show following message

This is an informational message, this message does not indicate a problem.

1) This is because even after timeout ( default 30 seconds )
    We did not flush the capture buffers.

or

2) We had idle sessions at the time we issued DBMS_WORKLOAD_CAPTURE.FINISH_CAPTURE().
     for example sqlplus connection connected to the database, but doing nothing.

Solution

1) To correct this specify high time out value like 60 seconds or 120 seconds or higher. You may still have this message even after increase timeout if there are idle sessions at the end of the capture.

2) To avoid this, do not have sessions that remain idle (waiting for the next user request) while invoking FINISH_CAPTURE. Either close the database session(s) before running FINISH_CAPTURE or send new database requests to those sessions during FINISH_CAPTURE.

3) You can safely ignore this information in most cases assuming you have captured enough workload.

Check Oracle documentation on DBMS_WORKLOAD_CAPTURE.FINISH_CAPTURE()


While doing the Database Replay one may get the following alert

A Replay client is not able to issue the captured workload at the requested rate


Database Replay monitors the WRC client statistics for i/o and cpu usage.
If the time spent by WRC clients is the above the set threshold values,
it sends an alert to the database server and the status metrics carry the status to EM and issues alert.

The following query gives the different alerts available for Database Replay clients.

AVG_IO_TIME average IO latency for Workload Replay Client threshold is 60 ms.
P_CPU percentage of threads on CPU for Workload Replay Client threshold is 3%.
P_IO percentage of threads doing IOs for Workload Replay Client threshold is 3%.

The idea here is WRC itself should not be spending much time in reading the capture files or CPU.
The alert means either the storage subsystem on which the capture files are located is slow or have bottlenecks or less throughput,
or you are executing too many WRC clients from the same machine.

Solution

1) Move capture files on storage subsystem which is fast or do not have bottleneck or have better throughput.
2) Execute WRC clients from more machines.

ORA-01458 "invalid length inside variable character string"

If you are getting ORA-01458 during get_replay_info or Initialize or prepare phase

WORKAROUND

Use alter system set "_wcr_control"=16384 before executing get_replay_info or do Initialize or prepare phase.

Solution

Apply patch for  bug 10076544.
Or  apply Patch 11870615 DBREPLAY PATCH BUNDLE 1 FOR 11.2.0.2.0 that  includes fix for bug 10076544.

ORA-15505  "cannot start workload capture because instance %s encountered errors while accessing directory %s"

 Example of ORA-15505

 
Reasons for ORA-15505

1) There is an existing file in the CAPTURE Directory. 
    There should be no files including no hidden files,
    Before the dbms_workload_capture.start_capture is executed.

2) The Instance is unable to access the CAPTURE Directory.

3)The Capture directory can not be on an ASM disk.
  In 11gR1 and 11gR2, DBMS_WORKLOAD_CAPTURE can not be used when the capture directory is on an ASM disk.  Currently, there is no exact fix release.

ORA-15506 "cannot prepare instance %s for replay"

1) Ensure proper shared directory during replay on a RAC database.
    Non-shared directories are not supported during replay on a RAC database.

2) Unpublished Bug 7260616
    Fixed in 11.2.0.1.0 and above.

3) ORA-15506 is one of the symptoms of bug 10327241.
    For 11.2 Database release apply one off patch for Bug 10327241
    Or apply one off patch for Patch 11870615 DBREPLAY PATCH BUNDLE 1 FOR 11.2.0.2.0 that includes includes fix for Bug 10327241
    Ensure to do preprocessing again after applying this patch.  

4) For 10.2.0.5.6, apply patch for bug 9373986 as per Document 560977.1.

ORA-15509 "workload replay has been cancelled"

On RAC databases:

For database replay you prepared for RAC database with capture_sts => TRUE

Then get following error:

Reason for this particular ORA-15509

The use of CAPTURE_STS=TRUE on RAC database has resulted in ORA-15509.

Solution

Do not use capture_sts => TRUE, use capture_sts => FALSE. capture_sts => TRUE which is not supported on RAC databases.

Another reason for ora-15509 may be bug 13043788.

Run workload replay again in debug mode to identify the bug:

Look for a trace file which contains similar data:

Solution

Request a patch for bug:13043788.

ORA-15510 "cannot perform operation when "STATISTICS_LEVEL is "BASIC" "

Example of  ORA-15510

In 9.2.0.X.0

And in the alert log you may see:

In 10.2.0.X

Reasons for ORA-15510

STATISTICS_LEVEL set to BASIC.

Solution

set STATISTICS_LEVEL=TYPICAL in the init.ora or dynamically.

For example 

ORA-15511 "cannot process workload capture because no user sessions were recorded"

Example of  ORA-15511

Reason for ORA-15511

1) No user sessions were recorded in the given workload capture.

ORA-15513 "cannot access the input directory"

During Initialize of Replay one gets an ORA-15513 error

Workaround

In the Replay directory check for directory rep*

for example

backup directory rep226671820 to some other directory and delete it. The rep* directory is generated during each Replay.deleting the rep* directory from previous replays will not impact the new replay.

or

and do the Initialize again.

Solution

Apply patch for bug 10076544.

ORA-15518 "parallel preprocessing encountered an unexpected error"

raises the following error:

along with  ORA 7445  kecppSeqEndCallProcessing()

Solution

Apply one off Patch 11691834.
  Or  apply Patch 11870615 DBREPLAY PATCH BUNDLE 1 FOR 11.2.0.2.0 that  includes these fixes .

ORA-15552 "workload replay client cannot login to database server"

The error ORA-15552 means workload replay client cannot login to database server.

Reasons for ORA-15552

1) The replay client used an invalid username or password;
2) The replay user does not have the privilege to switch user;
3) The recorded username does not exist in the replay database.
4) Incorrect format for wrc client used.

No password specified. The correct method is

ORA-15555 "workload replay client encountered unexpected error: %s

ORA-15555 "THE DIAGNOSABILITY CONTEXT CAN'T BE INITIALIZED"

Reason for ORA-15555

1) ADR setup issue.
2) DIAG_ADR_ENABLED=OFF set in sqlnet.ora. DIAG_ADR_ENABLED=ON should be set in sqlnet.ora.

ORA-15557 "workload replay client cannot access the replay directory or the database version do not match the preprocessing one"

Reasons for ORA-15557

1) Check the Replay Directory That was specified, make sure that the WRC client has an access to the directory.
2) The workload  was preprocessed using a different version than the version of the database it is replayed on currently.

ORA-15558 "replay thread encountered unexpected error"

While running the WRC client while doing the replay WRC got an error ORA-15558

ORA-15558 simply means that the replay thread lost connection to the database server; Error ORA-15558 is too generic.
ORA-15558 can occur for login failure of  user during replay. The workload replay client throws ORA-15558 when a user login fails during replay.

ORA-15563: workload replay client cannot spawn new threads

A possible reason for this error is the number of concurrent threads being run during replay.
Check the process limit (ulimit -u) and increase the soft limit  for the user running the replay by increasing the nproc limit in /etc/security/limits.conf

ORA-15567  "replay user %s encountered an error during a sanity check"

This error means that the user has insufficient privileges to replay the workload. To resolve the error, give the user the correct privileges or grant them DBA privilege and grant admin option in order to replay the workload.

ORA-15590 "encountered incomplete workload capture files"

 During preprocessing you encountered ORA-15590

This error basically informs the user that part of the workload may be missing.

Check the alert.log for specific output like this:

Error message in alert file

The capture file wcr_aw95ch0000488.rec is incomplete.

Ignore the error message ORA-15590, it is for information.
The preprocess has completed.

I/O Error while capture:Captured Data may still be usable.

During Capture if you get I/O Error while capture:Captured Data may still be usable.

This means the capture stopped automatically. Usually this would mean they run out of space on disk after capture had started or out of memory when trying to allocate PGA buffers. 
Now it is debatable if the capture is still usable as wcr_fcaptrue.wmd may not be there as  we ran out of disk space and capture may not have written file wcr_fcaptrue.wmd.

For Slow Preprocess issues

Apply one off Patch 11691834.
Or  apply Patch 11870615 DBREPLAY PATCH BUNDLE 1 FOR 11.2.0.2.0 that  includes fix for Bug 11691834.

AWR snapshots with error

While generating the Compare Period Report one may get the following message.

There were 2 AWR snapshots with error during Replay wrr-2011 with id 1

AWR had some problem flushing data and recorded the error(s) in dba_hist_snap_error (wrm$_snap_error).
Check and save the table_name and error_number in there:

This will return rows.
This is why the error is generated and that is what prevented us from generating the report.

WORKAROUND

Note depending upon the table involved in the error
you may miss having that information in AWR Reports
and Replay reports. If you delete those rows in wrm$_snap_error, the report will be generated.

ORA-20222 "Given capture_id %s is invalid or the corresponding capture failed"

Example of ORA-20222

1) 

2)

Reason for ORA-20222

ORA-20222 is a generic error message that can occur due other underlying error during Capture.
Check the error accompanying with ORA-20222.

• Ensure correct directory name is used including check the case-sensitivity of the directory name.The directory name is case sensitive.
• Ensure READ and WRITE privilege on the DIRECTORY object to the which is the capture is written. 

ORA-20223 "No replay clients have connected yet! Please issue START_REPLAY() after one or more WRC replay clients have been started!"

Example of ORA-20223

1)

2)

3)

A Database Replay is already in progress in the database.
You can not start multiple Database Replays at the same time on the same database.

4)

5)

6)

7)  This error can occur in RAC environment:

Replay on RAC using local replay directory fails even though mappings are all to the one node.

Failure is due to insert being parallel and slaves starting on remote node and hence cannot see replay directory.

Solution: set parallel_force_local = true or Use a shared directory instead of a local directory.

 Reason for ORA-20223

ORA-20223 is a generic error message that can occur due other underlying error during replay.
Check the error accompanying with ORA-20223.

ORA-39087 "directory name %s is invalid"

When attempting to implement a Grid Control AWR export after capturing a replay, the following error occurs:

Solution

Solution is the following:

ORA-21500: internal error code, arguments:[keccuCheckOCIError]

When running workload replay, the following error is dumped:

Solution is to apply the fix for Bug 13399236 which is fixed in 12.1.  If there is no patch available for your version, please request a one off patch be produced. See:

Timeout is called

1) The following message is in the WRC trace:

This is considered a normal message but is indicative of an issue likely unrelated to WRC. To further diagnose the issue, run the scripts from following note:

2) There is another timeout message that is considered normal but received due to the high capture process time:

For example, if the capture process is taken for days,the replay will take longer period of time and may time out.  In looking at the Replay report, it may be seen that processes are still running, but progress is slow due to the size of the capture.

It is recommended that Database Replay is started with a smaller duration of capture. As a thumb rule, it is recommend that Database Replay is one hour capture of representative workload. Do not jump directly to a larger duration capture like 12 hour or 24 hour. Once a one hour Database Replay run has worked well to satisfaction, one can move towards a larger duration of Database Replay. 

To find the offending workload capture file, see:

Please check the following document for a list of available patches for different releases:

References