ORA-8103 "object no longer exists" 診斷與解決

In this Document

Applies to:

Purpose

This article provides information about error ORA-8103 and possible actions.

Scope

This note is intended for general audience as initial starting point for beginning diagnosis of ORA-8103.

Details

Description

ORA-8103 is reporting that a SQL statement found a block that no longer belongs to the object referenced in the statement. 

Cause

ORA-8103 is caused by an invalid block type. The block header has an invalid block type or the block type inside the block is not expected; e.g. a data block (Type=6) was expected but the actual block information is not a data block (Type!=6).

ORA-8103 is also caused by an unexpected data_object_id where it is changing for the involved objects while the affected SQL statement is executed.

These two causes might be due to an expected behavior. or other problems.  Details are:

Expected behavior.

Tables are being dropped/truncated while a SQL statement for those tables is still in execution. In the case of an index, it might be caused by an index rebuild. In other words the object has been deleted by another session since the operation began.

Look if dba_objects.data_object_id is changing for the affected object while queries are being executed.

data_object_id is changed by DDL statements like:

truncate table
alter index .. rebuild
alter table .. move
etc.

For a truncate look for column TRUNCATED in DBA_TAB_MODIFICATIONS. Note that it indicates whether the table has been truncated since the last analyze.  See documentation.

In 11g parameter enable_ddl_logging can be set to TRUE to print DDL statements in the alert log and identify what DDL's are run that may potentially cause this error.

Block Corruption.

Hardware, IO subsystem or OS problems may cause block corruptions overwriting the Block Type in the block header causing the error ORA-8103. 

The block is healthy but it is an "OLD/STALE" block. These kind of corruptions might be caused by LOST IO/LOST WRITE or a bug in external non-oracle tools that migrate file systems while the database is OPEN.   In that case the data_object_id for the affected object could have changed but the actual block does not reflect it.

Note that the block may also be temporarily corrupted in the buffer cache (SGA Memory).

Overlapped Extents.

Overlapped Extents means that two or more segments incorrectly use the same block. This is a LMT Bitmap Extent inconsistency or Data Dictionary inconsistency.

To identify overlapped extents run the next checks:

For LMT (Locally Managed Tablespaces) run :

sqlplus / as sysdba

execute dbms_space_admin.tablespace_verify('&tablespace_name')
oradebug setmypid
oradebug tracefile_name

In 10gR2 and forward, for locally managed tablespaces using ASSM (DBA_TABLESPACES.SEGMENT_SPACE_MANAGEMENT=AUTO) run:

execute dbms_space_admin.assm_tablespace_verify('&tablespace_name',dbms_space_admin.TS_VERIFY_BITMAPS)
oradebug setmypid
oradebug tracefile_name

Note that the session running dbms_space_admin does not say in the screen if an inconsistency is detected.  dbms_space_admin will generate a trace file instead.   So check if "oradebug tracefile_name" provides a trace file.

For Data Dictionary managed tablespaces run the hcheck script. in: 

Note 136697.1 (it checks for overlapped extents through fet$/uet$).

Overlapped ASM extent.

If using ASM: overlapped ASM extents is when 2 different files and 2 different extents use the same Allocation Unit (AU) in the same ASM disk. 

From the ASM instance run:

SQL> alter diskgroup check all norepair;

In ASM alert log the next message is registered:

ERROR: file +data1.3551.721223631: F3551 PX3819 => D254 A78 => F3564 PX337: fnum mismatch

Meaning that ASM file number 3551 Physical Extent 3819 is using Allocation unit 78 in Disk 254 and at the same time ASM File Number 3564 has Physical Extent pointing to the same Allocation unit.

Oracle defect.

It can be an Oracle bug.  See section "known issues" below.

 

Identify the affected object

Sometimes the SQL statement producing the ORA-8103 involves several tables and the affected object can be an index.  Follow the next procedures to identify the affected object:

Get the SQL statement that is causing the error.

Reproduce the error from SQL*Plus if possible by running the affected SQL statement.

Identify the affected object with ANALYZE.

Is it a TABLE or an INDEX causing the error?:

For a TABLE run:

analyze table validate structure;

Alternatively  identify if a full table scan is producing the error.

For an INDEX run:

analyze index validate structure;

Identify the object with event 10236.

In 9.2.0.8 or above if ktrget is in the call stack trace for an ORA-8103, setting Event 10236 will include the block number causing the error:

alter session set events '8103 trace name errorstack level 3';
alter session set events '10236 trace name context forever, level 1';
alter session set tracefile_identifier='ORA8103';

run the query that produces the error ORA-8103 and identify the trace with the form. of _ora__ORA8103.trc

Open the trace file and if function ktrget is in the call stack trace, then the trace file may have:

KTRVAC: path typ=22, rdba=32810e06

Meaning that the error is produced in rdba=0x32810e06 . Convert that number to decimal and get the relative_fno, block#:

select dbms_utility.data_block_address_file(847318534) relative_fno
,dbms_utility.data_block_address_block(847318534) block#
from dual;

RELATIVE_FNO BLOCK#
------------ ----------
         202 69126

Then use Note <> to identify the object; section "Identify the Corrupt Object".

Identify the object with event 10200.

Event 10200 can be used to identify the last accessed block when this error is produced. event trace_buffer_on can be used to generate a smaller trace file.  Errorstack and event 10236 can be added to get a more complete trace file:

alter session set max_dump_file_size=unlimited;
alter session set db_file_multiblock_read_count=1;
alter session set events 'immediate trace name trace_buffer_on level 1048576';
alter session set events '10200 trace name context forever, level 1';
alter session set events '8103 trace name errorstack level 3';
alter session set events '10236 trace name context forever, level 1';
alter session set tracefile_identifier='ORA8103';

run the query that produces the error ORA-8103

alter session set events 'immediate trace name trace_buffer_off';
exit

Identify the trace with the form. of _ora__ORA8103.trc

In the trace file generated locate last information about:

"Consistent read started for block : "

where XXX is the tablespace number and YYY is the rdba for that block.  Normally this is the affected block.

Trace output example:

*** 2004-01-28 09:40:16.000
*** SESSION ID:(9.5) 2004-01-28 09:40:16.000
Consistent read started for block 0 : 00405302
env: (scn: 0x0000.001a9434 xid: 0x0000.000.00000000 uba: 0x00000000.0000.00 statement num=0 parent xid: xid: 0x0000.000.00000000 scn: 0x0000.00000000 0sch: scn: 0x0000.00000000)
CR exa ret 9 on: 01FEE950 scn: 0xffff.ffffffff xid: 0x0000.000.00000000 uba: 0x00000000.0000.00 scn: 0xffff.ffffffff sfl: 0
Consistent read finished for block 0 : 405302
Consistent read finished for block 0 : 405302
Consistent read started for block 0 : 00405303

The error is produced while reading rdba=0x00405303 (File=1,Block=21251) as there is NOT a message like:
"Consistent read finished for block 0 : 00405303"

Then use Note <> to identify the object; section "Identify the Corrupt Object".

Identify object affected by Block Corruption.

Use DBVerify or RMAN to identify corrupted blocks associated to the datafiles where the object is stored. 

A common corruption is when the block has been zeroed out (block type is also zero). DBVerify reports it as:

Completely zero block found during dbv

Use sections "DBVerify - Identify Datafile Block Corruptions" or "RMAN - Identify Datafile Block Corruptions" in Note 836658.1 and Note 819533.1 to identify the affected object.

Identify object affected by Overlapped Extents.

If overlapped extents in Locally Managed Tablespace is identified, use Note 887263.1

@ Using the call stack trace arguments to identify the block producing the ORA-8103.

Solution

Fix Block Corruption.

If error ORA-8103 is constantly reproduced by ANALYZE and if it has been determined that it is not the expected behavior, then it means that there is a possible block corruption. The possible solutions are:

Flush the buffer cache.

It might be a corruption only in the SGA memory (Buffer cache):

alter session set events 'immediate trace name flush_cache level 1';

In a RAC system, flushing the buffer cache may be needed in the additional rac instances.

If error still persists it means the the block is invalid also in disk, so continue with the next steps:

Index Corruption.

Drop and recreate the index.

Table Corruption.

1. Consider to apply media recovery if it is determined that the problem was caused by a corrupt block (Like zeroed out blocks). RMAN BLOCKRECOVER can be used to repair a zeroed out block. 

2. TRUNCATE or DROP it and reload the data from export dump. If the the same error is produced by dropping/truncating a PARTITION, then consider to EXCHANGE:

alter table . exchange partition with table .;

Note: has to have the same structure as .

3. If recover from a backup is not an option, table data can be saved skipping the blocks that are causing the ORA-8103 error:

OPTION 1 - No backup

REM Create a new table based on the affected one with no rows:

create table  
as
select * 
from
where 1=2;

REM Create the table to keep track of ROWIDs pointing to affected rows:

create table bad_rows (row_id rowid
                      ,oracle_error_code number);
set serveroutput on

DECLARE
  TYPE RowIDTab IS TABLE OF ROWID INDEX BY BINARY_INTEGER;

  CURSOR c1 IS select /*+ index(tab1) */ rowid
  from tab1
  where is NOT NULL;

  r RowIDTab;
  rows NATURAL := 20000;
  bad_rows number := 0 ;
  errors number;
  error_code number;
  myrowid rowid;
BEGIN
  OPEN c1;
  LOOP
   FETCH c1 BULK COLLECT INTO r LIMIT rows;
   EXIT WHEN r.count=0;
   BEGIN
    FORALL i IN r.FIRST..r.LAST SAVE EXCEPTIONS
     insert into  
     select /*+ ROWID(A) */
     from A where rowid = r(i);
   EXCEPTION
   when OTHERS then
    BEGIN
     errors := SQL%BULK_EXCEPTIONS.COUNT;
     FOR err1 IN 1..errors LOOP
      error_code := SQL%BULK_EXCEPTIONS(err1).ERROR_CODE;
      if error_code in (1410, 8103) then
       myrowid := r(SQL%BULK_EXCEPTIONS(err1).ERROR_INDEX);
       bad_rows := bad_rows + 1;
       insert into bad_rows values(myrowid, error_code);
      else
       raise;
      end if;
     END LOOP;
    END;
   END;
   commit;
  END LOOP;
  commit;
  CLOSE c1;
  dbms_output.put_line('Total Bad Rows: '||bad_rows);
END;
/

Notes:

• The idea is to get the rowid's from the index, then get all the columns from the table for each rowid and insert these rows into another table. Using the above "index" hint, will allow the optimizer to choose the most appropriated index to scan the table based on the indexed column.

• Make sure that the the select in the plsql above is using an index. One way to verify if the index is used is to get an execution plan from sqlplus:

set autotrace trace explain
select /*+ index(tab1) */ rowid
from tab1
where is NOT NULL;

• Note that the plsql executes an INSERT for 20000 rows and COMMIT. If it required to change this, adjust the value of rows. e.g.:

rows NATURAL := 50000; -> to insert 50000 rows in one execution of INSERT and commit every 50000 records.

OPTION 2 - No backup

If the table does not have an index, use script. provided in Note 422547.1

Another solution is to determine if dbms_repair can be used to skip these blocks or if procedure described in Note 61685.1 can be used to skip corrupted blocks using rowid scans.

Temporary Segment Corruption

If it is identified that the ORA-8103 is caused by a temporary segment, use dbms_space_admin to drop the temporary segment and rebuild the tablespace bitmap:

sqlplus / as sysdba

execute dbms_space_admin.segment_corrupt('&tablespace_name',&relative_fno,&block_number)
execute dbms_space_admin.segment_drop_corrupt('&tablespace_name',&relative_fno,&block_number)
execute dbms_space_admin.tablespace_rebuild_bitmaps('&tablespace_name')

Fix Overlapped Extents.

If the error is caused by Overlapped extents in a LOCALLY MANAGED Tablespace (LMT), dropping one of the affected objects and rebuilding the tablespace bitmaps (dbms_space_admin.tablespace_rebuild_bitmaps) can fix the overlapped extent problem but will not fix the current affected blocks. For a DICTIONARY MANAGED tablespace determine if the database has to be recreated.