Direct I/O (DIO) and Concurrent I/O (CIO) on AIX 5L

...........[@more@]

Subject:　 Direct I/O (DIO) and Concurrent I/O (CIO) on AIX 5L
Doc ID:　 Note:257338.1 Type:　 BULLETIN
　Last Revision Date:　 13-SEP-2005 Status:　 PUBLISHED

PURPOSE
-------
To discuss the use of Direct I/O (DIO) and Concurrent I/O (CIO) on AIX 5L
　
SCOPE & APPLICATION
-------------------
System Administrators managing AIX 5L that has Oracle RDBMS on top.
DIRECT I/O (DIO) AND CONCURRENT I/O (CIO) on AIX
------------------------------------------------
If you use a Journaled File System, it is easier to manage and maintain database
files than it is if you use raw devices. In prior versions of AIX, file systems
used to support only buffered read and write and added extra contention due to
undesirable inode locking. These two issues are solved by JFS2's Concurrent I/O
and GPFS' Direct I/O, allowing file systems to be used in place of raw devices
even when optimal performance is required.
AIX 5L includes Direct I/O support on JFS, JFS2 and GPFS. In addition, AIX 5L
version 5.2 and later includes Concurrent I/O support. Direct I/O and Concurrent
I/O support allows database files to exist on file systems while bypassing the
operating system buffer cache and removing inode locking operations that are
redundant with functionality provided by Oracle Database 9i. Where possible,
Oracle recommends isolating Oracle data files on their own dedicated file
system(s) on which the Concurrent I/O or Direct I/O option is enabled.
The following table lists file systems available on AIX and the recommended
setting:
　 File System　　　　　　　 Option　　　　　　　 Descrīption
　 -----------　　　　　　　 ------　　　　　　　 -----------
　 JFS　　　　　　　　 dio　　　　　　　　　　 Concurrent I/O is not available on JFS.
　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 Direct I/O (dio) is available, but performance
　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 is degraded compared to JFS2 with Concurrent I/O.
　 JFS large file　　　none　　　　　　　　　 Oracle does not recommend using JFS big file for
　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 Oracle Database because it's 128 KB alignment
　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 constraint prevents you from using Direct I/O.
　 JFS2　　　　　　　　cio　　　　　　　　　　 Concurrent I/O (cio) is a better setting than
　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 Direct I/O (dio) on JFS2 because it has support
　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 for multiple concurrent readers/multiple
　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 concurrent writers on the same file.
　 GPFS　　　　　　　　-　　　　　　　　　　　　 Oracle Database 9i release 2 and later silently
　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 enables Direct I/O on GPFS for optimum
　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 performance. GPFS already supports multiple
　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 reader/multiple writers/multiple nodes.
　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 Therefore, Direct I/O and Concurrent I/O are the
　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　 same thing on GPFS.
If GPFS is used, the same file system can be used for all purposes including
$ORACLE_HOME, datafiles, logs. A large GPFS block size (typically at least 512KB)
should be used for optimal performance. GPFS is designed for scalability and there
is no requirement to create multiple GPFS file systems as long as the amount of
data fits in a single GPFS file system.
If placing the Oracle Database's logs on a JFS2 filesystem, the optimal configuration
is to create the file system with option "agblksize=512" and to mount it with option
"cio". This delivers logging performance similar, within a few percents, to the
performance of a raw device.
In Oracle Database 9i, Direct I/O and/or Concurrent I/O can only be enabled at
filesystem level on JFS/JFS2 and therefore $ORACLE_HOME and datafiles have to be
placed in separate filesystems for optimal performance: $ORACLE_HOME would reside on
a file system mounted with default options, whereas datafiles and logs would reside
on file systems mounted with the "dio" or "cio" option.
Having on ORACLE_HOME on a filesystem mounted with "cio" option is not supported.
Such a configuration will cause, installation, relinking and other unexpected
problems.