SOLARIS: Asynchronous I/O (AIO) on Solaris (SPARC) servers (Doc ID 48769.1)

Asynchronous I/O (AIO) on Solaris (SPARC) servers
=================================================

Note that this article discusses normal asynchronous I/O (AIO) which is
not the same as kernel asynchronous I/O (KAIO).

It is documented in (at least) "Oracle for Sun Performance Tuning Tips,
A25584-1" that asynchronous I/O (AIO) is enabled by default on Solaris, and is
supported for database files created on BOTH raw devices and filesystem files.

This is TRUE, however it's not clear how this actually works (especially as
some other UNIX ports cannot support AIO on filesystem files). Also, output
from "truss" can be misleading, and suggest that AIO is not working on
filesystem files. See bug:532270 for an example of just such confusion.

This document attempts to explain the operation of AIO on Solaris version 2.4
and above.

1. Enabling AIO within Oracle
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Oracle 7 can use asynchronous writes for parallel database writes by DBWR.
This feature is turned ON by default (i.e. the init.ora parameter async_write =
TRUE).

Oracle 7 can use asynchronous reads (specifically for database recovery). This
feature is also turned ON by default (i.e. the init.ora parameter asuync_read =
TRUE).

2. AIO functions used by Oracle
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Assuming AIO is enabled within Oracle (see above), at relevent points, it will
use the libaio functions aioread() and aiowrite() (as opposed to the normal,
synchronous I/O routines read(), pread(), write(), pwrite() etc.).

This is irrespective of whether the file is a raw device, or a filesystem file
(the Solaris kernel manages the differences).

3. How the Solaris kernel reacts
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The use of the aioread() and aiowrite() functions result in calls to the kaio()
system call, with arguments of AIOREAD and AIOWRITE etc.

When the file being operated on is a raw device, these calls succeed and invoke
the kernalised AIO driver.

When the file being operated on is a filesystem file, the first call to
kaio() (with an argument of AIOREAD, or AIOWRITE etc.) will fail with error #48
(Operation not supported). As a result, the normal synchronous IO system calls
pread() and write() are called instead.

This behaviour can cleary be seen in a truss, and is the cause of most
confusion as it suggests AIO is NOT being used on the filesystem file.

4. Truss extract from asynchronously reading a raw device
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

28476/1:        open("/dev/rdsk/c0t3d0s1", O_RDONLY)            = 3
28476/1:        kaio(5, 0xFFFFFFE8, 0xFFFFFFFF, 0x00000000, 0x00000000, 0x000000
00, 0xEF7F4298) = 0
28476/1:        kaio(AIOREAD, 3, 0x00020FF0, 1024, 0, 0xEFFFF9F4) = 0
28476/1:        kaio(AIOWAIT, 0x00000000)                       = -268437004
28476/1:        kaio(AIOREAD, 3, 0x00020FF0, 1024, 1024, 0xEFFFF9F4) = 0
28476/1:        kaio(AIOWAIT, 0x00000000)                       = -268437004
28476/1:        kaio(AIOREAD, 3, 0x00020FF0, 1024, 2048, 0xEFFFF9F4) = 0
28476/1:        kaio(AIOWAIT, 0x00000000)                       = -268437004

5. Truss extract from asynchronously reading a filesystem file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

28463/1:        open("/etc/passwd", O_RDONLY)                   = 3
28463/1:        kaio(5, 0xFFFFFFE8, 0xFFFFFFFF, 0x00000000, 0x00000000, 0x000000
00, 0xEF7F4298) = 0
28463/1:        kaio(AIOREAD, 3, 0x00020FF0, 1024, 0, 0xEFFFF9FC) Err#48
28463/1:        lwp_create(0xEFFFF628, 0, 0xEF781F44)           = 3
28463/3:        lwp_create(0x00000000, 0, 0x00000000)           = -277341360
28463/3:        lwp_self()                                      = 3
28463/1:        lwp_create(0xEFFFF628, 0, 0xEF778F44)           = 4
28463/4:        lwp_create(0x00000000, 0, 0x00000000)           = -277378224
28463/4:        lwp_self()                                      = 4
28463/1:        lwp_create(0xEFFFF628, 0, 0xEF6BAF44)           = 5
28463/5:        lwp_create(0x00000000, 0, 0x00000000)           = -278156464
28463/5:        lwp_self()                                      = 5
28463/1:        lwp_create(0xEFFFF628, 0, 0xEF6B1F44)           = 6
28463/6:        lwp_create(0x00000000, 0, 0x00000000)           = -278193328
28463/6:        lwp_self()                                      = 6
28463/3:        pread(3, " r o o t : x : 0 : 1 : 0".., 1024, 0) = 1024
28463/4:        pread(3, " r y h a : x : 2 1 6 : 1".., 1024, 1024) = 1024
28463/5:        pread(3, " s 1 / j c a h i l l : /".., 1024, 2048) = 1024
28463/6:        pread(3, " s h\n s f a x : x : 5 0".., 1024, 3072) = 115

6. So how is AIO being implemented for the filesystem file?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This is the clever bit. The KAIO kernel driver cannot operate on a filesystem
file, so as a result of the first kaio(AIOREAD) failing, the aioread() function
spawns 4 lightweight processes (threads). Each of these threads is responsible
for reading a different section of the file, but in parallel; thereby SIMULATING
asynchronous I/O.

This can be seen in the second truss extract (the thread number follows the / in
the process number).

7. Implications of this behaviour
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Binding Oracle processes (specifically DBWR or server processes) to a CPU
when AIO is in use on filesystem files will impact performance, since all the
spawned threads will also be bound to this same CPU, and therefore be unable to
run in parallel.

SOLARIS: Asynchronous I/O (AIO) on Solaris (SPARC) servers_48769.1

Related

Products

相關文章