AIX下使用ASM

尛樣兒發表於2010-04-17
metalink note 282036.1
IBM Software Requirements and PTFs for AIX 5.3 support of Oracle Database 10g Release 2 (10.2.0.1 or higher)
 
The requirements for the software products certified on Oracle Database 10gR2 with AIX 5.3 are described in this section.
 
Use AIX 5.3 TL 02 or higher, make sure to include the fixes for the following APARs:
 
 IY68989: WRITE TO MMAPPED SPACE HANGS
 
 IY68874: An application that is run with mandatory large page data (LDR_CNTRL=LARGE_PAGE_DATA=M) may core-dump on the AIX V5.3 64-bit kernel in a LPAR environment.
 
 IY70031: CORRUPTION FROM SIMULTANEOUS CIO WRITES WITH O_DSYNC ON JFS2  If using the IBM Journal File System Version 2 (JFS2) for Oracle Database files.
 
NOTE: All Oracle 9i Database and Oracle 10g Database customers who are running on AIX 5.3 TL 05 must install the IBM AIX PTF for APAR IY89080.  In addition, Oracle customers should contact Oracle support to obtain the fix for Oracle Bug 5496862.
  
 IZ03260: LIO_LISTIO FAILS TO UPDATE AIO CONTROL BLOCKS ON ERROR APPLIES TO AIX 5300-06 (for AIX V5.3 TL06 customers ).
 
IZ03475: LIO_LISTIO FAILS TO UPDATE AIO CONTROL BLOCKS ON ERROR APPLIES TO AIX 5300-07 (for AIX V5.3 TL07 customers).
 

Basic System Setup
1. Install AIX 5.3 + latest maintenance level, and check metalink note 282036.1 for any additional
system prerequisites for Oracle
2. Verify the following filesets are installed, or install if not present:
?bos.adt.base
?bos.adt.lib
?bos.adt.libm
?bos.adt.syscalls
?bos.perf.libperfstat
?bos.perf.perfstat
?bos.perf.proctools
?bos.perf.gtools
?rsct.basic
?rsct.compat.basic
3. Create dba and oinstall groups with the same GID across all cluster nodes
4. Create oracle user with the same UID across all cluster nodes, primary group dba
5. set date and timezone (smit system)
6. start xntpd (smit xntpd)
7. implement tuning parameters from the Tuning Parameters and Settings for ASM section of this
document

AIX Parameters
The best practices for AIX system setup for an ASM environment are virtually the same as when
using raw logical volumes. The following initial settings are recommended for the operating
system, although environments do vary and additional tuning may be required.
1. System attributes (found in ‘smit system’)
The system maintains a parameter, maxuprocs, to identify the maximum number of user
processes. This parameter should be increased from the default to a number which will
exceed the number of oracle processes; typically a value of 4096 is sufficient.
2. Asynchronous I/O
AIX can use two types of asynchronous I/O, kernelized and threaded. All filesystem
implementations used threaded asynchronous I/O, and require the configuration of the
threaded asynchronous I/O subsystem through ‘smit aio’, typically increasing the number
of I/O servers (maximum # of servers) and the size of the request queue (maximum # of
requests). ASM, however, like raw, uses the kernelized asynchronous I/O subsystem,
which does not require configuration, although in order to install the Oracle software, the
‘STATE to be configured at system restart’ for asynchronous I/O threads must be set to
‘available’ from the ‘smit aio’ menu.
For Oracle to take advantage of asynchronous I/O, the spfile must include the
parameters ‘disk_asynch_io=TRUE and filesystemio_options=asynch’.
3. /etc/security/limits
This file sets process resource limits for users. Historically, gating resources with this
file has caused issues including inability to start CRS, inability to open database files, and
inability to create files greater than 2GB in size. It is highly recommended to set all
values other than core to unlimited (-1) for the oracle user.
4. Network Tunables (seen with no –a)
The size of the tcp send buffer (tcp_sendspace) dictates how much data the application
can send, and the tcp receive buffer (tcp_recvspace) indicates how much data can be
received before being read by the receiving application. Values which are too small
risk having application waits due to insufficient buffer space for tcp requests. The
default values for AIX are too small for most two or three tier application environments.
At a minimum, the following values are suggested:
- tcp_sendspace = 65536
- tcp_recvspace = 65536
These values may be insufficient for some environments, particularly where network
backups are implemented. In order to allow any sendspace or recvspace buffers to be
greater than 65536, the parameter ‘rfc1323=1’ must also be set.
When UDP over gigabit Ethernet is used as the cluster interconnect, the following
parameters should also be set, as indicated in the Oracle Database Administrator’s
Reference for UNIX Systems:
- udp_sendspace = db_block_size * db_file_multiblock_read_count +4k
- udp_recvspace = 10*(udp_sendspace)
- rfc1323 = 1
5. Virtual memory manager parameters (seen with vmo -a)
By default, AIX allocates up to 80% of real memory for use for filesystem buffer cache.
When a filesystem is not used, such as with ASM, or if filesystem buffer cache is
bypassed with options such as concurrent or direct I/O, it is advisable to decrease the
amount of memory which can be allocated to filesystem buffer cache. The value
minperm% dictates the minimum percentage of real memory that is allocated to
filesystem buffer cache, maxperm% dictates the maximum percentage of real memory
that can be allocated to filesystem buffer cache for JFS filesystems, and maxclient%
dictates the maximum percentage of real memory that can be allocated to filesystem
buffer cache for JFS2 filesystems. We typically set these values as follows in an
ASM or raw logical volume environment:
- minperm% = 5
- maxclient% = 15
- maxperm% = 15
With AIX5.2ML4 and above, we instead typically use the following values:
- lru_file_repage=0
- minperm% = 5
- maxclient% = 80
- maxperm% = 80
The lru_file_repage parameter was introduced in AIX5.2ML4, and changes the behavior
of the least recently used daemon (lrud) such that as long as the number of file pages in
use (numperm or numclient) is greater than minperm, only file pages, not computational
pages, will be paged out. When lru_file_repage=0, the values for maxperm% and
maxclient% can be left at the default value of 80%, which allows any available memory
up to 80% which is not in use to be used for filesystem buffer cache.
6. Paging space (smit chps)
The default allocation of paging space on AIX is 512MB, which is too small to support
Oracle installations. Typically a reasonable rule of thumb is 1x real memory, though
larger memory configurations will not need this much memory.

Oracle Parameters
The following parameters are suggested in the document Oracle Database 10g Release 2
Automatic Storage Management Overview and Technical Best Practices:
Parameters to be included in the spfile for databases using ASM:
1. Increase Processes by 16
2. Increase Large_Pool by 600k
3. Increase Shared Pool by (1M per 100GB of usable space) + 2M (assumes external
redundancy is chosen)
Parameters to be included in the spfile of the ASM instances:
1. ASM_POWER_LIMIT=1
The Best Practices guide suggests this value to make ASM rebalance operations a low
priority; however, this can potentially cause rebalance operations to occur during peak
volume times. This parameter can also be set to 0 to prevent accidental rebalance
operations from occurring during peak loads, which may be preferable for many
environments. This parameter can be raised specifically when rebalancing is desired.
2. Processes=25 + 15n, where “n” is the number of databases which will use ASM.

IBM eServer storage products:
DS8000, DS6800 and ESS storage models have been successfully implemented
using AIX MPIO for multipathing. Using SDDPCM is highly recommended.
SDD multipathing software does not allow non-root ownership of devices, and
may not be used at this time.
? In order to turn off device locking, all disk devices used for ASM, OCR, or
voting must set reserve_policy=no_reserve
# chdev –l hdisk# -a reserve_policy=no_reserve
? Verify that this is set correctly on all devices:
# lsattr –El hdisk# | grep reserve
reserve_policy no_reserve Reserve Policy True
DS4000 storage models use RDAC for path failover capability, and this is also
known to work with ASM, but the parameter in this case to turn off device
locking is reserve_lock=no
? # chdev –l hdisk# -a reserve_lock=no
? Verify that this is set correctly on all devices:
# lsattr –El hdisk# | grep reserve
reserve_lock no Reserve device on open True

        預設掛載的共享磁碟是不能被共享訪問的,只能被其中一個節點執行,如果不修改共享磁碟的共享屬性,那麼在執行另一個節點的root.sh指令碼的時候可能收到如下的錯誤:
/dev/hdisk13 device is busy.        
        執行lsattr –El | grep reserve命令檢視共享磁碟的共享屬性。修改屬性時注意以下步驟:
      1.根據使用的不同儲存可能看到的共享磁碟屬性是不同的,可能是reserve_policy或者是reserve_lock屬性。
      如果使用 檢視到的是reserve_policy屬性,那麼就執行chdev –l -a reserve_policy=no_reserve命令設定共享磁碟共享屬性。
      如果看到的是reserve_lock屬性執行chdev –l -a reserve_lock=no命令設定共享磁碟共享屬性。
      下面是修改多個共享磁碟的例子:
chdev -l hdisk2 -a reserve_policy=no_reserve
chdev -l hdisk3 -a reserve_policy=no_reserve
chdev -l hdisk4 -a reserve_policy=no_reserve
chdev -l hdisk5 -a reserve_policy=no_reserve
chdev -l hdisk6 -a reserve_policy=no_reserve
chdev -l hdisk7 -a reserve_policy=no_reserve
chdev -l hdisk8 -a reserve_policy=no_reserve
chdev -l hdisk9 -a reserve_policy=no_reserve
chdev -l hdisk10 -a reserve_policy=no_reserve
chdev -l hdisk11 -a reserve_policy=no_reserve
chdev -l hdisk13 -a reserve_policy=no_reserve
chdev -l hdisk14 -a reserve_policy=no_reserve
chdev -l hdisk15 -a reserve_policy=no_reserve
chdev -l hdisk16 -a reserve_policy=no_reserve
chdev -l hdisk17 -a reserve_policy=no_reserve
       2.以上的命令需要在RAC的所有節點都完成,在執行共享磁碟屬性修改的時候確保磁碟沒有被使用。
       3.在使用hdisk13作為共享磁碟的時候一定注意使用的檔案是/dev/rhdisk13,而不是/dev/hdisk13,rhdisk13是字元裝置,hdisk13是塊裝置。

        4.完成設定之後,修改/dev/rhdisk#的所有者為oracle:oinstall,/dev/rhdisk#就可作為asm disk使用。透過dbca來建立asm例項,新建asm group就可以正常使用asm了。

來自 “ ITPUB部落格 ” ,連結:http://blog.itpub.net/23135684/viewspace-659998/,如需轉載,請註明出處,否則將追究法律責任。

相關文章