ORACLE LARGE MEMORY(zt)

Solution

Follow steps below to perform the configuration:

(Note: For 9iR2 patch 3318884 (9.2.0.4) and patch 3566570 (9.2.0.5)are required respectively.)

1. Logon root

2. Configure Linux to mount ramfs over /dev/shm at every boot. Edit /etc/rc.local and add the following:

umount /dev/shm
mount -t ramfs ramfs /dev/shm
chown oracle:oinstall /dev/shm

where oracle is the Oracle owner and oinstall is the group for Oracle owner account.

3. Reboot server

4. Logon as root

5. Check whether /dev/shm is mounted with type ramfs

# mount | grep shm
ramfs on /dev/shm type ramfs (rw)

# ls -ld /dev/shm
drwxr-xr-x  3 oracle oinstall 0 Jul 13 12:12 /dev/shm

7. Increase max locked memory limit. Edit /etc/security/limits.conf and add

*          soft    memlock        3145728
*          hard    memlock        3145728    

9. Check max locked mem limit:

$ ulimit -l
3145728

11. Configure instance parameters for VLM:

a. ConvertDB_CACHE_SIZE, DB_xK_CACHE_SIZE parameters to DB_BLOCK_BUFFERS
b. Add parameter USE_INDIRECT_DATA_BUFFERS=TRUE
c. Configure SGA size according to the needs
d. Remove SGA_TARGET if set

12. Startup instance

13. Examine the memory allocation:

$ ls -l /dev/shm
$ ipcs -m

14. Configure hugepages

a. Get Hugepagesize from

	$ grep Hugepagesize /proc/meminfo

b. Compute nr_hugepages using Note 401749.1

c. Set kernel parameter:

	  # sysctl -w vm.nr_hugepages=

d. Set parameter for every boot.Edit/etc/sysctl.conf for vm.nr_hugepages=

15. Check available hugepages:

	 $ grep Huge /proc/meminfo

16. Restart instance

17. Check available hugepages (1 or 2 pages free)

	 $ grep Huge /proc/meminfo

Note: If the setting of nr_hugepages is not effective, you might need to reboot the server.

< A>). To check whether HugePages are supported/available on a running configuration, run:

    $ grep Huge /proc/meminfo

and check the
output:
    HugePages_Total:    
    HugePages_Free:

    Hugepagesize: 
Regardless of the values printed, if you can see the lines above, the system supports HugePages.


Why HugePages Should Be Used on 64-bit Linux?
Very Large Memory:
The 64-bit systems do have VLM. So the general VLM issues apply.
Not swappable: HugePages are not
swappable. Therefore there is no page replacement mechanism
overhead.HugePages are universally regarded as pinned.

Eliminated page table lookup overhead: Since the pages are not subject to replacement,
page table lookups are not required. 
Faster overall memory performance: On virtual memory systems each memory operation is actually two
abstract memory operations. Since there are less number of pages to
work on, the possible bottleneck on page table access is clearly
avoided.
kswapd: kswapd will get very busy if there is a very large area to be paged (i.e. 13 million page table entries for 50GB memory) and will use an incredible amount of CPU resource. When HugePages are used, kswapd is not involved in managing them.
Configuration/Setup
The configuration for HugePages on 64-bit Linuxes are not different
from the configuration on 32-bit systems. See Note 361323.1  section "Parameters/Setup" for the generic setup of
the feature.

On 64-bit for best practice, set the memlock user limit to a value larger than the total size of the sga. e.g.
For 5GB SGA:
/etc/security/limits.conf:
*          soft     memlock        5243000
*          hard     memlock        5243000