linux系統中配置hugepage，提升oracle資料庫效能

linux配置大記憶體技術

1、理論介紹

   系統程式透過虛擬地址訪問記憶體，CPU必須把它轉換成實體記憶體地址才能真正訪問記憶體。為了提高這個轉換效率,CPU會快取最近的虛擬記憶體地址和實體記憶體地址的對映關係，並儲存在一個

由CPU維護的對映表中。為了儘量提高記憶體的訪問速度，需要在對映表中儲存儘量多的對映關係。在linux系統中,所有記憶體都是以頁的形式劃分的，預設情況下每頁是4096B，這就意味著如果

實體記憶體很大，對映表的條目數會很多，系統cpu在檢索的過程中，需要檢索的條目數就會很。從另一個角度說，如果能夠讓cpu減少檢索記憶體條目的數量，可採取的辦法只有增加頁的尺寸，

hugepage應時而生。hugepage，一直被cache,不會被交換出去,也就是說使用hugepage的記憶體不能被其他的程式使用。隨著記憶體技術的發展，現在上T的記憶體配置也出現，所以記憶體已經不是問

題，在實際應用中，如果你的實體記憶體能夠達到8g以上，那麼都可以使用大記憶體技術。在oracle資料庫中，sga與大記憶體配合使用，將會對效能產生非常大的提升。

2、oracle中大記憶體技術的配置建議

   oracle針對大記憶體技術，提供了一個指令碼如下：
#!/bin/bash
#
# hugepages_settings.sh
#
# Linux bash script to compute values for the
# recommended HugePages/HugeTLB configuration
#
# Note: This script does calculation for all shared memory
# segments available when the script is run, no matter it
# is an Oracle RDBMS shared memory segment or not.
#
# This script is provided by Doc ID 401749.1 from My Oracle Support
#
 
# Welcome text
echo "
This script is provided by Doc ID 401749.1 from My Oracle Support
() where it is intended to compute values for
the recommended HugePages/HugeTLB configuration for the current shared
memory segments. Before proceeding with the execution please make sure
that:
 * Oracle Database instance(s) are up and running
 * Oracle Database 11g Automatic Memory Management (AMM) is not setup
   (See Doc ID 749851.1)
 * The shared memory segments can be listed by command:
     # ipcs -m
 
Press Enter to proceed..."
 
read
 
# Check for the kernel version
KERN=`uname -r | awk -F. '{ printf("%d.%d\n",$1,$2); }'`
 
# Find out the HugePage size
HPG_SZ=`grep Hugepagesize /proc/meminfo | awk '{print $2}'`
 
# Initialize the counter
NUM_PG=0
 
# Cumulative number of pages required to handle the running shared memory segments
for SEG_BYTES in `ipcs -m | awk '{print $5}' | grep "[0-9][0-9]*"`
do
   MIN_PG=`echo "$SEG_BYTES/($HPG_SZ*1024)" | bc -q`
   if [ $MIN_PG -gt 0 ]; then
      NUM_PG=`echo "$NUM_PG+$MIN_PG+1" | bc -q`
   fi
done
 
RES_BYTES=`echo "$NUM_PG * $HPG_SZ * 1024" | bc -q`
 
# An SGA less than 100MB does not make sense
# Bail out if that is the case
if [ $RES_BYTES -lt 100000000 ]; then
   echo "***********"
   echo "** ERROR **"
   echo "***********"
   echo "Sorry! There are not enough total of shared memory segments allocated for
HugePages configuration. HugePages can only be used for shared memory segments
that you can list by command:
 
   # ipcs -m
 
of a size that can match an Oracle Database SGA. Please make sure that:
 * Oracle Database instance is up and running
 * Oracle Database 11g Automatic Memory Management (AMM) is not configured"
   exit 1
fi
 
# Finish with results
case $KERN in
   '2.4') HUGETLB_POOL=`echo "$NUM_PG*$HPG_SZ/1024" | bc -q`;
          echo "Recommended setting: vm.hugetlb_pool = $HUGETLB_POOL" ;;
   '2.6') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;
    *) echo "Unrecognized kernel version $KERN. Exiting." ;;
esac
 
# End

[root@hdyydb1 u02]# chmod u+x hugepages_settings.sh
[root@hdyydb1 u02]# ./hugepages_settings.sh

This script is provided by Doc ID 401749.1 from My Oracle Support
() where it is intended to compute values for
the recommended HugePages/HugeTLB configuration for the current shared
memory segments. Before proceeding with the execution please make sure
that:
 * Oracle Database instance(s) are up and running
 * Oracle Database 11g Automatic Memory Management (AMM) is not setup
   (See Doc ID 749851.1)
 * The shared memory segments can be listed by command:
     # ipcs -m
 
Press Enter to proceed...

Recommended setting: vm.nr_hugepages = 2588

可以看到，執行指令碼顯示，將vm.nr_hugepages設定為2588

3、hugepage在oracle中的配置應用

--檢查系統是否啟用hugepage，或者是否可以啟用
[root@hdyydb1 u02]# cat /proc/meminfo | grep -i huge
AnonHugePages:     75776 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

可以看到，HugePages_Total,HugePages_Free均為零，證明沒有使用，但可以配置。如果上面什麼都沒有顯示，證明不能配置。

--檢查資料庫的引數設定是否支援
SQL> show parameter sga_target

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
sga_target                           big integer 0

大記憶體技術，不支援記憶體自動管理，必須關閉AMM(自動記憶體管理）特性才能使用hugepage

調整方法如下：

ALTER SYSTEM SET sga_max_size = xxxxM SCOPE=SPFILE;
ALTER SYSTEM SET sga_target = xxxxM SCOPE=SPFILE;
ALTER SYSTEM SET PGA_AGGREGATE_TARGET = xxxxM SCOPE=SPFILE;
ALTER SYSTEM SET memory_target = 0 SCOPE=SPFILE;
ALTER SYSTEM SET memory_max_target = 0 SCOPE=SPFILE;

--根據第二步的驗證結果進行配置

[root@hdyydb1 u02]# vi /etc/sysctl.conf
把vm.nr_hugepages = 2588 新增進去，並儲存

[root@hdyydb1 u02]# sysctl -p
net.ipv4.ip_forward = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0
kernel.msgmnb = 65536
kernel.msgmax = 65536
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 7247757312
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586
vm.min_free_kbytes = 51200
vm.nr_hugepages = 2588

檢查設定：
[root@hdyydb1 u02]# cat /proc/meminfo | grep -i huge
AnonHugePages:     75776 kB
HugePages_Total:     977
HugePages_Free:      977
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

--鎖定記憶體配置
如果不配置鎖定記憶體，在後alert_sid.log中會有類似如下的建議：
RECOMMENDATION:
  Total System Global Area size is 5 GB. For optimal performance,
  prior to the next instance restart:
  1. Large pages are automatically locked into physical memory.
 Increase the per process memlock (soft) limit to at least 5 GB to lock
 100% System Global Area's large pages into physical memory
並且，hugepage並沒有真正使用起來！

主要是配置
oracle soft memlock
oracle hard memlock
注意，這裡設定的值均以kb為單位的！
實際實體記憶體 > 鎖定記憶體 >=HugePages_Total*Hugepagesize

[root@hdyydb1 u02]# vim /etc/security/limits.conf
新增：
oracle soft memlock 6291456
oracle hard memlock 6291456

我的實體記憶體是8g，sga 5g左右

4、重啟作業系統、資料庫後生效

[oracle@hdyydb1 ~]$ cat /proc/meminfo | grep -i huge
AnonHugePages:     43008 kB
HugePages_Total:    2588
HugePages_Free:     2168
HugePages_Rsvd:     2165
HugePages_Surp:        0
Hugepagesize:       2048 kB

free比total小，證明已經生效