菜鳥學Linux 第111篇筆記 Memory

菜鳥學Linux 第111篇筆記 Memory

建議檢視原文（因為複製版的格式可能有問題)

原文出自 Winthcloud　連結：

內容總覽

記憶體子系統元件

Memory提升

Viewing system calls

Strategies for using memory

Tunning page allocation

Tuning overcommit

Slab cache

ARP cache

Page cache

調優策略(redhat 6 performance tuning guide官方文件)

程式間通訊相關的調優

記憶體子系統元件

slab allocator

buddy system

kswapd

pdflush

mmu

虛擬化環境

PA --> HA --> MA

虛擬機器轉換：PA-->HA

GuestOS, OS

Shadow PT

Memory提升

lbs 

Hugetlbfs

檢視是否啟用Hugetlbfs

cat /proc/meminfo | grep Huge

啟用大頁面(永久有效)

/etc/sysctl.conf

新增 vm.nr_hugepages = n

即時啟用

sysctl -w vm.nr_hugepages=n

Configure hugetlbfs if needed by application

建立hugepage並掛載

mkdir /hugepages

mount -t hugetlbfs none /hugepages

Viewing system calls

Trace every system call made by a program

strace -o /tmp/strace.out -p PID

grep mmap /tmp/strace.out

Summarize system calls

strace -c -p PID or

strace -c COMMAND

Strategies for using memory

Reduce overhead for tiny memory objects

Slab cache

cat /proc/slabinfo

Reduce or defer service time for slower subsystems

Filesystem metadata: buffer cache(slab cache)

Disk IO: page cache

Interprocess communications: shared memory

Network IO: buffer cache, arp cache, connection tracking

使用buffer cache 快取檔案後設資料

使用page cache快取Disk IO

使用shm完成程式間通訊

使用buffer cache, arp cache和connection tracking提升網路IO效能

Considerations when tunning memory

How should pages be reclaimed to avoid pressure?

Larger writes are usually more efficient due to re-sorting

Tunning page allocation

Set using 

vm.min_free_kbytes

Tuning vm.min_free_kbytes only be necessary when an application regularly

needs to allocate a large block of memory, then frees that same memory

It may well be the case that the system has too little disk bandwith, too

little CPU power, or too little memory to handle its load.

Consequences

Reduces service time for demand paging

Memory is not available for other useage

Can cause pressure on ZONE_NORMAL

記憶體耗盡會使系統崩潰

Tuning overcommit

Set using

cat /proc/sys/vm/overcommit_memory

vm.overcommit_memory

0 = heuristic overcommit

1 = always overcommit

2 = commit all RAM plus a percentage of swap (may be > 100)

vm.overcommit_ratio

specified the percentage of physical memory allowed to be 

overcommited when the vm.overcommit_memory set to 2

View Committed_AS in /proc/meminfo

An estimate of how much RAM is required to avoid an out of memory (OOM)

condition for the current workload on a system

OOM

Overcommit Of Memory

Slab cache

Tiny kernel objects are stored in slab

Extra overhead of tracking is better than using 1 page/object

Example: filesystem metadata(dentry and inode caches)

Monitoring

/proc/slabinfo

slabtop

vmstat -m

Tuning a particular slab cache

echo "cache_name limit batchcount shared" > /proc/slabinfo

limit the maximum number of objects that will be cached for each CPU

batchcount the maximum number of global cache objects that will be 

   trasferred to the per-CPU cache when it becomes empty

shared the sharing behavior for Symmetric MultiProcessing(SMP) systems

ARP cache

ARP entries map hardware address to protocol address

cached in /proc/net/arp

By default, the cache is limited to 512 entries as a soft limit 

and 1024 entries as a hard limit

Garbage collection removes stale or older entries

Insufficient ARP cache leads to

Intermittent timeouts between hosts

ARP thrashing

Too much ARP cache puts pressure on ZONE_NORMAL

List entries

ip neighbor list

Flush cache

ip neighbor flush dev ethX

Tuning ARP cache

Adjust where the gc will leave arp table alone

net.ipv4.neigh.default.gc_thresh1

default 128

Soft upper limit

net.ipv4.neigh.default.gc_thresh2

default 512

Becomes hard limit after 5 seconds

Hard upper limit

net.ipv4.neigh.default.gc_thresh3

Garbage collection frequency in seconds

net.ipv4.neigh.default.gc_interval

Page cache

A large percentage of paging activity is due to I/O requests

File reads: each page of file read from disk into memory

These pages form the page cache

Page cache is always checked for IO requests

Drectory reads

Reading and writing regular files

Reading and writing via block device files, DISK IO

Accessing memory mapped files, mmap

Accessing swapped out pages

Page in the page cache are associated with file data

Tuning page cache

View page cache allocation in /proc/meminfo

Tune length/size of memory

vm.lowmen_reserve_ratio

vm.vfs_cache_pressure

Tune arrival/completion rate

vm.page-cluster

vm.zone_reclaim_mode

vm.lowmen_reserve_ratio

For some specialised workloads on highmem machines it is dangerous for the 

kernel to allow process memory to be allocated from the "lowmem" zone

Linux page allocator has a mechanism which prevents allocations which could

use highmem from using too much lowmem

The 'lowmem_reserve_ratio' tunable determines how aggressive the kernel is 

in defending these lower zones

If you have a machine which uses highmem or ISA DMA and Your applications

are using mlock(), or if you are running with no swap then you probably 

should change the lowmem_reserve_ratio setting

vfs_cache_pressure

Controls the tendency of the kernel to reclaim the memory which is used for

caching of directory and inode objects

At the default value of vfs_cache_pressure=100 the kernel will attempt to 

reclaim dentries and inodes at a "fair" rate with respect to pagecache and 

swapcache reclaim

Decreasing vfs_cache_pressure causes the kernel to prefer to retain dentry 

and inode caches

When vfs_cache_pressure=0, the kernel will never reclaim dentries and 

inodes due to memory pressure and this can easily lead to out-of-memory

conditions

Increasing vfs_cache_pressure beyond 100 causes the kernel to prefer to 

reclaim dentries and inodes.

page-cluster

page-cluster controls the number of pages which are written to swap in 

single attempt

It is a logarithmic value-setting it to zero means "1 page", setting it to

1 means "2 pages", setting it to 2 means "4 pages", etc

The default value is three (eight pages at a time)

There may be some small benefits in tuning this to a different value if 

your workload is swap-intensive

zone_reclaim_mode

Zone_reclaim_mode allows someone to set more or less aggressive approaches

to reclaim memory when a zone runs out of memory

If it is set to zero then no zone reclaim occurs

Allocations will be satisfied from other zones/nodes in the system

This is value ORed together of

1 = Zone reclaim on

2 = Zone reclaim writes dirty pages out

4 = Zone reclaim swaps pages

Anonymous pages

Anonymous pages can be another large  consumer of data

Are not associated with a file, but instead contain:

Program data - arrays, head allocations, etc

Anonymous memory regions

Dirty memory mapped process private pages

IPC shared memory regions pages

View summary usage

grep Anon /proc/meminfo

cat /proc/PID/statm

Anonymous pages = RSS - Shared

Anonymous pages are eligible for swap

調優策略

硬體調優： 硬體選型

軟體調優： 核心調優 /proc, /sys

   應用調優

核心調優

1. 程式管理，CPU

2. 記憶體調優

3. I/O 調優

4. 檔案系統

5. 網路子系統

調優思路

1. 檢視各項效能指標，定位瓶頸

2. 調優

紅帽官方提供一份文件  redhat 6 performance tuning guide 可以搜尋到

程式間通訊相關的調優

ipcs (interprocess communication facilities)

程式間通訊管理命令

ipcs

ipcrm

shared memory

kernel.shmmni

Specifies the maximum number of shared memory segments 

system-wide, default = 4096

kernel.shmall

Specifies the total amount of shared memory, in pages, that

can be used at one time on the system, default=2097152

This should be at least kernel.shammax/PAGE_SIZE

kernel.shmmax

Specifies the maximum size of a shared memory segment that 

can be created

messages

kernel.msgmnb

Specifies the maximum number of bytes in single message 

queue, default = 16384

kernel.msgmni

Specifies the maximum number of message queue identifiers, 

default=16

kernel.msgmax

Specifies the maximum size of a message that can be passed 

between processes

This memory cannot be swapped, default=8192

本文轉自Winthcloud部落格51CTO部落格，原文連結http://blog.51cto.com/winthcloud/1930795如需轉載請自行聯絡原作者

Winthcloud