[20151110]Oracle Direct NFS Client.txt

[20151110]Oracle Direct NFS Client.txt

--摘要連結:
在Oracle 11g中引入了Direct Network File System(Oracle Direct NFS)的新特性，透過一個打包在Oracle核心中的NFS客戶機以改善實
例使用NFS時的效能，同時進一步完善了透過NFS實現RAC的解決方案。常規的NFS客戶端軟體一般由作業系統供應商提供，這類NFS客戶端
不會專門為Oracle資料檔案的IO做最佳化。而透過內建的Oracle Direct NFS，資料庫將可以直接訪問NFS伺服器上的檔案，避免由OS核心
NFS造成的額外開銷。Oracle宣稱由以上最佳化所帶來的效能提升，在DSS環境中超過40%，而在OLTP環境中超過10%(詳見<Oracle Database
11g  Direct NFS Client a white paper>)。

使用$ORACLE_HOME/dbs/oranfstab來配置Direct NFS客戶機；該oranfstab配置檔案可以包括Server,path,export以及mount引數，各引數
代表的屬性如下:

    Server:NFS伺服器名
    Path:到達NFS伺服器的最多4個網路路徑，可以是IP或者主機名
    Export:從NFS伺服器匯出的路徑
    Mount：NFS的本地裝載點

在正式啟用Direct NFS客戶機前，NFS檔案系統應當已由常規NFS方式mount並且可用。為了啟用Direct NFS client，我們還需要將標準的
Oracle磁碟管理庫(Oracle Disk Manager (ODM) library)替換為支援Direct NFS client的NFS ODM。可以透過建立從標準ODM庫指向NFS
ODM庫的符號連結來完成以上工作，但是需要注意的是以上操作僅能在例項關閉的情況下才能實施並且有效。

--說明:以前我測試過,不知道為什麼沒有成功,不知道問題在那裡,我僅僅估計當時的測試環境linux版本太低(rh 4,3),當時為了安裝11g,
--自己還升級了幾個包,也許在某個地方出現問題,再加上沒有這種需求,也就再也沒有測試過.

1.建立連結:
--注意在關閉資料庫下進行.
$ cd $ORACLE_HOME/lib
$ ls -l  libodm11.so
lrwxrwxrwx 1 oracle oinstall 12 Aug 17 15:58 libodm11.so -> libodmd11.so

$ ln -s libnfsodm11.so libodm11.so
$ ls -l  libodm11.so
lrwxrwxrwx 1 oracle oinstall 14 Nov 10 07:51 libodm11.so -> libnfsodm11.so

$ mkdir /mnt/ramdisk/nfs

2.建立nfs伺服器(IP=192.168.101.115):
--IP=192.168.101.115
# mkdir /u01/nfs
# chmod 777 /u01/nfs

--建立/exc/exports檔案,加入:
/u01/nfs *(rw,no_root_squash,insecure)
--注:我以前估計錯誤在這裡,我設定成/u01/nfs *(rw,sync,no_root_squash)

--啟動服務.
# service rpcbind restart
Stopping rpcbind:                                          [  OK  ]
Starting rpcbind:                                          [  OK  ]
--注:我使用centos 6.2,使用rpcbind代替portmap.

# service nfs start
Starting NFS services:                                     [  OK  ]
Starting NFS quotas:                                       [  OK  ]
Starting NFS daemon:                                       [  OK  ]
Starting NFS mountd:                                       [  OK  ]

# showmount -e 192.168.101.115
Export list for 192.168.101.115:
/u01/nfs *

--在資料庫機器上測試nfs是否正常:
# mount -t nfs 192.168.101.115:/u01/nfs /mnt/ramdisk/nfs/

# mount | grep nfs
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
192.168.101.115:/u01/nfs on /mnt/ramdisk/nfs type nfs (rw,addr=192.168.101.115)

--ok nfs建立訪問正常!

3.建立oranfstab檔案:

$ cd /u01/app/oracle/product/11.2.0.4/dbhome_1/dbs
$ cat oranfstab
server: 192.168.101.115
path: 192.168.101.115
export: /u01/nfs  mount: /mnt/ramdisk/nfs

4.開始啟動資料庫:
--在alert*.log存在如下內容:

Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 3.0
...

5.開始測試:
SYS@book> select * from v$dnfs_servers;
no rows selected

SYS@book> select * from v$dnfs_files;
no rows selected

--沒有檔案顯示是正常的,但是v$dnfs_servers這裡看不到內容是我困惑的地方.也許上次錯誤在這裡,沒有繼續下去.

SYS@book> create tablespace dnfs datafile '/mnt/ramdisk/nfs/dnfs01.dbf' size 2M;
Tablespace created.

SYS@book> column filename format a40
SYS@book> select * from v$dnfs_files;
FILENAME                                   FILESIZE       PNUM     SVR_ID
---------------------------------------- ---------- ---------- ----------
/mnt/ramdisk/nfs/dnfs01.dbf                 2105344         10          1

SYS@book> select * from v$dnfs_servers;
        ID SVRNAME                        DIRNAME                           MNTPORT    NFSPORT      WTMAX      RTMAX
---------- ------------------------------ ------------------------------ ---------- ---------- ---------- ----------
         1 192.168.101.115                /u01/nfs                            56759       2049     524288     524288

--ok!這次成功了.真不知道上次問題在那裡.在alert*.log記錄如下:
Tue Nov 10 09:09:10 2015
create tablespace dnfs datafile '/mnt/ramdisk/nfs/dnfs01.dbf' size 2M
Tue Nov 10 09:09:11 2015
Direct NFS: channel id [0] path [192.168.101.115] to filer [192.168.101.115] via local [] is UP
Direct NFS: channel id [1] path [192.168.101.115] to filer [192.168.101.115] via local [] is UP
Completed: create tablespace dnfs datafile '/mnt/ramdisk/nfs/dnfs01.dbf' size 2M

--從這裡說明這次成功了.

SYS@book> drop tablespace dnfs including contents and datafiles cascade constraints;
Tablespace dropped.

--總的感覺很慢,我改變mount引數再測試看看.

6.改變mount引數再測試:
# umount /mnt/ramdisk/nfs
# mount -t nfs 192.168.101.115:/u01/nfs /mnt/ramdisk/nfs/ -o rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,nfsvers=3,timeo=600,actimeo=0

SYS@book> column SVRNAME format a30
SYS@book> column DIRNAME format a30
SYS@book> select * from v$dnfs_servers;
        ID SVRNAME                        DIRNAME                           MNTPORT    NFSPORT      WTMAX      RTMAX
---------- ------------------------------ ------------------------------ ---------- ---------- ---------- ----------
         1 192.168.101.115                /u01/nfs                            56759       2049     524288     524288

SYS@book> create tablespace dnfs datafile '/mnt/ramdisk/nfs/dnfs01.dbf' size 2M;
Tablespace created.

SYS@book> drop tablespace dnfs including contents and datafiles cascade constraints;
Tablespace dropped.

--感覺沒有什麼變化.重啟在測試看看.效果也一樣.

--在nfs存在檔案的情況下,檢視alert*.log發現使用direct nfs:
Tue Nov 10 09:20:10 2015
ALTER DATABASE OPEN
Direct NFS: attempting to mount /u01/nfs on filer 192.168.101.115 defined in oranfstab
Direct NFS: channel config is:
     channel id [0] local [] path [192.168.101.115]
Direct NFS: mount complete dir /u01/nfs on 192.168.101.115 mntport 56759 nfsport 2049
Direct NFS: channel id [0] path [192.168.101.115] to filer [192.168.101.115] via local [] is UP
Direct NFS: channel id [1] path [192.168.101.115] to filer [192.168.101.115] via local [] is UP

SYS@book> @ &r/spid
       SID    SERIAL# SPID   C50
---------- ---------- ------ --------------------------------------------------
       580          5 60110  alter system kill session '580,5' immediate;

# lsof -i -P -n | grep 2049
oracle    60088  oracle   32u  IPv4 12090792      0t0  TCP 192.168.100.78:25917->192.168.101.115:2049 (ESTABLISHED)
oracle    60088  oracle   33u  IPv4 12094081      0t0  TCP 192.168.100.78:52324->192.168.101.115:2049 (ESTABLISHED)
oracle    60090  oracle   32u  IPv4 12097630      0t0  TCP 192.168.100.78:21175->192.168.101.115:2049 (ESTABLISHED)
oracle    60090  oracle   33u  IPv4 12094082      0t0  TCP 192.168.100.78:36396->192.168.101.115:2049 (ESTABLISHED)
oracle    60110  oracle   10u  IPv6 12090793      0t0  UDP *:20496
oracle    60110  oracle   32u  IPv4 12097629      0t0  TCP 192.168.100.78:8344->192.168.101.115:2049 (ESTABLISHED)

# ps -ef | egrep "60080|60090|60110" | grep -v grep
oracle   60080     1  0 09:20 ?        00:00:00 ora_diag_book
oracle   60090     1  0 09:20 ?        00:00:00 ora_lgwr_book
oracle   60110 59964  0 09:20 ?        00:00:01 oraclebook (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))