在exadata環境下配置dbfs檔案系統

kingsql發表於2017-04-08

在exadata環境下配置dbfs檔案系統

1.ORACLE的一體機環境下,通常原廠初始化完成後,都會有一個典型的dbfs磁碟組被建立,可用來建立oracle所支援的檔案系統來用。
  ORACLE的檔案系統需要一個獨立的資料庫來作為後臺支撐,在此庫內建立一個表空間,此空間的容量就是後續檔案系統可使用的容量。
  建立系統時涉及到2個方法:使用錢包或者不使用錢包,使用錢包由於複雜且有潛在影響,所以下文只針對非錢包方式進行描述。
 
  另外假設平臺是Linux平臺

2.建立單獨的資料庫
  由於Dbfs檔案系統是依託於某個表空間的,所以需要先為此目的單獨建立一個資料庫,資料庫的要求如下:
 
  2.1. Invoke?DBCA
  2.2. Choose Real Application Clusters database.
  2.3. Choose Create a Database?
  2.4. Choose the General Purpose or Transaction Processing template (include datafiles)
  2.5. Choose Admin-Managed and select all nodes
  2.6. Name your database (e.g. dbfs)
  2.7. Optionally configure Enterprise Manager and automatic maintenance tasks as per site requirements
  2.8. Choose Automatic Storage Management and Oracle-Managed Files
  2.9. Choose the desired Disk Group. In general, DBFS_DG is adequate for DBFS purposes
  2.10. De-select Flash Recovery Area
  2.11. Do not select Enable Archiving. Archivelog mode is not necessary for this use case.
  2.12. In the Memory Tab:
  2.12.1 Choose Custom and then Automatic Shared Memory Management
  2.12.2 Enter 1536 (and choose M Bytes for units) in the SGA box and enter 6656 and choose M Bytes for units) in the PGA Size box
  2.13. In the Character Sets tab choose AL32UTF8 as the Database Character Set
  2.14. Open the All Initialization Parameters dialogue. Select "Show Advanced arameters". Scroll down to parallel_max_servers and enter "2" under the Value column
  2.15. If diskgroup DBFS_DG's compatible.rdbms attribute is set to 11.2.0.2.0 then
        in DBCA set compatible=11.2.0.2.0 in All Initialization Parameters screen. If this isn't set properly, you will get "ORA-15204: database version 11.2.0.0.0 is
        incompatible with diskgroup DBFS_DG" while creating the database.
  2.16. Click on the Finish button

3.配置過程如下:(假設是四分之一配的一體機,存在兩個db server:dm01test01,dm02test02)
3.1 將oracle加入fuse 組
    (root)# dcli -g ~/dbs_group -l root usermod -a -G fuse oracle
    more dbfs_group:
    dm01test01
    dm01test02  
   
    建立/etc/fuse.conf 檔案並授權
    (root)# dcli -g ~/dbs_group -l root "echo user_allow_other > /etc/fuse.conf"
    (root)# dcli -g ~/dbs_group -l root chmod 644 /etc/fuse.conf   
3.2  建立檔案系統的掛載點(比如dbfsmnt)
   (root)# dcli -g ~/dbs_group -l root chown oracle:dba /dbfsmnt 
  
3.3叢集重啟

(root)# dcli -g ~/dbs_group -l root /u01/app/11.2.0.3/grid/bin/crsctl stop crs
(root)# dcli -g ~/dbs_group -l root /u01/app/11.2.0.3/grid/bin/crsctl start crs

3.4在第2步驟建立的書庫內建立使用者及表空間
表空間要求如下:
    Tablespaces for DBFS can be either SMALLFILE or BIGFILE. Optimally, the  administrator will determine the size of the staging area needed for the Data
     Warehouse activities and will create the tablespace with the required size as opposed to relying on autoextend.
    When creating a tablespace for DBFS, include the following options:
    NOLOGGING ONLINE PERMANENT EXTENT MANAGEMENT LOCAL AUTOALLOCATE  SEGMENT SPACE MANAGEMENT AUTO 
   
   
SQL> create bigfile tablespace dbfsts datafile '+DBFS_DG' size 32g autoextend on next 8g maxsize 300g NOLOGGING EXTENT MANAGEMENT LOCAL AUTOALLOCATE  SEGMENT SPACE MANAGEMENT AUTO ;
SQL> create user dbfs_user identified by test default tablespace dbfsts quota unlimited on dbfsts;
SQL> grant create session, create table, create view, create procedure, dbfs_role to dbfs_user;

3.5使用上述建立的使用者,執行指令碼儲存dbfs的物件
(oracle)$ sqlplus dbfs_user/dbfs_passwd
SQL> start dbfs_create_filesystem dbfsts FS1

dbfsts:是用於檔案系統的表空間名稱
FS1:檔案系統掛載點下面的第一子目錄,所有後續可用的檔案系統都需要在此子目錄下建立更內層的子目錄,比如/dbfsmnt/FS1/test

3.6.下載mount-dbfs.sh 檔案並進行轉換處理,比如檔案放到/tmp下

For Linux, run this:
(root)# dos2unix /tmp/mount-dbfs.sh

3.7 編輯/tmp/mount-dbfs.sh 檔案
 DBNAME
 MOUNT_POINT
 DBFS_USER
 ORACLE_HOME (should be the RDBMS ORACLE_HOME directory)
 LOGGER_FACILITY (used by syslog to log the messages/output from this script)
 MOUNT_OPTIONS
 DBFS_PASSWD (used only if WALLET=false)
 DBFS_PWDFILE_BASE (used only if WALET=false)
 WALLET (must be true or false)
 TNS_ADMIN (used only if WALLET=true)
 DBFS_LOCAL_TNSALIAS
 
 如上所示,由於示例未採用wallet,所以部分和wallet相關的屬性不用設定
 改後的檔案內容如下:
 more mount-dbfs.sh
#!/bin/bash

### This script. is from Note 1054431.1, ensure you have the latest version
### Note 1054431.1 provides information about the setup required to use this script

### updated 26-JUL-2012

###########################################
### Everyone must set these values
###########################################
### Database name for the DBFS repository as used in "srvctl status database -d $DBNAME"
DBNAME=test

### Mount point where DBFS should be mounted
MOUNT_POINT=/dbfsmnt

### Username of the DBFS repository owner in database $DBNAME
DBFS_USER=dbfs_user

### RDBMS ORACLE_HOME directory path
ORACLE_HOME=/u01/app/oracle/product/11.2.0.3/dbhome_1

### Syslog facility name (default user)
### Changed default from local3 to user for Solaris default support on 17-FEB-2012
### This will allow us to log messages to the syslog
###  (/var/log/messages on Linux, /var/adm/messages on Solaris)
LOGGER_FACILITY=user

### mount options for dbfs_client; these are used for both wallet and non-wallet mounting
MOUNT_OPTIONS=allow_other,direct_io
### if tracing is required, maybe consider parameter like the example below
### following example is commented out, only uncomment if directed by Oracle Support
### fix_control=32 added per bug 13340960 to allow async statfs response
#MOUNT_OPTIONS=allow_other,direct_io,fix_control=32,trace_level=1,trace_file=/tmp/dbfs_client_trace.$$.log,trace_size=100

### PERL_ALARM_TIMEOUT is number of seconds to wait for response from status command.
### After this, if no respnose, the script. will run clean.
### NOTE: If this is longer than the clusterware check interval, bad things may happen.
###       Adjust the CHECK_INTERVAL to ensure it is at least 2x as long as PERL_ALARM_TIMEOUT.
###       Example:
###       $ crsctl status res dbfs_mount -p|grep ^CHECK     
###       CHECK_INTERVAL=30
###       $ crsctl modify res dbfs_mount -attr "CHECK_INTERVAL=32"
###       $ crsctl status res dbfs_mount -p|grep ^CHECK
###       CHECK_INTERVAL=32
PERL_ALARM_TIMEOUT=14

###########################################
### If using password-based authentication, set these
###########################################
### This is the plain text password for the DBFS_USER user
DBFS_PASSWD=test

### The file used to temporarily store the DBFS_PASSWD so dbfs_client can read it
### This file is removed immediately after it is read by dbfs_client
### The actual filename used will have the PID appended to the name for uniqueness
### This variable should be a full pathname including a directory and the first part of a filename.
DBFS_PWDFILE_BASE=/tmp/.dbfs-passwd.txt

###########################################
### If using wallet-based authentication, modify these
###########################################
### WALLET should be true if using a wallet, otherwise, false
WALLET=false

### TNS_ADMIN is the directory containing tnsnames.ora and sqlnet.ora used by DBFS
TNS_ADMIN=/export/home/oracle/dbfs/tnsadmin

### TNS alias used for mounting with wallets
DBFS_LOCAL_TNSALIAS=fsdb.local

###########################################
### No editing is required below this point
###########################################
### determine platform
UNAME_S=`uname -s`
if   [ $UNAME_S = 'Linux' ]; then LINUX=1; SOLARIS=0;
elif [ $UNAME_S = 'SunOS' ]; then LINUX=0; SOLARIS=1;
fi

GREP=/bin/grep
AWK=/bin/awk
ECHO=/bin/echo
LOGGER="/bin/logger -t DBFS_${MOUNT_POINT}"
RMF='/bin/rm -f'
TOUCH=/bin/touch
CHMOD=/bin/chmod
PS=/bin/ps
SLEEP=/bin/sleep
KILL=/bin/kill
BASENAME=/bin/basename
STAT=/usr/bin/stat
ID=/usr/bin/id
WC=/usr/bin/wc
SRVCTL=$ORACLE_HOME/bin/srvctl
DBFS_CLIENT=$ORACLE_HOME/bin/dbfs_client
HN=/bin/hostname
PERL=/usr/bin/perl
MOUNT=/bin/mount
### ensure messages are displayed in English for pattern matching
LANG=en_US.UTF-8
NLS_LANG=American_America.US7ASCII

if [ -z "$STATUS_TIMEOUT" ]; then STATUS_TIMEOUT=0; fi

if [ $LINUX -eq 1 ]; then
  MOUNT=/bin/mount
  XARGS='/usr/bin/xargs -r'
  FUSERMOUNT=/bin/fusermount
  LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib64
elif [ $SOLARIS -eq 1 ]; then
  MOUNT=/sbin/mount
  XARGS=/usr/bin/xargs
  UMOUNT=/usr/sbin/umount
  LD_LIBRARY_PATH=$ORACLE_HOME/lib:$ORACLE_HOME/rdbms/lib:/usr/lib:/lib
fi

DBFS_PWDFILE=$DBFS_PWDFILE_BASE.$$

export ORACLE_HOME LD_LIBRARY_PATH TNS_ADMIN
export STAT MOUNT_POINT PERL_ALARM_TIMEOUT SOLARIS LINUX
export PATH=$ORACLE_HOME/bin:$PATH
export STATUS_TIMEOUT
export LANG NLS_LANG

logit () {
  ### type: info, error, debug
  type=$1
  msg=$2
  if [ "$type" = "info" ]; then
    $ECHO $msg
    $LOGGER -p ${LOGGER_FACILITY}.info "$msg"
  elif [ "$type" = "error" ]; then
    $ECHO $msg
    $LOGGER -p ${LOGGER_FACILITY}.error "$msg"
  elif [ "$type" = "debug" ]; then
    $ECHO $msg
    $LOGGER -p ${LOGGER_FACILITY}.debug "$msg"
  fi
}

### must not be root
if [ `$ID -u` -eq 0 ]; then
  logit error "Run this as the Oracle software owner, not root"
  exit 1
fi

### determine how we were called, derive location
SCRIPTPATH=$0
SCRIPTNAME=`$BASENAME $SCRIPTPATH`

echo $SCRIPTPATH | grep ^/ > /dev/null 2>&1
if [ $? -ne 0 ]; then
  MYDIR=`pwd`
  SCRIPTPATH=${MYDIR}/${SCRIPTPATH}
fi

### must cd to a directory where the oracle owner can get CWD
cd /tmp

case "$1" in
'start')
  logit info "$SCRIPTNAME mounting DBFS at $MOUNT_POINT from database $DBNAME"

  ### check to see if it is already mounted
  $SCRIPTPATH status > /dev/null 2>&1
  if [ $? -eq 0 ]; then
    logit error "$MOUNT_POINT already mounted, use $SCRIPTNAME stop before attempting to start"
    $SCRIPTPATH status
    exit 1
  fi

  ### set the ORACLE_SID dynamically based on OCR info, if it is running
  export ORACLE_SID=$($SRVCTL status instance -d $DBNAME -n `$HN` | \
                      $GREP 'is running' | $AWK '{print $2}' )
  logit info "ORACLE_SID is $ORACLE_SID"

  ### if there's no SID defined locally or it isn't running, stop
  if [ -z "$ORACLE_SID" -a "$WALLET" = 'false' ]; then
    logit error "No running ORACLE_SID available on this host, exiting"
    exit 2
  fi

  ### if using password-based startup, use this
  if [ "$WALLET" = 'false' -a -n "$DBFS_PASSWD" ]; then
    $RMF $DBFS_PWDFILE
    if [ -f $DBFS_PWDFILE ]; then
      logit error "please remove $DBFS_PWDFILE and try again"
      exit 1
    fi

    $TOUCH $DBFS_PWDFILE
    $CHMOD 600 $DBFS_PWDFILE
    $ECHO $DBFS_PASSWD > $DBFS_PWDFILE

    logit info "spawning dbfs_client command using SID $ORACLE_SID"

    (nohup $DBFS_CLIENT ${DBFS_USER}@ -o $MOUNT_OPTIONS \
          $MOUNT_POINT < $DBFS_PWDFILE | $LOGGER -p ${LOGGER_FACILITY}.info 2>&1 & ) &

    $RMF $DBFS_PWDFILE

  elif [ "$WALLET" = true ]; then
    logit info "doing mount $MOUNT_POINT using SID $ORACLE_SID with wallet now"
    (nohup $DBFS_CLIENT } -o $MOUNT_OPTIONS,wallet \
          $MOUNT_POINT | $LOGGER -p ${LOGGER_FACILITY}.info 2>&1 & ) &
  fi

  ### allow time for the mount table update before checking it
  $SLEEP 1
  ### set return code based on success of mounting
  $SCRIPTPATH status > /dev/null 2>&1
  if [ $? -eq 0 ]; then
    logit info "Start -- ONLINE"
    exit 0
  else
    logit info "Start -- OFFLINE"
    exit 1
  fi
  ;;
'stop')
  $SCRIPTPATH status > /dev/null
  if [ $? -eq 0 ]; then
    logit info "unmounting DBFS from $MOUNT_POINT"
    if [ $LINUX -eq 1 ]; then
      logit info "umounting the filesystem using '$FUSERMOUNT -u $MOUNT_POINT'"
      $FUSERMOUNT -u $MOUNT_POINT
    elif [ $SOLARIS -eq 1 ]; then
      logit info "umounting the filesystem using '$UMOUNT $MOUNT_POINT'"
      $UMOUNT $MOUNT_POINT > /dev/null 2>&1
    fi
    $SCRIPTPATH status > /dev/null
    if [ $? -eq 0 ]; then
      logit error "Stop - stopped, but still mounted, error"
      exit 1
    else
      logit info "Stop - stopped, now not mounted"
      exit 0
    fi
  else
    logit error "filesystem $MOUNT_POINT not currently mounted, no need to stop"
  fi
  ;;

'check'|'status')
  ### check to see if it is mounted
  ### fire off a short process in perl to do the check (need the alarm builtin)
  logit debug "Checking status now"
  $PERL <<'TOT'
    $timeout = $ENV{'PERL_ALARM_TIMEOUT'};
    $SIG{ALRM} = sub {
      ### we have a problem and need to cleanup
      exit 3;
      die "timeout" ;
    };
    alarm $timeout;
    eval {
      $STATUSOUT=`$ENV{'STAT'} -f -c "%T" $ENV{'MOUNT_POINT'} 2>&1 `;
      chomp($STATUSOUT);
      if ( ( $ENV{'SOLARIS'} == 1 && $STATUSOUT eq 'uvfs' ) ||
           ( $ENV{'LINUX'} == 1   && $STATUSOUT eq 'UNKNOWN (0x65735546)' ) ) {
        ### status is okay
        exit 0;
      } elsif ( $STATUSOUT =~ /Transport endpoint is not connected/ ) {
        ### we have a problem, need to clean up
        exit 2;
      } else {
        ### filesystem is offline
        exit 1;
      }
    };

TOT

  RC=$?
  ### process return codes from the perl block
  if [ $RC -eq 3 ]; then
    STATUS_TIMEOUT=$(( $STATUS_TIMEOUT + 1 ))
    logit error "Found timeout while checking status, cleaning mount automatically"
    $SCRIPTPATH clean
    logit debug "Check -- OFFLINE"
    exit 1
  elif [ $RC -eq 2 ]; then
    STATUS_TIMEOUT=$(( $STATUS_TIMEOUT + 1 ))
    logit error "Found error while checking status, cleaning mount automatically"
    $SCRIPTPATH clean
    logit debug "Check -- OFFLINE"
    exit 1
  elif [ $RC -eq 1 ]; then
    logit debug "Check -- OFFLINE"
    exit 1
  elif [ $RC -eq 0 ]; then
    logit debug "Check -- ONLINE"
    exit 0
  fi
  ;;

'restart')
  logit info "restarting DBFS"
  $SCRIPTPATH stop
  $SLEEP 2
  $SCRIPTPATH start
  ;;

'clean'|'abort')
  logit info "cleaning up DBFS nicely using (fusermount -u|umount)"
  if [ $LINUX -eq 1 ]; then
    $FUSERMOUNT -u $MOUNT_POINT
  elif [ $SOLARIS -eq 1 ]; then
    $UMOUNT $MOUNT_POINT > /dev/null 2>&1
  fi
  $SLEEP 1
  FORCE_CLEANUP=0
  if [ $STATUS_TIMEOUT -gt 1 ]; then
    FORCE_CLEANUP=1
  else
    $SCRIPTPATH status > /dev/null
    if [ $? -eq 0 ]; then FORCE_CLEANUP=1; fi
  fi
  if [ $FORCE_CLEANUP -eq 1 ]; then
    logit error "tried (fusermount -u|umount), still mounted, now cleaning with (fusermount -u -z|umount -f) and kill"
    if [ $LINUX -eq 1 ]; then
      $FUSERMOUNT -u -z $MOUNT_POINT
    elif [ $SOLARIS -eq 1 ]; then
      echo "running umount -f now"
      $UMOUNT -f $MOUNT_POINT > /dev/null 2>&1
    fi
    if [ $LINUX -eq 1 ]; then
      PIDS=`$PS -ef | $GREP -w "$MOUNT_POINT" | $GREP dbfs_client| $GREP -v grep | \
        $AWK '{print $2}'`
      if [ -n "$PIDS" ]; then $KILL -9 $PIDS; fi
      PIDS=`$PS -ef | $GREP -w "$MOUNT_POINT" | $GREP mount.dbfs | $GREP -v grep | \
        $AWK '{print $2}'`
      if [ -n "$PIDS" ]; then $KILL -9 $PIDS; fi
    elif [ $SOLARIS -eq 1 ]; then
      PIDS=`$PS -ef | $GREP dbfs_client| $GREP -v grep | $AWK '{print $2}'`
      REALPIDS=' '
      for pid in $PIDS
      do
        ARGS=`pargs $pid`
        echo $ARGS | grep  "$MOUNT_POINT$"  > /dev/null
        RET=$?
        if [ $RET -eq 0 ]; then REALPIDS="$REALPIDS $pid"; fi
      done
      if [ -n "$REALPIDS" ]; then $KILL -9 $REALPIDS; fi
      ### do it a 2nd time to clean up others
      if [ -n "$REALPIDS" ]; then $KILL -9 $REALPIDS; fi
      PIDS=`$PS -ef | $GREP dbfs_client| $GREP -v grep | $AWK '{print $2}'`
      REALPIDS=' '
      for pid in $PIDS
      do
        ARGS=`pargs $pid`
        echo $ARGS | grep  "$MOUNT_POINT$"  > /dev/null
        RET=$?
        if [ $RET -eq 0 ]; then REALPIDS="$REALPIDS $pid"; fi
      done
      if [ -n "$REALPIDS" ]; then $KILL -9 $REALPIDS; fi

    fi
    exit 1
  fi
  ;;

*)
  $ECHO "Usage: $SCRIPTNAME { start | stop | check | status | restart | clean | abort }"
  ;;

esac

3.8 將修改後的檔案複製grid的使用者的ORACLE_HOME/crs/scripts下,並複製到其它節點
(root)# dcli -g ~/dbs_group -l root -d /u01/app/11.2.0.3/grid/crs/script. -f /tmp/mount-dbfs.sh
(root)# dcli -g ~/dbs_group -l root chown oracle:dba /u01/app/11.2.0.3/grid/crs/script/mount-dbfs.sh
(root)# dcli -g ~/dbs_group -l root chmod 750 /u01/app/11.2.0.3/grid/crs/script/mount-dbfs.sh

3.9註冊dbfs資源到ocr庫
在grid的家目錄生成檔案,內容如下:

##### start script. add-dbfs-resource.sh
#!/bin/bash
ACTION_SCRIPT=/u01/app/11.2.0/grid/crs/script/mount-dbfs.sh
RESNAME=dbfs_mount
DBNAME=test
DBNAMEL=`echo $DBNAME | tr A-Z a-z`
ORACLE_HOME=/u01/app/11.2.0.3/grid
PATH=$ORACLE_HOME/bin:$PATH
export PATH ORACLE_HOME
crsctl add resource $RESNAME \
  -type local_resource \
  -attr "ACTION_SCRIPT=$ACTION_SCRIPT, \
         CHECK_INTERVAL=30,RESTART_ATTEMPTS=10, \
         START_DEPENDENCIES='hard(ora.$DBNAMEL.db)pullup(ora.$DBNAMEL.db)',\
         STOP_DEPENDENCIES='hard(ora.$DBNAMEL.db)',\
         SCRIPT_TIMEOUT=300"
##### end script. add-dbfs-resource.sh
注意上述檔案內容,ACTION_SCRIPT,RESNAME,DBNAME,ORACLE_HOME 需要根據實際需求 進行修改
Then run this as the Grid Infrastructure owner (typically oracle) on one database server only:
(oracle)$ sh ./add-dbfs-resource.sh

3.10(grid)$ sh ./add-dbfs-resource.sh

When successful, this command has no output.

3.11 啟動dbfs_mount 資源
(oracle)$ <GI_HOME>/bin/crsctl stat res dbfs_mount -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS      
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
dbfs_mount
               OFFLINE OFFLINE      test1                                    
               OFFLINE OFFLINE      test2

啟動
(oracle)$ <GI_HOME>/bin/crsctl start resource dbfs_mount
CRS-2672: Attempting to start 'dbfs_mount' on 'test1'
CRS-2672: Attempting to start 'dbfs_mount' on 'test2'
CRS-2676: Start of 'dbfs_mount' on 'test2' succeeded
CRS-2676: Start of 'dbfs_mount' on 'test1' succeeded
(oracle)$ <GI_HOME>/bin/crsctl stat res dbfs_mount -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS      
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
dbfs_mount
               ONLINE  ONLINE       test1                                    
               ONLINE  ONLINE       test2
              
結束                                 

來自 “ ITPUB部落格 ” ,連結:http://blog.itpub.net/28389881/viewspace-2136845/,如需轉載,請註明出處,否則將追究法律責任。

相關文章