刪除ACFS系統碰到的問題

首先嚐試UMOUNT檔案系統：

[oracle@xsh-server1 ~]$ df -k
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00 435653856 192338280 220828536 47% /
/dev/sda1                       101086    16919     78948     18% /boot
tmpfs                           65968984  195784    65773200  1%  /dev/shm
/dev/asm/acfsdgvol1-57          624951296 165746644 459204652 27% /backup

[root@xsh-server1 ~]# umount -t acfs /backup

在節點2上執行同樣的UMOUNT操作：

[root@xsh-server2 asm]# umount -t acfs /backup/

在節點1上取消acfs的註冊資訊：

[root@xsh-server1 ~]# acfsutil registry -m /dev/asm/acfsdgvol1-57
/backup
[root@xsh-server1 ~]# acfsutil registry -d /dev/asm/acfsdgvol1-57
acfsutil registry: successfully removed ACFS volume /dev/asm/acfsdgvol1-57 from Oracle Registry

但是隨後刪除acfs檔案系統的時候出現了錯誤：

[root@xsh-server1 ~]# acfsutil rmfs /dev/asm/acfsdgvol1-57
acfsutil rmfs: ACFS-00511: /dev/asm/acfsdgvol1-57 is mounted on at least one node of the cluster.
acfsutil rmfs: ACFS-03014: file system not removed

由於檔案系統無法刪除，是的VOLUME同樣無法刪除：

[root@xsh-server1 ~]# su - grid
[grid@xsh-server1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.2.0 Production on Thu Aug 11 11:15:02 2011

Copyright (c) 1982, 2010, Oracle. All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> SELECT GROUP_NUMBER, VOLUME_NAME, SIZE_MB, STATE, USAGE, VOLUME_DEVICE, MOUNTPATH
2 FROM V$ASM_VOLUME;

GROUP_NUMBER VOLUME_NAME SIZE_MB STATE   USAGE VOLUME_DEVICE          MOUNTPATH
------------ ----------- ------- ------- ----- ---------------------- ----------
           4 ACFSDGVOL1  610304  ENABLED ACFS  /dev/asm/acfsdgvol1-57 /backup

SQL> SELECT GROUP_NUMBER, NAME FROM V$ASM_DISKGROUP WHERE GROUP_NUMBER = 4;

GROUP_NUMBER NAME
------------ ------------------------------
           4 ACFSDG

SQL> ALTER DISKGROUP ACFSDG DISABLE VOLUME ACFSDGVOL1;
ALTER DISKGROUP ACFSDG DISABLE VOLUME ACFSDGVOL1
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15468: volume 'ACFSDGVOL1' in diskgroup 'ACFSDG' is currently being accessed

嘗試在ASMCMD中操作，同樣無法刪除VOLUME：

[grid@xsh-server1 ~]$ asmcmd
ASMCMD> volinfo -a
Diskgroup Name: ACFSDG

Volume Name: ACFSDGVOL1
Volume Device: /dev/asm/acfsdgvol1-57
State: ENABLED
Size (MB): 610304
Resize Unit (MB): 256
Redundancy: UNPROT
Stripe Columns: 4
Stripe Width (K): 128
Usage: ACFS
Mountpath: /backup

ASMCMD> voldelete -G ACFSDG ACFSDGVOL1
ORA-15032: not all alterations performed
ORA-15468: volume 'ACFSDGVOL1' in diskgroup 'ACFSDG' is currently being accessed (DBD ERROR: OCIStmtExecute)
ASMCMD> voldisable -a
ORA-15032: not all alterations performed
ORA-15468: volume 'ACFSDGVOL1' in diskgroup 'ACFSDG' is currently being accessed (DBD ERROR: OCIStmtExecute)
ASMCMD> quit
[grid@xsh-server1 ~]$ exit

嘗試各種方法，都無法以常規的方法刪除ACFS檔案系統和ASM中建立的VOLUME，沒有辦法，最終只是使用強制的方法來刪除：

[grid@xsh-server1 ~]$ id
uid=1300(grid) gid=1000(oinstall) groups=1000(oinstall),1200(dba) context=user_u:system_r:unconfined_t
[grid@xsh-server1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.2.0 Production on Thu Aug 11 12:59:35 2011

Copyright (c) 1982, 2010, Oracle. All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> set pages 100 lines 120
SQL> SELECT GROUP_NUMBER, NAME FROM V$ASM_DISKGROUP;

GROUP_NUMBER NAME
------------ ------------------------------
           1 CRSDG
           2 DATADG
           3 FRADG
           4 ACFSDG

SQL> DROP DISKGROUP ACFSDG;
DROP DISKGROUP ACFSDG
*
ERROR at line 1:
ORA-15039: diskgroup not dropped
ORA-15053: diskgroup "ACFSDG" contains existing files

SQL> DROP DISKGROUP ACFSDG INCLUDING CONTENTS;
DROP DISKGROUP ACFSDG INCLUDING CONTENTS
*
ERROR at line 1:
ORA-15039: diskgroup not dropped
ORA-15027: active use of diskgroup "ACFSDG" precludes its dismount

SQL> DROP DISKGROUP ACFSDG FORCE INCLUDING CONTENTS;
DROP DISKGROUP ACFSDG FORCE INCLUDING CONTENTS
*
ERROR at line 1:
ORA-15039: diskgroup not dropped
ORA-15230: diskgroup 'ACFSDG' does not require the FORCE option

SQL> ALTER DISKGROUP ACFSDG DISMOUNT;
ALTER DISKGROUP ACFSDG DISMOUNT
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15027: active use of diskgroup "ACFSDG" precludes its dismount

SQL> ALTER DISKGROUP ACFSDG DISMOUNT FORCE;

Diskgroup altered.

SQL> DROP DISKGROUP ACFSDG;
DROP DISKGROUP ACFSDG
*
ERROR at line 1:
ORA-15039: diskgroup not dropped
ORA-15001: diskgroup "ACFSDG" does not exist or is not mounted

SQL> DROP DISKGROUP ACFSDG FORCE INCLUDING CONTENTS;
DROP DISKGROUP ACFSDG FORCE INCLUDING CONTENTS
*
ERROR at line 1:
ORA-15039: diskgroup not dropped
ORA-15073: diskgroup ACFSDG is mounted by another ASM instance

利用DISMOUNT FORCE的選項，終於將磁碟組ACFSDG解除安裝，由於另外的ASM例項還載入這個磁碟組，因此這個磁碟組暫時還不能刪除。

登入另外的例項，執行DISMOUNT操作：

[root@xsh-server2 dev]# su - grid
[grid@xsh-server2 ~]$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.2.0 Production on Thu Aug 11 13:11:55 2011

Copyright (c) 1982, 2010, Oracle. All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> set pages 100 lines 120
SQL> select group_number, name, state from v$asm_diskgroup;

GROUP_NUMBER NAME STATE
------------ ------------------------------ -----------
1 CRSDG MOUNTED
2 DATADG MOUNTED
3 FRADG MOUNTED
4 ACFSDG MOUNTED

SQL> alter diskgroup acfsdg dismount;

Diskgroup altered.

下面會到第一個節點，再次執行刪除操作：

SQL> DROP DISKGROUP ACFSDG;
DROP DISKGROUP ACFSDG
*
ERROR at line 1:
ORA-15039: diskgroup not dropped
ORA-15001: diskgroup "ACFSDG" does not exist or is not mounted

SQL> DROP DISKGROUP ACFSDG FORCE INCLUDING CONTENTS;

Diskgroup dropped.

SQL> SELECT GROUP_NUMBER, NAME FROM V$ASM_DISKGROUP;

GROUP_NUMBER NAME
------------ ------------------------------
1 CRSDG
2 DATADG
3 FRADG

SQL> EXIT
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options
[grid@xsh-server1 ~]$ df -k
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00 435653856 192353508 220813308 47% /
/dev/sda1                       101086    16919     78948     18% /boot
tmpfs                           65968984  195784    65773200  1%  /dev/shm

最後利用ORACLEASM刪除對應的裝置：

[grid@xsh-server1 ~]$ su -
Password:
[root@xsh-server1 ~]# oracleasm listdisks
VOL1
VOL2
VOL3
VOLACFS
[root@xsh-server1 ~]# oracleasm deletedisk VOLACFS
Clearing disk header: done
Dropping disk: done

在另外的節點執行檢查ASM磁碟組和ORACLEASM裝置，確認節點1上的操作在兩個節點上都已經生效：

SQL> select group_number, name, state from v$asm_diskgroup;

GROUP_NUMBER NAME STATE
------------ ------------------------------ -----------
1 CRSDG MOUNTED
2 DATADG MOUNTED
3 FRADG MOUNTED

SQL> EXIT
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options
[grid@xsh-server2 ~]$ exit
logout

[root@xsh-server2 dev]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Cleaning disk "VOLACFS"
Scanning system for ASM disks...
[root@xsh-server2 dev]# oracleasm listdisks
VOL1
VOL2
VOL3

雖然最終透過強制的方式刪除了ACFS檔案系統和對應的磁碟組，但是這種強制的方式很可能在作業系統級別上還留下一下沒有清除的資訊。

顯然ACFS這項新技術還存在很多的bug，而且在處理這個問題時也發現，即使是在metalink上，關於ACFS的資訊也非常少，雖然存在一個和當前幾乎完全一樣的問題，但是並沒有下文，Oracle也沒有給出任何解決的方法。

如果打算在產品環境中使用ACFS，還是要經過大量的測試才行。