AsM丟魂記?(二)

anycall2010發表於2009-03-05

昨天的問題,貌似已經結束。今天朋友又打來電話,資料庫又當機了。難道ASM的磁碟不夠,整個系統在使用者寫入資料庫的時候,磁碟滿了,無法寫入,造成資料庫崩潰?

1、問題繼續分析:

經過我不懈的努力,將節點恢復到目前狀態:

[oracle@rac2 bin]$ crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora.orcl.db    application    ONLINE    OFFLINE
ora....l1.inst application    ONLINE    OFFLINE
ora....l2.inst application    ONLINE    OFFLINE
ora....SM1.asm application    ONLINE    ONLINE    rac1
ora....C1.lsnr application    ONLINE    ONLINE    rac1
ora.rac1.gsd   application    ONLINE    ONLINE    rac1
ora.rac1.ons   application    ONLINE    ONLINE    rac1
ora.rac1.vip   application    ONLINE    ONLINE    rac1
ora....SM2.asm application    ONLINE    ONLINE    rac2
ora....C2.lsnr application    ONLINE    ONLINE    rac2
ora.rac2.gsd   application    ONLINE    ONLINE    rac2
ora.rac2.ons   application    ONLINE    ONLINE    rac2
ora.rac2.vip   application    ONLINE    ONLINE    rac2

資料庫的例項始終是啟動不了。我一直在納悶,今天見鬼了。。。。。

嘗試手工啟動資料庫:

[oracle@rac2 bin]$ sqlplus /nolog

SQL*Plus: Release 10.2.0.2.0 - Production on Thu Mar 5 01:06:14 2009

Copyright (c) 1982, 2005, Oracle.  All Rights Reserved.

SQL> conn sys/oracle as sysdba
Connected to an idle instance.
SQL> startup
ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '+DATA/orcl/spfileorcl.ora'
ORA-17503: ksfdopn:2 Failed to open file +DATA/orcl/spfileorcl.ora
ORA-15077: could not locate ASM instance serving a required diskgroup
SQL>
SQL> exit
Disconnected

沒有磁碟?為什麼?

[oracle@rac2 bin]$ sqlplus /nolog

SQL*Plus: Release 10.2.0.2.0 - Production on Thu Mar 5 01:06:14 2009

Copyright (c) 1982, 2005, Oracle.  All Rights Reserved.

SQL> conn sys/oracle as sysdba
Connected to an idle instance.
SQL> startup
ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '+DATA/orcl/spfileorcl.ora'
ORA-17503: ksfdopn:2 Failed to open file +DATA/orcl/spfileorcl.ora
ORA-15077: could not locate ASM instance serving a required diskgroup
SQL>
SQL> exit
Disconnected

[oracle@rac2 bin]$ crs_start ora.orcl.orcl2.inst
Attempting to start `ora.orcl.orcl2.inst` on member `rac2`
Start of `ora.orcl.orcl2.inst` on member `rac2` failed.
rac1 : CRS-1018: Resource ora.rac2.vip (application) is already running on rac2


CRS-0215: Could not start resource 'ora.orcl.orcl2.inst'.

這樣貌似也啟動不了,報沒有看到資源的錯誤。為什麼?ASM已經啟動了!

確認一下:

[oracle@rac2 bin]$ ps -ef|grep ASM
oracle   30320     1  0 00:06 ?        00:00:00 asm_pmon_+ASM2
oracle   30322     1  0 00:06 ?        00:00:00 asm_diag_+ASM2
oracle   30324     1  0 00:06 ?        00:00:00 asm_psp0_+ASM2
oracle   30326     1  0 00:06 ?        00:00:01 asm_lmon_+ASM2
oracle   30328     1  0 00:06 ?        00:00:00 asm_lmd0_+ASM2
oracle   30330     1  0 00:06 ?        00:00:00 asm_lms0_+ASM2
oracle   30334     1  0 00:06 ?        00:00:00 asm_mman_+ASM2
oracle   30336     1  0 00:06 ?        00:00:00 asm_dbw0_+ASM2
oracle   30348     1  0 00:06 ?        00:00:00 asm_lgwr_+ASM2
oracle   30365     1  0 00:06 ?        00:00:00 asm_ckpt_+ASM2
oracle   30367     1  0 00:06 ?        00:00:00 asm_smon_+ASM2
oracle   30369     1  0 00:06 ?        00:00:00 asm_rbal_+ASM2
oracle   30371     1  0 00:06 ?        00:00:00 asm_gmon_+ASM2
oracle   30423     1  0 00:06 ?        00:00:00 asm_lck0_+ASM2
oracle   30491     1  0 00:06 ?        00:00:00 /u01/app/oracle/product/10.2.0/asm/bin/racgimon daemon ora.rac2.ASM2.asm
oracle   25371 16831  0 01:08 pts/2    00:00:00 grep ASM

看來是沒問題!ASM程式是啟動的。

root@rac2 ~]# /etc/init.d/oracleasm listdisks
[root@rac2 ~]# /etc/init.d/oracleasm enable
Writing Oracle ASM library driver configuration:           [  OK  ]
Scanning system for ASM disks:                             [  OK  ]
[root@rac2 ~]# /etc/init.d/oracleasm listdisks
掃描一下ASM是否啟動,發現找不到ASM新增的磁碟。

基於作業系統層面,檢視一下磁碟(其實這一步做不做都是多餘)

[root@rac2 ~]# df -m
Filesystem           1M-blocks      Used Available Use% Mounted on
/dev/sda2                10077       884      8681  10% /
/dev/sda1                  190        14       168   8% /boot
none                      3986         0      3986   0% /dev/shm
/dev/sda7                 4031        47      3779   2% /tmp
/dev/sda8                95178     30170     60174  34% /u01
/dev/sda6                10077      2151      7414  23% /usr
/dev/sda5                10077       159      9407   2% /var
172.16.1.160:/u01/rac1Arch
                         95186     22229     68122  25% /u01/rac1Arch
/dev/sdb1                 1024       112       913  11% /u03

但是,我做到這個步驟,就明白可能儲存有問題:

[root@rac2 ~]# fdisk  -l

Disk /dev/sda: 146.6 GB, 146695782400 bytes
255 heads, 63 sectors/track, 17834 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          25      200781   83  Linux
/dev/sda2              26        1330    10482412+  83  Linux
/dev/sda3            1331        2375     8393962+  82  Linux swap
/dev/sda4            2376       17834   124174417+   5  Extended
/dev/sda5            2376        3680    10482381   83  Linux
/dev/sda6            3681        4985    10482381   83  Linux
/dev/sda7            4986        5507     4192933+  83  Linux
/dev/sda8            5508       17834    99016596   83  Linux

Disk /dev/sdb: 1073 MB, 1073741824 bytes
34 heads, 61 sectors/track, 1011 cylinders
Units = cylinders of 2074 * 512 = 1061888 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        1011     1048376+  83  Linux

Disk /dev/sdd: 1073 MB, 1073741824 bytes
34 heads, 61 sectors/track, 1011 cylinders
Units = cylinders of 2074 * 512 = 1061888 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1               1        1011     1048376+  83  Linux

根本沒發現ASM的卷組!看來該卷組根本沒有掛載上來!

[root@rac2 ~]# ls -lat /dev/raw/raw*
ls: /dev/raw/raw*: No such file or directory

沒有這個目錄!

[root@rac2 dev]# su - oracle
[oracle@rac2 ~]$ srvctl config asm -n rac1
+ASM1 /u01/app/oracle/product/10.2.0/asm
[oracle@rac2 ~]$ srvctl config asm -n rac2
+ASM2 /u01/app/oracle/product/10.2.0/asm
[oracle@rac2 ~]$ export ORACLE_SID=+ASM2
[oracle@rac2 ~]$ sqlplus /nolog

SQL*Plus: Release 10.2.0.2.0 - Production on Thu Mar 5 01:42:05 2009

Copyright (c) 1982, 2005, Oracle.  All Rights Reserved.

SQL> conn / as sysdba
Connected to an idle instance.
SQL> exit
Disconnected
[oracle@rac2 ~]$ srvctl status asm -n rac1
ASM instance +ASM1 is running on node rac1.
[oracle@rac2 ~]$

這樣就明白了,ASM的卷組丟失,但是ASM的程式仍舊執行,看來這次ASM的魂魄是丟失啦。。。。。

來自 “ ITPUB部落格 ” ,連結:http://blog.itpub.net/8334342/viewspace-561445/,如需轉載,請註明出處,否則將追究法律責任。

相關文章