HP-UX 11.31 安裝RAC 新增共享磁碟的問題(兩種辦法)

tonyzhou_cn發表於2012-12-15
HPUX 11.31下更改裝置名(測試透過)
個人還是推薦用第二種辦法,第一種感覺危險,因為我不太熟悉HP-UX。不知道ioinit具體用法
 在HPUX(特別是HPUX 11.31,不能安裝其它多路徑軟體)下安裝ORACLE RAC時,建ASM磁碟組時有時會碰到兩個節點上儲存對映過來的磁碟磁碟機代號不一致,有時是物理路徑一致,但聚合後路徑不一致,有時甚至物理路徑也不一致,此時就需將兩個節點上的磁碟磁碟機代號修改為一致,上網找了很久,找到一篇HPUX網站上的一篇文件,將實施過程記錄下來,以備後查:
1.確認LUN ID與磁碟的對應關係,確認兩個節點磁碟的對應關係
首先找到儲存對映到主機的磁碟的LUN ID,並將兩臺主機之間相同LUN ID的磁碟對應關係找到,因為HP 11.31使用的是HP自己的多路徑軟體,如果儲存是HDS的話可以透過HDS提供的工具指令碼來找到磁碟和LUN ID的對應關係:
HPUX下Lun對應關係檢視方式如下:
請將附件的工具inqraid.hpux以二進位制模式ftp上傳到主機/tmp目錄,執行以下命令
a>cd /tmp
b>chmod +x /tmp/inqraid.hpux
c>ioscan -fun | grep -e rdisk -e rdsk | ./inqraid.hpux -CLI -fx
d>ioscan -fun | grep -e rdisk -e rdsk | ./inqraid.hpux -CLI -fx -fg
-h可以看到更多選項。。。。。。  -------------------------------------------因為沒有這個工具那麼可以使用查詢WWID來一一對應也可以
scsimgr lun_map -D /dev/rdisk/diskxx  |grep WWID   這裡是關鍵,不能弄錯了,每個盤一一對應
根據命令的輸出結果可以核對出主機裝置名與儲存裝置Lun的對應關係。

如果是EMC或者其它的需要找儲存廠商瞭解,如果找不到辦法就只有用最笨的辦法,一塊盤一塊盤的對映來找磁碟和LUN ID的對應關係
以下對映了5塊磁碟,儲存是EMC CX4-240,在兩個節點上磁碟與LUN ID的對應關係如下:
LUNID                                           節點1磁碟名                                                       節點2磁碟名        
lun 36                                            /dev/rdisk/disk216                                            /dev/rdisk/disk191
lun 37                                            /dev/rdisk/disk217                                            /dev/rdisk/disk192
lun 38                                            /dev/rdisk/disk218                                            /dev/rdisk/disk193
lun 39                                            /dev/rdisk/disk219                                            /dev/rdisk/disk194
lun 40                                            /dev/rdisk/disk220                                            /dev/rdisk/disk195

檢視兩個節點上聚合後和聚合前磁碟對應關係
                  節點1                                                                                                  節點2
nodeA#ioscan -m dsf                              nodeB#ioscan -m dsf   
Persistent DSF          Legacy DSF(s)       Persistent DSF     Legacy DSF(s)
===================================     =================================== 
/dev/rdisk/disk216 /dev/rdsk/c36t0d0            /dev/rdisk/disk191 /dev/rdsk/c37t0d0
                   /dev/rdsk/c35t0d0                       /dev/rdsk/c36t0d0
                   /dev/rdsk/c38t0d0                       /dev/rdsk/c38t0d0
                   /dev/rdsk/c37t0d0                       /dev/rdsk/c39t0d0
/dev/rdisk/disk217 /dev/rdsk/c36t0d1            /dev/rdisk/disk192 /dev/rdsk/c37t0d1
                   /dev/rdsk/c35t0d1                       /dev/rdsk/c36t0d1
                   /dev/rdsk/c38t0d1                       /dev/rdsk/c38t0d1
                   /dev/rdsk/c37t0d1                       /dev/rdsk/c39t0d1
/dev/rdisk/disk218 /dev/rdsk/c36t0d2        /dev/rdisk/disk193 /dev/rdsk/c37t0d2
                   /dev/rdsk/c35t0d2                       /dev/rdsk/c36t0d2
                   /dev/rdsk/c38t0d2                       /dev/rdsk/c38t0d2
                   /dev/rdsk/c37t0d2                       /dev/rdsk/c39t0d2
/dev/rdisk/disk219 /dev/rdsk/c36t0d3             /dev/rdisk/disk194 /dev/rdsk/c37t0d3
                   /dev/rdsk/c35t0d3                       /dev/rdsk/c36t0d3
                   /dev/rdsk/c38t0d3                       /dev/rdsk/c38t0d3
                   /dev/rdsk/c37t0d3                       /dev/rdsk/c39t0d3
/dev/rdisk/disk220 /dev/rdsk/c35t0d4             /dev/rdisk/disk195 /dev/rdsk/c37t0d4
                   /dev/rdsk/c36t0d4                       /dev/rdsk/c36t0d4
                  /dev/rdsk/c38t0d4                       /dev/rdsk/c38t0d4
                   /dev/rdsk/c37t0d4                       /dev/rdsk/c39t0d4
節點1                                                    節點2
 /dev/rdisk/disk216                     /dev/rdisk/disk191
 /dev/rdisk/disk217                     /dev/rdisk/disk192
 /dev/rdisk/disk218                     /dev/rdisk/disk193
 /dev/rdisk/disk219                     /dev/rdisk/disk194
 /dev/rdisk/disk220                     /dev/rdisk/disk195
根據以上情況決定以第一個節點上的磁碟機代號為準,將第二個節點上的磁碟機代號進行更改:
2.根據磁碟路徑資訊生成IO初始化的infile檔案
查詢磁碟聚合後路徑:
nodeB#ioscan -fnNkCdisk                                                   
Class    I H/W Path Driver        S/W State  H/W Type    Description    
===========================================================================   
disk 191 64000/0xfa00/0x1e esdisk        CLAIMED    DEVICE DGC CX4-240WDR5
                   /dev/disk/disk191            /dev/rdisk/disk191                      
disk 192 64000/0xfa00/0x1f esdisk        CLAIMED    DEVICE DGC CX4-240WDR5
                   /dev/disk/disk192          /dev/rdisk/disk192                      
disk 193 64000/0xfa00/0x20 esdisk        CLAIMED    DEVICE DGC CX4-240WDR5
                   /dev/disk/disk193          /dev/rdisk/disk193                      
disk 194 64000/0xfa00/0x21 esdisk        CLAIMED    DEVICE DGC CX4-240WDR5
                   /dev/disk/disk194          /dev/rdisk/disk194                      
disk 195 64000/0xfa00/0x22 esdisk        CLAIMED    DEVICE DGC CX4-240WDR5
                     /dev/disk/disk195        /dev/rdisk/disk195                                           
.....                                                                                                
查詢磁碟聚合前路徑:                                                                                               
nodeB#ioscan -fnkCdisk                                                       
Class    I H/W Path     Driver        S/W State  H/W Type    Description   
============================================================================     
disk 28 0/3/1/0.1.21.0.0.0.0 sdisk          CLAIMED    DEVICE DGC CX4-240WDR5
                      /dev/dsk/c37t0d0  /dev/rdsk/c37t0d0                       
disk 149 0/3/1/0.1.21.0.0.0.1 sdisk          CLAIMED    DEVICE DGC CX4-240WDR5
                      /dev/dsk/c37t0d1 /dev/rdsk/c37t0d1                       
disk 150 0/3/1/0.1.21.0.0.0.2 sdisk          CLAIMED    DEVICE DGC CX4-240WDR5
                      /dev/dsk/c37t0d2 /dev/rdsk/c37t0d2                       
disk 151 0/3/1/0.1.21.0.0.0.3 sdisk          CLAIMED    DEVICE DGC CX4-240WDR5
                      /dev/dsk/c37t0d3 /dev/rdsk/c37t0d3                       
disk 152 0/3/1/0.1.21.0.0.0.4 sdisk          CLAIMED    DEVICE DGC CX4-240WDR5
                      /dev/dsk/c37t0d4 /dev/rdsk/c37t0d4                       
disk 153 0/3/1/0.1.21.0.0.0.5 sdisk          CLAIMED    DEVICE DGC CX4-240WDR5
                      /dev/dsk/c37t0d5 /dev/rdsk/c37t0d5                       
disk 154 0/3/1/0.1.21.0.0.0.6 sdisk          CLAIMED    DEVICE DGC CX4-240WDR5
                      /dev/dsk/c37t0d6 /dev/rdsk/c37t0d6                       
disk 155 0/3/1/0.1.21.0.0.0.7 sdisk          CLAIMED    DEVICE DGC CX4-240WDR5
                      /dev/dsk/c37t0d7 /dev/rdsk/c37t0d7                       
disk 156 0/3/1/0.1.21.0.0.1.0 sdisk          CLAIMED    DEVICE DGC CX4-240WDR5
                      /dev/dsk/c37t1d0 /dev/rdsk/c37t1d0                       
disk 157 0/3/1/0.1.21.0.0.1.1 sdisk          CLAIMED    DEVICE DGC CX4-240WDR5
                      /dev/dsk/c37t1d1 /dev/rdsk/c37t1d1                       
disk 158 0/3/1/0.1.21.0.0.1.2 sdisk          CLAIMED    DEVICE DGC CX4-240WDR5
                      /dev/dsk/c37t1d2 /dev/rdsk/c37t1d2                       
....                                                                             
disk 174 0/7/1/0.1.23.0.0.1.4 sdisk          CLAIMED    DEVICE DGC CX4-240WDR5
                      /dev/dsk/c38t1d4 /dev/rdsk/c38t1d4                       
disk 175 0/7/1/0.1.23.0.0.1.5 sdisk          CLAIMED    DEVICE DGC CX4-240WDR5
                      /dev/dsk/c38t1d5 /dev/rdsk/c38t1d5                       
disk 176 0/7/1/0.1.23.0.0.1.6 sdisk          CLAIMED    DEVICE DGC CX4-240WDR5
                      /dev/dsk/c38t1d6 /dev/rdsk/c38t1d6
生成infile檔案
# cd /tmp
 # ioscan -kNf | grep ^disk | awk '{printf "%s %s %s\n",$3,$1,$2}' > infile
 # ioscan -kf | grep ^disk | awk '{printf "%s %s %s\n",$3,$1,$2}' >> infile
修改生成的infile檔案,以64000/0xfa00/0x1e開始的行,除了需要修改裝置名的,其它的都刪除掉:
 # vi infile                                        
 nodeBtmp/hpdisk#vi infile                      
"infile" 109 lines, 3144 characters                 
64000/0xfa00/0x1e disk 191
64000/0xfa00/0x1f disk 192
64000/0xfa00/0x20 disk 193
64000/0xfa00/0x21 disk 194
64000/0xfa00/0x22 disk 195                          
0/0/2/1.0.16 disk 3                          
0/3/1/0.1.21.0.0.0.0 disk 28                        
……                      
0/3/1/0.1.22.0.0.0.1 disk 118
0/3/1/0.1.22.0.0.0.2 disk 119                  
將紅色部分修改成以下內容:
64000/0xfa00/0x1e disk 216 ------------------這裡根據WWID對應的順序來變更。
64000/0xfa00/0x1f disk 217
64000/0xfa00/0x20 disk 218
64000/0xfa00/0x21 disk 219
64000/0xfa00/0x22 disk 220
0/0/2/1.0.16 disk 3
0/3/1/0.1.21.0.0.0.0 disk 28
……
0/3/1/0.1.22.0.0.0.1 disk 118
0/3/1/0.1.22.0.0.0.2 disk 119
3.初始化修改後裝置檔案
執行生成的檔案:
nodeB #ioinit -f ./infile
ioinit: Input is identical to kernel, line ignored
Input line 2: 0/0/2/1.0.16 disk 3

ioinit: Input is identical to kernel, line ignored  
Input line 83: 0/7/1/0.1.23.0.0.1.6 disk 176
 4.刪除之前舊的裝置
nodeB#rmsf /dev/disk/disk191       /dev/rdisk/disk191
nodeB#rmsf /dev/disk/disk192       /dev/rdisk/disk192
nodeB#rmsf /dev/disk/disk193       /dev/rdisk/disk193
nodeB#rmsf /dev/disk/disk194       /dev/rdisk/disk194
nodeB#rmsf /dev/disk/disk195       /dev/rdisk/disk195
5.驗證查詢                 
nodeBtmp#ioscan -m dsf
Persistent DSF          Legacy DSF(s)
========================================
/dev/rdisk/disk1           /dev/rdsk/c34t0d0
                        /dev/rdsk/c35t0d0
/dev/rdisk/disk41          /dev/rdsk/c34t0d2
                        /dev/rdsk/c35t0d2
.....
/dev/rdisk/disk216         /dev/rdsk/c37t0d0
                        /dev/rdsk/c36t0d0
                        /dev/rdsk/c38t0d0
                        /dev/rdsk/c39t0d0
/dev/rdisk/disk217         /dev/rdsk/c37t0d2
                        /dev/rdsk/c36t0d2
                        /dev/rdsk/c38t0d2
                        /dev/rdsk/c39t0d2
/dev/rdisk/disk218         /dev/rdsk/c37t0d4
                        /dev/rdsk/c36t0d4
                        /dev/rdsk/c38t0d4
                        /dev/rdsk/c39t0d4
/dev/rdisk/disk219         /dev/rdsk/c37t0d6
                        /dev/rdsk/c36t0d6
                        /dev/rdsk/c38t0d6
                        /dev/rdsk/c39t0d6
/dev/rdisk/disk220         /dev/rdsk/c37t1d0
                        /dev/rdsk/c36t1d0
                        /dev/rdsk/c38t1d0
                        /dev/rdsk/c39t1d0

第二種辦法
hp-ux 11g RAC asm 新增磁碟 .
system : hp-ux B.11.31   --檢視hp-ux版本 #uname -a
storage : hp
db : 10.2.0.5.0
由於RAC環境下的資料儲存空間快撐滿了,於是在儲存上劃了4個256G的LUN指定給RAC(IP1:10.1.10.56;IP2:10.1.10.57)
1、檢視各HOST下現有的磁碟路徑,並將LUN指定給HOST後,檢視指定後的磁碟路徑,對比:
IP1:10.1.10.56
透過#smh, 在SMH->Disks and File Systems->Disks下
Path(s)           Device File(s)    Use    VG Names   Paths   Size(GB) Description
----------------------------------------------------------------------------------------------------------------------
0/0/0/5/0/0/0.21.13.0.0.0.2 /dev/dsk/c22t0d2  unused  -         8       256.00  HP      HSV450
0/0/0/5/0/0/0.21.13.0.0.0.3 /dev/dsk/c22t0d3  unused  -         8       256.00  HP      HSV450
0/0/0/5/0/0/0.21.13.0.0.0.4 /dev/dsk/c22t0d4  unused  -         8       256.00  HP      HSV450
0/0/0/5/0/0/0.21.13.0.0.0.5 /dev/dsk/c22t0d5  unused  -         8       256.00  HP      HSV450
指定磁碟後
Path(s)           Device File(s)    Use    VG Names   Paths   Size(GB) Description
------------------------------------------------------------------------------------------------------
0/0/0/5/0/0/0.21.13.0.0.1.3 /dev/dsk/c48t1d3  unused  -         8       256.00  HP      HSV450
0/0/0/5/0/0/0.21.15.0.0.1.4 /dev/dsk/c50t1d4  unused  -         8       256.00  HP      HSV450
0/0/0/5/0/0/1.22.13.0.0.1.5 /dev/dsk/c60t1d5  unused  -         8       256.00  HP      HSV450     
0/0/0/5/0/0/0.21.13.0.0.1.6 /dev/dsk/c48t1d6  unused  -         8       256.00  HP      HSV450
0/0/0/5/0/0/0.21.13.0.0.0.3 /dev/dsk/c48t0d3  unused  -         8       256.00  HP      HSV450
0/0/0/5/0/0/0.21.13.0.0.0.4 /dev/dsk/c48t0d4  unused  -         8       256.00  HP      HSV450
0/0/0/5/0/0/0.21.13.0.0.0.5 /dev/dsk/c48t0d5  unused  -         8       256.00  HP      HSV450
0/0/0/5/0/0/0.21.13.0.0.0.6 /dev/dsk/c48t0d6  unused  -         8       256.00  HP      HSV450
IP2:10.1.194.57
                                  SMH->Disks and File Systems->Disks
------------------------------------------------------------------------------------------------------
H/W Scan: Thu Oct 11 09:12:14 2012
Path(s)           Device File(s)    Use    VG Names   Paths   Size(GB) Description
------------------------------------------------------------------------------------------------------
0/0/0/5/0/0/0.21.13.0.0.0.2 /dev/dsk/c22t0d2  unused  -         8       256.00  HP      HSV450
0/0/0/5/0/0/0.21.13.0.0.0.3 /dev/dsk/c22t0d3  unused  -         8       256.00  HP      HSV450
0/0/0/5/0/0/0.21.13.0.0.0.4 /dev/dsk/c22t0d4  unused  -         8       256.00  HP      HSV450
0/0/0/5/0/0/0.21.13.0.0.0.5 /dev/dsk/c22t0d5  unused  -         8       256.00  HP      HSV450
指定磁碟後
IP:10.1.194.57
                                  SMH->Disks and File Systems->Disks
------------------------------------------------------------------------------------------------------
H/W Scan: Thu Oct 11 14:25:26 2012
Path(s)           Device File(s)    Use    VG Names   Paths   Size(GB) Description
------------------------------------------------------------------------------------------------------
0/0/0/5/0/0/0.21.13.0.0.0.2 /dev/dsk/c22t0d2  unused  -         8       256.00  HP      HSV450
0/0/0/5/0/0/0.21.13.0.0.0.3 /dev/dsk/c22t0d3  unused  -         8       256.00  HP      HSV450
0/0/0/5/0/0/0.21.13.0.0.0.4 /dev/dsk/c22t0d4  unused  -         8       256.00  HP      HSV450
0/0/0/5/0/0/0.21.13.0.0.0.5 /dev/dsk/c22t0d5  unused  -         8       256.00  HP      HSV450
0/0/0/5/0/0/1.22.9.0.0.1.4 /dev/dsk/c26t1d4  unused  -         8       256.00  HP      HSV450
0/0/0/5/0/0/0.21.9.0.0.1.5 /dev/dsk/c18t1d5  unused  -         8       256.00  HP      HSV450
0/0/0/5/0/0/0.21.15.0.0.1.6 /dev/dsk/c24t1d6  unused  -         8       256.00  HP      HSV450
0/0/0/5/0/0/1.22.15.0.0.1.7 /dev/dsk/c32t1d7  unused  -         8       256.00  HP      HSV450
2、找出新增加的磁碟路徑,找出對應磁碟機代號及WWN:
第一種方法:ioscan -funNC disk
第二種方法:根據以上4個結果集,對比出新增加的磁碟路徑,找出對應磁碟機代號及相應的WWN(eg:c22t0d2前三位應為磁碟通道,對比後四位)
10.1.10.56
新增加的磁碟路徑
/dev/dsk/c26t1d4
/dev/dsk/c18t1d5
/dev/dsk/c24t1d6
/dev/dsk/c32t1d7
10.1.10.57
新增加的磁碟路徑
/dev/dsk/c48t1d3
/dev/dsk/c50t1d4
/dev/dsk/c60t1d5
/dev/dsk/c48t1d6
3、根據磁碟路徑,查詢ioscan -m dsf輸出結果,找出對應磁碟機代號
10.1.10.56  對應磁碟機代號
/dev/rdisk/disk121
/dev/rdisk/disk126
/dev/rdisk/disk133
/dev/rdisk/disk139
10.1.10.57  對應磁碟機代號
/dev/rdisk/disk93
/dev/rdisk/disk99
/dev/rdisk/disk108
/dev/rdisk/disk116

4、找出對應磁碟機代號的唯一ID,並找出其WWN(磁碟機代號應該是字元裝置檔案):
10.1.1.94.56
$ ll disk121 disk126 disk133 disk139
crw-r-----   1 bin        sys         13 0x000025 Oct 11 13:53 disk121
crw-r-----   1 bin        sys         13 0x000026 Oct 11 13:54 disk126
crw-r-----   1 bin        sys         13 0x000027 Oct 11 13:56 disk133
crw-r-----   1 bin        sys         13 0x000028 Oct 11 14:08 disk139
scsimgr lun_map -D /dev/rdisk/disk121 | grep WWID 
scsimgr lun_map -D /dev/rdisk/disk126 | grep WWID 
scsimgr lun_map -D /dev/rdisk/disk133 | grep WWID 
scsimgr lun_map -D /dev/rdisk/disk139 | grep WWID
WWN的結果省略
10.1.194.57
#cd /dev/rdisk
# ll disk93 disk99 disk108 disk116
crw-r-----   1 bin        sys         13 0x000015 Oct 11 13:53 disk108
crw-r-----   1 bin        sys         13 0x000016 Oct 11 14:06 disk116
crw-r-----   1 bin        sys         13 0x000013 Oct 11 13:50 disk93
crw-r-----   1 bin        sys         13 0x000014 Oct 11 13:51 disk99
scsimgr lun_map -D /dev/rdisk/disk93 | grep WWID  
scsimgr lun_map -D /dev/rdisk/disk99 | grep WWID  
scsimgr lun_map -D /dev/rdisk/disk108 | grep WWID 
scsimgr lun_map -D /dev/rdisk/disk116 | grep WWID 
WWN的結果省略
5、根據一一對應的WWN值,將相應WWN對應兩個節點的磁碟機代號設定成一致:
10.1.10.56
#mknod  /dev/rdisk/data5disk c 13 0x000013
#mknod  /dev/rdisk/data6disk c 13 0x000014
#mknod  /dev/rdisk/data7disk c 13 0x000015
#mknod  /dev/rdisk/data8disk c 13 0x000016
10.1.10.57
#mknod  /dev/rdisk/data5disk c 13 0x000025
#mknod  /dev/rdisk/data6disk c 13 0x000026
#mknod  /dev/rdisk/data7disk c 13 0x000027
#mknod  /dev/rdisk/data8disk c 13 0x000028
6、設定兩個節點對應磁碟機代號路徑的許可權:
#chown oracle:oinstall /dev/rdisk/data5disk
#chown oracle:oinstall /dev/rdisk/data6disk
#chown oracle:oinstall /dev/rdisk/data7disk
#chown oracle:oinstall /dev/rdisk/data8disk
7、以ORACLE使用者檢驗讀寫許可權
$dd if=/dev/zero f=/dev/rdisk/data5disk count=10 count=1024
8、在其中一節點上為ASM磁碟組新增磁碟
export ORACLE_SID=+ASM3
sqlplus / as sysdba
alter diskgroup DATADG add disk '/dev/rdisk/data5disk';
alter diskgroup DATADG add disk '/dev/rdisk/data6disk';
alter diskgroup DATADG add disk '/dev/rdisk/data7disk';
alter diskgroup DATADG add disk '/dev/rdisk/data8disk';
(注意: 檢查diskgroup新增前後的對比.
select a.GROUP_NUMBER, a.name, a.path, b.NAME
from v$asm_disk a,
     v$asm_diskgroup b
where a.GROUP_NUMBER = b.GROUP_NUMBER
)
 

 

來自 “ ITPUB部落格 ” ,連結:http://blog.itpub.net/27038344/viewspace-751163/,如需轉載,請註明出處,否則將追究法律責任。

相關文章