【新炬網路名師大講堂】記一次打PSU遇到的Copy failed的問題

（本文轉自“NCN新炬團隊部落格”）

好久沒做過安裝和打PSU的活了，心裡還有點小期待，本以為2個小時就可以結束的事情，結果輪到真正的打的時候，打了4個多小時才弄完，遇到了一些小問題，我這次打的環境是11.2.0.3.9，這裡做個記錄。廢話就不說了，我們來先看下日誌：

root@crmdb1rac1:/#/oracle/app/11.2.0/grid/OPatch/opatch auto /oraclelog/opatch_soft -ocmrf /oracle/app/grid/ocm0709.rsp

Executing /oracle/app/11.2.0/grid/perl/bin/perl /oracle/app/11.2.0/grid/OPatch/crs/patch11203.pl -patchdir /oraclelog -patchn opatch_soft -ocmrf /oracle/app/grid/ocm0709.rsp -paramfile /oracle/app/11.2.0/grid/crs/install/crsconfig_params

/oracle/app/11.2.0/grid/crs/install/crsconfig_params

/oracle/app/11.2.0/grid/crs/install/s_crsconfig_defs

 

This is the main log file: /oracle/app/11.2.0/grid/cfgtoollogs/opatchauto2014-07-10_00-18-14.log

This file will show your detected configuration and all the steps that opatchauto attempted to do on your system: /oracle/app/11.2.0/grid/cfgtoollogs/opatchauto2014-07-10_00-18-14.report.log

 

2014-07-10 00:18:14: Starting Clusterware Patch Setup

Using configuration parameter file: /oracle/app/11.2.0/grid/crs/install/crsconfig_params

patch /oraclelog/opatch_soft/17592127/custom/server/17592127  apply successful for home  /oracle/app/oracle/product/11.2.0/db 

 patch /oraclelog/opatch_soft/18031683  apply successful for home  /oracle/app/oracle/product/11.2.0/db 

 

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on ’’crmdb1rac1’’

CRS-2673: Attempting to stop ’’ora.crsd’’ on ’’crmdb1rac1’’

CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on ’’crmdb1rac1’’

CRS-2673: Attempting to stop ’’ora.LISTENER_SCAN1.lsnr’’ on ’’crmdb1rac1’’

CRS-2673: Attempting to stop ’’ora.oc4j’’ on ’’crmdb1rac1’’

CRS-2673: Attempting to stop ’’ora.cvu’’ on ’’crmdb1rac1’’

CRS-2673: Attempting to stop ’’ora.crmdb1rac1.vip’’ on ’’crmdb1rac1’’

CRS-2677: Stop of ’’ora.cvu’’ on ’’crmdb1rac1’’ succeeded

CRS-2672: Attempting to start ’’ora.cvu’’ on ’’crmdb1rac2’’

CRS-2677: Stop of ’’ora.LISTENER_SCAN1.lsnr’’ on ’’crmdb1rac1’’ succeeded

CRS-2673: Attempting to stop ’’ora.scan1.vip’’ on ’’crmdb1rac1’’

CRS-2676: Start of ’’ora.cvu’’ on ’’crmdb1rac2’’ succeeded

CRS-2677: Stop of ’’ora.crmdb1rac1.vip’’ on ’’crmdb1rac1’’ succeeded

CRS-2672: Attempting to start ’’ora.crmdb1rac1.vip’’ on ’’crmdb1rac2’’

CRS-2677: Stop of ’’ora.scan1.vip’’ on ’’crmdb1rac1’’ succeeded

CRS-2672: Attempting to start ’’ora.scan1.vip’’ on ’’crmdb1rac2’’

CRS-2676: Start of ’’ora.crmdb1rac1.vip’’ on ’’crmdb1rac2’’ succeeded

CRS-2676: Start of ’’ora.scan1.vip’’ on ’’crmdb1rac2’’ succeeded

CRS-2672: Attempting to start ’’ora.LISTENER_SCAN1.lsnr’’ on ’’crmdb1rac2’’

CRS-2676: Start of ’’ora.LISTENER_SCAN1.lsnr’’ on ’’crmdb1rac2’’ succeeded

CRS-2677: Stop of ’’ora.oc4j’’ on ’’crmdb1rac1’’ succeeded

CRS-2672: Attempting to start ’’ora.oc4j’’ on ’’crmdb1rac2’’

CRS-2676: Start of ’’ora.oc4j’’ on ’’crmdb1rac2’’ succeeded

CRS-2673: Attempting to stop ’’ora.ons’’ on ’’crmdb1rac1’’

CRS-2677: Stop of ’’ora.ons’’ on ’’crmdb1rac1’’ succeeded

CRS-2673: Attempting to stop ’’ora.net1.network’’ on ’’crmdb1rac1’’

CRS-2677: Stop of ’’ora.net1.network’’ on ’’crmdb1rac1’’ succeeded

CRS-2792: Shutdown of Cluster Ready Services-managed resources on ’’crmdb1rac1’’ has completed

CRS-2677: Stop of ’’ora.crsd’’ on ’’crmdb1rac1’’ succeeded

CRS-2673: Attempting to stop ’’ora.mdnsd’’ on ’’crmdb1rac1’’

CRS-2673: Attempting to stop ’’ora.crf’’ on ’’crmdb1rac1’’

CRS-2673: Attempting to stop ’’ora.ctssd’’ on ’’crmdb1rac1’’

CRS-2673: Attempting to stop ’’ora.evmd’’ on ’’crmdb1rac1’’

CRS-2673: Attempting to stop ’’ora.cluster_interconnect.haip’’ on ’’crmdb1rac1’’

CRS-2677: Stop of ’’ora.mdnsd’’ on ’’crmdb1rac1’’ succeeded

CRS-2677: Stop of ’’ora.evmd’’ on ’’crmdb1rac1’’ succeeded

CRS-2677: Stop of ’’ora.crf’’ on ’’crmdb1rac1’’ succeeded

CRS-2677: Stop of ’’ora.cluster_interconnect.haip’’ on ’’crmdb1rac1’’ succeeded

CRS-2677: Stop of ’’ora.ctssd’’ on ’’crmdb1rac1’’ succeeded

CRS-2673: Attempting to stop ’’ora.cssd’’ on ’’crmdb1rac1’’

CRS-2677: Stop of ’’ora.cssd’’ on ’’crmdb1rac1’’ succeeded

CRS-2673: Attempting to stop ’’ora.gipcd’’ on ’’crmdb1rac1’’

CRS-2677: Stop of ’’ora.gipcd’’ on ’’crmdb1rac1’’ succeeded

CRS-2673: Attempting to stop ’’ora.gpnpd’’ on ’’crmdb1rac1’’

CRS-2677: Stop of ’’ora.gpnpd’’ on ’’crmdb1rac1’’ succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on ’’crmdb1rac1’’ has completed

CRS-4133: Oracle High Availability Services has been stopped.

Successfully unlock /oracle/app/11.2.0/grid

patch /oraclelog/opatch_soft/17592127  apply failed  for home  /oracle/app/11.2.0/grid

CRS-4123: Oracle High Availability Services has been started.

在打PSU進行到Successfully unlock /oracle/app/11.2.0/grid這一步的時候。這個時候螢幕的輸出會非常非常的慢，我們可以觀察後臺的日誌輸出，我們發現在後面進行一個copy操作的時候出現了問題。如下所示：

 Patching component oracle.usm, 11.2.0.3.0...

 

 The following actions have failed:

 Copy failed from ’’/oracle/app/11.2.0/grid/.patch_storage/16619898_Jun_26_2013_21_46_53/files/lib/libeons.so’’ to ’’/oracle/app/11.2.0/grid/lib/libeons.so’’... 

 Copy failed from ’’/oracle/app/11.2.0/grid/.patch_storage/16619898_Jun_26_2013_21_46_53/files/lib/libhasgen11.so’’ to ’’/oracle/app/11.2.0/grid/lib/libhasgen11.so’’... 

 Copy failed from ’’/oracle/app/11.2.0/grid/.patch_storage/16619898_Jun_26_2013_21_46_53/files/lib/libocr11.so’’ to ’’/oracle/app/11.2.0/grid/lib/libocr11.so’’... 

 Copy failed from ’’/oracle/app/11.2.0/grid/.patch_storage/16619898_Jun_26_2013_21_46_53/files/lib/libocrb11.so’’ to ’’/oracle/app/11.2.0/grid/lib/libocrb11.so’’... 

 Copy failed from ’’/oracle/app/11.2.0/grid/.patch_storage/16619898_Jun_26_2013_21_46_53/files/lib/libocrutl11.so’’ to ’’/oracle/app/11.2.0/grid/lib/libocrutl11.so’’...

我們可以看到在copy這些檔案的時候是無法成功copy的，這是因為我們的Oracle的這些so檔案在記憶體中被佔用著，我們需要執行slibclean去清理這些快取。於是我做了回退的操作，然後把crs完整的停下來，做了幾次slibclean的操作，在正常的啟動crs。做完這些操作後，我發現打PSU仍然在這一步失敗了。於是我查了一些資料。我把這幾個coyp失敗的.so的檔案做了一個備份，移動到了其他的目錄裡面，然後crs就無法成功啟動了。crs無法啟動還怎麼打PSU呢？於是我又把檔案複製回去，我仔細觀察了下，在打PSU的時候，會自動把crs停掉，然後就會執行我們剛剛所說的Successfully unlock /oracle/app/11.2.0/grid。那我能不能在停掉CRS這個階段執行slibclean去清理這些快取呢？

這是在CRS還在啟動狀態下佔用的lib檔案:

root@crmdb1rac1:/oracle/app/11.2.0/grid/.patch_storage#genkld |grep oracle

 900000004d98000     b2c7 /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/libnio.a

 900000004d77000    203b1 /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/libnet.a

 900000004d5f000    17840 /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/libzip.a

 900000004d31000    21700 /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/libj9ute23.so

 900000004d1e000    12c7c /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/libiverel23.so

 900000004cb6000    6792f /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/libjclscar_23.so

 900000004c91000    2427d /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/libj9vrb23.so

 900000004c5e000    32c3f /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/libj9jvmti23.so

 900000004c2f000    2ed2c /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/libj9dyn23.so

 900000004b97000    97a31 /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/libj9gc23.so

 900000004b7f000    179e1 /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/libj9bcv23.so

 9000000046cf000   4af269 /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/libj9jit23.so

 9000000046be000    10eef /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/libj9trc23.so

 900000004681000    2ac84 /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/libj9dmp23.so

 9000000046ac000    11bab /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/libj9zlib23.so

 900000004649000    37b9e /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/libjava.a

 90000000091a000     1d41 /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/libwrappers.a

 900000004588000    6f0fe /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/libj9vm23.so

 9000000045f8000    50434 /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/libj9prt23.so

 900000000918000     1d2f /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/libj9hookable23.so

 90000000456c000    1b631 /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/j9vm/libjvm.so

 900000004059000     bf6a /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/libj9thr23.so

 900000000916000     1ff3 /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/libjsig.so

 900000004049000     f6aa /oracle/app/oracle/product/11.2.0/db/jdk/jre/bin/classic/libjvm.so

 90000000ff8e000  248bf1b /oracle/app/11.2.0/grid/lib/libclntsh.so

 900000004513000    5852a /oracle/app/11.2.0/grid/lib/libcell11.so

 90000000434a000   1b3f27 /oracle/app/11.2.0/grid/lib/libdb-4.6.so

 90000000428a000    bf3a1 /oracle/app/11.2.0/grid/lib/libcrf11.so

 9000000041d7000     ad7d /oracle/app/11.2.0/grid/lib/libeonsserver.so

 9000000041b1000    15c4e /oracle/app/11.2.0/grid/lib/libonsx.so

 900000004019000    2509a /oracle/app/11.2.0/grid/lib/libeons.so

 90000000123b000  2dd6039 /oracle/app/11.2.0/grid/lib/libttsh11.so

 900000004012000     6c16 /oracle/app/11.2.0/grid/lib/libskgxn2.so

 90000000406c000   1449e1 /oracle/app/11.2.0/grid/lib/libgns11.so

 90000000122e000     c5c5 /oracle/app/11.2.0/grid/lib/libocrutl11.so

 90000000103e000   1ef18b /oracle/app/11.2.0/grid/lib/libocrb11.so

 900000000f9f000    9e0a5 /oracle/app/11.2.0/grid/lib/libocr11.so

 900000000921000   67dc0c /oracle/app/11.2.0/grid/lib/libhasgen11.so

 90000000f7a9000   2adb17 /usr/openv/lib/libpioracle.so

這是在CRS關閉後，我執行了好幾次slibclean之後，佔用的lib檔案.

 root@crmdb1rac1:/usr/sbin#genkld |grep oracle

 900000001160000     b2c7 /oracle/app/11.2.0/grid/jdk/jre/bin/libnio.a

 90000000113f000    203b1 /oracle/app/11.2.0/grid/jdk/jre/bin/libnet.a

 900000001127000    17840 /oracle/app/11.2.0/grid/jdk/jre/bin/libzip.a

 9000000010f9000    21700 /oracle/app/11.2.0/grid/jdk/jre/bin/libj9ute23.so

 9000000010e6000    12c7c /oracle/app/11.2.0/grid/jdk/jre/bin/libiverel23.so

 90000000107e000    6792f /oracle/app/11.2.0/grid/jdk/jre/bin/libjclscar_23.so

 900000001059000    2427d /oracle/app/11.2.0/grid/jdk/jre/bin/libj9vrb23.so

 900000001026000    32c3f /oracle/app/11.2.0/grid/jdk/jre/bin/libj9jvmti23.so

 900000000ff7000    2ed2c /oracle/app/11.2.0/grid/jdk/jre/bin/libj9dyn23.so

 900000000f5f000    97a31 /oracle/app/11.2.0/grid/jdk/jre/bin/libj9gc23.so

 900000000a97000   4af269 /oracle/app/11.2.0/grid/jdk/jre/bin/libj9jit23.so

 900000000a86000    10eef /oracle/app/11.2.0/grid/jdk/jre/bin/libj9trc23.so

 900000000a49000    2ac84 /oracle/app/11.2.0/grid/jdk/jre/bin/libj9dmp23.so

 900000000a74000    11bab /oracle/app/11.2.0/grid/jdk/jre/bin/libj9zlib23.so

 900000000a0f000    37b9e /oracle/app/11.2.0/grid/jdk/jre/bin/libjava.a

 900000000a47000     1d41 /oracle/app/11.2.0/grid/jdk/jre/bin/libwrappers.a

 90000000094e000    6f0fe /oracle/app/11.2.0/grid/jdk/jre/bin/libj9vm23.so

 9000000009be000    50434 /oracle/app/11.2.0/grid/jdk/jre/bin/libj9prt23.so

 900000000432000     1d2f /oracle/app/11.2.0/grid/jdk/jre/bin/libj9hookable23.so

 900000000926000    1b631 /oracle/app/11.2.0/grid/jdk/jre/bin/j9vm/libjvm.so

 900000000942000     bf6a /oracle/app/11.2.0/grid/jdk/jre/bin/libj9thr23.so

 900000000430000     1ff3 /oracle/app/11.2.0/grid/jdk/jre/bin/libjsig.so

 900000000916000     f6aa /oracle/app/11.2.0/grid/jdk/jre/bin/classic/libjvm.so

 90000000f7a9000   2adb17 /usr/openv/lib/libpioracle.so

可以看到基本上全是和JDK相關的一些lib檔案，沒有我們需要copy的這些lib檔案了。做完這些操作之後，PSU補丁才安裝成功。不知道從何時開始起，非要折騰幾下才能把事情做完做好。

 

原文連結：