Weblogic BEA-141281 unable to get file lock, will retry 故障處理
今天兄弟單位的一臺應用伺服器需要從測試環境移交到機房並修改IP，在移交前關閉了應用伺服器，在移交後啟動weblogic時出現了問題，weblogic程式起來了，但是控制檯進不了不能修改資料來源設定(也就是修改jdbc連線串)。操作下如下:
手動執行啟動指令碼(但這裡其實另一個同事已經執行過一次)
[root@cdydtest bin]# ./startWebLogic.sh . . JAVA Memory arguments: -Xms300m -Xmx300m -XX:CompileThreshold=8000 -XX:PermSize=256m -XX:MaxPermSize=256m . WLS Start Mode=Development . CLASSPATH=/usr/bea/patch_wls1036/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/usr/bea/jdk1.6.0_20/lib/tools.jar:/usr/bea/wlserver_10.3/server/lib/weblogic_sp.jar:/usr/bea/wlserver_10.3/server/lib/weblogic.jar:/usr/bea/modules/features/weblogic.server.modules_10.3.6.0.jar:/usr/bea/wlserver_10.3/server/lib/webservices.jar:/usr/bea/modules/org.apache.ant_1.7.1/lib/ant-all.jar:/usr/bea/modules/net.sf.antcontrib_1.1.0.0_1-0b2/lib/ant-contrib.jar:/usr/bea/wlserver_10.3/common/derby/lib/derbyclient.jar:/usr/bea/wlserver_10.3/server/lib/xqrl.jar . PATH=/usr/bea/wlserver_10.3/server/bin:/usr/bea/modules/org.apache.ant_1.7.1/bin:/usr/bea/jdk1.6.0_20/jre/bin:/usr/bea/jdk1.6.0_20/bin:/usr/lib64/qt-3.3/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin . *************************************************** * To start WebLogic Server, use a username and * * password assigned to an admin-level user. For * * server administration, use the WebLogic Server * * console at * *************************************************** starting weblogic with Java version: java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) 64-Bit Server VM (build 16.3-b01, mixed mode) Starting WLS with line: /usr/bea/jdk1.6.0_20/bin/java -client -Xms300m -Xmx300m -XX:CompileThreshold=8000 -XX:PermSize=256m -XX:MaxPermSize=256m -Dweblogic.Name=AdminServer -Djava.security.policy=/usr/bea/wlserver_10.3/server/lib/weblogic.policy -Xverify:none -da -Dplatform.home=/usr/bea/wlserver_10.3 -Dwls.home=/usr/bea/wlserver_10.3/server -Dweblogic.home=/usr/bea/wlserver_10.3/server -Dweblogic.management.discover=true -Dwlw.iterativeDev= -Dwlw.testConsole= -Dwlw.logErrorsToConsole= -Dweblogic.ext.dirs=/usr/bea/patch_wls1036/profiles/default/sysext_manifest_classpath weblogic.Server
因為輸出日誌被重定向到了日誌檔案(47ggzj.log)中，所以這裡沒有顯示完整的日誌資訊。
抓取java程式來判斷是否weblogic已經啟動
[root@cdydtest bin]# ps -ef | grep java root 11933 1 0 14:23 ? 00:00:17 /usr/bea/jdk1.6.0_20/bin/java -client -Xms300m -Xmx300m -XX:CompileThreshold=8000 -XX:PermSize=256m -XX:MaxPermSize=256m -Dweblogic.Name=AdminServer -Djava.security.policy=/usr/bea/wlserver_10.3/server/lib/weblogic.policy -Xverify:none -da -Dplatform.home=/usr/bea/wlserver_10.3 -Dwls.home=/usr/bea/wlserver_10.3/server -Dweblogic.home=/usr/bea/wlserver_10.3/server -Dweblogic.management.discover=true -Dwlw.iterativeDev= -Dwlw.testConsole= -Dwlw.logErrorsToConsole= -Dweblogic.ext.dirs=/usr/bea/patch_wls1036/profiles/default/sysext_manifest_classpath weblogic.Server root 14675 1 20 15:10 pts/3 00:00:01 /usr/bea/jdk1.6.0_20/bin/java -client -Xms300m -Xmx300m -XX:CompileThreshold=8000 -XX:PermSize=256m -XX:MaxPermSize=256m -Dweblogic.Name=AdminServer -Djava.security.policy=/usr/bea/wlserver_10.3/server/lib/weblogic.policy -Xverify:none -da -Dplatform.home=/usr/bea/wlserver_10.3 -Dwls.home=/usr/bea/wlserver_10.3/server -Dweblogic.home=/usr/bea/wlserver_10.3/server -Dweblogic.management.discover=true -Dwlw.iterativeDev= -Dwlw.testConsole= -Dwlw.logErrorsToConsole= -Dweblogic.ext.dirs=/usr/bea/patch_wls1036/profiles/default/sysext_manifest_classpath weblogic.Server root 14695 14567 0 15:10 pts/3 00:00:00 grep java
從上面的資訊看出現了兩個weblogic程式(pid為11933,14675)在執行，這是因為兩個同事都手動執行了一次啟動指令碼。但登入不了weblogic控制檯，如是檢視weblogic日誌檔案。
[root@cdydtest base_domain]# cat 47ggzj.log
<nov 19, 2015 3:10:34 PM CST> <info> <security> <bea -090905> <disabling CryptoJ JCE Provider self-integrity check for better startup performance. To enable this check, specify -Dweblogic.security.allowCryptoJDefaultJCEVerification=true>
<nov 19, 2015 3:10:34 PM CST> <info> <security> <bea -090906> <changing the default Random Number Generator in RSA CryptoJ from ECDRBG to FIPS186PRNG. To disable this change, specify -Dweblogic.security.allowCryptoJDefaultPRNG=true>
<nov 19, 2015 3:10:35 PM CST> <info> <weblogicserver> <bea -000377> <starting WebLogic Server with Java HotSpot(TM) 64-Bit Server VM Version 16.3-b01 from Sun Microsystems Inc.>
<nov 19, 2015 3:10:45 PM CST> <info> <management> <bea -141281> <unable to get file lock, will retry ...>
<nov 19, 2015 3:10:55 PM CST> <info> <management> <bea -141281> <unable to get file lock, will retry ...>
<nov 19, 2015 3:11:05 PM CST> <info> <management> <bea -141281> <unable to get file lock, will retry ...>
<nov 19, 2015 3:11:15 PM CST> <info> <management> <bea -141281> <unable to get file lock, will retry ...>
<nov 19, 2015 3:11:25 PM CST> <info> <management> <bea -141281> <unable to get file lock, will retry ...>
<nov 19, 2015 3:11:35 PM CST> <info> <management> <bea -141281> <unable to get file lock, will retry ...>
<nov 19, 2015 3:11:45 PM CST> <info> <management> <bea -141281> <unable to get file lock, will retry ...>
<nov 19, 2015 3:11:55 PM CST> <info> <management> <bea -141281> <unable to get file lock, will retry ...>
<nov 19, 2015 3:12:05 PM CST> <info> <management> <bea -141281> <unable to get file lock, will retry ...>
<nov 19, 2015 3:12:15 PM CST> <info> <management> <bea -141281> <unable to get file lock, will retry ...>
<nov 19, 2015 3:12:25 PM CST> <info> <management> <bea -141281> <unable to get file lock, will retry ...>
<nov 19, 2015 3:12:35 PM CST> <info> <management> <bea -141281> <unable to get file lock, will retry ...>
<nov 19, 2015 3:12:45 PM CST> <info> <management> <bea -141281> <unable to get file lock, will retry ...>
<nov 19, 2015 3:12:55 PM CST> <info> <management> <bea -141281> <unable to get file lock, will retry ...>
<nov 19, 2015 3:13:06 PM CST> <info> <management> <bea -141281> <unable to get file lock, will retry ...>
從上面訊息可以看到不能獲得檔案鎖，雖然weblogic程式已經在執行，但是不能執行任何操作。這裡出現這個問題的原因是因為需要修改jdbc連線，因為資料庫伺服器的IP地址修改了，而原來在weblogic中配置的jdbc並沒有修改，那麼在啟動weblogic時，就會一直嘗試連線，在這時weblogic服務是沒有成功啟動的,也登入不了控制檯,但嘗試連線達到weblogic缺的次數後，就會放棄嘗試連線而執行後續的啟動操作，但這需要等待一定的時間，而這時業務人員說不能登入系統，一位同事如是登入weblogic控制不能登入，如是再次執行了一次啟動指令碼，所以出現了不能獲得檔案鎖的問題。現在的處理方法是kill掉這兩個weblogic程式，並刪除被鎖定的AdminServer.lok檔案，再次執行weblogic啟動指令碼就能正常啟動。
刪除被鎖定的AdminServer.lok檔案
[root@cdydtest /]#cd /usr/bea/user_projects/domains/base_domain/servers/AdminServer/tmp [root@cdydtest tmp]# ls AdminServer.lok WebServiceUtils.ser _WL_internal _WL_user [root@cdydtest tmp]# rm AdminServer.lok rm: remove regular empty file `AdminServer.lok'? y
手動執行啟動指令碼
[root@cdydtest bin]# ./startWebLogic.sh . . JAVA Memory arguments: -Xms300m -Xmx300m -XX:CompileThreshold=8000 -XX:PermSize=256m -XX:MaxPermSize=256m . WLS Start Mode=Development . CLASSPATH=/usr/bea/patch_wls1036/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/usr/bea/jdk1.6.0_20/lib/tools.jar:/usr/bea/wlserver_10.3/server/lib/weblogic_sp.jar:/usr/bea/wlserver_10.3/server/lib/weblogic.jar:/usr/bea/modules/features/weblogic.server.modules_10.3.6.0.jar:/usr/bea/wlserver_10.3/server/lib/webservices.jar:/usr/bea/modules/org.apache.ant_1.7.1/lib/ant-all.jar:/usr/bea/modules/net.sf.antcontrib_1.1.0.0_1-0b2/lib/ant-contrib.jar:/usr/bea/wlserver_10.3/common/derby/lib/derbyclient.jar:/usr/bea/wlserver_10.3/server/lib/xqrl.jar . PATH=/usr/bea/wlserver_10.3/server/bin:/usr/bea/modules/org.apache.ant_1.7.1/bin:/usr/bea/jdk1.6.0_20/jre/bin:/usr/bea/jdk1.6.0_20/bin:/usr/lib64/qt-3.3/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin . *************************************************** * To start WebLogic Server, use a username and * * password assigned to an admin-level user. For * * server administration, use the WebLogic Server * * console at * *************************************************** starting weblogic with Java version: java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) 64-Bit Server VM (build 16.3-b01, mixed mode) Starting WLS with line: /usr/bea/jdk1.6.0_20/bin/java -client -Xms300m -Xmx300m -XX:CompileThreshold=8000 -XX:PermSize=256m -XX:MaxPermSize=256m -Dweblogic.Name=AdminServer -Djava.security.policy=/usr/bea/wlserver_10.3/server/lib/weblogic.policy -Xverify:none -da -Dplatform.home=/usr/bea/wlserver_10.3 -Dwls.home=/usr/bea/wlserver_10.3/server -Dweblogic.home=/usr/bea/wlserver_10.3/server -Dweblogic.management.discover=true -Dwlw.iterativeDev= -Dwlw.testConsole= -Dwlw.logErrorsToConsole= -Dweblogic.ext.dirs=/usr/bea/patch_wls1036/profiles/default/sysext_manifest_classpath weblogic.Server
檢視日誌資訊
[root@cdydtest base_domain]# cat 47ggzj.log2015-11-19 15:23:46 Initializing Insur_CHANGDE from init-parameters: PowerSI Version v0.2.9(Build20140901) starttime:2015-11-19 15:23:46 trace_busiconn:1 trace_dbconn:0 hostname:cdydtest jdbclogger:/usr/bea/user_projects/domains/base_domain/applications/Insur_CHANGDE/WEB-INF/jdbclogger.properties homedir:/usr/bea/user_projects/domains/base_domain/applications/Insur_CHANGDE companyname:???? jdbclogger.maxBatchCount:10 jdbclogger.minRuntime:1000 jdbclogger.needCaller:true service_centerid: applicationname:Insur_CHANGDE scheduler_flag:0 logger:/usr/bea/user_projects/domains/base_domain/applications/Insur_CHANGDE/WEB-INF/log4j.properties log_level:0 instancename:cdydtest.Insur_CHANGDE <nov 19, 2015 3:23:46 PM CST> <notice> <loggingservice> <bea -320400> <the log file /usr/bea/user_projects/domains/base_domain/servers/AdminServer/logs/base_domain.log will be rotated. Reopen the log file if tailing has stopped. This can happen on some platforms like Windows.> <nov 19, 2015 3:23:46 PM CST> <notice> <loggingservice> <bea -320401> <the log file has been rotated to /usr/bea/user_projects/domains/base_domain/servers/AdminServer/logs/base_domain.log00027. Log messages will continue to be logged in /usr/bea/user_projects/domains/base_domain/servers/AdminServer/logs/base_domain.log.> <nov 19, 2015 3:23:46 PM CST> <notice> <log Management> <bea -170027> <the Server has established connection with the Domain level Diagnostic Service successfully.> <nov 19, 2015 3:23:46 PM CST> <notice> <weblogicserver> <bea -000365> <server state changed to ADMIN> <nov 19, 2015 3:23:46 PM CST> <notice> <weblogicserver> <bea -000365> <server state changed to RESUMING> <nov 19, 2015 3:23:46 PM CST> <notice> <server> <bea -002613> <channel "Default" is now listening on 10.138.130.251:7001 for protocols iiop, t3, ldap, snmp, http.> <nov 19, 2015 3:23:46 PM CST> <notice> <server> <bea -002613> <channel "Default[1]" is now listening on 127.0.0.1:7001 for protocols iiop, t3, ldap, snmp, http.> <nov 19, 2015 3:23:46 PM CST> <notice> <weblogicserver> <bea -000331> <started WebLogic Admin Server "AdminServer" for domain "base_domain" running in Development Mode> <nov 19, 2015 3:23:46 PM CST> <notice> <weblogicserver> <bea -000365> <server state changed to RUNNING> <nov 19, 2015 3:23:46 PM CST> <notice> <weblogicserver> <bea -000360> <server started in RUNNING mode>從上面資訊可知weblogic成功啟動。從這個故障的原因來看就是操作人員在處理問題時不夠細心，在不能登入weblogic控制檯時並沒有檢查當前已經啟動weblogic服務，而就執行了啟動指令碼才產生的問題。處理問題時一定要搞清狀況，瞭解必要的資訊，弄清原因才能操作。
來自 “ ITPUB部落格 ” ，連結：http://blog.itpub.net/26015009/viewspace-1841751/，如需轉載，請註明出處，否則將追究法律責任。
相關文章
- weblogic - <BEA-141281> <unable to get file lock, will retry ...>Web
- unable to get file lock, will retry ...
- Unable to create index.lock File exists錯誤Index
- Oracle Database on NFS : unable to lock file - already in use" ErrorOracleDatabaseNFSError
- kkjcre1p: unable to spawn jobq slave process故障處理
- 【WebLogic故障處理】一次嚴重的WebLogic記憶體洩漏問題處理過程Web記憶體
- 【故障處理】ORA-1688: unable to extend table AUDSYS.AUD$UNIFIEDNifi
- 【故障處理】告警日誌報“ORA-01565 Unable To open Spfile”
- VMware虛擬機器Reason: Failed to lock the file問題處理虛擬機AI
- 【rac故障】root.sh報錯Unable to get VIP info for new node
- 【故障處理】ORA-3113 "end of file on communication channel"
- unable to get oracle owner forOracle
- 【故障處理】一次RAC故障處理過程
- ORA-01157: cannot identify/lock data file %s - see DBWR trace file的處理IDE
- MongoDB故障處理MongoDB
- Unable to create git index lockGitIndex
- weblogic啟動不能鎖定AdminServer.lok的故障處理WebServer
- 故障分析 | Greenplum Segment 故障處理
- DFS lock handle事件處理事件
- GPON網路故障如何處理？GPON網路故障處理流程
- 【故障處理】ORA-600:[13013],[5001]故障處理
- 【故障處理】ORA- 2730*，status 12故障分析與處理
- linux故障處理Linux
- ora-故障處理
- apt could not get lockAPT
- unable to qualify my own domain name_處理AI
- 線上故障處理手冊
- MySQL show processlist故障處理MySql
- 微服務的故障處理微服務
- teams登入故障處理
- Oracle更新Opatch故障處理Oracle
- 如何快速處理線上故障
- Mysql故障處理2則MySql
- dataguard故障處理一則
- AIX系統故障處理AI
- 【Linux】 nfs 故障處理LinuxNFS
- 【故障處理】CRS-1153錯誤處理
- 【故障處理】ORA-19809錯誤處理