【VMware VCF】解決還原 SDDC Manager 備份後無法顯示映像管理中的可用映像問題。

JUNIOR_MU發表於2024-11-30

之前透過備份的配置檔案還原 SDDC Manager 元件之後發現有個問題,導航到生命週期管理->映像管理,在“可用的映像”管理檢視中無法正常顯示所有的映像,並出現如下圖所示的錯誤。嘗試測試匯入新的映像到 SDDC Manager 中,任務也能正常完成,但是始終無法在可用的映像中正常顯示。

檢索映像失敗。伺服器出現問題。請重試或聯絡服務提供商並提供參考令牌。
Failed to retrieve images. A problem has occurred on the server. Please retry or contact the service provider and provide the reference token.

【VMware VCF】解決還原 SDDC Manager 備份後無法顯示映像管理中的可用映像問題。

登入 SDDC Manager CLI 並檢視以下 /var/log/vmware/vcf/sddc-manager-ui-app/sddcManagerServer.log 日誌,可以看到 SDDC Manager UI 中所提示的相關錯誤資訊。這裡可以大概看出好像是從資料庫查詢什麼資訊結果失敗了,但是無法確定是什麼原因導致的。

2024-11-29T13:58:01.714+0000 ERROR [8a763d4e4be04857] [services/errorHandling.js, http-get-onboardinglandingpage, production
ErrorRoute:131] 
600.158: VError: Sending error response: Encountered error querying postgres database for workflow data: Failed to construct
 workflowEntry from database entry: Encountered exception while trying to decrypt data: error:1C800064:Provider routines::ba
d decrypt
    at Object.errorHandlerSend (/opt/vmware/vcf/sddc-manager-ui-app/server/src/errors/VCFError.js:104:5)
    at productionErrorRoute (/opt/vmware/vcf/sddc-manager-ui-app/server/src/services/errorHandling.js:118:34)
    at Layer.handle_error (/opt/vmware/vcf/sddc-manager-ui-app/server/node_modules/express/lib/router/layer.js:71:5)
    at trim_prefix (/opt/vmware/vcf/sddc-manager-ui-app/server/node_modules/express/lib/router/index.js:326:13)
    at /opt/vmware/vcf/sddc-manager-ui-app/server/node_modules/express/lib/router/index.js:286:9
    at Function.process_params (/opt/vmware/vcf/sddc-manager-ui-app/server/node_modules/express/lib/router/index.js:346:12)
    at next (/opt/vmware/vcf/sddc-manager-ui-app/server/node_modules/express/lib/router/index.js:280:10)
    at Layer.handle_error (/opt/vmware/vcf/sddc-manager-ui-app/server/node_modules/express/lib/router/layer.js:67:12)
    at trim_prefix (/opt/vmware/vcf/sddc-manager-ui-app/server/node_modules/express/lib/router/index.js:326:13)
    at /opt/vmware/vcf/sddc-manager-ui-app/server/node_modules/express/lib/router/index.js:286:9
Error Info: {"upn":"Administrator@VSPHERE.LOCAL","id":"9a844bc6-507b-40bf-baa9-cc09727b86e4","requestedPath":"/ui/api/intern
al/workflows/onboardinglandingpage","error":{"jse_shortmsg":"Encountered error querying postgres database for workflow data"
,"jse_cause":{"jse_shortmsg":"Failed to construct workflowEntry from database entry","jse_cause":{"jse_shortmsg":"Encountere
d exception while trying to decrypt data","jse_cause":{"library":"Provider routines","reason":"bad decrypt","code":"ERR_OSSL
_BAD_DECRYPT"},"jse_info":{"errorReference":"100.135","errorModule":100,"errorCode":135,"code":"ERR_OSSL_BAD_DECRYPT"},"mess
age":"Encountered exception while trying to decrypt data: error:1C800064:Provider routines::bad decrypt","code":"ERR_OSSL_BA
D_DECRYPT","databaseRowData":{"upn":"Administrator@VSPHERE.LOCAL","type":"onboardinglandingpage","label":"default","site":"d
efault","metadata":"{\"version\":1,\"timestamp\":1727334337422}","payload":"fdc856b8b9b5cdf75d573e9fb59075cc269524936569950d
2e67fca49350599555bb65f393db10593463afa20be4ca19b86e627bbee5e77446798f870e1365fd82641d0f46664323007a17e36cc8d5668c6463a985ff
03722b80a44818162b98b3e808ae4e52e5508b958f164aaa3112"}},"jse_info":{"errorReference":"400.168","errorModule":400,"errorCode"
:168,"code":"ERR_OSSL_BAD_DECRYPT"},"message":"Failed to construct workflowEntry from database entry: Encountered exception 
while trying to decrypt data: error:1C800064:Provider routines::bad decrypt","code":"ERR_OSSL_BAD_DECRYPT"},"jse_info":{"fai
ledQuery":"SELECT * FROM workflow_config_data WHERE upn = 'Administrator@VSPHERE.LOCAL' AND type = 'onboardinglandingpage' A
ND label = 'default' AND site = 'default'","errorReference":"400.169","errorModule":400,"errorCode":169,"code":"ERR_OSSL_BAD
_DECRYPT"},"message":"Encountered error querying postgres database for workflow data: Failed to construct workflowEntry from
 database entry: Encountered exception while trying to decrypt data: error:1C800064:Provider routines::bad decrypt","code":"
ERR_OSSL_BAD_DECRYPT"},"stack":"VError: Encountered error querying postgres database for workflow data: Failed to construct 
workflowEntry from database entry: Encountered exception while trying to decrypt data: error:1C800064:Provider routines::bad
 decrypt\n    at Object.encounterErrorQueryingPostgresDataBaseError (/opt/vmware/vcf/sddc-manager-ui-app/server/src/errors/V
CFError.js:104:5)\n    at getWorkflowEntry (/opt/vmware/vcf/sddc-manager-ui-app/server/src/services/workflows.js:69:31)\n   
 at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async /opt/vmware/vcf/sddc-manager-ui
-app/server/src/routes/api/workflows.js:49:18","message":"Encountered error querying postgres database for workflow data: Fa
iled to construct workflowEntry from database entry: Encountered exception while trying to decrypt data: error:1C800064:Prov
ider routines::bad decrypt","code":"ERR_OSSL_BAD_DECRYPT","status":500,"errorModule":600,"errorCode":158}
caused by:
400.169: VError: Encountered error querying postgres database for workflow data: Failed to construct workflowEntry from data
base entry: Encountered exception while trying to decrypt data: error:1C800064:Provider routines::bad decrypt
    at Object.encounterErrorQueryingPostgresDataBaseError (/opt/vmware/vcf/sddc-manager-ui-app/server/src/errors/VCFError.js
:104:5)
    at getWorkflowEntry (/opt/vmware/vcf/sddc-manager-ui-app/server/src/services/workflows.js:69:31)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /opt/vmware/vcf/sddc-manager-ui-app/server/src/routes/api/workflows.js:49:18
Error Info: {"failedQuery":"SELECT * FROM workflow_config_data WHERE upn = 'Administrator@VSPHERE.LOCAL' AND type = 'onboard
inglandingpage' AND label = 'default' AND site = 'default'","errorModule":400,"errorCode":169,"code":"ERR_OSSL_BAD_DECRYPT"}
caused by:
400.168: VError: Failed to construct workflowEntry from database entry: Encountered exception while trying to decrypt data: 
error:1C800064:Provider routines::bad decrypt
    at Object.failedToConstructWorkFlowEntryError (/opt/vmware/vcf/sddc-manager-ui-app/server/src/errors/VCFError.js:104:5)
    at databaseEntryToWorkflowEntry (/opt/vmware/vcf/sddc-manager-ui-app/server/src/services/workflows.js:39:31)
    at getWorkflowEntry (/opt/vmware/vcf/sddc-manager-ui-app/server/src/services/workflows.js:67:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /opt/vmware/vcf/sddc-manager-ui-app/server/src/routes/api/workflows.js:49:18
Error Info: {"errorModule":400,"errorCode":168,"code":"ERR_OSSL_BAD_DECRYPT"}
caused by:
100.135: VError: Encountered exception while trying to decrypt data: error:1C800064:Provider routines::bad decrypt
    at Object.encounterExceptionWhileTryingToDecryptDataError (/opt/vmware/vcf/sddc-manager-ui-app/server/src/errors/VCFErro
r.js:104:5)
    at decryptData (/opt/vmware/vcf/sddc-manager-ui-app/server/src/services/cryptoUtils.js:239:26)
    at databaseEntryToWorkflowEntry (/opt/vmware/vcf/sddc-manager-ui-app/server/src/services/workflows.js:35:33)
    at getWorkflowEntry (/opt/vmware/vcf/sddc-manager-ui-app/server/src/services/workflows.js:67:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /opt/vmware/vcf/sddc-manager-ui-app/server/src/routes/api/workflows.js:49:18
Error Info: {"errorModule":100,"errorCode":135,"code":"ERR_OSSL_BAD_DECRYPT"}
caused by:
Error: error:1C800064:Provider routines::bad decrypt
    at Decipheriv.final (node:internal/crypto/cipher:199:29)
    at decryptData (/opt/vmware/vcf/sddc-manager-ui-app/server/src/services/cryptoUtils.js:237:72)
    at databaseEntryToWorkflowEntry (/opt/vmware/vcf/sddc-manager-ui-app/server/src/services/workflows.js:35:33)
    at getWorkflowEntry (/opt/vmware/vcf/sddc-manager-ui-app/server/src/services/workflows.js:67:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /opt/vmware/vcf/sddc-manager-ui-app/server/src/routes/api/workflows.js:49:18
All Errors Info:
 Encountered error querying postgres database for workflow data: Failed to construct workflowEntry from database entry: Enco
untered exception while trying to decrypt data: error:1C800064:Provider routines::bad decrypt {"code":"ERR_OSSL_BAD_DECRYPT"
,"error":{"code":"ERR_OSSL_BAD_DECRYPT","jse_cause":{"code":"ERR_OSSL_BAD_DECRYPT","jse_cause":{"code":"ERR_OSSL_BAD_DECRYPT
","databaseRowData":{"label":"default","metadata":"{\"version\":1,\"timestamp\":1727334337422}","payload":"fdc856b8b9b5cdf75
d573e9fb59075cc269524936569950d2e67fca49350599555bb65f393db10593463afa20be4ca19b86e627bbee5e77446798f870e1365fd82641d0f46664
323007a17e36cc8d5668c6463a985ff03722b80a44818162b98b3e808ae4e52e5508b958f164aaa3112","site":"default","type":"onboardingland
ingpage","upn":"Administrator@VSPHERE.LOCAL"},"jse_cause":{"code":"ERR_OSSL_BAD_DECRYPT","library":"Provider routines","reas
on":"bad decrypt"},"jse_info":{"code":"ERR_OSSL_BAD_DECRYPT","errorCode":135,"errorModule":100},"jse_shortmsg":"Encountered 
exception while trying to decrypt data","message":"Encountered exception while trying to decrypt data: error:1C800064:Provid
er routines::bad decrypt"},"jse_info":{"code":"ERR_OSSL_BAD_DECRYPT","errorCode":168,"errorModule":400},"jse_shortmsg":"Fail
ed to construct workflowEntry from database entry","message":"Failed to construct workflowEntry from database entry: Encount
ered exception while trying to decrypt data: error:1C800064:Provider routines::bad decrypt"},"jse_info":{"code":"ERR_OSSL_BA
D_DECRYPT","errorCode":169,"errorModule":400,"failedQuery":"SELECT * FROM workflow_config_data WHERE upn = 'Administrator@VS
PHERE.LOCAL' AND type = 'onboardinglandingpage' AND label = 'default' AND site = 'default'"},"jse_shortmsg":"Encountered err
or querying postgres database for workflow data","message":"Encountered error querying postgres database for workflow data: 
Failed to construct workflowEntry from database entry: Encountered exception while trying to decrypt data: error:1C800064:Pr
ovider routines::bad decrypt"},"failedQuery":"SELECT * FROM workflow_config_data WHERE upn = 'Administrator@VSPHERE.LOCAL' A
ND type = 'onboardinglandingpage' AND label = 'default' AND site = 'default'","id":"9a844bc6-507b-40bf-baa9-cc09727b86e4","r
equestedPath":"/ui/api/internal/workflows/onboardinglandingpage","stack":"VError: Encountered error querying postgres databa
se for workflow data: Failed to construct workflowEntry from database entry: Encountered exception while trying to decrypt d
ata: error:1C800064:Provider routines::bad decrypt\n    at Object.encounterErrorQueryingPostgresDataBaseError (/opt/vmware/v
cf/sddc-manager-ui-app/server/src/errors/VCFError.js:104:5)\n    at getWorkflowEntry (/opt/vmware/vcf/sddc-manager-ui-app/se
rver/src/services/workflows.js:69:31)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n  
  at async /opt/vmware/vcf/sddc-manager-ui-app/server/src/routes/api/workflows.js:49:18","status":500,"upn":"Administrator@V
SPHERE.LOCAL"}
2024-11-29T13:58:01.730+0000 DEBUG [c61a0da2d8bb4830] [services/lcmBundlesService.js, http-get-upgrade-status, getInprogress
Upgrades:942] Request for LCM inprogress upgrades data successful. {"id":"1171e083-168d-442e-8b87-d10f9cebe7ec","upn":"Admin
istrator@VSPHERE.LOCAL"}
2024-11-29T13:58:01.730+0000 DEBUG [c61a0da2d8bb4830] [services/lcmBundlesService.js, http-get-upgrade-status, getAvailableU
pgradesWithBundles:408] start getAvailableUpgradesWithBundles() {"id":"1171e083-168d-442e-8b87-d10f9cebe7ec","upn":"Administ
rator@VSPHERE.LOCAL"}

透過檢視以下 /var/log/vmware/vcf/lcm/lcm.log 日誌,可以發現這裡的錯誤資訊會更加明確,看著好像是這個“06f28563-3ecd-4abe-a707-431397d889b9”映像不存在於映像目錄中,然後獲取映像失敗!看到這裡,大概能猜到是什麼原因了,由於透過配置備份檔案還原了 SDDC Manager,而這個 SDDC Manager 是新部署的虛擬機器,但是在之前舊的 SDDC Manager 裡是存在可用的叢集映像的,透過配置檔案還原後,新 SDDC Manager 保留了映像的配置檔案,但是實際並不存在舊的映像,最終導致了查詢失敗並提示開始的錯誤。

2024-11-29T14:29:03.622+0000 ERROR [vcf_lcm,1cab22ff8c824239,8dc7] [c.v.v.i.b.i.TranslationMessage,http-nio-127.0.0.1-7400-exec-9] Can't find resource for bundle java.util.PropertyResourceBundle, key VCF_ERROR_INTERNAL_SERVER_ERROR.remedy
2024-11-29T14:29:03.624+0000 INFO  [vcf_lcm,9e04e7402b6c490e,f059] [c.v.v.l.r.a.c.v.p.PersonalityController,http-nio-127.0.0.1-7400-exec-10] Get personalities
2024-11-29T14:29:03.624+0000 INFO  [vcf_lcm,9e04e7402b6c490e,f059] [c.v.v.l.r.a.c.v.p.PersonalityController,http-nio-127.0.0.1-7400-exec-10] Fetching all the personalities
2024-11-29T14:29:03.638+0000 INFO  [vcf_lcm,6749cfaf02986e11c8becaa292bbcf1f,4106,bundleId=7a70a555-74ec-4849-89d4-3e45b551c8d8] [c.v.e.s.l.a.r.c.b.BundleController,http-nio-127.0.0.1-7400-exec-8] get downloaded manifest for Bundle Id 7a70a555-74ec-4849-89d4-3e45b551c8d8
2024-11-29T14:29:03.640+0000 ERROR [vcf_lcm,6749cfaf02986e11c8becaa292bbcf1f,4106,bundleId=7a70a555-74ec-4849-89d4-3e45b551c8d8] [c.v.e.s.l.d.c.b.BundleClientImpl,http-nio-127.0.0.1-7400-exec-8] For bundleId 7a70a555-74ec-4849-89d4-3e45b551c8d8, no manifest files available in /nfs/vmware/vcf/nfs-mount/bundle/7a70a555-74ec-4849-89d4-3e45b551c8d8/manifest
2024-11-29T14:29:03.644+0000 ERROR [vcf_lcm,9e04e7402b6c490e,f059] [c.v.e.s.l.f.p.ExportedPersonalityParser,http-nio-127.0.0.1-7400-exec-10] Personality directory:/nfs/vmware/vcf/nfs-mount/personalities/06f28563-3ecd-4abe-a707-431397d889b9 does not exist and hence populating hardware and additional metadata failed
2024-11-29T14:29:03.644+0000 ERROR [vcf_lcm,9e04e7402b6c490e,f059] [c.v.v.l.r.a.c.v.p.PersonalityController,http-nio-127.0.0.1-7400-exec-10] Get Personalities Failed
com.vmware.evo.sddc.lcm.model.error.LcmException: Personality directory: /nfs/vmware/vcf/nfs-mount/personalities/06f28563-3ecd-4abe-a707-431397d889b9 does not exist and hence populating hardware and additional metadata failed.
	at com.vmware.evo.sddc.lcm.file.personality.ExportedPersonalityParser.populatePersonalityWithVsphereExportedFilePaths(ExportedPersonalityParser.java:402)
	at com.vmware.evo.sddc.lcm.file.personality.ExportedPersonalityParser.populateFilePathAndSetHardwareInfo(ExportedPersonalityParser.java:306)
	at com.vmware.evo.sddc.lcm.services.impl.PersonalityManagementServiceImpl.getAllPersonalities(PersonalityManagementServiceImpl.java:322)
	at com.vmware.vcf.lcm.rest.api.controller.v1.personality.PersonalityController.getPersonalities(PersonalityController.java:170)
	at jdk.internal.reflect.GeneratedMethodAccessor230.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
	at org.springframework.security.authorization.method.AuthorizationManagerBeforeMethodInterceptor.invoke(AuthorizationManagerBeforeMethodInterceptor.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:703)
	at com.vmware.vcf.lcm.rest.api.controller.v1.personality.PersonalityController$$SpringCGLIB$$0.getPersonalities(<generated>)
	at jdk.internal.reflect.GeneratedMethodAccessor230.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:118)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:892)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:798)
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1081)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:974)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014)
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:903)
	at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:564)
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885)
	at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:195)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:101)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:110)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:108)
	at org.springframework.security.web.FilterChainProxy.lambda$doFilterInternal$3(FilterChainProxy.java:231)
	at org.springframework.security.web.ObservationFilterChainDecorator$FilterObservation$SimpleFilterObservation.lambda$wrap$1(ObservationFilterChainDecorator.java:479)
	at org.springframework.security.web.ObservationFilterChainDecorator$AroundFilterObservation$SimpleAroundFilterObservation.lambda$wrap$1(ObservationFilterChainDecorator.java:340)
	at org.springframework.security.web.ObservationFilterChainDecorator.lambda$wrapSecured$0(ObservationFilterChainDecorator.java:82)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:128)
	at org.springframework.security.web.access.intercept.AuthorizationFilter.doFilter(AuthorizationFilter.java:100)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:126)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:120)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:131)
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:85)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:100)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:179)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at com.vmware.vcf.security.authorization.AuthorizationFilter.doFilterInternal(AuthorizationFilter.java:99)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:107)
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:93)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90)
	at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at org.springframework.security.web.context.SecurityContextHolderFilter.doFilter(SecurityContextHolderFilter.java:82)
	at org.springframework.security.web.context.SecurityContextHolderFilter.doFilter(SecurityContextHolderFilter.java:69)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:62)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at org.springframework.security.web.session.DisableEncodeUrlFilter.doFilterInternal(DisableEncodeUrlFilter.java:42)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$AroundFilterObservation$SimpleAroundFilterObservation.lambda$wrap$0(ObservationFilterChainDecorator.java:323)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:224)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:233)
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:191)
	at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:113)
	at org.springframework.web.servlet.handler.HandlerMappingIntrospector.lambda$createCacheFilter$3(HandlerMappingIntrospector.java:195)
	at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:113)
	at org.springframework.web.filter.CompositeFilter.doFilter(CompositeFilter.java:74)
	at org.springframework.security.config.annotation.web.configuration.WebMvcSecurityConfiguration$CompositeFilterChainProxy.doFilter(WebMvcSecurityConfiguration.java:230)
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:352)
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:268)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.springframework.web.filter.ServerHttpObservationFilter.doFilterInternal(ServerHttpObservationFilter.java:109)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:482)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:115)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:389)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:904)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1741)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
	at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1190)
	at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
	at java.base/java.lang.Thread.run(Thread.java:840)
2024-11-29T14:29:03.645+0000 ERROR [vcf_lcm,9e04e7402b6c490e,f059] [c.v.e.s.e.h.LocalizableRuntimeExceptionHandler,http-nio-127.0.0.1-7400-exec-10] [A61F8G] VCF_ERROR_INTERNAL_SERVER_ERROR A problem has occurred on the server. Please retry or contact the service provider and provide the reference token.
com.vmware.evo.sddc.common.core.error.InternalServerErrorException: A problem has occurred on the server. Please retry or contact the service provider and provide the reference token.
	at com.vmware.vcf.lcm.rest.api.controller.v1.common.ExceptionAssemblers.toRestExceptionFromLcmException(ExceptionAssemblers.java:277)
	at com.vmware.vcf.lcm.rest.api.controller.v1.personality.PersonalityController.getPersonalities(PersonalityController.java:174)
	at jdk.internal.reflect.GeneratedMethodAccessor230.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
	at org.springframework.security.authorization.method.AuthorizationManagerBeforeMethodInterceptor.invoke(AuthorizationManagerBeforeMethodInterceptor.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:751)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:703)
	at com.vmware.vcf.lcm.rest.api.controller.v1.personality.PersonalityController$$SpringCGLIB$$0.getPersonalities(<generated>)
	at jdk.internal.reflect.GeneratedMethodAccessor230.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:118)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:892)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:798)
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1081)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:974)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014)
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:903)
	at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:564)
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885)
	at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:195)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:101)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:110)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:108)
	at org.springframework.security.web.FilterChainProxy.lambda$doFilterInternal$3(FilterChainProxy.java:231)
	at org.springframework.security.web.ObservationFilterChainDecorator$FilterObservation$SimpleFilterObservation.lambda$wrap$1(ObservationFilterChainDecorator.java:479)
	at org.springframework.security.web.ObservationFilterChainDecorator$AroundFilterObservation$SimpleAroundFilterObservation.lambda$wrap$1(ObservationFilterChainDecorator.java:340)
	at org.springframework.security.web.ObservationFilterChainDecorator.lambda$wrapSecured$0(ObservationFilterChainDecorator.java:82)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:128)
	at org.springframework.security.web.access.intercept.AuthorizationFilter.doFilter(AuthorizationFilter.java:100)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:126)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:120)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:131)
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:85)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:100)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:179)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at com.vmware.vcf.security.authorization.AuthorizationFilter.doFilterInternal(AuthorizationFilter.java:99)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:107)
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:93)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90)
	at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at org.springframework.security.web.context.SecurityContextHolderFilter.doFilter(SecurityContextHolderFilter.java:82)
	at org.springframework.security.web.context.SecurityContextHolderFilter.doFilter(SecurityContextHolderFilter.java:69)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:62)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at org.springframework.security.web.session.DisableEncodeUrlFilter.doFilterInternal(DisableEncodeUrlFilter.java:42)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240)
	at org.springframework.security.web.ObservationFilterChainDecorator$AroundFilterObservation$SimpleAroundFilterObservation.lambda$wrap$0(ObservationFilterChainDecorator.java:323)
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:224)
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137)
	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:233)
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:191)
	at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:113)
	at org.springframework.web.servlet.handler.HandlerMappingIntrospector.lambda$createCacheFilter$3(HandlerMappingIntrospector.java:195)
	at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:113)
	at org.springframework.web.filter.CompositeFilter.doFilter(CompositeFilter.java:74)
	at org.springframework.security.config.annotation.web.configuration.WebMvcSecurityConfiguration$CompositeFilterChainProxy.doFilter(WebMvcSecurityConfiguration.java:230)
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:352)
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:268)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.springframework.web.filter.ServerHttpObservationFilter.doFilterInternal(ServerHttpObservationFilter.java:109)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:482)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:115)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:389)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:904)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1741)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
	at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1190)
	at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)
	at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: com.vmware.evo.sddc.lcm.model.error.LcmException: Personality directory: /nfs/vmware/vcf/nfs-mount/personalities/06f28563-3ecd-4abe-a707-431397d889b9 does not exist and hence populating hardware and additional metadata failed.
	at com.vmware.evo.sddc.lcm.file.personality.ExportedPersonalityParser.populatePersonalityWithVsphereExportedFilePaths(ExportedPersonalityParser.java:402)
	at com.vmware.evo.sddc.lcm.file.personality.ExportedPersonalityParser.populateFilePathAndSetHardwareInfo(ExportedPersonalityParser.java:306)
	at com.vmware.evo.sddc.lcm.services.impl.PersonalityManagementServiceImpl.getAllPersonalities(PersonalityManagementServiceImpl.java:322)
	at com.vmware.vcf.lcm.rest.api.controller.v1.personality.PersonalityController.getPersonalities(PersonalityController.java:170)
	... 149 common frames omitted

參考這篇文章,嘗試透過 PowerCLI 來管理 SDDC Manager 中的叢集映像,執行 PowerShell 並使用以下命令連線到 SDDC Manager。

Connect-VcfSddcManagerServer -Server vcf-mgmt01-sddc01.mulab.local -User administrator@vsphere.local -Password Vcf520@password

當執行 Invoke-VcfGetPersonalities 命令後,出現如下錯誤,這提示跟上面日誌中的報錯居然一樣,看來確實跟缺失的映像有關係。

Invoke-VcfGetPersonalities : Error calling GetPersonalities: {"errorCode":"VCF_ERROR_INTERNAL_SERVER_ERROR","arguments":[],"message":"A problem has occ
urred on the server. Please retry or contact the service provider and provide the reference token.","causes":[{"type":"com.vmware.evo.sddc.lcm.model.er
ror.LcmException","message":"Personality directory: /nfs/vmware/vcf/nfs-mount/personalities/06f28563-3ecd-4abe-a707-431397d889b9 does not exist and hen
ce populating hardware and additional metadata failed."}],"referenceToken":"RJ5JBH"}

於是,嘗試使用以下命令將這個缺失的映像從 SDDC Manager 中刪除。

Invoke-VcfDeletePersonality -PersonalityId 06f28563-3ecd-4abe-a707-431397d889b9

當完成刪除後,一切恢復正常!

相關文章