Communication: Agent to OMS Communication Fails if the Agent is 'Blocked' in the 10.2.0.5 Grid Console (Doc ID 799618.1)

In this Document

Cause

Solution

References

Applies to:

Enterprise Manager Base Platform - Version 10.2.0.5 to 10.2.0.5 [Release 10.2]
Information in this document applies to any platform.
***Checked for relevance on 06-Feb-2014***

Symptoms

After upgrading the OMS and Agent to 10.2.0.5 version, getting the following error in /sysman/log/emagent.trc :

Did not receive valid response to ping "ERROR-Agent is blocked. Blocked reason is: Agent is out-of-sync with repository. This most likely means that the agent was reinstalled or recovered. Please contact an EM administrator to unblock the agent by performing an agent resync from the console. Please contact EM adminstrator to unblock the agent"

The command: emctl upload agent

fails with :

Oracle Enterprise Manager 10g Release 5 Grid Control 10.2.0.5.0.
Copyright (c) 1996, 2009 Oracle Corporation. All rights reserved.
---------------------------------------------------------------
EMD upload error: uploadXMLFiles skipped :: OMS version not checked yet..

The Agent can however be secured successfully with the OMS.

Cause

The Agent Re-synchronisation is a new feature in 10.2.0.5 Grid Control which verifies whether an Agent that been uploading earlier to the OMS is re-installed or restored from a backup. If yes, the OMS blocks further updates from this Agent until the information about the Agent and all its targets are synchronized between the Repository and the Agent.

Factors which can cause the Agent to go out-of-sync with the Repository:

1. Agent is re-installed or restored from a backup, on the same port number as before.
2. The /sysman/emd/agntstmp.txt is manually deleted for some reason.

Solution

To solve the issue you have to perform following from grid console

+ Login to 10.2.0.5 Grid Console as sysman user

+ Navigate to Setup -> Agents -> Click on problematic Agent name

+ In Agent home page click on "Resynchronize" Button on right hand top of the page.
Choose the 'Unblock Agent' option and click Continue

Ensure that the Agent is up and running when attempting the Resynchronization.

Once the re-synchronization is completed successfully, the Agent should be able to communicate with the OMS.

Sometimes, the re-synchronization may complete with results such as:

Agent Operation completed with errors. For those targets that could not be saved, please go to the target's monitoring configuration page to save them. All other targets have been saved successfully. Agent has not been unblocked.

1. Error saving target database1.domain:oracle_database - Skipping target {database1.domain, oracle_database}: Missing properties - UserName, password
2. Error saving target database2.domain:oracle_database - Skipping target {database2.domain, oracle_database}: Missing properties - UserName, password
...
OR

Error saving target EnterpriseManager0.omsmachine.domain_Web
Cache:oracle_webcache - Skipping target
{EnterpriseManager0.omsmachine.domain_Web Cache, oracle_webcache}:
Missing properties - authpwd, authuser

The above indicates that the Repository has been unable to completely save the details of the all targets that the Agent currently has. In this case, the Agent was re-installed and has newly discovered all the Database targets and iAS targets, which are not completely configured as yet i.e the monitoring password is missing. To resolve the above:

- Navigate to the Agent home page in the Grid Console. Choose one database at a time and click on the 'Configure' button.
- In the Monitoring Configuration page, enter the password for the monitoring user and save.
- Perform the same for any other target that is shown in the result from the Agent re-synchronization.
- Once, all the targets are configured in this manner go to Setup -> Agents -> choose this Agent name and click on the 'Unblock' button.