Thursday, July 2, 2015

Agent status shows 'Availability Evaluation Error' on EM 12c Console

Background
During deploying agent 12.1.0.4 to new host for EM 12.1.0.4, the local firewall was enabled by accident and blocked incoming access. The deployment is successful. However, OEM shows error:
'Unable to connect to the agent at https://host01.dbaplus.ca:3872/emd/main/ [Connection establishment timed out])'

After stopping firewall on the new host, agent status is shown 'Availability Evaluation Error' on EM 12c Console.
Diagnose
On the new host:
$ /u01/app/oracle/product/12.1.0/agent/core/12.1.0.4.0/bin/emctl status agent
Oracle Enterprise Manager Cloud Control 12c Release 4
Copyright (c) 1996, 2014 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
Agent Version          : 12.1.0.4.0
OMS Version            : 12.1.0.4.0
Protocol Version       : 12.1.0.1.0
Agent Home             : /u01/app/oracle/product/12.1.0/agent/agent_inst
Agent Log Directory    : /u01/app/oracle/product/12.1.0/agent/agent_inst/sysman/log
Agent Binaries         : /u01/app/oracle/product/12.1.0/agent/core/12.1.0.4.0
Agent Process ID       : 8366
Parent Process ID      : 8280
Agent URL              : https://host01.dbaplus.ca:3872/emd/main/
Local Agent URL in NAT : https://host01.dbaplus.ca:3872/emd/main/
Repository URL         : https://omshost.dbaplus.ca:1159/empbs/upload
Started at             : 2015-07-02 11:29:46
Started by user        : oracle
Operating System       : Linux version 2.6.39-400.17.1.el6uek.x86_64 (amd64)
Last Reload            : (none)
Last successful upload                       : (none)
Last attempted upload                        : (none)
Total Megabytes of XML files uploaded so far : 0
Number of XML files pending upload           : 0
Size of XML files pending upload(MB)         : 0
Available disk space on upload filesystem    : 21.46%
Collection Status                            : Collections enabled
Heartbeat Status                             : Ok
Last attempted heartbeat to OMS              : 2015-07-02 11:30:49
Last successful heartbeat to OMS             : 2015-07-02 11:30:49
Next scheduled heartbeat to OMS              : 2015-07-02 11:31:49

---------------------------------------------------------------
Agent is Running and Ready

We can find that 'Last successful upload' is '(none)', it is supposed to be a valid date & time. Try to manually upload:
$ /u01/app/oracle/product/12.1.0/agent/core/12.1.0.4.0/bin/emctl upload agent
Oracle Enterprise Manager Cloud Control 12c Release 4
Copyright (c) 1996, 2014 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
EMD upload completed successfully

It does not help. 'Last successful upload' is still '(none)' and agent status is still shown 'Availability Evaluation Error'.
Try to re-synchronize the agent on OMS server:
$ . oraenv
ORACLE_SID = [oracle] ? OMS
ORACLE_HOME = [/home/oracle] ? /u01/app/oracle/product/12.1.0/OMS/oms
The Oracle base has been set to 
$ which emcli
/u01/app/oracle/product/12.1.0/OMS/oms/bin/emcli
$ 
$ emcli  login -username=SYSMAN
Enter password 

Login successful
$ emcli sync
Synchronized successfully
$ emcli resyncAgent -agent=host01.dbaplus.ca:3872
Resync job RESYNC_20150702142805361 successfully submitted

Now agent is shown UP on EM 12c console.

No comments: