Listener in INTERMEDIATE status with "Not All Endpoints Registered

In this Document

Symptoms

Cause

Solution

References

Oracle Server - Enterprise Edition - Version 11.2.0.1 and later
Information in this document applies to any platform.

2 node RAC cluster, trying to add 2nd instance using dbca, get error:

"Connection to the database cannot be established because the listener could be down. Please make sure that the service is registered with a listener and the listener is up."

$crsctl stat res -t shows the listeners are in INTERMEDIATE status on node 2.

ora.LISTENER.lsnr
       ONLINE   ONLINE               racdb1
       ONLINE   INTERMEDIATE         racdb2   Not All Endpoints Registered
ora.LISTENER_SCAN1.lsnr
   1   ONLINE   INTERMEDIATE         racdb2    Not All Endpoints Registered

 

The problem is caused by another listener defined statically in listener.ora, using the same port and IP is running from the RDBMS ORACLE_HOME, started manually causing the default listener starting from GRID_HOME can not register its endpoint. Hence the error reported in dbca.

ps -ef | grep tns:

grid   7222 1 0 Apr26 ? 00:00:13 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN1 -inherit
grid   7237 1 0 Apr26 ? 00:00:13 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER -inherit
oracle 7354 1 0 Apr26 ? 00:00:01 /u02/app/oracle/product/11.2.0/db/bin/tnslsnr LISTENER -inherit

  Another possible cause is the listener or scan listener being defined manually in listener.ora, for example:

LISTENER_SCAN3 =                          
 (DESCRIPTION =
   (ADDRESS = (PROTOCOL = TCP)(HOST = racnode1)(PORT = 1523))
 )

LISTENER_SCAN1 =                            
 (DESCRIPTION =
   (ADDRESS = (PROTOCOL = TCP)(HOST = racnode1)(PORT = 1521))

 

 

From 11.2 onwards, all listeners should be runing from GRID_HOME, listener and listener_scan entry should be added automatically into listener.ora, no manual editing is required for TCP definition.

1. Stop the listener running from RDBMS ORACLE_HOME

$/bin/lsnrctl stop LISTENER

2. stop the listener from GRID_HOME

$/bin/srvctl stop listener -n
$/bin/srvctl stop scan_listener -i

eg:

$/bin/srvctl stop listener -n racnode1
$/bin/srvctl stop scan_listener -i 1

If above command fails to stop the tnslsnr process, please use "kill -9 " to stop the LISTENER and LISTENER_SCAN1 process.

3. remove any manually added LISTENER definition from listener.ora if it exists

4. restart the LISTENER and LISTENER_SCAN1  from GRID_HOME

$/bin/srvctl start listener -n
$/bin/srvctl start scan_listener -i

5. check crsctl stat res -t output, they both should show ONLINE status now.