Remote Diagnostic Agent (RDA) - RAC Cluster Guide (Doc ID 359395.1)

Remote Diagnostic Agent (RDA) - RAC Cluster Guide (Doc ID 359395.1)

OSS Support Tools - Version 8.07 and later
Linux x86
HP-UX PA-RISC (32-bit)
z*OBSOLETE: Microsoft Windows 2000
Microsoft Windows (32-bit)

*Main Content
Enter the diagnostic tools text.

RDA - RAC Cluster/Multi-Node Users Guide

In This Document

Overview

Starting from release 4.2, RDA has the capability to perform remote data collections. RDA which is installed on one node can control the installation and execution of RDA on multiple nodes/Oracle homes.

This guide provides users with an overview of RDA, download instructions, and general steps on to how to execute RDA within anOracle RAC (Cluster or Multiple node) environment.

Supported Topologies

In the current release, only UNIX and Windows nodes are supported. RDA can be also installed on shared disk. In a shared disk context, all nodes must be able to write in that directory. Remote command execution and file transfer can only be possible under one of the following conditions:

• ssh/scp.ssh must be configured to work with security keys instead of passwords. In that context, RDA will use an existing (or its own) authentication agent to avoid asking the passphrase multiple times.
• rsh or remsh/rcp.[NOTE: appropriate configuration should be done to avoid providing passwords repetitively]

The collecting node could be outside the cluster if that node has a database connection within the cluster topology.

Platforms Supported

At this time, RDA for Oracle RAC cluster has been successfully tested to run on the following Supported platforms:

• IBM AIX
• Intel Linux (Oracle Enterprise, RedHat, and SuSE)
• HP-UX (10.* and 11.*)
• HP Tru64
• Mac OS X/Darwin
• Microsoft Windows (requires Cygwin or MKS)
• Sequent Dynix/Ptx
• Sun Solaris (2.6 - 2.10)

RDA can also be run on other platforms that support Perl 5.005 and above, however, Oracle Support recommends testing them on a non-production server first, as their performance is unpredictable. For example, you will receive errors when RDA attempts to run utilities and commands that are not supported on those platforms.

Versions Supported

RDA supports most supported versions of the above Oracle products. In most cases it will also run on desupported versions as well, although the information collected may not be as extensive.

Installation Instructions

Instructions for UNIX type operating systems:

1. From all your existing nodes, select a single node where you start the data collection and store the combined output on.
2. Choose or create a directory that will be used exclusively for RDA on your Unix server. Make sure you have ample space for RDA output in that location you have chosen, as this location will contain all data gathered from all nodes including local node as well as combined archive package. It does not matter where you create this directory or what it is named, but it should be owned by the same user that will run RDA. Do notuse a directory that contains an older version of RDA unless you have deleted the older version of RDA first; if need be prior setup files can be reused.

1.  
2. FTP the downloaded file from Knowledge Article 314422.1 to your Unix server in binary mode and place it in the directory created in step 1.
3. Extract the archive contents into a new directory, preserving the directory structure of the archive. Do not extract into a directory that contains an older RDA version.

Example:
unzip .zip
or
tar xvf .tar
or
gunzip .tar.gz
tar xvf.tar

1. Make sure the RDA command (rda.sh and rda.pl) is executable. To verify, enter the following command:

chmod +x

Instructions for Microsoft Windows based operating systems

1. Choose or create a directory that will be used exclusively for RDA on your Unix server. Make sure you have ample space for RDA output in that location you have chosen, as this location will contain all data gathered from all nodes including local node as well as combined archive package. It does not matter where you create this directory or what it is named, but it should be owned by the same user that will run RDA. Do not use a directory that contains an older version of RDA unless you have deleted the older version of RDA first; if need be prior setup files can be reused. 

1. Transfer the downloaded rda.zip file from Knowledge Article 314422.1 to your Windows Server and place it in the directory created in step 1. If transferring the zip file from one machine to another using FTP, remember to transfer it in binary mode.
2. extract the .zip archive contents into a new directory, preserving the directory structure of the archive. Do not extract into a directory that contains an older RDA version. 

Example:
unzip rda.zip

1. You can verify the RDA installation using the following command:
   rda.cmd -cv

Running RDA

Instructions for UNIX type operating systems:

1. If performing a single node collection: Log on as the UNIX user that owns the Oracle installation. On some operating systems, this user will not have the necessary permissions to run all of the commands and utilities called by RDA (e.g. sar, top, vmstat, etc). If you are running RDA to assist in resolving a tar, the analyst will most likely need the information pertaining to the Oracle owner. The exception to this rule is when RDA is used to assist in a performance related issue. In this case, Oracle support recommends that you run RDA as the UNIX user who owns the Oracle software.
   
   If performing a multiple node collection: Log on as the user the owns the RDA installation. Ensure this user will have the necessary permissions to run all of the commands and utilities called by RDA (e.g. sar, top, vmstat, etc), especially SSH/SCP or RSH/RCP. If you are running RDA to assist in resolving a tar, the analyst will most likely need the information pertaining to the Oracle owner.

1. Select the RDA command line script that you will be using, either: 

-- rda.sh Use this command if Perl is not available.

-- rda.pl Use the following command to verify Perl is installed and available in the path:

perl -V

Inspect the command output, checking that '.' (i.e. the current directory) is present in @INC section.

1. Verify the RAC node status (e.g. by using either olsnodes or lsnodes.) and take appropriate actions to ensure they are available for data colleciton. (NOTE: RDA can be used in case of failures too). If you are doing the data collection from a system outside the cluster, verify thet the cluster database is accessible by using sqlplus.
2. Ensure remote connectivity using SSH/SCP or RSH/RCP is available and works. Test remote connectively to each node that data collection is going to performed on. For example you can check to see if you list the environment variable on system2 from system1: 
   
   $ssh user@system2 env 
   
   In case of problems more information can be obtained using -T TSTssh. Specify the corresponding hostname when prompted. This test module tests different commands/options to test the connectivity. Please note which ones are successful (exit code 0).

1. The data collection requires an initial setup to determine which information should be collected. Enter the following command to initiate the set up:

RDA 4.3x: 
$./ -vX RDA::Remote setup_cluster []

 can be specified as argument to get the cluster nodes from the running database. Following user formats are supported:
user     -To connect with that user in the database referenced by the ORACLE_SID environment variable.
user@SID -To connect with that user in the specified database.
/        -To connect as "/ AS SYSDBA" in the database referenced by the ORACLE_SID environment variable. You must have enough privileges.
/@SID    -To connect as "/ AS SYSDBA" in the specified database. You must have enough privileges for doing it.

If  is not specified, the operating system command (olsnodes or lsnodes) is used to get the cluster nodes. In that case, the RDA collection must be initiated from one of the cluster nodes.
If you need to perform a clean setup add -f option to the command above (./ -vfX ...)

RDA 8.x:
$./ -vXRda start CLOUD -pRac_Assessment

1. Including the dot ensures that RDA is executed from the local directory.

After setup completes,you can review the result definition file output.cfg by opening it a text editor like vi.

You can also choose to collect only specific data. For more details, view the command usage help by specifying the -h option, or complete manual page with the -M option.

1. If any node are reported as inaccessible and are not really down, use the TSTssh module to identify the possible problem or which command variant could work. For instance if ssh does not work but ssh without timeout works for the node identified by NOD002, please adapt its type by

RDA 4.x: $./ -vX RDA::Remote set ssh0 NOD002

RDA 8.x: $./ -vC RDA.REMOTE set

1. At any moment if you want list the nodes that has been configured and to check their accessibility. The following example shows all remote nodes found by RDA and that N0D002 is the remote node. 
   
   RDA4.3x:
   
    $./ -vX RDA::Remote list
   Nodes: 
   . NOD001 db01-autoxyz
   . NOD002 db02-autoxyz
   1 nodes found
   
   RDA8.x: $./ -vC RDA.REMOTE list 
2. MAKE SURE: Make sure you have ample space for RDA output (inlcuding locale node data, remote node data, and finally combined RDA package) in the output location you have chosen in STEP 5. Sometimes depending on the size of your data files (alert.log, *.trc, install*.log, apache logs, networking logs, etc) of the Oracle installation of each of the nodes the final size of your RDA collection may reach over 1000 MB in size.
3. At this point, you can collect diagnostic information. sqlplus should connect to the database with the userid that you specified during the setup. Start the data collection using the following command:

RDA 4.3x: $./ -v -e REMOTE_TRACE=1
RDA 8.x: $./ -v -e TRC/TRACE=1

The -v option is optional, it allows you to view the collection progression. 
The -e REMOTE_TRACE=1 or -e TRC/TRACE=1 allows to you to follow the progress on the remote collections.
To retry the last step of remote data collections, when you recieve errors from the nodes:
RDA4.3x:   $./ -vX RDA::Remote retry [ ...]
RDA8.x: $./ -vC RDA.REMOTE retry [ ...]

1. To restart the remote node collection from the beginning:
   RDA4.x:   $./ -v -X RDA::Remote restart [ ...]
   RDA8.x: $./ -vC RDA.REMOTE restart [ ...]
   It is only performs when the node step is already defined.
2. The output is a set of HTML files that are located in the RDA output directory which you specified at setup. You can review the data collected by using a Web Browser to open the following file located in:

/__start.htm

1. The final output is also packaged in an archive located in the chosen during RDA setup -- the packaged output file will have a .zip, .tar, .tar.gz, or .tar.Z extension. If the data collection was generated to assist in resolving a Service Request, send the report archive (note: file name varies, but ends in .zip, .tar,.tar.gz, or .tar.Z ; e.a: RDA.RDA_db_autoxyz.tar) to Oracle Support by uploading the file via My Oracle Support. If FTP'ing the file, please be sure to FTP in BINARY format. Do not rename the file, as the file name helps Oracle Support quickly identify that RDA output is attached to the service request.

Instructions for Microsoft Windows based operating systems

1. Only single node RAC collections is supported at this time; therefore RDA must be install and executed on each RAC node manually.
2. Follow the instruction on the Main RDA Overview and User guide - (Note.314422.1)

Special Notes On Userids And Passwords

RDA can perform OS authentication, which eliminates having to enter a password for database information gathering.
It accepts "/" as a username to avoid entering a password when RDA is gathering database information.

If another user is being used for the remote collection, its password will be encoded in the remote setup files. To limit security vulnerabilities, the permissions of the remote RDA output directories should be set as restrictive as possible.

Special Note on the Output Directory

To limit security vulnerabilities, the permissions of the RDA output directory should be set as restrictive as possible. The output directory could contain sensitive configuration information and, when no other mechanism is available, temporary data collection files.

Remote collections output directories are always created in the remote top RDA directories structure. Their name correspond to the node identifiers.

Viewing RDA Report Output Files

RDA output is not encrypted and can be viewed by anyone. You can view the RDA report files using almost any web browser. We recommend using  1.x ( or Higher), Netscape 4.x (or higher), or Internet Explorer 4.x (or higher).

Reporting Problems or Feedback