DNS and DHCP Setup Example for Grid Infrastructure GNS

yyp2009發表於2014-07-01
原文網址 : http://blog.itpub.net/13750068/viewspace-1200415/
DNSASTStruct

In this Document

Purpose
Scope
Details
  DNS Setup Example
  To verify delegation
  DHCP Server Setup Example
  Validate DHCP Server from cluvfy
  Validating DHCP Server on OS level
  Solaris
  AIX
  Linux
References

Applies to: 

Oracle Database - Enterprise Edition - Version 11.2.0.1 and later
Information in this document applies to any platform.

Purpose

The Grid Naming Service (GNS) is a new feature in the 11.2 Grid Infrastructure (GI). Its design is to simplify the management of the network configuration of the Real Application Cluster (RAC). When GNS is used, it eliminates the manual allocation of the Node and Single Client Access Name (SCAN) Virtual Internet Protocol (VIP) addresses. It also eliminates the need to configure VIP names and SCAN names in a Domain Name Server (DNS) as GNS provides the name resolution for the cluster. What this means is the GNS is configured as a delegated authority in DNS for the Cluster Nodes.


1. Allocate one static IP address on public network for the GNS VIP.
2. Setup GNS as a delegate for the cluster's sub-domain on the DNS server.
3. Validate that there is a DHCP server that is addressable by the cluster nodes.

When installing GI, there are 2 options:

1. Configure GNS and let it handle name resolution in the cluster subdomain.
2. Choose not to configure GNS and configure each Node and SCAN name with IP addresses defined in DNS

This note does not address option (2) as it is intended to cover how to setup the DNS servers and DHCP servers to work with GNS.

Please refer to following for more info:

Oracle? Grid Infrastructure
Installation Guide

Checking the Network Requirements


This article is an example of how to configure DNS to delegate a sub-domain to GNS. It also covers how DHCP is to be configured to allocate IP address for SCAN and Node VIP addresses.

Example below should be limited to an isolated test Lab. It should be set up by the Network Administrator, as it covers the configuration of the DNS server and DHCP server. This note does not cover the configuration of any of the cluster nodes.

Scope

This document is intended for Network Administrator, Oracle Clusterware/RAC Database Administrators and Oracle support engineers. 

Details

DNS Setup Example

The example below is based on bind-9.3.4-10 on Linux though the setting should be generic for bind.
In this example, the GNS VIP is gns.us.eot.com with IP 120.0.0.200. The GNS subdomain is us.eot.com.

Main config /etc/named.conf:

options {
        directory "/etc/named";
        zone-statistics yes;
        allow-transfer {"none";};
};

controls {
        inet 127.0.0.1 allow { localhost; } keys { rndc; };
};

key rndc {
        algorithm hmac-md5;
        secret "tpyCbBuiElAdZMfkirg3RcrCtyql0kW42vT24LxXysM=";
};

zone "." {
        type hint;
        file "root";
};

zone "0.0.127.in-addr.arpa" {
        type master;
        file "127.0.0";
        notify no;
};

zone "eot.com" {
        type master;
        file "eot.com";
        notify no;
};

zone "0.0.120.in-addr.arpa" {
        type master;
        file "120.0.0";
        notify no;
};

The main config is for a single forward domain eot.com and reverse domain for subnet 120.0.0.0

eot.com zone config file under /etc/named as defined in named.conf file above: 


$TTL 3D @       IN      SOA     eot.com. root.eot.com. (

                        199609206       ; serial, todays date + todays serial #
                        8H              ; refresh, seconds
                        2H              ; retry, seconds
                        4W              ; expire, seconds
                        1D )            ; minimum, seconds
                NS      eot.com.
                MX      10 eot.com.  ; Primary Mail Exchanger
                TXT     "EOT Domain"

localhost       A       127.0.0.1

eot.com.        A       120.0.0.254
dns             CNAME   eot.com.


$ORIGIN         us.eot.com. @       IN      NS     gns.us.eot.com.

gns.us.eot.com.    IN      A      120.0.0.200

Above setting will delegate resolution of sub-domain us.eot.com to GNS VIP 120.0.0.200.

 

To verify delegation

To verify delegation is working as expected, execute the following nslookup and both should resolve to same set of IP addresses once GI is up and running: 

nslookup
nslookup

 For example: 

nslookup scan.us.eot.com 120.0.0.254
nslookup scan.us.eot.com 120.0.0.200

  

DHCP Server Setup Example

The example below is based on dhcp-3.0.5 on Linux:
/etc/dhcp/dhcp.conf

ddns-update-style interim;
ignore client-updates;

subnet 120.0.0.0 netmask 255.255.255.0 {

  range 120.0.0.201 120.0.0.220;
  default-lease-time 86400;
  option routers 120.0.0.1;
  option ip-forwarding off;
  option broadcast-address 120.0.0.255;
  option subnet-mask 255.255.255.0;
  option time-offset              -28800;
  option ntp-servers 120.0.0.254;
  option domain-name              "us.eot.com";
  option domain-name-servers 120.0.0.254;
}

The DHCP will provide dynamic IPs for client in the range of 120.0.0.201-120.0.0.220 with subnet mask of 255.255.255.0, the default lease time is 86400 seconds, timezone -28800 which is PST (GMT-8), with default gateway of 120.0.0.1 and default ntp and DNS server of 120.0.0.254

Note: When using 3rd party DHCP server, the DHCP server must support "Client Identifier" for GNS to work. For example, for Lucent DHCP Server, the option name is "SupportClientId". Please consult with the DHCP Server provider to confirm if this feature is available and how to enable it. However this does not apply to the Linux DHCP server as example above.

Without this feature, DHCP server will not allocate IP to a client which does not have a valid MAC address. As VIP and SCAN VIP are designed to failover, they will not have MAC address assigned. When Oracle request IP for these VIPs from the DHCP server, we only supply the Client Identifier, not the MAC address in the client request.

A tcpdump of the DHCP packet will show:
******************
15:59:09.165705  In 00:10:e0:3a:2c:56 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 414: vlan 2100, p 0, ethertype IPv4, (tos 0x0, ttl  64, id 0, offset 0, flags [DF] , proto: UDP (17), length: 396) 120.0.0.10.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:00:00:00:00:00, length 368, xid 0x89b5e43f, Flags [Broadcast]
         Client-Ethernet-Address 00:00:00:00:00:00
         sname "racnode1-vip"
         Vendor-rfc1048 Extensions
           Magic Cookie 0x63825363
           DHCP-Message Option 53, length 1: Discover
           MSZ Option 57, length 2: 8
           Client-ID Option 61, length 12: "racnode1-vip"
******************

The 3rd party DHCP server will report error:  "Oct  7 15:59:09 packet has an invalid hardware address" if it does not have support "Client Identifier" feature enabled.

 

Validate DHCP Server from cluvfy

To verify whether client can get IP from DHCP server, as root:

# /bin/cluvfy comp dhcp -clustername clu202

Verifying DHCP Check
Checking if any DHCP server exists on the network...
At least one DHCP server exists on the network and is listening on port 67
Checking if DHCP server has sufficient free IP addresses for all VIPs...
DHCP server was able to provide sufficient number of IP addresses
The DHCP server response time is within acceptable limits

Verification of DHCP Check was successful.

The verbose output looks like this:
# /bin/cluvfy comp dhcp -clustername clu202 -verbose

Verifying DHCP Check
Checking if any DHCP server exists on the network...
Checking if network CRS resource is configured and online
Network CRS resource is offline or not configured. Proceeding with DHCP checks.
CRS-10009: DHCP server returned server: 120.0.0.200, loan address : 120.0.0.250/255.255.255.0, lease time: 86400

At least one DHCP server exists on the network and is listening on port 67
Checking if DHCP server has sufficient free IP addresses for all VIPs...
Sending DHCP "DISCOVER" packets for client ID "clu202-scan1-vip"
CRS-10009: DHCP server returned server: 120.0.0.254, loan address : 120.0.0.250/255.255.255.0, lease time: 86400

Sending DHCP "REQUEST" packets for client ID "clu202-scan1-vip"
CRS-10009: DHCP server returned server: 120.0.0.254, loan address : 120.0.0.250/255.255.255.0, lease time: 86400

Sending DHCP "DISCOVER" packets for client ID "clu202-scan2-vip"
CRS-10009: DHCP server returned server: 120.0.0.254, loan address : 120.0.0.249/255.255.255.0, lease time: 86400

Sending DHCP "REQUEST" packets for client ID "clu202-scan2-vip"
CRS-10009: DHCP server returned server: 120.0.0.254, loan address : 120.0.0.249/255.255.255.0, lease time: 86400

Sending DHCP "DISCOVER" packets for client ID "clu202-scan3-vip"
CRS-10009: DHCP server returned server: 120.0.0.254, loan address : 120.0.0.248/255.255.255.0, lease time: 86400

Sending DHCP "REQUEST" packets for client ID "clu202-scan3-vip"
CRS-10009: DHCP server returned server: 120.0.0.254, loan address : 120.0.0.248/255.255.255.0, lease time: 86400

Sending DHCP "DISCOVER" packets for client ID "clu202-racnode1-vip"
CRS-10009: DHCP server returned server: 120.0.0.254, loan address : 120.0.0.247/255.255.255.0, lease time: 86400

Sending DHCP "REQUEST" packets for client ID "clu202-racnode1-vip"
CRS-10009: DHCP server returned server: 120.0.0.254, loan address : 120.0.0.247/255.255.255.0, lease time: 86400

CRS-10012: released DHCP server lease for client ID clu202-scan1-vip on port 67

CRS-10012: released DHCP server lease for client ID clu202-scan2-vip on port 67

CRS-10012: released DHCP server lease for client ID clu202-scan3-vip on port 67

CRS-10012: released DHCP server lease for client ID clu202-racnode1-vip on port 67

DHCP server was able to provide sufficient number of IP addresses
The DHCP server response time is within acceptable limits

Verification of DHCP Check was successful.


Note: "cluvfy comp dhcp" is supported in 11.2.0.2 and above, to get latest CVU, go to http://otn.oracle.com

* If above command is not executed as root, the following error will be reported:

PRVF-0055 : User "grid" does not have sufficient authorization to run this command


* If the following error is reported, refer to note 1316815.1

PRVF-4037 : CRS is not installed on any of the nodes
Verification cannot proceed


* If it fails to get IP from DHCP server, engage System Administrator or Network Administrator to verify DHCP server setup

Verifying DHCP Check
Checking if any DHCP server exists on the network...

PRVF-5801 : Failed to execute command "/ocw/grid/bin/crsctl discover dhcp -clientid giclu-scan0-vip" on all nodes
PRVF-5704 : No DHCP server were discovered on the public network listening on port 67

Verification of DHCP Check was unsuccessful on all the specified nodes.

crsctl discover dhcp -clientid giclu-scan0-vip
CRS-10010: unable to discover DHCP server in the network listening on port 67 for client ID giclu-scan0-vip
CRS-4000: Command discover failed, or completed with errors.

Agent may not get an IP address if DHCP runs on one of the cluster node, this is fixed in bug 14638061 which is included in 12.1.0.2

Validating DHCP Server on OS level

Testing DHCP could potentially change local network config and cause connection lost, engage System Administrator if you have any doubt. All commands need to be executed as root.

Solaris

Assuming e1000g0 is on public network, the following can be executed to test whether OS is able to get IP from DHCP server

# ifconifg e1000g0:9                  #### to confirm e1000g0:9 is unused; if it's used, try a different alias
ifconfig: status: SIOCGLIFFLAGS: e1000g0:9: no such interface    #### output confirms e1000g0:9 is not being used
# ifconfig e1000g0:9 plumb
# ifconfg e1000g0:9 dhcp start
# ifconfig e1000g0:9
e1000g0:9: flags=1004843DHCP,IPv4> mtu 1500 index 2
       inet 10.11.25.202 netmask fffffe00 broadcast 10.11.25.255
# ifconfig e1000g0:9 down
# ifconfig e1000g0:9 unplumb

 

AIX

# smitty dhcpcd

For details, please consult System Administrator

Linux

Assuming eth0 is on public network and eth0:9 is unused (ifconfig output should not have eth0:9, try a different one if eth0:9 is used), create /etc/sysconfig/network-scripts/ifcfg-eth0:9 with the following:

DEVICE=eth0:9
BOOTPROTO=dhcp


To start eth0:9:

# cp -p /etc/resolv.conf /etc/resolv.conf.dhcporig    #### backup the file as dhcp client may modify it
# /sbin/ifup eth0:9
Determining IP information for eth0:9... failed.    #### failed to get IP from DHCP server

Determining IP information for eth0:9... done.  #### succeeded to get IP from DHCP server


To stop eth0:9:

# ifconfig eth0:9 down
# ps -ef| grep eth0:9| grep -v grep
root      7930     1  0 17:48 ?        00:00:00 /sbin/dhclient -1 -q -lf /var/lib/dhclient/dhclient-eth0:9.leases -pf /var/run/dhclient-eth3:11.pid eth3:11
# kill -9
# cp -p /etc/resolv.conf.dhcporig /etc/resolv.conf    #### backup the file as dhcp client may modify it


Once the testing is done, remove /etc/sysconfig/network-scripts/ifcfg-eth0:9

來自 “ ITPUB部落格 ” ，連結：http://blog.itpub.net/13750068/viewspace-1200415/，如需轉載，請註明出處，否則將追究法律責任。

相關文章