How configure Multiples Public Network in a Grid Infrastructure 11g R2 (11.2) environment
Posted: 22/10/2011 Filed under: 11g R2, 11g R2, 11g R2, 11g R2 (11.2.0.1), Clusterware / Grid Infrastructure, Linux, New Features, Operating System, Oracle, Real Application Cluster One Node, Real Application Clusters 17 CommentsIntroduction
Using earlier versions of RAC Oracle 11.2 is not possible to create more than one public subnet to serve different database clients in different sites/network, to get the routing of connections we should have help from a software/hardware from third parties.
Now it’s possible configure multiples network segment (i.e different subnet) with GI/RAC 11.2.
Oracle GI/RAC 11.2 works like a router managing connections of multiple network, routing connections to the correct network/endpoint.
During the Grid Infrastructure installation, it’s not possible configure more than one public/vip network. So, the other networks should be configured manually after installation.
Purpose
The purpose of this post is to explain how to create multipe network/listener/services on multiple public network in an 11.2 Grid Infrastructure environment.
No downtime is required to perform this task. All tasks can be done with Clusterware and RAC online.
Concept before Starting
Please Note – Limitations of Grid Infrastructure 11g R2:
SCAN feature will work only in one public network, because we can configure only one SCAN for entire Clusterware Stack and SCAN must be associated whith only one public network. So, only one public network we can use all feature of GI 11.2 the others public networks will be required use the VIP in our Oracle Clients like in version 11g R1 and 10g.
I believe that Oracle will solve this issue in later versions.
Before start configuration you must learn how things work to know to properly configure our RAC. You need to know how they work to understand how to configure your Oracle RAC.
Basic Concept:
The listener is a server-side process that listens for incoming client connection requests and manages traffic to the database. When a database instance starts, and at various times during its life, the instance contacts a listener and establishes a communication pathway to this instance.
Service registration enables the listener to determine whether a database service and its service handlers are available. A service handler is a dedicated server process or dispatcher that acts as a connection point to a database. During registration, the PMON process provides the listener with the instance name, database service names, and the type and addresses of service handlers. This information enables the listener to start a service handler when a client request arrives.
A service name is a logical representation of a service used for client connections.
When a client connects to a listener, it requests a connection to a service.Thus, the listener acts as a mediator between the client and instances and routes the connection request to the right place.
To ensure service registration works properly, the initialization parameter file should contain the following parameters:
LOCAL_LISTENERfor the local listenerREMOTE_LISTENERfor the remote listener
LOCAL_LISTENER: To have PMON register with a local listener in RAC env we must configure the LOCAL_LISTENER parameter in the initialization parameter file to locate the local listener. Multiple addresses are supported, but connect-time failover and client load balancing features are not supported.
REMOTE_LISTENER: A remote listener is a listener residing on one computer that redirects connections to a database instance on another computer. In a dedicated server RAC environment, we must enable the PMON background process to register with a remote listener. You do this by configuring the REMOTE_LISTENER parameter.
Each network must have a dedicated LISTENER. It would be necessary to configure multiple addresses (Listener) in the parameter LOCAL_LISTENER from all instances to serve NET1 and NET2. Multiple addresses are supported, but connect-time failover and client load balancing features are not supported. (first problem) 😦
If we use only parameter LOCAL_LISTENER and REMOTE_LISTENER the connections can be redirected by REMOTE_LISTENER to LOCAL_LISTENER from others network. (second problem) 😦 😦
To ensure that connections to the remote listener are only redirected to the local listener on the same network Oracle create new Parameter LISTENER_NETWORKS :-), this parameter is avaliable only in version 11g R2.
LISTENER_NETWORKSspecifies one or more sets of local & remote listeners for cross-registration. All listeners within the same network_name will cross-register.
Using parameter LISTENER_NETWORKS we can configure multipes LOCAL & REMOTE Listener in your own network. It ensure wich connections incoming by network net1 will be redirected over RAC Database only on network net1.
This tasks will be done in 2 steps:
1. Create New Network and Listener in Oracle Clusterware
2. Configure RAC to support multipe Network and Create Service
Creating New Network and Listener in Oracle Clusterware
The Clusterware must be configured and online in all nodes.
Overview of Environment
- First Public network is called North.
- Second Public network is called South.
- Third Network is called East.
Infrastructure:
Servers Hostname 1° northora01 2° northora02 3° northora03 North - eth0 - 192.168.217.0 # Public Hostnames/IP North northora01 - 192.168.217.60 northora02 - 192.168.217.45 northora03 - 192.168.217.68 # Virtual Hostnames/IP North northora01-vip - 192.168.217.52 northora02-vip - 192.168.217.53 northora02-vip - 192.168.217.69 South - eth2 - 140.120.120.0 # Public Hostnames/IP South southora01 - 140.120.120.200 southora02 - 140.120.120.201 southora03 - 140.120.120.202 # Virtual Hostnames/IP South southora01-vip - 140.120.120.100 southora02-vip - 140.120.120.101 southora03-vip - 140.120.120.102 East - eth3 - 154.120.120.0 # Public Hostnames/IP East eastora01 - 154.120.120.200 eastora02 - 154.120.120.201 eastora03 - 154.120.120.202 # Virtual Hostnames/IP East eastora01-vip - 154.120.120.100 eastora02-vip - 154.120.120.101 eastora03-vip - 154.120.120.102
Before start the New Interfaces (eth2 and eth3) must be configured with IP,Mask and Gateway.
After setting up the new interfaces you must set VIP hostnames from new networks in hosts files (e.g /etc/hosts) from all nodes and add VIP-hostnames in your DNS.
# /etc/hosts # Do not remove the following line, or various programs # that require network functionality will fail. 127.0.0.1 localhost.localdomain localhost ::1 localhost6.localdomain6 localhost6 # Public North 192.168.217.60 northora01.north.com northora01 192.168.217.45 northora02.north.com northora02 192.168.217.68 northora03.north.com northora03 # VIP North 192.168.217.52 northora01-vip.north.com northora01-vip 192.168.217.53 northora02-vip.north.com northora02-vip 192.168.217.69 northora03-vip.north.com northora03-vip # Public South 140.120.120.200 southora01.south.com southora01 140.120.120.201 southora02.south.com southora02 140.120.120.202 southora03.south.com southora03 # VIP South 140.120.120.100 southora01-vip.south.com southora01-vip 140.120.120.101 southora02-vip.south.com southora02-vip 140.120.120.102 southora03-vip.south.com southora03-vip # Public East 154.120.120.200 eastora01.east.com eastora01 154.120.120.201 eastora02.east.com eastora02 154.120.120.202 eastora03.east.com eastora03 # Vip East 154.120.120.100 eastora01-vip.east.com eastora01-vip 154.120.120.101 eastora02-vip.east.com eastora02-vip 154.120.120.102 eastora03-vip.east.com eastora03-vip
Ensure Public and VIP address is defined for the 2nd and 3nd public network
Creating the CRS resource (network/vip) using srvctl
A new network and new vip resources for the new network are created by using the ‘-k’ switch.
The-k option indicates to which network it is. The public network has automatically installed the first number.
Using Oracle Grid Infrastucture 11.2.0.1 you can’t create network resource explicitly, when we create a new vip resource the network resource will be created implicitly and a dependency between resources (vip and network) will be configured.
Retrieving information about the environment
# Actual Network # Network Info crsctl status resource -f |grep NAME=ora.net -A 3 NAME=ora.net1.network TYPE=ora.network.type STATE=ONLINE TARGET=ONLINE # VIP Info crsctl status resource -f |grep NAME=ora.northora -A 3 |grep .vip -A 3 NAME=ora.northora01.vip TYPE=ora.cluster_vip_net1.type STATE=ONLINE TARGET=ONLINE -- NAME=ora.northora02.vip TYPE=ora.cluster_vip_net1.type STATE=ONLINE TARGET=ONLINE -- NAME=ora.northora03.vip TYPE=ora.cluster_vip_net1.type STATE=ONLINE TARGET=ONLINE # Getting Nodes Information olsnodes -n -s northora01 1 Active northora02 2 Active northora03 3 Active
Adding Network/VIP Resources using Clusterware 11.2.0.1
as root user: # srvctl add vip -n -k -A //[if1[|if2...]] [-v] srvctl add vip -n northora01 -A southora01-vip/255.255.255.0/eth2 -k 2 srvctl add vip -n northora02 -A southora02-vip/255.255.255.0/eth2 -k 2 srvctl add vip -n northora03 -A southora03-vip/255.255.255.0/eth2 -k 2 srvctl start vip -i southora01-vip srvctl start vip -i southora02-vip srvctl start vip -i southora03-vip # Checking Status Network 2 crsctl status resource -f |grep NAME=ora.net2 -A 3 NAME=ora.net2.network TYPE=ora.network.type STATE=ONLINE TARGET=ONLINE crsctl status resource -f |grep NAME=ora.south -A 3 |grep .vip -A 3 NAME=ora.southora01-vip.vip TYPE=ora.cluster_vip_net2.type STATE=ONLINE TARGET=ONLINE -- NAME=ora.ora.southora02-vip.vip.vip TYPE=ora.cluster_vip_net2.type STATE=ONLINE TARGET=ONLINE -- NAME=ora.ora.southora03-vip.vip.vip TYPE=ora.cluster_vip_net2.type STATE=ONLINE TARGET=ONLINE
From 11.2.0.2+, network resource can be created explicitly:
as root user: # srvctl add network [-k ] -S //[if1[|if2...]] [-w ] [-v] srvctl add network -k 3 -S 154.120.120.0/255.255.255.0/eth3 crsctl status resource -f |grep NAME=ora.net3 -A 3 NAME=ora.net3.network TYPE=ora.network.type STATE=OFFLINE TARGET=OFFLINE Then add vip resource for the 3nd network: srvctl add vip -n northora01 -A eastora01-vip/255.255.255.0/eth3 -k 3 srvctl add vip -n northora02 -A eastora02-vip/255.255.255.0/eth3 -k 3 srvctl add vip -n northora03 -A eastora03-vip/255.255.255.0/eth3 -k 3 And starting vip resource srvctl start vip -i eastora01-vip srvctl start vip -i eastora02-vip srvctl start vip -i eastora03-vip crsctl status resource -f |grep NAME=ora.east -A 3 |grep .vip -A 3 NAME=ora.eastora01-vip.vip TYPE=ora.cluster_vip_net3.type STATE=ONLINE TARGET=ONLINE -- NAME=ora.eastora02-vip.vip TYPE=ora.cluster_vip_net3.type STATE=ONLINE TARGET=ONLINE -- NAME=ora.eastora03-vip.vip TYPE=ora.cluster_vip_net3.type STATE=ONLINE TARGET=ONLINE
Creating New Listener for each network.
You can create new listener using srvctl, but I recommend create Listener using NETCA.
As the grid user invoke “netca” from the 11.2 GRID_HOME, the select “Cluster configuration” -> “Listener configuration” -> “Add”, enter Listener name as required.
Repeat the procedure above to create the Listener on the network South.
Validating Clusterware Configuration
Checking Listeners created
srvctl config listener -a Name: LISTENER Network: 1, Owner: oracle Home: /u01/app/11.2.0/grid on node(s) northora01,northora02,northora03 End points: TCP:1521 Name: LISTENER_SOUTH Network: 2, Owner: oracle Home: /u01/app/11.2.0/grid on node(s) northora01,northora02,northora03 End points: TCP:1522 Name: LISTENER_EAST Network: 3, Owner: oracle Home: /u01/app/11.2.0/grid on node(s) northora01,northora02,northora03 End points: TCP:1523 srvctl status listener Listener LISTENER is enabled Listener LISTENER is running on node(s): northora01,northora02,northora03 Listener LISTENER_EAST is enabled Listener LISTENER_EAST is running on node(s): northora01,northora02,northora03 Listener LISTENER_SOUTH is enabled Listener LISTENER_SOUTH is running on node(s): northora01,northora02,northora03 srvctl status scan_listener SCAN Listener LISTENER_SCAN1 is enabled SCAN listener LISTENER_SCAN1 is running on node northora01 SCAN Listener LISTENER_SCAN2 is enabled SCAN listener LISTENER_SCAN2 is running on node northora02 SCAN Listener LISTENER_SCAN3 is enabled SCAN listener LISTENER_SCAN3 is running on node northora03 srvctl config scan_listener SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1521 SCAN Listener LISTENER_SCAN2 exists. Port: TCP:1521 SCAN Listener LISTENER_SCAN3 exists. Port: TCP:1521
The Configuration of network and listener on Cluster-Wide is finished.
Configuring RAC Database “db11g” to support multipe Network and Creating Service
Now we need configure Oracle RAC Database (11.2) to provide service on all networks (network 1,2 and 3).
Prepare the database instance for the new listener
To ensure that connections to the remote listener are only redirected to the local listener on the same network, LISTENER_NETWORKS parameter needs to be set in the pfile or spfile for the database instance.
Use tnsnames.ora from Oracle Home (RAC) to resolve listener name alias for LISTENER_NETWORKS. All alias must be in tnsnames.ora from all nodes of RAC Database.
# tnsnames.ora Network Configuration File: /u01/app/oracle/product/11.2.0/dbhome_1/network/admin/tnsnames.ora
# Generated by Oracle configuration tools.
LISTENER_NORTH1=
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = northora01-vip.oracle.com)(PORT = 1521))
)
LISTENER_NORTH2=
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = northora02-vip.oracle.com)(PORT = 1521))
)
LISTENER_NORTH3=
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = northora03-vip.oracle.com)(PORT = 1521))
)
LISTENER_SOUTH1 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = southora01-vip.oracle.com)(PORT = 1522))
)
LISTENER_SOUTH2 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = southora02-vip.oracle.com)(PORT = 1522))
)
LISTENER_SOUTH3 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = southora03-vip.oracle.com)(PORT = 1522))
)
REMOTE_SOUTH =
(DESCRIPTION_LIST =
(DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = southora01-vip.oracle.com)(PORT = 1522)))
(DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = southora02-vip.oracle.com)(PORT = 1522)))
(DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = southora03-vip.oracle.com)(PORT = 1522)))
)
LISTENER_EAST1 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = eastora01-vip.east.com)(PORT = 1523))
)
LISTENER_EAST2 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = eastora02-vip.east.com)(PORT = 1523))
)
LISTENER_EAST3 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = eastora03-vip.east.com)(PORT = 1523))
)
REMOTE_EAST =
(DESCRIPTION_LIST =
(DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = eastora01-vip.east.com)(PORT = 1523)))
(DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = eastora02-vip.east.com)(PORT = 1523)))
(DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = eastora03-vip.east.com)(PORT = 1523)))
)
Don’t forget configure sqlnet.ora.
# sqlnet.ora Network Configuration File: /u01/app/oracle/product/11.2.0/dbhome_1/network/admin/sqlnet.ora # Generated by Oracle configuration tools. NAMES.DIRECTORY_PATH= (TNSNAMES, EZCONNECT) ADR_BASE = /u01/app/oracle
Testing connectivity of all entries in TNSNAMES.ora using utility tnsping
This step is very important, all of listener alias name must be validated in all nodes.
e.g
tnsping LISTENER_SOUTH1 TNS Ping Utility for Linux: Version 11.2.0.2.0 - Production on 18-OCT-2011 17:31:12 Copyright (c) 1997, 2010, Oracle. All rights reserved. Used parameter files: /u01/app/oracle/product/11.2.0/dbhome_1/network/admin/sqlnet.ora Used TNSNAMES adapter to resolve the alias Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = southora1-vip.oracle.com)(PORT = 1522))) OK (0 msec) # Testing REMOTE tnsping REMOTE_EAST TNS Ping Utility for Linux: Version 11.2.0.2.0 - Production on 18-OCT-2011 17:31:55 Copyright (c) 1997, 2010, Oracle. All rights reserved. Used parameter files: /u01/app/oracle/product/11.2.0/dbhome_1/network/admin/sqlnet.ora Used TNSNAMES adapter to resolve the alias Attempting to contact (DESCRIPTION_LIST = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = eastora01-vip.east.com)(PORT = 1523))) (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = eastora02-vip.east.com)(PORT = 1523))) (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = eastora03-vip.east.com)(PORT = 1523)))) OK (0 msec)
Configuring Oracle RAC to support multiples network
Please, don’t change current parameter of LOCAL_LISTENER and REMOTE_LISTENER of your spfile/pfile.
Setting LISTENER_NETWORKS is enough to RAC to work properly.
Recommendation : Set parameter using scope=both, it’s will validade if the all Listener alias is working. If you use scope=spfile, it will not validaded and if not was configured properly the error will be raised at moment of startup of database instance.
####### Instance 1 #############
ALTER SYSTEM
SET LISTENER_NETWORKS = '((NAME=NORTH_NET)(LOCAL_LISTENER=LISTENER_NORTH1)(REMOTE_LISTENER=global-scan.north.com:1521))',
'((NAME=SOUTH_NET)(LOCAL_LISTENER=LISTENER_SOUTH1)(REMOTE_LISTENER=REMOTE_SOUTH))',
'((NAME=EAST_NET)(LOCAL_LISTENER=LISTENER_EAST1)(REMOTE_LISTENER=REMOTE_EAST))'
SCOPE=BOTH SID='db11g1';
####### Instance 2 #############
System altered.
ALTER SYSTEM
SET LISTENER_NETWORKS='((NAME=NORTH_NET)(LOCAL_LISTENER=LISTENER_NORTH2)(REMOTE_LISTENER=global-scan.north.com:1521))',
'((NAME=SOUTH_NET)(LOCAL_LISTENER=LISTENER_SOUTH2)(REMOTE_LISTENER=REMOTE_SOUTH))',
'((NAME=EAST_NET)(LOCAL_LISTENER=LISTENER_EAST2)(REMOTE_LISTENER=REMOTE_EAST))'
SCOPE=BOTH SID='db11g2';
System altered.
####### Instance 3 #############
ALTER SYSTEM
SET LISTENER_NETWORKS='((NAME=NORTH_NET)(LOCAL_LISTENER=LISTENER_NORTH3)(REMOTE_LISTENER=global-scan.north.com:1521))',
'((NAME=SOUTH_NET)(LOCAL_LISTENER=LISTENER_SOUTH3)(REMOTE_LISTENER=REMOTE_SOUTH))',
'((NAME=EAST_NET)(LOCAL_LISTENER=LISTENER_EAST3)(REMOTE_LISTENER=REMOTE_EAST))'
SCOPE=BOTH SID='db11g3';
System altered.
Checking if default database service db11g was registered proprely
#################### Network 1 ##################### lsnrctl status LISTENER .. . Listening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.217.52)(PORT=1521))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.217.60)(PORT=1521))) Services Summary... .. . Service "db11g" has 1 instance(s). Instance "db11g2", status READY, has 1 handler(s) for this service... .. . The command completed successfully # In network 1 the remote_listener is registered in Listener SCAN lsnrctl status LISTENER_SCAN3 .. . Listening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_SCAN3))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.217.62)(PORT=1521))) Services Summary... .. . Service "db11g" has 3 instance(s). Instance "db11g1", status READY, has 2 handler(s) for this service... Instance "db11g2", status READY, has 2 handler(s) for this service... Instance "db11g3", status READY, has 2 handler(s) for this service... .. . The command completed successfully #################### Network 2 ##################### lsnrctl status LISTENER_SOUTH .. . Listening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_SOUTH))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=140.120.120.101)(PORT=1522))) Services Summary... .. . Service "db11g" has 3 instance(s). Instance "db11g1", status READY, has 1 handler(s) for this service... Instance "db11g2", status READY, has 2 handler(s) for this service... Instance "db11g3", status READY, has 1 handler(s) for this service... The command completed successfully #################### Network 2 ##################### lsnrctl status LISTENER_EAST .. . Listening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_EAST))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=154.120.120.101)(PORT=1523))) Services Summary... .. . Service "db11g" has 3 instance(s). Instance "db11g1", status READY, has 1 handler(s) for this service... Instance "db11g2", status READY, has 2 handler(s) for this service... Instance "db11g3", status READY, has 1 handler(s) for this service... The command completed successfully
Creating Service “db11g_inbound”
Using “srvctl add service” with option -k it’s possible create a dependency specifying the number of network. However, this has no effect on listeners which is registered with the service, the service will be registered in all listener configured in RAC (LOCAL,REMOTE and NETWORK Listener). It rather serves only to ensure that the service is started, when the corresponding subnet, or the corresponding ora.netX.network resource is present and started.
Bad news, each service can be configure with dependency to only one network. We can not specify multiple networks numbers.
I created a workaround to fix it. (It’s not supported by Oracle)
Creating new Service DB11G_INBOUND
In example below a dependency will be created with ora.net2.network
srvctl add service -s db11g_inbound -m BASIC -e SELECT -d db11g -k 2 -r "db11g1,db11g2,db11g3"
Solution not supported by Oracle. Creating dependency of resource manually.
# Create service without specifying flag "-k" srvctl will use default network 1 srvctl add service -s db11g_inbound -m BASIC -e SELECT -d db11g -r "db11g1,db11g2,db11g3" # Getting current START and STOP Dependencies. crsctl status resource ora.db11g.db11g_inbound.svc -f |grep -E 'START_DEPENDENCIES|STOP_DEPENDENCIES' START_DEPENDENCIES= hard(ora.db11g.db,type:ora.cluster_vip_net1.type) weak(type:ora.listener.type) pullup(type:ora.cluster_vip_net1.type) pullup:always(ora.db11g.db) STOP_DEPENDENCIES=hard(intermediate:ora.db11g.db,intermediate:type:ora.cluster_vip_net1.type) # Modifying START Dependencies adding type cluster_vip_net (cluster_vip_net2 and cluster_vip_net3) crsctl modify resource ora.db11g.db11g_inbound.svc -attr\ "START_DEPENDENCIES='hard(\ ora.db11g.db,\ type:ora.cluster_vip_net1.type,\ type:ora.cluster_vip_net2.type,\ type:ora.cluster_vip_net3.type)\ weak(type:ora.listener.type) \ pullup(type:ora.cluster_vip_net1.type) \ pullup(type:ora.cluster_vip_net2.type) \ pullup(type:ora.cluster_vip_net3.type) \ pullup:always(ora.db11g.db)'" # Modifying STOP Dependencies adding type cluster_vip_net (cluster_vip_net2 and cluster_vip_net3) crsctl modify resource ora.db11g.db11g_inbound.svc -attr\ "STOP_DEPENDENCIES='hard(\ intermediate:ora.db11g.db,\ intermediate:type:ora.cluster_vip_net1.type,\ intermediate:type:ora.cluster_vip_net2.type,\ intermediate:type:ora.cluster_vip_net3.type)'" # P.S The service db11g_inbound will not start if network (VIP IP) 1,2 and 3 is not avaliable, and you cannot stop network 1,2 and 3 if sevice db11g_inbound is online. # Starting Service Created srvctl start service -d db11g -s db11g_inbound
You must understand your environment and see if is better to create a service for each network or create a service to all networks.
Remember that the service created always will be available in all networks (listeners), but in network maintenance may affect services if you manually created dependency.
Checking if Service was registered in all Listeners
############### Network 1 ######################### lsnrctl status listener_scan1 .. . Listening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_SCAN1))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.217.63)(PORT=1521))) .. . Service "db11g_inbound" has 3 instance(s). Instance "db11g1", status READY, has 1 handler(s) for this service... Instance "db11g2", status READY, has 1 handler(s) for this service... Instance "db11g3", status READY, has 1 handler(s) for this service... .. . lsnrctl status listener .. . Listening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.217.60)(PORT=1521))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.217.52)(PORT=1521))) .. . Service "db11g_inbound" has 1 instance(s). Instance "db11g2", status READY, has 2 handler(s) for this service... .. . ############### Network 2 ######################### lsnrctl status listener_south .. . Listening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_SOUTH))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=140.120.120.101)(PORT=1522))) .. . Service "db11g_inbound" has 3 instance(s). Instance "db11g1", status READY, has 1 handler(s) for this service... Instance "db11g2", status READY, has 2 handler(s) for this service... Instance "db11g3", status READY, has 1 handler(s) for this service... .. . ############### Network 3 ######################### lsnrctl status listener_east .. . Listening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_EAST))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=154.120.120.101)(PORT=1523))) .. . Service "db11g_inbound" has 3 instance(s). Instance "db11g1", status READY, has 1 handler(s) for this service... Instance "db11g2", status READY, has 2 handler(s) for this service... Instance "db11g3", status READY, has 1 handler(s) for this service... .. .
Use the connect string above to connect your clients on RAC Database 11.2
########### Connection String to Network 1 #####################
### Using SCAN ###########
db11g_inbound =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = global-scan.north.com)(PORT = 1521))
(LOAD_BALANCE = YES)
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = db11g_inbound)
(FAILOVER_MODE =
(TYPE = SELECT)(METHOD = BASIC)(RETRIES = 180)(DELAY = 5)
)
)
### Or Using VIP ###########
db11g_inbound =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = northora01-vip.north.com)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = northora02-vip.north.com)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = northora03-vip.north.com)(PORT = 1521))
)
(LOAD_BALANCE = YES)
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = db11g_inbound)
(FAILOVER_MODE =
(TYPE = SELECT)(METHOD = BASIC)(RETRIES = 180)(DELAY = 5)
)
)
########### Connection String to Network 2 #####################
db11g_inbound =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = southora01-vip.north.com)(PORT = 1522))
(ADDRESS = (PROTOCOL = TCP)(HOST = southora02-vip.north.com)(PORT = 1522))
(ADDRESS = (PROTOCOL = TCP)(HOST = southora03-vip.north.com)(PORT = 1522))
)
(LOAD_BALANCE = YES)
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = db11g_inbound)
(FAILOVER_MODE =
(TYPE = SELECT)(METHOD = BASIC)(RETRIES = 180)(DELAY = 5)
)
)
########### Connection String to Network 3 #####################
db11g_inbound =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = eastora01-vip.north.com)(PORT = 1523))
(ADDRESS = (PROTOCOL = TCP)(HOST = eastora02-vip.north.com)(PORT = 1523))
(ADDRESS = (PROTOCOL = TCP)(HOST = eastora03-vip.north.com)(PORT = 1523))
)
(LOAD_BALANCE = YES)
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = db11g_inbound)
(FAILOVER_MODE =
(TYPE = SELECT)(METHOD = BASIC)(RETRIES = 180)(DELAY = 5)
)
)
Useful docs about how configure multiple network an RAC env.
Data Guard: Redo Transport Services – How to use a separate network in a RAC environment. [ID 1210153.1] (11.1 or above)
How to Configure A Second Listener on a Separate Network in 11.2 Grid Infrastructure [ID 1063571.1] (11.2)
Data Guard Physical Standby 11.2 RAC Primary to RAC Standby using a second network [ID 1349977.1] (11.2)
Hope this helps.
Enjoy
Levi Pereira | Oracle Blog 
[…] https://levipereira.wordpress.com/2011/10/22/how-configure-multiples-public-network-an-grid-infrastru… […]
LikeLike
thank you much for this post .much Appreciated your efforts for sharing the knowledge ..ur all posts are really very helpful
Best Regards
Shivanand Hindwan
LikeLike
[…] http://www.oracle-dba-online.com/sql/oracle_table_partition.htm https://levipereira.wordpress.com/2011/10/22/how-configure-multiples-public-network-an-grid-infrastru… Share this:TwitterFacebookLike this:LikeBe the first to like this […]
LikeLike
[…] work in one network the others network will provide services using only VIP hostnames. See it: How configure Multiples Public Network in a Grid Infrastructure 11g R2 (11.2) environment Regards, Reply With […]
LikeLike
Hi Levi,
I’m a little confused about the Oracle statement “Listeners specified by the LISTENER_NETWORKS parameter should not be used in the LOCAL_LISTENER and REMOTE_LISTENER”. But they advise to include the default scan listener both in the init parameter REMOTE_LISTENER and in the REMOTE_LISTENER clause of the LISTENER_NETWORKS.
Isn’t there some contradiction or did I miss something ?
LikeLike
Hey Jean,
This setup is exclusively for case that we use multiple network.
LISTENER_NETWORKS act as route (ensuring that each connection will go to the correct destination), So when we are using different networks we must use this parameter to allow cross-registration.
There is a restriction when we use LISTENER_NETWORKS. We cannot use SCAN feature over all network configured.
SCAN feature (REMOTE_LISTENER and LOCAL_LISTENER) will work only in one public network, because scan is configured over one public network, as we can configure only one SCAN for entire Clusterware Stack and SCAN must be associated whith only one public network. So, only one public network we can use all feature of GI 11.2 the others public networks will be required use the VIP in our Oracle Clients like in version 11g R1 and 10g.
About this question:
But they advise to include the default scan listener both in the init parameter REMOTE_LISTENER and in the REMOTE_LISTENER clause of the LISTENER_NETWORKS?
Who advised this? Where did you get this information?
Regards,
Levi Pereira
LikeLike
Hey Levi,
That was a very nice article.
I’m facing a similar problem with a bit different scenario: I have two databases that run on the
GI stack: North and South, both 3 instances. For security reasons, it has been decided that we need to separate the two databases to two different subnets.
Your solution is excellent and gave me some ideas, but it’s referring to a single database. I was wondering how can we modify it so it will solve our scenario as well?
Keep up the good work!
Zohar
LikeLike
Hi Zohar,
This post is useful when you need configure multiple network in a Database.
Far as I can understand you are configuring two network for two database, so each database in onw network.
If it’s true, you don’t need to use parameter LISTENER_NETWORKS, because you are not using cross-registration between different network. You can use REMOTE and LOCAL Listener parameter pointing each to own Listener/Network.
If I understood wrong, please correct me.
e.g
On Clusterware/RAC
* net1 VIP/SCAN/Listener 1521
* net2 VIP/Listener 1522
* db1 (3 instances) (local_listner= listener(net1):1521/ remote_listener = scan_listener(net1):1521)
* db2 (3 instances) (local_listener = listener(net2):1521/ remote_listener = tnsnames(local_listener(net2) from virtual IP node1,node2,node3))
On Single Instance
* net1 Listener 1521
* net2 Listener 1522
* db1 (local_listener = listener(net1):1521
* db2 (local_listener = listener(net2):1522
P.S You don’t need set remote_listener on single database, because does not existe remote instance on single.
LikeLike
Fantastic!!! Thanks a lot
LikeLike
Great article! But i have another scenario:
I have two nodes in a GI with two distinct database (ora1 and ora2 with each one shared home)
I need to configure two scan listener on the same network (one for ora1 and 1 for ora2).
It’s possible to do this ?
Thanks
Alessandro
LikeLike
No, Is not supported to have two SCAN on same cluster.
So, one database will support SCAN/VIP and the other database will support old fashion i.e using only VIP.
LikeLike
the problem is only when you are using oracle rac- ie using scan listener and listener cross registartion correct?
If I am on a single instance database (with no SCAN listener) and 2 local listeners, one for regular client connections and 2nd for data guard traffic I wouldnt have to specify”listener_networks”…is that a correct understanding?
LikeLike
[…] https://levipereira.wordpress.com/2011/10/22/how-configure-multiples-public-network-an-grid-infrastru… […]
LikeLike
Hello, I read carefully this article and I don’t understand what we can do with “south public”, “east public” addresses. Are these addresses usefull for something ?
Best regards.
LikeLike
Hi, this is excellent article. I have implemented 2nd network. Now, we want to reverse the change. Say, I want to remove the newly added 2nd network. Can you shed somelight on it?
I read the Oracle CRS manual in which I did not find srvctl add network or remove network options. Can share where do you find those ?
Thank you very much.
Xiaoqiang
LikeLike
Hi Xiaoqiang,
I’m sorry for the delayed response.
You are right it’s not documented “srvctl remove network”, but the command exists.
$ crsctl query crs activeversion
Oracle Clusterware active version on the cluster is [11.2.0.3.0]
$ srvctl remove network -h
Removes a network configuration from the Oracle Clusterware.
Usage: srvctl remove network {-k | -a} [-f] [-v]
-k network number (default number is 1)
-a All networks
-f Force remove
-h Print usage
-v Verbose output
I’ll contact Oracle Support to fix this issue.
Hope this help.
LikeLike
[…] listen on the new network interface also. RAC needs special attention and it is described in this blog post. Basically you need to configure new set of VIP addresses, create a new network resource in […]
LikeLike