RAC and HA Best Practices

What are the challenges of RAC with Dbvisit Replicate?

• The database instances can go up and down
• Instances can be added and dropped
• The machine running Replicate can go down
• Replicate can connect to database that goes down

The concepts are similar for any clusterware involved, examples are provided for Oracle Clusterware 11.2

Best Practices

Location of Dbvisit Replicate
To provide HA for Dbvsit Replicate, run it on a machine configured in a cluster. The simplest way is to run it on the cluster running the database itself.

Shared files

• For MINE, MINE_PLOG directory must be on shared filesystem. If FETCHER is used, then the MINE_STAGING_DIR must be shared as well.
• For APPLY, the APPLY_STAGING_DIR must be on shared filesystem.
• The dbvrep executable and the DDC locations must be identical on all nodes. If the DDC files are not on
• shared storage, they must be copied to each node.
• This requirement may be made obsolete in future versions.
• The archive logs must be on shared storage (cluster file system or ASM).

Database connectivity

Dbvisit Replicate keeps a connection to the source (for MINE and FETCHER) or target (APPLY) database.Console keeps connection to both of them, but only if it is used to change the configuration.

This ensures that when Replicate is running on a node, it is connecting to one instance only.This limits its dependencies to current node only. The database load imposed by MINE is very low on source, so no load balancing is needed. APPLY currently executes only one SQL at a time, so the load is also not high – but this will change in the future.

This is optional and it just limits the impact of a node going down on the console.

Application connectivity

For each cluster, set up an Application VIP mamaged by the clusterware. Use this address as the name of node when the Replicate process run. This IP will be always assigned to just one node and the Replicate process will follow it in case of node failure.

To create an application VIP, use the following syntax:

appvipcfg create -network=network_number -ip=ip_address -vipname=vip_name
-user=user_name [-group=group_name] [-failback=0 | 1]

Where network_number is the number of the network, ip_address is the IP address, vip_name is the nameof the VIP (we use dbvrepVIP further on),

user_name is root (as root privileges are needed to reconfigure network interfaces). 

The default value of the -failback option is 0. If you set the option to 1, then the VIP (and therefore Replicate as well) fails back to the original node when it becomes available again.

As root, allow the Oracle Grid infrastructure software owner (e.g. oracle) to run the script to start the VIP.

crsctl setperm resource dbvrepVIP -u user:oracle:r-x

To check the status, run:

crsctl status res dbvrepVIP -p

As oracle user, start the VIP.

crsctl start resource dbvrepVIP

Configure and test

Configure the Dbvisit Replication as usual, using the previously connected TNS identifiers for source and target databases and the Applicatio VIPs for location where processes will run (*_LISTEN_INTERFACE settings). 

Start the replication (use the nodes where the Application VIPs currently reside) and test that your configuration works.

Create action scripts
Create scripts that will manage (start/stop/check/clean/abort) the dbvrep processes. Use the script provided and:

• modify the settings at the top of the script (DDC name, process name, paths)
• save the script to a directory and make it executable by oracle user; do this on all cluster nodes

An example name of such script would be:

/home/oracle/REP/action_script_MINE.scr

Create resource for Replicate

As oracle, create the cluster resource for the dbvrep process:

crsctl add resource resource_name -type cluster_resource -attr
"ACTION_SCRIPT=resource_action_script.scr, CHECK_INTERVAL=30,
START_DEPENDENCIES='hard(dbvrepVIP,dbresource,ASMresource) pullup(dbvrepVIP)',
STOP_DEPENDENCIES='hard(dbvrepVIP)'"

Use any name for resource_name, e.g. dbvrepMINE_DDCname like dbvrepMINE_ERP. For action script, use the path to the script created earlier. For start dependencies, use resource name for your database (like ora.orcl.db) and if ASM is used, use ora.asm for ASMresource.

The resource was created with “restore” (the default) AUTO_START setting and thus clusterware will try to start it automatically if it was running last time the cluster was shut down. The resource was created with default placement policy and thus any node will be eligible to host the resource. Use placement=favored/restricted and hosting_members/server_pools to change this. How does this interact with the VIP?

Manage the processes
Stop the replication if was started manually.

To manage the replication processes, use:

crsctl start resource resource_name
crsctl stop resource resource_name

crsctl relocate resource resource_name -n nodename -f -- -f

Will relocate the VIP as well.

Unsupported operations

Dbvisit Replicate does not support online addition and dropping of cluster instances, or to be more precise, redo threads. For such operation, Dbvisit Replicate MINE process on such database must be stopped.

Known issues

Running appvipcfg fails with error:

CRS-0160: The attribute '"USR_ORA_VIP' is not supported in this resource type.

Reason: Oracle bug 9964188.

Workaround: 

edit appvipcfg.pl and remote \” in front of USR_ORA_VIP.

See also : patch shown in the bug description at My Oracle Support.