Oracle RAC Configurations

1.  Introduction

In this section, we will cover a few of the advanced topics with regard to Dbvisit Standby and Oracle RAC configurations.

In this section, the concepts will be explained using an example.

1.1.  Different types of Oracle RAC configurations

Dbvisit Standby version 10 handles Oracle RAC configurations similar to Dbvisit Standby versions 8 and 9. It is important to take note of this and not attempt a version 7.0.x to 10.0.x upgrade without knowing this as you will not be able to perform a direct upgrade.  

If you are moving from version 7 to version 10, you MUST create a new DDC file.  You will not be allowed to upgrade the current version 7 DDC file if Oracle RAC is being detected.

Dbvisit Standby version 10 is designed to be more Oracle RAC aware.  This include

  1. Run Dbvisit Standby on only one of the Oracle RAC nodes
  2. To allow for this use a new Virtual IP - to be created and assigned to run on the preferred node of your choosing.  This allows Dbvisit Standby to treat the cluster as "one" node.  Example, if the VIP is 10.0.2.89 and the name attached to this is kiwi812-vip.  We can anywhere on the network (as long as kiwi812-vip for 10.0.2.89 is in the DNS) ping or attempt a connection to dbv-vip and it will always run on one of the RAC nodes, it does not make a difference which one.  The VIP will run on the node where the Dbvisit Standby components are running.  That way you do not have to configure Dbvisit Standby on all the RAC nodes, but only on one.  Which is the much more effective and better use of resources?
  3. If this preferred node fails for some reason, switch the kiwi812-vip (Virtual IP) and the Dbvisit components (resources) to the second node (using action scripts and adding these components as cluster resources).
  4. Now to achieve this configuration, as you might have guessed, Shared storage is used.  This allows the software to be installed once, configured to use the Virtual IP (example kiwi812-vip) and then you can switch between the nodes (after resources have been configured and added to Clusterware).
  5. Shared Storage can be ACFS, NFS or possibly any other cluster filesystem.  This filesystem will store the Dbvisit Software as well as copies of the Archive Logs during certain phases of Dbvisit Standby use, depending on your configuration.
  6. It is recommended that the Dbvisit ARCHSOURCE and ARCHDEST be located on shared storage.  (Remember when a Graceful Switchover - role reversal is performed, the ARCHSOURCE and ARCHDEST parameter values are swapped.  So make sure that ARCHSOURCE is located on the shared storage.  This is where archive logs will be stored when the RAC cluster becomes a standby - and if on shared storage, both nodes can perform recovery (whichever one would be running the Dbvisit Standby resources at the time).
  7. Note: this does not mean you cannot use local storage.  The downside is if you use local storage and not shared storage, you can only run of one node unless you implement something like "rsync" to synchronize the local folder for the node 1 (maybe node 1 is your preferred node) to the secondary node 2.  That way you have the resources for Dbvisit Standby running on node 1 and all files are synced with node 2 (using "rsync" and a "CRON" schedule).  This is not a perfect solution but is an option if you really cannot implement the use of shared storage.



2.  Example 1:  Oracle RAC Primary to Single Instance Standby


Two Options are available, using shared storage on the Oracle RAC cluster for the Dbvisit Standby installation, or using local storage and run of only one node and optionally configure "rsync" to have the Dbvisit Standby folder synced between the primary nodes.

The two diagrams below explain the configuration in more detail:





2.2.  Option 2 - Using Local Storage


The diagram below provides an overview of the Architecture:




To explain the configuration further we will use an example.  Below is a diagram that provides high-level details for the environment:


The key parts to note:

  1.  The primary RAC Cluster consists of two nodes (kiwi81 and kiwi82)
  2.  The first node (kiwi81) will be used as the node to run Dbvisit Standby on local storage
  3.  The "Dbvisit Base" folder "/usr/dbvisit" will be synced between the primary node kiwi81 and kiwi82 using the "rsync" utility and a CRON schedule.
  4.  The standby server is a single instance system using Filesystem Bases storage:
    1. /u01/app/oracle/oradata1  --> this location will hold the database files (OMF - Oracle Managed Files will be used), this location will also hold a copy of the redo logs (OMF)
    2. /u01/app/oracle/oradata2   → This location will hold the second copy of the redo logs 
    3. /u01/app/oracle/fast_recovery_area . → This location will be used as the recovery area on the standby server


The Dbvisit Standby locations are:

  • DBVISIT_BASE = /usr/dbvisit
  • ARCHSOURCE = /u01/app/oracle/dbvisit_arch/DEV
  • ARCHDEST = /u01/app/oracle/dbvisit_arch/DEV


Now that we have an overview of the configuration, we can start with the installation. 

We will install Dbvisit Standby software on node 1 (kiwi81) and on the standby server (kiwi91).

The installation of the software on kiwi1 will be synced later (see 2.2.4 below).



2.2.1.  Installing the Software


Below is an example of installing the Dbvisit Standby software onto a local folder - the default which is /usr/dbvisit

Remember in this example, the VIP on the RAC Cluster is running on kiwi1 and is called "dbv-vip"

We first install the software on node 1 (kiwi81) and then on the standby node (kiwi91).

2.2.1.1.  Installing Dbvisit Standby on the RAC Node 1 (kiwi81)


The next step is to start the Dbvisit Components on kiwi1 - dbvnet and dbvagent:


2.2.1.2.  Installing Dbvisit Standby on Standby Server (kiwi91)

Below is the output of the installation on the standby server.  

IMPORTANT: In this example, the Central Console is installed on the Standby server.  Ideally, it should be installed on its own server, but if you do not have a 3rd system available for the central console, you can install it on either the primary or standby - we recommend the standby.


Following the installation Start the Dbvisit Standby components: dbvnet, dbvagent, dbvserver:




2.2.2.  Creating a DDC File

There are two options now available to you for creating the DDC file, using the Command Line Interface (CLI) or the Central Console (GUI)

2.2.2.1. Using the Command Line Interface

The DDC file creation must be run from the Primary server (kiwi1) in this example.

The command used is:  ./dbvctl -o setup --noprompt


From the above example, we can see that we now have a DDC file called "dbv_MYDEV.env" and a repository for this DDC file called "mydev.db".



2.2.2.2.  Using the Central Console

Below are the screenshots showing you how you can use the Central Console to create the DDC file for this configuration:

Step 1:  Open Main Screen, and navigate to manage hosts and first add the two hosts (dbv-vip and kiwi221)



Step 2:   Add a new host

Step 3: Add the RAC node 1 (kiwi18) - but not the host details - you must use the RAC VIP that we assigned - this allows us to move to kiwi2 without having to update names as we use the Virtual IP (kiwi812-vip) which moves between them.


Step 4: Review host and click add new to add standby host - kiwi91


Step 5:  Add kiwi91 host


Step 6:  Review hosts


Step 7: Add new DDC → navigate to Manage Configurations


Step 8:  Start new DDC process by selecting "NEW"


Step 9:  Fill in the required details:


Once the details are provided, click on Submit and the DDC file will be created.


Step 10:  Review the DDC file


Step 11:  Make sure you apply your License key for this newly created DDC before you move onto the Create Standby Database (CSD), this can be done while creating the DDC itself.



2.2.3.  Creating the Standby Database (CSD)

This example will only show using the Central Console.

It is possible to create the standby database either via the Command Line Interface (CLI) using the command:  ./dbvctl -d <DDC> --csd   in this example:  "./dbvctl -d DEV --csd"

To use the Central Console, navigate to the Central Console page and follow the steps as outlined below:


Step 1:  Navigate to the Create Standby Database (CSD) Menu


Step 2.1:  Select the DDC, Create New Database option and update the Database SPFILE parameters to match the Standby server configuration:

In this example, the standby is Filesystem Based Storage

As we are using OMF - Oracle Managed Files on ASM (Which is recommended) we have to adjust the OMF parameters according - see 3 and 4 as marked below:


Step 2.2: Review Temporary Backup Locations - Make sure that this location has sufficient space for a full RMAN compressed backup of the database.  In this example /usr/tmp has sufficient space on both the primary and standby.



2.2.4.  Initial Send/Apply Logs

This section will show you how you can send/apply logs following the CSD process.

This can be done either via the Command Line Interface or the Central Console.

2.2.4.1.  Using the Command Line Interface (Send/Apply)


From the primary node running the Dbvisit Standby processes (kiwi81)


The same command:  "./dbvctl -d DEV" can be executed on the standby server kiwi221 to apply the logs already shipped.

Running a log gap report is done from the primary node (kiwi81).  Here is an example:

2.2.4.2.  Using the Central Console (Send/Apply)

Step 1:  Navigate to Manage Databases


Step 2: Select the DDC, click on [2] Send Logs icon and an Active Task will appear [3] which will turn to a green tick when done.  View the details by clicking on the task [3] and you will see the details [4]


Step 3:  Select the DDC, click on the Apply Logs icon [2].  The active task will be created [3] and when selected the details will be displayed [4]




2.2.5.  Use "rsync" to sync local storage on RAC nodes

The next step in this example is to use the UNIX "rsync" utility to copy the files located in the /usr/dbvisit directory (The DBVISIT_BASE) from the "preferred node" which is the node where Dbvisit Standby normally is set to run, which in this example is "kiwi81".  It will be set to copy the files from this location to the secondary node kiwi82 in this example.  From kiwi82 there will be no Dbvisit Standby processes running as the Virtual IP (dbv-vip) will be running on kiwi1.  

You will only start the Dbvisit Processes on kiwi2 if the Virtual IP cluster resource (kiwi812-vip) has been switched over - (re-allocated or failed over) to run on kiwi82.  For more detail on the configuration of the VIP and the Action Scripts for Dbvnet and Dbvagent please see here - Oracle RAC Configurations#OracleRACConfigurations-5.Example2:AddDbvnetandDbvagentasClusterResources


Note:  The /usr/dbvisit directory (The Dbvisit Base) location must exist on both Primary RAC nodes and that the permission of this folder is set to have the "oracle" Unix account (the user that runs your database software) as the owner.  At this stage, the folder on the secondary server (kiwi82) will be empty as we have not synced the files yet.


Step 1:  Create a "rsync" script 

The first step now is to create a script that will be used to copy the files from node 1 (kiwi1 in this example) to node 2 (kiwi2).


Sample "rsync-dbvisit" script

Note - this script can be copied into a folder /home/oracle/bin and enabled via a UNIX CRON schedule to run on a regular basis.


Once you have created this script, make sure that it has sufficient permissions and that the owner "oracle" user, in this case, have a execute permission (example  "chmod +x oracle rsync-dbvisit" ).  For more detail please see the "chmod" command man pages or online help for your UNIX distribution.

The above script is making use of the UNIX "rsync" command.  For more details on the use of this command and all parameters please see the UNIX man pages or online documentation.  

Once you have created the script you can run it manually the first time to ensure the directories are kept in sync.  

oracle@kiwi81[/home/oracle]: cd bin
oracle@kiwi81[/home/oracle/bin]: ./rsync-dbvisit


Then review the log file in /tmp/dbvisit-rsync.log as well as the /usr/dbvisit directory on the secondary server (kiwi2 in this example)


Step 2:  Schedule the Script

In this example, we will schedule the script to run every 10 minutes.  This should be sufficient for most configurations. 

The schedule can be adjusted if required based on your configuration and environment.


##
# CRON: Synchronize the Dbvisit Base folder between kiwi81 and kiwi82
##
*/10 * * * * cd /home/oracle/bin; ./rsync-dbvisit >>/dev/null 2>&1


Step 3:  Monitor

Have Dbvisit Standby run either via a schedule or via the Daemon process for a period and then monitor the /usr/dbvisit folder on the standby server.

You should see files being updated in the secondary server following a run of the above schedule.


2.2.6.  Next Step - Scheduling

The next step is to schedule the Dbvisit Standby or to run the Daemon process.

Both methods can be used.  

Example if you want to run the Schedule using the Unix Cron, you can use the following example CRON entries:


On the Primary (kiwi1)

##
# CRON: Synchronize the Dbvisit Base folder between kiwi81 and kiwi82
##
*/10 * * * * cd /home/oracle/bin; ./rsync-dbvisit >>/dev/null 2>&1
##
##
# Dbvisit Standby Scheduled to run every 5 min
*/5 * * * * cd /usr/dbvisit/standby; ./dbvctl -d MYDEV >>/dev/null 2>&1


On the Standby (kiwi91) - apply logs every 15min, this can provide a buffer (delay in applying logs) to possibly stop Human error (deletion of data etc).  

##
# Dbvisit Standby Scheduled to run every 15 min
*/15 * * * * cd /usr/dbvisit/standby; ./dbvctl -d MYDEV >>/dev/null 2>&1


For more detail on scheduling or running Dbvisit Standby as a background Daemon/Process please see here - Dbvisit Standby Scheduling



2.2.7.  Example Graceful Switchover in this Configuration

This example is just to show Graceful Switchover that was performed on this environment.

Before the Graceful Switchover (GS) a log gap report was run to make sure that the primary and standby is up to date.


Following this process, kiwi91 is now a single instance primary with kiwi81/kiwi82 as the standby database.





3.  Example 2: Oracle RAC primary to Oracle RAC standby

In the example we will discuss in this section we will explain the configuration of a Standby Oracle RAC environment for a Primary Oracle RAC environment.

The Diagram below provide a summary of the environment:

3.1.  The Primary Oracle RAC Configuration - Overview

In the example the primary RAC configuration exists out of two Nodes:

  • Primary node 1: kiwi81
  • Primary node 2: kiwi82

The IP address details are listed below:

## Public IP addresses
10.0.2.81	kiwi81.oraclekiwi.co.nz kiwi81
10.0.2.82	kiwi82.oraclekiwi.co.nz kiwi82

## Virtual IP Addresses
10.0.2.83	kiwi81-vip.oraclekiwi.co.nz kiwi81-vip
10.0.2.84	kiwi82-vip.oraclekiwi.co.nz kiwi82-vip

## Private Interconnect Addresses
10.5.5.81	kiwi81-priv.oraclekiwi.co.nz kiwi81-priv
10.5.5.82	kiwi82-priv.oraclekiwi.co.nz kiwi82-priv

## SCAN address list in DNS Only
# 10.0.2.85       kiwi812-scan.oraclekiwi.co.nz kiwi812-scan
# 10.0.2.86       kiwi812-scan.oraclekiwi.co.nz kiwi812-scan
# 10.0.2.87       kiwi812-scan.oraclekiwi.co.nz kiwi812-scan

# Dbvisit Standby VIP 
10.0.2.89	kiwi812-vip.oraclekiwi.co.nz kiwi812-vip

These details are added to the /etc/hosts file as well as to the DNS.

Having the values in the DNS is highly recommended.  

The Oracle SCAN details must be in the DNS.  

3.1.1.  The new Virtual IP

Take note of the new Virtual IP that is specified.  This new Virtual IP is created and will be assigned to one specific node at a time.  But it can move between the RAC nodes.  Dbvisit Standby version 10 will be configured to use this address as the "cluster address".  That way we can decide on which node we want Dbvisit Standby components to run.

  • Get new IP address on the public network (network used by Primary/Standby)
  • Create new Cluster Resource – Virtual IP (as root user)
  • Review Oracle Documentation on creating a Virtual IP!

The steps to create the Virtual IP address is listed below (Note these are executed as the root user)

# appvipcfg create -network=1 -ip=10.0.2.89 -vipname=kiwi812-vip -user=root
# crsctl setperm resource kiwi812-vip -u user:oracle:r-x
# crsctl setperm resource kiwi812-vip -u group:grid:r-x
# crsctl start resource kiwi812-vip -n kiwi81

# crsctl status resource kiwi812-vip -p
# crsctl relocate resource kiwi812-vip -n kiwi81

The last step is to make sure you add to the DNS:  10.0.2.89 kiwi812-vip

3.1.2.  ASM Disk Groups

  • +DATA 
    • Datafiles, Redo Logs, Controlfile
  • +FRA 
    • Recovery Area, Redo Logs, Control file, Archivelogs
  • +ACFS 
    • Used for /acfs a shared filesystem

3.1.3.  Dbvisit Standby Installation

  • DBVISIT_BASE : /acfs/dbvisit
  • SOURCE: kiwi812-vip
  • DESTINATION: kiwi912-vip
  • ARCHSOURCE location : /acfs/dbvisit_ach/DEV
  • ARCHDEST Location : /acfs/dbvisit_ach/DEV

3.1.4.  Node 1 and 2 storage overview

Primary RAC Node 1: kiwi81Primary RAC Node 2: kiwi82
oracle@kiwi81[/home/oracle]: df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_oel6-lv_root
                       12G  8.2G  2.7G  76% /
tmpfs                 1.4G  117M  1.3G   9% /dev/shm
/dev/sda1             976M  126M  784M  14% /boot
/dev/mapper/vg_oel6-lv_u01
                       23G   13G  9.3G  57% /u01
tmpfs                 2.0G  4.3M  2.0G   1% /tmp
/dev/asm/acfsvol-260  9.0G  423M  8.6G   5% /acfs
oracle@kiwi82[/home/oracle]: df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_oel6-lv_root
                       12G  7.6G  3.4G  70% /
tmpfs                 1.4G  118M  1.3G   9% /dev/shm
/dev/sda1             976M  126M  784M  14% /boot
/dev/mapper/vg_oel6-lv_u01
                       23G   14G  8.1G  63% /u01
tmpfs                 2.0G  6.8M  2.0G   1% /tmp
/dev/asm/acfsvol-260  9.0G  423M  8.6G   5% /acfs


3.2.  The Standby Oracle RAC Configuration - Overview

In the example the standby RAC configuration exists out of two Nodes:

  • Standby node 1: kiwi91
  • Standby node 2: kiwi92

The IP address details are listed below:

## Public IP addresses
10.0.2.91	kiwi91.oraclekiwi.co.nz kiwi91
10.0.2.92	kiwi92.oraclekiwi.co.nz kiwi92

## Virtual IP Addresses
10.0.2.93	kiwi91-vip.oraclekiwi.co.nz kiwi91-vip
10.0.2.94	kiwi92-vip.oraclekiwi.co.nz kiwi92-vip

## Private Interconnect Addresses
10.5.5.91	kiwi91-priv.oraclekiwi.co.nz kiwi91-priv
10.5.5.92	kiwi92-priv.oraclekiwi.co.nz kiwi92-priv

## SCAN address list in DNS Only
# 10.0.2.95       kiwi912-scan.oraclekiwi.co.nz kiwi912-scan
# 10.0.2.96       kiwi912-scan.oraclekiwi.co.nz kiwi912-scan
# 10.0.2.97       kiwi912-scan.oraclekiwi.co.nz kiwi912-scan

# Dbvisit Standby VIP 
10.0.2.99	kiwi912-vip.oraclekiwi.co.nz kiwi912-vip

  • These details are added to the /etc/hosts file as well as to the DNS.
  • Having the values in the DNS is highly recommended.
  • The Oracle SCAN details must be in the DNS.

3.2.1.  The new Virtual IP

Take note of the new Virtual IP that is specified. This new Virtual IP is created and will be assigned to one specific node at a time. But it can move between the RAC nodes. Dbvisit Standby version 10 will be configured to use this address as the "cluster address". That way we can decide on which node we want Dbvisit Standby components to run.

  • Get new IP address on the public network (network used by Primary/Standby)
  • Create new Cluster Resource – Virtual IP (as root user)
  • Review Oracle Documentation on creating a Virtual IP!

The steps to create the Virtual IP address is listed below (Note these are executed as the root user)


# appvipcfg create -network=1 -ip=10.0.2.99 -vipname=kiwi912-vip -user=root
# crsctl setperm resource kiwi912-vip -u user:oracle:r-x
# crsctl setperm resource kiwi912-vip -u user:grid:r-x
# crsctl start resource kiwi912-vip -n kiwi91

# crsctl status resource kiwi912-vip -p
# crsctl relocate kiwi912-vip
# crsctl relocate resource kiwi912-vip -n kiwi91


The last step is to make sure you add to the DNS: 10.0.2.99 kiwi912-vip

3.2.2.  ASM Disk Groups

  • +DATA
    • Datafiles, Redo Logs, Controlfile
  • +FRA
    • Recovery Area, Redo Logs, Control file, Archivelogs
  • +ACFS
    • Used for /acfs a shared filesystem

3.2.3.  Dbvisit Standby Installation

  • DBVISIT_BASE : /acfs/dbvisit
  • SOURCE: kiwi812-vip
  • DESTINATION: kiwi912-vip
  • ARCHSOURCE location : /acfs/dbvisit_ach/DEV
  • ARCHDEST Location : /acfs/dbvisit_ach/DEV

3.2.4.  Node 1 and 2  storage overview


Standby RAC Node 1: kiwi91Standby RAC Node 1: kiwi92
oracle@kiwi91[/home/oracle]: df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_oel6-lv_root
                       12G  8.4G  2.6G  77% /
tmpfs                 1.4G  133M  1.3G  10% /dev/shm
/dev/sda1             976M  126M  784M  14% /boot
/dev/mapper/vg_oel6-lv_u01
                       23G   14G  7.7G  65% /u01
/dev/asm/acfsvol-229  9.0G  390M  8.7G   5% /acfs
oracle@kiwi92[/home/oracle]: df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_oel6-lv_root
                       12G   10G  889M  93% /
tmpfs                 1.4G  136M  1.3G  10% /dev/shm
/dev/sda1             976M  126M  784M  14% /boot
/dev/mapper/vg_oel6-lv_u01
                       23G   13G  9.2G  58% /u01
/dev/asm/acfsvol-229  9.0G  390M  8.7G   5% /acfs


4.  Example 2:  The Core Component Installation

4.1.  Primary Cluster - Node 1 (kiwi81)

  • The primary cluster Virtual IP is 10.0.2.89 and is using the hostname alias kiwi812-vip.  We will use this as the SOURCE address name for the Primary Cluster.
  • At the time of installing the software, the Virtual IP was configured to run on node 1 - kiwi81.
  • The steps below explain the installation of the Dbvisit Standby Core components - Dbvnet, Dbvagent and Dbvisit Standby CLI
  • Dbvisit Standby components are installed onto the shared storage which is mounted under /acfs directory
  • The directory /acfs/dbvisit and /acfs/dbvisit_arch is created and the "oracle" Unix account is made the owner
  • In this example, a basic passphrase of "kiwi123" is used for Dbvnet and the Dbvagent.

4.2.  Standby Cluster - Node 1 (kiwi91)

  • The standby cluster Virtual IP is 10.0.2.99 and is using the hostname alias kiwi912-vip.  We will use this as the SOURCE address name for the Primary Cluster.
  • At the time of installing the software, the Virtual IP was configured to run on node 1 - kiwi91.
  • The steps below explain the installation of the Dbvisit Standby Core components - Dbvnet, Dbvagent and Dbvisit Standby CLI
  • Dbvisit Standby components are installed onto the shared storage which is mounted under /acfs directory
  • The directory /acfs/dbvisit and /acfs/dbvisit_arch is created and the "oracle" Unix account is made the owner
  • In this example, a basic passphrase of "kiwi123" is used for Dbvnet and the Dbvagent


5. Example 2:  Add Dbvnet and Dbvagent as Cluster Resources

In this section, we will show you how to add Dbvnet and Dbvagent as cluster resources.

We will make use of an action script for each component, which will then be used to manage these two resources.

Note: these scripts are provided as examples and should be tested in your development or test environments before implementing it into production as each environment is different and they might need to be adjusted for your environment.


5.1.  Dbvnet Action Script

The script below is a sample action script that will be used to manage dbvnet.  

This script is created in the dbvnet folder /acfs/dbvisit/dbvnet

#!/bin/bash
#
# Dbvnet Action Script 

##############################################
#  Function to change database environments
# Description:
# Use oraenv to set the environment if needed
# This is optional to set the environment
##############################################

## set following to ensure oraenv is picked up from /usr/local/bin
export PATH=/usr/local/bin:$PATH

set_env ()
{
  export ORAENV_ASK=NO
  export ORACLE_SID=$1
  . oraenv >> /dev/null
  export ORAENV_ASK=YES
}

#################
## Main Section
#################

# This is logged to CRSD agent log file
echo "`date` Action script '$_CRS_ACTION_SCRIPT' for resource [$_CRS_NAME] called for action $1"

# set environment
set_env DEV
cd /acfs/dbvisit/dbvnet


case "$1" in
     'start')
           ./dbvnet -d start
           RET=0
           echo "Running start dbvnet resource with return code $RET"
     ;;

     'stop')
           NUM=`ps -ef | grep dbvnet | egrep -v 'grep|action-script|resource' | wc -l`
           if [ $NUM = 0 ]; then
             ## do a cleanup of pid
             ./dbvnet -d stop
             RET=0
           else
             ## now stop the dbvnet
             ./dbvnet -d stop
             NUM=`ps -ef | grep dbvnet | grep -v grep | wc -l`
             if [ $NUM = 0 ]; then
               RET=0
             else
               RET=1
             fi
           fi
           echo "Running stop dbvnet resource with return code $RET"
     ;;

     'check')
           NUM=`ps -ef | grep dbvnet | egrep -v 'grep|action-script|resource' | wc -l`
           if [ $NUM = 0 ]; then
             ## return code 1 for check means OFFLINE
             RET=1
           else
             ## return code 0 for check means ONLINE
             RET=0
           fi
           echo "Running check dbvnet resource with return code $RET"
     ;;

     'clean')
           for c1 in `ps -ef|grep dbvnet |egrep -v 'grep|action-script|resource'| awk '{print $2}'` ;
           do
             echo "...force kill dbvnet pid $c1"
             kill -9 $c1
           done
           ## do some cleanup 
           ./dbvnet -d stop
           RET=0
           echo "Running clean dbvnet resource with return code $RET"
     ;;
esac

if [ $RET -eq 0 ]; then
  exit 0
else
  exit 1
fi

5.2. Dbvagent Action Script


The script below is a sample action script that will be used to manage dbvagent.
This script is created in the dbvnet folder /acfs/dbvisit/dbvagent

#!/bin/bash
#
# Dbvagent Action Script

#############################################
#  Function to change database environments
# Description:
# Use oraenv to set the environment if needed
# This is optional to set the environment
#############################################

## set following to ensure oraenv is picked up from /usr/local/bin
export PATH=/usr/local/bin:$PATH

set_env ()
{
  export ORAENV_ASK=NO
  export ORACLE_SID=$1
  . oraenv >> /dev/null
  export ORAENV_ASK=YES
}

#################
## Main Section
#################

# This is logged to CRSD agent log file
echo "`date` Action script '$_CRS_ACTION_SCRIPT' for resource [$_CRS_NAME] called for action $1"

# set environment
set_env DEV
cd /acfs/dbvisit/dbvagent


case "$1" in
     'start')
           ./dbvagent -d start
           RET=0
           echo "Running start dbvagent resource with return code $RET"
     ;;

     'stop')
           NUM=`ps -ef | grep dbvagent | egrep -v 'grep|action-script|resource' | wc -l`
           if [ $NUM = 0 ]; then
             ## do a cleanup of pid
             ./dbvagent -d stop
             RET=0
           else
             ## now stop the agent
             ./dbvagent -d stop
             NUM=`ps -ef | grep dbvagent | grep -v grep | wc -l`
             if [ $NUM = 0 ]; then
               RET=0
             else
               RET=1
             fi
           fi
           echo "Running stop dbvagent resource with return code $RET"
     ;;

     'check')
           NUM=`ps -ef | grep dbvagent | egrep -v 'grep|action-script|resource' | wc -l`
           if [ $NUM = 0 ]; then
             ## return code 1 for check means OFFLINE
             RET=1
           else
             ## return code 0 for check means ONLINE
             RET=0
           fi
           echo "Running check dbvagent resource with return code $RET"
     ;;

     'clean')
           for c1 in `ps -ef|grep dbvagent |egrep -v 'grep|action-script|resource'| awk '{print $2}'` ;
           do
             echo "...force kill dbvagent pid $c1"
             kill -9 $c1
           done
           ## do some cleanup of pids
           ./dbvagent -d stop
           RET=0
           echo "Running clean dbvagent resource with return code $RET"
     ;;
esac

if [ $RET -eq 0 ]; then
  exit 0
else
  exit 1
fi


5.3.  Add the Cluster Resources

On the Primary cluster node 1 (kiwi81) as the root user run the following commands to add the two resources:

IMPORTANT: The lines below should be on one line, but are shown on separate lines to make it easier to read

crsctl add resource dbvnet -type cluster_resource 
   -attr "ACTION_SCRIPT=/acfs/dbvisit/dbvnet/action-script.scr, RESTART_ATTEMPTS=3, START_TIMEOUT=60, STOP_TIMEOUT=60, CHECK_INTERVAL=10
         ,START_DEPENDENCIES='hard(kiwi812-vip) pullup(kiwi812-vip)'
         ,STOP_DEPENDENCIES='hard(kiwi812-vip)' 
         ,ACL='owner:oracle:rwx,pgrp:oinstall:rwx,other::r--' 
         ,PLACEMENT='favored' 
         ,HOSTING_MEMBERS='kiwi81 kiwi82'"




crsctl add resource dbvagent -type cluster_resource 
   -attr "ACTION_SCRIPT=/acfs/dbvisit/dbvagent/action-script.scr, RESTART_ATTEMPTS=3, START_TIMEOUT=60, STOP_TIMEOUT=60, CHECK_INTERVAL=10
         ,START_DEPENDENCIES='hard(kiwi812-vip) pullup(kiwi812-vip)'
         ,STOP_DEPENDENCIES='hard(kiwi812-vip)' 
         ,ACL='owner:oracle:rwx,pgrp:oinstall:rwx,other::r--' 
         ,PLACEMENT='favored' 
         ,HOSTING_MEMBERS='kiwi81 kiwi82'"



On the Standby cluster node 1 (kiwi91) as the root user run the following commands to add the two resources:

IMPORTANT: The lines below should be on one line, but are shown on separate lines to make it easier to read

crsctl add resource dbvnet -type cluster_resource 
   -attr "ACTION_SCRIPT=/acfs/dbvisit/dbvnet/action-script.scr, RESTART_ATTEMPTS=3, START_TIMEOUT=60, STOP_TIMEOUT=60, CHECK_INTERVAL=10
         ,START_DEPENDENCIES='hard(kiwi912-vip) pullup(kiwi912-vip)'
         ,STOP_DEPENDENCIES='hard(kiwi912-vip)' 
         ,ACL='owner:oracle:rwx,pgrp:oinstall:rwx,other::r--' 
         ,PLACEMENT='favored' 
         ,HOSTING_MEMBERS='kiwi91 kiwi92'"


crsctl add resource dbvagent -type cluster_resource 
   -attr "ACTION_SCRIPT=/acfs/dbvisit/dbvagent/action-script.scr, RESTART_ATTEMPTS=3, START_TIMEOUT=60, STOP_TIMEOUT=60, CHECK_INTERVAL=10
         ,START_DEPENDENCIES='hard(kiwi912-vip) pullup(kiwi912-vip)'
         ,STOP_DEPENDENCIES='hard(kiwi912-vip)' 
         ,ACL='owner:oracle:rwx,pgrp:oinstall:rwx,other::r--' 
         ,PLACEMENT='favored' 
         ,HOSTING_MEMBERS='kiwi91 kiwi92'"

5.4.  Start the Dbvnet and Dbvagent Cluster Resources

The following commands can be used to check the status of the cluster resources as well as to start and stop them.

Remember that these two resources are dependant on the Virtual IP address that was added. 

If the Virtual IP address is moved (relocated) to the other node, the Dbvnet and Dbvagent resources will be stopped and moved to the other node and started.

  • Status Check
root@kiwi91[/root]: crsctl status resource dbvnet
NAME=dbvnet
TYPE=cluster_resource
TARGET=ONLINE
STATE=ONLINE on kiwi91

root@kiwi91[/root]: crsctl status resource dbvagent
NAME=dbvagent
TYPE=cluster_resource
TARGET=ONLINE
STATE=ONLINE on kiwi91

root@kiwi91[/root]: crsctl status resource kiwi912-vip
NAME=kiwi912-vip
TYPE=app.appvip_net1.type
TARGET=ONLINE
STATE=ONLINE on kiwi91

or

root@kiwi91[/root]: crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
dbvagent       clus...esource ONLINE    ONLINE    kiwi91
dbvnet         clus...esource ONLINE    ONLINE    kiwi91
kiwi912-vip    app....t1.type ONLINE    ONLINE    kiwi91
...
...
...


  • Start Resource
root@kiwi91[/root]: crsctl start resource kiwi912-vip
CRS-2672: Attempting to start 'kiwi912-vip' on 'kiwi91'
CRS-2676: Start of 'kiwi912-vip' on 'kiwi91' succeeded
root@kiwi91[/root]: crsctl start resource dbvnet
CRS-2672: Attempting to start 'dbvnet' on 'kiwi91'
CRS-2676: Start of 'dbvnet' on 'kiwi91' succeeded
root@kiwi91[/root]: crsctl start resource dbvagent
CRS-2672: Attempting to start 'dbvagent' on 'kiwi91'
CRS-2676: Start of 'dbvagent' on 'kiwi91' succeeded
root@kiwi91[/root]:


  • Stop Resource
root@kiwi91[/root]: crsctl stop resource dbvnet
CRS-2673: Attempting to stop 'dbvnet' on 'kiwi91'
CRS-2677: Stop of 'dbvnet' on 'kiwi91' succeeded
CRS-2679: Attempting to clean 'dbvnet' on 'kiwi91'
CRS-2681: Clean of 'dbvnet' on 'kiwi91' succeeded
root@kiwi91[/root]: crsctl stop resource dbvagent
CRS-2673: Attempting to stop 'dbvagent' on 'kiwi91'
CRS-2677: Stop of 'dbvagent' on 'kiwi91' succeeded
CRS-2679: Attempting to clean 'dbvagent' on 'kiwi91'
CRS-2681: Clean of 'dbvagent' on 'kiwi91' succeeded
root@kiwi91[/root]: crsctl stop resource kiwi912-vip
CRS-2673: Attempting to stop 'kiwi912-vip' on 'kiwi91'
CRS-2677: Stop of 'kiwi912-vip' on 'kiwi91' succeeded


  • Relocate Virtual IP 
root@kiwi91[/root]: crsctl relocate resource kiwi912-vip -f

CRS-2673: Attempting to stop 'dbvagent' on 'kiwi91'
CRS-2673: Attempting to stop 'dbvnet' on 'kiwi91'
CRS-2677: Stop of 'dbvnet' on 'kiwi91' succeeded
CRS-2679: Attempting to clean 'dbvnet' on 'kiwi91'
CRS-2677: Stop of 'dbvagent' on 'kiwi91' succeeded
CRS-2679: Attempting to clean 'dbvagent' on 'kiwi91'
CRS-2681: Clean of 'dbvnet' on 'kiwi91' succeeded
CRS-2681: Clean of 'dbvagent' on 'kiwi91' succeeded
CRS-2673: Attempting to stop 'kiwi912-vip' on 'kiwi91'
CRS-2677: Stop of 'kiwi912-vip' on 'kiwi91' succeeded
CRS-2672: Attempting to start 'kiwi912-vip' on 'kiwi92'
CRS-2676: Start of 'kiwi912-vip' on 'kiwi92' succeeded
CRS-2672: Attempting to start 'dbvagent' on 'kiwi92'
CRS-2676: Start of 'dbvagent' on 'kiwi92' succeeded
CRS-2672: Attempting to start 'dbvnet' on 'kiwi92'
CRS-2676: Start of 'dbvnet' on 'kiwi92' succeeded


6.  Example 2: Create the Dbvisit Standby DDC File

There are two methods to create the Dbvisit Standby DDC file.

In this example, we will show it via the Central Console.

The Central Console is installed onto its own system and have network access to both the Primary and the Standby RAC clusters.

6.1.  Add the two new hosts 

The two hosts to be added is the Dbvagent connections, which is to the Virtual IP addresses.



6.2.  Start the DDC creation process

Navigate to the Configurations screen and start the creation of the new DDC

The New Configuration will now start as seen below:

  1. Select the source as the primary Virtual IP - kiwi812-vip, then accept the license agreement
  2. Select the source Instance Name - this is obtained from the /etc/oratab file.  We need to make sure you have both the Database and the Instance on the specified system in the /etc/oratab file.  Select the instance name, in this case, it will be MYDEV1
  3. Specify the ARCSOURCE location.  This location should ideally be located on the shared storage location.  Remember when a Graceful Switchover is performed the roles are reversed between the ARCSOURCE and the ARCHDEST values in the DDC file
  4. Specify to use DBVNET by default (recommended)
  5. You will now be presented with the primary RAC database details, review and confirm the database and ASM instances match the servers they are running on.  Dbvisit Standby will detect these values based on the connection that was made to the primary instance that was selected in step 2 above.
  6. As we are going to create a standby RAC database, we will adjust this setting to YES.  See the next screenshot below


From the above image we can see:

  1. Specify the standby database will be a RAC enabled environment
  2. Select the standby Virtual IP hostname - kiwi912-vip
  3. You are then presented with the RAC Thread details for node 1 and 2 on the standby server, fill in the required detail
  4. Specify the ARCHDEST location - this should be on shared storage and is the location where Dbvisit Standby will transfer archive logs into on the standby server.
  5. Review and update the Dbvisit Base location on the standby server if different
  6. Review the Oracle Database Home on the standby server
  7. Review and update the DB Unique Name (db_unique_name) of the standby if required - default value recommended 
  8. Specify the Dbvisit Standby Configuration (DDC) name
  9. Provide the license key for the configuration, this is optional and can be provided after the DDC creation, but must be provided before the running the CSD(Create Standby Database)


The last step is to click on Submit.

Dbvisit Standby will now validate the parameters specified and create a new DDC file as well as a repository (stored in the DBVISIT_BASE/standby/conf directory)




7.  Example 2:  Create the Standby Database

The standby database can either be created via the command line interface (dbvctl -d <DDC> --csd) or via the Central Console.

In this example, we will show the use of the Central Console to create the standby database

  • Navigate to the "Create Standby Database" Menu
  • Select the MYDEV DDC file from the dropdown list.
  • Then specify the process option as "NEW DATABASE"


The server parameter file (spfile) section does contain a lot of information.  Make sure you review the locations of the datafiles, redo logs and spfile.

In this example, the primary and standby systems match exactly from a storage point of view (same disk groups etc) so we do not have to change any parameters.

From the above, we can see that the temporary location /user/tmp was used for the local and remote temp backup locations.  

Make sure the locations you specify has sufficient space to hold a full compressed database backup.

In this example, the "Create Standby Database & Template" option is selected.

The new task will be created as shown below:

8.  Example 2: Send Logs from Primary to Standby

  1. Navigate to the Database Actions Screen
  2. Click on the option to "Send Logs"
  3. Review the Task details


9.  Example 2: Apply Archivelog to Standby

  • Navigate to the Database Actions Screen
  • Click on the option to "Apply Logs"
  • Review the Task details



10.  Oracle RAC on Windows

This section will include additional information and recommendations when installing Dbvisit Standby on an Oracle RAC configuration on Windows.

10.1.  Installation Recommendations

On Windows, the setup should be similar to a UNIX based Oracle RAC configuration.  You have to install the Oracle software (Clusterware and Database) as per Oracle best practice.  

There is one important difference between UNIX based installations of Dbvisit Standby compared to Windows (when installing on a RAC cluster) and this is that on Windows - Services is used, which means we need to create Dbvnet and Dbvagent services on both nodes in the cluster.

Example steps would be:

  • Install the Oracle Clusterware and Configure ASM, including ASM diskgroups and ACFS, shared storage (recommended to place the Dbvisit Software on shared disk - as well as the Dbvisit ARCHDEST/ARCSOURCE locations)
  • Install the Oracle Database software (again following best practice from Oracle Documentation).
  • Once the database is created - in most cases step 1-3 here would already be done and you would be focussing on the next steps:
  • Download the latest Dbvisit Standby software
  • Make sure you configure the Virtual IP that will be used by Dbvisit Standby (please see above).  This is the IP address/host alias that the standby server will be used to communicate with the RAC cluster - meaning it will see the RAC cluster as one using this Virtual IP (host alias) and from its point of view does not care which node this Virtual IP and of course Dbvisit Standby will be running on.

Example - Below we created a Virtual IP 10.0.2.118 for the VIP Name (host alias) kiwiwin-vip (note this should be added to your DNS).  In this example, the RAC cluster was using local accounts and not domain accounts.  If domain accounts were used the -user flag should be updated to the domain account.

C:\app\11.2.0\grid\BIN>appvipcfg.bat create -network=1 -ip=10.0.2.118 -vipname=kiwiwin-vip -user=.\oracle
roduction Copyright 2007, 2008, Oracle.All rights reserved
017-10-30 02:01:14: Creating Resource Type
017-10-30 02:01:14: Executing C:\app\11.2.0\grid\bin\crsctl add type app.appvip_net1.type -basetype ora.cluster_vip_net1.type -file C:\app\11.2.0\grid/crs/template/appvip.type
017-10-30 02:01:14: Executing cmd: C:\app\11.2.0\grid\bin\crsctl add type app.appvip_net1.type -basetype ora.cluster_vip_net1.type -file C:\app\11.2.0\grid/crs/template/appvip.type
017-10-30 02:01:15: Create the Resource
017-10-30 02:01:15: Executing C:\app\11.2.0\grid\bin\crsctl add resource kiwiwin-vip -type app.appvip_net1.type -attr "USR_ORA_VIP=10.0.2.118,START_DEPENDENCIES=hard(ora.net1.network) pullup(ora.net1.network),STOP_DEPENDENCIES=hard(ora.net1.network),ACL='owner:.\oracle:rwx,pgrp::r-x,other::r--',HOSTING_MEMBERS=kiwiwin01,APPSVIP_FAILBACK="
017-10-30 02:01:15: Executing cmd: C:\app\11.2.0\grid\bin\crsctl add resource kiwiwin-vip -type app.appvip_net1.type -attr "USR_ORA_VIP=10.0.2.118,START_DEPENDENCIES=hard(ora.net1.network) pullup(ora.net1.network),STOP_DEPENDENCIES=hard(ora.net1.network),ACL='owner:.\oracle:rwx,pgrp::r-x,other::r--',HOSTING_MEMBERS=kiwiwin01,APPSVIP_FAILBACK="



c:\app\11.2.0\grid\BIN>crsctl status resource kiwiwin-vip -p
NAME=kiwiwin-vip
TYPE=app.appvip_net1.type
ACL=owner:.\oracle:rwx,pgrp::r-x,other::r--
ACTION_FAILURE_TEMPLATE=
ACTION_SCRIPT=
ACTIVE_PLACEMENT=1
AGENT_FILENAME=%CRS_HOME%/bin/orarootagent%CRS_EXE_SUFFIX%
APPSVIP_FAILBACK=0
AUTO_START=restore
CARDINALITY=1
CHECK_INTERVAL=1
CHECK_TIMEOUT=120
DEFAULT_TEMPLATE=PROPERTY(RESOURCE_CLASS=vip) ELEMENT(HOSTING_MEMBERS=%HOSTING_MEMBERS%)
DEGREE=1
DESCRIPTION=Application VIP
ENABLED=1
FAILOVER_DELAY=0
FAILURE_INTERVAL=0
FAILURE_THRESHOLD=0
GEN_USR_ORA_STATIC_VIP=
GEN_USR_ORA_VIP=
HOSTING_MEMBERS=kiwiwin01
LOAD=1
LOGGING_LEVEL=1
NLS_LANG=
NOT_RESTARTING_TEMPLATE=
OFFLINE_CHECK_INTERVAL=0
PLACEMENT=balanced
PROFILE_CHANGE_TEMPLATE=
RESTART_ATTEMPTS=0
SCRIPT_TIMEOUT=60
SERVER_POOLS=*
START_DEPENDENCIES=hard(ora.net1.network) pullup(ora.net1.network)
START_TIMEOUT=120
STATE_CHANGE_TEMPLATE=
STOP_DEPENDENCIES=hard(ora.net1.network)
STOP_TIMEOUT=0
TYPE_VERSION=2.2
UPTIME_THRESHOLD=7d
USR_ORA_ENV=
USR_ORA_VIP=10.0.2.118
VERSION=11.2.0.3.0



c:\app\11.2.0\grid\BIN>ping kiwiwin-vip

Pinging kiwiwin-vip.oraclekiwi.co.nz [10.0.2.118] with 32 bytes of data:
Control-C
^C


c:\app\11.2.0\grid\BIN>crsctl start resource kiwiwin-vip
CRS-2672: Attempting to start 'kiwiwin-vip' on 'kiwiwin02'
CRS-2676: Start of 'kiwiwin-vip' on 'kiwiwin02' succeeded



c:\app\11.2.0\grid\BIN>ping kiwiwin-vip

Pinging kiwiwin-vip.oraclekiwi.co.nz [10.0.2.118] with 32 bytes of data:
Reply from 10.0.2.118: bytes=32 time<1ms TTL=128
Reply from 10.0.2.118: bytes=32 time<1ms TTL=128

Ping statistics for 10.0.2.118:
    Packets: Sent = 2, Received = 2, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms
Control-C
^C


  • Once the Virtual IP is created, you can install Dbvisit Standby on the first node, using shared storage - example if ACFS is used, in this example the shared storage was mounted under C:\app\oracle\acfsmounts\acfs_data 


In this example the Dbvisit Base was: C:\app\oracle\acfsmounts\acfs_data\dbvisit

ONLY Dbvnet, Dbvagent and Standby Core components are installed.  It is not recommended to run the GUI (Central Console) from a RAC Cluster - but rather on a standalone small system or the standby server.


  • Install Dbvisit Standby first on node 1, then on node 2, in both cases install it into the same directory as mentioned above and let the services be created, but after installation makes sure you stop all services.

Also, make sure that when you install Dbvnet and the Agent that you configure the listening address as the Virtual IP hostname.  This is to ensure that when Dbvnet or the Agent is started, it will listen on this Virtual IP.


  • Now create a directory on both systems, example c:\dbv-scripts
  • Inside this directory create two batch scripts which will be used as the Action Scripts when the cluster resources are created.


Example Action Scripts - Please note these are samples and should be tested in your development or test environments prior to implementing into production:

Dbvnet Action Script: Dbvnet.cmd

@echo off

setlocal
pushd


if "%~1"=="" (
   echo Action cannot be empty ...
   echo:
   call :HELP
 ) & (goto :EOF)
 
REM Test if Usage/Help should be displayed
if /i {%1}=={/?} (call :HELP) & (goto :EOF)
 
goto :setEnvironment

REM ##########################################
REM   Function to set Environment values
REM ##########################################
:setEnvironment

    set action=%~1

    for /F "tokens=1,2,3,4 delims=/- "  %%a  in  ('date /t') do set CDATE=%%d%%c%%b
    for /F "tokens=1,2,3,4 delims=/: "  %%a  in  ('time /t') do set CTIME=%%a%%b%%c
    set LOGTIME=%CDATE%-%CTIME%
    set DBVISIT_BASE="C:\app\oracle\acfsmounts\acfs_data\Dbvisit"
    echo %LOGTIME%

GOTO :EndOfSetEnvironment
REM ##########################################


:EndOfSetEnvironment

if "%action%" == "start" goto :start
if "%action%" == "stop" goto :stop
if "%action%" == "check" goto :check
if "%action%" == "clean" goto :clean
goto :cleanexit

:start
  %DBVISIT_BASE%\dbvsmgr\dbvsmgr.exe -s dbvnet
  echo "Running start dbvnet resource with return code %ERRORLEVEL%"
  if %ERRORLEVEL% == 0 (goto :cleanexit)
  if %ERRORLEVEL% gtr 0 (goto :errorexit)

:stop
  %DBVISIT_BASE%\dbvsmgr\dbvsmgr.exe -t dbvnet
  echo "Running stop dbvnet resource with return code %ERRORLEVEL%"
  if %ERRORLEVEL% == 0 (goto :cleanexit)
  if %ERRORLEVEL% gtr 0 (goto :errorexit)
  
:check
  echo "Running status check on Dbvnet resource"
for /F "tokens=3 delims=: " %%H in ('sc query "Dbvnet" ^| findstr "        STATE"') do (
  if /I "%%H" NEQ "RUNNING" (
    set ERRORLEVEL=1
    goto :errorexit
  )
  if /I "%%H" == "RUNNING" (
    set ERRORLEVEL=0
    goto :cleanexit
  ) 
)

:clean
   %DBVISIT_BASE%\Dbvisit\dbvsmgr\dbvsmgr.exe -t dbvnet
  echo "Running clean dbvnet resource with return code %ERRORLEVEL%"
  if %ERRORLEVEL% == 0 (goto :cleanexit)
  if %ERRORLEVEL% gtr 0 (goto :errorexit)

:cleanexit
  exit /b 0
  
:errorexit
  exit /b 1

  
:HELP
  echo USAGE:
  echo  dbvnet.scr [start^|stop^|clean^|check]
goto :EOF

:EXIT
popd
endlocal
REM echo EXIT %ERRORLEVEL%
goto :EOF

Example Dbvagent action script:  dbvagent.cmd

@echo off

setlocal
pushd


if "%~1"=="" (
   echo Action cannot be empty ...
   echo:
   call :HELP
 ) & (goto :EOF)
 
REM Test if Usage/Help should be displayed
if /i {%1}=={/?} (call :HELP) & (goto :EOF)
 
goto :setEnvironment

REM ##########################################
REM   Function to set Environment values
REM ##########################################
:setEnvironment

    set action=%~1

    for /F "tokens=1,2,3,4 delims=/- "  %%a  in  ('date /t') do set CDATE=%%d%%c%%b
    for /F "tokens=1,2,3,4 delims=/: "  %%a  in  ('time /t') do set CTIME=%%a%%b%%c
    set LOGTIME=%CDATE%-%CTIME%
    set DBVISIT_BASE="C:\app\oracle\acfsmounts\acfs_data\Dbvisit"
    echo %LOGTIME%

GOTO :EndOfSetEnvironment
REM ##########################################


:EndOfSetEnvironment

if "%action%" == "start" goto :start
if "%action%" == "stop" goto :stop
if "%action%" == "check" goto :check
if "%action%" == "clean" goto :clean
goto :cleanexit

:start
  %DBVISIT_BASE%\dbvsmgr\dbvsmgr.exe -s dbvagent
  echo "Running start dbvagent resource with return code %ERRORLEVEL%"
  if %ERRORLEVEL% == 0 (goto :cleanexit)
  if %ERRORLEVEL% gtr 0 (goto :errorexit)

:stop
  %DBVISIT_BASE%\dbvsmgr\dbvsmgr.exe -t dbvagent
  echo "Running stop dbvagent resource with return code %ERRORLEVEL%"
  if %ERRORLEVEL% == 0 (goto :cleanexit)
  if %ERRORLEVEL% gtr 0 (goto :errorexit)
  
:check
  echo "Running status check on Dbvagent resource"
for /F "tokens=3 delims=: " %%H in ('sc query "Dbvagent" ^| findstr "        STATE"') do (
  if /I "%%H" NEQ "RUNNING" (
    set ERRORLEVEL=1
    goto :errorexit
  )
  if /I "%%H" == "RUNNING" (
    set ERRORLEVEL=0
    goto :cleanexit
  ) 
)

:clean
  %DBVISIT_BASE%\dbvsmgr\dbvsmgr.exe -t dbvagent
  echo "Running clean dbvagent resource with return code %ERRORLEVEL%"
  if %ERRORLEVEL% == 0 (goto :cleanexit)
  if %ERRORLEVEL% gtr 0 (goto :errorexit)

:cleanexit
  exit /b 0
  
:errorexit
  exit /b 1

  
:HELP
  echo USAGE:
  echo  dbvagent.scr [start^|stop^|clean^|check]
goto :EOF

:EXIT
popd
endlocal
REM echo EXIT %ERRORLEVEL%
goto :EOF

Once these scripts are on both RAC nodes in the exact same location, we can register the resource from the first node (Where the Dbvisit Virtual IP that will be used is running).

The commands to add the cluster resource is shown below:

Example: Add Dbvnet Cluster Resource

C:\dbv-scripts>set ORACLE_HOME=c:\app\11.2.0\grid

C:\dbv-scripts>C:\app\11.2.0\grid\bin\crsctl add resource dbvnet -type cluster_resource -attr "ACTION_SCRIPT=c:\dbv-scripts\dbvnet.cmd, RESTART_ATTEMPTS=3, START_TIMEOUT=60, STOP_TIMEOUT=60, CHECK_INTERVAL=10, START_DEPENDENCIES='hard(kiwiwin-vip) pullup(kiwiwin-vip)',STOP_DEPENDENCIES='hard(kiwiwin-vip)' PLACEMENT='favored' HOSTING_MEMBERS='kiwiwin01'"

Example: Add Dbvagent Cluster Resource

C:\dbv-scripts>set ORACLE_HOME=c:\app\11.2.0\grid

C:\dbv-scripts>%ORACLE_HOME%\bin\crsctl add resource dbvagent -type cluster_resource -attr "ACTION_SCRIPT=c:\dbv-scripts\dbvagent.cmd, RESTART_ATTEMPTS=3, START_TIMEOUT=60, STOP_TIMEOUT=60, CHECK_INTERVAL=10, START_DEPENDENCIES='hard(kiwiwin-vip) pullup(kiwiwin-vip)',STOP_DEPENDENCIES='hard(kiwiwin-vip)' PLACEMENT='favored' HOSTING_MEMBERS='kiwiwin01'"

More detail example output:

Example - Dbvnet:

C:\dbv-scripts>dbvnet.cmd
Action cannot be empty ...

USAGE:
 dbvnet.scr [start|stop|clean|check]

C:\dbv-scripts>dbvnet.cmd check
20170411-0713PM
Dbvnet service is running
Running check dbvnet resource with return code 0

C:\dbv-scripts>dbvnet.cmd stop
20170411-0713PM
Stopping Dbvnet service...........
Stop Complete
Running stop dbvnet resource with return code 0

C:\dbv-scripts>dbvnet.cmd start
20170411-0713PM
Starting Dbvnet service
Start Complete
Running start dbvnet resource with return code 0

C:\dbv-scripts>dbvnet.cmd clean
20170411-0713PM
Stopping Dbvnet service...........
Stop Complete
Running clean dbvnet resource with return code 0

C:\dbv-scripts>dbvnet.cmd check
20170411-0714PM
Dbvnet service is not running
Running check dbvnet resource with return code 10

C:\dbv-scripts>echo %ERRORLEVEL%
1

C:\dbv-scripts>dbvnet.cmd start
20170411-0714PM
Starting Dbvnet service
Start Complete
Running start dbvnet resource with return code 0

C:\dbv-scripts>set ORACLE_HOME=c:\app\11.2.0\grid

C:\dbv-scripts>C:\app\11.2.0\grid\bin\crsctl add resource dbvnet -type cluster_resource -attr "ACTION_SCRIPT=c:\dbv-scripts\dbvnet.cmd, RESTART_ATTEMPTS=3, START_TIMEOUT=60, STOP_TIMEOUT=60, CHECK_INTERVAL=10, START_DEPENDENCIES='hard(kiwiwin-vip) pullup(kiwiwin-vip)',STOP_DEPENDENCIES='hard(kiwiwin-vip)' PLACEMENT='favored' HOSTING_MEMBERS='kiwiwin01'"

C:\dbv-scripts>
C:\dbv-scripts>crsctl status resource dbvnet
NAME=dbvnet
TYPE=cluster_resource
TARGET=OFFLINE
STATE=OFFLINE


C:\dbv-scripts>dbvnet.cmd stop
20170411-0720PM
Stopping Dbvnet service...........
Stop Complete
Running stop dbvnet resource with return code 0

C:\dbv-scripts>crsctl start resource dbvnet
CRS-2672: Attempting to start 'dbvnet' on 'kiwiwin01'
CRS-2676: Start of 'dbvnet' on 'kiwiwin01' succeeded



C:\dbv-scripts>crsctl status resource dbvnet
NAME=dbvnet
TYPE=cluster_resource
TARGET=ONLINE
STATE=ONLINE on kiwiwin01


C:\dbv-scripts>


C:\dbv-scripts>crsctl status resource dbvnet -p
NAME=dbvnet
TYPE=cluster_resource
ACL=owner:oracle:rwx,pgrp::---,other::r--
ACTION_FAILURE_TEMPLATE=
ACTION_SCRIPT=c:\dbv-scripts\dbvnet.cmd
ACTIVE_PLACEMENT=0
AGENT_FILENAME=%CRS_HOME%\bin\scriptagent.exe
AUTO_START=restore
CARDINALITY=1
CHECK_INTERVAL=10
DEFAULT_TEMPLATE=
DEGREE=1
DESCRIPTION=
ENABLED=1
FAILOVER_DELAY=0
FAILURE_INTERVAL=0
FAILURE_THRESHOLD=0
HOSTING_MEMBERS=kiwiwin01
LOAD=1
LOGGING_LEVEL=1
NOT_RESTARTING_TEMPLATE=
OFFLINE_CHECK_INTERVAL=0
PLACEMENT=favored
PROFILE_CHANGE_TEMPLATE=
RESTART_ATTEMPTS=3
SCRIPT_TIMEOUT=60
SERVER_POOLS=
START_DEPENDENCIES=hard(kiwiwin-vip) pullup(kiwiwin-vip)
START_TIMEOUT=60
STATE_CHANGE_TEMPLATE=
STOP_DEPENDENCIES=hard(kiwiwin-vip)
STOP_TIMEOUT=60
UPTIME_THRESHOLD=1h


Dbvagent:

C:\dbv-scripts>dbvagent.cmd check
20170411-0723PM
Dbvnet service is running
Running check dbvagent resource with return code 0"

C:\dbv-scripts>dbvagent.cmd clean
20170411-0723PM
Stopping Dbvnet service...........
Stop Complete
Running clean dbvagent resource with return code 0"

C:\dbv-scripts>dbvagent.cmd start
20170411-0723PM
Starting Dbvnet service
Start Complete
Running start dbvagent resource with return code 0"

C:\dbv-scripts>dbvagent.cmd stop
20170411-0723PM
Stopping Dbvnet service...........
Stop Complete
Running stop dbvagent resource with return code 0"

C:\dbv-scripts>
C:\dbv-scripts>
C:\dbv-scripts>set ORACLE_HOME=c:\app\11.2.0\grid

C:\dbv-scripts>%ORACLE_HOME%\bin\crsctl add resource dbvagent -type cluster_resource -attr "ACTION_SCRIPT=c:\dbv-scripts\dbvagent.cmd, RESTART_ATTEMPTS=3, START_TIMEOUT=60, STOP_TIMEOUT=60, CHECK_INTERVAL=10, START_DEPENDENCIES='hard(kiwiwin-vip) pullup(kiwiwin-vip)',STOP_DEPENDENCIES='hard(kiwiwin-vip)' PLACEMENT='favored' HOSTING_MEMBERS='kiwiwin01'"

C:\dbv-scripts>
C:\dbv-scripts>crsctl status resource dbvagent -p
NAME=dbvagent
TYPE=cluster_resource
ACL=owner:oracle:rwx,pgrp::---,other::r--
ACTION_FAILURE_TEMPLATE=
ACTION_SCRIPT=c:\dbv-scripts\dbvagent.cmd
ACTIVE_PLACEMENT=0
AGENT_FILENAME=%CRS_HOME%\bin\scriptagent.exe
AUTO_START=restore
CARDINALITY=1
CHECK_INTERVAL=10
DEFAULT_TEMPLATE=
DEGREE=1
DESCRIPTION=
ENABLED=1
FAILOVER_DELAY=0
FAILURE_INTERVAL=0
FAILURE_THRESHOLD=0
HOSTING_MEMBERS=kiwiwin01
LOAD=1
LOGGING_LEVEL=1
NOT_RESTARTING_TEMPLATE=
OFFLINE_CHECK_INTERVAL=0
PLACEMENT=favored
PROFILE_CHANGE_TEMPLATE=
RESTART_ATTEMPTS=3
SCRIPT_TIMEOUT=60
SERVER_POOLS=
START_DEPENDENCIES=hard(kiwiwin-vip) pullup(kiwiwin-vip)
START_TIMEOUT=60
STATE_CHANGE_TEMPLATE=
STOP_DEPENDENCIES=hard(kiwiwin-vip)
STOP_TIMEOUT=60
UPTIME_THRESHOLD=1h


C:\dbv-scripts>crsctl start resource dbvagent
CRS-2672: Attempting to start 'dbvagent' on 'kiwiwin01'
CRS-2676: Start of 'dbvagent' on 'kiwiwin01' succeeded

C:\dbv-scripts>crsctl status resource dbvagent
NAME=dbvagent
TYPE=cluster_resource
TARGET=ONLINE
STATE=ONLINE on kiwiwin01


Testing Relocation of the Resources:

C:\dbv-scripts>crsctl relocate resource kiwiwin-vip
CRS-2529: Unable to act on 'kiwiwin-vip' because that would require stopping or relocating 'dbvnet', but the force option was not specified
CRS-4000: Command Relocate failed, or completed with errors.

C:\dbv-scripts>crsctl relocate resource kiwiwin-vip -f
CRS-2673: Attempting to stop 'dbvagent' on 'kiwiwin01'
CRS-2673: Attempting to stop 'dbvnet' on 'kiwiwin01'
CRS-2677: Stop of 'dbvnet' on 'kiwiwin01' succeeded
CRS-2677: Stop of 'dbvagent' on 'kiwiwin01' succeeded
CRS-2673: Attempting to stop 'kiwiwin-vip' on 'kiwiwin01'
CRS-2677: Stop of 'kiwiwin-vip' on 'kiwiwin01' succeeded
CRS-2672: Attempting to start 'kiwiwin-vip' on 'kiwiwin02'
CRS-2676: Start of 'kiwiwin-vip' on 'kiwiwin02' succeeded
CRS-2672: Attempting to start 'dbvagent' on 'kiwiwin02'
CRS-2676: Start of 'dbvagent' on 'kiwiwin02' succeeded
CRS-2672: Attempting to start 'dbvnet' on 'kiwiwin02'
CRS-2676: Start of 'dbvnet' on 'kiwiwin02' succeeded

C:\dbv-scripts>crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
dbvagent       clus...esource ONLINE    ONLINE    kiwiwin02
dbvnet         clus...esource ONLINE    ONLINE    kiwiwin02
kiwiwin-vip    app....t1.type ONLINE    ONLINE    kiwiwin02
ora.DATA.dg    ora....up.type ONLINE    ONLINE    kiwiwin01
ora....ER.lsnr ora....er.type ONLINE    ONLINE    kiwiwin01
ora....N1.lsnr ora....er.type ONLINE    ONLINE    kiwiwin02
ora....N2.lsnr ora....er.type ONLINE    ONLINE    kiwiwin01
ora....N3.lsnr ora....er.type ONLINE    ONLINE    kiwiwin01
ora.OCRVOT.dg  ora....up.type ONLINE    ONLINE    kiwiwin01
ora.asm        ora.asm.type   ONLINE    ONLINE    kiwiwin01
ora.cvu        ora.cvu.type   ONLINE    ONLINE    kiwiwin01
ora.dev.db     ora....se.type ONLINE    ONLINE    kiwiwin01
ora.gsd        ora.gsd.type   OFFLINE   OFFLINE
ora....SM1.asm application    ONLINE    ONLINE    kiwiwin01
ora....01.lsnr application    ONLINE    ONLINE    kiwiwin01
ora....n01.gsd application    OFFLINE   OFFLINE
ora....n01.ons application    ONLINE    ONLINE    kiwiwin01
ora....n01.vip ora....t1.type ONLINE    ONLINE    kiwiwin01
ora....SM2.asm application    ONLINE    ONLINE    kiwiwin02
ora....02.lsnr application    ONLINE    ONLINE    kiwiwin02
ora....n02.gsd application    OFFLINE   OFFLINE
ora....n02.ons application    ONLINE    ONLINE    kiwiwin02
ora....n02.vip ora....t1.type ONLINE    ONLINE    kiwiwin02
ora....network ora....rk.type ONLINE    ONLINE    kiwiwin01
ora.oc4j       ora.oc4j.type  ONLINE    ONLINE    kiwiwin01
ora.ons        ora.ons.type   ONLINE    ONLINE    kiwiwin01
ora....ry.acfs ora....fs.type ONLINE    ONLINE    kiwiwin01
ora.scan1.vip  ora....ip.type ONLINE    ONLINE    kiwiwin02
ora.scan2.vip  ora....ip.type ONLINE    ONLINE    kiwiwin01
ora.scan3.vip  ora....ip.type ONLINE    ONLINE    kiwiwin01

C:\dbv-scripts>

10.2 Example on How to Modify OS User Privileges for 11gR2 Grid Infrastructure and RAC Services

In some environments, without the following steps, the VIP resource would not start and give an error. 


Run on node 1:
Run on node 2: