Oracle RAC Configurations
1. Introduction
In this section, we will cover a few of the advanced topics with regard to Dbvisit Standby and Oracle RAC configurations.
In this section, the concepts will be explained using an example.
1.1. Different types of Oracle RAC configurations
Dbvisit Standby version 10 handles Oracle RAC configurations similar to Dbvisit Standby versions 8 and 9. It is important to take note of this and not attempt a version 7.0.x to 10.0.x upgrade without knowing this as you will not be able to perform a direct upgrade.
If you are moving from version 7 to version 10, you MUST create a new DDC file. You will not be allowed to upgrade the current version 7 DDC file if Oracle RAC is being detected.
Dbvisit Standby version 10 is designed to be more Oracle RAC aware. This include
- Run Dbvisit Standby on only one of the Oracle RAC nodes
- To allow for this use a new Virtual IP - to be created and assigned to run on the preferred node of your choosing. This allows Dbvisit Standby to treat the cluster as "one" node. Example, if the VIP is 10.0.2.89 and the name attached to this is kiwi812-vip. We can anywhere on the network (as long as kiwi812-vip for 10.0.2.89 is in the DNS) ping or attempt a connection to dbv-vip and it will always run on one of the RAC nodes, it does not make a difference which one. The VIP will run on the node where the Dbvisit Standby components are running. That way you do not have to configure Dbvisit Standby on all the RAC nodes, but only on one. Which is the much more effective and better use of resources?
- If this preferred node fails for some reason, switch the kiwi812-vip (Virtual IP) and the Dbvisit components (resources) to the second node (using action scripts and adding these components as cluster resources).
- Now to achieve this configuration, as you might have guessed, Shared storage is used. This allows the software to be installed once, configured to use the Virtual IP (example kiwi812-vip) and then you can switch between the nodes (after resources have been configured and added to Clusterware).
- Shared Storage can be ACFS, NFS or possibly any other cluster filesystem. This filesystem will store the Dbvisit Software as well as copies of the Archive Logs during certain phases of Dbvisit Standby use, depending on your configuration.
- It is recommended that the Dbvisit ARCHSOURCE and ARCHDEST be located on shared storage. (Remember when a Graceful Switchover - role reversal is performed, the ARCHSOURCE and ARCHDEST parameter values are swapped. So make sure that ARCHSOURCE is located on the shared storage. This is where archive logs will be stored when the RAC cluster becomes a standby - and if on shared storage, both nodes can perform recovery (whichever one would be running the Dbvisit Standby resources at the time).
- Note: this does not mean you cannot use local storage. The downside is if you use local storage and not shared storage, you can only run of one node unless you implement something like "rsync" to synchronize the local folder for the node 1 (maybe node 1 is your preferred node) to the secondary node 2. That way you have the resources for Dbvisit Standby running on node 1 and all files are synced with node 2 (using "rsync" and a "CRON" schedule). This is not a perfect solution but is an option if you really cannot implement the use of shared storage.
On this page:
2. Example 1: Oracle RAC Primary to Single Instance Standby
Two Options are available, using shared storage on the Oracle RAC cluster for the Dbvisit Standby installation, or using local storage and run of only one node and optionally configure "rsync" to have the Dbvisit Standby folder synced between the primary nodes.
The two diagrams below explain the configuration in more detail:
2.1. Option 1 - Recommended - Using Shared Storage
2.2. Option 2 - Using Local Storage
The diagram below provides an overview of the Architecture:
To explain the configuration further we will use an example. Below is a diagram that provides high-level details for the environment:
The key parts to note:
- The primary RAC Cluster consists of two nodes (kiwi81 and kiwi82)
- The first node (kiwi81) will be used as the node to run Dbvisit Standby on local storage
- The "Dbvisit Base" folder "/usr/dbvisit" will be synced between the primary node kiwi81 and kiwi82 using the "rsync" utility and a CRON schedule.
- The standby server is a single instance system using Filesystem Bases storage:
- /u01/app/oracle/oradata1 --> this location will hold the database files (OMF - Oracle Managed Files will be used), this location will also hold a copy of the redo logs (OMF)
- /u01/app/oracle/oradata2 → This location will hold the second copy of the redo logs
- /u01/app/oracle/fast_recovery_area . → This location will be used as the recovery area on the standby server
The Dbvisit Standby locations are:
- DBVISIT_BASE = /usr/dbvisit
- ARCHSOURCE = /u01/app/oracle/dbvisit_arch/DEV
- ARCHDEST = /u01/app/oracle/dbvisit_arch/DEV
Now that we have an overview of the configuration, we can start with the installation.
We will install Dbvisit Standby software on node 1 (kiwi81) and on the standby server (kiwi91).
The installation of the software on kiwi1 will be synced later (see 2.2.4 below).
2.2.1. Installing the Software
Below is an example of installing the Dbvisit Standby software onto a local folder - the default which is /usr/dbvisit
Remember in this example, the VIP on the RAC Cluster is running on kiwi1 and is called "dbv-vip"
We first install the software on node 1 (kiwi81) and then on the standby node (kiwi91).
From Dbvisit Standby V 9.0.10 onwards, Dbvnet installer prompt for remote hostname is removed.
Optional Dbvagent Parameter has been added. In most cases, the default value is recommended.
2.2.1.1. Installing Dbvisit Standby on the RAC Node 1 (kiwi81)
oracle@kiwi81[/home/oracle/dbvisit/installer]: ./install-dbvisit ----------------------------------------------------------- Welcome to the Dbvisit software installer. ----------------------------------------------------------- It is recommended to make a backup of our current Dbvisit software location (Dbvisit Base location) for rollback purposes. Installer Directory /home/oracle/dbvisit >>> Please specify the Dbvisit installation directory (Dbvisit Base). The various Dbvisit products and components - such as Dbvisit Standby, Dbvisit Dbvnet will be installed in the appropriate subdirectories of this path. Enter a custom value or press ENTER to accept default [/usr/dbvisit]: > DBVISIT_BASE = /usr/dbvisit ----------------------------------------------------------- Component Installer Version Installed Version ----------------------------------------------------------- standby 10.0.0RC_24_g94ba1d85 not installed dbvnet 10.0.0RC_24_g94ba1d85 (dbvnet v1.5.0-301-g3896e54)not installed dbvagent 10.0.0RC_24_g94ba1d85 not installed dbvserver 10.0.0RC_24_g94ba1d85 not installed observer 10.0.0RC_24_g94ba1d85 not installed ----------------------------------------------------------- What action would you like to perform? 1 - Install component(s) 2 - Uninstall component(s) 3 - Exit Your choice: 1 Choose component(s): 1 - Core Components (Dbvisit Standby Cli, Dbvnet, Dbvagent) 2 - Dbvisit Standby Core (Command Line Interface) 3 - Dbvnet (Dbvisit Network Communication) 4 - Dbvagent (Dbvisit Agent) 5 - Dbvserver (Dbvisit Central Console) - Not available on Solaris/AIX 6 - Dbvisit Observer (Automatic Failover Option) - Not available on Solaris/AIX Press ENTER to exit Installer Your choice: 1 >>> Please specify the Oracle Instant Client version. The Oracle Instant Client is a required library for Dbvisit products. You may choose 12.1 or 12.2, however the default version is 12.2. Enter a custom value or press ENTER to accept default [12.2]: > ----------------------------------------------------------- Summary of the Dbvisit STANDBY configuration ----------------------------------------------------------- DBVISIT_BASE /usr/dbvisit INSTANT_CLIENT_VERSION 12.2 Press ENTER to continue ----------------------------------------------------------- About to configure DBVISIT DBVNET ----------------------------------------------------------- >>> Please specify the Local host name to be used by Dbvnet on this server. Dbvnet will be listening on the local IP Address on this server which resolve to the host name specified here. If using a cluster or virtual IP make sure the host name or alias specified here resolve to the IP address local to where dbvnet is installed. The host name should resolve to IPv4 address, if not you can use an IPv4 IP address instead of host name. Enter a custom value or press ENTER to accept default [kiwi81]: > kiwi812-vip >>> Please specify the Local Dbvnet PORT to be used. Dbvnet will be listening on the specified port for incoming connections from remote dbvnet connections. Please make sure that this port is not already in use or blocked by any firewall. You may choose any value between 1024 and 65535, however the default of 7890 is recommended. Enter a custom value or press ENTER to accept default [7890]: > >>> Please specify the Dbvnet Passphrase to be used for secure connections. The passphrase provided must be the same in both the local and remote Dbvnet installations. It is used to establish a secure (encrypted) Dbvnet connections Enter a custom value: > kiwi123 ----------------------------------------------------------- Summary of the Dbvisit DBVNET configuration ----------------------------------------------------------- DBVISIT_BASE /usr/dbvisit DBVNET_LOCAL_HOST kiwi812-vip DBVNET_LOCAL_PORT 7890 DBVNET_PASSPHRASE kiwi123 Press ENTER to continue ----------------------------------------------------------- About to configure DBVISIT DBVAGENT ----------------------------------------------------------- >>> Please specify the IPv4 address or host name to be used for the Dbvisit Agent. The Dbvisit Agent (Dbvagent) will be listening on this local address. Connections from Dbvserver (GUI) will be established to this Dbvagent, thus the address must be visible from the Dbvserver location. If using a cluster or virtual IP, make sure the host name or alias specified here resolves to the IP address local to where the Dbvagent is installed. The host name should resolve to an IPv4 address, otherwise you can use an IPv4 IP address instead of a host name. Enter a custom value or press ENTER to accept default [kiwi81]: > kiwi812-vip >>> Please specify the listening PORT number for Dbvagent. The Dbvisit Agent (Dbvagent) will be listening on the specified port for incoming requests from Dbvserver (GUI). Please make sure that this port is not already in use, and is not blocked by any firewall. You may choose any value between 1024 and 65535, however the default of 7891 is recommended. Enter a custom value or press ENTER to accept default [7891]: > >>> Please specify the passphrase for Dbvagent Each Dbvisit Agent must have a passphrase specified. Think of this passphrase as the password for Dbvserver (GUI) to access each Dbvagent securely. The passphrase can be unique to each Dbvagent. It will be used to establish a secure connection between Dbvserver and the Dbvisit Agent. Enter a custom value: > kiwi123 >>> Please specify the IPv4 address or host name to be used for the Dbvserver Return Address (OPTIONAL) This is an advanced, and OPTIONAL parameter. If unsure, please accept the default value, which is blank. If necessary, you can use this parameter to specify a fixed IPv4 address for the Dbvisit Dbvserver (GUI). In the overwhelming majority of cases, our software will handle this for you automatically, and setting this parameter should NOT be necessary. If, however, your Dbvserver's externally-visible address differs from the locally-resolved address, you can specify it here. The most common scenario where this can happen is if you have address forwarding set up on a cloud-deployed environment. If set, Dbvagent will use this address instead of any other to connect to Dbvserver. Enter a custom value or press ENTER to accept default [ ]: > >>> Please specify the port number to be used as the Dbvserver Return Port (OPTIONAL) This is an advanced, and OPTIONAL parameter. This is the port value to match the OPTIONAL Dbvserver Return Address parameter immediately above. If unsure, please accept the default value, which is blank. Enter a custom value or press ENTER to accept default [ ]: > ----------------------------------------------------------- Summary of the Dbvisit DBVAGENT configuration ----------------------------------------------------------- DBVISIT_BASE /usr/dbvisit DBVAGENT_LOCAL_HOST kiwi812-vip DBVAGENT_LOCAL_PORT 7891 DBVAGENT_PASSPHRASE kiwi123 DBVAGENT_WEBSERVER_HOST DBVAGENT_WEBSERVER_PORT Press ENTER to continue ----------------------------------------------------------- About to install Dbvisit STANDBY ----------------------------------------------------------- NTF repo installed. Component standby installed. ----------------------------------------------------------- About to install Dbvisit DBVNET ----------------------------------------------------------- Component dbvnet installed. ----------------------------------------------------------- About to install Dbvisit DBVAGENT ----------------------------------------------------------- Component dbvagent installed. ----------------------------------------------------------- Component Installer Version Installed Version ----------------------------------------------------------- standby 10.0.0RC_24_g94ba1d85 10.0.0RC_24_g94ba1d85 dbvnet 10.0.0RC_24_g94ba1d85 (dbvnet v1.5.0-301-g3896e54)10.0.0RC_24_g94ba1d85 (dbvnet v1.5.0-301-g3896e54) dbvagent 10.0.0RC_24_g94ba1d85 10.0.0RC_24_g94ba1d85 dbvserver 10.0.0RC_24_g94ba1d85 not installed observer 10.0.0RC_24_g94ba1d85 not installed ----------------------------------------------------------- What action would you like to perform? 1 - Install component(s) 2 - Uninstall component(s) 3 - Exit Your choice: 3 >>> Installation completed Install log /tmp/dbvisit_install.log.202012141046.
The next step is to start the Dbvisit Components on kiwi1 - dbvnet and dbvagent:
oracle@kiwi81[/home/oracle/dbvisit/installer]: cd /usr/dbvisit/ oracle@kiwi81[/usr/dbvisit]: cd dbvnet/ oracle@kiwi81[/usr/dbvisit/dbvnet]: ./dbvnet -d start Dbvnet daemon started with pid 16889. oracle@kiwi81[/usr/dbvisit/dbvnet]: cd ../dbvagent/ oracle@kiwi81[/usr/dbvisit/dbvagent]: ./dbvagent -d start Dbvagent daemon started. oracle@kiwi81[/usr/dbvisit/dbvagent]: ps -ef |grep dbv oracle 16889 1 0 10:54 ? 00:00:00 ./dbvnet -d start oracle 16911 1 0 10:54 ? 00:00:00 ./dbvagent -d start oracle 17073 9702 0 10:55 pts/0 00:00:00 grep dbv oracle@kiwi81[/usr/dbvisit/dbvagent]:
2.2.1.2. Installing Dbvisit Standby on Standby Server (kiwi91)
Below is the output of the installation on the standby server.
IMPORTANT: In this example, the Central Console is installed on the Standby server. Ideally, it should be installed on its own server, but if you do not have a 3rd system available for the central console, you can install it on either the primary or standby - we recommend the standby.
oracle@kiwi91[/home/oracle/dbvisit/installer]: ./install-dbvisit ----------------------------------------------------------- Welcome to the Dbvisit software installer. ----------------------------------------------------------- It is recommended to make a backup of our current Dbvisit software location (Dbvisit Base location) for rollback purposes. Installer Directory /home/oracle/dbvisit >>> Please specify the Dbvisit installation directory (Dbvisit Base). The various Dbvisit products and components - such as Dbvisit Standby, Dbvisit Dbvnet will be installed in the appropriate subdirectories of this path. Enter a custom value or press ENTER to accept default [/usr/dbvisit]: > DBVISIT_BASE = /usr/dbvisit ----------------------------------------------------------- Component Installer Version Installed Version ----------------------------------------------------------- standby 10.0.0RC_24_g94ba1d85 not installed dbvnet 10.0.0RC_24_g94ba1d85 (dbvnet v1.5.0-301-g3896e54)not installed dbvagent 10.0.0RC_24_g94ba1d85 not installed dbvserver 10.0.0RC_24_g94ba1d85 not installed observer 10.0.0RC_24_g94ba1d85 not installed ----------------------------------------------------------- What action would you like to perform? 1 - Install component(s) 2 - Uninstall component(s) 3 - Exit Your choice: 1 Choose component(s): 1 - Core Components (Dbvisit Standby Cli, Dbvnet, Dbvagent) 2 - Dbvisit Standby Core (Command Line Interface) 3 - Dbvnet (Dbvisit Network Communication) 4 - Dbvagent (Dbvisit Agent) 5 - Dbvserver (Dbvisit Central Console) - Not available on Solaris/AIX 6 - Dbvisit Observer (Automatic Failover Option) - Not available on Solaris/AIX Press ENTER to exit Installer Your choice: 1 >>> Please specify the Oracle Instant Client version. The Oracle Instant Client is a required library for Dbvisit products. You may choose 12.1 or 12.2, however the default version is 12.2. Enter a custom value or press ENTER to accept default [12.2]: > ----------------------------------------------------------- Summary of the Dbvisit STANDBY configuration ----------------------------------------------------------- DBVISIT_BASE /usr/dbvisit INSTANT_CLIENT_VERSION 12.2 Press ENTER to continue ----------------------------------------------------------- About to configure DBVISIT DBVNET ----------------------------------------------------------- >>> Please specify the Local host name to be used by Dbvnet on this server. Dbvnet will be listening on the local IP Address on this server which resolve to the host name specified here. If using a cluster or virtual IP make sure the host name or alias specified here resolve to the IP address local to where dbvnet is installed. The host name should resolve to IPv4 address, if not you can use an IPv4 IP address instead of host name. Enter a custom value or press ENTER to accept default [kiwi91]: > kiwi912-vip >>> Please specify the Local Dbvnet PORT to be used. Dbvnet will be listening on the specified port for incoming connections from remote dbvnet connections. Please make sure that this port is not already in use or blocked by any firewall. You may choose any value between 1024 and 65535, however the default of 7890 is recommended. Enter a custom value or press ENTER to accept default [7890]: > >>> Please specify the Dbvnet Passphrase to be used for secure connections. The passphrase provided must be the same in both the local and remote Dbvnet installations. It is used to establish a secure (encrypted) Dbvnet connections Enter a custom value: > kiwi123 ----------------------------------------------------------- Summary of the Dbvisit DBVNET configuration ----------------------------------------------------------- DBVISIT_BASE /usr/dbvisit DBVNET_LOCAL_HOST kiwi912-vip DBVNET_LOCAL_PORT 7890 DBVNET_PASSPHRASE kiwi123 Press ENTER to continue ----------------------------------------------------------- About to configure DBVISIT DBVAGENT ----------------------------------------------------------- >>> Please specify the IPv4 address or host name to be used for the Dbvisit Agent. The Dbvisit Agent (Dbvagent) will be listening on this local address. Connections from Dbvserver (GUI) will be established to this Dbvagent, thus the address must be visible from the Dbvserver location. If using a cluster or virtual IP, make sure the host name or alias specified here resolves to the IP address local to where the Dbvagent is installed. The host name should resolve to an IPv4 address, otherwise you can use an IPv4 IP address instead of a host name. Enter a custom value or press ENTER to accept default [kiwi91]: > kiwi912-vip >>> Please specify the listening PORT number for Dbvagent. The Dbvisit Agent (Dbvagent) will be listening on the specified port for incoming requests from Dbvserver (GUI). Please make sure that this port is not already in use, and is not blocked by any firewall. You may choose any value between 1024 and 65535, however the default of 7891 is recommended. Enter a custom value or press ENTER to accept default [7891]: > >>> Please specify the passphrase for Dbvagent Each Dbvisit Agent must have a passphrase specified. Think of this passphrase as the password for Dbvserver (GUI) to access each Dbvagent securely. The passphrase can be unique to each Dbvagent. It will be used to establish a secure connection between Dbvserver and the Dbvisit Agent. Enter a custom value: > kiwi123 >>> Please specify the IPv4 address or host name to be used for the Dbvserver Return Address (OPTIONAL) This is an advanced, and OPTIONAL parameter. If unsure, please accept the default value, which is blank. If necessary, you can use this parameter to specify a fixed IPv4 address for the Dbvisit Dbvserver (GUI). In the overwhelming majority of cases, our software will handle this for you automatically, and setting this parameter should NOT be necessary. If, however, your Dbvserver's externally-visible address differs from the locally-resolved address, you can specify it here. The most common scenario where this can happen is if you have address forwarding set up on a cloud-deployed environment. If set, Dbvagent will use this address instead of any other to connect to Dbvserver. Enter a custom value or press ENTER to accept default [ ]: > >>> Please specify the port number to be used as the Dbvserver Return Port (OPTIONAL) This is an advanced, and OPTIONAL parameter. This is the port value to match the OPTIONAL Dbvserver Return Address parameter immediately above. If unsure, please accept the default value, which is blank. Enter a custom value or press ENTER to accept default [ ]: > ----------------------------------------------------------- Summary of the Dbvisit DBVAGENT configuration ----------------------------------------------------------- DBVISIT_BASE /usr/dbvisit DBVAGENT_LOCAL_HOST kiwi912-vip DBVAGENT_LOCAL_PORT 7891 DBVAGENT_PASSPHRASE kiwi123 DBVAGENT_WEBSERVER_HOST DBVAGENT_WEBSERVER_PORT Press ENTER to continue ----------------------------------------------------------- About to install Dbvisit STANDBY ----------------------------------------------------------- NTF repo installed. Component standby installed. ----------------------------------------------------------- About to install Dbvisit DBVNET ----------------------------------------------------------- Component dbvnet installed. ----------------------------------------------------------- About to install Dbvisit DBVAGENT ----------------------------------------------------------- Component dbvagent installed. ----------------------------------------------------------- Component Installer Version Installed Version ----------------------------------------------------------- standby 10.0.0RC_24_g94ba1d85 10.0.0RC_24_g94ba1d85 dbvnet 10.0.0RC_24_g94ba1d85 (dbvnet v1.5.0-301-g3896e54)10.0.0RC_24_g94ba1d85 (dbvnet v1.5.0-301-g3896e54) dbvagent 10.0.0RC_24_g94ba1d85 10.0.0RC_24_g94ba1d85 dbvserver 10.0.0RC_24_g94ba1d85 not installed observer 10.0.0RC_24_g94ba1d85 not installed ----------------------------------------------------------- What action would you like to perform? 1 - Install component(s) 2 - Uninstall component(s) 3 - Exit Your choice: 1 Choose component(s): 1 - Core Components (Dbvisit Standby Cli, Dbvnet, Dbvagent) 2 - Dbvisit Standby Core (Command Line Interface) 3 - Dbvnet (Dbvisit Network Communication) 4 - Dbvagent (Dbvisit Agent) 5 - Dbvserver (Dbvisit Central Console) - Not available on Solaris/AIX 6 - Dbvisit Observer (Automatic Failover Option) - Not available on Solaris/AIX Press ENTER to exit Installer Your choice: 5 ----------------------------------------------------------- About to configure DBVISIT DBVSERVER ----------------------------------------------------------- >>> Please specify the IPv4 address to be used for Dbvserver The Dbvisit Web Server (Dbvserver) will be listening on this local address. If using a cluster or virtual IP, make sure that the host name or alias specified here resolve to the IP address local to where Dbvserver is installed. It is recommended that you set this parameter to an IP address, as opposed to a host name. If you are unsure about the remote host name during installation, use the default value, which will be the current local hostname. Enter a custom value or press ENTER to accept default [kiwi91]: > 192.168.56.91 >>> Please specify the listening port number for Dbvserver on the local server You may choose any value between 1024 and 65535. The default recommended value is 4433. Note: if you can not access this port after the installation has finished, then please double-check your server firewall settings to ensure the selected port is open. Enter a custom value or press ENTER to accept default [4433]: > >>> Please specify the IPv4 address to be used for the Dbvserver public interface The public interface address will be used by Dbvagents for sending back information, e.g. Task progress. This should be set to the externally-visible IP address of the local host. Enter a custom value or press ENTER to accept default [192.168.56.91]: > >>> Please specify a publicly-available port number for Dbvserver. Like the Public Address setting previously, this port should be remotely accessible, and can potentially be different from the Dbvserver Listening Port, especially if your server is behind a DMZ. If you are unsure of the correct value for this parameter, it is usually fine to leave it the same as the Listener Port setting. You may choose any value between 1024 and 65535. The default recommended value is 4433. Note: if you can not access this port after the installation has finished, then please double-check your server firewall settings to ensure the selected port is open. Enter a custom value or press ENTER to accept default [4433]: > ----------------------------------------------------------- Summary of the Dbvisit DBVSERVER configuration ----------------------------------------------------------- DBVISIT_BASE /usr/dbvisit DBVSERVER_LOCAL_HOST 192.168.56.91 DBVSERVER_LOCAL_PORT 4433 DBVSERVER_PUBLIC_HOST 192.168.56.91 DBVSERVER_PUBLIC_PORT 4433 Press ENTER to continue ----------------------------------------------------------- About to install Dbvisit DBVSERVER ----------------------------------------------------------- Component dbvserver installed. ----------------------------------------------------------- Component Installer Version Installed Version ----------------------------------------------------------- standby 10.0.0RC_24_g94ba1d85 10.0.0RC_24_g94ba1d85 dbvnet 10.0.0RC_24_g94ba1d85 (dbvnet v1.5.0-301-g3896e54)10.0.0RC_24_g94ba1d85 (dbvnet v1.5.0-301-g3896e54) dbvagent 10.0.0RC_24_g94ba1d85 10.0.0RC_24_g94ba1d85 dbvserver 10.0.0RC_24_g94ba1d85 10.0.0RC_24_g94ba1d85 observer 10.0.0RC_24_g94ba1d85 not installed ----------------------------------------------------------- What action would you like to perform? 1 - Install component(s) 2 - Uninstall component(s) 3 - Exit Your choice: 3 >>> Installation completed Install log /tmp/dbvisit_install.log.202012141048.
Following the installation Start the Dbvisit Standby components: dbvnet, dbvagent, dbvserver:
oracle@kiwi91[/home/oracle/dbvisit/installer]: cd /usr/dbvisit/ oracle@kiwi91[/usr/dbvisit]: cd dbvnet/ oracle@kiwi91[/usr/dbvisit/dbvnet]: ./dbvnet -d start Dbvnet daemon started with pid 21029. oracle@kiwi91[/usr/dbvisit/dbvnet]: cd ../dbvagent/ oracle@kiwi91[/usr/dbvisit/dbvagent]: ./dbvagent -d start Dbvagent daemon started. oracle@kiwi91[/usr/dbvisit/dbvagent]: cd ../dbvserver/ oracle@kiwi91[/usr/dbvisit/dbvserver]: ./dbvserver -d start Dbvserver daemon started. oracle@kiwi91[/usr/dbvisit/dbvserver]: ps -ef |grep dbv oracle 21029 1 0 10:57 ? 00:00:00 ./dbvnet -d start oracle 21153 1 0 10:57 ? 00:00:00 ./dbvagent -d start oracle 21169 1 0 10:57 ? 00:00:00 ./dbvserver -d start oracle 21177 11186 0 10:57 pts/0 00:00:00 grep dbv oracle@kiwi91[/usr/dbvisit/dbvserver]:
2.2.2. Creating a DDC File
There are two options now available to you for creating the DDC file, using the Command Line Interface (CLI) or the Central Console (GUI)
2.2.2.1. Using the Command Line Interface
The DDC file creation must be run from the Primary server (kiwi1) in this example.
The command used is: ./dbvctl -o setup --noprompt
oracle@kiwi81[/usr/dbvisit/standby]: ./dbvctl -o setup ========================================================= Dbvisit Standby Database Technology (10.0.0RC_24_g94ba1d85) http://www.dbvisit.com ========================================================= =>dbvctl only needs to be run on the primary server. Is this the primary server? <Yes/No> [Yes]: The following Dbvisit Database configuration (DDC) file(s) found on this server: DDC === 1) Create New DDC 2) Cancel Please enter choice [] : 1 Is this correct? <Yes/No> [Yes]: END USER LICENSE AGREEMENT PLEASE READ THIS END USER LICENSE AGREEMENT (AGREEMENT) CAREFULLY BEFORE DOWNLOADING THE LICENSED PRODUCT. BY CLICKING I AGREE BELOW, YOU ARE REPRESENTING THAT YOU HAVE THE RIGHT AND AUTHORITY TO LEGALLY BIND YOURSELF OR YOUR COMPANY, AS APPLICABLE AND YOU ARE AGREEING TO BE BOUND BY THE .... .... <<------- These lines have been removed to make the output easier to read .... all other provisions shall remain in full force and effect. Waiver of any provision of this Agreement in one instance shall not preclude enforcement of it on future occasions. Headings are for reference purposes only and have no substantive effect. Continue ? <Yes/No> [No]: Yes ========================================================= Dbvisit Standby setup begins. ========================================================= The following Oracle instance(s) have been found on this server: SID ORACLE_HOME === =========== 1) MYDEV1 /u01/app/oracle/product/11.2.0/db_1 2) MYDEV /u01/app/oracle/product/11.2.0/db_1 3) Enter own ORACLE_SID and ORACLE_HOME Please enter choice [] : 1 Is this correct? <Yes/No> [Yes]: Yes =>ORACLE_SID will be: MYDEV1 =>ORACLE_HOME will be: /u01/app/oracle/product/11.2.0/db_1 ------------------------------------------------------------------------------ Enter the the virtual hostname (linked to a Virtual IP) for primary database. This virtual hostname is attached to a VIP that will be able to move between the RAC nodes. Dbvisit Standby will only run on the server where this virtual hostname (VIP) resource is running. The Virtual Hostname (and VIP) should be configured as a resouce in clusterware and must be enabled and started on one of the nodes before yo continue with the setup. NOTE: If you are not using a Virtual Hostname (Highly Recommended), Dbvisit Standby can only be configured to run on one dedicated node in the RAC configuration - specify the hosts name here if you do not have a Virtual Hostname (attached to VIP) for the cluster configured. =>SOURCE is []: kiwi812-vip Your input: kiwi812-vip Is this correct? <Yes/No> [Yes]: Choice is kiwi812-vip ------------------------------------------------------------------------------ Please enter a filesystem directory that Dbvisit Standby use to store (archive) log files. This directory is not the same as the database recovery area or archive destinations and should not be located in these areas. The ARCHSOURCE directory is located on the primary server and will become the ARCHDEST location when the primary database is converted to a standby database during a Graceful Switchover operation. This directory will ONLY contain (archive) log files related to this database. It should not contain any other (non archive log) files. Please ensure that this directory exists on =>ARCHSOURCE is [/u01/app/oracle/dbvisit_arch/MYDEV]: /acfs/dbvisit_arch/MYDEV Your input: /acfs/dbvisit_arch/MYDEV Is this correct? <Yes/No> [Yes]: Choice is /acfs/dbvisit_arch/MYDEV ------------------------------------------------------------------------------ Enter primary Oracle database instance for thread 1 =>RAC1_SID is [MYDEV1]: Your input: MYDEV1 Is this correct? <Yes/No> [Yes]: Choice is MYDEV1 ------------------------------------------------------------------------------ Enter primary host name for thread 1 =>RAC1_HOST is [kiwi81]: Your input: kiwi81 Is this correct? <Yes/No> [Yes]: Choice is kiwi81 ------------------------------------------------------------------------------ Enter primary ASM instance for thread 1. Leave null for ACFS =>RAC1_SID_ASM is [+ASM1]: Your input: +ASM1 Is this correct? <Yes/No> [Yes]: Choice is +ASM1 ------------------------------------------------------------------------------ Enter primary Oracle database instance for thread 2 =>RAC2_SID is [MYDEV2]: Your input: MYDEV2 Is this correct? <Yes/No> [Yes]: Choice is MYDEV2 ------------------------------------------------------------------------------ Enter primary host name for thread 2 =>RAC2_HOST is [kiwi82]: Your input: kiwi82 Is this correct? <Yes/No> [Yes]: Choice is kiwi82 ------------------------------------------------------------------------------ Enter primary ASM instance for thread 2. Leave null for ACFS =>RAC2_SID_ASM is [+ASM2]: Your input: +ASM2 Is this correct? <Yes/No> [Yes]: Choice is +ASM2 ------------------------------------------------------------------------------ Will the standby database be a RAC database? =>RAC_DR is [N]: Your input: N Is this correct? <Yes/No> [Yes]: Choice is N ------------------------------------------------------------------------------ Do you want to use SSH to connect to the standby server? Note that if you are using SSH, passwordless SSH authentication between the hosts must already be configured. By default Dbvnet will be used. =>USE_SSH is [N]: Your input: N Is this correct? <Yes/No> [Yes]: Choice is N ------------------------------------------------------------------------------ Specify the DBVNET or SSH port number on the primary server. The default value supplied is the dbvnet port 7890. If you specified the use of SSH, please specify the SSH port here. =>NETPORT is [7890]: Your input: 7890 Is this correct? <Yes/No> [Yes]: Choice is 7890 ------------------------------------------------------------------------------ Enter the standby database hostname. If the standby database will be Oracle RAC enabled: Enter the the Virtual Hostname (linked to a Virtual IP) for standby database. This virtual hostname is attached to a VIP that will be able to move between the RAC nodes. Dbvisit Standby will only run on the server where this virtual hostname (VIP) resource is running. The Virtual Hostname (and VIP) should be configured as a resouce in clusterware and must be enabled and started on one of the nodes before yo continue with the setup. If you are not using a Virtual Hostname (Highly Recommended), Dbvisit Standby can only be configured to run on one dedicated node in the RAC configuration - specify the hosts name here if you do not have a Virtual Hostname (attached to VIP) for the cluster configured. For non-RAC configurations specify the standby database server name here. =>DESTINATION is []: kiwi91 Your input: kiwi91 Is this correct? <Yes/No> [Yes]: Choice is kiwi91 ------------------------------------------------------------------------------ Specify the DBVNET or SSH port number on the standby server. The default value supplied is the dbvnet port 7890. If you specified the use of SSH, please specify the SSH port here. =>NETPORT_DR is [7890]: Your input: 7890 Is this correct? <Yes/No> [Yes]: Choice is 7890 ------------------------------------------------------------------------------ Enter Dbvisit Standby installation directory on the standby server =>DBVISIT_BASE_DR is [/usr/dbvisit]: Your input: /usr/dbvisit Is this correct? <Yes/No> [Yes]: Choice is /usr/dbvisit ------------------------------------------------------------------------------ Enter ORACLE_HOME directory on the standby server =>ORACLE_HOME_DR is [/u01/app/oracle/product/11.2.0/db_1]: Your input: /u01/app/oracle/product/11.2.0/db_1 Is this correct? <Yes/No> [Yes]: Choice is /u01/app/oracle/product/11.2.0/db_1 ------------------------------------------------------------------------------ Enter DB_UNIQUE_NAME on the standby server =>DB_UNIQUE_NAME_DR is [MYDEV]: Your input: MYDEV Is this correct? <Yes/No> [Yes]: Choice is MYDEV ------------------------------------------------------------------------------ Please enter the directory where Dbvisit Standby will transfer the (archive) log files to on standby server. This directory is not the same as the database recovery area or archive destinations and should not be located in these areas. This directory should ONLY contain (archive) log files related to this database. It should not contain any other (non archive log) files. Please ensure that this directory exists on the standby server =>ARCHDEST is [/acfs/dbvisit_arch/MYDEV]: /u01/app/oracle/dbvisit_arch/MYDEV Your input: /u01/app/oracle/dbvisit_arch/MYDEV Is this correct? <Yes/No> [Yes]: Choice is /u01/app/oracle/dbvisit_arch/MYDEV ------------------------------------------------------------------------------ Enter ORACLE_SID on the standby server =>ORACLE_SID_DR is [MYDEV1]: MYDEV Your input: MYDEV Is this correct? <Yes/No> [Yes]: Choice is MYDEV ------------------------------------------------------------------------------ Enter ASM instance name on the standby server, if your standby is using ASM. If you are not using ASM on the standby or standby is ACFS leave the value null. =>ORACLE_SID_ASM_DR is []: +ASM1 Your input: +ASM1 Is this correct? <Yes/No> [Yes]: Choice is +ASM1 ------------------------------------------------------------------------------ Please specify the name of the Dbvisit Database configuration (DDC) file. The DDC file is a plain text file that contains all the Dbvisit Standby settings. =>ENV_FILE is [MYDEV]: Your input: MYDEV Is this correct? <Yes/No> [Yes]: Choice is MYDEV ------------------------------------------------------------------------------ Provide license key. =>LICENSE_KEY is []: 4jo70-qwp4l-7gplh-g1cgo-7yz30-2hj2j-tdx6y Your input: 4jo70-qwp4l-7gplh-g1cgo-7yz30-2hj2j-tdx6y Is this correct? <Yes/No> [Yes]: Choice is 4jo70-qwp4l-7gplh-g1cgo-7yz30-2hj2j-tdx6y ------------------------------------------------------------------------------ Below are the list of configuration variables provided during the setup process: Configuration Variable Value Provided ====================== ============== ORACLE_SID MYDEV1 ORACLE_HOME /u01/app/oracle/product/11.2.0/db_1 SOURCE kiwi812-vip ARCHSOURCE /acfs/dbvisit_arch/MYDEV RAC1_THREAD 1 RAC1_SID MYDEV1 RAC1_HOST kiwi81 RAC1_SID_ASM +ASM1 RAC2_THREAD 2 RAC2_SID MYDEV2 RAC2_HOST kiwi82 RAC2_SID_ASM +ASM2 RAC_DR N USE_SSH N NETPORT 7890 DESTINATION kiwi91 NETPORT_DR 7890 DBVISIT_BASE_DR /usr/dbvisit ORACLE_HOME_DR /u01/app/oracle/product/11.2.0/db_1 DB_UNIQUE_NAME_DR MYDEV ARCHDEST /u01/app/oracle/dbvisit_arch/MYDEV ORACLE_SID_DR MYDEV ORACLE_SID_ASM_DR +ASM1 ENV_FILE MYDEV LICENSE_KEY 4jo70-qwp4l-7gplh-g1cgo-7yz30-2hj2j-tdx6y Are these variables correct? <Yes/No> [Yes]: Yes >>> Dbvisit Database configuration (DDC) file MYDEV created. >>> Dbvisit Database repository (DDR) MYDEV created. Repository Version 9.2 Software Version 9.2 Repository Status VALID PID:19715 TRACE:dbvisit_install.log oracle@kiwi81[/usr/dbvisit/standby/conf]: ls -lrt total 188 -rw-r--r-- 1 oracle oinstall 8861 Dec 14 10:46 dbv_message.html -rw-r--r-- 1 oracle oinstall 8872 Dec 14 10:46 dbv_error.html -rw-r--r-- 1 oracle oinstall 28672 Dec 14 11:00 dbv_notify.db -rw-r--r-- 1 oracle oinstall 26195 Dec 14 11:05 dbv_MYDEV.env -rw-r--r-- 1 oracle oinstall 102400 Dec 14 11:05 mydev.db oracle@kiwi81[/usr/dbvisit/standby/conf]:
From the above example, we can see that we now have a DDC file called "dbv_MYDEV.env" and a repository for this DDC file called "mydev.db".
2.2.2.2. Using the Central Console
Below are the screenshots showing you how you can use the Central Console to create the DDC file for this configuration:
Step 1: Open Main Screen, and navigate to manage hosts and first add the two hosts (dbv-vip and kiwi221)
Step 2: Add a new host
Step 3: Add the RAC node 1 (kiwi18) - but not the host details - you must use the RAC VIP that we assigned - this allows us to move to kiwi2 without having to update names as we use the Virtual IP (kiwi812-vip) which moves between them.
Step 4: Review host and click add new to add standby host - kiwi91
Step 5: Add kiwi91 host
Step 6: Review hosts
Step 7: Add new DDC → navigate to Manage Configurations
Step 8: Start new DDC process by selecting "NEW"
Step 9: Fill in the required details:
Once the details are provided, click on Submit and the DDC file will be created.
Step 10: Review the DDC file
Step 11: Make sure you apply your License key for this newly created DDC before you move onto the Create Standby Database (CSD), this can be done while creating the DDC itself.
2.2.3. Creating the Standby Database (CSD)
This example will only show using the Central Console.
It is possible to create the standby database either via the Command Line Interface (CLI) using the command: ./dbvctl -d <DDC> --csd in this example: "./dbvctl -d DEV --csd"
To use the Central Console, navigate to the Central Console page and follow the steps as outlined below:
Step 1: Navigate to the Create Standby Database (CSD) Menu
Step 2.1: Select the DDC, Create New Database option and update the Database SPFILE parameters to match the Standby server configuration:
In this example, the standby is Filesystem Based Storage
As we are using OMF - Oracle Managed Files on ASM (Which is recommended) we have to adjust the OMF parameters according - see 3 and 4 as marked below:
Step 2.2: Review Temporary Backup Locations - Make sure that this location has sufficient space for a full RMAN compressed backup of the database. In this example /usr/tmp has sufficient space on both the primary and standby.
2.2.4. Initial Send/Apply Logs
This section will show you how you can send/apply logs following the CSD process.
This can be done either via the Command Line Interface or the Central Console.
2.2.4.1. Using the Command Line Interface (Send/Apply)
From the primary node running the Dbvisit Standby processes (kiwi81)
oracle@kiwi81[/usr/dbvisit/standby]: ./dbvctl -d MYDEV ============================================================= Dbvisit Standby Database Technology (10.0.0RC_24_g94ba1d85) (pid 5004) dbvctl started on kiwi812-vip: Mon Dec 14 11:50:27 2020 ============================================================= >>> Obtaining information from standby database (RUN_INSPECT=Y)... done Thread: 1 Archive log gap: 0. Transfer log gap: 0 Thread: 2 Archive log gap: 1. Transfer log gap: 0 Note FORCE_LOGGING is disabled in the primary database. >>> Sending heartbeat message... skipped >>> Performing a log switch... done >>> Transferring Log file(s) from MYDEV1 on kiwi812-vip to kiwi91: thread 1 sequence 103 (thread_1_seq_103.366.1059133841)... done thread 2 sequence 71 (thread_2_seq_71.367.1059133839)... done ============================================================= dbvctl ended on kiwi812-vip: Mon Dec 14 11:50:45 2020 =============================================================
oracle@kiwi91[/usr/dbvisit/standby]: ./dbvctl -d MYDEV ============================================================= Dbvisit Standby Database Technology (10.0.0RC_24_g94ba1d85) (pid 15233) dbvctl started on kiwi91: Mon Dec 14 11:51:27 2020 ============================================================= >>> Applying Log file(s) from kiwi812-vip to MYDEV on kiwi91: thread 1 sequence 103 (1_103_1009025610.arc)... done thread 2 sequence 70 (2_70_1009025610.arc)... done thread 2 sequence 71 (2_71_1009025610.arc)... done thread 1 sequence 103 (1_103_1009025610.arc)... done Last applied log(s): thread 1 sequence 103 thread 2 sequence 71 Next SCN required for recovery 1809756 generated at 2020-12-14:11:50:38 +13:00. Next required log thread 2 sequence 72 ============================================================= dbvctl ended on kiwi91: Mon Dec 14 11:51:32 2020 =============================================================
The same command: "./dbvctl -d DEV" can be executed on the standby server kiwi221 to apply the logs already shipped.
Running a log gap report is done from the primary node (kiwi81). Here is an example:
oracle@kiwi81[/usr/dbvisit/standby]: ./dbvctl -d MYDEV -i ============================================================= Dbvisit Standby Database Technology (10.0.0RC_24_g94ba1d85) (pid 6069) dbvctl started on kiwi812-vip: Mon Dec 14 11:52:37 2020 ============================================================= Dbvisit Standby log gap report for MYDEV at 202012141152: ------------------------------------------------------------- Description | SCN | Timestamp ------------------------------------------------------------- Source 1809860 2020-12-14:11:52:39 +13:00 Destination 1809756 2020-12-14:11:50:38 +13:00 Standby database time lag (DAYS-HH:MI:SS): +00:02:01 Report for Thread 1 ------------------- SOURCE Current Sequence 104 Last Archived Sequence 103 Last Transferred Sequence 103 Last Transferred Timestamp 2020-12-14 11:50:44 DESTINATION Next Required Recovery Sequence 103 Transfer Log Gap 0 Apply Log Gap 1 Report for Thread 2 ------------------- SOURCE Current Sequence 72 Last Archived Sequence 71 Last Transferred Sequence 71 Last Transferred Timestamp 2020-12-14 11:50:45 DESTINATION Next Required Recovery Sequence 72 Transfer Log Gap 0 Apply Log Gap 0 ============================================================= dbvctl ended on kiwi812-vip: Mon Dec 14 11:52:43 2020 =============================================================
2.2.4.2. Using the Central Console (Send/Apply)
Step 1: Navigate to Manage Databases
Step 2: Select the DDC, click on [2] Send Logs icon and an Active Task will appear [3] which will turn to a green tick when done. View the details by clicking on the task [3] and you will see the details [4]
Step 3: Select the DDC, click on the Apply Logs icon [2]. The active task will be created [3] and when selected the details will be displayed [4]
2.2.5. Use "rsync" to sync local storage on RAC nodes
The next step in this example is to use the UNIX "rsync" utility to copy the files located in the /usr/dbvisit directory (The DBVISIT_BASE) from the "preferred node" which is the node where Dbvisit Standby normally is set to run, which in this example is "kiwi81". It will be set to copy the files from this location to the secondary node kiwi82 in this example. From kiwi82 there will be no Dbvisit Standby processes running as the Virtual IP (dbv-vip) will be running on kiwi1.
You will only start the Dbvisit Processes on kiwi2 if the Virtual IP cluster resource (kiwi812-vip) has been switched over - (re-allocated or failed over) to run on kiwi82. For more detail on the configuration of the VIP and the Action Scripts for Dbvnet and Dbvagent please see here - Oracle RAC Configurations#OracleRACConfigurations-5.Example2:AddDbvnetandDbvagentasClusterResources
Note: The /usr/dbvisit directory (The Dbvisit Base) location must exist on both Primary RAC nodes and that the permission of this folder is set to have the "oracle" Unix account (the user that runs your database software) as the owner. At this stage, the folder on the secondary server (kiwi82) will be empty as we have not synced the files yet.
Step 1: Create a "rsync" script
The first step now is to create a script that will be used to copy the files from node 1 (kiwi1 in this example) to node 2 (kiwi2).
IMPORTANT
Please note that this example of showing how "rsync" can be used.
These steps and scripts must be tested and modified to suit your environment prior to implementing it into production.
Make sure that you disable the rsync schedule prior to running a Graceful Switchover - it can be re-enabled again after that.
Sample "rsync-dbvisit" script
Note - this script can be copied into a folder /home/oracle/bin and enabled via a UNIX CRON schedule to run on a regular basis.
#!/bin/bash # # Note this is a sample script and it must be modified to suit your environment. # Test this script before implementing into a production environment. # Basic script to sync /usr/dbvisit local with remote /usr/dbvisit directory. # This script: rsync-dbvisit will be located in /home/oracle/bin folder. # # remote_server=kiwi82 dbv_base_log=/tmp/rsync-dbvisit-base.log dbv_arch_log=/tmp/rsync-dbvisit-arch.log echo "Start Sync: `date`" > ${dbv_base_log} ## Run the rsync command over SSH ## options used are to preservice times, have a 2min i/o timout and to exclude trace files ## rsync -aqzte ssh --exclude '*.trc' --log-file=${dbv_base_log} /usr/dbvisit/ oracle@${remote_server}:/usr/dbvisit/ echo "End Sync: `date`" >> ${dbv_base_log} echo "Start Sync: `date`" > ${dbv_arch_log} ## We also need to sync the ARCHSOURCE and ARCHDEST location if on local storage and not shared rsync -aqzte ssh --exclude '*.trc' --log-file=${dbv_arch_log} /u01/app/oracle/dbvisit_arch/MYDEV/ oracle@${remote_server}:/u01/app/oracle/dbvisit_arch/DEV/ echo "End Sync: `date`" >> ${dbv_arch_log}
Once you have created this script, make sure that it has sufficient permissions and that the owner "oracle" user, in this case, have a execute permission (example "chmod +x oracle rsync-dbvisit" ). For more detail please see the "chmod" command man pages or online help for your UNIX distribution.
The above script is making use of the UNIX "rsync" command. For more details on the use of this command and all parameters please see the UNIX man pages or online documentation.
Once you have created the script you can run it manually the first time to ensure the directories are kept in sync.
oracle@kiwi81[/home/oracle]: cd bin oracle@kiwi81[/home/oracle/bin]: ./rsync-dbvisit
Then review the log file in /tmp/dbvisit-rsync.log as well as the /usr/dbvisit directory on the secondary server (kiwi2 in this example)
Step 2: Schedule the Script
In this example, we will schedule the script to run every 10 minutes. This should be sufficient for most configurations.
The schedule can be adjusted if required based on your configuration and environment.
## # CRON: Synchronize the Dbvisit Base folder between kiwi81 and kiwi82 ## */10 * * * * cd /home/oracle/bin; ./rsync-dbvisit >>/dev/null 2>&1
To schedule, a cron use the "crontab -e" command and to view any existing schedules use "crontab -l"
Step 3: Monitor
Have Dbvisit Standby run either via a schedule or via the Daemon process for a period and then monitor the /usr/dbvisit folder on the standby server.
You should see files being updated in the secondary server following a run of the above schedule.
2.2.6. Next Step - Scheduling
The next step is to schedule the Dbvisit Standby or to run the Daemon process.
Both methods can be used.
Example if you want to run the Schedule using the Unix Cron, you can use the following example CRON entries:
On the Primary (kiwi1)
## # CRON: Synchronize the Dbvisit Base folder between kiwi81 and kiwi82 ## */10 * * * * cd /home/oracle/bin; ./rsync-dbvisit >>/dev/null 2>&1 ## ## # Dbvisit Standby Scheduled to run every 5 min */5 * * * * cd /usr/dbvisit/standby; ./dbvctl -d MYDEV >>/dev/null 2>&1
On the Standby (kiwi91) - apply logs every 15min, this can provide a buffer (delay in applying logs) to possibly stop Human error (deletion of data etc).
## # Dbvisit Standby Scheduled to run every 15 min */15 * * * * cd /usr/dbvisit/standby; ./dbvctl -d MYDEV >>/dev/null 2>&1
For more detail on scheduling or running Dbvisit Standby as a background Daemon/Process please see here - Dbvisit Standby Scheduling
2.2.7. Example Graceful Switchover in this Configuration
This example is just to show Graceful Switchover that was performed on this environment.
Before the Graceful Switchover (GS) a log gap report was run to make sure that the primary and standby is up to date.
oracle@kiwi81[/usr/dbvisit/standby]: ./dbvctl -d MYDEV -o switchover ============================================================= Dbvisit Standby Database Technology (10.0.0RC_23_gca959724) (pid 29392) dbvctl started on kiwi812-vip: Mon Dec 14 17:01:10 2020 ============================================================= >>> Starting Switchover between kiwi812-vip and kiwi91 Running pre-checks ... done Pre processing ... done Processing primary ... done Processing standby ... done Converting standby ... done Converting primary ... done Completing ... done Synchronizing ... done Post processing ... done >>> Graceful switchover completed. Primary Database Server: kiwi91 Standby Database Server: kiwi812-vip >>> Dbvisit Standby can be run as per normal: dbvctl -d MYDEV PID:29392 TRACE:29392_dbvctl_switchover_MYDEV_202012141701.trc ============================================================= dbvctl ended on kiwi812-vip: Mon Dec 14 17:06:53 2020 =============================================================
Following this process, kiwi91 is now a single instance primary with kiwi81/kiwi82 as the standby database.
3. Example 2: Oracle RAC primary to Oracle RAC standby
In the example we will discuss in this section we will explain the configuration of a Standby Oracle RAC environment for a Primary Oracle RAC environment.
The Diagram below provide a summary of the environment:
3.1. The Primary Oracle RAC Configuration - Overview
In the example the primary RAC configuration exists out of two Nodes:
- Primary node 1: kiwi81
- Primary node 2: kiwi82
The IP address details are listed below:
## Public IP addresses 10.0.2.81 kiwi81.oraclekiwi.co.nz kiwi81 10.0.2.82 kiwi82.oraclekiwi.co.nz kiwi82 ## Virtual IP Addresses 10.0.2.83 kiwi81-vip.oraclekiwi.co.nz kiwi81-vip 10.0.2.84 kiwi82-vip.oraclekiwi.co.nz kiwi82-vip ## Private Interconnect Addresses 10.5.5.81 kiwi81-priv.oraclekiwi.co.nz kiwi81-priv 10.5.5.82 kiwi82-priv.oraclekiwi.co.nz kiwi82-priv ## SCAN address list in DNS Only # 10.0.2.85 kiwi812-scan.oraclekiwi.co.nz kiwi812-scan # 10.0.2.86 kiwi812-scan.oraclekiwi.co.nz kiwi812-scan # 10.0.2.87 kiwi812-scan.oraclekiwi.co.nz kiwi812-scan # Dbvisit Standby VIP 10.0.2.89 kiwi812-vip.oraclekiwi.co.nz kiwi812-vip
These details are added to the /etc/hosts file as well as to the DNS.
Having the values in the DNS is highly recommended.
The Oracle SCAN details must be in the DNS.
3.1.1. The new Virtual IP
Take note of the new Virtual IP that is specified. This new Virtual IP is created and will be assigned to one specific node at a time. But it can move between the RAC nodes. Dbvisit Standby version 10 will be configured to use this address as the "cluster address". That way we can decide on which node we want Dbvisit Standby components to run.
- Get new IP address on the public network (network used by Primary/Standby)
- Create new Cluster Resource – Virtual IP (as root user)
- Review Oracle Documentation on creating a Virtual IP!
The steps to create the Virtual IP address is listed below (Note these are executed as the root user)
# appvipcfg create -network=1 -ip=10.0.2.89 -vipname=kiwi812-vip -user=root # crsctl setperm resource kiwi812-vip -u user:oracle:r-x # crsctl setperm resource kiwi812-vip -u group:grid:r-x # crsctl start resource kiwi812-vip -n kiwi81 # crsctl status resource kiwi812-vip -p # crsctl relocate resource kiwi812-vip -n kiwi81
The last step is to make sure you add to the DNS: 10.0.2.89 kiwi812-vip
3.1.2. ASM Disk Groups
- +DATA
- Datafiles, Redo Logs, Controlfile
- +FRA
- Recovery Area, Redo Logs, Control file, Archivelogs
- +ACFS
- Used for /acfs a shared filesystem
3.1.3. Dbvisit Standby Installation
- DBVISIT_BASE : /acfs/dbvisit
- SOURCE: kiwi812-vip
- DESTINATION: kiwi912-vip
- ARCHSOURCE location : /acfs/dbvisit_ach/DEV
- ARCHDEST Location : /acfs/dbvisit_ach/DEV
3.1.4. Node 1 and 2 storage overview
Primary RAC Node 1: kiwi81 | Primary RAC Node 2: kiwi82 |
---|---|
oracle@kiwi81[/home/oracle]: df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg_oel6-lv_root 12G 8.2G 2.7G 76% / tmpfs 1.4G 117M 1.3G 9% /dev/shm /dev/sda1 976M 126M 784M 14% /boot /dev/mapper/vg_oel6-lv_u01 23G 13G 9.3G 57% /u01 tmpfs 2.0G 4.3M 2.0G 1% /tmp /dev/asm/acfsvol-260 9.0G 423M 8.6G 5% /acfs | oracle@kiwi82[/home/oracle]: df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg_oel6-lv_root 12G 7.6G 3.4G 70% / tmpfs 1.4G 118M 1.3G 9% /dev/shm /dev/sda1 976M 126M 784M 14% /boot /dev/mapper/vg_oel6-lv_u01 23G 14G 8.1G 63% /u01 tmpfs 2.0G 6.8M 2.0G 1% /tmp /dev/asm/acfsvol-260 9.0G 423M 8.6G 5% /acfs |
3.2. The Standby Oracle RAC Configuration - Overview
In the example the standby RAC configuration exists out of two Nodes:
- Standby node 1: kiwi91
- Standby node 2: kiwi92
The IP address details are listed below:
## Public IP addresses 10.0.2.91 kiwi91.oraclekiwi.co.nz kiwi91 10.0.2.92 kiwi92.oraclekiwi.co.nz kiwi92 ## Virtual IP Addresses 10.0.2.93 kiwi91-vip.oraclekiwi.co.nz kiwi91-vip 10.0.2.94 kiwi92-vip.oraclekiwi.co.nz kiwi92-vip ## Private Interconnect Addresses 10.5.5.91 kiwi91-priv.oraclekiwi.co.nz kiwi91-priv 10.5.5.92 kiwi92-priv.oraclekiwi.co.nz kiwi92-priv ## SCAN address list in DNS Only # 10.0.2.95 kiwi912-scan.oraclekiwi.co.nz kiwi912-scan # 10.0.2.96 kiwi912-scan.oraclekiwi.co.nz kiwi912-scan # 10.0.2.97 kiwi912-scan.oraclekiwi.co.nz kiwi912-scan # Dbvisit Standby VIP 10.0.2.99 kiwi912-vip.oraclekiwi.co.nz kiwi912-vip
- These details are added to the /etc/hosts file as well as to the DNS.
- Having the values in the DNS is highly recommended.
- The Oracle SCAN details must be in the DNS.
3.2.1. The new Virtual IP
Take note of the new Virtual IP that is specified. This new Virtual IP is created and will be assigned to one specific node at a time. But it can move between the RAC nodes. Dbvisit Standby version 10 will be configured to use this address as the "cluster address". That way we can decide on which node we want Dbvisit Standby components to run.
- Get new IP address on the public network (network used by Primary/Standby)
- Create new Cluster Resource – Virtual IP (as root user)
- Review Oracle Documentation on creating a Virtual IP!
The steps to create the Virtual IP address is listed below (Note these are executed as the root user)
# appvipcfg create -network=1 -ip=10.0.2.99 -vipname=kiwi912-vip -user=root # crsctl setperm resource kiwi912-vip -u user:oracle:r-x # crsctl setperm resource kiwi912-vip -u user:grid:r-x # crsctl start resource kiwi912-vip -n kiwi91 # crsctl status resource kiwi912-vip -p # crsctl relocate kiwi912-vip # crsctl relocate resource kiwi912-vip -n kiwi91
The last step is to make sure you add to the DNS: 10.0.2.99 kiwi912-vip
3.2.2. ASM Disk Groups
- +DATA
- Datafiles, Redo Logs, Controlfile
- +FRA
- Recovery Area, Redo Logs, Control file, Archivelogs
- +ACFS
- Used for /acfs a shared filesystem
3.2.3. Dbvisit Standby Installation
- DBVISIT_BASE : /acfs/dbvisit
- SOURCE: kiwi812-vip
- DESTINATION: kiwi912-vip
- ARCHSOURCE location : /acfs/dbvisit_ach/DEV
- ARCHDEST Location : /acfs/dbvisit_ach/DEV
3.2.4. Node 1 and 2 storage overview
Standby RAC Node 1: kiwi91 | Standby RAC Node 1: kiwi92 |
---|---|
oracle@kiwi91[/home/oracle]: df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg_oel6-lv_root 12G 8.4G 2.6G 77% / tmpfs 1.4G 133M 1.3G 10% /dev/shm /dev/sda1 976M 126M 784M 14% /boot /dev/mapper/vg_oel6-lv_u01 23G 14G 7.7G 65% /u01 /dev/asm/acfsvol-229 9.0G 390M 8.7G 5% /acfs | oracle@kiwi92[/home/oracle]: df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg_oel6-lv_root 12G 10G 889M 93% / tmpfs 1.4G 136M 1.3G 10% /dev/shm /dev/sda1 976M 126M 784M 14% /boot /dev/mapper/vg_oel6-lv_u01 23G 13G 9.2G 58% /u01 /dev/asm/acfsvol-229 9.0G 390M 8.7G 5% /acfs |
4. Example 2: The Core Component Installation
4.1. Primary Cluster - Node 1 (kiwi81)
- The primary cluster Virtual IP is 10.0.2.89 and is using the hostname alias kiwi812-vip. We will use this as the SOURCE address name for the Primary Cluster.
- At the time of installing the software, the Virtual IP was configured to run on node 1 - kiwi81.
- The steps below explain the installation of the Dbvisit Standby Core components - Dbvnet, Dbvagent and Dbvisit Standby CLI
- Dbvisit Standby components are installed onto the shared storage which is mounted under /acfs directory
- The directory /acfs/dbvisit and /acfs/dbvisit_arch is created and the "oracle" Unix account is made the owner
In this example, a basic passphrase of "kiwi123" is used for Dbvnet and the Dbvagent.
From Dbvisit Standby V 9.0.10 onwards, dbvnet installer prompt for remote hostname is removed.
Optional Dbvagent Parameter has been added. In most cases, the default value is recommended.oracle@kiwi81[/home/oracle/dbvisit/installer]: ./install-dbvisit ----------------------------------------------------------- Welcome to the Dbvisit software installer. ----------------------------------------------------------- It is recommended to make a backup of our current Dbvisit software location (Dbvisit Base location) for rollback purposes. Installer Directory /home/oracle/dbvisit >>> Please specify the Dbvisit installation directory (Dbvisit Base). The various Dbvisit products and components - such as Dbvisit Standby, Dbvisit Dbvnet will be installed in the appropriate subdirectories of this path. Enter a custom value or press ENTER to accept default [/usr/dbvisit]: > /acfs/dbvisit DBVISIT_BASE = /acfs/dbvisit ----------------------------------------------------------- Component Installer Version Installed Version ----------------------------------------------------------- standby 10.0.0RC_23_gca959724 not installed dbvnet 10.0.0RC_23_gca959724 (dbvnet v1.5.0-301-g3896e54)not installed dbvagent 10.0.0RC_23_gca959724 not installed dbvserver 10.0.0RC_23_gca959724 not installed observer 10.0.0RC_23_gca959724 not installed ----------------------------------------------------------- What action would you like to perform? 1 - Install component(s) 2 - Uninstall component(s) 3 - Exit Your choice: 1 Choose component(s): 1 - Core Components (Dbvisit Standby Cli, Dbvnet, Dbvagent) 2 - Dbvisit Standby Core (Command Line Interface) 3 - Dbvnet (Dbvisit Network Communication) 4 - Dbvagent (Dbvisit Agent) 5 - Dbvserver (Dbvisit Central Console) - Not available on Solaris/AIX 6 - Dbvisit Observer (Automatic Failover Option) - Not available on Solaris/AIX Press ENTER to exit Installer Your choice: 1 >>> Please specify the Oracle Instant Client version. The Oracle Instant Client is a required library for Dbvisit products. You may choose 12.1 or 12.2, however the default version is 12.2. Enter a custom value or press ENTER to accept default [12.2]: > ----------------------------------------------------------- Summary of the Dbvisit STANDBY configuration ----------------------------------------------------------- DBVISIT_BASE /acfs/dbvisit INSTANT_CLIENT_VERSION 12.2 Press ENTER to continue ----------------------------------------------------------- About to configure DBVISIT DBVNET ----------------------------------------------------------- >>> Please specify the Local host name to be used by Dbvnet on this server. Dbvnet will be listening on the local IP Address on this server which resolve to the host name specified here. If using a cluster or virtual IP make sure the host name or alias specified here resolve to the IP address local to where dbvnet is installed. The host name should resolve to IPv4 address, if not you can use an IPv4 IP address instead of host name. Enter a custom value or press ENTER to accept default [kiwi81]: > kiwi812-vip >>> Please specify the Local Dbvnet PORT to be used. Dbvnet will be listening on the specified port for incoming connections from remote dbvnet connections. Please make sure that this port is not already in use or blocked by any firewall. You may choose any value between 1024 and 65535, however the default of 7890 is recommended. Enter a custom value or press ENTER to accept default [7890]: > >>> Please specify the Dbvnet Passphrase to be used for secure connections. The passphrase provided must be the same in both the local and remote Dbvnet installations. It is used to establish a secure (encrypted) Dbvnet connections Enter a custom value: > kiwi1234 ----------------------------------------------------------- Summary of the Dbvisit DBVNET configuration ----------------------------------------------------------- DBVISIT_BASE /acfs/dbvisit DBVNET_LOCAL_HOST kiwi812-vip DBVNET_LOCAL_PORT 7890 DBVNET_PASSPHRASE kiwi1234 Press ENTER to continue ----------------------------------------------------------- About to configure DBVISIT DBVAGENT ----------------------------------------------------------- >>> Please specify the IPv4 address or host name to be used for the Dbvisit Agent. The Dbvisit Agent (Dbvagent) will be listening on this local address. Connections from Dbvserver (GUI) will be established to this Dbvagent, thus the address must be visible from the Dbvserver location. If using a cluster or virtual IP, make sure the host name or alias specified here resolves to the IP address local to where the Dbvagent is installed. The host name should resolve to an IPv4 address, otherwise you can use an IPv4 IP address instead of a host name. Enter a custom value or press ENTER to accept default [kiwi81]: > kiwi812-vip >>> Please specify the listening PORT number for Dbvagent. The Dbvisit Agent (Dbvagent) will be listening on the specified port for incoming requests from Dbvserver (GUI). Please make sure that this port is not already in use, and is not blocked by any firewall. You may choose any value between 1024 and 65535, however the default of 7891 is recommended. Enter a custom value or press ENTER to accept default [7891]: > >>> Please specify the passphrase for Dbvagent Each Dbvisit Agent must have a passphrase specified. Think of this passphrase as the password for Dbvserver (GUI) to access each Dbvagent securely. The passphrase can be unique to each Dbvagent. It will be used to establish a secure connection between Dbvserver and the Dbvisit Agent. Enter a custom value: > kiwi1234 >>> Please specify the IPv4 address or host name to be used for the Dbvserver Return Address (OPTIONAL) This is an advanced, and OPTIONAL parameter. If unsure, please accept the default value, which is blank. If necessary, you can use this parameter to specify a fixed IPv4 address for the Dbvisit Dbvserver (GUI). In the overwhelming majority of cases, our software will handle this for you automatically, and setting this parameter should NOT be necessary. If, however, your Dbvserver's externally-visible address differs from the locally-resolved address, you can specify it here. The most common scenario where this can happen is if you have address forwarding set up on a cloud-deployed environment. If set, Dbvagent will use this address instead of any other to connect to Dbvserver. Enter a custom value or press ENTER to accept default [ ]: > >>> Please specify the port number to be used as the Dbvserver Return Port (OPTIONAL) This is an advanced, and OPTIONAL parameter. This is the port value to match the OPTIONAL Dbvserver Return Address parameter immediately above. If unsure, please accept the default value, which is blank. Enter a custom value or press ENTER to accept default [ ]: > ----------------------------------------------------------- Summary of the Dbvisit DBVAGENT configuration ----------------------------------------------------------- DBVISIT_BASE /acfs/dbvisit DBVAGENT_LOCAL_HOST kiwi812-vip DBVAGENT_LOCAL_PORT 7891 DBVAGENT_PASSPHRASE kiwi1234 DBVAGENT_WEBSERVER_HOST DBVAGENT_WEBSERVER_PORT Press ENTER to continue ----------------------------------------------------------- About to install Dbvisit STANDBY ----------------------------------------------------------- NTF repo installed. Component standby installed. ----------------------------------------------------------- About to install Dbvisit DBVNET ----------------------------------------------------------- Component dbvnet installed. ----------------------------------------------------------- About to install Dbvisit DBVAGENT ----------------------------------------------------------- Component dbvagent installed. ----------------------------------------------------------- Component Installer Version Installed Version ----------------------------------------------------------- standby 10.0.0RC_23_gca959724 10.0.0RC_23_gca959724 dbvnet 10.0.0RC_23_gca959724 (dbvnet v1.5.0-301-g3896e54)10.0.0RC_23_gca959724 (dbvnet v1.5.0-301-g3896e54) dbvagent 10.0.0RC_23_gca959724 10.0.0RC_23_gca959724 dbvserver 10.0.0RC_23_gca959724 not installed observer 10.0.0RC_23_gca959724 not installed ----------------------------------------------------------- What action would you like to perform? 1 - Install component(s) 2 - Uninstall component(s) 3 - Exit Your choice: 3 >>> Installation completed Install log /tmp/dbvisit_install.log.202012161751.
4.2. Standby Cluster - Node 1 (kiwi91)
- The standby cluster Virtual IP is 10.0.2.99 and is using the hostname alias kiwi912-vip. We will use this as the SOURCE address name for the Primary Cluster.
- At the time of installing the software, the Virtual IP was configured to run on node 1 - kiwi91.
- The steps below explain the installation of the Dbvisit Standby Core components - Dbvnet, Dbvagent and Dbvisit Standby CLI
- Dbvisit Standby components are installed onto the shared storage which is mounted under /acfs directory
- The directory /acfs/dbvisit and /acfs/dbvisit_arch is created and the "oracle" Unix account is made the owner
- In this example, a basic passphrase of "kiwi123" is used for Dbvnet and the Dbvagent
oracle@kiwi91[/home/oracle/dbvisit/installer]: ./install-dbvisit ----------------------------------------------------------- Welcome to the Dbvisit software installer. ----------------------------------------------------------- It is recommended to make a backup of our current Dbvisit software location (Dbvisit Base location) for rollback purposes. Installer Directory /home/oracle/dbvisit >>> Please specify the Dbvisit installation directory (Dbvisit Base). The various Dbvisit products and components - such as Dbvisit Standby, Dbvisit Dbvnet will be installed in the appropriate subdirectories of this path. Enter a custom value or press ENTER to accept default [/usr/dbvisit]: > /acfs/dbvisit DBVISIT_BASE = /acfs/dbvisit ----------------------------------------------------------- Component Installer Version Installed Version ----------------------------------------------------------- standby 10.0.0RC_23_gca959724 not installed dbvnet 10.0.0RC_23_gca959724 (dbvnet v1.5.0-301-g3896e54)not installed dbvagent 10.0.0RC_23_gca959724 not installed dbvserver 10.0.0RC_23_gca959724 not installed observer 10.0.0RC_23_gca959724 not installed ----------------------------------------------------------- What action would you like to perform? 1 - Install component(s) 2 - Uninstall component(s) 3 - Exit Your choice: 1 Choose component(s): 1 - Core Components (Dbvisit Standby Cli, Dbvnet, Dbvagent) 2 - Dbvisit Standby Core (Command Line Interface) 3 - Dbvnet (Dbvisit Network Communication) 4 - Dbvagent (Dbvisit Agent) 5 - Dbvserver (Dbvisit Central Console) - Not available on Solaris/AIX 6 - Dbvisit Observer (Automatic Failover Option) - Not available on Solaris/AIX Press ENTER to exit Installer Your choice: 1 >>> Please specify the Oracle Instant Client version. The Oracle Instant Client is a required library for Dbvisit products. You may choose 12.1 or 12.2, however the default version is 12.2. Enter a custom value or press ENTER to accept default [12.2]: > ----------------------------------------------------------- Summary of the Dbvisit STANDBY configuration ----------------------------------------------------------- DBVISIT_BASE /acfs/dbvisit INSTANT_CLIENT_VERSION 12.2 Press ENTER to continue ----------------------------------------------------------- About to configure DBVISIT DBVNET ----------------------------------------------------------- >>> Please specify the Local host name to be used by Dbvnet on this server. Dbvnet will be listening on the local IP Address on this server which resolve to the host name specified here. If using a cluster or virtual IP make sure the host name or alias specified here resolve to the IP address local to where dbvnet is installed. The host name should resolve to IPv4 address, if not you can use an IPv4 IP address instead of host name. Enter a custom value or press ENTER to accept default [kiwi91]: > kiwi912-vip >>> Please specify the Local Dbvnet PORT to be used. Dbvnet will be listening on the specified port for incoming connections from remote dbvnet connections. Please make sure that this port is not already in use or blocked by any firewall. You may choose any value between 1024 and 65535, however the default of 7890 is recommended. Enter a custom value or press ENTER to accept default [7890]: > >>> Please specify the Dbvnet Passphrase to be used for secure connections. The passphrase provided must be the same in both the local and remote Dbvnet installations. It is used to establish a secure (encrypted) Dbvnet connections Enter a custom value: > kiwi1234 ----------------------------------------------------------- Summary of the Dbvisit DBVNET configuration ----------------------------------------------------------- DBVISIT_BASE /acfs/dbvisit DBVNET_LOCAL_HOST kiwi912-vip DBVNET_LOCAL_PORT 7890 DBVNET_PASSPHRASE kiwi1234 Press ENTER to continue ----------------------------------------------------------- About to configure DBVISIT DBVAGENT ----------------------------------------------------------- >>> Please specify the IPv4 address or host name to be used for the Dbvisit Agent. The Dbvisit Agent (Dbvagent) will be listening on this local address. Connections from Dbvserver (GUI) will be established to this Dbvagent, thus the address must be visible from the Dbvserver location. If using a cluster or virtual IP, make sure the host name or alias specified here resolves to the IP address local to where the Dbvagent is installed. The host name should resolve to an IPv4 address, otherwise you can use an IPv4 IP address instead of a host name. Enter a custom value or press ENTER to accept default [kiwi91]: > kiwi912-vip >>> Please specify the listening PORT number for Dbvagent. The Dbvisit Agent (Dbvagent) will be listening on the specified port for incoming requests from Dbvserver (GUI). Please make sure that this port is not already in use, and is not blocked by any firewall. You may choose any value between 1024 and 65535, however the default of 7891 is recommended. Enter a custom value or press ENTER to accept default [7891]: > >>> Please specify the passphrase for Dbvagent Each Dbvisit Agent must have a passphrase specified. Think of this passphrase as the password for Dbvserver (GUI) to access each Dbvagent securely. The passphrase can be unique to each Dbvagent. It will be used to establish a secure connection between Dbvserver and the Dbvisit Agent. Enter a custom value: > kiwi1234 >>> Please specify the IPv4 address or host name to be used for the Dbvserver Return Address (OPTIONAL) This is an advanced, and OPTIONAL parameter. If unsure, please accept the default value, which is blank. If necessary, you can use this parameter to specify a fixed IPv4 address for the Dbvisit Dbvserver (GUI). In the overwhelming majority of cases, our software will handle this for you automatically, and setting this parameter should NOT be necessary. If, however, your Dbvserver's externally-visible address differs from the locally-resolved address, you can specify it here. The most common scenario where this can happen is if you have address forwarding set up on a cloud-deployed environment. If set, Dbvagent will use this address instead of any other to connect to Dbvserver. Enter a custom value or press ENTER to accept default [ ]: > >>> Please specify the port number to be used as the Dbvserver Return Port (OPTIONAL) This is an advanced, and OPTIONAL parameter. This is the port value to match the OPTIONAL Dbvserver Return Address parameter immediately above. If unsure, please accept the default value, which is blank. Enter a custom value or press ENTER to accept default [ ]: > ----------------------------------------------------------- Summary of the Dbvisit DBVAGENT configuration ----------------------------------------------------------- DBVISIT_BASE /acfs/dbvisit DBVAGENT_LOCAL_HOST kiwi912-vip DBVAGENT_LOCAL_PORT 7891 DBVAGENT_PASSPHRASE kiwi1234 DBVAGENT_WEBSERVER_HOST DBVAGENT_WEBSERVER_PORT Press ENTER to continue ----------------------------------------------------------- About to install Dbvisit STANDBY ----------------------------------------------------------- NTF repo installed. Component standby installed. ----------------------------------------------------------- About to install Dbvisit DBVNET ----------------------------------------------------------- Component dbvnet installed. ----------------------------------------------------------- About to install Dbvisit DBVAGENT ----------------------------------------------------------- Component dbvagent installed. ----------------------------------------------------------- Component Installer Version Installed Version ----------------------------------------------------------- standby 10.0.0RC_23_gca959724 10.0.0RC_23_gca959724 dbvnet 10.0.0RC_23_gca959724 (dbvnet v1.5.0-301-g3896e54)10.0.0RC_23_gca959724 (dbvnet v1.5.0-301-g3896e54) dbvagent 10.0.0RC_23_gca959724 10.0.0RC_23_gca959724 dbvserver 10.0.0RC_23_gca959724 not installed observer 10.0.0RC_23_gca959724 not installed ----------------------------------------------------------- What action would you like to perform? 1 - Install component(s) 2 - Uninstall component(s) 3 - Exit Your choice: 1 Choose component(s): 1 - Core Components (Dbvisit Standby Cli, Dbvnet, Dbvagent) 2 - Dbvisit Standby Core (Command Line Interface) 3 - Dbvnet (Dbvisit Network Communication) 4 - Dbvagent (Dbvisit Agent) 5 - Dbvserver (Dbvisit Central Console) - Not available on Solaris/AIX 6 - Dbvisit Observer (Automatic Failover Option) - Not available on Solaris/AIX Press ENTER to exit Installer Your choice: 5 ----------------------------------------------------------- About to configure DBVISIT DBVSERVER ----------------------------------------------------------- >>> Please specify the IPv4 address to be used for Dbvserver The Dbvisit Web Server (Dbvserver) will be listening on this local address. If using a cluster or virtual IP, make sure that the host name or alias specified here resolve to the IP address local to where Dbvserver is installed. It is recommended that you set this parameter to an IP address, as opposed to a host name. If you are unsure about the remote host name during installation, use the default value, which will be the current local hostname. Enter a custom value or press ENTER to accept default [kiwi91]: > 192.168.56.91 >>> Please specify the listening port number for Dbvserver on the local server You may choose any value between 1024 and 65535. The default recommended value is 4433. Note: if you can not access this port after the installation has finished, then please double-check your server firewall settings to ensure the selected port is open. Enter a custom value or press ENTER to accept default [4433]: > >>> Please specify the IPv4 address to be used for the Dbvserver public interface The public interface address will be used by Dbvagents for sending back information, e.g. Task progress. This should be set to the externally-visible IP address of the local host. Enter a custom value or press ENTER to accept default [192.168.56.91]: > >>> Please specify a publicly-available port number for Dbvserver. Like the Public Address setting previously, this port should be remotely accessible, and can potentially be different from the Dbvserver Listening Port, especially if your server is behind a DMZ. If you are unsure of the correct value for this parameter, it is usually fine to leave it the same as the Listener Port setting. You may choose any value between 1024 and 65535. The default recommended value is 4433. Note: if you can not access this port after the installation has finished, then please double-check your server firewall settings to ensure the selected port is open. Enter a custom value or press ENTER to accept default [4433]: > ----------------------------------------------------------- Summary of the Dbvisit DBVSERVER configuration ----------------------------------------------------------- DBVISIT_BASE /acfs/dbvisit DBVSERVER_LOCAL_HOST 192.168.56.91 DBVSERVER_LOCAL_PORT 4433 DBVSERVER_PUBLIC_HOST 192.168.56.91 DBVSERVER_PUBLIC_PORT 4433 Press ENTER to continue ----------------------------------------------------------- About to install Dbvisit DBVSERVER ----------------------------------------------------------- Component dbvserver installed. ----------------------------------------------------------- Component Installer Version Installed Version ----------------------------------------------------------- standby 10.0.0RC_23_gca959724 10.0.0RC_23_gca959724 dbvnet 10.0.0RC_23_gca959724 (dbvnet v1.5.0-301-g3896e54)10.0.0RC_23_gca959724 (dbvnet v1.5.0-301-g3896e54) dbvagent 10.0.0RC_23_gca959724 10.0.0RC_23_gca959724 dbvserver 10.0.0RC_23_gca959724 10.0.0RC_23_gca959724 observer 10.0.0RC_23_gca959724 not installed ----------------------------------------------------------- What action would you like to perform? 1 - Install component(s) 2 - Uninstall component(s) 3 - Exit Your choice: 1 Choose component(s): 1 - Core Components (Dbvisit Standby Cli, Dbvnet, Dbvagent) 2 - Dbvisit Standby Core (Command Line Interface) 3 - Dbvnet (Dbvisit Network Communication) 4 - Dbvagent (Dbvisit Agent) 5 - Dbvserver (Dbvisit Central Console) - Not available on Solaris/AIX 6 - Dbvisit Observer (Automatic Failover Option) - Not available on Solaris/AIX Press ENTER to exit Installer Your choice: 6 ----------------------------------------------------------- Summary of the Dbvisit OBSERVER configuration ----------------------------------------------------------- DBVISIT_BASE /acfs/dbvisit Press ENTER to continue ----------------------------------------------------------- About to install Dbvisit OBSERVER ----------------------------------------------------------- Component observer installed. ----------------------------------------------------------- Component Installer Version Installed Version ----------------------------------------------------------- standby 10.0.0RC_23_gca959724 10.0.0RC_23_gca959724 dbvnet 10.0.0RC_23_gca959724 (dbvnet v1.5.0-301-g3896e54)10.0.0RC_23_gca959724 (dbvnet v1.5.0-301-g3896e54) dbvagent 10.0.0RC_23_gca959724 10.0.0RC_23_gca959724 dbvserver 10.0.0RC_23_gca959724 10.0.0RC_23_gca959724 observer 10.0.0RC_23_gca959724 10.0.0RC_23_gca959724 ----------------------------------------------------------- What action would you like to perform? 1 - Install component(s) 2 - Uninstall component(s) 3 - Exit Your choice: 3 >>> Installation completed Install log /tmp/dbvisit_install.log.202012161755.
5. Example 2: Add Dbvnet and Dbvagent as Cluster Resources
In this section, we will show you how to add Dbvnet and Dbvagent as cluster resources.
We will make use of an action script for each component, which will then be used to manage these two resources.
Note: these scripts are provided as examples and should be tested in your development or test environments before implementing it into production as each environment is different and they might need to be adjusted for your environment.
5.1. Dbvnet Action Script
The script below is a sample action script that will be used to manage dbvnet.
This script is created in the dbvnet folder /acfs/dbvisit/dbvnet
#!/bin/bash # # Dbvnet Action Script ############################################## # Function to change database environments # Description: # Use oraenv to set the environment if needed # This is optional to set the environment ############################################## ## set following to ensure oraenv is picked up from /usr/local/bin export PATH=/usr/local/bin:$PATH set_env () { export ORAENV_ASK=NO export ORACLE_SID=$1 . oraenv >> /dev/null export ORAENV_ASK=YES } ################# ## Main Section ################# # This is logged to CRSD agent log file echo "`date` Action script '$_CRS_ACTION_SCRIPT' for resource [$_CRS_NAME] called for action $1" # set environment set_env DEV cd /acfs/dbvisit/dbvnet case "$1" in 'start') ./dbvnet -d start RET=0 echo "Running start dbvnet resource with return code $RET" ;; 'stop') NUM=`ps -ef | grep dbvnet | egrep -v 'grep|action-script|resource' | wc -l` if [ $NUM = 0 ]; then ## do a cleanup of pid ./dbvnet -d stop RET=0 else ## now stop the dbvnet ./dbvnet -d stop NUM=`ps -ef | grep dbvnet | grep -v grep | wc -l` if [ $NUM = 0 ]; then RET=0 else RET=1 fi fi echo "Running stop dbvnet resource with return code $RET" ;; 'check') NUM=`ps -ef | grep dbvnet | egrep -v 'grep|action-script|resource' | wc -l` if [ $NUM = 0 ]; then ## return code 1 for check means OFFLINE RET=1 else ## return code 0 for check means ONLINE RET=0 fi echo "Running check dbvnet resource with return code $RET" ;; 'clean') for c1 in `ps -ef|grep dbvnet |egrep -v 'grep|action-script|resource'| awk '{print $2}'` ; do echo "...force kill dbvnet pid $c1" kill -9 $c1 done ## do some cleanup ./dbvnet -d stop RET=0 echo "Running clean dbvnet resource with return code $RET" ;; esac if [ $RET -eq 0 ]; then exit 0 else exit 1 fi
5.2. Dbvagent Action Script
The script below is a sample action script that will be used to manage dbvagent.
This script is created in the dbvnet folder /acfs/dbvisit/dbvagent
#!/bin/bash # # Dbvagent Action Script ############################################# # Function to change database environments # Description: # Use oraenv to set the environment if needed # This is optional to set the environment ############################################# ## set following to ensure oraenv is picked up from /usr/local/bin export PATH=/usr/local/bin:$PATH set_env () { export ORAENV_ASK=NO export ORACLE_SID=$1 . oraenv >> /dev/null export ORAENV_ASK=YES } ################# ## Main Section ################# # This is logged to CRSD agent log file echo "`date` Action script '$_CRS_ACTION_SCRIPT' for resource [$_CRS_NAME] called for action $1" # set environment set_env DEV cd /acfs/dbvisit/dbvagent case "$1" in 'start') ./dbvagent -d start RET=0 echo "Running start dbvagent resource with return code $RET" ;; 'stop') NUM=`ps -ef | grep dbvagent | egrep -v 'grep|action-script|resource' | wc -l` if [ $NUM = 0 ]; then ## do a cleanup of pid ./dbvagent -d stop RET=0 else ## now stop the agent ./dbvagent -d stop NUM=`ps -ef | grep dbvagent | grep -v grep | wc -l` if [ $NUM = 0 ]; then RET=0 else RET=1 fi fi echo "Running stop dbvagent resource with return code $RET" ;; 'check') NUM=`ps -ef | grep dbvagent | egrep -v 'grep|action-script|resource' | wc -l` if [ $NUM = 0 ]; then ## return code 1 for check means OFFLINE RET=1 else ## return code 0 for check means ONLINE RET=0 fi echo "Running check dbvagent resource with return code $RET" ;; 'clean') for c1 in `ps -ef|grep dbvagent |egrep -v 'grep|action-script|resource'| awk '{print $2}'` ; do echo "...force kill dbvagent pid $c1" kill -9 $c1 done ## do some cleanup of pids ./dbvagent -d stop RET=0 echo "Running clean dbvagent resource with return code $RET" ;; esac if [ $RET -eq 0 ]; then exit 0 else exit 1 fi
5.3. Add the Cluster Resources
On the Primary cluster node 1 (kiwi81) as the root user run the following commands to add the two resources:
IMPORTANT: The lines below should be on one line, but are shown on separate lines to make it easier to read
crsctl add resource dbvnet -type cluster_resource -attr "ACTION_SCRIPT=/acfs/dbvisit/dbvnet/action-script.scr, RESTART_ATTEMPTS=3, START_TIMEOUT=60, STOP_TIMEOUT=60, CHECK_INTERVAL=10 ,START_DEPENDENCIES='hard(kiwi812-vip) pullup(kiwi812-vip)' ,STOP_DEPENDENCIES='hard(kiwi812-vip)' ,ACL='owner:oracle:rwx,pgrp:oinstall:rwx,other::r--' ,PLACEMENT='favored' ,HOSTING_MEMBERS='kiwi81 kiwi82'" crsctl add resource dbvagent -type cluster_resource -attr "ACTION_SCRIPT=/acfs/dbvisit/dbvagent/action-script.scr, RESTART_ATTEMPTS=3, START_TIMEOUT=60, STOP_TIMEOUT=60, CHECK_INTERVAL=10 ,START_DEPENDENCIES='hard(kiwi812-vip) pullup(kiwi812-vip)' ,STOP_DEPENDENCIES='hard(kiwi812-vip)' ,ACL='owner:oracle:rwx,pgrp:oinstall:rwx,other::r--' ,PLACEMENT='favored' ,HOSTING_MEMBERS='kiwi81 kiwi82'"
On the Standby cluster node 1 (kiwi91) as the root user run the following commands to add the two resources:
IMPORTANT: The lines below should be on one line, but are shown on separate lines to make it easier to read
crsctl add resource dbvnet -type cluster_resource -attr "ACTION_SCRIPT=/acfs/dbvisit/dbvnet/action-script.scr, RESTART_ATTEMPTS=3, START_TIMEOUT=60, STOP_TIMEOUT=60, CHECK_INTERVAL=10 ,START_DEPENDENCIES='hard(kiwi912-vip) pullup(kiwi912-vip)' ,STOP_DEPENDENCIES='hard(kiwi912-vip)' ,ACL='owner:oracle:rwx,pgrp:oinstall:rwx,other::r--' ,PLACEMENT='favored' ,HOSTING_MEMBERS='kiwi91 kiwi92'" crsctl add resource dbvagent -type cluster_resource -attr "ACTION_SCRIPT=/acfs/dbvisit/dbvagent/action-script.scr, RESTART_ATTEMPTS=3, START_TIMEOUT=60, STOP_TIMEOUT=60, CHECK_INTERVAL=10 ,START_DEPENDENCIES='hard(kiwi912-vip) pullup(kiwi912-vip)' ,STOP_DEPENDENCIES='hard(kiwi912-vip)' ,ACL='owner:oracle:rwx,pgrp:oinstall:rwx,other::r--' ,PLACEMENT='favored' ,HOSTING_MEMBERS='kiwi91 kiwi92'"
5.4. Start the Dbvnet and Dbvagent Cluster Resources
The following commands can be used to check the status of the cluster resources as well as to start and stop them.
Remember that these two resources are dependant on the Virtual IP address that was added.
If the Virtual IP address is moved (relocated) to the other node, the Dbvnet and Dbvagent resources will be stopped and moved to the other node and started.
IMPORTANT
After the dbvagent resource is started, check USER variable in its environment, for example:
xargs --null --max-args=1 < /proc/<pid>/environ | grep USER
If the result is:
USER=root
Then you will need to add one extra line to change this variable to the dbvagent action script:
... case "$1" in 'start') export USER=oracle ./dbvagent -d start ...
Restart dbvagent resource afterwards
- Status Check
root@kiwi91[/root]: crsctl status resource dbvnet NAME=dbvnet TYPE=cluster_resource TARGET=ONLINE STATE=ONLINE on kiwi91 root@kiwi91[/root]: crsctl status resource dbvagent NAME=dbvagent TYPE=cluster_resource TARGET=ONLINE STATE=ONLINE on kiwi91 root@kiwi91[/root]: crsctl status resource kiwi912-vip NAME=kiwi912-vip TYPE=app.appvip_net1.type TARGET=ONLINE STATE=ONLINE on kiwi91
or
root@kiwi91[/root]: crs_stat -t Name Type Target State Host ------------------------------------------------------------ dbvagent clus...esource ONLINE ONLINE kiwi91 dbvnet clus...esource ONLINE ONLINE kiwi91 kiwi912-vip app....t1.type ONLINE ONLINE kiwi91 ... ... ...
- Start Resource
root@kiwi91[/root]: crsctl start resource kiwi912-vip CRS-2672: Attempting to start 'kiwi912-vip' on 'kiwi91' CRS-2676: Start of 'kiwi912-vip' on 'kiwi91' succeeded root@kiwi91[/root]: crsctl start resource dbvnet CRS-2672: Attempting to start 'dbvnet' on 'kiwi91' CRS-2676: Start of 'dbvnet' on 'kiwi91' succeeded root@kiwi91[/root]: crsctl start resource dbvagent CRS-2672: Attempting to start 'dbvagent' on 'kiwi91' CRS-2676: Start of 'dbvagent' on 'kiwi91' succeeded root@kiwi91[/root]:
- Stop Resource
root@kiwi91[/root]: crsctl stop resource dbvnet CRS-2673: Attempting to stop 'dbvnet' on 'kiwi91' CRS-2677: Stop of 'dbvnet' on 'kiwi91' succeeded CRS-2679: Attempting to clean 'dbvnet' on 'kiwi91' CRS-2681: Clean of 'dbvnet' on 'kiwi91' succeeded root@kiwi91[/root]: crsctl stop resource dbvagent CRS-2673: Attempting to stop 'dbvagent' on 'kiwi91' CRS-2677: Stop of 'dbvagent' on 'kiwi91' succeeded CRS-2679: Attempting to clean 'dbvagent' on 'kiwi91' CRS-2681: Clean of 'dbvagent' on 'kiwi91' succeeded root@kiwi91[/root]: crsctl stop resource kiwi912-vip CRS-2673: Attempting to stop 'kiwi912-vip' on 'kiwi91' CRS-2677: Stop of 'kiwi912-vip' on 'kiwi91' succeeded
- Relocate Virtual IP
root@kiwi91[/root]: crsctl relocate resource kiwi912-vip -f CRS-2673: Attempting to stop 'dbvagent' on 'kiwi91' CRS-2673: Attempting to stop 'dbvnet' on 'kiwi91' CRS-2677: Stop of 'dbvnet' on 'kiwi91' succeeded CRS-2679: Attempting to clean 'dbvnet' on 'kiwi91' CRS-2677: Stop of 'dbvagent' on 'kiwi91' succeeded CRS-2679: Attempting to clean 'dbvagent' on 'kiwi91' CRS-2681: Clean of 'dbvnet' on 'kiwi91' succeeded CRS-2681: Clean of 'dbvagent' on 'kiwi91' succeeded CRS-2673: Attempting to stop 'kiwi912-vip' on 'kiwi91' CRS-2677: Stop of 'kiwi912-vip' on 'kiwi91' succeeded CRS-2672: Attempting to start 'kiwi912-vip' on 'kiwi92' CRS-2676: Start of 'kiwi912-vip' on 'kiwi92' succeeded CRS-2672: Attempting to start 'dbvagent' on 'kiwi92' CRS-2676: Start of 'dbvagent' on 'kiwi92' succeeded CRS-2672: Attempting to start 'dbvnet' on 'kiwi92' CRS-2676: Start of 'dbvnet' on 'kiwi92' succeeded
6. Example 2: Create the Dbvisit Standby DDC File
IMPORTANT
Before creating DDC file double check if both (or more) of your cluster nodes have entries in /etc/oratab for node specific ASM instance and node specific database instance. That means for example that on RAC cluster on second node in /etc/oratab you would have:
PROD2:/u01/app/oracle/product/19.7/dbhome_1:N +ASM2:/u01/app/19.7/grid:N
Again, you need to modify /etc/oratab on all your nodes. These entries must be present at all times otherwise synchronization (sending / applying of archivelogs will not work)
There are two methods to create the Dbvisit Standby DDC file.
In this example, we will show it via the Central Console.
The Central Console is installed onto its own system and have network access to both the Primary and the Standby RAC clusters.
IMPORTANT
When using 19c SEHA feature (which replaced RAC in 19c and onwards) during creation of DDC file the SEHA system is NOT detected as RAC by the GUI. That's because the database type is cluster_database=false. When setting up DDC with SEHA you won't be given the extra option "Source RAC Settings" in the GUI as shown below. Don't worry about this and create DDC only as a DDC for single database with no special options and finish the whole process as described below.
We'll however still need to enter the SEHA information, otherwise dbvisit would not work if your database and dbvisit VIP would failover to 2nd SEHA node. What you need to do is manually edit freshly created DDC file manually. DDC file is located in $DBVISIT_BASE/standby/conf/dbv_<DDC>.env and you need to modify following parameter like so:
ORACLE_SID_ASM = RAC1 = 1:<node1>:<instance_name>:+ASM1 RAC2 = 1:<node2>:<instance_name>:+ASM2
node1 and node2 names must be exact node names as they appear in OCR. You can quickly execute "crsctl check cluster -all" to see node names. Instance_name is the instance name of your primary database - it will be the same on both lines as with SEHA you've got only single instance.
If your standby site is also RAC, you'll need to make the same ammendments for DDC variables:
ORACLE_SID_ASM_DR = RAC1_DR = 1:<standby_node1>:<standby_instance_name>:+ASM1 RAC2_DR = 1:<standby_node2>:<standby_instance_name>:+ASM2
6.1. Add the two new hosts
The two hosts to be added is the Dbvagent connections, which is to the Virtual IP addresses.
6.2. Start the DDC creation process
Navigate to the Configurations screen and start the creation of the new DDC
The New Configuration will now start as seen below:
- Select the source as the primary Virtual IP - kiwi812-vip, then accept the license agreement
- Select the source Instance Name - this is obtained from the /etc/oratab file. We need to make sure you have both the Database and the Instance on the specified system in the /etc/oratab file. Select the instance name, in this case, it will be MYDEV1
- Specify the ARCSOURCE location. This location should ideally be located on the shared storage location. Remember when a Graceful Switchover is performed the roles are reversed between the ARCSOURCE and the ARCHDEST values in the DDC file
- Specify to use DBVNET by default (recommended)
- You will now be presented with the primary RAC database details, review and confirm the database and ASM instances match the servers they are running on. Dbvisit Standby will detect these values based on the connection that was made to the primary instance that was selected in step 2 above.
- As we are going to create a standby RAC database, we will adjust this setting to YES. See the next screenshot below
From the above image we can see:
- Specify the standby database will be a RAC enabled environment
- Select the standby Virtual IP hostname - kiwi912-vip
- You are then presented with the RAC Thread details for node 1 and 2 on the standby server, fill in the required detail
- Specify the ARCHDEST location - this should be on shared storage and is the location where Dbvisit Standby will transfer archive logs into on the standby server.
- Review and update the Dbvisit Base location on the standby server if different
- Review the Oracle Database Home on the standby server
- Review and update the DB Unique Name (db_unique_name) of the standby if required - default value recommended
- Specify the Dbvisit Standby Configuration (DDC) name
- Provide the license key for the configuration, this is optional and can be provided after the DDC creation, but must be provided before the running the CSD(Create Standby Database)
The last step is to click on Submit.
Dbvisit Standby will now validate the parameters specified and create a new DDC file as well as a repository (stored in the DBVISIT_BASE/standby/conf directory)
7. Example 2: Create the Standby Database
The standby database can either be created via the command line interface (dbvctl -d <DDC> --csd) or via the Central Console.
In this example, we will show the use of the Central Console to create the standby database
- Navigate to the "Create Standby Database" Menu
- Select the MYDEV DDC file from the dropdown list.
- Then specify the process option as "NEW DATABASE"
The server parameter file (spfile) section does contain a lot of information. Make sure you review the locations of the datafiles, redo logs and spfile.
In this example, the primary and standby systems match exactly from a storage point of view (same disk groups etc) so we do not have to change any parameters.
From the above, we can see that the temporary location /user/tmp was used for the local and remote temp backup locations.
Make sure the locations you specify has sufficient space to hold a full compressed database backup.
In this example, the "Create Standby Database & Template" option is selected.
The new task will be created as shown below:
srvctl modify database -db <db_name> -acfspath /u01/app/oradata
8. Example 2: Send Logs from Primary to Standby
- Navigate to the Database Actions Screen
- Click on the option to "Send Logs"
- Review the Task details
9. Example 2: Apply Archivelog to Standby
- Navigate to the Database Actions Screen
- Click on the option to "Apply Logs"
- Review the Task details
10. Oracle RAC on Windows
This section will include additional information and recommendations when installing Dbvisit Standby on an Oracle RAC configuration on Windows.
Please review Oracle Support Note: Doc ID 1339053.1
Running the command "crsuser add <windows-user>" might be required in your environment for the Action scripts to work. Refer 10.2 for example
10.1. Installation Recommendations
On Windows, the setup should be similar to a UNIX based Oracle RAC configuration. You have to install the Oracle software (Clusterware and Database) as per Oracle best practice.
There is one important difference between UNIX based installations of Dbvisit Standby compared to Windows (when installing on a RAC cluster) and this is that on Windows - Services is used, which means we need to create Dbvnet and Dbvagent services on both nodes in the cluster.
Example steps would be:
- Install the Oracle Clusterware and Configure ASM, including ASM diskgroups and ACFS, shared storage (recommended to place the Dbvisit Software on shared disk - as well as the Dbvisit ARCHDEST/ARCSOURCE locations)
- Install the Oracle Database software (again following best practice from Oracle Documentation).
- Once the database is created - in most cases step 1-3 here would already be done and you would be focussing on the next steps:
- Download the latest Dbvisit Standby software
- Make sure you configure the Virtual IP that will be used by Dbvisit Standby (please see above). This is the IP address/host alias that the standby server will be used to communicate with the RAC cluster - meaning it will see the RAC cluster as one using this Virtual IP (host alias) and from its point of view does not care which node this Virtual IP and of course Dbvisit Standby will be running on.
Example - Below we created a Virtual IP 10.0.2.118 for the VIP Name (host alias) kiwiwin-vip (note this should be added to your DNS). In this example, the RAC cluster was using local accounts and not domain accounts. If domain accounts were used the -user flag should be updated to the domain account.
C:\app\11.2.0\grid\BIN>appvipcfg.bat create -network=1 -ip=10.0.2.118 -vipname=kiwiwin-vip -user=.\oracle roduction Copyright 2007, 2008, Oracle.All rights reserved 017-10-30 02:01:14: Creating Resource Type 017-10-30 02:01:14: Executing C:\app\11.2.0\grid\bin\crsctl add type app.appvip_net1.type -basetype ora.cluster_vip_net1.type -file C:\app\11.2.0\grid/crs/template/appvip.type 017-10-30 02:01:14: Executing cmd: C:\app\11.2.0\grid\bin\crsctl add type app.appvip_net1.type -basetype ora.cluster_vip_net1.type -file C:\app\11.2.0\grid/crs/template/appvip.type 017-10-30 02:01:15: Create the Resource 017-10-30 02:01:15: Executing C:\app\11.2.0\grid\bin\crsctl add resource kiwiwin-vip -type app.appvip_net1.type -attr "USR_ORA_VIP=10.0.2.118,START_DEPENDENCIES=hard(ora.net1.network) pullup(ora.net1.network),STOP_DEPENDENCIES=hard(ora.net1.network),ACL='owner:.\oracle:rwx,pgrp::r-x,other::r--',HOSTING_MEMBERS=kiwiwin01,APPSVIP_FAILBACK=" 017-10-30 02:01:15: Executing cmd: C:\app\11.2.0\grid\bin\crsctl add resource kiwiwin-vip -type app.appvip_net1.type -attr "USR_ORA_VIP=10.0.2.118,START_DEPENDENCIES=hard(ora.net1.network) pullup(ora.net1.network),STOP_DEPENDENCIES=hard(ora.net1.network),ACL='owner:.\oracle:rwx,pgrp::r-x,other::r--',HOSTING_MEMBERS=kiwiwin01,APPSVIP_FAILBACK=" c:\app\11.2.0\grid\BIN>crsctl status resource kiwiwin-vip -p NAME=kiwiwin-vip TYPE=app.appvip_net1.type ACL=owner:.\oracle:rwx,pgrp::r-x,other::r-- ACTION_FAILURE_TEMPLATE= ACTION_SCRIPT= ACTIVE_PLACEMENT=1 AGENT_FILENAME=%CRS_HOME%/bin/orarootagent%CRS_EXE_SUFFIX% APPSVIP_FAILBACK=0 AUTO_START=restore CARDINALITY=1 CHECK_INTERVAL=1 CHECK_TIMEOUT=120 DEFAULT_TEMPLATE=PROPERTY(RESOURCE_CLASS=vip) ELEMENT(HOSTING_MEMBERS=%HOSTING_MEMBERS%) DEGREE=1 DESCRIPTION=Application VIP ENABLED=1 FAILOVER_DELAY=0 FAILURE_INTERVAL=0 FAILURE_THRESHOLD=0 GEN_USR_ORA_STATIC_VIP= GEN_USR_ORA_VIP= HOSTING_MEMBERS=kiwiwin01 LOAD=1 LOGGING_LEVEL=1 NLS_LANG= NOT_RESTARTING_TEMPLATE= OFFLINE_CHECK_INTERVAL=0 PLACEMENT=balanced PROFILE_CHANGE_TEMPLATE= RESTART_ATTEMPTS=0 SCRIPT_TIMEOUT=60 SERVER_POOLS=* START_DEPENDENCIES=hard(ora.net1.network) pullup(ora.net1.network) START_TIMEOUT=120 STATE_CHANGE_TEMPLATE= STOP_DEPENDENCIES=hard(ora.net1.network) STOP_TIMEOUT=0 TYPE_VERSION=2.2 UPTIME_THRESHOLD=7d USR_ORA_ENV= USR_ORA_VIP=10.0.2.118 VERSION=11.2.0.3.0 c:\app\11.2.0\grid\BIN>ping kiwiwin-vip Pinging kiwiwin-vip.oraclekiwi.co.nz [10.0.2.118] with 32 bytes of data: Control-C ^C c:\app\11.2.0\grid\BIN>crsctl start resource kiwiwin-vip CRS-2672: Attempting to start 'kiwiwin-vip' on 'kiwiwin02' CRS-2676: Start of 'kiwiwin-vip' on 'kiwiwin02' succeeded c:\app\11.2.0\grid\BIN>ping kiwiwin-vip Pinging kiwiwin-vip.oraclekiwi.co.nz [10.0.2.118] with 32 bytes of data: Reply from 10.0.2.118: bytes=32 time<1ms TTL=128 Reply from 10.0.2.118: bytes=32 time<1ms TTL=128 Ping statistics for 10.0.2.118: Packets: Sent = 2, Received = 2, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 0ms, Maximum = 0ms, Average = 0ms Control-C ^C
- Once the Virtual IP is created, you can install Dbvisit Standby on the first node, using shared storage - example if ACFS is used, in this example the shared storage was mounted under C:\app\oracle\acfsmounts\acfs_data
In this example the Dbvisit Base was: C:\app\oracle\acfsmounts\acfs_data\dbvisit
ONLY Dbvnet, Dbvagent and Standby Core components are installed. It is not recommended to run the GUI (Central Console) from a RAC Cluster - but rather on a standalone small system or the standby server.
- Install Dbvisit Standby first on node 1, then on node 2, in both cases install it into the same directory as mentioned above and let the services be created, but after installation makes sure you stop all services.
Also, make sure that when you install Dbvnet and the Agent that you configure the listening address as the Virtual IP hostname. This is to ensure that when Dbvnet or the Agent is started, it will listen on this Virtual IP.
- Now create a directory on both systems, example c:\dbv-scripts
- Inside this directory create two batch scripts which will be used as the Action Scripts when the cluster resources are created.
Example Action Scripts - Please note these are samples and should be tested in your development or test environments prior to implementing into production:
Dbvnet Action Script: Dbvnet.cmd
@echo off setlocal pushd if "%~1"=="" ( echo Action cannot be empty ... echo: call :HELP ) & (goto :EOF) REM Test if Usage/Help should be displayed if /i {%1}=={/?} (call :HELP) & (goto :EOF) goto :setEnvironment REM ########################################## REM Function to set Environment values REM ########################################## :setEnvironment set action=%~1 for /F "tokens=1,2,3,4 delims=/- " %%a in ('date /t') do set CDATE=%%d%%c%%b for /F "tokens=1,2,3,4 delims=/: " %%a in ('time /t') do set CTIME=%%a%%b%%c set LOGTIME=%CDATE%-%CTIME% set DBVISIT_BASE="C:\app\oracle\acfsmounts\acfs_data\Dbvisit" echo %LOGTIME% GOTO :EndOfSetEnvironment REM ########################################## :EndOfSetEnvironment if "%action%" == "start" goto :start if "%action%" == "stop" goto :stop if "%action%" == "check" goto :check if "%action%" == "clean" goto :clean goto :cleanexit :start %DBVISIT_BASE%\dbvsmgr\dbvsmgr.exe -s dbvnet echo "Running start dbvnet resource with return code %ERRORLEVEL%" if %ERRORLEVEL% == 0 (goto :cleanexit) if %ERRORLEVEL% gtr 0 (goto :errorexit) :stop %DBVISIT_BASE%\dbvsmgr\dbvsmgr.exe -t dbvnet echo "Running stop dbvnet resource with return code %ERRORLEVEL%" if %ERRORLEVEL% == 0 (goto :cleanexit) if %ERRORLEVEL% gtr 0 (goto :errorexit) :check echo "Running status check on Dbvnet resource" for /F "tokens=3 delims=: " %%H in ('sc query "Dbvnet" ^| findstr " STATE"') do ( if /I "%%H" NEQ "RUNNING" ( set ERRORLEVEL=1 goto :errorexit ) if /I "%%H" == "RUNNING" ( set ERRORLEVEL=0 goto :cleanexit ) ) :clean %DBVISIT_BASE%\Dbvisit\dbvsmgr\dbvsmgr.exe -t dbvnet echo "Running clean dbvnet resource with return code %ERRORLEVEL%" if %ERRORLEVEL% == 0 (goto :cleanexit) if %ERRORLEVEL% gtr 0 (goto :errorexit) :cleanexit exit /b 0 :errorexit exit /b 1 :HELP echo USAGE: echo dbvnet.scr [start^|stop^|clean^|check] goto :EOF :EXIT popd endlocal REM echo EXIT %ERRORLEVEL% goto :EOF
Example Dbvagent action script: dbvagent.cmd
@echo off setlocal pushd if "%~1"=="" ( echo Action cannot be empty ... echo: call :HELP ) & (goto :EOF) REM Test if Usage/Help should be displayed if /i {%1}=={/?} (call :HELP) & (goto :EOF) goto :setEnvironment REM ########################################## REM Function to set Environment values REM ########################################## :setEnvironment set action=%~1 for /F "tokens=1,2,3,4 delims=/- " %%a in ('date /t') do set CDATE=%%d%%c%%b for /F "tokens=1,2,3,4 delims=/: " %%a in ('time /t') do set CTIME=%%a%%b%%c set LOGTIME=%CDATE%-%CTIME% set DBVISIT_BASE="C:\app\oracle\acfsmounts\acfs_data\Dbvisit" echo %LOGTIME% GOTO :EndOfSetEnvironment REM ########################################## :EndOfSetEnvironment if "%action%" == "start" goto :start if "%action%" == "stop" goto :stop if "%action%" == "check" goto :check if "%action%" == "clean" goto :clean goto :cleanexit :start %DBVISIT_BASE%\dbvsmgr\dbvsmgr.exe -s dbvagent echo "Running start dbvagent resource with return code %ERRORLEVEL%" if %ERRORLEVEL% == 0 (goto :cleanexit) if %ERRORLEVEL% gtr 0 (goto :errorexit) :stop %DBVISIT_BASE%\dbvsmgr\dbvsmgr.exe -t dbvagent echo "Running stop dbvagent resource with return code %ERRORLEVEL%" if %ERRORLEVEL% == 0 (goto :cleanexit) if %ERRORLEVEL% gtr 0 (goto :errorexit) :check echo "Running status check on Dbvagent resource" for /F "tokens=3 delims=: " %%H in ('sc query "Dbvagent" ^| findstr " STATE"') do ( if /I "%%H" NEQ "RUNNING" ( set ERRORLEVEL=1 goto :errorexit ) if /I "%%H" == "RUNNING" ( set ERRORLEVEL=0 goto :cleanexit ) ) :clean %DBVISIT_BASE%\dbvsmgr\dbvsmgr.exe -t dbvagent echo "Running clean dbvagent resource with return code %ERRORLEVEL%" if %ERRORLEVEL% == 0 (goto :cleanexit) if %ERRORLEVEL% gtr 0 (goto :errorexit) :cleanexit exit /b 0 :errorexit exit /b 1 :HELP echo USAGE: echo dbvagent.scr [start^|stop^|clean^|check] goto :EOF :EXIT popd endlocal REM echo EXIT %ERRORLEVEL% goto :EOF
Once these scripts are on both RAC nodes in the exact same location, we can register the resource from the first node (Where the Dbvisit Virtual IP that will be used is running).
The commands to add the cluster resource is shown below:
Example: Add Dbvnet Cluster Resource
C:\dbv-scripts>set ORACLE_HOME=c:\app\11.2.0\grid C:\dbv-scripts>C:\app\11.2.0\grid\bin\crsctl add resource dbvnet -type cluster_resource -attr "ACTION_SCRIPT=c:\dbv-scripts\dbvnet.cmd, RESTART_ATTEMPTS=3, START_TIMEOUT=60, STOP_TIMEOUT=60, CHECK_INTERVAL=10, START_DEPENDENCIES='hard(kiwiwin-vip) pullup(kiwiwin-vip)',STOP_DEPENDENCIES='hard(kiwiwin-vip)' PLACEMENT='favored' HOSTING_MEMBERS='kiwiwin01'"
Example: Add Dbvagent Cluster Resource
C:\dbv-scripts>set ORACLE_HOME=c:\app\11.2.0\grid C:\dbv-scripts>%ORACLE_HOME%\bin\crsctl add resource dbvagent -type cluster_resource -attr "ACTION_SCRIPT=c:\dbv-scripts\dbvagent.cmd, RESTART_ATTEMPTS=3, START_TIMEOUT=60, STOP_TIMEOUT=60, CHECK_INTERVAL=10, START_DEPENDENCIES='hard(kiwiwin-vip) pullup(kiwiwin-vip)',STOP_DEPENDENCIES='hard(kiwiwin-vip)' PLACEMENT='favored' HOSTING_MEMBERS='kiwiwin01'"
More detail example output:
Example - Dbvnet:
C:\dbv-scripts>dbvnet.cmd Action cannot be empty ... USAGE: dbvnet.scr [start|stop|clean|check] C:\dbv-scripts>dbvnet.cmd check 20170411-0713PM Dbvnet service is running Running check dbvnet resource with return code 0 C:\dbv-scripts>dbvnet.cmd stop 20170411-0713PM Stopping Dbvnet service........... Stop Complete Running stop dbvnet resource with return code 0 C:\dbv-scripts>dbvnet.cmd start 20170411-0713PM Starting Dbvnet service Start Complete Running start dbvnet resource with return code 0 C:\dbv-scripts>dbvnet.cmd clean 20170411-0713PM Stopping Dbvnet service........... Stop Complete Running clean dbvnet resource with return code 0 C:\dbv-scripts>dbvnet.cmd check 20170411-0714PM Dbvnet service is not running Running check dbvnet resource with return code 10 C:\dbv-scripts>echo %ERRORLEVEL% 1 C:\dbv-scripts>dbvnet.cmd start 20170411-0714PM Starting Dbvnet service Start Complete Running start dbvnet resource with return code 0 C:\dbv-scripts>set ORACLE_HOME=c:\app\11.2.0\grid C:\dbv-scripts>C:\app\11.2.0\grid\bin\crsctl add resource dbvnet -type cluster_resource -attr "ACTION_SCRIPT=c:\dbv-scripts\dbvnet.cmd, RESTART_ATTEMPTS=3, START_TIMEOUT=60, STOP_TIMEOUT=60, CHECK_INTERVAL=10, START_DEPENDENCIES='hard(kiwiwin-vip) pullup(kiwiwin-vip)',STOP_DEPENDENCIES='hard(kiwiwin-vip)' PLACEMENT='favored' HOSTING_MEMBERS='kiwiwin01'" C:\dbv-scripts> C:\dbv-scripts>crsctl status resource dbvnet NAME=dbvnet TYPE=cluster_resource TARGET=OFFLINE STATE=OFFLINE C:\dbv-scripts>dbvnet.cmd stop 20170411-0720PM Stopping Dbvnet service........... Stop Complete Running stop dbvnet resource with return code 0 C:\dbv-scripts>crsctl start resource dbvnet CRS-2672: Attempting to start 'dbvnet' on 'kiwiwin01' CRS-2676: Start of 'dbvnet' on 'kiwiwin01' succeeded C:\dbv-scripts>crsctl status resource dbvnet NAME=dbvnet TYPE=cluster_resource TARGET=ONLINE STATE=ONLINE on kiwiwin01 C:\dbv-scripts> C:\dbv-scripts>crsctl status resource dbvnet -p NAME=dbvnet TYPE=cluster_resource ACL=owner:oracle:rwx,pgrp::---,other::r-- ACTION_FAILURE_TEMPLATE= ACTION_SCRIPT=c:\dbv-scripts\dbvnet.cmd ACTIVE_PLACEMENT=0 AGENT_FILENAME=%CRS_HOME%\bin\scriptagent.exe AUTO_START=restore CARDINALITY=1 CHECK_INTERVAL=10 DEFAULT_TEMPLATE= DEGREE=1 DESCRIPTION= ENABLED=1 FAILOVER_DELAY=0 FAILURE_INTERVAL=0 FAILURE_THRESHOLD=0 HOSTING_MEMBERS=kiwiwin01 LOAD=1 LOGGING_LEVEL=1 NOT_RESTARTING_TEMPLATE= OFFLINE_CHECK_INTERVAL=0 PLACEMENT=favored PROFILE_CHANGE_TEMPLATE= RESTART_ATTEMPTS=3 SCRIPT_TIMEOUT=60 SERVER_POOLS= START_DEPENDENCIES=hard(kiwiwin-vip) pullup(kiwiwin-vip) START_TIMEOUT=60 STATE_CHANGE_TEMPLATE= STOP_DEPENDENCIES=hard(kiwiwin-vip) STOP_TIMEOUT=60 UPTIME_THRESHOLD=1h
Dbvagent:
C:\dbv-scripts>dbvagent.cmd check 20170411-0723PM Dbvnet service is running Running check dbvagent resource with return code 0" C:\dbv-scripts>dbvagent.cmd clean 20170411-0723PM Stopping Dbvnet service........... Stop Complete Running clean dbvagent resource with return code 0" C:\dbv-scripts>dbvagent.cmd start 20170411-0723PM Starting Dbvnet service Start Complete Running start dbvagent resource with return code 0" C:\dbv-scripts>dbvagent.cmd stop 20170411-0723PM Stopping Dbvnet service........... Stop Complete Running stop dbvagent resource with return code 0" C:\dbv-scripts> C:\dbv-scripts> C:\dbv-scripts>set ORACLE_HOME=c:\app\11.2.0\grid C:\dbv-scripts>%ORACLE_HOME%\bin\crsctl add resource dbvagent -type cluster_resource -attr "ACTION_SCRIPT=c:\dbv-scripts\dbvagent.cmd, RESTART_ATTEMPTS=3, START_TIMEOUT=60, STOP_TIMEOUT=60, CHECK_INTERVAL=10, START_DEPENDENCIES='hard(kiwiwin-vip) pullup(kiwiwin-vip)',STOP_DEPENDENCIES='hard(kiwiwin-vip)' PLACEMENT='favored' HOSTING_MEMBERS='kiwiwin01'" C:\dbv-scripts> C:\dbv-scripts>crsctl status resource dbvagent -p NAME=dbvagent TYPE=cluster_resource ACL=owner:oracle:rwx,pgrp::---,other::r-- ACTION_FAILURE_TEMPLATE= ACTION_SCRIPT=c:\dbv-scripts\dbvagent.cmd ACTIVE_PLACEMENT=0 AGENT_FILENAME=%CRS_HOME%\bin\scriptagent.exe AUTO_START=restore CARDINALITY=1 CHECK_INTERVAL=10 DEFAULT_TEMPLATE= DEGREE=1 DESCRIPTION= ENABLED=1 FAILOVER_DELAY=0 FAILURE_INTERVAL=0 FAILURE_THRESHOLD=0 HOSTING_MEMBERS=kiwiwin01 LOAD=1 LOGGING_LEVEL=1 NOT_RESTARTING_TEMPLATE= OFFLINE_CHECK_INTERVAL=0 PLACEMENT=favored PROFILE_CHANGE_TEMPLATE= RESTART_ATTEMPTS=3 SCRIPT_TIMEOUT=60 SERVER_POOLS= START_DEPENDENCIES=hard(kiwiwin-vip) pullup(kiwiwin-vip) START_TIMEOUT=60 STATE_CHANGE_TEMPLATE= STOP_DEPENDENCIES=hard(kiwiwin-vip) STOP_TIMEOUT=60 UPTIME_THRESHOLD=1h C:\dbv-scripts>crsctl start resource dbvagent CRS-2672: Attempting to start 'dbvagent' on 'kiwiwin01' CRS-2676: Start of 'dbvagent' on 'kiwiwin01' succeeded C:\dbv-scripts>crsctl status resource dbvagent NAME=dbvagent TYPE=cluster_resource TARGET=ONLINE STATE=ONLINE on kiwiwin01
Testing Relocation of the Resources:
C:\dbv-scripts>crsctl relocate resource kiwiwin-vip CRS-2529: Unable to act on 'kiwiwin-vip' because that would require stopping or relocating 'dbvnet', but the force option was not specified CRS-4000: Command Relocate failed, or completed with errors. C:\dbv-scripts>crsctl relocate resource kiwiwin-vip -f CRS-2673: Attempting to stop 'dbvagent' on 'kiwiwin01' CRS-2673: Attempting to stop 'dbvnet' on 'kiwiwin01' CRS-2677: Stop of 'dbvnet' on 'kiwiwin01' succeeded CRS-2677: Stop of 'dbvagent' on 'kiwiwin01' succeeded CRS-2673: Attempting to stop 'kiwiwin-vip' on 'kiwiwin01' CRS-2677: Stop of 'kiwiwin-vip' on 'kiwiwin01' succeeded CRS-2672: Attempting to start 'kiwiwin-vip' on 'kiwiwin02' CRS-2676: Start of 'kiwiwin-vip' on 'kiwiwin02' succeeded CRS-2672: Attempting to start 'dbvagent' on 'kiwiwin02' CRS-2676: Start of 'dbvagent' on 'kiwiwin02' succeeded CRS-2672: Attempting to start 'dbvnet' on 'kiwiwin02' CRS-2676: Start of 'dbvnet' on 'kiwiwin02' succeeded C:\dbv-scripts>crs_stat -t Name Type Target State Host ------------------------------------------------------------ dbvagent clus...esource ONLINE ONLINE kiwiwin02 dbvnet clus...esource ONLINE ONLINE kiwiwin02 kiwiwin-vip app....t1.type ONLINE ONLINE kiwiwin02 ora.DATA.dg ora....up.type ONLINE ONLINE kiwiwin01 ora....ER.lsnr ora....er.type ONLINE ONLINE kiwiwin01 ora....N1.lsnr ora....er.type ONLINE ONLINE kiwiwin02 ora....N2.lsnr ora....er.type ONLINE ONLINE kiwiwin01 ora....N3.lsnr ora....er.type ONLINE ONLINE kiwiwin01 ora.OCRVOT.dg ora....up.type ONLINE ONLINE kiwiwin01 ora.asm ora.asm.type ONLINE ONLINE kiwiwin01 ora.cvu ora.cvu.type ONLINE ONLINE kiwiwin01 ora.dev.db ora....se.type ONLINE ONLINE kiwiwin01 ora.gsd ora.gsd.type OFFLINE OFFLINE ora....SM1.asm application ONLINE ONLINE kiwiwin01 ora....01.lsnr application ONLINE ONLINE kiwiwin01 ora....n01.gsd application OFFLINE OFFLINE ora....n01.ons application ONLINE ONLINE kiwiwin01 ora....n01.vip ora....t1.type ONLINE ONLINE kiwiwin01 ora....SM2.asm application ONLINE ONLINE kiwiwin02 ora....02.lsnr application ONLINE ONLINE kiwiwin02 ora....n02.gsd application OFFLINE OFFLINE ora....n02.ons application ONLINE ONLINE kiwiwin02 ora....n02.vip ora....t1.type ONLINE ONLINE kiwiwin02 ora....network ora....rk.type ONLINE ONLINE kiwiwin01 ora.oc4j ora.oc4j.type ONLINE ONLINE kiwiwin01 ora.ons ora.ons.type ONLINE ONLINE kiwiwin01 ora....ry.acfs ora....fs.type ONLINE ONLINE kiwiwin01 ora.scan1.vip ora....ip.type ONLINE ONLINE kiwiwin02 ora.scan2.vip ora....ip.type ONLINE ONLINE kiwiwin01 ora.scan3.vip ora....ip.type ONLINE ONLINE kiwiwin01 C:\dbv-scripts>
10.2 Example on How to Modify OS User Privileges for 11gR2 Grid Infrastructure and RAC Services
In some environments, without the following steps, the VIP resource would not start and give an error.
set ORACLE_HOME=C:\app\11.2.0\grid C:\app\11.2.0\grid\BIN>crsuser add kiwiwin01\oracle Enter the password for kiwiwin01\oracle: << DO NOT ENTER PASSWORD, JUST PRESS ENTER>> ERROR: The account name is invalid or does not exist, or the password is invalid for the account name specified. CreateService for OracleCRSToken_oracle on node kiwiwin02 failed C:\app\11.2.0\grid\BIN>crsuser list Available nodes kiwiwin01 kiwiwin02 Improperly configured users oracle kiwiwin02: The specified service does not exist as an installed service.
set ORACLE_HOME=C:\app\11.2.0\grid C:\app\11.2.0\grid\BIN>crsuser add kiwiwin02\oracle Enter the password for kiwiwin02\oracle: << DO NOT ENTER PASSWORD, JUST PRESS ENTER>> ERROR: The account name is invalid or does not exist, or the password is invalid for the account name specified. CreateService for OracleCRSToken_oracle on node kiwiwin01 failed C:\app\11.2.0\grid\BIN>crsuser list Available nodes kiwiwin01 kiwiwin02 Properly configured users oracle