Activate (Failover) as Standby Database

1.  Introduction

When disaster strikes and the primary database is no longer available the standby database must be activated to become the new primary database to continue operation.  This is also called failover to the standby database.

The steps to activate the standby database are:

  1. Stop the scheduling of Dbvisit Standby.
  2. Change the network configuration (or DNS) so that users will connect to the standby database (or server) instead of the primary database (or server).
  3. Activate and open the standby database for normal operation as per instructions below.  Note that as soon as the standby database is activated and becomes the new primary database that is open read/write.  The link to the original primary database is lost and it is no longer possible to apply new logs to the original standby database.

To activate and open the standby database for continued operation in the event of a disaster the following procedure discussed in this section must be used.


2.  Activate (Failover) - Using the Command Line Interface

The command line options available to activate the standby database are: 

dbvctl -d <ddc> -o activate [--force] 

This command is used to Activate the standby database, during which process it becomes the new primary database. Important, this operation MUST be done with caution as the process is NOT reversible.  This action is performed in the event that the primary database is no longer available. Dbvisit Standby will prompt before activating the standby database.  If the --force an option is provided then Dbvisit Standby will no longer prompt before activating the standby database (this option can be used in batch mode).

Note: On the event that APPLY_DELAY_LAG_MINUTES (i.e. APPLY_DELAY_LAG_MINUTES = 180) is set in the DDC, and standby is activated in batch mode, it will most likely fail or cancelled. If APPLY_DELAY_LAG_MINUTES is set in the DDC, the correct way of activating the standby is via interactive mode and not using the option --force.

Before you activate the standby database, running a quick test by ensuring the standby database is in a consistent state (datafile headers and controlfile are in sync) and ready to be activated.  This can be done by opening the standby database read-only.

This is done by running the following command: "./dbvctl -d DEV -o read"

Example:

As we can see in the example above, the standby database can be opened read-only without any errors.

If you did get any errors, example stating file 1 needs more recovery, you will need to apply more archive logs before you will be able to successfully activate (open the standby database read/write).

Now that we know we can open the standby read-only, let's start it back into recovery mode (mount state), by running: "./dbvctl -d DEV -o restart"

Example:

Now that we know the standby database can be opened read-only and that it is in a consistent state that it can be opened, we can execute the "Activation"

The example below shows the process is executed to activate the standby database for DDC file DEV:

[oracle@dbv2 /usr/dbvisit/standby]$ ./dbvctl -d DEV -o activate
=============================================================
Dbvisit Standby Database Technology (8.0.04.18169) (pid 3420)
dbvctl started on dbv2: Sun Nov 13 10:52:38 2016
=============================================================

Activating means this database will become a Primary Database.
It will no longer be a Standby Database for DEV on dbv1.
Activation cannot be reversed.
=>Activate Standby Database on dbv2? <Yes/No> [No]: Yes
Are you sure? <Yes/No> [No]: Yes
>>> Activating now...

File /usr/dbvisit/standby/conf/dbv_DEV.env copied to /usr/dbvisit/standby/conf/dbv_DEV.env.201611131052.

>>> Activation complete. Please ensure a backup of this Database is made
>>> Old archives from before the activation should be removed to avoid mix-up between new
    and old archive logs


>>> If the Dbvisit Standby process is to be reversed, then database on dbv1 will need to be
    rebuilt as a Standby Database

=============================================================
dbvctl ended on dbv2: Sun Nov 13 10:53:35 2016
=============================================================

Next Steps:

As soon as possible, back up your new production database.

At this point, the former standby database is now your production database.

This task, while not required, is a recommended safety measure because you cannot recover changes made after activation without a backup.



3.  Disaster Recovery Actions - Using the Central Console

In case of disaster strikes, you will want to activate (failover) to the standby database.  During this process, a resetlogs is performed and the standby database is opened read/write.

The standby database is then converted to a primary database and cannot be used as a standby database anymore.

In version 9 of Dbvisit Standby, a number of new options have been introduced in this section.  First select "Disaster Recovery Actions" from the main menu [1]


Once you select a DDC on which the operation is to be performed, there are 3 options are now provided as listed below:





  1. Activate Now
    1. Select this option if you want to activate the standby database now - this is usually in the case when the disaster happened and you need to activate (failover) to the standby database.
  2. Perform DR Test
    1. It is always recommended that sites perform DR testing, activating a standby database and making use if a disaster did happen everything work as expected.  This option will allow you to perform a backup of the standby database prior to activating it. 
    2. Two options will be available:
      1. Create a backup to a local disk using RMAN Backupsets
      2. Create a backup to a local disk using RMAN Image Copies
        1. This option can be useful to quickly switch back to your image copy instead of running a restore process.

  3. Standby Read Only Test
    1. This option allows you to quickly test if your standby database can be opened read-only.  This is an excellent test to see if the standby database is in a consistent state.  If a standby database is not consistent, you will not be able to open it read-only or activate it.  In this case, you will have to send and apply more logs before you will be able to activate or open the standby read-only.

The screen capture below shows what the Perform DR Test screen looks like:



  1. Select "Perform DR Test"
  2. If you are running this the first time you will need to select "RUN DR TEST" - if you already performed this task, you will be given the option to "REINSTATE" if you have backups (backupset or image copies) available
  3. Confirm if you want to create a backup of the standby database prior to the test - NOTE: if you select NO, you will not have backups unless you create them manually
  4. Select the backup method type - Image Copy or Backupset
  5. Specify the format - this is making use of the RMAN format options - for more detail please see the Oracle RMAN documentation 
  6. Specify the location for the backup - this location must have at least enough space to hold a full copy of the standby database. 
  7. You can specify the process to stop between the backup and activation stages
  8. Submit to continue


If you have selected the option to create an Image copy backup, once done with the first step of the image copy creation you will get a completed task indicating the image backup was created and validated.

You will then be presented with the option to resume and continue with the activation.  Once that is complete you will get the task that indicates the completion of the activation/failover process.

Once the above is complete, you will be able to perform you tests on the newly activate standby.  


When you are ready to re-instantiate the standby again you can navigate to the Disaster Recovery Actions page and select the DDC.

You will then select the Perform DR Test and you will get the option REINSTATE, which will allow you to get to the next screen that allows you to continue restoring from the backup taken in the previous step.



...

...

...


  • You will get two options if using Image Copy
    • Switch to the image copy
      • This will be fast as we will switch to the image copy instead of restoring.  You will need to be aware that in this scenario the database will be switched to the copy.
    • Restore from the image copy
      • This method will perform a full restore from the image copy - this process is not as fast but will restore the database into the original location.


  • Once this is selected, click on Submit to start the operation.  As mentioned the switching to the image copy is faster.  


Once the re-instate is complete, for example, if you have picked to switch to copy the following task details, will be displayed.


4. Disaster Recovery Actions - Using CLI

The DR test and the Read-Only test can also be performed using the command line. The below example will show how it is done. dbvctl -o -h will display the help on the commands which can be used .


4.1 Perform a Read-Only test

This option will help in determining if the database is standby is in a consistent state. This would open the database in read-only mode and the restart the database to mount mode so that the archivelogs can be applied.

4.2 Perform a DR Test and Re-instate


Status of the database

Now the next step is to reinstate the database back to the standby database

Status of the database

Below is another example where the DR test is performed by using image copy as backup and we switch to copy when reinstating the DR to standby.
Reinstate using the switch to copy function

5. Video - How to Perform a DR Test in Dbvisit Standby v9