This page describes how to perform standby database activation (Failover). This task is done in case of real disasters when primary site is lost, or when performing DR Test. The activation process is the same for test and real life situations, the only difference is in recovering from the activation.

For performing DR Tests, Dbvisit StandbyMP has a guided process “Test Standby Activation” which is described here: https://dbvisit.atlassian.net/wiki/spaces/DSMPV12/pages/4462018590 , but this process requires that the primary site remains reachable at all times.

If you need to perform DR test with primary site unreachable or perform DR test in isolated environment, you have to use Activation (Failover) process described on this page.

1. Standby Database Activation

When your primary database isn’t accessible anymore and you need to failover to the standby, you can follow this procedure to successfully activate your standby database to read-write mode and use it as new primary.

1.1 Prechecks

Before activating standby database, we can verify its consistency by opening in read only mode and restarting it back to mount mode. On standby server run:

The result for “alter database open” command must be “Database altered”. You will need to restart the standby database back to MOUNT mode (in this example startup mount force;) Only then you can proceed with standby database activation.

Result similar to this:

Means that your standby database requires recovery (applying one or more archivelogs) and can’t be activated.

You can also put your standby database to read-only mode and back to MOUNT using dbvcontrol GUI:

Miscellaneous Operation Tasks | 6. Start/Stop Database

You can also manually check the standby database last recovery timestamp:

The standby database will contain data till this date.

1.2 Activation With Dbvcontrol GUI

Once you perform the prechecks, you can use dbvserver GUI for the activation. From the ACTIONS pane choose activate standby database:

Open

You will see a summary indicating whenter your standby DB is ready for activation and it’s current recovery SCN:

Open

Once you click on activate and confirm, you will see a new task:

And when clicking on it, you can see the details:

1.3 Activation with dbvctl

If Dvbcontrol GUI is not available, or if you want to use command line instead of GUI for Activation, it is also possible with following dbvctl command on standby server:

Example with full output:

2. Standby Database Activation with apply delay set (Intelligent Activate)

If you have set the apply log delay on standby server, there will be unapplied archivelogs on your standby server present when you want to activate the standby database. Dbvisit GUI would then automatically detect this and will provide you with comprehensive PITR (Poin In Time Recovery) guide.

To trigger the activation, use the same action as for case without any delay set:

You will be afterwards presented with an overview of unapplied archivelogs (1). You’ll be able to select up to which archivelog you want to recover your standby database before activation (2):

Once you decide and select the last archivelog you want to apply, click on Activate (3).

The Activation Task will appear and you will be able to see in details the information about which SCN was used to roll forward:

As with activate without delay, it’s also possible to use dbvctl command line for intelligent activate. You can trigger the intelligent activate in same way as when the delay is not set and the choose the PITR. The command you need to run is:

The full output is:

4. Post-Activation steps

Once the standby database is activated, its mode is changed to read-write and it will get a new incarnation. You will be able to connect your application to the database without issues.

We reccomend however that before connecting your application, you verify that everything is working correctly from DBA perspective.

The most important checks include (but are not limited to):

• checking all temporary datafiles are created

• observing alert log of new primary database

• cycle through all redo log groups by running “alter system switch logfile” and verify all archivelogs are correctly created

• cross check backups and archivelogs in RMAN repository

• verify that listener lists all necessary database services

Without these steps you risk that there will be issues in the near future with the activated standby database.

5. Recovery from Standby Database Activation

At some point in time after activation after get your old primary site online, you will need to get back to the original state as it was before the activation. This process is different for test failovers (when you don’t want to preserve your activated standby) and for real failover (when you want to preserve your activated standby because it’s new production).

5.1 Recovery from real disaster recovery event

There’s no way how to directly re-establish synchronization in the opposite direction to your old primary database (= get back to the original state). Your old primary database can’t be used anymore and must be deleted. This is dictated by Oracle RDBMS Concept and can’t be overcome in any way.

You will need to perform following high-level steps in order to get back to the state before the activation:

• Delete old primary

• Recover DDC configuration from failover

• Create standby database in place of the old primary

• Perform Graceful Switchover (GS)

Note that all the steps (except the switchover) happen without any downtime for your new production database (activated standby).

A. Delete your Old Primary

This is intentionally a manual step for which Dbvisit doesn’t offer any GUI or command line interface. You have to manually delete your old primary database, for example as described in:

Prerequisites | 6. Cleanup Standby Server

B. Recover DDC configuration from failover

To create standby database on the old primary server, we must first fix our DDC in GUI Dashboard. After failover, the dashboard woud look like so:

The configuration will be visible as “IN FAILOVER STATE” (1). You may see leftover apply delay icon (2) in case you had delay set and did activation without primary server reachable, but you can ignore it. In order to continue:

• The Primary and Standby server roles must be correct. In this example czlin0232 is the new primary server (activated standby) and czlin0231 was former primary server

• The status of database on standby server (in this case czlin0231) must be “NOT PRESENT”. If you see status “DOWN” it might mean that not all files are deleted properly - especially spfile

To continue, click on “recover from failover”:

And click on the button “Recover From Failover”

• once recovery from failover is finished, you will be able to see DDC configuration as if it was before you created the standby database for the first time:

Once you get to this point, you’re in good position to create standby database.

D. Create standby database in place of the old primary

This process follows exactly the same principles and is the same as normal CSD process:

https://dbvisit.atlassian.net/wiki/spaces/DSMPV12/pages/4446715916

The only exception is that any parameters you already set in DDC file are preserved, so for example your AMM will be already set. After CSD you should see result similar to this:

At this point the synchronization is established in opposite direction than before you executed the Failover. You’re able now to continue to the last step: Graceful Switchover

E. Perform Graceful Switchover (GS)

This process follows exactly the same principles and is the same as normal GS process:

Graceful Switchover (GS)

Executing GS will mean short downtime for your production. On GS is finished, you will be back to the original state:

5.2 Recovery from test failover

When doing the test failover, the tasks to recover are different, because obviously we want to preserve our old primary and we want to recreate our standby. The high level steps are:

• Shutdown and Delete activated standby database

• Recover DDC configuration from failover

• Re-create the standby database from scratch

The typical state you would see your DDC configuration in GUI after test failover would be this:

Primary server = czlin0231. This would be the original production server to which you production application is connected and which you don’t want to touch.

Activated standby = czlin0232. This is your standby server where you activated standby database and performed DR tests.

Our goal here is to recreate the standby database on czlin0232 and change the roles in the GUI without affecting primary database on czlin0231

A. Shutdown and Delete activated standby database

This is intentionally a manual step for which Dbvisit doesn’t offer any GUI or command line interface. You have to manually delete your activated standby database, for example as described in:

Prerequisites | 6. Cleanup Standby Server

B. Recover DDC configuration from failover

There’s no way how to directly change the direction of the replication in the GUI. You have to remove the configuration and import it again as specified in the Import DDC procedure:

Removing and Importing DDC file

Once you successfully import the DDC file, you GUI should look like this:

The server roles are correct now and we can proceed to create standby database.

C. Re-create the standby database from scratch

This process follows exactly the same principles and is the same as normal CSD process:

https://dbvisit.atlassian.net/wiki/spaces/DSMPV12/pages/4446715916

The only exception is that any parameters you already set in DDC file are preserved, so for example your AMM will be already set. After CSD you should see result similar to this:

which means that you are back to your original configuration from before when you started the DR Test by Failover.