Activate (Failover) as Standby Database
- Deepika Kumar (Deactivated)
- Shalini Anbu
- Charmaigne Bezuidenhout (Deactivated)
1. Introduction
When disaster strikes and the primary database is no longer available the standby database must be activated to become the new primary database to continue operation. This is also called failover to the standby database.
The steps to activate the standby database are:
- Stop the scheduling of Dbvisit Standby.
- Change the network configuration (or DNS) so that users will connect to the standby database (or server) instead of the primary database (or server).
- Activate and open the standby database for normal operation as per instructions below. Note that as soon as the standby database is activated and becomes the new primary database that is open read/write. The link to the original primary database is lost and it is no longer possible to apply new logs to the original standby database.
To activate and open the standby database for continued operation in the event of a disaster the following procedure discussed in this section must be used.
2. Activate (Failover) - Using the Command Line Interface
The command line options available to activate the standby database are:
dbvctl -d <ddc> -o activate [--force]
This command is used to Activate the standby database, during which process it becomes the new primary database. Important, this operation MUST be done with caution as the process is NOT reversible. This action is performed in the event that the primary database is no longer available. Dbvisit Standby will prompt before activating the standby database. If the --force an option is provided then Dbvisit Standby will no longer prompt before activating the standby database (this option can be used in batch mode).
Note: On the event that APPLY_DELAY_LAG_MINUTES (i.e. APPLY_DELAY_LAG_MINUTES = 180) is set in the DDC, and standby is activated in batch mode, it will most likely fail or cancelled. If APPLY_DELAY_LAG_MINUTES is set in the DDC, the correct way of activating the standby is via interactive mode and not using the option --force.
Before you activate the standby database, running a quick test by ensuring the standby database is in a consistent state (datafile headers and controlfile are in sync) and ready to be activated. This can be done by opening the standby database read-only.
This is done by running the following command: "./dbvctl -d DEV -o read"
Example:
[oracle@dbv2 standby]$ ./dbvctl -d DEV -o read ============================================================= Dbvisit Standby Database Technology (9.0.0_1271_g4b00fb49) (pid 2555) dbvctl started on dbv2: Tue May 7 11:36:30 2019 ============================================================= >>> Running pre-checks please wait... done Open Standby instance DEV in READ ONLY mode... Standby instance DEV on dbv2 opened in READ ONLY mode. Log files cannot be applied to Database while in READ ONLY mode. Database tempfile(s) may need to be added to this database. ============================================================= dbvctl ended on dbv2: Tue May 7 11:36:31 2019 =============================================================
As we can see in the example above, the standby database can be opened read-only without any errors.
If you did get any errors, example stating file 1 needs more recovery, you will need to apply more archive logs before you will be able to successfully activate (open the standby database read/write).
Now that we know we can open the standby read-only, let's start it back into recovery mode (mount state), by running: "./dbvctl -d DEV -o restart"
Example:
[oracle@dbv2 standby]$ ./dbvctl -d DEV -o restart ============================================================= Dbvisit Standby Database Technology (9.0.0_1271_g4b00fb49) (pid 2574) dbvctl started on dbv2: Tue May 7 11:37:16 2019 ============================================================= Stopping instance DEV... Standby Database DEV shutdown successfully on dbv2. Starting instance DEV... Standby Database DEV started on dbv2. ============================================================= dbvctl ended on dbv2: Tue May 7 11:37:24 2019 =============================================================
Now that we know the standby database can be opened read-only and that it is in a consistent state that it can be opened, we can execute the "Activation"
The example below shows the process is executed to activate the standby database for DDC file DEV:
[oracle@dbv2 /usr/dbvisit/standby]$ ./dbvctl -d DEV -o activate
=============================================================
Dbvisit Standby Database Technology (8.0.04.18169) (pid 3420)
dbvctl started on dbv2: Sun Nov 13 10:52:38 2016
=============================================================
Activating means this database will become a Primary Database.
It will no longer be a Standby Database for DEV on dbv1.
Activation cannot be reversed.
=>Activate Standby Database on dbv2? <Yes/No> [No]: Yes
Are you sure? <Yes/No> [No]: Yes
>>> Activating now...
File /usr/dbvisit/standby/conf/dbv_DEV.env copied to /usr/dbvisit/standby/conf/dbv_DEV.env.201611131052.
>>> Activation complete. Please ensure a backup of this Database is made
>>> Old archives from before the activation should be removed to avoid mix-up between new
and old archive logs
>>> If the Dbvisit Standby process is to be reversed, then database on dbv1 will need to be
rebuilt as a Standby Database
=============================================================
dbvctl ended on dbv2: Sun Nov 13 10:53:35 2016
=============================================================
Next Steps:
As soon as possible, back up your new production database.
At this point, the former standby database is now your production database.
This task, while not required, is a recommended safety measure because you cannot recover changes made after activation without a backup.
3. Disaster Recovery Actions - Using the Central Console
In case of disaster strikes, you will want to activate (failover) to the standby database. During this process, a resetlogs is performed and the standby database is opened read/write.
The standby database is then converted to a primary database and cannot be used as a standby database anymore.
In version 9 of Dbvisit Standby, a number of new options have been introduced in this section. First select "Disaster Recovery Actions" from the main menu [1]
Once you select a DDC on which the operation is to be performed, there are 3 options are now provided as listed below:
- Activate Now
- Select this option if you want to activate the standby database now - this is usually in the case when the disaster happened and you need to activate (failover) to the standby database.
- Perform DR Test
- It is always recommended that sites perform DR testing, activating a standby database and making use if a disaster did happen everything work as expected. This option will allow you to perform a backup of the standby database prior to activating it.
- Two options will be available:
- Create a backup to a local disk using RMAN Backupsets
- Create a backup to a local disk using RMAN Image Copies
This option can be useful to quickly switch back to your image copy instead of running a restore process.
Use Filesystem/ACFS locations to take backups and image copies of the database.The current release does not support taking backups and image copies in ASM diskgroups
- Standby Read Only Test
- This option allows you to quickly test if your standby database can be opened read-only. This is an excellent test to see if the standby database is in a consistent state. If a standby database is not consistent, you will not be able to open it read-only or activate it. In this case, you will have to send and apply more logs before you will be able to activate or open the standby read-only.
The screen capture below shows what the Perform DR Test screen looks like:
- Select "Perform DR Test"
- If you are running this the first time you will need to select "RUN DR TEST" - if you already performed this task, you will be given the option to "REINSTATE" if you have backups (backupset or image copies) available
- Confirm if you want to create a backup of the standby database prior to the test - NOTE: if you select NO, you will not have backups unless you create them manually
- Select the backup method type - Image Copy or Backupset
- Specify the format - this is making use of the RMAN format options - for more detail please see the Oracle RMAN documentation
- Specify the location for the backup - this location must have at least enough space to hold a full copy of the standby database.
- You can specify the process to stop between the backup and activation stages
- Submit to continue
If you have selected the option to create an Image copy backup, once done with the first step of the image copy creation you will get a completed task indicating the image backup was created and validated.
You will then be presented with the option to resume and continue with the activation. Once that is complete you will get the task that indicates the completion of the activation/failover process.
Once the above is complete, you will be able to perform you tests on the newly activate standby.
When you are ready to re-instantiate the standby again you can navigate to the Disaster Recovery Actions page and select the DDC.
You will then select the Perform DR Test and you will get the option REINSTATE, which will allow you to get to the next screen that allows you to continue restoring from the backup taken in the previous step.
...
...
...
- You will get two options if using Image Copy
- Switch to the image copy
- This will be fast as we will switch to the image copy instead of restoring. You will need to be aware that in this scenario the database will be switched to the copy.
- Restore from the image copy
- This method will perform a full restore from the image copy - this process is not as fast but will restore the database into the original location.
- Switch to the image copy
- Once this is selected, click on Submit to start the operation. As mentioned the switching to the image copy is faster.
Once the re-instate is complete, for example, if you have picked to switch to copy the following task details, will be displayed.
4. Disaster Recovery Actions - Using CLI
The DR test and the Read-Only test can also be performed using the command line. The below example will show how it is done. dbvctl -o -h will display the help on the commands which can be used .
[oracle@dbv2 standby]$ ./dbvctl -o -h
...
dbvctl -d <ddc> -o ro_test
dbvctl -d <ddc> -o activate [--force] [--noprompt]
dbvctl -d <ddc> -o dr_test [--backup --backup_type image|backupset --backup_location bck_dir] [--noprompt]
dbvctl -d <ddc> -o reinstate [--switch] [--noprompt]
....
...
4.1 Perform a Read-Only test
This option will help in determining if the database is standby is in a consistent state. This would open the database in read-only mode and the restart the database to mount mode so that the archivelogs can be applied.
[oracle@dbv2 standby]$ ./dbvctl -d DEV -o ro_test ============================================================= Dbvisit Standby Database Technology (9.0.00_0_g28d7024c) (pid 1245) dbvctl started on dbv2: Fri May 31 11:53:44 2019 ============================================================= >>> Running pre-checks please wait... done Open Standby database DEV in READ ONLY mode... Standby database DEV on dbv2 opened in READ ONLY mode. Log files cannot be applied to Database while in READ ONLY mode. Database tempfile(s) may need to be added to this database. Stopping database DEV... Standby Database DEV shutdown successfully on dbv2. Starting database DEV... Standby Database DEV started on dbv2. ============================================================= dbvctl ended on dbv2: Fri May 31 11:53:54 2019 =============================================================
4.2 Perform a DR Test and Re-instate
Use Filesystem/ACFS locations to take backups and image copies of the database. The current release does not support taking backups and image copies in ASM diskgroups
[oracle@dbv2 standby]$ ./dbvctl -d DEV -o dr_test
=============================================================
Dbvisit Standby Database Technology (9.0.00_0_g28d7024c) (pid 1383)
dbvctl started on dbv2: Fri May 31 12:07:47 2019
=============================================================
>>> Running pre-checks please wait... done
>>> No backup of the standby database DEV is available
-------------------------------------------------------------------------------
=>Do you want to backup the standby database prior to activation? [yes]: yes
Your input: 1
Is this correct? <Yes/No> [Yes]:
-------------------------------------------------------------------------------
=>Specify backup type: 1 for image copy, 2 for backupset [1]: 2
Your input: 2
Is this correct? <Yes/No> [Yes]:
-------------------------------------------------------------------------------
=>Specify backup location: []: /home/oracle/temp
Your input: /home/oracle/temp
Is this correct? <Yes/No> [Yes]:
-------------------------------------------------------------------------------
=>Specify backup format: [%U.bkp]:
Your input: %U.bkp
Is this correct? <Yes/No> [Yes]:
-------------------------------------------------------------------------------
=>Do you want to be prompted to confirm activation on successful completion of the backup? [yes]:
Your input: 1
Is this correct? <Yes/No> [Yes]:
>>> Standby database size is 1.39GB. Backing up database might take some time.
>>> Backing up standby database... done
>>> Validating backup of the standby database... done
-------------------------------------------------------------------------------
Standby Backup summary
-------------------------------------------------------------------------------
tag DBV_DEV_1
location /home/oracle/temp
type backupset
db_unique_name DEV
server dbv2
resetlogs_id 1004372670
resetlogs_time 2019-03-31:16:24:30
created_on 2019-05-31 12:09:00
status VALID
Files:
DBV_DEV_1_03u2u0tk_1_1.bkp
DBV_DEV_1_04u2u0ud_1_1.bkp
DBV_DEV_1_CONTROL.CTL
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
=>Do you want to proceed with activating the standby database? [no]: yes
Your input: 1
Is this correct? <Yes/No> [Yes]:
>>> Activating now...
>>> Activation complete. Please ensure a backup of this Database is made
>>> Old archives from before the activation should be removed to avoid mix-up between new
and old archive logs
>>> Process complete
=============================================================
dbvctl ended on dbv2: Fri May 31 12:09:29 2019
=============================================================
Status of the database
[oracle@dbv2 standby]$ ./dbvctl -d DEV -o status ============================================================= Dbvisit Standby Database Technology (9.0.00_0_g28d7024c) (pid 1670) dbvctl started on dbv2: Fri May 31 12:11:29 2019 ============================================================= Database Instance DEV on dbv2 status: Regular database open in read write mode ============================================================= dbvctl ended on dbv2: Fri May 31 12:11:29 2019 =============================================================
Now the next step is to reinstate the database back to the standby database
[oracle@dbv2 standby]$ ./dbvctl -d DEV -o reinstate ============================================================= Dbvisit Standby Database Technology (9.0.00_0_g28d7024c) (pid 1689) dbvctl started on dbv2: Fri May 31 12:12:48 2019 ============================================================= >>> Running pre-checks please wait... >>> Validating backup of the standby database... done done ------------------------------------------------------------------------------- Standby Backup summary ------------------------------------------------------------------------------- tag DBV_DEV_1 location /home/oracle/temp type backupset db_unique_name DEV server dbv2 resetlogs_id 1004372670 resetlogs_time 2019-03-31:16:24:30 created_on 2019-05-31 12:09:00 status VALID Files: DBV_DEV_1_03u2u0tk_1_1.bkp DBV_DEV_1_04u2u0ud_1_1.bkp DBV_DEV_1_CONTROL.CTL ------------------------------------------------------------------------------- ------------------------------------------------------------------------------- >>> Reinstating standby will result in permanent removal of current database files. =>Do you want to view a list of files to be removed? [yes]: yes Your input: 1 Is this correct? <Yes/No> [Yes]: REINSTATE will remove the following files: datafile: /u01/app/oracle/oradata/DEV/system01.dbf /u01/app/oracle/oradata/DEV/sysaux01.dbf /u01/app/oracle/oradata/DEV/undotbs01.dbf /u01/app/oracle/oradata/DEV/users01.dbf tempfile: /u01/app/oracle/oradata/DEV/temp01.dbf controlfile: /u01/app/oracle/oradata/DEV/control01.ctl /u01/app/oracle/fast_recovery_area/DEV/control02.ctl redolog: /u01/app/oracle/oradata/DEV/redo01.log /u01/app/oracle/oradata/DEV/redo02.log /u01/app/oracle/oradata/DEV/redo03.log other: ------------------------------------------------------------------------------- =>Proceed with reinstating? [no]: yes Your input: 1 Is this correct? <Yes/No> [Yes]: >>> Shutting down the database... done >>> Removing database files... done >>> Restoring standby control file... done >>> Starting standby database... done >>> Restoring standby database... done Standby database reinstated. ============================================================= dbvctl ended on dbv2: Fri May 31 12:13:59 2019 =============================================================
Status of the database
[oracle@dbv2 standby]$ ./dbvctl -d DEV -o status ============================================================= Dbvisit Standby Database Technology (9.0.00_0_g28d7024c) (pid 1855) dbvctl started on dbv2: Fri May 31 12:15:10 2019 ============================================================= Database Instance DEV on dbv2 status: Standby Database in recovery mode ============================================================= dbvctl ended on dbv2: Fri May 31 12:15:10 2019 =============================================================
Below is another example where the DR test is performed by using image copy as backup and we switch to copy when reinstating the DR to standby.
[oracle@dbv2 standby]$ ./dbvctl -d DEV -o dr_test
=============================================================
Dbvisit Standby Database Technology (9.0.00_0_g28d7024c) (pid 1266)
dbvctl started on dbv2: Fri May 31 16:22:08 2019
=============================================================
>>> Running pre-checks please wait... done
>>> No backup of the standby database DEV is available
-------------------------------------------------------------------------------
=>Do you want to backup the standby database prior to activation? [yes]:
Your input: 1
Is this correct? <Yes/No> [Yes]:
-------------------------------------------------------------------------------
=>Specify backup type: 1 for image copy, 2 for backupset [1]: 1
Your input: 1
Is this correct? <Yes/No> [Yes]:
-------------------------------------------------------------------------------
=>Specify backup location: []: /home/oracle/temp
Your input: /home/oracle/temp
Is this correct? <Yes/No> [Yes]:
-------------------------------------------------------------------------------
=>Specify backup format: [%U.dbf]:
Your input: %U.dbf
Is this correct? <Yes/No> [Yes]:
-------------------------------------------------------------------------------
=>Do you want to be prompted to confirm activation on successful completion of the backup? [yes]:
Your input: 1
Is this correct? <Yes/No> [Yes]:
>>> Standby database size is 1.39GB. Backing up database might take some time.
>>> Backing up standby database... done
>>> Validating backup of the standby database... done
-------------------------------------------------------------------------------
Standby Backup summary
-------------------------------------------------------------------------------
tag DBV_DEV_1
location /home/oracle/temp
type image
db_unique_name DEV
server dbv2
resetlogs_id 1004372670
resetlogs_time 2019-03-31:16:24:30
created_on 2019-05-31 16:23:29
status VALID
Files:
DBV_DEV_1_08u2ufri_1_1.dbf
DBV_DEV_1_cf_D-DEV_id-4082711483_06u2ufrg.dbf
DBV_DEV_1_CONTROL.CTL
DBV_DEV_1_data_D-DEV_I-4082711483_TS-SYSAUX_FNO-2_04u2ufrc.dbf
DBV_DEV_1_data_D-DEV_I-4082711483_TS-SYSTEM_FNO-1_03u2ufr9.dbf
DBV_DEV_1_data_D-DEV_I-4082711483_TS-UNDOTBS1_FNO-3_05u2ufrf.dbf
DBV_DEV_1_data_D-DEV_I-4082711483_TS-USERS_FNO-4_07u2ufrh.dbf
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
=>Do you want to proceed with activating the standby database? [no]: yes
Your input: 1
Is this correct? <Yes/No> [Yes]:
>>> Activating now...
>>> Activation complete. Please ensure a backup of this Database is made
>>> Old archives from before the activation should be removed to avoid mix-up between new
and old archive logs
>>> Process complete
=============================================================
dbvctl ended on dbv2: Fri May 31 16:24:05 2019
=============================================================
Reinstate using the switch to copy function
[oracle@dbv2 standby]$ ./dbvctl -d DEV -o reinstate --switch ============================================================= Dbvisit Standby Database Technology (9.0.00_0_g28d7024c) (pid 1559) dbvctl started on dbv2: Fri May 31 16:24:33 2019 ============================================================= >>> Running pre-checks please wait... >>> Validating backup of the standby database... done done ------------------------------------------------------------------------------- Standby Backup summary ------------------------------------------------------------------------------- tag DBV_DEV_1 location /home/oracle/temp type image db_unique_name DEV server dbv2 resetlogs_id 1004372670 resetlogs_time 2019-03-31:16:24:30 created_on 2019-05-31 16:23:29 status VALID Files: DBV_DEV_1_08u2ufri_1_1.dbf DBV_DEV_1_cf_D-DEV_id-4082711483_06u2ufrg.dbf DBV_DEV_1_CONTROL.CTL DBV_DEV_1_data_D-DEV_I-4082711483_TS-SYSAUX_FNO-2_04u2ufrc.dbf DBV_DEV_1_data_D-DEV_I-4082711483_TS-SYSTEM_FNO-1_03u2ufr9.dbf DBV_DEV_1_data_D-DEV_I-4082711483_TS-UNDOTBS1_FNO-3_05u2ufrf.dbf DBV_DEV_1_data_D-DEV_I-4082711483_TS-USERS_FNO-4_07u2ufrh.dbf ------------------------------------------------------------------------------- ------------------------------------------------------------------------------- >>> Reinstating standby will result in permanent removal of current database files. =>Do you want to view a list of files to be removed? [yes]: Your input: 1 Is this correct? <Yes/No> [Yes]: REINSTATE will remove the following files: datafile: /u01/app/oracle/oradata/DEV/system01.dbf /u01/app/oracle/oradata/DEV/sysaux01.dbf /u01/app/oracle/oradata/DEV/undotbs01.dbf /u01/app/oracle/oradata/DEV/users01.dbf tempfile: /u01/app/oracle/oradata/DEV/temp01.dbf controlfile: /u01/app/oracle/oradata/DEV/control01.ctl /u01/app/oracle/fast_recovery_area/DEV/control02.ctl redolog: /u01/app/oracle/oradata/DEV/redo01.log /u01/app/oracle/oradata/DEV/redo02.log /u01/app/oracle/oradata/DEV/redo03.log other: ------------------------------------------------------------------------------- =>Do you want to switch standby database to the latest image copy at /home/oracle/temp? [yes]: Your input: 1 Is this correct? <Yes/No> [Yes]: ------------------------------------------------------------------------------- =>Proceed with reinstating? [no]: yes Your input: 1 Is this correct? <Yes/No> [Yes]: >>> Shutting down the database... done >>> Removing database files... done >>> Starting up database nomount... done >>> Renaming control file... done >>> Shutting database down... done >>> Starting standby database... done >>> Switching database to copy... done Standby database reinstated. ============================================================= dbvctl ended on dbv2: Fri May 31 16:25:20 2019 =============================================================
5. Video - How to Perform a DR Test in Dbvisit Standby v9