1. Introduction

In this section, we will cover specific topics around troubleshooting Dbvisit Standby version 9.

2. Creating a support package

One of the most important steps you should perform when logging a support call is to generate a support package.

This section will cover the process in more detail and show you how easy it is to create a support package.

2.1. Using the Command Line Interface (dbvctl)

When running Dbvisit Standby processes, the Process ID or "pid" is displayed in the header. In the example below, we can see the (pid 4085).

=============================================================
Dbvisit Standby Database Technology (9.0.0_1271_g4b00fb49) (pid 4085)
dbvctl started on dbv1: Tue May 14 12:45:36 2019
=============================================================
...
...

Now that we have the process id, we can create a support package with the command: dbvctl -d <ddc> -f support_package [-a pid=<pid>]

Unless you are aware of the specific PID process that is having a problem use this command: dbvctl -d <ddc> -f support_package. This would create a generic support package, collecting database, Dbvisit Standby configuration details required by the support team.

Example:

In the example below support, the package is created for process id 4085.

[oracle@dbv1 standby]$ ./dbvctl -d DEV -f support_package -a pid=4085
Support package /usr/dbvisit/standby/support/4085_DEV_201905141247.tar.gz created.
[oracle@dbv1 standby]$ ls -lrt support/4085_DEV_201905141247.tar.gz
-rw-r--r-- 1 oracle oinstall 235048 May 14 12:47 support/4085_DEV_201905141247.tar.gz
[oracle@dbv1 standby]$

Once the support package is created in the DBVISIT_BASE/standby/support subfolder, you can upload this to the Dbvisit Support team to assist.

2.2. Using the Central Console

This section will show you how to create a support package using the Central Console.

There are two methods for creating a support package:

On completion or failure of a task
1. At the completion of each Active task or if any error is detected in the Central Console the option to create and then download the support package will be presented at the top of the task output.
Using the Create Support Package option from the Active Task List menu (bottom bar)
1. Using this option you can create a support package for a specific server (primary or standby) for a specific DDC file, or you can create a support package for a specific PID for a DDC file.
2. The steps to do this is described using the images below:

Step 1: Click on "Create Support Package" (1), then select the Dbvisit Standby Configuration (DDC) (2) - see image below:

Step 2: Select the specific Host (on which the Support Package will be created) for the specific DDC (2) - see image below:

Step 3: If you want to create a support package for a specific PID (4 - below) specify the pid, otherwise, click on submit to create a generic support package.

Step 4: Download the support package (3) below. Once downloaded upload the support package to the support ticket if you have one logged with the Dbvisit Support team.

DEV

3. General Troubleshooting

3.1. Error opening file for writing

During an upgrade on a Windows-based system (if you are installing version 8.0.06 and above) you need to make sure that all Dbvisit Standby Schedules and Services are stopped prior to starting the installer.

If you get an error "Error opening file for writing" while running the Windows installer, you have not stopped the Dbvisit services, which means the dbvsmgr.exe application is in use and cannot be updated.

Stop the installer, stop the Dbvisit services and restart the installer.

3.2. Troubleshooting - "specify valid username and password"

When installing Dbvisit Standby version 9, it is important to make sure the user that is running the installer is in the Local Administrator group.

The user that is used to Run the dbvisit services - username and password are asked during the installer - must be in the following groups:

Local Administrators
ORA_DBA
USERS

This user must also have the "Logon as a service" permission.

For more detail on how to do this please see the example below:

4. Graceful Switchover

This section will contain notes with regards to Graceful Switchover and possible errors and their fixes.

4.1. GS Fail due to NON-OMF redo logs and OMF redo logs expected

The following error is shown when performing a Graceful Switchover (GS) between two Windows-based Databases. This is the second Graceful Switchover, meaning a switch was performed between primary and standby, where the primary database was using ASM based storage and standby not (filesystem based). The second switch over is when we want to move from the filesystem based primary database to the ASM standby.

Error message:

Dbvisit Standby terminated... 
Error Code: 1 
Remote execution error on KIWI301. 

========================Remote Output: KIWI301======================== 
Dbvisit Standby terminated... 
Error Code: 2147 
Dbvisit Standby cannot proceed: 
Cannot create standby redo log 
C:\APP\ORACLE\ORADATA\DEV\ONLINELOG\GROUP_1.272.933526145 for primary log 
C:\APP\ORACLE\ORADATA\DEV\ONLINELOG\GROUP_1.272.933526145 as ASM file. 
Cannot create standby redo log 
C:\APP\ORACLE\ORADATA\DEV\ONLINELOG\GROUP_1.276.933526145 for primary log 
C:\APP\ORACLE\ORADATA\DEV\ONLINELOG\GROUP_1.276.933526145 as ASM file. 
Cannot create standby redo log 
C:\APP\ORACLE\ORADATA\DEV\ONLINELOG\GROUP_2.268.933526147 for primary log 
C:\APP\ORACLE\ORADATA\DEV\ONLINELOG\GROUP_2.268.933526147 as ASM file. 
Cannot create standby redo log 
C:\APP\ORACLE\ORADATA\DEV\ONLINELOG\GROUP_2.269.933526147 for primary log 
C:\APP\ORACLE\ORADATA\DEV\ONLINELOG\GROUP_2.269.933526147 as ASM file. 
Cannot create standby redo log 
C:\APP\ORACLE\ORADATA\DEV\ONLINELOG\GROUP_3.265.933526151 for primary log 
C:\APP\ORACLE\ORADATA\DEV\ONLINELOG\GROUP_3.265.933526151 as ASM file. 
Cannot create standby redo log 
C:\APP\ORACLE\ORADATA\DEV\ONLINELOG\GROUP_3.266.933526151 for primary log 
C:\APP\ORACLE\ORADATA\DEV\ONLINELOG\GROUP_3.266.933526151 as ASM file. 
Review the following standby database parameters: 
	db_create_file_dest = +DATA 
	db_create_online_log_dest_1 = 
	db_create_online_log_dest_2 = 
	db_create_online_log_dest_3 = 
	db_create_online_log_dest_4 = 
	db_create_online_log_dest_5 = 
	db_recovery_file_dest = +DATA 
	log_file_name_convert = 
Tracefile from server: KIWI301 (PID:2796) 
2796_dbvctl_f_gs_get_info_standby_DEV_201701251733.trc

In this example, the primary database is running on Windows and using ASM storage with Oracle Managed Files configured (OMF)

The standby database was created on a Windows system using Filesystem Based storage.

But during the Standby Creation instead of having proper OMF configuration, the convert parameters where set.

Standby Server Details: Using ASM (this was the original primary)

SQL> select * from v$logfile;

    GROUP# STATUS  TYPE    MEMBER                                             IS_     CON_ID
---------- ------- ------- -------------------------------------------------- --- ----------
         1         ONLINE  +DATA                                              YES          0
         1         ONLINE  +DATA                                              YES          0
         2         ONLINE  +DATA                                              YES          0
         2         ONLINE  +DATA                                              YES          0
         3         ONLINE  +DATA                                              YES          0
         3         ONLINE  +DATA                                              YES          0

6 rows selected.

SQL> show parameter convert

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_file_name_convert                 string
log_file_name_convert                string
pdb_file_name_convert                string
SQL> show parameter create

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
create_bitmap_area_size              integer     8388608
create_stored_outlines               string
db_create_file_dest                  string      +DATA
db_create_online_log_dest_1          string
db_create_online_log_dest_2          string
db_create_online_log_dest_3          string
db_create_online_log_dest_4          string
db_create_online_log_dest_5          string
SQL> show parameter recovery

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_recovery_file_dest                string      +DATA
db_recovery_file_dest_size           big integer 6420M
recovery_parallelism                 integer     0

Primary System Details: Using Filesystem (this was the original standby)

SQL> show parameter create

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
create_bitmap_area_size              integer     8388608
create_stored_outlines               string
db_create_file_dest                  string      c:\app\oracle\oradata
db_create_online_log_dest_1          string
db_create_online_log_dest_2          string
db_create_online_log_dest_3          string
db_create_online_log_dest_4          string
db_create_online_log_dest_5          string
SQL> show parameter recovery

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_recovery_file_dest                string      c:\app\oracle\oradata
db_recovery_file_dest_size           big integer 6420M
recovery_parallelism                 integer     0
SQL> show parameter convert

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_file_name_convert                 string      +DATA, c:\app\oracle\oradata
log_file_name_convert                string      +DATA, c:\app\oracle\oradata
pdb_file_name_convert                string

One way to show that these files are not OMF files are to add a new one, example below - you will see how the new OMF file names are different:

SQL> alter database add logfile group 4 size 50M;

Database altered.

SQL> select * from v$logfile;

    GROUP# STATUS  TYPE    MEMBER                                                                        IS_     CON_ID
---------- ------- ------- -------------------------------------------------------------------------------- --- ----------
         1         ONLINE  C:\APP\ORACLE\ORADATA\DEV\ONLINELOG\GROUP_1.272.933526145                        NO           0
         1         ONLINE  C:\APP\ORACLE\ORADATA\DEV\ONLINELOG\GROUP_1.276.933526145                        NO           0
         2         ONLINE  C:\APP\ORACLE\ORADATA\DEV\ONLINELOG\GROUP_2.268.933526147                        NO           0
         2         ONLINE  C:\APP\ORACLE\ORADATA\DEV\ONLINELOG\GROUP_2.269.933526147                        NO           0
         3         ONLINE  C:\APP\ORACLE\ORADATA\DEV\ONLINELOG\GROUP_3.265.933526151                        NO           0
         3         ONLINE  C:\APP\ORACLE\ORADATA\DEV\ONLINELOG\GROUP_3.266.933526151                        NO           0
         4         ONLINE  C:\APP\ORACLE\ORADATA\DEV\ONLINELOG\O1_MF_4_D8LODWLK_.LOG                        NO           0
         4         ONLINE  C:\APP\ORACLE\ORADATA\DEV\ONLINELOG\O1_MF_4_D8LODWTC_.LOG                        YES          0

8 rows selected.

SQL>

The Solution:

The solution in this scenario is to do the following:

From the Primary which is currently using Filesystem Based Storage, remove (drop) redo log groups 1,2 and 3 which is not true OMF files, then recreate them as OMF redo (same as group 4 above)
Remove the db_file_name_convert and log_file_name_convert parameters
Run a number of log switches
Send logs to the standby
Apply logs
Run Log Gap Report
If the report shows 0 archive log gap, re-start the Graceful Switchover.

Dbvisit Standby 9.0 User Guide

Troubleshooting