Page Comparison

In Oracle to SQL Server replication, If if source Oracle database is running on Linux then setup wizard is executed on target Windows machine where SQL Server database is running.  

...

3rd party products such as MS SSMA or Navicat Premium 

...

(Navicat is recommended only for small tables) can be used to load data from oracle to SQL Server in order to sync source and target environment before initiating replication.

Setup Wizard Example

This example shows an Oracle-to-SQL Server one-way replication using the following information.

First of all open the Dbvisit Replicate command console from the start menu and execute 'setup wizard' command.

dbvrep> setup wizard

This wizard configures Dbvisit Replicate to start a replication process.

The setup wizard creates configuration scripts, which need to be run after the wizard ends. No changes to the databases are

made before that.

The progress is saved every time a list of databases, replications, etc. is shown. It will be re-read if wizard is restarted

and the same DDC name and script path is selected.

Run the wizard now? [yes] yes

Accept end-user license agreement? (view/yes/no) [view] yes

Before starting the actual configuration, some basic information is needed. The DDC name and script path determines where

all files created by the wizard go (and where to reread them if wizard is rerun) and the license key determines which

options are available for this configuration.

(DDC_NAME) - Please enter a name for this replication (suggestion: use the name of the source database): [] reptest2

(LICENSE_KEY) - Please enter your license key (or just enter "(trial)"): [(trial)]

(SETUP_SCRIPT_PATH) - Please enter a directory for location of configuration scripts on this machine:

[C:\Users\oracle\Documents\reptest2]

Network configuration (TNS)

Dbvisit Replicate will detect tnsnames.ora locations on your system. Choose the TNS configuration that is applicable.

Network configuration files were detected on this system in these locations:

C:\app\oracle\product\11.2.0\grid\network\admin

C:\app\oracle\product\11.2.0\dbhome_1\network\admin

<oracle\product\11.2.0\grid\network\admin] C:\app\oracle\product\11.2.0\dbhome_1\network\admin

Describing the databases

During this step information on the databases used in the replication is collected. The result is a SQL*Plus  SQLCMD scripts that recreates the schema user and grants it required privileges.

Please note that being a SQL*Plus/SQLCMD script, the resulting file contains plain text passwords.

Describe the first database:

Step 1 - Describe databases

========================================

The first step is to describe databases used in the replication. There are usually two of them (source and target); however,

there can be just one (when source and target is the same) or more than two (one-to-many or other complex configurations.)

Let's configure the database, describing it's type, connectivity, user names etc.

What type of database is this? (Oracle/MySQL/MSSQL): [Oracle]

Please enter database TNS alias: [] reptest2

Please enter SYSDBA user name: [SYS]

Please enter password for this user: [change_on_install] ******

Please enter user with DBA role: [SYSTEM]

Please enter password for this user: [manager] ******

Connecting to database reptest2 as SYSTEM to query list of tablespaces and to detect ASM (by looking whether any redo logs

or archived logs are stored in ASM).

Enter the Dbvisit Replicate owner and apply user (this user will be created by this script): [dbvrep]

Please enter password for this user: [dbvpasswd]

Permanent tablespaces detected on the database: USERS.

Please enter default permanent tablespace for this user: [USERS]

Temporary tablespaces detected on the database: TEMP.

Please enter default temporary tablespace for this user: [TEMP]

Following databases are now configured:

1: Oracle reptest2, SYS/***, SYSTEM/***, dbvrep/***, USERS/TEMP, dbvrep/, ASM:NO, TZ: +00:00

Enter number of database to modify it, or "add", or "done": [add]

Adding the second database:

Let's configure the database, describing it's type, connectivity, user names etc.

What type of database is this? (Oracle/MySQL/MSSQL): [Oracle] MSSQL

Please enter server name (as specified in "sqlcmd -S", e.g. WINSVR1\SQLEXPRESS): [localhost] dbvisit420

Please enter database ODBC DNS: [] mstest

Please enter user name of an administrator: [SA] dbvuser

Please enter password for this user: [password] *******

Enter the user to log into apply database: [SA] dbvuser

Please enter password for this user: [password] *******

Enter the database to use: [dbvrep]

Enter the schema the table are created by the user specified: [dbo]

Enter the database with the user data: [] w420g

Following databases are now configured:

1: Oracle reptest2, SYS/***, SYSTEM/***, dbvrep/***, USERS/TEMP, dbvrep/, ASM:NO, TZ: +00:00

2: MSSQL mstest, dbvuser/***, dbvuser/***, dbvuser/***, /, dbvrep/dbo, ASM:n/a, TZ:

Enter number of database to modify it, or "add", or "done": [done]

Replication pairs

During this step the source and target databases are set for each replication pair. In Oracle to SQL Server replication, Only Oracle database can ne source. 

Step 2 - Replication pairs

========================================

The second step is to set source and targets for each replication pair. This is usually just choosing the first database a

source and the second one as target, but many more configurations are possible.

Let's configure the replication pair, selecting source and target.

Following databases are described:

1: reptest2 (Oracle)

2: mstest (MSSQL) (cannot be source, is not Oracle)

Select source database: [1]

Select target database: [2]

Use fetcher to offload the mining to a different server? (yes/no) [no]

(NETWORK_QUALITY) - Please specify your network type (LAN or WAN). Autoconfigures timeouts, use of compression etc. [LAN]

Lock and copy the data initially one-by-one or at a single SCN? (one-by-one/single-scn/ddl-only/resetlogs) [single-scn]

Following replication pairs are now configured:

1: reptest2 (Oracle) ==> mstest (MSSQL), DDL: no, fetcher: no, process suffix: (no suffix), network: LAN, prepare type:

single-scn, data load: ddl_file

Enter number of replication pair to modify it, or "add", or "done": [done]

Schema/Tables to be replicated

The third step is to choose the schemas and tables to be replicated.You can specify coma separated list of multiple tables or/and schema.

Step 3 - Replicated tables

========================================

The third step is to choose the schemas and tables to be replicated. If the databases are reachable, the tables are checked

for existence, datatype support, etc., schemas are queried for tables. Note that all messages are merely hints/warnings and

may be ignored if issues are rectified before the scripts are actually executed.

Note the following assumptions are made in this wizard ΓÇô which can be modified by editing the resulting script:

1. All replicated tables have a primary key defined.

2. All columns of the tables and all tables of the specified schemas are replicated, and if DDL support is enabled, whenever

a new column/table is added, this should be replicate as well.

3. If an apply conflict arises, the default option is to try again repeatedly, until a different option is given or the

underlying issue is resolved.

4. If an apply conflict arises, no more data will be replicated until the issue is resolved or ignored.

You can also specify some advanced options:

1. Rename schemas or tables.

2. Specify filtering conditions.

3. (Tables only) Configure Change Data Capture; this does not maintain a copy of the source table, but logs all operations

as separate entries. This is useful for ETL or as an audit trail. This usually requires adding of new columns (timestamps,

old/new values etc.) to the target table. Dbvisit Replicate can create the target table with the additional columns during

setup by selecting "ddl_run" in the data copy section of Step 2.

Following tables are defined for replication pairs:

1: reptest2 (Oracle) ==> mstest (MSSQL), DDL: no, suffix: (no suffix), prepare: single-scn

 No tables defined.

Enter number of replication pair to modify it, or "done": [1]

Please enter list of all individual tables to be replicated. Enter schema name(s) only to replicate all tables in that

schema. Use comma or space to delimit the entries.

Enter the tables and schemas: [] avi.sample

Selected tables: avi.sample

Add more tables or schemas? (YES/NO) [NO]

Specify rename name, filter condition, CDC/Audit/ETL for any of the specified tables (yes/no): [no] yes

Rename SCHEMA name for avi.sample (empty means no rename): [w420g.dbo] 

Rename TABLE name for avi.sample (empty means no rename): []

Filter the data to be replicated? If yes, use single condition for ALL DML operations or use CUSTOM conditions for each

operation? (NO/ALL/CUSTOM) [NO]

Configure change data capture for change auditing or real-time BI? (NO/YES) [NO]

Following tables are defined for replication pairs:

1: reptest2 (Oracle) ==> mstest (MSSQL), DDL: no, suffix: (no suffix), prepare: single-scn

 avi.sample => w420g.dbo.sample

Enter number of replication pair to modify it, or "done": [done]

...

Add process information for the MINE process:

Step 4 - Process configuration

========================================

The fourth step is to configure the replication processes for each replication. Although most options have reasonable

defaults, manual input is required.

Following processes are defined:

1: MINE on reptest2

 Not configured.

2: APPLY on mstest

 Not configured.

Enter number of process to modify it, or "done": [1]

Fully qualified name of the server for the process (usually co-located with the database, unless mine is offloaded using

fetcher): [dbvldemo102]

Server type (Windows/Linux/Unix): [Linux]

Enable email notifications about problems (yes/no)? [YES] no

Enable SNMP traps/notifications about problems (yes/no)? [NO]

Directory with DDC file and default where to create log files etc. (recommended: same as global setting, if possible)?

[/home/oracle/reptest2]

Following settings were pre-filled with defaults or your reloaded settings:

----------------------------------------

[MINE_REMOTE_INTERFACE]: Network remote interface: dbvldemo102:7901

[MINE_DATABASE]: Database TNS: reptest2

[TNS_ADMIN]: tnsnames.ora path: C:\app\oracle\product\11.2.0\dbhome_1\network\admin

[MINE_USER]: Dbvisit Replicate database username: dbvrep

[MINE_PASSWORD]: Dbvisit Replicate database password: *********

[MINE_PLOG]: Filemask for generated plogs: /home/oracle/reptest2/mine/%S.%E (%S is sequence, %T thread, %F original filename

(stripped extension), %P process type, %N process name, %E default extension)

[LOG_FILE]: General log file: /home/oracle/reptest2/log/dbvrep_%N_%D.%E

[LOG_FILE_TRACE]: Error traces: /home/oracle/reptest2/log/trace/dbvrep_%N_%D_%I_%U.%E

Checking that these settings are valid...

Do you want change any of the settings? [no]

Following processes are defined:

1: MINE on reptest2

 Host: dbvldemo102, SMTP: NO, SNMP: NO

2: APPLY on mstest

 Not configured.

Add process information for the APPLY process:

Enter number of process to modify it, or "done": [2]

Fully qualified name of the server for the process (usually co-located with the database, unless mine is offloaded using

fetcher): [] dbvisit420.dbvisitdomain.dbvisit.com

Server type (Windows/Linux/Unix): [] Windows

Enable email notifications about problems (yes/no)? [YES] no

Enable SNMP traps/notifications about problems (yes/no)? [NO]

Directory with DDC file and default where to create log files etc. (recommended: same as global setting, if possible)?

[C:\Users\oracle\Documents\reptest2]

Following settings were pre-filled with defaults or your reloaded settings:

----------------------------------------

[APPLY_REMOTE_INTERFACE]: Network remote interface: dbvisit420.dbvisitdomain.dbvisit.com:7902

[APPLY_DATABASE]: Database ODBC DNS: mstest

[TNS_ADMIN]: tnsnames.ora path: C:\app\oracle\product\11.2.0\dbhome_1\network\admin

[APPLY_USER]: Dbvisit Replicate database username: dbvuser

[APPLY_PASSWORD]: Dbvisit Replicate database password: ********

[APPLY_SCHEMA]: Dbvisit Replicate database: dbvrep

[APPLY_SCHEMA2]: Dbvisit Replicate schema: dbo

[APPLY_STAGING_DIR]: Directory for received plogs: C:\Users\oracle\Documents\reptest2\apply

[LOG_FILE]: General log file: C:\Users\oracle\Documents\reptest2\log\dbvrep_%N_%D.%E

[LOG_FILE_TRACE]: Error traces: C:\Users\oracle\Documents\reptest2\log\trace\dbvrep_%N_%D_%I_%U.%E

Checking that these settings are valid...

Do you want change any of the settings? [no]

Following processes are defined:

1: MINE on reptest2

 Host: dbvldemo102, SMTP: NO, SNMP: NO

2: APPLY on mstest

 Host: dbvisit420.dbvisitdomain.dbvisit.com, SMTP: NO, SNMP: NO

Enter number of process to modify it, or "done": [done]

Final wrap-up

All the information about the replication process has been obtained and now the replication environment can be created.

Created file C:\Users\oracle\Documents\reptest2\reptest2-APPLY.ddc.

Created file C:\Users\oracle\Documents\reptest2\reptest2-MINE.ddc.

Created file C:\Users\oracle\Documents\reptest2\reptest2-setup.dbvrep.

Created file C:\Users\oracle\Documents\reptest2\reptest2-dbsetup_reptest2.sql.

Created file C:\Users\oracle\Documents\reptest2\reptest2-dbsetup_mstest.sql.

Created file C:\Users\oracle\Documents\reptest2\reptest2-grants_reptest2.sql.

Created file C:\Users\oracle\Documents\reptest2\reptest2-grants_mstest.sql.

Created file C:\Users\oracle\Documents\reptest2\reptest2-onetime.ddc.

Created file C:\Users\oracle\Documents\reptest2\start-console.bat.

Created file C:\Users\oracle\Documents\reptest2\reptest2-run-dbvisit420.dbvisitdomain.dbvisit.com.bat.

Created file C:\Users\oracle\Documents\reptest2\reptest2-win_services-dbvisit420.dbvisitdomain.dbvisit.com.bat.

Created file C:\Users\oracle\Documents\reptest2\reptest2-run-dbvldemo102.sh.

Created file C:\Users\oracle\Documents\reptest2\Nextsteps.txt.

Created file C:\Users\oracle\Documents\reptest2\reptest2-all.bat.

===========================================================================================================================

Dbvisit Replicate wizard completed

Script C:\Users\oracle\Documents\reptest2\reptest2-all.bat created. This runs all the above created scripts. Please exit out

of dbvrep, review and run script as current user to setup and start Dbvisit Replicate.

===========================================================================================================================

dbvrep> exit

...

Running the reptest2-all.sh bat script will configure and initiate the replication. It does the following:

• Create the Dbvisit Replicate schemas in the source and target databases.
• Grant the necessary privileges to the Dbvisit Replicate schema.
• Create the Dbvisit Replicate repository in the source and target databases.
• Load the Dbvisit Replicate configuration file (DDC) into the Dbvisit Replicate repository. The DDC file is like the init.ora for Oracle and contains the settings for the replication.
• Prepare the schemas or objects for replication.

Example of running the reptest1the reptest2-all.bat

C:\Users\oracle\Documents\reptest2>reptest2-all.bat

Setting up Dbvisit Replicate configuration

Configure database reptest2...

Configure database mstest...

Msg 3701, Level 11, State 1, Server DBVISIT420\SQLTEST, Line 1

Cannot drop the database 'dbvrep', because it does not exist or you do not have permission.

Object grants for database reptest2...

Object grants for database mstest...

Setting up the configuration

Initializing......done

WARN-1850: No DDC DB available, dictionary table does not exist.

DDC loaded from database (0 variables).

Dbvisit Replicate version 2.

7
Copyright (C) Dbvisit Software Limited. All rights reserved.

DDC file C:\Users\oracle\Documents\reptest2\reptest2-onetime.ddc

loaded.

MINE: Cannot determine Dbvisit Replicate dictionary version. (no

dictionary exists)

APPLY: Cannot determine Dbvisit Replicate dictionary version. (no

dictionary exists)

dbvrep> set ON_WARNING SKIP

Variable ON_WARNING set to SKIP for process *.

dbvrep> set ON_ERROR EXIT

Variable ON_ERROR set to EXIT for process *.

dbvrep> ENGINE SWITCH_REDOLOG

dbvrep> ENGINE SETUP MINE DROP DICTIONARY

0 dictionary objects dropped.

dbvrep> ENGINE SETUP MINE CREATE DICTIONARY

dbvrep> ENGINE SETUP MINE LOAD DICTIONARY

Supplemental logging on database set.

Loading dictionary table DBRSCOL$

Loading dictionary table DBRSOBJ$

Loading dictionary table DBRSTAB$

Loading dictionary table DBRSUSER$

Loading dictionary table DBRSV_$DATABASE

dbvrep> ENGINE SETUP APPLY DROP DICTIONARY

0 dictionary objects dropped.

dbvrep> ENGINE SETUP APPLY CREATE DICTIONARY

dbvrep> ENGINE SETUP APPLY LOAD DICTIONARY

dbvrep> ENGINE SETUP PAIR MINE AND APPLY

ID of mine proces is 8EFF00AA-6D9B-1014-B197-EC251B966082. If not using

DDC in database, set MINE_UNIQUE_ID to this value.

1 applier SCN set.

dbvrep> SET APPLY.INSTANTIATE_SCN NOW

Variable INSTANTIATE_SCN set to NOW for process APPLY.

dbvrep> memory_set IGNORE_APPLY_DDL_DIFFERENCES YES

Variable IGNORE_APPLY_DDL_DIFFERENCES set to YES for process *.

dbvrep> ENGINE SUPPLEMENTAL LOGGING TABLE avi.sample ENABLE PRIMARY KEY

dbvrep> ENGINE SWITCH_REDOLOG

dbvrep> #single-scn instantiation: lock all tables and schemas

dbvrep> ENGINE LOCK TABLES avi.sample

Locking all tables.

Lock done.

dbvrep> #single-scn instantiation: unlock all tables and schemas, but

keep the SCN

dbvrep> ENGINE LOCK RELEASE LOCKS

dbvrep>

dbvrep> #prepare the tables (we use OFFLINE as neither MINE not APPLY

is running; with OFFLINE we won't wait on network timeout)

dbvrep> PREPARE OFFLINE TABLE avi.sample NODDL RENAME TO

w420g.dbo.sample

Table avi.sample instantiated at SCN 1413199

dbvrep> #single-scn instantiation: unlock all tables and schemas,

forget the SCN (so it does not affect any further PREPARE statements)

dbvrep> ENGINE LOCK CLEAR SCN

dbvrep> ENGINE SWITCH_REDOLOG

dbvrep> #prepare script for instantiation

dbvrep> ENGINE PREPARE_DP WRITE DDL_FILE FILE

C:\Users\oracle\Documents\reptest2\APPLY.sql USERID

SYSTEM/

password@reptest2
Created DDL script C:\Users\oracle\Documents\reptest2\APPLY.sql.

dbvrep> create ddcdb from ddcfile

DDC loaded into database (204 variables).

dbvrep> set ON_WARNING SKIP

Variable ON_WARNING set to SKIP for process *.

dbvrep> set ON_ERROR SKIP

Variable ON_ERROR set to SKIP for process *.

OK-0: Completed successfully.

These steps are required after the reptest2-all.bat script runs:

1) Create the necessary directory(ies) on the servers:

dbvisit420.dbvisitdomain.dbvisit.com: C:\Users\oracle\Documents\reptest2

dbvldemo102: /home/oracle/reptest2

2) Copy the DDC files to the server(s) where the processes will run:

dbvisit420.dbvisitdomain.dbvisit.com: C:\Users\oracle\Documents\reptest2\reptest2-APPLY.ddc

dbvldemo102: C:\Users\oracle\Documents\reptest2\reptest2-MINE.ddc

3) Review that path to dbvrep executable is correct in the run scripts:

C:\Users\oracle\Documents\reptest2\reptest2-run-dbvisit420.dbvisitdomain.dbvisit.com.bat

C:\Users\oracle\Documents\reptest2\reptest2-run-dbvldemo102.sh

4) Review and run the scripts for Windows to create services (run with Administrator privileges):

C:\Users\oracle\Documents\reptest2\reptest2-win_services-dbvisit420.dbvisitdomain.dbvisit.com.bat

5) Copy the run script to the server(s) where the processes will run:

C:\Users\oracle\Documents\reptest2\reptest2-run-dbvisit420.dbvisitdomain.dbvisit.com.bat

C:\Users\oracle\Documents\reptest2\reptest2-run-dbvldemo102.sh

6) Ensure firewall is open for listen interfaces 0.0.0.0:7902, 0.0.0.0:7901 used by the processes.

7) Make sure the data on apply are in sync as of time when setup was run.

Scripts for Data Pump/export/DDL were created as requested:

C:\Users\oracle\Documents\reptest2\APPLY.sql

Create referenced database links (if any) before running the scripts.

8) Start the replication processes on all servers:

C:\Users\oracle\Documents\reptest2\reptest2-run-dbvisit420.dbvisitdomain.dbvisit.com.bat

C:\Users\oracle\Documents\reptest2\reptest2-run-dbvldemo102.sh

9) Start the console to monitor the progress:

C:\Users\oracle\Documents\reptest2\start-console.bat

The above list is stored in C:\Users\oracle\Documents\reptest2\Nextsteps.txt.

Press any key to continue . . .

The replication has been configured and installed. Steps required to start replication are saved in Nextsteps.txt file.

...

Starting the console

The dbvrep console can be started by executing start-console.bat script to monitor the real time status of the replication.

C:\Users\oracle\Documents\reptest2>start-console.

bat 

The console can only be started on Windows machine (Target). Future release will overcome setup-wizard and console limitations.  

Following is the reptest2 replication console showing current status of replication.

...

...

...

...

 Initialization of APPLY process may take 5 to 7 minutes

-MINE IS running. Currently at plog 425 and SCN 12377134 (09/05/2012 21:58:15).

APPLY IS running. Currently at plog 425 and SCN 12377168 (09/05/2012 21:58:27).

Progress of replication reptest1:MINE->APPLY: total/this execution

---------------------------------------------------------------------------------------------------------------------------------

AVI.SAMPLE:

                   100%  Mine:1/1       

 Unrecov:0/0     

 Applied:1/1 

 Conflicts:0/0    

 Last:05/09/2012 21:58:30/OK

---------------------------------------------------------------------------------------------------------------------------------

1 tables listed.