Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Current »

In Oracle to SQL Server replication,If both source and target databases are running on Windows OS then setup wizard is executed on source Windows machine. Navicat Premium  can be used to load data from oracle to SQL Server in order to sync source and target environment before initiating replication.

Click here for prerequisite checklist.

Setup Wizard Example

This example show an Oracle-to-SQL Server one-way replication using the following information.

Server/DatabaseName
Source databasereptest2 (Oracle)
Source serverdbvisit410 (Windows 2008)
Target databasew420g (SQL Server)
Target serverdbvisit420 (Windows 2008)
TNS alias for source database   reptest2
System DSNmstest

 

Throughout the wizard, defaults are shown in [brackets]. Just press enter to accept them. The replication name in this example is "reptest2".

 

dbvrep> setup wizard
This wizard configures Dbvisit Replicate to start a replication process.
The setup wizard creates configuration scripts, which need to be run after the wizard ends. No changes to the databases are
made before that.
The progress is saved every time a list of databases, replications, etc. is shown. It will be re-read if wizard is restarted
and the same DDC name and script path is selected.
Run the wizard now? [yes]
Accept end-user license agreement? (view/yes/no) [view] yes
Before starting the actual configuration, some basic information is needed. The DDC name and script path determines where
all files created by the wizard go (and where to reread them if wizard is rerun) and the license key determines which
options are available for this configuration.
(DDC_NAME) - Please enter a name for this replication (suggestion: use the name of the source database): [] reptest2
(LICENSE_KEY) - Please enter your license key (or just enter "(trial)"): [(trial)]
(SETUP_SCRIPT_PATH) - Please enter a directory for location of configuration scripts on this machine:
[C:\Users\oracle\Documents\reptest2]

 

Network configuration (TNS)

Dbvisit Replicate will detect tnsnames.ora locations on your system. Choose the TNS configuration that is applicable.

Network configuration files were detected on this system in these locations:
C:\app\oracle\product\11.2.0\grid\network\admin
C:\app\oracle\product\11.2.0\dbhome_1\network\admin
<oracle\product\11.2.0\grid\network\admin] C:\app\oracle\product\11.2.0\dbhome_1\network\admin

 

Describing the databases

During this step information on the databases used in the replication is collected. 

The result is a SQL*Plus  SQLCMD scripts that recreates the schema user and grants it required privileges.

Please note that being a SQL*Plus/SQLCMD script, the resulting file contains plain text passwords.

 

Describe the first database:

Step 1 - Describe databases
========================================
The first step is to describe databases used in the replication. There are usually two of them (source and target); however,
there can be just one (when source and target is the same) or more than two (one-to-many or other complex configurations.)
Let's configure the database, describing it's type, connectivity, user names etc.
What type of database is this? (Oracle/MySQL/MSSQL): [Oracle]
Please enter database TNS alias: [] reptest2
Please enter SYSDBA user name: [SYS]
Please enter password for this user: [change_on_install] password

Please enter user with DBA role: [SYSTEM]
Please enter password for this user: [manager] password

Connecting to database reptest2 as SYSTEM to query list of tablespaces and to detect ASM (by looking whether any redo logs
or archived logs are stored in ASM).
Enter the Dbvisit Replicate owner and apply user (this user will be created by this script): [dbvrep]
Please enter password for this user: [dbvpasswd]

Permanent tablespaces detected on the database: USERS.
Please enter default permanent tablespace for this user: [USERS]
Temporary tablespaces detected on the database: TEMP.
Please enter default temporary tablespace for this user: [TEMP]
Following databases are now configured:
1: Oracle reptest2, SYS/***, SYSTEM/***, dbvrep/***, USERS/TEMP, dbvrep/, ASM:NO, TZ: +00:00
Enter number of database to modify it, or "add", or "done": [add]

 

Adding the second database:

Let's configure the database, describing it's type, connectivity, user names etc.
What type of database is this? (Oracle/MySQL/MSSQL): [Oracle] MSSQL
Please enter server name (as specified in "sqlcmd -S", e.g. WINSVR1\SQLEXPRESS): [localhost] dbvisit420
Please enter database ODBC DNS: [] ms
Please enter user name of an administrator: [SA] dbvuser
Please enter password for this user: [password] password??

Enter the user to log into apply database: [SA] dbvuser
Please enter password for this user: [password] password??

Enter the database to use: [dbvrep]
Enter the schema the table are created by the user specified: [dbo]
Enter the database with the user data: [] w420g
Following databases are now configured:
1: Oracle reptest2, SYS/***, SYSTEM/***, dbvrep/***, USERS/TEMP, dbvrep/, ASM:NO, TZ: +00:00
2: MSSQL ms, dbvuser/***, dbvuser/***, dbvuser/***, /, dbvrep/dbo, ASM:n/a, TZ:
Enter number of database to modify it, or "add", or "done": [done]

 

Replication pairs

During this step the source and target databases are set for each replication pair. In Oracle to SQL Server replication, Only Oracle database can ne source. 

Step 2 - Replication pairs
========================================
The second step is to set source and targets for each replication pair. This is usually just choosing the first database as
source and the second one as target, but many more configurations are possible.
Let's configure the replication pair, selecting source and target.
Following databases are described:
1: reptest2 (Oracle)
2: ms (MSSQL) (cannot be source, is not Oracle)
Select source database: [1]
Select target database: [2]
Use fetcher to offload the mining to a different server? (yes/no) [no]
(NETWORK_QUALITY) - Please specify your network type (LAN or WAN). Autoconfigures timeouts, use of compression etc. [LAN]
Lock and copy the data initially one-by-one or at a single SCN? (one-by-one/single-scn/ddl-only/resetlogs) [single-scn]
Following replication pairs are now configured:
1: reptest2 (Oracle) ==> ms (MSSQL), DDL: no, fetcher: no, process suffix: (no suffix), network: LAN, prepare type:
single-scn, data load: ddl_file
Enter number of replication pair to modify it, or "add", or "done": [done]

 

Schema/Tables to be replicated

The third step is to choose the schemas and tables to be replicated.You can specify coma separated list of multiple tables or/and schema.

Step 3 - Replicated tables
========================================
The third step is to choose the schemas and tables to be replicated. If the databases are reachable, the tables are checked
for existence, datatype support, etc., schemas are queried for tables. Note that all messages are merely hints/warnings and
may be ignored if issues are rectified before the scripts are actually executed.
Note the following assumptions are made in this wizard ΓÇô which can be modified by editing the resulting script:
1. All replicated tables have a primary key defined.
2. All columns of the tables and all tables of the specified schemas are replicated, and if DDL support is enabled, whenever
a new column/table is added, this should be replicate as well.
3. If an apply conflict arises, the default option is to try again repeatedly, until a different option is given or the
underlying issue is resolved.
4. If an apply conflict arises, no more data will be replicated until the issue is resolved or ignored.
You can also specify some advanced options:
1. Rename schemas or tables.
2. Specify filtering conditions.
3. (Tables only) Configure Change Data Capture; this does not maintain a copy of the source table, but logs all operations
as separate entries. This is useful for ETL or as an audit trail. This usually requires adding of new columns (timestamps,
old/new values etc.) to the target table. Dbvisit Replicate can create the target table with the additional columns during
setup by selecting "ddl_run" in the data copy section of Step 2.
Following tables are defined for replication pairs:
1: reptest2 (Oracle) ==> ms (MSSQL), DDL: no, suffix: (no suffix), prepare: single-scn
 No tables defined.
Enter number of replication pair to modify it, or "done": [1]
Please enter list of all individual tables to be replicated. Enter schema name(s) only to replicate all tables in that
schema. Use comma or space to delimit the entries.
Enter the tables and schemas: [] avi.sample
Selected tables: avi.sample
Add more tables or schemas? (YES/NO) [NO]
Specify rename name, filter condition, CDC/Audit/ETL for any of the specified tables (yes/no): [no] y
Rename SCHEMA name for avi.sample (empty means no rename): [w420g.dbo] 
Rename TABLE name for avi.sample (empty means no rename): []
Filter the data to be replicated? If yes, use single condition for ALL DML operations or use CUSTOM conditions for each
operation? (NO/ALL/CUSTOM) [NO]
Configure change data capture for change auditing or real-time BI? (NO/YES) [NO]
Following tables are defined for replication pairs:
1: reptest2 (Oracle) ==> ms (MSSQL), DDL: no, suffix: (no suffix), prepare: single-scn
 avi.sample => w420g.dbo.sample
Enter number of replication pair to modify it, or "done": [done]

 

 

Rename SCHEMA name for avi.sample (empty means no rename): [] w420g.dbo
Here, w420g is the target SQL Server database name and dbo is schema. Please use <targetDB>.<schema> format in "Rename Schema"

Process configuration

This steps configures the replication processes for each replication. This includes setting the

  • hostnames
  • port numbers
  • Notification settings and thresholds

Add process information for the MINE process:

Step 4 - Process configuration
========================================
The fourth step is to configure the replication processes for each replication. Although most options have reasonable
defaults, manual input is required.
Following processes are defined:
1: MINE on reptest2
 Not configured.
2: APPLY on ms
 Not configured.
Enter number of process to modify it, or "done": [1]
Fully qualified name of the server for the process (usually co-located with the database, unless mine is offloaded using
fetcher): [DBVISIT410]
Server type (Windows/Linux/Unix): [Windows]
Enable email notifications about problems (yes/no)? [YES] n
Enable SNMP traps/notifications about problems (yes/no)? [NO]
Directory with DDC file and default where to create log files etc. (recommended: same as global setting, if possible)?
[C:\Users\oracle\Documents\reptest2]
Following settings were pre-filled with defaults or your reloaded settings:
----------------------------------------
[MINE_REMOTE_INTERFACE]: Network remote interface: DBVISIT410:7901
[MINE_DATABASE]: Database TNS: reptest2
[TNS_ADMIN]: tnsnames.ora path: C:\app\oracle\product\11.2.0\dbhome_1\network\admin
[MINE_USER]: Dbvisit Replicate database username: dbvrep
[MINE_PASSWORD]: Dbvisit Replicate database password: *********
[MINE_PLOG]: Filemask for generated plogs: C:\Users\oracle\Documents\reptest2\mine\%S.%E (%S is sequence, %T thread, %F
original filename (stripped extension), %P process type, %N process name, %E default extension)
[LOG_FILE]: General log file: C:\Users\oracle\Documents\reptest2\log\dbvrep_%N_%D.%E
[LOG_FILE_TRACE]: Error traces: C:\Users\oracle\Documents\reptest2\log\trace\dbvrep_%N_%D_%I_%U.%E
Checking that these settings are valid...
Do you want change any of the settings? [no]
Following processes are defined:
1: MINE on reptest2
 Host: DBVISIT410, SMTP: NO, SNMP: NO
2: APPLY on ms
 Not configured.

 

 

Keep the TNS_ADMIN unchanged. Here, It is pointing to TNS_ADMIN location on windows but as MINE process runs on Linux, this values must be changed later before starting MINE process.

 

Add process information for the APPLY process:

Enter number of process to modify it, or "done": [2]
Fully qualified name of the server for the process (usually co-located with the database, unless mine is offloaded using
fetcher): [] dbvisit420
Server type (Windows/Linux/Unix): [] Windows
Enable email notifications about problems (yes/no)? [YES] n
Enable SNMP traps/notifications about problems (yes/no)? [NO]
Directory with DDC file and default where to create log files etc. (recommended: same as global setting, if possible)?
[C:\Users\oracle\Documents\reptest2]
Following settings were pre-filled with defaults or your reloaded settings:
----------------------------------------
[APPLY_REMOTE_INTERFACE]: Network remote interface: dbvisit420:7902
[APPLY_DATABASE]: Database ODBC DNS: ms
[TNS_ADMIN]: tnsnames.ora path: C:\app\oracle\product\11.2.0\dbhome_1\network\admin
[APPLY_USER]: Dbvisit Replicate database username: dbvuser
[APPLY_PASSWORD]: Dbvisit Replicate database password: ********
[APPLY_SCHEMA]: Dbvisit Replicate database: dbvrep
[APPLY_SCHEMA2]: Dbvisit Replicate schema: dbo
[APPLY_STAGING_DIR]: Directory for received plogs: C:\Users\oracle\Documents\reptest2\apply
[LOG_FILE]: General log file: C:\Users\oracle\Documents\reptest2\log\dbvrep_%N_%D.%E
[LOG_FILE_TRACE]: Error traces: C:\Users\oracle\Documents\reptest2\log\trace\dbvrep_%N_%D_%I_%U.%E
Checking that these settings are valid...
Do you want change any of the settings? [no]
Following processes are defined:
1: MINE on reptest2
 Host: DBVISIT410, SMTP: NO, SNMP: NO
2: APPLY on ms
 Host: dbvisit420, SMTP: NO, SNMP: NO
Enter number of process to modify it, or "done": [done]

 

Final wrap-up

All the information about the replication process has been obtained and now the replication environment can be created.

Created file C:\Users\oracle\Documents\reptest2\reptest2-APPLY.ddc.
Created file C:\Users\oracle\Documents\reptest2\reptest2-MINE.ddc.
Created file C:\Users\oracle\Documents\reptest2\reptest2-setup.dbvrep.
Created file C:\Users\oracle\Documents\reptest2\reptest2-dbsetup_reptest2.sql.
Created file C:\Users\oracle\Documents\reptest2\reptest2-dbsetup_ms.sql.
Created file C:\Users\oracle\Documents\reptest2\reptest2-grants_reptest2.sql.
Created file C:\Users\oracle\Documents\reptest2\reptest2-grants_ms.sql.
Created file C:\Users\oracle\Documents\reptest2\reptest2-onetime.ddc.
Created file C:\Users\oracle\Documents\reptest2\start-console.bat.
Created file C:\Users\oracle\Documents\reptest2\reptest2-run-DBVISIT410.bat.
Created file C:\Users\oracle\Documents\reptest2\reptest2-win_services-DBVISIT410.bat.
Created file C:\Users\oracle\Documents\reptest2\reptest2-run-dbvisit420.bat.
Created file C:\Users\oracle\Documents\reptest2\reptest2-win_services-dbvisit420.bat.
Created file C:\Users\oracle\Documents\reptest2\Nextsteps.txt.
Created file C:\Users\oracle\Documents\reptest2\reptest2-all.bat.
===========================================================================================================================
Dbvisit Replicate wizard completed
Script C:\Users\oracle\Documents\reptest2\reptest2-all.bat created. This runs all the above created scripts. Please exit out
of dbvrep, review and run script as current user to setup and start Dbvisit Replicate.
===========================================================================================================================
dbvrep>exit

 

 

the Dbvisit Replicate wizard has completed and has created the necessary replication environment.
Exit out of dbvrep and run <replication name>-all.bat script created by the wizard
In the above example the script is called reptest2-all.bat. This script can be run at any time to reset or recreate the replication environment. The Nextsteps.txt explains the next steps to initiate the replication.
This completes the Dbvisit Replicate setup and configuration.
Before running the script, review the created scripts as indicated by the Nextsteps.txt (shown at the end of all.* script). For example:

  • Review the location of dbvrep(.exe) if it differs among the nodes if RAC is used.
  • Check TNS_ADMIN and ORACLE_HOME paths and make sure they are correct on all the different servers involved.

Outcome of running the scripts created by the setup wizard.

Running the reptest2-all.sh script will configure and initiate the replication. It does the following:

  • Create the Dbvisit Replicate schemas in the source and target databases.
  • Grant the necessary privileges to the Dbvisit Replicate schema.
  • Create the Dbvisit Replicate repository in the source and target databases.
  • Load the Dbvisit Replicate configuration file (DDC) into the Dbvisit Replicate repository. The DDC file is like the init.ora for Oracle and contains the settings for the replication.
  • Prepare the schemas or objects for replication.

Example of running the reptest1-all.bat

C:\Users\oracle\Documents\reptest2>reptest2-all.bat
Setting up Dbvisit Replicate configuration
Configure database reptest2...
Configure database ms...
Object grants for database reptest2...
Object grants for database ms...
Setting up the configuration
Initializing......done
WARN-1850: No DDC DB available, dictionary table does not exist.
DDC loaded from database (0 variables).
Dbvisit Replicate version 2.7
Copyright (C) Dbvisit Software Limited. All rights reserved.
DDC file C:\Users\oracle\Documents\reptest2\reptest2-onetime.ddc
loaded.
MINE: Cannot determine Dbvisit Replicate dictionary version. (no
dictionary exists)
APPLY: Cannot determine Dbvisit Replicate dictionary version. (no
dictionary exists)
dbvrep> set ON_WARNING SKIP
Variable ON_WARNING set to SKIP for process *.
dbvrep> set ON_ERROR EXIT
Variable ON_ERROR set to EXIT for process *.
dbvrep> ENGINE SWITCH_REDOLOG
dbvrep> ENGINE SETUP MINE DROP DICTIONARY
0 dictionary objects dropped.
dbvrep> ENGINE SETUP MINE CREATE DICTIONARY
dbvrep> ENGINE SETUP MINE LOAD DICTIONARY
Supplemental logging on database set.
Loading dictionary table DBRSCOL$
Loading dictionary table DBRSOBJ$
Loading dictionary table DBRSTAB$
Loading dictionary table DBRSUSER$
Loading dictionary table DBRSV_$DATABASE
dbvrep> ENGINE SETUP APPLY DROP DICTIONARY
0 dictionary objects dropped.
dbvrep> ENGINE SETUP APPLY CREATE DICTIONARY
dbvrep> ENGINE SETUP APPLY LOAD DICTIONARY
dbvrep> ENGINE SETUP PAIR MINE AND APPLY
ID of mine proces is DBD0D988-6DAD-1014-AEB7-034420638A48. If not using
DDC in database, set MINE_UNIQUE_ID to this value.
1 applier SCN set.
dbvrep> SET APPLY.INSTANTIATE_SCN NOW
Variable INSTANTIATE_SCN set to NOW for process APPLY.
dbvrep> memory_set IGNORE_APPLY_DDL_DIFFERENCES YES
Variable IGNORE_APPLY_DDL_DIFFERENCES set to YES for process *.
dbvrep> ENGINE SUPPLEMENTAL LOGGING TABLE avi.sample ENABLE PRIMARY KEY
dbvrep> ENGINE SWITCH_REDOLOG
dbvrep> #single-scn instantiation: lock all tables and schemas
dbvrep> ENGINE LOCK TABLES avi.sample
Locking all tables.
Lock done.
dbvrep> #single-scn instantiation: unlock all tables and schemas, but
keep the SCN
dbvrep> ENGINE LOCK RELEASE LOCKS
dbvrep>
dbvrep> #prepare the tables (we use OFFLINE as neither MINE not APPLY
is running; with OFFLINE we won't wait on network timeout)
dbvrep> PREPARE OFFLINE TABLE avi.sample NODDL RENAME TO
w420g.dbo.sample
Table avi.sample instantiated at SCN 1077447
dbvrep> #single-scn instantiation: unlock all tables and schemas,
forget the SCN (so it does not affect any further PREPARE statements)
dbvrep> ENGINE LOCK CLEAR SCN
dbvrep> ENGINE SWITCH_REDOLOG
dbvrep> #prepare script for instantiation
dbvrep> ENGINE PREPARE_DP WRITE DDL_FILE FILE
C:\Users\oracle\Documents\reptest2\APPLY.sql USERID
SYSTEM/password@reptest2
Created DDL script C:\Users\oracle\Documents\reptest2\APPLY.sql.
dbvrep> create ddcdb from ddcfile
DDC loaded into database (204 variables).
dbvrep> set ON_WARNING SKIP
Variable ON_WARNING set to SKIP for process *.
dbvrep> set ON_ERROR SKIP
Variable ON_ERROR set to SKIP for process *.
OK-0: Completed successfully.
These steps are required after the reptest2-all.bat script runs:
1) Create the necessary directory(ies) on the servers:
DBVISIT410: C:\Users\oracle\Documents\reptest2
dbvisit420: C:\Users\oracle\Documents\reptest2
2) Copy the DDC files to the server(s) where the processes will run:
dbvisit420: C:\Users\oracle\Documents\reptest2\reptest2-APPLY.ddc
DBVISIT410: C:\Users\oracle\Documents\reptest2\reptest2-MINE.ddc
3) Review that path to dbvrep executable is correct in the run scripts:
C:\Users\oracle\Documents\reptest2\reptest2-run-DBVISIT410.bat
C:\Users\oracle\Documents\reptest2\reptest2-run-dbvisit420.bat
4) Review and run the scripts for Windows to create services (run with Administrator privileges):
C:\Users\oracle\Documents\reptest2\reptest2-win_services-DBVISIT410.bat
C:\Users\oracle\Documents\reptest2\reptest2-win_services-dbvisit420.bat

5) Copy the run script to the server(s) where the processes will run:
C:\Users\oracle\Documents\reptest2\reptest2-run-DBVISIT410.bat
C:\Users\oracle\Documents\reptest2\reptest2-run-dbvisit420.bat
6) Ensure firewall is open for listen interfaces 0.0.0.0:7902, 0.0.0.0:7901 used by the processes.
7) Make sure the data on apply are in sync as of time when setup was run.
Scripts for Data Pump/export/DDL were created as requested:
C:\Users\oracle\Documents\reptest2\APPLY.sql
Create referenced database links (if any) before running the scripts.
8) Start the replication processes on all servers:
C:\Users\oracle\Documents\reptest2\reptest2-run-DBVISIT410.bat
C:\Users\oracle\Documents\reptest2\reptest2-run-dbvisit420.bat
9) Start the console to monitor the progress:
C:\Users\oracle\Documents\reptest2\start-console.bat
The above list is stored in C:\Users\oracle\Documents\reptest2\Nextsteps.txt.
Press any key to continue . . .

 

The replication has been configured and installed. Steps required to start replication are saved in Nextsteps.txt file.

If you have empty source tables then you can execute APPLY.sql to create tables in SQL server. Otherwise you can use Navicat Premium  to load data from Oracle into SQL Server.

Executing reptest2-all.bat script creates dbvrep database in SQL Server to store metadata of replication. Set correct snapshot isolation for dbvrep. Click here for more info.

Starting the replication

As indicated by the Nextsteps.txt , the replication can be started by following the steps that are listed above. 
Starting the replication does the following:

  • Starts the MINE and APPLY process in the background
  • Creates the necessary log and plog directories.
  • Once the APPLY and MINE process has been started, there is in INITIALIZATION process. This replicates the contents of the Dbvisit Replication repository from the source to the target database. The replication can not be started until this phase has been completed.

Starting the console

The dbvrep console can be started by executing start-console.bat script on source server to monitor the real time status of the replication.

C:\Users\oracle\Documents\reptest2>start-console.bat

 

Following is the reptest2 replication console showing current status of replication.

-MINE IS running. Currently at plog 425 and SCN 12377134 (09/05/2012 21:58:15).
APPLY IS running. Currently at plog 425 and SCN 12377168 (09/05/2012 21:58:27).
Progress of replication reptest1:MINE->APPLY: total/this execution
---------------------------------------------------------------------------------------------------------------------------------
AVI.SAMPLE:                   100%  Mine:1/1             Unrecov:0/0         Applied:1/1         Conflicts:0/0       Last:05/09/2012 21:58:30/OK
---------------------------------------------------------------------------------------------------------------------------------
1 tables listed.

 

 

Initialization of APPLY process can take 5 to 7 minutes

  • No labels