In all replications, source must be an Oracle database. Thus, two-way replication with MySQL is not supported.
All target tables must be stored in InnoDB engine, as Dbvisit Replicate depends on correct transaction handling to reliably restart after a shutdown or failure. The internal tables are stored in InnoDB engine as well.
Currently, no DDL replication is supported for MySQL.
Oracle → MySQL replication setup example
The following example shows Oracle to MySQL replication. In this example, Oracle and MySQL are running on a single server.
Environment: Virtualbox using Pre-Built Developer VMs (for Oracle VM VirtualBox)
Template name: Database App Development VM
Oracle Linux 5 (32bit)
Oracle RDBMS 11gR2
MySQL Server 5.0.77
MySQL is not installed on the pre-built VM, but this can be installed with:
- yum mysql mysql-server
Start MySQL
/etc/init.d/mysqld start
Install Dbvisit Replicate using the 32bit RPM installation file (as root): - rpm -ivh dbvisit_replicate-2.0.02-1.i386.rpm
Preparing... ########################################## [100%]
1:dbvisit_replicate ########################################### [100%]
The following Oracle table is going to be replicated. This is a copy of the sys.dba_objects table but created in the SCOTT schema.
Create this table in Oracle:
CREATE TABLE "SCOTT"."AVI_OBJECTS"
( "OWNER" VARCHAR2(30),
"OBJECT_NAME" VARCHAR2(128),
"SUBOBJECT_NAME" VARCHAR2(30),
"OBJECT_ID" NUMBER,
"DATA_OBJECT_ID" NUMBER,
"OBJECT_TYPE" VARCHAR2(19),
"CREATED" DATE,
"LAST_DDL_TIME" DATE,
"TIMESTAMP" VARCHAR2(19),
"STATUS" VARCHAR2(7),
"TEMPORARY" VARCHAR2(1),
"GENERATED" VARCHAR2(1),
"SECONDARY" VARCHAR2(1),
"NAMESPACE" NUMBER,
"EDITION_NAME" VARCHAR2(30)
) SEGMENT CREATION IMMEDIATE
PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 NOCOMPRESS LOGGING
STORAGE(INITIAL 65536 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645
PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT FLASH_CACHE DEFAULT CELL_FLASH_CACHE DEFAULT)
TABLESPACE "USERS"
Create this same table in MySQL. The syntax is slightly different to Oracle.
First user scott (or database) has to be created first in MySQL.
mysql> create database scott;
Query OK, 1 row affected (0.00 sec)
mysql> use scott;
Database changed
Now the MySQL table can be created:
DROP TABLE IF EXISTS `avi_objects`;
SET @saved_cs_client = @@character_set_client;
SET character_set_client = utf8;
CREATE TABLE `avi_objects` (
`OWNER` varchar(30) default NULL,
`OBJECT_NAME` varchar(128) default NULL,
`SUBOBJECT_NAME` varchar(30) default NULL,
`OBJECT_ID` bigint(20) default NULL,
`DATA_OBJECT_ID` bigint(20) default NULL,
`OBJECT_TYPE` varchar(19) default NULL,
`CREATED` date default NULL,
`LAST_DDL_TIME` date default NULL,
`TIMESTAMP` varchar(19) default NULL,
`STATUS` varchar(7) default NULL,
`TEMPORARY` varchar(1) default NULL,
`GENERATED` varchar(1) default NULL,
`SECONDARY` varchar(1) default NULL,
`NAMESPACE` bigint(20) default NULL,
`EDITION_NAME` varchar(30) default NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Start Dbvisit Replicate:
$ dbvrep
Initializing.....done
Dbvisit Replicate
Copyright (C) Dbvisit Software Limited. All rights reserved.
No DDC file loaded.
Run "setup wizard" to start the configuration wizard or try "help" to see all
commands available.
dbvrep>
Start the setup wizard and follow the instructions. Most of the defaults can be chosen. Note that MySQL is entered as the target database:
dbvrep> setup wizard
This wizard configures Dbvisit Replicate to start a replication process.
The setup wizard creates configuration scripts, which need to be run after the
wizard ends. No changes to the databases are made before that.
The progress is saved every time a list of databases, replications, etc. is
shown. It will be re-read if wizard is restarted and the same DDC name and
script path is selected.
Run the wizard now? [yes]
Accept end-user license agreement? (view/yes/no) [view] yes
Before starting the actual configuration, some basic information is needed. The
DDC name and script path determines where all files created by the wizard go
(and where to reread them if wizard is rerun) and the license key determines
which options are available for this configuration.
(DDC_NAME) - Please enter a name for this replication (suggestion: use the name of the source database): [] orcl
(LICENSE_KEY) - Please enter your license key (or just enter "(trial)"): [(trial)]
(SETUP_SCRIPT_PATH) - Please enter a directory for location of configuration scripts on this machine: [/home/oracle/orcl]
Network configuration files were detected on this system in these locations:
/home/oracle/app/oracle/product/11.2.0/dbhome_1/network/admin
(TNS_ADMIN) - Please enter TNS configuration directory for this machine: [/home/oracle/app/oracle/product/11.2.0/dbhome_1/network/admin]
Step 1 - Describe databases
========================================
The first step is to describe databases used in the replication. There are
usually two of them (source and target); however, there can be just one (when source and target is the same) or more than two (one-to-many or other complex configurations.)
Let's configure the database, describing it's type, connectivity, user names
etc.
What type of database is this? (Oracle/MySQL/MSSQL): [Oracle]
Please enter database TNS alias: [] ttorcl
Please enter SYSDBA user name: [SYS]
Please enter password for this user: [change_on_install] ******
Please enter user with DBA role on the target database: [SYSTEM]
Please enter password for this user: [manager] ******
Connecting to database ttorcl as SYSTEM to query list of tablespaces and to
detect ASM (by looking whether any redo logs or archived logs are stored in
ASM).
Enter the Dbvisit Replicate owner and apply user (this user will be created by this script): [dbvrep]
Please enter password for this user: [dbvpasswd]
Permanent tablespaces detected on the database: APEX_1295922881855015, EXAMPLE,
FLOW_1170420963682633, FLOW_1194425963955800, FLOW_1218408858999342,
FLOW_1242310449730067, FLOW_1266412439758696, USERS, SYSAUX, SYSTEM.
Please enter default permanent tablespace for this user: [APEX_1295922881855015] USERS
Temporary tablespaces detected on the database: TEMP.
Please enter default temporary tablespace for this user: [TEMP]
Does the database use ASM? (yes/no): [NO]
Following databases are now configured:
1: Oracle ttorcl, SYS/**, SYSTEM/*, dbvrep/**, USERS/TEMP, dbvrep/, ASM:NO
Enter number of database to modify it, or "add", or "done": [add]
Let's configure the database, describing it's type, connectivity, user names
etc.
What type of database is this? (Oracle/MySQL/MSSQL): [Oracle] MySQL
Please enter database hostname: [localhost]
Please enter user name of an administrator: [root]
Please enter password for this user: [password] ******
Enter the user to log into apply database: [root]
Please enter password for this user: [password]
Enter the database (schema) to use for Dbvisit Replicate internal data (will be created in the script): [dbvrep]
Following databases are now configured:
1: Oracle ttorcl, SYS/**, SYSTEM/*, dbvrep/**, USERS/TEMP, dbvrep/, ASM:NO
2: MySQL database=dbvrep;host=localhost, root/**, root/*, root/**, /,
dbvrep/, ASM:n/a
Enter number of database to modify it or "done": [done]
Step 2 - Replication pairs
========================================
The second step is to set source and targets for each replication pair. This is
usually just choosing the first database as source and the second one as
target, but many more configurations are possible.
Let's configure the replication pair, selecting source and target.
Following databases are described:
1: ttorcl (Oracle)
2: database=dbvrep;host=localhost (MySQL) (cannot be source, is not Oracle)
Select source database: [1]
Select target database: [2]
Use fetcher to offload the mining to a different server? (yes/no) [no]
(NETWORK_QUALITY) - Please specify your network type (LAN or WAN). Autoconfigures timeouts, use of compression etc. [LAN]
Following replication pairs are now configured:
1: ttorcl (Oracle) ==> database=dbvrep;host=localhost (MySQL), DDL: no,
fetcher: no, process suffix: (no suffix), network: LAN
Enter "1" to modify or "done": [done]
Step 3 - Replicated tables
========================================
The third step is to choose the schemas and tables to be replicated. If the
databases are reachable, the tables are checked for existence, datatype
support, etc., schemas are queried for tables. Note that all messages are
merely hints/warnings and may be ignored if issues are rectified before the
scripts are actually executed.
Note the following assumptions are made in this wizard – which can be
modified by editing the resulting script:
1. All replicated tables have a primary key defined.
2. All columns of the tables and all tables of the specified schemas are
replicated, and if DDL support is enabled, whenever a new column/table is
added, this should be replicate as well.
3. If an apply conflict arises, the default option is to try again repeatedly,
until a different option is given or the underlying issue is resolved.
4. If an apply conflict arises, no more data will be replicated until the issue
is resolved or ignored.
Following tables are defined for replication pairs:
1: ttorcl (Oracle) ==> database=dbvrep;host=localhost (MySQL), DDL: no, suffix:
(no suffix)
No tables defined.
Enter number of replication pair to modify it, or "done": [1]
To replicate an entire schema, please enter the schemas to be replicated. Enter
through a comma-delimited list, or enter one by one. Hit Enter when finished
Enter the list of schemas: []
To replicate individual tables, please enter the table names. Do not enter
tables in quotes (" ").
Enter through a comma-delimited list, or enter one by one. Hit Enter when
finished.
Please use fully qualified names (ie owner.table_name).
Enter the list of tables: [] scott.avi_objects
Enter the list of tables: []
Specify rename name or filter condition for any of the specified tables (yes/no): [no]
Following tables are defined for replication pairs:
1: ttorcl (Oracle) ==> database=dbvrep;host=localhost (MySQL), DDL: no, suffix:
(no suffix)
scott.avi_objects
Enter number of replication pair to modify it, or "done": [done]
Step 4 - Process configuration
========================================
The fourth step is to configure the replication processes for each replication.
Although most options have reasonable defaults, manual input will be required.
Following processes are defined:
1: MINE on ttorcl
Not configured.
2: APPLY on database=dbvrep;host=localhost
Not configured.
Enter number of process to modify it, or "done": [1]
Fully qualified name of the server for the process (usually co-located with the database, unless mine is offloaded using fetcher): [] dhcppc13
Enable email notifications about problems (yes/no)? [YES] NO
Enable SNMP traps/notifications about problems (yes/no)? [NO]
Directory with DDC file and default where to create log files etc. (recommended: same as global setting, if possible)? [/home/oracle/orcl]
Following settings were pre-filled with defaults or your reloaded settings:
----------------------------------------
[MINE_LISTEN_INTERFACE]: Network listen interface: dhcppc13:7901
[MINE_DATABASE]: Database TNS: ttorcl
[TNS_ADMIN]: tnsnames.ora path:
/home/oracle/app/oracle/product/11.2.0/dbhome_1/network/admin
[MINE_USER]: Replicate database username: dbvrep
[MINE_PASSWORD]: Replicate database password: *********
[MINE_PLOG]: Filemask for generated plogs: /home/oracle/orcl/mine/%S.%E
[LOG_FILE]: General log file: /home/oracle/orcl/log/dbvrep_%N_%D.%E
[LOG_FILE_TRACE]: Error traces:
/home/oracle/orcl/log/trace/dbvrep_%N_%D_%I_%U.%E
Checking that these settings are valid...
Do you want change any of the settings? [no]
Following processes are defined:
1: MINE on ttorcl
Host: localhost, SMTP: NO, SNMP: NO
2: APPLY on database=dbvrep;host=localhost
Not configured.
Enter number of process to modify it, or "done": [2]
Fully qualified name of the server for the process (usually co-located with the database, unless mine is offloaded using fetcher): [] dhcppc13
Enable email notifications about problems (yes/no)? [YES] NO
Enable SNMP traps/notifications about problems (yes/no)? [NO]
Directory with DDC file and default where to create log files etc. (recommended: same as global setting, if possible)? [/home/oracle/orcl]
Following settings were pre-filled with defaults or your reloaded settings:
----------------------------------------
[APPLY_LISTEN_INTERFACE]: Network listen interface: dhcppc13:7902
[APPLY_DATABASE]: Database MySQL connection string:
database=dbvrep;host=localhost
[APPLY_USER]: Replicate database username: root
[APPLY_PASSWORD]: Replicate database password: ********
[APPLY_SCHEMA]: Replicate database (schema): dbvrep
[APPLY_STAGING_DIR]: Directory for received plogs: /home/oracle/orcl/apply
[LOG_FILE]: General log file: /home/oracle/orcl/log/dbvrep_%N_%D.%E
[LOG_FILE_TRACE]: Error traces:
/home/oracle/orcl/log/trace/dbvrep_%N_%D_%I_%U.%E
Checking that these settings are valid...
Do you want change any of the settings? [no]
Following processes are defined:
1: MINE on ttorcl
Host: localhost, SMTP: NO, SNMP: NO
2: APPLY on database=dbvrep;host=localhost
Host: localhost, SMTP: NO, SNMP: NO
Enter number of process to modify it, or "done": [done]
Created file /home/oracle/orcl/orcl-MINE.ddc.
Created file /home/oracle/orcl/orcl-APPLY.ddc.
Created file /home/oracle/orcl/orcl-dbsetup_ttorcl.sql.
Created file /home/oracle/orcl/orcl-dbsetup_database_dbvrep_host_localhost.sql.
Created file /home/oracle/orcl/orcl-setup.dbvrep.
Created file /home/oracle/orcl/orcl-grants_ttorcl.sql.
Created file /home/oracle/orcl/orcl-grants_database_dbvrep_host_localhost.sql.
Created file /home/oracle/orcl/orcl-onetime.ddc.
Created file /home/oracle/orcl/orcl-run-dhcppc13.sh.
Created file /home/oracle/orcl/Nextsteps.txt.
Created file /home/oracle/orcl/orcl-all.sh.
==============================================================================
Dbvisit Replicate wizard completed
Script /home/oracle/orcl/orcl-all.sh created. This runs all the above created
scripts. Please exit out of dbvrep, review and run script as current user to
setup and start Dbvisit Replicate.
==============================================================================
Optionally, the script can be invoked now by this wizard.
Run this script now? (yes/no) [NO]
Exit out of Dbvisit Replicate and start the script that has been generated by the setup wizard:
$ ./orcl-all.sh
Setting up Dbvisit Replicate configuration
Configure database ttorcl...
Configure database database=dbvrep
Object grants for database ttorcl...
Object grants for database database=dbvrep
Setting up the configuration
Initializing.....done
WARN-1850: No DDC DB available, dictionary table does not exist.
DDC loaded from database (0 variables).
OK-9056: Directory /home/oracle/orcl/ddc_backup set by variable DDC_BACKUP_DIR does not exist on this system.
OK-9056: Directory /home/oracle/orcl/ddc_backup set by variable DDC_BACKUP_DIR does not exist on this system.
Dbvisit Replicate version 2.1.04.1047
Copyright (C) Dbvisit Software Limited. All rights reserved.
DDC file /home/oracle/orcl/orcl-onetime.ddc loaded.
MINE: Cannot determine Dbvisit Replicate dictionary version. (no dictionary exists)
APPLY: Cannot determine Dbvisit Replicate dictionary version. (no dictionary exists)
dbvrep> set ON_WARNING SKIP
Variable ON_WARNING set to SKIP for process *.
dbvrep> set ON_ERROR EXIT
Variable ON_ERROR set to EXIT for process *.
dbvrep> ENGINE SETUP MINE DROP DICTIONARY
0 dictionary objects dropped.
dbvrep> ENGINE SETUP MINE CREATE DICTIONARY
dbvrep> ENGINE SETUP MINE LOAD DICTIONARY
Supplemental logging on database set.
Loading dictionary table DBRSCOL$
Loading dictionary table DBRSOBJ$
Loading dictionary table DBRSTAB$
Loading dictionary table DBRSUSER$
Loading dictionary table DBRSV_$DATABASE
dbvrep> ENGINE SETUP APPLY DROP DICTIONARY
0 dictionary objects dropped.
dbvrep> ENGINE SETUP APPLY CREATE DICTIONARY
dbvrep> ENGINE SETUP APPLY LOAD DICTIONARY
dbvrep> ENGINE SETUP PAIR MINE AND APPLY
ID of mine proces is 276F6266-243F-11E1-A450-BD91EEFD88F8. If not using DDC in database, set MINE_UNIQUE_ID to this value.
Table dbvrep.DBRSCOL$ instantiated at SCN 7436161
Table dbvrep.DBRSOBJ$ instantiated at SCN 7436161
Table dbvrep.DBRSTAB$ instantiated at SCN 7436161
Table dbvrep.DBRSUSER$ instantiated at SCN 7436161
Table dbvrep.DBRSV_$DATABASE instantiated at SCN 7436161
1 applier SCN set.
dbvrep> PREPARE OFFLINE TABLE scott.avi_objects NODDL
Table scott.avi_objects instantiated at SCN 7437937
dbvrep> create ddcdb from ddcfile
DDC loaded into database (131 variables).
dbvrep> set ON_WARNING SKIP
Variable ON_WARNING set to SKIP for process *.
dbvrep> set ON_ERROR SKIP
Variable ON_ERROR set to SKIP for process *.
OK-0: Completed successfully.
1) Create the necessary directory(ies) on the servers:
2) Copy the DDC files to the server(s) where the processes will run:
/home/oracle/orcl/orcl-MINE.ddc
/home/oracle/orcl/orcl-APPLY.ddc
3) Review that path to dbvrep executable is correct in the run scripts:
/home/oracle/orcl/orcl-run-dhcppc13.sh
4) Copy the run script to the server(s) where the processes will run:
/home/oracle/orcl/orcl-run-dhcppc13.sh
5) Ensure firewall is open for listen interfaces dhcppc13:7901, dhcppc13:7902 used by the processes.
6) Start the replication processes on all servers:
/home/oracle/orcl/orcl-run-dhcppc13.sh
7) Start the console to monitor the progress:
/usr/bin/dbvrep --ddcfile /home/oracle/orcl/orcl-MINE.ddc
Start the replication process by running the orcl-run-dhcppc13.sh script. This starts both the Mine and Apply process:
$ ./orcl-run-dhcppc13.sh
Initializing.....done
OK-9056: Directory /home/oracle/orcl/ddc_backup set by variable DDC_BACKUP_DIR does not exist on this system.
OK-9056: Directory /home/oracle/orcl/ddc_backup set by variable DDC_BACKUP_DIR does not exist on this system.
DDC loaded from database (131 variables).
Dbvisit Replicate version 2.1.04.1047
Copyright (C) Dbvisit Software Limited. All rights reserved.
DDC file /home/oracle/orcl/orcl-APPLY.ddc loaded.
Starting process APPLY...Created directory /home/oracle/orcl/ddc_backup
Created directory /home/oracle/orcl/log/
Created directory /home/oracle/orcl/log/trace/
Created directory /home/oracle/orcl/apply
started
Initializing.....[oracle@dhcppc13 orcl]$ done
DDC loaded from database (131 variables).
Dbvisit Replicate version 2.1.04.1047
Copyright (C) Dbvisit Software Limited. All rights reserved.
DDC file /home/oracle/orcl/orcl-MINE.ddc loaded.
Starting process MINE...Created directory /home/oracle/orcl/mine/
started
Dbvisit Replicate has now been configured and started in the background. The Dbvisit Replicate can be started in the foreground so that progress can be monitored:
$ /usr/bin/dbvrep --ddcfile /home/oracle/orcl/orcl-MINE.ddc
Initializing.....done
DDC loaded from database (134 variables).
Dbvisit Replicate version 2.1.04.1047
Copyright (C) Dbvisit Software Limited. All rights reserved.
-MINE IS running, initialization NOT yet complete. Currently at plog 264 and SCN 6355055 (08/12/2011 20:11:24).
APPLY IS running, initialization NOT yet complete. Currently at plog 263 and SCN 6353874 (08/12/2011 20:11:20).
DDC file /home/oracle/orcl/orcl.ddc loaded.
Try "help"
dbvrep>
When Dbvisit Replicate is first configured, it needs to replicate its own internal tables and this is known as the initialization process. To monitor the progress of initialization, the LIST PROGRESS ALL command can be used.
dbvrep> list progress all
Progress of replication: total/this execution
------------------------------------------------------------------------------------
DBVREP.DBRSCOL$/dbvrep.DBRSCOL$: 59% Mine:109211/109211 Applied:65363/65363
DBVREP.DBRSOBJ$: ---% Mine:78355/783 Applied:0/0
DBVREP.DBRSTAB$: ---% Mine:3516/3516 Applied:0/0
DBVREP.DBRSUSER$: ---% Mine:109/109 Applied:0/0
DBVREP.DBRSV_$DATABASE: ---% Mine:1/1 Applied:0/0
------------------------------------------------------------------------------------
5 tables listed.
(not all columns are displayed above)
Replication cannot be started until the initialization has been completed. This is completed when the console show MINE and APPLY is running:
/MINE IS running. Currently at plog 269 and SCN 6365268 (08/12/2011 21:01:09).
APPLY IS running. Currently at plog 269 and SCN 6365270 (08/12/2011 21:01:10).
Insert records into the scott.avi_objects table. A script can be created to do this:
sqlplus scott/tiger@ttorcl<<SQL
insert into scott.avi_objects select * from dba_objects where rownum <= 2000;
commit;
exit
SQL
On the Dbvisit Replicate console the replication can be monitored:
Progress of replication: total/this execution
------------------------------------------------------------------------------------
SCOTT.AVI_OBJECTS/scott.avi_objects:100% Mine:2000/2000 Applied:2000/2000
------------------------------------------------------------------------------------
1 tables listed.
dbvrep>
(not all columns are displayed above)
If the replication shows 100%, then it is fully replicated. If the progress is not shown on the command console then LIST PROGRESS can be used to display the progress.
The insert script can be repeatedly run to insert more records which are then automatically replicated to the MySQL database.
The Mine and Apply processes are run in the background and can be viewed with ps
$ ps -ef | grep dbvrep
oracle 4098 1 6 14:27 ? 00:01:17 dbvrep APPLY or –daemon --ddcfile /home/oracle/orcl/orcl-APPLY.ddc start APPLY
oracle 4107 1 6 14:27 ? 00:01:16 dbvrep MINE orc –daemon --ddcfile /home/oracle/orcl/orcl-MINE.ddc start MINE