...
Code Block | ||
---|---|---|
| ||
SOURCE_DB = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = sourceserver.com)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = SOURCE_DB) ) ) EC2_TARGET = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = awsdb.cu1fmi2nxxxx.us-east-1.amazonaws.com)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = EC2_TARGET) ) ) |
...
Amazon EC2 Instances (Apply)
In this example we created one AWS EC2 instance to show the concept of loading data into EC2 using Replicate. The The Source database is on-premise. The EC2 target instance (EC2_TARGET) will be configured to perform the APPLY inside the AWS environment.
...
Code Block | ||
---|---|---|
| ||
Run "setup wizard" to start the configuration
wizard or try "help" to see all commands available.
dbvrep> setup wizard
This wizard configures Dbvisit Replicate.
The setup wizard creates configuration scripts, which need
to be run after the wizard ends. No changes to the databases are made before
that.
The progress is saved every time a list of databases,
replications, etc. is shown. It will be re-read if wizard is restarted and the
same DDC name and script path is selected.
Run the wizard now? [yes]
Accept end-user license agreement? (view/yes/no) [view] yes
Before starting the actual configuration, some basic
information is needed. The DDC name and script path determines where all files
created by the wizard go (and where to reread them if wizard is
rerun) and the license key determines which options are
available for this configuration.
(DDC_NAME) - Please enter a name for this replication
(suggestion: use the name of the source database): [] TEST-EC2
(LICENSE_KEY) - Please enter your license key: [(trial)]
Which Replicate edition do you want to trial (LTD/XTD/MAX):[MAX]
(SETUP_SCRIPT_PATH) - Please enter a directory for location
of configuration scripts on this machine: [/home/oracle/TEST-EC2] /home/oracle/TEST
Network configuration files were detected on this system in
these locations:
/u01/app/oracle/product/11.2.0/db_1/network/admin
/u01/app/oracle/product/10.2.0/db_1/network/admin
/u01/app/oracle/product/11.2.0/db_1/network/admin
(TNS_ADMIN) - Please enter TNS configuration directory for
this machine: [/u01/app/oracle/product/11.2.0/db_1/network/admin]
Step 1 - Describe databases
========================================
The first step is to describe databases used in the
replication. There are usually two of them (source and target).
Store SYSDBA and DBA passwords? Passwords only required
during setup and initialization? (yes/no) [yes]
Let's configure the database, describing it's type,
connectivity, user names etc.
What type of database is this? (Oracle/MySQL/SQL
Server/Oracle AWS RDS/CSV/Hadoop): [Oracle]
Please enter database TNS alias: [] SOURCE_DB
Please enter SYSDBA user name: [SYS]
Please enter password for this user: [change_on_install] ******
Please enter user with DBA role: [SYSTEM]
Please enter password for this user: [manager] ******
Connecting to database DZ11G as SYSTEM to query list of
tablespaces and to detect ASM (by looking whether any redo logs or archived
logs are stored in ASM).
Enter the Dbvisit Replicate owner (this user will be created
by this script): [dbvrep]
Please enter password for this user: [dbvpasswd]
Permanent tablespaces detected on the database: USERS.
Please enter default permanent tablespace for this user:[USERS]
Temporary tablespaces detected on the database: TEMP.
Please enter default temporary tablespace for this user:[TEMP]
Following databases are now configured:
1: Oracle DZ11G, SYS/***, SYSTEM/***, dbvrep/***,
USERS/TEMP, dbvrep/, ASM:NO, TZ: +00:00
Enter the number of the database to modify it, or
"add", or "done": [add]
Let's configure the database, describing it's type,
connectivity, user names etc.
What type of database is this? (Oracle/MySQL/SQL
Server/Oracle AWS RDS/CSV/Hadoop): [Oracle]
Please enter database TNS alias: [] EC2_TARGET
Please enter SYSDBA user name: [SYS]
Please enter password for this user: [change_on_install] *******
Please enter user with DBA role: [SYSTEM]
Please enter password for this user: [manager] **********
Connecting to database EC2_TARGET as SYSTEM to query list of
tablespaces and to detect ASM (by looking whether any redo logs or archived
logs are stored in ASM).
Enter the Dbvisit Replicate owner (this user will be created
by this script): [dbvrep]
Please enter password for this user: [dbvpasswd]
Permanent tablespaces detected on the database: USERS.
Please enter default permanent tablespace for this user: [USERS]
Temporary tablespaces detected on the database: TEMP.
Please enter default temporary tablespace for this user: [TEMP]
Following databases are now configured:
1: Oracle SOURCE_DB, SYS/***, SYSTEM/***, dbvrep/***,
USERS/TEMP, dbvrep/, ASM:NO, TZ: +00:00
2: Oracle EC2_TARGET, SYS/***, SYSTEM/***, dbvrep/***, USERS/TEMP,
dbvrep/***, ASM:NO, TZ: +00:00
Enter the number of the database to modify it, or
"add", or "done": [done]
Step 2 - Replication pairs
========================================
The second step is to set source and targets for each
replication pair.
Let's configure the replication pair, selecting source and
target.
Following databases are described:
1: SOURCE_DB (Oracle)
2: EC2_TARGET (Oracle)
Select source database: [1]
Select target database: [2]
Will be DDL replication enabled? [yes]
Use fetcher to offload the mining to a different server?
(yes/no) [no]
Would you like to encrypt the data across the network (yes
or no) [no]
Would you like to compress the data across the network (yes
or no) [no]
How long do you want to set the network timeouts.
Recommended range between 60-300 seconds [60]
Lock and copy the data initially one-by-one or at a single SCN?
one-by-one : Lock tables one by one and capture SCN
single-scn : One SCN for all tables
ddl-only : Only DDL
script for target objects
resetlogs : Use SCN
from last resetlogs operation (standby activation, rman incomplete
recovery)
(one-by-one/single-scn/ddl-only/resetlogs) [single-scn]
What data instantiation script to create?
dp_networklink : Data Pump with network link. No export
datafile is created (APPLY.sh)
dp_exp : Data Pump with export datafile. Creates an export datafile (APPLY.sh)
exp : Pre-datapump exp/imp with export datafile (APPLY.sh)
ddl_file : DDL file created (APPLY.sql)
ddl_run : DDL is automatically executed on target
load : All replicated data is created and loaded automatically
none
(dp_networklink/dp_exp/exp/ddl_file/ddl_run/load/none) [dp_networklink]
Following replication pairs are now configured:
1: SOURCE_DB (Oracle) ==> EC2_TARGET (Oracle), DDL: yes, fetcher: no,
process suffix: (no suffix), compression: no, encryption: no, network timeout:
60, prepare type: single-scn, data load: dp_networklink
Enter number of replication pair to modify it, or
"add", or "done": [done]
Step 3 - Replicated tables
========================================
The third step is to choose the schemas and tables to be
replicated. If the databases are reachable, the tables are checked for
existence, datatype support, etc., schemas are queried for tables. Note
that all messages are merely hints/warnings and may be
ignored if issues are rectified before the scripts are actually executed.
Following tables are defined for replication pairs:
1: SOURCE_DB (Oracle) ==> EC2_TARGET (Oracle), DDL: yes, suffix: (no
suffix), prepare: single-scn
No tables defined.
Enter number of replication pair to modify it, or
"done": [1]
Please enter list of all individual tables to be replicated.
Enter schema name(s) only to replicate all tables in that schema. Use comma or
space to delimit the entries.
Enter the tables and schemas: [] SCOTT
Add more tables or schemas? (YES/NO) [NO] NO
To replicate changes of PL/SQL objects in schema(s), please
enter the schemas to be replicated. Note that specifying any entry will cause
additional privileges to be granted to dbvrep. Enter through a
comma or space-delimited list.
Enter the list of schemas (PL/SQL): []
You can also specify some advanced options:
1. Rename schemas or tables.
2. Specify filtering conditions.
3. (Tables only) Configure Change Data Capture; this does
not maintain a copy of the source table, but logs all operations as separate
entries. This is useful for ETL or as an audit trail. This usually
requires adding of new columns (timestamps, old/new values
etc.) to the target table.
Specify rename name or filter condition for any of the
specified schemas (YES/NO): [no]
Following tables are defined for replication pairs:
1: SOURCE_DB (Oracle) ==> EC2_TARGET (Oracle), DDL: yes, suffix: (no
suffix), prepare: single-scn SCOTT(tables)
Enter number of replication pair to modify it, or
"done": [done]
Step 4 - Process configuration
========================================
The fourth step is to configure the replication processes
for each replication.
Following processes are defined:
1: MINE on SOURCE_DB
Not configured.
2: APPLY on EC2_TARGET
Not configured.
Enter number of process to modify it, or "done": [1]
Fully qualified name of the server for the process (usually
co-located with the database, unless mine is offloaded using fetcher):[sourceserver.com]
Server type (Windows/Linux/Unix): [Linux]
Enable email notifications about problems (yes/no)? [no]
Enable SNMP traps/notifications about problems (yes/no)?[no]
Directory with DDC file and default where to create log
files etc. (recommended: same as global setting, if possible)? [/home/oracle/TEST]
Following settings were pre-filled with defaults or your
reloaded settings:
----------------------------------------
[MINE_REMOTE_INTERFACE]: Network remote interface:sourceserver.com:7901
[MINE_DATABASE]: Database TNS: SOURCE_DB
[TNS_ADMIN] tnsnames.ora path: /u01/app/oracle/product/11.2.0/db_1/network/admin
[MINE_PLOG]: Filemask for generated plogs:
/home/oracle/TEST/mine/%S.%E (%S is sequence, %T thread, %F original
filename (stripped extension), %P process type, %N process name, %E default extension)
[LOG_FILE]: General log file: /home/oracle/TEST/log/dbvrep_%N_%D.%E
[LOG_FILE_TRACE]: Error traces: /home/oracle/TEST/log/trace/dbvrep_%N_%D_%I_%U.%E
Checking that these settings are valid...
Do you want change any of the settings? [no]
Following processes are defined:
1: MINE on SOURCE_DB
Host: sourceserver.com, SMTP: no, SNMP: no
2: APPLY on EC2_TARGET
Not configured.
Enter number of process to modify it, or "done": [2]
Fully qualified name of the server for the process (usually
co-located with the database, unless mine is offloaded using fetcher):
[ip-10-47-141-210] ec2-54-82-3-111.compute-1.amazonaws.com
Server type (Windows/Linux/Unix): [Linux]
Enable email notifications about problems (yes/no)? [no]
Enable SNMP traps/notifications about problems (yes/no)?[no]
Directory with DDC file and default where to create log
files etc. (recommended: same as global setting, if possible)? [/home/oracle/TEST]
Following settings were pre-filled with defaults or your
reloaded settings:
----------------------------------------
[APPLY_REMOTE_INTERFACE]: Network remote interface:
ec2-54-82-3-111.compute-1.amazonaws.com:7902
[APPLY_DATABASE]: Database TNS: EC2_TARGET
[TNS_ADMIN]: tnsnames.ora path: /u01/app/oracle/product/11.2.0/db_1/network/admin
[APPLY_STAGING_DIR]: Directory for received plogs: /home/oracle/TEST/apply
[LOG_FILE]: General log file: /home/oracle/TEST/log/dbvrep_%N_%D.%E
[LOG_FILE_TRACE]: Error traces: /home/oracle/TEST/log/trace/dbvrep_%N_%D_%I_%U.%E
Checking that these settings are valid...
Do you want change any of the settings? [no] no
Following processes are defined:
1: MINE on SOURCE_DB
Host: sourceserver.com, SMTP: no, SNMP: no
2: APPLY on EC2_TARGET
Host: ec2-54-82-3-111.compute-1.amazonaws.com, SMTP: no, SNMP: no
Enter number of process to modify it, or "done": [done]
Created file /home/oracle/TEST/TEST-EC2-APPLY.ddc.
Created file /home/oracle/TEST/TEST-EC2-MINE.ddc.
Created file /home/oracle/TEST/config/TEST-EC2-setup.dbvrep.
Created file /home/oracle/TEST/config/TEST-EC2-dbsetup_SOURCE_DB.sql.
Created file /home/oracle/TEST/config/TEST-EC2-dbsetup_EC2_TARGET.sql.
Created file /home/oracle/TEST/config/TEST-EC2-grants_SOURCE_DB.sql.
Created file /home/oracle/TEST/config/TEST-EC2-grants_EC2_TARGET.sql.
Created file /home/oracle/TEST/config/TEST-EC2-onetime.ddc.
Created file /home/oracle/TEST/start-console.sh.
Created file /home/oracle/TEST/TEST-EC2-run-sourceserver.com.sh.
Created file /home/oracle/TEST/TEST-EC2-run-ec2-54-82-3-111.compute-1.amazonaws.com.sh.
Created file /home/oracle/TEST/Nextsteps.txt.
Created file /home/oracle/TEST/TEST-EC2-all.sh.
========================================================================================================================================================================================================
Dbvisit Replicate wizard completed
Script /home/oracle/TEST/TEST-EC2-all.sh created. This
runs all the above created scripts. Please exit out of dbvrep, review and run
script as current user to setup and start Dbvisit Replicate.
========================================================================================================================================================================================================
Optionally, the script can be invoked now by this wizard.
Run this script now? (yes/no) [NO]
dbvrep>
oracle@dbvlin210[/home/oracle/TEST]: ./TEST-EC2-all.sh
Setting up Dbvisit Replicate configuration
Configure database SOURCE_DB...
This check fails if the DBID is not the expected one...
Ok, check passed.
Configure database EC2_TARGET...
This check fails if the DBID is not the expected one...
Ok, check passed.
Object grants for database SOURCE_DB...
Object grants for database EC2_TARGET...
Setting up the configuration
Initializing......done
WARN-1850: No DDC DB available, dictionary table does not
exist.
DDC loaded from database (0 variables).
Dbvisit Replicate version 2.6.01.3880
Copyright (C) Dbvisit Software Limited. All rights reserved.
DDC file /home/oracle/TEST/config/TEST-EC2-onetime.ddc loaded.
MINE: Cannot determine Dbvisit Replicate dictionary version.
(no dictionary exists)
APPLY: Cannot determine Dbvisit Replicate dictionary
version. (no dictionary exists)
dbvrep> #clear the no-DDC-DB-available warning
dbvrep> engine clear previous warnings
dbvrep> set ON_WARNING SKIP
Variable ON_WARNING set to SKIP for process *.
dbvrep> set ON_ERROR EXIT
Variable ON_ERROR set to EXIT for process *.
dbvrep> ENGINE SWITCH_REDOLOG
dbvrep> ENGINE SETUP MINE DROP DICTIONARY
0 dictionary objects dropped.
dbvrep> ENGINE SETUP MINE CREATE DICTIONARY
dbvrep> ENGINE SETUP MINE LOAD DICTIONARY
Supplemental logging on database set.
Loading dictionary table DBRSCOL$
Loading dictionary table DBRSIND$
Loading dictionary table DBRSOBJ$
Loading dictionary table DBRSTAB$
Loading dictionary table DBRSUSER$
Loading dictionary table DBRSV_$DATABASE
ID of mine proces is 03C8E202-A92F-11E3-BCBE-75B651CB262D.
If not using DDC in database, set MINE_UNIQUE_ID to this value.
dbvrep> ENGINE SETUP APPLY DROP DICTIONARY
0 dictionary objects dropped.
dbvrep> ENGINE SETUP APPLY CREATE DICTIONARY
dbvrep> ENGINE SETUP APPLY LOAD DICTIONARY
dbvrep> ENGINE PREPARE_DP SETUP CLEAR
dbvrep> ENGINE SETUP PAIR MINE AND APPLY
1 applier SCN set.
dbvrep> SET APPLY.INSTANTIATE_SCN NOW
Variable INSTANTIATE_SCN set to NOW for process APPLY.
dbvrep> ENGINE SUPPLEMENTAL LOGGING SCHEMA SCOTT ENABLE
PRIMARY KEY
dbvrep> ENGINE SWITCH_REDOLOG
dbvrep> ENGINE WAIT_SCN_FLIP
Waited 3 seconds until scn_to_timestamp changed.
dbvrep> #single-scn instantiation: lock all tables and schemas
dbvrep> ENGINE LOCK SCHEMAS SCOTT
Locking all schemas.
Lock done.
dbvrep> #single-scn instantiation: unlock all tables and
schemas, but keep the SCN
dbvrep> ENGINE LOCK RELEASE LOCKS
dbvrep>
dbvrep> #prepare the tables (we use OFFLINE as neither
MINE not APPLY is running; with OFFLINE we won't wait on network timeout)
dbvrep> PREPARE OFFLINE SCHEMA SCOTT
Table SCOTT.BONUS instantiated at SCN 4564856
Table SCOTT.DEPT instantiated at SCN 4564856
Table SCOTT.EMP instantiated at SCN 4564856
Table SCOTT.SALGRADE instantiated at SCN 4564856
dbvrep> #single-scn instantiation: unlock all tables and
schemas, forget the SCN (so it does not affect any further PREPARE
statements)
dbvrep> ENGINE LOCK CLEAR SCN
dbvrep> ENGINE SWITCH_REDOLOG
dbvrep> #prepare script for instantiation
dbvrep> ENGINE PREPARE_DP WRITE DP_NETWORKLINK DIRECTORY
DATA_PUMP_DIR FILE /home/oracle/TEST/APPLY.sh DBLINK TEST_DB_LINK USERID SYSTEM/*****@EC2_TARGET
Created Data Pump script /home/oracle/TEST/APPLY.sh, using network import.
dbvrep> create ddcdb from ddcfile
DDC loaded into database (316 variables).
dbvrep> set ON_WARNING SKIP
Variable ON_WARNING set to SKIP for process *.
dbvrep> set ON_ERROR SKIP
Variable ON_ERROR set to SKIP for process *.
OK-0: Completed successfully.
These steps are required after the TEST-EC2-all.sh script runs:
1) Create the necessary directory(ies) on the servers:
ec2-54-82-3-111.compute-1.amazonaws.com: /home/oracle/TEST
2) Copy the DDC files to the server(s) where the processes will run:
ec2-54-82-3-111.compute-1.amazonaws.com: /home/oracle/TEST/TEST-EC2-APPLY.ddc
sourceserver.com: /home/oracle/TEST/TEST-EC2-MINE.ddc
3) Review that path to dbvrep executable is correct in the run scripts:
/home/oracle/TEST/TEST-EC2-run-sourceserver.com
/home/oracle/TEST/TEST-EC2-run-ec2-54-82-3-111.compute-1.amazonaws.com.sh
4) Copy the run script to the server(s) where the processes will run:
sourceserver.com: /home/oracle/TEST/TEST-EC2-run-sourceserver.com.sh
ec2-54-82-3-111.compute-1.amazonaws.com: /home/oracle/TEST/TEST-EC2-run-ec2-54-82-3-111.compute-1.amazonaws.com.sh
5) Ensure firewall is open for listen interfaces
0.0.0.0:7902, 0.0.0.0:7901 used by the processes.
6) Make sure the data on apply are in sync as of time when setup was run.
Scripts for Data Pump/export/DDL were created as requested: /home/oracle/TEST/APPLY.sh
Create referenced database links (if any) before running the scripts.
7) Start the replication processes on all servers:
sourceserver.com: /home/oracle/TEST/TEST-EC2-run-sourceserver.com.sh
ec2-54-82-3-111.compute-1.amazonaws.com: /home/oracle/TEST/TEST-EC2-run-ec2-54-82-3-111.compute-1.amazonaws.com.sh
8) Start the console to monitor the progress:
/home/oracle/TEST/start-console.sh
The above list is stored in
/home/oracle/TEST/Nextsteps.txt.
oracle@[/home/oracle/TEST]: ./APPLY.sh
Import: Release 11.2.0.4.0 - Production on Wed Mar 12 04:22:48 2014
Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved.
Connected to: Oracle Database 11g Express Edition Release
11.2.0.2.0 - 64bit Production
Starting "SYSTEM"."D_0001": SYSTEM/********@EC2_TARGET
table_exists_action=TRUNCATE network_link=DZ11G directory=DATA_PUMP_DIR
flashback_scn=4564856 tables=SCOTT.BONUS,SCOTT.DEPT,SCOTT.EMP,SCOTT.SALGRADE
logfile=SCOTT_SALGRADE.log JOB_NAME=DP_DZ11G_0001
Estimate in progress using BLOCKS method...
Processing object type TABLE_EXPORT/TABLE/TABLE_DATA
Total estimation using BLOCKS method: 192 KB
Processing object type TABLE_EXPORT/TABLE/TABLE
. . imported "SCOTT"."DEPT" 4 rows
. . imported "SCOTT"."EMP" 14 rows
. . imported "SCOTT"."SALGRADE" 5 rows
. . imported "SCOTT"."BONUS" 0 rows
Processing object type TABLE_EXPORT/TABLE/GRANT/OWNER_GRANT/OBJECT_GRANT
Processing object type TABLE_EXPORT/TABLE/INDEX/INDEX
Processing object type
TABLE_EXPORT/TABLE/CONSTRAINT/CONSTRAINT
Processing object type
TABLE_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS
Processing object type TABLE_EXPORT/TABLE/CONSTRAINT/REF_CONSTRAINT
Processing object type
TABLE_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS
Job "SYSTEM"."D0001"
successfully completed at 15:43:33
oracle@[/home/oracle/TEST]: ./TEST-EC2-run-sourceserver.sh
Initializing......done
DDC loaded from database (316 variables).
Dbvisit Replicate version 2.6.01.3880
Copyright (C) Dbvisit Software Limited. All rights reserved.
DDC file /home/oracle/TEST/TEST-EC2-MINE.ddc loaded.
Starting process MINE...Created directory /home/oracle/TEST/ddc_backup
Created directory /home/oracle/TEST/log/trace/
Created directory /home/oracle/TEST/mine/
started
*************am I missing something here?***************************
oracle@[/home/oracle/TEST]: ./start-console.sh
\ Dbvisit Replicate 2.6.01.3880(MAX edition) - Evaluation
License expires in 30 days
MINE IS running. Currently at plog 141 and SCN 4587877 (03/12/2014 05:36:19).
APPLY IS running. Currently at plog 141 and SCN 4587867 (03/12/2014 05:36:17).
Progress of replication DZ11G:MINE->APPLY: total/this execution
--------------------------------------------------------------------------------------------------------------------------------------------
SCOTT.DEPT:
100% Mine:70/70 Unrecov:0/0 Applied:70/70 Conflicts:0/0 Last:11/03/2014 15:57:08/OK
SCOTT.EMP:
100% Mine:1/1 Unrecov:0/0 Applied:1/1 Conflicts:0/0 Last:11/03/2014 15:58:43/OK
--------------------------------------------------------------------------------------------------------------------------------------------
oracle@[/home/oracle/TEST]: scp -i
/home/oracle/aws/dbv-lab-keypair.pem /home/oracle/SOURCE_DB/DZ11G-APPLY.ddc
root@ec2-54-82-3-191.compute-1.amazonaws.com:/home/oracle/TEST/DZ11G-APPLY.ddc
DZ11G-APPLY.ddc
oracle@dbvlin210[/home/oracle/SOURCE_DB]: scp -i
/home/oracle/aws/dbv-lab-keypair.pem
/home/oracle/SOURCE_DB/DZ11G-run-ec2-54-82-3-191.compute-1.amazonaws.com.sh
root@ec2-54-82-3-191.compute-1.amazonaws.com:/home/oracle/SOURCE_DB/DZ11G-run-ip-10-47-141-210.sh
DZ11G-run-ec2-54-82-3-191.compute-1.amazonaws.com.sh
100% 101
0.1KB/s 00:00
On EC2 Intance as root:
root@ip-10-47-141-210[/home/oracle/SOURCE_DB] # ls -l
total 8
-rw-r----- 1 root root 524 Mar 11 15:37 DZ11G-APPLY.ddc
-rwxr----- 1 root root 101 Mar 11 15:37 DZ11G-run-ip-10-47-141-210.sh
root@ip-10-47-141-210[/home/oracle/SOURCE_DB] # chown
oracle:oinstall DZ*
root@ip-10-47-141-210[/home/oracle/SOURCE_DB] # ls -l
total 8
-rw-r----- 1 oracle oinstall 524 Mar 11 15:37
DZ11G-APPLY.ddc
-rwxr----- 1 oracle oinstall 101 Mar 11 15:37
DZ11G-run-ip-10-47-141-210.sh
dbvrep> setup wizard
This wizard configures Dbvisit Replicate to start a replication process.
The setup wizard creates configuration scripts, which need to be run after the wizard ends. No changes to the databases are made before that.
The progress is saved every time a list of databases, replications, etc. is shown. It will be re-read if wizard is restarted and the same DDC name and
script path is selected.
Run the wizard now? [yes]
Accept end-user license agreement? (view/yes/no) [view] yes
Before starting the actual configuration, some basic information is needed. The DDC name and script path determines where all files created by the wizard
go (and where to reread them if wizard is rerun) and the license key determines which options are available for this configuration.
(DDC_NAME) - Please enter a name for this replication (suggestion: use the name of the source database): [] RDSPOC
(LICENSE_KEY) - Please enter your license key (or just enter "(trial)"): [(trial)]
Which Replicate edition do you want to trial (LTD/XTD/MAX): [MAX]
(SETUP_SCRIPT_PATH) - Please enter a directory for location of configuration scripts on this machine: [/home/oracle/RDSPOC]
Network configuration files were detected on this system in these locations:
/u01/app/oracle/product/11.2.0/xe/network/admin
(TNS_ADMIN) - Please enter TNS configuration directory for this machine: [/u01/app/oracle/product/11.2.0/xe/network/admin]
Step 1 - Describe databases
========================================
The first step is to describe databases used in the replication. There are usually two of them (source and target); however, there can be just one (when
source and target is the same) or more than two (one-to-many or other complex configurations.)
Store SYSDBA and DBA passwords? Passwords only required during setup and initialization? (yes/no) [yes]
Let's configure the database, describing it's type, connectivity, user names etc.
What type of database is this? (Oracle/MySQL/SQL Server): [Oracle]
Please enter database TNS alias: [] XE
Please enter SYSDBA user name: [SYS]
Please enter password for this user: [change_on_install] <<- Test system is using default passwords
Please enter user with DBA role: [SYSTEM]
Please enter password for this user: [manager]
Connecting to database XE as SYSTEM to query list of tablespaces and to detect ASM (by looking whether any redo logs or archived logs are stored in ASM).
Enter the Dbvisit Replicate owner and apply user (this user will be created by this script): [dbvrep]
Please enter password for this user: [dbvpasswd]
Permanent tablespaces detected on the database: DATA, USERS.
Please enter default permanent tablespace for this user: [DATA] USERS
Temporary tablespaces detected on the database: TEMP.
Please enter default temporary tablespace for this user: [TEMP]
Following databases are now configured:
1: Oracle XE, SYS/***, SYSTEM/***, dbvrep/***, USERS/TEMP, dbvrep/, ASM:NO, TZ: +00:00
Enter number of database to modify it, or "add", or "done": [add]
Let's configure the database, describing it's type, connectivity, user names etc.
What type of database is this? (Oracle/MySQL/SQL Server): [Oracle]
Please enter database TNS alias: [] AWSDB
Please enter SYSDBA user name: [SYS] dbvadmin <<- Specify AWS RDS instance DBA user Account
Please enter password for this user: [change_on_install] ********
Please enter user with DBA role: [SYSTEM] dbvadmin <<- Specify AWS RDS instance DBA user Account
Please enter password for this user: [manager] ********
Connecting to database AWSDB as dbvadmin to query list of tablespaces and to detect ASM (by looking whether any redo logs or archived logs are stored in
ASM).
Enter the Dbvisit Replicate owner and apply user (this user will be created by this script): [dbvrep]
Please enter password for this user: [dbvpasswd]
Permanent tablespaces detected on the database: DATA, RDSADMIN, USERS.
Please enter default permanent tablespace for this user: [DATA] USERS
Temporary tablespaces detected on the database: TEMP.
Please enter default temporary tablespace for this user: [TEMP]
Following databases are now configured:
1: Oracle XE, SYS/***, SYSTEM/***, dbvrep/***, USERS/TEMP, dbvrep/, ASM:NO, TZ: +00:00
2: Oracle AWSDB, dbvadmin/***, dbvadmin/***, dbvrep/***, USERS/TEMP, dbvrep/, ASM:NO, TZ: +00:00
Enter number of database to modify it, or "add", or "done": [done]
Step 2 - Replication pairs
========================================
The second step is to set source and targets for each replication pair. This is usually just choosing the first database as source and the second one as
target, but many more configurations are possible.
Let's configure the replication pair, selecting source and target.
Following databases are described:
1: XE (Oracle)
2: AWSDB (Oracle)
Select source database: [1]
Select target database: [2]
Will be DDL replication enabled? (If YES, the script will grant more privileges to the Dbvisit Replicate users and enable database-wide supplemental logging): [yes]
Use fetcher to offload the mining to a different server? (yes/no) [no] NO
(NETWORK_QUALITY) - Please specify your network type (LAN or WAN). Autoconfigures timeouts, use of compression etc. [LAN] WAN
Lock and copy the data initially one-by-one or at a single SCN? (one-by-one/single-scn/ddl-only/resetlogs) [single-scn] single-scn
What data copy script to create? (dp_networklink/dp_exp/exp/ddl_file/ddl_run/load/none) [dp_networklink] dp_networklink
Following replication pairs are now configured:
1: XE (Oracle) ==> AWSDB (Oracle), DDL: yes, fetcher: NO, process suffix: (no suffix), network: WAN, prepare type: single-scn, data load: dp_networklink
Enter number of replication pair to modify it, or "add", or "done": [done]
Step 3 - Replicated tables
========================================
The third step is to choose the schemas and tables to be replicated. If the databases are reachable, the tables are checked for existence, datatype
support, etc., schemas are queried for tables. Note that all messages are merely hints/warnings and may be ignored if issues are rectified before the
scripts are actually executed.
Note the following assumptions are made in this wizard – which can be modified by editing the resulting script:
1. All replicated tables have a primary key defined.
2. All columns of the tables and all tables of the specified schemas are replicated, and if DDL support is enabled, whenever a new column/table is added,
this should be replicate as well.
3. If an apply conflict arises, the default option is to try again repeatedly, until a different option is given or the underlying issue is resolved.
4. If an apply conflict arises, no more data will be replicated until the issue is resolved or ignored.
You can also specify some advanced options:
1. Rename schemas or tables.
2. Specify filtering conditions.
3. (Tables only) Configure Change Data Capture; this does not maintain a copy of the source table, but logs all operations as separate entries. This is
useful for ETL or as an audit trail. This usually requires adding of new columns (timestamps, old/new values etc.) to the target table. Dbvisit Replicate
can create the target table with the additional columns during setup by selecting "ddl_run" in the data copy section of Step 2.
Following tables are defined for replication pairs:
1: XE (Oracle) ==> AWSDB (Oracle), DDL: yes, suffix: (no suffix), prepare: single-scn
No tables defined.
Enter number of replication pair to modify it, or "done": [1]
Please enter list of all individual tables to be replicated. Enter schema name(s) only to replicate all tables in that schema. Use comma or space to
delimit the entries.
Enter the tables and schemas: [] OE
Selected schemas: OE
Add more tables or schemas? (YES/NO) [NO] NO
To replicate changes of PL/SQL objects in schema(s), please enter the schemas to be replicated. Note that specifying any entry will cause additional
privileges to be granted to dbvrep. Enter through a comma or space-delimited list.
Enter the list of schemas (PL/SQL): []
Specify rename name or filter condition for any of the specified schemas (YES/NO): [no]
Following tables are defined for replication pairs:
1: XE (Oracle) ==> AWSDB (Oracle), DDL: yes, suffix: (no suffix), prepare: single-scn
OE(tables)
Enter number of replication pair to modify it, or "done": [done]
Step 4 - Process configuration
========================================
The fourth step is to configure the replication processes for each replication. Although most options have reasonable defaults, manual input is required.
Following processes are defined:
1: MINE on XE
Not configured.
2: APPLY on AWSDB
Not configured.
Enter number of process to modify it, or "done": [1]
Fully qualified name of the server for the process (usually co-located with the database, unless mine is offloaded using fetcher): [dbvrds01]
Server type (Windows/Linux/Unix): [Linux]
Enable email notifications about problems (yes/no)? [YES] NO
Enable SNMP traps/notifications about problems (yes/no)? [NO] NO
Directory with DDC file and default where to create log files etc. (recommended: same as global setting, if possible)? [/home/oracle/RDSPOC]
Following settings were pre-filled with defaults or your reloaded settings:
----------------------------------------
[MINE_REMOTE_INTERFACE]: Network remote interface: dbvrds01:7901
[MINE_DATABASE]: Database TNS: XE
[TNS_ADMIN]: tnsnames.ora path: /u01/app/oracle/product/11.2.0/xe/network/admin
[MINE_USER]: Dbvisit Replicate database username: dbvrep
[MINE_PASSWORD]: Dbvisit Replicate database password: *********
[MINE_PLOG]: Filemask for generated plogs: /home/oracle/RDSPOC/mine/%S.%E (%S is sequence, %T thread, %F original filename (stripped extension), %P process
type, %N process name, %E default extension)
[LOG_FILE]: General log file: /home/oracle/RDSPOC/log/dbvrep_%N_%D.%E
[LOG_FILE_TRACE]: Error traces: /home/oracle/RDSPOC/log/trace/dbvrep_%N_%D_%I_%U.%E
Checking that these settings are valid...
Do you want change any of the settings? [no]
Following processes are defined:
1: MINE on XE
Host: dbvrds01, SMTP: NO, SNMP: NO
2: APPLY on AWSDB
Not configured.
Enter number of process to modify it, or "done": [2]
<usually co-located with the database, unless mine is offloaded using fetcher): [ip-10-143-181-189] dbvrds02
Server type (Windows/Linux/Unix): [Linux]
Enable email notifications about problems (yes/no)? [YES] NO
Enable SNMP traps/notifications about problems (yes/no)? [NO] NO
Directory with DDC file and default where to create log files etc. (recommended: same as global setting, if possible)? [/home/oracle/RDSPOC]
Following settings were pre-filled with defaults or your reloaded settings:
----------------------------------------
[APPLY_REMOTE_INTERFACE]: Network remote interface: dbvrds02:7902
[APPLY_DATABASE]: Database TNS: AWSDB
[TNS_ADMIN]: tnsnames.ora path: /u01/app/oracle/product/11.2.0/xe/network/admin
[APPLY_USER]: Dbvisit Replicate database username: dbvrep
[APPLY_PASSWORD]: Dbvisit Replicate database password: *********
[APPLY_STAGING_DIR]: Directory for received plogs: /home/oracle/RDSPOC/apply
[LOG_FILE]: General log file: /home/oracle/RDSPOC/log/dbvrep_%N_%D.%E
[LOG_FILE_TRACE]: Error traces: /home/oracle/RDSPOC/log/trace/dbvrep_%N_%D_%I_%U.%E
Checking that these settings are valid...
Do you want change any of the settings? [no]
Following processes are defined:
1: MINE on XE
Host: dbvrds01, SMTP: NO, SNMP: NO
2: APPLY on AWSDB
Host: dbvrds02, SMTP: NO, SNMP: NO
Enter number of process to modify it, or "done": [done]
Created file /home/oracle/RDSPOC/RDSPOC-APPLY.ddc.
Created file /home/oracle/RDSPOC/RDSPOC-MINE.ddc.
Created file /home/oracle/RDSPOC/config/RDSPOC-setup.dbvrep.
Created file /home/oracle/RDSPOC/config/RDSPOC-dbsetup_XE.sql.
Created file /home/oracle/RDSPOC/config/RDSPOC-dbsetup_AWSDB.sql.
Created file /home/oracle/RDSPOC/config/RDSPOC-grants_XE.sql.
Created file /home/oracle/RDSPOC/config/RDSPOC-grants_AWSDB.sql.
Created file /home/oracle/RDSPOC/config/RDSPOC-onetime.ddc.
Created file /home/oracle/RDSPOC/start-console.sh.
Created file /home/oracle/RDSPOC/RDSPOC-run-dbvrds01.sh.
Created file /home/oracle/RDSPOC/RDSPOC-run-dbvrds02.sh.
Created file /home/oracle/RDSPOC/Nextsteps.txt.
Created file /home/oracle/RDSPOC/RDSPOC-all.sh.
==========================================================================================================================================================
Dbvisit Replicate wizard completed
Script /home/oracle/RDSPOC/RDSPOC-all.sh created. This runs all the above created scripts. Please exit out of dbvrep, review and run script as current user
to setup and start Dbvisit Replicate.
==========================================================================================================================================================
Optionally, the script can be invoked now by this wizard.
Run this script now? (yes/no) [NO] NO
dbvrep> exit
|
Review APPLY DDC file
...