dbvrep> setup wizard
This wizard configures Dbvisit Replicate.

The setup wizard creates configuration scripts, which need to be run after the wizard ends. No changes to the databases are made before that.

The progress is saved every time a list of databases, replications, etc. is shown. It will be re-read if wizard is restarted and the same DDC name and script path is selected.
Run the wizard now? [Yes] Yes
Accept end-user license agreement? (View/Yes/No) [View] Yes

Before starting the actual configuration, some basic information is needed. The DDC name and script path determines where all files created by the wizard go (and where to reread them if wizard is rerun) and the license key determines which options are available for this configuration.
(DDC_NAME) - Please enter a name for this replication: [] ora2pg
(LICENSE_KEY) - Please enter your license key: [(trial)] trial
Which Replicate edition do you want to trial (LTD/XTD/MAX): [MAX] MAX
(SETUP_SCRIPT_PATH) - Please enter a directory for location of configuration scripts on this machine: [/home/oracle/Documents/ora2pg] /home/oracle/ora2pg

Network configuration files were detected on this system in these locations:
/u01/app/oracle/product/12.1.0.2/db_1/network/admin

(TNS_ADMIN) - Please enter TNS configuration directory for this machine: [/u01/app/oracle/product/12.1.0.2/db_1/network/admin] 


Step 1 - Describe databases
========================================
The first step is to describe databases used in the replication. There are usually two of them (source and target).
Store SYSDBA and DBA passwords? Passwords only required during setup and initialization? (Yes/No) [Yes] Yes
Yes
Let's configure the database, describing its type, connectivity, user names etc.
What type of database is this? (Oracle/MySQL/Google Cloud SQL/SQL Server/Tibero/Postgres/Oracle AWS RDS/CSV/Hadoop): [Oracle] oracle
oracle
Please enter database TNS alias: [] unitsrc
Please enter SYSDBA user name: [SYS] SYS
Please enter password for user SYS: [change_on_install] ******
Please enter user with DBA role: [SYSTEM] SYSTEM
Please enter password for user SYSTEM: [manager] ******

Connecting to database unitsrc as SYSTEM to query list of tablespaces and to detect ASM (by looking whether any redo logs or archived logs are stored in ASM).

Dbvisit Replicate requires the use of database user to store metadata.
Please enter the Dbvisit Replicate user name: [dbvrep] 
Please enter password for user dbvrep: [dbvpasswd] 

Dbvisit Replicate user dbvrep is recommended to store metadata. It is also possible to store metadata in another (non-source) database.
Store Dbvisit Replicate metadata in unitsrc database? This is a typical / recommended configuration. (Yes/No) [Yes] Yes

Dbvisit Replicate user dbvrep will be created by install scripts and used to store metadata.

Permanent tablespaces detected in database: USERS.
Please enter default permanent tablespace for user dbvrep: [USERS] USERS

Temporary tablespaces detected in database: TEMP.
Please enter default temporary tablespace for user dbvrep: [TEMP] TEMP

Following databases are now configured:
1: Oracle unitsrc, SYS/***, SYSTEM/***, dbvrep/***, USERS/TEMP, dbvrep/, ASM:No, TZ: -07:00

Enter the number of the database to modify it, or "add", or "done": [add] add

Let's configure the database, describing its type, connectivity, user names etc.
What type of database is this? (Oracle/MySQL/Google Cloud SQL/SQL Server/Tibero/Postgres/Oracle AWS RDS/CSV/Hadoop): [Oracle] postgres
Please enter Postgres database name: [postgres] 

Please enter database hostname: [localhost] oel712ctrg
Please enter user name of an administrator: [postgres] dbvrep_admin
Please enter password for this user: [] ******
Enter the user to log into apply database: [dbvrep] 
Please enter password for this user: [password] 
Enter the database (schema) to use for Dbvisit Replicate internal data (will be created in the script): [dbvrep] 

Following databases are now configured:
1: Oracle unitsrc, SYS/***, SYSTEM/***, dbvrep/***, USERS/TEMP, dbvrep/, ASM:No, TZ: -07:00
2: Postgres postgres, dbvrep_admin/***, dbvrep_admin/***, dbvrep/***, /, dbvrep/, ASM:n/a, TZ: 

Enter the number of the database to modify it, or "add", or "done": [done] done

Step 2 - Replication pairs
========================================
The second step is to set source and targets for each replication pair.
Let's configure the replication pair, selecting source and target.
Following databases are described:
1: UNITSRC#DBVREP (Oracle)
2: DBNAME=POSTGRES;HOST=OEL712CTRG#DBVREP (Postgres) (cannot be source: not an Oracle database)
Select source database: [1] 1
Select target database: [2] 2
Will limited DDL replication be enabled? (Yes/No) [Yes] Y
Use fetcher to offload the mining to a different server? (Yes/No) [No] No
Should where clauses (and Event Streaming) include all columns, not just changed and PK? (Yes/No) [No] No
Would you like to encrypt the data across the network? (Yes/No) [No] No
Would you like to compress the data across the network? (Yes/No) [No] No
How long do you want to set the network timeouts. Recommended range between 60-300 seconds [60] 60
Lock and copy the data initially one-by-one or at a single SCN?
one-by-one : Lock tables one by one and capture SCN
single-scn : One SCN for all tables
ddl-only   : Only DDL script for target objects
resetlogs  : Use SCN from last resetlogs operation (standby activation, rman incomplete recovery)
no-lock    : Do not lock tables. Captures previous SCN of oldest active transaction. Requires pre-requisite running of pre-all.sh script
(one-by-one/single-scn/ddl-only/resetlogs/no-lock) [single-scn] 

What data instantiation script to create?
ddl_file       : DDL file created (APPLY.sql)
ddl_run        : DDL is automatically executed on target
load           : All replicated data is created and loaded automatically
none
(ddl_file/ddl_run/load/none) [ddl_file] 

Following replication pairs are now configured:
1: unitsrc (Oracle) ==> postgres (Postgres), DDL: YES, fetcher: NO, process suffix: (no suffix), compression: NO, encryption: NO, network timeout: 60, prepare type: single-scn, data load: ddl_file

Enter number of replication pair to modify it, or "add", or "done": [done] done

Step 3 - Replicated tables
========================================
The third step is to choose the schemas and tables to be replicated. If the databases are reachable, the tables are checked for existence, datatype support, etc., schemas are queried for tables. Note that all messages are merely hints/warnings and may be ignored if issues are rectified before the scripts are actually executed.

Following tables are defined for replication pairs:
1: unitsrc (Oracle) ==> postgres (Postgres), DDL: YES, suffix: (no suffix), prepare: single-scn
  No tables defined.

Enter number of replication pair to modify it, or "done": [1] 1

Please enter list of all individual tables to be replicated. Enter schema name(s) only to replicate all tables in that schema. Use comma or space to delimit the entries.
Enter the tables and schemas: [] ora2pg

Selected schemas: ORA2PG
Add more tables or schemas? (Yes/No) [No] No

You can also specify some advanced options:
1. Exclude some tables from schema-level replication
2. Rename schemas or tables.
3. Specify filtering conditions.
4. (Tables only) Configure Event Streaming; this does not maintain a copy of the source table, but logs all operations as separate entries. This is useful for ETL or as an audit trail. This usually requires adding of new columns (timestamps, old/new values etc.) to the target table.
Specify rename name, filter condition or audit for any of the specified schemas? (Yes/No) [No] No
(PREPARE_SCHEMA_EXCEPTIONS) - Specify tables to exclude from PREPARE SCHEMA, if any: [] 

Following tables are defined for replication pairs:
1: unitsrc (Oracle) ==> postgres (Postgres), DDL: YES, suffix: (no suffix), prepare: single-scn
  ORA2PG(tables)

Enter number of replication pair to modify it, or "done": [done] done
done

Step 4 - Process configuration
========================================
The fourth step is to configure the replication processes for each replication.

Following processes are defined:
1: MINE on unitsrc
  Not configured.
2: APPLY on postgres
  Not configured.

Enter number of process to modify it, or "done": [1] 1
Fully qualified name of the server for the process (usually co-located with the database, unless mine is offloaded using fetcher): [oel712csrc] 
Server type (Windows/Linux/Unix): [Linux] linux
Enable email notifications about problems? (Yes/No) [No] 
Enable SNMP traps/notifications about problems? (Yes/No) [No] 
Directory with DDC file and default where to create log files etc. (recommended: same as global setting, if possible)? [/home/oracle/ora2pg] 

Following settings were pre-filled with defaults or your reloaded settings:
----------------------------------------
[MINE_REMOTE_INTERFACE]: Network remote interface: oel712csrc:7901 
[MINE_DATABASE]: Database TNS: unitsrc 
[TNS_ADMIN]: tnsnames.ora path: /u01/app/oracle/product/12.1.0.2/db_1/network/admin 
[MINE_PLOG]: Filemask for generated plogs: /home/oracle/ora2pg/mine/%S.%E (%S is sequence, %T thread, %F original filename (stripped extension), %P process type, %N process name, %E default extension)
[LOG_FILE]: General log file: /home/oracle/ora2pg/log/dbvrep_%N_%D.%E 
[LOG_FILE_TRACE]: Error traces: /home/oracle/ora2pg/log/trace/dbvrep_%N_%D_%I_%U.%E 

Checking that these settings are valid...
Do you want to change any of the settings? [No] No
No

Following processes are defined:
1: MINE on unitsrc
  Host: oel712csrc, SMTP: No, SNMP: No
2: APPLY on postgres
  Not configured.

Enter number of process to modify it, or "done": [2] 2
Fully qualified name of the server for the process (usually co-located with the database, unless mine is offloaded using fetcher): [oel712ctrg] oel712ctrg
Server type (Windows/Linux/Unix): [] linux
Enable email notifications about problems? (Yes/No) [No] 
Enable SNMP traps/notifications about problems? (Yes/No) [No] 
Directory with DDC file and default where to create log files etc. (recommended: same as global setting, if possible)? [/home/oracle/ora2pg] 

Following settings were pre-filled with defaults or your reloaded settings:
----------------------------------------
[APPLY_REMOTE_INTERFACE]: Network remote interface: oel712ctrg:7902 
[APPLY_DATABASE]: Database Postgres connection string: dbname=postgres;host=oel712ctrg 
[TNS_ADMIN]: tnsnames.ora path: /u01/app/oracle/product/12.1.0.2/db_1/network/admin 
[APPLY_SCHEMA]: Dbvisit Replicate database (schema): dbvrep 
[APPLY_STAGING_DIR]: Directory for received plogs: /home/oracle/ora2pg/apply 
[LOG_FILE]: General log file: /home/oracle/ora2pg/log/dbvrep_%N_%D.%E 
[LOG_FILE_TRACE]: Error traces: /home/oracle/ora2pg/log/trace/dbvrep_%N_%D_%I_%U.%E 

Checking that these settings are valid...
Do you want to change any of the settings? [No] No

Following processes are defined:
1: MINE on unitsrc
  Host: oel712csrc, SMTP: No, SNMP: No
2: APPLY on postgres
  Host: oel712ctrg, SMTP: No, SNMP: No

Enter number of process to modify it, or "done": [done] done
Created file /home/oracle/ora2pg/ora2pg-APPLY.ddc.
Created file /home/oracle/ora2pg/ora2pg-MINE.ddc.
Created file /home/oracle/ora2pg/config/ora2pg-setup.dbvrep.
Created file /home/oracle/ora2pg/config/ora2pg-dbsetup_UNITSRC_DBVREP.sql.
Created file /home/oracle/ora2pg/config/ora2pg-dbsetup_DBNAME_POSTGRES_HOST_OEL712CTRG_DBVREP.sql.
Created file /home/oracle/ora2pg/config/ora2pg-grants_UNITSRC_DBVREP.sql.
Created file /home/oracle/ora2pg/config/ora2pg-grants_DBNAME_POSTGRES_HOST_OEL712CTRG_DBVREP.sql.
Created file /home/oracle/ora2pg/config/ora2pg-onetime.ddc.
Created file /home/oracle/ora2pg/start-console.sh.
Created file /home/oracle/ora2pg/ora2pg-run-oel712csrc.sh.
Created file /home/oracle/ora2pg/scripts/ora2pg-oel712csrc-start-MINE.sh.
Created file /home/oracle/ora2pg/scripts/ora2pg-oel712csrc-stop-MINE.sh.
Created file /home/oracle/ora2pg/scripts/ora2pg-oel712csrc-dbvrep-MINE.sh.
Created file /home/oracle/ora2pg/scripts/systemd-dbvrep-MINE_ora2pg.service.
Created file /home/oracle/ora2pg/scripts/upstart-dbvrep-MINE_ora2pg.conf.
Created file /home/oracle/ora2pg/ora2pg-run-oel712ctrg.sh.
Created file /home/oracle/ora2pg/scripts/ora2pg-oel712ctrg-start-APPLY.sh.
Created file /home/oracle/ora2pg/scripts/ora2pg-oel712ctrg-stop-APPLY.sh.
Created file /home/oracle/ora2pg/scripts/ora2pg-oel712ctrg-dbvrep-APPLY.sh.
Created file /home/oracle/ora2pg/scripts/systemd-dbvrep-APPLY_ora2pg.service.
Created file /home/oracle/ora2pg/scripts/upstart-dbvrep-APPLY_ora2pg.conf.
Created file /home/oracle/ora2pg/ora2pg-sample_response_file.txt.
Created file /home/oracle/ora2pg/Nextsteps.txt.
Created file /home/oracle/ora2pg/ora2pg-all.sh.
======================================================================

Dbvisit Replicate wizard completed

Script /home/oracle/ora2pg/ora2pg-all.sh created. This runs all the above created scripts. Please exit out of dbvrep, review and run script as current user to setup and start Dbvisit Replicate.
======================================================================
Optionally, the script can be invoked now by this wizard.
Run this script now? (Yes/No) [No] No
dbvrep> exit
OK-0: Completed successfully.

[oracle@oel712csrc ora2pg]$ ./ora2pg-all.sh
Setting up Dbvisit Replicate configuration
Configure database unitsrc...

This check fails if the DBID is not the expected one...


Ok, check passed.
Configure database dbname=postgres
Object grants for database unitsrc...

Object grants for database dbname=postgres
Setting up the configuration
Initializing......done
DDC loaded from database (0 variables).
Dbvisit Replicate version 2.9.00
Copyright (C) Dbvisit Software Limited. All rights reserved.
DDC file /home/oracle/ora2pg/config/ora2pg-onetime.ddc loaded.
MINE: Cannot determine Dbvisit Replicate dictionary version. (no
dictionary exists)
APPLY: Cannot determine Dbvisit Replicate dictionary version. (no
dictionary exists)
dbvrep> #clear the no-DDC-DB-available warning
dbvrep> process clear previous warnings
dbvrep> set ON_WARNING SKIP
Variable ON_WARNING set to SKIP for process *.
dbvrep> set ON_ERROR EXIT
Variable ON_ERROR set to EXIT for process *.
dbvrep>
dbvrep> # Configuring default processes
dbvrep> choose process MINE
Process type MINE set to: MINE.
dbvrep> choose process APPLY
Process type APPLY set to: APPLY.
dbvrep> PROCESS SWITCH_REDOLOG
Redo log switch requested.dbvrep> PROCESS SETUP MINE DROP DICTIONARY
0 dictionary objects dropped.
dbvrep> PROCESS SETUP MINE CREATE DICTIONARY
dbvrep> PROCESS SETUP MINE LOAD DICTIONARY
Oldest active transaction SCN: 15142472 (no active transaction)
Supplemental logging on database set.
dbvrep> PROCESS SETUP APPLY DROP DICTIONARY
0 dictionary objects dropped.
dbvrep> PROCESS SETUP APPLY CREATE DICTIONARY
dbvrep> PROCESS SETUP APPLY LOAD DICTIONARY
dbvrep> PROCESS SETUP PAIR MINE AND APPLY
Applier SCN set (start=15142485, current=15142485).
dbvrep> SET APPLY.INSTANTIATE_SCN NOW
Variable INSTANTIATE_SCN set to NOW for process APPLY.
dbvrep> SET MINE._PREPARE_SUPLOG_TYPE PK
Variable _PREPARE_SUPLOG_TYPE set to PK for process MINE.
dbvrep> EXCLUDE CREATE TABLE %.DBMS_TABCOMP_TEMP_UNCMP #Ignore tables
created by Compression Advisor
Exclude rule created.
dbvrep> EXCLUDE CREATE TABLE %.DBMS_TABCOMP_TEMP_CMP #Ignore tables
created by Compression Advisor
Exclude rule created.
dbvrep> EXCLUDE CREATE TABLE %.SCHEDULER$_% #Ignore tables created by
Oracle scheduler (also used by schema/full expdp/impdp)
Exclude rule created.
dbvrep> EXCLUDE CREATE TABLE %.CMP1$% #Ignore tables created by
Compression Advisor since 11.2.0.4
Exclude rule created.
dbvrep> EXCLUDE CREATE TABLE %.CMP2$% #Ignore tables created by
Compression Advisor since 11.2.0.4
Exclude rule created.
dbvrep> EXCLUDE CREATE TABLE %.CMP3$% #Ignore tables created by
Compression Advisor since 11.2.0.4
Exclude rule created.
dbvrep> EXCLUDE CREATE TABLE %.CMP4$% #Ignore tables created by
Compression Advisor since 11.2.0.4
Exclude rule created.
dbvrep> memory_set IGNORE_APPLY_DDL_DIFFERENCES Yes
Variable IGNORE_APPLY_DDL_DIFFERENCES set to YES for process *.
dbvrep> SET PREPARE_SCHEMA_EXCEPTIONS none
Variable PREPARE_SCHEMA_EXCEPTIONS set to none for process *.
dbvrep> PROCESS SUPPLEMENTAL LOGGING SCHEMA "ORA2PG" ENABLE PRIMARY KEY
dbvrep> PROCESS SWITCH_REDOLOG
Redo log switch requested.dbvrep> PROCESS WAIT_SCN_FLIP
Waited 1 seconds until scn_to_timestamp changed.
dbvrep> #single-scn instantiation: lock all tables and schemas
dbvrep> PROCESS LOCK SCHEMAS "ORA2PG"
Locking all schemas.
...locked 0 of 0 tables from ORA2PG schema.
Lock done.
dbvrep> #single-scn instantiation: unlock all tables and schemas, but
keep the SCN
dbvrep> PROCESS LOCK RELEASE LOCKS
dbvrep>
dbvrep> #prepare the tables (we use OFFLINE as neither MINE nor APPLY
is running; with OFFLINE we won't wait on network timeout)
dbvrep> PREPARE OFFLINE SCHEMA "ORA2PG"
dbvrep>
dbvrep> #single-scn instantiation: unlock all tables and schemas,
forget the SCN (so it does not affect any further PREPARE statements)
dbvrep> PROCESS LOCK CLEAR SCN
dbvrep> PROCESS SWITCH_REDOLOG
Redo log switch requested.dbvrep> #prepare script for instantiation
dbvrep> PROCESS PREPARE_DP WRITE DDL_FILE FILE
/home/oracle/ora2pg/APPLY.sql USERID SYSTEM/odb10g@unitsrc
Created DDL script /home/oracle/ora2pg/APPLY.sql.
dbvrep> create ddcdb from ddcfile
DDC loaded into database (426 variables).
dbvrep> load ddcdb
DDC loaded from database (426 variables).
dbvrep> set ON_WARNING SKIP
Variable ON_WARNING set to SKIP for process *.
dbvrep> set ON_ERROR SKIP
Variable ON_ERROR set to SKIP for process *.
OK-0: Completed successfully.
WARN-1850: No DDC DB available, dictionary table does not exist.
These steps are required after the ora2pg-all.sh script runs:

1) Create the necessary directory(ies) on the servers:
oel712ctrg: /home/oracle/ora2pg

2) Copy the DDC files to the server(s) where the processes will run:
oel712ctrg: /home/oracle/ora2pg/ora2pg-APPLY.ddc
oel712csrc: /home/oracle/ora2pg/ora2pg-MINE.ddc

3) Review that path to dbvrep executable is correct in the run scripts:
/home/oracle/ora2pg/ora2pg-run-oel712csrc.sh
/home/oracle/ora2pg/ora2pg-run-oel712ctrg.sh

4) Copy the run script to the server(s) where the processes will run:
oel712csrc: /home/oracle/ora2pg/ora2pg-run-oel712csrc.sh
oel712ctrg: /home/oracle/ora2pg/ora2pg-run-oel712ctrg.sh

5) Ensure firewall is open for listen interfaces 0.0.0.0:7902, 0.0.0.0:7901 used by the processes.

6) Make sure the data on apply are in sync as of time when setup was run.
Scripts for Data Pump/export/DDL were created as requested:

Create referenced database links (if any) before running the scripts.
/home/oracle/ora2pg/APPLY.sql

7) Start the replication processes on all servers:
oel712csrc: /home/oracle/ora2pg/ora2pg-run-oel712csrc.sh
oel712ctrg: /home/oracle/ora2pg/ora2pg-run-oel712ctrg.sh

8) Start the console to monitor the progress:
/home/oracle/ora2pg/start-console.sh

The above list is stored in /home/oracle/ora2pg/Nextsteps.txt.

[oracle@oel712ctrg ora2pg]$ ls -ln
total 8
-rwxr----- 1 54321 54321   0 Apr 19 14:01 APPLY.sql
-rw-r----- 1 54321 54321 529 Apr 19 14:01 ora2pg-APPLY.ddc
-rwxr----- 1 54321 54321  97 Apr 19 14:01 ora2pg-run-oel712ctrg.sh

[oracle@oel712csrc ora2pg]$ pwd
/home/oracle/ora2pg


[oracle@oel712csrc ora2pg]$ ./ora2pg-run-oel712csrc.sh
Initializing......done
DDC loaded from database (426 variables).
Dbvisit Replicate version 2.9.00
Copyright (C) Dbvisit Software Limited. All rights reserved.
DDC file /home/oracle/ora2pg/ora2pg-MINE.ddc loaded.
Starting process MINE...Created directory /home/oracle/ora2pg/ddc_backup
Created directory /home/oracle/ora2pg/log/trace/
Created directory /home/oracle/ora2pg/mine/
started

[oracle@oel712ctrg ora2pg]$ pwd
/home/oracle/ora2pg


[oracle@oel712ctrg ora2pg]$ ./ora2pg-run-oel712ctrg.sh
Initializing......done
DDC loaded from database (427 variables).
Dbvisit Replicate version 2.9.00
Copyright (C) Dbvisit Software Limited. All rights reserved.
DDC file /home/oracle/ora2pg/ora2pg-APPLY.ddc loaded.
Starting process APPLY...Created directory /home/oracle/ora2pg/ddc_backup
Created directory /home/oracle/ora2pg/log/
Created directory /home/oracle/ora2pg/log/trace/
Created directory /home/oracle/ora2pg/apply
started

$ dbvrep --ddcfile /home/oracle/ora2pg/ora2pg-APPLY.ddc

| Dbvisit Replicate 2.9.00 (MAX edition) - Evaluation License expires in 30 days
MINE is running. Currently at plog 1002 and SCN 15154445 (04/19/2017 14:20:42).
APPLY is running. Currently at plog 1002 and SCN 15154423 (04/19/2017 14:20:38).
Progress of replication ora2pg:MINE->APPLY: total/this execution
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
0 tables listed.
dbvrep>

dbvrep> LIST PREPARE
List of prepared schemas:
ORA2PG (DDL)
List of prepared tables:
DBVREP.DBRSCOMMON_HEARTBEAT -> dbvrep.dbrscommon_heartbeat (table rename)
DBVREP.DBRSOBJ$ -> dbvrep.dbrsobj$ (table rename)
DBVREP.DBRSUSER$ -> dbvrep.dbrsuser$ (table rename)