Dbvisit Replicate 2.3.14.1603 For Linux/Unix and Windows Readme.txt Last Updated: $Date: 2012-05-11 01:30:05 -0700 (Fri, 11 May 2012) $ $Rev: 1599 $ www.dbvisit.com ========================= WHAT IS DBVISIT REPLICATE ========================= Dbvisit Replicate is a software product enabling real time Oracle replication. Dbvisit Replicate uses efficient redo log-based change data capture (CDC) technology to detect changes to replicate and distribute the changed data in real-time across Oracle and non Oracle databases. Dbvisit Replicate uses its own internal redo-log mining technology and is not dependent on any third-party software. One-way replication is possible allowing: - Real time reporting across distributed reader farms reducing the performance impact on the production databases. - Minimal downtime migration across different Oracle versions and platforms. - Source data change detection allowing real time loading of data warehouses. - Replication of critical data to other locations and databases. Two-way replication is possible allowing master-to-master replication to provide real time information across multiple applications and sites. Dbvisit Replicate can replicate from: Oracle -> Oracle Oracle -> MySQL Oracle -> SQL Server This software is fully functional commercial software and includes a 30-day evaluation license. A license key is needed to activate the software to change it from an evaluation version to a fully licensed version. To continue using Dbvisit Replicate after 30 days please go to www.dbvisit.com to order a license key. This README contains both install and upgrade instructions for Windows and Linux. This software can be used for new installs and upgrades. For complete instructions and user manual please see: http://www.dbvisit.com/content/online/dbvisit_replicate_userguide ========================== NLS considerations ========================== Dbvisit Replicate relies on the target database client settings for handling of NLS issues. For an Oracle target database, set the NLS_LANG environment variableto "AMERICAN_AMERICA.source_db_charset". The AMERICAN_AMERICA part ensures unified number and date formats and specifying the source database charset ensures that the client libraries handle any non-ASCII characters properly. ========================== DBVISIT REPLICATE COMMANDS ========================== Linux/Unix: To use Dbvisit Replicate type: i) If installed through tar file cd /usr/local/dbvisit/replicate (this may be different) ./dbvrep OR: ii) If installed through RPM dbvrep This brings up the Dbvisit Replicate command console. For help type "help" in the Dbvisit Replicate command console: dbvrep> help To configure Dbvisit Replicate: dbvrep> setup wizard Windows: Start -> Dbvisit Replicate Command Console For help type "help" in the Dbvisit Replicate command console: dbvrep> help To configure Dbvisit Replicate: dbvrep> setup wizard ========= WHATS NEW ========= Please see below in section CHANGES ======================================== WINDOWS - NEW INSTALLATIONS ======================================== 1) Unzip the dbvisit_replicate2.3.x_win.zip to a temporary location. 2) Double click the Dbvisit Replicate Windows installer: dbvrep2.2.x-Inst.exe 3) Follow the on-screen instructions to complete the software installation. 4) The steps above need to be done on both the source (mine) database server and the target (apply) database server. 5) On the source database server: Start the Dbvisit Replicate Command Console: Start -> Dbvisit Replicate Command Console 6) To configure Dbvisit Replicate, start the setup wizard: dbvrep> setup wizard 7) Follow the on-screen instructions. For complete instructions and user manual please see: http://www.dbvisit.com/content/online/dbvisit_replicate_userguide ======================================== LINUX/UNIX - NEW INSTALLATIONS (TAR FILE) ======================================== Dbvisit Replicate can be installed with any OS user id. However, if ASM is used on a source database, a bequeath connection must be made to the ASM instance and the user must be member of the dba or sysasm group. 1) Unzip the dbvisit_replicate2.3.x_.zip file to a directory on your local PC if you have not done so already. (where is the Operating System - linux, solaris etc) 2) Create the main Dbvisit install directory on your server. We recommend /usr/local/dbvisit/ - Make sure this directory has the appropriate owner and permissions. - The tar file will create the "replicate" subdirectory under the main dbvisit directory. This is where Dbvisit Replicate will be installed. 3) Copy or ftp the dbvrep2.3.x.tar file to your main Dbvisit directory (ie /usr/local/dbvisit) 4) Untar the dbvrep2.3.x.tar file with the command: tar xvf dbvrep2.3.x.tar This will have created the "replicate" subdirectory. This is where Dbvisit Replicate will be installed. 5) cd into the "replicate" directory. 6) Ensure all Dbvisit Replicate is executable with the command: chmod 750 dbvrep 7) The steps above need to be done on both the source (mine) database server and the target (apply) database server. 8) On the source database server: cd /usr/local/dbvisit/replicate Start Dbvisit Replicate Command Console: ./dbvrep 9) To configure Dbvisit Replicate, start the setup wizard: dbvrep> setup wizard 10) Follow the on-screen instructions. For complete instructions and user manual please see: http://www.dbvisit.com/content/online/dbvisit_replicate_userguide ======================================== LINUX x86 - NEW INSTALLATIONS (RPM) ======================================== Root privilege is needed to install the RPM. dbvisit_replicate-2.3.x-1.i386.rpm is the 32bit version. dbvisit_replicate-2.3.x-1.x86_64.rpm is the 64bit version. 1) Copy or ftp the dbvisit_replicate-2.3.x-1.i386.rpm file to a temp directory on your server (example: /usr/tmp) 2) su - Switch user to root. 3) Install the RPM package. rpm -ivh /usr/tmp/dbvisit_replicate-2.3.x-1.x86_64.rpm 4) This installs Dbvisit Replicate in: /usr/dbvisit/replicate/ and creates the following softlink: /usr/bin/dbvrep -> /usr/dbvisit/replicate/dbvrep 5) The steps above need to be done on both the source (mine) database server and the target (apply) database server. 6) On the source database server: Login as the user that is going to run Dbvisit Replicate and cd to a directory where you want to create the Dbvisit Replication Database Configuration (DDC) file. Example: cd /home/oracle/dbvrep 7) Start Dbvisit Replicate Command Console: dbvrep 8) To configure Dbvisit Replicate, start the setup wizard: dbvrep> setup wizard 9) Follow the on-screen instructions. For complete instructions and user manual please see: http://www.dbvisit.com/content/online/dbvisit_replicate_userguide ======================================== ======================================== WINDOWS - UPGRADE ======================================== 1) Unzip the dbvisit_replicate2.3.x_win.zip to a temporary location. 2) Double click the Dbvisit Replicate Windows installer: dbvrep2.3.x-Inst.exe 3) Follow the on-screen instructions to complete the software installation. 4) The steps above need to be done on both the source (mine) database server and the target (apply) database server. For complete upgrade instructions and user manual please see: http://www.dbvisit.com/content/online/dbvisit_replicate_userguide ======================================== LINUX/UNIX - UPGRADE (TAR FILE) ======================================== Dbvisit Replicate can be installed as the same user that owns the Oracle software. 1) Unzip the dbvisit_replicate2.3.x_.zip file to a directory on your local PC if you have not done so already. (where is the Operating System - linux, solaris etc) 2) Create the main Dbvisit install directory on your server. We recommend /usr/local/dbvisit/ - Make sure this directory has the appropriate owner and permissions. - The tar file will create the "replicate" subdirectory under the main dbvisit directory. This is where Dbvisit Replicate will be installed. 3) Copy or ftp the dbvrep2.3.x.tar file to your main Dbvisit directory (ie /usr/local/dbvisit) 4) Untar the dbvrep2.3.x.tar file with the command: tar xvf dbvrep2.3.x.tar This will have created the "replicate" subdirectory. This is where Dbvisit Replicate will be installed. 5) cd into the "replicate" directory. 6) Ensure all Dbvisit Replicate is executable with the command: chmod 750 dbvrep 7) The steps above need to be done on both the source (mine) database server and the target (apply) database server. For complete upgrade instructions and user manual please see: http://www.dbvisit.com/content/online/dbvisit_replicate_userguide ======================================== LINUX x86 - UPGRADE (RPM) ======================================== Root privilege is needed to upgrade the RPM. dbvisit_replicate-2.3.x-1.i386.rpm is the 32bit version. dbvisit_replicate-2.3.x-1.x86_64.rpm is the 64bit version. 1) Copy or ftp the dbvisit_replicate-2.3.x-1.i386.rpm file to a temp directory on your server (example: /usr/tmp) 2) su - Switch user to root. 3) Remove the current RPM package. rpm -e `rpm -qa | grep dbvisit_replicate` 4) Install the RPM package. rpm -ivh /usr/tmp/dbvisit_replicate-2.3.x-1.x86_64.rpm 5) The steps above need to be done on both the source (mine) database server and the target (apply) database server. For complete upgrade instructions and user manual please see: http://www.dbvisit.com/content/online/dbvisit_replicate_userguide ======================================== CONTACT INFORMATION ======================================== Web: http://www.dbvisit.com Support: http://www.dbvisit.com/support.php Dbvisit Replicate is a product of Dbvisit Software Limited and Avisit Solutions Limited. Dbvisit is a registered trademark of Avisit Solutions Limited. Oracle is a registered trademark of Oracle Corporation. Copyright (C) 2012 Dbvisit Software Limited and Avisit Solutions Limited. Except as specifically set out the Dbvisit Replicate license agreement, nothing in this Documentation constitutes a warranty as to the operation of the Dbvisit Replicate software. ======================================== COPYRIGHT NOTICES ======================================== This distribution contains implementation of the MD5 algorithm: Copyright (C) 1991-2, RSA Data Security, Inc. Created 1991. All rights reserved. License to copy and use this software is granted provided that it is identified as the "RSA Data Security, Inc. MD5 Message-Digest Algorithm" in all material mentioning or referencing this software or this function. License is also granted to make and use derivative works provided that such works are identified as "derived from the RSA Data Security, Inc. MD5 Message-Digest Algorithm" in all material mentioning or referencing the derived work. RSA Data Security, Inc. makes no representations concerning either the merchantability of this software or the suitability of this software for any particular purpose. It is provided "as is" without express or implied warranty of any kind. These notices must be retained in any copies of any part of this documentation and/or software. This distribution contains code derived from the sample UUID implementation in RFC 4122, which contains the following clause. /* ** Copyright (c) 1990- 1993, 1996 Open Software Foundation, Inc. ** Copyright (c) 1989 by Hewlett-Packard Company, Palo Alto, Ca. & ** Digital Equipment Corporation, Maynard, Mass. ** Copyright (c) 1998 Microsoft. ** To anyone who acknowledges that this file is provided "AS IS" ** without any express or implied warranty: permission to use, copy, ** modify, and distribute this file for any purpose is hereby ** granted without fee, provided that the above copyright notices and ** this notice appears in all source code copies, and that none of ** the names of Open Software Foundation, Inc., Hewlett-Packard ** Company, Microsoft, or Digital Equipment Corporation be used in ** advertising or publicity pertaining to distribution of the software ** without specific, written prior permission. Neither Open Software ** Foundation, Inc., Hewlett-Packard Company, Microsoft, nor Digital ** Equipment Corporation makes any representations about the ** suitability of this software for any purpose. */ ======================================== CHANGES ======================================== 2.3.14 (10 May 2012) 1. Fixed fetcher issue when current redo was overwritten, now correctly falls back to archived copy 2. DDL scripts and CTAS did not handle correctly RAW columns 3. CTAS could cause incorrect PLOG CRC checksum 4. Improve mine speed (limit flushing to disk, do not write progress so often to log) 5. Support for HP-UX 6. Support for 4k redo block size (Avanced Disk Format) 7. Fallback console when termcap does not work, LIST STATUS command 8. Speed-up prepare schema/table if single-scn is used 9. Fix in row-migration handling 2.3.12 (2 May 2012) 1. Fixes in fetcher (could crash in some scenarios) 2. Fixed crash when audit data was not preseentt in the source (e.g. DDL) 3. Fixed crash when mine re-read CTAS for table that has been unprepared 4. Table was still present in list progress/notify emails even when unprepared 5. Removed incorrect warning about SCN mismatch on big-endian platforms 6. Fixed error for overwrite/newer/older conflict handlers 7. Changed how much of a long error message is printed on crash 2.3.10 (30 April 2012) 1. Setup wizard now defaults to use INADR_ANY (0.0.0.0) for listen interfaces, thus making dbvrep processes listen on all network interfaces 2. Added directory for dp import 3. Check tables on apply only if no import scripts are created (e.g. for resetlogs instantiation) 4. Default conflict handlers 5. Streamline setup wizard questions for tables and schemas 6. Backup re-downloaded plogs (and redologs from fetcher). Can be useful for support. 7. Create table as select support (without LOB columns) 8. Fix various small bugs with table/schema rename in setup wizard and ddl-create 9. Fix various small bugs in set_conflict_handlers command 10. DBRSAPPLY_PKG now has audit information variables (this version thus requires repository uprade) 2.3.08 (16 April 2012) 1. If exp or expdp (using file) is used to load the data, the script now contains the import, too 2. Fix: disappearing list progress in console 3. DDL scripts (created by *-all script) now include primary keys (if CDC is not used) 4. Fixed default path names in setup wizard if target was on different platform (Windows x Un*x) 5. Disable hostname detection if multiple hostnames are found (on RAC, we could autodetect wrong node) 6. Fix for 2-way (and other shared-database) replication setup.dbvrep script 7. Support can enable special dynamic library handling 8. Bundle the mysql libraries 9. Fixed memory leak introduced in 2.3.04 10. LIST CONFLICT command 11. DDL commands get a proper id (shown and used with conflicts) 12. Fixed truncate of long commands at 2kB 13. Do not generate support package for "Could not open port" error (usually shown when dbvrep was already running) 14. Fixes in online PREPARE 15. Fixes in parsing new user being created 16. PREPARE could fail if executed as first command just after console started 17. Let SHOW_CONFLICT_HANDLERS handle properly tables that are prepared on mine only 2.3.06 (9 April 2012) 1. RAC - fixed bugs in RAC degraded mode (when some of the node(s) are down) 2. Setup wizard checks whether the specified tables/schemas exist 3. Variable APPLY_COLUMN_NAMES_MAP can be used to rename columns globally 2.3.04 (30 March 2012) 1. Fix handling of NULL LOBs 2. Self-locked apply session: we try to go on with other sessions (fixes crashes due to UK/FK/PK locking on index duplicates) 3. Self-fixing conflict with incorrect date format 4. RAC - fixed bugs in thread merging (caused mine not to detect log switch) 5. RAC - fixed bugs in log switching (could skip some LCRs near the redolog end) 6. RAC - fixed bugs in thread merging (idle thread could skip some LCRs when becomes non-idle; strand flushes could cause incorrect order of SCNs) 7. RAC - fixed bugs in mine restart (stats and plog) 2.3.02 (16 March 2012) 1. One-to-many and two-way support 2. Filtering/subsetting support 3. CDC/Audit 4. PL/SQL replication 5. Apply threaded to improve performance (Oracle OCI only) 6. Prepare as of SCN 7. Support for AL32UTF16/UTF8 NCHAR/NVARCHAR2 8. Initial data copy now supports DDL including additional CDC/Audit columns if applicable 9. FLOAT datatype support 10. RESOLVE as IGNOREALL 11. PAUSE is now default handling for non-specified cases, e.g. error on COMMIT, DDL etc. 12. SMTP TLS support for email sending 13. Autodetect NLS_LANG in setup wizard and let apply honour it 14. LIST OBSOLETE REDO command 15. Try to skip applying changes that are going to be rolled back 16. Support package tries to use asmcmd copy 17. /tmp/par-*/cache/inc/BUILD_VERSION identifies version that uses this particular cache directory 18. Memory usage considerably lowered 19. Improved support package contents on Windows 20. code cleanup 21. Cluster key move fix 22. Setup wizard defaults hostname to database hostname from v$database 23. Apply OCI statement cache improved 24. Show PL/SQL DDL commands replicated 25. Fixed notification peer down 26. Notification emails now include the image inline 27. Setup wizard supports two-way and one-to-many replication 28. Windows: list progress did not show last apply date 29. On Oracle targets (OCI) sets module/action to show which transaction is being applied (and start SCN) 30. Single-scn instantiation is now default in setup wizard 31. LONG and LONG RAW datatype support 2.2.18 - (16 March 2012) 1. Fixed race condition in mine that could cause incorrect column/datatype in apply SQL 2. DDL CREATE TABLE as SELECT should not be replicated (some syntax sugar cause mine to not detect it is actually a CTAS) 3. Fixed that sometimes plog did not became obsolete and stayed thus on the filesystem (and were reread on apply startup) 4. Fixed empty LOB handling (caused ORA-24801: illegal parameter value in OCI lob function) 5. Fixed 9i query to get current incarnation 6. Redo log parsing 9i fixes 2.2.16 - (6 March 2012) 1. 2.2.14 did not print correctly error message on exit and generated support package even from console errors 2. Handle case when init happened on end of redo log 3. Fix of LWN reorder (private redo strands) 4. Add support package on apply conflict via hidden parameter 5. Fix of notification (complained about "OK" table not being replicated) 6. Fix of set_conflict_handlers (would cause apply to abort when used from console) 7. NLS_LANG can be set in ddc file 8. Fix: last LCR in redo log was sometimes ignored 9. Fix: varchar column was occasionally treated as number 10. Handle resetlogs (ignore logs from past incarnations) 11. Fix: last few LCRs in plog were attributed to next plog 12. Fix: occassional two-task error in propagate thread 13. Prevent apply loop on DDL: set DDL_IGNORE_USER 14. Rollback could be attributed to wrong transaction, resulting in conflict 2.2.14 - (10 February 2012) 1. Automatic invoke of support package on fatal crash 2. Fix of 11gR2 row chaining bug 2.2.12 - (3 February 2012) 1. Fix of missed changes on 11.2 (multiple in-flight row pieces) 2. Enable supplemental logging would fail with syntax error if no tables were found 3. Caching fix, fixing ORA-1036 and 0 rows affected transient errors on apply 4. Fix of start SCN handling on apply that could cause skipped changes in conflicting transactions after apply restart 5. Prepare schema raised syntax error when invoked on empty schema 6. Fix of Oracle missing to include proper row piece information when inserting a migrated row 2.2.10 - (20 January 2012) 1. Unprepare affects datapump script 2. Datapump script race condition with supplemental logging fix (schema level) 3. Fix of RESET variable command 4. Fix of PAUSE/RESUME for process with non-default names 5. Do not rename table to BIN$... in list progress 6. Fix of TIMESTAMP handling in lie of Oracle implicit timezone conversions 7. Fix of decreasing SCN 8. Datapump export/old export ways of data copy 9. Single SCN instantiation support for data instantiation 2.2.08 - (6 January 2012) 1. Datapump script fix - dot in dblink is acceptable character 2. Datapump script race condition with supplemental logging fix (table level) 3. Support package fix for logs in ASM 2.2.06 - (2 January 2012) 1. LOB support 2. Improved startup speed of dbvrep executable 3. RAC support for node shutdown/startup 4. Ability to disable apply loop prevention (support simple cascaded destonations config) 5. Loop&sleep when cannot create new pooled connection to apply database 6. .trc files to collect debug info instead of using .log file; added %I (PID) and %U (unique string) placeholders 7. License type (R*) 8. One-to-many support 9. New setup wizard to support one-to-many 10. Email FATAL errors 2.1.08 - (30 December 2011) 1. Fix of PREPARE command 2. Fix of dictionary update when view is created 2.1.06 - (13 December 2011) 1. Fix of trailling blanks being trimmed in character fields 2. Support for delayed initial target table load (e.g. data pump import) 2.1.04 - (4 December 2011) 1. Handle multiple dictionary changes at same SCN 2. LogPackaging fix with ASM 3. Added time information into trace files 4. Fixed RAC thread syncing when conflicted with LWN 5. New default for MAIL_FROM if the usual default fails validity checks 2.1.02 - (30 November 2011) 1. AIX and SPARC support 2. Apply fix for case when two transactions are committed at the same SCN 3. Data type support extended to RAW, BFILE, TIMESTAMPs, INTERVALs 4. Resolve conflict as ROLLBACK 5. Partitioning 6. Support for non-ordered redo logs/multiple redo streams (11.2 default feature) 7. Crash report informations is written to .trc files 8. New Setup wizard 9. Improved network outage handling 10. Upgrade/downgrade repository 11. ASM instance name autodetection 2.0.18 - (11 November 2011) 1. Networking robustness improvements 2. CPU load lowered 3. Handle apply plog corruption 4. Licensing on date/duration fixed 5. Setup wizard change to status_bar default 2.0.16 - (4 November 2011) 1. NULLs were improperly quoted in non-bind SQL in conflict log 2. LIST APPLY PLOGS fix of SQL 3. Notifications were not enabled even if configured 4. SHUTDOWN command for non-existent process was a fatal error, changed to warning. 5. Ligthweight thread feedback now included in log file 6. Improved recovery after various network issues 7. START process can't be run form interactive console 8. SUPPORT package zip name now includes DDC name 9. Dropped tables are removed from LIST PROGRESS when status is 100% 10. Notifications include sender information (process, server) 11. Setup scrips include full name to dbvrep executable 12. SHOW_CONFLICT_HANDLERS had incorrect help text 13. INSERT conflict was always treated as ABORT 2.0.14 - (14 October 2011) 1. Fix: SCN handling when wrapped 2. Out-of-order mine dictionary changes 3. Improved mine dictionary debugging 2.0.12 - (23 September 2011) 1. START SERVICE command treated as START_SERVICE in order not to confuse Windows users by the mandatory underscore. 2. KTB alignment fix for 10.2.0.5. 3. MINE_STAGING_DIR created if needed on startup (only if fetcher is enabled) 4. Ignored DDL conflict did not change current schema back and apply failed with FATAL-6615 5. Mine now skips DDL DROP "BIN$..." commands. 2.0.10 - (20 September 2011) 1. Autocreation of directories when creating Windows service. 2. New defaults for script and log directories if the current user has no home directory. 3. Basic support for initial data load using Data Pump. 4. Fix for Index-Organized Table overflow segment crash. 2.0.08 - (11 September 2011) Fixes: 1. Autocreation of trace directories fix. 2. Redologs not closed fix. 3. Assembly fix (caused merge of multiple updates to single one). 4. Direct insert compression detection fix. 5. Apply conflict log not included in support package fix. Known limitations: 1. With RAC all nodes must be available, and all the threads must be opened by the instances. 2. No current support for following data types: - LOBs, XMLType. - Table cluster related DDL. - IOTs (index-organized tables). - NCHAR, NVARCHAR, NVARCHAR2. - INTERVAL, TIMESTAMP datatypes. - Object datatypes (includes ADT, VARRAY, nested tables etc.). 3. No support for compression. 4. Currently for RAC 11.2 the _log_parallelism_max = 1 parameter needs to be set to disable redo optimisation. 2.0.06 - (2 September 2011) Oracle 11.2 - block cleanout fixed. Oracle 11.2 - DDL parsing fixed. 2.0.04 - (25 August 2011) Error reporting before process initialization is complete and DDC is loaded. Setup wizard instructions text improved. Added checks for incomplete redo change records. Fixed handling of non-existing DDC file. 2.0.02 - (15 August 2011) New release.