The release notes of the last release may be found on README.md.
- Code editors occasionally loosing changes when saving
- Dependency searchbar closing randomly
- Replacing code using the
Search/Replace
tab does not update the code in the editor without refreshing
- Format overly long migration effort estimates on the migration list
Certificate could not be obtained: no SSL error reported
when using SSL for the PostgreSQL connection
- Migration List:
- Structure stage:
- Detect Index Organized Tables and automatically exclude their overflow tables
- Data stage:
- Show total bytes and bytes per seconds for a finished transfer
- Indicate whether a child process died from a signal, such as from the out-of-memory killer
- Gracefully handle
INT
signals to hasten abort request
- Migration Settings:
- Changed data progress interval unit from milliseconds to seconds
- Migration passwords are now encrypted within the database
- Code editors occasionally loose changes while typing
- Renaming a table column does not display the new name in the constraint tab
- PostgreSQL reserved identifiers are not read if the amount of CPU cores can not be retrieved
- Replacing trigger code using the
Search
tab does not update the code in the editor without refreshing ORA-01652: unable to grow temp segment in tablespace TEMP by 1MB during operation
when reading indexes from some Oracle databases- Enabling the diff mode on a view-editor causes a page crash
DEFAULT is not allowed in this context
when migrating list partitions with a non-uppercasedefault
statement
- Upon opening the Trial edition, the user must acknowledge the usage restrictions prior to using the application.
- Package
curl
andpsql
with thecore
image for troubleshooting purposes
- Package
ping
,telnet
andtraceroute
with thecore
image for troubleshooting purposes
unable to connect to internal database: Error: Entity metadata for Schema#packages was not found. Check if you specified a correct entity object and if it's connected in the connection options.
Fresh installation fails during the database setup.
-
Automatic Code Transpilation: Receive suggestions from the CYBERTEC Migrator when rewriting functions, procedures, triggers and views.
For more information, see Transpiler Features.
Beginning with this release, the Docker images include the edition in their name (e.g. cybertecpostgresql/cybertec_migrator-professional-core
).
Furthermore, the Standard
edition has been renamed to Trial
; its features have been adapted accordingly.
For users operating the installer: Execute ./migrator configure --edition <edition>
.
Users operating on Kubernetes or OpenShift: Please modify your configurations accordingly.
-
Dependency Control: Manually manage object dependencies to precisely control the order of database object creation
-
Migration Creation: Enhanced error handling now allows you to identify and address issues, at the same time allowing the creation process to continue
-
Bulk Migration Creation: Provide expected JSON schema and downloadable template
Error during data transfer: Failed to connect to database: could not translate host name "@localhost" to address: Name or service not known
Data stage fails if the PostgreSQL username, password or database contain special characters such as@
- Creating a migration for Oracle 23 fails during the Analyze step
- Drop Docker Compose V1 support (EOL June 2023) in favour of Compose V2
- Creating and cloning a migration may become unresponsive during schema selection
-
Migration overview: View the assessed migration effort for each migration alongside performance improvements
-
Bulk migration creation: Rapidly create a multitude of migrations
Example JSON:
{ "migrations": [ { "name": "Migration", "description": "An optional description", "source": { "connection_string": "oracle://localhost:1521/pdb1", "username": "system", "password": "oracle" }, "target": { "connection_string": "postgresql://localhost:5432/postgres", "username": "postgres", "password": "postgres" }, "schema_selection": [ "TOOLS", "ONLINE_STORE" ] } ] }
-
Data transfer prioritization: Customize the order in which tables are transferred
-
Custom SCN: Force the migrator to use a fixed SCN for the data stage
- Removed upper limit when generating table chunks
- Removed deprecated C data migrator (and the correlating
CORE_DATA_MIGRATOR_USE_RUST
environment variable).
- Resuming data stage after chunking a table would truncate other chunked tables
-
Table chunking: Speed up your data transfers through parallelization
At present, chunking is limited to tables with
- a primary/unique key or index
- over exactly one column
- of a numeric type
Using a key/index, the migrator will create approximate chunks with it's
min
andmax
values. -
View triggers: Show, edit and migrate view triggers
-
Improved migration creation: Enhanced keyboard navigation and introduced
Go Back
button -
Sidebar: Show or hide excluded DBOs
- Importing a migration archive on Windows causes
File must be a .zip archive
- Switching between procedure editors causes their contents to be overwritten
-
Live Data stage overview: Determine the migration status of each table at a glance
-
Import migration: Create a new migration using a migration export, as an alternative to cloning
Note: Only migration exports created with v3.14.0 and later are supported.
-
Log Panel Timezone: Choose a custom timezone for migration log timestamps
-
Sequence Editor: View and edit the details of sequences
-
Rename table columns:
-
Migration Settings:
- Regression: Can't create table and column comments containing single quotes
- Tables containing columns named after Oracle-only reserved identifiers (e.g.
SIZE
) fail during the data stage
-
Migration Assessment:
- Quickly determine how much effort an individual migration encompasses
- Inspect the cost for discrete database objects and identify outliers
Note: The figures given by the assessment are only a rough estimation.
We will continue to tweak and improve these values as we collect experience and introduce new technologies, such as a PL/SQL parser. -
Migration Overview: List empty and non-empty schemas separately
Note: Empty schemas are now excluded per default
-
Migration Creation: Improve feedback for invalid connection strings
-
Migration Lifecycle:
- Defer the creation of functions after those of tables to allow for the use of the
%TYPE
attribute - Enhance the migration log of the structure, integrity and logic stages to be on a par with those of the data stage
- Configure the
COPY FREEZE
functionality introduced inv3.12.0
for each individual table instead of the whole system. TheCORE_DATA_MIGRATOR_USE_COPY_FREEZE
environment variable has subsequently been removed.
- Defer the creation of functions after those of tables to allow for the use of the
-
Miscellaneous: Rework the log output format and fidelity throughout the
core
container
- The Analyze step of the migration creation always shows
Jobs
as pending - LOBs exceeding the maximum size of 500MB do not fail with an appropriate error message
- Setting the sidebar filter to
Index
does not include all indexes in the filter result - Usernames and passwords containing special characters cause the target connection check during migration creation to fail
- Deleting a migration after executing a stage fails with an unexpected error
-
Enhance migration log in the data stage:
-
Improve summary of the start and end logs
resumed data stage from oracle://localhost:1521/pdb1 to postgresql://localhost:5432/postgres: using 8 workers excluded tables: 1 successful transfers: 1 (out of 7) remaining transfers: 6 ... failed executing data stage: stage run-time 00:01:534 excluded tables: 1 successful transfers: 6 (out of 7) remaining transfers: 1 failed transfers: 1
-
Print a warning if the source and/or target connection are not encrypted
Info Data using secure connection to read data from oracle://pdb_sec [oracle://10.0.0.127:2484/pdb?protocol=tcps] Warning Data using insecure connection to write data to postgresql://localhost:5432/postgres
-
Periodically log
data-migrator
transfer statistics for each table. This interval may be set using theCORE_DATA_MIGRATOR_PROGRESS_INTERVAL
environment variable (default:600000
milliseconds,10
minutes)Verbose Data loading data for table:"HR"."EMPLOYEES" [hr.employees]: processed 982 rows in 01:20.000 - 45 rows/sec (total: 41 rows/sec in 05:00.000) Verbose Data finished loading data for table:"HR"."EMPLOYEES" [hr.employees]: processed 1012 rows in 00:02:01 - 40 rows/sec
-
-
Quote reserved identifiers when generating data queries for both Oracle and PostgreSQL
-
Add support for secured communication (TCPS) access to Oracle databases without using a net service name. For details, check out our FAQ - How do I configure TCPS for Oracle databases section
-
Configure the
FREEZE
parameter during the data transfer by setting theCORE_DATA_MIGRATOR_USE_COPY_FREEZE
environment variable (default:false
). Only use this setting if you are aware of its implications
- An error on the target connection during the structure, integrity or logic stage may cause the Migrator core to crash
- Data stage reported as successful for certain corner cases when transfer failed with
No space left on device
- Connectivity check to PostgreSQL with connection parameters fails when port is omitted (
FATAL: database does not exist
) - Failed to read data from Oracle for tables with lowercase identifiers
- Regression: can not provide hostname with '-' in connection string
-
Add support for Oracle 11 (closes #14).
-
We changed the default configuration of the reverse proxy to use HTTPS.
This means you have to install a TLS/SSL certificate or create a self signed certificate, otherwise the web-server will refuse to start. To facilitate the installation of the certificate we added a new option--tls
to themigrator configure
command. Check out the FAQ for detailed information how to install a TLS/SSL certificate.migrator configure --tls self-signed-cert Generate self-signed TLS/SSL certificate migrator configure --tls cert:<file-location> Install TLS/SSL certificate migrator configure --tls key:<file-location> Install private key of TLS/SSL certificate
Attention
If you upgrade from a previous Migrator version you have to create the TLS/SSL certificate after upgrading to the new version, before restarting the new Migrator../migrator update ./migrator upgrade # Don't forget the create or install a TLS/SSL certificate ./migrator configure --tls self-signed-cert ./migrator up
-
Add support for secured communication (TCPS) access to Oracle databases. For details check our FAQ - How do I configure TCPS for Oracle databases?.
-
Read meta-data of additional Oracle database object types: jobs and operators
-
Users may provide a custom Data Query, thus manipulating the
A few use cases where this feature may be useful:SELECT
statement executed on the source database to migrate the data.- Migrate the table partially by filtering rows with
WHERE
- Migrate only the table structure but no data (
WHERE ROWID < 0
) - You want to keep the table structure, but not the data of a specific column (see screenshot above)
- Retrieve (and thus insert) the data in a specific
ORDER BY
- Convert the column data of an unsupported data type into a string representation which can be βcastβ into a valid PostgreSQL type via the COPY statement
- Migrate the table partially by filtering rows with
-
Improve output for erroneous SQL execution when reading Oracle meta-data.
- Columns starting with
SYS
are not included in primary keys or unique constraints - Data stage fails when column is named after an SQL reserved keywords (for example
IN
)
- Creating a migration for Oracle 11 and below fails on reading out sequences due to a non-existent table
- The
lower_bound
of range subpartitions is not determined correctly during migration creation - Running the Structure stage with
REFERENCE
partitions results in a non-descriptive error - Regression in v3.10.0 causing the data of individual partitions to not be migrated correctly
- Code editors mark their complete content as erroneous instead of only the faulty section
- Revert style changes in the search and replace panel
- Improve Search Panel:
- Show information for Oracle Queues in the Sidebar
- DBO creation fails due to global
lock_timeout
setting on target database - Running
ps
inside the core container during the data stage reveals the database passwords - Abort of data stage may sometimes hang indefinitely
- Improve Sidebar:
- Migration Overview: Additional index types on Indices drill down
- Switch Migrator core to the native
libpq
PostgreSQL driver to support additional authentication methods (e.g. GSSAPI) - Recurring security maintenance: upgrade package dependencies with known vulnerabilities to newest version
- PostgreSQL authentication with AuthenticationGSS causes a crash
- Importing an Oracle View with a Null-Byte results in an unresponsive state
- Unexpected shutdown of the target PostgreSQL connection may cause a crash
- Configure the maximal number of parallel data transfers and index creation workers for each migration.
The Migrator picks reasonable default values when a new migration is created:Data Tranfers
: number of CPUs where the Migrator is hostedDatabase Workers
: number of CPUs of the target database. Make sure to tune this parameter with the PostgreSQL configuration of max_worker_processes, max_parallel_workers, max_parallel_maintenance_workers
- Enhance sidebar: show synonym meta-data as pop-up
- Set/clear object type filter by selecting an
Object Type
in the overview tab. Use theCTRL
key to filter for more than one database object type. - Add issue templates for asking questions.
- Empty connection check error message on
504 Gateway Time-Out
- Broken GUI workflow in the Analyze step on
504 Gateway Time-Out
- Error on migrating
REVERSE
indexes: PostgreSQL does not have/need an equivalent feature - Opening a non-existent migration endlessly shows the loading animation
- Error on migrating sequences with negative
INCREMENT
-START value cannot be greater than MAXVALUE
- Migration controls (resume, continue, etc.) occasionally do not behave as expected
- Views depending on other views are not created in the correct order
- Spacing between the migration log time and level is jumping
- Incorrect name conflict detection for tables and triggers with the same identifier
Do you want to know if the Migrator can migrate your Oracle database to PostgreSQL?
Then get the Migrator Trial Edition, a free version (as in beer) of the CYBERTEC Migrator, follow the offline instructions provided in Getting Started section, and try it out.
- Improve migration job execution (which removed the Redis job queue as a dependency)
- Provide help menu to reach out to CYBERTEC
- Provide a Migrator demo database environment to facilitate a test-run of the Migrator Trial Edition
- Added License information
- Error when attempting to edit a function, procedure, trigger or view containing a
#
in its name - Trigger Type and Level can be changed even if the trigger is excluded
- Column data-types qualified with
SYS
are not translated properly
- Add support for stage post-hook SQL scripts to adapt the migration with functionalities not provided by the Migrator. The scripts are executed at the end of a stage. Typical use cases for such scripts are to create database objects which are not present in the source database, or to define ownership and access permissions.
- Improve Data stage: list and range partitions and sub-partitions of partitioned tables are migrated in parallel.
- Support migration of
GENERATED AS IDENTITY
constraint. - Improve user experience:
- Failed to created migration of an Oracle database containing a column that uses a data-type from the
SYS
schema - Starting a migration job with more than one stage and incorrect target connection bricks the migration
- Sidebar filter showing schemas that contain none of the filter results
- Misleading console log entries
No metadata found. There is more than once class-validator version installed probably ...
- Several improvements in the execution of a migration:
- Swap execution order of Integrity and Logic stage. Having indices in the Logic stage makes it easier to test performance of views, stored procedures and triggers.
- Move creation of check constraints from Logic into Structure stage. Checking the data during the data bulk load is negligible compared to the time needed to re-read the table from disk in the Logic stage.
- As a consequence, functions are created in the Structure stage prior to tables, since they may be used by check constraints. The Logic stage re-creates the functions once again still providing the means for fast change-test round-trips.
- The Migration Overview was updated to reflect the changes in the migration execution:
- Enhance migration configuration:
- Improve handling of implicitly created indices via Unique Constraints or Primary Keys:
- Log view shows detailed information about the started migration job:
- Improve Sidebar:
- Object Type Filter: add option for User Defined Types (UDT).
- Cloning a migration fails due to missing database object in the source database (dropped or renamed table, dropped column, etc.)
- Integrity stage
ERROR: timeout exceeded when trying to connect
- Integrity stage keeps processing workers after an error even with enabled "Abort stage on first error"
- Resume of an aborted job fails when the Migrator core was forcefully shut down
- Core dump due to repeated, large error message (
warn: Unable to process subpartition ... table not found
) - Functional index with an expression containing a number was not migrated
- Overview page does now show 0 database links, synonyms, packages
This is a bugfix release for v3.4.0.
Note: v3.4.1 and v3.4.2 were not released.
- Foreign keys are not created in parallel
- Import fails for databases with a view on a view
- Import fails for databases with a partition of a non-imported table (i.e. temporary, secondary, nested, or dropped table)
Not released.
Not released.
The main focus of this release is improving the performance for reading the source database meta-data (we tested databases with approximately 400.000 database objects). This includes improvements to the GUI to be responsive with a large number of database objects.
NOTE
After starting the migrator the dashboard may show an error until the Migrator converted existing migrations in its internal database. Depending on the number and size of existing migrations this may take some time.
- Reading and analyzing the source database meta-data is now faster
- Overview page: drill down to show specific data types
- Improve Sidebar:
- Show number of database objects
- Add new
Constraints
Object Type filter (Check
,Foreign Keys
, andUnique/Primary Keys
) - Add
Disable Partitioning
for multiple tables
- Improve Search and Replace
- Include index names
- Scope search with sidebar filter
- Highlight element in the table editor when selected in the sidebar or when navigated from the log view
- Log View:
- Scope log entries based on job execution:
ALL
,CURRENT MIGRATION
, andLAST EXECUTED JOB
- Download content of log view
- Scope log entries based on job execution:
- Creating a migration of a database with a large number of DBOs fails
- Unable to save large configuration changes
Partitions
tab not responsive for a table with large number of partitions (>500)- Resuming data stage results in migrating the data from a different SCN
- Data stage was not aborted properly by user request
- Show information of additional Oracle database object types
- Database links
- Packages (specification and body)
- Synonyms
- Show procedures and functions as separate types (until now both types were shown under
Functions
) - Improve data transfer for Oracle
BLOB, CLOB, NCLOB
andBFILE
columns for rows with big LOBs. This may fix out-of-memory errors on the Oracle side, namelyMSG: ORA-01062: unable to allocate memory for define buffer
. It also significantly reduces the Migrator memory footprint. - The transfer of a row containing a LOB bigger than 500MB will abort with an informative error message.
- Creating a migration of a database with invisible columns causes a failure during meta-data import
(
null value in column "position" of relation "table_column" violates not-null constraint
)
- Enhanced migration configuration:
- Index Renaming
Rename indices in case there are naming collisions with other database objects - Primary/Unique Key Modification
Add, remove and change the order of table columns in a primary/unique key
- Index Renaming
- Browser Tab Syncing
Work on the same migration in multiple browser tabs, with them being synced in real-time - Search and Replace on check constraints and expression indexes
- Usability Improvements
- Improved sidebar functionalities:
- Object Type filter: add options for indices, triggers, and sequences
- Bulk exclude of schemata and tables
- The tabs in the table editor (Columns, Constraints, etc.) display the identifiers used in the target database
- Provide examples for source and target connection strings when creating new migrations
- Performance improvements when running migration jobs
- Provide more concise error details when writing database objects to the target database (e.g. miss-matching foreign key data-types)
- Improved sidebar functionalities:
- Creating a migration of a database with a view containing numerous columns causes a failure during meta-data import (
ORA-40478: output value too large (maximum: 4000)
) - Creating a migration with an unreachable host shows a non-descriptive error
- Replace All on different database object types sometimes doesn't replace all search results
-
Search and Replace
Perform bulk changes on column data types / default values, functions, views, and triggers (with support for even more object properties coming soon) -
Significant Speed Improvements
Feel the difference -
Resume Failed Stages
Spend less time fixing those last few migration errors -
Realtime Syntax Checking
Get immediate feedback about the syntactical correctness of code -
Partition Renaming
Rename partitions as part of a migration instead of altering the source database -
Code Export
Export code of function / views / triggers (with code import coming soon) -
Migration Deletion
Delete of completed migrations -
Migration Cloning
Prepare migrations against your staging environment and then execute them against production
- Keys for excluded columns are not excluded automatically
- Column aliases in view code are not migrated
- Migration status is sometimes displayed incorrectly
- Excluding partition columns via the sidebar prevents the Structure stage from succeeding
With CYBERTEC Migrator v3 we've rebuilt the GUI from the ground up to simplify database migrations even further
- Get a brief look at the structure of your source database by using the migration Overview
- Quickly drill down into your data model with the help of the new Sidebar
- Stages (natural synchronization points of a migration process) now guide you through the migration process
- Structure stage: create database objects necessary to migrate the table data
- Data stage: migrate table data in parallel
- Logic stage: create functions, triggers, views
- Integrity stage: parallel creation of primary keys, indices, foreign keys, and constraint checks
- Enjoy quick round trips and gain confidence in the correctness of your migration thanks to the reworked Migration Controls
- Start migration from the beginning
- Rerun stage in case of an error
- Continue to next stage
- Abort migration and restart at any time
- Explore the inner workings of your migration with the new, and easily accessible Log View
- Log entries are inter-linked to the database objects configuration
- Filter on log level (
ERROR
,WARNING
,INFO
,VERBOSE
)
- Extensive configuration
- Exclude database objects (schemas, tables, columns, indexes, ...) from migration
- Table editor to configure columns, constraints, indices, triggers, and partitions
- Bulk change of data types (for example change all
NUMBER(4)
toint
, instead ofsmallint
) - Integrated code editor for functions, stored procedures, and views with diff feature