Skip to content

Commit

Permalink
Merge pull request #315 from liquibase/Readme_update
Browse files Browse the repository at this point in the history 
updated Readme file
  • Loading branch information
@AlexanderSashchenko
AlexanderSashchenko authored Aug 8, 2022
2 parents 1e59dab + 516ffad commit cdfd95f
Showing 1 changed file with 51 additions and 36 deletions.
87 changes: 51 additions & 36 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,26 +4,28 @@

## Test-Harness Support Matrix

| Database | Versions Tested|
| ----------- | ----------- |
| Postgres | `9, 9.5, 10, 11, 12, 13, 14` |
| AWS Postgres RDS | `10, 11, 12, 13, 14` |
| MySQL | `5.6, 5.7, 8` |
| MariaDB | `10.2, 10.3 , 10.4, 10.5, 10.6, 10.7` |
| SQL Server | `2017`, `2019` |
| Percona XtraDB | `5.7`, `8.0`|
| Oracle | `18.3.0, 18.4.0, 19.9.0, 21.3.0` |
| AWS Oracle RDS | `19.0` |
| CockroachDB | `20.2, 21.1, 21.2, 22.1` |
| EDB | `9.5, 9.6, 10, 11, 12, 13, 14` |
| DB2 on z/OS | `11.1` |
| H2 | `2.1.210` |
| SQLite | `3.34.0` |
| Apache Derby | `10.14.2.0` |
| Firebird | `3.0, 4.0` |
| HSQLDB | `2.3.4`, `2.5` |
| Snowflake | `6.23.0` |
| Azure SQL DB | `latest` |
| Database | Versions Tested |
|---------------------------|---------------------------------------|
| Postgres | `9, 9.5, 10, 11, 12, 13, 14` |
| AWS Postgres RDS | `10, 11, 12, 13, 14` |
| MySQL | `5.6, 5.7, 8` |
| MariaDB | `10.2, 10.3 , 10.4, 10.5, 10.6, 10.7` |
| SQL Server | `2017`, `2019` |
| Percona XtraDB | `5.7`, `8.0` |
| Oracle | `18.3.0, 18.4.0, 21.3.0` |
| AWS Oracle RDS | `19.0` |
| CockroachDB | `20.2, 21.1, 21.2, 22.1` |
| EDB | `9.5, 9.6, 10, 11, 12, 13, 14` |
| DB2 on z/OS | `11.1` |
| DB2 on Linux/Unix/Windows | `11.5.7` |
| H2 | `2.1.210` |
| SQLite | `3.34.0` |
| Apache Derby | `10.14.2.0` |
| Firebird | `3.0, 4.0` |
| HSQLDB | `2.4, 2.5` |
| Snowflake | `latest` |
| Azure SQL DB | `latest` |
| Azure SQL MI | `latest` |

## Framework

Expand All @@ -48,9 +50,10 @@ The general pattern is that for each directory containing configuration files:

At each level in that hierarchy, new configurations can be added and/or can override configurations from a lower level.

Currently, there are four test types defined in the test harness:
Currently, there are five test types defined in the test harness:
* Change Object Tests
* Change Data Tests
* Snapshot Command Test
* Diff Command Test
* Base level test

Expand Down Expand Up @@ -78,24 +81,24 @@ For more information on using the test harness in your extension, see [README.ex

## Change Objects Test

The test-harness validates most of the Change Types as listed on [Home Page](https://docs.liquibase.com/change-types/home.html).
The test-harness validates most of the Data Definition Language related Change Types as listed on [Home Page](https://docs.liquibase.com/change-types/home.html).
The primary focus is on add, create, drop & rename database objects.

The `groovy/liquibase/harness/ChangeObjectsTests.groovy` test executes changelogs against the database and validates the SQL generated by them as well as
whether they make the expected changes.

* The test behavior is as follows:
* It reads the changesets from the changelogs provided in `src/main/resources/liquibase/harness/change/changelogs` folders (recursively)
* Runs the changeset through the SqlGeneratorFactory to generate SQL
* Compares the generated SQL with the expected SQL (provided in `src/main/resources/liquibase/harness/change/expectedSql`)
* If the SQL generation is correct, the test then runs `liquibase update` to deploy the
changeset to the DB
* The test takes a snapshot of the database after deployment
* The deployed changes are then rolled back by either using `rollbackToDate` (default) or `rollback` by tag (**test-harness-tag**). See `-DrollbackStrategy` option below for more information.
* Finally, the actual DB snapshot is compared to the expected DB snapshot (provided in `src/main/resources/liquibase/harness/change/expectedSnapshot`)
* Runs Liquibase 'updateSql' command to generate query
* Compares generated query with expected query (provided in `src/main/resources/liquibase/harness/change/expectedSql`)
* If the query generation is correct, the test then runs `liquibase update` to deploy the changeset to the DB
* The test takes a snapshot of the database after deployment by running Liquibase 'snapshot' command
* Actual DB snapshot is compared to expected DB snapshot (provided in `src/main/resources/liquibase/harness/change/expectedSnapshot`)
* Finally, deployed changes are then rolled back by either using `rollbackToDate` (default) or `rollback` by tag (**test-harness-tag**). See `-DrollbackStrategy` option below for more information.


#### Types of input files
* The tests work with the 4 types of input files that are supported by liquibase itself - xml, yaml, json, sql.
* The tests work with 4 types of input files that are supported by Liquibase itself - xml, yaml, json, sql.
Thus files with extensions 'xml', 'sql', 'json', 'yml', 'yaml' are taken into account, but not all formats together in the same run.
* The default format is xml, so by default only changelogs with xml file extension are executed.
To change it to another format, like 'sql' for instance, specify `-DinputFormat=sql` as the command line argument for Maven or as VM option to your JUnit test run config.
Expand All @@ -104,13 +107,13 @@ To change it to another format, like 'sql' for instance, specify `-DinputFormat=
### Adding a change object test
1) Go to `src/main/resources/liquibase/harness/change/changelogs` and add the xml (or other) changeset for the change type you
want to test.
- The framework tries to rollback changes after deploying them to DB. If liquibase knows how to do a rollback for that particular changeset, it will automatically do that.
- The framework tries to rollback changes after deploying it to DB. If Liquibase knows how to do a rollback for that particular changeset, it will automatically do that.
If not, you will need to provide the rollback by yourself. To learn more about rollbacks read [Rolling back changesets](https://docs.liquibase.com/workflows/liquibase-community/using-rollback.html) article.
2) Go to `src/main/resources/liquibase/harness/change/expectedSql` and add the expected generated SQL.
2) Go to `src/main/resources/liquibase/harness/change/expectedSql` and add expected query.
- You will need to add this under the database specific folder.
- NOTE: If your changeSet will generate multiple SQL statements, you should add each SQL statement as a separate line. (See `renameTable.sql` in the postgres folder for an example.)
- If you would like to test another DB type, please add the requisite folder.
3) Go to `src/main/resources/liquibase/harness/change/expectedSnapshot` and add the expected DB Snapshot results.
3) Go to `src/main/resources/liquibase/harness/change/expectedSnapshot` and add expected DB Snapshot results.
- To verify the absence of an object in a snapshot (such as with drop* commands) add `"_noMatch": true,` to that tree level where the missing object should be verified.
See [dropSequence.json](src/main/resources/liquibase/harness/change/expectedSnapshot/postgresql/dropSequence.json) as an example.
Additionally the `_noMatchField` parameter can be used to define the exact property which should be absent or different for that particular database object (for example Column, Table etc.)
Expand All @@ -124,13 +127,13 @@ see [createTableWithNumericColumn.json](src/main/resources/liquibase/harness/cha
The primary goal of these tests is to validate change types related to DML (Data Manipulation Language) aspect.
Generally it is similar to ChangeObjectTests except it doesn't use Liquibase snapshot to verify data but obtains result set via JDBC.
- `src/main/resources/liquibase/harness/data/changelogs` - add DML related changelogs here;
- `src/main/resources/liquibase/harness/data/checkingSql` - add select query which will obtain a result set from required table;
- `src/main/resources/liquibase/harness/data/expectedResultSet` - add JSON formatted expected result set from required table
- `src/main/resources/liquibase/harness/data/checkingSql` - add select query which will obtain a result set from required DB object;
- `src/main/resources/liquibase/harness/data/expectedResultSet` - add JSON formatted expected result set from required DB object;
where left part of a JSON node is the name of a change type and right part is JSON Array with a result set;
- `src/main/resources/liquibase/harness/data/expectedSql` - add query which is expected to be generated by Liquibase;
To run the test go to your IDE and run the test class `ChangeDataTests.groovy`.

## DiffCommandTest

This test executes the following steps:
* Reads `src/test/resources/harness-config.yml` and `src/main/resources/liquibase/harness/diff/diffDatabases.yml` to locate the
databases that need to be compared
Expand All @@ -142,6 +145,18 @@ This test executes the following steps:

#### Warning: This is a destructive test -- it will alter the state of targetDatabase to match the referenceDatabase.

## SnapshotCommandTests

This test validates work of Liquibase 'snapshot' command by comparing expected and generated snapshots
after a DB object was created.

## BaseLevelTest

This test validates work of basic Liquibase logic. It runs simple 'create table' sql-formatted query using Liquibase 'update' command and
checks whether databasechangelog and databasechangeloglock tables were created and filled with correct data.



## Running the Tests

### Minimum Requirements
Expand Down

0 comments on commit cdfd95f

Please sign in to comment.