Skip to content

Commit

Permalink
Support Sqlalchemy 1.4 part 2
Browse files Browse the repository at this point in the history
* Disable unimplemented sql_exprssion_limit_offset tests
* Adjust function call to comply with latest api changes
* Disable tests requiring an explicit index which isn't supported by exasol
* Make sure metadata queries do not create a second connection
* Fix args for report command
* Make sure test schemas are in place before integration tests get executed
* Fix setup of test cases
* Temporarily add migration notes and todos to PR/changes
* Mark test depending on driver update as xfail
    see also #232
* run sphinx via poetry in nox task
* Update dependencies
* Suppress failing turbodbc test
* Disable incompatible unit tests
* Disable merge tests
  • Loading branch information
Nicoretti committed Nov 30, 2022
1 parent b0044d3 commit 08aa8c0
Show file tree
Hide file tree
Showing 15 changed files with 874 additions and 174 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/ci-cd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ jobs:
ITDE_TAG: "0.11.0"

- name: Run Test for Python ${{ matrix.python }} using ${{ matrix.connector }}
run: poetry run nox -s integration-tests -- --connector ${{ matrix.connector }} --db-version ${{ matrix.exasol_version }}
run: poetry run nox -s integration-tests -- -- --connector ${{ matrix.connector }} --db-version ${{ matrix.exasol_version }}

upload_to_pypi:
runs-on: ubuntu-latest
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ jobs:
ITDE_TAG: "0.11.0"

- name: Run Test for Python ${{ matrix.python }} using ${{ matrix.connector }}
run: poetry run nox -s integration-tests -- --connector ${{ matrix.connector }} --db-version ${{ matrix.exasol_version }}
run: poetry run nox -s integration-tests -- -- --connector ${{ matrix.connector }} --db-version ${{ matrix.exasol_version }}

build_package:
runs-on: ubuntu-latest
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/gh-pages.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:
poetry run python -m nox -s build-docs
touch doc/build/.nojekyll
- name: Deploy
uses: JamesIves/github-pages-deploy-action@v4.3.0
uses: JamesIves/github-pages-deploy-action@v4.4.1
with:
branch: gh-pages
folder: ./doc/build
Expand Down
49 changes: 49 additions & 0 deletions Migration-Notes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Current Status

```
# Status Update
## Test(s) Status
### sqlalchemy
* pyodbc 3 :no_entry_sign: failed, 190 :heavy_check_mark: passed , 359 :yellow_circle: skipped, 0 :boom: errors
* 1 test is caused by a potentially by an escaping issue
* 2 failing tests are caused by incorrectly returned rowcounts
* turbodbc 6 :no_entry_sign: failed, 185 :heavy_check_mark: passed , 361 :yellow_circle: skipped, 0 :boom: errors
* 1 test is caused by a potentially by an escaping issue
* 5 failing tests are caused by incorrectly returned rowcounts
### exasol
* pyodbc 9 :no_entry_sign: failed, 202 :heavy_check_mark: passed , 10 :yellow_circle: skipped, 0 :boom: errors
* All the failing 9 tests, are caused by the custom merge statement.
* turbodbc 9 :no_entry_sign: failed, 189 :heavy_check_mark: passed , 23 :yellow_circle: skipped, 0 :boom: errors
* All the failing 9 tests, are caused by the custom merge statement.
## Next Steps
* Investigate the escaping issue (DifficultParametersTest_*::test_round_trip[q?marks])
* Investigate the invalid returned row counts
* Refactor/Implement Merge statement in respect to new sqla syntax/api
```

# TODO's:

* TODO: Investigate/Fix the escaping issue (DifficultParametersTest_*::test_round_trip[q?marks])
* TODO: Investigate/Fix the invalid returned row counts
* TODO: Refactor/Implement custom merge statement based on new sqla api
* TODO: Move DB setup/preperation from noxfile to conftest.py

# Post Migration TODO's (tickets)

* TODO: Create ticket to Update to ITDE to 1.1.0
* TODO: Create ticket for migrating exasol tests to non sqla based fixture setup (e.g. pyexasol)
* TODO: Create ticket to additional details to developer guide
# Architecture
* https://www.aosabook.org/en/sqlalchemy.html
# Custom SQL Constructs
* https://docs.sqlalchemy.org/en/14/core/compiler.html
# Other dialects with custom commands
* https://github.com/SAP/sqlalchemy-hana/blob/master/sqlalchemy_hana/dialect.py
* https://github.com/snowflakedb/snowflake-sqlalchemy
* https://github.com/clach04/ingres_sa_dialect
* https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/blob/main/sqlalchemy_redshift/ddl.py
* https://github.com/IBM/nzalchemy/blob/master/sqlalchemy-netezza/nzalchemy/base.py
8 changes: 8 additions & 0 deletions doc/_static/data/features.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
sqla requirement,test case,supported by exasol db,supported by exasol sqla dialect,test status,comment
sql_expression_limit_offset,FetchLimitOffsetTest*.test_expr_limit,yes,no,skipped,not implemented
sql_expression_limit_offset,FetchLimitOffsetTest*.test_expr_limit_offset,yes,no,skipped,not implemented
sql_expression_limit_offset,FetchLimitOffsetTest*.test_expr_limit_simple_offset,yes,no,skipped,not implemented
sql_expression_limit_offset,FetchLimitOffsetTest*.test_expr_offset,partially,no,skipped,not implemented
sql_expression_limit_offset,FetchLimitOffsetTest*.test_simple_limit_expr_offset,yes,no,skipped,not implemented
,HasIndexTest*.*,no,no,skipped,not supported
,ComponentReflectionTest*.test_get_indexes*,no,no,skipped,not supported
15 changes: 12 additions & 3 deletions noxfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -178,8 +178,9 @@ def populate() -> None:
transaction(
connection,
(
"CREATE SCHEMA TEST_SCHEMA;",
# "CREATE SCHEMA TEST_SCHEMA_2;",
"CREATE SCHEMA IF NOT EXISTS TEST",
"CREATE SCHEMA IF NOT EXISTS TEST_SCHEMA;",
"CREATE SCHEMA IF NOT EXISTS TEST_SCHEMA_2;",
),
)
connection.close()
Expand Down Expand Up @@ -318,7 +319,15 @@ def clean(session: Session) -> None:
def build(session: Session) -> None:
"""Build the documentation"""
session.run(
"sphinx-build", "-b", "html", "-W", f"{DOC}", f"{DOC_BUILD}", external=True
"poetry",
"run",
"sphinx-build",
"-b",
"html",
"-W",
f"{DOC}",
f"{DOC_BUILD}",
external=True,
)


Expand Down
Loading

0 comments on commit 08aa8c0

Please sign in to comment.