#37 Added the pytest-backend project (#38)

* #37 Added the pytest-backend project

* #37 Added the pytest-backend project

* #37 Added the pytest-backend project

* #37 Running only itde tests

* #37 Restored pytest-itde as it was

* #37 Improved readability of parameterized test

* #37 Made all fixtures the session level

* Update .gitattributes

* #37 Changed dependencies on other plugins to released versions

* Update justfile

* #37 referencing pytest-exasol-saas 0.2.2

* #37 Added a guard for a repeated call of a session fixture

---------

Co-authored-by: Christoph Kuhnke <[email protected]>

.gitattributes

@@ -0,0 +1 @@
+*/poetry.lock  linguist-generated

README.md

@@ -14,10 +14,11 @@ Whether you're looking to use database interactions, enhance test reporting, or
 
 ## Plugins
 
-| Plugin               | Description                                                                                                                | PYPI                                                               |
-|----------------------|----------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------|
-| `pytest-exasol-itde` | Fixture to enable simple usage with Exasol's project [ITDE](https://github.com/exasol/integration-test-docker-environment) | [pytest-exasol-itde](https://pypi.org/project/pytest-exasol-itde/) |
-| `pytest-exasol-saas` | Fixture to enable simple usage with Exasol's project [saas-api-python](https://github.com/exasol/saas-api-python/)         | [pytest-exasol-saas](https://pypi.org/project/pytest-exasol-saas/) |
+| Plugin                  | Description                                                                                                                | PYPI                                                                     |
+|-------------------------|----------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------|
+| `pytest-exasol-itde`    | Fixture to enable simple usage with Exasol's project [ITDE](https://github.com/exasol/integration-test-docker-environment) | [pytest-exasol-itde](https://pypi.org/project/pytest-exasol-itde/)       |
+| `pytest-exasol-saas`    | Fixture to enable simple usage with Exasol's project [saas-api-python](https://github.com/exasol/saas-api-python/)         | [pytest-exasol-saas](https://pypi.org/project/pytest-exasol-saas/)       |
+| `pytest-exasol-backend` | Fixture aggregating functionality of both of the above plugins                                                             | [pytest-exasol-backend](https://pypi.org/project/pytest-exasol-backend/) |
 
 
 ## Installation

justfile

@@ -1,4 +1,4 @@
-PROJECTS := "pytest-saas pytest-itde"
+PROJECTS := "pytest-backend pytest-saas pytest-itde"
 
 # Default target
 default:

pytest-backend/.gitignore

@@ -0,0 +1 @@
+.html-documentation

pytest-backend/README.md

@@ -0,0 +1,69 @@
+# pytest-exasol-backend Plugin
+
+The `pytest-exasol-backend` plugin is a collection of pytest fixtures commonly used for testing 
+projects related to Exasol. In particular, it provides unified access to both Exasol On-Prem and
+SaaS backends. This eliminates the need to build different sets of tests for different backends.
+
+## Features
+
+* Provides session level fixtures that can be turned into connection factories for the database and the BucketFS.
+* Automatically makes the tests running on the selected backends.
+* Allows selecting either or both backends from the CLI that executes the pytest.
+
+## Installation
+
+The pytest-exasol-saas plugin can be installed using pip:
+
+```shell
+pip install pytest-exasol-backend
+```
+
+## Usage in Tests
+
+Below is an example of a test that requires access to the database. Note, that by default
+this test will run twice - once for each backend.
+
+```python
+import pyexasol
+
+def test_number_of_rows_in_my_table(backend_aware_database_params):
+    with pyexasol.connect(**backend_aware_database_params, schema='MY_SCHEMA') as conn:
+        num_of_rows = conn.execute('SELECT COUNT(*) FROM MY_TABLE;').fetchval()
+        assert num_of_rows == 5
+```
+
+Here is an example of a test that requires access to the BucketFS. Again, this test will
+run for each backend, unless one of them is disabled in the CLI.
+
+```python
+import exasol.bucketfs as bfs
+
+def test_my_file_exists(backend_aware_bucketfs_params):
+    my_bfs_dir = bfs.path.build_path(**backend_aware_bucketfs_params, path='MY_BFS_PATH')
+    my_bfs_file = my_bfs_dir / 'my_file.dat' 
+    assert my_bfs_file.exists()
+```
+
+Sometimes it may be necessary to know which backend the test is running with. In such
+a case the `backend` fixture can be used, as in the example below.
+
+```python
+def test_something_backend_sensitive(backend):
+    if backend == 'onprem':
+        # Do something special for the On-Prem database.
+        pass
+    elif backend == 'saas':
+        # Do something special for the SaaS database.
+        pass
+    else:
+        raise RuntimeError(f'Unknown backend {backend}')
+```
+
+# Selecting Backends in CLI
+
+By default, both backends are selected for testing. To run the tests on one backed only, 
+the `--backend` option can be used. The command below runs the tests on an on-prem database.
+
+```shell
+pytest --backend=onprem my_test_suite.py
+```

pytest-backend/doc/changes/changelog.md

@@ -0,0 +1,12 @@
+# Changes
+
+* [unreleased](unreleased.md)
+
+<!--- This MyST Parser Sphinx directive is necessary to keep Sphinx happy. We need list here all release letters again, because release droid and other scripts assume Markdown --->
+```{toctree}
+---
+hidden:
+---
+unreleased
+
+```

pytest-backend/doc/changes/unreleased.md

@@ -0,0 +1,5 @@
+# Unreleased
+
+## Feature
+
+* #37: Added the pytest-backend project

pytest-backend/exasol/pytest_backend/__init__.py

@@ -0,0 +1,230 @@
+from __future__ import annotations
+from typing import Any
+from datetime import timedelta
+from contextlib import ExitStack
+import ssl
+from urllib.parse import urlparse
+import pytest
+
+from exasol_integration_test_docker_environment.lib import api
+from exasol.saas.client.api_access import (
+    OpenApiAccess,
+    create_saas_client,
+    get_connection_params
+)
+
+_BACKEND_OPTION = '--backend'
+_BACKEND_ONPREM = 'onprem'
+_BACKEND_SAAS = 'saas'
+
+_onprem_stash_key = pytest.StashKey[bool]()
+_saas_stash_key = pytest.StashKey[bool]()
+
+
+def pytest_addoption(parser):
+    parser.addoption(
+        _BACKEND_OPTION,
+        action="append",
+        default=[],
+        help=f"""List of test backends (onprem, saas). By default, the tests will be
+            run on both backends. To select only one of the backends add the
+            argument {_BACKEND_OPTION}=<name-of-the-backend> to the command line. Both
+            backends can be selected like ... {_BACKEND_OPTION}=onprem {_BACKEND_OPTION}=saas,
+            but this is the same as the default.
+            """,
+    )
+
+
+@pytest.fixture(scope='session', params=[_BACKEND_ONPREM, _BACKEND_SAAS])
+def backend(request) -> str:
+    backend_options = request.config.getoption(_BACKEND_OPTION)
+    if backend_options and (request.param not in backend_options):
+        pytest.skip()
+    return request.param
+
+
+def _is_backend_selected(request, backend: str) -> bool:
+    backend_options = request.config.getoption(_BACKEND_OPTION)
+    if backend_options:
+        return backend in backend_options
+    else:
+        return True
+
+
+@pytest.fixture(scope='session')
+def use_onprem(request) -> bool:
+    return _is_backend_selected(request, _BACKEND_ONPREM)
+
+
+@pytest.fixture(scope='session')
+def use_saas(request) -> bool:
+    return _is_backend_selected(request, _BACKEND_SAAS)
+
+
+@pytest.fixture(scope="session")
+def backend_aware_onprem_database(request,
+                                  use_onprem,
+                                  itde_config,
+                                  exasol_config,
+                                  bucketfs_config,
+                                  ssh_config,
+                                  database_name) -> None:
+    if use_onprem and (itde_config.db_version != "external"):
+        # Guard against a potential issue with repeated call of a parameterised fixture
+        if _onprem_stash_key in request.session.stash:
+            raise RuntimeError(('Repeated call of the session level fixture '
+                                'backend_aware_onprem_database'))
+        request.session.stash[_onprem_stash_key] = True
+
+        bucketfs_url = urlparse(bucketfs_config.url)
+        _, cleanup_function = api.spawn_test_environment(
+            environment_name=database_name,
+            database_port_forward=exasol_config.port,
+            bucketfs_port_forward=bucketfs_url.port,
+            ssh_port_forward=ssh_config.port,
+            db_mem_size="4GB",
+            docker_db_image_version=itde_config.db_version,
+        )
+        yield
+        cleanup_function()
+    else:
+        yield
+
+
+@pytest.fixture(scope="session")
+def backend_aware_saas_database_id(request,
+                                   use_saas,
+                                   database_name,
+                                   saas_host,
+                                   saas_pat,
+                                   saas_account_id) -> str:
+    if use_saas:
+        # Guard against a potential issue with repeated call of a parameterised fixture
+        if _saas_stash_key in request.session.stash:
+            raise RuntimeError(('Repeated call of the session level fixture '
+                                'backend_aware_saas_database_id'))
+        request.session.stash[_saas_stash_key] = True
+
+        db_id = request.config.getoption("--saas-database-id")
+        keep = request.config.getoption("--keep-saas-database")
+        idle_hours = float(request.config.getoption("--saas-max-idle-hours"))
+
+        with ExitStack() as stack:
+            # Create and configure the SaaS client.
+            client = create_saas_client(host=saas_host, pat=saas_pat)
+            api_access = OpenApiAccess(client=client, account_id=saas_account_id)
+            stack.enter_context(api_access.allowed_ip())
+
+            if db_id:
+                # Return the id of an existing database if it's provided
+                yield db_id
+            else:
+                # Create a temporary database and waite till it becomes operational
+                db = stack.enter_context(api_access.database(
+                    name=database_name,
+                    keep=keep,
+                    idle_time=timedelta(hours=idle_hours)))
+                api_access.wait_until_running(db.id)
+                yield db.id
+    else:
+        yield ''
+
+
+@pytest.fixture(scope="session")
+def backend_aware_onprem_database_params(use_onprem,
+                                         backend_aware_onprem_database,
+                                         exasol_config) -> dict[str, Any]:
+    if use_onprem:
+        return {
+            'dsn': f'{exasol_config.host}:{exasol_config.port}',
+            'user': exasol_config.username,
+            'password': exasol_config.password,
+            'websocket_sslopt': {'cert_reqs': ssl.CERT_NONE}
+        }
+    return {}
+
+
+@pytest.fixture(scope="session")
+def backend_aware_saas_database_params(use_saas,
+                                       saas_host,
+                                       saas_pat,
+                                       saas_account_id,
+                                       backend_aware_saas_database_id) -> dict[str, Any]:
+    if use_saas:
+        conn_params = get_connection_params(host=saas_host,
+                                            account_id=saas_account_id,
+                                            database_id=backend_aware_saas_database_id,
+                                            pat=saas_pat)
+        conn_params['websocket_sslopt'] = {'cert_reqs': ssl.CERT_NONE}
+        return conn_params
+    return {}
+
+
+@pytest.fixture(scope="session")
+def backend_aware_onprem_bucketfs_params(use_onprem,
+                                         backend_aware_onprem_database,
+                                         bucketfs_config) -> dict[str, Any]:
+    if use_onprem:
+        return {
+            'backend': _BACKEND_ONPREM,
+            'url': bucketfs_config.url,
+            'username': bucketfs_config.username,
+            'password': bucketfs_config.password,
+            'service_name': 'bfsdefault',
+            'bucket_name': 'default',
+            'verify': False
+        }
+    return {}
+
+
+@pytest.fixture(scope="session")
+def backend_aware_saas_bucketfs_params(use_saas,
+                                       saas_host,
+                                       saas_pat,
+                                       saas_account_id,
+                                       backend_aware_saas_database_id) -> dict[str, Any]:
+    if use_saas:
+        return {
+            'backend': _BACKEND_SAAS,
+            'url': saas_host,
+            'account_id': saas_account_id,
+            'database_id': backend_aware_saas_database_id,
+            'pat': saas_pat
+        }
+    return {}
+
+
+@pytest.fixture(scope="session")
+def backend_aware_database_params(backend,
+                                  backend_aware_onprem_database_params,
+                                  backend_aware_saas_database_params) -> dict[str, Any]:
+    """
+    Returns a set of parameters sufficient to open a pyexasol connection to the
+    current testing backend.
+    Usage example:
+    connection = pyexasol.connect(**backend_aware_database_params, compression=True)
+    """
+    if backend == _BACKEND_ONPREM:
+        return backend_aware_onprem_database_params
+    elif backend == _BACKEND_SAAS:
+        return backend_aware_saas_database_params
+    else:
+        ValueError(f'Unknown backend {backend}')
+
+
+@pytest.fixture(scope="session")
+def backend_aware_bucketfs_params(backend,
+                                  backend_aware_onprem_bucketfs_params,
+                                  backend_aware_saas_bucketfs_params) -> dict[str, Any]:
+    """
+    Returns a set of parameters sufficient to open a PathLike bucket-fs connection to the
+    current testing backend.
+    Usage example:
+    bfs_path = exasol.bucketfs.path.build_path(**backend_aware_bucketfs_params, path=path_in_bucket)
+    """
+    if backend == _BACKEND_ONPREM:
+        return backend_aware_onprem_bucketfs_params
+    elif backend == _BACKEND_SAAS:
+        return backend_aware_saas_bucketfs_params
+    else:
+        ValueError(f'Unknown backend {backend}')

pytest-backend/exasol/pytest_backend/version.py

@@ -0,0 +1,10 @@
+# ATTENTION:
+# This file is generated by exasol/toolbox/pre_commit_hooks/package_version.py when using:
+#   * either "poetry run nox -s fix"
+#   * or "poetry run version-check <path/version.py> --fix"
+# Do not edit this file manually!
+# If you need to change the version, do so in the project.toml, e.g. by using `poetry version X.Y.Z`.
+MAJOR = 0
+MINOR = 1
+PATCH = 0
+VERSION = f"{MAJOR}.{MINOR}.{PATCH}"

pytest-backend/noxconfig.py

@@ -0,0 +1,39 @@
+"""Configuration for nox based task runner"""
+from __future__ import annotations
+
+from dataclasses import dataclass
+from pathlib import Path
+from typing import (
+    Any,
+    Iterable,
+    MutableMapping,
+)
+
+from nox import Session
+
+
+@dataclass(frozen=True)
+class Config:
+    """Project specific configuration used by nox infrastructure"""
+
+    root: Path = Path(__file__).parent
+    doc: Path = Path(__file__).parent / "doc"
+    version_file: Path = Path(__file__).parent / "exasol" / "pytest_backend" / "version.py"
+    path_filters: Iterable[str] = ("dist", ".eggs", "venv", "metrics-schema")
+
+    @staticmethod
+    def pre_integration_tests_hook(
+        _session: Session, _config: Config, _context: MutableMapping[str, Any]
+    ) -> bool:
+        """Implement if project specific behaviour is required"""
+        return True
+
+    @staticmethod
+    def post_integration_tests_hook(
+        _session: Session, _config: Config, _context: MutableMapping[str, Any]
+    ) -> bool:
+        """Implement if project specific behaviour is required"""
+        return True
+
+
+PROJECT_CONFIG = Config()