Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use pytest-backend-plugin to test against SaaS and on-prem and fix putfile SaaS bug #129

Open
tkilias opened this issue Aug 26, 2024 · 1 comment · May be fixed by #145 or #148
Open

Use pytest-backend-plugin to test against SaaS and on-prem and fix putfile SaaS bug #129

tkilias opened this issue Aug 26, 2024 · 1 comment · May be fixed by #145 or #148
Assignees
Labels
bug Unwanted / harmful behavior

Comments

@tkilias
Copy link
Collaborator

tkilias commented Aug 26, 2024

This issue is more related to cloudstorage-extension setup in the ai lab, rather than the AI Lab itself.

Proposed fix:

  • use pytest plugin in notebook-connector
  • then the error will be observable in the notebook-connector

Background

In https://github.com/exasol/ai-lab/actions/runs/10525178182/job/29163449570 we had the following error

E           ------------------
E           from exasol.nb_connector import github, bfs_utils, cloud_storage
E           from exasol.nb_connector.connections import open_bucketfs_connection, open_pyexasol_connection
E           
E           jar_local_path = github.retrieve_jar(github.Project.CLOUD_STORAGE_EXTENSION, use_local_cache=True)
E           bfs_bucket = open_bucketfs_connection(ai_lab_config)
E           bfs_path = bfs_utils.put_file(bfs_bucket, jar_local_path)
E           
E           with open_pyexasol_connection(ai_lab_config) as conn:
E               cloud_storage.setup_scripts(conn, ai_lab_config.db_schema, bfs_path)
E               
E           print("Could Storage Extension was initialized")
E           ------------------
E           
E           
E           ---------------------------------------------------------------------------
E           TypeError                                 Traceback (most recent call last)
E           Cell In[2], line 6
E                 4 jar_local_path = github.retrieve_jar(github.Project.CLOUD_STORAGE_EXTENSION, use_local_cache=True)
E                 5 bfs_bucket = open_bucketfs_connection(ai_lab_config)
E           ----> 6 bfs_path = bfs_utils.put_file(bfs_bucket, jar_local_path)
E                 8 with open_pyexasol_connection(ai_lab_config) as conn:
E                 9     cloud_storage.setup_scripts(conn, ai_lab_config.db_schema, bfs_path)
E           
E           File ~/jupyterenv/lib/python3.10/site-packages/exasol/nb_connector/bfs_utils.py:24, in put_file(bucket, file_path, skip_if_exists)
E                22     raise ValueError(f"Local file doesn't exist: {file_path}")
E                23 local_name = file_path.name
E           ---> 24 if skip_if_exists and local_name in list(bucket):
E                25     _logger.info("File %s is already present in the bucketfs", local_name)
E                26 else:
E           
E           TypeError: 'SaaSBucket' object is not iterable

job-logs.txt

The reason that we recognized the error that late was first, that test was disabled in the ai-lab, but also because we don't test here against saas.

Acceptance Criteria

  • use pytest-backend-plugin for testing agains onprem and SaaS
  • fix bug
@tkilias tkilias added the refactoring Code improvement without behavior change label Aug 26, 2024
@tkilias tkilias changed the title Use pytest-backend-plugin to test against SaaS and on-prem Use pytest-backend-plugin to test against SaaS and on-prem and fix putfile SaaS bug Aug 26, 2024
@tkilias tkilias added bug Unwanted / harmful behavior and removed refactoring Code improvement without behavior change labels Aug 26, 2024
@Shmuma Shmuma self-assigned this Dec 6, 2024
@Shmuma Shmuma linked a pull request Dec 6, 2024 that will close this issue
3 tasks
@Shmuma Shmuma linked a pull request Dec 18, 2024 that will close this issue
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Unwanted / harmful behavior
Projects
None yet
3 participants
@Shmuma @tkilias and others