Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#174: Added get_language_definition to the language container deployer #175

Merged
merged 4 commits into from
Jan 5, 2024
Merged

#174: Added get_language_definition to the language container deployer #175

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
8d572f6
Add get_language_activation to LanguageActivationLevel
tkilias Jan 5, 2024
ceb4245
Rename get_language_activation to get_language_definition
tkilias Jan 5, 2024
7e088f5
Update changelog
tkilias Jan 5, 2024
12437e6
[CodeBuild]
tkilias Jan 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions doc/changes/changes_0.8.0.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
# Transformers Extension 0.8.0, T.B.D
# Transformers Extension 0.8.0, 2024-01-05

Code name: T.B.D
Code name: Get Language Definition in LanguageContainerDeployer


## Summary

T.B.D
This release added the get_language_definition function to the LanguageContainerDeployer.

### Features

- n/a
- #174: Added get_language_definition to the language container deployer

### Bug Fixes

Expand All @@ -24,5 +24,6 @@ T.B.D
- n/a

### Security

- Update paramiko version to 3.4.0
- Updated transformers to 4.36.2
10 changes: 10 additions & 0 deletions exasol_transformers_extension/deployment/language_container_deployer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,16 @@ def _update_previous_language_settings(self, alter_type: LanguageActivationLevel
path_in_udf, prev_lang_aliases)
return new_definitions_str

def get_language_definition(self, bucket_file_path: str):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it be cleaner to split _generate_new_language_settings into a function which generates the " other_definitions" and one which generates "new_language_alias_definition"? then we do not have to call with an empty list, but instead have one calling function calling both, and one only calling one?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in general yes, but because we haven't extracted this code into a central place, yet, we need to manually copy changes to sagemaker-extension, for that reason, I would like to keep the changes simple for the moment

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

alright. maybe add a comment or something to do later?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will create a ticket

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#176

"""
Generate a language definition (ALIAS=URL) for the specified bucket file path.

bucket_file_path - Path within the designated bucket where the container is uploaded.
"""
path_in_udf = self._bucketfs_location.generate_bucket_udf_path(bucket_file_path)
result = self._generate_new_language_settings(path_in_udf=path_in_udf, prev_lang_aliases=[])
return result

def _generate_new_language_settings(self, path_in_udf: PurePosixPath,
prev_lang_aliases: List[str]) -> str:
other_definitions = [
Expand Down
28 changes: 18 additions & 10 deletions tests/unit_tests/deployment/test_language_container_deployer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,11 @@
#########################################################
from pathlib import Path, PurePosixPath
from unittest.mock import create_autospec, MagicMock, patch

import pytest
from pyexasol import ExaConnection
from exasol_bucketfs_utils_python.bucketfs_location import BucketFSLocation
from pyexasol import ExaConnection

from exasol_transformers_extension.deployment.language_container_deployer import (
LanguageContainerDeployer, LanguageActivationLevel)

Expand Down Expand Up @@ -77,14 +79,13 @@ def test_slc_deployer_activate(container_deployer, container_file_name, containe
@patch('exasol_transformers_extension.deployment.language_container_deployer.get_language_settings')
def test_slc_deployer_generate_activation_command(mock_lang_settings, container_deployer, language_alias,
container_file_name, container_bfs_path):

mock_lang_settings.return_value = 'R=builtin_r JAVA=builtin_java PYTHON3=builtin_python3'

alter_type = LanguageActivationLevel.Session
expected_command = f"ALTER {alter_type.value.upper()} SET SCRIPT_LANGUAGES='" \
"R=builtin_r JAVA=builtin_java PYTHON3=builtin_python3 " \
f"{language_alias}=localzmq+protobuf:///{container_bfs_path}?" \
f"lang=python#/buckets/{container_bfs_path}/exaudf/exaudfclient_py3';"
"R=builtin_r JAVA=builtin_java PYTHON3=builtin_python3 " \
f"{language_alias}=localzmq+protobuf:///{container_bfs_path}?" \
f"lang=python#/buckets/{container_bfs_path}/exaudf/exaudfclient_py3';"

command = container_deployer.generate_activation_command(container_file_name, alter_type)
assert command == expected_command
Expand All @@ -93,7 +94,6 @@ def test_slc_deployer_generate_activation_command(mock_lang_settings, container_
@patch('exasol_transformers_extension.deployment.language_container_deployer.get_language_settings')
def test_slc_deployer_generate_activation_command_override(mock_lang_settings, container_deployer, language_alias,
container_file_name, container_bfs_path):

current_bfs_path = 'bfsdefault/default/container_abc'
mock_lang_settings.return_value = \
'R=builtin_r JAVA=builtin_java PYTHON3=builtin_python3 ' \
Expand All @@ -102,9 +102,9 @@ def test_slc_deployer_generate_activation_command_override(mock_lang_settings, c

alter_type = LanguageActivationLevel.Session
expected_command = f"ALTER {alter_type.value.upper()} SET SCRIPT_LANGUAGES='" \
"R=builtin_r JAVA=builtin_java PYTHON3=builtin_python3 " \
f"{language_alias}=localzmq+protobuf:///{container_bfs_path}?" \
f"lang=python#/buckets/{container_bfs_path}/exaudf/exaudfclient_py3';"
"R=builtin_r JAVA=builtin_java PYTHON3=builtin_python3 " \
f"{language_alias}=localzmq+protobuf:///{container_bfs_path}?" \
f"lang=python#/buckets/{container_bfs_path}/exaudf/exaudfclient_py3';"

command = container_deployer.generate_activation_command(container_file_name, alter_type, allow_override=True)
assert command == expected_command
Expand All @@ -113,7 +113,6 @@ def test_slc_deployer_generate_activation_command_override(mock_lang_settings, c
@patch('exasol_transformers_extension.deployment.language_container_deployer.get_language_settings')
def test_slc_deployer_generate_activation_command_failure(mock_lang_settings, container_deployer, language_alias,
container_file_name):

current_bfs_path = 'bfsdefault/default/container_abc'
mock_lang_settings.return_value = \
'R=builtin_r JAVA=builtin_java PYTHON3=builtin_python3 ' \
Expand All @@ -123,3 +122,12 @@ def test_slc_deployer_generate_activation_command_failure(mock_lang_settings, co
with pytest.raises(RuntimeError):
container_deployer.generate_activation_command(container_file_name, LanguageActivationLevel.Session,
allow_override=False)


def test_slc_deployer_get_language_definition(container_deployer, language_alias,
container_file_name, container_bfs_path):
expected_command = f"{language_alias}=localzmq+protobuf:///{container_bfs_path}?" \
f"lang=python#/buckets/{container_bfs_path}/exaudf/exaudfclient_py3"

command = container_deployer.get_language_definition(container_file_name)
assert command == expected_command
Loading