Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow direct execution #41

Merged
merged 8 commits into from
Oct 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/development/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,9 @@ venvstacks can then be executed via the ``-m`` switch:

.. code-block:: console

$ .venv/bin/python -m venvstacks --help
$ .venv/bin/venvstacks --help

Usage: python -m venvstacks [OPTIONS] COMMAND [ARGS]...
Usage: venvstacks [OPTIONS] COMMAND [ARGS]...

Lock, build, and publish Python virtual environment stacks.

Expand Down
21 changes: 14 additions & 7 deletions docs/overview.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,14 @@ Installing
----------

``venvstacks`` is available from the :pypi:`Python Package Index <venvstacks>`,
and can be installed with :pypi:`pip`:
and can be installed with :pypi:`pipx` (or similar tools):

.. code-block:: console

$ pipx install venvstacks

Alternatively, it can be installed as a user level package (although this may
make future Python version upgrades more irritating):

.. code-block:: console

Expand All @@ -27,9 +34,9 @@ The command line help also provides additional usage information:

.. code-block:: console

$ .venv/bin/python -m venvstacks --help
$ venvstacks --help

Usage: python -m venvstacks [OPTIONS] COMMAND [ARGS]...
Usage: venvstacks [OPTIONS] COMMAND [ARGS]...

Lock, build, and publish Python virtual environment stacks.

Expand Down Expand Up @@ -100,7 +107,7 @@ Locking environment stacks

.. code-block:: console

$ python -m venvstacks lock sklearn_demo/venvstacks.toml
$ venvstacks lock sklearn_demo/venvstacks.toml

The ``lock`` subcommand takes the defined layer requirements from the specification,
and uses them to perform a complete combined resolution of all of the environment stacks
Expand All @@ -116,7 +123,7 @@ Building environment stacks

.. code-block:: console

$ python -m venvstacks build sklearn_demo/venvstacks.toml
$ venvstacks build sklearn_demo/venvstacks.toml

The ``build`` subcommand performs the step of converting the layer specifications
and their locked requirements into a working Python environment
Expand All @@ -133,7 +140,7 @@ Publishing environment layer archives

.. code-block:: console

$ python -m venvstacks publish --tag-outputs --output-dir demo_artifacts sklearn_demo/venvstacks.toml
$ venvstacks publish --tag-outputs --output-dir demo_artifacts sklearn_demo/venvstacks.toml

Once the environments have been successfully built,
the ``publish`` command allows each layer to be converted to a separate
Expand All @@ -154,7 +161,7 @@ Locally exporting environment stacks

.. code-block:: console

$ python -m venvstacks local-export --output-dir demo_export sklearn_demo/venvstacks.toml
$ venvstacks local-export --output-dir demo_export sklearn_demo/venvstacks.toml

Given that even considering the use of ``venvstacks`` implies that some layer archives may be of
significant size (a fully built `pytorch` archive weighs in at multiple gigabytes, for example),
Expand Down
3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,9 @@ rich-cli = [
"typer>=0.12.4",
]

[project.scripts]
venvstacks = "venvstacks.cli:main"

[build-system]
requires = ["pdm-backend"]
build-backend = "pdm.backend"
Expand Down
15 changes: 8 additions & 7 deletions src/venvstacks/_util.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

from contextlib import contextmanager
from pathlib import Path
from typing import Any, Generator
from typing import Any, Generator, Mapping

WINDOWS_BUILD = hasattr(os, "add_dll_directory")

Expand Down Expand Up @@ -78,23 +78,24 @@ def get_env_python(env_path: Path) -> Path:


def run_python_command_unchecked(
# Narrow list/dict type specs here due to the way `subprocess.run` params are typed
# Narrow list type spec here due to the way `subprocess.run` params are typed
command: list[str],
*,
env: dict[str, str] | None = None,
env: Mapping[str, str] | None = None,
**kwds: Any,
) -> subprocess.CompletedProcess[str]:
if env is None:
env = os.environ.copy()
env.update(_SUBPROCESS_PYTHON_CONFIG)
run_env = os.environ.copy()
if env is not None:
run_env.update(env)
run_env.update(_SUBPROCESS_PYTHON_CONFIG)
result: subprocess.CompletedProcess[str] = subprocess.run(
command, env=env, text=True, **kwds
)
return result


def run_python_command(
# Narrow list/dict type specs here due to the way `subprocess.run` params are typed
# Narrow list type spec here due to the way `subprocess.run` params are typed
command: list[str],
**kwds: Any,
) -> subprocess.CompletedProcess[str]:
Expand Down
23 changes: 22 additions & 1 deletion src/venvstacks/cli.py
Original file line number Diff line number Diff line change
@@ -1,21 +1,42 @@
"""Command line interface implementation"""

import os.path
import sys

from typing import Annotated

import typer

from .stacks import StackSpec, BuildEnvironment, _format_json, IndexConfig

# Inspired by the Python 3.13+ `argparse` feature,
# but reports `python -m venvstacks` whenever `__main__`
# refers to something other than the entry point script,
# rather than trying to infer anything from the main
# module's `__spec__` attributes.
_THIS_PACKAGE = __spec__.parent


def _get_usage_name() -> str:
exec_name = os.path.basename(sys.argv[0]).removesuffix(".exe")
if exec_name == _THIS_PACKAGE:
# Entry point wrapper, suggest direct execution
return exec_name
# Could be `python -m`, could be the test suite,
# could be something else calling `venvstacks.cli.main`,
# but treat it as `python -m venvstacks` regardless
py_name = os.path.basename(sys.executable).removesuffix(".exe")
return f"{py_name} -m {_THIS_PACKAGE}"


_cli = typer.Typer(
add_completion=False,
pretty_exceptions_show_locals=False,
no_args_is_help=True,
)


@_cli.callback(name="python -m venvstacks")
@_cli.callback(name=_get_usage_name())
def handle_app_options() -> None:
"""Lock, build, and publish Python virtual environment stacks."""
# TODO: Handle app logging config via main command level options
Expand Down
2 changes: 1 addition & 1 deletion tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ Updating metadata and examining built artifacts
To generate a full local sample project build to help debug failures:

$ cd /path/to/repo/
$ pdm run python -m venvstacks build --publish \
$ pdm run venvstacks build --publish \
tests/sample_project/venvstacks.toml ~/path/to/output/folder

This assumes `pdm sync --dev` has been used to set up a local development venv.
Expand Down
29 changes: 28 additions & 1 deletion tests/test_cli_invocation.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
"""Test cases for CLI invocation"""

import subprocess
import sys

from contextlib import contextmanager
from dataclasses import dataclass, field
from pathlib import Path
Expand All @@ -17,6 +20,7 @@

from venvstacks import cli
from venvstacks.stacks import BuildEnvironment, EnvironmentLock, IndexConfig
from venvstacks._util import run_python_command_unchecked


def report_traceback(exc: BaseException | None) -> str:
Expand Down Expand Up @@ -128,6 +132,8 @@ def mocked_runner() -> Generator[MockedRunner, None, None]:
class TestTopLevelCommand:
def test_implicit_help(self, mocked_runner: MockedRunner) -> None:
result = mocked_runner.invoke([])
# Usage message should suggest indirect execution
assert "Usage: python -m venvstacks [" in result.stdout
# Top-level callback docstring is used as the overall CLI help text
cli_help = cli.handle_app_options.__doc__
assert cli_help is not None
Expand All @@ -142,8 +148,29 @@ def test_implicit_help(self, mocked_runner: MockedRunner) -> None:
assert result.exception is None, report_traceback(result.exception)
assert result.exit_code == 0

# See https://github.com/lmstudio-ai/venvstacks/issues/42
@pytest.mark.xfail(
sys.platform == "win32", reason="UnicodeDecodeError parsing output"
)
def test_entry_point_help(self) -> None:
if sys.prefix == sys.base_prefix:
pytest.skip("Entry point test requires test execution in venv")
expected_entry_point = Path(sys.executable).parent / "venvstacks"
command = [str(expected_entry_point), "--help"]
result = run_python_command_unchecked(
command, stdout=subprocess.PIPE, stderr=subprocess.PIPE
)
# Usage message should suggest direct execution
assert "Usage: venvstacks [" in result.stdout
# Top-level callback docstring is used as the overall CLI help text
cli_help = cli.handle_app_options.__doc__
assert cli_help is not None
assert cli_help.strip() in result.stdout
# Check operation result last to ensure test results are as informative as possible
assert result.returncode == 0


EXPECTED_USAGE_PREFIX = "Usage: python -m venvstacks"
EXPECTED_USAGE_PREFIX = "Usage: python -m venvstacks "
EXPECTED_SUBCOMMANDS = ["lock", "build", "local-export", "publish"]
NO_SPEC_PATH: list[str] = []
NEEDS_SPEC_PATH = sorted(set(EXPECTED_SUBCOMMANDS) - set(NO_SPEC_PATH))
Expand Down