Skip to content

Commit

Permalink
feat: optionally install Scenario with ops[testing] and expose the …
Browse files Browse the repository at this point in the history
…names in ops.testing (canonical#1381)

Add a new optional install `testing`, e.g. `pip install ops[testing]`.
This pulls in a compatible version of `ops-scenario`, and exposes the
Scenario names in the `ops.testing` namespace, alongside Harness.

`pip install ops[harness]` is also supported to ease the (far off,
presumably 3.0) transition to Harness being moved out of the base
install. It currently installs no extra dependencies, so is the same as
`pip install ops` but a forward-looking charm would use `pip install
ops[harness]` (or `pip install ops[harness, testing]`) if using Harness.

Requires ops-scenario 7.0.5, which has the required adjustments to
support insertion into `ops.testing`.

The `ActionFailed` name exists in both `ops._private.harness` and
`scenario.context`. This is handled by adjusting the Harness class to
support the functionality of both and monkeypatching that into Scenario
until Scenario starts using it. It's compatible with both Harness and
Scenario, but will have empty data in an attribute (which attribute
depends on which framework is used).

The `Container` name in `ops.testing`, which is only present for
backwards compatibility, is also overwritten if ops-scenario is
installed. If anyone is actually using `ops.testing.Container` instead
of `ops.Container` then they'll need to fix their code before using
`ops[testing]` (or ops-scenario combined with the release of ops with
this change).

A very basic unit test is added to make sure that Scenario tests work
(all the actual Scenario tests are in the ops-scenario repo) if
ops-scenario/ops[testing] is installed (this is the case for `tox -e
unit`). A test is also added to ensure that all of the Scenario names
are documented, since `automodule` isn't used any more.

Also adjusts the documentation to include the new framework in
ops.testing.

---------

Co-authored-by: Ben Hoyt <[email protected]>
Co-authored-by: Dima Tisnek <[email protected]>
  • Loading branch information
3 people authored Sep 25, 2024
1 parent d5eadcb commit 4a14764
Show file tree
Hide file tree
Showing 14 changed files with 510 additions and 117 deletions.
55 changes: 35 additions & 20 deletions docs/custom_conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,33 +18,17 @@

from docutils import nodes

import inspect
import sphinx.ext.autodoc
from sphinx import addnodes
from sphinx.util.docutils import SphinxDirective

import furo
import furo.navigation

sys.path.insert(0, str(pathlib.Path(__file__).parent.parent))

# Furo patch to get local TOC to show in sidebar (as sphinx-rtd-theme did)
# See https://github.com/pradyunsg/furo/blob/490527b2aef00b1198770c3389a1979911ee1fcb/src/furo/__init__.py#L115-L128

_old_compute_navigation_tree = furo._compute_navigation_tree


def _compute_navigation_tree(context):
tree_html = _old_compute_navigation_tree(context)
if not tree_html and context.get('toc'):
tree_html = furo.navigation.get_navigation_tree(context['toc'])
return tree_html


furo._compute_navigation_tree = _compute_navigation_tree

# Pull in fix from https://github.com/sphinx-doc/sphinx/pull/11222/files to fix
# "invalid signature for autoattribute ('ops.pebble::ServiceDict.backoff-delay')"
import re # noqa: E402
import sphinx.ext.autodoc # noqa: E402
import re

sphinx.ext.autodoc.py_ext_sig_re = re.compile(
r"""^ ([\w.]+::)? # explicit module name
Expand Down Expand Up @@ -216,7 +200,9 @@ def _compute_navigation_tree(context):
# pyspelling, sphinx, sphinx-autobuild, sphinx-copybutton, sphinx-design,
# sphinx-notfound-page, sphinx-reredirects, sphinx-tabs, sphinxcontrib-jquery,
# sphinxext-opengraph
custom_required_modules = []
custom_required_modules = [
'ops-scenario>=7.0.5,<8',
]

# Add files or directories that should be excluded from processing.
custom_excludes = [
Expand Down Expand Up @@ -315,6 +301,8 @@ def _compute_navigation_tree(context):
# Please keep this list sorted alphabetically.
('py:class', '_ChangeDict'),
('py:class', '_CheckInfoDict'),
('py:class', '_EntityStatus'),
('py:class', '_Event'),
('py:class', '_FileInfoDict'),
('py:class', '_NoticeDict'),
('py:class', '_ProgressDict'),
Expand All @@ -326,6 +314,8 @@ def _compute_navigation_tree(context):
('py:class', '_TextOrBinaryIO'),
('py:class', '_WarningDict'),
('py:class', '_Writeable'),
('py:class', 'AnyJson'),
('py:class', 'CharmType'),
('py:obj', 'ops._private.harness.CharmType'),
('py:class', 'ops._private.harness.CharmType'),
('py:class', 'ops.charm._ContainerBaseDict'),
Expand All @@ -345,9 +335,34 @@ def _compute_navigation_tree(context):
('py:class', 'ops.testing._ConfigOption'),
('py:class', 'ops.testing.CharmType'),
('py:obj', 'ops.testing.CharmType'),
('py:class', 'scenario.state._EntityStatus'),
('py:class', 'scenario.state._Event'),
('py:class', 'scenario.state._max_posargs.<locals>._MaxPositionalArgs'),
]


# Monkeypatch Sphinx to look for __init__ rather than __new__ for the subclasses
# of _MaxPositionalArgs.
_real_get_signature = sphinx.ext.autodoc.ClassDocumenter._get_signature


def _custom_get_signature(self):
if any(p.__name__ == '_MaxPositionalArgs' for p in self.object.__mro__):
signature = inspect.signature(self.object)
parameters = []
for position, param in enumerate(signature.parameters.values()):
if position >= self.object._max_positional_args:
parameters.append(param.replace(kind=inspect.Parameter.KEYWORD_ONLY))
else:
parameters.append(param)
signature = signature.replace(parameters=parameters)
return None, None, signature
return _real_get_signature(self)


sphinx.ext.autodoc.ClassDocumenter._get_signature = _custom_get_signature


# This is very strongly based on
# https://github.com/sphinx-doc/sphinx/blob/03b9134ee00e98df4f8b5f6d22f345cdafe31870/sphinx/domains/changeset.py#L46
# Unfortunately, the built-in class is not easily subclassable without also
Expand Down
39 changes: 39 additions & 0 deletions docs/harness.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
.. _harness:

Harness (legacy unit testing)
=============================

.. deprecated:: 2.17
The Harness framework is deprecated and will be moved out of the base
install in a future ops release. Charm authors that don't want to upgrade
will still be able to use it with ``pip install ops[harness]``.

The Harness API includes:

- :class:`ops.testing.Harness`, a class to set up the simulated environment,
that provides:

- :meth:`~ops.testing.Harness.add_relation` method, to declare a relation
(integration) with another app.
- :meth:`~ops.testing.Harness.begin` and :meth:`~ops.testing.Harness.cleanup`
methods to start and end the testing lifecycle.
- :meth:`~ops.testing.Harness.evaluate_status` method, which aggregates the
status of the charm after test interactions.
- :attr:`~ops.testing.Harness.model` attribute, which exposes e.g. the
:attr:`~ops.Model.unit` attribute for detailed assertions on the unit's state.

.. warning:: The Harness API has flaws with resetting the charm state between
Juju events. Care must be taken when emitting multiple events with the same
Harness object.

.. note::
Unit testing is only one aspect of a comprehensive testing strategy. For more
on testing charms, see `Charm SDK | Testing <https://juju.is/docs/sdk/testing>`_.


.. autoclass:: ops.testing.ActionFailed
:noindex:
.. autoclass:: ops.testing.ActionOutput
.. autoclass:: ops.testing.ExecArgs
.. autoclass:: ops.testing.ExecResult
.. autoclass:: ops.testing.Harness
49 changes: 21 additions & 28 deletions docs/index.rst
Original file line number Diff line number Diff line change
@@ -1,79 +1,72 @@

ops library API reference
=========================
API reference
=============

The `ops` library is a Python framework for writing and testing Juju charms.

See more: `Charm SDK documentation <https://juju.is/docs/sdk>`_

The library provides:
The library (`available on PyPI`_) provides:

- :ref:`ops_main_entry_point`, used to initialise and run your charm;
- :ref:`ops_module`, the API to respond to Juju events and manage the application;
- :ref:`ops_main_entry_point`, used to initialise and run your charm;
- :ref:`ops_pebble_module`, the Pebble client, a low-level API for Kubernetes containers;
- :ref:`ops_testing_module`, the framework for unit testing charms in a simulated environment;
- the APIs for unit testing charms in a simulated environment:

- :doc:`State-transition testing </state-transition-testing>`. This is the
recommended approach (it was previously known as 'Scenario').
- :doc:`Harness </harness>`. This is a deprecated framework, and has issues,
particularly with resetting the charm state between Juju events.

You can structure your charm however you like, but with the `ops` library, you
get a framework that promotes consistency and readability by following best
practices. It also helps you organise your code better by separating different
aspects of the charm, such as managing the applications state, handling
aspects of the charm, such as managing the application's state, handling
integrations with other services, and making the charm easier to test.

.. _available on PyPI: https://pypi.org/project/ops/

.. toctree::
:hidden:
:maxdepth: 2
:caption: Contents:

self
state-transition-testing
harness

.. _ops_module:

ops module
==========
ops
---

.. automodule:: ops
:exclude-members: main


.. _ops_main_entry_point:

ops.main entry point
====================
--------------------

The main entry point to initialise and run your charm.

.. autofunction:: ops.main


legacy main module
------------------

.. automodule:: ops.main
:noindex:


.. _ops_pebble_module:

ops.pebble module
=================
ops.pebble
----------

.. automodule:: ops.pebble


.. _ops_testing_module:

ops.testing module
==================

.. autoclass:: ops.testing.ActionFailed
.. autoclass:: ops.testing.ActionOutput
.. autoclass:: ops.testing.ExecArgs
.. autoclass:: ops.testing.ExecResult
.. autoclass:: ops.testing.Harness


Indices
=======

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
12 changes: 10 additions & 2 deletions docs/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# This file is autogenerated by pip-compile with Python 3.12
# This file is autogenerated by pip-compile with Python 3.11
# by the following command:
#
# pip-compile --extra=docs --output-file=docs/requirements.txt pyproject.toml
Expand Down Expand Up @@ -73,6 +73,10 @@ mdurl==0.1.2
# via markdown-it-py
myst-parser==4.0.0
# via ops (pyproject.toml)
ops==2.16.1
# via ops-scenario
ops-scenario==7.0.5
# via ops (pyproject.toml)
packaging==24.1
# via sphinx
pygments==2.18.0
Expand All @@ -85,7 +89,9 @@ pyspelling==2.10
pyyaml==6.0.2
# via
# myst-parser
# ops
# ops (pyproject.toml)
# ops-scenario
# pyspelling
requests==2.32.3
# via
Expand Down Expand Up @@ -160,6 +166,8 @@ wcmatch==9.0
webencodings==0.5.1
# via html5lib
websocket-client==1.8.0
# via ops (pyproject.toml)
# via
# ops
# ops (pyproject.toml)
websockets==12.0
# via sphinx-autobuild
Loading

0 comments on commit 4a14764

Please sign in to comment.