Skip to content

Commit

Permalink
docs: update how-to guides to use the new unit testing framework (#1507)
Browse files Browse the repository at this point in the history 
Adjustments to the how-to guides:
* Links to the API reference are updated to the new location. I assume
that these can be references that will automatically resolve somehow,
but I'm not sure how to do that, so I opened #1506 and just fixed them
as full links for now (at least they'll work again).
* A few minor whitespace cleanups.
* Removed all the Harness sections, moving the Scenario sections up to
be "unit tests"
* Updated the Scenario sections to use ops.testing and Scenario 7
* Updated the secrets how-to to be more explicit about needing to remove
revisions if you create new ones.

[Live preview](https://ops--1507.org.readthedocs.build/en/1507/)
  • Loading branch information
@tonyandrewmeyer
tonyandrewmeyer authored Dec 19, 2024
1 parent dde98d9 commit dea45c1
Show file tree
Hide file tree
Showing 13 changed files with 254 additions and 482 deletions.
4 changes: 2 additions & 2 deletions docs/howto/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ Manage secrets <manage-secrets>
Manage the charm version <manage-the-charm-version>
Manage the workload version <manage-the-workload-version>
Get started with charm testing <get-started-with-charm-testing>
Write unit tests for a charm <write-unit-tests-for-a-charm>
Write scenario tests for a charm <write-scenario-tests-for-a-charm>
Write unit tests for a charm <write-scenario-tests-for-a-charm>
Write integration tests for a charm <write-integration-tests-for-a-charm>
Write legacy unit tests for a charm <write-unit-tests-for-a-charm>
Turn a hooks-based charm into an ops charm <turn-a-hooks-based-charm-into-an-ops-charm>
```
Expand Down
70 changes: 22 additions & 48 deletions docs/howto/manage-actions.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@

To tell users what actions can be performed on the charm, define an `actions` section in `charmcraft.yaml` that lists the actions and information about each action. The actions should include a short description that explains what running the action will do. Normally, all parameters that can be passed to the action are also included here, including the type of parameter and any default value. You can also specify that some parameters are required when the action is run.
For example:

```yaml
actions:
snapshot:
Expand Down Expand Up @@ -41,8 +42,6 @@ actions:
additionalProperties: false
```
### Observe the action event and define an event handler
In the `src/charm.py` file, in the `__init__` function of your charm, set up an observer for the action event associated with your action and pair that with an event handler. For example:
Expand All @@ -51,10 +50,9 @@ In the `src/charm.py` file, in the `__init__` function of your charm, set up an
self.framework.observe(self.on.grant_admin_role_action, self._on_grant_admin_role_action)
```
Now, in the body of the charm definition, define the action event handler. For example:
```
```python
def _on_grant_admin_role_action(self, event):
"""Handle the grant-admin-role action."""
# Fetch the user parameter from the ActionEvent params dict
Expand All @@ -77,7 +75,6 @@ def _on_grant_admin_role_action(self, event):

More detail below:

#### Use action params

To make use of action parameters, either ones that the user has explicitly passed, or default values, use the `params` attribute of the event object that is passed to the handler. This is a dictionary of parameter name (string) to parameter value. For example:
Expand All @@ -87,8 +84,8 @@ def _on_snapshot(self, event: ops.ActionEvent):
filename = event.params["filename"]
...
```
> See more: [`ops.ActionEvent.params`](https://ops.readthedocs.io/en/latest/#ops.ActionEvent.params)

> See more: [`ops.ActionEvent.params`](https://ops.readthedocs.io/en/latest/reference/ops.html#ops.ActionEvent.params)
#### Report that an action has failed

Expand All @@ -108,22 +105,24 @@ def _on_snapshot(self, event: ops.ActionEvent):
...
```

> See more: [`ops.ActionEvent.fail`](https://ops.readthedocs.io/en/latest/#ops.ActionEvent.fail)
> See more: [`ops.ActionEvent.fail`](https://ops.readthedocs.io/en/latest/reference/ops.html#ops.ActionEvent.fail)
#### Return the results of an action

To pass back the results of an action to the user, use the `set_results` method of the action event. These will be displayed in the `juju run` output. For example:

```python
def _on_snapshot(self, event: ops.ActionEvent):
size = self.do_snapshot(event.params['filename'])
event.set_results({'snapshot-size': size})
```
> See more: [`ops.ActionEvent.set_results`](https://ops.readthedocs.io/en/latest/#ops.ActionEvent.set_results)

> See more: [`ops.ActionEvent.set_results`](https://ops.readthedocs.io/en/latest/reference/ops.html#ops.ActionEvent.set_results)
#### Log the progress of an action

In a long-running action, to give the user updates on progress, use the `.log()` method of the action event. This is sent back to the user, via Juju, in real-time, and appears in the output of the `juju run` command. For example:

```python
def _on_snapshot(self, event: ops.ActionEvent):
event.log('Starting snapshot')
Expand All @@ -133,75 +132,51 @@ def _on_snapshot(self, event: ops.ActionEvent):
event.log('Table2 complete')
self.snapshot_table3()
```
> See more: [`ops.ActionEvent.log`](https://ops.readthedocs.io/en/latest/#ops.ActionEvent.log)

> See more: [`ops.ActionEvent.log`](https://ops.readthedocs.io/en/latest/reference/ops.html#ops.ActionEvent.log))
#### Record the ID of an action task

When a unique ID is needed for the action task - for example, for logging or creating temporary files, use the `.id` attribute of the action event. For example:

```python
def _on_snapshot(self, event: ops.ActionEvent):
temp_filename = f'backup-{event.id}.tar.gz'
logger.info("Using %s as the temporary backup filename in task %s", filename, event.id)
self.create_backup(temp_filename)
...
```
> See more: [`ops.ActionEvent.id`](https://ops.readthedocs.io/en/latest/#ops.ActionEvent.id)
> See more: [`ops.ActionEvent.id`](https://ops.readthedocs.io/en/latest/reference/ops.html#ops.ActionEvent.id)
## Test the feature

> See first: {ref}`get-started-with-charm-testing`
What you need to do depends on what kind of tests you want to write.


### Write unit tests

> See first: {ref}`write-unit-tests-for-a-charm`
When using Harness for unit tests, use the `run_action` method to verify that charm actions have the expected behaviour. This method will either raise an `ActionFailed` exception (if the charm used the `event.fail()` method) or return an `ActionOutput` object. These can be used to verify the failure message, logs, and results of the action. For example:

```python
def test_backup_action():
harness = ops.testing.Harness()
harness.begin()
try:
out = harness.run_action('snapshot', {'filename': 'db-snapshot.tar.gz'})
except ops.testing.ActionFailed as e:
assert "Could not backup because" in e.message
else:
assert out.logs == ['Starting snapshot', 'Table1 complete', 'Table2 complete']
assert 'snapshot-size' in out.results
finally:
harness.cleanup()


```

> See more: [`ops.testing.Harness.run_action`](https://ops.readthedocs.io/en/latest/#ops.testing.Harness.run_action)

### Write scenario tests
> See first: {ref}`write-scenario-tests-for-a-charm`
When using Scenario for unit tests, to verify that the charm state is as expected after executing an action, use the `run_action` method of the Scenario `Context` object. The method returns an `ActionOutput` object that contains any logs and results that the charm set.
To verify that the charm state is as expected after executing an action, use the `run` method of the `Context` object, with `ctx.on.action`. The context contains any logs and results that the charm set.

For example:

```python
from ops import testing

def test_backup_action():
action = scenario.Action('snapshot', params={'filename': 'db-snapshot.tar.gz'})
ctx = scenario.Context(MyCharm)
out = ctx.run_action(action, scenario.State())
assert out.logs == ['Starting snapshot', 'Table1 complete', 'Table2 complete']
if out.success:
assert 'snapshot-size' in out.results
else:
assert 'Failed to run' in out.failure
ctx = testing.Context(MyCharm)
ctx.run(ctx.on.action('snapshot', params={'filename': 'db-snapshot.tar.gz'}), testing.State())
assert ctx.action_logs == ['Starting snapshot', 'Table1 complete', 'Table2 complete']
assert 'snapshot-size' in ctx.action_results
```
> See more: [Scenario action testing](https://github.com/canonical/ops-scenario?tab=readme-ov-file#actions)

> See more: [`Context.action_logs`](https://ops.readthedocs.io/en/latest/reference/ops-testing.html#ops.testing.Context.action_logs), [`Context.action_results`](https://ops.readthedocs.io/en/latest/reference/ops-testing.html#ops.testing.Context.action_results), [`ActionFailed`](https://ops.readthedocs.io/en/latest/reference/ops-testing.html#ops.testing.ActionFailed)

### Write integration tests

> See first: {ref}`write-integration-tests-for-a-charm`
To verify that an action works correctly against a real Juju instance, write an integration test with `pytest_operator`. For example:
Expand All @@ -215,4 +190,3 @@ async def test_logger(ops_test):
assert action.status == 'completed'
assert action.results['snapshot-size'].isdigit()
```

46 changes: 8 additions & 38 deletions docs/howto/manage-configurations.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ config:
type: string
```


### Observe the `config-changed` event and define the event handler

In the `src/charm.py` file of the charm project, in the `__init__` function of the charm, set up an observer for the config changed event and pair that with an event handler:
Expand All @@ -47,14 +46,14 @@ def _on_config_changed(self, event):
logger.debug("New application port is requested: %s", port)
self._update_layer_and_restart(None)
```
> See more: [`ops.CharmBase.config`](https://ops.readthedocs.io/en/latest/#ops.CharmBase.config)

> See more: [`ops.CharmBase.config`](https://ops.readthedocs.io/en/latest/reference/ops.html#ops.CharmBase.config)
```{caution}
- Multiple configuration values can be changed at one time through Juju, resulting in only one `config_changed` event. Thus, your charm code must be able to process more than one config value changing at a time.
- If `juju config` is run with values the same as the current configuration, the `config_changed` event will not run. Therefore, if you have a single config value, there is no point in tracking its previous value -- the event will only be triggered if the value changes.
- Configuration cannot be changed from within the charm code. Charms, by design, aren't able to mutate their own configuration by themselves (e.g., in order to ignore an admin-provided configuration), or to configure other applications. In Ops, one typically interacts with config via a read-only facade.
```

### (If applicable) Update and restart the Pebble layer
Expand All @@ -69,56 +68,27 @@ https://github.com/canonical/juju-sdk-tutorial-k8s/compare/01_create_minimal_cha

> See first: {ref}`get-started-with-charm-testing`
You'll want to add two levels of tests: unit and scenario.

### Write unit tests

> See first: {ref}`write-unit-tests-for-a-charm`
To use a unit test to verify that the configuration change is handled correct, the test needs to trigger the `config-changed` event and then check that the update method was called. In your `tests/unit/test_charm.py` file, add the following test functions to the file:
To verify that the `config-changed` event validates the port, pass the new config to the `State`, and, after running the event, check the unit status. For example, in your `tests/unit/test_charm.py` file, add the following test function:

```python
def test_invalid_port_configuration():
harness = ops.testing.Harness()
harness.begin()

harness.update_config({"server-port": 22})
assert isinstance(harness.model.unit.status, ops.BlockedStatus)

def test_port_configuration(monkeypatch):
update_called = False
def mock_update(*args):
update_called = True
monkeypatch.setattr(MyCharm, "_update_layer_and_restart", mock_update)

harness = ops.testing.Harness()
harness.begin()

harness.update_config({"server-port": 8080})
from ops import testing

assert update_called
```

### Write scenario tests

> See first: {ref}`write-scenario-tests-for-a-charm`
To use a Scenario test to verify that the `config-changed` event validates the port, pass the new config to the `State`, and, after running the event, check the unit status. For example, in your `tests/scenario/test_charm.py` file, add the following test function:

```python
def test_open_port():
ctx = scenario.Context(MyCharm)
ctx = testing.Context(MyCharm)

state_out = ctx.run("config_changed", scenario.State(config={"server-port": 22}))
state_out = ctx.run(ctx.on.config_changed(), testing.State(config={"server-port": 22}))

assert isinstance(state_out.unit_status, ops.BlockedStatus)
assert isinstance(state_out.unit_status, testingZ.BlockedStatus)
```

### Test-deploy
### Manually test

To verify that the configuration option works as intended, pack your charm, update it in the Juju model, and run `juju config` followed by the name of the application deployed by your charm and then your newly defined configuration option key set to some value. For example, given the `server-port` key defined above, you could try:

```text
juju config <name of application deployed by your charm> server-port=4000
```

50 changes: 5 additions & 45 deletions docs/howto/manage-leadership-changes.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ In the `src/charm.py` file, in the `__init__` function of your charm, set up an
self.framework.observe(self.on.leader_elected, self._on_leader_elected)
```

> See more: [`ops.LeaderElectedEvent`](https://ops.readthedocs.io/en/latest/#ops.LeaderElectedEvent)
> See more: [`ops.LeaderElectedEvent`](https://ops.readthedocs.io/en/latest/reference/ops.html#ops.LeaderElectedEvent)
Now, in the body of the charm definition, define the event handler. For example, the handler below will update a configuration file:

Expand Down Expand Up @@ -52,48 +52,11 @@ event or an `is-leader` check. If the charm code may run longer, then extra

> See first: {ref}`get-started-with-charm-testing`

### Write unit tests

> See first: {ref}`write-unit-tests-for-a-charm`
When using Harness for unit tests, use the `set_leader()` method to control whether the unit is the leader. For example, to verify that leadership change is handled correctly:

```python
@pytest.fixture()
def harness():
yield ops.testing.Harness(MyCharm)
harness.cleanup()


def test_new_leader(harness):
# Before the test, the unit is not leader.
harness.set_leader(False)
harness.begin()
# Simulate Juju electing the unit as leader.
harness.set_leader(True)
# Assert that it was handled correctly.
assert ...


def test_leader_sets_secrets(harness):
# The unit is the leader throughout the test, and no leader-elected event
# is emitted.
harness.set_leader(True)
harness.begin()
secret_id = harness.add_model_secret(APP_NAME, content={"secret": "sssh"})
harness.update_config(secret_option=secret_id)
# Assert that the config-changed handler set additional secret metadata:
assert ...
```

> See more: [`ops.testing.Harness.set_leader`](https://ops.readthedocs.io/en/latest/harness.html#ops.testing.Harness.set_leader)
## Write scenario tests

> See first: {ref}`write-scenario-tests-for-a-charm`
When using Scenario for unit tests, pass the leadership status to the `State`. For example:
To verify behaviour when leadership has changed, pass the leadership status to the `State`. For example:

```python
class MyCharm(ops.CharmBase):
Expand All @@ -110,12 +73,11 @@ class MyCharm(ops.CharmBase):

@pytest.mark.parametrize('leader', (True, False))
def test_status_leader(leader):
ctx = scenario.Context(MyCharm, meta={"name": "foo"})
out = ctx.run('start', scenario.State(leader=leader))
assert out.unit_status == ops.ActiveStatus('I rule' if leader else 'I am ruled')
ctx = testing.Context(MyCharm, meta={"name": "foo"})
out = ctx.run(ctx.on.start(), testing.State(leader=leader))
assert out.unit_status == testing.ActiveStatus('I rule' if leader else 'I am ruled')
```


## Write integration tests

> See first: {ref}`write-integration-tests-for-a-charm`
Expand Down Expand Up @@ -146,5 +108,3 @@ async def get_leader_unit(ops_test, app, model=None):
> Examples: [Zookeeper testing upgrades](https://github.com/canonical/zookeeper-operator/blob/106f9c2cd9408a172b0e93f741d8c9f860c4c38e/tests/integration/test_upgrade.py#L22), [postgresql testing password rotation action](https://github.com/canonical/postgresql-k8s-operator/blob/62645caa89fd499c8de9ac3e5e9598b2ed22d619/tests/integration/test_password_rotation.py#L38)
> See more: [`juju.unit.Unit.is_leader_from_status`](https://pythonlibjuju.readthedocs.io/en/latest/api/juju.unit.html#juju.unit.Unit.is_leader_from_status)

Loading

0 comments on commit dea45c1

Please sign in to comment.