Skip to content

Commit

Permalink
fixed tests, new charm tracing version
Browse files Browse the repository at this point in the history
  • Loading branch information
PietroPasotti committed Jun 21, 2024
1 parent 76f84c1 commit e8abede
Show file tree
Hide file tree
Showing 10 changed files with 250 additions and 687 deletions.
55 changes: 40 additions & 15 deletions lib/charms/loki_k8s/v1/loki_push_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,20 +16,24 @@
send log to Loki by implementing the consumer side of the `loki_push_api` relation interface.
For instance, a Promtail or Grafana agent charm which needs to send logs to Loki.
- `LogProxyConsumer`: This object can be used by any Charmed Operator which needs to
send telemetry, such as logs, to Loki through a Log Proxy by implementing the consumer side of the
`loki_push_api` relation interface.
- `LogProxyConsumer`: DEPRECATED.
This object can be used by any Charmed Operator which needs to send telemetry, such as logs, to
Loki through a Log Proxy by implementing the consumer side of the `loki_push_api` relation
interface.
In order to be able to control the labels on the logs pushed this object adds a Pebble layer
that runs Promtail in the workload container, injecting Juju topology labels into the
logs on the fly.
This object is deprecated. Consider migrating to LogForwarder with the release of Juju 3.6 LTS.
- `LogForwarder`: This object can be used by any Charmed Operator which needs to send the workload
standard output (stdout) through Pebble's log forwarding mechanism, to Loki endpoints through the
`loki_push_api` relation interface.
In order to be able to control the labels on the logs pushed this object updates the pebble layer's
"log-targets" section with Juju topology.
Filtering logs in Loki is largely performed on the basis of labels. In the Juju ecosystem, Juju
topology labels are used to uniquely identify the workload which generates telemetry like logs.
In order to be able to control the labels on the logs pushed this object adds a Pebble layer
that runs Promtail in the workload container, injecting Juju topology labels into the
logs on the fly.
## LokiPushApiProvider Library Usage
Expand All @@ -42,13 +46,14 @@
- `charm`: A reference to the parent (Loki) charm.
- `relation_name`: The name of the relation that the charm uses to interact
with its clients, which implement `LokiPushApiConsumer` or `LogProxyConsumer`.
with its clients, which implement `LokiPushApiConsumer` `LogForwarder`, or `LogProxyConsumer`
(note that LogProxyConsumer is deprecated).
If provided, this relation name must match a provided relation in metadata.yaml with the
`loki_push_api` interface.
The default relation name is "logging" for `LokiPushApiConsumer` and "log-proxy" for
`LogProxyConsumer`.
The default relation name is "logging" for `LokiPushApiConsumer` and `LogForwarder`, and
"log-proxy" for `LogProxyConsumer` (note that LogProxyConsumer is deprecated).
For example, a provider's `metadata.yaml` file may look as follows:
Expand Down Expand Up @@ -223,6 +228,9 @@ def __init__(self, *args):
## LogProxyConsumer Library Usage
> Note: This object is deprecated. Consider migrating to LogForwarder with the release of Juju 3.6
> LTS.
Let's say that we have a workload charm that produces logs, and we need to send those logs to a
workload implementing the `loki_push_api` interface, such as `Loki` or `Grafana Agent`.
Expand Down Expand Up @@ -519,7 +527,7 @@ def _alert_rules_error(self, event):

# Increment this PATCH version before using `charmcraft publish-lib` or reset
# to 0 if you are raising the major API version
LIBPATCH = 8
LIBPATCH = 11

PYDEPS = ["cosl"]

Expand All @@ -534,13 +542,21 @@ def _alert_rules_error(self, event):
# To update Promtail version you only need to change the PROMTAIL_VERSION and
# update all sha256 sums in PROMTAIL_BINARIES. To support a new architecture
# you only need to add a new key value pair for the architecture in PROMTAIL_BINARIES.
PROMTAIL_VERSION = "v2.5.0"
PROMTAIL_VERSION = "v2.9.7"
PROMTAIL_ARM_BINARY = {
"filename": "promtail-static-arm64",
"zipsha": "c083fdb45e5c794103f974eeb426489b4142438d9e10d0ae272b2aff886e249b",
"binsha": "4cd055c477a301c0bdfdbcea514e6e93f6df5d57425ce10ffc77f3e16fec1ddf",
}

PROMTAIL_BINARIES = {
"amd64": {
"filename": "promtail-static-amd64",
"zipsha": "543e333b0184e14015a42c3c9e9e66d2464aaa66eca48b29e185a6a18f67ab6d",
"binsha": "17e2e271e65f793a9fbe81eab887b941e9d680abe82d5a0602888c50f5e0cac9",
"zipsha": "6873cbdabf23062aeefed6de5f00ff382710332af3ab90a48c253ea17e08f465",
"binsha": "28da9b99f81296fe297831f3bc9d92aea43b4a92826b8ff04ba433b8cb92fb50",
},
"arm64": PROMTAIL_ARM_BINARY,
"aarch64": PROMTAIL_ARM_BINARY,
}

# Paths in `charm` container
Expand Down Expand Up @@ -1585,7 +1601,8 @@ def __init__(
the Loki API endpoint to push logs. It is intended for workloads that can speak
loki_push_api (https://grafana.com/docs/loki/latest/api/#push-log-entries-to-loki), such
as grafana-agent.
(If you only need to forward a few workload log files, then use LogProxyConsumer.)
(If you need to forward workload stdout logs, then use LogForwarder; if you need to forward
log files, then use LogProxyConsumer.)
`LokiPushApiConsumer` can be instantiated as follows:
Expand Down Expand Up @@ -1760,6 +1777,9 @@ class LogProxyEvents(ObjectEvents):
class LogProxyConsumer(ConsumerBase):
"""LogProxyConsumer class.
> Note: This object is deprecated. Consider migrating to LogForwarder with the release of Juju
> 3.6 LTS.
The `LogProxyConsumer` object provides a method for attaching `promtail` to
a workload in order to generate structured logging data from applications
which traditionally log to syslog or do not have native Loki integration.
Expand Down Expand Up @@ -1831,7 +1851,12 @@ def __init__(

# architecture used for promtail binary
arch = platform.processor()
self._arch = "amd64" if arch == "x86_64" else arch
if arch in ["x86_64", "amd64"]:
self._arch = "amd64"
elif arch in ["aarch64", "arm64", "armv8b", "armv8l"]:
self._arch = "arm64"
else:
self._arch = arch

events = self._charm.on[relation_name]
self.framework.observe(events.relation_created, self._on_relation_created)
Expand Down
38 changes: 22 additions & 16 deletions lib/charms/observability_libs/v1/cert_handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@

import logging

from ops.charm import CharmBase, RelationBrokenEvent
from ops.charm import CharmBase
from ops.framework import EventBase, EventSource, Object, ObjectEvents
from ops.jujuversion import JujuVersion
from ops.model import Relation, Secret, SecretNotFoundError
Expand All @@ -67,7 +67,7 @@

LIBID = "b5cd5cd580f3428fa5f59a8876dcbe6a"
LIBAPI = 1
LIBPATCH = 9
LIBPATCH = 11

VAULT_SECRET_LABEL = "cert-handler-private-vault"

Expand Down Expand Up @@ -260,7 +260,13 @@ def retrieve(self) -> Dict[str, str]:

def clear(self):
"""Clear the vault."""
self._backend.clear()
try:
self._backend.clear()
except SecretNotFoundError:
# guard against: https://github.com/canonical/observability-libs/issues/95
# this is fine, it might mean an earlier hook had already called .clear()
# not sure what exactly the root cause is, might be a juju bug
logger.debug("Could not clear vault: secret is gone already.")


class CertHandler(Object):
Expand All @@ -274,6 +280,7 @@ def __init__(
*,
key: str,
certificates_relation_name: str = "certificates",
peer_relation_name: str = "peers",
cert_subject: Optional[str] = None,
sans: Optional[List[str]] = None,
):
Expand All @@ -285,7 +292,11 @@ def __init__(
charm: The owning charm.
key: A manually-crafted, static, unique identifier used by ops to identify events.
It shouldn't change between one event to another.
certificates_relation_name: Must match metadata.yaml.
certificates_relation_name: Name of the certificates relation over which we obtain TLS certificates.
Must match metadata.yaml.
peer_relation_name: Name of a peer relation used to store our secrets.
Only used on older Juju versions where secrets are not supported.
Must match metadata.yaml.
cert_subject: Custom subject. Name collisions are under the caller's responsibility.
sans: DNS names. If none are given, use FQDN.
"""
Expand All @@ -309,7 +320,7 @@ def __init__(
# self.framework.observe(self.charm.on.secret_remove, self._rotate_csr)

else:
vault_backend = _RelationVaultBackend(charm, relation_name="peers")
vault_backend = _RelationVaultBackend(charm, relation_name=peer_relation_name)
self.vault = Vault(vault_backend)

self.certificates_relation_name = certificates_relation_name
Expand Down Expand Up @@ -339,10 +350,6 @@ def __init__(
self.certificates.on.all_certificates_invalidated, # pyright: ignore
self._on_all_certificates_invalidated,
)
self.framework.observe(
self.charm.on[self.certificates_relation_name].relation_broken, # pyright: ignore
self._on_certificates_relation_broken,
)
self.framework.observe(
self.charm.on.upgrade_charm, # pyright: ignore
self._on_upgrade_charm,
Expand Down Expand Up @@ -514,7 +521,7 @@ def _csr(self) -> Optional[str]:
# ignoring all but the last one.
if len(csrs) > 1:
logger.warning(
"Multiple CSRs found in `certificates` relation. "
f"Multiple CSRs found in {self.certificates_relation_name!r} relation. "
"cert_handler is not ready to expect it."
)

Expand Down Expand Up @@ -569,14 +576,13 @@ def _on_certificate_invalidated(self, event: CertificateInvalidatedEvent) -> Non
self.on.cert_changed.emit() # pyright: ignore

def _on_all_certificates_invalidated(self, _: AllCertificatesInvalidatedEvent) -> None:
# Do what you want with this information, probably remove all certificates
# Note: assuming "limit: 1" in metadata
self._generate_csr(overwrite=True, clear_cert=True)
self.on.cert_changed.emit() # pyright: ignore

def _on_certificates_relation_broken(self, _: RelationBrokenEvent) -> None:
"""Clear all secrets data when removing the relation."""
# Note: assuming "limit: 1" in metadata
# The "certificates_relation_broken" event is converted to "all invalidated" custom
# event by the tls-certificates library. Per convention, we let the lib manage the
# relation and we do not observe "certificates_relation_broken" directly.
self.vault.clear()
# We do not generate a CSR here because the relation is gone.
self.on.cert_changed.emit() # pyright: ignore

def _check_juju_supports_secrets(self) -> bool:
Expand Down
Loading

0 comments on commit e8abede

Please sign in to comment.