Skip to content

Commit

Permalink
docs(auditlogs): minor changes in wording
Browse files Browse the repository at this point in the history
  • Loading branch information
mlenkeit committed Nov 22, 2024
1 parent 8b38626 commit 066501b
Showing 1 changed file with 13 additions and 12 deletions.
25 changes: 13 additions & 12 deletions projects/audit-logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Audit logging describes the capability of capturing audit-trail relevant events

Unlike regular application logs, audit logs are usually subject to long retention periods and software providers must guarantee their completeness (i.e. guarantee of delivery).

Examples of audit logs include: (see [Appendix B: Sample Audit Log Events])
Examples of audit logs include: (see [Appendix B: Examples of Audit Log Events])
- failed login attempts
- permission changes (e.g. of a service account or application user)
- accessing sensitive information
Expand All @@ -18,7 +18,7 @@ OpenTelemetry does not have a good solution for audit logging

- no semantic conventions for audit logs in OTel
- OTel APIs/SDKs do not provide feedback to the application level whether data (in particular logs) have been successfully delivered to a remote endpoint. To guarantee delivery, either the SDK has to give those guarantees, or provide feedback to the application so that it can take care of guaranteed delivery itself.
- OTel collectors may lose audit logs in transit (i.e. no guarantee of delivery)
- OTel Collector instances may lose audit logs in transit (i.e. no guarantee of delivery)

See [Appendix A: Guarantee of Delivery] for more details

Expand All @@ -29,15 +29,15 @@ The goal of this project is to make OTel fit for audit logging purposes that mee
- REQ-CONV-01: Semantic conventions for application-level audit logs are defined

Check warning on line 29 in projects/audit-logging.md

View workflow job for this annotation

GitHub Actions / spelling-check

Unknown word (CONV)
- REQ-CONV-02: Semantic conventions for infrastructure-level audit logs are defined

Check warning on line 30 in projects/audit-logging.md

View workflow job for this annotation

GitHub Actions / spelling-check

Unknown word (CONV)
- REQ-APPL-01: Guaranteed delivery of audit logs exported via OpenTelemetry SDK

Check warning on line 31 in projects/audit-logging.md

View workflow job for this annotation

GitHub Actions / spelling-check

Unknown word (APPL)
- REQ-PIPE-01: OTel collector must provide guaranteed delivery of audit logs, including when its process is interrupted
- REQ-PIPE-01: OTel Collector instances must provide guaranteed delivery of audit logs, including when its process is interrupted

See [Appendix A: Guarantee of Delivery] for more details

## Deliverables

- semantic convention for audit logs
- extend OTel APIs/SDKs for audit logging purposes (in collaboration with the respective SIG)
- extend OTel collector for audit logging purposes (in collaboration with the respective SIG)
- extend OTel Collector for audit logging purposes (in collaboration with the respective SIG)

## Staffing / Help Wanted

Expand All @@ -54,8 +54,9 @@ Other vendors are invited to join the discussion.
- @reyang
- tbd
* GC liaison: @svrnm
* Engineers:
* Engineers for API/SDK:
* SAP will provide a prototype in two languages (tbd; likely two of Java, JavaScript, Go)
* Engineers for OTel Collector: tbd
* Maintainers/approvers: tbd

## Timeline
Expand Down Expand Up @@ -84,23 +85,23 @@ Messaging protocols that support different levels of delivery guarantees may ref

We assume that every component that is involved in the delivery of audit logs from source to destination must support guarantee of delivery individually, rather than assuming that this ability can be provided by e.g. only the collector or SDK.

The implications of guarantee of delivery can be illustrated with an example consisting of a workload, an OTel collector, and a durable storage. The workload acts as the source and produces audit logs via the OTel SDK/API. It writes the data via OTLP to the OTel collector. The OTel collector is configured to export audit logs to a durable storage that acts as the destination such as an S3 bucket.
The implications of guarantee of delivery can be illustrated with an example consisting of a workload, an OTel Collector instance, and a durable storage. The workload acts as the source and produces audit logs via the OTel API/SDK. It writes the data via OTLP to the collector. The collector is configured to export audit logs to a durable storage that acts as the destination such as an S3 bucket.

The following implications would apply:

- workload produces an audit-relevant event:

The workload emits the event via the OTel SDK/API. It may wait for acknowledgement of receipt from the collector before proceeding. If the event is rejected or receipt is not acknowledged in time, the workload or SDK may act accordingly, e.g. retry, rollback a database transaction, inform the user, etc.
The workload emits the event via the OTel API/SDK. It may wait for acknowledgement of receipt from the collector before proceeding. If the event is rejected or receipt is not acknowledged in time, the workload or SDK may act accordingly, e.g. retry, rollback a database transaction, inform the user, etc.

- OTel collector receives the event:
- OTel Collector receives the event:

To ensure that the event is not lost even if the collector process is terminated or crashes, the collector may need to persist the event before acknowledging receipt to the workload or SDK. If the event cannot be persisted, receipt must be rejected.

- OTel collector exports the event:
- OTel Collector exports the event:

Once the event is exported and the target (i.e. S3) acknowledges receipt, the event can dropped from the collector's persistence.

- S3 receives the event:
- the target (i.e. S3) receives the event:

Acknowledges receipt after persisting the event.

Expand All @@ -112,7 +113,7 @@ The example is kept simple for illustration purposes. Many edge cases need to be

It may turn out that all OTel receivers, processors, or exporters can be made compatible with guarantee of delivery for audit logging purposes.

### Appendix B: Sample Audit Log Events
### Appendix B: Examples of Audit Log Events

The following list contains sample audit log events in a YAML format for better readability and intentionally do not follow any OTel-related schema.

Expand Down Expand Up @@ -193,4 +194,4 @@ An event consists of the event name, event-specific data, and general metadata.
<!-- links -->
[Appendix A: Guarantee of Delivery]: #appendix-a-guarantee-of-delivery
[Appendix B: Sample Audit Log Events]: #appendix-b-sample-audit-log-events
[Appendix B: Examples of Audit Log Events]: #appendix-b-examples-of-audit-log-events

0 comments on commit 066501b

Please sign in to comment.