Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logs API to be user-facing #4225

Closed
wants to merge 24 commits into from
Closed
Show file tree
Hide file tree
Changes from 23 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ release.
([#4203](https://github.com/open-telemetry/opentelemetry-specification/pull/4203))
- Make all fields as identifying for Logger. Previously attributes were omitted from being identifying.
([#4161](https://github.com/open-telemetry/opentelemetry-specification/pull/4161))
- Make Logs Bridge API as the Logs API and do not discourage from direct usage.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this PR should be about either

  • Merging the entire Logs Bridge API AND the Events API (as is) and marking ALL as in-progress
  • Creating a new EMPTY Logs API stating with comments in the Events and Bridge API stating that the new Logs API will encompass ALL of the functionality from the currently represented in the current API's (with the exact details to be worked on)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

creating a new logs/api.md and leaving both the existing (stable) logs/bridge-api.md and (experimental) logs/event-api.md alone sounds like a good path forward.

the logs/api.md page can explain that one of its goals is to replace the Event API and possibly also the Bridge API.

Copy link
Member Author

@pellared pellared Sep 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer not creating yet another API. The specification was foreseeing that Logs Bridge API may be evolved into Logs API. I thing that having them separated in the specification will make it harder to follow and maintain. This is the main motivation behind this PR.

I strongly prefer having a separate, alternate PR instead of changing this one. @MSNev, can you create a new PR?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer not creating yet another API. The specification was foreseeing that Logs Bridge API may be evolved into Logs API. I thing that having them separated in the specification will make it harder to follow and maintain. This is the main motivation behind this PR.

makes sense, do you think we need to change the page to Mixed stability and call out anything?

Copy link
Member Author

@pellared pellared Sep 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there are good reasons to have the changes as Development then yes. I did my best to limit the scope so that it may be seen as acceptable in the Stable portion of the specification.

Copy link
Member Author

@pellared pellared Sep 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I should mark Convenience section as Development as this is a new "functional' addition to logs. The rest is more about the purpose of the API.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added a topic to tomorrow's Spec SIG agenda:

  • Any gotchas to renaming the “Logs Bridge API” to “Logs API” inside of a stable document?

([#4225](https://github.com/open-telemetry/opentelemetry-specification/pull/4225))

### Events

Expand Down
6 changes: 3 additions & 3 deletions specification/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,13 +28,13 @@ path_base_for_github_subdir:
- [Baggage](baggage/api.md)
- [Tracing](trace/api.md)
- [Metrics](metrics/api.md)
- [Logs](logs/README.md)
- [Bridge API](logs/bridge-api.md)
- [Event API](logs/event-api.md)
- [Logs](logs/api.md)
- [Events](logs/event-api.md)
- SDK Specification
- [Tracing](trace/sdk.md)
- [Metrics](metrics/sdk.md)
- [Logs](logs/sdk.md)
- [Events](logs/event-sdk.md)
- [Resource](resource/sdk.md)
- [Configuration](configuration/README.md)
- Data Specification
Expand Down
4 changes: 2 additions & 2 deletions specification/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,7 @@ concept the attributes of the scope cannot change at runtime.

This refers to the `name` and (optional) `version` arguments specified when
creating a new `Tracer` or `Meter` (see
[Obtaining a Tracer](trace/api.md#tracerprovider)/[Obtaining a Meter](metrics/api.md#meterprovider))/[Obtaining a Logger](logs/bridge-api.md#loggerprovider).
[Obtaining a Tracer](trace/api.md#tracerprovider)/[Obtaining a Meter](metrics/api.md#meterprovider))/[Obtaining a Logger](logs/api.md#loggerprovider).
The name/version pair identifies the
[Instrumentation Scope](#instrumentation-scope), for example the
[Instrumentation Library](#instrumentation-library) or another unit of
Expand Down Expand Up @@ -250,7 +250,7 @@ recommended to call it out specifically.
### Log Appender / Bridge

A log appender or bridge is a component which bridges logs from an existing log
API into OpenTelemetry using the [Log Bridge API](./logs/bridge-api.md). The
API into OpenTelemetry using the [Logs (Bridge) API](./logs/api.md). The
terms "log bridge" and "log appender" are used interchangeably, reflecting that
these components bridge data into OpenTelemetry, but are often called appenders
in the logging domain.
34 changes: 17 additions & 17 deletions specification/logs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,16 +140,18 @@ Given the above state of the logging space we took the following approach:
OpenTelemetry log data model. OpenTelemetry Collector can read such logs and
translate them to OpenTelemetry log data model.

- OpenTelemetry defines a Logs Bridge API
for [emitting LogRecords](./bridge-api.md#emit-a-logrecord). Application developers are
NOT encouraged to call this API directly. It is provided for library authors
to build [log appender](../glossary.md#log-appender--bridge), which use
the API to bridge between existing logging libraries and the OpenTelemetry log
data model. Existing logging libraries generally provide a much richer set of
features than what is defined in OpenTelemetry. It is NOT a goal of
OpenTelemetry to ship a feature-rich logging library.

- OpenTelemetry defines an [SDK](./sdk.md) implementation of the [Bridge API](./bridge-api.md),
- OpenTelemetry defines a Logs API for [emitting LogRecords](./api.md#emit-a-logrecord).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This NEEDS to include comments about also emitting events as that is the intention

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is described in API and Data Model. Do we need to repeat it? Can you suggest a change?

It is provided for library authors to build [log appenders](../glossary.md#log-appender--bridge),
which use the API to bridge between existing logging libraries and the
OpenTelemetry log data model.
The [API](./api.md) is also provided for instrumentation libraries to emit log
records following semantic conventions (also known as OpenTelemetry Events).
pellared marked this conversation as resolved.
Show resolved Hide resolved
Existing logging libraries generally provide a much richer set of features
than what is defined in OpenTelemetry. It is NOT a goal of OpenTelemetry to
ship a feature-rich logging library. However, nothing prevents application
cijothomas marked this conversation as resolved.
Show resolved Hide resolved
developers to use it directly.

- OpenTelemetry defines an [SDK](./sdk.md) implementation of the [API](./api.md),
which enables configuration of [processing](./sdk.md#logrecordprocessor)
and [exporting](./sdk.md#logrecordexporter) LogRecords.

Expand Down Expand Up @@ -230,10 +232,9 @@ processor.
### Infrastructure Logs

These are logs generated by various infrastructure components, such as
Kubernetes events (if you are wondering why events are discussed in the context
of logs see [Event API Data model](./event-api.md#event-data-model)). Like system logs, the
infrastructure logs lack a trace context and can be enriched by the resource
context - information about the node, pod, container, etc.
Kubernetes events. Like system logs, the infrastructure logs lack a trace
context and can be enriched by the resource context - information about
the node, pod, container, etc.

OpenTelemetry Collector or other agents can be used to query logs from most
common infrastructure controllers.
Expand Down Expand Up @@ -359,7 +360,7 @@ as parsers, log tailing and rotation. It also enables the possibility to send
logs directly to the logging backend without using a log collection agent.

To facilitate both approaches described above OpenTelemetry provides
a [Bridge API](./bridge-api.md) and [SDK](./sdk.md), which can be used together with existing
a [API](./api.md) and [SDK](./sdk.md), which can be used together with existing
logging libraries to automatically inject the trace context in the emitted logs,
and provide an easy way to send the logs via OTLP. Instead of
modifying each logging statement, [log appenders](../glossary.md#log-appender--bridge)
Expand Down Expand Up @@ -444,10 +445,9 @@ standard output.

## Specifications

* [Logs Bridge API](./bridge-api.md)
* [Logs API](./api.md)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about having links to the old API's stating that the this new API will encompass the existing functionality.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not want to create yet another API.

* [Logs SDK](./sdk.md)
* [Logs Data Model](./data-model.md)
* [Event API](./event-api.md)
* [Trace Context in non-OTLP Log Formats](../compatibility/logging_trace_context.md)

## References
Expand Down
46 changes: 30 additions & 16 deletions specification/logs/bridge-api.md → specification/logs/api.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Logs Bridge API
# Logs API

**Status**: [Stable](../document-status.md), except where otherwise specified

Expand All @@ -9,6 +9,7 @@

<!-- toc -->

- [Overview](#overview)
- [LoggerProvider](#loggerprovider)
* [LoggerProvider operations](#loggerprovider-operations)
+ [Get a Logger](#get-a-logger)
Expand All @@ -18,21 +19,32 @@
+ [Enabled](#enabled)
- [Optional and required parameters](#optional-and-required-parameters)
- [Concurrency requirements](#concurrency-requirements)
- [Artifact Naming](#artifact-naming)
- [Convenience](#convenience)
- [Logs Bridge API](#logs-bridge-api)
- [References](#references)

<!-- tocstop -->

</details>

<b>Note: this document defines a log *backend* API. The API is not intended
to be called by application developers directly. It is provided for logging
library authors to build
[log appenders](./supplementary-guidelines.md#how-to-create-a-log4j-log-appender),
## Overview

This document defines a log API.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As part of OTep #265 it was determined that there SHOULD be a public facing "Logging" API to enable the emitting of "Events" (as a type of OpenTelemetry Log) as well as other general "Logs" <- I'm implicitly inferring based on discussions that occurred as part of #265 (ie. while all events can be represented as Logs, not all Logs are events)

One of the key words here is type, there are many types of Logs and each type fundamentally defines the "intent" (or the "why") the log is being emitted.

These can include "Event", "Audit", "Security", "Transaction", "Message", "Server", "File" and many, many more, and just as "Events are a type of Log", several of these can also be sub-classified as a type of event.

As part of any Logging API we SHOULD provide a front-end API and guidance for all of the CORE fundamental concepts to "try" to make it clear about what will and won't occur when you do or don't use the specific API.
(Right NOW we don't have this clearly defined (because we are always trying to take small steps) which always lands us into the argument of "But it's just a Log", which is not a productive argument as it ignores the requirements by different teams / use cases that there are different needs and different types of Logs that should have some uniform structure (Semantic Conventions / processing) around them.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the PR so that Logs API can be used to emit events but I have not removed the Events API.

The API serves following use cases.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are many additional features / requirements that exist and are yet to be defined as we move this API forward (some of which was starting to occur in the Event API) as it was moving (or trying to move) towards stability.
Some of these will include different processing requirements for the different types and as it moves towards a stabilized state it may cause implementation changes, so for now (as a small step) I'll try to define a starting point.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should rather require refining the SDK and not the API.

It is provided for logging library authors to build
[log appenders/bridges](./supplementary-guidelines.md#how-to-create-a-log4j-log-appender),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • emitBridgeLog (just making the name different from above -- needs a better one), just here to have the distinction explicit.
    • Exists to "bridge" into the OpenTelemetry Logs pipeline only
    • Used by log Appenders ONLY to create OpenTelemetry LogRecords and have them processed
    • NO special processing (beyond existing Log Processors ??)*
      • (*)There may be some special use cases where some Log Processors may be required to be skipped (like a processor that links OTel Logs -> Native Logging)
      • Should we change this and have "explicit" Appender/Bridge Log Processors
    • IF an existing language / logging library supports the creation of "Events" or specific "Structured" Logs then this API MAY be used to "Simulate" (generate) a LogRecord that looks like those created by the above API's.
      • HOWEVER, this WILL NOT re-apply / perform any special processing that may be configured by the SDK instance for the other API methods
      • So it's entirely 100% up to the appender (caller of this API) to ensure that it is complying with any necessary semantic conventions to avoid any downstream complications around deterministically identifying / processing the resulting LogRecord.
      • It also MUST NOT just convert everything into an Event UNLESS
        • it can EXPLICITLY identify that the intent of the LogRecord was to identify (signal) that the specifically named event occurred vs general Detailed Logging
        • or it create a general semantic convention that identifies that this LogRecord that looks like an event was created by or follows the semantic conventions of X. ie. it has a specific name like log.*, applog.* or event.ilogger again so the intent (type) can be deterministically evaluated AND it would avoid any "clashes" with defined semantic conventions.
        • To phrase another way as long as the Appender "Knows" that the user emitting the LogRecord WANTS this specific log to be called XXXXX then this is 100% acceptable as long as the control (setting) of this name is explicitly defined by the user. eg. Application has their own logger for their own events, but they want to explicitly emit an OpenTelemetry defined event with a Specific name via their own logger and not via the OpenTelemetry emitEvent with the understanding (implicit or otherwise) that any configured OpenTelemetry "Event" processing will NOT be performed via this path.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MSNev can you highlight any specific blocking concerns you have about this PR beyond #4225 (comment), and open tracking issues for the rest? Once we get this initial restructuring PR landed then we'll be able to start sending PRs to try to address those. Thanks.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is up to the bridge implementation how the logging library would be mapped to OTel Logs Data Model. I think we should rather create some "guidance" / "recommendations" for log bridge authors. We cannot control how the bridges are going to be implemented.

For sure this should be out of scope of this PR.

which use this API to bridge between existing logging libraries and the
OpenTelemetry log data model.</b>
OpenTelemetry log data model. See also: [Logs Bridge API](#logs-bridge-api)

It is provided for OpenTelemetry (instrumentation) libraries to emit
[OpenTelemetry Events](data-model.md#events).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe by making Otel Logs API user-facing, we should not limit instrumentation to events. It should be fine to emit nameless log records.

So maybe

Suggested change
[OpenTelemetry Events](data-model.md#events).
logs.

?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe by making Otel Logs API user-facing, we should not limit instrumentation to events. It should be fine to emit nameless log records.

So maybe

?

ABSOLUTELY!

And whether that are called "nameless" or there is some other "name" (not event.name) that describes that type (options discussed include log.name, log.record.name maybe even a new log.type (But the enumeration of this is extremely problematic as what "level" do you define)) of Log entry is specifically being written is just part of the unanswered questions.

Copy link
Member Author

@pellared pellared Sep 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is about OTel instrumentation libraries (not any "instrumentation").

When OpenTelemetry instrumentation emits logs, those logs SHOULD be formatted as Events.

from https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/event-api.md#event-data-model

as well as https://github.com/open-telemetry/oteps/blob/58d6420237091bcfb739f329e09adaa7ec03f702/text/0265-event-vision.md?plain=1#L29-L32

The next paragraph is about using the API to emit any logs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And both of those links talked about them be "represented" by the same Data Model, or as a "type" of Log, it is NOT stating that this makes them the SAME thing (beyond the transport / representation), they still HAVE different Semantics.
Circling back to the OTep this is the HOW are they SENT not WHAT are they.

What is an Event https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/events.md
What is a Log https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/logs.md

Nowhere is it saying that a Log is an Event

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And as an example of additional cases where I'm saying that we are talking past each other on the possible solutions,
Please go and see my comment from Sept 9th IN THE OTEP
open-telemetry/oteps#265 (comment), in the design section and the "Combining API's" (the exact thing you are trying to define here)

Nothing over there was "simply" ranting, it was about trying to inform and provide workable solutions that will work for ALL of US and the community.

Copy link
Member Author

@pellared pellared Sep 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there could also be things I just want to let my users know of via logs. E.g. certain configuration was picked, connection has dropped, etc. I could use 3rd party logging library, but why if I can use OTel logging API without adding a new dependency

How would the instrumentation library report a problem if the OTel Logs Provider is not configured?

Some OTel libraries have already some ways to report problems. E.g. in Go we have https://pkg.go.dev/go.opentelemetry.io/otel#SetErrorHandler.

I am not against the idea, but I think it would increase the impact (and scope) of this PR. How about a separate issue after this one is (hopefully) merged?

Copy link
Contributor

@lmolkova lmolkova Sep 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would the instrumentation library report a problem if the OTel Logs Provider is not configured?

it won't - I mean someone could configure default provider (stdout), but in general case if you use 3rd party logging library, it's not a responsibility of something that emits logs to notify you that your logging provider is not configured.

We can borrow some ideas from logging libs. E.g. SLF4J notifies you if provider is not configured with something like this in stderr:

image

I agree that it's not a viable choice for instrumentation today to rely on OTel logs being configured as a main source of logging, but 3 years from now it could be a nice option to report all telemetry with otel instead of taking dependency on something else.

And we're not restricting anything - we're adding more options.

In any case, I'd like to have this discussion tomorrow on the spec call, but if you feel strongly about making progress on this PR without addressing it, I'm fine doing it as a follow up.

Copy link
Member Author

@pellared pellared Sep 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created #4234 please feel free to update the description to make it more complete

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Imagine I'm instrumenting kafka:

  • there could be some events I'd emit - e.g. cluster leader is unreachable - which are documented in the semconv

  • there could also be things I just want to let my users know of via logs. E.g. certain configuration was picked, connection has dropped, etc. I could use 3rd party logging library, but why if I can use OTel logging API without adding a new dependency?

The library logs are emitted with is not relevant to users - they'd get the same logs if lib used 3rd party logger or OTel API. So I don't see why we should limit instrumentations from using OTel API to emit undocumented regular (but structured) logs.

I think this is a good example. You can emit an untyped event (which is a nameless logrecord), and it would bypass any event pipelines set up, but it would still give you a single api surface at the low level.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW @pellared my main focus with this is native instrumentations where now I need to pick one API for logging and use OTel for everything else. I.e. I don't mean instrumentation logs, I mean the underlying library logs.

The Logs Bridge API consist of these main components:
It is provided for application developers to emit structured log records
(including OpenTelemetry Events).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should call out the "difference" between the processing / handling / semantics of an event vs a structured log

  • emitLog
    • Used to emit a structured Log (without (necessarily) any implicit meaning
    • These logs will include type "Server", "Transaction", "Message", "Audit", "Debug", "Diagnostic" etc.
    • To be used to provide "Detailed Information" which may be used to understand operational issues, often used for post mortem analysis rather than operational dashboards or monitoring.
      • Should be used by Applications, Instrumentations and SDK implementations.
    • If there is any explicit event processing, this API won't perform any explicit processing beyond general common Log Processors

Problem Statement and confusion

  • As "structured" logs generally also need some form of "identifier" about the "type", so these will also need some form of name, schema or identifying attributes as these (vs general unstructured logging) generally have some semantic conventions on their structure and beyond just have "OTLP" AnyValues processors will likely want a simple mechanism for identifying them.
  • The MUST however, NOT be identified in the same manner as events, as then there is no deterministic mechanism for identifying "How" the type of LogRecord was produced (as an Event or as a Detailed Information record).
  • As such should this include or be restricted to have a general (optional) name
  • If a Language / runtime has a standard logging library with a supported log appender then there will be confusion between using this API vs their specific (preferred) logging library.
    • The difference should be isolated around configured specific Log Processing (if any) -- which of course doesn't exist today.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should call out the "difference" between the processing / handling / semantics of an event vs a structured log

I find it as out of scope. We need such mechanism for all log records. From last SIG meeting agenda:

We need filtering and separate log processing pipelines. E.g. distinct processing for different exporters or different processing for events and other log records. Also needed for Specify how Logs SDK implements Enabled #4207

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Am I stating anywhere that we don't -- NO.
But I am stating that we ALSO need this for Events (The Event API has this for that exact purpose), we could have "just" ignored it and said hey go just use a genera Log processor on your Log definition.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

based on the current version of spec, it's totally valid to emit events using log bridge. We don't have a requirement to support separate pipelines today and this PR does not change status quo

If a logging library is capable of creating logs which correctly map
to the Event data model, logging in this manner is also an acceptable way to
create Events.


The Logs API consists of these main components:

* [LoggerProvider](#loggerprovider) is the entry point of the API. It provides access to `Logger`s.
* [Logger](#logger) is responsible for emitting logs as
Expand Down Expand Up @@ -167,21 +179,23 @@ provide it.

## Concurrency requirements

For languages which support concurrent execution the Logs Bridge APIs provide
For languages which support concurrent execution the Logs APIs provide
specific guarantees and safeties.

**LoggerProvider** - all methods are safe to be called concurrently.

**Logger** - all methods are safe to be called concurrently.

## Artifact Naming
## Convenience

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is PART of the issue with this process.

Events are MORE than just a Convenience!

I would be fine with wording along the lines of

For languages that do not need or want to provide any OpenTelemetry Event support because they have their own native or common libraries, then they MAY choose to not implement the emitEvent functionality and support.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Events are MORE than just a Convenience

I find this as an opinionated statement based on the current definition of Opentelemetry Events

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find this as an opinionated statement based on the current definition of Opentelemetry Events

And THAT is the crux of the disagreement

You (and several others) DON'T believe and either DON'T understand or WANT to understand that "Events" are NOT JUST Logs, so your Opinionated view / statement is to remove something that you DON'T understand.

As stated in the Oel (this portion I agree with)

OpenTelemetry Events are a type of OpenTelemetry Log that requires an event name and follows a specific structure implied by that event name.

They are a TYPE of Log, it does not state that ALL Logs are Events and it does not state the there is ONLY 1 TYPE of Log (Events). Calling Every LOG and Event destroys the Semantic meaning of what is an Event.

That does not mean that there isn't room to "define" the appropriate Semantics to keep Both, but that is the discussions and part of the agreements that we need to make BEFORE we can conclude this PR (as I've mentioned eariler)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we call this section "Optional methods" so it works for everyone?

Copy link
Member Author

@pellared pellared Sep 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather remove the whole section.

I think it is needed not only for sake of events but e.g. Java may want to have convenient methods like Info, Warn, Error. I find a EmitEvent as a similar convience method as it is not necessary for the users that want to emit events.

Calling Every LOG and Event destroys the Semantic meaning of what is an Event.

Nobody does it. Currently, just call every log with event.name attribute as an Event. For the SDK/Data Model/Collector perspective it does not matter how it was created. This is currently the way an Event is defined and I am not making any change to it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another problem, I see in defining Emit an Event method (in scope of this PR) is that there would be an ambiguity what should happen if a user adds an additional event.name attribute. Should the name parameter or event.name attribute have precedence?

PS. The same problem currently exists in https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/event-api.md#emit-event

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created #4233

And also committed cc0f262 to avoid this issue in scope of this PR

Languages MAY provide additional ergonomics and convinence APIs. For instance,
a language may provide an `Info` method for `Logger` which emits a log record
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Languages MAY provide additional ergonomics and convinence APIs. For instance,
Languages MAY provide additional ergonomics and convenience APIs. For instance,

with `SeverityNumber` equal to `9`.

## Logs Bridge API

The Logs Bridge API is not intended to be called by application developers
directly, and SHOULD include documentation that discourages direct use. However,
in the event OpenTelemetry were to add a user facing API, the Logs Bridge API would
be a natural starting point. Therefore, Log Bridge API artifact, package, and class
names MUST NOT include the terms "bridge", "appender", or any other qualifier
that would prevent evolution into a user facing API.
Languages MAY provide a separate Logs Bridge API if they need different
ergonomics for consumers that are building log appenders/bridges.

## References

Expand Down
23 changes: 23 additions & 0 deletions specification/logs/data-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
* [Field: `InstrumentationScope`](#field-instrumentationscope)
* [Field: `Attributes`](#field-attributes)
+ [Errors and Exceptions](#errors-and-exceptions)
* [Events](#events)
- [Example Log Records](#example-log-records)
- [Example Mappings](#example-mappings)
- [References](#references)
Expand Down Expand Up @@ -466,6 +467,28 @@ of the record.
If included, they MUST follow the OpenTelemetry
[semantic conventions for exception-related attributes](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/exceptions/exceptions-logs.md).

### Events
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this section be marked with stability Development?


Wikipedia’s [definition of log file](https://en.wikipedia.org/wiki/Log_file):

>In computing, a log file is a file that records either events that occur in an
>operating system or other software runs.

From OpenTelemetry's perspective LogRecords and Events are both represented
using the same [data model](./data-model.md). An Event is a specialized type
of LogRecord, not a separate concept.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the not a separate concept or rephase that OpenTelemetry encapsulates and transmits as a LogRecord

From OpenTelemetry's perspective LogRecords and Events are both represented
using the same data model. As such an Event is sent as a LogRecord
with an identifying event.name attribute.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. This is an extract from Events API. I have not changed a single word. Out of scope.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the framing of the Event API as a separate entity and therefore the creation and sending of the resulting Event as a LogRecord (and not a separate Event concept) it make sense there. If does NOT make sense as part of folding into a SINGLE Api which emits both because now they ARE 2 separate concepts that are emitted from the same API.

Copy link
Member Author

@pellared pellared Sep 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they ARE 2 separate concepts

(As of today's understanding of the specification) They are not. Event is a specialization of a LogRecord. I never have seen in spec that Events and Logs are separate signals (this would mean for me that these are separate concepts).


Events are OpenTelemetry's standardized semantic formatting for LogRecords.
Beyond the structure provided by the LogRecord data model, it is helpful for
logs to have a common format within that structure. When OpenTelemetry
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And this is the crux of the problem

When OpenTelemetry instrumentation emits logs, those logs SHOULD be formatted as Events.

While Events are represented as LogRecords not all Logs are Events

Copy link
Member Author

@pellared pellared Sep 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not follow what this comment is about

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again the crux of the issue that needs to be discussed during the meeting synchronously and not asynchronously where everyone only has a narrow view of these things.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See not just my comments but also @jkwatson open-telemetry/oteps#265 (comment) in the OTep.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to be a copy-paste from the existing

Events are OpenTelemetry's standardized semantic formatting for LogRecords.
Beyond the structure provided by the LogRecord data model, it is helpful for
logs to have a common format within that structure. When OpenTelemetry
instrumentation emits logs, those logs SHOULD be formatted as Events. All
semantic conventions defined for logs MUST be formatted as Events.

but I wonder if we should allow instrumentations to emit logs or events now that logging api is going to be user-facing.

Left another comment on it - https://github.com/open-telemetry/opentelemetry-specification/pull/4225/files#r1781573289

instrumentation emits logs, those logs SHOULD be formatted as Events. All
semantic conventions defined for logs MUST be formatted as Events.

The Event format is as follows. All Events have a
[`event.name` attribute](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/events.md),
and all Events with the same `event.name` MUST conform to the same schema for
both their `Attributes` and their `Body`.

## Example Log Records

For example log records see
Expand Down
35 changes: 7 additions & 28 deletions specification/logs/event-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@

<!-- toc -->
cijothomas marked this conversation as resolved.
Show resolved Hide resolved

- [Event Data model](#event-data-model)
- [Event API use cases](#event-api-use-cases)
- [EventLoggerProvider](#eventloggerprovider)
* [EventLoggerProvider operations](#eventloggerprovider-operations)
Expand All @@ -30,35 +29,13 @@ The Event API consists of these main components:
provides access to `EventLogger`s.
* [EventLogger](#eventlogger) is the component responsible for emitting events.

## Event Data model

Wikipedia’s [definition of log file](https://en.wikipedia.org/wiki/Log_file):

>In computing, a log file is a file that records either events that occur in an
>operating system or other software runs.

From OpenTelemetry's perspective LogRecords and Events are both represented
using the same [data model](./data-model.md). An Event is a specialized type
of LogRecord, not a separate concept.

Events are OpenTelemetry's standardized semantic formatting for LogRecords.
Beyond the structure provided by the LogRecord data model, it is helpful for
logs to have a common format within that structure. When OpenTelemetry
instrumentation emits logs, those logs SHOULD be formatted as Events. All
semantic conventions defined for logs MUST be formatted as Events.

The Event format is as follows. All Events have a
[`event.name` attribute](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/events.md),
and all Events with the same `event.name` MUST conform to the same schema for
both their `Attributes` and their `Body`.

## Event API use cases

The Events API was designed to allow shared libraries to emit high quality
logs without needing to depend on a third party logger. Unlike the
[Logs Bridge API](./bridge-api.md), instrumentation authors and application
developers are encouraged to call this API directly. It is appropriate to
use the Event API when these properties fit your requirements:
The Events API was designed to allow shared libraries to emit
[events](data-model.md#events) without needing to depend on a third party logger.
Comment on lines +34 to +35
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps this part is changing now.

Maybe

Suggested change
The Events API was designed to allow shared libraries to emit
[events](data-model.md#events) without needing to depend on a third party logger.
The Events API was designed to allow shared libraries and applications to emit named and structured logs with consistent semantics.

?

Or maybe just remove this sentence?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. We can the apply your suggestion if there are no counter arguments.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was designed to emit named and structured EVENTS that happen to be sent / represented as a LogRecord, they are still EVENTS and not JUST A LOG!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MSNev can you make a suggestion that would work for you? The original sentence is no longer applicable, what would be a good replacement?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But they are still logs 😉

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it no longer applicable, the original sentence is still valid. If the Events API is being merged into a new common Log API then nothing has changed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @MSNev that the current sentence is still applicable/valid. @lmolkova, how about resolving?

Copy link
Contributor

@lmolkova lmolkova Sep 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's not relevant since with this PR we will have user-facing logging API.

I.e. the second part of this sentence no longer makes sense: "The Events API was designed to allow shared libraries to emit events without needing to depend on a third party logger"

Copy link
Member Author

@pellared pellared Sep 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then maybe

Suggested change
The Events API was designed to allow shared libraries to emit
[events](data-model.md#events) without needing to depend on a third party logger.
The Events API was designed to allow shared libraries to emit
[events](data-model.md#events).

Copy link
Contributor

@lmolkova lmolkova Sep 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this sounds precise and resolves my concern.

I feel this statement ("X API was designed to .. emit X") does not carry any useful information though😄 but if you believe that it does, I don't have any objections against it

Instrumentation authors and application developers are encouraged can call this
cijothomas marked this conversation as resolved.
Show resolved Hide resolved
API directly. It is appropriate to use the Event API when these properties fit
your requirements:

* Logging from a shared library that must run in many applications.
* A semantic convention needs to be defined. We do not define semantic
Expand All @@ -80,6 +57,8 @@ If a logging library is capable of creating logs which correctly map
to the Event data model, logging in this manner is also an acceptable way to
create Events.

Emitting events via [Logs API](api.md) is also acceptable.

## EventLoggerProvider

`EventLogger`s can be accessed with a `EventLoggerProvider`.
Expand Down
8 changes: 3 additions & 5 deletions specification/logs/event-sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,6 @@ OpenTelemetry API to actually produce telemetry. The OpenTelemetry SDK
(henceforth referred to as the SDK) is an implementation of the OpenTelemetry
API that provides users with this functionally.

All implementations of the OpenTelemetry API MUST provide an SDK.

## Overview

From OpenTelemetry's perspective LogRecords and Events are both represented
Expand Down Expand Up @@ -81,7 +79,7 @@ Emit a `LogRecord` representing an `Event`.
**Implementation Requirements:**

The implementation MUST use the parameters
to [emit a logRecord](./bridge-api.md#emit-a-logrecord) as follows:
to [emit a logRecord](./api.md#emit-a-logrecord) as follows:

* The `Name` MUST be used to set
the `event.name` [Attribute](./data-model.md#field-attributes). If
Expand All @@ -94,10 +92,10 @@ to [emit a logRecord](./bridge-api.md#emit-a-logrecord) as follows:
the [Timestamp](./data-model.md#field-timestamp). If not provided, `Timestamp`
MUST be set to the current time when [emit](#emit-event) was called.
* The [Observed Timestamp](./data-model.md#field-observedtimestamp) MUST not be
set. (NOTE: [emit a logRecord](./bridge-api.md#emit-a-logrecord) will
set. (NOTE: [emit a logRecord](./api.md#emit-a-logrecord) will
set `ObservedTimestamp` to the current time when unset.)
* If provided by the user, the `Context` MUST be used to set
the [Context](./bridge-api.md#emit-a-logrecord). If not provided, `Context`
the [Context](./api.md#emit-a-logrecord). If not provided, `Context`
MUST be set to the current Context.
* If provided by the user, the `SeverityNumber` MUST be used to set
the [Severity Number](./data-model.md#field-severitynumber) when emitting the
Expand Down
10 changes: 5 additions & 5 deletions specification/logs/noop.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
linkTitle: No-Op
--->

# Logs Bridge API No-Op Implementation
# Logs API No-Op Implementation

**Status**: [Stable](../document-status.md)

Expand All @@ -23,12 +23,12 @@ linkTitle: No-Op
Users of OpenTelemetry need a way to disable the API from actually
performing any operations. The No-Op OpenTelemetry API implementation
(henceforth referred to as the No-Op) provides users with this
functionally. It implements the [OpenTelemetry Logs Bridge API](./bridge-api.md)
functionally. It implements the [OpenTelemetry Logs API](./api.md)
so that no telemetry is produced and computation resources are minimized.

All language implementations of OpenTelemetry MUST provide a No-Op.

The [Logs Bridge API](./bridge-api.md) defines components with various operations.
The [Logs API](./api.md) defines components with various operations.
All No-Op components MUST NOT hold configuration or operational state. All No-op
operations MUST accept all defined parameters, MUST NOT validate any arguments
received, and MUST NOT return any non-empty error or log any message.
Expand All @@ -42,7 +42,7 @@ provide the same `LoggerProvider` instances to all creation requests.

### Logger Creation

New `Logger` instances are always created with a [LoggerProvider](./bridge-api.md#loggerprovider).
New `Logger` instances are always created with a [LoggerProvider](./api.md#loggerprovider).
Therefore, `LoggerProvider` MUST allow for the creation of `Logger`s.
All `Logger`s created MUST be an instance of the [No-Op Logger](#logger).

Expand All @@ -54,4 +54,4 @@ return the same `Logger` instances to all creation requests.
### Emit LogRecord

The No-Op `Logger` MUST allow
for [emitting LogRecords](./bridge-api.md#emit-a-logrecord).
for [emitting LogRecords](./api.md#emit-a-logrecord).
Loading
Loading