Blog post for OpenTelemetry Generative AI updates

[drewby]

Title: OpenTelemetry for Generative AI

This blog post introduces enhancements to OpenTelemetry specifically tailored for generative AI technologies, focusing on the development of Semantic Conventions and the Python Instrumentation Library.

Samples are in Python

SIG: GenAI Observability

Sponsors: @tedsuo @lmolkova

Closes: #5581

[theletterf]

Cool one! Added some comments.

[codefromthecrypt]

The below is about if we can make it zero code or not, and what's remaining to do so:

---

I want to run dotenv run -- opentelemetry-instrument python main.py and not have any telemetry code in the foreground or even noticed (zero code approach)

I have a similar example, I've tried locally, and I don't see a way to implicitly configure the logging provider, yet. I'm not sure if we want to make a hybrid to reduce the amount of code or just leave the explicit tracing and logging stuff in until logging can be env configured.

cc @anuraaga and @xrmx in case I got below wrong also

requirements

openai~=1.54.3

opentelemetry-sdk==1.28.1
opentelemetry-exporter-otlp-proto-http==1.28.1
opentelemetry-distro==0.49b1
opentelemetry-instrumentation-openai-v2==2.0b0

env

# Override default ENV variables for Ollama
OPENAI_BASE_URL=http://localhost:11434/v1
OPENAI_API_KEY=unused
CHAT_MODEL=qwen2.5:0.5b

# Set to false or remove to disable log events
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_SERVICE_NAME=python-opentelemetry-openai
OTEL_METRIC_EXPORT_INTERVAL=100
OTEL_BSP_SCHEDULE_DELAY=100

Best I could manage was to add hooks only for the log/event stuff

import os

from openai import OpenAI

# NOTE: OpenTelemetry Python Logs and Events APIs are in beta
from opentelemetry import _logs, _events
from opentelemetry.sdk._logs import LoggerProvider
from opentelemetry.sdk._events import EventLoggerProvider

from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
from opentelemetry.exporter.otlp.proto.http._log_exporter import OTLPLogExporter

_logs.set_logger_provider(LoggerProvider())
_logs.get_logger_provider().add_log_record_processor(BatchLogRecordProcessor(OTLPLogExporter()))
_events.set_event_logger_provider(EventLoggerProvider())

def main():
    client = OpenAI()
    messages = [
        {
            "role": "user",
            "content": "Answer in up to 3 words: Which ocean contains the falkland islands?",
        },
    ]
    model = os.getenv("CHAT_MODEL", "gpt-4o-mini")
    chat_completion = client.chat.completions.create(model=model, messages=messages)
    print(chat_completion.choices[0].message.content)


if __name__ == "__main__":
    main()

then, I get a warning about overriding the event provider, but at least the events do show up

$ dotenv run -- opentelemetry-instrument python main.py
Overriding of current EventLoggerProvider is not allowed
Indian Ocean

[codefromthecrypt]

I guess what's out of scope is metrics, as they aren't implemented yet. One concern is that folks follow this, then later when metrics are implemented, they need more instructions, or people need to remember to go back and change the docs etc.

Since there are recent developments like asyncapi etc going in fairly quickly, if metrics were added quickly also, would it make sense to hold the blog until they are released? or would it make more sense to do a second blog and revisit the setup instructions once that's supported?

[drewby]

I guess what's out of scope is metrics, as they aren't implemented yet. One concern is that folks follow this, then later when metrics are implemented, they need more instructions, or people need to remember to go back and change the docs etc.

Since there are recent developments like asyncapi etc going in fairly quickly, if metrics were added quickly also, would it make sense to hold the blog until they are released? or would it make more sense to do a second blog and revisit the setup instructions once that's supported?

Metrics are discussed as part of the semantic conventions, but correct, they are not yet implemented in the library yet. I think it's worth getting an article out there earlier than later. We might even attract some contributors for the Metrics implementation.

[codefromthecrypt]

We might even attract some contributors for the Metrics implementation.

good point!

[svrnm]

Great read, thanks for writing this!

[svrnm]

Putting this comment here to keep an eye on setting the date right when we finally publish. Do not resolve.

Suggested change

	date: 2024-11-09
	date: 2024-12-02

[svrnm]

I would like to have @chalin re-review the PR, and we shouldn't publish this week anyways due to thanksgiving

[chalin]

Initial drive-by comments, with a suggestion for fixing the build:

> hugo --cleanDestinationDir -e dev --buildDrafts --baseURL https://deploy-preview-5575--opentelemetry.netlify.app/ --minify
...
Error: error building site: process: readAndProcessContent: "/opt/build/repo/content/en/blog/2024/otel-generative-ai/index.md:8:1": failed to unmarshal YAML: yaml: line 8: did not find expected ',' or ']'
Failed during stage 'building site': Build script returned non-zero exit code: 2 (https://ntl.fyi/exit-code-2)
...

[chalin]

/fix:refcache

[opentelemetrybot]

You triggered fix:refcache action run at https://github.com/open-telemetry/opentelemetry.io/actions/runs/11987016837

[chalin]

This PR will need to be manually rebased (after pulling the latest from GitHub):

[opentelemetrybot]

fix:refcache was successful.

IMPORTANT: (RE-)RUN /fix:all to ensure that there are no remaining check issues.

[drewby]

This PR will need to be manually rebased (after pulling the latest from GitHub):

Up to date now.

[lmolkova]

Thanks for following up! It looks great!