Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(csharp/src/Apache.Arrow.Adbc): add OpenTelemetry compatible tracing support #2388

Draft
wants to merge 86 commits into
base: main
Choose a base branch
from

Conversation

birschick-bq
Copy link
Contributor

@birschick-bq birschick-bq commented Dec 20, 2024

This PR adds OpenTelemetry-compatible tracing support on the CSharp library.

Under the hood it uses Activity and ActivitySource from the net System.Diagnostics.DiagnosticSource package.

All the pertinent Adbc* classes now inherit from the TracingBase class.

To instrument tracing, use one of the following overrides of

  • TraceActivity
  • TraceActivityAsync

Example:

public void Connect(...)
{
    TraceActivity((activity) =>
    {
        driver.OpenSession(...);
    });
}

Each of these overrides creates a new Activity which will be non-null only if there is an active ActivityListener or OpenTelemetry exporter. The Activity is passed to the delegate Func/Action in case it need to add ActivityEvent, ActivityLink or Tags (KeyValuePair). When instrumenting tracing, you should always use the null-conditional operator (?. ) when accessing members on the passed delegate parameter, activity.

Example:

public void Connect(...)
{
    TraceActivity((activity) =>
    {
        activity?.AddEvent("opensession.start");
        var result = driver.OpenSession(...);
        activity?.AddEvent("opensession.end", ["status", result.Status]);
    });
}

The default behavior is to invoke the delegate and if successful (i.e., no exception thrown), the Activity.SetStatus is set to Ok. If an exception is observed, then the Activity.SetStatus is set to Error and the exception is traced (Activity.AddException) as an event in the current Activity.

Callers can pass a traceparent string to any of the TraceActivity[Async] methods using the optional traceParent parameter, or to the inherited classes using the TraceParent property. The parameter takes precedence over the property. The traceId from the traceParent parameter or TraceParent property will be adopted as the rootId for all trace Activity on that call or object. If TraceParent is null (initial or set later), then the Activity creates a new rootId for the beginning of the initial Activity in the stack.

Example:

public void Connect(..., string? traceParent = default)
{
    TraceActivity((activity) =>
    {
        activity?.AddEvent("opensession.start");
        var result = driver.OpenSession(...);
        activity?.AddEvent("opensession.end", ["status", result.Status]);
    }, traceParent: traceParent);
}

The PR also includes a FileExporter that inherits from the OpenTelemetry class BaseExporter<Activity>.

To enable the exporter, use the OpenTelemetry SDK as follows:

TracerProvider tracerProvider = Sdk.CreateTracerProviderBuilder()
    .AddSource(ActivitySource.Name)
    .AddAdbcFileExporter(ActivitySource.Name, tracingLocation, maxTraceFileSizeKb, maxTraceFiles)
    .Build();

Ensure to Dispose the TracerProvider when you no longer need to listen for Activity messages.

@lidavidm lidavidm removed this from the ADBC Libraries 16 milestone Dec 26, 2024
@github-actions github-actions bot added this to the ADBC Libraries 16 milestone Jan 3, 2025
@lidavidm lidavidm removed this from the ADBC Libraries 16 milestone Jan 6, 2025
@birschick-bq birschick-bq marked this pull request as draft January 9, 2025 21:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants