[feature request] Filter at end of activity

[DannyRichardsonWG]

Component

OpenTelemetry.Instrumentation.AspNetCore

Is your feature request related to a problem?

I'd like the ability to be able to filter logged items based on the status code of the HTTP Response.

What is the expected behavior?

At the end of the activity processing have a filter that is able to exclude items. Similar to the filter that is applied at the start of the activity.

Which alternative solutions or features have you considered?

The goal is to drop any logging for a request to a specific endpoint with a certain status code

e.g. anything sent to / with a 401 response code.

So I call UseAzureMonitor

OpenTelemetryBuilder monitor = services.AddOpenTelemetry()
            .UseAzureMonitor(options =>
            {
                options.ConnectionString = appInsightsConnectionString;
                options.SamplingRatio = samplingRatio;
            }).ConfigureResource(resource => { resource.AddService(serviceName); });

And added a processor

public class SuperAwesomeProcessor : BaseProcessor<Activity>
{
    public override void OnEnd(Activity activity)
    {

        int responseCode = -99;
        if (activity.TryGetTagObject("url.path", out string? url))
        {
            if (activity.TryGetTagObject("http.response.status_code", out responseCode, -123))
            {
                // if it's a root call and 401 response we can ignore it
                if (url == "/"
                    && responseCode == (int)HttpStatusCode.Unauthorized)
                {
                    activity.ActivityTraceFlags &= ~ActivityTraceFlags.Recorded;
                    return;
                }

            }
        }

        base.OnEnd(activity);
    }
}


public static class SuperAwesomeExtensions{

  public static bool TryGetTagObject<T>(this Activity? activity, string key, out T? value, T defaultValue)
    {
        value = defaultValue;
        if (activity?.TagObjects == null)
        {
            return false;
        }
        
        foreach (var tag in activity.TagObjects)
        {
            if (tag.Key == key)
            {
                if (!(tag.Value is T tagValue))
                {
                    return false;
                }

                value = tagValue;
                return true;
            }
        }

        return false;
    }

}

Which when hooked up partially worked, it removes the dependencies of the request but the parent request was still getting logged in the requests table in application insights.

After a bit more digging I found you can add a filter to the AddAspNetCoreInstrumentation call but as we're calling UseAzureMonitor we need to call configure using AspNetCoreTraceInstrumentationOptions

        services.Configure<AspNetCoreTraceInstrumentationOptions>(options =>
        {
            options.Filter = (context) =>
            {
                var path = context.Request.Path.ToString();
                if (path.StartsWithIgnoreCase("/") 
                    && context.Response.StatusCode == (int)HttpStatusCode.Unauthorized
                    )
                    return false;

                return true;
            };
        });

The problem is the StatusCode is 200 and not 401, I'm assuming because the logging/filter fires before the authentication middleware has kicked in as the filter runs in OnStartActivity in HttpInListener.

Is there a filter that fires at the end of the request like the OnEnd of the processor or is there a better way of achieving the goal?

Cheers,

Danny

Additional context

No response

[TimothyMothra]

HI @DannyRichardsonWG,
Let me try to summarize this, please correct me if I get something wrong.

So for context we're talking about; Incoming Http Request --> Your Application --> Outbound Http Request

You added an Activity Processor, which is successfully filtering the Outbound Requests, but not the root Incoming Request.
You then tried a custom Filter for the Instrumentation.AspNetCore library. This isn't filtering anything because we think it's running before the middleware.

---

Next Steps,

Can you share a minimal repro app?
I tried to reproduce this myself but wasn't seeing your exact circumstances.
You can use the ConsoleExporter instead of the AzureMonitorExporter.

[DannyRichardsonWG]

Hi @TimothyMothra

Thanks for taking a look. Yeah your summary sounds correct. I think the filter is running before the request gets processed by the authentication middleware so the status code isn't 401 when the filter looks.

I've created a small demo here: LogFilteringExample.zip

Rather than implement authentication I'm just having one of the endpoints return a 401.

I setup the filter to stop logging on either of these two conditions

1. The status is 401
2. The endpoint path is /neverLog

So calling /weatherforecast works as expected. The log appears

Calling /testUnauth shouldn't log as it returns 401 but it still ends up in the log. If you put a breakpoint on the filter you'll see it hits before the endpoint sets the status code so neither condition is met for preventing logging.

Calling /neverLog the filter successfully prevents the trace ending up in the log

So it's like there needs to be an additional EndFilter on AspNetCoreTraceInstrumentationOptions that gets called when the activity is about to stop and export

Cheers,

Danny

[TimothyMothra]

Hi @DannyRichardsonWG

So I got an answer about the current Filter method. This is intended to be head-based filtering, meaning by design it's used to make a decision at the very beginning of an incoming request. Many hosting providers have healthcheck or heartbeat requests to confirm that an app is alive, and the Filter method was intended to help filter these kinds of requests so it can also propagate to any dependency calls associated with that request.

The Activity Processor is going to be the recommend solution since this runs after the request.

You had another question about also filtering the dependencies of your unauthorized request. I would expect that if the request is unauthorized than the dependencies wouldn't execute. Regardless, if you need to make a filtering decision about a dependencies parent, you can do that in an Activity Processor as well by referencing activity.Parent.

For example:

activity.TryGetTagObject("http.response.status_code", out int responseCode)
|| activity.Parent.TryGetTagObject("http.response.status_code", out responseCode)

Please let me know if you have any other questions.