Clarify default sampler

[seemk]

OpenTelemetry defaults to parentbased_always_on, but this is not consistent across our distributions:

• parentbased_always_on is used by Node.js, .NET, Python
• always_on is used by Java, Go

If we are advertising as a no sampling solution, we should default to always_on, but still allow for customers to change it.
Using parentbased_always_on can cause data loss if customers have previously set up cloud tracing, which injects a traceparent with 00 for traceflags.

[seemk]

@akubik-splunk

[pellared]

Wouldn't changing the default be a breaking change for a distribution?

I see it as a breaking change from semantic point of view. If we change the default in Node.js, .NET, Python distributions then I would advise to bump the major version.

[akubik-splunk]

In which distros this is marked as stable?
If this is marked as stable we must consider releasing this as braking change and bump the major release
As option we can consider aligning the Node.js and making it braking change only there
This is another reason we need compatibility matrix

[pellared]

In which distros this is marked as stable?

For sure in .NET it is stable.

In Python it is part of its SDK which is stable.

[akubik-splunk]

In which distros this is marked as stable?

For sure in .NET it is stable.

In Python it is part of its SDK which is stable.

So it seems to me that we should prep for the major bump and if possible bundle it up with other changes e.g. HTTP semantics :) for .NET and for Python as well. If there are other major changes that we would like to do let's use this as opportunity to do so. We would target such changes for the Jan-Feb next year though

[Kielek]

Blocking for now. We need agreement if we really should update it in all Splunk distros.

It is against OTel spec defaults.

[pellared]

It is against OTel spec defaults.

Correct, this is why we want to change the default in the GDI spec.

We need agreement if we really should update it in all Splunk distros.

I think we should. We intentionally decided to use always_on in Go to address the issue from the PR description:

If we are advertising as a no sampling solution, we should default to always_on, but still allow for customers to change it.
Using parentbased_always_on can cause data loss if customers have previously set up cloud tracing, which injects a traceparent with 00 for traceflags.

AFAIK Node.js plans to do the change in their upcoming v3 release. cc @seemk
I think it would be good to do the same in Python and .NET distributions.
Personally, I would not require a major bump for such change. Initially I had a different opinion on this, but I changed my mind. I think changing the other way could be disturbing for the users.

@akubik-splunk, do you have an opinion here?

[akubik-splunk]

Pleaser go with always_on in our distros
Temporarily .NET will be only distro with different default which should be fixed next time we do major release

[Kielek]

I do not like the idea changing default behavior from OTel. It can be problematic, especially in terms of promoting upstream distributions (for non-most-popular technologies.

Approving, as you agreed on this.

[pellared]

especially in terms of promoting upstream distributions

One can always set the always_on while using upstream. This is just for make it more convenient for our users.