Add SDK integration testing framework

[MrAlias]

Complete the sdk/metric/metrictest package.

• Create a testing framework to end-to-end test the SDK
  • Needs to be extensible. When new aggregators or interfaces are added the SDK it should not be challenging to test their addition.
  • Test all functionality the metric API will send to the SDK
  • Test the configuration of the SDK

[MrAlias]

This issues is referenced by open-telemetry/opentelemetry-go-contrib#2755, open-telemetry/opentelemetry-go-contrib#2757, open-telemetry/opentelemetry-go-contrib#2759, and open-telemetry/opentelemetry-go-contrib#2761 as adding integration testing utilities. It is not defined to accomplish this.

I'm going to create a separate issue to address that.

[MrAlias]

I wonder if this issue can't be moved to the post-GA project?

We can add an sdk/metric/internal/metrictest package post 1.0 that builds out the integration testing of the SDK. If another SDK requests access to that package we could then "stabilize" it by moving it to sdk/metric/metrictest.

The downside to delaying this is that it will likely never get done. Putting off adding tests for later is a good way to ensure they are never created.

[MrAlias]

Moving to the "Metric SDK (GA)" project. We should complete this prior to releasing a GA, but should also wait for the SDK to stabilize in the beta release so we don't churn updating the developed tests.

[joshcarp]

Hello. I came across the go.opentelemetry.io/otel/sdk/metric/[email protected] and I was wondering why it got removed because there doesn't seem to be any alternative and there seems to be some repos already using it

[MrAlias]

Hello. I came across the go.opentelemetry.io/otel/sdk/metric/[email protected] and I was wondering why it got removed because there doesn't seem to be any alternative and there seems to be some repos already using it

It was built against the v0.31.0 SDK which was replaced in v0.32.0. Use a manual reader and the metricdatatest package to test expected metric values: #3407

[MrAlias]

Unit testing metric instrumentation can be accomplished using a manual reader and the metricdatatest package.

Having a full "test suite" would duplicate much of the dedicated testing we already have built for our SDK. Closing as it will not be useful way to spend development effort to further this project in building testing we already have but making it generic for other SDKs.

[krak3n]

@MrAlias is there an example of how to do this?

[sk-]

@krak3n take a look at this file https://github.com/open-telemetry/opentelemetry-go-contrib/blob/f998e3f7c81927b5e7626551863e8db668b9e591/instrumentation/net/http/otelhttp/test/handler_test.go#L60