Update to OTel Collector v0.105.0 (#1362)

* Update OTel to v0.105.0 * Use "0.0.0.0:4317" as the receiver endpoint in the OTLP integration test. This is required due to a recent breaking change in the config. --------- Co-authored-by: Clayton Cornell <[email protected]>
grafana · Jul 30, 2024 · bda038d · bda038d
1 parent 28221a4
commit bda038d
Show file tree

Hide file tree

Showing 78 changed files with 1,471 additions and 956 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -10,6 +10,16 @@ internal API changes are not present.
 Main (unreleased)
 -----------------
 
+### Breaking changes
+
+- `otelcol.receiver.otlp`, `otelcol.receiver.jaeger`, `otelcol.extension.jaeger_remote_sampling`, `otelcol.receiver.zipkin` 
+  will now configure `endpoint` using `localhost` by default instead of `0.0.0.0`. 
+  This may break the receiver in containerized environments like Kubernetes. 
+  If you depend on `0.0.0.0`, configure the `endpoint` attribute to explicitly use `0.0.0.0`.
+- [`otelcol.exporter.otlp`,`otelcol.exporter.loadbalancing`]: Change the default gRPC load balancing strategy.
+  The default value for the `balancer_name` attribute has changed to `round_robin`
+  https://github.com/open-telemetry/opentelemetry-collector/pull/10319
+
 ### Breaking changes to non-GA functionality
 
 - Update Public preview `remotecfg` argument from `metadata` to `attributes`. (@erikbaranowski)
@@ -47,6 +57,61 @@ Main (unreleased)
 
 - `mimir.rules.kubernetes` is now able to add extra labels to the Prometheus rules. (@psychomantys)
 
+- Upgrade from OpenTelemetry v0.102.1 to v0.105.0.
+  - [`otelcol.receiver.*`] A new `compression_algorithms` attribute to configure which 
+    compression algorithms are allowed by the HTTP server.
+    https://github.com/open-telemetry/opentelemetry-collector/pull/10295
+  - [`otelcol.exporter.*`] Fix potential deadlock in the batch sender.
+    https://github.com/open-telemetry/opentelemetry-collector/pull/10315
+  - [`otelcol.exporter.*`] Fix a bug when the retry and timeout logic was not applied with enabled batching.
+    https://github.com/open-telemetry/opentelemetry-collector/issues/10166
+  - [`otelcol.exporter.*`] Fix a bug where an unstarted batch_sender exporter hangs on shutdown.
+    https://github.com/open-telemetry/opentelemetry-collector/issues/10306
+  - [`otelcol.exporter.*`] Fix small batch due to unfavorable goroutine scheduling in batch sender.
+    https://github.com/open-telemetry/opentelemetry-collector/issues/9952
+  - [`otelcol.exporter.otlphttp`] A new `cookies` block to store cookies from server responses and reuse them in subsequent requests.
+    https://github.com/open-telemetry/opentelemetry-collector/issues/10175
+  - [`otelcol.exporter.otlp`] Fixed a bug where the receiver's http response was not properly translating grpc error codes to http status codes.
+    https://github.com/open-telemetry/opentelemetry-collector/pull/10574
+  - [`otelcol.processor.tail_sampling`] Simple LRU Decision Cache for "keep" decisions.
+    https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/33533
+  - [`otelcol.processor.tail_sampling`] Fix precedence of inverted match in and policy.
+    Previously if the decision from a policy evaluation was `NotSampled` or `InvertNotSampled` 
+    it would return a `NotSampled` decision regardless, effectively downgrading the result.
+    This was breaking the documented behaviour that inverted decisions should take precedence over all others.
+    https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/33671
+  - [`otelcol.exporter.kafka`,`otelcol.receiver.kafka`] Add config attribute to disable Kerberos PA-FX-FAST negotiation.
+    https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/26345
+  - [`OTTL`]: Added `keep_matching_keys` function to allow dropping all keys from a map that don't match the pattern.
+    https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/32989
+  - [`OTTL`]: Add debug logs to help troubleshoot OTTL statements/conditions
+    https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/33274
+  - [`OTTL`]: Introducing `append` function for appending items into an existing array.
+    https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/32141
+  - [`OTTL`]: Introducing `Uri` converter parsing URI string into SemConv
+    https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/32433
+  - [`OTTL`]: Added a Hex() converter function
+    https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/33450
+  - [`OTTL`]: Added a IsRootSpan() converter function.
+    https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/33729
+  - [`otelcol.processor.probabilistic_sampler`]: Add Proportional and Equalizing sampling modes.
+    https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/31918
+  - [`otelcol.processor.deltatocumulative`]: Bugfix to properly drop samples when at limit.
+    https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/33285
+  - [`otelcol.receiver.vcenter`] Fixes errors in some of the client calls for environments containing multiple datacenters.
+    https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/33735
+  - [`otelcol.processor.resourcedetection`] Fetch CPU info only if related attributes are enabled.
+    https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/33774
+  - [`otelcol.receiver.vcenter`] Adding metrics for CPU readiness, CPU capacity, and network drop rate.
+    https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/33607
+  - [`otelcol.receiver.vcenter`] Drop support for vCenter 6.7.
+    https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/33607
+  - [`otelcol.processor.attributes`] Add an option to extract value from a client address 
+    by specifying `client.address` value in the `from_context` field.
+    https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/34048
+  - `otelcol.connector.spanmetrics`: Produce delta temporality span metrics with StartTimeUnixNano and TimeUnixNano values representing an uninterrupted series.
+    https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/31780
+
 ### Bugfixes
 
 - Fixed a clustering mode issue where a fatal startup failure of the clustering service

diff --git a/docs/developer/updating-otel.md → docs/developer/updating-otel/README.md b/docs/developer/updating-otel.md → docs/developer/updating-otel/README.md
@@ -99,19 +99,19 @@ for an example on how to update Alloy.
 
 ### Testing a tracing pipeline locally
 
-You can use the resources in the [Tempo repository](https://github.com/grafana/tempo/tree/main/example/docker-compose/agent) to create a local source of traces using k6. You can also start your own Tempo and Grafana instances.
-
-1. Comment out the "agent" and "prometheus" sections in the [docker-compose](https://github.com/grafana/tempo/blob/main/example/docker-compose/agent/docker-compose.yaml). We don't need this - instead, we will start our own locally built Alloy collector.
-2. Change the "k6-tracing" endpoint to send traces on the localhost, outside of the Docker container.
-   * For example, use `ENDPOINT=host.docker.internal:4320`.
-   * Then our local Alloy should be configured to accept traces on `0.0.0.0:4320`.
-3. Optionally, e.g. if you prefer Grafana Cloud, comment out the "tempo" and "grafana" sections of the docker-compose file.
-4. Add a second k6 instance if needed - for example, if multiple receivers are configured.
-
-The "otelcol"
-[components](https://grafana.com/docs/alloy/latest/reference/components/) are
-the only components which use OTel. Try to test as many of them as possible
-using a config file like this one:
+Firstly, start a K6 trace generator to simulate an application instrumented for tracing:
+```
+cd docs/developer/updating-otel/k6-trace-gen/
+docker compose up -d
+```
+
+K6 will be configured to send traces on `ENDPOINT=host.docker.internal:4320`.
+This means that the local Alloy instance must be configured to accept traces on `0.0.0.0:4320`.
+
+The ["otelcol" components][otelcol-components]  are the only components which use OTel. 
+Try to test as many of them as possible using a config file like this one:
+
+[otelcol-components](https://grafana.com/docs/alloy/latest/reference/components/otelcol/)
 
 <details>
   <summary>Example Alloy config</summary>

diff --git a/docs/developer/updating-otel/k6-trace-gen/docker-compose.yaml b/docs/developer/updating-otel/k6-trace-gen/docker-compose.yaml
@@ -0,0 +1,7 @@
+services:
+
+  k6-tracing:
+    image: ghcr.io/grafana/xk6-client-tracing:v0.0.5
+    environment:
+      - ENDPOINT=host.docker.internal:4320
+    restart: always
diff --git a/docs/sources/_index.md b/docs/sources/_index.md
@@ -5,7 +5,7 @@ description: Grafana Alloy is a a vendor-neutral distribution of the OTel Collec
 weight: 350
 cascade:
   ALLOY_RELEASE: v1.3.0
-  OTEL_VERSION: v0.87.0
+  OTEL_VERSION: v0.105.0
   FULL_PRODUCT_NAME: Grafana Alloy
   PRODUCT_NAME: Alloy
 hero:

diff --git a/docs/sources/_index.md.t b/docs/sources/_index.md.t
@@ -5,7 +5,7 @@ description: Grafana Alloy is a a vendor-neutral distribution of the OTel Collec
 weight: 350
 cascade:
   ALLOY_RELEASE: $ALLOY_VERSION
-  OTEL_VERSION: v0.87.0
+  OTEL_VERSION: v0.105.0
   FULL_PRODUCT_NAME: Grafana Alloy
   PRODUCT_NAME: Alloy
 hero:

diff --git a/docs/sources/reference/components/otelcol/otelcol.connector.spanmetrics.md b/docs/sources/reference/components/otelcol/otelcol.connector.spanmetrics.md
@@ -67,6 +67,7 @@ otelcol.connector.spanmetrics "LABEL" {
 | `exclude_dimensions`              | `list(string)` | List of dimensions to be excluded from the default set of dimensions.                      | `[]`           | no       |
 | `metrics_flush_interval`          | `duration`     | How often to flush generated metrics.                                                      | `"60s"`        | no       |
 | `metrics_expiration`              | `duration`     | Time period after which metrics are considered stale and are removed from the cache.       | `"0s"`         | no       |
+| `metric_timestamp_cache_size`     | `number`       | Controls the size of a cache used to keep track of the last time a metric was flushed.     | `1000`         | no       |
 | `namespace`                       | `string`       | Metric namespace.                                                                          | `""`           | no       |
 | `resource_metrics_cache_size`     | `number`       | The size of the cache holding metrics for a service.                                       | `1000`         | no       |
 | `resource_metrics_key_attributes` | `list(string)` | Limits the resource attributes used to create the metrics.                                 | `[]`           | no       |
@@ -84,6 +85,11 @@ Setting `metrics_expiration` to `"0s"` means that the metrics will never expire.
 
 `resource_metrics_cache_size` is mostly relevant for cumulative temporality. It helps avoid issues with increasing memory and with incorrect metric timestamp resets.
 
+`metric_timestamp_cache_size` is only relevant for delta temporality span metrics. 
+It controls the size of a cache used to keep track of the last time a metric was flushed. 
+When a metric is evicted from the cache, its next data point will indicate a "reset" in the series. 
+Downstream components converting from delta to cumulative may handle these resets by setting cumulative counters back to 0.
+
 `resource_metrics_key_attributes` can be used to avoid situations where resource attributes may change across service restarts,
 causing metric counters to break (and duplicate). A resource does not need to have all of the attributes.
 The list must include enough attributes to properly identify unique resources or risk aggregating data from more than one service and span.

diff --git a/docs/sources/reference/components/otelcol/otelcol.exporter.loadbalancing.md b/docs/sources/reference/components/otelcol/otelcol.exporter.loadbalancing.md
@@ -205,16 +205,16 @@ The endpoints used by the client block are the ones from the `resolver` block
 
 The following arguments are supported:
 
-Name                | Type                       | Description                                                                      | Default      | Required
---------------------|----------------------------|----------------------------------------------------------------------------------|--------------|---------
-`compression`       | `string`                   | Compression mechanism to use for requests.                                       | `"gzip"`     | no
-`read_buffer_size`  | `string`                   | Size of the read buffer the gRPC client to use for reading server responses.     |              | no
-`write_buffer_size` | `string`                   | Size of the write buffer the gRPC client to use for writing requests.            | `"512KiB"`   | no
-`wait_for_ready`    | `boolean`                  | Waits for gRPC connection to be in the `READY` state before sending data.        | `false`      | no
-`headers`           | `map(string)`              | Additional headers to send with the request.                                     | `{}`         | no
-`balancer_name`     | `string`                   | Which gRPC client-side load balancer to use for requests.                        | `pick_first` | no
-`authority`         | `string`                   | Overrides the default `:authority` header in gRPC requests from the gRPC client. |              | no
-`auth`              | `capsule(otelcol.Handler)` | Handler from an `otelcol.auth` component to use for authenticating requests.     |              | no
+Name                | Type                       | Description                                                                      | Default       | Required
+--------------------|----------------------------|----------------------------------------------------------------------------------|---------------|---------
+`compression`       | `string`                   | Compression mechanism to use for requests.                                       | `"gzip"`      | no
+`read_buffer_size`  | `string`                   | Size of the read buffer the gRPC client to use for reading server responses.     |               | no
+`write_buffer_size` | `string`                   | Size of the write buffer the gRPC client to use for writing requests.            | `"512KiB"`    | no
+`wait_for_ready`    | `boolean`                  | Waits for gRPC connection to be in the `READY` state before sending data.        | `false`       | no
+`headers`           | `map(string)`              | Additional headers to send with the request.                                     | `{}`          | no
+`balancer_name`     | `string`                   | Which gRPC client-side load balancer to use for requests.                        | `round_robin` | no
+`authority`         | `string`                   | Overrides the default `:authority` header in gRPC requests from the gRPC client. |               | no
+`auth`              | `capsule(otelcol.Handler)` | Handler from an `otelcol.auth` component to use for authenticating requests.     |               | no
 
 {{< docs/shared lookup="reference/components/otelcol-compression-field.md" source="alloy" version="<ALLOY_VERSION>" >}}
 

diff --git a/docs/sources/reference/components/otelcol/otelcol.exporter.otlp.md b/docs/sources/reference/components/otelcol/otelcol.exporter.otlp.md
@@ -66,17 +66,17 @@ The `client` block configures the gRPC client used by the component.
 
 The following arguments are supported:
 
-Name                | Type                       | Description                                                                      | Default      | Required
---------------------|----------------------------|----------------------------------------------------------------------------------|--------------|---------
-`endpoint`          | `string`                   | `host:port` to send telemetry data to.                                           |              | yes
-`compression`       | `string`                   | Compression mechanism to use for requests.                                       | `"gzip"`     | no
-`read_buffer_size`  | `string`                   | Size of the read buffer the gRPC client to use for reading server responses.     |              | no
-`write_buffer_size` | `string`                   | Size of the write buffer the gRPC client to use for writing requests.            | `"512KiB"`   | no
-`wait_for_ready`    | `boolean`                  | Waits for gRPC connection to be in the `READY` state before sending data.        | `false`      | no
-`headers`           | `map(string)`              | Additional headers to send with the request.                                     | `{}`         | no
-`balancer_name`     | `string`                   | Which gRPC client-side load balancer to use for requests.                        | `pick_first` | no
-`authority`         | `string`                   | Overrides the default `:authority` header in gRPC requests from the gRPC client. |              | no
-`auth`              | `capsule(otelcol.Handler)` | Handler from an `otelcol.auth` component to use for authenticating requests.     |              | no
+Name                | Type                       | Description                                                                      | Default       | Required
+--------------------|----------------------------|----------------------------------------------------------------------------------|---------------|---------
+`endpoint`          | `string`                   | `host:port` to send telemetry data to.                                           |               | yes
+`compression`       | `string`                   | Compression mechanism to use for requests.                                       | `"gzip"`      | no
+`read_buffer_size`  | `string`                   | Size of the read buffer the gRPC client to use for reading server responses.     |               | no
+`write_buffer_size` | `string`                   | Size of the write buffer the gRPC client to use for writing requests.            | `"512KiB"`    | no
+`wait_for_ready`    | `boolean`                  | Waits for gRPC connection to be in the `READY` state before sending data.        | `false`       | no
+`headers`           | `map(string)`              | Additional headers to send with the request.                                     | `{}`          | no
+`balancer_name`     | `string`                   | Which gRPC client-side load balancer to use for requests.                        | `round_robin` | no
+`authority`         | `string`                   | Overrides the default `:authority` header in gRPC requests from the gRPC client. |               | no
+`auth`              | `capsule(otelcol.Handler)` | Handler from an `otelcol.auth` component to use for authenticating requests.     |               | no
 
 {{< docs/shared lookup="reference/components/otelcol-compression-field.md" source="alloy" version="<ALLOY_VERSION>" >}}
 

diff --git a/docs/sources/reference/components/otelcol/otelcol.exporter.otlphttp.md b/docs/sources/reference/components/otelcol/otelcol.exporter.otlphttp.md
@@ -48,6 +48,7 @@ Hierarchy        | Block                | Description
 -----------------|----------------------|----------------------------------------------------------------------------|---------
 client           | [client][]           | Configures the HTTP server to send telemetry data to.                      | yes
 client > tls     | [tls][]              | Configures TLS for the HTTP client.                                        | no
+client > cookies | [cookies][]          | Store cookies from server responses and reuse them in subsequent requests. | no
 sending_queue    | [sending_queue][]    | Configures batching of data before sending.                                | no
 retry_on_failure | [retry_on_failure][] | Configures retry mechanism for failed requests.                            | no
 debug_metrics    | [debug_metrics][]    | Configures the metrics that this component generates to monitor its state. | no
@@ -57,6 +58,7 @@ For example, `client > tls` refers to a `tls` block defined inside a `client` bl
 
 [client]: #client-block
 [tls]: #tls-block
+[cookies]: #cookies-block
 [sending_queue]: #sending_queue-block
 [retry_on_failure]: #retry_on_failure-block
 [debug_metrics]: #debug_metrics-block
@@ -98,6 +100,18 @@ If `http2_read_idle_timeout` is unset or set to `0s`, then no health check will
 
 {{< docs/shared lookup="reference/components/otelcol-compression-field.md" source="alloy" version="<ALLOY_VERSION>" >}}
 
+### cookies block
+
+The `cookies` block allows the HTTP client to store cookies from server responses and reuse them in subsequent requests.
+
+This could be useful in situations such as load balancers relying on cookies for sticky sessions and enforcing a maximum session age.
+
+The following arguments are supported:
+
+Name      | Type   | Description                               | Default    | Required
+----------|--------|-------------------------------------------|------------|---------
+`enabled` | `bool` | The target URL to send telemetry data to. | `false`    | no
+
 ### tls block
 
 The `tls` block configures TLS settings used for the connection to the HTTP server.