-
Notifications
You must be signed in to change notification settings - Fork 239
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
otelcol.processor.deltatocumulative: new component (#1044)
Introduce a new `otelcol` component which tracks metric streams with the delta temporality and converts them into the cumulative temporality. This component is initially introduced at the experimental level. Co-authored-by: Carrie Edwards <[email protected]> Co-authored-by: Clayton Cornell <[email protected]>
- Loading branch information
3 people
authored
Jun 14, 2024
1 parent
3f18ccc
commit c22b4a4
Showing
20 changed files
with
403 additions
and
47 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
159 changes: 159 additions & 0 deletions
159
docs/sources/reference/components/otelcol.processor.deltatocumulative.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,159 @@ | ||
| --- | ||
| canonical: https://grafana.com/docs/alloy/latest/reference/components/otelcol.processor.deltatocumulative/ | ||
| description: Learn about otelcol.processor.deltatocumulative | ||
| title: otelcol.processor.deltatocumulative | ||
| --- | ||
|
|
||
| <span class="badge docs-labels__stage docs-labels__item">Experimental</span> | ||
|
|
||
| # otelcol.processor.deltatocumulative | ||
|
|
||
| {{< docs/shared lookup="stability/experimental.md" source="alloy" version="<ALLOY_VERSION>" >}} | ||
|
|
||
| `otelcol.processor.deltatocumulative` accepts metrics from other `otelcol` components and converts metrics with the delta temporality to cumulative. | ||
|
|
||
| {{< admonition type="note" >}} | ||
| `otelcol.processor.deltatocumulative` is a wrapper over the upstream OpenTelemetry Collector `deltatocumulative` processor. | ||
| Bug reports or feature requests will be redirected to the upstream repository, if necessary. | ||
| {{< /admonition >}} | ||
|
|
||
| You can specify multiple `otelcol.processor.deltatocumulative` components by giving them different labels. | ||
|
|
||
| ## Usage | ||
|
|
||
| ```alloy | ||
| otelcol.processor.deltatocumulative "LABEL" { | ||
| output { | ||
| metrics = [...] | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ## Arguments | ||
|
|
||
| `otelcol.processor.deltatocumulative` supports the following arguments: | ||
|
|
||
| Name | Type | Description | Default | Required | ||
| ------------- | ---------- | ------------------------------------------------------------------- | ------- | -------- | ||
| `max_stale` | `duration` | How long to wait for a new sample before marking a stream as stale. | `"5m"` | no | ||
| `max_streams` | `number` | Upper limit of streams to track. Set to `0` to disable. | `0` | no | ||
|
|
||
| `otelcol.processor.deltatocumulative` tracks incoming metric streams. | ||
| Sum and exponential histogram metrics with delta temporality are tracked and converted into cumulative temporality. | ||
|
|
||
| If a new sample hasn't been received since the duration specified by `max_stale`, tracked streams are considered stale and dropped. `max_stale` must be set to a duration greater than `"0s"`. | ||
|
|
||
| The `max_streams` attribute configures the upper limit of streams to track. | ||
| If the limit of tracked streams is reached, new incoming streams are dropped. | ||
| You can disable this behavior by setting `max_streams` to `0`. | ||
|
|
||
| ## Blocks | ||
|
|
||
| The following blocks are supported inside the definition of `otelcol.processor.deltatocumulative`: | ||
|
|
||
| Hierarchy | Block | Description | Required | ||
| ------------- | ----------------- | -------------------------------------------------------------------------- | -------- | ||
| output | [output][] | Configures where to send received telemetry data. | yes | ||
| debug_metrics | [debug_metrics][] | Configures the metrics that this component generates to monitor its state. | no | ||
|
|
||
| [output]: #output-block | ||
| [debug_metrics]: #debug_metrics-block | ||
|
|
||
| ### output block | ||
|
|
||
| {{< docs/shared lookup="reference/components/output-block.md" source="alloy" version="<ALLOY_VERSION>" >}} | ||
|
|
||
| ### debug_metrics block | ||
|
|
||
| {{< docs/shared lookup="reference/components/otelcol-debug-metrics-block.md" source="alloy" version="<ALLOY_VERSION>" >}} | ||
|
|
||
| ## Exported fields | ||
|
|
||
| The following fields are exported and can be referenced by other components: | ||
|
|
||
| Name | Type | Description | ||
| --------|--------------------|----------------------------------------------------------------- | ||
| `input` | `otelcol.Consumer` | A value that other components can use to send telemetry data to. | ||
|
|
||
| `input` accepts `otelcol.Consumer` data for metrics. | ||
|
|
||
| ## Component health | ||
|
|
||
| `otelcol.processor.deltatocumulative` is only reported as unhealthy if given an invalid configuration. | ||
|
|
||
| ## Debug information | ||
|
|
||
| `otelcol.processor.deltatocumulative` does not expose any component-specific debug information. | ||
|
|
||
| ## Debug metrics | ||
|
|
||
| * `processor_deltatocumulative_streams_tracked` (gauge): Number of streams currently tracked by the aggregation state. | ||
| * `processor_deltatocumulative_streams_limit` (gauge): Upper limit of tracked streams. | ||
| * `processor_deltatocumulative_streams_evicted` (counter): Total number of streams removed from tracking to ingest newer streams. | ||
| * `processor_deltatocumulative_streams_max_stale` (gauge): Duration without new samples after which streams are dropped. | ||
| * `processor_deltatocumulative_datapoints_processed` (counter): Total number of datapoints processed (successfully or unsuccessfully). | ||
| * `processor_deltatocumulative_datapoints_dropped` (counter): Faulty datapoints that were dropped due to the reason given in the `reason` label. | ||
| * `processor_deltatocumulative_gaps_length` (counter): Total length of all gaps in the streams, such as being due to lost in transit. | ||
|
|
||
| ## Examples | ||
|
|
||
| ### Basic usage | ||
|
|
||
| This example converts delta temporality metrics to cumulative before sending it to [otelcol.exporter.otlp][] for further processing: | ||
|
|
||
| ```alloy | ||
| otelcol.processor.deltatocumulative "default" { | ||
| output { | ||
| metrics = [otelcol.exporter.otlp.production.input] | ||
| } | ||
| } | ||
| otelcol.exporter.otlp "production" { | ||
| client { | ||
| endpoint = env("OTLP_SERVER_ENDPOINT") | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| [otelcol.exporter.otlp]: ../otelcol.exporter.otlp/ | ||
|
|
||
| ### Exporting Prometheus data | ||
|
|
||
| This example converts delta temporality metrics to cumulative metrics before it is converted to Prometheus data, which requires cumulative temporality: | ||
|
|
||
| ```alloy | ||
| otelcol.processor.deltatocumulative "default" { | ||
| output { | ||
| metrics = [otelcol.exporter.prometheus.default.input] | ||
| } | ||
| } | ||
| otelcol.exporter.prometheus "default" { | ||
| forward_to = [prometheus.remote_write.default.receiver] | ||
| } | ||
| prometheus.remote_write "default" { | ||
| endpoint { | ||
| url = env("PROMETHEUS_SERVER_URL") | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| <!-- START GENERATED COMPATIBLE COMPONENTS --> | ||
|
|
||
| ## Compatible components | ||
|
|
||
| `otelcol.processor.deltatocumulative` can accept arguments from the following components: | ||
|
|
||
| - Components that export [OpenTelemetry `otelcol.Consumer`](../../compatibility/#opentelemetry-otelcolconsumer-exporters) | ||
|
|
||
| `otelcol.processor.deltatocumulative` has exports that can be consumed by the following components: | ||
|
|
||
| - Components that consume [OpenTelemetry `otelcol.Consumer`](../../compatibility/#opentelemetry-otelcolconsumer-consumers) | ||
|
|
||
| {{< admonition type="note" >}} | ||
| Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. | ||
| Refer to the linked documentation for more details. | ||
| {{< /admonition >}} | ||
|
|
||
| <!-- END GENERATED COMPATIBLE COMPONENTS --> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
103 changes: 103 additions & 0 deletions
103
internal/component/otelcol/processor/deltatocumulative/deltatocumulative.go
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,103 @@ | ||
| // Package deltatocumulative provides an otelcol.processor.deltatocumulative | ||
| // component. | ||
| package deltatocumulative | ||
|
|
||
| import ( | ||
| "fmt" | ||
| "time" | ||
|
|
||
| "github.com/grafana/alloy/internal/component" | ||
| "github.com/grafana/alloy/internal/component/otelcol" | ||
| otelcolCfg "github.com/grafana/alloy/internal/component/otelcol/config" | ||
| "github.com/grafana/alloy/internal/component/otelcol/processor" | ||
| "github.com/grafana/alloy/internal/featuregate" | ||
| "github.com/open-telemetry/opentelemetry-collector-contrib/processor/deltatocumulativeprocessor" | ||
| otelcomponent "go.opentelemetry.io/collector/component" | ||
| otelextension "go.opentelemetry.io/collector/extension" | ||
| ) | ||
|
|
||
| func init() { | ||
| component.Register(component.Registration{ | ||
| Name: "otelcol.processor.deltatocumulative", | ||
| Stability: featuregate.StabilityExperimental, | ||
| Args: Arguments{}, | ||
| Exports: otelcol.ConsumerExports{}, | ||
|
|
||
| Build: func(opts component.Options, args component.Arguments) (component.Component, error) { | ||
| fact := deltatocumulativeprocessor.NewFactory() | ||
| return processor.New(opts, fact, args.(Arguments)) | ||
| }, | ||
| }) | ||
| } | ||
|
|
||
| // Arguments configures the otelcol.processor.deltatocumulative component. | ||
| type Arguments struct { | ||
| MaxStale time.Duration `alloy:"max_stale,attr,optional"` | ||
| MaxStreams int `alloy:"max_streams,attr,optional"` | ||
|
|
||
| // Output configures where to send processed data. Required. | ||
| Output *otelcol.ConsumerArguments `alloy:"output,block"` | ||
|
|
||
| // DebugMetrics configures component internal metrics. Optional. | ||
| DebugMetrics otelcolCfg.DebugMetricsArguments `alloy:"debug_metrics,block,optional"` | ||
| } | ||
|
|
||
| var ( | ||
| _ processor.Arguments = Arguments{} | ||
| ) | ||
|
|
||
| // DefaultArguments holds default settings for Arguments. | ||
| var DefaultArguments = Arguments{ | ||
| MaxStale: 5 * time.Minute, | ||
|
|
||
| // NOTE(rfratto): 0 means disable, and there is an ongoing effort upstream to | ||
| // identify a good non-zero default: | ||
| // | ||
| // https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/31603 | ||
| MaxStreams: 0, | ||
| } | ||
|
|
||
| // SetToDefault implements syntax.Defaulter. | ||
| func (args *Arguments) SetToDefault() { | ||
| *args = DefaultArguments | ||
| args.DebugMetrics.SetToDefault() | ||
| } | ||
|
|
||
| // Validate implements syntax.Validator. | ||
| func (args *Arguments) Validate() error { | ||
| if args.MaxStale <= 0 { | ||
| return fmt.Errorf("max_stale must be a positive duration (got %s)", args.MaxStale) | ||
| } | ||
| if args.MaxStreams < 0 { | ||
| return fmt.Errorf("max_streams must be a positive number or zero (got %d)", args.MaxStreams) | ||
| } | ||
| return nil | ||
| } | ||
|
|
||
| // Convert implements processor.Arguments. | ||
| func (args Arguments) Convert() (otelcomponent.Config, error) { | ||
| return &deltatocumulativeprocessor.Config{ | ||
| MaxStale: args.MaxStale, | ||
| MaxStreams: args.MaxStreams, | ||
| }, nil | ||
| } | ||
|
|
||
| // Extensions implements processor.Arguments. | ||
| func (args Arguments) Extensions() map[otelcomponent.ID]otelextension.Extension { | ||
| return nil | ||
| } | ||
|
|
||
| // Exporters implements processor.Arguments. | ||
| func (args Arguments) Exporters() map[otelcomponent.DataType]map[otelcomponent.ID]otelcomponent.Component { | ||
| return nil | ||
| } | ||
|
|
||
| // NextConsumers implements processor.Arguments. | ||
| func (args Arguments) NextConsumers() *otelcol.ConsumerArguments { | ||
| return args.Output | ||
| } | ||
|
|
||
| // DebugMetricsConfig implements processor.Arguments. | ||
| func (args Arguments) DebugMetricsConfig() otelcolCfg.DebugMetricsArguments { | ||
| return args.DebugMetrics | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.