Skip to content

Commit

Permalink
MAIL for documentation and merge
Browse files Browse the repository at this point in the history
  • Loading branch information
aidanleuck committed Jan 3, 2025
1 parent 4665aaf commit 3913159
Show file tree
Hide file tree
Showing 34 changed files with 322 additions and 157 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/helm-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ jobs:
run: ct lint --config ./operations/helm/ct.yaml

- name: Create kind cluster
uses: helm/kind-action@v1.10.0
uses: helm/kind-action@v1.11.0
if: steps.list-changed.outputs.changed == 'true'

- name: Add dependency chart repos
Expand Down
6 changes: 4 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,12 +50,14 @@ Main (unreleased)
- Change processlist query to support ONLY_FULL_GROUP_BY sql_mode
- Add perf_schema quantile columns to collector

- Live Debugging button should appear in UI only for supported components (@ravishankar15)
- Add three new stdlib functions to_base64, from_URLbase64 and to_URLbase64 (@ravishankar15)
- Add `ignore_older_than` option for local.file_match (@ravishankar15)
- Add livedebugging support for `discover.relabel` (@ravishankar15)

- Use a forked `github.com/goccy/go-json` module which reduces the memory consumption of an Alloy instance by 20MB.
- Upgrade `github.com/goccy/go-json` to v0.10.4, which reduces the memory consumption of an Alloy instance by 20MB.
If Alloy is running certain otelcol components, this reduction will not apply. (@ptodev)

- Update `prometheus.write.queue` library for performance increases in cpu. (@mattdurham)

### Bugfixes
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ title: otelcol.auth.basic
`otelcol.auth.basic` exposes a `handler` that can be used by other `otelcol`
components to authenticate requests using basic authentication.

This extension supports both server and client authentication.
This `component` supports both server and client authentication.

> **NOTE**: `otelcol.auth.basic` is a wrapper over the upstream OpenTelemetry
> Collector `basicauth` extension. Bug reports or feature requests will be
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ title: otelcol.auth.bearer
`otelcol.auth.bearer` exposes a `handler` that can be used by other `otelcol`
components to authenticate requests using bearer token authentication.

This extension supports both server and client authentication.
This `component` supports both server and client authentication.

{{< admonition type="note" >}}
`otelcol.auth.bearer` is a wrapper over the upstream OpenTelemetry Collector `bearertokenauth` extension.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ title: otelcol.auth.headers
`otelcol.auth.headers` exposes a `handler` that can be used by other `otelcol`
components to authenticate requests using custom headers.

This extension only supports client authentication.
This `component` only supports client authentication.

{{< admonition type="note" >}}
`otelcol.auth.headers` is a wrapper over the upstream OpenTelemetry Collector `headerssetter` extension.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ title: otelcol.auth.oauth2

`otelcol.auth.oauth2` exposes a `handler` that can be used by other `otelcol` components to authenticate requests using OAuth 2.0.

This extension only supports client authentication.
This `component` only supports client authentication.

The authorization tokens can be used by HTTP and gRPC based OpenTelemetry exporters.
This component can fetch and refresh expired tokens automatically.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ title: otelcol.auth.sigv4
components to authenticate requests to AWS services using the AWS Signature Version 4 (SigV4) protocol.
For more information about SigV4 see the AWS documentation about [Signing AWS API requests][].

This extension only supports client authentication.
This `component` only supports client authentication.

[Signing AWS API requests]: https://docs.aws.amazon.com/general/latest/gr/signing-aws-api-requests.html

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -296,9 +296,14 @@ otelcol.extension.jaeger_remote_sampling "example" {
}
```

## Enabling Authentication

You can create a `jaeger_remote_sampling` extensions that requires authentication for requests. This is useful for limiting access to the sampling document. Note that not all OpenTelemetry Collector (otelcol) authentication plugins support receiver authentication. Please refer to the documentation for each `otelcol.auth.*` plugin to determine its compatibility.
## Enable authentication

You can use `jaeger_remote_sampling` to authenticate requests.
This allows you to limit access to the sampling document.
{{< admonition type="note" >}}
Not all OpenTelemetry Collector authentication plugins support receiver authentication.
Refer to the [documentation](https://grafana.com/docs/alloy/<ALLOY_VERSION>/reference/components/otelcol/) for each `otelcol.auth.*` component to determine its compatibility.
{{< /admonition >}}

```alloy
otelcol.extension.jaeger_remote_sampling "default" {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -136,9 +136,14 @@ otelcol.exporter.otlp "default" {
}
```

## Enabling Authentication
## Enable authentication

You can create a `datadog` receiver that requires authentication for requests. This is useful for limiting who can push data to the server. Note that not all OpenTelemetry Collector (otelcol) authentication plugins support receiver authentication. Please refer to the documentation for each `otelcol.auth.*` plugin to determine its compatibility.
You can create a `otelcol.receiver.datadog` component that requires authentication for requests. This is useful for limiting who can push data to the server.

{{< admonition type="note" >}}
Not all OpenTelemetry Collector authentication plugins support receiver authentication.
Refer to the [documentation](https://grafana.com/docs/alloy/<ALLOY_VERSION>/reference/components/otelcol/) for each `otelcol.auth.*` component to determine its compatibility.
{{< /admonition >}}

```alloy
otelcol.receiver.datadog "default" {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -151,9 +151,14 @@ prometheus.remote_write "mimir" {
}
```

## Enabling Authentication
## Enable authentication

You can create a `influxdb` receiver that requires authentication for requests. This is useful for limiting who can push data to the server. Note that not all OpenTelemetry Collector (otelcol) authentication plugins support receiver authentication. Please refer to the documentation for each `otelcol.auth.*` plugin to determine its compatibility.
You can create a `otelcol.receiver.influxdb` component that requires authentication for requests. This is useful for limiting who can push data to the server.

{{< admonition type="note" >}}
Not all OpenTelemetry Collector authentication plugins support receiver authentication.
Refer to the [documentation](https://grafana.com/docs/alloy/<ALLOY_VERSION>/reference/components/otelcol/) for each `otelcol.auth.*` component to determine its compatibility.
{{< /admonition >}}

```alloy
otelcol.receiver.influxdb "influxdb_metrics" {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -264,9 +264,18 @@ otelcol.exporter.otlp "default" {

`otelcol.receiver.jaeger` supports [Gzip](https://en.wikipedia.org/wiki/Gzip) for compression.

## Enabling Authentication
## Enable authentication

You can create a `jaeger` receiver that requires authentication for requests. This is useful for limiting who can push data to the server. Note that not all OpenTelemetry Collector (otelcol) authentication plugins support receiver authentication. Please refer to the documentation for each `otelcol.auth.*` plugin to determine its compatibility. This functionality is currently limited to the GRPC/HTTP blocks.
You can create a `otelcol.receiver.jaeger` component that requires authentication for requests. This is useful for limiting who can push data to the server.

{{< admonition type="note" >}}
This functionality is currently limited to the GRPC/HTTP blocks.
{{< /admonition >}}

{{< admonition type="note" >}}
Not all OpenTelemetry Collector authentication plugins support receiver authentication.
Refer to the [documentation](https://grafana.com/docs/alloy/<ALLOY_VERSION>/reference/components/otelcol/) for each `otelcol.auth.*` component to determine its compatibility.
{{< /admonition >}}

```alloy
otelcol.receiver.jaeger "default" {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -210,9 +210,14 @@ otelcol.exporter.otlp "default" {
}
```

## Enabling Authentication
## Enable authentication

You can create a `opencensus` receiver that requires authentication for requests. This is useful for limiting who can push data to the server. Note that not all OpenTelemetry Collector (otelcol) authentication plugins support receiver authentication. Please refer to the documentation for each `otelcol.auth.*` plugin to determine its compatibility.
You can create a `otelcol.receiver.opencensus` component that requires authentication for requests. This is useful for limiting who can push data to the server.

{{< admonition type="note" >}}
Not all OpenTelemetry Collector authentication plugins support receiver authentication.
Refer to the [documentation](https://grafana.com/docs/alloy/<ALLOY_VERSION>/reference/components/otelcol/) for each `otelcol.auth.*` component to determine its compatibility.
{{< /admonition >}}

```alloy
otelcol.receiver.opencensus "default" {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -243,9 +243,14 @@ otelcol.exporter.otlp "default" {

`otelcol.receiver.otlp` supports [gzip](https://en.wikipedia.org/wiki/Gzip) for compression.

## Enabling Authentication
## Enable authentication

You can create a `otlp` receiver that requires authentication for requests. This is useful for limiting who can push data to the server. Note that not all OpenTelemetry Collector (otelcol) authentication plugins support receiver authentication. Please refer to the documentation for each `otelcol.auth.*` plugin to determine its compatibility.
You can create a `otelcol.reciever.otlp` component that requires authentication for requests. This is useful for limiting who can push data to the server.

{{< admonition type="note" >}}
Not all OpenTelemetry Collector authentication plugins support receiver authentication.
Refer to the [documentation](https://grafana.com/docs/alloy/<ALLOY_VERSION>/reference/components/otelcol/) for each `otelcol.auth.*` component to determine its compatibility.
{{< /admonition >}}

```alloy
otelcol.receiver.otlp "default" {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -143,9 +143,14 @@ otelcol.exporter.otlp "default" {
}
```

## Enabling Authentication
## Enable authentication

You can create a `zipkin` receiver that requires authentication for requests. This is useful for limiting who can push data to the server. Note that not all OpenTelemetry Collector (otelcol) authentication plugins support receiver authentication. Please refer to the documentation for each `otelcol.auth.*` plugin to determine its compatibility.
You can create a `otelcol.receiver.zipkin` component that requires authentication for requests. This is useful for limiting who can push data to the server.

{{< admonition type="note" >}}
Not all OpenTelemetry Collector authentication plugins support receiver authentication.
Refer to the [documentation](https://grafana.com/docs/alloy/<ALLOY_VERSION>/reference/components/otelcol/) for each `otelcol.auth.*` component to determine its compatibility.
{{< /admonition >}}

```alloy
otelcol.receiver.zipkin "default" {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -427,7 +427,7 @@ To troubleshoot, take the following steps in order:
You can use [Promtool][promtool] to inspect it and find out which metric series were sent by this {{< param "PRODUCT_NAME" >}} instance since the last WAL truncation event.
For example:
```
./promtool tsdb dump --match='{__name__=\"otelcol_connector_spanmetrics_duration_seconds_bucket\", http_method=\"GET\", job=\"ExampleJobName\"' /path/to/wal/
./promtool tsdb dump --match='{__name__="otelcol_connector_spanmetrics_duration_seconds_bucket", http_method="GET", job="ExampleJobName"}' /path/to/wal/
```

[clustering]: ../../configure/clustering
Expand Down
1 change: 1 addition & 0 deletions docs/sources/troubleshoot/debug.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,7 @@ Supported components:
* `otelcol.receiver.*`
* `prometheus.relabel`
{{< /admonition >}}
* `discovery.relabel`

## Debug using the UI

Expand Down
5 changes: 1 addition & 4 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -518,7 +518,7 @@ require (
github.com/go-viper/mapstructure/v2 v2.2.1 // indirect
github.com/go-zookeeper/zk v1.0.3 // indirect
github.com/gobwas/glob v0.2.3 // indirect
github.com/goccy/go-json v0.10.3 // indirect
github.com/goccy/go-json v0.10.4 // indirect
github.com/godbus/dbus v0.0.0-20190726142602-4481cbc300e2 // indirect
github.com/godbus/dbus/v5 v5.1.0 // indirect
github.com/gogo/googleapis v1.4.1 // indirect
Expand Down Expand Up @@ -945,6 +945,3 @@ exclude (
)

replace github.com/prometheus/procfs => github.com/prometheus/procfs v0.12.0

// TODO(ptodev): Remove when this PR has been merged: https://github.com/goccy/go-json/pull/490
replace github.com/goccy/go-json => github.com/grafana/go-json v0.0.0-20241106155216-71a03f133f5c
5 changes: 3 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -1619,6 +1619,9 @@ github.com/goburrow/modbus v0.1.0/go.mod h1:Kx552D5rLIS8E7TyUwQ/UdHEqvX5T8tyiGBT
github.com/goburrow/serial v0.1.0/go.mod h1:sAiqG0nRVswsm1C97xsttiYCzSLBmUZ/VSlVLZJ8haA=
github.com/gobwas/glob v0.2.3 h1:A4xDbljILXROh+kObIiy5kIaPYD8e96x1tgBhUI5J+Y=
github.com/gobwas/glob v0.2.3/go.mod h1:d3Ez4x06l9bZtSvzIay5+Yzi0fmZzPgnTbPcKjJAkT8=
github.com/goccy/go-json v0.9.11/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
github.com/goccy/go-json v0.10.4 h1:JSwxQzIqKfmFX1swYPpUThQZp/Ka4wzJdK0LWVytLPM=
github.com/goccy/go-json v0.10.4/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
github.com/godbus/dbus v0.0.0-20190726142602-4481cbc300e2 h1:ZpnhV/YsD2/4cESfV5+Hoeu/iUR3ruzNvZ+yQfO03a0=
github.com/godbus/dbus v0.0.0-20190726142602-4481cbc300e2/go.mod h1:bBOAhwG1umN6/6ZUMtDFBMQR8jRg9O75tm9K00oMsK4=
github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
Expand Down Expand Up @@ -1853,8 +1856,6 @@ github.com/grafana/dskit v0.0.0-20240104111617-ea101a3b86eb h1:AWE6+kvtE18HP+lRW
github.com/grafana/dskit v0.0.0-20240104111617-ea101a3b86eb/go.mod h1:kkWM4WUV230bNG3urVRWPBnSJHs64y/0RmWjftnnn0c=
github.com/grafana/go-gelf/v2 v2.0.1 h1:BOChP0h/jLeD+7F9mL7tq10xVkDG15he3T1zHuQaWak=
github.com/grafana/go-gelf/v2 v2.0.1/go.mod h1:lexHie0xzYGwCgiRGcvZ723bSNyNI8ZRD4s0CLobh90=
github.com/grafana/go-json v0.0.0-20241106155216-71a03f133f5c h1:yKBKEC347YZpgii1KazRCfxHsTaxMqWZzoivM1OTT50=
github.com/grafana/go-json v0.0.0-20241106155216-71a03f133f5c/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
github.com/grafana/go-offsets-tracker v0.1.7 h1:2zBQ7iiGzvyXY7LA8kaaSiEqH/Yx82UcfRabbY5aOG4=
github.com/grafana/go-offsets-tracker v0.1.7/go.mod h1:qcQdu7zlUKIFNUdBJlLyNHuJGW0SKWKjkrN6jtt+jds=
github.com/grafana/gocql v0.0.0-20200605141915-ba5dc39ece85/go.mod h1:crI9WX6p0IhrqB+DqIUHulRW853PaNFf7o4UprV//3I=
Expand Down
2 changes: 1 addition & 1 deletion internal/cmd/integration-tests/docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ services:
- "9009:9009"

tempo:
image: grafana/tempo:latest
image: grafana/tempo:2.6.1
command: [ "-config.file=/etc/tempo.yaml" ]
volumes:
- ./configs/tempo/tempo.yaml:/etc/tempo.yaml
Expand Down
43 changes: 23 additions & 20 deletions internal/component/component_provider.go
Original file line number Diff line number Diff line change
Expand Up @@ -123,9 +123,10 @@ type Info struct {
ComponentName string // Name of the component.
Health Health // Current component health.

Arguments Arguments // Current arguments value of the component.
Exports Exports // Current exports value of the component.
DebugInfo interface{} // Current debug info of the component.
Arguments Arguments // Current arguments value of the component.
Exports Exports // Current exports value of the component.
DebugInfo interface{} // Current debug info of the component.
LiveDebuggingEnabled bool
}

// MarshalJSON returns a JSON representation of cd. The format of the
Expand All @@ -139,19 +140,20 @@ func (info *Info) MarshalJSON() ([]byte, error) {
}

componentDetailJSON struct {
Name string `json:"name"`
Type string `json:"type,omitempty"`
LocalID string `json:"localID"`
ModuleID string `json:"moduleID"`
Label string `json:"label,omitempty"`
References []string `json:"referencesTo"`
ReferencedBy []string `json:"referencedBy"`
Health *componentHealthJSON `json:"health"`
Original string `json:"original"`
Arguments json.RawMessage `json:"arguments,omitempty"`
Exports json.RawMessage `json:"exports,omitempty"`
DebugInfo json.RawMessage `json:"debugInfo,omitempty"`
CreatedModuleIDs []string `json:"createdModuleIDs,omitempty"`
Name string `json:"name"`
Type string `json:"type,omitempty"`
LocalID string `json:"localID"`
ModuleID string `json:"moduleID"`
Label string `json:"label,omitempty"`
References []string `json:"referencesTo"`
ReferencedBy []string `json:"referencedBy"`
Health *componentHealthJSON `json:"health"`
Original string `json:"original"`
Arguments json.RawMessage `json:"arguments,omitempty"`
Exports json.RawMessage `json:"exports,omitempty"`
DebugInfo json.RawMessage `json:"debugInfo,omitempty"`
CreatedModuleIDs []string `json:"createdModuleIDs,omitempty"`
LiveDebuggingEnabled bool `json:"liveDebuggingEnabled"`
}
)

Expand Down Expand Up @@ -196,10 +198,11 @@ func (info *Info) MarshalJSON() ([]byte, error) {
Message: info.Health.Message,
UpdatedTime: info.Health.UpdateTime,
},
Arguments: arguments,
Exports: exports,
DebugInfo: debugInfo,
CreatedModuleIDs: info.ModuleIDs,
Arguments: arguments,
Exports: exports,
DebugInfo: debugInfo,
CreatedModuleIDs: info.ModuleIDs,
LiveDebuggingEnabled: info.LiveDebuggingEnabled,
})
}

Expand Down
24 changes: 21 additions & 3 deletions internal/component/discovery/relabel/relabel.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,14 @@ package relabel

import (
"context"
"fmt"
"sync"

"github.com/grafana/alloy/internal/component"
alloy_relabel "github.com/grafana/alloy/internal/component/common/relabel"
"github.com/grafana/alloy/internal/component/discovery"
"github.com/grafana/alloy/internal/featuregate"
"github.com/grafana/alloy/internal/service/livedebugging"
"github.com/prometheus/prometheus/model/labels"
"github.com/prometheus/prometheus/model/relabel"
)
Expand Down Expand Up @@ -46,13 +48,23 @@ type Component struct {

mut sync.RWMutex
rcs []*relabel.Config

debugDataPublisher livedebugging.DebugDataPublisher
}

var _ component.Component = (*Component)(nil)
var _ component.LiveDebugging = (*Component)(nil)

// New creates a new discovery.relabel component.
func New(o component.Options, args Arguments) (*Component, error) {
c := &Component{opts: o}
debugDataPublisher, err := o.GetServiceData(livedebugging.ServiceName)
if err != nil {
return nil, err
}
c := &Component{
opts: o,
debugDataPublisher: debugDataPublisher.(livedebugging.DebugDataPublisher),
}

// Call to Update() to set the output once at the start
if err := c.Update(args); err != nil {
Expand Down Expand Up @@ -81,9 +93,13 @@ func (c *Component) Update(args component.Arguments) error {

for _, t := range newArgs.Targets {
lset := componentMapToPromLabels(t)
lset, keep := relabel.Process(lset, relabelConfigs...)
relabelled, keep := relabel.Process(lset, relabelConfigs...)
if keep {
targets = append(targets, promLabelsToComponent(lset))
targets = append(targets, promLabelsToComponent(relabelled))
}
componentID := livedebugging.ComponentID(c.opts.ID)
if c.debugDataPublisher.IsActive(componentID) {
c.debugDataPublisher.Publish(componentID, fmt.Sprintf("%s => %s", lset.String(), relabelled.String()))
}
}

Expand All @@ -95,6 +111,8 @@ func (c *Component) Update(args component.Arguments) error {
return nil
}

func (c *Component) LiveDebugging(_ int) {}

func componentMapToPromLabels(ls discovery.Target) labels.Labels {
res := make([]labels.Label, 0, len(ls))
for k, v := range ls {
Expand Down
2 changes: 1 addition & 1 deletion internal/component/otelcol/auth/auth.go
Original file line number Diff line number Diff line change
Expand Up @@ -304,7 +304,7 @@ func (a *Auth) Update(args component.Arguments) error {
)

// Schedule the components to run once our component is running.
a.sched.Schedule(host, components...)
a.sched.Schedule(a.ctx, func() {}, host, components...)
return nil
}

Expand Down
Loading

0 comments on commit 3913159

Please sign in to comment.