Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport release-v3] feat(metrics): add additionalServiceMonitors setting #3293

Merged
merged 1 commit into from
Sep 21, 2023
Merged

[Backport release-v3] feat(metrics): add additionalServiceMonitors setting #3293

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .changelog/3292.added.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
feat(metrics): add additionalServiceMonitors setting
1 change: 1 addition & 0 deletions deploy/helm/sumologic/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,7 @@ The following table lists the configurable parameters of the Sumo Logic chart an
| `sumologic.metrics.remoteWriteProxy.podAnnotations` | Additional annotations for for the remote write proxy container. | `{}` |
| `sumologic.metrics.remoteWriteProxy.config.port` | Port on which remote write proxy is going to be exposed | `8080` |
| `sumologic.metrics.serviceMonitors` | Configuration of Sumo Logic Kubernetes Collection components serviceMonitors | See [values.yaml] |
| `sumologic.metrics.additionalServiceMonitors` | List of ServiceMonitor objects to create. | See [values.yaml] |
| `sumologic.metrics.collector.otelcol.enabled` | Enable experimental otelcol metrics collector | See [values.yaml] |
| `sumologic.metrics.collector.otelcol.scrapeInterval` | The default scrape interval for the collector. | `30s` |
| `sumologic.metrics.collector.otelcol.replicaCount` | Replica count for the experimental otelcol metrics collector | `1` |
Expand Down
5 changes: 3 additions & 2 deletions deploy/helm/sumologic/templates/metrics/prometheus/servicemonitors.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
{{- $kps := get .Values "kube-prometheus-stack" -}}
{{- if or (and (typeIs "<nil>" $kps.enabled) (.Values.sumologic.metrics.enabled)) (and (typeIs "bool" $kps.enabled) ($kps.enabled)) -}}
{{- $serviceMonitors := concat .Values.sumologic.metrics.serviceMonitors .Values.sumologic.metrics.additionalServiceMonitors -}}
---
{{- if .Values.sumologic.metrics.serviceMonitors }}
{{- if $serviceMonitors }}
apiVersion: v1
kind: List
items:
{{- range .Values.sumologic.metrics.serviceMonitors }}
{{- range $serviceMonitors }}
- apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
Expand Down
3 changes: 3 additions & 0 deletions deploy/helm/sumologic/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -739,6 +739,9 @@ sumologic:
matchLabels:
app: kube-prometheus-stack-prometheus

## User-defined ServiceMonitors go here
additionalServiceMonitors: []

## The type of source we send to in Sumo. The possible values are http and otlp.
## Consult the documentation for more information.
sourceType: http
Expand Down
41 changes: 28 additions & 13 deletions docs/collecting-application-metrics.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,17 @@ annotations:

### Application metrics are exposed (multiple enpoints scenario)

> [!NOTE]
> Use `sumologic.metrics.additionalServiceMonitors` instead of `kube-prometheus-stack.prometheus.additionalServiceMonitors`. They have
> identical behaviour and can even be used in tandem, but the latter only works if Prometheus is enabled, and won't work with the Otel
> metrics collector which is the default in v4 of the Chart.

If you want to scrape metrics from multiple endpoints in a single Pod, you need a Service which points to the Pod and also to configure
`kube-prometheus-stack.prometheus.additionalServiceMonitors` in your `user-values.yaml`:
`sumologic.metrics.additionalServiceMonitors` in your `user-values.yaml`:

```yaml
kube-prometheus-stack:
prometheus:
sumologic:
metrics:
additionalServiceMonitors:
- name: <service monitor name>
endpoints:
Expand Down Expand Up @@ -157,8 +162,8 @@ spec:
In order to scrape metrics from the above objects, the following configuration should be applied to `user-values.yaml`:

```yaml
kube-prometheus-stack:
prometheus:
sumologic:
metrics:
additionalServiceMonitors:
- name: my-custom-app-service-monitor
endpoints:
Expand Down Expand Up @@ -347,14 +352,24 @@ sumologic:

If you do not see your metrics in Sumo Logic, please check the following stages:

- [Check if metrics are in Prometheus](#check-if-metrics-are-in-prometheus)

- [Investigate Prometheus scrape configuration](#investigate-prometheus-scrape-configuration)
- [Pod is visible in Prometheus targets](#pod-is-visible-in-prometheus-targets)
- [There is no target for serviceMonitor](#there-is-no-target-for-servicemonitor)
- [Pod is not visible in target for custom serviceMonitor](#pod-is-not-visible-in-target-for-custom-servicemonitor)

- [Check if Prometheus knows how to send metrics to Sumo Logic](#check-if-prometheus-knows-how-to-send-metrics-to-sumo-logic)
- [Collecting Application Metrics](#collecting-application-metrics)
- [Scraping metrics](#scraping-metrics)
- [Application metrics are exposed (one endpoint scenario)](#application-metrics-are-exposed-one-endpoint-scenario)
- [Application metrics are exposed (multiple enpoints scenario)](#application-metrics-are-exposed-multiple-enpoints-scenario)
- [Example](#example)
- [Application metrics are not exposed](#application-metrics-are-not-exposed)
- [Metrics modifications](#metrics-modifications)
- [Filtering metrics](#filtering-metrics)
- [Default attributes](#default-attributes)
- [Renaming metric](#renaming-metric)
- [Adding or renaming metadata](#adding-or-renaming-metadata)
- [Investigation](#investigation)
- [Check if metrics are in Prometheus](#check-if-metrics-are-in-prometheus)
- [Investigate Prometheus scrape configuration](#investigate-prometheus-scrape-configuration)
- [Pod is visible in Prometheus targets](#pod-is-visible-in-prometheus-targets)
- [There is no target for serviceMonitor](#there-is-no-target-for-servicemonitor)
- [Pod is not visible in target for custom serviceMonitor](#pod-is-not-visible-in-target-for-custom-servicemonitor)
- [Check if Prometheus knows how to send metrics to Sumo Logic](#check-if-prometheus-knows-how-to-send-metrics-to-sumo-logic)

### Check if metrics are in Prometheus

Expand Down
108 changes: 108 additions & 0 deletions tests/helm/metrics_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -187,3 +187,111 @@ sumologic:
assert.Contains(t, otelConfig.Service.Pipelines.Metrics.Processors, "transform/drop_routing_attribute")
assert.Equal(t, otelConfig.Service.Pipelines.Metrics.Exporters, []string{"sumologic/default"})
}

func TestNoPrometheusServiceMonitors(t *testing.T) {
t.Parallel()
allTemplatePaths := []string{
"templates/metrics/prometheus/servicemonitors.yaml",
}

testCases := []struct {
Name string
ValuesYaml string
ExpectedNames []string
TemplatePaths []string
}{
{
Name: "additionalServiceMonitor",
ValuesYaml: `
sumologic:
metrics:
additionalServiceMonitors:
- name: collection-sumologic-fluentd-logs-test
additionalLabels:
sumologic.com/app: fluentd-logs
endpoints:
- port: metrics
namespaceSelector:
matchNames:
- $(NAMESPACE)
selector:
matchLabels:
sumologic.com/app: fluentd-logs
sumologic.com/scrape: "true"
`,
ExpectedNames: []string{
"collection-sumologic-fluentd-logs",
"collection-sumologic-otelcol-logs",
"collection-sumologic-fluentd-metrics",
"collection-sumologic-otelcol-metrics",
"collection-sumologic-metrics-collector",
"collection-sumologic-fluentd-events",
"collection-sumologic-fluent-bit",
"collection-sumologic-otelcol-logs-collector",
"collection-sumologic-otelcol-events",
"collection-sumologic-otelcol-traces",
"collection-sumologic-prometheus",
"collection-sumologic-fluentd-logs-test",
},
TemplatePaths: allTemplatePaths,
},
{
Name: "onlyAdditionalServiceMonitor",
ValuesYaml: `
sumologic:
metrics:
serviceMonitors: []
additionalServiceMonitors:
- name: collection-sumologic-fluentd-logs-test
additionalLabels:
sumologic.com/app: fluentd-logs
endpoints:
- port: metrics
namespaceSelector:
matchNames:
- $(NAMESPACE)
selector:
matchLabels:
sumologic.com/app: fluentd-logs
sumologic.com/scrape: "true"
`,
ExpectedNames: []string{
"collection-sumologic-fluentd-logs-test",
},
TemplatePaths: allTemplatePaths,
},
{
Name: "default",
ValuesYaml: "",
ExpectedNames: []string{
"collection-sumologic-fluentd-logs",
"collection-sumologic-otelcol-logs",
"collection-sumologic-fluentd-metrics",
"collection-sumologic-otelcol-metrics",
"collection-sumologic-metrics-collector",
"collection-sumologic-fluentd-events",
"collection-sumologic-fluent-bit",
"collection-sumologic-otelcol-logs-collector",
"collection-sumologic-otelcol-events",
"collection-sumologic-otelcol-traces",
"collection-sumologic-prometheus",
},
TemplatePaths: allTemplatePaths,
},
}

for _, tt := range testCases {
t.Run(tt.Name, func(t *testing.T) {
names := []string{}
for _, templatePath := range tt.TemplatePaths {
servicemonitors := GetServiceMonitors(t, tt.ValuesYaml, templatePath)
for _, sm := range servicemonitors {
names = append(names, sm.Name)
}
}

assert.Equal(t, tt.ExpectedNames, names)
})
}

}