Proposal: Support JSON and YAML forms in place of River

[rfratto]

While I (personally, with obvious bias) believe the learning curve of River isn't too high:

1. Some people may still be uncomfortable learning something new.
2. River doesn't integrate as well with tooling like Helm or Jsonnet and requires using helper libraries like river-jsonnet.

One possible way to solve these problems is to introduce JSON and YAML representation that can be used in place of River, but still support River expressions instead of them.

Proposal

There are three challenges with mapping JSON and YAML to River:

• The representation of blocks (e.g., being able to distinguish between YAML creating a block vs creating an attribute)
• The representation of River expressions.
• The representation of multiple blocks of the same name (e.g., rule blocks in prometheus.relabel).

I propose that blocks be given a special syntax of block BLOCK_NAME BLOCK_LABEL. For example: block prometheus.remote_write default. Note the lack of quotes for the label; quotes are avoided to make it easier for JSON, which would otherwise require escaping the double quotes. This introduces the risk of improperly formatting the block, such as not putting a space between the block name and block label.

Expressions will be represented by the interpolated string syntax, ${EXPR}, as proposed in grafana/river#15.

If there is content before or after the expression, the expression is converted into a string and concatenated to the strings before and after it:

field: "Hello, ${2022 + 1}!"

would evaluate to:

field: "Hello, 2023!"

A string which contains only the interpolated string syntax is treated as a pure expression:

field: "${2022}"

would evaluate to a number, not a string:

field: 2022 

If the expression cannot be converted into a string, a VM error would be produced.

Finally, blocks may be defined as an array if there are multiple blocks of the same name:

block rule:
  - # relabel rule 1 
  - # relabel rule 2 
  - ... 

Internally, JSON and YAML files will be parsed into a River AST. AST elements which store position (like the position of { for a block) will have to get creative with where a { hypothetically would be for the start of the block.

Examples

The following examples are all three equivalent Flow configuration files using River, YAML, and JSON:

logging {
	level  = "debug"
	format = "logfmt"
}

tracing {
	sampling_fraction = 1
	write_to          = [otelcol.exporter.otlp.tempo.input]
}

otelcol.exporter.otlp "tempo" {
	client {
		endpoint = "localhost:4317"

		tls {
			insecure = true
		}
	}
}

prometheus.integration.node_exporter { /* use defaults */ }

prometheus.scrape "default" {
	targets    = prometheus.integration.node_exporter.targets
	forward_to = [prometheus.remote_write.default.receiver]
}

prometheus.remote_write "default" {
	endpoint {
		url = "http://localhost:9009/api/prom/push"
	}
}

block logging:
  level: debug
  format: logfmt

block tracing:
  sampling_fraction: 1
  write_to: 
    - ${otelcol.exporter.otlp.tempo.input}

block otelcol.exporter.otlp tempo:
  block client:
    endpoint: localhost:4317
    block tls:
      insecure: true

block prometheus.integration.node_exporter: {} # use defaults

block prometheus.scrape default:
  targets: ${prometheus.integration.node_exporter.targets}
  forward_to: 
    - ${prometheus.remote_write.default.receiver}

block prometheus.remote_write default:
  block endpoint:
    url: http://localhost:9009/api/prom/push

{
  "block logging": {
    "level": "debug",
    "format": "logfmt"
  }, 

  "block tracing": {
    "sampling_fraction": 1,
    "write_to": [
      "${otelcol.exporter.otlp.tempo.input}"
    ]
  },

  "block otelcol.exporter.otlp tempo": {
    "block client": {
      "endpoint": "localhost:4317",
      "block tls": {
        "insecure": true
      }
    }
  },

  "block prometheus.integration.node_exporter": {},

  "block prometheus.scrape default": {
    "targets": "${prometheus.integration.node_exporter.targets}",
    "forward_to": [
      "${prometheus.remote_write.default.receiver}"
    ]
  },

  "block prometheus.remote_write default": {
    "block endpoint": [{
      "url": "http://localhost:9009/api/prom/push"
    }]
  }
}

Potentially removing the "block" keyword

The block keyword is needed to identify at parse time whether a key in a JSON or YAML object is referring to a block or an attribute. Other keywords could potentially be used which are less verbose.

Alternatively, the River AST could be modified to include an ambiguous AST node which either represents a block or an attribute. The River VM would have to be responsible for resolving this AST node into its final form, and fail the evaluation if an expression was used to assign a block. This shouldn't have any significant overhead, but would require introducing an AST element for something which isn't River.

Modifying the AST to remove the block keyword would change the above YAML to the following:

logging:
  level: debug
  format: logfmt

tracing:
  sampling_fraction: 1
  write_to: 
    - ${otelcol.exporter.otlp.tempo.input}

otelcol.exporter.otlp tempo:
  client:
    endpoint: localhost:4317
    tls:
      insecure: true

prometheus.integration.node_exporter: {} # use defaults

prometheus.scrape default:
  targets: ${prometheus.integration.node_exporter.targets}
  forward_to: 
    - ${prometheus.remote_write.default.receiver}

prometheus.remote_write default:
  endpoint:
    url: http://localhost:9009/api/prom/push

Counterarguments

There are some counterarguments I can imagine to this entire proposal:

• We haven't really gotten much complaints about River, so doing this might not be fully justified yet.
• River is still used here for expressions, and only the base syntax of the file themselves is JSON/YAML, so users would still need to learn River.
• River would still be preferable in the future for concepts which couldn't translate to JSON/YAML, such as variables.

[rfratto]

cc @Logiraptor for thoughts since you mentioned wanting to be able to use YAML in Helm charts for #2604.

[mattdurham]

Biased viewpoint but, I would prefer to invest time into the conversion of yaml -> river instead of supporting yaml. I believe as river and flow expands it's going to be harder to maintain parity.

[rfratto]

Biased viewpoint but, I would prefer to invest time into the conversion of yaml -> river instead of supporting yaml. I believe as river and flow expands it's going to be harder to maintain parity.

Just to make sure I understand, you're talking about the conversion of the static agent config to a Flow config? Just want to make sure; this proposal could also be interpreted as supporting "yaml -> river" in a way.

[mattdurham]

Correct, you can convert the static agent config as it exists today from yaml to river, but you cannot use expressions in the yaml or any other river-specific features or any new features/components introduced in flow.

[Logiraptor]

I think any of the proposals here could technically work for my use-case. I'll leave some more details here to hopefully help. I'll reduce this to a simple case to avoid too much redundant information:

In the Mimir helm chart, I'd like to make the work from #2604 work out of the box. That means I need a few things to happen:

1. I need a way to deploy the agent in flow mode (assuming this is solved by Introduce a Helm chart for Grafana Agent #2695)
2. I need to wire the "known" facts into the flow config like the Mimir address at mimir-gateway:80 or whatever
3. I need to give the user a way to set the unknown facts. For example they may want to send to grafana cloud instead of the local Mimir or they may want to customize the label selectors.

With the current agent chart as a subchart, I think it would look something like this:

I would add the agent chart as a subchart and in mimir values.yaml:

agent:
  mode: 'flow'

  configMap:
    # disable this configmap so I can generate my own
    create: false
    name: mimir-agent-config

Then I would add a template to our helm chart to generate the new configmap:

# UNTESTED, probably wrong somehow:
{{- if .Values.metaMonitoring.rules.enabled }}
apiVersion: v1
kind: ConfigMap
metadata:
  name: mimir-agent-config
  labels:
    # ...
data:
  config.river: |
    mimir.rules.kubernetes "default" {

      {{ if .Values.metaMonitoring.rules.address }}
      address = "http://mimir-gateway.mimir.svc.cluster.local"
      {{ end }}

      {{ if .Values.metaMonitoring.rules.tenant_id }}
      tenant_id = {{ .Values.metaMonitoring.rules.tenant_id }}
      {{ end }}

      {{ with .Values.metaMonitoring.rules.client }}
      http_client_config {
        {{ with .basic_auth }}
        basic_auth {
          {{ if .username }}
          username = {{ .username }}
          {{ end }}
          {{ if .username }}
          password = {{ .password }}
          {{ end }}
        }
        {{ end }}
      }
      {{ end }}

      # TODO: also support rule_selector and rule_namespace_selector, as well as another dozen fields in http_client_config
    }
{{- end }}

What I'm specifically trying to avoid is the need to generate correct river syntax. I think this problem extends to any environment where configuration is generated by machines, as evidenced by the existence of a river-jsonnet library.

If I needed to build this feature today I would probably take a different approach and let the user define a snippet of river which just gets pasted into the agent config map. For example, check out what we do with the nginx configuration: https://github.com/grafana/mimir/blob/main/operations/helm/charts/mimir-distributed/values.yaml#L2090. Importantly, you can use helm templating in that string and we provide a reasonable default value. FYI The helm templating is enabled by a function in helm, so it doesn't automatically apply to every string.

[mattdurham]

Closing per May community call conversation

[mattdurham]

Closing per May community call conversation