From 5f09452d45cebe97a13d15d570a83d9271c5d3f7 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Tue, 21 May 2024 11:13:30 -0700 Subject: [PATCH] reorganize configuration syntax intro (#849) (#897) * reorganize configuration syntax intro * Apply suggestions from code review accepted @clayton-cornell edits Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com> --------- Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com> (cherry picked from commit 8afa9728e9c3067aae074ee18991cbada01b7a92) Co-authored-by: David Allen --- .../concepts/configuration-syntax/_index.md | 116 +++++++++++------- 1 file changed, 71 insertions(+), 45 deletions(-) diff --git a/docs/sources/concepts/configuration-syntax/_index.md b/docs/sources/concepts/configuration-syntax/_index.md index a07b94cb97..741fb298a3 100644 --- a/docs/sources/concepts/configuration-syntax/_index.md +++ b/docs/sources/concepts/configuration-syntax/_index.md @@ -9,50 +9,81 @@ weight: 10 # {{% param "PRODUCT_NAME" %}} configuration syntax {{< param "FULL_PRODUCT_NAME" >}} dynamically configures and connects components with the {{< param "PRODUCT_NAME" >}} configuration syntax. +{{< param "PRODUCT_NAME" >}} handles the collection, transformation, and delivery of telemetry data. +Each component in the configuration handles one of those tasks or specifies how data flows and how the components are bound together. -The {{< param "PRODUCT_NAME" >}} syntax aims to reduce errors in configuration files by making configurations easier to read and write. -{{< param "PRODUCT_NAME" >}} configurations use blocks that can be easily copied and pasted from the documentation to help you get started as quickly as possible. - -An {{< param "PRODUCT_NAME" >}} configuration file tells {{< param "PRODUCT_NAME" >}} which components to launch and how to bind them together into a pipeline. +{{< figure src="/media/docs/alloy/flow-diagram-small-alloy.png" alt="Alloy flow diagram" >}} -The {{< param "PRODUCT_NAME" >}} syntax uses blocks, attributes, and expressions. +The following simple example shows the basic concepts and how an {{< param "PRODUCT_NAME" >}} configuration comes together into a pipeline. ```alloy -// Create a local.file component labeled my_file. -// This can be referenced by other components as local.file.my_file. -local.file "my_file" { - filename = "/tmp/my-file.txt" +// Collection: mount a local directory with a certain path spec +local.file_match "applogs" { + path_targets = [{"__path__" = "/tmp/app-logs/app.log"}] } -// Pattern for creating a labeled block, which the above block follows: -BLOCK_NAME "BLOCK_LABEL" { - // Block body - IDENTIFIER = EXPRESSION // Attribute +// Collection: Take the file match as input, and scrape those mounted log files +loki.source.file "local_files" { + targets = local.file_match.applogs.targets + + // This specifies which component should process the logs next, the "link in the chain" + forward_to = [loki.process.add_new_label.receiver] } -// Pattern for creating an unlabeled block: -BLOCK_NAME { - // Block body - IDENTIFIER = EXPRESSION // Attribute +// Transformation: pull some data out of the log message, and turn it into a label +loki.process "add_new_label" { + stage.logfmt { + mapping = { + "extracted_level" = "level", + } + } + + // Add the value of "extracted_level" from the extracted map as a "level" label + stage.labels { + values = { + "level" = "extracted_level", + } + } + + // The next link in the chain is the local_loki "receiver" (receives the telemetry) + forward_to = [loki.write.local_loki.receiver] +} + +// Anything that comes into this component gets written to the loki remote API +loki.write "local_loki" { + endpoint { + url = "http://loki:3100/loki/api/v1/push" + } } ``` -{{< param "PRODUCT_NAME" >}} is designed with the following requirements in mind: +The {{< param "PRODUCT_NAME" >}} syntax aims to reduce errors in configuration files by making configurations easier to read and write. +The {{< param "PRODUCT_NAME" >}} syntax uses blocks, attributes, and expressions. +The blocks can be copied and pasted from the documentation to help you get started as quickly as possible. -* _Fast_: The configuration language must be fast so the component controller can quickly evaluate changes. -* _Simple_: The configuration language must be easy to read and write to minimize the learning curve. -* _Debuggable_: The configuration language must give detailed information when there's a mistake in the configuration file. +The {{< param "PRODUCT_NAME" >}} syntax is declarative, so ordering components, blocks, and attributes does not matter. +The relationship between components determines the order of operations in the pipeline. -The {{< param "PRODUCT_NAME" >}} configuration syntax is a distinct language with custom syntax and features, such as first-class functions. +## Blocks -* Blocks are a group of related settings and usually represent creating a component. - Blocks have a name that consists of zero or more identifiers separated by `.`, an optional user label, and a body containing attributes and nested blocks. -* Attributes appear within blocks and assign a value to a name. -* Expressions represent a value, either literally or by referencing and combining other values. - You use expressions to compute a value for an attribute. +You use _Blocks_ to configure components and groups of attributes. +Each block can contain any number of attributes or nested blocks. +Blocks are steps in the overall pipeline expressed by the configuration. -The {{< param "PRODUCT_NAME" >}} syntax is declarative, so ordering components, blocks, and attributes within a block isn't significant. -The relationship between components determines the order of operations. +```alloy +prometheus.remote_write "default" { + endpoint { + url = "http://localhost:9009/api/prom/push" + } +} +``` + +The preceding example has two blocks: + +* `prometheus.remote_write "default"`: A labeled block which instantiates a `prometheus.remote_write` component. + The label is the string `"default"`. +* `endpoint`: An unlabeled block inside the component that configures an endpoint to send metrics to. + This block sets the `url` attribute to specify the endpoint. ## Attributes @@ -84,26 +115,21 @@ The most common expression is to reference the exports of a component, for examp You form a reference to a component's exports by merging the component's name (for example, `local.file`), label (for example, `password_file`), and export name (for example, `content`), delimited by a period. -## Blocks +## Configuration syntax design goals -You use _Blocks_ to configure components and groups of attributes. -Each block can contain any number of attributes or nested blocks. +{{< param "PRODUCT_NAME" >}} is: -```alloy -prometheus.remote_write "default" { - endpoint { - url = "http://localhost:9009/api/prom/push" - } -} -``` +* _Fast_: The configuration language is fast, so the component controller can quickly evaluate changes. +* _Simple_: The configuration language is easy to read and write to minimize the learning curve. +* _Easy to debug_: The configuration language gives detailed information when there's a mistake in the configuration file. -The preceding example has two blocks: - -* `prometheus.remote_write "default"`: A labeled block which instantiates a `prometheus.remote_write` component. - The label is the string `"default"`. -* `endpoint`: An unlabeled block inside the component that configures an endpoint to send metrics to. - This block sets the `url` attribute to specify the endpoint. +The {{< param "PRODUCT_NAME" >}} configuration syntax is a distinct language with custom syntax and features, such as first-class functions. +* Blocks are a group of related settings and usually represent creating a component. + Blocks have a name that consists of zero or more identifiers separated by `.`, an optional user label, and a body containing attributes and nested blocks. +* Attributes appear within blocks and assign a value to a name. +* Expressions represent a value, either literally or by referencing and combining other values. + You use expressions to compute a value for an attribute. ## Tooling