Skip to content

Commit

Permalink
[release/v1.0] Clean up word use, reduce passive voice, fix spelling …
Browse files Browse the repository at this point in the history 
…and capitalisation use #1094 (#1104)
  • Loading branch information
@clayton-cornell
clayton-cornell authored Jun 21, 2024
1 parent cd8a980 commit 6f98bc3
Show file tree
Hide file tree
Showing 5 changed files with 265 additions and 304 deletions.
95 changes: 41 additions & 54 deletions docs/sources/reference/components/loki.source.file.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,20 +8,19 @@ title: loki.source.file

`loki.source.file` reads log entries from files and forwards them to other `loki.*` components.

Multiple `loki.source.file` components can be specified by giving them
different labels.
You can specify multiple `loki.source.file` components by giving them different labels.

{{< admonition type="note" >}}
`loki.source.file` does not handle file discovery. You can use `local.file_match` for file discovery.
Refer to the [File Globbing](#file-globbing) example for more information.
`loki.source.file` doesn't handle file discovery. You can use `local.file_match` for file discovery.
Refer to the [File globbing](#file-globbing) example for more information.
{{< /admonition >}}

## Usage

```alloy
loki.source.file "LABEL" {
targets = TARGET_LIST
forward_to = RECEIVER_LIST
loki.source.file "<LABEL>" {
targets = <TARGET_LIST>
forward_to = <RECEIVER_LIST>
}
```

Expand All @@ -31,13 +30,13 @@ The component starts a new reader for each of the given `targets` and fans out l

`loki.source.file` supports the following arguments:

| Name | Type | Description | Default | Required |
| ------------------------| -------------------- | ----------------------------------------------------------------------------------- | ------- | -------- |
| `targets` | `list(map(string))` | List of files to read from. | | yes |
| `forward_to` | `list(LogsReceiver)` | List of receivers to send log entries to. | | yes |
| `encoding` | `string` | The encoding to convert from when reading files. | `""` | no |
| `tail_from_end` | `bool` | Whether a log file should be tailed from the end if a stored position is not found. | `false` | no |
| `legacy_positions_file` | `string` | Allows conversion from legacy positions file. | `""` | no |
| Name | Type | Description | Default | Required |
|-------------------------|----------------------|-----------------------------------------------------------------------------|---------|----------|
| `targets` | `list(map(string))` | List of files to read from. | | yes |
| `forward_to` | `list(LogsReceiver)` | List of receivers to send log entries to. | | yes |
| `encoding` | `string` | The encoding to convert from when reading files. | `""` | no |
| `tail_from_end` | `bool` | Whether a log file is tailed from the end if a stored position isn't found. | `false` | no |
| `legacy_positions_file` | `string` | Allows conversion from legacy positions file. | `""` | no |

The `encoding` argument must be a valid [IANA encoding][] name. If not set, it
defaults to UTF-8.
Expand All @@ -47,11 +46,11 @@ When set to true, only new logs will be read, ignoring the existing ones.


{{< admonition type="note" >}}
The `legacy_positions_file` argument is used when you are transitioning from legacy. The legacy positions file will be rewritten into the new format.
This operation will only occur if the new positions file does not exist and the `legacy_positions_file` is valid.
Once converted successfully, the `legacy_positions_file` will be deleted.
If you add any labels before `loki.source.file`, then the positions file will conversion will not work.
The legacy positions file did not have a concept of labels in the positions file, so the conversion assumes no labels.
The `legacy_positions_file` argument is used when you are transitioning from legacy. The legacy positions file is rewritten into the new format.
This operation only occurs if the positions file doesn't exist and the `legacy_positions_file` is valid.
After the configuration is successfully converted, the `legacy_positions_file` is deleted.
If you add any labels before `loki.source.file`, then the positions file conversion won't work.
The legacy positions file didn't have a concept of labels in the positions file, so the conversion assumes no labels.
{{< /admonition >}}

## Blocks
Expand All @@ -77,18 +76,17 @@ compressed files. The following arguments are supported:
| `initial_delay` | `duration` | Time to wait before starting to read from new compressed files. | 0 | no |
| `format` | `string` | Compression format. | | yes |

If you compress a file under a folder being scraped, `loki.source.file` might
try to ingest your file before you finish compressing it. To avoid it, pick
an `initial_delay` that is enough to avoid it.
If you compress a file under a folder being scraped, `loki.source.file` might try to ingest your file before you finish compressing it.
To avoid it, pick an `initial_delay` that's long enough to avoid it.

Currently supported compression formats are:

- `gz` - for gzip
- `gz` - for Gzip
- `z` - for zlib
- `bz2` - for bzip2

The component can only support one compression format at a time, in order to
handle multiple formats, you will need to create multiple components.
The component can only support one compression format at a time.
To handle multiple formats, you must create multiple components.

### file_watch block

Expand All @@ -106,20 +104,19 @@ If file changes are detected, the poll frequency is reset to `min_poll_frequency

## Exported fields

`loki.source.file` does not export any fields.
`loki.source.file` doesn't export any fields.

## Component health

`loki.source.file` is only reported as unhealthy if given an invalid
configuration.
`loki.source.file` is only reported as unhealthy if given an invalid configuration.

## Debug information

`loki.source.file` exposes some target-level debug information per reader:

- The tailed path.
- Whether the reader is currently running.
- What is the last recorded read offset in the positions file.
- Whether the reader is running.
- The last recorded read offset in the positions file.

## Debug metrics

Expand All @@ -131,40 +128,33 @@ configuration.

## Component behavior

If the decompression feature is deactivated, the component will continuously monitor and 'tail' the files.
In this mode, upon reaching the end of a file, the component remains active, awaiting and reading new entries in real-time as they are appended.
If the decompression feature is deactivated, the component continuously monitors and tails the files.
The component remains active after reaching the end of a file, and reads new entries in real-time as they're appended to the file.

Each element in the list of `targets` as a set of key-value pairs called _labels_.
The set of targets can either be _static_, or dynamically provided periodically
by a service discovery component. The special label `__path__` _must always_ be
present and must point to the absolute path of the file to read from.
The set of targets can either be _static_, or dynamically provided periodically by a service discovery component.
The special label `__path__` _must always_ be present and must contain the absolute path of the file to read from.

<!-- TODO(@tpaschalis) refer to local.file_match -->

The `__path__` value is available as the `filename` label to each log entry
the component reads. All other labels starting with a double underscore are
considered _internal_ and are removed from the log entries before they're
passed to other `loki.*` components.
The `__path__` value is available as the `filename` label to each log entry the component reads.
All other labels starting with a double underscore are considered _internal_ and are removed from the log entries before they're passed to other `loki.*` components.

The component uses its data path, a directory named after the domain's
fully qualified name, to store its _positions file_. The positions file is used
to store read offsets, so that if a component or {{< param "PRODUCT_NAME" >}} restarts,
`loki.source.file` can pick up tailing from the same spot.
The component uses its data path, a directory named after the domain's fully qualified name, to store its _positions file_.
The positions file stores read offsets, so that if a component or {{< param "PRODUCT_NAME" >}} restarts, `loki.source.file` can pick up tailing from the same spot.

The data path is inside the directory configured by the `--storage.path` [command line argument][cmd-args].

If a file is removed from the `targets` list, its positions file entry is also
removed. When it's added back on, `loki.source.file` starts reading it from the
beginning.
If a file is removed from the `targets` list, its positions file entry is also removed.
When it's added back on, `loki.source.file` starts reading it from the beginning.

[cmd-args]: ../../cli/run/

## Examples

### Static targets

This example collects log entries from the files specified in the targets
argument and forwards them to a `loki.write` component to be written to Loki.
This example collects log entries from the files specified in the targets argument and forwards them to a `loki.write` component.

```alloy
loki.source.file "tmpfiles" {
Expand All @@ -185,9 +175,8 @@ loki.write "local" {

### File globbing

This example collects log entries from the files matching `*.log` pattern
using `local.file_match` component. When files appear or disappear, the list of
targets will be updated accordingly.
This example collects log entries from the files matching `*.log` pattern using `local.file_match` component.
When files appear or disappear, the list of targets is updated accordingly.

```alloy
Expand All @@ -211,9 +200,7 @@ loki.write "local" {

### Decompression

This example collects log entries from the compressed files matching `*.gz`
pattern using `local.file_match` component and the decompression configuration
on the `loki.source.file` component.
This example collects log entries from the compressed files matching `*.gz` pattern using `local.file_match` component and the decompression configuration on the `loki.source.file` component.

```alloy
Expand Down
45 changes: 22 additions & 23 deletions docs/sources/reference/components/loki.source.windowsevent.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ title: loki.source.windowsevent

`loki.source.windowsevent` reads events from Windows Event Logs and forwards them to other `loki.*` components.

Multiple `loki.source.windowsevent` components can be specified by giving them different labels.
You can specify multiple `loki.source.windowsevent` components by giving them different labels.

## Usage

Expand All @@ -20,34 +20,34 @@ loki.source.windowsevent "LABEL" {
```

## Arguments
The component starts a new reader and fans out
log entries to the list of receivers passed in `forward_to`.

The component starts a reader and fans out log entries to the list of receivers passed in `forward_to`.

`loki.source.windowsevent` supports the following arguments:

Name | Type | Description | Default | Required
------------------------ |----------------------|--------------------------------------------------------------------------------|----------------------------| --------
`locale` | `number` | Locale ID for event rendering. 0 default is Windows Locale. | `0` | no
`eventlog_name` | `string` | Event log to read from. | | See below.
`xpath_query` | `string` | Event log to read from. | `"*"` | See below.
`bookmark_path` | `string` | Keeps position in event log. | `"DATA_PATH/bookmark.xml"` | no
`poll_interval` | `duration` | How often to poll the event log. | `"3s"` | no
`exclude_event_data` | `bool` | Exclude event data. | `false` | no
`exclude_user_data` | `bool` | Exclude user data. | `false` | no
`exclude_event_message` | `bool` | Exclude the human-friendly event message. | `false` | no
`use_incoming_timestamp` | `bool` | When false, assigns the current timestamp to the log when it was processed. | `false` | no
`forward_to` | `list(LogsReceiver)` | List of receivers to send log entries to. | | yes
`labels` | `map(string)` | The labels to associate with incoming logs. | | no
Name | Type | Description | Default | Required
-------------------------|----------------------|-------------------------------------------------------------|----------------------------|-----------
`locale` | `number` | Locale ID for event rendering. 0 default is Windows Locale. | `0` | no
`eventlog_name` | `string` | Event log to read from. | | See below.
`xpath_query` | `string` | Event log to read from. | `"*"` | See below.
`bookmark_path` | `string` | Keeps position in event log. | `"DATA_PATH/bookmark.xml"` | no
`poll_interval` | `duration` | How often to poll the event log. | `"3s"` | no
`exclude_event_data` | `bool` | Exclude event data. | `false` | no
`exclude_user_data` | `bool` | Exclude user data. | `false` | no
`exclude_event_message` | `bool` | Exclude the human-friendly event message. | `false` | no
`use_incoming_timestamp` | `bool` | When false, assigns the current timestamp to the log. | `false` | no
`forward_to` | `list(LogsReceiver)` | List of receivers to send log entries to. | | yes
`labels` | `map(string)` | The labels to associate with incoming logs. | | no

{{< admonition type="note" >}}
`eventlog_name` is required if `xpath_query` does not specify the event log.
You can define `xpath_query` in [short or xml form](https://docs.microsoft.com/en-us/windows/win32/wes/consuming-events).
When using the XML form you can specify `event_log` in the `xpath_query`.
If using short form, you must define `eventlog_name`.
`eventlog_name` is required if `xpath_query` doesn't specify the event log.
You can define `xpath_query` in [short or XML form](https://docs.microsoft.com/windows/win32/wes/consuming-events).
When you use the XML form you can specify `event_log` in the `xpath_query`.
If you use the short form, you must define `eventlog_name`.
{{< /admonition >}}

{{< admonition type="note" >}}
`legacy_bookmark_path` is used to convert the Grafana Agent Static to a {{< param "PRODUCT_NAME" >}} bookmark, if `bookmark_path` does not exist.
`legacy_bookmark_path` converts the legacy Grafana Agent Static bookmark to a {{< param "PRODUCT_NAME" >}} bookmark, if `bookmark_path` doesn't exist.
{{< /admonition >}}

## Component health
Expand All @@ -56,8 +56,7 @@ If using short form, you must define `eventlog_name`.

## Example

This example collects log entries from the Event Log specified in `eventlog_name` and
forwards them to a `loki.write` component so they are written to Loki.
This example collects log entries from the Event Log specified in `eventlog_name` and forwards them to a `loki.write` component.

```alloy
loki.source.windowsevent "application" {
Expand Down
Loading

0 comments on commit 6f98bc3

Please sign in to comment.