Skip to content

Commit

Permalink
Linked back to query granularity docs (apache#16883)
Browse files Browse the repository at this point in the history
* Linked back to query granularity docs

* Update ingestion-spec.md

clairfy about query granularities in the spec.

* Update docs/design/storage.md

Co-authored-by: Charles Smith <[email protected]>

* Update docs/ingestion/ingestion-spec.md

Co-authored-by: Charles Smith <[email protected]>

* Update docs/querying/granularities.md

Co-authored-by: Charles Smith <[email protected]>

* Apply suggestions from code review

---------

Co-authored-by: Charles Smith <[email protected]>
  • Loading branch information
hevansDev and techdocsmith authored Aug 23, 2024
1 parent 0603d51 commit 60d4317
Show file tree
Hide file tree
Showing 3 changed files with 10 additions and 9 deletions.
2 changes: 1 addition & 1 deletion docs/design/storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ On the Coordinator / Historical side:
Segments all have a four-part identifier with the following components:

- Datasource name.
- Time interval (for the time chunk containing the segment; this corresponds to the `segmentGranularity` specified at ingestion time).
- Time interval for the time chunk containing the segment; this corresponds to the `segmentGranularity` specified at ingestion time. Uses the same format as [query granularity](../querying/granularities.md).
- Version number (generally an ISO8601 timestamp corresponding to when the segment set was first started).
- Partition number (an integer, unique within a datasource+interval+version; may not necessarily be contiguous).

Expand Down
14 changes: 7 additions & 7 deletions docs/ingestion/ingestion-spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -301,15 +301,15 @@ An example `metricsSpec` is:

### `granularitySpec`

The `granularitySpec` is located in `dataSchema``granularitySpec` and is responsible for configuring
the following operations:
The `granularitySpec`, located in `dataSchema``granularitySpec`, specifies the following:

1. Partitioning a datasource into [time chunks](../design/storage.md) (via `segmentGranularity`).
2. Truncating the timestamp, if desired (via `queryGranularity`).
3. Specifying which time chunks of segments should be created, for batch ingestion (via `intervals`).
4. Specifying whether ingestion-time [rollup](./rollup.md) should be used or not (via `rollup`).
1. `segmentGranularity` to partitioning a datasource into [time chunks](../design/storage.md).
2. `queryGranularity` to optionally truncate the timestamp.
3. `intervals` to define the time chunks of segments to create for batch ingestion.
4. `rollup` to enable ingestion-time [rollup](./rollup.md) or not.

Other than `rollup`, these operations are all based on the [primary timestamp](./schema-model.md#primary-timestamp).
Use the format from [Query granularities] to specify both `segmentGranualarity` and `queryGranularity`.

An example `granularitySpec` is:

Expand Down Expand Up @@ -561,4 +561,4 @@ For example:
}
}
}
```
```
3 changes: 2 additions & 1 deletion docs/querying/granularities.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,8 @@ sidebar_label: "Granularities"
[SQL documentation](sql-scalar.md#date-and-time-functions).
:::

Granularity determines how to bucket data across the time dimension, or how to aggregate data by hour, day, minute, etc.
Granularity determines how to bucket data across the time dimension, or how to aggregate data by hour, day, minute, etc and defines how it is stored.
The granularity formats here apply also to `segmentGranularity` and `queryGranularity` in the `granularitySpec` section of the the [ingestion spec](../ingestion/ingestion-spec.md#granularityspec).

For example, use time granularities in [native queries](querying.md) to bucket results by time, and in the `dataSchema` \\ [`granularitySpec`](../ingestion/ingestion-spec.md#granularityspec) section of ingestion specifications to segment incoming data.

Expand Down

0 comments on commit 60d4317

Please sign in to comment.