Skip to content

Commit

Permalink
Merge branch 'current' into patch-10
Browse files Browse the repository at this point in the history
  • Loading branch information
@matthewshaver
matthewshaver authored Oct 31, 2024
2 parents abdda24 + 7004dc5 commit 7befd76
Show file tree
Hide file tree
Showing 37 changed files with 321 additions and 670 deletions.
2 changes: 1 addition & 1 deletion contributing/single-sourcing-content.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ This component can be added directly to a markdown file in a similar way as othe
Both properties can be used together to set a range where the content should show. In the example below, this content will only show if the selected version is between **0.21** and **1.0**:

```markdown
<VersionBlock lastVersion="1.6">
<VersionBlock lastVersion="1.9">

Versioned content here

Expand Down
134 changes: 1 addition & 133 deletions website/dbt-versions.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,7 @@ exports.versions = [
},
{
version: "1.7",
EOLDate: "2024-10-30",
},
{
version: "1.6",
EOLDate: "2024-07-31",
EOLDate: "2024-11-01",
},
];

Expand Down Expand Up @@ -66,134 +62,6 @@ exports.versionedPages = [
page: "reference/global-configs/indirect-selection",
firstVersion: "1.8",
},
{
page: "reference/resource-configs/store_failures_as",
firstVersion: "1.7",
},
{
page: "docs/build/build-metrics-intro",
firstVersion: "1.6",
},
{
page: "docs/build/sl-getting-started",
firstVersion: "1.6",
},
{
page: "docs/build/about-metricflow",
firstVersion: "1.6",
},
{
page: "docs/build/join-logic",
firstVersion: "1.6",
},
{
page: "docs/build/validation",
firstVersion: "1.6",
},
{
page: "docs/build/semantic-models",
firstVersion: "1.6",
},
{
page: "docs/build/group-by",
firstVersion: "1.6",
},
{
page: "docs/build/entities",
firstVersion: "1.6",
},
{
page: "docs/build/metrics-overview",
firstVersion: "1.6",
},
{
page: "docs/build/cumulative",
firstVersion: "1.6",
},
{
page: "docs/build/derived",
firstVersion: "1.6",
},
{
page: "docs/build/measure-proxy",
firstVersion: "1.6",
},
{
page: "docs/build/ratio",
firstVersion: "1.6",
},
{
page: "reference/commands/clone",
firstVersion: "1.6",
},
{
page: "docs/collaborate/govern/project-dependencies",
firstVersion: "1.6",
},
{
page: "reference/dbt-jinja-functions/thread_id",
firstVersion: "1.6",
},
{
page: "reference/resource-properties/deprecation_date",
firstVersion: "1.6",
},
{
page: "reference/commands/retry",
firstVersion: "1.6",
},
{
page: "docs/build/groups",
firstVersion: "1.5",
},
{
page: "docs/collaborate/govern/model-contracts",
firstVersion: "1.5",
},
{
page: "reference/commands/show",
firstVersion: "1.5",
},
{
page: "docs/collaborate/govern/model-access",
firstVersion: "1.5",
},
{
page: "docs/collaborate/govern/model-versions",
firstVersion: "1.5",
},
{
page: "reference/programmatic-invocations",
firstVersion: "1.5",
},
{
page: "reference/resource-configs/contract",
firstVersion: "1.5",
},
{
page: "reference/resource-configs/group",
firstVersion: "1.5",
},
{
page: "reference/resource-properties/access",
firstVersion: "1.5",
},
{
page: "reference/resource-properties/constraints",
firstVersion: "1.5",
},
{
page: "reference/resource-properties/latest_version",
firstVersion: "1.5",
},
{
page: "reference/resource-properties/versions",
firstVersion: "1.5",
},
{
page: "reference/resource-configs/on_configuration_change",
firstVersion: "1.6",
},
];

/**
Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/build/environment-variables.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ dbt Cloud has a number of pre-defined variables built in. Variables are set auto
The following environment variable is set automatically for the dbt Cloud IDE:

- `DBT_CLOUD_GIT_BRANCH` &mdash; Provides the development Git branch name in the [dbt Cloud IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud).
- Available in dbt v 1.6 and later.
- Available in dbt v1.6 and later.
- The variable changes when the branch is changed.
- Doesn't require restarting the IDE after a branch change.
- Currently not available in the [dbt Cloud CLI](/docs/cloud/cloud-cli-installation).
Expand Down
7 changes: 5 additions & 2 deletions website/docs/docs/build/incremental-microbatch.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ Several configurations are relevant to microbatch models, and some are required:
| `event_time` | Column (required) | The column indicating "at what time did the row occur." Required for your microbatch model and any direct parents that should be filtered. | N/A |
| `begin` | Date (required) | The "beginning of time" for the microbatch model. This is the starting point for any initial or full-refresh builds. For example, a daily-grain microbatch model run on `2024-10-01` with `begin = '2023-10-01` will process 366 batches (it's a leap year!) plus the batch for "today." | N/A |
| `batch_size` | String (required) | The granularity of your batches. Supported values are `hour`, `day`, `month`, and `year` | N/A |
| `lookback` | Integer (optional) | Process X batches prior to the latest bookmark to capture late-arriving records. | `0` |
| `lookback` | Integer (optional) | Process X batches prior to the latest bookmark to capture late-arriving records. | `1` |

<Lightbox src="/img/docs/building-a-dbt-project/microbatch/event_time.png" title="The event_time column configures the real-world time of this record"/>

Expand All @@ -192,12 +192,15 @@ During standard incremental runs, dbt will process batches according to the curr

Whether to fix erroneous source data or retroactively apply a change in business logic, you may need to reprocess a large amount of historical data.

Backfilling a microbatch model is as simple as selecting it to run or build, and specifying a "start" and "end" for `event_time`. As always, dbt will process the batches between the start and end as independent queries.
Backfilling a microbatch model is as simple as selecting it to run or build, and specifying a "start" and "end" for `event_time`. Note that `--event-time-start` and `--event-time-end` are mutually necessary, meaning that if you specify one, you must specify the other.

As always, dbt will process the batches between the start and end as independent queries.

```bash
dbt run --event-time-start "2024-09-01" --event-time-end "2024-09-04"
```


<Lightbox src="/img/docs/building-a-dbt-project/microbatch/microbatch_backfill.png" title="Configure a lookback to reprocess additional batches during standard incremental runs"/>

### Retry
Expand Down
139 changes: 0 additions & 139 deletions website/docs/docs/build/metricflow-time-spine.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -124,42 +124,6 @@ For an example project, refer to our [Jaffle shop](https://github.com/dbt-labs/j

<File name="metricflow_time_spine.sql">

<VersionBlock lastVersion="1.6">

```sql
{{
config(
materialized = 'table',
)
}}
with days as (
{{
dbt_utils.date_spine(
'day',
"to_date('01/01/2000','mm/dd/yyyy')",
"to_date('01/01/2025','mm/dd/yyyy')"
)
}}
),
final as (
select cast(date_day as date) as date_day
from days
)
select * from final
-- filter the time spine to a specific range
where date_day > dateadd(year, -4, current_timestamp())
and date_hour < dateadd(day, 30, current_timestamp())
```

</VersionBlock>

<VersionBlock firstVersion="1.7">

```sql
{{
config(
Expand Down Expand Up @@ -189,42 +153,9 @@ where date_day > dateadd(year, -4, current_timestamp())
and date_hour < dateadd(day, 30, current_timestamp())
```

</VersionBlock>

### Daily (BigQuery)

Use this model if you're using BigQuery. BigQuery supports `DATE()` instead of `TO_DATE()`:
<VersionBlock lastVersion="1.6">

<File name="metricflow_time_spine.sql">

```sql
{{config(materialized='table')}}
with days as (
{{dbt_utils.date_spine(
'day',
"DATE(2000,01,01)",
"DATE(2025,01,01)"
)
}}
),
final as (
select cast(date_day as date) as date_day
from days
)
select *
from final
-- filter the time spine to a specific range
where date_day > dateadd(year, -4, current_timestamp())
and date_hour < dateadd(day, 30, current_timestamp())
```

</File>
</VersionBlock>

<VersionBlock firstVersion="1.7">

<File name="metricflow_time_spine.sql">

Expand Down Expand Up @@ -253,7 +184,6 @@ and date_hour < dateadd(day, 30, current_timestamp())
```

</File>
</VersionBlock>

</File>

Expand Down Expand Up @@ -306,42 +236,6 @@ To create this table, you need to create a model in your dbt project called `met

### Daily

<VersionBlock lastVersion="1.6">
<File name='metricflow_time_spine.sql'>

```sql
{{
config(
materialized = 'table',
)
}}
with days as (
{{
dbt_utils.date_spine(
'day',
"to_date('01/01/2000','mm/dd/yyyy')",
"to_date('01/01/2025','mm/dd/yyyy')"
)
}}
),
final as (
select cast(date_day as date) as date_day
from days
)
select * from final
-- filter the time spine to a specific range
where date_day > dateadd(year, -4, current_timestamp())
and date_hour < dateadd(day, 30, current_timestamp())
```
</File>
</VersionBlock>

<VersionBlock firstVersion="1.7">
<File name='metricflow_time_spine.sql'>


Expand Down Expand Up @@ -375,43 +269,11 @@ and date_hour < dateadd(day, 30, current_timestamp())
```

</File>
</VersionBlock>

### Daily (BigQuery)

Use this model if you're using BigQuery. BigQuery supports `DATE()` instead of `TO_DATE()`:

<VersionBlock lastVersion="1.6">

<File name="metricflow_time_spine.sql">

```sql
{{config(materialized='table')}}
with days as (
{{dbt_utils.date_spine(
'day',
"DATE(2000,01,01)",
"DATE(2025,01,01)"
)
}}
),
final as (
select cast(date_day as date) as date_day
from days
)
select *
from final
-- filter the time spine to a specific range
where date_day > dateadd(year, -4, current_timestamp())
and date_hour < dateadd(day, 30, current_timestamp())
```
</File>
</VersionBlock>

<VersionBlock firstVersion="1.7">

<File name="metricflow_time_spine.sql">

```sql
Expand All @@ -438,7 +300,6 @@ and date_hour < dateadd(day, 30, current_timestamp())
```

</File>
</VersionBlock>

You only need to include the `date_day` column in the table. MetricFlow can handle broader levels of detail, but finer grains are only supported in versions 1.9 and higher.

Expand Down
Loading

0 comments on commit 7befd76

Please sign in to comment.