From b1f9c688f04ef473670014cad9bf448157c1f65e Mon Sep 17 00:00:00 2001
From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com>
Date: Tue, 29 Oct 2024 23:18:15 +0000
Subject: [PATCH] add microbatch updates (#6351)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
this pr adds updates to incremental microbatch per core prs:
- [#10878](https://github.com/dbt-labs/dbt-core/pull/10878) - makes it
so --event-time-start adn --event-time-end are mutually required.
- [#10876](https://github.com/dbt-labs/dbt-core/pull/10876) - changes
lookback default window to 1 (from 0)
[ X ] dbt Core PRs must get merged first before docs pr is merged
---
🚀 Deployment available! Here are the direct links to the updated files:
-
https://docs-getdbt-com-git-update-microbatch-dbt-labs.vercel.app/docs/build/incremental-microbatch
---
website/docs/docs/build/incremental-microbatch.md | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/website/docs/docs/build/incremental-microbatch.md b/website/docs/docs/build/incremental-microbatch.md
index 38e0edddfd6..6d80007e2d8 100644
--- a/website/docs/docs/build/incremental-microbatch.md
+++ b/website/docs/docs/build/incremental-microbatch.md
@@ -165,7 +165,7 @@ Several configurations are relevant to microbatch models, and some are required:
| `event_time` | Column (required) | The column indicating "at what time did the row occur." Required for your microbatch model and any direct parents that should be filtered. | N/A |
| `begin` | Date (required) | The "beginning of time" for the microbatch model. This is the starting point for any initial or full-refresh builds. For example, a daily-grain microbatch model run on `2024-10-01` with `begin = '2023-10-01` will process 366 batches (it's a leap year!) plus the batch for "today." | N/A |
| `batch_size` | String (required) | The granularity of your batches. Supported values are `hour`, `day`, `month`, and `year` | N/A |
-| `lookback` | Integer (optional) | Process X batches prior to the latest bookmark to capture late-arriving records. | `0` |
+| `lookback` | Integer (optional) | Process X batches prior to the latest bookmark to capture late-arriving records. | `1` |
@@ -192,12 +192,15 @@ During standard incremental runs, dbt will process batches according to the curr
Whether to fix erroneous source data or retroactively apply a change in business logic, you may need to reprocess a large amount of historical data.
-Backfilling a microbatch model is as simple as selecting it to run or build, and specifying a "start" and "end" for `event_time`. As always, dbt will process the batches between the start and end as independent queries.
+Backfilling a microbatch model is as simple as selecting it to run or build, and specifying a "start" and "end" for `event_time`. Note that `--event-time-start` and `--event-time-end` are mutually necessary, meaning that if you specify one, you must specify the other.
+
+As always, dbt will process the batches between the start and end as independent queries.
```bash
dbt run --event-time-start "2024-09-01" --event-time-end "2024-09-04"
```
+
### Retry