Skip to content

Commit

Permalink
clarify
Browse files Browse the repository at this point in the history
  • Loading branch information
kwannoel committed May 30, 2024
1 parent 7739612 commit fd94d78
Showing 1 changed file with 8 additions and 4 deletions.
12 changes: 8 additions & 4 deletions docs/backfill.md
Original file line number Diff line number Diff line change
Expand Up @@ -361,14 +361,18 @@ and arrangement backfill will consume this historical data snapshot:
#### Initialization

Something to note is that for the first snapshot,
upstream may not have committed that epoch.
Additionally, we also have not replicated any upstream records
during that epoch.
upstream may not have finished committing data in that epoch to s3.

Additionally, we have not replicated any upstream records
during that epoch, only in the subsequent ones.

As such, we must wait for that first checkpoint to be committed,
before reading.
before reading, or we risk missing the uncommitted data in our backfill.

This is supported internally inside `init_epoch` for replicated state table.
```shell
upstream_table.init_epoch(first_epoch).await?;
```

### Recovery

Expand Down

0 comments on commit fd94d78

Please sign in to comment.