Skip to content

Commit

Permalink
safekeeper: short-circuit interpreted wal sender (#10202)
Browse files Browse the repository at this point in the history
## Problem

Safekeeper may currently send a batch to the pageserver even if it
hasn't decoded a new record.
I think this is quite unlikely in the field, but worth adressing.

## Summary of changes

Don't send anything if we haven't decoded a full record. Once this
merges and releases, the `InterpretedWalRecords` struct can be updated
to remove the Option wrapper for `next_record_lsn`.
  • Loading branch information
VladLazar authored Dec 19, 2024
1 parent 502d512 commit 628451d
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion safekeeper/src/send_interpreted_wal.rs
Original file line number Diff line number Diff line change
Expand Up @@ -94,9 +94,14 @@ impl<IO: AsyncRead + AsyncWrite + Unpin> InterpretedWalSender<'_, IO> {
}
}

let max_next_record_lsn = match max_next_record_lsn {
Some(lsn) => lsn,
None => { continue; }
};

let batch = InterpretedWalRecords {
records,
next_record_lsn: max_next_record_lsn
next_record_lsn: Some(max_next_record_lsn),
};

tx.send(Batch {wal_end_lsn, available_wal_end_lsn, records: batch}).await.unwrap();
Expand Down

1 comment on commit 628451d

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

7245 tests run: 6937 passed, 1 failed, 307 skipped (full report)


Failures on Postgres 16

  • test_storage_controller_many_tenants[github-actions-selfhosted]: release-x86-64
# Run all failed tests locally:
scripts/pytest -vv -n $(nproc) -k "test_storage_controller_many_tenants[release-pg16-github-actions-selfhosted]"
Flaky tests (3)

Postgres 17

  • test_subscriber_synchronous_commit: release-x86-64
  • test_timeline_archival_chaos: release-arm64
  • test_delete_timeline_exercise_crash_safety_failpoints[Check.RETRY_WITHOUT_RESTART-timeline-delete-before-rm]: release-arm64

Code coverage* (full report)

  • functions: 31.2% (8396 of 26876 functions)
  • lines: 47.9% (66634 of 139015 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
628451d at 2024-12-19T16:26:03.731Z :recycle:

Please sign in to comment.