Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: spilled data may be skipped on storage get #14598

Closed
hzxa21 opened this issue Jan 16, 2024 · 0 comments · Fixed by #14607
Closed

bug: spilled data may be skipped on storage get #14598

hzxa21 opened this issue Jan 16, 2024 · 0 comments · Fixed by #14607
Assignees
Labels
type/bug Something isn't working
Milestone

Comments

@hzxa21
Copy link
Collaborator

hzxa21 commented Jan 16, 2024

Describe the bug

When introducing memtable spill, we have structure kv-entry epoch to be (epoch_from_barrier, spilled_offset). For each epoch_from_barrier, we increment the spilled_offset per memtable spill to ensure that data spilled within the same epoch_from_barrier has different spilled_offset

However, in storage get, the read epoch provided is purely epoch_from_barrier with spilled_offset == 0 and we use it to construct a full key, which later will be used in a SstableIterator::seek. This will caused data with spill_offset > 0 to be skipped.

Example:

Assume we have a SST:

key1 (epoch_from_barrier = 1, spilled_offset = 0)
key2 (epoch_from_barrier = 1, spilled_offset = 1)
key3 (epoch_from_barrier = 1, spilled_offset = 1)

get key2 with (epoch_from_barrier = 1) should see key2 in the SST but SstableIterator::seek will point to key3 because seek key is key2 (epoch_from_barrier = 1, spilled_offset = 0)

Error message/log

No response

To Reproduce

No response

Expected behavior

No response

How did you deploy RisingWave?

No response

The version of RisingWave

No response

Additional context

No response

@hzxa21 hzxa21 added the type/bug Something isn't working label Jan 16, 2024
@github-actions github-actions bot added this to the release-1.7 milestone Jan 16, 2024
@wcy-fdu wcy-fdu self-assigned this Jan 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants