Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The subscription doesn't appear to be handling the vnode bitmap update correctly. #16174

Closed
shanicky opened this issue Apr 6, 2024 · 4 comments
Assignees
Labels
type/bug Something isn't working
Milestone

Comments

@shanicky
Copy link
Contributor

shanicky commented Apr 6, 2024

Describe the bug

commit d0f8e58

./risedev p

run sql

dev=> create table t(v int);
CREATE_TABLE
dev=> create subscription subscription1 from t with (retention = '1D');
CREATE_SUBSCRIPTION
dev=> alter table t set parallelism = 1;

It could lead to a panic.

2024-04-06T19:06:29.940053+08:00  INFO      rw-main risingwave_meta::barrier::state: update paused state current=Some(ConfigChange) new=None
thread 'rw-streaming' panicked at src/storage/src/hummock/store/local_hummock_storage.rs:451:9:
There is uncommitted staging data in read version table_id TableId { table_id: 3 } instance_id 2 on vnode bitmap update

Error message/log

No response

To Reproduce

No response

Expected behavior

No response

How did you deploy RisingWave?

No response

The version of RisingWave

No response

Additional context

No response

@shanicky shanicky added the type/bug Something isn't working label Apr 6, 2024
@github-actions github-actions bot added this to the release-1.8 milestone Apr 6, 2024
@hzxa21
Copy link
Collaborator

hzxa21 commented Apr 7, 2024

It is because subscription will write to state store on seeing a barrier even after pause:

for vnode in self.serde.vnodes().iter_vnodes() {

Discussed offline with @xxhZs. It will be fixed in #15180

@shanicky
Copy link
Contributor Author

shanicky commented Apr 7, 2024

It is because subscription will write to state store on seeing a barrier even after pause:

for vnode in self.serde.vnodes().iter_vnodes() {

Discussed offline with @xxhZs. It will be fixed in #15180

Can we make it in time for the 1.8.1 release?

@hzxa21
Copy link
Collaborator

hzxa21 commented Apr 7, 2024

We can hotfix it in main/release-1.8 but subscription is an unreleased feature so no user should be using it. So I think it is okay to not block 1.8.1. Is the assertion caught by CI tests?

@shanicky shanicky modified the milestones: release-1.8, release-1.10 May 8, 2024
@xxhZs
Copy link
Contributor

xxhZs commented May 11, 2024

test it , fixed

@xxhZs xxhZs closed this as completed May 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants