Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: should use executor_id to count mview creation progress #6236

Open
BugenZhao opened this issue Nov 7, 2022 · 4 comments
Open

bug: should use executor_id to count mview creation progress #6236

BugenZhao opened this issue Nov 7, 2022 · 4 comments
Labels
component/meta Meta related issue. component/streaming Stream processing related issue. no-issue-activity type/bug Something isn't working

Comments

@BugenZhao
Copy link
Member

BugenZhao commented Nov 7, 2022

a fragment contains more than one chain operator and we need to take care

Yes. I believe there will some problems with the current MV on MV creation reporting since we're using the actor id. 😢

Originally posted by @BugenZhao in #6233 (comment)

@github-actions github-actions bot added this to the release-0.1.14 milestone Nov 7, 2022
@BugenZhao BugenZhao added component/meta Meta related issue. component/streaming Stream processing related issue. type/bug Something isn't working labels Nov 7, 2022
@BugenZhao
Copy link
Member Author

#6270 will be a prerequisite.

@BugenZhao BugenZhao removed this from the release-0.1.14 milestone Nov 9, 2022
@github-actions
Copy link
Contributor

github-actions bot commented Jan 9, 2023

This issue has been open for 60 days with no activity. Could you please update the status? Feel free to continue discussion or close as not planned.

@BugenZhao
Copy link
Member Author

This is not a problem anymore, as there's no chance that two Backfill executors reside in the same actor. Otherwise,

  • If they are for the same table (like self-join), they should be shared.
  • If they are for different tables, it's impossible because different upstream tables have different distribution in the optimizer and won't be backfilled in the same fragment.

@xxchan
Copy link
Member

xxchan commented Sep 6, 2024

In theory, we can have multiple singleton executors in the same fragment (e.g., count(*) mv, or StreamValues). In our current code, it seems this won't happen, since JOIN/UNION will force a hash shard to inputs, and doesn't have the specialization for 2 singleton inputs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/meta Meta related issue. component/streaming Stream processing related issue. no-issue-activity type/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants