Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Optimize the performance of scaling in the SQL backend. #16722

Merged
merged 5 commits into from
Aug 19, 2024

Conversation

shanicky
Copy link
Contributor

@shanicky shanicky commented May 13, 2024

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

This PR attempts to use the native Fragment/Actor structure to replace the original list_all_table_fragments during the scaling process.

Checklist

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • I have added test labels as necessary. See details.
  • I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
  • My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
  • All checks passed in ./risedev check (or alias, ./risedev c)
  • My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)
  • My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

@shanicky shanicky force-pushed the peng/scale-sql-version branch from 60116d3 to 3d78569 Compare May 14, 2024 06:18
@shanicky shanicky force-pushed the peng/scale-sql-version branch from 3d78569 to bb0e444 Compare May 27, 2024 06:24
@shanicky shanicky force-pushed the peng/scale-sql-version branch 2 times, most recently from e52067b to 0dae38a Compare July 23, 2024 09:57
@shanicky shanicky force-pushed the peng/scale-sql-version branch from 0dae38a to a7d6b1d Compare July 30, 2024 10:38
@shanicky shanicky marked this pull request as ready for review July 31, 2024 07:44
@shanicky shanicky force-pushed the peng/scale-sql-version branch 3 times, most recently from 0acc646 to d3451bf Compare August 6, 2024 07:38
@fuyufjh fuyufjh self-requested a review August 7, 2024 02:43
@shanicky shanicky force-pushed the peng/scale-sql-version branch 2 times, most recently from c68a0cf to ec8a750 Compare August 8, 2024 05:58
@shanicky shanicky force-pushed the peng/scale-sql-version branch from 824889d to a443e1b Compare August 15, 2024 08:34
…dates

Refactor imports in scale.rs across controller & stream

Signed-off-by: Shanicky Chen <[email protected]>
…ed vars in `generate_table_resize_plan`.

Signed-off-by: Shanicky Chen <[email protected]>
…dd MetaMgrV2, async build_index_v2.

Signed-off-by: Shanicky Chen <[email protected]>
@shanicky shanicky force-pushed the peng/scale-sql-version branch from a443e1b to 4c2f413 Compare August 16, 2024 09:33
@shanicky shanicky requested a review from yezizp2012 August 16, 2024 10:29
@shanicky
Copy link
Contributor Author

It looks like it's ready for review. 🤔

Copy link
Member

@yezizp2012 yezizp2012 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉 🎉 LGTM, thx for the job! We can add or change the madsim integration tests after #15612 .

@shanicky shanicky added this pull request to the merge queue Aug 19, 2024
Merged via the queue into main with commit f54c7fe Aug 19, 2024
30 of 31 checks passed
@shanicky shanicky deleted the peng/scale-sql-version branch August 19, 2024 06:58
github-actions bot pushed a commit that referenced this pull request Aug 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants