Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(stream): stateless 2-phase agg, state local hash agg #16179

Open
st1page opened this issue Apr 7, 2024 · 3 comments
Open

perf(stream): stateless 2-phase agg, state local hash agg #16179

st1page opened this issue Apr 7, 2024 · 3 comments
Assignees
Labels
help wanted Issues that need help from contributors no-issue-activity

Comments

@st1page
Copy link
Contributor

st1page commented Apr 7, 2024

We have had a stateless 2-phase simple agg to prevent single-point bottlenecks in simple aggregation with less costs.

// Stateless 2-phase simple agg
// can be applied on stateless simple agg calls,
// with input distributed by [`Distribution::AnyShard`]
if self.group_key().is_empty()
&& self
.core
.all_local_aggs_are_stateless(stream_input.append_only())
&& input_dist.satisfies(&RequiredDist::AnyShard)
{
return self.gen_stateless_two_phase_streaming_agg_plan(stream_input);
}

pub struct StatelessSimpleAggExecutor {

But we now found when a hotspot key exists. that is found in NEXMark q15. https://risingwave-labs.slack.com/archives/C0543LNH0KU/p1699324490070239?thread_ts=1699323766.819059&cid=C0543LNH0KU.
The stateless 2-phase optmization is also needed.

@github-actions github-actions bot added this to the release-1.8 milestone Apr 7, 2024
@st1page st1page self-assigned this Apr 7, 2024
@st1page st1page added the help wanted Issues that need help from contributors label Apr 7, 2024
@stdrc
Copy link
Member

stdrc commented Apr 8, 2024

Any more information?

@st1page
Copy link
Contributor Author

st1page commented Apr 8, 2024

Any more information?

Yes I will add it.

Copy link
Contributor

github-actions bot commented Aug 1, 2024

This issue has been open for 60 days with no activity.

If you think it is still relevant today, and needs to be done in the near future, you can comment to update the status, or just manually remove the no-issue-activity label.

You can also confidently close this issue as not planned to keep our backlog clean.
Don't worry if you think the issue is still valuable to continue in the future.
It's searchable and can be reopened when it's time. 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Issues that need help from contributors no-issue-activity
Projects
None yet
Development

No branches or pull requests

2 participants