Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(network): make streamhashmap not end #2249

Closed
wants to merge 1 commit into from

Conversation

eitanm-starkware
Copy link
Contributor

@eitanm-starkware eitanm-starkware commented Jul 24, 2024

This change is Reviewable

Copy link

codecov bot commented Jul 24, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 66.33%. Comparing base (3bcfcc6) to head (5f09916).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2249      +/-   ##
==========================================
- Coverage   66.34%   66.33%   -0.01%     
==========================================
  Files         139      139              
  Lines       18379    18374       -5     
  Branches    18379    18374       -5     
==========================================
- Hits        12194    12189       -5     
  Misses       4890     4890              
  Partials     1295     1295              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@ShahakShama ShahakShama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r1, all commit messages.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @eitanm-starkware)


-- commits line 2 at r1:
This is fix, not chore


crates/papyrus_network/src/utils.rs line 39 at r1 (raw file):

    pub fn insert(&mut self, key: K, value: V) -> Option<V> {
        self.finished_streams.remove(&key);

I think you can remove finished_streams entirely and just remove from the map all finished streams. Use HashMap::retain for that


crates/papyrus_network/src/utils.rs line 58 at r1 (raw file):

                }
                Poll::Pending => {}
            }

As a general rule of thumb, if there is a flow in which you return pending and don't call poll on some other future, you should be worried that your future can reach a status where it's never gets awake

In this case, it can happen if self.map is empty.

Add to the struct a field of type Vec called wakers_waiting_for_new_stream. Check at the start of the function if the map is empty. If yes, add cx.waker().clone() to this vector. Inside the insert function, call wake() on all the wakers in this vector and clear the vector (you can use drain(..) for that)

There are many examples for this in our code. Just look for the word waker

Copy link
Contributor Author

@eitanm-starkware eitanm-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @ShahakShama)


crates/papyrus_network/src/utils.rs line 39 at r1 (raw file):

Previously, ShahakShama wrote…

I think you can remove finished_streams entirely and just remove from the map all finished streams. Use HashMap::retain for that

Done.


crates/papyrus_network/src/utils.rs line 58 at r1 (raw file):

Previously, ShahakShama wrote…

As a general rule of thumb, if there is a flow in which you return pending and don't call poll on some other future, you should be worried that your future can reach a status where it's never gets awake

In this case, it can happen if self.map is empty.

Add to the struct a field of type Vec called wakers_waiting_for_new_stream. Check at the start of the function if the map is empty. If yes, add cx.waker().clone() to this vector. Inside the insert function, call wake() on all the wakers in this vector and clear the vector (you can use drain(..) for that)

There are many examples for this in our code. Just look for the word waker

Done. had to use a set to track finished streams though

@github-actions github-actions bot locked and limited conversation to collaborators Jul 30, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants