Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactoring: Move topology and arrow bookkeeping into JScheduler #259

Merged
merged 5 commits into from
Nov 1, 2023

Conversation

nathanwbrei
Copy link
Collaborator

Previously, our design attempted to separate the logic for shutting down individual arrows (and ultimately the whole topology) from the scheduling logic. These two things are not in fact independent, and this led to many headaches and issues such as #227. This refactoring moves all shutdown logic and bookkeeping state into the scheduler so that everyone sees a consistent view of the entire topology status. This gets rid of a number of atomic variables and puts everything behind one mutex. This improves the legibility of the design because the JScheduler class now encapsulates all state that needs the mutex's protection. This is not expected to hurt performance because the topology shutdown only happens at the very end of processing, and is ~O(number of arrows), which in typical use cases ranges from 2 to 6.

@nathanwbrei nathanwbrei merged commit 2db622a into master Nov 1, 2023
3 checks passed
@nathanwbrei nathanwbrei deleted the nbrei_scheduler_refactoring branch November 1, 2023 02:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant