Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create scheduler state module #968

Conversation

adam-singer
Copy link
Contributor

@adam-singer adam-singer commented Jun 4, 2024

Description

Moving enough code out of simple_scheduler for decoupling stateful structures to support trait based interfaces in the future. There is no logical flow change in this refactor, all tests pass. Some structures will have pub(crate) visable modules and fields to help with the transition of code reordering.

Fixes # #982

Type of change

  • Refactor (non-breaking change which no functionality added)

How Has This Been Tested?

Checklist

  • Tests added/amended
  • bazel test //... passes locally
  • PR is contained in a single commit, using git amend see some docs

This change is Reviewable

@adam-singer adam-singer added the enhancement New feature or request label Jun 4, 2024
@adam-singer adam-singer requested review from allada and zbirenbaum June 4, 2024 21:54
@adam-singer adam-singer self-assigned this Jun 4, 2024
@adam-singer adam-singer force-pushed the adams/create-scheduler-state-manager branch 2 times, most recently from 7c21633 to 7736560 Compare June 4, 2024 22:59
Copy link
Member

@allada allada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love this PR!

:lgtm_strong:

Reviewed 8 of 8 files at r1, all commit messages.
Reviewable status: 0 of 1 LGTMs obtained, and 5 discussions need to be resolved (waiting on @adam-singer)


nativelink-scheduler/src/scheduler_state/awaited_action.rs line 26 at r1 (raw file):

/// An action that is being awaited on and last known state.
pub struct AwaitedAction {
    pub(crate) action_info: Arc<ActionInfo>,

nit: Lets document these.


nativelink-scheduler/src/scheduler_state/completed_action.rs line 24 at r1 (raw file):

pub struct CompletedAction {
    pub(crate) completed_time: SystemTime,

nit: ditto.


nativelink-scheduler/src/scheduler_state/manager.rs line 25 at r1 (raw file):

use crate::scheduler_state::workers::Workers;

pub(crate) struct Manager {

nit: Lets just name this StateManager.


nativelink-scheduler/src/scheduler_state/manager.rs line 36 at r1 (raw file):

    // Important: These two fields must be kept in-sync, so if you modify one, you likely need to
    // modify the other.
    pub(crate) queued_actions_set: HashSet<Arc<ActionInfo>>,

nit: ditto.


nativelink-scheduler/src/scheduler_state/workers.rs line 24 at r1 (raw file):

use crate::worker::{Worker, WorkerId, WorkerTimestamp};

pub struct Workers {

nit: ditto.

Copy link
Contributor

@zbirenbaum zbirenbaum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! Super clean - really like the crate level visibility, definitely something that I'll want to include on my side. I think the comment nits Blaise left are the only real things to address.

:lgtm_strong:

Reviewable status: 0 of 1 LGTMs obtained, and 5 discussions need to be resolved (waiting on @adam-singer)

Copy link
Contributor Author

@adam-singer adam-singer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 1 LGTMs obtained, and 1 discussions need to be resolved


nativelink-scheduler/src/scheduler_state/awaited_action.rs line 26 at r1 (raw file):

Previously, allada (Nathan (Blaise) Bruer) wrote…

nit: Lets document these.

Done.


nativelink-scheduler/src/scheduler_state/completed_action.rs line 24 at r1 (raw file):

Previously, allada (Nathan (Blaise) Bruer) wrote…

nit: ditto.

Done.


nativelink-scheduler/src/scheduler_state/manager.rs line 25 at r1 (raw file):

Previously, allada (Nathan (Blaise) Bruer) wrote…

nit: Lets just name this StateManager.

Done.


nativelink-scheduler/src/scheduler_state/manager.rs line 36 at r1 (raw file):

Previously, allada (Nathan (Blaise) Bruer) wrote…

nit: ditto.

Done.

@adam-singer adam-singer force-pushed the adams/create-scheduler-state-manager branch from 7736560 to c17bf9b Compare June 7, 2024 05:19
Moving enough code out of `simple_scheduler` for decoupling stateful
structures to support trait based interfaces in the future. There is
no logical flow change in this refactor, all tests pass. Some
structures will have `pub(crate)` visable modules and fields to help
with the transition of code reordering.
@adam-singer adam-singer force-pushed the adams/create-scheduler-state-manager branch from c17bf9b to 5334be4 Compare June 7, 2024 05:24
Copy link
Contributor Author

@adam-singer adam-singer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 1 LGTMs obtained, and pending CI: Vercel, docker-compose-compiles-nativelink (20.04), pre-commit-checks, ubuntu-20.04 / stable, vale (waiting on @adam-singer)


nativelink-scheduler/src/scheduler_state/workers.rs line 24 at r1 (raw file):

Previously, allada (Nathan (Blaise) Bruer) wrote…

nit: ditto.

Done.

Copy link
Contributor Author

@adam-singer adam-singer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: :shipit: complete! 1 of 1 LGTMs obtained

@adam-singer adam-singer merged commit 264edb7 into TraceMachina:main Jun 7, 2024
28 checks passed
@adam-singer adam-singer deleted the adams/create-scheduler-state-manager branch June 7, 2024 13:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants