Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Workload Management] QueryGroupService that orchestrates Resource Tracking & Cancellation #1

Open
wants to merge 9 commits into
base: kp/wlm-cancellation-1
Choose a base branch
from

Conversation

kiranprakash154
Copy link
Owner

@kiranprakash154 kiranprakash154 commented Aug 28, 2024

Description

[Workload Management] QueryGroupService that orchestrates Resource Tracking & Cancellation

Related Issues

Resolves - opensearch-project#14883

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.
Signed-off-by: Kiran Prakash [email protected]Signed-off-by: Kiran Prakash [email protected]Signed-off-by: Kiran Prakash [email protected]

@kiranprakash154 kiranprakash154 changed the title Update CHANGELOG.md [DRAFT] QueryGroupService Aug 28, 2024
Comment on lines 48 to 50
private final ClusterService clusterService;
private final ClusterSettings clusterSettings;
private final Settings settings;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should only need settings defined for the feature in WorkloadManagementSettings class. Hence these member variables are unnecessary instead should only keep WorkloadManagementSettings instance.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm having ClusterSettings & Settings only for setting up the NodeDuressTrackers, aer you proposing to move the node duress tracker into WorkloadManagementSettings class ?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it makes sense to include NodeDuressTrackers setup in WorkloadManagementSettings instead of this class, what do you think ?

private final QueryGroupResourceUsageTrackerService queryGroupUsageTracker;
private final TimeValue queryGroupServiceRunIntervalInMillis;
private volatile Scheduler.Cancellable scheduledFuture;
private final ThreadPool threadPool;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also have setting which controls whether the feature is enabled or not

private final ClusterService clusterService;
private final ClusterSettings clusterSettings;
private final Settings settings;
private Set<QueryGroup> activeQueryGroups = new HashSet<>();

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should init the query groups from the state metadata because in come cases metadata might have some querygroups since metadata is durable.

Copy link
Owner Author

@kiranprakash154 kiranprakash154 Aug 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I'm doing this in

protected Set<QueryGroup> getActiveQueryGroups() {
        Map<String, QueryGroup> queryGroups = Metadata.builder(clusterService.state().metadata()).getQueryGroups();
        return new HashSet<>(queryGroups.values());
    }

which is invoked from doRun()

Signed-off-by: Kiran Prakash <[email protected]>
Signed-off-by: Kiran Prakash <[email protected]
Signed-off-by: Kiran Prakash <[email protected]>
Signed-off-by: Kiran Prakash <[email protected]>
@kiranprakash154 kiranprakash154 force-pushed the kp/queryGroup-Service-from-cancellation branch from d8c31ee to eb7ad77 Compare August 31, 2024 07:57
Signed-off-by: Kiran Prakash <[email protected]>
Signed-off-by: Kiran Prakash <[email protected]>
Signed-off-by: Kiran Prakash <[email protected]>
@kiranprakash154 kiranprakash154 marked this pull request as ready for review September 5, 2024 08:30
@kiranprakash154 kiranprakash154 changed the title [DRAFT] QueryGroupService [Workload Management] QueryGroupService that orchestrates Resource Tracking & Cancellation Sep 5, 2024
@kaushalmahi12
Copy link

kaushalmahi12 commented Sep 20, 2024

@kiranprakash154 Can we close this in favor of: opensearch-project#15925

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants