Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Experimental: Add query type definition and schemas #897

Merged
merged 79 commits into from
Mar 6, 2024
Merged

Conversation

ryantxu
Copy link
Member

@ryantxu ryantxu commented Feb 13, 2024

This PR has two key parts:

  1. Defines usable go structs for the query request. Currently we only have internal objects or and crafted datasource specific ones.
  2. Add a tool that inspects a go struct and creates a spec from it.

In an effort to discover if the shapes are useful and easy enough to integrate, these changes are exercised in a few projects:

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file was generated by dropping the folder in grafana/grafana/pkg/apis and running codegen

Eventually this should not require any manual steps, but as an initial pass it seems OK

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add instructions about how to update this in the README?

@ryantxu ryantxu changed the title Experimental: Add query type handler and schemas Experimental: Add query type definition and schemas Mar 4, 2024
@ryantxu ryantxu marked this pull request as ready for review March 4, 2024 20:34
@ryantxu ryantxu requested a review from a team as a code owner March 4, 2024 20:34
@ryantxu ryantxu requested review from wbrowne, andresmgot and oshirohugo and removed request for a team March 4, 2024 20:34
Copy link
Contributor

@andresmgot andresmgot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see no harm (increased size) for existing plugins so we can continue with this

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add instructions about how to update this in the README?

@@ -0,0 +1,19 @@
package v0alpha1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this is going to hold only query schemas for the time being we can call the group query (apis/query/v0alpha1/). If the plan is to add more things here then data would also be fine.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, why not adding this to experimental first?

Copy link
Member Author

@ryantxu ryantxu Mar 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the plan is to add more things here then data would also be fine.

The next thing is settings (app + datasource... maybe the same object?) so want to avoid "query" -- i'll use data; it is a bit awkward since that is also the package for data.Frame, but eventually Frame should be in something clients would want access to

why not adding this to experimental first?

I have bounced between them a few times 🤣 -- the question is really how compatible we want to be with k8s. The directory structure /apis/{group}/{version} is standard in k8s and deviating from it makes life hard. In that world "v0*" is experimental, and also has a clear pattern for version evolution.

Working from the assumption that we will eventually land in k8s style apis seems like the right path here (though I do not feel too strongly about it).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the purpose of not confusing/communicating to plugin authors whether this is experimental or not it would be good to put this in the experimental package I think for now until things has settled/this is officially supported.

headers []string
}

func NewQueryDataClient(url string, client *http.Client, headers ...string) QueryDataClient {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure I understand the purpose of this new client? Is it an example?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is what alert ruler would call (currently we say "call /ds/query" but there is no typed client).

It is also the abstraction used inside the query service that will execute query requests. We have a few implementations:

We don't necessarily need the implementation here (could be internal) but then not sure how to best have alert ruler use the client.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it may make sense in its own package (and perhaps with a generic for the query type) but given this is all in experimental/v0alpha I don't think it matters too much yet

@ryantxu ryantxu requested a review from andresmgot March 6, 2024 14:47
@ryantxu
Copy link
Member Author

ryantxu commented Mar 6, 2024

@marefr -- i move the package, can you take another look?

Copy link
Contributor

@marefr marefr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ryantxu ryantxu merged commit a8003ef into main Mar 6, 2024
4 checks passed
@ryantxu ryantxu deleted the query-type-handler branch March 6, 2024 15:11

type QueryDataRequest struct {
// Time range applied to each query (when not included in the query body)
TimeRange `json:",inline"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be good to remove this if we can since I don't think it adds any additional functionality - but does add complexity to the Request and handling of the Request.

If there is no additional functionality, and it is just for convenience, then I think it would be better to have Code (e.g. Methods) to set a single on all the queries, rather then have the Type that supports both.

// TimeRange represents a time range for a query and is a property of DataQuery.
type TimeRange struct {
// From is the start time of the query.
From string `json:"from" jsonschema:"example=now-1h,default=now-6h"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • From and Two should be typed beyond being a string so we can have valid and valid values for what they are.
  • If relative times are supported, there should also be an (optional) Timestamp to indicate an absolute value for "now".
  • We should have methods or function that return time.Time for these, so that does not vary between receivers.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If not, then things like following are valid, and if they don't work it is a bug ;-)

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants