-
Notifications
You must be signed in to change notification settings - Fork 66
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Experimental: Add query type definition and schemas #897
Merged
Merged
Changes from 71 commits
Commits
Show all changes
79 commits
Select commit
Hold shift + click to select a range
8781f2f
add query type support
ryantxu 0ec8629
move values
ryantxu 556b816
no refs
ryantxu fa45a90
add k8s placeholder
ryantxu 645173e
parallel k8s
ryantxu ff55c53
Merge remote-tracking branch 'origin/main' into query-type-handler
ryantxu f6bb323
now with examples
ryantxu e2341d6
ok with dataframe now
ryantxu 992463d
ok with dataframe now
ryantxu 58b32ee
no versions
ryantxu d791fc9
add apiVersion
ryantxu 0d526d8
Merge remote-tracking branch 'origin/main' into query-type-handler
ryantxu 4ae1f0c
add settings example
ryantxu b9382d8
Merge remote-tracking branch 'origin/main' into query-type-handler
ryantxu 238f7c9
fix build
ryantxu 7ed252c
Merge remote-tracking branch 'origin/main' into query-type-handler
ryantxu 1fe487a
multiple discriminators
ryantxu f5a35b3
difference between save and post
ryantxu dc41e8c
rename to spec
ryantxu 1202776
rename to spec
ryantxu 10b145d
more reference stuff
ryantxu e854402
more reference stuff
ryantxu 074b8f8
more reference stuff
ryantxu c673243
with definitions
ryantxu 940bc8e
with definitions
ryantxu 5c2de27
cleanup
ryantxu 64c06c2
cleanup
ryantxu 36b841c
use kube-openapi
ryantxu 2c8c4ad
fix error state
ryantxu 2b78392
refactor (again)
ryantxu e717941
refactor (again)
ryantxu d9f700e
now with panel schema
ryantxu ee12be2
now with panel schema
ryantxu 77f2055
now with dataFrame type
ryantxu f8eb48f
now with dataFrame type
ryantxu 8f647ba
lint
ryantxu 9e14771
refactor
ryantxu 29beda6
refactor
ryantxu a09ab12
another constructor
ryantxu 81d51b3
merge main
ryantxu 2151c09
update query parser
ryantxu 5af1ec4
update query parser
ryantxu 1e64757
merge main
ryantxu f18d1cb
use real type for JSONSchema object
ryantxu ff38e88
use real type for JSONSchema object
ryantxu 34c635a
use real type for JSONSchema object
ryantxu 0847e1d
add spec definition
ryantxu 8482109
more specs
ryantxu 8dbdba0
more specs
ryantxu bb0cd98
lint
ryantxu 2b3cc45
now with unstructured
ryantxu 0b0e1ea
less verbose
ryantxu a804614
now with unstructured
ryantxu 424be9a
now with unstructured
ryantxu 3a97c12
more copy methods
ryantxu f8336e6
fix lint remove parser
ryantxu 2a7ba03
fix lint
ryantxu 2ec1471
panel cleanup
ryantxu 0237a40
panel cleanup
ryantxu d35e870
inline timerange
ryantxu a7986a7
more deepcopy
ryantxu 71374e7
hopefully last big package rename
ryantxu d2c2e90
loading frame types
ryantxu 0e568d6
loading frame types
ryantxu e7e1b68
fix test
ryantxu 81d7990
remove unused settings configs
ryantxu 0fe8c9c
move panel to schema builder
ryantxu bd58a3b
Merge remote-tracking branch 'origin/main' into query-type-handler
ryantxu e9b643e
use k8s style paths
ryantxu c52d81c
use codegen
ryantxu 91df7c5
use codegen
ryantxu 66438c4
lint
ryantxu 557e208
rename to data
ryantxu 283c5df
update readme
ryantxu 782e1f5
update headers
ryantxu 342bf4e
update headers
ryantxu 0fd38ec
Merge remote-tracking branch 'origin/main' into query-type-handler
ryantxu 86b43fa
move package
ryantxu 95b4050
move package
ryantxu File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
This package helps exposes objects from the plugin SDK into k8s based apis. | ||
|
||
This should most likely include the k8s.io/apimachinery dependency -- however this initial work | ||
will just mock that values and see how well that works. Avoiding the need for https://github.com/grafana/grafana-plugin-sdk-go/pull/909 | ||
|
||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
package v0alpha1 | ||
|
||
// ObjectMeta is a struct that aims to "look" like a real kubernetes object when | ||
// written to JSON, however it does not require the pile of dependencies | ||
// This is really an internal helper until we decide which dependencies make sense | ||
// to require within the SDK | ||
type ObjectMeta struct { | ||
// The name is for k8s and description, but not used in the schema | ||
Name string `json:"name,omitempty"` | ||
// Changes indicate that *something * changed | ||
ResourceVersion string `json:"resourceVersion,omitempty"` | ||
// Timestamp | ||
CreationTimestamp string `json:"creationTimestamp,omitempty"` | ||
} | ||
|
||
type TypeMeta struct { | ||
Kind string `json:"kind"` // "QueryTypeDefinitionList", | ||
APIVersion string `json:"apiVersion"` // "query.grafana.app/v0alpha1", | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,82 @@ | ||
package v0alpha1 | ||
|
||
import ( | ||
"embed" | ||
|
||
"k8s.io/kube-openapi/pkg/common" | ||
spec "k8s.io/kube-openapi/pkg/validation/spec" | ||
) | ||
|
||
//go:embed query.schema.json query.definition.schema.json | ||
var f embed.FS | ||
|
||
func GetOpenAPIDefinitions(ref common.ReferenceCallback) map[string]common.OpenAPIDefinition { | ||
return map[string]common.OpenAPIDefinition{ | ||
"github.com/grafana/grafana-plugin-sdk-go/backend.DataResponse": schemaDataResponse(ref), | ||
"github.com/grafana/grafana-plugin-sdk-go/data.Frame": schemaDataFrame(ref), | ||
"github.com/grafana/grafana-plugin-sdk-go/apis/sdkapi/v0alpha1.DataQuery": schemaDataQuery(ref), | ||
"github.com/grafana/grafana-plugin-sdk-go/apis/sdkapi/v0alpha1.QueryTypeDefinitionSpec": schemaQueryTypeDefinitionSpec(ref), | ||
} | ||
} | ||
|
||
// Individual response | ||
func schemaDataResponse(_ common.ReferenceCallback) common.OpenAPIDefinition { | ||
return common.OpenAPIDefinition{ | ||
Schema: spec.Schema{ | ||
SchemaProps: spec.SchemaProps{ | ||
Description: "todo... improve schema", | ||
Type: []string{"object"}, | ||
AdditionalProperties: &spec.SchemaOrBool{Allows: true}, | ||
}, | ||
}, | ||
} | ||
} | ||
|
||
func schemaDataFrame(_ common.ReferenceCallback) common.OpenAPIDefinition { | ||
return common.OpenAPIDefinition{ | ||
Schema: spec.Schema{ | ||
SchemaProps: spec.SchemaProps{ | ||
Description: "any object for now", | ||
Type: []string{"object"}, | ||
Properties: map[string]spec.Schema{}, | ||
AdditionalProperties: &spec.SchemaOrBool{Allows: true}, | ||
}, | ||
}, | ||
} | ||
} | ||
|
||
func schemaQueryTypeDefinitionSpec(_ common.ReferenceCallback) common.OpenAPIDefinition { | ||
s, _ := loadSchema("query.definition.schema.json") | ||
if s == nil { | ||
s = &spec.Schema{} | ||
} | ||
return common.OpenAPIDefinition{ | ||
Schema: *s, | ||
} | ||
} | ||
|
||
func schemaDataQuery(_ common.ReferenceCallback) common.OpenAPIDefinition { | ||
s, _ := DataQuerySchema() | ||
if s == nil { | ||
s = &spec.Schema{} | ||
} | ||
s.SchemaProps.Type = []string{"object"} | ||
s.SchemaProps.AdditionalProperties = &spec.SchemaOrBool{Allows: true} | ||
return common.OpenAPIDefinition{Schema: *s} | ||
} | ||
|
||
// Get the cached feature list (exposed as a k8s resource) | ||
func DataQuerySchema() (*spec.Schema, error) { | ||
return loadSchema("query.schema.json") | ||
} | ||
|
||
// Get the cached feature list (exposed as a k8s resource) | ||
func loadSchema(path string) (*spec.Schema, error) { | ||
body, err := f.ReadFile(path) | ||
if err != nil { | ||
return nil, err | ||
} | ||
s := &spec.Schema{} | ||
err = s.UnmarshalJSON(body) | ||
return s, err | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
package v0alpha1 | ||
|
||
import ( | ||
"encoding/json" | ||
"os" | ||
"testing" | ||
|
||
"github.com/stretchr/testify/assert" | ||
"github.com/stretchr/testify/require" | ||
"k8s.io/kube-openapi/pkg/validation/spec" | ||
"k8s.io/kube-openapi/pkg/validation/strfmt" | ||
"k8s.io/kube-openapi/pkg/validation/validate" | ||
) | ||
|
||
func TestOpenAPI(t *testing.T) { | ||
//nolint:gocritic | ||
defs := GetOpenAPIDefinitions(func(path string) spec.Ref { // (unlambda: replace ¯\_(ツ)_/¯) | ||
return spec.MustCreateRef(path) // placeholder for tests | ||
}) | ||
|
||
def, ok := defs["github.com/grafana/grafana-plugin-sdk-go/backend.DataResponse"] | ||
require.True(t, ok) | ||
require.Empty(t, def.Dependencies) // not yet supported! | ||
|
||
validator := validate.NewSchemaValidator(&def.Schema, nil, "data", strfmt.Default) | ||
|
||
body, err := os.ReadFile("./testdata/sample_query_results.json") | ||
require.NoError(t, err) | ||
unstructured := make(map[string]any) | ||
err = json.Unmarshal(body, &unstructured) | ||
require.NoError(t, err) | ||
|
||
result := validator.Validate(unstructured) | ||
for _, err := range result.Errors { | ||
assert.NoError(t, err, "validation error") | ||
} | ||
for _, err := range result.Warnings { | ||
assert.NoError(t, err, "validation warning") | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,72 @@ | ||
{ | ||
"$schema": "https://json-schema.org/draft-04/schema#", | ||
"properties": { | ||
"discriminators": { | ||
"items": { | ||
"properties": { | ||
"field": { | ||
"type": "string", | ||
"description": "DiscriminatorField is the field used to link behavior to this specific\nquery type. It is typically \"queryType\", but can be another field if necessary" | ||
}, | ||
"value": { | ||
"type": "string", | ||
"description": "The discriminator value" | ||
} | ||
}, | ||
"additionalProperties": false, | ||
"type": "object", | ||
"required": [ | ||
"field", | ||
"value" | ||
] | ||
}, | ||
"type": "array", | ||
"description": "Multiple schemas can be defined using discriminators" | ||
}, | ||
"description": { | ||
"type": "string", | ||
"description": "Describe whe the query type is for" | ||
}, | ||
"schema": { | ||
"$ref": "https://json-schema.org/draft-04/schema#", | ||
"type": "object", | ||
"description": "The query schema represents the properties that can be sent to the API\nIn many cases, this may be the same properties that are saved in a dashboard\nIn the case where the save model is different, we must also specify a save model" | ||
}, | ||
"examples": { | ||
"items": { | ||
"properties": { | ||
"name": { | ||
"type": "string", | ||
"description": "Version identifier or empty if only one exists" | ||
}, | ||
"description": { | ||
"type": "string", | ||
"description": "Optionally explain why the example is interesting" | ||
}, | ||
"saveModel": { | ||
"additionalProperties": true, | ||
"type": "object", | ||
"description": "An example value saved that can be saved in a dashboard" | ||
} | ||
}, | ||
"additionalProperties": false, | ||
"type": "object" | ||
}, | ||
"type": "array", | ||
"description": "Examples (include a wrapper) ideally a template!" | ||
}, | ||
"changelog": { | ||
"items": { | ||
"type": "string" | ||
}, | ||
"type": "array", | ||
"description": "Changelog defines the changed from the previous version\nAll changes in the same version *must* be backwards compatible\nOnly notable changes will be shown here, for the full version history see git!" | ||
} | ||
}, | ||
"additionalProperties": false, | ||
"type": "object", | ||
"required": [ | ||
"schema", | ||
"examples" | ||
] | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if this is going to hold only query schemas for the time being we can call the group
query
(apis/query/v0alpha1/
). If the plan is to add more things here thendata
would also be fine.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, why not adding this to
experimental
first?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The next thing is settings (app + datasource... maybe the same object?) so want to avoid "query" -- i'll use data; it is a bit awkward since that is also the package for data.Frame, but eventually Frame should be in something clients would want access to
I have bounced between them a few times 🤣 -- the question is really how compatible we want to be with k8s. The directory structure /apis/{group}/{version} is standard in k8s and deviating from it makes life hard. In that world "v0*" is experimental, and also has a clear pattern for version evolution.
Working from the assumption that we will eventually land in k8s style apis seems like the right path here (though I do not feel too strongly about it).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For the purpose of not confusing/communicating to plugin authors whether this is experimental or not it would be good to put this in the experimental package I think for now until things has settled/this is officially supported.