Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Experimental: Add query type definition and schemas #897

Merged
merged 79 commits into from
Mar 6, 2024
Merged
Show file tree
Hide file tree
Changes from 71 commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
8781f2f
add query type support
ryantxu Feb 12, 2024
0ec8629
move values
ryantxu Feb 13, 2024
556b816
no refs
ryantxu Feb 13, 2024
fa45a90
add k8s placeholder
ryantxu Feb 13, 2024
645173e
parallel k8s
ryantxu Feb 13, 2024
ff55c53
Merge remote-tracking branch 'origin/main' into query-type-handler
ryantxu Feb 13, 2024
f6bb323
now with examples
ryantxu Feb 13, 2024
e2341d6
ok with dataframe now
ryantxu Feb 13, 2024
992463d
ok with dataframe now
ryantxu Feb 13, 2024
58b32ee
no versions
ryantxu Feb 14, 2024
d791fc9
add apiVersion
ryantxu Feb 14, 2024
0d526d8
Merge remote-tracking branch 'origin/main' into query-type-handler
ryantxu Feb 16, 2024
4ae1f0c
add settings example
ryantxu Feb 16, 2024
b9382d8
Merge remote-tracking branch 'origin/main' into query-type-handler
ryantxu Feb 17, 2024
238f7c9
fix build
ryantxu Feb 17, 2024
7ed252c
Merge remote-tracking branch 'origin/main' into query-type-handler
ryantxu Feb 20, 2024
1fe487a
multiple discriminators
ryantxu Feb 20, 2024
f5a35b3
difference between save and post
ryantxu Feb 21, 2024
dc41e8c
rename to spec
ryantxu Feb 21, 2024
1202776
rename to spec
ryantxu Feb 21, 2024
10b145d
more reference stuff
ryantxu Feb 24, 2024
e854402
more reference stuff
ryantxu Feb 24, 2024
074b8f8
more reference stuff
ryantxu Feb 25, 2024
c673243
with definitions
ryantxu Feb 25, 2024
940bc8e
with definitions
ryantxu Feb 25, 2024
5c2de27
cleanup
ryantxu Feb 25, 2024
64c06c2
cleanup
ryantxu Feb 25, 2024
36b841c
use kube-openapi
ryantxu Feb 25, 2024
2c8c4ad
fix error state
ryantxu Feb 25, 2024
2b78392
refactor (again)
ryantxu Feb 25, 2024
e717941
refactor (again)
ryantxu Feb 25, 2024
d9f700e
now with panel schema
ryantxu Feb 25, 2024
ee12be2
now with panel schema
ryantxu Feb 25, 2024
77f2055
now with dataFrame type
ryantxu Feb 26, 2024
f8eb48f
now with dataFrame type
ryantxu Feb 26, 2024
8f647ba
lint
ryantxu Feb 26, 2024
9e14771
refactor
ryantxu Feb 26, 2024
29beda6
refactor
ryantxu Feb 26, 2024
a09ab12
another constructor
ryantxu Feb 26, 2024
81d51b3
merge main
ryantxu Feb 28, 2024
2151c09
update query parser
ryantxu Feb 29, 2024
5af1ec4
update query parser
ryantxu Feb 29, 2024
1e64757
merge main
ryantxu Feb 29, 2024
f18d1cb
use real type for JSONSchema object
ryantxu Feb 29, 2024
ff38e88
use real type for JSONSchema object
ryantxu Feb 29, 2024
34c635a
use real type for JSONSchema object
ryantxu Feb 29, 2024
0847e1d
add spec definition
ryantxu Feb 29, 2024
8482109
more specs
ryantxu Feb 29, 2024
8dbdba0
more specs
ryantxu Feb 29, 2024
bb0cd98
lint
ryantxu Feb 29, 2024
2b3cc45
now with unstructured
ryantxu Mar 1, 2024
0b0e1ea
less verbose
ryantxu Mar 1, 2024
a804614
now with unstructured
ryantxu Mar 1, 2024
424be9a
now with unstructured
ryantxu Mar 1, 2024
3a97c12
more copy methods
ryantxu Mar 1, 2024
f8336e6
fix lint remove parser
ryantxu Mar 2, 2024
2a7ba03
fix lint
ryantxu Mar 2, 2024
2ec1471
panel cleanup
ryantxu Mar 2, 2024
0237a40
panel cleanup
ryantxu Mar 2, 2024
d35e870
inline timerange
ryantxu Mar 3, 2024
a7986a7
more deepcopy
ryantxu Mar 3, 2024
71374e7
hopefully last big package rename
ryantxu Mar 4, 2024
d2c2e90
loading frame types
ryantxu Mar 4, 2024
0e568d6
loading frame types
ryantxu Mar 4, 2024
e7e1b68
fix test
ryantxu Mar 4, 2024
81d7990
remove unused settings configs
ryantxu Mar 4, 2024
0fe8c9c
move panel to schema builder
ryantxu Mar 4, 2024
bd58a3b
Merge remote-tracking branch 'origin/main' into query-type-handler
ryantxu Mar 4, 2024
e9b643e
use k8s style paths
ryantxu Mar 4, 2024
c52d81c
use codegen
ryantxu Mar 4, 2024
91df7c5
use codegen
ryantxu Mar 4, 2024
66438c4
lint
ryantxu Mar 4, 2024
557e208
rename to data
ryantxu Mar 5, 2024
283c5df
update readme
ryantxu Mar 6, 2024
782e1f5
update headers
ryantxu Mar 6, 2024
342bf4e
update headers
ryantxu Mar 6, 2024
0fd38ec
Merge remote-tracking branch 'origin/main' into query-type-handler
ryantxu Mar 6, 2024
86b43fa
move package
ryantxu Mar 6, 2024
95b4050
move package
ryantxu Mar 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions apis/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
This package helps exposes objects from the plugin SDK into k8s based apis.

This should most likely include the k8s.io/apimachinery dependency -- however this initial work
will just mock that values and see how well that works. Avoiding the need for https://github.com/grafana/grafana-plugin-sdk-go/pull/909



19 changes: 19 additions & 0 deletions apis/sdkapi/v0alpha1/metaV1.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
package v0alpha1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this is going to hold only query schemas for the time being we can call the group query (apis/query/v0alpha1/). If the plan is to add more things here then data would also be fine.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, why not adding this to experimental first?

Copy link
Member Author

@ryantxu ryantxu Mar 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the plan is to add more things here then data would also be fine.

The next thing is settings (app + datasource... maybe the same object?) so want to avoid "query" -- i'll use data; it is a bit awkward since that is also the package for data.Frame, but eventually Frame should be in something clients would want access to

why not adding this to experimental first?

I have bounced between them a few times 🤣 -- the question is really how compatible we want to be with k8s. The directory structure /apis/{group}/{version} is standard in k8s and deviating from it makes life hard. In that world "v0*" is experimental, and also has a clear pattern for version evolution.

Working from the assumption that we will eventually land in k8s style apis seems like the right path here (though I do not feel too strongly about it).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the purpose of not confusing/communicating to plugin authors whether this is experimental or not it would be good to put this in the experimental package I think for now until things has settled/this is officially supported.


// ObjectMeta is a struct that aims to "look" like a real kubernetes object when
// written to JSON, however it does not require the pile of dependencies
// This is really an internal helper until we decide which dependencies make sense
// to require within the SDK
type ObjectMeta struct {
// The name is for k8s and description, but not used in the schema
Name string `json:"name,omitempty"`
// Changes indicate that *something * changed
ResourceVersion string `json:"resourceVersion,omitempty"`
// Timestamp
CreationTimestamp string `json:"creationTimestamp,omitempty"`
}

type TypeMeta struct {
Kind string `json:"kind"` // "QueryTypeDefinitionList",
APIVersion string `json:"apiVersion"` // "query.grafana.app/v0alpha1",
}
82 changes: 82 additions & 0 deletions apis/sdkapi/v0alpha1/openapi.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
package v0alpha1

import (
"embed"

"k8s.io/kube-openapi/pkg/common"
spec "k8s.io/kube-openapi/pkg/validation/spec"
)

//go:embed query.schema.json query.definition.schema.json
var f embed.FS

func GetOpenAPIDefinitions(ref common.ReferenceCallback) map[string]common.OpenAPIDefinition {
return map[string]common.OpenAPIDefinition{
"github.com/grafana/grafana-plugin-sdk-go/backend.DataResponse": schemaDataResponse(ref),
"github.com/grafana/grafana-plugin-sdk-go/data.Frame": schemaDataFrame(ref),
"github.com/grafana/grafana-plugin-sdk-go/apis/sdkapi/v0alpha1.DataQuery": schemaDataQuery(ref),
"github.com/grafana/grafana-plugin-sdk-go/apis/sdkapi/v0alpha1.QueryTypeDefinitionSpec": schemaQueryTypeDefinitionSpec(ref),
}
}

// Individual response
func schemaDataResponse(_ common.ReferenceCallback) common.OpenAPIDefinition {
return common.OpenAPIDefinition{
Schema: spec.Schema{
SchemaProps: spec.SchemaProps{
Description: "todo... improve schema",
Type: []string{"object"},
AdditionalProperties: &spec.SchemaOrBool{Allows: true},
},
},
}
}

func schemaDataFrame(_ common.ReferenceCallback) common.OpenAPIDefinition {
return common.OpenAPIDefinition{
Schema: spec.Schema{
SchemaProps: spec.SchemaProps{
Description: "any object for now",
Type: []string{"object"},
Properties: map[string]spec.Schema{},
AdditionalProperties: &spec.SchemaOrBool{Allows: true},
},
},
}
}

func schemaQueryTypeDefinitionSpec(_ common.ReferenceCallback) common.OpenAPIDefinition {
s, _ := loadSchema("query.definition.schema.json")
if s == nil {
s = &spec.Schema{}
}
return common.OpenAPIDefinition{
Schema: *s,
}
}

func schemaDataQuery(_ common.ReferenceCallback) common.OpenAPIDefinition {
s, _ := DataQuerySchema()
if s == nil {
s = &spec.Schema{}
}
s.SchemaProps.Type = []string{"object"}
s.SchemaProps.AdditionalProperties = &spec.SchemaOrBool{Allows: true}
return common.OpenAPIDefinition{Schema: *s}
}

// Get the cached feature list (exposed as a k8s resource)
func DataQuerySchema() (*spec.Schema, error) {
return loadSchema("query.schema.json")
}

// Get the cached feature list (exposed as a k8s resource)
func loadSchema(path string) (*spec.Schema, error) {
body, err := f.ReadFile(path)
if err != nil {
return nil, err
}
s := &spec.Schema{}
err = s.UnmarshalJSON(body)
return s, err
}
40 changes: 40 additions & 0 deletions apis/sdkapi/v0alpha1/openapi_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
package v0alpha1

import (
"encoding/json"
"os"
"testing"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"k8s.io/kube-openapi/pkg/validation/spec"
"k8s.io/kube-openapi/pkg/validation/strfmt"
"k8s.io/kube-openapi/pkg/validation/validate"
)

func TestOpenAPI(t *testing.T) {
//nolint:gocritic
defs := GetOpenAPIDefinitions(func(path string) spec.Ref { // (unlambda: replace ¯\_(ツ)_/¯)
return spec.MustCreateRef(path) // placeholder for tests
})

def, ok := defs["github.com/grafana/grafana-plugin-sdk-go/backend.DataResponse"]
require.True(t, ok)
require.Empty(t, def.Dependencies) // not yet supported!

validator := validate.NewSchemaValidator(&def.Schema, nil, "data", strfmt.Default)

body, err := os.ReadFile("./testdata/sample_query_results.json")
require.NoError(t, err)
unstructured := make(map[string]any)
err = json.Unmarshal(body, &unstructured)
require.NoError(t, err)

result := validator.Validate(unstructured)
for _, err := range result.Errors {
assert.NoError(t, err, "validation error")
}
for _, err := range result.Warnings {
assert.NoError(t, err, "validation warning")
}
}
72 changes: 72 additions & 0 deletions apis/sdkapi/v0alpha1/query.definition.schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
{
"$schema": "https://json-schema.org/draft-04/schema#",
"properties": {
"discriminators": {
"items": {
"properties": {
"field": {
"type": "string",
"description": "DiscriminatorField is the field used to link behavior to this specific\nquery type. It is typically \"queryType\", but can be another field if necessary"
},
"value": {
"type": "string",
"description": "The discriminator value"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"field",
"value"
]
},
"type": "array",
"description": "Multiple schemas can be defined using discriminators"
},
"description": {
"type": "string",
"description": "Describe whe the query type is for"
},
"schema": {
"$ref": "https://json-schema.org/draft-04/schema#",
"type": "object",
"description": "The query schema represents the properties that can be sent to the API\nIn many cases, this may be the same properties that are saved in a dashboard\nIn the case where the save model is different, we must also specify a save model"
},
"examples": {
"items": {
"properties": {
"name": {
"type": "string",
"description": "Version identifier or empty if only one exists"
},
"description": {
"type": "string",
"description": "Optionally explain why the example is interesting"
},
"saveModel": {
"additionalProperties": true,
"type": "object",
"description": "An example value saved that can be saved in a dashboard"
}
},
"additionalProperties": false,
"type": "object"
},
"type": "array",
"description": "Examples (include a wrapper) ideally a template!"
},
"changelog": {
"items": {
"type": "string"
},
"type": "array",
"description": "Changelog defines the changed from the previous version\nAll changes in the same version *must* be backwards compatible\nOnly notable changes will be shown here, for the full version history see git!"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"schema",
"examples"
]
}
Loading