Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Column alias tables for simplified querying #862

Merged
merged 6 commits into from
Jun 16, 2024

Conversation

SpencerTorres
Copy link
Collaborator

Purpose

Some schemas have complicated maps such as OTel LogAttributes. This feature allows configuring a table like this:

CREATE TABLE otel_logs_query_aliases (
	`alias` String,
	`select` String,
	`type` String
)
Engine = MergeTree
ORDER BY (alias);
INSERT INTO otel_logs_query_aliases (`alias`, `select`, `type`) VALUES
('productId', 'LogAttributes[\'productId\']', 'UInt64'),
('quantity', 'LogAttributes[\'quantity\']', 'String'),
('userId', 'LogAttributes[\'userId\']', 'UInt64'),
('code.filepath', 'LogAttributes[\'code.filepath\']', 'String'),
('code.function', 'LogAttributes[\'code.function\']', 'String'),
('code.lineno', 'LogAttributes[\'code.lineno\']', 'Int64'),
. . .

This table can then be configured in the plugin:
config query alias tables

And then be used for the target table's column suggestions:
column suggestions

With the column selection simplified, the correct SQL will be generated using the underlying select syntax:
generated SQL

Notes:

  • Tables without a matching alias will default to the original DESC TABLE for column suggestions.
  • This capability is already built-in to ClickHouse via the ALIAS column type, but this feature allows for easier maintenance without having to run bulk updates on your primary table (some schemas may have thousands of alias entries).

Changes

  • added configuration for alias tables
  • updated query builder to use a label, which defaults to the field name if an alias doesn't override it.
  • added tests for affected files/features
  • updated changelog

@SpencerTorres SpencerTorres requested a review from a team as a code owner June 13, 2024 19:53
asimpson
asimpson previously approved these changes Jun 14, 2024
Copy link
Contributor

@asimpson asimpson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🏆 love it, nicely done!

@SpencerTorres
Copy link
Collaborator Author

The array change you suggested worked locally, I didn't get any errors when unit testing or in-editor, but the build seemed to be failing. I tried updating to a typescript version that supports that extension and it lead to a complete collapse of the plugin's package ecosystem. I think I even saw leftpad mentioned in the logs.

Anyway, let me know if you want me to try again. I want to use that new feature, but there's a lot of dependencies affected by updating TS.

Copy link
Collaborator

@bossinc bossinc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@bossinc bossinc enabled auto-merge (squash) June 16, 2024 15:32
@bossinc bossinc merged commit 5db6b9f into grafana:main Jun 16, 2024
16 checks passed
@SpencerTorres SpencerTorres deleted the flat-querying branch June 17, 2024 17:20
@SpencerTorres SpencerTorres mentioned this pull request Jun 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants