Slim down the ClickHouse database #6352
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
bnaecker
commented
Aug 15, 2024
bnaecker
commented
- Add TTLs to all field tables, by using a materialized column with the time each record is inserted. ClickHouse will retain the latest timestamp, so when we stop inserting, the TTL clock will start counting down on those timeseries records.
- Update Dropshot dependency.
- Add operation ID to HTTP service timeseries, remove other fields. Expunge the old timeseries too.
- Remove unnecessary stingifying of URIs in latency tracking.
bnaecker
force-pushed
the
slim-down-http-latency-histograms
branch
from
550ebec to
2d4b7d8
Compare
- Add TTLs to all field tables, by using a materialized column with the time each record is inserted. ClickHouse will retain the latest timestamp, so when we stop inserting, the TTL clock will start counting down on those timeseries records. - Update Dropshot dependency. - Add operation ID to HTTP service timeseries, remove other fields. Expunge the old timeseries too. - Remove unnecessary stingifying of URIs in latency tracking. - Fixes #6328 and #6331
bnaecker
force-pushed
the
slim-down-http-latency-histograms
branch
from
2d4b7d8 to
1ab13b4
Compare
andrewjstone
approved these changes
Aug 19, 2024
| @@ -14,7 +14,7 @@ description = "Duration for the server to handle a request" | |||
| units = "seconds" | |||
| datum_type = "histogram_f64" | |||
| versions = [ | |||
| { added_in = 1, fields = [ "route", "method", "status_code" ] } | |||
| { added_in = 1, fields = [ "operation_id", "status_code" ] } | |||
There was a problem hiding this comment.
Does the added_in version need to be bumped?
There was a problem hiding this comment.
I'm still partway through supporting actual updates to these schema. Until then, we're getting away with this because we drop the data from the old schema. When I do finish that work, then we'd need to add a new versions entry, describing the new fields.
bnaecker
commented
Aug 19, 2024
This was referenced Aug 19, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.