-
Notifications
You must be signed in to change notification settings - Fork 897
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug]: Timescale scans all chunks if timestamp is not computable at query time, even though it is bounded #5352
Comments
I think this issue doesn't affect the correctness of the resultset - so this is not Bug; its more like a feature request for a missed optimization opportunity; to detect and optionally rewrite conditions like:
a bit more self contained repro; (checked with 2.9.1)
Thank you for reporting this - we will get back to it! |
@kgyrtkirk Do you by chance have any potential prio information for this feature request? |
no - I don't know about that; maybe @horzsolt has some info |
@akuzm I saw this is getting abandoned, however, I do still run into this all the time and am surprised it is not a more common issue.
Without the additional constraint on c2 |
What type of bug is this?
Performance issue
What subsystems and features are affected?
Query executor, Query planner
What happened?
We are experiencing bad performance when the time bounds of a query are not passe in as constants. This happens frequently, for example when we need to query daily aggregates in the timezone of a particular location. As a workaround, we can add additional constraints to query data within +/- 1 day of the specified time interval. It would be great if this could be automated though, as it should be possible to deduct these additional time exclusions from the timezone statements. The improved query runs 15 times faster
An example is illustrated below with queries and resulting plans
Good plan:
This query adds additional time bounds with +/- 24 hours to have constant time bounds that will include the needed time interval guaranteed. Adding these redundant constraints improves performance significantly
EXPLAIN (ANALYSE, FORMAT JSON)
SELECT *
FROM node_measurements
join positions p ON node_measurements.position_id = p.position_id
join locations l on p.location_id = l.location_id
-- due to the timezone conversion that depends on reading the timezone from the DB the timestamps can not be computed or known before execution.
WHERE time between timezone(l.timezone , '2023-01-02 00:00:00+0000'::timestamptz) and timezone(l.timezone,'2023-01-03 23:59:59+0000'::timestamptz)
-- The below constraint is not needed but improves performance significantly as it allows the planner to exclude chunks at planning time due to these timestamps being constants
AND time between '2023-01-01 00:00:00+0000'::timestamptz and '2023-01-04 23:59:59+0000'::timestamptz
AND location_name = 'Max Home'
AND measurement_type = 'temperature_set'
AND src_mac_id = 'FFFF14B4577FE5F0'
ORDER by time DESC limit 100;
Bad plan:
Without the extra constant time bounds in the query, the resulting plan is looking at every single chunk in the table instead of excluding them according to the time bounds of the query. The query runs significantly slower because of this.
EXPLAIN (ANALYSE, FORMAT JSON)
SELECT *
FROM node_measurements
join positions p ON node_measurements.position_id = p.position_id
join locations l on p.location_id = l.location_id
WHERE time between timezone(l.timezone , '2023-01-02 00:00:00+0000'::timestamptz) and timezone(l.timezone,'2023-01-03 23:59:59+0000'::timestamptz)
AND location_name = 'Max Home'
AND measurement_type = 'temperature_set'
AND src_mac_id = 'FFFF14B4577FE5F0'
ORDER by time DESC limit 100;
I did report this before here: #2670 but opening it again since it has been a while and is on a completely new version now. feel free
TimescaleDB version affected
2.8.1 and below, likely latest also
PostgreSQL version used
13.9
What operating system did you use?
whatever timescale powered by aiven is running on
What installation method did you use?
Other
What platform did you run on?
Amazon Web Services (AWS)
Relevant log output and stack trace
How can we reproduce the bug?
The text was updated successfully, but these errors were encountered: