Fix error: invalid child of chunk append #7514
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
erimatnor
force-pushed
the
fix-chunk-append-error
branch
3 times, most recently
from
c21e648 to
c3a271b
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #7514 +/- ##
==========================================
+ Coverage 80.06% 82.18% +2.11%
==========================================
Files 190 230 +40
Lines 37181 43230 +6049
Branches 9450 10875 +1425
==========================================
+ Hits 29770 35527 +5757
- Misses 2997 3378 +381
+ Partials 4414 4325 -89 ☔ View full report in Codecov by Sentry. |
akuzm
reviewed
Dec 4, 2024
src/nodes/chunk_append/planner.c
Outdated
| * nodes with one child, recurse into the child to find a baserel | ||
| * scan. It is not clear what a CustomScan with multiple children | ||
| * would mean here so we don't handle it. */ | ||
| if (list_length(custom->custom_plans) == 1) |
There was a problem hiding this comment.
This part is not needed for the fix? I'd leave it as is, not sure that recursing into arbitrary custom scan nodes is always correct for the ChunkAppend optimization. Technically this might even be a plan node from another extension.
There was a problem hiding this comment.
Yes, it is not needed to avoid the error. But why would it not be correct? It is a recursive call that will just do nothing if it doesn't find a baserel plan node it recognizes for the chunk append purpose.
There was a problem hiding this comment.
I removed it for now, but I don't think this special case is needed.
akuzm
approved these changes
Dec 4, 2024
fabriziomello
approved these changes
Dec 4, 2024
When doing startup and runtime chunk exclusion, the chunk append node could sometimes throw an error: "invalid child of chunk append: Sort". Unfortunately, this error hasn't been successfully reproduced in a test but has been reported by users. However, it seems clear that the error happens in ts_chunk_append_get_scan_plan() when it can't find a "known" plan node to use for chunk exclusion. This function should ideally never throw and error and instead just return NULL, which means that chunk append falls back to not doing any exclusion instead of throwing an error. It is also possible to improve the code and make it properly handle Sort and Result nodes by not special-casing them. By inspecting ts_chunk_append_get_scan_plan(), it is clear that it can only throw the error if it encounters a Result node with a Sort child, because in those two cases it didn't descend down the lefttree child node using a recursive call. Therefore, remove the special case and instead do a recursive call similar to how other nodes are handled.
erimatnor
force-pushed
the
fix-chunk-append-error
branch
from
c3a271b to
e9a8fda
Compare
pallavisontakke
added a commit
to pallavisontakke/timescaledb
that referenced
this pull request
Dec 13, 2024
This release contains performance improvements and bug fixes since the 2.17.2 release. We recommend that you upgrade at the next available opportunity. **Features** * timescale#6901 Add hypertable support for transition tables * timescale#7104 Hypercore table access method * timescale#7271 Push down ORDER BY in real time continuous aggregate queries * timescale#7295: Support ALTER TABLE SET ACCESS METHOD on hypertable. * timescale#7390 Disable custom hashagg planner code * timescale#7411 Change parameter name to enable Hypercore TAM * timescale#7412 Add GUC for hypercore_use_access_method default * timescale#7413: Add GUC for segmentwise recompression. * timescale#7443 Add Hypercore function and view aliases * timescale#7455: Support DROP NOT NULL on compressed hypertables * timescale#7486 Prevent building against postgres versions with broken ABI **Bugfixes** * timescale#7378 Remove obsolete job referencing policy_job_error_retention * timescale#7409 Update bgw job table when altering procedure * timescale#7410 "aggregated compressed column not found" error on aggregation query. * timescale#7426 Fix datetime parsing error in chunk constraint creation * timescale#7432 Verify that heap tuple is valid before using * timescale#7434 Fixes segfault when internally set the replica identity for a given chunk * timescale#7488 Emit error for transition table trigger on chunks * timescale#7514 Fix error: invalid child of chunk append **Thanks** * @bharrisau for reporting the segfault when creating chunks * @pgloader for reporting an issue an internal background job * @uasiddiqi for reporting the "aggregated compressed column not found" error.
pallavisontakke
added a commit
to pallavisontakke/timescaledb
that referenced
this pull request
Dec 16, 2024
This release contains performance improvements and bug fixes since the 2.18.0 release. We recommend that you upgrade at the next available opportunity. **Features** * timescale#6901 Add hypertable support for transition tables * timescale#7104 Hypercore table access method * timescale#7271 Push down ORDER BY in real time continuous aggregate queries * timescale#7295: Support ALTER TABLE SET ACCESS METHOD on hypertable. * timescale#7390 Disable custom hashagg planner code * timescale#7411 Change parameter name to enable Hypercore TAM * timescale#7412 Add GUC for hypercore_use_access_method default * timescale#7413: Add GUC for segmentwise recompression. * timescale#7443 Add Hypercore function and view aliases * timescale#7455: Support DROP NOT NULL on compressed hypertables * timescale#7486 Prevent building against postgres versions with broken ABI **Bugfixes** * timescale#7378 Remove obsolete job referencing policy_job_error_retention * timescale#7409 Update bgw job table when altering procedure * timescale#7410 "aggregated compressed column not found" error on aggregation query. * timescale#7426 Fix datetime parsing error in chunk constraint creation * timescale#7432 Verify that heap tuple is valid before using * timescale#7434 Fixes segfault when internally set the replica identity for a given chunk * timescale#7488 Emit error for transition table trigger on chunks * timescale#7514 Fix error: invalid child of chunk append **Thanks** * @bharrisau for reporting the segfault when creating chunks * @pgloader for reporting an issue an internal background job * @uasiddiqi for reporting the "aggregated compressed column not found" error.
pallavisontakke
added a commit
to pallavisontakke/timescaledb
that referenced
this pull request
Dec 16, 2024
This release contains performance improvements and bug fixes since the 2.17.2 release. We recommend that you upgrade at the next available opportunity. **Features** * timescale#6901 Add hypertable support for transition tables * timescale#7104 Hypercore table access method * timescale#7271 Push down ORDER BY in real time continuous aggregate queries * timescale#7295: Support ALTER TABLE SET ACCESS METHOD on hypertable. * timescale#7390 Disable custom hashagg planner code * timescale#7411 Change parameter name to enable Hypercore TAM * timescale#7412 Add GUC for hypercore_use_access_method default * timescale#7413: Add GUC for segmentwise recompression. * timescale#7443 Add Hypercore function and view aliases * timescale#7455: Support DROP NOT NULL on compressed hypertables * timescale#7486 Prevent building against postgres versions with broken ABI **Bugfixes** * timescale#7378 Remove obsolete job referencing policy_job_error_retention * timescale#7409 Update bgw job table when altering procedure * timescale#7410 "aggregated compressed column not found" error on aggregation query. * timescale#7426 Fix datetime parsing error in chunk constraint creation * timescale#7432 Verify that heap tuple is valid before using * timescale#7434 Fixes segfault when internally set the replica identity for a given chunk * timescale#7488 Emit error for transition table trigger on chunks * timescale#7514 Fix error: invalid child of chunk append **Thanks** * @bharrisau for reporting the segfault when creating chunks * @pgloader for reporting an issue an internal background job * @uasiddiqi for reporting the "aggregated compressed column not found" error.
Closed
pallavisontakke
added a commit
to pallavisontakke/timescaledb
that referenced
this pull request
Dec 16, 2024
This release contains performance improvements and bug fixes since the 2.17.2 release. We recommend that you upgrade at the next available opportunity. **Features** * timescale#6901 Add hypertable support for transition tables * timescale#7104 Hypercore table access method * timescale#7271 Push down ORDER BY in real time continuous aggregate queries * timescale#7295: Support ALTER TABLE SET ACCESS METHOD on hypertable. * timescale#7390 Disable custom hashagg planner code * timescale#7411 Change parameter name to enable Hypercore TAM * timescale#7412 Add GUC for hypercore_use_access_method default * timescale#7413: Add GUC for segmentwise recompression. * timescale#7443 Add Hypercore function and view aliases * timescale#7455: Support DROP NOT NULL on compressed hypertables * timescale#7486 Prevent building against postgres versions with broken ABI **Bugfixes** * timescale#7378 Remove obsolete job referencing policy_job_error_retention * timescale#7409 Update bgw job table when altering procedure * timescale#7410 "aggregated compressed column not found" error on aggregation query. * timescale#7426 Fix datetime parsing error in chunk constraint creation * timescale#7432 Verify that heap tuple is valid before using * timescale#7434 Fixes segfault when internally set the replica identity for a given chunk * timescale#7488 Emit error for transition table trigger on chunks * timescale#7514 Fix error: invalid child of chunk append **Thanks** * @bharrisau for reporting the segfault when creating chunks * @pgloader for reporting an issue an internal background job * @uasiddiqi for reporting the "aggregated compressed column not found" error.
Closed
pallavisontakke
added a commit
to pallavisontakke/timescaledb
that referenced
this pull request
Dec 16, 2024
This release contains performance improvements and bug fixes since the 2.17.2 release. We recommend that you upgrade at the next available opportunity. **Features** * timescale#6901 Add hypertable support for transition tables * timescale#7104 Hypercore table access method * timescale#7271 Push down ORDER BY in real time continuous aggregate queries * timescale#7295: Support ALTER TABLE SET ACCESS METHOD on hypertable. * timescale#7390 Disable custom hashagg planner code * timescale#7411 Change parameter name to enable Hypercore TAM * timescale#7412 Add GUC for hypercore_use_access_method default * timescale#7413: Add GUC for segmentwise recompression. * timescale#7443 Add Hypercore function and view aliases * timescale#7455: Support DROP NOT NULL on compressed hypertables * timescale#7486 Prevent building against postgres versions with broken ABI **Bugfixes** * timescale#7378 Remove obsolete job referencing policy_job_error_retention * timescale#7409 Update bgw job table when altering procedure * timescale#7410 "aggregated compressed column not found" error on aggregation query. * timescale#7426 Fix datetime parsing error in chunk constraint creation * timescale#7432 Verify that heap tuple is valid before using * timescale#7434 Fixes segfault when internally set the replica identity for a given chunk * timescale#7488 Emit error for transition table trigger on chunks * timescale#7514 Fix error: invalid child of chunk append **Thanks** * @bharrisau for reporting the segfault when creating chunks * @pgloader for reporting an issue an internal background job * @uasiddiqi for reporting the "aggregated compressed column not found" error.
pallavisontakke
added a commit
to pallavisontakke/timescaledb
that referenced
this pull request
Dec 17, 2024
This release contains performance improvements and bug fixes since the 2.17.2 release. We recommend that you upgrade at the next available opportunity. **Features** * timescale#6901 Add hypertable support for transition tables * timescale#7104 Hypercore table access method * timescale#7271 Push down ORDER BY in real time continuous aggregate queries * timescale#7295: Support ALTER TABLE SET ACCESS METHOD on hypertable. * timescale#7390 Disable custom hashagg planner code * timescale#7411 Change parameter name to enable Hypercore TAM * timescale#7412 Add GUC for hypercore_use_access_method default * timescale#7413: Add GUC for segmentwise recompression. * timescale#7443 Add Hypercore function and view aliases * timescale#7455: Support DROP NOT NULL on compressed hypertables * timescale#7458 Support vecorized aggregation with aggregate FILTER clauses that are also vectorizable * timescale#7486 Prevent building against postgres versions with broken ABI **Bugfixes** * timescale#7378 Remove obsolete job referencing policy_job_error_retention * timescale#7409 Update bgw job table when altering procedure * timescale#7410 "aggregated compressed column not found" error on aggregation query. * timescale#7426 Fix datetime parsing error in chunk constraint creation * timescale#7432 Verify that heap tuple is valid before using * timescale#7434 Fixes segfault when internally set the replica identity for a given chunk * timescale#7488 Emit error for transition table trigger on chunks * timescale#7514 Fix error: invalid child of chunk append **Thanks** * @bharrisau for reporting the segfault when creating chunks * @pgloader for reporting an issue an internal background job * @uasiddiqi for reporting the "aggregated compressed column not found" error.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When doing startup and runtime chunk exclusion, the chunk append node could sometimes throw an error: "invalid child of chunk append: Sort". Unfortunately, this error hasn't been successfully reproduced in a test but has been reported by users.
However, it seems clear that the error happens in
ts_chunk_append_get_scan_plan() when it can't find a "known" plan node to use for chunk exclusion. This function should ideally never throw and error and instead just return NULL, which means that chunk append falls back to not doing any exclusion instead of throwing an error.
It is also possible to improve the code and make it properly handle Sort and Result nodes by not special-casing them. By inspecting ts_chunk_append_get_scan_plan(), it is clear that it can only throw the error if it encounters a Result node with a Sort child, because in those two cases it didn't descend down the lefttree child node using a recursive call. Therefore, remove the special case and instead do a recursive call similar to how other nodes are handled. Also remove the special case for the vector agg node, since it should be OK to recursively process all CustomScan nodes.