optimize bucket level monitor to resolve alias to query only those time-series indices that contain docs within timeframe of range query filter in search input #1701
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
eirsep
requested review from
lezzago,
AWSHurneyt,
sbcd90,
getsaurabh02,
praveensameneni,
bowenlan-amzn,
rishabhmaurya,
engechas,
riysaxen-amzn,
jowg-amazon,
amsiglan and
goyamegh
as code owners
eirsep
changed the title
AWSHurneyt
reviewed
Oct 18, 2024
AWSHurneyt
approved these changes
Oct 18, 2024
sbcd90
reviewed
Oct 19, 2024
| @@ -1190,6 +1191,133 @@ class MonitorRunnerServiceIT : AlertingRestTestCase() { | |||
| assertEquals("Incorrect search result", 2, buckets.size) | |||
| } | |||
|
|
|||
| fun `test execute bucket-level monitor with alias optimization - indices not skipped`() { | |||
There was a problem hiding this comment.
can we add a test case with partial index consideration e.g. after the current monitor run, 5 new docs are added to index1, index1 is rolled over to index2, 5 new docs added to index2, index2 is rolled over to index3, 5 new docs added to index3, index3 is rolled over to index4, 5 new docs added to index4 & now bucket-level monitor run happens. So, this run considers, index1, index2, index3 & index4.
There was a problem hiding this comment.
added 2 test scenarios
scenario one
12 indices in alias
1st index skipped as creation time out of time range of query contains 3 docs (3 new buckets in response)
2nd index NOT SKIPPED with creation time out of time range of query contains 3 docs (3 new buckets in response)
10 indices contains 2 docs (2 new buckets in response)
Response has only 5 buckets
scenario 2
12 indices in alias
1st index NOT SKIPPED as creation time in time range of query contains 3 docs (3 new buckets in response)
2nd index NOT SKIPPED with creation time in time range of query contains 3 docs (3 new buckets in response)
10 indices contains 2 docs (2 new buckets in response)
Response has only 8 buckets
…me-series indices that contain docs within timeframe of range query filter in search input Signed-off-by: Surya Sashank Nistala <[email protected]>
Signed-off-by: Surya Sashank Nistala <[email protected]>
…ces being skipped and not skipped Signed-off-by: Surya Sashank Nistala <[email protected]>
eirsep
force-pushed
the
agg_optimization
branch
from
4de98fc to
4578bc8
Compare
sbcd90
approved these changes
Oct 21, 2024
Signed-off-by: Surya Sashank Nistala <[email protected]>
AWSHurneyt
approved these changes
Oct 21, 2024
opensearch-trigger-bot bot
pushed a commit
that referenced
this pull request
Oct 21, 2024
…me-series indices that contain docs within timeframe of range query filter in search input (#1701) * optimize bucket level monitor to resolve alias to query only those time-series indices that contain docs within timeframe of range query filter in search input Signed-off-by: Surya Sashank Nistala <[email protected]> * add tests to verify alias based optimziation scenarios Signed-off-by: Surya Sashank Nistala <[email protected]> * fix tests to verify aggregation query on alias optimziation with indices being skipped and not skipped Signed-off-by: Surya Sashank Nistala <[email protected]> * null check addded Signed-off-by: Surya Sashank Nistala <[email protected]> --------- Signed-off-by: Surya Sashank Nistala <[email protected]> (cherry picked from commit e3c9fc2) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
opensearch-trigger-bot bot
pushed a commit
that referenced
this pull request
Oct 21, 2024
…me-series indices that contain docs within timeframe of range query filter in search input (#1701) * optimize bucket level monitor to resolve alias to query only those time-series indices that contain docs within timeframe of range query filter in search input Signed-off-by: Surya Sashank Nistala <[email protected]> * add tests to verify alias based optimziation scenarios Signed-off-by: Surya Sashank Nistala <[email protected]> * fix tests to verify aggregation query on alias optimziation with indices being skipped and not skipped Signed-off-by: Surya Sashank Nistala <[email protected]> * null check addded Signed-off-by: Surya Sashank Nistala <[email protected]> --------- Signed-off-by: Surya Sashank Nistala <[email protected]> (cherry picked from commit e3c9fc2) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
opensearch-trigger-bot bot
pushed a commit
that referenced
this pull request
Oct 21, 2024
…me-series indices that contain docs within timeframe of range query filter in search input (#1701) * optimize bucket level monitor to resolve alias to query only those time-series indices that contain docs within timeframe of range query filter in search input Signed-off-by: Surya Sashank Nistala <[email protected]> * add tests to verify alias based optimziation scenarios Signed-off-by: Surya Sashank Nistala <[email protected]> * fix tests to verify aggregation query on alias optimziation with indices being skipped and not skipped Signed-off-by: Surya Sashank Nistala <[email protected]> * null check addded Signed-off-by: Surya Sashank Nistala <[email protected]> --------- Signed-off-by: Surya Sashank Nistala <[email protected]> (cherry picked from commit e3c9fc2) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
eirsep
pushed a commit
that referenced
this pull request
Oct 21, 2024
…me-series indices that contain docs within timeframe of range query filter in search input (#1701) (#1712) * optimize bucket level monitor to resolve alias to query only those time-series indices that contain docs within timeframe of range query filter in search input * add tests to verify alias based optimziation scenarios * fix tests to verify aggregation query on alias optimziation with indices being skipped and not skipped * null check addded --------- (cherry picked from commit e3c9fc2) Signed-off-by: Surya Sashank Nistala <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
eirsep
pushed a commit
that referenced
this pull request
Oct 21, 2024
…me-series indices that contain docs within timeframe of range query filter in search input (#1701) (#1713) * optimize bucket level monitor to resolve alias to query only those time-series indices that contain docs within timeframe of range query filter in search input * add tests to verify alias based optimziation scenarios * fix tests to verify aggregation query on alias optimziation with indices being skipped and not skipped * null check addded --------- (cherry picked from commit e3c9fc2) Signed-off-by: Surya Sashank Nistala <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
eirsep
pushed a commit
that referenced
this pull request
Oct 21, 2024
…me-series indices that contain docs within timeframe of range query filter in search input (#1701) (#1711) * optimize bucket level monitor to resolve alias to query only those time-series indices that contain docs within timeframe of range query filter in search input * add tests to verify alias based optimziation scenarios * fix tests to verify aggregation query on alias optimziation with indices being skipped and not skipped * null check addded --------- (cherry picked from commit e3c9fc2) Signed-off-by: Surya Sashank Nistala <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
optimize bucket level monitor to resolve alias to query only those time-series indices that contain docs within timeframe of range query filter in search input
Related Issues
#1710
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.