feat(workflow_engine): Add process_data_packets method
#82002
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
the
Scope: Backend
saponifi3d
force-pushed
the
jcallender/aci/process_data_packets
branch
from
923a9fc to
623b1e8
Compare
❌ 1 Tests Failed:
View the top 1 failed tests by shortest run time
To view more test analytics, go to the Test Analytics Dashboard |
mifu67
reviewed
Dec 12, 2024
| query_id = BoundedBigIntegerField() | ||
|
|
||
| # TODO - Add a type here |
There was a problem hiding this comment.
Colleen creates an enum for this in #81953
| **kwargs, | ||
| ) -> tuple[Workflow, Detector, DetectorWorkflow, DataConditionGroup]: | ||
| """ | ||
| Create a Worfkllow, Detector, DetectorWorkflow, and DataConditionGroup for testing. |
There was a problem hiding this comment.
I wouldn't normally nitpick typos like this, but since some tests are failing anyway I thought I would point out this docstring 😅
ceorourke
approved these changes
Dec 12, 2024
saponifi3d
force-pushed
the
jcallender/post_process_workflow_engine
branch
from
9d3e9bf to
ee1fbf6
Compare
saponifi3d
force-pushed
the
jcallender/aci/process_data_packets
branch
from
623b1e8 to
b9f8d52
Compare
saponifi3d
force-pushed
the
jcallender/aci/process_data_packets
branch
from
e958e36 to
74e70de
Compare
saponifi3d
force-pushed
the
jcallender/post_process_workflow_engine
branch
from
6e30368 to
41c64a8
Compare
saponifi3d
force-pushed
the
jcallender/aci/process_data_packets
branch
from
74e70de to
39105c8
Compare
wedamija
reviewed
Dec 13, 2024
Comment on lines
+8
to
+10
| def process_data_packets( | ||
| data_packets: list[DataPacket], query_type: str | ||
| ) -> list[tuple[Detector, dict[DetectorGroupKey, DetectorEvaluationResult]]]: |
There was a problem hiding this comment.
Is this idea that the snuba subscription result consumer will call this with each packet?
There was a problem hiding this comment.
Yep! That said, it could also work as a batch processor since it takes a list of packets.
saponifi3d
force-pushed
the
jcallender/aci/process_data_packets
branch
from
39105c8 to
1cdccbf
Compare
saponifi3d
force-pushed
the
jcallender/aci/process_data_packets
branch
from
1cdccbf to
881615d
Compare
Base automatically changed from
jcallender/post_process_workflow_engine
to
master
December 13, 2024 22:28
saponifi3d
force-pushed
the
jcallender/aci/process_data_packets
branch
from
881615d to
7d88677
Compare
…ocess data sources / process detectors
saponifi3d
force-pushed
the
jcallender/aci/process_data_packets
branch
from
d29b3ce to
b623aa7
Compare
evanh
pushed a commit
that referenced
this pull request
Dec 17, 2024
## Description Adding a wrapper method, `process_data_packets` to merge the process_data_sources and process_detectors. Still thinking through higher level abstractions to simplify using the workflow_engine, but this should simplify integrations a little further for now.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Adding a wrapper method,
process_data_packetsto merge the process_data_sources and process_detectors.Still thinking through higher level abstractions to simplify using the workflow_engine, but this should simplify integrations a little further for now.