Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(similarity): Do not send over 30 system-only frames to seer #80946

Merged
merged 8 commits into from
Nov 20, 2024

Conversation

jangjodi
Copy link
Member

@jangjodi jangjodi commented Nov 18, 2024

If an event has no in-app frames and over 30 system frames, return an empty formatted stacktrace string
This will prevent the stacktrace from being sent to seer for the backfill
This will prevent the stacktrace from being sent to seer in ingest after this change is merged #80796
Add metrics for which platform this happens for and new referrer for did_call_seer metric

@jangjodi jangjodi requested a review from a team as a code owner November 18, 2024 21:56
@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Nov 18, 2024
Copy link
Member

@JoshFerge JoshFerge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to work on the interface of this function. adding another parameter for metrics purposes is a sign that this function get_grouping_info_from_variants is doing too many things and getting bloated.

I would recommend that if this case is reached, we instead raise an exception which we can then catch, call this metric, and set the value to empty string.

you can declare another function which holds this exception logic, which calls get_stacktrace_string. and you can keep the endpoint call the same as we do not need metrics on that call.

IMO,i

@jangjodi jangjodi requested a review from JoshFerge November 19, 2024 17:41
@jangjodi
Copy link
Member Author

I think we need to work on the interface of this function. adding another parameter for metrics purposes is a sign that this function get_grouping_info_from_variants is doing too many things and getting bloated.

I would recommend that if this case is reached, we instead raise an exception which we can then catch, call this metric, and set the value to empty string.

you can declare another function which holds this exception logic, which calls get_stacktrace_string. and you can keep the endpoint call the same as we do not need metrics on that call.

IMO,i

Thank you for your review!

Copy link

codecov bot commented Nov 19, 2024

Codecov Report

Attention: Patch coverage is 93.10345% with 2 lines in your changes missing coverage. Please review.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...ssues/endpoints/group_similar_issues_embeddings.py 60.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #80946      +/-   ##
==========================================
+ Coverage   78.48%   78.50%   +0.01%     
==========================================
  Files        7214     7206       -8     
  Lines      319710   319378     -332     
  Branches    44008    43959      -49     
==========================================
- Hits       250937   250715     -222     
+ Misses      62374    62272     -102     
+ Partials     6399     6391       -8     

Copy link
Member

@armenzg armenzg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks very good!

Minor changes and some questions.

tests/sentry/grouping/ingest/test_seer.py Show resolved Hide resolved
tests/sentry/grouping/ingest/test_seer.py Outdated Show resolved Hide resolved
tests/sentry/seer/similarity/test_utils.py Outdated Show resolved Hide resolved
@@ -261,3 +262,55 @@ def test_returns_no_grouphash_and_empty_metadata_if_no_similar_group_found(self)
expected_metadata,
None,
)

@patch("sentry.seer.similarity.utils.metrics")
def test_too_many_only_system_frames(self, mock_metrics: Mock) -> None:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a test that tracks what happens if you have N exceptions and the sum of their frames is over MAX_FRAMES?

Do we have a test that handles multiple exceptions but only 1 frame is in-app?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion! I've added a test for the first case.
Regarding the second case, if we have app data, it's used instead of system data (see here) and we have a test for it here

tests/sentry/tasks/test_backfill_seer_grouping_records.py Outdated Show resolved Hide resolved
tests/sentry/tasks/test_backfill_seer_grouping_records.py Outdated Show resolved Hide resolved
Copy link
Member

@JoshFerge JoshFerge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good. thanks for the changes. small comments!

@jangjodi jangjodi merged commit 5b7dad4 into master Nov 20, 2024
49 of 50 checks passed
@jangjodi jangjodi deleted the jodi/similarity-over-30-system-frames-ineligible branch November 20, 2024 21:05
@jangjodi jangjodi added the Trigger: Revert Add to a merged PR to revert it (skips CI) label Nov 20, 2024
@getsentry-bot
Copy link
Contributor

PR reverted: 4b1562c

getsentry-bot added a commit that referenced this pull request Nov 20, 2024
harshithadurai pushed a commit that referenced this pull request Nov 25, 2024
…0946)

If an event has no in-app frames and over 30 system frames, return an
null formatted stacktrace string
harshithadurai pushed a commit that referenced this pull request Nov 25, 2024
evanh pushed a commit that referenced this pull request Nov 25, 2024
…0946)

If an event has no in-app frames and over 30 system frames, return an
null formatted stacktrace string
evanh pushed a commit that referenced this pull request Nov 25, 2024
jangjodi added a commit that referenced this pull request Nov 25, 2024
…0946)

If an event has no in-app frames and over 30 system frames, return an
null formatted stacktrace string
andrewshie-sentry pushed a commit that referenced this pull request Dec 2, 2024
…0946)

If an event has no in-app frames and over 30 system frames, return an
null formatted stacktrace string
andrewshie-sentry pushed a commit that referenced this pull request Dec 2, 2024
@github-actions github-actions bot locked and limited conversation to collaborators Dec 6, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Scope: Backend Automatically applied to PRs that change backend components Trigger: Revert Add to a merged PR to revert it (skips CI)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants