Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cron monitor slugs created via auto-instrumentation are truncated at 50 characters causing a single monitor to be shared #77642

Open
peterfarrell opened this issue Sep 17, 2024 · 4 comments

Comments

@peterfarrell
Copy link

Problem Statement

This is in regards to cron monitors created by auto-instrumentation (in our case Celerybeat). Monitor names created by auto-instrumentation that exceed 50 characters result in a slug that is truncated at 50 characters. This can cause separate scheduled Celerybeat jobs with logn name to check-in on a single Sentry cron monitor because the slug names are truncated and essentially are the same.

Also, each cron monitor check-in changes the configuration of the monitor. This causes confusion because there are more check-ins received than configured. Essentially the last job to check-in wins the configuration.

The issue occurs when monitors are automatically created by the Celerybeat auto-instrumentation via the Python SDK (it does NOT appear to be a Python SDK issue - #77692). When the monitor is created via this route and the slug exceeds 50 characters, Sentry truncates the slug to 50 characters.

For example, we have several Celerybeat periodic tasks are executed 4 times per day:

letter-send-not-signed-or-declined-reminder-notifications-0700
letter-send-not-signed-or-declined-reminder-notifications-0800
letter-send-not-signed-or-declined-reminder-notifications-0900
letter-send-not-signed-or-declined-reminder-notifications-1000

However, all check-in slugs are truncated at 50 characters to and therefore only a single monitor is created:
letter_send_not_signed_or_declined_reminder_notifi

We expected 4 monitors to be created, but only 1 monitor was created. No error or warning is displayed as a result of this issue.

Also, the Sentry UI does not enforce any character limits even though it appears it is limited to 50 characters:

Image

Solution Brainstorm

Auto-instrumentation Creation:

  • Increase the acceptable length of the slug name. django-celerybeat allows up to 200 characters however this makes the fix work for celery without regard for other solutions.
  • Do not allow to create monitors when the slug exceeds 50 instead of truncating the slug silently. This change may break existing users as folks may unknowingly rely on this behavior.

UI:

  • Ensure the UI enforces a slug character limit.

Product Area

Crons

@getsantry
Copy link
Contributor

getsantry bot commented Sep 17, 2024

Assigning to @getsentry/support for routing ⏲️

@getsantry
Copy link
Contributor

getsantry bot commented Sep 17, 2024

Routing to @getsentry/product-owners-crons for triage ⏲️

@getsantry getsantry bot moved this from Waiting for: Support to Waiting for: Product Owner in GitHub Issues with 👀 3 Sep 17, 2024
@gaprl
Copy link
Member

gaprl commented Sep 18, 2024

Thank you @peterfarrell, we'll look into increasing the maximum length of the monitor's slug.

@DaveRosenthalSentry
Copy link

Looks like this issue would not be resolved by increasing the character count. It sounds like there are also just plain bugs and we need to consistently enforce whatever truncation happens up front, before any other logic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants