Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable self-serve portion of feature and monitor uptake #214

Open
Tracked by #153
azaslavsky opened this issue Nov 7, 2023 · 0 comments
Open
Tracked by #153

Enable self-serve portion of feature and monitor uptake #214

azaslavsky opened this issue Nov 7, 2023 · 0 comments

Comments

@azaslavsky
Copy link

azaslavsky commented Nov 7, 2023 •
edited
Loading

This includes both a dashboard to track usage and some error reporting (via sentry, of course) so that we know when something is going wrong.
@azaslavsky azaslavsky mentioned this issue Nov 7, 2023
Closed
@azaslavsky azaslavsky mentioned this issue Nov 8, 2023
Merged
azaslavsky added a commit to getsentry/sentry that referenced this issue Nov 9, 2023
@azaslavsky
feat(backup): Create standalone encryption command (#59634)
7ae8afc 
This also moves general backup tools into the `sentry backup` command,
so `sentry compare` is now spelled `sentry backup compare`.

Issue: getsentry/team-ospo#214
azaslavsky added a commit to getsentry/sentry that referenced this issue Dec 6, 2023
@azaslavsky
ref(relocation): Swap feature flag for option [1/2]
4b1bce8 
This is the first of 2 PRs to remove the `relocation:enabled` feature
flag, and replace it with the `relocation.enabled` option. This PR adds
the option, but leaves the feature flag untouched. Once this is picked
up by `getsentry`, the follow up PR removing the feature flag will roll
as well.

Issue: getsentry/team-ospo#214
This was referenced Dec 6, 2023
Merged
Merged
azaslavsky added a commit to getsentry/sentry that referenced this issue Dec 6, 2023
@azaslavsky
ref(relocation): Swap feature flag for option [1/2] (#61229)
668d974 
This is the first of 2 PRs to remove the `relocation:enabled` feature
flag, and replace it with the `relocation.enabled` option. This PR adds
the option, but leaves the feature flag untouched. Once this is picked
up by `getsentry`, the follow up PR removing the feature flag will roll
as well.

Issue: getsentry/team-ospo#214
azaslavsky added a commit to getsentry/sentry that referenced this issue Dec 6, 2023
@azaslavsky
ref(relocation): Swap feature flag for option [2/2]
dfae002 
This completes the swap by changing the feature flag to be INTERNAL.

Issue: getsentry/team-ospo#214
azaslavsky added a commit to getsentry/sentry that referenced this issue Dec 6, 2023
@azaslavsky
ref(relocation): Swap feature flag for option [2/2] (#61230)
679c298 
This completes the swap by removing the now-unused feature flag.

Issue: getsentry/team-ospo#214
@azaslavsky azaslavsky mentioned this issue Dec 12, 2023
Merged
azaslavsky added a commit to getsentry/sentry that referenced this issue Dec 12, 2023
@azaslavsky
feat(relocation): Public key download screen
5d73e02 
Add the public key download screen.

Issue: getsentry/team-ospo#214
azaslavsky added a commit to getsentry/sentry that referenced this issue Dec 12, 2023
@azaslavsky
ref(relocation): Organize options a bit
8d1b10e 
Issue: getsentry/team-ospo#214
@azaslavsky azaslavsky mentioned this issue Dec 12, 2023
Merged
@azaslavsky azaslavsky mentioned this issue Dec 13, 2023
Merged
azaslavsky added a commit to getsentry/sentry that referenced this issue Dec 13, 2023
@azaslavsky
feat(relocation): Public key download screen
0d912a6 
Add the public key download screen.

Issue: getsentry/team-ospo#214
azaslavsky added a commit to getsentry/sentry that referenced this issue Dec 13, 2023
@azaslavsky
feat(relocation): Public key download screen (#61578)
2d4cb79 
Add the public key download screen.

![Screenshot 2023-12-08 at 15 42
57](https://github.com/getsentry/sentry/assets/3709945/2056c56f-8f25-420f-88a8-11ec4d0a1fc8)
![Screenshot 2023-12-08 at 15 42
20](https://github.com/getsentry/sentry/assets/3709945/6f4edc60-c85f-4f92-8836-113ca321081b)
![Screenshot 2023-12-08 at 15 41
37](https://github.com/getsentry/sentry/assets/3709945/23fad0f8-18a2-4a58-bf0e-66bdb5a79327)


Issue: getsentry/team-ospo#214
azaslavsky added a commit to getsentry/sentry that referenced this issue Dec 13, 2023
@azaslavsky
ref(relocation): Use separate filestore for relocations
994b804 
Previously, relocations were stored in the same filestore (local
directory, GCP/S3 bucket, etc) as all other Django-tracked files, but in
a special `/relocation` top-level directory. For operational reasons,
we've decided to move relocation data into an entirely separate store.
Thus, we make the following changes:

1. The `filestore` configuration now has an extra property,
   `filestore.relocation`. This closely mirrors the existing
   `filetore.options`: where the latter is used to configure the storage
   backend for regular Django-tracked files, this new config object does
   the same for the relocation bucket. A new storage getter called
   `get_relocation_storage` is created to provide access to this bespoke
   bucket in the same manner we use `get_storage` to directly access the
   main bucket today.
2. The `baseline-config.tar` and `colliding-users.tar` exports are no
   longer tracked in the Django FS. Instead, we write them directly to
   the new relocation bucket at creation time, and forego entering them
   into the DB-bakced filesystem altogether.
3. A new preprocessing step, `PREPROCESSING_TRANSFER`, is added directly
   after the completion of the scan. This copies the user-supplied
   `raw-relocation.tar` from the DB-backed filesystem into the
   relocation bucket. The `PREPROCESSING_COMPLETE` task is modified to
   just do a quick check ensuring that all necessary files are gettable
   by CloudBuild (ie, not only has the write finished, but they are
   available for read as well).

Issue: getsentry/team-ospo#214
azaslavsky added a commit to getsentry/sentry that referenced this issue Dec 13, 2023
@azaslavsky
ref(relocation): Organize options a bit (#61635)
aeed6f3 
This PR introduces a `relocation.self-serve` option, but it is currently
unused. Future PRs will gate the self-serve UI on it, separating the
self-service toggle from the global feature toggle.

Issue: getsentry/team-ospo#214
azaslavsky added a commit to getsentry/sentry that referenced this issue Dec 14, 2023
@azaslavsky
ref(relocation): Use separate filestore for relocations (#61706)
4413316 
Previously, relocations were stored in the same filestore (local
directory, GCP/S3 bucket, etc) as all other Django-tracked files, but in
a special `/relocation` top-level directory. For operational reasons,
we've decided to move relocation data into an entirely separate store.
Thus, we make the following changes:

1. The `filestore` configuration now has an extra property,
`filestore.relocation`. This closely mirrors the existing
`filetore.options`: where the latter is used to configure the storage
backend for regular Django-tracked files, this new config object does
the same for the relocation bucket. A new storage getter called
`get_relocation_storage` is created to provide access to this bespoke
bucket in the same manner we use `get_storage` to directly access the
main bucket today.
2. The `baseline-config.tar` and `colliding-users.tar` exports are no
longer tracked in the Django FS. Instead, we write them directly to the
new relocation bucket at creation time, and forego entering them into
the DB-bakced filesystem altogether.
3. A new preprocessing step, `PREPROCESSING_TRANSFER`, is added directly
after the completion of the scan. This copies the user-supplied
`raw-relocation.tar` from the DB-backed filesystem into the relocation
bucket. The `PREPROCESSING_COMPLETE` task is modified to just do a quick
check ensuring that all necessary files are gettable by CloudBuild (ie,
not only has the write finished, but they are available for read as
well).

Issue: getsentry/team-ospo#214
@azaslavsky azaslavsky mentioned this issue Dec 15, 2023
Merged
azaslavsky added a commit to getsentry/sentry that referenced this issue Dec 15, 2023
@azaslavsky
feat(relocation): Enable globally for superusers
137e966 
Superusers (aka admins) will no be able to hit the relocation endpoint
successfully in all deployments. The `relocation:enabled` flag, and the
`relocation.enabled` option that controls it, will instead refer only to
the self-serve endpoint, which will remain contained to the test region
for now.

This means that admins can now trigger relocations in any region for any
user, and are not throttled when they do so. Regular users may only
trigger relocations for themselves in regions where
`relocations.enabled` is set to `True`, and are subject to the global
daily throttles.

Issue: getsentry/team-ospo#214
azaslavsky added a commit to getsentry/sentry that referenced this issue Dec 15, 2023
@azaslavsky
feat(relocation): Enable globally for superusers (#61822)
c3faedd 
Superusers (aka admins) will now be able to hit the relocation endpoint
successfully in all deployments. The `relocation:enabled` flag, and the
`relocation.enabled` option that controls it, will instead refer only to
the self-serve endpoint, which will remain contained to the test region
for now.

This means that admins can now trigger relocations in any region for any
user, and are not throttled when they do so. Regular users may only
trigger relocations for themselves in regions where
`relocations.enabled` is set to `True`, and are subject to the global
daily throttles.

Issue: getsentry/team-ospo#214
azaslavsky added a commit to getsentry/sentry that referenced this issue Dec 18, 2023
@azaslavsky
feat(backup): Add creator/owner email/username to GET /relocations/
1ed4b90 
This makes the list a bit easier to render in the admin panel, and
minimizes the number of round trips to the server.

Issue: getsentry/team-ospo#214
@azaslavsky azaslavsky mentioned this issue Dec 18, 2023
Merged
azaslavsky added a commit to getsentry/sentry that referenced this issue Dec 19, 2023
@azaslavsky
feat(backup): Add creator/owner email/username to GET /relocations/ (#…
f8d938e 
…61969)

This makes the list a bit easier to render in the admin panel, and
minimizes the number of round trips to the server.

Issue: getsentry/team-ospo#214
azaslavsky added a commit to getsentry/sentry that referenced this issue Dec 21, 2023
@azaslavsky
fix(relocation): Change task scheduling to always be last action
af82efb 
This ensures that the current task is marked "done" in the database
before the next task is kicked off. Since this is handled by the
`retry_or_fail_relocation` context manager, all next task calls must be
moved outside of said context manager, becoming the very last procedures
completed in their owning task.

Because this change required genericizing over the "schedule the next
task" operation, all such operations have been converted in
`.apply_async()` calls, and a new `NextTask` class has been added to
track task invocations that we want to kick off at the end of the
current task.

Issue: getsentry/team-ospo#214
@azaslavsky azaslavsky mentioned this issue Dec 21, 2023
Merged
azaslavsky added a commit to getsentry/sentry that referenced this issue Dec 21, 2023
@azaslavsky
fix(relocation): Change task scheduling to always be last action (#62144
05ebd4e 
)

This ensures that the current task is marked "done" in the database
before the next task is kicked off. Since this is handled by the
`retry_or_fail_relocation` context manager, all next task calls must be
moved outside of said context manager, becoming the very last procedures
completed in their owning task.

Because this change required genericizing over the "schedule the next
task" operation, all such operations have been converted in
`.apply_async()` calls, and a new `NextTask` class has been added to
track task invocations that we want to kick off at the end of the
current task.

Issue: getsentry/team-ospo#214
@azaslavsky azaslavsky mentioned this issue Dec 21, 2023
Merged
azaslavsky added a commit to getsentry/sentry that referenced this issue Dec 21, 2023
@azaslavsky
fix(relocation): Repair routes and owner field
4a3de39 
We want the `owner` field of the POST API to be a username, not email.

An unused route is also removed.

Issue: getsentry/team-ospo#214
@azaslavsky azaslavsky mentioned this issue Dec 21, 2023
Merged
azaslavsky added a commit to getsentry/sentry that referenced this issue Dec 22, 2023
@azaslavsky
fix(relocation): Repair routes and owner field (#62244)
d373ff8 
We want the `owner` field of the POST API to be a username, not email.

An unused route is also removed.

Issue: getsentry/team-ospo#214
azaslavsky added a commit to getsentry/sentry that referenced this issue Dec 22, 2023
@azaslavsky
fix(relocation): Handle expired superuser gracefully (#62243)
cbfdce1 
Issue: getsentry/team-ospo#214
trillville pushed a commit to getsentry/sentry that referenced this issue Jan 19, 2024
@azaslavsky @trillville
fix(relocation): Repair routes and owner field (#62244)
0f11512 
We want the `owner` field of the POST API to be a username, not email.

An unused route is also removed.

Issue: getsentry/team-ospo#214
trillville pushed a commit to getsentry/sentry that referenced this issue Jan 19, 2024
@azaslavsky @trillville
fix(relocation): Handle expired superuser gracefully (#62243)
11f8787 
Issue: getsentry/team-ospo#214
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@azaslavsky