Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[db] Move towards sync deletion (away from PeriodicDeleter) - step I/II #18833

Merged
merged 8 commits into from
Sep 29, 2023

Conversation

geropl
Copy link
Member

@geropl geropl commented Sep 28, 2023

Description

PeriodicDeleter is a leftover of the time we had multiple databases + db-sync, which forced us to have an async deletion mechanism. Since this is no longer true, we should move away from this pattern as it started to rot really badly.

It's step I/II, and defensively only switches to sync deletion for the entities that don't do that already. Column deletion is done in a 2nd step (#18826).

For details see the commits overview.

Summary generated by Copilot

🤖 Generated by Copilot at c7d19e6

This pull request refactors the database layer of the gitpod-io/gitpod repository to use hard deletion instead of soft deletion for several tables and entities. This simplifies the code, improves performance and maintainability, and avoids unnecessary data retention. It also removes the unused deleted columns and properties from the d_b_team, d_b_project, Project, and Organization tables and interfaces. It includes the corresponding migration files to apply the schema changes to the existing database.

Related Issue(s)

Fixes EXP-718

How to test

  • check server logs, and verify that there are no errors (from garbage collectors or PeriodicDeleter)

Documentation

Preview status

Gitpod was successfully deployed to your preview environment.

Build Options

Build
  • /werft with-werft
    Run the build with werft instead of GHA
  • leeway-no-cache
  • /werft no-test
    Run Leeway with --dont-test
Publish
  • /werft publish-to-npm
  • /werft publish-to-jb-marketplace
Installer
  • analytics=segment
  • with-dedicated-emulation
  • workspace-feature-flags
    Add desired feature flags to the end of the line above, space separated
Preview Environment / Integration Tests
  • /werft with-local-preview
    If enabled this will build install/preview
  • /werft with-preview
  • /werft with-large-vm
  • /werft with-gce-vm
    If enabled this will create the environment on GCE infra
  • with-integration-tests=all
    Valid options are all, workspace, webapp, ide, jetbrains, vscode, ssh. If enabled, with-preview and with-large-vm will be enabled.
  • with-monitoring

/hold

name: "d_b_user_storage_resource",
primaryKeys: ["userId", "uri"],
timeColumn: "_lastModified",
deletionColumn: "deleted",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't use this table for years now... 🙈

@geropl
Copy link
Member Author

geropl commented Sep 29, 2023

/unhold

@roboquat roboquat merged commit d00f9c0 into main Sep 29, 2023
@roboquat roboquat deleted the gpl/716-deleted-1 branch September 29, 2023 06:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants