Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Investigate cause of slow performance for TOMs with large databases #668

Open
2 tasks
rachel3834 opened this issue Jun 28, 2023 · 0 comments · May be fixed by #619
Open
2 tasks

Investigate cause of slow performance for TOMs with large databases #668

rachel3834 opened this issue Jun 28, 2023 · 0 comments · May be fixed by #619
Labels
bug Something isn't working enhancement New feature or request Epic Overview topic for high level discussion
Milestone
TOMToolkit versio...

Comments

@rachel3834
Copy link
Contributor

rachel3834 commented Jun 28, 2023 •
edited by jchate6
Loading

Is your feature request related to a problem? Please describe.
A number of projects have build TOMs with very large databases, including some that have customised the TOM DB.
A good example is the DESC-TOM, which ingested alert and classification information for a large set of simulated LSST alerts from several different brokers. The goal was to compare the classification data for the same alerts from different brokers, necessitating a large table join. Queries against this DB were very slow.

Describe the solution you'd like
Explore alternative DB structures that might offer better performance for these kinds of projects, for example no-SQL.

Additional context
This has been highlighed as an issue for the DESC-TOM and Sagguro projects.

This is related to:
@rachel3834 rachel3834 added the enhancement New feature or request label Jun 28, 2023
@github-project-automation github-project-automation bot moved this to Backlog in TOM Toolkit Jun 28, 2023
@rachel3834 rachel3834 moved this from Backlog to Icebox in TOM Toolkit Oct 30, 2023
@jchate6 jchate6 mentioned this issue Jan 17, 2024
Open
@jchate6 jchate6 linked a pull request Oct 1, 2024 that will close this issue
Fix/slow target list #619
Draft
@jchate6 jchate6 added bug Something isn't working Epic Overview topic for high level discussion labels Oct 1, 2024
@jchate6 jchate6 moved this from Icebox to Triage in TOM Toolkit Oct 1, 2024
@jchate6 jchate6 added this to the TOMToolkit version 3.0 milestone Oct 11, 2024
@jchate6 jchate6 moved this from Triage to Backlog in TOM Toolkit Oct 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working enhancement New feature or request Epic Overview topic for high level discussion
Projects
TOM Toolkit
Status: Backlog
Milestone
TOMToolkit version 3.0
Development

Successfully merging a pull request may close this issue.

Fix/slow target list TOMToolkit/tom_base
2 participants
@rachel3834 @jchate6