New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Efficient sequence upload #496

Open

atc3 opened this issue Jan 18, 2022 · 0 comments

Assignees

Labels

data

Member

atc3 commented Jan 18, 2022

Right now all sequences are uploaded every day - this will start to overrun our 24-hour update schedule

Proposal:

Upload sequence hash alongside sequence
Before upload, download complete list of Accession IDs + hashes
Locally compare hashes and only upload new Accession IDs and different hashes

atc3 added the data label

atc3 self-assigned this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment