Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TM-1520] POC Unified DB Bulk Upload #36

Merged
merged 12 commits into from
Jan 7, 2025

Conversation

roguenet
Copy link
Collaborator

@roguenet roguenet commented Jan 6, 2025

https://gfw.atlassian.net/browse/TM-1520

This ticket ended up being a lot bigger than I originally thought because I also fleshed out the full Project record for Airtable, which turned out to be a lot more involved than I thought it would be. I'm also going to be making some changes to how this works in the next ticket because the process as it stands is optimized to limit the number of queries as much as possible, but this ends up causing a different inefficiency: It means that there is a row returned from the DB per included record, which can mean thousands of rows for only a couple of projects once you get down to the tree species and workdays data. So, that system is going to need a change as well.

As with the previous ticket, this is a POC and does not include tests. The next ticket will end up getting those whole project to a stable state and will include tests.

Copy link

nx-cloud bot commented Jan 6, 2025

View your CI Pipeline Execution ↗ for commit 39c5b09.

Command Status Duration Result
nx run-many -t test --coverage --passWithNoTests ✅ Succeeded 1m 22s View ↗
nx test database --no-cache ✅ Succeeded 6s View ↗
nx run-many -t lint build ✅ Succeeded 28s View ↗

☁️ Nx Cloud last updated this comment at 2025-01-06 20:53:36 UTC

@roguenet roguenet merged commit 594ce2d into staging Jan 7, 2025
2 checks passed
@roguenet roguenet deleted the feat/TM-1520-unified-db-bulk-upload branch January 7, 2025 17:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants