Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: soilDataPush mutation #1472

Merged
merged 9 commits into from
Oct 31, 2024
Merged

feat: soilDataPush mutation #1472

merged 9 commits into from
Oct 31, 2024

Conversation

shrouxm
Copy link
Member

@shrouxm shrouxm commented Sep 26, 2024

Description

This is a Big Ol' PR ™️ which implements the soilDataPush mutation. Roughly speaking, this PR:

  • makes some changes to graphql/schema/commons.py and soil_id/graphql/soil_data/mutations.py to expose some functionality for re-use
  • creates a new model SoilDataHistory to store changes to soil data over time
  • creates a new file soil_id/graphql/soil_data/push_mutation.py defining the new mutation, which accepts:
    • a list of sites with associated soil data changes, including:
      • top level soil data updates
      • depth dependent data updates
      • depth interval updates and deletions
      • depth interval preset changes
  • the mutation logs all data it receives regardless of whether it gets applied, applies the data to each site for which it is valid, and returns a rejection reason for each site with invalid data
  • adds tests for that functionality (still need to be fleshed out more)

Implementation decisions/notes

  • permissions: there are two permissions related to this mutation, SiteActions.UPDATE_DEPTH_INTERVAL and SiteActions.ENTER_DATA. since i believe right now a user will never have one but not both of them, for simplicity i just check if the user has both for each site rather than checking whether the individual update requires each permission on its own.
  • previously, deleting a depth interval which doesn't exist would fail and return an error. in the push endpoint, this is instead a no-op which seems like the better UX for a "coming back online" scenario.
  • a permissions thing to note: right now users with no permissions for a site can write data to its history table. it will be marked as a failed update, and the history table is not public, but we will need to think through this more carefully if we ever decide to expose that data to users
  • right now the endpoint is forgiving if the user sends repeated data. e.g. if the user sends the same site twice in one push, or the same depth interval, or adds and deletes a depth interval in the same push. i think it would be more principled to reject such pushes since they may indicate a client-side bug, but it's more work to do so i'm going to skip it unless anyone feels strongly!
    • follow-up tech debt ticket
  • depth interval presets are weird. i think i understand why we manually delete all depth intervals when we change presets, but since this is now happening in simultaneously with other operations it raises some questions:
    • should we do that deletion before or after applying other depth interval related changes? i implemented "before"
    • should we just not implement this server-side and rely on clients to send the appropriate set of depth intervals to delete? it seems like we'd be more likely to end up with the client's intended data this way, but not sure how that interacts with concurrent changes. but now that i'm writing this out, i'm thinking that since we're recommending to early users to not make concurrent changes we should prioritize user intent
    • needs follow-up ticket
  • there is a lot of duplicated implementation and test code now, because the pushSoilData function duplicates the functionality of four separate previous mutations, which have complicated constraints/permissions conditions and extensive test code of those permissions
    • it would be nice to also have such extensive tests for pushSoilData, but i haven't implemented it yet
    • once this change is out and all clients are running on the pushSoilData endpoint, do we feel good about deleting the old soil data mutations (which i believe will now be unused) to remove this source of duplication? or do we want to have both around
    • test graphql layer now, move other tests to model layer once we remove old mutation code

Checklist

  • Corresponding issue has been opened

Related Issues

Related to #1399 and #1485

@shrouxm shrouxm force-pushed the prototype/data-sync branch 5 times, most recently from 91e4831 to d29c727 Compare October 2, 2024 20:41
result: SoilDataBulkUpdateResult!
}

union SoilDataBulkUpdateResult = SoilDataBulkUpdateSuccess | SoilDataBulkUpdateFailure
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there's one of these per entry, maybe make that clear in the name. Imo without looking at the code around it, this name kinda makes it sound like there's one per bulk update

Comment on lines 2362 to 2305
enum SoilDataBulkUpdateFailureReason {
DOES_NOT_EXIST
NOT_ALLOWED
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a rather niche case that maybe we don't care about, but just a thought:
If this is on a per-site level: is there ever a case where some edits should be allowed, but others not? Like, you're a manager, you edit X and Y, someone changes you to a contributor (who is allowed to edit X but not Y), and then you push?

Copy link
Member Author

@shrouxm shrouxm Oct 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

currently there's no difference in how a manager and contributor can modify soil data (as opposed to sites and projects), so i think this niche case does not exist as things stand (though it could in the future, but i like to live in the present 😎 )

@shrouxm shrouxm force-pushed the prototype/data-sync branch from d29c727 to 2142d4e Compare October 15, 2024 19:13
@shrouxm shrouxm changed the title PROTOTYPE: backend data sync WIP: backend data sync Oct 16, 2024
@shrouxm shrouxm force-pushed the prototype/data-sync branch from d679c5b to 71992cb Compare October 16, 2024 00:04
@shrouxm shrouxm force-pushed the prototype/data-sync branch 2 times, most recently from a04af27 to 5594ee3 Compare October 30, 2024 00:50
@shrouxm shrouxm changed the base branch from main to refactor/split-soil-data-graphql October 30, 2024 00:51
Base automatically changed from refactor/split-soil-data-graphql to main October 30, 2024 16:04
@shrouxm shrouxm force-pushed the prototype/data-sync branch from 5594ee3 to b5fe246 Compare October 30, 2024 16:10
@shrouxm shrouxm marked this pull request as ready for review October 30, 2024 18:29
@shrouxm shrouxm changed the title WIP: backend data sync feat: soilDataPush mutation Oct 30, 2024


class SoilDataPushEntrySuccess(graphene.ObjectType):
site = graphene.Field(SiteNode, required=True)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: change to SoilDataNode

@shrouxm shrouxm force-pushed the prototype/data-sync branch from 65391c7 to 8360648 Compare October 31, 2024 18:26
@shrouxm shrouxm requested review from knipec and tm-ruxandra October 31, 2024 18:29
# an individual site's updates are invalid, we reject all of that site's updates
# NOTE: changing a depth interval preset causes all depth intervals for that site to be
# deleted. we haven't yet thought through the implications of when/whether to apply
# that change in the context of this mutation
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could link to a ticket if you've made one for follow-up work

assert history_3.soil_data_changes["slope_aspect"] == 15


def test_push_soil_data_success(client, user):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: from a readability perspective it might be easier to have the success case before the mixed case?

@shrouxm shrouxm merged commit 74fc2fc into main Oct 31, 2024
7 checks passed
@shrouxm shrouxm deleted the prototype/data-sync branch October 31, 2024 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants