Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enh/2/behavior sessions 2 #16

Merged
merged 15 commits into from
Jul 9, 2024

Conversation

mochic
Copy link
Collaborator

@mochic mochic commented Jun 29, 2024

Addresses #2 and #3.

Uses a different pattern in behavior_session which I'd like to port over to other fetching/writing methods and also partially add to the core client so that table references can be resolved by the core client code.

Also, I'd like to return all records fetched from all fetching code and maybe have the return thresholding as some client level option (ideally to fetch less via the query params if that exists on the slims api ) or for that to be on the user also returning a list of validated and unvalidated data like fetch_behavior_session_content_events

Also, we should change the way Record name aliases are serialized/deserialized for better naming but that might be better for a separate pr.

@mochic
Copy link
Collaborator Author

mochic commented Jul 1, 2024

Can't add reviewers, if either of y'all could take a look at this it would be helpful. @dyf @patricklatimer @jtyoung84 @bruno-f-cruz

Copy link
Collaborator

@patricklatimer patricklatimer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's talk it through, but I now kind of think the client.fetch_models should commit to returning validated models, then you don't have to handle the multiple return types all over.

"""Model for an instance of the Behavior Session ContentEvent"""

pk: int | None = Field(default=None, alias="cnvn_pk")
mouse_name: int | None = Field(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might name this mouse_fk (or mouse_pk?) or something, mouse_name implies it'll match the barcode or labtracks id rather than the pk.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 74e32b8


def fetch_behavior_session_content_events(
client: SlimsClient,
mouse: SlimsSingletonFetchReturn,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be better to accept either a SlimsMouseContent or the parameters to fetch it (i.e. barcode)

Copy link
Collaborator Author

@mochic mochic Jul 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that fetching logic for other records shouldn't be embedded in other fetch functions when necessary, it creates unnecessary coupling within our fetching functions. This function is designed to be used like this:

fetch_behavior_session_content_events(client, fetch_mouse_content(client, labtracks_id), ...)

I would prefer to use some record reference abstraction in the future over this pattern but adding record references seems like it goes outside of the bounds of this pr. The references would allow us to centralize and control when pks get resolved, how they get resolved, and also minimize an end user's experience with that.

validated.append(model.model_validate(record))
except ValidationError as e:
logger.error(f"SLIMS data validation failed, {repr(e)}")
unvalidated.append(record.json_entity)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that I think about it a little more, I don't think this method should return two lists. Doing that kind of splits the difference between using models and using dicts, and I think if we want to use the models we should fully commit.

Might be useful to think through the use cases of when model validation might fail, because ideally the database has proper data (especially if it's been written from the aind_slims_api).

  1. Data got fed to the database manually through ui or through some other process (not this library), and our models are more restricted than the SLIMS fields.
  2. The models got updated and old data doesn't match
  3. The database schema got updated and the models don't match.

I think these are all things that we'd want to catch with integration test errors, but shouldn't expect users to have to deal with.

Anyway, maybe more discussion is warranted, but I'm now thinking validation errors should raise exceptions (maybe we'd want to raise one error for all the failures like Pydantic does; maybe TypeAdapter is appropriate?)

Dictionaries representations of objects that failed validation
"""
response = self.fetch(
model._slims_table.default, # TODO: consider changing fetch method
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

model._slims_table should be a str, right?
model._slims_table,

Copy link
Collaborator Author

@mochic mochic Jul 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a consequence of the way SlimsBaseModel has it's table definition defined, which we should probably amend but in a separate pr.

@jtyoung84 jtyoung84 self-requested a review July 3, 2024 22:07
@jtyoung84 jtyoung84 merged commit 8e5d33b into AllenNeuralDynamics:main Jul 9, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants