Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Comply with table specs v1 #613

Merged
merged 59 commits into from
Dec 11, 2023
Merged

Comply with table specs v1 #613

merged 59 commits into from
Dec 11, 2023

Conversation

Copy link

github-actions bot commented Nov 22, 2023

Coverage report

The coverage rate went from 90.48% to 90.74% ⬆️
The branch rate is 84%.

None of the new lines are part of the tested code. Therefore, there is no coverage data about them.

@jluethi
Copy link
Collaborator

jluethi commented Nov 30, 2023

I'm starting to review this PR. One thing I noticed:
The table .zattrs for our well_ROI_table & FOV_ROI_table don't have that "type": "roi_table", that the spec says they should have. Neither in the 3D Zarr nor in the 2D MIP Zarr.

@tcompa
Copy link
Collaborator Author

tcompa commented Nov 30, 2023

The table .zattrs for our well_ROI_table & FOV_ROI_table don't have that "type": "roi_table", that the spec says they should have. Neither in the 3D Zarr nor in the 2D MIP Zarr.

Thanks!
This is now tested with 73e4a87, and fixed with 1361c79.

The same fix is now needed at least within import_ome_zarr.

@tcompa
Copy link
Collaborator Author

tcompa commented Nov 30, 2023

The same fix is now needed at least within import_ome_zarr.

And in create-ome-zarr-multiplex

@tcompa
Copy link
Collaborator Author

tcompa commented Nov 30, 2023

The same fix is now needed at least within import_ome_zarr.

And in create-ome-zarr-multiplex

Both fixed with 62ddae0.

Copy link
Collaborator

@jluethi jluethi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great, our table writing is becoming much more structured.

The PR does open a few questions for me though, I wonder whether we can address them here as well:
Are we expecting most users of the API to provide the attrs or is this a special case? Are we building up the correct attrs when users don't provide them? So far, we are doing this in the individual tasks, right? Should we document more in the docstrings of the table writer functions what valid attrs are that we expect (/where to find that info)?
The attrs are often optional parameters, but core info like table_type seems to be in them.

When are there potential conflicts between attrs & other parameters? (see also #619) What takes precedence?

My overarching question: Which parameters should be exposed as function parameters and what goes into attrs? And are those attrs really optional?

fractal_tasks_core/lib_input_models.py Outdated Show resolved Hide resolved
fractal_tasks_core/lib_tables/v1.py Outdated Show resolved Hide resolved
fractal_tasks_core/lib_tables/__init__.py Show resolved Hide resolved
fractal_tasks_core/lib_tables/v1.py Outdated Show resolved Hide resolved
fractal_tasks_core/tasks/create_ome_zarr.py Show resolved Hide resolved
@jluethi
Copy link
Collaborator

jluethi commented Dec 6, 2023

Scenarios for type:

  1. No type is provided (neither optional new arg, nor in attrs) => Fail
  2. Type is provided with new arg => set the type in attrs to that new type
  3. No type is provided in new arg, but it's in attrs => Fine

@jluethi
Copy link
Collaborator

jluethi commented Dec 6, 2023

Let's also tackle this one here: #619

@jluethi
Copy link
Collaborator

jluethi commented Dec 6, 2023

  • _write_table_v1 should raise errors for masking ROI table or feature table if they don't follow the write attrs structure

@tcompa tcompa merged commit cea992d into main Dec 11, 2023
17 checks passed
@tcompa tcompa deleted the comply-with-table-specs-v1 branch December 11, 2023 09:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment