[DAR-4924][External] Resolving issues with import & export of NifTI annotations
#979
Conversation
| slot_affine_map = {} | ||
| for slot in remote_file.slots: | ||
| slot_affine_map[slot["slot_name"]] = np.array( | ||
| slot["metadata"]["medical"]["affine"], | ||
| dtype=np.float64, | ||
| ) | ||
| remote_files_that_require_legacy_scaling[ | ||
| Path(remote_file.full_path) | ||
| ] = slot_affine_map |
There was a problem hiding this comment.
If we need to re-orient annotations upon import, we have to know what the orientation of the in-platform file is, so we return it from this function and use it downstream
| remote_file_path = Path(nifti_annotation["image"]) | ||
| if not str(remote_file_path).startswith("/"): | ||
| remote_file_path = Path("/" + str(remote_file_path)) |
There was a problem hiding this comment.
Allow import of NifTI annotation to files in dataset folders
| return dt.AnnotationFile( | ||
| path=json_path, | ||
| filename=str(filename), | ||
| remote_path="/", | ||
| remote_path=str(remote_path), | ||
| annotation_classes=annotation_classes, |
There was a problem hiding this comment.
Also allowing import of NifTI annotations to dataset folders
| @@ -353,7 +362,7 @@ def nifti_to_video_polygon_annotation( | |||
| if len(all_frame_ids) == 1: | |||
| segments = [[all_frame_ids[0], all_frame_ids[0] + 1]] | |||
| elif len(all_frame_ids) > 1: | |||
| segments = [[min(all_frame_ids), max(all_frame_ids)]] | |||
| segments = [[min(all_frame_ids), max(all_frame_ids) + 1]] | |||
| video_annotation = dt.make_video_annotation( | |||
There was a problem hiding this comment.
1 line bugfix to resolve issue where imported NifTI polygon annotations longer than 1 frame would always omit the last frame
| ornt: (n,2) orientation array. It defines a transformation from RAS. | ||
| ornt: (n,2) orientation array. It defines a transformation to LPI |
There was a problem hiding this comment.
nibabel and ITK disagree on what orientation is represented by LPI, RAS, etc. We process NifTI annotations exclusively with nibabel, which represents LPI as:
[[0.0, -1.0], [1.0, -1.0], [2.0, -1.0]]
| @@ -531,9 +546,13 @@ def process_nifti( | |||
| img = correct_nifti_header_if_necessary(input_data) | |||
| orig_ax_codes = nib.orientations.aff2axcodes(img.affine) | |||
| orig_ornt = nib.orientations.axcodes2ornt(orig_ax_codes) | |||
| if remote_file_path in remote_files_that_require_legacy_scaling: | |||
| slot_affine_map = remote_files_that_require_legacy_scaling[remote_file_path] | |||
| affine = slot_affine_map[next(iter(slot_affine_map))] # Take the 1st slot | |||
There was a problem hiding this comment.
There was work done in MED-13 to allow export of NifTI annotations from multi-slotted items, but it has never been possible to import NifTI annotations to multi-slotted items. We therefore take the 1st slot in this case
| img = nib.Nifti1Image( | ||
| dataobj=np.flip(volume.pixel_array, (0, 1, 2)).astype(np.int16), | ||
| dataobj=volume.pixel_array.astype(np.int16), | ||
| affine=volume.affine, | ||
| ) |
There was a problem hiding this comment.
Until this PR, this line was flipping the orientation of every axis in every NifTI export
| The volume containing the affine and original affine | ||
| """ | ||
| if volume.original_affine is not None: | ||
| img_ax_codes = nib.orientations.aff2axcodes(volume.affine) |
There was a problem hiding this comment.
I don't think this is correct for pre MED_2D due to the volume.affine being (or expected to be at least) RAS
So this flow will try to go from RAS -> original orientation.
But really its meant to go from LPI -> original orientation. I think this is why dataobj=np.flip(volume.pixel_array, (0, 1, 2)).astype(np.int16) used to be invoked: to go from LPI to RAS, so that then this logic would make sense for pre MED_2D files
Problem
This PR tackles 3 issues related to importing & exporting annotations in the
NifTIformat. Issue 1 is major, issues 2 & 3 are minorNifTIannotations are being incorrectly oriented upon import and export in some situationsNifTIannotations to items in dataset foldersNifTIannotations as polygons, the last frame is always omittedSolution
To tackle these problems:
NifTIannotations incorrectly, and in places we didn't need to be. This PR changes the logic to be the following:When importing
NifTIannotations: If importing to a postMED_2D_VIEWERfile, re-orient toLPIaccording tonibabel. This is the exact same logic we apply to medical file processing in the backend. Otherwise, if importing to a preMED_2D_VIEWERfile, look up current affine of the target dataset item, and re-orient the annotations based on thatWhen exporting
NifTIannotations: Always re-orient the exportedNifTIfile based on the original affine of the dataset item it came fromNifTIannotations to items in dataset foldersNifTIpolygon annotationsIMPORTANT NOTE: Due to what I believe were some issues in the pre
MED_2D_VIEWERbackend processing pipeline, we may have incorrectly stored affines and original affines for some preMED_2D_VIEWERfiles. Since this is all we have to go off when re-orientingNifTIannotations, there may be issues in these cases, and we should keep an eye out for themChangelog
Resolved issues with automatic scaling & orientation of
NifTIannotations