Scribbles labelmap mismatch?

[adamaji]

Hi folks, I'm trying to get scribbles working with the Slicer plugin and dev MONAILabel but having some trouble -- calling the scribbles update after adding some foreground scribbles doesn't update the organ label as expected.

I'm running all the sample stuff

# download radiology app and sample dataset
monailabel apps --download --name radiology --output apps
monailabel datasets --download --name Task09_Spleen --output datasets

# start server using radiology app with deepedit model enabled
monailabel start_server --app apps/radiology --studies datasets/Task09_Spleen/imagesTr --conf models deepedit

and a recent preview build of Slicer: 2022-03-28. My installed MONAILabel plugin is via adding the module path to the pulled down repo at f9e37fe.

From there in Slicer I

1. Connect to the server
2. Click next sample to load in an image
3. Paint some "foreground" from the scribbles tool approximately where the spleen should be
4. Click update in scribbles

and no changes are made to the spleen segment. If I select "spleen" instead of "background" in SmartEdit / Deepgrow > Label and then press update in scribbles, then the segment does stuff added to it, but it just ends up segmenting the entire space of the image.

In the server logs I see that the background/foreground scribbles are expected to be labels 2/3

[2022-03-31 14:14:54,056] [24800] [MainThread] [INFO] (root:89) - warning: no background scribbles received with label 2, available in scribbles [0. 4.]
[2022-03-31 14:14:54,088] [24800] [MainThread] [INFO] (root:96) - warning: no foreground scribbles received with label 3, available in scribbles [0. 4.]

which tracks with the infer call here

MONAILabel/monailabel/scribbles/infer.py

Lines 103 to 104 in f9e37fe

	scribbles_bg_label=2,
	scribbles_fg_label=3,

however, checking the created scribbles labelmap, the foreground scribbles have been assigned as 4. It seems that the labelmap export call

MONAILabel/plugins/slicer/MONAILabel/MONAILabel.py

Line 1282 in f9e37fe

slicer.modules.segmentations.logic().ExportVisibleSegmentsToLabelmapNode(

assigns "background" - 1, "spleen" - 2, "background_scribbles" - 3, "foreground_scribbles" - 4 based on the generated segments in the label segmentation.

MONAILabel/plugins/slicer/MONAILabel/MONAILabel.py

Lines 1522 to 1524 in f9e37fe

	for label in labels:
	if not segmentation.GetSegmentIdBySegmentName(label):
	segmentation.AddEmptySegment(label, label, self.getLabelColor(label))

Am I missing something here? Is background not supposed to be added as a separate segment?

[SachidanandAlle]

@masadcv can you help here..

[masadcv]

Thanks @SachidanandAlle I will have a look into this.
Was there any recent changes to Slicer plugin that could result in additional labelmaps in what is sent to server?

[adamaji]

I think I see the issue here, which is the use of ExportVisibleSegmentsToLabelmapNode while the "background" label is visible. It isn't hidden automatically, so it gets exported as its own label class. Rather it seems ExportSegmentsToLabelmapNode should be used to specify exactly which segments are transferred. This is also a problem in onSaveLabel (

MONAILabel/plugins/slicer/MONAILabel/MONAILabel.py

Line 1282 in 3f4fef4

slicer.modules.segmentations.logic().ExportVisibleSegmentsToLabelmapNode(

) where the exported label sets the organ class to "2" and "background" to "1" if "background" is visible.

I can provide a fix for this if this isn't the expected behavior.

[masadcv]

Hello @adamaji, Many thanks for reporting this. It is not expected behavior.
I am debugging this, will update once I reproduce and step through the code.

[adamaji]

I've posted #716 as a potential fix

[masadcv]

Many thanks for posting a potential fix. It seems like a temporary workaround that may only work for this specific case, given the user has selected required label in SmartEdit tab.

I have a fix for this which will address this, without needing to select a specific label. I am testing it now, will have a PR once it passes all tests. I will update here once I do this.

[masadcv]

@adamaji can you please try this: #717 and let me know if this solves your issue?

Many thanks for detailed comments as well as looking into this for a possible a workaround!

[adamaji]

Thanks for the help. Continued the discussion in the PR, though I will have to come back to this tomorrow

[adamaji]

Marking this as the answer since the linked PR should resolve this issue.