Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blocky segmentation masks #55

Open
aleksmirosh opened this issue Feb 14, 2024 · 6 comments
Open

Blocky segmentation masks #55

aleksmirosh opened this issue Feb 14, 2024 · 6 comments

Comments

@aleksmirosh
Copy link

aleksmirosh commented Feb 14, 2024

Hello, thank you for your incredible work. This model shows great IoU on my data, much better than all I tested before.
But even after finetune masks look blocky. Like here, like small squares:
mask

Do you have some suggestions please how I can deal with it?
I finetuned with sen1floods11 config and checkpoint.

@aleksmirosh
Copy link
Author

any suggestions?

@HamedAlemo
Copy link
Collaborator

@aleksmirosh we had another user trying this and having similar pattern in their prediction. The issue for them turned out to be that their input data was between [0, 1] but you need to multiply them by 10000 to be integers.
cc @paolofraccaro

@aleksmirosh
Copy link
Author

thank you @HamedAlemo for your response, I much appreciate it.
I tried the model on satellite but not Sentinal data, it is already between 0 and 60000.
I fond that this block is 16x16 size. Is this the problem with the encoder/decoder? Should I finetune it?
I much appreciate your help with the issue

@HamedAlemo
Copy link
Collaborator

I think @thesujitroy or @paolofraccaro should be able to help with this.

@CarlosGomes98
Copy link
Collaborator

Hi @aleksmirosh . This is somewhat confusing to me, as the predictions should be either 0 or 1 for each pixel, so completely dark or completely white, So I suspect at least some of this is due to distortion being introduced by compression in your image (are you saving it as JPEG perhaps?)

Its possible the output of the image is also being affected by the 16x16 patch size though, as you observed. You could try using a smaller patch size on your data, although it may take some more training epochs.

@aleksmirosh
Copy link
Author

Hi @CarlosGomes98 thank you for the response. I much appreciate your time.
My predictions are probabilities, I tested the original finetune script on the provided sentinel floods, and predictions were probabilities between 0 and 1 too. I attached mask I got after original finetune process on flood data.
mask_flood

For to my custom finetune the input is TIFF and I save it as TIFF.
I tried different constants and normalizations, but the results are the same.

I will try to experiment with patch size. Does it make sense to try a bigger patch size? In this case, I must use batch size = 1, just memory limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants