Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Nan padding in blockwise coregistration stats #663

Merged
merged 4 commits into from
Dec 16, 2024

Conversation

vschaffn
Copy link
Contributor

@vschaffn vschaffn commented Nov 26, 2024

Resolves #604

Description

This PR introduces several bug fixes to the BlockwiseCoreg functionnality, focusing on handling missing data and improving robustness in the stats method.

Key changes

In the section of the code where the statistics are collected, for chunks that fail (i.e., where points are not present in chunk._meta), the center coordinates are retrieved, and NaN values are now stored for the relevant statistics, replacing the absence of values.

Example of the BlockwiseCoreg.stats() method output with failing chunks 0 and 1:

chunk center_x center_y center_z x_off y_off z_off inlier_count nmad median
0 1.131960e+06 -644298.185858 NaN NaN NaN NaN NaN NaN NaN
1 1.132980e+06 -644298.185858 NaN NaN NaN NaN NaN NaN NaN
2 1.132080e+06 -644358.185858 152.551422 2.563047 8.209118 -4.270799 911.0 1.124144 0.032944
3 1.132960e+06 -644298.185858 209.978958 6.146258 4.459189 -4.719252 1666.0 1.055472 -0.023361
4 1.133760e+06 -644598.185858 185.525940 34.531841 181.556056 27.067570 309.0 3.279520 -5.745003

Tests

  • Added a test to ensure that NaN padding is applied if a chunk fails.
  • Added a test to verify the correct behavior of BlockwiseCoreg.stats when all chunks are processed successfully.
  • Modified the test_blockwise_coreg_large_gaps test to ensure that the len of stats is the same than the number of subdivision (i.e. each chunk has stats, even if the chunk fails -> NaN stats).

Documentation

Updated the docstring for BlockwiseCoreg.stats to reflect the NaN padding when a chunck fails.

Copy link

@quinto-clara quinto-clara left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two requests :

  • The index used for the loop that build statistics needs to be changed
  • The center_x, center_y and center_z values needs to be kept when a chunk fails

xdem/coreg/base.py Outdated Show resolved Hide resolved
xdem/coreg/base.py Show resolved Hide resolved
@vschaffn vschaffn force-pushed the 604-nan_padding branch 6 times, most recently from 4c03b3b to 936e20b Compare December 5, 2024 10:23
Copy link

@quinto-clara quinto-clara left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The BlockwiseCoreg method handles now missing data, improving robustness of the stats method.

The changes regarding the the two previous requests are approved :

  • The index used for the loop that build statistics is now correct. All the chunks, even failed ones are taken into account.

  • The center_x and center_y values are now kept when a chunk fails. Thus we can show where the NaN values are :

Example with 64 subdivisions and an altered DEM :
Capture d’écran du 2024-12-13 13-35-01

Example with 65 subdivisions (no altered DEM). We can visualize which chunk failed :

Capture d’écran du 2024-12-13 13-36-03

@rhugonnet
Copy link
Member

Great!
I have nothing to comment on, all good for me 🙂 (I assume the disabling of license headers in the last commit is temporary?)

Tagging @erikmannerfelt (who largely coded BlockwiseCoreg) so that he sees this when he has the time.
He is likely not available for the rest of the month, so don't wait on it for merging 😉

@adebardo
Copy link

Thanks @rhugonnet, Yes, it’s temporary, just until we find a solution to avoid putting the CNES copyright on code it didn’t develop.

@adebardo adebardo merged commit cd86d69 into GlacioHack:main Dec 16, 2024
19 checks passed
@vschaffn vschaffn deleted the 604-nan_padding branch December 16, 2024 10:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[POC]: Nan padding for statistics when the blockwise coregistration fails on certain subdivisions
4 participants