From raw to real depth values

[salykova]

Hello INGP developers and INGP community!

First of all, I would like to thank you for sharing the code.

May I ask you, is it possible to obtain real depth values (in mm) from raw depths? From previous issues and discussions I understand, that raw depth values are expressed in scene units. But what is transformation between scene and real coordinates? I tried to multiply raw depth values by a factor of scene_scale = 0.33 (I also tried 1/0.33), but it seems to be wrong.

Should I use aabb_scale parameter to obtain real depth values?

[Tom94]

Hi there, aabb_scale does not influence the depth values. You want to divide by NERF_SCALE, which will then return depth values in the units that your transforms were in. testbed.render() already returns depth values in the units that your transforms were in. At that point, obtaining millimeters is a question of understanding where the transforms came from -- in the case of colmap, the scale is arbitrary if the cameras weren't calibrated beforehand.

[salykova]

Hi Thomas! Thank you for your answer! I noticed, that translation part of COLMAP poses is divided by average of camera distances from origin.

instant-ngp/scripts/colmap2nerf.py

Line 316 in 3fa15bf

f["transform_matrix"][0:3,3] *= 4.0 / avglen # scale to "nerf sized"

I already have calibrated extrinsics, then I applied colmap2nerf.py on my extrinsics, to transform them to your format. I think if I multiply depth values by (average of camera distances from origin)/4 and by a factor of 1/0.33, I will get true depth values in mm. I will test it. Thank you!

[salykova]

Hi Thomas!
I figured out how to get real depth values. Thank you again for your reply!

By the way, one doesn't need to divide by NERF_SCALE. python binding testbed.render() returns depth values already in "real" units (nerf was loaded from a checkpoint .msgpack)

[Tom94]

Glad to hear & thank you for the clarification! I edited my answer above with the correct info to avoid misleading others.