Problem about Custom Dataset

[xi8zhu]

Hi! Thank you for your fantastic work! And I have a question about how to apply the model to the custom dataset. How can I get the UV images and the static assets?

[una-dinosauria]

Hi!,

All the assets are part of the release. You should email [email protected] to get access to the dataset.

Cheers,

[xi8zhu]

Thank you for answering my quesion! We got the dataset goliath uses and want to train the model on our own datasets. However, our dataset only consists of images captured on different cameras under diffenert light patterns. We are confused about how to get the uv images and static assets in goliath datasets!
Could you give any suggestions? Thank you again!

[una-dinosauria]

Ah, for that you need some sort of registration of the canonical topology we provide wrt to your data -- then get average textures per frame by unwrapping the registration.

We don't have plans to release code to do those things in custom datasets at the moment.

[jryebread]

@una-dinosauria any future possibility of that code being released? strange to release code with no way to reproduce the results. Atleast some guide on how to get what is needed for training on a custom dataset.

[una-dinosauria]

@jryebread

@una-dinosauria any future possibility of that code being released?

Maybe in the future. Consider opening another issue with that request, so I can show this to the people in charge of those decisions.

strange to release code with no way to reproduce the results

Our results are reproducible with the data that we provide. We are not responsible for our code running on other people's data -- good data for avatar construction is hard and expensive to collect! This is not strange, but rather standard in academic settings. Thank your for your understanding.

Atleast some guide on how to get what is needed for training on a custom dataset.

Like I mentioned, you would need registration with respect to the canonical mesh that we provide. After that, texture unwrapping averaging all the cameras.

Non-rigid registration is a classical problem in computer vision that is studied separately from avatar reconstruction. Maybe see this recent paper from CVPR24 and references there to learn more about the state of the art in that area.

Cheers,

[xi8zhu]

Thank you once again for your impressive work and sincere responses.