Quality tradeoff between dmtet 128 and 64

[iraj465]

Hi,

@jmunkberg i tried with different quartet configs as you suggested and 64 and 128 works best for most use-cases i found on different aspects. I also tried adjusting the absolute laplace scale but it wasn't giving a good tradeoff.

The dmtet 64 gives immaculate geometry but obviously the texture is a bit washed out as you can see below.

Whereas the dmtet 128 gives good texture maps but not good geometry (jagged), see below:

3d8-7169-465c-b238-530da07bc269.png">

Now it is not possible to use geometry of 64 with textures of 128 due to uv mapping. Is there any workaround to use the 64 level geometry and 128 dmtet level texture maps? Or does the e2e training process needs to be adapted somewhat?

Any suggestions is appreciated!

[jmunkberg]

Texture resolution (config flag:"texture_res": [ 2048, 2048 ],) and mesh tessellation rate (config flag: "dmtet_grid" : 128,) are mostly decoupled.
That said, we use xatlas to UV-map our extracted mesh after the first pass, and with higher tessellation rates, sometimes it is harder to generate a nice UV-parametrization, and there may be more small islands.

Also, it is sometimes harder to optimize the shape for higher tessellation rates, particularly if you have a small number of camera views,

[iraj465]

Would the uv mapping change somewhat is we use the dmtet 64 as base mesh and run again with dmtet 128 and turning off the geometry optimization (namely, lock_pos)

[jmunkberg]

lock pos is only applied in the second pass, after UV mapping is performed. In the first pass, we use DMTet with a grid resolution specified with the dmtet_grid flag. The UV mapping is only a result of the output from the first pass (including the dmtet resolution), and is unaffected by lock_pos.

It is only in the second pass that we use 2D textures, so the "texture_res": [ 2048, 2048 ], config flag only affect the second pass.

[iraj465]

Got it, so since uv mapping is applied after the first pass this would be a problem. Let me look into how this can be solved

@jmunkberg any tips or workaround that can make the overall quality better?

[iraj465]

@jmunkberg is it possible to only improve the texture after training a model for both the passes?

[jmunkberg]

If the geometry and lighting are both good after the first two passes, you should be able to continue finetuning only the textures using the flags

"base_mesh": "path to your geometry from the first or second pass",
"learn_light" : false, // do not optimize lighting
"lock_pos" : true, // to not optimize geometry
"envmap": "path to your light probe from the first or second pass",

However, in my experience, it is often coupled. Bad geometry makes material optimization much harder, so if the geometry is good, the materials are often nice as well in the joint optimization. Trying to fine-tune materials on locked bad geometry will not work great.