Offscreen rendering doesn't work on server & workarounds & fixes

[jimeiyang]

Hi Davis,

I found the offscreen rendering doesn't work with the required installation. My workaround is to 1) re-installing pyrender with mesa and 2) adding os.environ["PYOPENGL_PLATFORM"] = "osmesa" to viz_fitting_rgb.py.

Then there is a related issue. Even if it is set as "RGBA" by default, my renderer only generates the color_image with three channels, which leads to an error of computing the valid_mask here and then the composition here.

My fix in mesh_viewer.py is as follows:

input_img = self.cur_bg_img
if color_img.shape[2] == 4:
    output_img = (color_img[:, :, :-1] * color_img[:,:,3:] +(1.0 - color_img[:,:,3:])*input_img)
else:
    valid_mask = (depth_img > 0)[:, :, np.newaxis]
     output_img = (color_img * valid_mask + (1 - valid_mask) * input_img)

-Jimei

[AIML]

When use offscreen render mode in Pyrender with platform='osmesa', it will produce 3 channel image, which means the alpha channel is lossed. I don't konw why, but it is. Instead you can do like follow trickly:

# valid_mask = (color_img[:, :, -1] > 0)[:, :, np.newaxis]
valid_mask = (np.sum(color_img, axis=-1) > 0)[:, :, np.newaxis]