The goal of this 3D prediction from single RGB image is to predict the 3D geometry and structure of objects from a single RGB image using deep learning techniques. This long standing ill-posed problem is crucial to numerous applications such as robot navigation, object recognition and scene understanding, medical diagnosis, and 3D modeling and animatio n. The advancement of deep learning techniques and the increasing availability of large 3D training data sets, have lead to a new generatio n of 3D shape inference methods that are able to predict the 3D geometry and structure of objects from a single view. In this work, we have experimented on different output representations, i.e. voxel, mesh, and point cloud, and two coordinate systems, commonly known as object-centered and viewer-centered. We have developed an end-to-end learning framework with Variational Autoencoder network, where the recognition network maps both the input image and silhouette, autogenerated using U-Net, to a latent representation and the generative network is expected to perform non-trivial reasoning about the 3D structure of the object, which has been tried on chair category of ShapeNet dataset and outperforms the state of the art.
Note: Since the paper is under process for publication details of the implemetation are not discused here.
