NaN values in the data

[Baboom-l]

Hello,

I see that Nan exists in the data in the FAQ of the website of the dataset. Do I want to know whether the Nan value exists in disparity or input image?How do you handle these error values in the dataset?

[nikolausmayer]

Hi, the NaN data elements exist in disparity and optical flow images. They do not exist in the input RGB images.
We filter out NaN pixels by excluding them from the error calculation and backpropagation: NaN pixels simply have no effect on the training process.

[whuwuteng]

Hello, I am also curious about the processing of NaN dataset.
Because I have a dataset, the LiDAR is sparse, so there are a lot of NaN pixels in the disparity dataset.
During the training, I found that the flow_loss3 flow_loss4 flow_loss5 flow_loss6 are "-nan".
From the issue, the early step only uses flow_loss5 and flow_loss6, so I think this means it is impossible to train.

In order to know the reason, I also randomly select the pixels(259200 from 960*540) from the disparity ground truth to make a sparse dataset from the sceneflow dataset.
An example is shown in the next. The original disparity image :

The randomly selected disparity image :

The same problem occurs, the training output is :

I use the DispNetCorr1D model to train the dataset.
So do you have any idea about this problem, thank you very much.
@Baboom-l and @nikolausmayer

[Baboom-l]

I found that PSMNet is avoiding the effect of Nan values in the data by mask when calculating the loss @whuwuteng

[whuwuteng]

Thank you for your reply @Baboom-l .
Yes, I know that the PSMnet has a mask for the Nan dataset, because most works give an evaluation on KITTI dataset, but @nikolausmayer said that they can handle the sparse disparity dataset(for example KITTI), refer to the issue.

[Baboom-l]

I went to check this out and according to nikolausmayer's reply it should simply ignore the pixels, and the 77QIQI description probably refers to replacing Nan's place with the value of the maximum disparity in GT or in cost volume, but I'm not sure about that. @whuwuteng

[whuwuteng]

Yes, I finally understood the issue when converting my dataset into the DispNet dataset, they use the maximum for the Nan value, so I do the same thing when randomly selecting the pixels.
But I do not know the difference when handling from loss1 to loss 6 if there is Nan value.

Anyway, thank you very much for your reply @Baboom-l .