Validation Loss Increasing

[JiangWenPL]

Hi authors, thank you for your great work.

I tried to train your network from scratch with ./configs/img/onet.yaml. I find something weird that the validation loss is increasing during training though the validation IOU is getting better. I also evaluated the model and the result looks fine(0.584 mean IOU). I am wondering if the validation loss would be normal.

Here is the screen shot for the loss curves.

[fangchuan]

@JiangWenPL Hi, have you solved the problem? I found same problems when i finetune the pretrained model on iGibson dataset, the training loss is declining while the validation loss is rising, though the validation IOU seems fine. At begining, I thought the network was overfitting, then I stop the iteration as early as possible, but it did little a help the evaluation result in test dataset.