Paper, Tags: #nlp
We show a method for training models that learn to ignore these problematic correlations. Our approach relies on the observation that models with limited capacity primarily learn to exploit biases in the dataset.
Our method doesn't require an explicit formulation of the dataset biases. We show how a model with limited capacity (weak learner) trained with a standard cross-entropy loss learns to exploit biases in the dataset. We then investigate the biases on which this weak learner relies and show that they match several previously manually identified biases.
We present an effective method for training models robust to dataset biases. We show that dataset biases don't need to be explicitly known or modeled to be able to train models that can generalize significantly better to out-of-distribution examples.