Joint training with classification and masked language modeling objectives

[saroyehun]

Not exactly an issue.

Please, can you advice on how to configure joint training of classification and masked language modeling objectives such that the stopping criterion will be when there is no improvement on the classification objective for N evaluation runs? Thank you!

[stefanik12]

Hi @ast123 ,

I've noticed that before we did not have any code demonstration of actually using the SequenceClassification objective, so I've added a test your case within the end2end_usecases_test.

In addition, to perform stopping after N (say, 5) evaluations on a first objective, pass in the following parameters to AdaptationArguments:

from adaptor.utils import AdaptationArguments, StoppingStrategy

args = AdaptationArguments(stopping_strategy=StoppingStrategy.FIRST_OBJECTIVE_CONVERGED,
                           stopping_patience=5)

Currently, Adaptor does not support stopping according to a specific objective, as we did not have this functional request before. This could be implemented relatively easily, with another StoppingStrategy and a corresponding logic in Schedule._should_stop().
However, I would be quite sure that any Classification objective will converge faster than MLM, so it is likely not necessary even in your case ;)

Please take a look at the test and feel free to ask if you need any more info!