Training with gradient checkpointing?

[benjamin-marie]

Is there a way to benchmark training with gradient checkpointing enabled?

I tried to pass
"gradient_checkpointing": True,
"gradient_checkpointing_kwargs": {"use_reentrant": True},

as training_arguments of TrainingConfig but it doesn't have any effect it seems.

[IlyasMoutawwakil]

Everything in training_arguments is passed directly to the TrainingArguments instance, which is passed to the trainer.

[benjamin-marie]

Thanks for your reply. This is what I was doing. I'll try again. If it still doesn't work, I'll share my full code.