Skip to content

Commit

Permalink
add validation for earl;y stopping patience
Browse files Browse the repository at this point in the history
  • Loading branch information
winglian committed Sep 8, 2023
1 parent e6d13b7 commit 0811514
Showing 1 changed file with 9 additions and 0 deletions.
9 changes: 9 additions & 0 deletions src/axolotl/utils/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,15 @@ def validate_config(cfg):
"sample_packing not compatible with xformers_attention. Use flash_attention"
)

if cfg.early_stopping_patience:
if not cfg.save_steps or not cfg.eval_steps:
raise ValueError(
"`early_stopping_patience` requires save_steps and eval_steps to be set. eval_steps should evenly divide save_steps."
)
if not cfg.save_steps % cfg.eval_steps != 0:
raise ValueError(
"`early_stopping_patience` requires that eval_steps should evenly divide save_steps."
)
# TODO
# MPT 7b
# https://github.com/facebookresearch/bitsandbytes/issues/25
Expand Down

0 comments on commit 0811514

Please sign in to comment.