Monitor Training

Using WandB

Add wandb.init() beginning of your training script as well as your evaluation script, and each piece would be tracked as a run in W&B. wandb.init() returns a run object, and you can also access the run object via wandb.run.

run = (
    wandb.init(
        project=project,
        name=run_name,
        config=config,
        dir=output_dir,
        resume=resume,
        **kwargs,
    )
    if not wandb.run
    else wandb.run
)

# you can then log your stats by passing dict
run.log(log_dict)

wandb.init() takes arguments including:

Arguments
project	(str, optional) The name of the project where you're sending the new run. If the project is not specified, the run is put in an "Uncategorized" project.
name	(str, optional) A short display name for this run, which is how you'll identify this run in the UI. By default, we generate a random two-word name that lets you easily cross-reference runs from the table to charts. Keeping these run names short makes the chart legends and tables easier to read. If you're looking for a place to save your hyperparameters, we recommend saving those in config.
group	(str, optional) Specify a group to organize individual runs into a larger experiment. For example, you might be doing cross validation, or you might have multiple jobs that train and evaluate a model against different test sets. Group gives you a way to organize runs together into a larger whole, and you can toggle this on and off in the UI. For more details, see our guide to grouping runs.
job_type	(str, optional) Specify the type of run, which is useful when you're grouping runs together into larger experiments using group. For example, you might have multiple jobs in a group, with job types like train and eval. Setting this makes it easy to filter and group similar runs together in the UI so you can compare apples to apples.
tags	(list, optional) A list of strings, which will populate the list of tags on this run in the UI. Tags are useful for organizing runs together, or applying temporary labels like "baseline" or "production". It's easy to add and remove tags in the UI, or filter down to just runs with a specific tag.
id	(str, optional) A unique ID for this run, used for resuming. It must be unique in the project, and if you delete a run you can't reuse the ID. Use the name field for a short descriptive name, or config for saving hyperparameters to compare across runs. The ID cannot contain the following special characters: /#?%:. See our guide to resuming runs.

Reference

1. https://docs.wandb.ai/ref/python/init