save configuration to wandb

[peterdudfield]

Pull Request

Description

• save configruation,
• hydra and wandb should save it, but there is an issue with _target_ dicts, and tricky to find out where the error was. Easier to save the file directly

#168

How Has This Been Tested?

Ran it locally

• Yes

Checklist:

• My code follows OCF's coding style guidelines
• I have performed a self-review of my own code
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works
• I have checked my code and corrected any misspellings

[dfulu]

Could we not just upload f"{callback.dirpath}/model_config.yaml" and f"{config.datamodule.batch_dir}/data_configuration.yaml" since these already exist rather than saving them to ./configuration first?

[peterdudfield]

The file structure in wandb matches the file strcutre of the files uploaded, and I wanted it neat, rather than some mnt/storage4tb_b/..../ for the data configuration. But perhaps there is a neater way to do it?

[dfulu]

I tried running this:

logger.experiment.save("/home/jamesfulton/repos/PVNet_summation/checkpoints/znoi18se/model_config.yaml")

and the file appears here without any subdirectories. So that suggests logger.experiment.save(f"{callback.dirpath}/model_config.yaml") should work and save without subdirs?

But maybe I'm missing something, and if that doesn't work can we just delete ./configuration after uploading to wandb to keep things tidy?

[peterdudfield]

hmm, yea, ok I could try that.
How do you feel about override the current config.yaml in wandb with a new one? My feeling is that we should save both

[dfulu]

Yeh I'd save both

[peterdudfield]

It looks like I have to use wandb_logger.experiment.save(f"{callback.dirpath}/data_config.yaml", callback.dirpath) to make sure its not in a sub folder. Im on a slightly high wandb version that you @dfulu

[peterdudfield]

For example https://wandb.ai/openclimatefix/wandb_test/runs/b98z16i9/files?nw=nwuserpeterdudfield