Description of wandb logs

[anuragkumar95]

It would be nice if there is some description of what's been sent to log on the wandb server. As far as I know the following are being logged:

1. feature_loss
2. loss
3. all_commitment_loss
4. adversarial_loss
5. recon_loss
6. stft
7. multi_spectral_recon_loss
8. disc_loss with different scales

Upon tallying these from the Soundstream original paper I could only find the disc_loss, feature_loss and recon_loss and multi_spectral_recon_loss description. Can someone explain what the scales mean for disc_loss? What is stft? and how should these loss look upon training? I have attached a screenshot of my wandb dashboard..

Why does it look like the losses are stuck? Any suggestion I can use to make this training better?

Here are my configs:

model = AudioLMSoundStream(
        codebook_size = 1024,
        rq_num_quantizers = 12,
        rq_groups = 2,                       
        use_lookup_free_quantizer = False,    
        use_finite_scalar_quantizer = False, 
        attn_window_size = 128,              
        attn_depth = 2                      
    )
    
    #Load checkpoint if resume_pt is present
    if args.resume_pt is not None:
        model.load(args.resume_pt)

    trainer = SoundStreamTrainer(
        model,
        folder = args.ds,
        save_model_every=1000,
        save_results_every=100,
        batch_size = 8,
        grad_accum_every = 4,         # effective batch size of 32
        data_max_length_seconds = 3,  # train on 3 second audio
        num_train_steps = 1_000_000,
        use_wandb_tracking = True,
        results_folder = args.save_dir
    ).cuda()

    with trainer.wandb_tracker(project="SoundStream"):
        trainer.train()