Code

Validation loss #1864

Code

Merged

kohya-ss merged 78 commits intokohya-ss:val-losskohya-ss/sd-scripts:val-lossfrom rockerBOO:validation-loss-upstreamrockerBOO/sd-scripts:validation-loss-upstreamCopy head branch name to clipboardJan 26, 2025

+670-267

Commits

Commits on Nov 5, 2023

• Add validation loss

  rockerBOO

  committed
  5b19bda

  Copy full SHA for 5b19bda

• new ratio code

  rockerBOO

  committed
  33c311e

  Copy full SHA for 33c311e

• Add validation split of datasets

  rockerBOO

  committed
  3de9e6c

  Copy full SHA for 3de9e6c

• Update args to validation_seed and validation_split

  rockerBOO

  committed
  a93c524

  Copy full SHA for a93c524

• Add process_batch for train_network

  rockerBOO

  committed
  c892521

  Copy full SHA for c892521

• Removed/cleanup a line

  rockerBOO

  committed
  e545fdf

  Copy full SHA for e545fdf

• Remove unnecessary subset line from collate

  rockerBOO

  committed
  9c591bd

  Copy full SHA for 9c591bd

Commits on Nov 7, 2023

• Set grad enabled if is_train and train_text_encoder

  Show description for 569ca72
  rockerBOO

  committed
  569ca72

  Copy full SHA for 569ca72

Commits on Mar 9, 2024

• val

  gesen2egee

  committed
  b558a5b

  Copy full SHA for b558a5b

Commits on Mar 10, 2024

• improve

  gesen2egee

  committed
  78cfb01

  Copy full SHA for 78cfb01

• Update train_network.py

  gesen2egee

  committed
  923b761

  Copy full SHA for 923b761

• Update train_network.py

  gesen2egee

  committed
  47359b8

  Copy full SHA for 47359b8

Commits on Mar 11, 2024

• fix timesteps

  gesen2egee

  committed
  a51723c

  Copy full SHA for a51723c

• only use train subset to val

  gesen2egee

  committed
  7d84ac2

  Copy full SHA for 7d84ac2

• Update train_network.py

  gesen2egee

  committed
  befbec5

  Copy full SHA for befbec5

• Update train_network.py

  gesen2egee

  committed
  63e58f7

  Copy full SHA for 63e58f7

• Update train_network.py

  gesen2egee

  committed
  a6c41c6

  Copy full SHA for a6c41c6

Commits on Mar 13, 2024

• fix

  gesen2egee

  committed
  bd7e229

  Copy full SHA for bd7e229

• Merge remote-tracking branch 'kohya-ss/dev' into val

  gesen2egee

  committed
  5d7ed0d

  Copy full SHA for 5d7ed0d

• Update train_network.py

  gesen2egee

  committed
  d05965d

  Copy full SHA for d05965d

Commits on Mar 16, 2024

• fix control net

  gesen2egee

  committed
  b5e8045

  Copy full SHA for b5e8045

Commits on Apr 10, 2024

• Merge branch 'main' into val

  gesen2egee

  authored
  086f600

  Copy full SHA for 086f600

• Update config_util.py

  gesen2egee

  authored
  36d4023

  Copy full SHA for 36d4023

• Update train_util.py

  gesen2egee

  authored
  229c5a3

  Copy full SHA for 229c5a3

• Update config_util.py

  gesen2egee

  authored
  3b251b7

  Copy full SHA for 3b251b7

• Update config_util.py

  gesen2egee

  authored
  459b125

  Copy full SHA for 459b125

Commits on Apr 11, 2024

• Update train_util.py

  gesen2egee

  authored
  89ad69b

  Copy full SHA for 89ad69b

• Update config_util.py

  gesen2egee

  authored
  fde8026

  Copy full SHA for fde8026

Commits on Aug 2, 2024

• Remove unnecessary is_train changes and use apply_debiased_estimation to calculate validation loss. Balances the influence of different time steps on training performance (without affecting actual …

  Show description for 31507b9
  gesen2egee

  authored
  31507b9

  Copy full SHA for 31507b9

Commits on Aug 4, 2024

• Update train_db.py

  gesen2egee

  authored
  1db4951

  Copy full SHA for 1db4951

• Update train_db.py

  gesen2egee

  authored
  6816217

  Copy full SHA for 6816217

• Update train_db.py

  gesen2egee

  authored
  96eb74f

  Copy full SHA for 96eb74f

• Update train_network.py

  gesen2egee

  authored
  b9bdd10

  Copy full SHA for b9bdd10

• Update train_db.py

  gesen2egee

  authored
  3d68754

  Copy full SHA for 3d68754

• Update train_network.py

  gesen2egee

  authored
  a593e83

  Copy full SHA for a593e83

• Update train_network.py

  gesen2egee

  authored
  f6dbf7c

  Copy full SHA for f6dbf7c

• Update train_network.py

  gesen2egee

  authored
  aa850aa

  Copy full SHA for aa850aa

• Update train_network.py

  gesen2egee

  authored
  cdb2d9c

  Copy full SHA for cdb2d9c

Commits on Oct 4, 2024

• Update train_network.py

  gesen2egee

  committed
  3028027

  Copy full SHA for 3028027

• Update train_db.py

  gesen2egee

  committed
  dece2c3

  Copy full SHA for dece2c3

Commits on Dec 27, 2024

• Add Validation loss for LoRA training

  hinablue

  committed
  05bb918

  Copy full SHA for 05bb918

• Change val loss calculate method

  hinablue

  committed
  62164e5

  Copy full SHA for 62164e5

Commits on Dec 28, 2024

• Split val latents/batch and pick up val latents shape size which equal to training batch.

  hinablue

  committed
  64bd531

  Copy full SHA for 64bd531

• Change val latent loss compare

  hinablue

  committed
  cb89e02

  Copy full SHA for cb89e02

Commits on Jan 2, 2025

• val

  

  gesen2egee

  authored and

  rockerBOO

  committed
  8743532

  Copy full SHA for 8743532

• Adding modified train_util and config_util

  rockerBOO

  committed
  449c1c5

  Copy full SHA for 449c1c5

Commits on Jan 3, 2025

• Merge branch 'gesen2egee/val' into validation-loss-upstream

  Show description for 7f6e124
  rockerBOO

  committed
  7f6e124

  Copy full SHA for 7f6e124

• Merge remote-tracking branch 'hina/feature/val-loss' into validation-loss-upstream

  Show description for d23c732
  rockerBOO

  committed
  d23c732

  Copy full SHA for d23c732

• Remove defunct code for train_controlnet.py

  rockerBOO

  committed
  7470173

  Copy full SHA for 7470173

• Typos and lingering is_train

  rockerBOO

  committed
  534059d

  Copy full SHA for 534059d

• Cleanup order, types, print to logger

  rockerBOO

  committed
  c8c3569

  Copy full SHA for c8c3569

• Update text for train/reg with repeats

  rockerBOO

  committed
  fbfc275

  Copy full SHA for fbfc275

• Add seed help clarifying info

  rockerBOO

  committed
  58bfa36

  Copy full SHA for 58bfa36

• Remove duplicate assignment

  rockerBOO

  committed
  6604b36

  Copy full SHA for 6604b36

• Fix training, validation split, revert to using upstream implemenation

  rockerBOO

  committed
  0522070

  Copy full SHA for 0522070

• Move get_huber_threshold_if_needed

  rockerBOO

  committed
  695f389

  Copy full SHA for 695f389

• Add step break for validation epoch. Remove unused variable

  rockerBOO

  committed
  1f9ba40

  Copy full SHA for 1f9ba40

• Add missing functions for training batch

  rockerBOO

  committed
  1c0ae30

  Copy full SHA for 1c0ae30

Commits on Jan 6, 2025

• Use self.get_noise_pred_and_target and drop fixed timesteps

  rockerBOO

  committed
  bbf6bbd

  Copy full SHA for bbf6bbd

• Revert train_db.py

  rockerBOO

  committed
  f4840ef

  Copy full SHA for f4840ef

• Cleanup unused code and formatting

  rockerBOO

  committed
  1c63e7c

  Copy full SHA for 1c63e7c

• Add validate_every_n_epochs, change name validate_every_n_steps

  rockerBOO

  committed
  c64d1a2

  Copy full SHA for c64d1a2

• Fix validate epoch, cleanup imports

  rockerBOO

  committed
  f885029

  Copy full SHA for f885029

• Clean up some validation help documentation

  rockerBOO

  committed
  fcb2ff0

  Copy full SHA for fcb2ff0

• Set validation steps in multiple lines for readability

  rockerBOO

  committed
  742bee9

  Copy full SHA for 742bee9

Commits on Jan 8, 2025

• Remove unused train_util code, fix accelerate.log for wandb, add init_trackers library code

  rockerBOO

  committed
  1231f51

  Copy full SHA for 1231f51

• Fix documentation, remove unused function, fix bucket reso for sd1.5, fix multiple datasets

  rockerBOO

  committed
  556f3f1

  Copy full SHA for 556f3f1

• Handle tuple return from generate_dataset_group_by_blueprint

  rockerBOO

  committed
  9fde0d7

  Copy full SHA for 9fde0d7

• Revert bucket_reso_steps to correct 64

  rockerBOO

  committed
  1e61392

  Copy full SHA for 1e61392

• Fix incorrect destructoring for load_abritrary_dataset

  rockerBOO

  committed
  d6f158d

  Copy full SHA for d6f158d

Commits on Jan 9, 2025

• Apply is_training_dataset only to DreamBoothDataset. Add validation_split check and warning

  rockerBOO

  committed
  264167f

  Copy full SHA for 264167f

Commits on Jan 12, 2025

• Add divergence to logs

  Show description for 4c61adc
  rockerBOO

  committed
  4c61adc

  Copy full SHA for 4c61adc

• Fix regularization images with validation

  Show description for 2bbb40c
  rockerBOO

  committed
  2bbb40c

  Copy full SHA for 2bbb40c

• Fix validate_every_n_steps always running first step

  rockerBOO

  committed
  0456858

  Copy full SHA for 0456858

• Fix validate_every_n_steps for gradient accumulation

  rockerBOO

  committed
  ee9265c

  Copy full SHA for ee9265c

• Remove Validating... print to fix output layout

  rockerBOO

  committed
  25929dd

  Copy full SHA for 25929dd

• Disable repeats for validation datasets

  rockerBOO

  committed
  b489082

  Copy full SHA for b489082

Commits on Jan 23, 2025

• Fix loss recorder on 0. Fix validation for cached runs. Assert on validation dataset

  rockerBOO

  committed
  c04e5df

  Copy full SHA for c04e5df