Training Your First LoRA (Local)

This guide assumes that you are already familiar with how to generate pictures with Stable Diffusion, and how LoRAs are used. If you have not generated pictures with Stable Diffusion yet, then it is too early for you to follow these instructions; come back later.

Before everything else:

• Ensure your graphics processing unit (GPU) is of Nvidia branding (if you don't know how, open Task Manager and go to your "Performance" tab), and has at least 6GB of VRAM. (You can check your GPU's VRAM spec through TechPowerUp's GPU database.)
• You should turn off sysmem fallback; if you use almost or all of your VRAM, you will encounter a very severe slowdown if you leave this enabled. Turning this off is not required, but strongly recommended. This step assumes your driver version is 546 or newer.
  • Open NVIDIA Control Panel
  • Go into "Manage 3D Settings"
  • Look for "CUDA - Sysmem Fallback Policy"
  • Choose "Prefer No Sysmem Fallback"
  • Click "Apply"
  • Close NVIDIA Control Panel

Kohya-ss installation:

IMPORTANT: THIS IS NOT AN EXTENSION FOR STABLE DIFFUSION. IT IS A SEPARATE PROGRAM. NEVER INSTALL THIS PROGRAM INSIDE OF YOUR STABLE DIFFUSION FOLDER, IT MAY BREAK IT.

• Follow install instructions:
  • Our method (includes automatic use-me.ps1 download)
  • Original method (fp8 base now works as of 16-05-2024)

Setting up use-me.ps1:

• Go into sd-scripts
• Make three new folders in sd-scripts: .dataset, .output, .class_img.
• If you used our method for installation of sd-scripts, look for use-me.ps1. If you didn't, then manually save use-me.ps1 into your sd-scripts folder.
• Inside of use-me.ps1 change:
  • $base_model_dir to your stable-diffusion-webui\models\Stable-diffusion folder. (or whatever folder you use for your checkpoints. This must not have the name of your checkpoint of choice, only the folder containing it)
  • $base_model to your checkpoint's filename (not your LoRA).
  • $clip_skip, $v_prediction, $sdxl to whatever your chosen checkpoint uses.
  • $lora_name to the name of the thing you want to train.
  • Any other settings you know how to change (if unsure, leave rest as default)

Preparing to train:

• Go into .dataset, make new folder of format 1_NAME, where NAME is the name of the concept being trained.
• Put dataset (images + captions) into your 1_NAME
  • If you do not have a dataset yet, please see "Preparing a Dataset".

Running training:

• Right click on use-me.ps1 and click "run in Powershell" to begin training.
• If your training performs one step, then it's working fine. If it runs an error, please share a screenshot of the error here.
• Once training finishes/error is shared, close Powershell window.