Enhanced_Stable_Diffusion_k_lms.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Skquark/structured-prompt-generator/blob/main/Enhanced_Stable_Diffusion_k_lms.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "cdyweXLq09gU"
      },
      "source": [
        "# [STABLE-K-4D](https://github.com/DamascusGit/stable-diffusion) - **Enhanced Stable Diffusion**\n",
        "\n",
        "Jupyter/Colab implementation of [Stable Diffusion](https://github.com/CompVis/stable-diffusion) and Katherine Crowson's [k-diffusion wrapper](https://github.com/crowsonkb/k-diffusion) in order to use the k_lms sampler  by [karan4d](https://twitter.com/karan4d) aka [DamascusGit](https://github.com/DamascusGit). The notebook utilizes CPU draw manual seeding with k_lms and has a weird quantize.py adjustment in an attempt to best replicate the results from Simulacra/Dreambot using the stable-diffusion weights.\n",
        "\n",
        "implemented with the help of [Katherine Crowson](https://https://twitter.com/rivershavewings), palp, wbrown, and litexev. shoutout SimulacraBot/DreamBot and their team, as well as the good folks over at EleutherAI + StabilityAI.\n",
        "\n",
        "---\n",
        "Mods and Flow Enhancements by [**Skquark**](https://www.Skquark.com)\n",
        "\n",
        "You can use the updated implementation of [**Enhanced Stable Diffusion**](https://colab.research.google.com/github/Skquark/structured-prompt-generator/blob/main/Enhanced_Stable_Diffusion_with_diffusers.ipynb) instead..\n",
        "\n",
        "Try our other useful notebooks [Structured Prompt Generator](https://colab.research.google.com/github/Skquark/structured-prompt-generator/blob/main/Structured_Prompt_Generator.ipynb) and [Enhanced DiscoArt](https://colab.research.google.com/github/Skquark/structured-prompt-generator/blob/main/DiscoArt_%5B_w_Batch_Prompts_%26_GPT_3_Generator%5D.ipynb)"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#@title Check GPU Status (A100 > G100 > V100 > P100 > T4 > K8)\n",
        "import subprocess\n",
        "simple_nvidia_smi_display = True#@param {type:\"boolean\"}\n",
        "if simple_nvidia_smi_display:\n",
        "    #!nvidia-smi\n",
        "    nvidiasmi_output = subprocess.run(['nvidia-smi', '-L'], stdout=subprocess.PIPE).stdout.decode('utf-8')\n",
        "    print(nvidiasmi_output)\n",
        "else:\n",
        "    #!nvidia-smi -i 0 -e 0\n",
        "    nvidiasmi_output = subprocess.run(['nvidia-smi'], stdout=subprocess.PIPE).stdout.decode('utf-8')\n",
        "    print(nvidiasmi_output)\n",
        "    nvidiasmi_ecc_note = subprocess.run(['nvidia-smi', '-i', '0', '-e', '0'], stdout=subprocess.PIPE).stdout.decode('utf-8')\n",
        "    print(nvidiasmi_ecc_note)"
      ],
      "metadata": {
        "cellView": "form",
        "id": "S4-gh_HSbAlS"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#@title ## Mount your Google Drive (Colab only)\n",
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EMZ1nT2B1nG1",
        "outputId": "b91feda4-2b14-4a60-ddba-5926458ba1c2",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "\n",
        "#@title ## Choose Installation Options\n",
        "#@markdown You can save the project to your Google Drive, which takes about 8GB, for faster startup. Also saves your image outputs to your GDrive path (or elsewhere) for convienience. If you already have sd-v1-3-full-ema.ckpt include path to file.\n",
        "install_type = \"Google Colab\" #@param ['Google Colab', 'Jupyter Notebook']\n",
        "copy_to_GDrive = False #@param {'type': 'boolean'}\n",
        "image_output = '/content/drive/MyDrive/AI/Stable_Diffusion/images_out' #@param {'type': 'string'}\n",
        "download_sd_model = True #@param {'type': 'boolean'}\n",
        "download_HuggingFace_model  = False #@param {'type': 'boolean'}\n",
        "HuggingFace_api_key = '' #@param {'type': 'string'}\n",
        "#save_models_to_google_drive = True #@param {'type': 'boolean'}\n",
        "ckpt_file = '' #@param {'type': 'string'}\n",
        "#ckpt_file = '/content/drive/MyDrive/StableRun/stable-diffusion/models/ldm/stable-diffusion-v1/model.ckpt'\n",
        "root_dir = '/content/'\n",
        "if install_type is 'Google Colab':\n",
        "  root_dir = '/content/'\n",
        "elif install_type is 'Jupyter Notebook':\n",
        "  root_dir = '/workspace/'\n",
        "stable_dir = root_dir\n",
        "if copy_to_GDrive:\n",
        "  stable_dir += 'drive/MyDrive/AI/'\n",
        "stable_dir += 'Stable_Diffusion'\n",
        "if not os.path.exists(stable_dir):\n",
        "  os.makedirs(stable_dir)\n",
        "if not os.path.exists(image_output):\n",
        "  os.makedirs(image_output)\n",
        "os.chdir(stable_dir)"
      ],
      "metadata": {
        "id": "ZDTBgn_eJiAu",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vBAxxY1rG-vH",
        "outputId": "1c468f1e-9665-4e9f-85b5-82a21af5e2f6",
        "cellView": "form"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "⏬ Downloading https://github.com/jaimergp/miniforge/releases/latest/download/Mambaforge-colab-Linux-x86_64.sh...\n",
            "📦 Installing...\n",
            "📌 Adjusting configuration...\n",
            "🩹 Patching environment...\n",
            "⏲ Done in 0:00:29\n",
            "🔁 Restarting kernel...\n",
            "conda 4.9.2\n",
            "condacolab_install.log\n"
          ]
        }
      ],
      "source": [
        "#@title ## Install Conda Colab Libraries\n",
        "!pip install -q condacolab\n",
        "import condacolab\n",
        "condacolab.install()\n",
        "\n",
        "!conda --version\n",
        "#If !conda --version returns no results, install conda with :\n",
        "#!pip install -q condacolab\n",
        "#import condacolab\n",
        "#condacolab.install()\n",
        "\n",
        "!ls"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "#@title # First-time Setup and SD Model Download\n",
        "os.chdir(stable_dir)\n",
        "!git clone https://github.com/DamascusGit/stable-diffusion\n",
        "import os\n",
        "import shutil\n",
        "\n",
        "model_ckpt = stable_dir + '/stable-diffusion/models/ldm/stable-diffusion-v1/model.ckpt'\n",
        "if not os.path.exists(f'{stable_dir}/stable-diffusion/models/ldm/stable-diffusion-v1/'):\n",
        "  os.mkdir(f'{stable_dir}/stable-diffusion/models/ldm/stable-diffusion-v1/')\n",
        "if download_sd_model:\n",
        "  if not os.path.exists(model_ckpt):\n",
        "    !wget -q --show-progress --no-cache --backups=1 'https://drinkordiecdn.lol/sd-v1-3-full-ema.ckpt'\n",
        "    shutil.move(stable_dir + '/sd-v1-3-full-ema.ckpt', model_ckpt)\n",
        "else:\n",
        "  if bool(ckpt_file):\n",
        "    if os.path.exists(ckpt_file):\n",
        "      if not os.path.exists(model_ckpt):\n",
        "        shutil.copy(ckpt_file, model_ckpt)\n",
        "    else: print(\"Couldn't find your sd checkpoint model\")\n",
        "  elif not os.path.exists(model_ckpt):\n",
        "    print(\"Put in the path to your sd-v1-3-full-ema.ckpt file, or click download leaked model\")\n",
        "#!ln -s /content/sd-v1-3-full-ema.ckpt models/ldm/stable-diffusion-v1/model.ckpt\n",
        "!ls -l {stable_dir}/stable-diffusion/models/ldm/stable-diffusion-v1\n",
        "os.chdir(f'{stable_dir}/stable-diffusion')"
      ],
      "metadata": {
        "id": "8DVqmcouybOd",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "268jN0yHYt2x",
        "cellView": "form"
      },
      "outputs": [],
      "source": [
        "#@title # Redirect to /stable-diffusion path if do a Restart Runtime\n",
        "os.chdir(f'{stable_dir}/stable-diffusion')\n",
        "# use this for Jupyter\n",
        "#%cd /workspace/StableRun/stable-diffusion\n",
        "#%cd /content/StableRun/stable-diffusion\n",
        "# use this for Colab\n",
        "#%cd /content/drive/MyDrive/StableRun/stable-diffusion"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "E_UAtsmbEQ5X",
        "cellView": "form"
      },
      "outputs": [],
      "source": [
        "#@title # Install Environment and Dependency Code\n",
        "os.chdir(f'{stable_dir}/stable-diffusion')\n",
        "!conda env create -f environment.yaml\n",
        "!conda activate ldm\n",
        "\n",
        "!conda install pytorch torchvision -c pytorch -y\n",
        "!pip install transformers==4.19.2\n",
        "!pip install -e .\n",
        "!conda install omegaconf\n",
        "!pip install omegaconf\n",
        "!pip install einops\n",
        "!pip install pytorch_lightning\n",
        "#!pip install git+https://github.com/Compvis/latent-diffusion.git\n",
        "#!conda install pytorch==1.9.0 torchvision==0.10.1 -c pytorch -c conda-forge\n",
        "#!pip install ldm\n",
        "#!pip install git+https://github.com/CompVis/latent-diffusion/tree/main/models/ldm.git\n",
        "#!pip install dlib\n",
        "#!pip install pillow==4.0.0\n",
        "!pip install scikit-image\n",
        "!conda install Pillow==6.1 #USE THIS WITH COLAB\n",
        "!pip install opencv-python\n",
        "!pip install ai-tools\n",
        "!pip install cognitive-face\n",
        "!pip install zprint\n",
        "#!pip install setuptools==59.5.0\n",
        "!pip install -e git+https://github.com/CompVis/taming-transformers.git@master#egg=taming-transformers\n",
        "!pip install git+https://github.com/openai/CLIP.git\n",
        "!pip install kornia"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "dL2D5mld03yI",
        "cellView": "form"
      },
      "outputs": [],
      "source": [
        "#@title ## Install k-diffusion requirements\n",
        "#@markdown Run to make sure you install requirements appropriate for Jupyter or Colab.\n",
        "%pip install lpips\n",
        "%pip install keras\n",
        "%pip install git+https://github.com/crowsonkb/k-diffusion/\n",
        "# Can't find requirements file.. Is it mistake?\n",
        "#%pip install -r /content/drive/MyDrive/requirements.txt #IF USING COLAB\n",
        "#%pip install -r /workspace/StableRun/requirements.txt #IF USING JUPYTR"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#@markdown ### To use **Real-ESRGAN AI Upscaling**, run this first\n",
        "# # Clone Real-ESRGAN and enter the Real-ESRGAN\n",
        "%cd {root_dir}\n",
        "!git clone https://github.com/xinntao/Real-ESRGAN.git\n",
        "%cd Real-ESRGAN\n",
        "# Set up the environment\n",
        "!pip install basicsr\n",
        "!pip install facexlib\n",
        "!pip install gfpgan\n",
        "!pip install -r requirements.txt\n",
        "!python setup.py develop\n",
        "# Download the pre-trained model\n",
        "!wget https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth -P experiments/pretrained_models\n",
        "%cd .."
      ],
      "metadata": {
        "cellView": "form",
        "id": "2mk8FXMLT05o"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "y_6MvdU4AMyS",
        "tags": []
      },
      "source": [
        "## Parameter Cheatsheet"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "```\n",
        "usage: txt2img.py [-h] [--prompt [PROMPT]] [--outdir [OUTDIR]] [--skip_grid] [--skip_save] [--ddim_steps DDIM_STEPS] [--plms] [--laion400m] [--fixed_code] [--ddim_eta DDIM_ETA] [--n_iter N_ITER] [--H H] [--W W] [--C C] [--f F] [--n_samples N_SAMPLES] [--n_rows N_ROWS]\n",
        "                   [--scale SCALE] [--from-file FROM_FILE] [--config CONFIG] [--ckpt CKPT] [--seed SEED] [--precision {full,autocast}]\n",
        "\n",
        " optional arguments:\n",
        "   -h, --help            show this help message and exit\n",
        "   --prompt [PROMPT]     the prompt to render\n",
        "   --outdir [OUTDIR]     dir to write results to\n",
        "   --skip_grid           do not save a grid, only individual samples. Helpful when evaluating lots of samples\n",
        "   --skip_save           do not save individual samples. For speed measurements.\n",
        "   --ddim_steps DDIM_STEPS\n",
        "                         number of ddim sampling steps\n",
        "   --plms                use plms sampling\n",
        "   --laion400m           uses the LAION400M model\n",
        "   --fixed_code          if enabled, uses the same starting code across samples\n",
        "   --ddim_eta DDIM_ETA   ddim eta (eta=0.0 corresponds to deterministic sampling\n",
        "   --n_iter N_ITER       sample this often\n",
        "   --H H                 image height, in pixel space (multiple of 64)\n",
        "   --W W                 image width, in pixel space (multiple of 64)\n",
        "   --C C                 latent channels\n",
        "   --f F                 downsampling factor\n",
        "   --n_samples N_SAMPLES\n",
        "                         how many samples to produce for each given prompt. A.k.a. batch size\n",
        "   --n_rows N_ROWS       rows in the grid (default: n_samples)\n",
        "   --scale SCALE         unconditional guidance scale: eps = eps(x, empty) + scale * (eps(x, cond) - eps(x, empty))\n",
        "   --from-file FROM_FILE\n",
        "                         if specified, load prompts from this file\n",
        "   --config CONFIG       path to config which constructs model\n",
        "   --ckpt CKPT           path to checkpoint of model\n",
        "   --seed SEED           the seed (for reproducible sampling)\n",
        "   --precision           {full,autocast} evaluate at this precision\n"
      ],
      "metadata": {
        "id": "i6K6d_9xjMis"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "---\n",
        "# Set your Stability.ai Parameters"
      ],
      "metadata": {
        "id": "l4Ks5KGnSOEg"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#prompt = \"\" #@param {'type': 'string'}\n",
        "n_samples = 1 #@param {'type': 'integer'}\n",
        "n_iterations = 1 #@param {'type': 'integer'}\n",
        "ddim_steps  = 50 #@param {'type': 'integer'}\n",
        "ddim_eta = 0.0 #@param {'type': 'number'}\n",
        "scale = 7 #@param {'type': 'integer'}\n",
        "seed = 22276 #@param {'type': 'integer'}\n",
        "width = 512 #@param {'type': 'integer'}\n",
        "height = 512 #@param {'type': 'integer'}\n",
        "lms_sampling = True #@param {'type': 'boolean'}\n",
        "plms_sampling = False #@param {'type': 'boolean'}\n",
        "init_image = None #@param {'type': 'string'}\n",
        "init_image_strength = 0.4 #@param {'type': 'number'}\n",
        "#@markdown ---\n",
        "#@markdown If you want to apply upscaling, initiallize ESRGAN library first\n",
        "apply_ESRGAN_upscale = False #@param {type:\"boolean\"}\n",
        "enlarge_scale = 2 #@param {type:'slider', min:1, max:4, step:0.5}\n",
        "#@markdown ---\n",
        "file_allowSpace = False #@param {type:\"boolean\"}\n",
        "file_prefix = 'sd-' #@param {type:\"string\"}\n",
        "save_image_metadata = False #@param {type:\"boolean\"}\n",
        "meta_ArtistName = \"Alan Bedian\" #@param {type:\"string\"}\n",
        "meta_Copyright = \"Skquark, Inc.\" #@param {type:\"string\"}\n",
        "params = ''\n",
        "if init_image:\n",
        "  params += ' --init_img ' + init_image + ' --strength ' + init_image_strength\n",
        "params += f' --n_samples {n_samples}'\n",
        "params += f' --n_iter {n_iterations}'\n",
        "params += f' --ddim_steps {ddim_steps}'\n",
        "params += f' --W {width} --H {height}'\n",
        "params += f' --scale {scale}'\n",
        "params += f' --seed {seed}'\n",
        "if ddim_eta != 0.0:\n",
        "  params += f' --ddim_eta {ddim_eta}'\n",
        "if plms_sampling:\n",
        "  params += ' --plms'\n",
        "print('Params:' + params)"
      ],
      "metadata": {
        "cellView": "form",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2iEgHUBWXTMG",
        "outputId": "ed5c8576-b22d-4874-8b1a-d83ff27455aa"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Params: --n_samples 1 --n_iter 1 --ddim_steps 50 --W 512 --H 512 --scale 7 --seed 22276\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Put one or many text prompts in the list, then run:"
      ],
      "metadata": {
        "id": "V4AsOjrylAaE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "prompts = [\n",
        "    \"Intergalactic stargate portal in the middle of a forest, with animals and birds all around it\",\n",
        "    \"Intergalactic stargate portal in the middle of a city, with people all around it by Frederic Church and Larry Elmore, style of stonepunk\",\n",
        "]"
      ],
      "metadata": {
        "id": "1o6hqohzgN7M"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#@title ## **> Run Stable Diffusion** on prompts\n",
        "import string, shutil, random, gc\n",
        "from collections import ChainMap\n",
        "from PIL import Image\n",
        "from PIL.PngImagePlugin import PngInfo\n",
        "#import torch\n",
        "\n",
        "os.chdir(f'{stable_dir}/stable-diffusion')\n",
        "def format_filename(s):\n",
        "    valid_chars = \"-_.() %s%s\" % (string.ascii_letters, string.digits)\n",
        "    filename = ''.join(c for c in s if c in valid_chars)\n",
        "    if file_allowSpace: filename = filename.replace(' ','_')\n",
        "    return filename[:248]\n",
        "\n",
        "for p in prompts:\n",
        "  try:\n",
        "    if lms_sampling:\n",
        "      !python scripts/txt2img_k.py --prompt \"{p}\"{params}\n",
        "    else: # Ignore Invalid character error, still works, their bug\n",
        "      !python scripts/txt2img.py --prompt \"{p}\"{params}\n",
        "  except RuntimeError as e:\n",
        "    if 'out of memory' in str(e):\n",
        "      print(f\"\\u001b[31m\\u001b[1m\\u001b[4mCRITICAL ERROR\\u001b[0m: GPU ran out of memory! Flushing memory to save session...\")\n",
        "      pass\n",
        "  finally:\n",
        "    gc.collect()\n",
        "    #torch.cuda.empty_cache()\n",
        "  txt2img_output = f'{stable_dir}/stable-diffusion/outputs/txt2img-samples'\n",
        "  if not os.path.exists(txt2img_output):\n",
        "    os.makedirs(txt2img_output)\n",
        "    print(\"Couldn't find output folder \" + txt2img_output)\n",
        "  else:\n",
        "    filenames = os.listdir(txt2img_output)\n",
        "    filename = format_filename(p)\n",
        "    idx = 0\n",
        "    for fname in filenames:\n",
        "      if fname.endswith('png'):\n",
        "        fpath = os.path.join(txt2img_output, fname)\n",
        "        if apply_ESRGAN_upscale:\n",
        "          os.chdir(f'{root_dir}Real-ESRGAN')\n",
        "          upload_folder = 'upload'\n",
        "          result_folder = 'results'     \n",
        "          if os.path.isdir(upload_folder):\n",
        "              shutil.rmtree(upload_folder)\n",
        "          if os.path.isdir(result_folder):\n",
        "              shutil.rmtree(result_folder)\n",
        "          os.mkdir(upload_folder)\n",
        "          os.mkdir(result_folder)\n",
        "          dst_path = os.path.join(f'{root_dir}Real-ESRGAN/{upload_folder}', fname)\n",
        "          #print(f'Moving {fpath} to {dst_path}')\n",
        "          shutil.move(fpath, dst_path)\n",
        "          # Ignore Invalid character error, still works, their bug\n",
        "          !python inference_realesrgan.py -n RealESRGAN_x4plus -i upload --outscale {enlarge_scale}\n",
        "          out_file = fname.rpartition('.')[0] + '_out.png'\n",
        "          #print(f'move {root_dir}Real-ESRGAN/{result_folder}/{out_file} to {fpath}')\n",
        "          shutil.move(f'{root_dir}Real-ESRGAN/{result_folder}/{out_file}', fpath)\n",
        "          # !python inference_realesrgan.py --model_path experiments/pretrained_models/RealESRGAN_x4plus.pth --input upload --netscale 4 --outscale 3.5 --half --face_enhance\n",
        "          os.chdir(f'{stable_dir}/stable-diffusion')\n",
        "        if save_image_metadata:\n",
        "          img = Image.open(fpath)\n",
        "          metadata = PngInfo()\n",
        "          metadata.add_text(\"artist\", meta_ArtistName)\n",
        "          metadata.add_text(\"copyright\", meta_Copyright)\n",
        "          metadata.add_text(\"software\", \"Stable Diffusion 1.3\" + f\", upscaled {enlarge_scale}x with ESRGAN\" if apply_ESRGAN_upscale else \"\")\n",
        "          metadata.add_text(\"title\", p)\n",
        "          #metadata.add_text(\"prompt\", p)\n",
        "          #metadata.add_text(\"config\", str(d.tags))\n",
        "          img.save(fpath, pnginfo=metadata)\n",
        "        new_file = f'{file_prefix}{filename}'\n",
        "        if os.path.isfile(os.path.join(image_output, f'{new_file}-{idx}.png')):\n",
        "          new_file += '-' + random.choice(string.ascii_letters) + random.choice(string.ascii_letters)\n",
        "        new_file += f'-{idx}.png'\n",
        "        shutil.move(fpath, os.path.join(image_output, new_file))\n",
        "        idx += 1\n"
      ],
      "metadata": {
        "id": "9TNTqeqSd2cE",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "biug_Qh4PC37",
        "cellView": "form"
      },
      "outputs": [],
      "source": [
        "#@title ### Out of memory? Try running this...\n",
        "\n",
        "#import torch\n",
        "#torch.cuda.empty_cache()\n",
        "\n",
        "#taskkill /F /PID <your PID here>\n",
        "\n",
        "#CUDA torch not enabled? try this:\n",
        "!pip uninstall torch\n",
        "!pip install torch --pre --extra-index-url https://download.pytorch.org/whl/nightly/cu116"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Manually Run txt2img scripts"
      ],
      "metadata": {
        "id": "-EVvd3DB_9Yw"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "i9e6r3Wj09gk",
        "outputId": "34d15926-8ec4-4f99-b704-b3290bb3afbe"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Global seed set to 1134143\n",
            "Loading model from models/ldm/stable-diffusion-v1/model.ckpt\n",
            "Global Step: 440000\n",
            "LatentDiffusion: Running in eps-prediction mode\n",
            "DiffusionWrapper has 859.52 M params.\n",
            "making attention of type 'vanilla' with 512 in_channels\n",
            "Working with z of shape (1, 4, 32, 32) = 4096 dimensions.\n",
            "making attention of type 'vanilla' with 512 in_channels\n",
            "^C\n"
          ]
        }
      ],
      "source": [
        "#USING COLAB PRO HIGH-RAM (CHANGE SEED EACH TIME):\n",
        "\n",
        "#txt2img no k_lms sampling\n",
        "#!python scripts/txt2img.py --prompt \"photorealistic award-winning photograph of a dog on a unicycle \" --n_samples 1 --n_iter 1 --ddim_steps 50 --H 640 --scale 7 --seed 4314123 --plms\n",
        "\n",
        "#txt2img with k_lms sampling\n",
        "!python scripts/txt2img_k.py --prompt \"photorealistic award-winning photograph of a dog on a unicycle \" --n_samples 1 --n_iter 1 --ddim_steps 50 --H 640 --scale 7 --seed 1134143 "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Wn0Wx7iQJYxQ"
      },
      "outputs": [],
      "source": [
        "#USING 1xA6000 OR EQUIVALENT (CHANGE SEED EACH TIME):\n",
        "\n",
        "#txt2img no k_lms sampling\n",
        "!python scripts/txt2img.py --prompt \"photorealistic award-winning photograph of a dog on a unicycle \" --n_samples 3 --n_iter 1 --ddim_steps 50 --H 832 --W 576 --scale 10 --seed 9235342 --plms\n",
        "#txt2img with k_lms sampling\n",
        "!python scripts/txt2img_k.py --prompt \"photorealistic award-winning photograph of a dog on a unicycle \" --n_samples 3 --n_iter 1 --ddim_steps 50 --H 832 --W 576 --scale 10 --seed 9235342"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NJJUVeQEXjIj"
      },
      "outputs": [],
      "source": [
        "#image2image script\n",
        "\n",
        "!python scripts/img2img.py --prompt \"photorealistic award-wining photograph of a dog on a unicycle\" --init-img /imagepath/image.png --strength 0.4 --seed 1574552011 --scale 7 --n_samples 1 --n_iter 1"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wJScwX1UAEng",
        "jp-MarkdownHeadingCollapsed": true,
        "tags": []
      },
      "source": [
        "# DEBUG CODE - IGNORE"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wq3_Mr_sJGid"
      },
      "outputs": [],
      "source": [
        "\n",
        "!mkdir -p models/ldm/stable-diffusion-v1/\n",
        "\n",
        "%cd models/ldm/stable-diffusion-v1/\n",
        "\n",
        "!ln -s /content/drive/My Drive/stable-diffusion-v-1-3/sd-v1-3-full-ema.ckpt /content/drive/My\\ Drive/Stable\\ Run/stable-diffusion/models/ldm/stable-diffusion-v1/\n",
        "\n",
        "#OUTDATED:\n",
        "#REPLACE ldm.py in usr/local/lib/python3.7/site-packages/ldm.py WITH the one in /content/drive/My Drive/Stable\\ Run/ldm.py\n",
        "#REPLACE microsoft_demo.py in /usr/local/lib/python3.7/site-packages/ai_tools/microsoft_demo.py WITH the one in /content/drive/My Drive/Stable\\ Run/microsoft_demo.py"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "CGxaCFQThjco"
      },
      "outputs": [],
      "source": [
        "!pip freeze"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "c5g2z9xDZin9",
        "outputId": "73620e5a-62ac-405e-8977-c823188e4132"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Fri Aug 19 06:52:36 2022       \n",
            "+-----------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 460.32.03    Driver Version: 460.32.03    CUDA Version: 11.2     |\n",
            "|-------------------------------+----------------------+----------------------+\n",
            "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                               |                      |               MIG M. |\n",
            "|===============================+======================+======================|\n",
            "|   0  Tesla P100-PCIE...  Off  | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   35C    P0    26W / 250W |      0MiB / 16280MiB |      0%      Default |\n",
            "|                               |                      |                  N/A |\n",
            "+-------------------------------+----------------------+----------------------+\n",
            "                                                                               \n",
            "+-----------------------------------------------------------------------------+\n",
            "| Processes:                                                                  |\n",
            "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
            "|        ID   ID                                                   Usage      |\n",
            "|=============================================================================|\n",
            "|  No running processes found                                                 |\n",
            "+-----------------------------------------------------------------------------+\n"
          ]
        }
      ],
      "source": [
        "!nvidia-smi"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [
        "-EVvd3DB_9Yw",
        "wJScwX1UAEng"
      ],
      "machine_shape": "hm",
      "name": "Enhanced_Stable_Diffusion_k_lms.ipynb",
      "provenance": [],
      "include_colab_link": true
    },
    "gpuClass": "standard",
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.13"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}