Sketch2Manga

Apply screentone to line drawings or colored illustrations with diffusion models.

Sketch2Manga - Drag and drop into ComfyUI to load the workflow (Source @ini_pmh)

Illustration2Manga - Drag and drop into ComfyUI to load the workflow (Source @curecu8)

Usage

Preperation

Install pytorch

Install ComfyUI or stable-diffusion-webui, download a diffusion model for colorization (the demo used meinapastel for ComfyUI, anything-v4.5 for sd-webui) and control_v11p_sd15_lineart.
Download the finetuned vae and diffusion model for screening.

ComfyUI

Clone this repo to the ComfyUI directory and install dependencies:

git clone https://github.com/dmMaze/sketch2manga [ComfyUI Directory]/custom_nodes/sketch2manga
cd [ComfyUI Directory]/custom_nodes/sketch2manga 
pip install -r requirements.txt

Launch ComfyUI, drag and drop the figure above to load the workflow.

Gradio Demo

Clone this repo and install dependencies, launch sd-webui with argument --api, and run

python gradio_demo/launch.py

SD-WebUI API

There is an example webuiapi_demo.ipynb showcasing inference using SD-WebUI API, it is a bit outdated though, but the logic applied is the same.

Comparsion

Our Illustration to Manga method compared with Mimic Manga (considered as SOTA)

Illustration (Input)	Mimic Manga	Ours