Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve "Add Noise" node to support subtle noise generation (#3048) #3055

Merged
merged 1 commit into from
Jan 4, 2025

Conversation

Arcitec
Copy link
Contributor

@Arcitec Arcitec commented Dec 5, 2024

Implements fine-grained control over the noise amount, by switching to a float-based noise multiplier instead.

(The previous implementation was only able to generate noise in increments of +2.55 on the 0-255 pixel luminance scale.)


Detailed description:

  • The node's previous noise implementation was not granular enough, causing huge jumps between the noise amounts generated by different slider values. The problem was the division factor (an integer scale of 0-100). Think about it in terms of the 0-255 range of normal uint8 image channels. The previous algorithm divided the input slider by 100, and the lowest input value was 1, meaning that the lowest possible noise amount "step" was 0.01. Now let's look at what happens when we generate such noise: 0.01 * 255 = 2.55. This means the smallest noise amount step was causing roughly 3 brightness steps for each pixel, which made the entire node very heavy-handed and not very useful. It was most noticeable when generating monochromatic noise on RGB images, where the equal impact on every RGB channel made the heavy noise very invasive (because ChaiNNer adds the same monochromatic noise texture equally to all 3 channels then).

  • To fix the node in a backwards-compatible way, the slider has been changed to a float with a single decimal, ranging from 0.0 to 100.0. By adding a single decimal to the slider, we thereby make the noise amount go in steps of 0.1 / 100 = 0.001 instead, which represents 0.001 * 255 = 0.255 on the regular uint8 scale. So to generate noise that adds +1 on the uint8 scale, you would now just need to go roughly +0.4 on the float slider.

This improvement finally makes the noise node granular enough to be useful for subtle, artistic noise.


Closes #3048

…-org#3048)

Implements fine-grained control over the noise amount, by switching to a float-based noise multiplier instead.

(The previous implementation was only able to generate noise in increments of +2.55 on the 0-255 pixel luminance scale.)
@Arcitec
Copy link
Contributor Author

Arcitec commented Dec 5, 2024

@FNsi Since you expressed interest on the ticket, you may be interested in downloading this patch. Just letting you know. :)

PS for anyone who wants to add camera-accurate noise, adding texture and realism without ever becoming distracting, you should use Gaussian noise, Color, Amount 0.8. The difference with/without adding noise after "image cleanup" models such as Swin2SR is like looking at plastic vs a real photo. The importance of noise becomes extremely apparent when zooming into a processed image. Have fun!

And yes, "Color" is the correct mode even when the input image is a 1-channel (grayscale) image. It will add the correct amount of (monochrome) noise to the grayscale image too, and will look completely equivalent to the perceived noise being added on RGB images. Sure, RGB means 3 channels, where each channel gets its own individual noise pattern, but the overall effect still ends up looking basically the same when just adding a single noise pattern to a single-channel grayscale image. So grayscale images are fixed with these settings too, meaning that you can use these values universally and don't have to worry about it. I verified this in-depth by taking an RGB image, converting it to a 1-channel grayscale image, and running it through the same settings as the color variant. Then looking very closely at the noise. The perceived noise levels looked absolutely identical.

So, again: Use Add Noise with Gaussian noise, Color, Amount 0.8 for all images, universally for both RGB and Grayscale input.

And if you're looking for the absolute best "JPEG compression artifact removal" workflow, I'll share it right here. There are two variants, artifact_removal.chn is the actual workflow you'll be using. There's also artifact_removal_compare.chn which performs the same work but outputs 3 PNG images for every input image: The original, the artifact removal, and the re-noised image. You can then see for yourself that noise is extremely important. If you want to use it for actual processing, you should just remove everything that isn't part of the "Save Image: Re-Noised" output path.

Here are the workflow files:

chainner_artifact_removal.zip

To be able to use these workflows, you will need to have the code patch from this pull request, by the way. Until it's merged, the easiest way to get the code is git clone -b fix-noise-node https://github.com/Arcitec/chaiNNer.git.

Furthermore, since the workflows were exported in "Share Workflow" mode, all of the input files, output directory and model paths are blank, so I recommend opening the presets, setting your model path to Swin2SR_RealworldSR_X4_64_BSRGAN_PSNR.pth in the Load Model node, and then re-saving the presets locally so that your own path always exists.

Another thing to note is that the "Load Single Image" node is connected by default. In real-world usage, you'll most likely want to re-route the Image and Name inputs to come from the "Load Images: Batch" node instead, so that you can use the workflow for batch processing.

And just in case the workflow files don't work for someone or is missing in the future, the correct workflow for image compression artifact removal is Swin2SR (Realworld 4x) -> Image Resize with Lanczos to 25% to get back to 1x which will un-hallucinate the upscaling artifacts -> Add noise with Gaussian noise, Color, Amount 0.8. You can then optionally add your own upscaler after that (Swin2SR is a great artifact remover but a terrible upscaler).

Enjoy!

@Arcitec
Copy link
Contributor Author

Arcitec commented Dec 5, 2024

@joeyballentine @RunDevelopment Thank you both so incredibly much for creating ChaiNNer. It's a stunningly beautiful piece of software, with great backend and frontend engineering. Truly a work of art.

I love the way you re-implement algorithms and neural networks in clever ways (spandrel, your genius noise generator code, and so on...). Seriously, thank you both so much.

This was the only issue I had with the application. The heavy-handed noise generator has now been fixed, and it's a small and uncontroversial and fully backwards-compatible change, so hopefully you find the time to merge it at your leisure sometime. :)

@FNsi
Copy link
Contributor

FNsi commented Dec 10, 2024

Haven't checked my account days , nice job man, I'd try it tomorrow!

@O-J1
Copy link

O-J1 commented Dec 10, 2024

Just tried it and can confirm it works.

Copy link
Member

@joeyballentine joeyballentine left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this

@joeyballentine joeyballentine merged commit 204a567 into chaiNNer-org:main Jan 4, 2025
14 checks passed
@Arcitec
Copy link
Contributor Author

Arcitec commented Jan 5, 2025

@joeyballentine Thank you too, man, for everything you did creating this amazing software. I hope you have a great new year! :)

@Arcitec Arcitec deleted the fix-noise-node branch January 5, 2025 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] Add Noise node is way too powerful
4 participants