Skip to content

Commit

Permalink
remove the word "noise" from readme
Browse files Browse the repository at this point in the history
  • Loading branch information
ljleb authored Dec 13, 2023
1 parent 39abf95 commit ccc2aed
Showing 1 changed file with 11 additions and 11 deletions.
22 changes: 11 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,15 @@ Neutral prompt is an a1111 webui extension that adds alternative composable diff

### Keyword `AND_PERP`

The `AND_PERP` keyword, standing for "PERPendicular `AND`", integrates the orthogonalization process described in the Perp-Neg paper. Essentially, `AND_PERP` allows for prompting concepts that highly overlap with regular prompts, by negating contradicting noise.
The `AND_PERP` keyword, standing for "PERPendicular `AND`", integrates the orthogonalization process described in the Perp-Neg paper. Essentially, `AND_PERP` allows for prompting concepts that highly overlap with regular prompts, by negating contradicting concepts.

You could visualize it as such: if `AND` prompts are "greedy" (taking as much space as possible in the output), `AND_PERP` prompts are opposite, relinquishing control as soon as there is a disagreement in the generated output.

### Keyword `AND_SALT`

Saliency-aware noise blending is made possible using the `AND_SALT` keyword, shorthand for "SALienT `AND`". In essence, `AND_SALT` monitors high noise activity during denoising and dominates any high-activation regions in the output.
Saliency-aware blending is made possible using the `AND_SALT` keyword, shorthand for "SALienT `AND`". In essence, `AND_SALT` keeps the highest activation pixels at each denoising step.

Think of it as a territorial dispute: the noise generated by the `AND` prompts is one country, and the noise(s) generated by `AND_SALT` prompts represent neighbouring nations. They're all vying for the same land - whoever strikes the strongest at a given time and location claims it.
Think of it as a territorial dispute: the image generated by the `AND` prompts is one country, and the images generated by `AND_SALT` prompts represent neighbouring nations. They're all vying for the same land - whoever strikes the strongest at a given time (denoising step) and location (latent pixel) claims it.

### Keyword `AND_TOPK`

Expand Down Expand Up @@ -51,7 +51,7 @@ Key observations:

### Using the `AND_SALT` Keyword

The `AND_SALT` keyword can be used to invoke saliency-aware noise blending. It spotlights and accentuates areas of high-activation in the output.
The `AND_SALT` keyword can be used to invoke saliency-aware blending. It spotlights and accentuates areas of high-activation in the output.

Consider this example prompt utilizing `AND_SALT`:

Expand All @@ -60,7 +60,7 @@ a vibrant rainforest with lush green foliage
AND_SALT the glimmering rays of a golden sunset piercing through the trees
```

In this case, the extension identifies and isolates the most salient regions in the noise of the sunset prompt. Then, the extension applies this salient noise to the noise of the rainforest prompt. Only the portions of the rainforest noise that coincide with the salient areas of the sunset noise are affected. These areas are replaced by noise from the sunset prompt.
In this case, the extension identifies and isolates the most salient regions in the sunset prompt. Then, the extension applies this marsked image to the rainforest prompt. Only the portions of the rainforest prompt that coincide with the salient areas of the sunset prompt are affected. These areas are replaced by pixels from the sunset prompt.

This is an XY grid with prompt S/R `AND_SALT, AND, AND_PERP`:

Expand Down Expand Up @@ -90,15 +90,15 @@ AND_SALT [
]
```

To generate the final noise from the diffusion model:
To generate the final image from the diffusion model:

1. The extension first processes the root `AND` prompts. In this case, it's just `magical tree forests, eternal city`
2. It then processes the `AND_SALT` prompt `small nocturne companion` in the context of `electrical pole voyage`. This enhances salient features in the `electrical pole voyage` noise
3. This new noise is orthogonalized with the noise from `magical tree forests, eternal city`, blending the details of the 'electrical pole voyage' into the main scene without creating conflicts
4. The extension then turns to the second `AND_SALT` group. It processes `electric arcs, bzzz, sparks` in the context of `electrical tornado`, amplifying salient features in the electrical tornado noise
5. The noise from this `AND_SALT` group is then combined with the noise of `magical tree forests, eternal city`. The final output retains the strongest features from both the `electrical tornado` (enhanced by 'electric arcs, bzzz, sparks') and the earlier 'magical tree forests, eternal city' scene influenced by the 'electrical pole voyage'
2. It then processes the `AND_SALT` prompt `small nocturne companion` in the context of `electrical pole voyage`. This enhances salient features in the `electrical pole voyage` image
3. This new image is orthogonalized with the image from `magical tree forests, eternal city`, blending the details of the 'electrical pole voyage' into the main scene without creating conflicts
4. The extension then turns to the second `AND_SALT` group. It processes `electric arcs, bzzz, sparks` in the context of `electrical tornado`, amplifying salient features in the electrical tornado image
5. The image from this `AND_SALT` group is then combined with the `magical tree forests, eternal city` image. The final output retains the strongest features from both the `electrical tornado` (enhanced by 'electric arcs, bzzz, sparks') and the earlier 'magical tree forests, eternal city' scene influenced by the 'electrical pole voyage'

Each keyword can define a distinct denoising space within its square brackets `[...]`. Prompts inside it merge into a single noise map before further processing down the prompt tree.
Each keyword can define a distinct denoising space within its square brackets `[...]`. Prompts inside it merge into a single image before further processing down the prompt tree.

While there's no strict limit on the depth of nesting, experimental evidence suggests that going beyond a depth of 2 is generally unnecessary. We're still exploring the added precision from deeper nesting. If you discover innovative ways of controlling the generations using nested prompts, please share in the discussions!

Expand Down

0 comments on commit ccc2aed

Please sign in to comment.