From 18065673fa5646c0e00e7269741142ddec771ec4 Mon Sep 17 00:00:00 2001 From: Marco Buono Date: Tue, 31 Oct 2023 17:59:02 -0300 Subject: [PATCH] `StandardMaterial` Light Transmission (#8015) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit # Objective Screenshot 2023-04-26 at 01 07 34 This PR adds both diffuse and specular light transmission capabilities to the `StandardMaterial`, with support for screen space refractions. This enables realistically representing a wide range of real-world materials, such as: - Glass; (Including frosted glass) - Transparent and translucent plastics; - Various liquids and gels; - Gemstones; - Marble; - Wax; - Paper; - Leaves; - Porcelain. Unlike existing support for transparency, light transmission does not rely on fixed function alpha blending, and therefore works with both `AlphaMode::Opaque` and `AlphaMode::Mask` materials. ## Solution - Introduces a number of transmission related fields in the `StandardMaterial`; - For specular transmission: - Adds logic to take a view main texture snapshot after the opaque phase; (in order to perform screen space refractions) - Introduces a new `Transmissive3d` phase to the renderer, to which all meshes with `transmission > 0.0` materials are sent. - Calculates a light exit point (of the approximate mesh volume) using `ior` and `thickness` properties - Samples the snapshot texture with an adaptive number of taps across a `roughness`-controlled radius enabling “blurry” refractions - For diffuse transmission: - Approximates transmitted diffuse light by using a second, flipped + displaced, diffuse-only Lambertian lobe for each light source. ## To Do - [x] Figure out where `fresnel_mix()` is taking place, if at all, and where `dielectric_specular` is being calculated, if at all, and update them to use the `ior` value (Not a blocker, just a nice-to-have for more correct BSDF) - To the _best of my knowledge, this is now taking place, after 964340cdd. The fresnel mix is actually "split" into two parts in our implementation, one `(1 - fresnel(...))` in the transmission, and `fresnel()` in the light implementations. A surface with more reflectance now will produce slightly dimmer transmission towards the grazing angle, as more of the light gets reflected. - [x] Add `transmission_texture` - [x] Add `diffuse_transmission_texture` - [x] Add `thickness_texture` - [x] Add `attenuation_distance` and `attenuation_color` - [x] Connect values to glTF loader - [x] `transmission` and `transmission_texture` - [x] `thickness` and `thickness_texture` - [x] `ior` - [ ] `diffuse_transmission` and `diffuse_transmission_texture` (needs upstream support in `gltf` crate, not a blocker) - [x] Add support for multiple screen space refraction “steps” - [x] Conditionally create no transmission snapshot texture at all if `steps == 0` - [x] Conditionally enable/disable screen space refraction transmission snapshots - [x] Read from depth pre-pass to prevent refracting pixels in front of the light exit point - [x] Use `interleaved_gradient_noise()` function for sampling blur in a way that benefits from TAA - [x] Drill down a TAA `#define`, tweak some aspects of the effect conditionally based on it - [x] Remove const array that's crashing under HLSL (unless a new `naga` release with https://github.com/gfx-rs/naga/pull/2496 comes out before we merge this) - [ ] Look into alternatives to the `switch` hack for dynamically indexing the const array (might not be needed, compilers seem to be decent at expanding it) - [ ] Add pipeline keys for gating transmission (do we really want/need this?) - [x] Tweak some material field/function names? ## A Note on Texture Packing _This was originally added as a comment to the `specular_transmission_texture`, `thickness_texture` and `diffuse_transmission_texture` documentation, I removed it since it was more confusing than helpful, and will likely be made redundant/will need to be updated once we have a better infrastructure for preprocessing assets_ Due to how channels are mapped, you can more efficiently use a single shared texture image for configuring the following: - R - `specular_transmission_texture` - G - `thickness_texture` - B - _unused_ - A - `diffuse_transmission_texture` The `KHR_materials_diffuse_transmission` glTF extension also defines a `diffuseTransmissionColorTexture`, that _we don't currently support_. One might choose to pack the intensity and color textures together, using RGB for the color and A for the intensity, in which case this packing advice doesn't really apply. --- ## Changelog - Added a new `Transmissive3d` render phase for rendering specular transmissive materials with screen space refractions - Added rendering support for transmitted environment map light on the `StandardMaterial` as a fallback for screen space refractions - Added `diffuse_transmission`, `specular_transmission`, `thickness`, `ior`, `attenuation_distance` and `attenuation_color` to the `StandardMaterial` - Added `diffuse_transmission_texture`, `specular_transmission_texture`, `thickness_texture` to the `StandardMaterial`, gated behind a new `pbr_transmission_textures` cargo feature (off by default, for maximum hardware compatibility) - Added `Camera3d::screen_space_specular_transmission_steps` for controlling the number of “layers of transparency” rendered for transmissive objects - Added a `TransmittedShadowReceiver` component for enabling shadows in (diffusely) transmitted light. (disabled by default, as it requires carefully setting up the `thickness` to avoid self-shadow artifacts) - Added support for the `KHR_materials_transmission`, `KHR_materials_ior` and `KHR_materials_volume` glTF extensions - Renamed items related to temporal jitter for greater consistency ## Migration Guide - `SsaoPipelineKey::temporal_noise` has been renamed to `SsaoPipelineKey::temporal_jitter` - The `TAA` shader def (controlled by the presence of the `TemporalAntiAliasSettings` component in the camera) has been replaced with the `TEMPORAL_JITTER` shader def (controlled by the presence of the `TemporalJitter` component in the camera) - `MeshPipelineKey::TAA` has been replaced by `MeshPipelineKey::TEMPORAL_JITTER` - The `TEMPORAL_NOISE` shader def has been consolidated with `TEMPORAL_JITTER` --- Cargo.toml | 6 +++++ src/loader.rs | 63 +++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+) diff --git a/Cargo.toml b/Cargo.toml index eac5769..354c5ac 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -8,6 +8,9 @@ repository = "https://github.com/bevyengine/bevy" license = "MIT OR Apache-2.0" keywords = ["bevy"] +[features] +pbr_transmission_textures = [] + [dependencies] # bevy bevy_animation = { path = "../bevy_animation", version = "0.12.0-dev", optional = true } @@ -30,6 +33,9 @@ bevy_utils = { path = "../bevy_utils", version = "0.12.0-dev" } # other gltf = { version = "1.3.0", default-features = false, features = [ "KHR_lights_punctual", + "KHR_materials_transmission", + "KHR_materials_ior", + "KHR_materials_volume", "KHR_materials_unlit", "KHR_materials_emissive_strength", "extras", diff --git a/src/loader.rs b/src/loader.rs index fa3085e..abdbc1a 100644 --- a/src/loader.rs +++ b/src/loader.rs @@ -763,6 +763,56 @@ fn load_material( texture_handle(load_context, &info.texture()) }); + #[cfg(feature = "pbr_transmission_textures")] + let (specular_transmission, specular_transmission_texture) = + material.transmission().map_or((0.0, None), |transmission| { + let transmission_texture: Option> = transmission + .transmission_texture() + .map(|transmission_texture| { + // TODO: handle transmission_texture.tex_coord() (the *set* index for the right texcoords) + texture_handle(load_context, &transmission_texture.texture()) + }); + + (transmission.transmission_factor(), transmission_texture) + }); + + #[cfg(not(feature = "pbr_transmission_textures"))] + let specular_transmission = material + .transmission() + .map_or(0.0, |transmission| transmission.transmission_factor()); + + #[cfg(feature = "pbr_transmission_textures")] + let (thickness, thickness_texture, attenuation_distance, attenuation_color) = material + .volume() + .map_or((0.0, None, f32::INFINITY, [1.0, 1.0, 1.0]), |volume| { + let thickness_texture: Option> = + volume.thickness_texture().map(|thickness_texture| { + // TODO: handle thickness_texture.tex_coord() (the *set* index for the right texcoords) + texture_handle(load_context, &thickness_texture.texture()) + }); + + ( + volume.thickness_factor(), + thickness_texture, + volume.attenuation_distance(), + volume.attenuation_color(), + ) + }); + + #[cfg(not(feature = "pbr_transmission_textures"))] + let (thickness, attenuation_distance, attenuation_color) = + material + .volume() + .map_or((0.0, f32::INFINITY, [1.0, 1.0, 1.0]), |volume| { + ( + volume.thickness_factor(), + volume.attenuation_distance(), + volume.attenuation_color(), + ) + }); + + let ior = material.ior().unwrap_or(1.5); + StandardMaterial { base_color: Color::rgba_linear(color[0], color[1], color[2], color[3]), base_color_texture, @@ -782,6 +832,19 @@ fn load_material( emissive: Color::rgb_linear(emissive[0], emissive[1], emissive[2]) * material.emissive_strength().unwrap_or(1.0), emissive_texture, + specular_transmission, + #[cfg(feature = "pbr_transmission_textures")] + specular_transmission_texture, + thickness, + #[cfg(feature = "pbr_transmission_textures")] + thickness_texture, + ior, + attenuation_distance, + attenuation_color: Color::rgb_linear( + attenuation_color[0], + attenuation_color[1], + attenuation_color[2], + ), unlit: material.unlit(), alpha_mode: alpha_mode(material), ..Default::default()