Add a Bevy-native tool to generate (and bake) global illumination for light maps #12233
Labels
A-Assets
Load files from disk to use for things like images, models, and sounds
A-Editor
Graphical tools to make Bevy games
A-Rendering
Drawing game state to the screen
C-Feature
A new feature, making something new possible
D-Complex
Quite challenging from either a design or technical perspective. Ask for help!
Comments
alice-i-cecile
added
C-Feature
|
In order of difficulty, what would need to be done is:
|
|
I made a PR to replace gltf-ibl-sampler, but it didn't work well on all environment maps and I've shelved it for now due to lack of motivation #9414. |
|
I've contemplated posting my environment maps to Discord and asking if anyone would be willing to convert them :) |
|
BTW, three.js has a PMREMGenerator written in JavaScript which does something similar to what we want: https://github.com/mrdoob/three.js/blob/master/src/extras/PMREMGenerator.js |
pcwalton
added a commit
to pcwalton/bevy
that referenced
this issue
Jun 13, 2024
This commit introduces a new type of camera, the *omnidirectional* camera. These cameras render to a cubemap texture, and as such extract into six different cameras in the render world, one for each side. The cubemap texture can be attached to a reflection probe as usual, which allows reflections to contain moving objects. To use an omnidirectional camera, create an [`OmnidirectionalCameraBundle`]. Because omnidirectional cameras extract to six different sub-cameras in the render world, render world extraction code that targets components present on cameras now needs to be aware of this fact and extract components to the individual sub-cameras, not the root camera component. They also need to run after omnidirectional camera extraction, as only then will the sub-cameras be present in the render world. New plugins, `ExtractCameraComponentPlugin` and `ExtractCameraInstancesPlugin`, are available to assist with this. Each side of an omnidirectional camera can be individually marked as active via the `ActiveCubemapSides` bitfield. This allows for the common technique of rendering only one (or two, or three) sides of the cubemap per frame, to reduce rendering overhead. It also allows for on-demand rendering, so that an application that wishes to optimize further can choose sides to refresh. For example, an application might wish to only rerender sides whose frusta contain moving entities. In addition to real-time reflection probes, this patch introduces much of the infrastructure necessary to support baking reflection probes from within Bevy as opposed to in an external program such as Blender, which has been the status quo up to this point. Even with this patch, there are still missing pieces needed to make this truly convenient, however: 1. Baking a reflection probe requires more than just saving a cubemap: it requires pre-filtering the cubemap into diffuse and specular parts in the same way that the [glTF IBL Sampler] does. This is not yet implemented in Bevy; see bevyengine#9414 for a previous attempt. 2. The cubemap needs to be saved in `.ktx2` format, as that's the only format that Bevy presently knows how to load. There's no comprehensive Rust crate for this, though note that my [glTF IBL Sampler UI] has code to do it for the specific case of cubemaps. 3. An editor UI is necessary for convenience, as otherwise every application will have to create some sort of bespoke tool that arranges scenes and saves the reflection cubemaps. The `reflection_probes` example has been updated in order to add an option to enable dynamic reflection probes, as well as an option to spin the cubes so that the impact of the dynamic reflection probes is visible. Additionally, the static reflection probe, which was previously rendered in Blender, has been changed to one rendered in Bevy. This results in a change in appearance, as Blender and Bevy render somewhat differently. Partially addresses bevyengine#12233. [glTF IBL Sampler]: https://github.com/KhronosGroup/glTF-IBL-Sampler [glTF IBL Sampler UI]: https://github.com/pcwalton/gltf-ibl-sampler-egui
pcwalton
added a commit
to pcwalton/bevy
that referenced
this issue
Jun 14, 2024
This commit introduces a new type of camera, the *omnidirectional* camera. These cameras render to a cubemap texture, and as such extract into six different cameras in the render world, one for each side. The cubemap texture can be attached to a reflection probe as usual, which allows reflections to contain moving objects. To use an omnidirectional camera, create an [`OmnidirectionalCameraBundle`]. Because omnidirectional cameras extract to six different sub-cameras in the render world, render world extraction code that targets components present on cameras now needs to be aware of this fact and extract components to the individual sub-cameras, not the root camera component. They also need to run after omnidirectional camera extraction, as only then will the sub-cameras be present in the render world. New plugins, `ExtractCameraComponentPlugin` and `ExtractCameraInstancesPlugin`, are available to assist with this. Each side of an omnidirectional camera can be individually marked as active via the `ActiveCubemapSides` bitfield. This allows for the common technique of rendering only one (or two, or three) sides of the cubemap per frame, to reduce rendering overhead. It also allows for on-demand rendering, so that an application that wishes to optimize further can choose sides to refresh. For example, an application might wish to only rerender sides whose frusta contain moving entities. In addition to real-time reflection probes, this patch introduces much of the infrastructure necessary to support baking reflection probes from within Bevy as opposed to in an external program such as Blender, which has been the status quo up to this point. Even with this patch, there are still missing pieces needed to make this truly convenient, however: 1. Baking a reflection probe requires more than just saving a cubemap: it requires pre-filtering the cubemap into diffuse and specular parts in the same way that the [glTF IBL Sampler] does. This is not yet implemented in Bevy; see bevyengine#9414 for a previous attempt. 2. The cubemap needs to be saved in `.ktx2` format, as that's the only format that Bevy presently knows how to load. There's no comprehensive Rust crate for this, though note that my [glTF IBL Sampler UI] has code to do it for the specific case of cubemaps. 3. An editor UI is necessary for convenience, as otherwise every application will have to create some sort of bespoke tool that arranges scenes and saves the reflection cubemaps. The `reflection_probes` example has been updated in order to add an option to enable dynamic reflection probes, as well as an option to spin the cubes so that the impact of the dynamic reflection probes is visible. Additionally, the static reflection probe, which was previously rendered in Blender, has been changed to one rendered in Bevy. This results in a change in appearance, as Blender and Bevy render somewhat differently. Partially addresses bevyengine#12233. [glTF IBL Sampler]: https://github.com/KhronosGroup/glTF-IBL-Sampler [glTF IBL Sampler UI]: https://github.com/pcwalton/gltf-ibl-sampler-egui
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What problem does this solve or what need does it fill?
Light maps (cube maps, irradiance volumes, reflection probes) are a class of assets used to "fake" pretty but expensive global illumination. Bevy has support for these now, but as #10057 says, we have no way to generate them ourselves:
What solution would you like?
This is tagged with
A-Editor, as while it is possible to do this using a non-GUI solution, it makes more sense as a component of a larger scene editor.What alternative(s) have you considered?
Existing third-party light baking tools exist, and can be used with Bevy.
These are generally harder to customize / integrate, and cannot be used to dynamically generate lightmaps as part of procedurally generated environments.
Additional context
Pre-existing work:
The text was updated successfully, but these errors were encountered: