Add UI Materials

[MarkusTheOrt]

Objective

• Add Ui Materials so that UI can render more complex and animated widgets.
• Fixes Add support for applying a shader to a UI Image #5607

Solution

• Create a UiMaterial trait for specifying a Shader Asset and Bind Group Layout/Data.
• Create a pipeline for rendering these Materials inside the Ui layout/tree.
• Create a MaterialNodeBundle for simple spawning.

Changelog

• Created a UiMaterial trait for specifying a Shader asset and Bind Group.
• Created a UiMaterialPipeline for rendering said Materials.
• Added Example ui_material for example usage.
• Created UiVertexOutput export as VertexData for shaders.
• Created material_ui shader as default for both Vertex and Fragment shaders.

[github-actions]

Welcome, new contributor!

Please make sure you've read our contributing guide and we look forward to reviewing your pull request shortly ✨

[IceSentry]

Could you add a simple example showing how to use this? I can figure it out fairly easily but I believe this feature deserves a standalone example.

[MarkusTheOrt]

I just added a simple example showcasing the feature. (its called ui_material)
I am thinking about ditching the use of UiPipeline inside the UiMaterialPipeline so that I can get rid of the default image bindings. Not every shader requires a texture, and if one is required it can be passed using the material.

Next up I do some renaming and documenting

[ickshonpe]

Looks really good. Maybe the material module would be better split up, with UiMaterialBundle in node_bundles.rs for discoverability, and the rendering-specific code moved to render/material.rs or render_material/mod.rs etc.

[MarkusTheOrt]

I did now remove the usage of UiPipeline, and copied the relevant code into the UiMaterialPipeline.
I also did split up the additions, with essentially only the UiMaterial trait and its accompanying UiMaterialKey being inside the material.rs and the rest being in render/material_pipeline.rs.

[ickshonpe]

Ah noticed a snag, ideally we want some sort of MaterialImageBundle as well:

/// A Ui Node that renders a shader
#[derive(Bundle, Debug)]
pub struct MaterialImageBundle<M: UiMaterial> {
    /// Describes the logical size of the node
    pub node: Node,
    /// Styles which control the layout (size and position) of the node and it's children
    /// In some cases these styles also affect how the node drawn/painted.
    pub style: Style,
    /// The Material of the Node.
    pub material: Handle<M>,
    /// Whether this node should block interaction with lower nodes
    pub focus_policy: FocusPolicy,
    /// The transform of the node
    ///
    /// This field is automatically managed by the UI layout system.
    /// To alter the position of the `NodeBundle`, use the properties of the [`Style`] component.
    pub transform: Transform,
    /// The global transform of the node
    ///
    /// This field is automatically managed by the UI layout system.
    /// To alter the position of the `NodeBundle`, use the properties of the [`Style`] component.
    pub global_transform: GlobalTransform,
    /// Describes the visibility properties of the node
    pub visibility: Visibility,
    /// Algorithmically-computed indication of whether an entity is visible and should be extracted for rendering
    pub computed_visibility: ComputedVisibility,
    /// Indicates the depth at which the node should appear in the UI
    pub z_index: ZIndex,
    /// The size of the image in pixels
    ///
    /// This field is set automatically
    pub image_size: UiImageSize,
    /// The calculated size based on the given image
    pub content_size: ContentSize,

}

impl<M: UiMaterial> Default for MaterialImageBundle<M> {
    fn default() -> Self {
        Self {
            node: Default::default(),
            style: Default::default(),
            material: Default::default(),
            focus_policy: Default::default(),
            transform: Default::default(),
            global_transform: Default::default(),
            visibility: Default::default(),
            computed_visibility: Default::default(),
            z_index: Default::default(),
            content_size: Default::default(),
        }
    }
}

The ContentSize component contains a measure func that is used to negotiate with the layout algorithm to find an optimal size for a node's content given that content's properties (like its aspect ratio and pixel density) and the space constraints of the UI layout.

ImageBundle has a matching system update_image_content_size_system that retrieves the size of the image from its texture asset and sets a measure func that preserves its aspect ratio. But we can't do that here because UiMaterial doesn't expose its texture asset publically (if it has one).

[ickshonpe]

I think it would be fine not to worry about measurefuncs for this PR though, the implementation looks great, really nice and clean.

It's not essential as users can create their own system to add a measurefunc, which isn't very hard. We do need a custom measurefunc example, it would make sense to base one around a UiMaterial.

[MarkusTheOrt]

Great suggestion, I agree with you there, I wasn't aware of ContentSize

I'd like to get this PR done and would be happy to take on the MaterialImageBundle as a followup once it is ready

[superdump]

Couldn’t the image just be a member of the material type? @ickshonpe as well as other data needed - the image size.

[MarkusTheOrt]

I did Change some of the names like @ickshonpe suggested.

I am thinking that it would be a good addition to add the bevy_render::globals into the default shader bindings.
I believe it is a sensible default since time can be useful when doing anything animated within a shader.

[ickshonpe]

Another problem this time with batching and draw ordering, I modified the ui_material example a bit to demonstrate:

use bevy::prelude::*;
use bevy_internal::{
    reflect::{TypePath, TypeUuid},
    render::render_resource::*,
};

fn main() {
    App::new()
        .add_plugins(DefaultPlugins)
        .add_plugins(UiMaterialPlugin::<CustomUiMaterial>::default())
        .add_systems(Startup, setup)
        .add_systems(Update, update)
        .run();
}

fn update(time: Res<Time>, mut ui_materials: ResMut<Assets<CustomUiMaterial>>) {
    for (_, material) in ui_materials.iter_mut() {
        material.fill_amount = time.elapsed_seconds() % 1.0;
        let new_color = Color::WHITE;
        material.color = new_color.into();
    }
}

fn setup(mut commands: Commands, mut ui_materials: ResMut<Assets<CustomUiMaterial>>) {
    // Camera so we can see UI
    commands.spawn(Camera2dBundle::default());


    commands
        .spawn(NodeBundle {
            style: Style {
                width: Val::Percent(100.0),
                align_items: AlignItems::Center,
                justify_content: JustifyContent::Center,
                ..default()
            },
            background_color: Color::BLACK.into(),
            ..default()
        })
        .with_children(|parent| {
            parent.spawn(NodeBundle {
                style: Style {
                    width: Val::Px(500.0),
                    height: Val::Px(500.0),
                    border: UiRect::all(Val::Px(10.)),
                    align_items: AlignItems::Center,
                    justify_content: JustifyContent::Center,
                    ..Default::default()
                },
                border_color: BorderColor(Color::WHITE),
                ..Default::default()
            }).with_children(|parent| {
                parent.spawn(NodeBundle {
                    style: Style {
                        position_type: PositionType::Absolute,
                        width: Val::Px(500.0),
                        height: Val::Px(25.0),
                        ..Default::default()
                    },
                    background_color: BackgroundColor(Color::GREEN),
                ..Default::default()
                });
                parent.spawn(MaterialNodeBundle {
                    style: Style {
                        position_type: PositionType::Absolute,
                        width: Val::Px(250.0),
                        height: Val::Px(250.0),
                        ..default()
                    },
                    material: ui_materials.add(CustomUiMaterial {
                        fill_amount: 0.0,
                        color: Color::WHITE.into(),
                    }),
                    ..default()
                });
                parent.spawn(NodeBundle {
                    style: Style {
                        position_type: PositionType::Absolute,
                        width: Val::Px(25.0),
                        height: Val::Px(500.0),
                        ..Default::default()
                    },
                    background_color: BackgroundColor(Color::RED),
                ..Default::default()
                });
            });
        });
}

#[derive(AsBindGroup, TypeUuid, TypePath, Debug, Clone)]
#[uuid = "7b5569c8-36d4-4c9d-acb7-d1754b385ab1"]
struct CustomUiMaterial {
    #[uniform(0)]
    fill_amount: f32,
    #[uniform(0)]
    color: Vec4,
}

impl UiMaterial for CustomUiMaterial {
    fn fragment_shader() -> ShaderRef {
        "shaders/circle_shader.wgsl".into()
    }
}

The circular section should be drawn in front of the green bar and behind the red bar, but it's drawn on top instead:

I managed to force it to render correctly by disabling batching completely and merging #9062 which is a fix for some UI draw order bugs.

The issue is that prepare_uinodes and prepare_uimaterial_nodes are working independently of each other.

prepare_ui_nodes puts the black background, the border rects, the green bar, and the red bar altogether into one batch.
prepare_ui_material puts the circular section ui material node into another batch.
Then the batches are sorted and then drawn in order. The batch with regular nodes first, and then the UI material batch on top.
prepare_ui_nodes has no idea that the red bar needs to be batched separately.

[MarkusTheOrt]

The issue is that prepare_uinodes and prepare_uimaterial_nodes are working independently of each other.

I am not sure if there is anything that can be done here. Disabling all batching is sub-optimal to say the least.
Maybe there can be batching for each layer in the Ui?

[MarkusTheOrt]

I’ll resolve these as soon as possible (in about one hour or so)

[JMS55]

Code looks generally fine, although I'm not familiar with any of the UI stuff. Some small doc nits.

[alice-i-cecile]

I'm less sleep-deprived now and feeling comfortable re-reviewing this. I'm happy with the quality of code, the docs are great, and this is useful abstraction. Merging for 0.12 :)