Ray Tracing Support #1040
Comments
kvark
added
area: api
|
See also:
It's too early to consider this an essential feature on all backends, but rolling it out at least where the backends do have support for it seems very useful today. |
|
Are there any updates on this? |
|
Not currently. There's interest from upstream WebGPU, but that would be well after v1. We could make it a native extension and I think it would be great, but the apis are quite large (including shader side transformations), and given very few of us actually have hardware to support this, the chance of it happening any time soon without a champion is low. |
|
Hi, when is this comming out? |
|
There's no one actively working on it to my knowledge. |
|
It would be nice to atleast get native extension or some bindings. Would be a nice start |
|
I'm curious about getting basic ray-tracing support working. I think that a first draft wouldn't be too hard, with some limitations. I think we'd initially want to:
The remaining part is then creating an abstraction for acceleration structures (https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VK_KHR_acceleration_structure.html). You probably want to focus on getting a simple acceleration structure abstraction for triangle mesh and instance ASes that are built on the GPU. Anything else can come later. I can't promise anything, but I'll try and look into this when I'm back with my home setup. |
|
My progress so far is at master...expenses:wgpu:hal-acceleration-structures. I plan on merging in wgpu-hal support first, probably with a 'hello world' ray-traced triangle example like https://github.com/SaschaWillems/Vulkan#basic-ray-tracing. |
|
@expenses thank you for championing Ray Tracing! It's very exciting for the community to get access to it 🚀 . Our main concerns are the maintenance costs for a large API surface added to the fact it will need to change in order to abstract over DXR efficiently. Ideally, there would be a proper investigation on the API differences between VkRT and DXR before wgpu-hal API is prototyped. However, we are somewhat confident that the amount of changes needed for DXR will be limited, and it would be fine to do as the next step. My advice, if I may, would be to not try to copy Vulkan into wgpu-hal. Our HAL is low level and zero/low overhead, but it doesn't have to be extremely low level. For example, it doesn't have the API for allocating memory and binding objects to it, like Vulkan. So if you have a choice of 1) put complexity in the Vulkan backend, or 2) expose it in wgpu-hal, please put a bigger weight on 1). We can make it more complex as a follow-up for DXR if needed. Again, thank you for all the amazing contributions. Looking forward to see the opportunities that your work opens to all of us! |
|
My impression is that as Vulkan raytracing was based on DXR, the APIs should be fairly similar. Beyond acceleration structures, the main thing I've focused on is the ray query extension: https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VK_KHR_ray_query.html. This allows ray tracing in the normal vertex/fragment/compute shader stages as opposed to requiring a new ray tracing shader stage and shader binding tables and all that stuff. This is equivalent to inline ray tracing in DXR 1.1: https://devblogs.microsoft.com/directx/dxr-1-1/#inline-raytracing |
Looking at the example. The setup on rust part is insane. 1k lines for raytracing few triangles. And unsafe blocks? |
|
Hello. I am a bit confused o.o. |
|
Hello!
Yes. This issue is for DXR, VK_KHR_ray_tracing support.
There's no set timeframe, just people working on it when they are able. There is a lot of components that go into raytracing, so full wgpu will take a while. Some small first steps have been made (like implementing RT in wgpu-hal with no shader support) |
|
Thank you for the response! |
|
If you want to play a bit with compute shaders and raytracing to create visibility buffer to the apply your shading, I've played a bit with the idea in my raytracing_visibility shader 😄 https://github.com/gents83/INOX/blob/master/data_raw/shaders/wgsl/raytracing_visibility.wgsl |
|
@expenses I'm adding ray query support to WGSL in gfx-rs/naga#2256 |
|
I have continued to work @expenses first implementation. (now PR #3507) The Specifications (Dx,Vk) are fairly compatible in regards to acceleration structures, Can anyone confirm if DX12 is unable to build acceleration structures indirectly? |
Definitely sounds like there is no indirect. |
|
@cwfitzgerald thanks for the assurance, (quite odd, that it is impossible). |
|
My Proposal for the api in wgpu: // Ray tracing api proposal for wgpu (underlining Vulkan, Metal and DX12 implementations)
// The general design goal is to come up with an simpler Api, which allows for validation.
// Since this validation would have high overhead in some cases,
// I decided to provide more limited functions and unsafe functions for the same action, to evade this tradeoff.
// Error handling and traits like Debug are omitted.
// Core structures with no public members
pub struct Blas {}
pub struct Tlas {}
pub struct BlasRequirements {}
pub struct TlasInstances{}
// Size descriptors used to describe the size requirements of blas geometries.
// Also used internally for strict validation
pub struct BlasTriangleGeometrySizeDescriptor{
pub vertex_format: wgt::VertexFormat,
pub vertex_count: u32,
pub index_format: Option<wgt::IndexFormat>,
pub index_count: Option<u32>,
pub flags: AccelerationStructureGeometryFlags,
}
pub struct BlasProceduralGeometrySizeDescriptor{
pub count: u32,
pub flags: AccelerationStructureGeometryFlags,
}
// Procedural geometry contains AABBs
pub struct BlasProceduralGeometry{
pub size: BlasTriangleGeometrySize,
pub bounding_box_buffer: Buffer,
pub bounding_box_buffer_offset: wgt::BufferAddress,
pub bounding_box_stride: wgt::BufferAddress,
}
// Triangle Geometry contains vertices, optionally indices and transforms
pub struct BlasTriangleGeometry{
pub size: BlasTriangleGeometrySize,
pub vertex_buffer: Buffer
pub first_vertex: u32,
pub vertex_stride: wgt::BufferAddress,
pub index_buffer: Option<Buffer>,
pub index_buffer_offset: Option<wgt::BufferAddress>,
pub transform_buffer: Option<Buffer>,
pub transform_buffer_offset: Option<wgt::BufferAddress>,
}
// Build flags
pub struct AccelerationStructureFlags{
// build_speed, small_size, ...
}
// Geometry flags
pub struct AccelerationStructureGeometryFlags{
// opaque, no_duplicate_any_hit, ...
}
// Descriptors used to determine the memory requirements and validation of a acceleration structure
pub enum BlasGeometrySizeDescriptors{
Triangles{desc: Vec<BlasTriangleGeometrySizeDescriptor>},
Procedural(desc: Vec<BlasProceduralGeometrySize>)
}
// With prefer update, we decide if an update is possible, else we rebuild.
// Maybe a force update option could be useful
pub enum UpdateMode{
Build,
// Update,
PreferUpdate,
}
// General descriptor for the size requirements,
// since the required size depends on the contents and build flags
pub struct GetBlasRequirementsDescriptor{
pub flags: AccelerationStructureFlags,
}
// Creation descriptors, we provide flags, and update_mode.
// We store it in the structure, so we don't need to pass it every build.
pub struct CreateBlasDescriptor<'a>{
pub requirements: &'a BlasRequirements
pub flags: AccelerationStructureFlags,
pub update_mode: UpdateMode,
}
pub struct CreateTlasDescriptor{
pub max_instances: u32,
pub flags: AccelerationStructureFlags,
pub update_mode: UpdateMode,
}
// Secure instance entry for tlas
struct TlasInstance{
transform: [f32; 12],
custom_index: u32,
mask: u8,
shader_binding_table_record_offset: u32,
flags: u8 //bitmap
blas: Blas
}
impl Device {
// Retrieves the size requirements for an acceleration structure.
// BlasRequirements stores the BlasGeometrySizeDescriptors for validation (thats why we take ownership)
// These descriptors are required for strict validation, because the underling (e.g. Vulkan) specifications doesn't
// make many guaranties about the ordering of size requirements between different geometries.
// By storing the sizes of all geometries we can validate that the different list of geometries is guarantied to fit.
// If we would just query if the requirement are satisfied for the new geometries, it may fit on some systems and not others.
pub fn get_blas_size_requirements(&self, desc: &GetBlasRequirementsDescriptor, entries: BlasGeometrySizeDescriptors) -> BlasRequirements;
// Creates a new bottom level accelerations structures and sets internal states for validation(reference to BlasGeometrySizeDescriptors)
// and builds (e.g update mode)
pub fn create_blas(&self, desc: &CreateBlasDescriptor) -> Blas;
// Creates a new top level accelerations structures and sets internal states for builds (e.g update mode)
pub fn create_tlas(&self, desc: &CreateTlasDescriptor) -> Tlas;
}
// Enum for the different types of geometries inside a single blas build
enum BlasGeometries<'a>{
TriangleGeometries(&'a [BlasTriangleGeometry])
ProceduralGeometries(&'a [BlasProceduralGeometry])
}
impl CommandEncoder {
// Build multiple bottom level acceleration structures.
// Validates that the geometries are guarantied to produce an acceleration structure that fits inside the allocated buffers (with strict validation).
// Ensures that all used buffers are valid and synchronized.
pub fn build_blas<'a>(&self, blas: impl IntoIterator<Item=&'a Blas>,
triangle_geometries: impl IntoIterator<Item=&'a [BlasGeometries]>,
scratch_buffers: impl IntoIterator<Item=&'a Buffer>);
// Build multiple top level acceleration structures.
// Validates the instances, (e.g. ensures that blas entries are valid and synchronized)
// Uploads The part of the instances that changed in a staging buffer and
// enqueues a command to copy from that staging buffer into the internal index buffer.
// (Splitting building of bottom level and top level, makes validation easier).
pub fn build_tlas(&self, tlas: impl IntoIterator<Item=&'a Tlas>,
instances: impl IntoIterator<Item=&'a TlasInstances>,
scratch_buffers: impl IntoIterator<Item=&'a Buffer>);
// Build multiple top level acceleration structures.
// Uses the provided instance buffer directly (minimal validation).
pub unsafe fn build_tlas_unsafe(&self, tlas: impl IntoIterator<Item=&'a Tlas>,
raw_instances: impl IntoIterator<Item=&'a Buffer>,
scratch_buffers: impl IntoIterator<Item=&'a Buffer>);
// Creates a new blas and copies (in a compacting way) the contents of the provided blas
// into the new one (compaction flag must be set).
pub fn compact_blas(&self, blas: &Blas) -> Blas;
}
impl BlasRequirements {
// To use the same acceleration structure for multiple blas build (after each build we compact into a new one)
// we need to allocate buffers big enough for all.
// This function allows this, in a safe way (with strict validation enabled).
pub fn find_smallest_shared_requirements(requirements: &[BlasRequirements]) -> BlasRequirements;
// getter for the required scratch_buffer_size
pub fn required_scratch_buffer_size() -> BufferAddress;
}
// trait on blas/tlas
trait AccelerationStructure {
// modify flags before a build
pub fn set_flags_mode(mode: AccelerationStructureFlags);
// modify the update mode before a build
pub fn set_update_mode(mode: UpdateMode);
// getter for the required scratch_buffer_size
pub fn required_scratch_buffer_size() -> BufferAddress;
}
// Safe Tlas Instance
impl TlasInstances{
pub fn new(max_instances: u32) -> Self;
// gets instances to read from
pub fn get(&self) -> &[TlasInstance];
// gets instances to modify, we keep track of the range to determine what we need to validate and copy
pub fn get_mut_range(&mut self, range: Range<u32>) -> &mut [TlasInstance];
// get the number of instances which will be build
pub fn active(&self) -> u32;
// set the number of instances which will be build
pub fn set_active(&mut self, active: u32);
}Previous discussion in a wgpu matrix room thread |
|
Thank you for coming up with a concrete proposal! Is there any way we can reduce the API surface here? it would increase the chances for it to make into an WebGPU extension. For example, would it be reasonable to have the first version of this API not supporting the "update" operation for acceleration structures at all? Without the update, managing the scratch buffers may be simplified - just allocate it when creating an acceleration structure, and then free when it's built, all internally. |
|
I have been looking at the formats acceleration structure building can support, and for the base acceleration structure feature it appears that it should only support f32x3 because this is all metal supports.
There could be another feature that additionally provides f32x2, f16x2, f16x4, snorm16x2, snorm16x4 support which vulkan and dx12 support (and maybe unorm16x2, unorm16x4, rgb10a2, unorm8x2 unorm8x4, snorm8x2, snorm8x4 which dxr 1.1 - needed for ray query - supports and buffer features can be queried on vulkan), however additional format features can probably be post MVP if they are required. From dx12:
From vulkan:
|
|
@Vecvec just f32x3 is good enough. I encourage you to start opening small PRs and getting an MVP merged - ray tracing is something I'm super excited about (for Bevy), and I'm happy that you're working on it, but it doesn't do much good sitting in a fork unfortunately :(. It's too difficult to maintain a forked version of Bevy. |
|
Hello guys. I saw that you support mostly all the inline ray query features. Although, have you had any advance on the ray tracing pipeline and shader binding table (SBT)? Keep it up! |
|
I've previously looked into adding naga support for a basic version of the ray-tracing pipeline (no callable shaders). I have a branch for it that works but I think it is lacking validation in certain parts (at least I haven't checked if it is lacking validation). It's probably quite out of date now, but some other pieces of work are taking priority right now, so in effect it's waiting on #6291. |
Are GLSL shaders still supported in WGPU? I mean, is it possible to extend those RT features without naga supporting them? I ask these questions because I don't know what Naga's tasks are. Cheers. |
|
I don't think it would work because naga would have to convert the shader to the correct backend because no backend that supports GLSL also supports ray-tracing pipelines. WGPU also doesn't support the ray-tracing pipeline which makes the problem even harder. In "theory" you could convert the GLSL shader to spirv using spirv-cross (or something similar) and then code your own raytracing pipeline code using vulkan + wgpu-hal, but at that point it would probably be better to emulate it using software rt (still on the gpu). |
|
Thank you for your response Vecvec. I think I see the whole picture now. RT pipeline + shader stages + shading language functions, inline variables, ... sounds like a LOT. Nevertheless, I am going to drop here a link about how important supporting accelerated by hardware ray tracing is becoming: I am trying to use the compute pipeline + inline ray tracing and directly write onto the swapchain texture and present it. Is it possible to do this currently in WGPU? I haven't seen an example showing this. Best regards. |
Not in current wgpu. There's an open PR (#6291) you can try. I'm not sure what the status of it is, but it's presumably blocked on either code changes or waiting for reviews atm. |
|
Hi @Vecvec, I forked your repo I've being working on trying to get procedural geometries to work for a couple of days by copying previous example (ray_cube_compute) but many validation errors (although those occur in any example I've tried so far, for instance, hello triangle) which prevent me to debug it using Renderdoc or Nvidia Nsight Graphics (they crash), then I am not sure if I am building the acceleration structures properly. No validation errors regarding cmd building accel structs are thrown though and vulkan API dump looks about right to me:
I copied some code from another example I have something like this: but nothing was intersected. Nonetheless I think inline ray query in WGSL is not prepared for this yet. This is the code that I'd like to mimic using WGSL in a compute shader like the following in GLSL: I hope I didn't overwelmed you too much and you could point it out how to go forward. |
|
First of all procedural geometry is not intended for the basic version of the acceleration structures, while these are very important there are three reasons for them not being in the first
I'm not sure why N-Sight doesn't work, I've used it lots to debug ray-tracing. RenderDoc (as of my knowledge) does not yet support debugging raytracing, and so anything run on it will cause an error because it will not show the ray-tracing feature. It says on your vulkan trace |
|
Thank you for answering! I was worried to hear that. I could PR but I though it would be overwelming right for the first attempt as you point out. Branch's in my github. Here is the error that I got at the begining of every example and I think is related to this thread: Anyway, I am not sure if I would be able to extend Naga to support proc geometry, I have no experience doing such task. |
|
You would likely want spirv passthrough feature + a compiler to spirv or this. also you seem to have used my get-vertex branch, the branch that is up to date with trunk is ray-tracing-new, which may have fixed some bugs. The get vertex branch is for additional triangle features which I suspect you do not need |
Alright. I did the following: I compiled my shader using glslangvalidator and then I loaded it like this: As a result I got this errors:
I am afraid I didn't understand what I supposed to do to in order to passthrough unsupported features. |
|
No, that is an error by naga saying it doesn't support reading that feature in your shader, all normal shaders go through naga to convert them to IR (intermediary representation). Shader passthrough is separate from create_shader_module using It's worth noting this limits you to vulkan, but currently Ray-tracing is only supported there. |
|
I could go forward but now I am stuck at matching resources between descriptor layout and shader bindings but I got some errors:
This is rust (vulkan descriptors): let shader;
// Creating a shader module spirv should fail.
unsafe {
shader = device.create_shader_module_spirv(&wgpu::include_spirv_raw!("shader.comp.spv"));
}
let compute_pipeline = device.create_compute_pipeline(&wgpu::ComputePipelineDescriptor {
label: Some("rt"),
layout: None, // <--- should I create a bind group layout previously and pass it as a parameter here?
module: &shader,
entry_point: Some("main"),
compilation_options: Default::default(),
cache: None,
});
let compute_bind_group_layout = compute_pipeline.get_bind_group_layout(0); // <-- it throws an error here
let compute_bind_group = device.create_bind_group(&wgpu::BindGroupDescriptor {
label: None,
layout: &compute_bind_group_layout,
entries: &[
wgpu::BindGroupEntry {
binding: 0,
resource: wgpu::BindingResource::TextureView(&rt_view),
},
wgpu::BindGroupEntry {
binding: 1,
resource: uniform_buf.as_entire_binding(),
},
wgpu::BindGroupEntry {
binding: 2,
resource: wgpu::BindingResource::AccelerationStructure(&tlas),
},
],
});This is WGSL code which works (for triangle version): @group(0) @binding(0)
var output: texture_storage_2d<rgba8unorm, write>;
@group(0) @binding(1)
var<uniform> uniforms: Uniforms;
@group(0) @binding(2)
var acc_struct: acceleration_structure;This is my attemp of GLSL which should work: // Output image at binding 0
layout(set = 0, binding = 0, rgba8) uniform image2D output_image;
// Uniform buffer for inverse view/projection matrices at binding 1
layout(set = 0, binding = 1) uniform Uniforms {
mat4 view_inv;
mat4 proj_inv;
} uniforms;
// Acceleration structure at binding 2
layout(set = 0, binding = 2) uniform accelerationStructureEXT acc_struct;I think wgpu does many things for me when we are using wgsl as a language, like creating and managing descriptors. I am sorry I am asking here many questions but here it goes: Do I need to create those descriptor structs by hand? Thank you for helping. |
|
I think naga auto-generates reflection info, allowing you to have an implict pipeline layout, since this bypasses naga there may be no reflection information. This means you should provide an explicit pipeline layout (in ComputePipelineDescriptor layout). |
|
|
|
Hi @Vecvec. I want to PR my little contribution. Should I try to do it to the RT branch or yours? |
|
I can't speak for @Vecvec, but imo we shouldn't add more stuff to the existing PR. It's already a large PR. |
|
Since I got lavapipe installed I've been testing it with ray-tracing, and it seems that if a section of the intersection struct is accessed that needs a triangle to be hit (not done within the current examples) lavapipe dereferences a null pointer and crashed. I think this is actually our issue, as we gather these members without caring whether we have hit anything or not (this is technically undefined behavior). I think that the best way to deal with this is to emit a function for it for the three different blocks required. |
cwfitzgerald
added
feature: bindless
|
Closing in favor of #6762 |
Khronos has recently released the final specification for ray tracing on Vulkan. At this point, DX12, Vulkan, and Metal all seem to have some form of acceleration for ray tracing. Are there any plans to eventually consolidate and expose these APIs in
wgpuandwgpu-rs?I foresee real-time rendering making increasing use of ray tracing in the future, so this may be an essential feature to have. However, I imagine it may be very difficult to support this on the DX11 and OpenGL backends through software emulation.
The text was updated successfully, but these errors were encountered: