What is the recommended way for embedding WebGPU in an existing event loop?

[zcbenz]

The WGPUFuture issue (#199) mentioned providing solutions for embedding WebGPU in an existing event loop, like the GPU process of Chromium, or an GUI app. (In my case, I'm adding WebGPU as a new backend to an machine learning framework.)

I suppose the WaitAny is the answer, i.e. I pass the futures to a polling thread, which calls WaitAny and notifies the main thread if any future is completed. However I can not find any living example do that, all the code I found in Chromium were just tests using it to wait for results in the same thread.

Is WaitAny supposed to support my use case? And is there any existing project actually using it for embedding WebGPU?

[kainino0x]

It's unfortunately a little bit theoretical still! We have not yet done the work to try to integrate this into Chromium's event loop yet; some people have been looking at Kotlin bindings, but I don't know if they have an integration into the Kotlin runtime yet.

With just WaitAny, you do indeed need a thread to sit and wait on things, though unfortunately we don't yet really have a way to interrupt WaitAny early so you can add more futures to the list to wait on, so it still needs to have unnecessary wakes.

There's a more integrated idea where we'll expose these as OS events, like D3D does. Explained here:
https://docs.google.com/document/d/1qJRTJRY318ZEqhK6ryw4P-91FumYQfOnNP6LpANYy6A/edit?tab=t.0#heading=h.o77zi44jdz5a
We have a pretty good idea of how this will work (polyfilled with a background thread where it can't be supported directly) but we haven't tried to expose it as an API yet in Dawn.

[kainino0x]

unfortunately we don't yet really have a way to interrupt WaitAny early

I filed #495 about this because we should have it.

[kainino0x]

There's a more integrated idea where we'll expose these as OS events, like D3D does.

And #496 about this.

[zcbenz]

I have been playing with the native dawn APIs to implement the idea of polling in a thread, and I think the current public APIs are quite cumbersome for the purpose.

To use WaitAny for polling I have to store all the futures created in main thread, and erase the completed ones in the polling thread, which requires synchronization and manual bookkeeping, while all the information are already in dawn::native::EventManager. By using the native dawn APIs I can easily implement polling with just while (true) { GetEventManager()->ProcessPollEvents(); }.

Would it be possible to expose EventManager::ProcessPollEvents as a public API? I also tried to use wgpuInstanceProcessEvents but it does not support running in multi-threaded env because it ticks device.

(Which led me to another question: is it fine if my app never calls wgpuInstanceProcessEvents or wgpuDeviceTick? If it is not, when should they be called?)

[kainino0x]

cc @lokokung who knows more about Dawn.

I also tried to use wgpuInstanceProcessEvents but it does not support running in multi-threaded env because it ticks device.

That's a deficiency of Dawn's multithreading support (which is not very good); this header says it should work.
I actually thought this would be fine already in Dawn but maybe Tick is not mutexed.

By using the native dawn APIs I can easily implement polling with just while (true) { GetEventManager()->ProcessPollEvents(); }.

Beware of two things:

• This is a Dawn internal thing, not part of its API, so it's not stable
• Doing a spin loop like this is probably very inefficient? I don't think this function sleeps at any point, so this is a spin loop. I'm pretty sure that's true (in the API contract) of wgpuInstanceProcessEvents(), which is meant to be used more like glfwPollEvents() where you call it when you have an opportunity, like at the end of your main loop.

(Which led me to another question: is it fine if my app never calls wgpuInstanceProcessEvents or wgpuDeviceTick? If it is not, when should they be called?)

In theory yes, it's fine, as long as every event is either waited or set to Spontaneous. (I'm sure I've written this down somewhere, but I don't see it in the docs, so we need to document this. (EDIT: #498))

In Dawn right now I'm not sure, it might be possible for things to get stuck if you don't call something that Ticks (ProcessEvents, Tick, Submit, etc.)

[zcbenz]

Thanks for your insights!

That's a deficiency of Dawn's multithreading support (which is not very good); this header says it should work. I actually thought this would be fine already in Dawn but maybe Tick is not mutexed.

Calling wgpuInstanceProcessEvents in polling thread resulted in a crash for me, not sure if it is a bug.

Error: Assertion failure at /Users/zcbenz/codes/betann/dawn/src/dawn/native/metal/CommandRecordingContext.mm:150 (BeginCompute): mCommands != nullptr

Doing a spin loop like this is probably very inefficient? I don't think this function sleeps at any point, so this is a spin loop.

Ahh I didn't realize the method does not actually wait!

From embedder's side I want a simple API that polls all futures, and another API that interrupts the polling. I'm currently using a custom event for interruption and bookkeeping all the futures created, which is really tedious:

  struct InterruptEvent final : public dawn::native::EventManager::TrackedEvent {
   ...
  };

[kainino0x]

Jan 23 meeting:

• See also: notes on Futures: There should be a way to interrupt wgpuInstanceWaitAny via CPU-timeline event #495
• Discuss: Do we actually need WaitAny(timeout=0)? Should ProcessEvents() take a timeout?
  • LK: Right now if someone wants to run their own event loop thread, they have to do one of two bad options:
  • track all futures so they can wait on them
    • do a spin-loop on ProcessEvents
  • LK: Do we even need WaitAny(timeout=0)?
    • In wasm it won't do anything if it doesn't yield to the JS event loop (which you can't without asyncify/jspi)
    • KN: Only lets you poll whether something happened during a previous yield.
  • KN: Q is basically is polling useful. It's never great, because it always has latency.
    • In design doc, likened ProcessEvents to glfwPollEvents. Call it once per frame, find out what happened. But are our events analogous?
  • LK: Want to add a timeout to ProcessEvents. Otherwise you have to track all futures, nice for us to do it for you.
  • KN: Could give a list of futures in the ProcessEvents list?? (LK: Requires FreeMembers)
  • KN: Can imagine reworking whole thing so there can me multiple ProcessEvents pools. But that's not really easier to use than just tracking every future manually.
  • LK: Another issue is ProcessEvents would now need to care about the limit of 64. Lots of events will coalesce into one queue wait, but not everything.
  • KN: Good point, feel we'd be obligated to make that work, like with the Windows thread pool thing
  • LK: ProcessEvents would now also have the problem of mixed source waiting.
  • KN: Lot of hurdles to be able to add a timeout to ProcessEvents.
  • LK: Spontaneous also requires a thread to be able to guarantee things happen?
    • KN: Only guarantee if application is doing stuff with the API (processevents, submit, etc.), or using WaitAny.
    • LK: Think we should make AllowSpontaneous have better guarantees
    • KN: We could do that later, for now not sure it's needed
  • …
  • LK: Instance callback for every time a new future is created?

[kainino0x]

• KN: Good point, feel we'd be obligated to make that work, like with the Windows thread pool thing

https://learn.microsoft.com/en-us/windows/win32/api/synchapi/nf-synchapi-waitformultipleobjects#:~:text=To%20wait%20on%20more%20than%20MAXIMUM_WAIT_OBJECTS%20handles%2C%20use%20one%20of%20the%20following%20methods