-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MediaStream Destination Node #91
Comments
Hey, as said in #27 , the issue is more about what it would mean to have a handle to a stream in a JS environment (meaning that another library would be able to use the handle and underlying stream properly), rather than just providing a wrapper for the upstream object. |
I don't actually want to handle the MediaStream directly in the JavaScript, I need to hand it off to another library (WebRTC) that is expecting specifically a MediaStream (not a NodeJS Stream). |
Hey @justin0mcateer, thanks for bringing this up. The problem is, like @b-ma states, there is no such thing as the On the web-audio-api-rs side of things I acknowledge this issue, and we solve it by provinding a real-time safe MediaStream shim that allows you to extract the audio samples ( https://docs.rs/web-audio-api/latest/web_audio_api/media_streams/struct.MediaStreamTrack.html#method.iter ) and it allows you to construct an instance from raw audio data ( https://docs.rs/web-audio-api/latest/web_audio_api/media_streams/struct.MediaStreamTrack.html#method.from_iter ). The NodeJs ecosystem should provide the same so application authors will be able to glue libraries such as node-web-audio-api and node-webrtc manually. So perhaps this issue could be rewritten to "How to extract raw audio without resorting to AudioWorkletProcessor"? |
Thanks @orottier for the further details I guess the best solution would be to have some compliant C++/Rust/Node implementation of https://www.w3.org/TR/mediacapture-streams/ that everyone agrees to use as middleware and build upon... but this looks like really non-trivial (and a bit like daydreaming for now :) |
Would it be possible to change the audiocontext destination and convert it to a readable stream? How do we extract a stream from node-web-audio-api without it playing from the speakers of the device? If we can extract the writable stream then in @roamhq/wrtc we could probably use audioSource.onData and pump in the raw chunks. |
Hey, thanks for re-opening the discussion
I'm really not convinced that's the way to go actually. In my opinion, this question must be solved at the lowest possible level and not on the JS side (maybe it could, but it would probably create some kind of problems that nobody wants to spend time on...) As @orottier mentioned, on the rust side this is quite opened as Then, once done creating a JS facade over that should be rather straightforward (hopefully) |
I'm pretty sure that the whole implementation of @roamhq/wrtc is just a wrapper around a C++ API with bindings for Node. |
You mention the MediaStreams API is somewhat out of scope for this library, but there is an implementation (wrapper) of the MediaStream Source Node. Also, the upstream/wrapped crate appears to implement the MediaStream Destination node:
https://github.com/orottier/web-audio-api-rs/blob/main/src/node/media_stream_destination.rs
Aside from perhaps needing to be written manually (vs generated) what are the challenges with creating a MediaStream Desitnation Node wrapper? Is there anyone able to take this on if it were to be sponsored?
The text was updated successfully, but these errors were encountered: