-
Notifications
You must be signed in to change notification settings - Fork 23
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(Call): Dynascale support for Plain-JS SDK (#914)
Introduces `call.bindVideoElement` and `call.bindAudioElement` methods which apply dynascale handlers to the provided video and audio elements, simplifies SDK-specific implementations. ### Breaking Changes - `call.viewportTracker` has been replaced with `call.dynascaleManager` - `call.updateSubscriptionsPartial` - `video` and `screen` kinds are deprecated and will be removed soon. Please switch to `videoTrack` or `screenShareTrack` - `<Audio />` component: - `audioStream` prop is dropped and now it requires a `participant` prop - `sinkId` doesn't have to be provided anymore, as our SDK handles this - `<Video />` - `kind` prop has been replaced with `trackType`. Accepted values: `videoTrack`, `screenShareTrack` and `none` - `<ParticipantView />` - `videoMode` has been renamed to `trackType`. Accepted values: `videoTrack`, `screenShareTrack` and `none` --------- Co-authored-by: Oliver Lazoroski <[email protected]> Co-authored-by: Zita Szupera <[email protected]> Co-authored-by: Zita Szupera <[email protected]> Co-authored-by: Khushal Agarwal <[email protected]>
- Loading branch information
5 people
authored
Aug 30, 2023
1 parent
809f2f7
commit d295fd3
Showing
60 changed files
with
2,032 additions
and
2,987 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
78 changes: 78 additions & 0 deletions
78
...ages/client/docusaurus/docs/javascript/02-guides/11-playing-video-and-audio.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| --- | ||
| title: Playing Video and Audio | ||
| description: Learn how to correctly play participants' video and audio. | ||
| --- | ||
|
|
||
| In this guide, we'll learn how to render and play participants' video and audio by using the JS SDK provided primitives. | ||
|
|
||
| ## Playing Video | ||
|
|
||
| Our JS SDK exposes a low-level method that binds a video element to a participant's video track. | ||
| This method can be found in `call.bindVideoElement`. It takes three arguments: | ||
|
|
||
| - the video element to bind to | ||
| - the participant's `sessionId` | ||
| - the kind of video track to bind to (either `videoTrack` or `screenShareTrack` for screen sharing) | ||
|
|
||
| This method needs to be called only once, usually after the element is mounted in the DOM. | ||
|
|
||
| ```js | ||
| let videoElement = document.getElementById('my-video-element'); | ||
| if (!videoElement) { | ||
| videoElement = document.createElement('video'); | ||
| videoElement.id = 'my-video-element'; | ||
| document.body.appendChild(videoElement); | ||
|
|
||
| // bind the video element to the participant's video track | ||
| // use the returned `unbind()` function to unbind the video element | ||
| const unbind = call.bindVideoElement( | ||
| videoElement, | ||
| participant.sessionId, | ||
| 'videoTrack', | ||
| ); | ||
| } | ||
| ``` | ||
|
|
||
| ### Playing Screen Sharing | ||
|
|
||
| Similar to the _Rendering Video_, a screenshare track can be rendered like this: | ||
|
|
||
| ```js | ||
| let screenElement = document.getElementById('my-screenshare-element'); | ||
| if (!screenElement) { | ||
| screenElement = document.createElement('video'); | ||
| screenElement.id = 'my-screenshare-element'; | ||
| document.body.appendChild(screenElement); | ||
|
|
||
| // bind the video element to the participant's screen track | ||
| // use the returned `unbind()` function to unbind the video element | ||
| const unbind = call.bindVideoElement( | ||
| screenElement, | ||
| participant.sessionId, | ||
| 'screenShareTrack', // note the 'screenShareTrack' argument | ||
| ); | ||
| } | ||
| ``` | ||
|
|
||
| ### Playing Audio | ||
|
|
||
| Our JS SDK exposes a low-level method that binds an audio element to a participant's audio track. | ||
| This method can be found in `call.bindAudioElement`. It takes two arguments: | ||
|
|
||
| - the audio element to bind to | ||
| - the participant's `sessionId` | ||
|
|
||
| This method needs to be called only once, usually after the element is mounted in the DOM. | ||
|
|
||
| ```js | ||
| let audioElement = document.getElementById('my-audio-element'); | ||
| if (!audioElement) { | ||
| audioElement = document.createElement('audio'); | ||
| audioElement.id = 'my-audio-element'; | ||
| document.body.appendChild(audioElement); | ||
|
|
||
| // bind the audio element to the participant's audio track | ||
| // use the returned `unbind()` function to unbind the audio element | ||
| const unbind = call.bindAudioElement(audioElement, participant.sessionId); | ||
| } | ||
| ``` |
85 changes: 85 additions & 0 deletions
85
packages/client/docusaurus/docs/javascript/02-guides/12-visibility-tracking.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| --- | ||
| title: Participant visibility tracking | ||
| description: Learn how to set up participant visibility tracking the screen. | ||
| --- | ||
|
|
||
| Depending on the UI layout of your call, some participants may not be visible on the screen at all times. | ||
| This is especially true for large calls with many participants. We can optimize the performance and bandwidth usage of our SDK | ||
| by only subscribing to video for participants that are visible on the screen or within a certain viewport. | ||
|
|
||
| To help you keep track of which participants are visible on the screen, our SDK provides a few helpers. | ||
| Once you set up visibility tracking to a participant, the client can automatically subscribe and unsubscribe to the video and screen stream of the given participant. | ||
|
|
||
| ## Setup a visibility tracker | ||
|
|
||
| Any DOM element can be tracked for visibility. In a typical scenario, that would be the element that wraps the participant's "box". | ||
| You can register a visibility tracker with the SDK by calling the `call.trackElementVisibility` method: | ||
|
|
||
| ```js | ||
| const myParticipantElement = document.getElementById('my-participant-element'); | ||
|
|
||
| // Track the visibility of the participant's element | ||
| // you can use the returned function to stop tracking an element. | ||
| const untrack = call.trackElementVisibility( | ||
| myParticipantElement, | ||
| participant.sessionId, | ||
| 'videoTrack', // or 'screenShareTrack' if you want to track screen share visibility | ||
| ); | ||
| ``` | ||
|
|
||
| ## Setup a viewport | ||
|
|
||
| In our context, a _Viewport_ is a section/container on the screen that wraps participant's video elements. | ||
| Typically, this is a scrollable container. You can register a viewport with the SDK by calling the `call.setViewport` method: | ||
|
|
||
| ```js | ||
| const viewport = document.getElementById('my-participant-container'); | ||
|
|
||
| // sets the viewport | ||
| // you can use the returned function to unset the viewport. | ||
| const unset = call.setViewport(viewport); | ||
| ``` | ||
|
|
||
| ## Access the visible participants | ||
|
|
||
| Every participant can have three visibility states for its video and screen share tracks: | ||
|
|
||
| - `VISIBLE` - the track is visible on the screen | ||
| - `INVISIBLE` - the track is not visible on the screen | ||
| - `UNKNOWN` - the track's visibility is unknown (e.g. the participant is not tracked) | ||
|
|
||
| Visibility state is available in the participant's state: | ||
|
|
||
| ```js | ||
| import { VisibilityState } from '@stream-io/video-client'; | ||
|
|
||
| call.state.participants.forEach((participant) => { | ||
| const { viewportVisibilityState } = participant; | ||
|
|
||
| // The participant's video visibility in the viewport | ||
| switch (viewportVisibilityState.videoTrack) { | ||
| case VisibilityState.VISIBLE: | ||
| // The participant's video track is visible in the viewport | ||
| break; | ||
| case VisibilityState.INVISIBLE: | ||
| // The participant's video track is not visible in the viewport | ||
| break; | ||
| case VisibilityState.UNKNOWN: | ||
| // The participant's video track visibility in the viewport is unknown | ||
| break; | ||
| } | ||
|
|
||
| // The participant's screen share visibility in the viewport | ||
| switch (viewportVisibilityState.screenShareTrack) { | ||
| case VisibilityState.VISIBLE: | ||
| // The participant's screen share track is visible in the viewport | ||
| break; | ||
| case VisibilityState.INVISIBLE: | ||
| // The participant's screen share track is not visible in the viewport | ||
| break; | ||
| case VisibilityState.UNKNOWN: | ||
| // The participant's screen share track visibility in the viewport is unknown | ||
| break; | ||
| } | ||
| }); | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.