Merge branch 'release/rtc-ng/4.2.3' of github.com:AgoraIO/agora_doc_s…

…ource into release/rtc-ng/4.2.3

markdown/RTC 4.x/release-notes/en-US/framework/release_electron_ng.md

@@ -1,3 +1,86 @@
+## v4.2.3
+
+v4.2.3 was released on September xx, 2023.
+
+#### New features
+
+1. **Update video screenshot and upload**
+
+   To facilitate the integration of third-party video moderation services from Agora Extensions Marketplace, this version has the following changes:
+
+   - An optional parameter `serverConfig` is added in `ContentInspectConfig`, which is for server-side configuration related to video screenshot and upload via extensions from Agora Extensions Marketplace. By configuring this parameter, you can integrate multiple third-party moderation extensions and achieve flexible control over extension switches and other features. For more details, please contact [technical support](mailto:[email protected]).
+
+   In addition, this version also introduces the `enableContentInspectEx` method, which supports taking screenshots for multiple video streams and uploading them.
+
+2. **ID3D11Texture2D Rendering**
+
+  As of this release, the SDK supports video formats of type ID3D11Texture2D, improving the rendering effect of video frames in game scenarios. You can set **format** to `VideoTextureId3d11texture2d` when pushing external raw video frames to the SDK by calling `pushVideoFrame`. By setting the **d3d11_texture_2d** and **texture_slice_index** properties, you can determine the ID3D11Texture2D texture object to use.
+
+3. **Local video status error code update**
+
+  In order to help users understand the exact reasons for local video errors in screen sharing scenarios, the following sets of enumerations have been added to the `onLocalVideoStateChanged` callback:
+
+   - `LocalVideoStreamErrorScreenCapturePaused`(23): Screen capture has been paused. Common scenarios for reporting this error code: The current screen may have been switched to a secure desktop, such as a UAC dialog box or Winlogon desktop.
+   - `LocalVideoStreamErrorScreenCaptureResumed`(24): Screen capture has resumed from the paused state.
+   - `LocalVideoStreamErrorScreenCaptureWindowHidden`(25): The window being captured on the current screen is in a hidden state and is not visible on the current screen.
+   - `LocalVideoStreamErrorScreenCaptureWindowRecoverFromHidden`(26): The window for screen capture has been restored from the hidden state.
+   - `LocalVideoStreamErrorScreenCaptureWindowRecoverFromMinimized`(27): The window for screen capture has been restored from the minimized state.
+
+4. **Check device support for advanced features**
+
+   This version adds the `isFeatureAvailableOnDevice` method to check whether the capability of the current device meet the requirements of the specified advanced feature, such as virtual background and image enhancement.
+
+   Before using advanced features, you can check whether the current device supports these features based on the call result. This helps to avoid performance degradation or unavailable features when enabling advanced features on low-end devices. Based on the return value of this method, you can decide whether to display or enable the corresponding feature button, or notify the user when the device's capabilities are insufficient.
+
+   In addition, since this version, calling `enableVirtualBackground` and `setBeautyEffectOptions` automatically triggers a test on the capability of the current device. When the device is considered underperformed, the error code `-4: ErrNotSupported` is returned, indicating the device does not support the feature.  
+
+#### Improvements
+
+1. **Optimize virtual background memory usage**
+
+   This version has upgraded the virtual background algorithm, reducing the memory usage of the virtual background feature. Compared to the previous version, the memory consumption of the app during the use of the virtual background feature on low-end devices has been reduced by approximately 4% to 10% (specific values may vary depending on the device model and platform).
+
+2. **Screen sharing scenario optimization**
+
+   This release optimizes the performance and encoding efficiency in ultra-high-definition (4K, 60 fps) game sharing scenarios, effectively reducing the system resource usage during screen sharing.
+
+**Other Improvements**
+
+This release includes the following additional improvements:
+
+- Optimizes the logic of handling invalid parameters. When you call the `setPlaybackSpeed` method to set the playback speed of audio files, if you pass an invalid parameter, the SDK returns the error code -2, which means that you need to reset the parameter.
+- Optimizes the logic of Token parsing, in order to prevent an app from crash when an invalid token is passed in.
+
+#### Issues fixed
+
+This release fixed the following issues:
+
+- Occasional crashes and dropped frames occured in screen sharing scenarios. (macOS)
+- Occasional crashes when joining a channel. (macOS)
+- When calling the `playEffect` method to play two audio files using the same `soundId`, the first audio file was somtimes played repeatedly.
+- Calling `takeSnapshotEx` once receives the `onSnapshotTaken` callback for multiple times.
+
+#### API changes
+
+**Added**
+
+- `enableContentInspectEx`
+- `contentInspectImageModeration` in `ContentInspectType`
+- `serverConfig` in `ContentInspectConfig` 
+- `onLocalVideoStateChanged` adds the following enumerations:
+
+  - `LocalVideoStreamErrorScreenCapturePaused`
+  - `LocalVideoStreamErrorScreenCaptureResumed`
+  - `LocalVideoStreamErrorScreenCaptureWindowHidden`
+  - `LocalVideoStreamErrorScreenCaptureWindowRecoverFromHidden`
+  - `LocalVideoStreamErrorScreenCaptureWindowRecoverFromMinimized`
+
+- `d3d11_texture_2d` and `texture_slice_index` in `ExternalVideoFrame` 
+
+- `VideoTextureId3d11texture2d` in `VideoPixelFormat` 
+- `isFeatureAvailableOnDevice`
+- `FeatureType`
+
 ## v4.2.2
 
 v4.2.2 was released on July xx, 2023.
@@ -20,7 +103,7 @@ v4.2.2 was released on July xx, 2023.
 
    In this release, the `backgroundColor` member has been added to `VideoCanvas`, which allows you to customize the background color of the video canvas when setting the properties of local or remote video display.
 
-4. **Publishing video streams from different sources** (Windows, macOS)
+4. **Publishing video streams from different sources**
 
    This release adds the following members in `ChannelMediaOptions` to allow you publish video streams captured from the third and fourth camera or screen:
 

markdown/RTC 4.x/release-notes/en-US/framework/release_rn_ng.md

@@ -1,3 +1,125 @@
+## v4.2.3
+
+v4.2.3 was released on September xx, 2023.
+
+#### New features
+
+1. **Update video screenshot and upload**
+
+   To facilitate the integration of third-party video moderation services from Agora Extensions Marketplace, this version has the following changes:
+
+   - An optional parameter `serverConfig` is added in `ContentInspectConfig`, which is for server-side configuration related to video screenshot and upload via extensions from Agora Extensions Marketplace. By configuring this parameter, you can integrate multiple third-party moderation extensions and achieve flexible control over extension switches and other features. For more details, please contact [technical support](mailto:[email protected]).
+
+   In addition, this version also introduces the `enableContentInspectEx` method, which supports taking screenshots for multiple video streams and uploading them.
+
+2. **Check device support for advanced features**
+
+   This version adds the `isFeatureAvailableOnDevice` method to check whether the capability of the current device meet the requirements of the specified advanced feature, such as virtual background and image enhancement.
+
+   Before using advanced features, you can check whether the current device supports these features based on the call result. This helps to avoid performance degradation or unavailable features when enabling advanced features on low-end devices. Based on the return value of this method, you can decide whether to display or enable the corresponding feature button, or notify the user when the device's capabilities are insufficient.
+
+   In addition, since this version, calling `enableVirtualBackground` and `setBeautyEffectOptions` automatically triggers a test on the capability of the current device. When the device is considered underperformed, the error code `-4: ErrNotSupported` is returned, indicating the device does not support the feature.  
+
+3. **Wildcard token**
+
+   This release introduces wildcard tokens. Agora supports setting the channel name used for generating a token as a wildcard character. The token generated can be used to join any channel if you use the same user id. In scenarios involving multiple channels, such as switching between different channels, using a wildcard token can avoid repeated application of tokens every time users joining a new channel, which reduces the pressure on your token server. See [Wildcard tokens](https://docportal.shengwang.cn/cn/live-streaming-premium-4.x/wildcard_token?platform=All%20Platforms).
+
+   <div class="alert note">All 4.x SDKs support using wildcard tokens.</div>
+
+4. **Preloading channels**
+
+   This release adds `preloadChannel` and `preloadChannelWithUserAccount` methods, which allows a user whose role is set as audience to preload channels before joining one. Calling the method can help shortening the time of joining a channel, thus reducing the time it takes for audience members to hear and see the host.
+
+   When preloading more than one channels, Agora recommends that you use a wildcard token for preloading to avoid repeated application of tokens every time you joining a new channel, thus saving the time for switching between channels. See [Wildcard tokens](https://docportal.shengwang.cn/cn/live-streaming-premium-4.x/wildcard_token?platform=All%20Platforms).
+
+5. **Customized background color of video canvas**
+
+   In this release, the `backgroundColor` member has been added to `VideoCanvas`, which allows you to customize the background color of the video canvas when setting the properties of local or remote video display.
+
+
+#### Improvements
+
+1. **Improved camera capture effect**
+
+   This release has improved camera capture effect in the following aspects:
+
+   1. Support for camera exposure adjustment
+
+      This release adds `isCameraExposureSupported` to query whether the device supports exposure adjustment and `setCameraExposureFactor` to set the exposure ratio of the camera.
+
+   2. Optimization of default camera selection (iOS)
+
+      Since this release, the default camera selection behavior of the SDK is aligned with that of the iOS system camera. If the device has multiple rear cameras, better shooting perspectives, zooming capabilities, low-light performance, and depth sensing can be achieved during video capture, thereby improving the quality of video capture.
+
+
+2. **Virtual Background Algorithm Upgrade**
+
+   This version has upgraded the portrait segmentation algorithm of the virtual background, which comprehensively improves the accuracy of portrait segmentation, the smoothness of the portrait edge with the virtual background, and the fit of the edge when the person moves. In addition, it optimizes the precision of the person's edge in scenarios such as meetings, offices, homes, and under backlight or weak light conditions.
+
+   This version reduces the memory usage of the virtual background feature. Compared to the previous version, the memory consumption of the app during the use of the virtual background feature on low-end devices has been reduced by approximately 4% to 10% (specific values may vary depending on the device model and platform).
+
+3. **Screen sharing scenario optimization**
+
+   This release also optimizes the video encoding configuration in screen sharing scenarios. When users customize the `width` and `height` properties of the video, the SDK rounds down the actual encoding resolution while maintaining the aspect ratio of the video and the screen, ensuring that the final encoding resolution does not exceed the user-defined encoding resolution, thereby improving the accuracy of billing for screen sharing streams.
+
+4. **Channel media relay**
+
+   The number of target channels for media relay has been increased to 6. When calling `startOrUpdateChannelMediaRelay` and `startOrUpdateChannelMediaRelayEx`, you can specify up to 6 target channels.
+
+5. **Enhancement in video codec query capability**
+
+   To improve the video codec query capability, this release adds the `codecLevels` member in `CodecCapInfo`. After successfully calling `queryCodecCapability`, you can obtain the hardware and software decoding capability levels of the device for H.264 and H.265 video formats through `codecLevels`.
+
+
+**Other Improvements**
+
+This release includes the following additional improvements:
+
+- Optimizes the logic of handling invalid parameters. When you call the `setPlaybackSpeed` method to set the playback speed of audio files, if you pass an invalid parameter, the SDK returns the error code -2, which means that you need to reset the parameter.
+- Optimizes the logic of Token parsing, in order to prevent an app from crash when an invalid token is passed in.
+- To improve the switching experience between multiple audio routes, this release adds the `setRouteInCommunicationMode` method. This method can switch the audio route from a Bluetooth headphone to the earpiece, wired headphone or speaker in communication volume mode ([`MODE_IN_COMMUNICATION`](https://developer.android.google.cn/reference/kotlin/android/media/AudioManager?hl=en#mode_in_communication)). (Android)
+- The SDK automacially adjusts the frame rate of the sending end based on the screen sharing scenario. Especially in document sharing scenarios, this feature avoids exceeding the expected video bitrate on the sending end to improve transmission efficiency and reduce network burden.
+- To help users understand the reasons for more types of remote video state changes, the `remoteVideoStateReasonCodecNotSupport` enumeration has been added to the `onRemoteVideoStateChanged` callback, indicating that the local video decoder does not support decoding the received remote video stream.
+
+#### Issues fixed
+
+This release fixed the following issues:
+
+- Occasional failure of joining a channel when the local system time was not set correctly.
+- When calling the `playEffect` method to play two audio files using the same `soundId`, the first audio file was somtimes played repeatedly.
+- Calling `takeSnapshotEx` once receives the `onSnapshotTaken` callback for multiple times.
+- When the host called the `startAudioMixing` method to play music, sometimes the host couldn't hear the music while the remote users could hear it. (Android)
+- Occasional crashes occured on certain Android devices. (Android)
+- In channels joined by calling `joinChannelEx` exclusively, calling `setEnableSpeakerphone` is unable to switch audio route from the speaker to the headphone. (Android)
+- Occasionally, noise occurred when the local user listened to their own and remote audio after joining the channel. (macOS)
+- Slow channel reconnection after the connection was interrupted due to network reasons.
+- In screen sharing scenarios, the delay of seeing the shared screen was occasionally higher than expected on some devices.
+- In custom video capturing scenarios, `setBeautyEffectOptions`, `setLowlightEnhanceOptions`, `setVideoDenoiserOptions`, and `setColorEnhanceOptions` could not load extensions automatically.
+
+
+#### API changes
+
+**Added**
+
+- `enableContentInspectEx`
+- `contentInspectImageModeration` in `ContentInspectType`
+- `serverConfig` in `ContentInspectConfig` 
+- `isFeatureAvailableOnDevice`
+- `FeatureType`
+- `setCameraExposureFactor` 
+- `isCameraExposureSupported` 
+- `preloadChannel`
+- `preloadChannelWithUserAccount`
+- `updatePreloadChannelToken`
+- `setRouteInCommunicationMode` (Android)
+- `CodecCapLevels`
+- `VideoCodecCapabilityLevel`
+- `backgroundColor` in `VideoCanvas`
+- `codecLevels` in `CodecCapInfo`
+- `remoteVideoStateReasonCodecNotSupport` in `RemoteVideoStateReason`
+
+
+
 ## v4.2.1
 
 This version was released on June 21, 2023.