Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

4.2.3 ld #3334

Merged
merged 11 commits into from
Sep 20, 2023
Merged

4.2.3 ld #3334

Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
32a0b0d
add
Sep 12, 2023
85c5b39
1
Sep 13, 2023
096886f
1
jinyuagora Sep 18, 2023
284bd05
4.2.3 ✌️
Suri539 Sep 19, 2023
6df170b
Revert "4.2.3 ✌️"
Suri539 Sep 19, 2023
7028d4b
Merge branch '4.2.3-ld' of github.com:AgoraIO/agora_doc_source into 4…
Sep 19, 2023
5a65a31
1
Sep 19, 2023
2eabc49
Update class_externalvideoframe.dita
jinyuagora Sep 19, 2023
c968df5
Update api_imediaengine_pushvideoframe.dita
Sep 19, 2023
2c2e950
Merge branch '4.2.3-ld' of github.com:AgoraIO/agora_doc_source into 4…
Sep 19, 2023
dec16e4
review comments
Sep 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions dita/RTC-NG/API/api_imediaengine_pushvideoframe.dita
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,10 @@
<p id="desc1"> <ph props="android ios mac">调用 <xref keyref="setExternalVideoSource"/> 方法，设置 <parmname>enabled</parmname> 参数为 <codeph><ph keyref="true"/></codeph>、<parmname>encodedFrame</parmname> 参数为 <codeph><ph keyref="false"/></codeph> 后，你可以调用本方法将未编码的外部视频帧推送到 SDK。</ph> <ph props="unity cpp unreal bp flutter rn electron cs">调用 <xref keyref="createCustomVideoTrack"/> 方法获得视频轨道 ID，在每个频道的 <xref keyref="ChannelMediaOptions"/> 中，将 <parmname>customVideoTrackId</parmname> 参数设置为你想要发布的视频轨道 ID，并将 <parmname>publishCustomVideoTrack</parmname> 设置为 <codeph><ph keyref="true"/></codeph>后，你可以调用本方法将未编码的外部视频帧推送到 SDK。</ph></p>
<p props="android">调用该方法或 <xref keyref="pushVideoFrame2"/> 均能将视频帧数据传递给SDK。区别为 <apiname keyref="pushVideoFrame"/> 方法不支持 texture 格式的视频数据。</p>
</section>
<section props="cpp">
<title>适用场景</title>
<p>SDK 自 v4.2.3 起支持 ID3D11Texture2D 类型的视频格式，该类型在游戏场景下应用较为广泛。当你需要将该类型的视频帧推送的 SDK 时，需要调用该方法，并设置 <parmname>frame</parmname> 中的 <parmname>format</parmname> 为 <codeph>VIDEO_TEXTURE_ID3D11TEXTURE2D</codeph>、设置 <parmname>*d3d11_texture_2d</parmname> 和 <parmname>texture_slice_index</parmname> 成员，将视频帧的格式设置为 ID3D11Texture2D。</p>
</section>
<section id="parameters">
<title>参数</title>
<parml>
Expand Down
154 changes: 88 additions & 66 deletions dita/RTC-NG/API/class_externalvideoframe.dita
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,49 +50,54 @@
@property(assign, nonatomic) int rotation;
@end</codeblock>
<codeblock props="cpp unreal bp" outputclass="language-cpp">struct ExternalVideoFrame {
ExternalVideoFrame()
: type(VIDEO_BUFFER_RAW_DATA),
format(VIDEO_PIXEL_DEFAULT),
buffer(NULL),
stride(0),
height(0),
cropLeft(0),
cropTop(0),
cropRight(0),
cropBottom(0),
rotation(0),
timestamp(0),
eglContext(NULL),
eglType(EGL_CONTEXT10),
textureId(0),
metadata_buffer(NULL),
metadata_size(0){}
enum EGL_CONTEXT_TYPE {
EGL_CONTEXT10 = 0,
EGL_CONTEXT14 = 1,
};
enum VIDEO_BUFFER_TYPE {
VIDEO_BUFFER_RAW_DATA = 1,
VIDEO_BUFFER_ARRAY = 2,
VIDEO_BUFFER_TEXTURE = 3,
};
VIDEO_BUFFER_TYPE type;
VIDEO_PIXEL_FORMAT format;
void* buffer;
int stride;
int height;
int cropLeft;
int cropTop;
int cropRight;
int cropBottom;
int rotation;
long long timestamp;
void *eglContext;
EGL_CONTEXT_TYPE eglType;
int textureId;
uint8_t* metadata_buffer;
int metadata_size;
};</codeblock>
ExternalVideoFrame()
: type(VIDEO_BUFFER_RAW_DATA),
format(VIDEO_PIXEL_DEFAULT),
buffer(NULL),
stride(0),
height(0),
cropLeft(0),
cropTop(0),
cropRight(0),
cropBottom(0),
rotation(0),
timestamp(0),
eglContext(NULL),
eglType(EGL_CONTEXT10),
textureId(0),
metadata_buffer(NULL),
metadata_size(0),
d3d11_texture_2d(NULL),
texture_slice_index(0){}
enum EGL_CONTEXT_TYPE {
EGL_CONTEXT10 = 0,
EGL_CONTEXT14 = 1,
};
enum VIDEO_BUFFER_TYPE {
VIDEO_BUFFER_RAW_DATA = 1,
VIDEO_BUFFER_ARRAY = 2,
VIDEO_BUFFER_TEXTURE = 3,
};
VIDEO_BUFFER_TYPE type;
VIDEO_PIXEL_FORMAT format;
void* buffer;
int stride;
int height;
int cropLeft;
int cropTop;
int cropRight;
int cropBottom;
int rotation;
long long timestamp;
void *eglContext;
EGL_CONTEXT_TYPE eglType;
int textureId;
float matrix[16];
uint8_t* metadata_buffer;
int metadata_size;
void *d3d11_texture_2d;
int texture_slice_index;
};</codeblock>
<codeblock props="electron" outputclass="language-typescript">export class ExternalVideoFrame {
type?: VideoBufferType;
format?: VideoPixelFormat;
Expand Down Expand Up @@ -190,8 +195,8 @@
metadata_buffer?: Uint8Array;
metadata_size?: number;
}</codeblock>
<codeblock props="flutter" outputclass="language-dart">class ExternalVideoFrame {

<codeblock props="flutter" outputclass="language-dart">@JsonSerializable(explicitToJson: true, includeIfNull: false)
class ExternalVideoFrame {
const ExternalVideoFrame(
{this.type,
this.format,
Expand All @@ -208,60 +213,68 @@
this.textureId,
this.matrix,
this.metadataBuffer,
this.metadataSize});
this.metadataSize,
this.alphaBuffer,
this.textureSliceIndex});

@JsonKey(name: 'type')
@JsonKey(name: &#x27;type&#x27;)
final VideoBufferType? type;

@JsonKey(name: 'format')
@JsonKey(name: &#x27;format&#x27;)
final VideoPixelFormat? format;

@JsonKey(name: 'buffer', ignore: true)
@JsonKey(name: &#x27;buffer&#x27;, ignore: true)
final Uint8List? buffer;

@JsonKey(name: 'stride')
@JsonKey(name: &#x27;stride&#x27;)
final int? stride;

@JsonKey(name: 'height')
@JsonKey(name: &#x27;height&#x27;)
final int? height;

@JsonKey(name: 'cropLeft')
@JsonKey(name: &#x27;cropLeft&#x27;)
final int? cropLeft;

@JsonKey(name: 'cropTop')
@JsonKey(name: &#x27;cropTop&#x27;)
final int? cropTop;

@JsonKey(name: 'cropRight')
@JsonKey(name: &#x27;cropRight&#x27;)
final int? cropRight;

@JsonKey(name: 'cropBottom')
@JsonKey(name: &#x27;cropBottom&#x27;)
final int? cropBottom;

@JsonKey(name: 'rotation')
@JsonKey(name: &#x27;rotation&#x27;)
final int? rotation;

@JsonKey(name: 'timestamp')
@JsonKey(name: &#x27;timestamp&#x27;)
final int? timestamp;

@JsonKey(name: 'eglType')
@JsonKey(name: &#x27;eglType&#x27;)
final EglContextType? eglType;

@JsonKey(name: 'textureId')
@JsonKey(name: &#x27;textureId&#x27;)
final int? textureId;

@JsonKey(name: 'matrix')
final List&lt;double>? matrix;
@JsonKey(name: &#x27;matrix&#x27;)
final List&lt;double&gt;? matrix;

@JsonKey(name: 'metadata_buffer', ignore: true)
@JsonKey(name: &#x27;metadata_buffer&#x27;, ignore: true)
final Uint8List? metadataBuffer;

@JsonKey(name: 'metadata_size')
@JsonKey(name: &#x27;metadata_size&#x27;)
final int? metadataSize;

factory ExternalVideoFrame.fromJson(Map&lt;String, dynamic> json) =>
@JsonKey(name: &#x27;alphaBuffer&#x27;, ignore: true)
final Uint8List? alphaBuffer;

@JsonKey(name: &#x27;texture_slice_index&#x27;)
final int? textureSliceIndex;

factory ExternalVideoFrame.fromJson(Map&lt;String, dynamic&gt; json) =&gt;
_$ExternalVideoFrameFromJson(json);

Map&lt;String, dynamic> toJson() => _$ExternalVideoFrameToJson(this);
Map&lt;String, dynamic&gt; toJson() =&gt; _$ExternalVideoFrameToJson(this);
}</codeblock>
<codeblock props="reserve" outputclass="language-cpp"></codeblock></p>
</section>
Expand Down Expand Up @@ -401,6 +414,15 @@
<pt props="flutter">metadataSize</pt>
<pd>该参数仅适用于 Texture 格式的视频数据。指 MetaData 的大小，默认值为 <codeph>0</codeph>。</pd>
</plentry>
<plentry props="cpp unity electron">
<pt>*d3d11_texture_2d</pt>
<pd>该参数仅适用于 Windows Texture 格式的视频数据。为一个指向 <codeph>ID3D11Texture2D</codeph> 类型对象的指针，该类型对象被视频帧所使用。</pd>
</plentry>
<plentry props="cpp unity electron flutter">
<pt props="cpp unity electron">texture_slice_index</pt>
<pt props="flutter">textureSliceIndex</pt>
<pd>该参数仅适用于 Windows Texture 格式的视频数据。表示在 <codeph>ID3D11Texture2D</codeph> 数组中，被视频帧使用的某一个 <codeph>ID3D11Texture2D</codeph> 纹理对象的索引。</pd>
</plentry>
<plentry>
<pt>cropLeft</pt>
<pd>原始数据相关字段。指定左边裁剪掉的像素数量。默认为 0。</pd>
Expand All @@ -415,7 +437,7 @@
</plentry>
<plentry>
<pt>cropRight</pt>
<pd> 原始数据相关字段。指定右边裁剪掉的像素数量。默认为 0。</pd>
<pd>原始数据相关字段。指定右边裁剪掉的像素数量。默认为 0。</pd>
<pd props="android">该参数仅适用于非 Texture 格式的视频数据。</pd>
<pd props="cpp unreal bp unity electron rn flutter cs">该参数仅适用于原始视频数据。</pd>
</plentry>
Expand Down
2 changes: 1 addition & 1 deletion dita/RTC-NG/API/class_screenvideoparameters.dita
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@
</plentry>
<plentry props="android">
<pt>height</pt>
<pd>视频编码分辨率中的高 (px)。默认值为 720。如果 width 和 height 的宽高比与屏幕像素值 的宽高比不一致，SDK 会按照如下规则调节视频编码分辨率，以 width × height 是 1280 × 720 为例：
<pd>视频编码分辨率中的高 (px)。默认值为 720。如果 width 和 height 的宽高比与屏幕像素值的宽高比不一致，SDK 会按照如下规则调节视频编码分辨率，以 width × height 是 1280 × 720 为例：
<ul>
<li>当屏幕宽和高的像素值均小于 width 和 height 时，例如屏幕像素值为 640 × 360，SDK 以 640 × 360 编码。</li>
<li>当屏幕宽或高的像素值大于 width 或 height 时，例如屏幕像素值为 2000 × 1500，SDK 以屏幕像素值的宽高比，即 4:3，取 width × height 内最大分辨率进行编码，即 960 × 720。</li>
Expand Down
4 changes: 2 additions & 2 deletions dita/RTC-NG/API/enum_localaudiostreamerror.dita
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,9 @@
<pt><ph keyref="LOCAL_AUDIO_STREAM_ERROR_NO_PLAYOUT_DEVICE"/></pt>
<pd>7:（仅适用于 Windows）无本地音频播放设备。请提示用户在设备的控制面板中检查扬声器是否与设备连接正常，检查扬声器是否正常工作。</pd>
</plentry>
<plentry props="cpp unreal bp unity flutter rn cs">
<plentry props="cpp unreal bp ios unity flutter rn cs">
<pt><ph keyref="LOCAL_AUDIO_STREAM_ERROR_INTERRUPTED"/> </pt>
<pd props="cpp unreal bp unity flutter">8:（仅适用于 Android 和 iOS）本地音频采集被系统来电、Siri、闹钟中断。如需恢复本地音频采集，请用户中止电话、Siri、闹钟。</pd>
<pd props="cpp unreal bp ios unity flutter">8:<ph props="cpp unreal bp unity flutter">（仅适用于 Android 和 iOS）</ph>本地音频采集被系统来电、Siri、闹钟中断。如需恢复本地音频采集，请用户中止电话、Siri、闹钟。</pd>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里应该还漏了一个 rn 的 props，上面那俩贴一起的应该也是漏了一个
image

<pd props="cs">预留。</pd>
</plentry>
<plentry props="cpp unreal bp unity flutter electron cs">
Expand Down
24 changes: 24 additions & 0 deletions dita/RTC-NG/API/enum_localvideostreamerror.dita
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,30 @@
<pt><ph keyref="LOCAL_VIDEO_STREAM_ERROR_SCREEN_CAPTURE_FAILURE"/></pt>
<pd>21：（仅适用于 Windows）当前采集的窗口无数据。</pd>
</plentry>
<plentry props="cpp flutter unity">
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

新增的这些枚举要不要+electron

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

因为 electron 这次不发，所以就不加，等到下次 electron发版的时候再做适配。
否则后面拉 bugfix 分支，上面的 electron 内容会超前，就出事了

<pt><ph keyref="LOCAL_VIDEO_STREAM_ERROR_SCREEN_CAPTURE_NO_PERMISSION"/></pt>
<pd>22：（仅适用于 Windows 和 macOS）不具备采集屏幕的权限。</pd>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

如果上面+electron，这里或许要+个props把elec排除掉

</plentry>
<plentry props="cpp flutter unity">
<pt><ph keyref="LOCAL_VIDEO_STREAM_ERROR_SCREEN_CAPTURE_PAUSED"/></pt>
<pd>23：（仅适用于 Windows）屏幕采集已暂停。报告该错误码的常见场景：该屏幕已切换到安全桌面，例如 UAC 对话框，或者 Winlogon 桌面。</pd>
</plentry>
<plentry props="cpp flutter unity">
<pt><ph keyref="LOCAL_VIDEO_STREAM_ERROR_SCREEN_CAPTURE_RESUMED"/></pt>
<pd>24：（仅适用于 Windows）屏幕采集已从暂停状态中恢复。</pd>
</plentry>
<plentry props="cpp flutter unity">
<pt><ph keyref="LOCAL_VIDEO_STREAM_ERROR_SCREEN_CAPTURE_WINDOW_HIDDEN"/></pt>
<pd>25：（仅适用于 Windows）当前屏幕采集的窗口处于隐藏状态，并且在当前屏幕上不可见。</pd>
</plentry>
<plentry props="cpp flutter unity">
<pt><ph keyref="LOCAL_VIDEO_STREAM_ERROR_SCREEN_CAPTURE_WINDOW_RECOVER_FROM_HIDDEN"/></pt>
<pd>26：（仅适用于 Windows）进行屏幕采集的窗口已经从隐藏状态中恢复。</pd>
</plentry>
<plentry props="cpp flutter unity">
<pt><ph keyref="LOCAL_VIDEO_STREAM_ERROR_SCREEN_CAPTURE_WINDOW_RECOVER_FROM_MINIMIZED"/></pt>
<pd>27：（仅适用于 Windows）进行屏幕采集的窗口已经从最小化状态中恢复。</pd>
</plentry>
<plentry props="cpp unreal bp unity electron flutter cs">
<pt><ph keyref="LOCAL_VIDEO_STREAM_ERROR_DEVICE_SYSTEM_PRESSURE"/></pt>
<pd>101：由于系统压力过大，导致当前视频采集设备不可用。</pd>
Expand Down
4 changes: 4 additions & 0 deletions dita/RTC-NG/API/enum_videopixelformat.dita
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,10 @@
<pt><ph keyref="VIDEO_PIXEL_I422"/></pt>
<pd>16: I422 格式。</pd>
</plentry>
<plentry props="cpp unity flutter">
<pt><ph keyref="VIDEO_TEXTURE_ID3D11TEXTURE2D"/></pt>
<pd>17: ID3D11TEXTURE2D 格式。目前支持的类型有 <codeph>DXGI_FORMAT_B8G8R8A8_UNORM</codeph>, <codeph>DXGI_FORMAT_B8G8R8A8_TYPELESS</codeph>, <codeph>DXGI_FORMAT_NV12</codeph>。</pd>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

是不是按照最新的 SG 要用顿号、（无所谓

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

嗯改一下好了~

</plentry>
</parml> </section>
</refbody>
</reference>
49 changes: 49 additions & 0 deletions dita/RTC-NG/config/keys-rtc-ng-api-cpp.ditamap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9144,6 +9144,48 @@
</keywords>
</topicmeta>
</keydef>
<keydef keys="LOCAL_VIDEO_STREAM_ERROR_SCREEN_CAPTURE_NO_PERMISSION">
<topicmeta>
<keywords>
<keyword>LOCAL_VIDEO_STREAM_ERROR_SCREEN_CAPTURE_NO_PERMISSION</keyword>
</keywords>
</topicmeta>
</keydef>
<keydef keys="LOCAL_VIDEO_STREAM_ERROR_SCREEN_CAPTURE_PAUSED">
<topicmeta>
<keywords>
<keyword>LOCAL_VIDEO_STREAM_ERROR_SCREEN_CAPTURE_PAUSED</keyword>
</keywords>
</topicmeta>
</keydef>
<keydef keys="LOCAL_VIDEO_STREAM_ERROR_SCREEN_CAPTURE_RESUMED">
<topicmeta>
<keywords>
<keyword>LOCAL_VIDEO_STREAM_ERROR_SCREEN_CAPTURE_RESUMED</keyword>
</keywords>
</topicmeta>
</keydef>
<keydef keys="LOCAL_VIDEO_STREAM_ERROR_SCREEN_CAPTURE_WINDOW_HIDDEN">
<topicmeta>
<keywords>
<keyword>LOCAL_VIDEO_STREAM_ERROR_SCREEN_CAPTURE_WINDOW_HIDDEN</keyword>
</keywords>
</topicmeta>
</keydef>
<keydef keys="LOCAL_VIDEO_STREAM_ERROR_SCREEN_CAPTURE_WINDOW_RECOVER_FROM_HIDDEN">
<topicmeta>
<keywords>
<keyword>LOCAL_VIDEO_STREAM_ERROR_SCREEN_CAPTURE_WINDOW_RECOVER_FROM_HIDDEN</keyword>
</keywords>
</topicmeta>
</keydef>
<keydef keys="LOCAL_VIDEO_STREAM_ERROR_SCREEN_CAPTURE_WINDOW_RECOVER_FROM_MINIMIZED">
<topicmeta>
<keywords>
<keyword>LOCAL_VIDEO_STREAM_ERROR_SCREEN_CAPTURE_WINDOW_RECOVER_FROM_MINIMIZED</keyword>
</keywords>
</topicmeta>
</keydef>
<keydef keys="LOCAL_VIDEO_STREAM_ERROR_SCREEN_CAPTURE_FAILURE">
<topicmeta>
<keywords>
Expand Down Expand Up @@ -12077,6 +12119,13 @@
</keywords>
</topicmeta>
</keydef>
<keydef keys="VIDEO_TEXTURE_ID3D11TEXTURE2D">
<topicmeta>
<keywords>
<keyword>VIDEO_TEXTURE_ID3D11TEXTURE2D</keyword>
</keywords>
</topicmeta>
</keydef>
<keydef keys="VIDEO_TEXTURE_2D">
<topicmeta>
<keywords>
Expand Down
Loading