Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

4.5.0 j #3802

Open
wants to merge 112 commits into
base: release/rtc-ng/4.5.0
Choose a base branch
from
Open

4.5.0 j #3802

Changes from 1 commit
Commits
Show all changes
112 commits
Select commit Hold shift + click to select a range
881bfc1
add proto
jinyuagora Aug 21, 2024
a1fa012
1
Cilla-luodan Aug 23, 2024
19bbec8
1
Cilla-luodan Aug 23, 2024
1b7e7d3
1
Cilla-luodan Aug 23, 2024
6ad334b
Merge pull request #3757 from AgoraIO/jjj-proto
Cilla-luodan Aug 23, 2024
f0db5f1
1
Cilla-luodan Aug 23, 2024
cd7ad6a
Merge branch 'harmony-ga' of github.com:AgoraIO/agora_doc_source into…
Cilla-luodan Aug 26, 2024
ef23739
1
Cilla-luodan Aug 26, 2024
44965d5
1
Cilla-luodan Aug 27, 2024
2f4a573
1
Cilla-luodan Aug 27, 2024
151179a
New translations
Cilla-luodan Aug 29, 2024
7766ff4
New translations
Cilla-luodan Aug 29, 2024
e7d0f2c
New translations
Cilla-luodan Aug 29, 2024
532874d
1
Cilla-luodan Aug 30, 2024
e1ad0a3
New translations
Cilla-luodan Sep 5, 2024
5f4208d
add missing APIs
Cilla-luodan Sep 11, 2024
f136aee
Merge branch 'master' into harmony-ga
Cilla-luodan Sep 11, 2024
76f22db
fix
Cilla-luodan Sep 11, 2024
f62a4c3
1
Cilla-luodan Sep 11, 2024
d55689e
to audio_basic
Cilla-luodan Sep 12, 2024
7249f8b
New translations
Cilla-luodan Sep 12, 2024
8fe3821
New translations
Cilla-luodan Sep 12, 2024
16b1c13
New translations
Cilla-luodan Sep 13, 2024
7f75cc5
to screencapture
Cilla-luodan Sep 13, 2024
22079f3
all APIs are done
Cilla-luodan Sep 14, 2024
e8a7391
that's all
Cilla-luodan Sep 14, 2024
707ddc3
fix publish errors
Cilla-luodan Sep 18, 2024
8f8a39c
fix publish errors
Cilla-luodan Sep 23, 2024
de1e575
1
Cilla-luodan Sep 23, 2024
2d2a6b6
New translations
Cilla-luodan Sep 27, 2024
12b9f7e
1
Cilla-luodan Sep 29, 2024
609d583
Merge branch 'master' into harmony-ga
Cilla-luodan Sep 29, 2024
1dbd2aa
Merge pull request #3782 from AgoraIO/harmony-ga
Cilla-luodan Sep 29, 2024
daeb2d3
Sync cn prototype to en prototype
Cilla-luodan Sep 29, 2024
ea1dd5b
Merge pull request #3783 from AgoraIO/auto/sync-cn-proto-to-en-patch-…
github-actions[bot] Sep 29, 2024
842b1e1
1
Cilla-luodan Sep 29, 2024
c0c4c68
Merge branch 'master' of github.com:AgoraIO/agora_doc_source
Cilla-luodan Sep 29, 2024
c212e50
Sync cn prototype to en prototype
Cilla-luodan Sep 29, 2024
a656594
Merge pull request #3784 from AgoraIO/auto/sync-cn-proto-to-en-patch-…
github-actions[bot] Sep 29, 2024
f17a79a
New translations
Cilla-luodan Sep 29, 2024
da6cb79
1
Cilla-luodan Oct 9, 2024
7d90b94
Merge branch 'master' of github.com:AgoraIO/agora_doc_source
Cilla-luodan Oct 9, 2024
83780ff
sync with guide
Cilla-luodan Oct 9, 2024
6c999df
Update callback_irtcengineeventhandler_onstreammessage.dita
jinyuagora Oct 10, 2024
8a9231a
Electron/RN: update proto
Nero-Hu Oct 11, 2024
f493ad6
Electron: remove BackgroundNone
Nero-Hu Oct 11, 2024
33305c5
RN: hide background none
Nero-Hu Oct 11, 2024
e392ee8
bugfix
Suri539 Oct 11, 2024
25cf7ed
update props
Nero-Hu Oct 11, 2024
42a9553
Revert "update props"
Nero-Hu Oct 11, 2024
bbe960c
1
Cilla-luodan Oct 11, 2024
f92c34d
Merge branch '4.4.0-bugfix-1010' of github.com:AgoraIO/agora_doc_sour…
Cilla-luodan Oct 11, 2024
d20cc62
Update callback_irtcengineeventhandler_onstreammessage.dita
Suri539 Oct 11, 2024
99d1b2a
Merge pull request #3785 from AgoraIO/4.4.0-bugfix-1010
Cilla-luodan Oct 11, 2024
4a34de8
Sync cn prototype to en prototype
Cilla-luodan Oct 11, 2024
3ad4717
Merge pull request #3786 from AgoraIO/auto/sync-cn-proto-to-en-patch-…
github-actions[bot] Oct 11, 2024
17f48a8
New translations
Cilla-luodan Oct 14, 2024
4a08206
api change
Cilla-luodan Oct 14, 2024
508e82c
New translations
Cilla-luodan Oct 14, 2024
a0222dc
liveshow
jinyuagora Oct 15, 2024
7adacea
resolve comments
jinyuagora Oct 15, 2024
eab1db6
eglcontext
jinyuagora Oct 16, 2024
b9878db
resolve comments
jinyuagora Oct 16, 2024
e63fe17
Update api_irtcengine_sendstreammessage.dita
jinyuagora Oct 17, 2024
6686fab
Update api_irtcengine_enablelocalaudio.dita
jinyuagora Oct 17, 2024
16620d7
bugfix
Cilla-luodan Oct 17, 2024
5da70c8
add quality_detecting for android
Nero-Hu Oct 17, 2024
0a5fd2a
1
jinyuagora Oct 17, 2024
fe23210
Merge branch '4.4.0-bugfix-1017' of github.com:AgoraIO/agora_doc_sour…
jinyuagora Oct 17, 2024
15998fb
Merge pull request #3791 from AgoraIO/4.4.0-bugfix-1017
Cilla-luodan Oct 17, 2024
09e015f
Sync cn prototype to en prototype
Cilla-luodan Oct 17, 2024
7ef2e9f
Merge pull request #3792 from AgoraIO/auto/sync-cn-proto-to-en-patch-…
github-actions[bot] Oct 17, 2024
ea1cb06
Update relations-rtc-ng-api.ditamap
jinyuagora Oct 17, 2024
34689f6
New translations
Cilla-luodan Oct 17, 2024
5a9fdec
colorspace
jinyuagora Oct 18, 2024
48407b7
encoding -> capturing
jinyuagora Oct 18, 2024
7147adf
resolve comments
jinyuagora Oct 21, 2024
0434377
audiotrack
jinyuagora Oct 21, 2024
65d11f2
minor fix
jinyuagora Oct 22, 2024
7f6fbe0
delete redundant desc
Cilla-luodan Oct 22, 2024
07a4d83
Update api_imusiccontentcenter_getcaches.dita
jinyuagora Oct 24, 2024
022ab6f
add props for flutter
Nero-Hu Oct 24, 2024
0f4243d
New translations
Cilla-luodan Oct 24, 2024
29a1768
Update class_rtcimage.dita
jinyuagora Oct 24, 2024
735c59d
Sync cn prototype to en prototype
jinyuagora Oct 24, 2024
f3ec7b6
Merge pull request #3795 from AgoraIO/auto/sync-cn-proto-to-en-patch-…
github-actions[bot] Oct 24, 2024
a46fb6c
Update class_rtcimage.dita
jinyuagora Oct 24, 2024
9b35f51
Sync cn prototype to en prototype
jinyuagora Oct 24, 2024
db388f9
Merge pull request #3796 from AgoraIO/auto/sync-cn-proto-to-en-patch-…
github-actions[bot] Oct 24, 2024
97dd6de
New translations
Cilla-luodan Oct 24, 2024
2085181
New translations
Cilla-luodan Oct 24, 2024
65d8f32
fix
Cilla-luodan Oct 25, 2024
ff954cb
New translations
Cilla-luodan Oct 25, 2024
7ca1e38
Merge pull request #3770 from AgoraIO/translation/4.x-api-ref
Cilla-luodan Oct 25, 2024
544a17d
bugfix
Suri539 Oct 25, 2024
456a0f4
Merge pull request #3797 from AgoraIO/4.4.1-harmony
Cilla-luodan Oct 25, 2024
5aedf65
Sync cn prototype to en prototype
Cilla-luodan Oct 25, 2024
de8b64a
Merge pull request #3798 from AgoraIO/auto/sync-cn-proto-to-en-patch-…
github-actions[bot] Oct 25, 2024
6550f2e
New translations
Cilla-luodan Oct 25, 2024
b9b2a93
New translations
Cilla-luodan Oct 25, 2024
c56c17f
Merge pull request #3799 from AgoraIO/translation/4.x-api-ref
Cilla-luodan Oct 25, 2024
373e9d8
Update api_imediaplayer_preloadsrc.dita
Suri539 Oct 25, 2024
eeff53e
Merge pull request #3794 from AgoraIO/4.4.0-bugfix-1024
Cilla-luodan Oct 25, 2024
9bbff5e
fix
Cilla-luodan Oct 25, 2024
dd4b088
resolve comments
jinyuagora Oct 28, 2024
aad24e2
合图 & 双流
jinyuagora Oct 28, 2024
bb192f7
媒体播放器缓存
jinyuagora Oct 28, 2024
5b496b1
check
jinyuagora Oct 28, 2024
8ba183e
1
jinyuagora Oct 28, 2024
866aef9
fix
jinyuagora Oct 29, 2024
9265362
1
jinyuagora Oct 29, 2024
f7f09b8
Merge branch 'master' into 4.5.0-j
jinyuagora Oct 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
colorspace
jinyuagora committed Oct 18, 2024
commit 5a9fdec9da751c6915eb9152381cea29129cbc26
129 changes: 102 additions & 27 deletions dita/RTC-NG/API/class_externalvideoframe.dita
Original file line number Diff line number Diff line change
@@ -7,20 +7,32 @@
<section id="prototype">
<p outputclass="codeblock">
<codeblock props="android" outputclass="language-java">public class AgoraVideoFrame {

public static final int FORMAT_NONE = -1;

public static final int FORMAT_TEXTURE_2D = 10;

public static final int FORMAT_TEXTURE_OES = 11;

public static final int FORMAT_I420 = 1;

public static final int FORMAT_BGRA = 2;

public static final int FORMAT_NV21 = 3;

public static final int FORMAT_RGBA = 4;

public static final int FORMAT_I422 = 16;

public static final int BUFFER_TYPE_NONE = -1;

public static final int BUFFER_TYPE_BUFFER = 1;

public static final int BUFFER_TYPE_ARRAY = 2;

public static final int BUFFER_TYPE_TEXTURE = 3;
public AgoraVideoFrame() {
format = 10;
format = 10;
timeStamp = 0;
stride = 0;
height = 0;
@@ -37,53 +49,78 @@
rotation = 0;
alphaStitchMode = 0;
}

public int format;

public long timeStamp;

public int stride;

public int height;

public int textureID;

public boolean syncMode;

public float[] transform;

public javax.microedition.khronos.egl.EGLContext eglContext10;
public android.opengl.EGLContext eglContext14;

public android.opengl.EGLContext eglContext14;

public byte[] buf;

public int cropLeft;

public int cropTop;

public int cropRight;

public int cropBottom;

public int rotation;

public int alphaStitchMode;
@Override
public String toString() {
return "AgoraVideoFrame{"
+ "format=" + format + ", timeStamp=" + timeStamp + ", stride=" + stride
+ ", height=" + height + ", textureID=" + textureID
+ ", buf.length=" + (buf != null ? buf.length : 0) + ", cropLeft=" + cropLeft
+ ", cropTop=" + cropTop + ", cropRight=" + cropRight + ", cropBottom=" + cropBottom
+ ", rotation=" + rotation + ", alphaStitchMode=" + alphaStitchMode + '}';
return &quot;AgoraVideoFrame{&quot;
+ &quot;format=&quot; + format + &quot;, timeStamp=&quot; + timeStamp + &quot;, stride=&quot; + stride
+ &quot;, height=&quot; + height + &quot;, textureID=&quot; + textureID
+ &quot;, buf.length=&quot; + (buf != null ? buf.length : 0) + &quot;, cropLeft=&quot; + cropLeft
+ &quot;, cropTop=&quot; + cropTop + &quot;, cropRight=&quot; + cropRight + &quot;, cropBottom=&quot; + cropBottom
+ &quot;, rotation=&quot; + rotation + &quot;, alphaStitchMode=&quot; + alphaStitchMode + &#x27;}&#x27;;
}

}</codeblock>
<codeblock props="hmos" outputclass="language-arkts"></codeblock>
<codeblock props="ios mac" outputclass="language-objectivec">__attribute__((visibility("default"))) @interface AgoraVideoFrame : NSObject
<codeblock props="ios mac" outputclass="language-objectivec">
__attribute__((visibility(&quot;default&quot;))) @interface AgoraVideoFrame : NSObject
@property(assign, nonatomic) NSInteger format;
@property(assign, nonatomic) CMTime time;
@property(assign, nonatomic) int stride DEPRECATED_MSG_ATTRIBUTE("use strideInPixels instead");
@property(assign, nonatomic) int strideInPixels;
@property(assign, nonatomic) int height;

@property(assign, nonatomic) CMTime time;
@property(assign, nonatomic) int stride DEPRECATED_MSG_ATTRIBUTE(&quot;use strideInPixels instead&quot;);

@property(assign, nonatomic) int strideInPixels;
@property(assign, nonatomic) int height;
@property(assign, nonatomic) CVPixelBufferRef _Nullable textureBuf;

@property(strong, nonatomic) IMAGE_CLASS * _Nullable image;
@property(strong, nonatomic) NSData *_Nullable dataBuf;

@property(strong, nonatomic) NSData *_Nullable dataBuf;
@property(strong, nonatomic) NSData *_Nullable alphaBuf;
@property(assign, nonatomic) AgoraAlphaStitchMode alphaStitchMode;
@property(assign, nonatomic) int cropLeft;
@property(assign, nonatomic) int cropTop;
@property(assign, nonatomic) int cropRight;
@property(assign, nonatomic) int cropBottom;
@property(assign, nonatomic) int rotation;

@property(assign, nonatomic) int cropLeft;
@property(assign, nonatomic) int cropTop;
@property(assign, nonatomic) int cropRight;
@property(assign, nonatomic) int cropBottom;
@property(assign, nonatomic) int rotation;
@property(strong, nonatomic) AgoraColorSpace *_Nullable colorSpace;

- (void)fillAlphaData;
@end</codeblock>
<codeblock props="cpp unreal" outputclass="language-cpp">struct ExternalVideoFrame {
<codeblock props="cpp unreal" outputclass="language-cpp">
struct ExternalVideoFrame {
ExternalVideoFrame()
: type(VIDEO_BUFFER_RAW_DATA),
format(VIDEO_PIXEL_DEFAULT),
@@ -99,44 +136,78 @@
eglContext(NULL),
eglType(EGL_CONTEXT10),
textureId(0),
fenceObject(0),
metadataBuffer(NULL),
metadataSize(0),
alphaBuffer(NULL),
fillAlphaBuffer(false),
alphaStitchMode(0),
alphaStitchMode(NO_ALPHA_STITCH),
d3d11Texture2d(NULL),
textureSliceIndex(0){}

enum EGL_CONTEXT_TYPE {

EGL_CONTEXT10 = 0,

EGL_CONTEXT14 = 1,
};

enum VIDEO_BUFFER_TYPE {

VIDEO_BUFFER_RAW_DATA = 1,

VIDEO_BUFFER_ARRAY = 2,

VIDEO_BUFFER_TEXTURE = 3,
};

VIDEO_BUFFER_TYPE type;

VIDEO_PIXEL_FORMAT format;

void* buffer;

int stride;

int height;

int cropLeft;

int cropTop;

int cropRight;

int cropBottom;

int rotation;

long long timestamp;
void *eglContext;

void* eglContext;

EGL_CONTEXT_TYPE eglType;

int textureId;

long long fenceObject;

float matrix[16];

uint8_t* metadataBuffer;

int metadataSize;

uint8_t* alphaBuffer;

bool fillAlphaBuffer;
int alphaStitchMode;

ALPHA_STITCH_MODE alphaStitchMode;

void *d3d11Texture2d;

int textureSliceIndex;

ColorSpace colorSpace;
};</codeblock>
<codeblock props="electron" outputclass="language-typescript">export class ExternalVideoFrame {
type?: VideoBufferType;
@@ -403,8 +474,8 @@ class ExternalVideoFrame {
<pd>Texture 帧额外的转换。该参数仅适用于 Texture 格式的视频数据。</pd>
</plentry>
<plentry props="android hmos">
<pt>eglContext11</pt>
<pd>EGLContext11。该参数仅适用于 Texture 格式的视频数据。</pd>
<pt>eglContext10</pt>
<pd>EGLContext10。该参数仅适用于 Texture 格式的视频数据。</pd>
</plentry>
<plentry props="android hmos">
<pt>eglContext14</pt>
@@ -414,8 +485,8 @@ class ExternalVideoFrame {
<pt>eglContext</pt>
<pd>该参数仅适用于 Texture 格式的视频数据。
<ul id="ul_fsv_shd_krb" props="cpp unreal bp unity rn electron flutter">
<li>当使用 Khronos 定义的 OpenGL 接口 (javax.microedition.khronos.egl.*)时,需要将 eglContext 设置给这个字段。</li>
<li>当使用 Android 定义的 OpenGL 接口 (android.opengl.*)时,需要将 eglContext 设置给这个字段。</li>
<li>当使用 Khronos 定义的 OpenGL 接口 (javax.microedition.khronos.egl.*) 时,需要将 eglContext 设置给这个字段。</li>
<li>当使用 Android 定义的 OpenGL 接口 (android.opengl.*) 时,需要将 eglContext 设置给这个字段。</li>
</ul></pd>
</plentry>
<plentry props="cpp unreal bp unity electron rn flutter cs">
@@ -506,6 +577,10 @@ class ExternalVideoFrame {
<pt props="ios mac">time</pt>
<pd>传入的视频帧的时间戳,以毫秒为单位。不正确的时间戳会导致丢帧或者音视频不同步。</pd>
</plentry>
<plentry props="apple cpp" conkeyref="VideoFrame/colorspace">
<pt/>
<pd/>
</plentry>
</parml> </section>
</refbody>
</reference>
253 changes: 200 additions & 53 deletions dita/RTC-NG/API/class_videoframe.dita
Original file line number Diff line number Diff line change
@@ -7,42 +7,61 @@
<section id="prototype">
<p outputclass="codeblock">
<codeblock props="android" outputclass="language-java">public class VideoFrame implements RefCounted {

public interface Buffer extends RefCounted {
@CalledByNative("Buffer") int getWidth();
@CalledByNative("Buffer") int getHeight();
@CalledByNative("Buffer") I420Buffer toI420();
@Override @CalledByNative("Buffer") void release();
@Override @CalledByNative("Buffer") void retain();
@CalledByNative("Buffer")

@CalledByNative(&quot;Buffer&quot;) int getWidth();

@CalledByNative(&quot;Buffer&quot;) int getHeight();

@CalledByNative(&quot;Buffer&quot;) I420Buffer toI420();

@Override @CalledByNative(&quot;Buffer&quot;) void release();

@Override @CalledByNative(&quot;Buffer&quot;) void retain();

@CalledByNative(&quot;Buffer&quot;)
Buffer cropAndScale(
int cropX, int cropY, int cropWidth, int cropHeight, int scaleWidth, int scaleHeight);
@CalledByNative("Buffer") @Nullable Buffer mirror(int frameRotation);
@CalledByNative("Buffer") @Nullable Buffer rotate(int frameRotation);
@CalledByNative("Buffer")

@CalledByNative(&quot;Buffer&quot;) @Nullable Buffer mirror(int frameRotation);

@CalledByNative(&quot;Buffer&quot;) @Nullable Buffer rotate(int frameRotation);

@CalledByNative(&quot;Buffer&quot;)
@Nullable
Buffer transform(int cropX, int cropY, int cropWidth, int cropHeight, int scaleWidth,
int scaleHeight, int frameRotation);
}

public interface I420Buffer extends Buffer {
@CalledByNative("I420Buffer") ByteBuffer getDataY();
@CalledByNative("I420Buffer") ByteBuffer getDataU();
@CalledByNative("I420Buffer") ByteBuffer getDataV();
@CalledByNative("I420Buffer") int getStrideY();
@CalledByNative("I420Buffer") int getStrideU();
@CalledByNative("I420Buffer") int getStrideV();

@CalledByNative(&quot;I420Buffer&quot;) ByteBuffer getDataY();

@CalledByNative(&quot;I420Buffer&quot;) ByteBuffer getDataU();

@CalledByNative(&quot;I420Buffer&quot;) ByteBuffer getDataV();
@CalledByNative(&quot;I420Buffer&quot;) int getStrideY();
@CalledByNative(&quot;I420Buffer&quot;) int getStrideU();
@CalledByNative(&quot;I420Buffer&quot;) int getStrideV();
}

public interface I422Buffer extends Buffer {
@CalledByNative("I422Buffer") ByteBuffer getDataY();
@CalledByNative("I422Buffer") ByteBuffer getDataU();
@CalledByNative("I422Buffer") ByteBuffer getDataV();
@CalledByNative("I422Buffer") int getStrideY();
@CalledByNative("I422Buffer") int getStrideU();
@CalledByNative("I422Buffer") int getStrideV();
}
public interface RgbaBuffer extends Buffer { @CalledByNative("RgbaBuffer") ByteBuffer getData(); }
@CalledByNative(&quot;I422Buffer&quot;) ByteBuffer getDataY();
@CalledByNative(&quot;I422Buffer&quot;) ByteBuffer getDataU();
@CalledByNative(&quot;I422Buffer&quot;) ByteBuffer getDataV();
@CalledByNative(&quot;I422Buffer&quot;) int getStrideY();
@CalledByNative(&quot;I422Buffer&quot;) int getStrideU();
@CalledByNative(&quot;I422Buffer&quot;) int getStrideV();
}
public interface RgbaBuffer extends Buffer { @CalledByNative(&quot;RgbaBuffer&quot;) ByteBuffer getData(); }

public interface TextureBuffer extends Buffer {

enum Type {

OES(GLES11Ext.GL_TEXTURE_EXTERNAL_OES),

RGB(GLES20.GL_TEXTURE_2D);
private final int glTarget;
private Type(final int glTarget) {
@@ -57,22 +76,26 @@
EGL_CONTEXT_14;
}
Type getType();
@CalledByNative("TextureBuffer") int getTextureId();

@CalledByNative(&quot;TextureBuffer&quot;) int getTextureId();

Matrix getTransformMatrix();
@CalledByNative("TextureBuffer") EglBase.Context getEglBaseContext();
@CalledByNative("TextureBuffer") Object getSourceTexturePool();
@CalledByNative("TextureBuffer") long getNativeEglContext();
@CalledByNative("TextureBuffer") int getEglContextType();
@CalledByNative("TextureBuffer") float[] getTransformMatrixArray();
@CalledByNative("TextureBuffer") int getSequence();
@CalledByNative("TextureBuffer") boolean is10BitTexture();

@CalledByNative(&quot;TextureBuffer&quot;) EglBase.Context getEglBaseContext();
@CalledByNative(&quot;TextureBuffer&quot;) Object getSourceTexturePool();
@CalledByNative(&quot;TextureBuffer&quot;) long getNativeEglContext();
@CalledByNative(&quot;TextureBuffer&quot;) int getEglContextType();
@CalledByNative(&quot;TextureBuffer&quot;) float[] getTransformMatrixArray();

@CalledByNative(&quot;TextureBuffer&quot;) int getSequence();
@CalledByNative(&quot;TextureBuffer&quot;) long getFenceObject();
@CalledByNative(&quot;TextureBuffer&quot;) boolean is10BitTexture();
}
public interface ColorSpace {
enum Range {
Invalid(0),
Limited(1),
Full(2),
Derived(3);
Full(2);
private final int range;
private Range(int range) {
this.range = range;
@@ -161,32 +184,55 @@
kBackCamera,
kUnspecified,
}
public enum AlphaStitchMode {
ALPHA_NO_STITCH(0),
ALPHA_STITCH_UP(1),
ALPHA_STITCH_BELOW(2),
ALPHA_STITCH_LEFT(3),
ALPHA_STITCH_RIGHT(4);
private final int stitchMode;
private AlphaStitchMode(int stitchMode) {
this.stitchMode = stitchMode;
}
public int value() {
return stitchMode;
}
}

private Buffer buffer;

private int rotation;

private long timestampNs;
private ColorSpace colorSpace;
private SourceType sourceType;
private float sampleAspectRatio;

private AlphaStitchMode alphaStitchMode = AlphaStitchMode.ALPHA_NO_STITCH;
private VideoFrameMetaInfo metaInfo = new VideoFrameMetaInfo();

private @Nullable ByteBuffer alphaBuffer;
private int alphaStitchMode;
private long nativeAlphaBuffer;

public VideoFrame(Buffer buffer, int rotation, long timestampNs) {
this(buffer, rotation, timestampNs, new WrappedNativeColorSpace(), null, 0L, 1.0f, SourceType.kUnspecified.ordinal());
this(buffer, rotation, timestampNs, new WrappedNativeColorSpace(), null, 0L, 1.0f,
SourceType.kUnspecified.ordinal());
}
@CalledByNative
public VideoFrame(Buffer buffer, int rotation, long timestampNs, ColorSpace colorSpace,
ByteBuffer alphaBuffer, float sampleAspectRatio, int sourceType) {
ByteBuffer alphaBuffer, long nativeAlphaBuffer, float sampleAspectRatio, int sourceType) {
if (buffer == null) {
throw new IllegalArgumentException("buffer not allowed to be null");
throw new IllegalArgumentException(&quot;buffer not allowed to be null&quot;);
}
if (rotation % 90 != 0) {
throw new IllegalArgumentException("rotation must be a multiple of 90");
throw new IllegalArgumentException(&quot;rotation must be a multiple of 90&quot;);
}
this.buffer = buffer;
this.rotation = rotation;
this.timestampNs = timestampNs;
this.colorSpace = colorSpace;
this.alphaBuffer = alphaBuffer;
this.nativeAlphaBuffer = nativeAlphaBuffer;
this.sampleAspectRatio = sampleAspectRatio;
this.sourceType = SourceType.values()[sourceType];
}
@@ -197,18 +243,25 @@
public float getSampleAspectRatio() {
return sampleAspectRatio;
}

@CalledByNative
public Buffer getBuffer() {
return buffer;
}

@CalledByNative
public int getRotation() {
return rotation;
}
@CalledByNative
public int getAlphaStitchMode() {
return alphaStitchMode;
return alphaStitchMode.value();
}
@CalledByNative
public void setAlphaStitchMode(int stitchMode) {
alphaStitchMode = AlphaStitchMode.values()[stitchMode];
}

@CalledByNative
public long getTimestampNs() {
return timestampNs;
@@ -217,41 +270,93 @@
public VideoFrameMetaInfo getMetaInfo() {
return metaInfo;
}

public int getRotatedWidth() {
if (rotation % 180 == 0) {
return buffer.getWidth();
return (alphaStitchMode == AlphaStitchMode.ALPHA_STITCH_LEFT
|| alphaStitchMode == AlphaStitchMode.ALPHA_STITCH_RIGHT)
? buffer.getWidth() / 2
: buffer.getWidth();
}
return buffer.getHeight();
return (alphaStitchMode == AlphaStitchMode.ALPHA_STITCH_UP
|| alphaStitchMode == AlphaStitchMode.ALPHA_STITCH_BELOW)
? buffer.getHeight() / 2
: buffer.getHeight();
}

public int getRotatedHeight() {
if (rotation % 180 == 0) {
return buffer.getHeight();
return (alphaStitchMode == AlphaStitchMode.ALPHA_STITCH_UP
|| alphaStitchMode == AlphaStitchMode.ALPHA_STITCH_BELOW)
? buffer.getHeight() / 2
: buffer.getHeight();
}
return buffer.getWidth();
return (alphaStitchMode == AlphaStitchMode.ALPHA_STITCH_LEFT
|| alphaStitchMode == AlphaStitchMode.ALPHA_STITCH_RIGHT)
? buffer.getWidth() / 2
: buffer.getWidth();
}

public void replaceBuffer(Buffer buffer, int rotation, long timestampNs) {
release();
this.buffer = buffer;
this.rotation = rotation;
this.timestampNs = timestampNs;
}
@CalledByNative
public ColorSpace getColorSpace() {
return colorSpace;
}
public void setColorSpace(ColorSpace colorSpace) {
this.colorSpace = colorSpace;
}
@CalledByNative
private int getColorSpaceRange() {
if (colorSpace == null) {
return ColorSpace.Range.Invalid.getRange();
}
return colorSpace.getRange().getRange();
}
@CalledByNative
private int getColorSpaceMatrix() {
if (colorSpace == null) {
return ColorSpace.Matrix.Unspecified.getMatrix();
}
return colorSpace.getMatrix().getMatrix();
}
@CalledByNative
private int getColorSpaceTransfer() {
if (colorSpace == null) {
return ColorSpace.Transfer.Unspecified.getTransfer();
}
return colorSpace.getTransfer().getTransfer();
}
@CalledByNative
private int getColorSpacePrimary() {
if (colorSpace == null) {
return ColorSpace.Primary.Unspecified.getPrimary();
}
return colorSpace.getPrimary().getPrimary();
}
@CalledByNative
public ByteBuffer getAlphaBuffer() {
return alphaBuffer;
}
public void retainAlphaBuffer() {
JniCommon.nativeAddRef(nativeAlphaBuffer);
}
public void releaseAlphaBuffer() {
JniCommon.nativeReleaseRef(nativeAlphaBuffer);
}
public void fillAlphaData(ByteBuffer buffer) {
alphaBuffer = buffer;
}
public void setAlphaStitchMode(int mode) {
this.alphaStitchMode = mode;
}

@Override
public void retain() {
buffer.retain();
}

@Override
@CalledByNative
public void release() {
@@ -278,7 +383,8 @@
this.format = format;
}
}</codeblock>
<codeblock props="ios mac" outputclass="language-objectivec">__attribute__((visibility("default"))) @interface AgoraOutputVideoFrame : NSObject
<codeblock props="ios mac" outputclass="language-objectivec">__attribute__((visibility(&quot;default&quot;))) @interface AgoraOutputVideoFrame : NSObject

@property (nonatomic, assign) NSInteger type;
@property (nonatomic, assign) int width;
@property (nonatomic, assign) int height;
@@ -291,12 +397,16 @@
@property (nonatomic, assign) int rotation;
@property (nonatomic, assign) int64_t renderTimeMs;
@property (nonatomic, assign) int avSyncType;

@property(assign, nonatomic) CVPixelBufferRef _Nullable pixelBuffer;
@property (nonatomic, assign) uint8_t* _Nullable alphaBuffer;
@property (nonatomic, assign) AgoraAlphaStitchMode alphaStitchMode;

@property(nonatomic, strong) NSDictionary *_Nonnull metaInfo;
@property(nonatomic, strong) AgoraColorSpace* _Nullable colorSpace;
@end</codeblock>
<codeblock props="cpp unreal" outputclass="language-cpp">struct VideoFrame {
<codeblock props="cpp unreal" outputclass="language-cpp">
struct VideoFrame {
VideoFrame():
type(VIDEO_PIXEL_DEFAULT),
width(0),
@@ -316,33 +426,57 @@
textureId(0),
d3d11Texture2d(NULL),
alphaBuffer(NULL),
alphaStitchMode(0),
alphaStitchMode(NO_ALPHA_STITCH),
pixelBuffer(NULL),
metaInfo(NULL){
memset(matrix, 0, sizeof(matrix));
}

VIDEO_PIXEL_FORMAT type;

int width;

int height;

int yStride;

int uStride;

int vStride;

uint8_t* yBuffer;

uint8_t* uBuffer;

uint8_t* vBuffer;

int rotation;

int64_t renderTimeMs;

int avsync_type;

uint8_t* metadata_buffer;

int metadata_size;

void* sharedContext;

int textureId;

void* d3d11Texture2d;

float matrix[16];

uint8_t* alphaBuffer;
int alphaStitchMode;

ALPHA_STITCH_MODE alphaStitchMode;

void* pixelBuffer;

IVideoFrameMetaInfo* metaInfo;

ColorSpace colorSpace;
};</codeblock>
<codeblock props="bp" outputclass="language-cpp">USTRUCT(BlueprintType)
struct FVideoFrame {
@@ -608,6 +742,19 @@ class VideoFrame {
<section id="detailed_desc">
<p>缓冲区给出的是指向指针的指针,该接口不能修改缓冲区的指针,只能修改缓冲区的内容。</p>
</section>
<section id="sub-method" props="android">
<title>方法</title>
<parml>
<plentry>
<pt>getColorSpace</pt>
<pd>获取视频帧的色彩空间属性。</pd>
</plentry>
<plentry>
<pt>setColorSpace</pt>
<pd>设置视频帧的色彩空间属性。</pd>
</plentry>
</parml>
</section>
<section id="parameters">
<title><text conref="../conref/conref_api_metadata.dita#conref_api_metadata/property"/></title>
<parml>
@@ -727,9 +874,9 @@ class VideoFrame {
<pt>pixelBuffer</pt>
<pd>将数据填充到 CVPixelBuffer。</pd>
</plentry>
<plentry props="android hmos">
<pt>colorSpace</pt>
<pd>表示视频帧的色彩空间。详见 <xref keyref="videocolorspace-link"/>。</pd>
<plentry id="colorspace">
<pt>colorSpace</pt>
<pd>视频帧的色彩空间属性,默认情况下会应用 Full Range 和 BT.709 标准配置。你可以根据自采集、自渲染的业务需求进行自定义设置,详见 <xref keyref="videocolorspace-link"/>。</pd>
</plentry>
<plentry props="android hmos">
<pt>sourceType</pt>