diff --git a/index.html b/index.html index d48ea70..b609a68 100644 --- a/index.html +++ b/index.html @@ -9,7 +9,7 @@ // See https://github.com/w3c/respec/wiki/ for how to configure ReSpec var respecConfig = { group: "webrtc", - xref: ["geometry-1", "html", "infra", "permissions", "dom", "image-capture", "mediacapture-streams", "webaudio", "webcodecs", "webidl"], + xref: ["geometry-1", "html", "infra", "permissions", "dom", "hr-time", "image-capture", "mediacapture-streams", "screen-capture", "webaudio", "webcodecs", "webidl"], edDraftURI: "https://w3c.github.io/mediacapture-extensions/", editors: [ {name: "Jan-Ivar Bruaroey", company: "Mozilla Corporation", w3cid: 79152}, @@ -58,6 +58,9 @@

Terminology

The terms [=permission state=], [=request permission to use=], and prompt the user to choose are defined in [[!permissions]].

+

+ {{Performance.now()}} is defined in [[!hr-time]]. +

@@ -1151,7 +1154,112 @@

Constrainable Properties

-
+
+

Video timestamp concepts

+

+ Video media flowing inside media stream tracks comprises of a sequence of video frames, where + the frames are sampled from the media at instants spread out over time. +

+

+ Each video frame must have a presentation timestamp + which is relative to a source specific origin. + A source of frames can define how this timestamp is set. A sink of frames + can define how this timestamp is used. +

+

+ The timestamp is present for sinks to be able to define an absolute presentation timeline of the frames + relative to a clock reference, for example for playback. +

+

+ Each frame may have an absolute capture timestamp representing + the instant the frame capture process began, which is useful for example for + delay measurements and synchronization. + A source of frames can define how this timestamp is set, otherwise it is unset. A + sink of frames can define how this timestamp is used if set. +

+

+ Each frame may have an absolute receive timestamp representing + the last received timestamp of packets used to produce this video frame was received in its entirety. + The timestamp is useful for example for network jitter measurements. + A source of frames can define how this timestamp is set, otherwise it is unset. A sink of + frames can define how this timestamp is used if set. +

+

+ Each frame may have a RTP timestamp representing the packet RTP + timestamp used to produce this video frame. The timestamp is useful for example for frame + identification and playback quality measurements. A source of frames can define how the + timestamp is set, otherwise it is unset. A sink of frames can define how this timestamp is + used if set. + The packet RTP timestamp concept is defined in [[?RFC3550]] Section 5.1. +

+

Timestamp clock relations

+

+ The [=capture timestamp=] and [=receive timestamp=] are using the same clock and offset. + The [=presentation timestamp=] and [=capture timestamp=] are using the same clock and + have an offset which can be arbitrarily chosen by the user agent since it isn't + directly observable by script. +

+

{{VideoFrameMetadata}}

+ 
+partial dictionary VideoFrameMetadata {
+  DOMHighResTimeStamp captureTime;
+  DOMHighResTimeStamp receiveTime;
+  unsigned long rtpTimestamp;
+};
+
+
Members
+
+
captureTime of type DOMHighResTimeStamp
+
+

The capture timestamp of the frame relative to {{Performance}}.{{Performance/timeOrigin}}. It corresponds to + the [=capture timestamp=] of {{MediaStreamTrack}} video frames. +

+
+
receiveTime of type DOMHighResTimeStamp
+
+

The receive time of the corresponding encoded frame relative to {{Performance}}.{{Performance/timeOrigin}}. + It corresponds to the [=receive timestamp=] of {{MediaStreamTrack}} video frames.

+
+
rtpTimestamp of type unsigned long
+
+

The RTP timestamp of the corresponding encoded frame. It corresponds to [=RTP timestamp=] of + {{MediaStreamTrack}} video frames.

+
+
+
+

Algorithms

+ When the Initialize Video Frame Timestamps From Internal MediaStreamTrack Video Frame + algorithm is invoked with |frame| and |offset| as input, run the following steps. +
    +
  1. Set {{VideoFrame/timestamp}} from [=presentation timestamp=] minus |offset|.
    2. +
  2. Set {{VideoFrameMetadata/captureTime}} from [=capture timestamp=] if set.
    3. +
  3. Set {{VideoFrameMetadata/receiveTime}} from [=receive timestamp=] if set.
    4. +
  4. Set {{VideoFrameMetadata/rtpTimestamp}} from [=RTP timestamp=] if set.
    5. +
+ When the Copy Video Frame Timestamps To Internal MediaStreamTrack Video Frame + algorithm runs with |frame| as input, run the following steps. +
    +
  1. Set [=presentation timestamp=] from {{VideoFrame/timestamp}}.
    2. +
  2. Set [=capture timestamp=] from {{VideoFrameMetadata/captureTime}} if [=map/exist|present=].
    3. +
  3. Set [=receive timestamp=] from {{VideoFrameMetadata/receiveTime}} if [=map/exist|present=].
    4. +
  4. Set [=RTP timestamp=] from {{VideoFrameMetadata/rtpTimestamp}} if [=map/exist|present=].
    5. +
+
+
+

Local video capture timestamps

+

+ The user agent MUST set the [=capture timestamp=] of each video frame that is sourced from + {{MediaDevices/getUserMedia()}} and {{MediaDevices/getDisplayMedia()}} to its best estimate of the time that + the frame was captured. + This value MUST be monotonically increasing. +

+
+ Local capture tracks have a fixed offset between [=presentation timestamp=] and [=capture timestamp=]. The + user agent may let this be zero with the result that [=presentation timestamp=] is the same as [=capture timestamp=]. +
+
+ +

Exposing MediaStreamTrack source heuristic reactions support

Some platforms or User Agents may provide built-in support for video effects triggered by user motion heuristics, in particular for camera video streams.