index.bs

<pre class="metadata">
Shortname: webxr-body-tracking
Title: WebXR Body Tracking Module - Level 1
Group: immersivewebwg
Status: UD
TR: https://www.w3.org/TR/webxr-body-tracking-1/
ED: https://immersive-web.github.io/webxr-body-tracking/
Previous Version: https://www.w3.org/TR/2020/WD-webxr-body-tracking-1-20210824/
Repository: immersive-web/webxr-body-tracking
Level: 1
Mailing List Archives: https://lists.w3.org/Archives/Public/public-immersive-web-wg/

!Participate: <a href="https://github.com/immersive-web/webxr-body-tracking/issues/new">File an issue</a> (<a href="https://github.com/immersive-web/webxr-body-tracking/issues">open issues</a>)
!Participate: <a href="https://lists.w3.org/Archives/Public/public-immersive-web-wg/">Mailing list archive</a>
!Participate: <a href="irc://irc.w3.org:6665/">W3C's #immersive-web IRC</a>

Editor: Rik Cabanier, Meta, cabanier@meta.com

Abstract: The WebXR Body Tracking module expands the <a href="https://www.w3.org/TR/webxr/">WebXR Device API</a> with the functionality to track articulated body poses.

Status Text: This WebXR Augmented Reality Module is designed as a module to be implemented in addition to <a href="https://www.w3.org/TR/webxr/">WebXR Device API</a>, and is originally included in WebXR Device API which was divided into core and modules.
</pre>


<link rel="icon" type="image/png" sizes="32x32" href="favicon-32x32.png">
<link rel="icon" type="image/png" sizes="96x96" href="favicon-96x96.png">

<style>
  .unstable::before {
    content: "This section is not stable";
    display: block;
    font-weight: bold;
    text-align: right;
    color: red;
  }
  .unstable {
    border: thin solid pink;
    border-radius: .5em;
    padding: .5em;
    margin: .5em calc(-0.5em - 1px);
    background-image: url("data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' width='300' height='290'><text transform='rotate(-45)' text-anchor='middle' font-family='sans-serif' font-weight='bold' font-size='70' y='210' opacity='.1'>Unstable</text></svg>");
    background-repeat: repeat;
    background-color: #FFF4F4;
  }
  .unstable h3:first-of-type {
    margin-top: 0.5rem;
  }

  .unstable.example:not(.no-marker)::before {
    content: "Example " counter(example) " (Unstable)";
    float: none;
  }

  .non-normative::before {
    content: "This section is non-normative.";
    font-style: italic;
  }
  .tg {
    border-collapse: collapse;
    border-spacing: 0;
  }
  .tg th {
    border-style: solid;
    border-width: 1px;
    background: var(--def-bg);
    font-family: sans-serif;
    font-weight: bold;
    border-color: var(--def-border);
  }
  .tg td {
    padding: 4px 5px;
    background-color: var(--def-bg);
    font-family: monospace;
    border-style: solid;
    border-width: 1px;
    border-color: var(--def-border);
    overflow: hidden;
    word-break: normal;
  }
</style>


<pre class="link-defaults">
spec: webxr;
    type: dfn; text: xr device; for: /
    type: dfn; text: xr device; for: XRSession
    type: dfn; text: animation frame; for: XRSession
    type: dfn; text: explicit consent
    type: dfn; text: native origin
    type: dfn; text: feature descriptor
    type: dfn; text: populate the pose
    type: dfn; text: session; for: XRFrame
    type: dfn; text: time; for: XRFrame
    type: dfn; text: session; for: XRSpace
spec:infra; type:dfn; text:list
spec:webxr-ar-module-1; type:enum-value; text:"immersive-ar"
spec:webidl;
    type:dfn; text:value pairs
</pre>

<pre class="anchors">
spec:html; urlPrefix: https://html.spec.whatwg.org/multipage/
    type: dfn; text: browsing context; url: browsers.html#browsing-context
</pre>


Introduction {#intro}
============

<section class="non-normative">
On some [=/XR device|XR devices=] it is possible to get fully articulated information about the user's body.

This API exposes the poses of each of the users' [=body joints=]. This can be used to render an avatar in VR scenarios.

</section>


Initialization {#initialization}
==============

If an application wants to get access to body information during a session,
the session MUST be requested with an appropriate [=feature descriptor=]. The string "<dfn for="feature descriptor">body-tracking</dfn>" is introduced
by this module as a new valid [=feature descriptor=] for articulated body tracking.

The "[=body-tracking=]" [=feature descriptor=] should only be granted for an {{XRSession}} when its [=XRSession/XR device=] has support for body tracking.

XRFrame {#xrinputsource-interface}
-------------

<pre class="idl">
partial interface XRFrame {
   [SameObject] readonly attribute XRBody? body;
};
</pre>

The <dfn attribute for=XRFrame>body</dfn> attribute on an [=XR Device=] that has support for body tracking will be an {{XRBody}} object giving access to the underlying body-tracking capabilities. {{XRFrame/body}} will be set to [=this=].

If the {{XRFrame}} belongs to an {{XRSession}} that has not been requested with the "[=body-tracking=]" [=feature descriptor=], {{XRFrame/body}} MUST be <code>null</code>.

Body Joints {#body-joints-section}
---------------

A {{XRFrame/body}} is made up of many <dfn>body joints</dfn>.

A [=body joint=] for a given body can be uniquely identified by a body joint name, which is an enum of type {{XRBodyJoint}}.

A [=body joint=] may have an <dfn>associated bone</dfn> that it is named after and used to orient its <code>-Z</code> axis. The [=associated bone=] of a [=body joint=] is the bone that comes after the joint when moving towards the fingertips. The tip and wrist joints have no [=associated bones=].

XRBody {#xrbody-interface}
------

<pre class="idl">
enum XRBodyJoint {
  "root",
  "hips",
  "spine-lower",
  "spine-middle",
  "spine-upper",
  "chest",
  "neck",
  "head",
  "left-shoulder",
  "left-scapula",
  "left-arm-upper",
  "left-arm-lower",
  "left-hand-wrist-twist",
  "right-shoulder",
  "right-scapula",
  "right-arm-upper",
  "right-arm-lower",
  "right-hand-wrist-twist",
  "left-hand-palm",
  "left-hand-wrist",
  "left-hand-thumb-metacarpal",
  "left-hand-thumb-phalanx-proximal",
  "left-hand-thumb-phalanx-distal",
  "left-hand-thumb-tip",
  "left-hand-index-metacarpal",
  "left-hand-index-phalanx-proximal",
  "left-hand-index-phalanx-intermediate",
  "left-hand-index-phalanx-distal",
  "left-hand-index-tip",
  "left-hand-middle-metacarpal",
  "left-hand-middle-phalanx-proximal",
  "left-hand-middle-phalanx-intermediate",
  "left-hand-middle-phalanx-distal",
  "left-hand-middle-tip",
  "left-hand-ring-metacarpal",
  "left-hand-ring-phalanx-proximal",
  "left-hand-ring-phalanx-intermediate",
  "left-hand-ring-phalanx-distal",
  "left-hand-ring-tip",
  "left-hand-little-metacarpal",
  "left-hand-little-phalanx-proximal",
  "left-hand-little-phalanx-intermediate",
  "left-hand-little-phalanx-distal",
  "left-hand-little-tip",
  "right-hand-palm",
  "right-hand-wrist",
  "right-hand-thumb-metacarpal",
  "right-hand-thumb-phalanx-proximal",
  "right-hand-thumb-phalanx-distal",
  "right-hand-thumb-tip",
  "right-hand-index-metacarpal",
  "right-hand-index-phalanx-proximal",
  "right-hand-index-phalanx-intermediate",
  "right-hand-index-phalanx-distal",
  "right-hand-index-tip",
  "right-hand-middle-metacarpal",
  "right-hand-middle-phalanx-proximal",
  "right-hand-middle-phalanx-intermediate",
  "right-hand-middle-phalanx-distal",
  "right-hand-middle-tip",
  "right-hand-ring-metacarpal",
  "right-hand-ring-phalanx-proximal",
  "right-hand-ring-phalanx-intermediate",
  "right-hand-ring-phalanx-distal",
  "right-hand-ring-tip",
  "right-hand-little-metacarpal",
  "right-hand-little-phalanx-proximal",
  "right-hand-little-phalanx-intermediate",
  "right-hand-little-phalanx-distal",
  "right-hand-little-tip",
  "left-upper-leg",
  "left-lower-leg",
  "left-foot-ankle-twist",
  "left-foot-ankle",
  "left-foot-subtalar",
  "left-foot-transverse",
  "left-foot-ball",
  "right-upper-leg",
  "right-lower-leg",
  "right-foot-ankle-twist",
  "right-foot-ankle",
  "right-foot-subtalar",
  "right-foot-transverse",
  "right-foot-ball"
};

[Exposed=Window]
interface XRBody {
    iterable&lt;XRBodyJoint, XRBodySpace>;

    readonly attribute unsigned long size;
    XRBodySpace get(XRBodyJoint key);
};
</pre>

The {{XRBodyJoint}} enum defines the various joints that each {{XRBody}} MUST contain.

<div algorithm="XRBody.[[joints]]">
    Each {{XRBody}} object has a <dfn attribute for="XRBody">\[[joints]]</dfn> internal slot,
    which is an [=ordered map=] of pairs with the key of type {{XRBodyJoint}} and the value of type {{XRBodySpace}}.

    The ordering of the {{[[joints]]}} internal slot is given by the {{XRBodyJoint|list of joints}} under [=body joints=].

    {{[[joints]]}} MUST NOT change over the course of a session.
</div>

<div algorithm="XRBody.iterator">
    The [=value pairs to iterate over=] for an {{XRBody}} object are the list of [=value pairs=] with the key being
    the {{XRBodyJoint}} and the value being the {{XRBodySpace}} corresponding to that {{XRBodyJoint}}, ordered by {{XRBodyJoint|list of joints}}}
    under [=body joints=].
</div>

If an individual device does not support a joint defined in this specification, it MUST emulate it instead.

The <dfn attribute for=XRBody>size</dfn> attribute MUST return the number <code>84</code>.

<div class="algorithm" data-algorithm="index-joint-space">
The <dfn method for="XRBody">get(|jointName|)</dfn> method when invoked on an {{XRBody}} [=this=] MUST run the following steps:

  1. Let |joints| be the value of [=this=]'s {{[[joints]]}} internal slot.
  2. Return |joints|[|jointName|]. (This implies returning <code>undefined</code> for unknown |jointName|.)

</div>

XRBodySpace {#xrjointspace-interface}
-------------

<pre class="idl">
[Exposed=Window]
interface XRBodySpace: XRSpace {
  readonly attribute XRBodyJoint jointName;
};
</pre>

The [=native origin=] of an {{XRBodySpace}} is the position and orientation of the underlying [=XRBodySpace/joint=].

The [=native origin=] of the {{XRBodySpace}} may only be reported when [=native origins=] of all other {{XRBodySpace}}s on the same [=XRBodySpace/body=] are being reported. When a body is partially obscured the user agent MUST either emulate the obscured joints, or report null poses for all of the joints.

Note: This means that when fetching poses you will either get an entire body or none of it.

The [=native origin=] has its <code>-Y</code> direction pointing perpendicular to the skin, outwards from the palm, and <code>-Z</code> direction pointing along their associated bone, away from the wrist.

For tip [=body joints=] where there is no [=associated bone=], the <code>-Z</code> direction is the same as that for the associated distal joint, i.e. the direction is along that of the previous bone. 

The {{XRBodyJoint/left-hand-wrist}} and {{XRBodyJoint/right-hand-wrist}} [=body joints=] are located at the pivot point of the wrist, which is location invariant when twisting the hand without moving the forearm. The backward (+Z) direction is parallel to the line from wrist joint to middle finger metacarpal joint, and points away from the finger tips. The up (+Y) direction points out towards back of the hand and perpendicular to the skin at wrist. The X direction is perpendicular to the Y and Z directions and follows the right hand rule.

The {{XRBodyJoint/left-hand-palm}} and {{XRBodyJoint/right-hand-palm}} [=body joints=] are located at the center of the middle finger’s metacarpal bone. The backward (+Z) direction is parallel to the middle finger’s metacarpal bone, and points away from the finger tips. The up (+Y) direction is perpendicular to palm surface and pointing towards the back of the hand. The X direction is perpendicular to the Y and Z directions and follows the right hand rule.

Every {{XRBodySpace}} has an associated <dfn for=XRBodySpace>body</dfn>, which is the {{XRBody}} that created it.

<dfn for=XRBodySpace>jointName</dfn> returns the joint name of the joint it tracks.

Every {{XRBodySpace}} has an associated <dfn for=XRBodySpace>joint</dfn>, which is the [=body joint=] corresponding to the [=XRBodySpace/jointName=].


Privacy & Security Considerations {#privacy-security}
=================================
The WebXR Body Tracking API is a powerful feature that carries significant privacy risks.

Since this feature MAY return new sensor data, the User Agent MUST ask for [=explicit consent=] from the user at session creation time.

Data returned from this API, MUST NOT be so specific that one can detect individual users.
If the underlying hardware returns data that is too precise, the User Agent MUST anonymize this data
before revealing it through the WebXR Body Tracking API.

This API MUST only be supported in XRSessions created with XRSessionMode of {{XRSessionMode/"immersive-vr"}}
or {{XRSessionMode/"immersive-ar"}}. {{XRSessionMode/"inline"}} sessions MUST not support this API.

<div class="note">
When anonymizing the body data, the UA can follow these guidelines:
 * Noising is discouraged in favour of rounding.
 * If the UA uses rounding, each joint must not be rounded independently. Instead the correct way to round is to map each body to a static body-model.
 * If noising, the noised data must not reveal any information over time:
    - Each new WebXR session in the same [=browsing context=] must use the same noise to make sure that the data cannot be de-noised by creating multiple sessions.
    - Each new [=browsing context=] must use a different noise vector.
    - Any seed used to initialize the noise must not be predictable.
 * Anonymization must be done in a trusted environment.

</div>

<h2 id="changes" class="no-num">
Changes</h2>