diff --git a/website/docs/open_datasets/aria_digital_twin_dataset/aria_digital_twin_dataset.mdx b/website/docs/open_datasets/aria_digital_twin_dataset/aria_digital_twin_dataset.mdx index a68986e9f..91a4b7fbc 100644 --- a/website/docs/open_datasets/aria_digital_twin_dataset/aria_digital_twin_dataset.mdx +++ b/website/docs/open_datasets/aria_digital_twin_dataset/aria_digital_twin_dataset.mdx @@ -6,24 +6,41 @@ title: Overview ## Overview -Project Aria Tools provides Python and C++ APIs to access the [Aria Digital Twin (ADT) dataset](https://www.projectaria.com/datasets/adt/) ([paper](https://arxiv.org/abs/2306.06362)). +Project Aria Tools provides Python and C++ APIs to access the [Aria Digital Twin (ADT) dataset](https://www.projectaria.com/datasets/adt/) +([paper](https://openaccess.thecvf.com/content/ICCV2023/html/Pan_Aria_Digital_Twin_A_New_Benchmark_Dataset_for_Egocentric_3D_ICCV_2023_paper.html)). ## About the data -ADT provides raw and synthesized sensor data from Project Aria glasses, combined with groundtruth data generated using a motion capture system including depth images, device trajectories, object trajectories and bounding boxes, and human tracking. We also provide processed sensor data from our [Machine Perception Services](/ARK/mps/mps.mdx). Go to [ADT Data Format](data_format.mdx) to see a full list of the data we provide. +ADT provides raw and synthesized sensor data from Project Aria glasses, combined with groundtruth data generated using a motion capture +system including depth images, device trajectories, object trajectories and bounding boxes, and human tracking. +We also provide processed sensor data from our [Machine Perception Services](/ARK/mps/mps.mdx). +Go to [ADT Data Format](data_format.mdx) to see a full list of the data we provide. -The ADT dataset contains 222 sequences recording single and dual-person activities. The data was recorded in two spaces: an apartment and a single room office. There are 74 single-instance dynamic objects shared between the two spaces. +The ADT dataset contains 236 sequences recording single and dual-person activities. +The data was recorded in two spaces: an apartment and a single room office. +There are 74 single-instance dynamic objects shared between the two spaces. -Go to the [Getting Started Tutorial](getting_started.mdx) to explore the sample dataset (available on Google colab, no download necessary) or the [Dataset Download page](dataset_download.mdx) to get started. +Go to the [Getting Started Tutorial](getting_started.mdx) to explore the sample dataset (available on Google colab, no download necessary) +or the [Dataset Download page](dataset_download.mdx) to get started. -The sample dataset is a single-user dataset with body pose in the Apartment. It is a pretty representative example that should give you an idea of the dataset. +The sample dataset is a single-user dataset with body pose in the Apartment. +It is a pretty representative example that should give you an idea of the dataset. + +In addition to the sensor data and annotated ground truth data, we also provide high quality 3D object models (in .glb) for each of the objects in ADT. +For more info on downloading and viewing these models, see [Object Model](object_models.mdx) page. + +:::info NEW UPDATE! +3D object model release is new as of September 2024 +::: ### Apartment scene -170 sequences were recorded in the apartment scene. The apartment comprised of a living room, kitchen, dining room and bedroom and contained 281 unique stationary objects. +284 sequences were recorded in the apartment scene. The apartment comprised of a living room, kitchen, dining room and bedroom and contained +281 unique stationary objects. Given some objects have multiple instances that may differ slightly, the apartment has 324 stationary object instances in total. ### Office scene -52 sequences were recorded in the office scene, a single room with minimal office furniture. The office room contained 15 unique stationary objects and 20 stationary object instances. +52 sequences were recorded in the office scene, a single room with minimal office furniture. +The office room contained 15 unique stationary objects and 20 stationary object instances. ### Activities @@ -46,13 +63,6 @@ The dual-person activities included: Every activity has 10 to 50 sequences and the activity names are embedded into the sequence_names. -### Other statistics: - -* Number of multi-person sequences: 77 -* Number of sequences with no skeletons: 110 -* Number of sequences with 1 skeleton: 92 -* Number of sequences with 2 skeleton: 20 - :::info We provide a mix of datasets where users may or may not be wearing an Aria and/or a bodysuit. Please refer to the `skeleton_aria_association.json` to see the case for each specific sub-sequence. @@ -64,7 +74,9 @@ The ADT section of the wiki covers: * [Getting Started](getting_started.mdx) * A quickstart tutorial available as a Google colab project or install Project Aria Tools Python package to run locally, run the ADT notebook to access and visualize ground-truth data. * [Dataset Download](dataset_download.mdx) - * A walkthrough of using `adt_benchmark_dataset_downloader` to download the published ADT dataset. + * A walkthrough of using `aria_dataset_downloader` to download the ADT dataset. +* [Object Models](object_models.mdx) + * Everything you need to know about the released ADT object models including: a walkthrough of using `dtc_object_downloader` to download the ADT object models, and brief description of data content and how to visualize the models * [Data Format](data_format.mdx) * How ADT data is organized and stored * [Data Loader](data_loader.mdx) diff --git a/website/docs/open_datasets/aria_digital_twin_dataset/dataset_download.mdx b/website/docs/open_datasets/aria_digital_twin_dataset/dataset_download.mdx index 4c07b2571..1f3929be8 100644 --- a/website/docs/open_datasets/aria_digital_twin_dataset/dataset_download.mdx +++ b/website/docs/open_datasets/aria_digital_twin_dataset/dataset_download.mdx @@ -7,6 +7,7 @@ title: Dataset Download ## Overview This page covers how to download the sample Aria Digital Twin (ADT) sequence as well as how to download specific sequences and types of data. Follow the instructions to download the sample datasets and from there you'll be able to use the CLI to download more data. +For information on how to download ADT object models, see [Object Models](object_models.mdx) page The sample dataset is a single-user dataset with body pose in the Apartment. This is a pretty representative example that should give you an idea of the dataset. @@ -39,7 +40,7 @@ Scroll down to the bottom of the page. Enter your email and select **Access the Once you've selected **Access the Datasets** you'll be taken back to the top of the ADT page. -Scroll down the page to select **Aria Digital Twin Download Links** and download the file to the folder $HOME/Downloads. +Scroll down the page to select **ADT Sequences** link and download the file to the folder $HOME/Downloads.
@@ -56,8 +57,6 @@ You can redownload the download links whenever they expire ```bash mkdir -p $HOME/Documents/projectaria_tools_adt_data - -mv $HOME/Downloads/aria_digital_twin_dataset_download_urls.json $HOME/Documents/projectaria_tools_adt_data/ ``` ### Step 3 : Download the sample sequence (~500MB) via CLI: @@ -77,7 +76,7 @@ For more information on the content in the other sequences, see the Data Content ## Download the Aria Digital Twin (ADT) benchmark dataset ### Data size -The Aria Digital Twin dataset consists of 217 sequences. It is split into 4 data types that can be downloaded individually, plus MPS data. Go to [Project Aria Machine Perception Services](/ARK/mps/mps.mdx) for more information about MPS data. The MPS data is also broken into chunks that can be included or excluded at download time. The size of each data type is shown below. ADT data without MPS is approximately 3.5TB. +The Aria Digital Twin dataset consists of 236 sequences. It is split into 4 data types that can be downloaded individually, plus MPS data. Go to [Project Aria Machine Perception Services](/ARK/mps/mps.mdx) for more information about MPS data. The MPS data is also broken into chunks that can be included or excluded at download time. The size of each data type is shown below. ADT data without MPS is approximately 3.5TB. | **Data type** | What's included | Per sequence size | Total size for all sequences | | --- | --- | --- | --- | @@ -91,7 +90,6 @@ The Aria Digital Twin dataset consists of 217 sequences. It is split into 4 data | MPS SLAM online calibration | Online calibrations | < 20 MB | 1.2 GB | - ### Download via CLI ADT uses the general Aria dataset downloader, which is available in the [`projectaria_tools` PyPI (pip install) package](/data_utilities/installation/installation_python.mdx). diff --git a/website/docs/open_datasets/aria_digital_twin_dataset/object_models.mdx b/website/docs/open_datasets/aria_digital_twin_dataset/object_models.mdx new file mode 100644 index 000000000..00f5d2ddb --- /dev/null +++ b/website/docs/open_datasets/aria_digital_twin_dataset/object_models.mdx @@ -0,0 +1,115 @@ +--- +sidebar_position: 35 +title: Object Models +--- +# ADT Object Models + +## Overview + +This page covers how to download the 3D object models associated with the Aria Digital Twin (ADT) dataset, +and briefly describe the data content and how to visualize the models. + +By downloading the object models as described in this page, you agree that you have read and accepted the +terms of the [Aria Digital Twin Object Models License Agreement](https://www.projectaria.com/datasets/adt/license/). + +## Download the ADT Object Models + +### Step 0: install project_aria_tools package and create venv if not done before + +Follow Step 0 to Step 3 in [Getting Started](/docs/open_datasets/aria_digital_twin_dataset/getting_started). + +### Step 1 : Signup and get the download links file + +To use the downloader CLI, you need to download a file which contains all data URLs plus some metadata for the downloader script. +We currently offer two ways of getting this file: + +Visit [ADT website](https://www.projectaria.com/datasets/adt/) and sign up. + +Scroll down to the bottom of the page. Enter your email and select **Access the Datasets**. + +![Screenshot from website showing download dataset](/img/open_datasets/aria_digital_twin_dataset/download_page.png) + +Once you've selected **Access the Datasets** you'll be taken back to the top of the ADT page. + +Scroll down the page to select **ADT Object 3D Models** link and download the file to the folder $HOME/Downloads. + +
+ +![ADT Website Signup Image](/img/open_datasets/aria_digital_twin_dataset/download_button.png) + +
+ +:::info The download-links file will expire in 14 days +You can redownload the download links whenever they expire +::: + + +### Step 2 : Set up a folder for object models + +```bash +mkdir -p $HOME/Documents/adt_object_library +``` + +### Step 3 : Download the object models via CLI: + +To download all ADT models (~5.5 GB), run the following command from your Python virtual environment: + +```bash +dtc_object_downloader -r ADT -c {PATH_TO_YOUR_CDN_FILE} -o $HOME/Documents/adt_object_library +``` + +To download only a select subset of ADT models, run the following command from your Python virtual environment: + +```bash +dtc_object_downloader -r ADT -c {PATH_TO_YOUR_CDN_FILE} -o $HOME/Documents/adt_object_library -l {SELECTED_OBJECT_NAMES} +``` + +You can retrieve object names from the ADT data providers using your downloaded sequence data, or have a look at the instances.json file in your sequence data. + +:::warning +For legal reasons, we have **removed** the object **DinoToy** from the available models, however, this object is still present in the ADT sequence data. +::: + +### Troubleshooting + +Check [Project Aria Tools troubleshooting](/docs/data_utilities/installation/troubleshooting) if you are having issues using this guide. + +## Object Model Data Format + +When downloading the object models using dtc_object_downloader, each object will download into its own folder, named using the instance name of the object. +Under each object folder, there will be three files as shown below. + +``` +|InstanceName1| + ├──3d-asset.glb # model file + ├──CC_BY-SA.txt # license file + ├──metadata.json +|InstanceName2| + {...} +``` + +All poses associated with the objects are stored in each sequence ground truth data, not in the downloaded object data. +For example code to learn how to load object models and their associated poses (including static and dynamic objects), +refer to the [ADT Rerun Viewer](https://github.com/facebookresearch/projectaria_tools/blob/main/projectaria_tools/utils/viewer_projects_adt.py) + +## Visualizing Object Models + +The GLB files for each object model can be viewed using any off-the-shelf GLB viewer. +For example, the open web browser based [Babylon viewer](https://sandbox.babylonjs.com/) can be used by dragging and dropping the GLB file. + +To visualize multiple object models, along with their ground truth poses, the ADT rerun viewer can be used. +To run the viewer, enter the following command from your virtual environment with projectaria_tools installed: + + +``` +viewer_projects_adt --sequence_path {PATH_TO_YOUR_SEQUENCE} --object_library_path $HOME/Documents/adt_object_library --display_objects {INSTANCE_NAMES} +``` +![ADT Model Viewer Screenshot](/img/open_datasets/aria_digital_twin_dataset/rerun-adt-models.png) + +Example command to produce the above visualization: + +``` +viewer_projects_adt --sequence_path $HOME/Documents/adt_sequence_data/Apartment_release_golden_skeleton_seq100_M1292 \ +--object_library_path $HOME/Documents/adt_object_library \ +--display_objects BlackCoffeeTable BirdHouseToy YellowArmChair GreySofa RedArmChair +``` diff --git a/website/static/img/open_datasets/aria_digital_twin_dataset/download_button.png b/website/static/img/open_datasets/aria_digital_twin_dataset/download_button.png index cc3df760b..b1d739d3a 100644 Binary files a/website/static/img/open_datasets/aria_digital_twin_dataset/download_button.png and b/website/static/img/open_datasets/aria_digital_twin_dataset/download_button.png differ diff --git a/website/static/img/open_datasets/aria_digital_twin_dataset/rerun-adt-models.png b/website/static/img/open_datasets/aria_digital_twin_dataset/rerun-adt-models.png new file mode 100644 index 000000000..2196504aa Binary files /dev/null and b/website/static/img/open_datasets/aria_digital_twin_dataset/rerun-adt-models.png differ