1.5.6

🎉We’re excited to announce Project Aria Tools v1.5.6🎉

[CodeSample]

• [MPS]
  • Added a code sample to show how to load and use semi dense point clouds (enable you to easily know points visible at a given timestamps on a given image stream id)
• [Core]
  • Vignette masks (enable you to de-vignette image streams)

[ARK]
Added support of mps-cli for Windows platform

1.5.5

🎉We’re excited to announce Project Aria Tools v1.5.5🎉

[PyPI]

• Added Windows support to our wheels on PyPI

[MPS]

• Shown how to best use SLAM online calibration and a time offset for the RGB camera
• MPSDataProvider can now provide the version ID of the MPS service

[Documentation]

• How to install projectaria_tools from pypi for windows
• How to install VRS tools and VRS Player
  • How to use VRSPlayer
  • How to use VRS tools to trim VRS file in time or stream wise

[ADT]

• All objects in ADT now have downloadable 3D models in glb format
• ADT rerun visualizer now supports visualizing 3D object models

1.5.4

🎉 We’re excited to announce Project Aria Tools v1.5.4 🎉

[Core]

Projectaria_tools can now be compiled for native windows support. Assuming Visual Studio Compiler is setup on the machine (default to Visual Studio 16 2019 but Visual Studio 17 2022 is also supported, if generator is updated in the cmake command line -G)

# C++ compilation + unit test run with Pixi package manager
pixi run run_c
# Python compilation + unit test run with Pixi package manager
pixi run run_python

[Dataset]
ADTv2 Tutorial notebook update

[Documentation]
Added HOT3D to the OpenDataset wiki page
Enhanced documentation of ARK TimeSync feature

[MPS/MPS-CLI]
Add support of Normals for wrist and palm helping to get the orientation of hands

[MPS Code Sample]
Python notebook showing how to align MediaPipe Hand tracking with MPS wrist and palm tracking

1.5.3

{Release} 1.5.2 -> 1.5.3

Summary:
Prepare for release 1.5.3

- Update version number
- Update links for GoogleColab files

Reviewed By: vijay609, danielyan86129, PiotrBrzyski

Differential Revision: D60532286

fbshipit-source-id: 78a507532014f58ccf4b1680afc16bfd1c341fec

1.5.2

🎉 We’re excited to announce Project Aria Tools v1.5.2. 🎉

[API]

• VrsDataProvider
  • Add support of VRS metadata/file tags and time sync mode
  • Calibration
    • Add better support for rotating calibration data (intrinsics, extrinsics)
      • See usage either on wiki or in the AEA dataset viewer
        Expose camera calibration rescaling function

[Code Samples]

• C++
  • VRS Image mutation
• Python
  • TicSync notebook tutorial
  • MPS EyeGaze Depth Estimation
  • ADT - Notebook on converting depth maps to point clouds

[Tools - Python]

• VRS to MP4
  • Improved timing accuracy during export
  • Updated to include timestamps in the metadata

[MPS-CLI]

• API update to mps-cli to support Aria Studio

[Build]

• Continuous Integration (adapt to changing software/hardware environment)
  • C++ - Fix for Pangolin
  • Python wheels - Fix cibuildwheel version
  • Python wheels - Fix GitHub runner to be Intel or Mac Silicon

[Visualization]

• MPS Viewer - Enable showing RGB images upright, undistorted, or unchanged

[Projects/Dataset]

• ADT
  • Data & Tooling Updated to v2.0
    • We have removed the notion of subsequences. All subsequence are now separated into their own sequence folder
    • Tooling including visualization and downloaders have been updated to new file structure, but are still backwards compatible
    • We have removed access to challenge data now that the challenge has completed
  • Visualization
    • Python - Add option to add image rectification to the viewer
    • C++ - Update Pangolin viewer to better aspect ratio
• AEA
  • Visualization - Python - Ease image rotation/undistortion

[Documentation]

• Various improvements, including:
  • New Pairing Additional Glasses and Troubleshooting
  • New Project Aria Glasses Cable Clip instructions
  • Updated Aria Digital Twin (ADT) v2.0 updated to support Aria Dataset Explorer
  • New Aria Dataset Explorer
  • New Aria Studio
  • New Time Synchronized Recordings with Multiple Aria Devices
  • New Project Aria FAQ
  • Updated Info page and About ARK page

1.5.1

{Release} 1.5.0 -> 1.5.1

Summary:
Prepare for release 1.5.1
- Update version number

Reviewed By: prithvirb

Differential Revision: D58143830

fbshipit-source-id: 2fe55bafe176747c5b8e1d47e796e6ec9c6c2c6d

1.5.0

🎉 We’re excited to announce Project Aria Tools v1.5.0. 🎉

[Tools - Python]

• ASR code sample shows how to use Faster Whisper to run Whisper Speech Recognition on an Aria audio stream. The ASR outputs are time aligned with Aria Device Time. Go to the Automated Speech Recognition Readme for how to get started.

[MPS CLI]

• Support for new MPS service, hand tracking
• Updated Eye Tracking to support depth estimation

[Documentation]

• Various improvements, including:
• New MPS Troubleshooting page
• New Collaborative Tools Page, did you know you can use Aria data with Nerfstudio?
• New Hand Tracking Data Formats
• Updated MPS Google Colab tutorials

1.4.0

🎉 We are excited to announce the release of our new 1.4.0 version! 🎉

• MPS CLI (ARK|Project Aria Research partners)
  • New (and recommended) workflow for requesting Machine Perception services.
  • MPS Multi-SLAM can only be requested by the MPS CLI.
• MPS Multi-SLAM (ARK|Project Aria Research partners)
  • Compute SLAM MPS outputs in a shared coordinate frame for multiple VRS files

Here is the complete changelog:

[API]

• DataProvider - {Feature} - Make DeliverQueue start and end timestamp logic more robust
• Calibration - {Feature} calibration - expose imu rectification matrix and bias vector
• Viewer MPS - {Visualizer} Python - viewer_mps - Add mps_folder option

[MPS CLI]

• aria_mps - Addition of a Python CLI for
  • Project Aria Research partners to manage their Machine Perception Services requests (run and monitor upload, run and results retrieval). Improvements on the Desktop app include:
    • Auto run health checks prior to upload
    • Recordings will not be uploaded if they are not valid for any of the MPS services requested
    • Resumable uploads
    • Concurrent processing
    • Automatically downloads outputs once processing is complete
    • Recordings are processed once
    • Uploaded data is stored for 24 hours
      • Additional MPS can be requested without needing to upload again
      • Data can be reprocessed without needing to upload again
    • CLI can be integrated into automated workflows
• Request Multi-Recording outputs
  • Compute SLAM MPS outputs in a shared coordinate frame for multiple VRS files
• Projectaria_tools must be installed using pip to access the CLI

[Documentation]

• aria_mps CLI documentation
• MPS Data Formats documentation refactored to align with MPS CLI output structure
  Minor improvements to other documentation

1.3.3

🎉 We are excited to announce the release of version 1.3.3 🎉
This release includes several new features and improvements, including:

• Features
  • A MpsDataPathsProvider & MPSDataProvider API
• Dataset support
  • Aria Everyday Activities (AEA) Dataset support
• NEW - Code Sample
  • Demo to run EfficientSAM with eye gaze prompt
• CI/CD/Build
  • Improved our PyPI Python wheel generation workflow

Here is the complete changelog:

[Visualization]

• viewer_mps update to load multi sequence datasets (AEA, ADT)
• viewer_projects_aea

[API]

• MPS
  • Add MpsDataPathsProvider & MPSDataProvider API
    Allowing to easily retrieve MPS file assets and query them their metadata by timestamp

[Dataset]
AEA Aria EveryDay Activities dataset
ADT Aria Digital Twin dataset is updated with MPS data

[Build/CI/CD]

• {Build} Improve FMT compatibility (#54)
• {Build} GitHub CI - Python Wheels generation
• {Build} Fix python wheel build workflow for PyPI

[Documentation]

• AEA documentation
  Minor improvements to other documentation

[Thank you to our new contributors]
@selcuk-meta
@eric-fb

1.3.0

🎉 We are excited to announce the release of version 1.3.0 🎉

This release includes several major new features and improvements, including:

• New python visualization samples
• A VRS_to_MP4 tool to help quickly review (RGB + Sound) for data collection
• New C++ and Python tutorials on MPS point cloud colorization & how to use ADT depth map to generate point cloud
  …

Here is the complete changelog:

[Visualization]

• [Python] New rerun visualization samples to help debug and visualize Aria and MPS temporal data & states
  • viewer_aria_sensors
  • viewer_mps
  • viewer_projects_adt
  • viewer_projects_ase

[API]

• [Python] Introduce an projectaria_tools.mps.utils module to help query and filter loaded MPS data

import projectaria_tools.mps.utils

# Retrieve Pose/Eye Gaze data by timestamp 
get_nearest_eye_gaze, get_nearest_pose

# Reproject eyegaze vector in image
get_gaze_vector_reprojection

# Filter Point Cloud data
filter_points_from_confidence
filter_points_from_count

• [Python - C++] Image undistortion update distort_by_calibration
  • API update to perform bilinear or nearest neighbor multithread interpolation to better select the right interpolation for depth (bilinear) or segmentation mask (nearest) -> see (distort_by_calibration, distort_depth_by_calibration & distort_label_by_calibration)
• [Python - C++] Calibration rotation
  • Ease accessibility to upright pinhole calibration data for RGB/SLAM images rotate_camera_calib_cw90deg

[Tools]

• [Python] Tool to create an MP4 file from VRS RGB and audio data (code, documentation)
• [C++] PointCloud Colorization Sample
• [C++] Aria Viewer - Enable plot buffer for AriaViewer to Avoids slow down/ speed up when audio is enabled/disabled
• [C++/Python] Vrs health check
  add health check for VRS streams (audio, barometer, bluetooth, camera, gps, imu, wifi) -> Reads all records from all streams and check the health of each record in each stream
• [Python] Projects/Aria Digital Twin - Add notebook tutorial to create and merge point clouds from depth maps data

[Continuous integration - GitHub]

• Various cleanup in GitHub actions
• Improved CI code coverage by adding testing python notebooks

[BugFix]

• [Core] fix support of multiple gps streams (coming from Aria and cell phone)

[Known Issues]

• Machine Perception Services (MPS) outputs have been renamed, so that they more clearly communicate what is in the outputs:

SLAM/Trajectory
  - global_points.csv.gz -> semidense_points.csv.gz
Eye Gaze
  - generalized_eye_gaze.csv -> general_eye_gaze.csv
  - calibrated_eye_gaze.csv -> personalized_eye_gaze.csv

[Documentation]

• Aria Digital Twin, new landing and challenge page
• Aria Synthetic Environments, new landing and challenge page
• Data Formats - how Project Aria uses VRS
• Data Utilities - refactored our visualization guide to have aPython and a C++ page
• Refactored downloading MPS sample data into a single download page

[Thank you to our new contributors]
@baderouaich