Skip to content

Latest commit

 

History

History
242 lines (171 loc) · 15.5 KB

ARCHITECTURE.md

File metadata and controls

242 lines (171 loc) · 15.5 KB

Rerun architecture

This document describes the technical architecture of Rerun.

See also

The major components

Logging APIs

It all starts with logging. You can log rich data (point clouds, images, etc) with either our Python SDK or our Rust SDK.

The logging SDK:s encodes the data using Apache Arrow (see more below).

The logging data can be written to disk as .rrd files, or transmitted over TCP to either a Rerun Viewer or a Rerun Server.

Rerun Viewer

The Rerun Viewer is where log data is visualized. It is usually run as a native app, but can also be compiled to WebAssembly (Wasm) and run in a browser.

Native Viewer

The easiest way to launch the Viewer is directly from the logging API with rr.init("rerun_example_app", spawn=True). However, the standalone Viewer can also be run from the command line, for example to view an .rrd file: rerun mydata.rrd.

Web viewer

You can try running the Viewer in a browser using rr.serve() in Python, or using rerun --web-viewer mydata.rrd.

The web viewer consists of just a few small files - a thin .html, a .wasm blob, and an auto-generated .js bridge for the wasm. These files are served using the re_web_viewer_server crate.

The web viewer can load .rrd files (just drag-drop them into the browser), or read logging data streamed over WebSockets.

.rrd files

.rrd ("Rerun Data") is just a bunch of log messages appended one after the other to a file.

NOTE: .rrd files do not yet guarantee any backwards or forwards compatibility. One version of Rerun will likely not be able to open an .rrd file generated by another Rerun version.

Technologies we use

Apache Arrow

Apache Arrow is a language-independent columnar memory format for arbitrary data. We use it to encode the log data when transmitting it over the network or storing it in an .rrd file. We also use it in our in-RAM data store, re_chunk_store.

In Rust, we use the arrow2 crate.

wgpu

The Rerun Viewer uses the wgpu graphics API. It provides a high-performance abstraction over Vulkan, Metal, D3D12, D3D11, OpenGLES, WebGL and WebGPU. This lets us write the same code graphics code for native as for web.

On web builds, we use WebGPU when available on the Web, but automatically fall back to a WebGL based emulation layer (with a more limited feature set).

We have written our own high-level rendering crate on top of wgpu, called re_renderer.

egui

The GUI in the Rerun Viewer is using egui, a cross-platform, immediate mode GUI.

We use eframe, the egui framework, to run egui on both native and web.

Wasm

Wasm (short for WebAssembly) is a binary instruction format supported by all major browser. The Rerun Viewer can be compiled to Wasm and run in a browser.

Threading support in Wasm is nascent, so care must we taken that we don't spawn any threads when compiling for wasm32.

Wasm has no access to the host system, except via JS calls (something that may change once WASI rolls out), so when compiling for wasm32 you can NOT use the Rust standard library to:

  • Access files
  • Read environment variables
  • Get the current time (use instant instead)
  • Use networking (use ehttp, reqwest, or ewebsock instead)
  • etc

Immediate mode

The Rerun Viewer uses an immediate mode GUI, egui. This means that each frame the entire GUI is being laid out from scratch.

In fact, the whole of the Rerun Viewer is written in an immediate mode style. Each rendered frame it will query the in-RAM data store, massage the results, and feed it to the renderer.

The advantage of immediate mode is that is removes all state management. There is no callbacks that are called when some state has already changed, and the state of the blueprint is always in sync with what you see on screen.

Immediate mode is also a forcing function, forcing us to relentlessly optimize our code. This leads to a very responsive GUI, where there is no "hickups" when switching data source or doing time scrubbing.

Of course, this will only take us so far. In the future we plan on caching queries and work submitted to the renderer so that we don't perform unnecessary work each frame. We also plan on doing larger operation in background threads. This will be necessary in order to support viewing large datasets, e.g. several million points. The plan is still to do so within an immediate mode framework, retaining most of the advantages of stateless code.

Crates

Here is an overview of the crates included in the project:

SDK/CLI/Wasm top-level crates

Crate Description
rerun-cli Rerun native CLI binary crate
Rerun Rerun Rust SDK and Viewer shim crate
rerun_c Rerun C SDK
rerun_py Rerun Python SDK
re_sdk Rerun logging SDK

Viewer crates

UI crates
Crate Description
re_blueprint_tree The UI for the blueprint tree in the left panel.
re_chunk_store_ui A chunk store browser UI.
re_component_ui Provides UI editors for Rerun component data for registration with the Rerun Viewer component UI registry.
re_selection_panel The UI for the selection panel.
re_space_view Types & utilities for defining Space View classes and communicating with the Viewport.
re_space_view_bar_chart A Space View that shows a single bar chart.
re_space_view_dataframe A Space View that shows the data contained in entities in a table.
re_space_view_map A Space View that shows geospatial data on a map.
re_space_view_spatial Space Views that show entities in a 2D or 3D spatial relationship.
re_space_view_tensor A Space View dedicated to visualizing tensors with arbitrary dimensionality.
re_space_view_text_document A simple Space View that shows a single text box.
re_space_view_text_log A Space View that shows text entries in a table and scrolls with the active time.
re_space_view_time_series A Space View that shows plots over Rerun timelines.
re_time_panel The time panel of the Rerun Viewer, allowing to control the displayed timeline & time.
re_viewer The Rerun Viewer
re_viewport The central viewport panel of the Rerun viewer.
UI support crates
Crate Description
re_context_menu Support crate for context menu and actions.
re_data_ui Provides UI elements for Rerun component data for the Rerun Viewer.
re_renderer A wgpu-based renderer for all your visualization needs.
re_ui Rerun GUI theme and helpers, built around egui
re_viewer_context Rerun Viewer state that is shared with the viewer's code components.
re_viewport_blueprint The data model description of the viewport panel.

Application-level store

Crate Description
re_entity_db In-memory storage of Rerun entities
re_query Querying data in the re_chunk_store
re_dataframe The Rerun public data APIs.
re_remote_store_types Rerun remote store gRPC API types
re_types The built-in Rerun data types, component types, and archetypes.
re_types_blueprint The core traits and types that power Rerun's Blueprint sub-system.
re_log_encoding Helpers for encoding and transporting Rerun log messages

Low-level store

Crate Description
re_chunk A chunk of Rerun data, encoded using Arrow. Used for logging, transport, storage and compute.
re_chunk_store An in-memory time series database for Rerun log data, based on Apache Arrow.
re_log_types The basic building blocks of the Rerun data types and tables.
re_types_core The core traits and types that power Rerun's data model.
re_format_arrow Formatting of Apache Arrow tables.

Data flow

Crate Description
re_sdk_comms TCP communication between Rerun SDK and Rerun Server
re_web_viewer_server Serves the Rerun web viewer (Wasm and HTML) over HTTP
re_ws_comms WebSocket communication library (encoding, decoding, client, server) between a Rerun server and Viewer
re_data_loader Handles loading of Rerun data from file using data loader plugins
re_data_source Handles loading of Rerun data from different sources

Build support

Crate Description
re_build_info Information about the build. Use together with re_build_tools
re_build_tools build.rs helpers for generating build info
re_remote_store_types_builder Generates code for Rerun remote store gRPC API
re_types_builder Generates code for Rerun's SDKs from flatbuffers definitions.
re_dev_tools Various tools for Rerun development. Each tool has a subcommand.

Utilities

Crate Description
re_analytics Rerun's analytics SDK
re_case Case conversions, the way Rerun likes them
re_crash_handler Detect panics and signals, logging them and optionally sending them to analytics.
re_error Helpers for handling errors.
re_format Miscellaneous tools to format and parse numbers, durations, etc.
re_int_histogram A histogram with i64 keys and u32 counts, supporting both sparse and dense uses.
re_log Helpers for setting up and doing text logging in the Rerun crates.
re_memory Run-time memory tracking and profiling.
re_smart_channel A channel that keeps track of latency and queue length.
re_string_interner Yet another string interning library
re_tracing Helpers for tracing/spans/flamegraphs and such.
re_tuid 128-bit Time-based Unique Identifier
re_video Video decoding library

Dependencies and docs

In order to get a dependency graph for our in-house crates and their docs, we recommend you run:

cargo install cargo-depgraph
cargo depgraph --all-deps --workspace-only --all-features --dedup-transitive-deps | dot -Tpng > deps.png
open deps.png

and:

cargo doc --no-deps --open

and then browse through the re_* crates.