Support for logging & plotting multi-dimensional "Signal" types

[jleibs]

Context

Many datasets use state vectors to capture robot state. For example lerobot uses a 7-dimensional vector of motor positions as their state.

Needing to break this up into a collection of separate columns/entities is somewhat awkward compared to just being able to log and plot using a ListArray instead.

New Archetype Proposal

Since Scalar vs Scalars as archetypes has the potential to be somewhat confusing, it seems worth introducing a new archetype here.

Current proposal is to name this archetype Signals which generally aligns with the use-cases we would expect to see.

Although we don't provide name in the Scalar archetype, forcing users to use SeriesLine / SeriesPoint to name things, the need here is more evident as all these signals belong to the same entity path.

That said doing "name-based" joining of sparse signals adds a lot of complexity, and so we should still document this as being informational-only and simply produce a warning if users accidentally use mixed-name signals.

archetypes/signals.fbs

/// A collection of synchronous scalar signal values.
///
/// For example, a trace of the individual axes in the output of an IMU.
///
/// This archetype does not allow for null-valued or otherwise non-uniform-length signals.
table Signals (
  "attr.rust.derive": "PartialEq",
  "attr.docs.category": "Plotting",
  "attr.docs.view_types": "TimeSeriesView"
) {
  // --- Required ---

  /// Multiple channels worth of signal data.
  /// 
  /// The logical identity of a signal is based on its index in this array and is not impacted by the
  /// value of labels component.
  ///
  /// The length of this array should be constant for an entire dataset. If multiple signal
  /// records have different lengths, any unset signal values shall be assumed to be 0.
  /// 
  channels: [rerun.components.Scalar] ("attr.rerun.component_required", order: 1000);

  /// --- Optional ---

  /// Optional names for each of the signals in the signal array
  ///
  /// This component is exposed for convenience of data annotation at the log-site. Specifying
  /// inconsistent names over time is undefined behavior. However, the Rerun viewer will detect
  /// that this has happened and produce a warning.
  /// 
  /// In situations where the performance and data overhead is relevant, you should only log this
  /// component once via blueprint or static logging APIs.
  names: [rerun.components.Name] ("attr.rerun.component_optional", nullable, order: 2000);
}

Visualizer Consequences

The SeriesLine and SeriesPoint visualizers should both be compatible with the new archetype.

This will require updating both archetypes to support batch, rather than mono- components for attributes such as color and width.

Known Limitations

Since visualizers act on entities (and not sub-entity selections), you will have to choose whether to plot all the signals as lines, or all the signals as points. There is not currently an effective way to mix and match.

Future work

It would be nice to somehow be able to treat these signal channels as "virtual" entities in a nested entity-path-like representation and just configure Lines/Points using a path-selection syntax. This concept needs a lot more work though and doesn't need to be a blocker for making this MVP work.

[emilk]

I'm not sold on the name. "Signals" is often used to mean "callbacks/delegates", especially in the context of UIs.

[jleibs]

I'm not sold on the name. "Signals" is often used to mean "callbacks/delegates", especially in the context of UIs.

My original thought is the signals/slots terminology was pretty removed from our particular use-case, but I went and took a look at Vega which also happens to use signals in the way you're describing (https://vega.github.io/vega/docs/signals/) as a specification related to reactive UI input events.

@emilk any alternative suggestions?

[jleibs]

Other names that have come up:

• Vector (concerns that it's too overloaded with spatial, e.g. Vec3)
• Tensor1D (not super obvious to users that haven't heard the tensor terminology before)
• Array (probably too generic)
• ScalarArray 🤷

[oceanusxiv]

+1 on this feature, having a for loop to log state vectors on a timeline is really not ideal. Ideally the viewer can internally do the unpacking. My 2c is ScalarArray is not bad as a name. if the idea is that this is just a bundle of the original Scalar archetype.