Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add experimental Dataframe Space View #4468

Merged
merged 14 commits into from
Dec 13, 2023
11 changes: 6 additions & 5 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,11 +88,11 @@ Of course, this will only take us so far. In the future we plan on caching queri
Here is an overview of the crates included in the project:

<picture>
<img src="https://static.rerun.io/crates/eaea8b78fd7efbefd76c0d6a09086ef9cd742c8b/full.png" alt="">
<source media="(max-width: 480px)" srcset="https://static.rerun.io/crates/eaea8b78fd7efbefd76c0d6a09086ef9cd742c8b/480w.png">
<source media="(max-width: 768px)" srcset="https://static.rerun.io/crates/eaea8b78fd7efbefd76c0d6a09086ef9cd742c8b/768w.png">
<source media="(max-width: 1024px)" srcset="https://static.rerun.io/crates/eaea8b78fd7efbefd76c0d6a09086ef9cd742c8b/1024w.png">
<source media="(max-width: 1200px)" srcset="https://static.rerun.io/crates/eaea8b78fd7efbefd76c0d6a09086ef9cd742c8b/1200w.png">
<img src="https://static.rerun.io/crates/21c577a05570720e96b850e8da21b5aa1fcd6c93/full.png" alt="">
<source media="(max-width: 480px)" srcset="https://static.rerun.io/crates/21c577a05570720e96b850e8da21b5aa1fcd6c93/480w.png">
<source media="(max-width: 768px)" srcset="https://static.rerun.io/crates/21c577a05570720e96b850e8da21b5aa1fcd6c93/768w.png">
<source media="(max-width: 1024px)" srcset="https://static.rerun.io/crates/21c577a05570720e96b850e8da21b5aa1fcd6c93/1024w.png">
<source media="(max-width: 1200px)" srcset="https://static.rerun.io/crates/21c577a05570720e96b850e8da21b5aa1fcd6c93/1200w.png">
</picture>

<!-- !!! IMPORTANT!!!
Expand Down Expand Up @@ -134,6 +134,7 @@ Update instructions:
| re_renderer | A wgpu-based renderer for all your visualization needs. |
| re_space_view | Types & utilities for defining Space View classes and communicating with the Viewport. |
| re_space_view_bar_chart | A Space View that shows a single bar chart. |
| re_space_view_dataframe | A Space View that shows the data contained in entities in a table. |
| re_space_view_spatial | Space Views that show entities in a 2D or 3D spatial relationship. |
| re_space_view_tensor | A Space View dedicated to visualizing tensors with arbitrary dimensionality. |
| re_space_view_text_document | A simple Space View that shows a single text box. |
Expand Down
21 changes: 21 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ re_sdk_comms = { path = "crates/re_sdk_comms", version = "=0.12.0-alpha.1", defa
re_smart_channel = { path = "crates/re_smart_channel", version = "=0.12.0-alpha.1", default-features = false }
re_space_view = { path = "crates/re_space_view", version = "=0.12.0-alpha.1", default-features = false }
re_space_view_bar_chart = { path = "crates/re_space_view_bar_chart", version = "=0.12.0-alpha.1", default-features = false }
re_space_view_dataframe = { path = "crates/re_space_view_dataframe", version = "=0.12.0-alpha.1", default-features = false }
re_space_view_spatial = { path = "crates/re_space_view_spatial", version = "=0.12.0-alpha.1", default-features = false }
re_space_view_tensor = { path = "crates/re_space_view_tensor", version = "=0.12.0-alpha.1", default-features = false }
re_space_view_text_log = { path = "crates/re_space_view_text_log", version = "=0.12.0-alpha.1", default-features = false }
Expand Down
33 changes: 33 additions & 0 deletions crates/re_space_view_dataframe/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
[package]
authors.workspace = true
description = "A Space View that shows the data contained in entities in a table."
edition.workspace = true
homepage.workspace = true
license.workspace = true
name = "re_space_view_dataframe"
publish = true
readme = "README.md"
repository.workspace = true
rust-version.workspace = true
version.workspace = true
include = ["../../LICENSE-APACHE", "../../LICENSE-MIT", "**/*.rs", "Cargo.toml"]

[package.metadata.docs.rs]
all-features = true

[dependencies]
re_arrow_store.workspace = true
re_data_store.workspace = true
re_data_ui.workspace = true
re_log.workspace = true
re_log_types.workspace = true
re_query.workspace = true
re_renderer.workspace = true
re_tracing.workspace = true
re_types.workspace = true
re_ui.workspace = true
re_viewer_context.workspace = true

egui_extras.workspace = true
egui.workspace = true
itertools.workspace = true
10 changes: 10 additions & 0 deletions crates/re_space_view_dataframe/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# re_space_view_dataframe

Part of the [`rerun`](https://github.com/rerun-io/rerun) family of crates.

[![Latest version](https://img.shields.io/crates/v/re_space_view_dataframe.svg)](https://crates.io/crates/re_space_view_dataframe?speculative-link)
[![Documentation](https://docs.rs/re_space_view_dataframe/badge.svg)](https://docs.rs/re_space_view_dataframe?speculative-link)
![MIT](https://img.shields.io/badge/license-MIT-blue.svg)
![Apache](https://img.shields.io/badge/license-Apache-blue.svg)

A Space View that shows the data contained in entities in a table.
8 changes: 8 additions & 0 deletions crates/re_space_view_dataframe/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
//! Rerun `Data` Space View
//!
//! A Space View that shows the data contained in entities in a table.

mod space_view_class;
mod view_part_system;

pub use space_view_class::DataframeSpaceView;
190 changes: 190 additions & 0 deletions crates/re_space_view_dataframe/src/space_view_class.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,190 @@
use crate::view_part_system::EmptySystem;
use egui_extras::Column;
use itertools::Itertools;
use re_data_store::{EntityProperties, InstancePath};
use re_log_types::EntityPath;
use re_query::get_component_with_instances;
use re_viewer_context::{
AutoSpawnHeuristic, PerSystemEntities, SpaceViewClass, SpaceViewClassRegistryError,
SpaceViewId, SpaceViewSystemExecutionError, SystemExecutionOutput, UiVerbosity, ViewQuery,
ViewerContext,
};

#[derive(Default)]
pub struct DataframeSpaceView;

impl SpaceViewClass for DataframeSpaceView {
type State = ();

const IDENTIFIER: &'static str = "Dataframe";
const DISPLAY_NAME: &'static str = "Dataframe";

fn icon(&self) -> &'static re_ui::Icon {
//TODO(ab): fix that icon
&re_ui::icons::SPACE_VIEW_TEXTBOX
}

fn help_text(&self, _re_ui: &re_ui::ReUi) -> egui::WidgetText {
"Show the data contained in entities in a table.".into()
}

fn on_register(
&self,
system_registry: &mut re_viewer_context::SpaceViewSystemRegistry,
) -> Result<(), SpaceViewClassRegistryError> {
system_registry.register_part_system::<EmptySystem>()
}

fn preferred_tile_aspect_ratio(&self, _state: &Self::State) -> Option<f32> {
None
}

fn layout_priority(&self) -> re_viewer_context::SpaceViewClassLayoutPriority {
re_viewer_context::SpaceViewClassLayoutPriority::Low
}

fn auto_spawn_heuristic(
&self,
_ctx: &ViewerContext<'_>,
_space_origin: &EntityPath,
_ent_paths: &PerSystemEntities,
) -> re_viewer_context::AutoSpawnHeuristic {
AutoSpawnHeuristic::NeverSpawn
}

fn selection_ui(
&self,
_ctx: &ViewerContext<'_>,
_ui: &mut egui::Ui,
_state: &mut Self::State,
_space_origin: &EntityPath,
_space_view_id: SpaceViewId,
_root_entity_properties: &mut EntityProperties,
) {
}

fn ui(
&self,
ctx: &ViewerContext<'_>,
ui: &mut egui::Ui,
_state: &mut Self::State,
_root_entity_properties: &EntityProperties,
query: &ViewQuery<'_>,
_system_output: SystemExecutionOutput,
) -> Result<(), SpaceViewSystemExecutionError> {
re_tracing::profile_function!();

let entities: Vec<_> = query
abey79 marked this conversation as resolved.
Show resolved Hide resolved
.iter_all_data_results()
.filter(|data_result| data_result.resolved_properties.visible)
.map(|data_result| &data_result.entity_path)
.unique()
.cloned()
.collect();
emilk marked this conversation as resolved.
Show resolved Hide resolved

let store = ctx.store_db.store();
let latest_at_query = query.latest_at_query();

// for each entity, this does the union of all instance keys of all components
let all_instances: Vec<_> = entities
.iter()
.flat_map(|entity| {
store
.all_components(&query.timeline, entity)
.unwrap_or_default()
.into_iter()
.filter(|comp| !comp.is_indicator_component())
.flat_map(|comp| {
get_component_with_instances(store, &latest_at_query, entity, comp)
.map(|(_, comp_inst)| comp_inst.instance_keys())
.unwrap_or_default()
})
.filter(|instance_key| !instance_key.is_splat())
.map(|instance_key| InstancePath::instance(entity.clone(), instance_key))
})
.unique()
.collect();
abey79 marked this conversation as resolved.
Show resolved Hide resolved

let all_components: Vec<_> = entities
.iter()
.flat_map(|entity| {
store
.all_components(&query.timeline, entity)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You've already made that same query just above!

.unwrap_or_default()
})
.unique()
.filter(|comp| !comp.is_indicator_component())
.collect();
abey79 marked this conversation as resolved.
Show resolved Hide resolved

egui::ScrollArea::both()
.auto_shrink([false, false])
.show(ui, |ui| {
egui::Frame {
inner_margin: egui::Margin::same(5.0),
..Default::default()
}
.show(ui, |ui| {
egui_extras::TableBuilder::new(ui)
.columns(
Column::auto_with_initial_suggestion(200.0),
all_components.len() + 1,
)
.resizable(true)
.vscroll(false)
.auto_shrink([false, true])
.striped(true)
abey79 marked this conversation as resolved.
Show resolved Hide resolved
.header(re_ui::ReUi::table_line_height(), |mut row| {
row.col(|ui| {
ui.strong("Entity");
});

for comp in &all_components {
row.col(|ui| {
ui.strong(comp.short_name());
});
}
})
.body(|body| {
body.rows(
abey79 marked this conversation as resolved.
Show resolved Hide resolved
re_ui::ReUi::table_line_height(),
all_instances.len(),
|idx, mut row| {
let instance = &all_instances[idx];

row.col(|ui| {
ui.label(format!("{instance}"));
});
abey79 marked this conversation as resolved.
Show resolved Hide resolved

for comp in &all_components {
row.col(|ui| {
if let Some((_, comp_inst)) =
get_component_with_instances(
store,
&latest_at_query,
&instance.entity_path,
*comp,
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Am I reading this right that this is redoing the full query for each and every instance? Also we've already queried the data at the start anyway (for all_instances).

All these get_component_instances queries should be done once at the start and put in a map, then everything can be done cheaply.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As per our discussion yesterday, this is acceptable because the inner table closure is called only for the displayed rows. I added a comment to that effect.

{
ctx.component_ui_registry.ui(
ctx,
ui,
UiVerbosity::Small,
&latest_at_query,
&instance.entity_path,
&comp_inst,
&instance.instance_key,
);
} else {
ui.weak("-");
}
});
}
},
);
});
});
});

Ok(())
}
}
38 changes: 38 additions & 0 deletions crates/re_space_view_dataframe/src/view_part_system.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
use re_types::ComponentNameSet;
use re_viewer_context::{
IdentifiedViewSystem, SpaceViewSystemExecutionError, ViewContextCollection, ViewPartSystem,
ViewQuery, ViewerContext,
};

/// An empty system to accept all entities in the space view
#[derive(Default)]
pub struct EmptySystem {}

impl IdentifiedViewSystem for EmptySystem {
fn identifier() -> re_viewer_context::ViewSystemIdentifier {
"Empty".into()
}
}

impl ViewPartSystem for EmptySystem {
fn required_components(&self) -> ComponentNameSet {
std::iter::empty().collect()
}

fn indicator_components(&self) -> ComponentNameSet {
std::iter::empty().collect()
}

fn execute(
&mut self,
_ctx: &ViewerContext<'_>,
_query: &ViewQuery<'_>,
_view_ctx: &ViewContextCollection,
) -> Result<Vec<re_renderer::QueueableDrawData>, SpaceViewSystemExecutionError> {
Ok(vec![])
}

fn as_any(&self) -> &dyn std::any::Any {
self
}
}
1 change: 1 addition & 0 deletions crates/re_viewer/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ re_renderer = { workspace = true, default-features = false }
re_smart_channel.workspace = true
re_space_view.workspace = true
re_space_view_bar_chart.workspace = true
re_space_view_dataframe.workspace = true
re_space_view_spatial.workspace = true
re_space_view_tensor.workspace = true
re_space_view_text_document = { workspace = true, features = ["markdown"] }
Expand Down
Loading
Loading