Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better naming of gRPC-related crates and URIs #8016

Merged
merged 8 commits into from
Nov 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,11 +88,11 @@ Of course, this will only take us so far. In the future we plan on caching queri
Here is an overview of the crates included in the project:

<picture>
<img src="https://static.rerun.io/crates/a1291641639f4dc3864e84259c70089a96bfcb1a/full.png" alt="">
<source media="(max-width: 480px)" srcset="https://static.rerun.io/crates/a1291641639f4dc3864e84259c70089a96bfcb1a/480w.png">
<source media="(max-width: 768px)" srcset="https://static.rerun.io/crates/a1291641639f4dc3864e84259c70089a96bfcb1a/768w.png">
<source media="(max-width: 1024px)" srcset="https://static.rerun.io/crates/a1291641639f4dc3864e84259c70089a96bfcb1a/1024w.png">
<source media="(max-width: 1200px)" srcset="https://static.rerun.io/crates/a1291641639f4dc3864e84259c70089a96bfcb1a/1200w.png">
<img src="https://static.rerun.io/crates/35c2f18eb941ec321c6d19f5a763f0ac542ca2ed/full.png" alt="">
<source media="(max-width: 480px)" srcset="https://static.rerun.io/crates/35c2f18eb941ec321c6d19f5a763f0ac542ca2ed/480w.png">
<source media="(max-width: 768px)" srcset="https://static.rerun.io/crates/35c2f18eb941ec321c6d19f5a763f0ac542ca2ed/768w.png">
<source media="(max-width: 1024px)" srcset="https://static.rerun.io/crates/35c2f18eb941ec321c6d19f5a763f0ac542ca2ed/1024w.png">
<source media="(max-width: 1200px)" srcset="https://static.rerun.io/crates/35c2f18eb941ec321c6d19f5a763f0ac542ca2ed/1200w.png">
</picture>


Expand Down Expand Up @@ -161,13 +161,13 @@ Update instructions:

| Crate | Description |
|-----------------------|--------------------------------------------------------------------------|
| re_dataframe | The Rerun public data APIs. |
| re_entity_db | In-memory storage of Rerun entities |
| re_log_encoding | Helpers for encoding and transporting Rerun log messages |
| re_protos | Rerun remote store gRPC API types |
| re_query | Querying data in the re_chunk_store |
| re_dataframe | The Rerun public data APIs. |
| re_remote_store_types | Rerun remote store gRPC API types |
| re_types | The built-in Rerun data types, component types, and archetypes. |
| re_types_blueprint | The core traits and types that power Rerun's Blueprint sub-system. |
| re_log_encoding | Helpers for encoding and transporting Rerun log messages |


### Low-level store
Expand All @@ -187,10 +187,10 @@ Update instructions:
|----------------------|--------------------------------------------------------------------------------------------------------|
| re_data_loader | Handles loading of Rerun data from file using data loader plugins |
| re_data_source | Handles loading of Rerun data from different sources |
| re_grpc_client | Communicate with the Rerun Data Platform over gRPC |
| re_sdk_comms | TCP communication between Rerun SDK and Rerun Server |
| re_web_viewer_server | Serves the Rerun web viewer (Wasm and HTML) over HTTP |
| re_ws_comms | WebSocket communication library (encoding, decoding, client, server) between a Rerun server and Viewer |
| re_rrdp_comms | Communicate with the Rerun Data Platform over gRPC |


### Build support
Expand All @@ -199,9 +199,9 @@ Update instructions:
|-------------------------------|------------------------------------------------------------------|
| re_build_info | Information about the build. Use together with re_build_tools |
| re_build_tools | build.rs helpers for generating build info |
| re_remote_store_types_builder | Generates code for Rerun remote store gRPC API |
| re_types_builder | Generates code for Rerun's SDKs from flatbuffers definitions. |
| re_dev_tools | Various tools for Rerun development. Each tool has a subcommand. |
| re_protos_builder | Generates code for Rerun remote store gRPC API |
| re_types_builder | Generates code for Rerun's SDKs from flatbuffers definitions. |


### Utilities
Expand Down
86 changes: 43 additions & 43 deletions Cargo.lock
Original file line number Diff line number Diff line change
Expand Up @@ -5536,10 +5536,10 @@ dependencies = [
"rayon",
"re_build_tools",
"re_data_loader",
"re_grpc_client",
"re_log",
"re_log_encoding",
"re_log_types",
"re_rrdp_comms",
"re_smart_channel",
"re_tracing",
"re_ws_comms",
Expand Down Expand Up @@ -5678,6 +5678,24 @@ dependencies = [
"re_types_core",
]

[[package]]
name = "re_grpc_client"
version = "0.20.0-alpha.1+dev"
dependencies = [
"re_chunk",
"re_error",
"re_log",
"re_log_types",
"re_protos",
"re_smart_channel",
"thiserror",
"tokio",
"tokio-stream",
"tonic",
"tonic-web-wasm-client",
"wasm-bindgen-futures",
]

[[package]]
name = "re_int_histogram"
version = "0.20.0-alpha.1+dev"
Expand Down Expand Up @@ -5812,6 +5830,28 @@ dependencies = [
"thiserror",
]

[[package]]
name = "re_protos"
version = "0.20.0-alpha.1+dev"
dependencies = [
"prost",
"re_arrow2",
"re_dataframe",
"re_log_types",
"thiserror",
"tonic",
"tonic-web-wasm-client",
]

[[package]]
name = "re_protos_builder"
version = "0.20.0-alpha.1+dev"
dependencies = [
"camino",
"re_log",
"tonic-build",
]

[[package]]
name = "re_query"
version = "0.20.0-alpha.1+dev"
Expand Down Expand Up @@ -5866,28 +5906,6 @@ dependencies = [
"zerocopy",
]

[[package]]
name = "re_remote_store_types"
version = "0.20.0-alpha.1+dev"
dependencies = [
"prost",
"re_arrow2",
"re_dataframe",
"re_log_types",
"thiserror",
"tonic",
"tonic-web-wasm-client",
]

[[package]]
name = "re_remote_store_types_builder"
version = "0.20.0-alpha.1+dev"
dependencies = [
"camino",
"re_log",
"tonic-build",
]

[[package]]
name = "re_renderer"
version = "0.20.0-alpha.1+dev"
Expand Down Expand Up @@ -5965,24 +5983,6 @@ dependencies = [
"zip",
]

[[package]]
name = "re_rrdp_comms"
version = "0.20.0-alpha.1+dev"
dependencies = [
"re_chunk",
"re_error",
"re_log",
"re_log_types",
"re_remote_store_types",
"re_smart_channel",
"thiserror",
"tokio",
"tokio-stream",
"tonic",
"tonic-web-wasm-client",
"wasm-bindgen-futures",
]

[[package]]
name = "re_sdk"
version = "0.20.0-alpha.1+dev"
Expand Down Expand Up @@ -6526,13 +6526,13 @@ dependencies = [
"re_entity_db",
"re_error",
"re_format",
"re_grpc_client",
"re_log",
"re_log_encoding",
"re_log_types",
"re_memory",
"re_query",
"re_renderer",
"re_rrdp_comms",
"re_sdk_comms",
"re_selection_panel",
"re_smart_channel",
Expand Down Expand Up @@ -6989,7 +6989,7 @@ dependencies = [
"re_log",
"re_log_types",
"re_memory",
"re_remote_store_types",
"re_protos",
"re_sdk",
"re_video",
"re_web_viewer_server",
Expand Down
6 changes: 3 additions & 3 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ version = "0.20.0-alpha.1+dev"
re_build_info = { path = "crates/build/re_build_info", version = "=0.20.0-alpha.1", default-features = false }
re_build_tools = { path = "crates/build/re_build_tools", version = "=0.20.0-alpha.1", default-features = false }
re_dev_tools = { path = "crates/build/re_dev_tools", version = "=0.20.0-alpha.1", default-features = false }
re_remote_store_types_builder = { path = "crates/build/re_remote_store_types_builder", version = "=0.20.0-alpha.1", default-features = false }
re_protos_builder = { path = "crates/build/re_protos_builder", version = "=0.20.0-alpha.1", default-features = false }
re_types_builder = { path = "crates/build/re_types_builder", version = "=0.20.0-alpha.1", default-features = false }

# crates/store:
Expand All @@ -48,11 +48,11 @@ re_data_source = { path = "crates/store/re_data_source", version = "=0.20.0-alph
re_dataframe = { path = "crates/store/re_dataframe", version = "=0.20.0-alpha.1", default-features = false }
re_entity_db = { path = "crates/store/re_entity_db", version = "=0.20.0-alpha.1", default-features = false }
re_format_arrow = { path = "crates/store/re_format_arrow", version = "=0.20.0-alpha.1", default-features = false }
re_grpc_client = { path = "crates/store/re_grpc_client", version = "=0.20.0-alpha.1", default-features = false }
re_protos = { path = "crates/store/re_protos", version = "=0.20.0-alpha.1", default-features = false }
re_log_encoding = { path = "crates/store/re_log_encoding", version = "=0.20.0-alpha.1", default-features = false }
re_log_types = { path = "crates/store/re_log_types", version = "=0.20.0-alpha.1", default-features = false }
re_query = { path = "crates/store/re_query", version = "=0.20.0-alpha.1", default-features = false }
re_rrdp_comms = { path = "crates/store/re_rrdp_comms", version = "=0.20.0-alpha.1", default-features = false }
re_remote_store_types = { path = "crates/store/re_remote_store_types", version = "=0.20.0-alpha.1", default-features = false }
re_sdk_comms = { path = "crates/store/re_sdk_comms", version = "=0.20.0-alpha.1", default-features = false }
re_types = { path = "crates/store/re_types", version = "=0.20.0-alpha.1", default-features = false }
re_types_blueprint = { path = "crates/store/re_types_blueprint", version = "=0.20.0-alpha.1", default-features = false }
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[package]
name = "re_remote_store_types_builder"
name = "re_protos_builder"
authors.workspace = true
edition.workspace = true
homepage.workspace = true
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# re_remote_store_types_builder
# re_protos_builder

Part of the [`rerun`](https://github.com/rerun-io/rerun) family of crates.

[![Latest version](https://img.shields.io/crates/v/re_remote_store_types_builder.svg)](https://crates.io/crates/re_remote_store_types_builder?speculative-link)
[![Documentation](https://docs.rs/re_remote_store_types_builder/badge.svg)](https://docs.rs/re_remote_store_types_builder?speculative-link)
[![Latest version](https://img.shields.io/crates/v/re_protos_builder.svg)](https://crates.io/crates/re_protos_builder?speculative-link)
[![Documentation](https://docs.rs/re_protos_builder/badge.svg)](https://docs.rs/re_protos_builder?speculative-link)
![MIT](https://img.shields.io/badge/license-MIT-blue.svg)
![Apache](https://img.shields.io/badge/license-Apache-blue.svg)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@

use camino::Utf8Path;

const PROTOBUF_DEFINITIONS_DIR_PATH: &str = "crates/store/re_remote_store_types/proto";
const PROTOBUF_DEFINITIONS_DIR_PATH: &str = "crates/store/re_protos/proto";
const PROTOBUF_REMOTE_STORE_V0_RELATIVE_PATH: &str = "rerun/v0/remote_store.proto";
const RUST_V0_OUTPUT_DIR_PATH: &str = "crates/store/re_remote_store_types/src/v0";
const RUST_V0_OUTPUT_DIR_PATH: &str = "crates/store/re_protos/src/v0";

fn main() {
re_log::setup_logging();
Expand All @@ -35,7 +35,7 @@ fn main() {
"Running codegen for storage node types",
);

re_remote_store_types_builder::generate_rust_code(
re_protos_builder::generate_rust_code(
definitions_dir_path,
&[PROTOBUF_REMOTE_STORE_V0_RELATIVE_PATH],
rust_generated_output_dir_path,
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
//! This crate contains logic for generating remote store gRPC API types as defined in
//! `re_remote_store_types` proto files. We are currently generating both client and server
//! `re_protos` proto files. We are currently generating both client and server
//! definitions in the same file.
//!

Expand Down
8 changes: 4 additions & 4 deletions crates/store/re_data_source/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,10 @@ all-features = true


[features]
default = ["rrdp"]
default = ["grpc"]

## Enable the rrdp data source.
rrdp = ["dep:re_rrdp_comms"]
## Enable the gRPC Rerun Data Platform data source.
grpc = ["dep:re_grpc_client"]


[dependencies]
Expand All @@ -43,7 +43,7 @@ itertools.workspace = true
rayon.workspace = true

# Optional dependencies:
re_rrdp_comms = { workspace = true, optional = true }
re_grpc_client = { workspace = true, optional = true }

[build-dependencies]
re_build_tools.workspace = true
22 changes: 11 additions & 11 deletions crates/store/re_data_source/src/data_source.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,9 @@ pub enum DataSource {
#[cfg(not(target_arch = "wasm32"))]
Stdin,

/// A remote file, served over RRDP interface.
#[cfg(feature = "rrdp")]
RrdpUrl { url: String },
/// A file on a Rerun Data Platform server, over `rerun://` gRPC interface.
#[cfg(feature = "grpc")]
RerunGrpcUrl { url: String },
}

impl DataSource {
Expand Down Expand Up @@ -90,9 +90,9 @@ impl DataSource {

let path = std::path::Path::new(&uri).to_path_buf();

#[cfg(feature = "rrdp")]
if uri.starts_with("rrdp://") {
return Self::RrdpUrl { url: uri };
#[cfg(feature = "grpc")]
if uri.starts_with("rerun://") {
return Self::RerunGrpcUrl { url: uri };
}

if uri.starts_with("file://") || path.exists() {
Expand Down Expand Up @@ -136,8 +136,8 @@ impl DataSource {
Self::WebSocketAddr(_) => None,
#[cfg(not(target_arch = "wasm32"))]
Self::Stdin => None,
#[cfg(feature = "rrdp")]
Self::RrdpUrl { .. } => None, // TODO(jleibs): This needs to come from the RRDP server.
#[cfg(feature = "grpc")]
Self::RerunGrpcUrl { .. } => None, // TODO(jleibs): This needs to come from the server.
}
}

Expand Down Expand Up @@ -245,9 +245,9 @@ impl DataSource {
Ok(rx)
}

#[cfg(feature = "rrdp")]
Self::RrdpUrl { url } => {
re_rrdp_comms::stream_recording(url, on_msg).map_err(|err| err.into())
#[cfg(feature = "grpc")]
Self::RerunGrpcUrl { url } => {
re_grpc_client::stream_recording(url, on_msg).map_err(|err| err.into())
}
}
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[package]
name = "re_rrdp_comms"
name = "re_grpc_client"
authors.workspace = true
description = "Client comms for RRDP gRPC server"
description = "gRCP client for the Rerun Data Platform gRPC prototocl"
edition.workspace = true
homepage.workspace = true
include.workspace = true
Expand All @@ -24,7 +24,7 @@ re_chunk.workspace = true
re_error.workspace = true
re_log_types.workspace = true
re_log.workspace = true
re_remote_store_types.workspace = true
re_protos.workspace = true
re_smart_channel.workspace = true

thiserror.workspace = true
Expand Down
10 changes: 10 additions & 0 deletions crates/store/re_grpc_client/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# re_grpc_client

Part of the [`rerun`](https://github.com/rerun-io/rerun) family of crates.

[![Latest version](https://img.shields.io/crates/v/re_grpc_client.svg)](https://crates.io/crates/re_grpc_client?speculative-link)
[![Documentation](https://docs.rs/re_grpc_client/badge.svg)](https://docs.rs/re_grpc_client?speculative-link)
![MIT](https://img.shields.io/badge/license-MIT-blue.svg)
![Apache](https://img.shields.io/badge/license-Apache-blue.svg)

Client-side gRPC communication library with a Rerun Data Platform server.
Loading
Loading