Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migration kernel for the blueprint space-view-related breaking changes #8439

Merged
merged 1 commit into from
Dec 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions crates/store/re_chunk/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ mod id;
mod iter;
mod latest_at;
mod merge;
mod migration;
mod range;
mod shuffle;
mod slice;
Expand Down
95 changes: 95 additions & 0 deletions crates/store/re_chunk/src/migration.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
use arrow2::array::{Array, Utf8Array};

use itertools::Itertools;
use nohash_hasher::IntMap;

use crate::Chunk;

impl Chunk {
/// A temporary migration kernel for blueprint data.
///
/// Deals with all the space-view terminology breaking changes (`SpaceView`->`View`, `space_view`->`view`, etc).
#[inline]
pub fn patched_for_blueprint_021_compat(&self) -> Self {
let mut chunk = self.clone();

const PATCHES: &[(&str, &str)] = &[("SpaceView", "View"), ("space_view", "view")];

// First, patch any entity path that still use space-view terminology.
// We expect this to only ever be called for blueprint data, so these entity paths
// are all builtin ones -- we're not overriding any user data.
let mut entity_path = chunk.entity_path.to_string();
for (from, to) in PATCHES {
entity_path = entity_path.replace(from, to);
}
chunk.entity_path = entity_path.into();

let mut components_patched = IntMap::default();
for (component_name, per_desc) in chunk.components.iter_mut() {
if PATCHES
.iter()
.any(|(from, _to)| component_name.contains(from))
{
// Second, patch all descriptors that still use space-view terminology.
for (mut descriptor, list_array) in std::mem::take(per_desc) {
for (from, to) in PATCHES {
if let Some(archetype_name) = descriptor.archetype_name.as_mut() {
*archetype_name = archetype_name.replace(from, to).into();
}
descriptor.component_name =
descriptor.component_name.replace(from, to).into();
}
components_patched.insert(descriptor, list_array.clone());
}
}

// Finally, patch actual data that still uses space-view terminology.
// As far as we know, this only concerns `IncludedContent` specifically.
if component_name == "rerun.blueprint.components.IncludedContent" {
for list_array in per_desc.values_mut() {
let arrays = list_array
.iter()
.map(|utf8_array| {
utf8_array.map(|array| {
let Some(array) = array.as_any().downcast_ref::<Utf8Array<i32>>()
else {
// Unreachable, just avoiding unwraps.
return array;
};

array
.iter()
.map(|s| {
s.map(|s| {
let mut s = s.to_owned();
for (from, to) in PATCHES {
s = s.replace(from, to);
}
s
})
})
.collect::<Utf8Array<i32>>()
.to_boxed()
})
})
.collect_vec();
let arrays = arrays
.iter()
.map(|a| a.as_deref() as Option<&dyn Array>)
.collect_vec();

if let Some(list_array_patched) = crate::util::arrays_to_list_array_opt(&arrays)
{
*list_array = list_array_patched;
}
}
}
}

for (desc, list_array) in components_patched {
chunk.components.insert_descriptor(desc, list_array);
}

chunk
}
}
4 changes: 3 additions & 1 deletion crates/store/re_chunk_store/src/writes.rs
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,9 @@ impl ChunkStore {
// * Somebody loads an old .rbl from somewhere and starts logging new blueprint data to it.
// * Etc.
if self.id.kind == re_log_types::StoreKind::Blueprint {
chunk = Arc::new(chunk.clone_as_untagged());
let patched = chunk.patched_for_blueprint_021_compat();
let patched = patched.clone_as_untagged();
chunk = Arc::new(patched);
}
Comment on lines 64 to 68
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wish we had a version tag on the store so we're not applying this indefinitely into the future ...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah we need to pull the StoreInfos all the way in at some point. We can do that later though, that's fine (I don't expect this particular change to be particularly version specific -- SpaceView is not making a come back anytime soon I hope 😛).


#[cfg(debug_assertions)]
Expand Down
Loading