Tensor shape and dimension names as separate arrow fields #8376
Conversation
emilk
added
codegen/idl
exclude from changelog
|
Latest documentation preview deployed successfully.
Note: This comment is updated whenever you push a commit. |
|
Web viewer built successfully. If applicable, you should also test it:
Note: This comment is updated whenever you push a commit. |
emilk
force-pushed
the
emilk/refactor-tensor
branch
from
7d68a74 to
416ba00
Compare
|
@rerun-bot full-check |
|
Started a full build: https://github.com/rerun-io/rerun/actions/runs/12258005654 |
|
@rerun-bot full-check |
|
Started a full build: https://github.com/rerun-io/rerun/actions/runs/12258330176 |
|
@rerun-bot full-check |
|
Started a full build: https://github.com/rerun-io/rerun/actions/runs/12271591874 |
There was a problem hiding this comment.
Nice 👍
My only gripe is that there doesn't seem to exist any centralized place that communicates clearly what happens when the number of names doesn't match the number of dimensions.
Actually, I'm not even sure the behavior itself is consistent all over the place, since it also feels pretty decentralized (which might very well be unavoidable).
But maybe I'm wrong and I'm missing the logic, it's a large diff and I suck at reviews.
| pub fn from_tensor_data(tensor_data: &re_types::datatypes::TensorData) -> Vec<Self> { | ||
| tensor_data | ||
| .shape | ||
| .iter() | ||
| .enumerate() | ||
| .map(|(dim_idx, dim_len)| Self { | ||
| size: *dim_len, | ||
| name: tensor_data.dim_name(dim_idx).cloned(), | ||
| }) | ||
| .collect() | ||
| } |
There was a problem hiding this comment.
So this one clamps from the dimensions's pov 🤔
|
DNM: Let's wait for the 0.21 release before we merge this |
Related
TensorDimensioninto shape and names #6830What
In
TensorDatawe used to haveshape: [TensorDimension]withstruct TensorDimension { size: u64, name: Option<String> }.Now
TensorDatahasshape: [u64], names: Option<[String]>instead.So basically a AoS -> SoA change
TODO
@rerun-bot full-check