-
Notifications
You must be signed in to change notification settings - Fork 370
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Video frame references #7368
Comments
6 tasks
Wumpf
added a commit
that referenced
this issue
Sep 11, 2024
…n on it (#7396) ### What * Large chunk of #7368 --- * Introduce `VideoFrameReference` archetype * comes with a bunch of subtypes * Make the video visualizer use this * in the process I did some general iteration on how this visualizer ticks * note that I departed from the usual quite encrusted patterns of how to implement visualizers that naturally found its way into here as well. Imho the patterns don't hold up all that well in general (we had to grind over them many times in quick succession) and less so in particular for "special" archetypes like video or mesh (etc.) * video data loader emits `VideoFrameReference` instead of the adhoc `VideoTick Still missing for completion of the video frame reference task: * examples to use this from SDKs (by directly logging video frames) * this surely will show that we need more extensions * pretty/edit ui for video timestamps * @Wumpf are you saying that you can then scrub videos from a blueprint slider?! Yes. That's exactly what will end up happening ;D * pretty/edit ui for entity paths * exposed utilities to SDKs for generation of video frame references @ reviewers: Special attention please on the fbs definition, thank you! <img width="1813" alt="image" src="https://github.com/user-attachments/assets/d221ff1e-7ec0-4067-8916-7a8dd2ed8002"> ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [x] I have tested the web demo (if applicable): * Using examples from latest `main` build: [rerun.io/viewer](https://rerun.io/viewer/pr/7396?manifest_url=https://app.rerun.io/version/main/examples_manifest.json) * Using full set of examples from `nightly` build: [rerun.io/viewer](https://rerun.io/viewer/pr/7396?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json) * [x] The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG * [x] If applicable, add a new check to the [release checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)! * [x] If have noted any breaking changes to the log API in `CHANGELOG.md` and the migration guide - [PR Build Summary](https://build.rerun.io/pr/7396) - [Recent benchmark results](https://build.rerun.io/graphs/crates.html) - [Wasm size tracking](https://build.rerun.io/graphs/sizes.html) To run all checks from `main`, comment on the PR with `@rerun-bot full-check`.
jprochazk
pushed a commit
that referenced
this issue
Sep 11, 2024
…n on it (#7396) ### What * Large chunk of #7368 --- * Introduce `VideoFrameReference` archetype * comes with a bunch of subtypes * Make the video visualizer use this * in the process I did some general iteration on how this visualizer ticks * note that I departed from the usual quite encrusted patterns of how to implement visualizers that naturally found its way into here as well. Imho the patterns don't hold up all that well in general (we had to grind over them many times in quick succession) and less so in particular for "special" archetypes like video or mesh (etc.) * video data loader emits `VideoFrameReference` instead of the adhoc `VideoTick Still missing for completion of the video frame reference task: * examples to use this from SDKs (by directly logging video frames) * this surely will show that we need more extensions * pretty/edit ui for video timestamps * @Wumpf are you saying that you can then scrub videos from a blueprint slider?! Yes. That's exactly what will end up happening ;D * pretty/edit ui for entity paths * exposed utilities to SDKs for generation of video frame references @ reviewers: Special attention please on the fbs definition, thank you! <img width="1813" alt="image" src="https://github.com/user-attachments/assets/d221ff1e-7ec0-4067-8916-7a8dd2ed8002"> ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [x] I have tested the web demo (if applicable): * Using examples from latest `main` build: [rerun.io/viewer](https://rerun.io/viewer/pr/7396?manifest_url=https://app.rerun.io/version/main/examples_manifest.json) * Using full set of examples from `nightly` build: [rerun.io/viewer](https://rerun.io/viewer/pr/7396?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json) * [x] The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG * [x] If applicable, add a new check to the [release checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)! * [x] If have noted any breaking changes to the log API in `CHANGELOG.md` and the migration guide - [PR Build Summary](https://build.rerun.io/pr/7396) - [Recent benchmark results](https://build.rerun.io/graphs/crates.html) - [Wasm size tracking](https://build.rerun.io/graphs/sizes.html) To run all checks from `main`, comment on the PR with `@rerun-bot full-check`.
7 tasks
Wumpf
added a commit
that referenced
this issue
Sep 13, 2024
### What * Part of #7368 * Based on: * #7401 * #7402 A bit of a first: I'm referencing the same examples on both the `AssetVideo` and the `VideoFrameReference` archetype, seems to make sense :) ![screenshot](https://static.rerun.io/video_manual_frames/320a44e1e06b8b3a3161ecbbeae3e04d1ccb9589/1200w.png) ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [x] I have tested the web demo (if applicable): * Using examples from latest `main` build: [rerun.io/viewer](https://rerun.io/viewer/pr/7403?manifest_url=https://app.rerun.io/version/main/examples_manifest.json) * Using full set of examples from `nightly` build: [rerun.io/viewer](https://rerun.io/viewer/pr/7403?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json) * [x] The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG * [x] If applicable, add a new check to the [release checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)! * [x] If have noted any breaking changes to the log API in `CHANGELOG.md` and the migration guide * [x] green main build - [PR Build Summary](https://build.rerun.io/pr/7403) - [Recent benchmark results](https://build.rerun.io/graphs/crates.html) - [Wasm size tracking](https://build.rerun.io/graphs/sizes.html) To run all checks from `main`, comment on the PR with `@rerun-bot full-check`.
This was referenced Sep 16, 2024
Wumpf
added a commit
that referenced
this issue
Sep 17, 2024
### What * Fixes #7368 * creating new issues for the previously remaining items * Fixes #6532 Had to patch this through to all 3 sdks which naturally takes quite a bit of extra machinery. Decided to go with raw ns here since it's easiest to pass through FFI and is the most versatile (hopefully our `VideoFrameReference` typing will be solved with tags instead of per component enums in the future, making this more equivalent). Changed snippets around. There's now: * a sample with two single frozen frames, showing next to each other. Demonstrating that video frames can be used individually and without any extra utilities * this doesn't quite work yet due to #7420, so this can be regarded a bit of a work in progress * sample using the new frame extraction utility, essentially a "send full video" which we may encapsulate into a higher level utility at some point. It's quite short though in Python, so I'm not too worried about this! Asset drag & drop got updated to also use this utility. All this churn also led me to changing how `re_video` is used a bit, making the entry point more highlevel. Media type got a bit annoying there because we can't afford `re_types` dependencies in the C++ & Rust SDKs. The solution here is to have independent media type parsing for videos in `re_video`. ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [x] I have tested the web demo (if applicable): * Using examples from latest `main` build: [rerun.io/viewer](https://rerun.io/viewer/pr/7421?manifest_url=https://app.rerun.io/version/main/examples_manifest.json) * Using full set of examples from `nightly` build: [rerun.io/viewer](https://rerun.io/viewer/pr/7421?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json) * [x] The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG * [x] If applicable, add a new check to the [release checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)! * [x] If have noted any breaking changes to the log API in `CHANGELOG.md` and the migration guide * [x] pass `main` ci - [PR Build Summary](https://build.rerun.io/pr/7421) - [Recent benchmark results](https://build.rerun.io/graphs/crates.html) - [Wasm size tracking](https://build.rerun.io/graphs/sizes.html) To run all checks from `main`, comment on the PR with `@rerun-bot full-check`.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Implement design from:
Data model
SDK
Our logging SDK should had a helper for generating all
VideoFrame
for a video.Viewer
Our
VideoVisualizer
queries forVideoFrame
.We follow a
VideoFrame
via another latest-at query on the given entity path.A
VideoAsset
withoutVideoFrame
will not show up in any space view.Selecting a video should ideally show that video in the selection view (with independent playback).
It should also be able to show the actual frame timestamps in the video.
The text was updated successfully, but these errors were encountered: