From 953de54d693dc5c96a8165213dab790f5a366883 Mon Sep 17 00:00:00 2001 From: Emil Ernerfeldt Date: Wed, 16 Oct 2024 18:36:28 +0200 Subject: [PATCH 01/10] Improve changelog-script --- scripts/generate_changelog.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/generate_changelog.py b/scripts/generate_changelog.py index 5ae09bcbb66b..166a27719b2d 100755 --- a/scripts/generate_changelog.py +++ b/scripts/generate_changelog.py @@ -187,7 +187,7 @@ def main() -> None: labels = pr_info.labels if pr_info else [] - if "exclude from changelog" in labels: + if "include in changelog" not in labels: continue summary = f"{title} [#{pr_number}](https://github.com/{OWNER}/{REPO}/pull/{pr_number})" @@ -212,7 +212,7 @@ def main() -> None: added = False # Some PRs can show up under multiple sections: - if "πŸͺ΅ Log-API" in labels: + if "πŸͺ΅ Log & send APIs" in labels: log_api.append(summary) added = True else: From e964918e6e9159b4e12eb55c2a6a11de548439c0 Mon Sep 17 00:00:00 2001 From: Emil Ernerfeldt Date: Wed, 16 Oct 2024 18:59:39 +0200 Subject: [PATCH 02/10] Remove done TODO --- docs/content/howto/dataframe-api.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/content/howto/dataframe-api.md b/docs/content/howto/dataframe-api.md index 3a01ab4d3c4a..22c00d4bbe03 100644 --- a/docs/content/howto/dataframe-api.md +++ b/docs/content/howto/dataframe-api.md @@ -82,8 +82,6 @@ view = recording.view(index="frame_nr", contents={ A view has several APIs to further filter the rows it will return. - - #### Filtering by time range Rows may be filtered to keep only a given range of values from its index column: From 46826aaa11a5d29b35daa719c40c148287ca5356 Mon Sep 17 00:00:00 2001 From: Emil Ernerfeldt Date: Wed, 16 Oct 2024 19:00:03 +0200 Subject: [PATCH 03/10] Add first draft of 0.19 changelog / release notes --- CHANGELOG.md | 105 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 104 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 76d3b692058a..2ea3fa729024 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,7 +1,110 @@ # Rerun changelog -## [Unreleased](https://github.com/rerun-io/rerun/compare/latest...HEAD) +## [Unreleased](https://github.com/rerun-io/rerun/compare/latest...HEAD) - Dataframe & Video + +TODO(emilk): insert a screenshot and/or code sample here + +πŸ“– Release blogpost: TODO(emilk): add link + +🧳 Migration guide: No breaking changes in this release! + +### ✨ Overview & highlights +This release introduces two powerful features: a dataframe API (and view), as well as video support. + +#### ☰ Dataframe Python API +We now have a Python API for querying the contents of an .rrd file. This integrates with popular packages such as [Pandas](https://pandas.pydata.org), [Polars](https://pola.rs), and [DuckDB](https://duckdb.org). + +You can read more in [the Dataframe API how-to guide](https://rerun.io/docs/content/howto/dataframe-api?speculative-link). + +We have also added a matching dataframe view inside the Rerun Viewer. +Read more [here](https://rerun.io/docs/content/reference/types/views/dataframe_view?speculative-link). + +#### 🎬 Video +Rerun now supports logging MP4 videos using the new [`AssetVideo`](https://rerun.io/docs/reference/types/archetypes/asset_video?speculative-link) archetype. +This can greatly reduce bandwidth and storage requirements. + +Currently, only the AV1 codec is supported in the native viewer, but we plan to support H.264 in the near future. +Read more about our video supports (and its limits) [in our video docs](https://rerun.io/docs/reference/video). + +### ⚠️ Breaking changes +None this time! + +### πŸ”Ž Details + +#### πŸͺ΅ Log API +- Refactor `MediaType` guessing [#7326](https://github.com/rerun-io/rerun/pull/7326) +- Tensor & depth image value ranges can now be configured, from UI & code [#7549](https://github.com/rerun-io/rerun/pull/7549) +- New planar pixel formats: `Y_U_V24`/`Y_U_V16`/`Y_U_V12` - `_LimitedRange`/`FullRange` [#7666](https://github.com/rerun-io/rerun/pull/7666) + +#### 🌊 C++ API +- Add `nullptr` check when forwarding from component to datatype [#7430](https://github.com/rerun-io/rerun/pull/7430) + +#### 🐍 Python API +- Add missing `show_labels` and `draw_order` arguments in Python API [#7363](https://github.com/rerun-io/rerun/pull/7363) (thanks [@kpreid](https://github.com/kpreid)!) +- Allow component selectors to be specified as strings with flexible matching [#7695](https://github.com/rerun-io/rerun/pull/7695) +- Allow logging to a recording without first calling `rr.init()` [#7698](https://github.com/rerun-io/rerun/pull/7698) +- Add support for NumPy arrays to the arrow serializer for string datatypes [#7689](https://github.com/rerun-io/rerun/pull/7689) +- Add column property accessors for selectors and descriptors [#7752](https://github.com/rerun-io/rerun/pull/7752) +- Include a `view.schema()` API and improved support for static data [#7754](https://github.com/rerun-io/rerun/pull/7754) + +#### πŸ¦€ Rust API +- Fix Rust's `TimeColumn::new_seconds/new_nanos` creating sequence timelines [#7402](https://github.com/rerun-io/rerun/pull/7402) +- Update ndarray to 0.16 and ndarray-rand to 0.15 [#7358](https://github.com/rerun-io/rerun/pull/7358) (thanks [@benliepert](https://github.com/benliepert)!) +- Replace `host_web_viewer` method with `WebViewerConfig::host_web_viewer` [#7553](https://github.com/rerun-io/rerun/pull/7553) +- Update MSRV to Rust 1.79 [#7563](https://github.com/rerun-io/rerun/pull/7563) +- Implement `From` for `bool` [#7686](https://github.com/rerun-io/rerun/pull/7686) (thanks [@grtlr](https://github.com/grtlr)!) + +#### πŸͺ³ Bug Fixes +- Purge the query cache to prevent GC livelocks [#7370](https://github.com/rerun-io/rerun/pull/7370) +- Bug fix: always show latest data in follow-mode [#7425](https://github.com/rerun-io/rerun/pull/7425) +- Fix encoded image being suggested for non-image blobs (like video) [#7428](https://github.com/rerun-io/rerun/pull/7428) +- Chunk store: support for overlapped range queries [#7586](https://github.com/rerun-io/rerun/pull/7586) +- Fix image & video cache creating new entries when selecting data without explicit media type [#7590](https://github.com/rerun-io/rerun/pull/7590) + +#### 🌁 Viewer improvements +- The viewer will tail an .rrd that's is being written to [#7475](https://github.com/rerun-io/rerun/pull/7475) +- Native video support for AV1 [#7557](https://github.com/rerun-io/rerun/pull/7557) + +#### πŸš€ Performance improvements +- Improve performance for scenes with many entities & transforms [#7456](https://github.com/rerun-io/rerun/pull/7456) +- `Caches` per recording [#7513](https://github.com/rerun-io/rerun/pull/7513) +- Automatic removal of unreachable static chunks [#7518](https://github.com/rerun-io/rerun/pull/7518) +- Invalidate hub-wide caches on deletions and overwrites [#7525](https://github.com/rerun-io/rerun/pull/7525) +- Do not cache static entries in the query-time latest-at cache [#7654](https://github.com/rerun-io/rerun/pull/7654) +- Make sure Arrow `filter` and `take` kernels early out where it makes sense [#7704](https://github.com/rerun-io/rerun/pull/7704) +- Bump `re_rav1d` to 0.1.2 [#7730](https://github.com/rerun-io/rerun/pull/7730) + +#### πŸ§‘β€πŸ« Examples +- Add drone LiDAR example [#7336](https://github.com/rerun-io/rerun/pull/7336) +- add instant_splat example [#7751](https://github.com/rerun-io/rerun/pull/7751) (thanks [@pablovela5620](https://github.com/pablovela5620)!) + +#### πŸ“š Docs +- Add video reference docs [#7533](https://github.com/rerun-io/rerun/pull/7533) +- Document that Rerun does not support left-handed coords [#7690](https://github.com/rerun-io/rerun/pull/7690) +- Add a How-to guide for the dataframe API [#7727](https://github.com/rerun-io/rerun/pull/7727) +- Docs: move "roadmap" down to "development" [#7775](https://github.com/rerun-io/rerun/pull/7775) +- Add a "Getting started" guide for the dataframe API [#7643](https://github.com/rerun-io/rerun/pull/7643) +- Docs: clean up reference menu [#7776](https://github.com/rerun-io/rerun/pull/7776) +- Updating "Navigating the viewer" [#7757](https://github.com/rerun-io/rerun/pull/7757) (thanks [@gavrelina](https://github.com/gavrelina)!) + +#### πŸ–Ό UI improvements +- Add a hook for views to add additional UI in the tab title bar [#7438](https://github.com/rerun-io/rerun/pull/7438) +- Text fields in the selection panel now span the available width [#7487](https://github.com/rerun-io/rerun/pull/7487) +- Do not deselect on ESC when it was used to close some other UI element [#7548](https://github.com/rerun-io/rerun/pull/7548) +- Add UI for precisely picking an exact sequence time [#7673](https://github.com/rerun-io/rerun/pull/7673) +- Remove the feature flag for plot query clamping [#7664](https://github.com/rerun-io/rerun/pull/7664) + +#### πŸ•ΈοΈ Web +- Support h.264 video decoding on web [#7511](https://github.com/rerun-io/rerun/pull/7511) + +#### 🎨 Renderer improvements +- Introduce image data conversion pipeline, taking over existing YUV conversions [#7640](https://github.com/rerun-io/rerun/pull/7640) + +#### πŸ§‘β€πŸ’» Dev-experience +- Add a command palette action to reset egui's memory (debug build only) [#7446](https://github.com/rerun-io/rerun/pull/7446) +- Add NOLINT block to `lint.py` [#7720](https://github.com/rerun-io/rerun/pull/7720) + ## [0.18.2](https://github.com/rerun-io/rerun/compare/0.18.1...0.18.2) - Even more bug fixes From b4fbe1333a9dbfa4fc6eb0a6121e3f413f671bef Mon Sep 17 00:00:00 2001 From: Emil Ernerfeldt Date: Wed, 16 Oct 2024 19:07:41 +0200 Subject: [PATCH 04/10] Trim slightly --- CHANGELOG.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2ea3fa729024..617afca7bb02 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -33,20 +33,17 @@ None this time! ### πŸ”Ž Details #### πŸͺ΅ Log API -- Refactor `MediaType` guessing [#7326](https://github.com/rerun-io/rerun/pull/7326) - Tensor & depth image value ranges can now be configured, from UI & code [#7549](https://github.com/rerun-io/rerun/pull/7549) - New planar pixel formats: `Y_U_V24`/`Y_U_V16`/`Y_U_V12` - `_LimitedRange`/`FullRange` [#7666](https://github.com/rerun-io/rerun/pull/7666) +- Refactor `MediaType` guessing [#7326](https://github.com/rerun-io/rerun/pull/7326) #### 🌊 C++ API - Add `nullptr` check when forwarding from component to datatype [#7430](https://github.com/rerun-io/rerun/pull/7430) #### 🐍 Python API - Add missing `show_labels` and `draw_order` arguments in Python API [#7363](https://github.com/rerun-io/rerun/pull/7363) (thanks [@kpreid](https://github.com/kpreid)!) -- Allow component selectors to be specified as strings with flexible matching [#7695](https://github.com/rerun-io/rerun/pull/7695) - Allow logging to a recording without first calling `rr.init()` [#7698](https://github.com/rerun-io/rerun/pull/7698) - Add support for NumPy arrays to the arrow serializer for string datatypes [#7689](https://github.com/rerun-io/rerun/pull/7689) -- Add column property accessors for selectors and descriptors [#7752](https://github.com/rerun-io/rerun/pull/7752) -- Include a `view.schema()` API and improved support for static data [#7754](https://github.com/rerun-io/rerun/pull/7754) #### πŸ¦€ Rust API - Fix Rust's `TimeColumn::new_seconds/new_nanos` creating sequence timelines [#7402](https://github.com/rerun-io/rerun/pull/7402) From 9a66f92dbd07053420b970c1649be392e1843fb5 Mon Sep 17 00:00:00 2001 From: Emil Ernerfeldt Date: Wed, 16 Oct 2024 19:10:35 +0200 Subject: [PATCH 05/10] Fix title casing/whitespace --- CHANGELOG.md | 6 +++--- scripts/generate_changelog.py | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 617afca7bb02..0aef0b320e38 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -46,13 +46,13 @@ None this time! - Add support for NumPy arrays to the arrow serializer for string datatypes [#7689](https://github.com/rerun-io/rerun/pull/7689) #### πŸ¦€ Rust API -- Fix Rust's `TimeColumn::new_seconds/new_nanos` creating sequence timelines [#7402](https://github.com/rerun-io/rerun/pull/7402) +- Fix Rust's `TimeColumn::new_seconds/new_nanos` creating sequence timelines [#7402](https://github.com/rerun-io/rerun/pull/7402) - Update ndarray to 0.16 and ndarray-rand to 0.15 [#7358](https://github.com/rerun-io/rerun/pull/7358) (thanks [@benliepert](https://github.com/benliepert)!) - Replace `host_web_viewer` method with `WebViewerConfig::host_web_viewer` [#7553](https://github.com/rerun-io/rerun/pull/7553) - Update MSRV to Rust 1.79 [#7563](https://github.com/rerun-io/rerun/pull/7563) - Implement `From` for `bool` [#7686](https://github.com/rerun-io/rerun/pull/7686) (thanks [@grtlr](https://github.com/grtlr)!) -#### πŸͺ³ Bug Fixes +#### πŸͺ³ Bug fixes - Purge the query cache to prevent GC livelocks [#7370](https://github.com/rerun-io/rerun/pull/7370) - Bug fix: always show latest data in follow-mode [#7425](https://github.com/rerun-io/rerun/pull/7425) - Fix encoded image being suggested for non-image blobs (like video) [#7428](https://github.com/rerun-io/rerun/pull/7428) @@ -341,7 +341,7 @@ _All four tetrahedron meshes on this screen share the same vertices and are inst - Recommend install rerun-cli with `--locked` [#6868](https://github.com/rerun-io/rerun/pull/6868) - Remove `TensorBuffer::JPEG`, `DecodedTensor`, `TensorDecodeCache` [#6884](https://github.com/rerun-io/rerun/pull/6884) -#### πŸͺ³Bug Fixes +#### πŸͺ³ Bug fixes - Respect 0.0 for start and end boundaries of scalar axis [#6887](https://github.com/rerun-io/rerun/pull/6887) (thanks [@amidabucu](https://github.com/amidabucu)!) - Fix text log/document view icons [#6855](https://github.com/rerun-io/rerun/pull/6855) - Fix outdated use of view coordinates in `Spaces and Transforms` doc page [#6955](https://github.com/rerun-io/rerun/pull/6955) diff --git a/scripts/generate_changelog.py b/scripts/generate_changelog.py index 166a27719b2d..953e9d3602c7 100755 --- a/scripts/generate_changelog.py +++ b/scripts/generate_changelog.py @@ -284,7 +284,7 @@ def main() -> None: print_section("🌊 C++ API", cpp) print_section("🐍 Python API", python) print_section("πŸ¦€ Rust API", rust) - print_section("πŸͺ³ Bug Fixes", bugs) + print_section("πŸͺ³ Bug fixes", bugs) print_section("🌁 Viewer improvements", viewer) print_section("πŸš€ Performance improvements", performance) print_section("πŸ§‘β€πŸ« Examples", examples) From 5657366711e4ccd6bbe03defde9b2f97bc5619b7 Mon Sep 17 00:00:00 2001 From: Emil Ernerfeldt Date: Wed, 16 Oct 2024 19:36:22 +0200 Subject: [PATCH 06/10] expand video section Co-authored-by: Andreas Reich --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0aef0b320e38..2544e346ef69 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -24,7 +24,7 @@ Read more [here](https://rerun.io/docs/content/reference/types/views/dataframe_v Rerun now supports logging MP4 videos using the new [`AssetVideo`](https://rerun.io/docs/reference/types/archetypes/asset_video?speculative-link) archetype. This can greatly reduce bandwidth and storage requirements. -Currently, only the AV1 codec is supported in the native viewer, but we plan to support H.264 in the near future. +While the web viewer supports a variety of codecs, the native viewer supports only the AV1 codec for the moment, but we plan to support H.264 in the near future as well. Read more about our video supports (and its limits) [in our video docs](https://rerun.io/docs/reference/video). ### ⚠️ Breaking changes From 0c4f303d367f34decd081e4a1c1fc41d06f3196a Mon Sep 17 00:00:00 2001 From: Emil Ernerfeldt Date: Wed, 16 Oct 2024 19:37:13 +0200 Subject: [PATCH 07/10] Remove web/video PR link --- CHANGELOG.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2544e346ef69..bb7e4b48669a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -92,9 +92,6 @@ None this time! - Add UI for precisely picking an exact sequence time [#7673](https://github.com/rerun-io/rerun/pull/7673) - Remove the feature flag for plot query clamping [#7664](https://github.com/rerun-io/rerun/pull/7664) -#### πŸ•ΈοΈ Web -- Support h.264 video decoding on web [#7511](https://github.com/rerun-io/rerun/pull/7511) - #### 🎨 Renderer improvements - Introduce image data conversion pipeline, taking over existing YUV conversions [#7640](https://github.com/rerun-io/rerun/pull/7640) From 1cc65bb0c1c35771ec7715a60bc40e682aae76de Mon Sep 17 00:00:00 2001 From: Emil Ernerfeldt Date: Wed, 16 Oct 2024 19:37:58 +0200 Subject: [PATCH 08/10] No thanks to the designer --- CHANGELOG.md | 2 +- scripts/generate_changelog.py | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bb7e4b48669a..dab7a583eac8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -83,7 +83,7 @@ None this time! - Docs: move "roadmap" down to "development" [#7775](https://github.com/rerun-io/rerun/pull/7775) - Add a "Getting started" guide for the dataframe API [#7643](https://github.com/rerun-io/rerun/pull/7643) - Docs: clean up reference menu [#7776](https://github.com/rerun-io/rerun/pull/7776) -- Updating "Navigating the viewer" [#7757](https://github.com/rerun-io/rerun/pull/7757) (thanks [@gavrelina](https://github.com/gavrelina)!) +- Updating "Navigating the viewer" [#7757](https://github.com/rerun-io/rerun/pull/7757) #### πŸ–Ό UI improvements - Add a hook for views to add additional UI in the tab title bar [#7438](https://github.com/rerun-io/rerun/pull/7438) diff --git a/scripts/generate_changelog.py b/scripts/generate_changelog.py index 953e9d3602c7..d64252b5d521 100755 --- a/scripts/generate_changelog.py +++ b/scripts/generate_changelog.py @@ -28,6 +28,7 @@ OFFICIAL_RERUN_DEVS = [ "abey79", "emilk", + "gavrelina", "jleibs", "jprochazk", "nikolausWest", From 7f375dcb6840e0087078153bf2d5a457014fdccc Mon Sep 17 00:00:00 2001 From: Emil Ernerfeldt Date: Wed, 16 Oct 2024 19:38:34 +0200 Subject: [PATCH 09/10] Rermove another line --- CHANGELOG.md | 1 - 1 file changed, 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index dab7a583eac8..a3f3c85db854 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -70,7 +70,6 @@ None this time! - Invalidate hub-wide caches on deletions and overwrites [#7525](https://github.com/rerun-io/rerun/pull/7525) - Do not cache static entries in the query-time latest-at cache [#7654](https://github.com/rerun-io/rerun/pull/7654) - Make sure Arrow `filter` and `take` kernels early out where it makes sense [#7704](https://github.com/rerun-io/rerun/pull/7704) -- Bump `re_rav1d` to 0.1.2 [#7730](https://github.com/rerun-io/rerun/pull/7730) #### πŸ§‘β€πŸ« Examples - Add drone LiDAR example [#7336](https://github.com/rerun-io/rerun/pull/7336) From e4e52efc8dfe187e13b5b88a8e2b30d496602d27 Mon Sep 17 00:00:00 2001 From: Emil Ernerfeldt Date: Wed, 16 Oct 2024 19:40:29 +0200 Subject: [PATCH 10/10] Remove `ShowLabel` line --- CHANGELOG.md | 1 - 1 file changed, 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a3f3c85db854..5cc89b4943c7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -50,7 +50,6 @@ None this time! - Update ndarray to 0.16 and ndarray-rand to 0.15 [#7358](https://github.com/rerun-io/rerun/pull/7358) (thanks [@benliepert](https://github.com/benliepert)!) - Replace `host_web_viewer` method with `WebViewerConfig::host_web_viewer` [#7553](https://github.com/rerun-io/rerun/pull/7553) - Update MSRV to Rust 1.79 [#7563](https://github.com/rerun-io/rerun/pull/7563) -- Implement `From` for `bool` [#7686](https://github.com/rerun-io/rerun/pull/7686) (thanks [@grtlr](https://github.com/grtlr)!) #### πŸͺ³ Bug fixes - Purge the query cache to prevent GC livelocks [#7370](https://github.com/rerun-io/rerun/pull/7370)