New planar pixel formats: Y_U_V24/Y_U_V16/Y_U_V12 - `_LimitedRa…

…nge`/`FullRange` (#7666)

### What

Adds a whole bunch of new pixel formats with limited & full range YUV
variants. All of these are needed by our AV1 decoder (branch is a
misnomer though, this is coming in another pr!)

----

Naming & categorizing these formats is quite the struggle just as
expected:
If limited to YUV variants, the overall picutre becomes sharper (ironic,
eh) and it's possible to specify the combinatorics in a deceptively
simple pattern:
* planarity & downsample ratio
* limited/full value range
* bit depth (we only do 8 bit right now)
* color primaries

That's what is happening in `re_renderer`'s YUV converter now and is
very useful for writing the conversions themselves!

However (!!!), in the wild names make these properties often convoluted
with each other, making the categorization a lot harder. E.g. NV12 is
almost always (afaik just _almost_) implying `LimitedRange` YUV (which
is generally a common default unless you're dealing with jpeg derived
data [...]).
This, and the fact that we already set out with `PixelFormat` for a flat
list of names, strongly compelled me to continue the current naming
scheme (and Ocean's numbering as far as possible) in the user facing
API. Worth revisiting in the future.

The primary motivation to expose these formats right away was to be able
to easily test these variants. However, I refrained from throwing color
primaries into the mix here because I think they should in fact be
separately defined, causing a bigger expansion of the APIs. The docs are
intentionally kept vague on color space for the moment (trying hard to
not further get entangled here).
As a result, our internal processing is now fairly explicit on the
matter, but the input interpretation is rather vague.


<img width="1405" alt="image"
src="https://github.com/user-attachments/assets/883aee03-9ef1-46f8-bec6-96af0f0713f3">
(screenshot of the release checklist which is part of this pr (moved
from a previous test). I manually tested the color inspection feature
against a reference color picker - slight rounding errors occasionally,
but they are hard to avoid since GPU/CPU don't always do the exact same
thing :/)


https://rerun.io/viewer/pr/7666?url=https%3A%2F%2Fstatic.rerun.io%2Frrd%2F0.19.0%2Fchroma_4688549ca3c20018fa2fb658b56482b955817b6a.rrd

### Checklist
* [ ] chrome/firefox windows
* [x] chrome/firefox/safari mac
* [ ] chrome/firefox linux
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested the web demo (if applicable):
* Using examples from latest `main` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/7666?manifest_url=https://app.rerun.io/version/main/examples_manifest.json)
* Using full set of examples from `nightly` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/7666?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json)
* [x] The PR title and labels are set such as to maximize their
usefulness for the next release's CHANGELOG
* [x] If applicable, add a new check to the [release
checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)!
* [x] If have noted any breaking changes to the log API in
`CHANGELOG.md` and the migration guide

- [PR Build Summary](https://build.rerun.io/pr/7666)
- [Recent benchmark results](https://build.rerun.io/graphs/crates.html)
- [Wasm size tracking](https://build.rerun.io/graphs/sizes.html)

To run all checks from `main`, comment on the PR with `@rerun-bot
full-check`.

.gitignore

@@ -31,6 +31,7 @@ _deps
 # Python build artifacts:
 __pycache__
 *.pyc
+*.pyd
 *.so
 **/.pytest_cache
 

crates/build/re_types_builder/src/codegen/rust/api.rs

@@ -161,6 +161,11 @@ fn generate_object_file(
         code.push_str("#![allow(deprecated)]\n");
     }
 
+    if obj.is_enum() {
+        // Needed for PixelFormat. Should we limit this via attribute to just that?
+        code.push_str("#![allow(non_camel_case_types)]\n");
+    }
+
     code.push_str("\n\n");
 
     code.push_str("use ::re_types_core::external::arrow2;\n");

crates/store/re_types/definitions/rerun/datatypes/pixel_format.fbs

@@ -1,5 +1,9 @@
 namespace rerun.datatypes;
 
+// TODO(andreas): Clarify relationship to color primaries. Right now there's some hardcoded differences between formats.
+// See `image_to_gpu.rs`
+// Suggestion: guides heuristic but doesn't specify it unless noted.
+
 /// Specifieds a particular format of an [archetypes.Image].
 ///
 /// Most images can be described by a [datatypes.ColorModel] and a [datatypes.ChannelDatatype],
@@ -24,13 +28,82 @@ enum PixelFormat: ubyte {
     // this organization and subsequently reduce the chance we may find ourselves wanting to
     // change the values in the future.
 
-    /// `NV12` (aka `Y_UV12`) is a YUV 4:2:0 chroma downsampled format with 12 bits per pixel and 8 bits per channel.
+    /// `Y_U_V24` is a YUV 4:4:4 fully planar YUV format without chroma downsampling, also known as `I444`.
+    ///
+    /// This uses limited range YUV, i.e. Y is expected to be within [16, 235] and U/V within [16, 240].
+    ///
+    /// First comes entire image in Y in one plane, followed by the U and V planes.
+    Y_U_V24_LimitedRange = 39,
+
+    /// `Y_U_V24` is a YUV 4:4:4 fully planar YUV format without chroma downsampling, also known as `I444`.
+    ///
+    /// This uses full range YUV with all components ranging from 0 to 255
+    /// (as opposed to "limited range" YUV as used e.g. in NV12).
+    ///
+    /// First comes entire image in Y in one plane, followed by the U and V planes.
+    Y_U_V24_FullRange = 40,
+
+    /// `Y_U_V16` is a YUV 4:2:2 fully planar YUV format without chroma downsampling, also known as `I422`.
+    ///
+    /// This uses limited range YUV, i.e. Y is expected to be within [16, 235] and U/V within [16, 240].
+    ///
+    /// First comes entire image in Y in one plane, followed by the U and V planes, which each only have half
+    /// the horizontal resolution of the Y plane.
+    Y_U_V16_LimitedRange = 49, // Ocean doesn't have a short codes for this
+
+    /// `Y_U_V16` is a YUV 4:2:2 fully planar YUV format without chroma downsampling, also known as `I422`.
+    ///
+    /// This uses full range YUV with all components ranging from 0 to 255
+    /// (as opposed to "limited range" YUV as used e.g. in NV12).
+    ///
+    /// First comes entire image in Y in one plane, followed by the U and V planes, which each only have half
+    /// the horizontal resolution of the Y plane.
+    Y_U_V16_FullRange = 50, // Ocean doesn't have a short codes for this
+
+    /// `Y_U_V12` is a YUV 4:2:0 fully planar YUV format without chroma downsampling, also known as `I420`.
+    ///
+    /// This uses limited range YUV, i.e. Y is expected to be within [16, 235] and U/V within [16, 240].
+    ///
+    /// First comes entire image in Y in one plane, followed by the U and V planes, which each only have half
+    /// the resolution of the Y plane.
+    Y_U_V12_LimitedRange = 20,
+
+    /// `Y_U_V12` is a YUV 4:2:0 fully planar YUV format without chroma downsampling, also known as `I420`.
+    ///
+    /// This uses full range YUV with all components ranging from 0 to 255
+    /// (as opposed to "limited range" YUV as used e.g. in NV12).
+    ///
+    /// First comes entire image in Y in one plane, followed by the U and V planes, which each only have half
+    /// the resolution of the Y plane.
+    Y_U_V12_FullRange = 44,
+
+    /// Monochrome Y plane only, essentially a YUV 4:0:0 planar format.
+    ///
+    /// Also known as just "gray".
+    ///
+    /// This uses limited range YUV, i.e. Y is expected to be within [16, 235].
+    /// If not for this range limitation/remapping, this is almost identical to 8bit luminace/grayscale (see [datatypes.ColorModel]).
+    Y8_LimitedRange = 41,
+
+    /// Monochrome Y plane only, essentially a YUV 4:0:0 planar format.
+    ///
+    /// Also known as just "gray". This is virtually identical to a 8bit luminance/grayscale (see [datatypes.ColorModel]).
+    ///
+    /// This uses entire range YUV, i.e. Y is expected to be within [0, 255].
+    /// (as opposed to "limited range" YUV as used e.g. in NV12).
+    Y8_FullRange = 30,
+
+    /// `NV12` (aka `Y_UV12`) is a YUV 4:2:0 chroma downsampled form at with 12 bits per pixel and 8 bits per channel.
+    ///
+    /// This uses limited range YUV, i.e. Y is expected to be within [16, 235] and U/V within [16, 240].
     ///
     /// First comes entire image in Y in one plane,
     /// followed by a plane with interleaved lines ordered as U0, V0, U1, V1, etc.
     NV12 = 26 (default), // _something_ has to be the default 🤷‍♀️
 
-    /// `YUY2` (aka `YUYV` or `YUYV16`), is a YUV 4:2:2 chroma downsampled format with 16 bits per pixel and 8 bits per channel.
+    /// `YUY2` (aka `YUYV`, `YUYV16` or `NV21`), is a YUV 4:2:2 chroma downsampled format with 16 bits per pixel and 8 bits per channel.
+    ///
+    /// This uses limited range YUV, i.e. Y is expected to be within [16, 235] and U/V within [16, 240].
     ///
     /// The order of the channels is Y0, U0, Y1, V0, all in the same plane.
     YUY2 = 27,