Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support HDR video (shows wrong color or glitches today) #7594

Open
Wumpf opened this issue Oct 4, 2024 · 5 comments
Open

Support HDR video (shows wrong color or glitches today) #7594

Wumpf opened this issue Oct 4, 2024 · 5 comments

Comments

@Wumpf
Copy link
Member

Wumpf commented Oct 4, 2024

Some of it might be shortcomings in the (web) decoder, but we need to take a more serious look at this. Meanwhile HDR/BT.2020 video should not be supported.

Firefox @ Mac on av1
image

Chrome @ Mac on the same video
image
(colors are still wrong)

AV1 native:
image

Possibly related to:

@emilk
Copy link
Member

emilk commented Oct 4, 2024

Can we detect HDR videos and log a warning about it?

@Wumpf
Copy link
Member Author

Wumpf commented Oct 4, 2024

given that for all these cases I see BT.2020 (instead of (SDR) BT.709) on the "media information" in VLC, I suspect that this is a piece of information provided by mp4 🤔 . In any case, yes there gotta be some way and a warning would be in order!

emilk added a commit that referenced this issue Oct 8, 2024
### What
* Uses rerun-io/re_mp4#10

Improves the video blob info in two ways:
* Adds bit depth (if known)
* Shows human-redable name of codec (e.g. AV1 or H.264)

![Screenshot 2024-10-08 at 15 25
24](https://github.com/user-attachments/assets/8a104b85-32fa-4176-8fa3-bd20bca01df6)

![Screenshot 2024-10-08 at 15 25
35](https://github.com/user-attachments/assets/db377cb7-d960-4edc-ad6b-ad60a3363e12)

![Screenshot 2024-10-08 at 15 27
41](https://github.com/user-attachments/assets/cebcb06b-7e16-4101-b9c1-dee549b6038e)


A warning will also be printed if trying to decode a HDR video (since we
don't yet support that).

Related:
* #7594

### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested the web demo (if applicable):
* Using examples from latest `main` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/7633?manifest_url=https://app.rerun.io/version/main/examples_manifest.json)
* Using full set of examples from `nightly` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/7633?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json)
* [x] The PR title and labels are set such as to maximize their
usefulness for the next release's CHANGELOG
* [x] If applicable, add a new check to the [release
checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)!
* [x] If have noted any breaking changes to the log API in
`CHANGELOG.md` and the migration guide

- [PR Build Summary](https://build.rerun.io/pr/7633)
- [Recent benchmark results](https://build.rerun.io/graphs/crates.html)
- [Wasm size tracking](https://build.rerun.io/graphs/sizes.html)

To run all checks from `main`, comment on the PR with `@rerun-bot
full-check`.
@Wumpf
Copy link
Member Author

Wumpf commented Oct 11, 2024

For reference: We do warn by now in a few places. In some even about the technically distinct cases of:

  • HDR primaries
  • high bit depth

(the distinction is kinda silly, but it looks like depending on the format it's real!)

emilk added a commit that referenced this issue Nov 2, 2024
### What
* Closes #7973
* Part of #7594

Review with ignoring whitespace changes

#### Before
<img width="1639" alt="Screenshot 2024-11-02 at 12 02 13"
src="https://github.com/user-attachments/assets/500e5b10-c82f-483b-9e2d-c5cc75f29e52">


#### After
<img width="1639" alt="Screenshot 2024-11-02 at 12 00 59"
src="https://github.com/user-attachments/assets/c7067210-bb47-4cf7-8633-30a8b3edb42a">

#### VLC (for reference)
<img width="1552" alt="Screenshot 2024-11-02 at 16 43 10"
src="https://github.com/user-attachments/assets/f85be55c-3878-4fa0-b6fd-3b54f015622a">

### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested the web demo (if applicable):
* Using examples from latest `main` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/7978?manifest_url=https://app.rerun.io/version/main/examples_manifest.json)
* Using full set of examples from `nightly` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/7978?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json)
* [x] The PR title and labels are set such as to maximize their
usefulness for the next release's CHANGELOG
* [x] If applicable, add a new check to the [release
checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)!
* [x] If have noted any breaking changes to the log API in
`CHANGELOG.md` and the migration guide

- [PR Build Summary](https://build.rerun.io/pr/7978)
- [Recent benchmark results](https://build.rerun.io/graphs/crates.html)
- [Wasm size tracking](https://build.rerun.io/graphs/sizes.html)

To run all checks from `main`, comment on the PR with `@rerun-bot
full-check`.

---------

Co-authored-by: Andreas Reich <[email protected]>
@Wumpf
Copy link
Member Author

Wumpf commented Nov 2, 2024

the distinction [of HDR bit depth & HDR color spaces] is kinda silly

turns out it is not! As demonstrated there's high bit depth Bt709 video material out there! See #7973

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants