Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-write in C using WASM #6

Closed
Arcus92 opened this issue Jul 3, 2024 · 3 comments
Closed

Re-write in C using WASM #6

Arcus92 opened this issue Jul 3, 2024 · 3 comments
Assignees
Labels
enhancement New feature or request

Comments

@Arcus92
Copy link
Owner

Arcus92 commented Jul 3, 2024

Parsing the PGS files could be slow on low-end devices using JavaScript. I'm working on a C implementation using WebAssembly with Emscripten and WebWorker support.

@Arcus92 Arcus92 added the enhancement New feature or request label Jul 3, 2024
@Arcus92 Arcus92 self-assigned this Jul 3, 2024
@ferferga
Copy link

ferferga commented Jul 7, 2024

I'd advise being cautious about this one. Currently, WASM works flawlessly in Firefox (regarding raw performance), but in Chromium browsers it's really slow. You can do a quick test here with an intensive workload, it's night and day: https://ffmpegwasm.netlify.app/docs/getting-started/usage/

In my thesis I did quite a few tests with WASM and I came to the conclusion that WASM doesn't give noticeable improvements for performance-critical scenerios, in it's current state it's main purpose is for those applications that can't be ported/developed in JavaScript, like our libass in SubtitlesOctopus or Google Earth and Photoshop (for some public app examples).

If you want to do some benchmarks in a real application, you can install blurhash-wasm into our current web client or Jellyfin Vue and compare it with the current WebWorker implementation (there might be differences in the algorithm implementation, but I think it's one simple enough test that you could try)

What if you split up the current worker into 2 workers? One for OffscreenCanvas and another for the other? So they don't block each other. If it's still slow in some devices, you can still give it a go with memoization.

@Arcus92
Copy link
Owner Author

Arcus92 commented Jul 7, 2024

@ferferga Thanks for your advice.
You are just confirming my doubts on this topic. I spend the last week looking into and learning WebAssembly and discovered the big overhead WASM is adding to this project. The only area I saw an issue was allocating many Color instances when parsing the palette. Structs could solve this. However, most subtitles only use a hand full of colors. Also browsers are optimized to handle many instances in an array of the same type. For fun I'll try to store the pixel data in one big byte array and measure if it makes any difference.
I also got a Raspberry PI 4 to test. If I can view a 1080p video with the current PGS implementation on Chromium and Firefox without adding to much CPU load I'll call it a day for now.

@Arcus92
Copy link
Owner Author

Arcus92 commented Jul 9, 2024

Made some good changed in #9 to improve the performance.
It runs smoothly with 1080p - 10 Mbps on a Raspberry PI 4 (4gb, no overclocking) on Chromium.
Firefox already struggled with 1080p video streaming. Subtitle rendering or clearing can make the video lag. Pixel data encoding is not the issue though. clearRect and drawImage are blocking the browser - even from a web-worker. I guess the software renderer is just that slow.

@Arcus92 Arcus92 closed this as completed Aug 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants