For some movies, QSV transcoding produces unplayable HLS segments

[chenxiaolong]

Describe The Bug

For some movies, when transcoding to 1080p and playing back in Firefox, Chromium, or the Jellyfin Android app, it downloads about ~7 of the .mp4 HLS segments but fails to play any of them.

In both Firefox and Chrome, the console logs:

HLS Error: Type: mediaError Details: fragParsingError Fatal: false
HLS Error: Type: mediaError Details: fragParsingError Fatal: false
HLS Error: Type: mediaError Details: fragParsingError Fatal: false
HLS Error: Type: mediaError Details: fragParsingError Fatal: false
HLS Error: Type: mediaError Details: fragParsingError Fatal: false
HLS Error: Type: mediaError Details: fragParsingError Fatal: false
HLS Error: Type: mediaError Details: fragParsingError Fatal: false
HLS Error: Type: mediaError Details: fragParsingError Fatal: true

It seems that this only happens when using Intel QSV hardware encoding and only for some files.

The file I was using for testing was a rip of the UHD bluray of Inception (2010). To make sure this isn't a result of a bad disc, I purchased a second copy, ripped it, and got a byte-for-byte identical video stream (after demuxing it from the .mkv).

Steps To Reproduce

1. Enable Intel QSV hardware encoding with default settings
2. Play back an affected file (eg. UHD bluray of Inception)
3. Change the quality to trigger transcoding (I tested with 1080p @ 20Mbps)
4. Seek to the middle of the movie
5. If playing in a browser, open the dev tools and observe the HLS fragParsingErrors in the console log

Expected Behavior

Intel QSV encoded HLS segments should play fine like software-encoded HLS segments.

System (please complete the following information):

• Platform: Linux
• Operating System: Fedora 40 x86_64
• Virtualization: Bare metal
• Jellyfin Version: 10.9.6
• Jellyfin-FFmpeg Version: 6.0.1-7
• Hardware Acceleration: QSV
• CPU & GPU Model: i5-13600K (13th gen Raptor Lake)
• Kernel version: 6.8.9-300.fc40.x86_64
• intel-media-driver version: 24.1.5-1.fc40.x86_64
• intel-vpl-gpu-rt version: 24.2.4-1.fc40.x86_64
• libvpl version: 2.11.0-1.fc40.x86_64

MediaInfo

mediainfo.txt

FFmpeg Logs

When using Intel QSV (does not work):

Command:

[2024-07-09 19:13:20.331 -04:00] [INF] [88] MediaBrowser.MediaEncoding.Transcoding.TranscodeManager: "/usr/libexec/jellyfin-ffmpeg/ffmpeg" "-analyzeduration 200M -probesize 1G -init_hw_device vaapi=va:,kernel_driver=i915,driver=iHD -init_hw_device qsv=qs@va -filter_hw_device qs -hwaccel vaapi -hwaccel_output_format vaapi -noautorotate -i file:\"/media/media/Movies/Inception (2010)/Inception (2010) - 2160p.mkv\" -noautoscale -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 h264_qsv -preset veryfast -look_ahead 0 -b:v 19616000 -maxrate 19616000 -bufsize 39232000 -g:v:0 72 -keyint_min:v:0 72 -vf \"setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale_vaapi=w=1920:h=1080:format=nv12:extra_hw_frames=24,hwmap=derive_device=qsv,format=qsv\" -codec:a:0 libfdk_aac -ac 2 -ab 256000 -af \"volume=2\" -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type fmp4 -hls_fmp4_init_filename \"eea7566d1f659fdbdb3be2167e7b35d8-1.mp4\" -start_number 0 -hls_segment_filename \"/var/lib/jellyfin/transcodes/eea7566d1f659fdbdb3be2167e7b35d8%d.mp4\" -hls_playlist_type vod -hls_list_size 0 -y \"/var/lib/jellyfin/transcodes/eea7566d1f659fdbdb3be2167e7b35d8.m3u8\""

Logs: ffmpeg_qsv_encoding.txt

When using software encoding (works):

Command:

"/usr/libexec/jellyfin-ffmpeg/ffmpeg" "-analyzeduration 200M -probesize 1G -f matroska,webm -noautorotate -i file:\"/media/media/Movies/Inception (2010)/Inception (2010) - 2160p.mkv\" -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 libx264 -preset veryfast -crf 23 -maxrate 19616000 -bufsize 39232000 -x264opts:0 subme=0:me_range=4:rc_lookahead=10:me=dia:no_chroma_me:8x8dct=0:partitions=none -force_key_frames:0 \"expr:gte(t,n_forced*3)\" -sc_threshold:v:0 0 -vf \"setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale=trunc(min(max(iw\,ih*a)\,1920)/2)*2:trunc(ow/a/2)*2,format=yuv420p\" -codec:a:0 libfdk_aac -ac 2 -ab 256000 -af \"volume=2\" -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type fmp4 -hls_fmp4_init_filename \"665d94cfdf6a6f5e56203d1257e4d207-1.mp4\" -start_number 0 -hls_segment_filename \"/var/lib/jellyfin/transcodes/665d94cfdf6a6f5e56203d1257e4d207%d.mp4\" -hls_playlist_type vod -hls_list_size 0 -y \"/var/lib/jellyfin/transcodes/665d94cfdf6a6f5e56203d1257e4d207.m3u8\""

Logs: ffmpeg_software_encoding.txt

Additional Context

I'm not super familiar with the mp4 format, but I tried to poke around the HLS segments with fq. I don't see any difference in structure. The QSV output has much larger sample sizes, but I'm not sure that's actually the cause of the issue.

[chenxiaolong]

It looks like the same behavior happens if I use VAAPI instead of QSV too.

[nyanmisaka]

This is file specific, please provide a clip so we can investigate.

[chenxiaolong]

Sure thing. I double checked to make sure the issue is still reproducible for me with a sample clip and it looks like it still is.

• 2 minute sample clip produced by ffmpeg -to 120 -i input.mkv -c copy sample.mkv
  • (621.1 MiB) https://files.pub.chiller3.com/issues/jellyfin-ffmpeg/issue413/sample.mkv
• Transcoded HLS output with QSV encoding
  • (287.5 MiB) https://files.pub.chiller3.com/issues/jellyfin-ffmpeg/issue413/transcodes_qsv.tar
• Transcoded HLS output with software encoding
  • (45.1 MiB) https://files.pub.chiller3.com/issues/jellyfin-ffmpeg/issue413/transcodes_software.tar

EDIT: Links deleted

[nyanmisaka]

Received. You can remove the link.

[Anonymo]

Just want to applaud your very detailed and thorough investigation. I usually don't see bug reports done this well.

[chenxiaolong]

I've been meaning to retest this after the recent jellyfin-ffmpeg 7.x releases, but I unfortunately can no longer do so. My motherboard (Supermicro X13SAE-F) failed and I replaced it with a system that uses an AMD iGPU.

[nyanmisaka]

I've been meaning to retest this after the recent jellyfin-ffmpeg 7.x releases, but I unfortunately can no longer do so. My motherboard (Supermicro X13SAE-F) failed and I replaced it with a system that uses an AMD iGPU.

I tried to reproduce this issue on an Arc GPU with no success. Perhaps a driver/ffmpeg update has fixed it.