Support for multiple audio tracks

[saerdnaer]

The files from the congress have multiple audio tracks: an orignal and translated version. The web player should display which one is selected and allow to change the track. Extra credits for the possibility to play both tracks with different volume, e.g. the translated track with low volume original for more atmosphere.

Some browsers/players e.g. Firefox on Windows or the native ios player (used by most browsers on that platform) automatically select the track in the system language. So watching http://media.ccc.de/browse/congress/2013/30C3_-_5543_-_en_-_saal_2_-_201312291245_-_id_cards_in_china_your_worst_nightmare_-_kate_krauss.html# in on a german os plays the translated audio track and an new user isn't aware that there is an original english sound track as Firefox it self doesn't display any message.

[irrweg]

With the Firefox Browser, without any Plug-In like VLC, its playing the translated version per default.
It does not matter, if the Presentation was in German or in English.

If the presentation is in German, he plays the English audio, If the presentation is in English, he plays the German audio. Really annoying.

Can't see any option in the drop-down menu for changing the audio source or even on the web player.
But the webm version plays the O-Ton, also no options for the audio.

Maybe It's an issues with Firefox...

[saerdnaer]

Same problem in Safari 7.1 on Mac OS X 10.9.5

[saerdnaer]

@MaZderMind can you add your findings to this ticket?

[MaZderMind]

Well, it's not more then the results of a quick google-around.
Aparently there's an API specified in HTML5to work with multiple audio/video tracks:
http://www.w3.org/TR/html5/embedded-content-0.html#dom-media-audiotracks

but no browser implements it:
http://www.w3schools.com/tags/av_prop_audiotracks.asp
https://bugzilla.mozilla.org/show_bug.cgi?id=744896

Apart from IE10:
http://msdn.microsoft.com/de-de/library/windows/apps/xaml/hh772507(v=vs.85).aspx

For that Reason mediaelement.js, our web-player, does not provide support for it either. It may however be possible to add the translated audio as separate <audio>-Player which is somehow synced to the video-player, because a little delay should not be a problem with translated audio. This would probably not work with the flash fallback and it would require a separate mp3 with the translated audio.

In the end I'd not recommend such a hack bit instead try to force browsers to implement the audioTracks-Feature. With todays Update-Cycles on Webkit & FF this really becomes an option (especially as IE10 already has it)

[MaZderMind]

As it seems FF33, which was released 4 days ago, does implement this when one enables media.track.enabled in about:config. A quick test reveals that it does indeed enable the audioTracks property but it still only contains a single track.

At least there seems to be a chance to get this feature sooner then later.

[saerdnaer]

If we create audio only files for the translated tracks we use an synchronized audio player and use the video player only for the video track. This would also give the possibility to play two audio tracks with different volumes at the same time, e.g. the original audio with low volume at the background of a high volume translated track at the 30C3 recordings.

Further readings:

• http://blog.denivip.ru/index.php/2013/10/alternative-audio-tracks-in-html5-video/?lang=en
• http://gingertech.net/2011/05/01/html5-multi-track-audio-or-video/

[saerdnaer]

@line-o FYI

[manno]

Is this happening with all formats (mp4-sd, mp4-hd,webm)? Does this affect the flash fallback, too?

[manno]

I remuxed the mp4 files for 30c3/31c3 and dropped the second audio track. I filtered the source tags of the video tag to only include one source of each mime type and gave preference to the remuxed version.

[MaZderMind]

it would probably an option to only distribute the native audio with the video files and play translated audio via a separate -Element, as @saerdnaer suggested. For translated Audio lipsync is not a blocking requirement as it is for native audio.

This would also fix most of the iOS / Android player problems, although it would violate the idea to push the player-devs into fixing their players by annoying their users with good but unplayable content ^^

[saerdnaer]

From todays meeting:
It should be possible to implement this feature via https://en.wikipedia.org/wiki/Media_Source_Extensions

Support in the major browsers was added during this year