Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move audio import into Trade #326

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

Move audio import into Trade #326

wants to merge 1 commit into from

Conversation

mosra
Copy link
Owner

@mosra mosra commented Mar 9, 2019

Halfway through adding file callback support to Audio::AbstractImporter for consistency with scene and font importers I realized it just doesn't make sense to have audio import separate in the Audio library. Reasons:

  • Audio and the BufferFormat enum depend on OpenAL, while the actual data import doesn't, at all, basically preventing users from using the importer plugins together with, let's say, fmod, SoLoud or other implementations
  • the importers are built for importing just one audio track -- what about playlists etc.? once we add multi-track support, we're even closer to Trade::AbstractImporter
  • complex scenes that have animations usually also have audio data for the animations, audio attached to various objects, having various orientation etc. -- but at the moment, there's no builtin way to connect these two apart from digging manually into importerState() -- and then, if the audio data are embedded in the file, no way of passing those to AudioImporters
  • there's the proposed MSFT_audio_emitter extension for glTF (or this for OGEX), would be nice to have that supported in the Magnum Player, for example

A possible follow-up question could be if Text::AbstractFont should get merged into Trade as well. In my opinion nope, since text rendering is a very specific thing and it's far from "having a buffer with data that you pass to the GPU or the audio card". Besides that, text objects are usually not embedded in scenes (unlike audio) and if such need arises, it can be always done via a format-specific extension.

Things to do:

  • drop the WIP commit I have here, as nothing there is useful anyway
  • introduce a new Magnum::AudioFormat zero-based enum
    • provide its mapping to Audio::BufferFormat
    • what about all the weird mu-Law things? Supported by the Wav importer, but ...
  • new Trade::AudioData class, wrapping format, frequency and buffer data
  • add audioCount(), audioName(), audioForName(), audio() accessors to Trade::AbstractImporter
  • rewrite AnyAudioImporter and WavAudioImporter to be a part of Trade
  • start testing audio import on Android (was blocked by Audio library and OpenAL on other platforms #149 until now, with this the AL dependency would no longer be a problem)
  • rewrite all other audio importer plugins
  • implement streaming import (will need to be done for Video/audio streaming APIs #360 as well)
  • stop depending on Audio and OpenAL in the plugins repo
    • simplify the CIs
    • test audio plugins on Android there
  • think about how can we provide backwards compatibility? Audio::AbstractImporter hooking into Trade? ugh

@mosra mosra self-assigned this Mar 9, 2019
@hsdk123
Copy link
Contributor

hsdk123 commented Nov 15, 2019

Looking forward to this! (audio streaming)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: TODO
Development

Successfully merging this pull request may close these issues.

2 participants