Soundfont2 (SF2) support

[psi29a]

https://en.wikipedia.org/wiki/SoundFont

At the very least, we should be able to handle the config file that ships with timitity:
/etc/timidity/fluidr3_gm.cfg

[kotc]

hi, i would love to have support for sf2!
can i help you by pointing you to unsf utility? i have used it to produce guspatches that i'm already using in my app with libwildmidi. if you need source it can be found inside 'gt' in fedora repos (ie. ftp://ftp.icm.edu.pl/vol/rzm5/linux-fedora/linux/releases/21/Everything/source/SRPMS/g/gt-0.4-19.fc21.src.rpm )

[kotc]

btw. unsf author pointed out features of sf2 not existing in gus format and proposed way to extend it. still, as i said, produced patches are already usable and sounding good

[psi29a]

If the unsf utility is LGPL then we might be able to use parts of it. The idea is to support sf2 in libwildmidi itself without having to dump first to disk then to playback. This means playing the patch natively from out the sf2 bank file, taking into account sf2's special features as well.

We might do support on phases.

[psi29a]

http://alsa.opensrc.org/Unsf
^-- this also talks about extended GUS format too.... interesting. :)

@chrisisonwildcode does this tickle your fancy ? ;)

[kotc]

i think it's available freely.

from the unsf.c:
/*

• unsf breaks up sound fonts into GUS type patch files.
  *
• usage: unsf
  *
• compile: cc -o unsf unsf.c -lm
  *
• Credits:
• unsf.c is derived from the Allegro tool pat2dat.c, and
• uses, presumably, the part of that program which was written
• originally by George Foot. The original file intro of pat2dat.c
• and information on copyright is given below. Also, some code
• is adapted from routines in sndfont.c, whose header has
• the notice:
  *================================================================
• SoundFont file extension

•  written by Takashi Iwai [email protected]
  

  ================================================================
  *
  *

•                                          Greg Lee, 6/25/04
  

  /
  / ______ ___ ___

•    /\  _  \ /_ \  /_ \
  

•    \ \ \L\ \//\ \ \//\ \      __     __   _ __   ___
  

•     \ \  __ \ \ \ \  \ \ \   /'**`\ /'_`\/`'**\/ __`\
  

•      \ \ \/\ \ _\ _ _\ _/\  __//\ \L\ \ \ \//\ \L\ \
  

•       \ _\ _\/_**_\/_**_\ _**_\ _**_ \ _\ ____/
  

•        \/_/\/_/\/____/\/____/\/____/\/**_L\ \/_/ \/_**/
  

•                                       /____/
  

•                                       _/__/
  

•  MIDI patch (for DIGMID) grabbing utility for the Allegro library.
  

•  By Shawn Hargreaves.
  

•  SoundFont loader based on code provided by George Foot.
  

•  See readme.txt for copyright information.
  

  /
  /

* (The above mentioned file readme.txt has the following about copyright:)

============ Copyright ============

Allegro is gift-ware. It was created by a number of people working in
cooperation, and is given to you freely as a gift. You may use, modify,
redistribute, and generally hack it about in any way you like, and you do
not have to give us anything in return. However, if you like this product
you are encouraged to thank us by making a return gift to the Allegro
community. This could be by writing an add-on package, providing a useful
bug report, making an improvement to the library, or perhaps just
releasing the sources of your program so that other people can learn from
them. If you redistribute parts of this code or make a game using it, it
would be nice if you mentioned Allegro somewhere in the credits, but you
are not required to do this. We trust you not to abuse our generosity.
*/

[chrisisonwildcode]

It does on so many levels ... I will look into this after everything seems happy for 0.4 ... maybe sneaking it in under 0.4.1-0.4.2

Sent from my iPhone

On 22 Jan 2015, at 6:40 am, Bret Curtis [email protected] wrote:

http://alsa.opensrc.org/Unsf
^-- this also talks about extended GUS format too.... interesting. :)

@chrisisonwildcode does this tickle your fancy ? ;)

—
Reply to this email directly or view it on GitHub.

[psi29a]

@chrisisonwildcode yeah.. i thought it would, we should really have a project plan :) we should abuse the milestone feature more to plan out our releases. we have feature creep in our 0.4 release ;)

[chrisisonwildcode]

looks innocently around the room

Sent from my iPhone

On 24 Jan 2015, at 2:59 am, Bret Curtis [email protected] wrote:

@chrisisonwildcode yeah.. i thought it would, we should really have a project plan :) we should abuse the milestone feature more to plan out our releases. we have feature creep in our 0.4 release ;)

—
Reply to this email directly or view it on GitHub.

[chrisisonwildcode]

You don't have to implement everything in -updates onto 0.4, It's why I have my own branch so you can worry about releases while I worry about features :)

Sent from my iPhone

On 24 Jan 2015, at 2:59 am, Bret Curtis [email protected] wrote:

@chrisisonwildcode yeah.. i thought it would, we should really have a project plan :) we should abuse the milestone feature more to plan out our releases. we have feature creep in our 0.4 release ;)

—
Reply to this email directly or view it on GitHub.

[psi29a]

hahaha :)

I think 0.4.2 is perfect for sf2 support.

[psi29a]

I stumbled across this recently:
https://csharpsynthproject.codeplex.com

It contains information on sf2 support and is MIT licensed... wooo!

[chrisisonwildcode]

Does that mean you don't want me to look at the sf2 -> gus converter you recently found?

Personally when I overhaul the sample file handling there are going to be changes (and some slowdowns) that will make it easier to add more sample formats :)

Sent from my iPhone

On 26 Feb 2015, at 7:18 pm, Bret Curtis [email protected] wrote:

I stumbled across this recently:
https://csharpsynthproject.codeplex.com

It contains information on sf2 support and is MIT licensed... wooo!

—
Reply to this email directly or view it on GitHub.

[psi29a]

honestly... (and this is what I would like to see happen), native support for sf2... no conversions.
We'll need to do the same for DLS and other formats that put their instruments in with the playing file (like trackers).

[kotc]

but please don't drop gus format support because most systems have freepats package available

[psi29a]

@kotc: we never said that, where did you get this idea and why does this have to do with a sf2 thread?

[kotc]

usually it's easier to just keep 1 format than multiple, one can also invent internal format that would suit all of them (which means some kind of conversion too)

[psi29a]

Creating temp files is out of the question. We'll not do that. Also think of the differences between patches and sf2 and other bank formats... converting to one or the other will lose a bit in translation. As for an internal format, that would require to either convert on the fly, which is additional CPU work, and/or explode memory use of WildMIDI to keep it cached. Bad ideas.

The best, quality, solution is to support reading from different bank formats and/or falling back to a CPU bound synthesized FM output.

[kotc]

by internal format i mean structures you have in memory to store samples and their parameters

[psi29a]

yes, i mentioned that... memory usage is at a premium, we'll not store patches in memory.

[kotc]

but you have to load sample when you load midi (not all, but the ones used in the song)

[psi29a]

we do a copy, render it, flush it to output, then release. since libwildmidi runs on low end hardware from SiS for midi jukeboxes, this is ideal

we do not store it in memory and keep it there.

[sezero]

By the way, is there any way to convert soundfont to patchsets ?

Google for the 'unsf' tool

[psi29a]

This has bubbled back up again, as a feature request for XLengine so that they can dump fluidsynth and use wildmidi for everything. This feature is apparently above (more urgent) than OPL emulation as there are many other SF2 files out there.

[chrisisonwildcode]

I can look into it, it's going to take a little time as this time of year is busy for me supporting others through tough times. This means personal time is limited and cherished.

Once I can grab a good sf2 and a good format spec I'll start nutting it out but no timeframe at this stage.

Chris

Sent from my iPhone

On 15 Jan 2016, at 6:56 PM, Bret Curtis [email protected] wrote:

This has bubbled back up again, as a feature request for XLengine so that they can dump fluidsynth and use wildmidi for everything. This feature is apparently above (more urgent) than OPL emulation as there are many other SF2 files out there.

—
Reply to this email directly or view it on GitHub.

[psi29a]

To quote:

FluidSynth has a pretty decent one, which I think is a good quality baseline (FluidR3_GM.sf2). It's 143MB, so a bit on the large side, but it sounds pretty good with most things I've thrown at it (it's sanely licensed too, part of most/all Linux distros). I also have one called SGM which I think is a bit better, although it's notably larger at 236MB uncompressed: http://www.mediafire.com/download/zo8l3dgf2989266/SGM-V2.01.7z (I'm also not sure what it's license is, I just remember finding it in a youtube video).

The main thing to take note of when adding sf2 support for WildMIDI, is properly implementing the modulators. IIRC, the GUS pat format that WildMIDI (and Timidity before it) are based on is that it doesn't really have a concept of modulators like sf2 has, but they can have a clear impact on the resulting sound. That's the reason why Timidity's sf2 support is considered rather poor, because it effectively just converts the sf2 to pat internally and uses it that way, ignoring all the extra bits that the pat format has no concept of but the sf2 is expecting.

[RyanBram]

For additional information.

If you want simple and small soundfont, I recommend using GMGSx.sf2. It is public domain soundfont with very small in size (about 3MB).

Download

[psi29a]

I'm going to look into this myself for 0.4.1

I will do the poor-man's route first then deal with modulators and other sf2 specific effects later.

[chrisisonwildcode]

Keep in mind that either the resamplers were designed for guspat. Ideally coming up with a single internal format that can handle different sample types would be great, otherwise we need resamplers for each sample type, something I would like to avoid in the interest of size and performance.

Sent from my iPhone

On 16 Jun 2016, at 9:25 PM, Bret Curtis [email protected] wrote:

I'm going to look into this myself for 0.4.1

I will do the poor-man's route first then deal with modulators and other sf2 specific effects later.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[psi29a]

I'm far from knowledgeable about these things, so I'll be starting from ground zero. Is this something you are still interested, if there was time available? ;)

Otherwise I'll focus my efforts in supporting additional formats as well as an 'off-line' hmp/hmi 2 mid function.

[psi29a]

Since we're not adding any additional formats at the moment, I'm looking at this again.

I've been working with unsf, it is public domain and I've cleaned up the bit-rot and hosting it here under the CC0 license here: https://github.com/psi29a/unsf

I'm currently in the process of creating a library and program split, removing globals and making things thread safe. Once this is finished, we can use this in WildMIDI to support SF2.

The plan is to check the wildmidi.cfg file, if the first thing it reads is the file path to a soundfont then we have unsf create a configuration file for us and extract/convert the SF into a GUS Patch on the fly for use by WildMIDI. This plays on our strong point, which is guspat.

We know that we don't honour all the features of GUS Patches, such as modulation (in another issue that is tracked), but with unsf there are extensions to the GUS Patches that we can use to improve the sound of the patches, including those converted from SF.

Now, if someone else wants to add support for SoundFonts another way, they are free to do so and send a PR. :) I believe there is more than one way to skin this cat.

[chrisisonwildcode]

I've spent a lot of time mulling over this as parts of the library will need rework to accommodate this.

I would like to basically accommodate this in a way that allows easier adding of other sample formats. This would probably look like separate:

• sample loading functions
• pitch adjust functions
• resampling functions

The main loop for events needs to be sample independent so probably the note data and pitch change events need to store a function pointer.

This also needs to be robust enough handle multiple formats at once and to easily switch sample formats on the fly when a patch change occurs, so that playback of a midi is not restricted to one sample type.

Thoughts...

Sent from my iPhone

On 20 Jun 2016, at 7:01 PM, Bret Curtis [email protected] wrote:

I'm far from knowledgeable about these things, so I'll be starting from ground zero. Is this something you are still interested, if there was time available? ;)

Otherwise I'll focus my efforts in supporting additional formats as well as an 'off-line' hmp/hmi 2 mid function.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[psi29a]

I would really want to see this happening as you describe. It would help when dealing with other formats as you said, with embedded samples (MMF and DLS). So please do, this will help in the long run. Is there anyway you can make a plan for work that can be split up and given out to people to work on? Or is this something you would like to do yourself?

In the short term, my approach would allow us to help ship unsf along with wildmidi so that others can use it to to convert sf2 to guspat for use with wildmidi which is immediately useful. With a small bit of work, we can automate the process so that it is seamless to the end-user when they provide a sf2 file for use in wildmidi.

I see both approaches dovetailing nicely together, no conflicts. What do you think?

[chrisisonwildcode]

I see that working fine in the short term. Some people will be after a sf2->gus option. Out of random curiosity does unsf do the reverse, gus->sf?

Give me a little time to mess around with the option I described as the main resampling loops will need to be reconfigured and I want to attempt to do it in a way that will be workable as we consider other formats.

Sent from my iPhone

On 10 Aug 2016, at 12:14 AM, Bret Curtis [email protected] wrote:

I would really want to see this happening as you describe. It would help when dealing with other formats as you said, with embedded samples (MMF and DLS). So please do, this will help in the long run. Is there anyway you can make a plan for work that can be split up and given out to people to work on? Or is this something you would like to do yourself?

In the short term, my approach would allow us to help ship unsf along with wildmidi so that others can use it to to convert sf2 to guspat for use with wildmidi which is immediately useful. With a small bit of work, we can automate the process so that it is seamless to the end-user when they provide a sf2 file for use in wildmidi.

I see both approaches dovetailing nicely together, no conflicts. What do you think?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[psi29a]

unsf at the moment just does sf2->gus, but the code can become two-directional with a bit of work. We could have unsf and sf2gus that both use the same library for example. So I see it useful to users of WildMIDI.

No worries Chris, I know you have a lot on your plate but I believe your method is the best way to go forward for supporting multiple formats 'correctly'. I'm guessing this is something for after looping is taking care of?

[kotc]

may i chime in? imo majority of soundbanks for midi (that are available to download) are in gus and sf2 formats, so i would like to see wildmidi able to support both (and as i've said at the top of the thread, currently i just use standalone unsf to convert sf2 to format known to wildmidi. on-the-fly loader for sf2 would be sweet). are there other significiant formats out there?

[psi29a]

@kotc there are formats that carry not only their MIDI component but also their samples used for playback such as MMF and DLS

#9
#93
#118

[chrisisonwildcode]

cough_mod files_cough

Sent from my iPhone

On 11 Aug 2016, at 4:42 PM, Bret Curtis [email protected] wrote:

@kotc there are formats that carry not only their MIDI component but also their samples used for playback such as MMF and DLS

#9
#93
#118

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[psi29a]

Not to demotivate anyone, but most mod files are handled by adplug/adplay anyway.
https://adplug.github.io/ <-- supported formats

[psi29a]

Yesterday I finished up splitting up unsf into library and application parts. It isn't perfect, but it works and gdb is happy. Valgrind is finding the same errors as before, so at least we are status quo there.

Next steps:

• solve valgrind's issues
• convert to cmake
• add support for building on osx and windows
• use contexts to handle multi-thread sceneroes
• import into wildmidi and make it LGPL as a result. (Original is still hosted as CC0).

I thought about adding this as a sub-module, but past experience has shown that git submodule is more headache than not.

[chrisisonwildcode]

Keep your up the good work.

Sent from my iPhone

On 16 Aug 2016, at 8:34 PM, Bret Curtis [email protected] wrote:

Yesterday I finished up splitting up unsf into library and application parts. It isn't perfect, but it works and gdb is happy. Valgrind is finding the same errors as before, so at least we are status quo there.

Next steps:

solve valgrind's issues
convert to cmake
add support for building on osx and windows
use contexts to handle multi-thread sceneroes
import into wildmidi and make it LGPL as a result. (Original is still hosted as CC0).
I thought about adding this as a sub-module, but past experience has shown that git submodule is more headache than not.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[RyanBram]

Thanks for your hardwork.
It makes SF2 support become closer.
If you don't mind, is it possible to keep the resulting SF2 library as CC0 if separated from the rest of WildMidi?
I want to see my favorite tracker to have better support when handling SF2 format. Recently SAGA MUSIX has improved MIDI Files support in OpenMPT, and I hope improvement of SF2 support will come next. The World of MIDI and Module music will become closer.

https://forum.openmpt.org/index.php?topic=4703.msg43413#msg43413

[psi29a]

As I posted earlier, unsf will remain CC0.

The real/native sf library that chris is working on will be LGPL. It can be linked against other projects without a problem.

[psi29a]

Update: my unsf to library branch has been merged into master. It still needs a bit of work to clean up the interfaces to easier to use for a developer, but it works! I even found a potentially dangerous bug from the original code that was fixed. It was reading at least 2 bytes after a malloced area, this can lead to bad things of course.

So in theory, we can begin with integrating it into WildMIDI.

I'm thinking about grafting the two repos together, but that would of course require people to do dark magic with their repos. I don't want to lose the commit history. Another option is submodule.

[sezero]

Remember that timidity++ has support for soundfonts in timidity.cfg. If wildmidi wants to continue supporting timidity.cfg format, it needs to replicate that for soundfont support too.

[RyanBram]

Is it possible to use https://github.com/schellingb/TinySoundFont for providing SF2 support in WildMidi?

[psi29a]

That's pretty awesome, thanks for that.

I'll see what is possible.

[a8underscore]

how is the project going

[psi29a]

As much as I would like to work on this, my plate has been full lately. I hope that anyone else that is interested will throw up a PR.