Sound fonts vs .wav or modern libraries ?

BradHowes

@wim

At present, SoundFonts app relies entirely on Apple's AVAudioUnitSampler for generating sounds from an SF2 source. I don't know for sure that they don't just load everything at once, but knowing them and their focus on minimizing memory and cpu processing footprints everywhere on iOS, I would guess that they only load what is necessary for playing a particular preset.

When the SoundFonts app first sees a new SF2 file, it reads it all to make sure it is has a valid RIFF structure and to sanitize the presets it contains, but that is about it. After that, all SF2 loading and playback is done by AVAudioUnitSampler.

Also, regarding playback of SF2. The SF2 files not only have the samples, but they also have a set of ~50 parameters (known as generators) that affect the playback of the sound. There are two envelopes with 6 segments each (delay, attack, hold, decay, sustain, release), two LFO that can affect pitch (vibrato) and envelopes, a low-pass filter (primarily used to keep high-frequency artifacts from aliasing into the audio spectrum), and two effect channels (chorus and delay) and settings for them and their mixing levels. An SF2 preset (aka patch) consists of a collection preset zones which can have adjustments to pretty much any of these 50 or so parameters, and a zone is defined by MIDI key note and MIDI velocity. Each zone points to an SF2 instrument which is more or less the same as a preset except that the parameter values in their zone definitions are absolute values, while the preset zone values are additions or adjustments. Instruments then point to sample configurations which specify where the first and last samples are in the file for the sound it makes, and if there is a looping point to be used to generate sound indefinitely as long as a note is playing (and the envelopes allow it) The result being that an SF2 sound generation system can become quite elaborate with many zones at both the instrument and preset level (even multiple zones matching a particular key/velocity pair, so one could conceivably have more than one sample playing for any key/velocity pair). But for any preset, it will only use the samplers that are attached to the instruments that the preset zones use.

My own code base at https://github.com/bradhowes/SoundFonts now has some of the pieces necessary to do its own sound generation without relying on Apple's AVAudioUnitSampler. Most of the SF*.hpp files pertain to SF2 file format and the SF2 spec. Again, it is not being used right now, only the Parser which quickly validates the RIFF integrity of the file.

Brad

wim

Thanks so much for that very generous and helpful answer. I took a brief dip into the source code earlier this week. It’s such a great gift to anyone trying to get into iOS app programming as I am.

I suspect the memory footprint taken by a given sound font does depend on the app implementation. Hopefully Apple has been efficient with AVAudioUnitSampler. I suspect they have since they do have an instance method which can target individual patches.

👍 @BradHowes

rs2000

Thanks @BradHowes!

BradHowes

Also if you are curious about how all of this stuff works under the hood there are some other code bases to ponder. One of the best of course is the AudioKit repo. They don't use SF2 or support it directly, but their sample synth engine works in a very similar fashion as I described above. If you are more of a C fan, then FluidSynth could be interesting. They have a very performant SF2 engine which is probably the most faithful to the SF2 spec. It is under the GPL license though. Finally, there is the material at basicsynth by Daniel R. Mitchell which is just great. And his book is very accessible and an excellent source of information on a variety of topics including various synthesizing techniques, filtering, and effects. Oh, and speaking of books, "Designing Software Synthesizer Plugins in C++" by Will C. Pirkle is an excellent (though pricey if new) addition to your bookshelf. There is also source available for it but I don't have the link handy.

Oh, and if you are really into this kind of stuff: https://ccrma.stanford.edu/links

Regards,

Brad

wim

Thanks again 😎

The real struggle for me right now is with the basics of getting audio units up and running. I wish there were as many resources for that basic foundation as there are for the advanced stuff.

(Apologies to the OP for hijacing the thread. )

BradHowes

@wim said:
The real struggle for me right now is with the basics of getting audio units up and running. I wish there were as many resources for that basic foundation as there are for the advanced stuff.

I agree. There is a lot of old info about AudioUnits, and much less accurate info on the AUv3 stuff. My repo https://github.com/bradhowes/LPF is my attempt to have working code that shows how to do this correctly (based on Apple's code but heavily modified).

rs2000

Great work @BradHowes 👍🏼

noob

yaman @BradHowes your app is a gem

ahallam

@noob said:
The Line - yes.

I assumed something like this

Fingolfinzzz

@BradHowes I really appreciate the knowledge, that was super informative and makes a lot of sense. I’ll be going down a rabbit hole of research now

Reuben

I am the author of the Soundfonts 4U site https://sites.google.com/site/soundfonts4u/
Assuming you have access to a PC, its actually petty easy to manage your soundfonts size using a free program called Polyphone. With it you can extract or combine the sounds you need and toward the bottom-right of my site I've included instructions on how to do it. If you need help send me a message.

McD

@Reuben said:
I am the author of the Soundfonts 4U site https://sites.google.com/site/soundfonts4u/
Assuming you have access to a PC, its actually petty easy to manage your soundfonts size using a free program called Polyphone. With it you can extact or combine the sounds you need and toward the bottom-right of my site I've included instructions on how to do it. If you need help send me a message.

Good point. In the history of sound fonts they were typically included 128 instruments in a "General MIDI" standard set of instruments. But a soundfont can contain as few as 3 instruments or less and provide a savings in storage and RAM consumption. But someone has to make the "envelope" holding the instruments and Polyphone does that.

Apple's ESX Sampler in Logic and MainStage makes ESX24 packages which is Apple's take on the concept of a SoundFont targeted to run in that specific sampler but also supported in a few other places like the AudioLayer app.

Telstar5

@Reuben said:
I am the author of the Soundfonts 4U site https://sites.google.com/site/soundfonts4u/
Assuming you have access to a PC, its actually petty easy to manage your soundfonts size using a free program called Polyphone. With it you can extract or combine the sounds you need and toward the bottom-right of my site I've included instructions on how to do it. If you need help send me a message.

Is there a “poly phone “ equivalent on iOS ? I guess Bismarck 16?

uncledave

@Telstar5 said:

@Reuben said:
I am the author of the Soundfonts 4U site https://sites.google.com/site/soundfonts4u/
Assuming you have access to a PC, its actually petty easy to manage your soundfonts size using a free program called Polyphone. With it you can extract or combine the sounds you need and toward the bottom-right of my site I've included instructions on how to do it. If you need help send me a message.

Is there a “poly phone “ equivalent on iOS ? I guess Bismarck 16?

Bismark 16 is just a multitimbral soundfont player. The referenced Polyphone program is a soundfont file editor, not the same at all. There are other iOS soundfont players, notably SoundFonts and KQ Sampei.

Telstar5

@uncledave :Oh, ok.. Thanks for that .. I didn’t know . So once I get some sound fonts, either of those would be my next stop?

uncledave

@Telstar5 said:
@uncledave :Oh, ok.. Thanks for that .. I didn’t know . So once I get some sound fonts, either of those would be my next stop?

Yes. They all have at least one sf2 included, usually a GM one with 128 instruments, so you can get started with just the app. Soundfonts 4U (referenced above) is a really good collection of mostly recently-created soundfonts.

Edit: Btw, some DAWs include a soundfont "instrument", don't know which. So you may already have one.

Telstar5

Thanks again @uncledave : Yes Audio Evolution a d multi track studio have them. Digital Sound Factory sells a bunch of them, a bit pricey but we’ll organized .

uncledave

@Telstar5 said:
Thanks again @uncledave : Yes Audio Evolution a d multi track studio have them. Digital Sound Factory sells a bunch of them, a bit pricey but we’ll organized .

You should def check out Soundfonts 4U, all free. Includes some remarkable pianos, lots more.