Audio about foster HOT 17 OPEN

Apologies, I haven't updated this thread with the latest.

I made a miniaudio wrapper to be used with Foster with the following:

• Automated builds for Linux/Mac/Windows
• Supported features
  • Loading WAV, MP3, FLAC, QOA, OGG, and raw PCM data
  • Essential operations/settings: Play, Pause, Stop, Seek, Volume, Pitch, Pan, Looping, Spatialization
  • Sound groups to manage multiple sound instances (useful for sound category volume management)
  • Garbage free managed sound instances

There's still a bit more I want to do with effects and writing audio to file (likely breaking changes), but it's pretty well rounded for now. Also included is a small music visualizer example:

https://github.com/MrBrixican/Foster.Audio

from foster.

miniaudio is more mature than I remember it being. It's high level (engine) api seems like it could be a good fit. Seems like the only component that's a bit iffy is having to init a sound per instance and then deiniting once it's no longer in use (what defines that would be up to us). Also not sure about ogg support.

from foster.

Yeah I definitely want some form of Audio support & agree with everything you've written up here. The main reason I don't have it yet is I 1) know very little about how the internals work and 2) work with audio teams that usually want to use existing proprietary tools like FMOD.

But that said I do want something simple and open source that can be used for small games. I am definitely on the same page regarding note 2 - Trying to implement a large scale generic system is not something I want to try to maintain, especially when people will likely just not use it anyway if they need more advanced features.

So that said, I do feel like a wrapper around cute_sound or miniaudio makes sense, similar to how the platform API currently wraps stb_image/stb_truetype. I also agree it should be in its own namespace, but probably still exist in the Framework folder (so Foster.Framework.Audio, like you suggested).

If you're interesting in taking a swing at this I'd be more than happy to give feedback and merge that in once ready. I'm not super sure whether cute_sound or miniaudio makes more sense, although I do agree I like both of these over soloud just due to their single-file nature.

from foster.

Oh also it might make sense to let App.Run take a flags parameter, for various initialization flags such as disabling audio. I'm not sure what else it would have yet, though.

from foster.

Sounds good, I'll work on this later this week. I'll probably do an initial implementation with cute_sound just to get something quickly working, but I have a feeling that miniaudio will be better featured if I can get instancing implemented correctly.

from foster.

Sounds good, I'll work on this later this week. I'll probably do an initial implementation with cute_sound just to get something quickly working, but I have a feeling that miniaudio will be better featured if I can get instancing implemented correctly.

I would go with miniaudio, as it offers more stuff than cute_audio, or FAudio as it integrates greatly with SDL

from foster.

I agree with you on miniaudio, it would probably be the ideal long term solution. However, FAudio retains a lot of bloat in order to maintain full XNA compatability (not saying that's bad but not the same requirements we have).

from foster.

Yeah I am in favor of miniaudio I think. I also feel like FAudio is a great project but it falls into the "larger audio" systems for me, and I feel like I want to try to keep the core of this framework as light as possible. If someone wants to use that instead they can just disable our default audio implementation.

from foster.

Sounds good, at this point I would suggest to turn off unused subsystems from SDL (like audio, renderer and probably others) this way resulting library is smaller

from foster.

That's a good idea! I can probably control those with CMake compile flags somehow.

from foster.

I have been messing around with miniaudio and must say, it's a fantastic library. It should be pretty trivial to create some sound management Platform methods that are very thin layers over what is already built in.

I do have a couple of thoughts I'd like to cover before I begin my implementation:

Sound Instancing

miniaudio has a very minimal implementation of sound instancing. For the most part, it is very similar to OpenAL in that you must manually create and manage sounds (equivalent to OpenAL sources). In order to play the same audio multiple times concurrently, you would need to instantiate multiple sounds. Fire and forget audio would obviously require a layer that sits above miniaudio. Additionally, by default the resource manager will deallocate resources based on reference counts, so if you uninit all sounds pointing to an audio file, it will unload the data and further dispatches of the same audio would incur a file load.

Manual sound management is an unfortunate burden to place on the user, who should probably not be required to have knowledge of these inner workings. Thus, I will create a set of classes/structs that will provide a further abstraction for audio, similar to how Batch abstracts rendering.

Mainly, it will allow for loading an audio file one time, and then being able to create sound instances that can optionally be manipulated. An instance will automatically be disposed of behind the scenes when it stops playing.

Scope

I will aim to support basic sound features:

• play
• pause
• stop
• volume
• pitch
• pan
• current position / play time
• length
• looping

I may add spatialization, fading, and scheduled stop/start seeing as they're already implemented in miniaudio.

I will not be adding:

• Effects
  • The node based effect system in miniaudio is pretty complex and this would add a significant amount of complexity to an already complex first pass
• Microphone support
  • Most games don't use mic input, so this is not a priority
• Directly writing pcm data to output device
  • Again, most games don't require this functionality

Misc

The docs and issues made note of potential issues with ogg vorbis files (specifically get length methods), however I didn't notice any issues at all (possible my test files are exceptions).

There are some intricacies around the config settings you can use for the built in resource manager and the effect that has on runtime memory/cpu usage. However, those are easily configured so I'll leave that for the final pass.

If you have any thoughts you'd like to share, please chime in.

from foster.

Hey this all sounds super good to me, thanks for working on this! I'm on the same page regarding implementation and features.

Various thoughts:

• I was thinking it might be worth making it possible to exclude the audio stuff from the native lib under an optional flag in case you want to run this on a weird platform that miniaudio doesn't support, but it's probably not necessary. I assume miniaudio just won't do anything on platforms it can't handle.
• I am also definitely in favor of handling the memory/instancing internally. I think the use case of this API should definitely favor simplicity and easy of use. If people want more complicated audio they can use a big external library.
• One consideration, depending on how this is set up, is trying to avoid GC allocations when playing new sound instances. If possible I could see the API working more like you get a struct handle back when playing a new instance, instead of allocating a class object. I'm not sure how realistic this is, but it would be nice to avoid allocating C# objects every time the user plays a sound.

from foster.

Regarding unsupported platforms, miniaudio defaults to a null backend which simply does nothing, as you say. Optionally, if it's a really weird platform miniaudio can be configured to back onto sdl. That should really only be consoles though, since miniaudio supports emscripten, desktop, and mobile platforms out of the box.

I 100% agree on the GC allocations, struct handles were what I had in mind, as it was something that bothered me with the XNA implementation. There will be plenty of unavoidable c allocations, but those will not touch the .NET GC.

from foster.

Yeah, that makes sense to me! Sounds great.

from foster.

Hi, I created a very basic audio c# wrapper for miniaudio to be included in my foster game, some things still to be implemented/improved like instance management and adding more features but I guess this is more or less the approach you were discussing about using handles isn't it?
https://github.com/Flip120/miniaudio_csharp/tree/main

Here the platform part:
https://github.com/Flip120/miniaudio_csharp/blob/main/platform/audio_api.h
https://github.com/Flip120/miniaudio_csharp/blob/main/platform/audio_api.c

The c# wrapper:
https://github.com/Flip120/miniaudio_csharp/blob/main/C_Lib_Test/AudioPlatform.cs

And a little test program:
https://github.com/Flip120/miniaudio_csharp/blob/main/C_Lib_Test/Program.cs

from foster.

Hm, while I like the idea for built-in audio interface, mayby we should consider opaque interface of sorts? I agree with Noel on the fact that we should keep this framework lightweight, and audio is rather controversial topic with lots of different opinions.

In case of opaque audio, Foster could provide an Audio class for use in third party extensions to provide swappable playback engines to the end users.

from foster.

The problem is that even playback is opinionated. For example, FMOD uses the concept of events, which can actually be a collection of sounds and effects that can organized in a timeline based fashion. Whereas miniaudio and other low level audio libraries work more on the concept that a sound is just one specific sound being played at a specific time.

Due to how opinionated Audio can be, the most Foster can do to abstract Audio is provide life cycle hooks (Startup, Update, Shutdown) which it already does through Modules.

After talking with Noel privately, the intent is for my wrapper to be separate (not built in) from Foster Framework as an option to those who don't have more complex audio use cases.

from foster.