Features pt. 2

[MarkKremer]

Time to have a good look at what Beep can an can't do.

Here are some old issues with desired features:

• Audio faiface/pixel#32
• Features #115

I haven't read them completely yet, but I'll update the current issue with wishes I find in them I think are important.

Below are a couple of scenario's I expect Beep to be used in. I've listed a bunch of requirements for each. Some of these requirements are my idea of what is probably needed, although I don't necessarily have experience with them so feedback is welcome. Other points are requests I found in issues. I'll try to link those issues when I find them. However, I don't necessarily place them within the scenario the original issue was about.

This issue will be updated with more requirements as I spit through all the old issues.

Game

• Background music
  • Tracks can be long and should not be stored in memory. They can be streamed from the drive.
  • Different songs become louder depending on the area you're in.
  • For a level-based game, songs are (cross)-faded between levels.
  • 1-3 tracks are looped.
  • Stereo or more channels.
• Event based sounds
  • Short audio clips that must be played as soon as possible when you make a movement, walk on grass, an enemy shoots, someone got hit, some other event occurs.
  • Single channel audio source.
  • The in-game position is encoded in the sound to make it multichannel.
• Environment based
  • Similar to event based sounds
  • Medium-long loops of audio for objects in the environment like the sound of water nearby a river.
  • Fade-out together with the background sound when going to another level.
• Accurate events/callbacks. E.g. when you have a track with dialog and something in the world must happen at an exact time in sync with the audio.

Music player

• Plays long songs in different file formats.
• Playback position can be changed / seeked.
• Can go to any other long in the playlist
  • Implement Forward/Backward for Seq #16
• The songs on the playlist can be modified on the fly.
  • How To Cross-fade #112
• Crossfading between songs.
  • How To Cross-fade #112
  • https://github.com/Programmerino/beepFade
• It is known when a song ends and a new one begins, e.g. when you want to print the song name that's currently playing.
  • Feature Request: Be able to cancel a Seq #101
• Surround sound
  • Support 5.1 Vorbis files #18
  • Support variable number of channels #9
• Accurate playback position information. Having a buffer in between the audio source and the speaker makes this more difficult.

BeepBox

Something like BeepBox.

• Programatically created sounds (sine waves etc.)
• Sounds are scheduled at specific times.

Audio streaming over the network

I saw some issues dealing with this and I don't currently know enough which formats could properly support this. It would be nice to have a second look at this at some point to have Beep support this the proper way.

Beep shouldn't have to deal with sockets. But it should be able to produce an theoretically infinite bytestream using the format's encode function and decode it on the other end.

• Be able to encode a streamable format.
  • wav file header is fully inconsistent before seek fix #38

---

Everything above should work well. No stuttering audio, all realizable with out-of-the-box components.

For this, we need to re-imagine the components we need in Beep. A lot can be done with Beep, but I'm struggling with how some of the above requirements could be fulfilled using only the current composites. Two things are needed:

• More components. A birds eye view should be taken to choose a set of components that work well together without creating a thousand ways to do the same thing.
• Recipes in the wiki to explain how to build certain structures optimally.

I wrote this issue and then @dusk125 said that he appreciated the easy low level access to the data in Beep. I think that should be kept in mind to not overdo it when implementing these things.

[MarkKremer]

Two ideas for composites I have floating in my mind:

Event listener

This is similar to a callback, but it is wrapped around a streamer so information about the current position is known and can be hooked into.

streamer, format, err := vorbis.Decode(f)
if err != nil {
	log.Fatal(err)
}
defer streamer.Close()

watchStreamer := beep.Watch(streamer)
watchStreamer.Started(func () {
	fmt.Println("Started")
})
watchStreamer.At(format.SampleRate.N(time.Second * 10), func () {
	fmt.Println("Passed the 10 second mark.")
})
watchStreamer.Ended(func () {
	fmt.Println("Finished")
})

speaker.Play(watchStreamer)

This doesn't yet make it easy to get accurate time information when speaker is buffered.

Scheduler

scheduler := NewScheduler()
scheduler.Now(streamerA)

speaker.Play(scheduler)

time.Sleep(time.Second * 2)

scheduler.StartAt(scheduler.Position() + format.SampleRate.N(time.Second * 5), streamerB) // played after 5 seconds from now

scheduler.Append(streamerC) // played after both streamerA and streamerB are finished.

time.Sleep(time.Second * 10)

scheduler.Now(streamerD) // played ASAP, will be mixed together with sounds that are  playing already.

The scheduler could also return a handle to cancel certain sounds:

handle := scheduler.Now(streamer)

// some time later

handle.Abort()

The scheduler and event listener can be combined to schedule a stream, and then get events on when it actually started.

[MarkKremer]

Swapper

// Play streamerA for 10 seconds, then play streamerB.
swapper := NewSwapper(streamerA)

speaker.Play(swapper)

time.Sleep(time.Second * 10)

swapper.Swap(streamerB)

Alternatively, this could be part of the Mixer.