Playing sound

[Jordan Johnson]

Hi all,

I'm looking at audio again because I have a student doing a game project for which he wants background music to play — and I'm feeling some trepidation based on most of the audio-tagged packages on pkgs.racket-lang.org showing some test failures or other build problems.

His minimal goal is to loop a sound file from the start of play until the end, and have it work on Windows and Mac. We tried play-sound from racket/gui/base, and it almost fits: we just need to be able to kill the sound at the end, and it seems that killing the sound-playing thread is not enough. Is there a way to stop the audio?

His ideal goal is to be able to play different background music depending on the room the player is in. AFAICT this is beyond play-sound's ability, at least if he wants to do smooth transitions. Is there another library which would facilitate cross-fading from one audio file to another?

More generally, what are currently our best tools for incorporating music and sound effects into Racket games?

Best,

Jordan

[Lehi Toskin]

This may be larger than the scope of the project, but there is an audio library called RSound, which uses PortAudio (via a wrapper). It's more robust and I'm certain there are things you could do with it that you couldn't with play-sound.

[John Clements]

Hi there! Author of rsound here.

If you’re concerned about test failures in pkg.racket-lang.org… well, my test cases try to load the shared libraries and play sound, so that’s not possible in pkg.racket-lang.org’s environment. I suppose I should refactor those tests so that only the ones that can run without portaudio get run.

RSound can certainly give you the ability to start, stop, and cross-fade sounds.

I’m guessing you’re looking for an imperative API that looks something like this:

(start-playing! stream sound at-time #:fade-in time-duration) => token
(stop-playing! stream token at-time #:fade-out time-duration)


… right?

John

[Jordan Johnson]

Hi John,

Thanks for the reply.

> If you’re concerned about test failures in pkg.racket-lang.org… well, my test cases try to load the shared libraries and play sound, so that’s not possible in pkg.racket-lang.org’s environment.

Oh, good. That's less of a worry then.

Will the separate shared libraries likely cause any extra difficulties when he bundles his code up into Win/Mac executables?

> I’m guessing you’re looking for an imperative API that looks something like this:
>
> (start-playing! stream sound at-time #:fade-in time-duration) => token
> (stop-playing! stream token at-time #:fade-out time-duration)

Yep, that should do what he wants.


Best,
Jordan

[Jordan Johnson]

Hey, so I’m now looking at how to set up background music in rsound with the ability to switch tracks. Below is the suggestion from John Clements that seems right:

I’m guessing you’re looking for an imperative API that looks something like this:

(start-playing! stream sound at-time #:fade-in time-duration) => token
(stop-playing! stream token at-time #:fade-out time-duration)

AFAICT this isn’t already in existence in the rsound library. My impression is that we will need to build this on top of signals and networks — but I want to check that, before we pour the time into doing it.

I’m thinking signals & networks because we tried to build it using pstreams, and ran into a snag when trying to stop playback and switch to a new track:

;; Given a pstream bgmusic and rsounds bgsong1, bgsong2, the following Interactions

;; occur:

> (pstream-play bgmusic bgsong1)

"sound is queued"

> (stop)

> (pstream-play bgmusic bgsong2)

"sound is queued"                ;; BUT NO SOUND PLAYS.

> 

;; - end interactions -

So, am I correct in thinking that pstreams aren’t sufficient for what I’m attempting? Is there another way to stop playback, other than calling (stop)?

Thanks,

Jordan

[John Clements]

You are correct. with one tiny caveat.

1) The “stop” procedure is a giant end-of-the-world hammer; I wouldn’t expect it to be useful while a program is running.
2) I hesitate to suggest this, but there is a “pstream-set-volume!” function that can silence a pstream. (that was the caveat.)
3) You are also correct in your supposition that the interface I proposed is not implemented. It looks like it could be done quickly, meaning probably on the order of 10-20 hours of work for me. The relevant code to look at here would probably be

https://github.com/jbclements/RSound/blob/master/rsound/sequencer.rkt

In fact, the pstream library (in stream-play.rkt) is just a thin skin over the sequencer code.

Adding the stop-playing! call would probably involve adding some king of message queue to the sequencer to receive fade-out messages, and updating the ‘entry’ structure to contain information about sounds that are currently fading in or out.

John

>
> Thanks,
> Jordan
>

[Jordan Johnson]

On Apr 6, 2016, at 15:18, John Clements <clem...@brinckerhoff.org> wrote:

1) The “stop” procedure is a giant end-of-the-world hammer; I wouldn’t expect it to be useful while a program is running.

OK. It had that feel.

2) I hesitate to suggest this, but there is a “pstream-set-volume!” function that can silence a pstream. (that was the caveat.)

I think this points to a hackish solution that’ll work for us for now:

https://gist.github.com/RenaissanceBug/da7027d8c55a1819f9bae274e189ccac

The only concern I have thought of so far is whether this’ll cause memory leakage over the course of a number of switches between pstreams; I don’t know if there’s anything that will keep the GC from cleaning up the pstream when it becomes unreachable.

3) You are also correct in your supposition that the interface I proposed is not implemented. It looks like it could be done quickly, meaning probably on the order of 10-20 hours of work for me.

I’m glad I asked rather than just dive in trying to figure it out in the middle of a busy semester.

The relevant code to look at here would probably be
https://github.com/jbclements/RSound/blob/master/rsound/sequencer.rkt

[...]

Adding the stop-playing! call would probably involve adding some king of message queue to the sequencer to receive fade-out messages, and updating the ‘entry’ structure to contain information about sounds that are currently fading in or out.

I’ll look into it at some point, time permitting. But I think the gist above will obviate my need for it, at least for the time being.

Thanks again!

Best,

jmj

[John Clements]

> On Apr 6, 2016, at 4:31 PM, Jordan Johnson <j...@fellowhuman.com> wrote:
>
> On Apr 6, 2016, at 15:18, John Clements <clem...@brinckerhoff.org> wrote:
>> 1) The “stop” procedure is a giant end-of-the-world hammer; I wouldn’t expect it to be useful while a program is running.
>
> OK. It had that feel.
>
>> 2) I hesitate to suggest this, but there is a “pstream-set-volume!” function that can silence a pstream. (that was the caveat.)
>
> I think this points to a hackish solution that’ll work for us for now:
> https://gist.github.com/RenaissanceBug/da7027d8c55a1819f9bae274e189ccac
> The only concern I have thought of so far is whether this’ll cause memory leakage over the course of a number of switches between pstreams; I don’t know if there’s anything that will keep the GC from cleaning up the pstream when it becomes unreachable.

Yes, I would expect it to be a problem. I wouldn’t worry so much about memory leakage as I would about the sound system gradually grinding to a halt. An ongoing pstream will continue to generate data, and sequencer.rkt will continue to compute it, multiply it all by zero, and then add it to the out-going stream.

>
>> 3) You are also correct in your supposition that the interface I proposed is not implemented. It looks like it could be done quickly, meaning probably on the order of 10-20 hours of work for me.
>
> I’m glad I asked rather than just dive in trying to figure it out in the middle of a busy semester.
>
>> The relevant code to look at here would probably be
>> https://github.com/jbclements/RSound/blob/master/rsound/sequencer.rkt
>> [...]
>> Adding the stop-playing! call would probably involve adding some king of message queue to the sequencer to receive fade-out messages, and updating the ‘entry’ structure to contain information about sounds that are currently fading in or out.
>
> I’ll look into it at some point, time permitting. But I think the gist above will obviate my need for it, at least for the time being.

Let me know how it goes.

Best,

John Clements