Sound scrubbing with rsound

[John Clements]

To Advait Raykar:

In response to a Slack thread about a scrubbing interface for sound playback:

Looks like you folks covered all the bases on rsound. It sounds like the "right" interface for this would be something that plays a file from start to finish but also allows you to asynchronously alter the playback point.

There are several ways to do this. There’s a way that uses the existing interface, and there’s a lower-level way. Hmm… actually, there are multiple ways, for multiple definitions of “lower-level”.

I think the right way to go here is with the “pstream” interface; it should allow you to queue sounds to be played, and to then change your mind and schedule different sounds to be played (as e.g. when the user moves a scroll bar). In fact, I have a pretty distinct recollection that students have implemented this using big-bang, and it works pretty well. Let me look …

Yes, in fact, there’s an example file called “katy-sliders” that’s a part of the rsound package. which does something similar. Specifically, it uses a slider to control the speed of playback, allowing both forward and backward play. This is a whole lot more computationally intensive than a scrubbing interface such as the one you describe, so this essentially constitutes a proof-of-possibility. This example actually uses the low-level “signal” interface, which is required because you’re computing the samples as you go, rather than just queueing existing sounds.

I should mention that there was one small issue in the “katy-sliders” example, calling 2htdp/image’s “text” function with a third argument of “solid”. I conjecture that this used to work, but that the interface has probably changed slightly (though the change might just be better enforcement of an existing contract). I’ve pushed an update to this, but you can make the change easily if you try running the file. Also, the file depends has a hard-coded dependency on a “/tmp/partygoing.wav” file, but you should be able to update this and point it to any wav you like.

I should also note that it looks like low-level manipulations of signed 16-bit numbers is either different or missing in CS racket, so for the moment, this code doesn’t run correctly in Racket CS.

Let me finish by suggesting a sequence of actions:

1) Convert your mp3 to a wav file.
2) Try the rsound/examples/katy-sliders.rkt file, to convince yourself that something like this is possible.
3) Write your own program using the pstream interface to implement a scrubber.

Let me know if you run into problems!

John Clements