HeadSpace: an HRTF library in PYO

[Christopher Rabl]

Hello fellow PYO-ers!

My colleague Curtis Litchfield and I have been working on a project
this semester at the University of Lethbridge, and we wish to share
the results with you. Using the PYO library, we created a simple
library that allows you to position an audio source anywhere in a 3D
space around the head. We used the HRTF impulse response dataset
created by Gardner and Martin at the MIT Media Lab to create the
illusion. When we made the decision to use PYO for our implementation,
we were looking for a DSP library that was simple, fast, and above
all, open-source. We are very thankful for all of you who have put
such a great amount of effort into creating this awesome library. As
well, we are firm believers in open-source and its ability to impact
the way that software is developed and consumed. As a result, we are
contributing our project to the open-source community so that we can
develop it further, and make it comparable to commercial HRTF plugins.

You can check the project out on Github if you are interested:
https://github.com/crabl/HeadSpace
It will run on Windows, Linux, and OS X, so feel free to mess around
with it!

Again, a big thank you to Olivier and everyone else who contributed
their time and effort into creating this fantastic library! Merci
beaucoup!

Chris

[Drew Harris]

This is awesome! Cannot wait to try it, was looking for something like this for a project.

I see it's using a UI, is there any way I can run without a UI?

Also, are there architecture dependencies? Would love to get this going on ARM.

[Christopher Rabl]

Absolutely, the UI is just there to allow people to test the library
without defining their own UI. All you need to do is create a new
SfPlayer with your sound, create a HeadSpace object by passing in your
player, azimuth, and elevation, and you're off to the races. You can
set the azimuth and elevation in real time by using the setAzimuth and
setElevation methods.

HS should have no other architectural dependencies. If PYO runs, it
should run.

April 29, 11:23 am, Drew Harris <drewbhar...@gmail.com> wrote:
> This is awesome! Cannot wait to try it, was looking for something like this for a project.
>
> I see it's using a UI, is there any way I can run without a UI?
>
> Also, are there architecture dependencies? Would love to get this going on ARM.
>

[Olivier Bélanger]

Wow, nice work! Thanks for sharing and for choosing pyo as your development platform. Just tried it here on a WindowsXP machine and it runs perfectly!

One thing that could be very nice is to use PyoObject as the parent class for HeadSpace. Doing so will allow to use your object in the same way as any other PyoObject (audio input to other objects, crossfading when changing the input source, mul and add attributes, play(), stop(), out(), mix(), ctrl(), etc.). I'll try something as soon as I can and send it back to you to see if you like it!

Thanks again!

Olivier

2012/4/29 Christopher Rabl <rabl.chr...@gmail.com>

[Christopher Rabl]

Very cool :-) Yeah, I just started playing around with PYO a few weeks
ago since we already had a MAX/MSP patch to do this same process. I
haven't really experimented with the PyoObject class (in the name of
simplicity) but I'd like to see how we can make use of it. The more
extensible this tool is, the better!

One of the biggest bottlenecks we found is reading in the impulse
responses every time we want to insert the next response into our
SndTable objects. Would be nice to be able to load all the impulse
responses into memory and perform the insertion that way, rather than
having to call SndTable.insert() with a new file each time.

On Apr 29, 4:38 pm, Olivier Bélanger <belan...@gmail.com> wrote:
> Wow, nice work! Thanks for sharing and for choosing pyo as your development
> platform. Just tried it here on a WindowsXP machine and it runs perfectly!
>
> One thing that could be very nice is to use PyoObject as the parent class
> for HeadSpace. Doing so will allow to use your object in the same way as
> any other PyoObject (audio input to other objects, crossfading when
> changing the input source, mul and add attributes, play(), stop(), out(),
> mix(), ctrl(), etc.). I'll try something as soon as I can and send it back
> to you to see if you like it!
>
> Thanks again!
>
> Olivier
>

> 2012/4/29 Christopher Rabl <rabl.christop...@gmail.com>

[Olivier Bélanger]

Hi Christopher,

2012/4/29 Christopher Rabl <rabl.chr...@gmail.com>

Very cool :-) Yeah, I just started playing around with PYO a few weeks
ago since we already had a MAX/MSP patch to do this same process. I
haven't really experimented with the PyoObject class (in the name of
simplicity) but I'd like to see how we can make use of it. The more
extensible this tool is, the better!

Come back on this later...
 

One of the biggest bottlenecks we found is reading in the impulse
responses every time we want to insert the next response into our
SndTable objects. Would be nice to be able to load all the impulse
responses into memory and perform the insertion that way, rather than
having to call SndTable.insert() with a new file each time.

First of all, I don't think .insert() is what you want here. The insert method adds samples to the previously stored samples in the table and crossfade mixes them (that modifies your impulses). I believe you simply want to replace the impulse. In this case .setSound() is what you need. As you can see, with this modified swapImpulse() method:

    def swapImpulse(self):
        leftImp, rightImp = self.helper.getImpulse(self.elevation, self.azimuth)
        # Load the new impulses into the tables for each ear (with crossfading)
        self.impulse_l.setSound(leftImp) # crossfade 0.05 seconds
        self.impulse_r.insert(rightImp, crossfade=0.05)
        print "left impulse length (sampes)", self.impulse_l.getSize()
        print "right impulse length (sampes)", self.impulse_r.getSize()

I got this results while changing azimut or elevation values:
----------------
left impulse length (sampes) 128
right impulse length (sampes) 129
left impulse length (sampes) 128
right impulse length (sampes) 130
left impulse length (sampes) 128
right impulse length (sampes) 131
left impulse length (sampes) 128
right impulse length (sampes) 132
left impulse length (sampes) 128
right impulse length (sampes) 133
left impulse length (sampes) 128
right impulse length (sampes) 134
left impulse length (sampes) 128
right impulse length (sampes) 135
---------------

By the way, I'm agree with you that loading sounds during perf is not the most efficient way. A better way should be to load every sounds in SndTable(s) at initialization and to tell Convolve objects which table to use. Here is the part of the code I modified to accomplish this:

==> HeadSpaceHelper.__init__ (create lists of SndTable)

        ### Init lists of SndTable objects ###
        self.elevData_l_tables = []
        self.elevData_r_tables = []

        for items in self.elevData_l:
            ### add sub list ###
            self.elevData_l_tables.append([])
            items.sort()
            ### populate the sub list with SndTable objects ###
            for item in items:
                self.elevData_l_tables[-1].append(SndTable(item))
        for items in self.elevData_r:
            self.elevData_r_tables.append([])
            items.sort()
            for item in items:
                self.elevData_r_tables[-1].append(SndTable(item))

==> HeadSpaceHelper.getImpulse (return SndTable instead of file path)

        # END MAGIC
        # wavLeft and wavRight are SndTable instead of file path
        wavLeft = self.elevData_l_tables[elIndex][azIndex]
        wavRight = self.elevData_r_tables[elIndex][azIndex]

==> HeadSpace.__init__ (removed SndTable objects from this class (no more used))

        # Load the impulse responses into sound tables for each ear
        leftImp, rightImp = self.helper.getImpulse(self.elevation, self.azimuth)
        #self.impulse_l = SndTable(leftImp)
        #self.impulse_r = SndTable(rightImp)
       
        # Perform convolution on each ear
        self.convolved_l = Convolve(self.signal, leftImp, size=self.bufferSize, mul=.5).out(0)
        self.convolved_r = Convolve(self.signal, rightImp, size=self.bufferSize, mul=.5).out(1)

==> HeadSpace.swapImpulse (assigned new tables to Convolve objects)

        leftImp, rightImp = self.helper.getImpulse(self.elevation, self.azimuth)
        self.convolved_l.table = leftImp
        self.convolved_r.table = rightImp

The modified headspace.py is attached to this mail.

Nice work. I will use it in my new piece!

Olivier