Dealing with long running (i.e. background) FFI functions

[Sam Aaron]

Hi all,

I've been working on a music live coding system called Sonic Pi which provides a Ruby DSL interface to the SuperCollider synthesis engine. Currently the comms between Ruby (MRI) and SuperCollider's server happens via UDP packets and everything works swimmingly.

However, the target platform is the Raspberry Pi, and it appears that loopback socket comms are extremely expensive. I've therefore made efforts to bypass UDP comms by binding SuperCollider's server via FFI. Unfortunately I can't get things working correctly. I wonder if the GIL and lack of real threads is my issue and whether or not there is a path forward. Hopefully one of you might be able to shed some light.

Here's the current FFI declaration:

    attach_function :World_New, [WorldOptions.ptr ], World.ptr

    attach_function :World_WaitForQuit, [World.ptr], :void, :blocking => true

    attach_function :World_SendPacket, [World.ptr, :int, :pointer, :reply_callback], :int

Interaction workflow should be as follows:

1. Create a World pointer using World_New

2. Start server with World_WaitForQuit in a separate thread

3. From arbitrary MRI threads, call World_SendPacket to trigger and control synths

4. Handle return messages via the callback function which returns a string of data

In a stand-alone script, I can get this working perfectly. You can see the script I cobbled together here:

https://github.com/samaaron/sonic-pi/blob/bfab1a91f0b0d58c14b54c9222d0eef015b05a71/app/server/sonicpi/scsynthffi.rb

However, when attempting to integrate it with the rest of Sonic Pi, I seem to be running into a problem with other threads not running (Sonic Pi uses a large number of MRI threads). For example, I have a web socket interface, but the web socket server doesn't seem to want to start up. 

Is there something I should be aware of? Is the kind of workflow I'm suggesting impossible with MRI and FFI? Ideally I'd be able to switch between these two classes with no change in behaviour of the system (other than performance):

https://github.com/samaaron/sonic-pi/blob/master/app/server/sonicpi/scsynthexternal.rb

https://github.com/samaaron/sonic-pi/blob/master/app/server/sonicpi/scsynthnative.rb

Sam