considering JSyn for a big project

[Curtis Ullerich]

Hi all,

I'm implementing several instruments for a large work for laptop ensemble and I would really appreciate some input on whether JSyn is appropriate for my use case. I prototyped a representative instrument with JSyn and it does run on our setup (not sure how many channels are actually getting used yet though), but I'm hoping to avoid finding gotchas after I put a hundred hours of work into the system.

In particular:

• I see these two items in the JSyn roadmap and would love to hear what the status is because they sound pretty relevant to our setup:
• Add optional native audio device interface to replace JavaSound. This will fix the clicks on Mac, and give JSyn better support for FireWire devices and multi-channel devices.
• Add multi-threading support in the engine to increase the audio rendering power on multi-core CPUs.
• When I run my prototype on a macbook, CPU usage revs up and stays high. Is this expected? Could I be doing something dumb in my code (I'm sure the answer to that is always "yes" lol)? Can I mitigate it?

Here's our system: We have 5 several-year-old macbooks for performers and another for the conductor, all communicating using OSC on a wired LAN. Each performer machine is connected to a GameTrak (6 joystick axes and a button, reporting at a device-limited 125 Hz). I'm using jinput for interfacing with this. I implemented a JSyn UnitGenerator that outputs control values on a 6-part port and map this to various synthesis parameters. We have some instruments that use oscillator synthesis and some that do sampler synthesis.

Each laptop is connected to a motu audio interface over firewire, and the motus are connected to a 6-channel speaker and a sub.

In some instruments we're sending GameTrak data to Wekinator and using the resultant continuous or discrete params to map to e.g. gain values for several simultaneously-playing wav files, or to control event triggers. In other cases we're using GameTrak data in a state machine to determine latency-sensitive note on/off events.

So far, audible latency is not noticeable in my prototypes that map live GameTrak input to a sine wave frequency and to gain values via Wekinator. This was really exciting, because the former application is extremely latency sensitive. (I implemented the same thing in Beads and the update period was very noticeable audibly.)

We've already implemented (and performed with) many of the synthesis parts of this system in ChucK which works fine in that domain, but I've grown frustrated by the lack of general purpose language features and run into some memory leaks/other issues that sucked a lot of debugging time out of me. We have some key needs like being able to control Wekinator programmatically (it's in Java) and some semblance of modularity/testing/CI that I've found onerous to do with ChucK in the mix. So -- I'm really hoping JSyn will work for us!

Thanks so much for any input from the JSyn community.

Curtis

[Phil Burk]

Hello Curtis,

> I see these two items in the JSyn roadmap

Where did you find that roadmap? It may be out of date.

>  Add optional native audio device interface to replace JavaSound. This will fix the clicks on Mac, and give JSyn better support for FireWire devices and multi-channel devices.

This is the JPortAudio effort. JPortAudio allows us to output sound through native PortAudio. That gives us somewhat lower latency and multi-channel devices.
I think the clicks on Mac was fixed a while ago in JavaSound. So that is no longer an issue.
Do you need multi-channel devices?

I have not worked on JPortAudio in a while. But we have been working on PortAudio. I have started working on extracting a new GitHub repo just for JPortAudio but it has not been a high priority. If you need it then we can bump the priority.

> Add multi-threading support in the engine to increase the audio rendering power on multi-core CPUs.

This would distribute the processing across multiple threads, which would feed their output through FIFOs into a mixer. It will increase latency somewhat but it will also give you more horse power.

>  I implemented a JSyn UnitGenerator that outputs control values on a 6-part port and map this to various synthesis parameters. We have some instruments that use oscillator synthesis and some that do sampler synthesis.

JSyn works as a giant flow graph. The data is pulled through the flow graph from the sinks, usually a LineOut. Everything that is connected runs all the time, generally.

A unit can be disabled using unit.setEnabled(false). Then that unit and everything upstream will not get executed.

You can specify a unit to be disabled when an EnvelopeDAHDSR goes to zero. 

    ampEnv.setupAutoDisable(unit);

This lets you have lots of "voices" and then only execute the ones that are audible.

Phil Burk

--
You received this message because you are subscribed to the Google Groups "jsyn" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jsyn+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jsyn/CADqA8Nfyi0vpG5y0SrJjSjOv1WwPPJyhXRvGO_SMk%2BGQ6GGZTw%40mail.gmail.com.

[Curtis Ullerich]

Hi Phil!

I found the roadmap at http://www.softsynth.com/jsyn/docs/roadmap.php. Our speakers have six channels and ideally we would be able to address them individually. My current demo is just outputting to one channel so next I'll see what I can do about sending that mono signal to all six channels as a starting point. I'm hearing some clicks/pops currently and haven't managed to get a LinearRamp to work as my first attempt at addressing that. So, if I can get audio out of all six channels without pops then I think I feel good about continuing to build out the rest of the instruments. Thanks for your answers on the other points!

Curtis

[Curtis Ullerich]

I hit `send` too soon! I see now that LineOut just has a 2-part port. Does that mean that multi-port audio isn't supported yet?

[Phil Burk]

Hello Curtis,

On Sat, Jul 30, 2022 at 11:21 PM Curtis Ullerich <curtisu...@gmail.com> wrote:

I see now that LineOut just has a 2-part port. Does that mean that multi-port audio isn't supported yet?

It is supported. LineOut is a convenience for stereo. For more than 2 channels use multiple ChannelOut units and setChannelIndex(n). Also you need to request more than 2 channels when you start() the synth.

You may not get more than 2 channels unless you use JPortAudio. I need to get JPortAudio working in a separate repo outside the PortAudio repo. I will give that a high priority. I also have an example of doing multi-channel audio that I will port.

On Sat, Jul 30, 2022 at 11:11 PM Curtis Ullerich <curtisu...@gmail.com> wrote:

I found the roadmap at http://www.softsynth.com/jsyn/docs/roadmap.php.

 I will update that and then move it to the GitHub JSyn pages.

> I'm hearing some clicks/pops currently and haven't managed to get a LinearRamp to work as my first attempt at addressing that.

At what point does the click occur? It can occur when there is a sudden amplitude change, or a sudden frequency change on a filter, or when start() or stop() is called on a unit.

Phil Burk

 

[Curtis Ullerich]

Hi Phil,

> You may not get more than 2 channels unless you use JPortAudio. I need to get JPortAudio working in a separate repo outside the PortAudio repo. I will give that a high priority. I also have an example of doing multi-channel audio that I will port.

I'll hook it up to all six channels and see how it behaves currently. Thanks so much for considering the JPortAudio work! If there are parts of that where I can be useful I'm happy to pitch in.

> At what point does the click occur? It can occur when there is a sudden amplitude change, or a sudden frequency change on a filter, or when start() or stop() is called on a unit.

My demo is 5 wav files with gains controlled by continuous parameters coming out of an ML model every 8 ms. The delta between update values is theoretically small but I need my audio graph to be tolerant to large deltas. I did get a LinearRamp plugged in and tried experimenting with the time. Even using a <8ms lag I hear some aliasing(?), perhaps unsurprising given the update frequency? Maybe I should implement a UnitInputPort that does a moving average? Some kind of interpolation between values seems ideal though. This rate of update/continuous tracking is a common pattern for our instruments so I'm trying to build something resilient and reusable.

Thanks!

Curtis

[Phil Burk]

Hello Curtis,

> Even using a <8ms lag I hear some aliasing(?), perhaps unsurprising given the update frequency?

8 ms for a LinearRamp time is very fast. I use a minimum of 10 msec to avoid clicks. 20 or 30 msec is even better. Just set the time port once to 0.010 and then update the input port whenever you want.

> continuous parameters coming out of an ML model every 8 ms.

For a smooth ramp with periodic updates the time port value should be greater than the period.

Phil Burk

 

[Curtis Ullerich]

After I used the LinearRamp with 20 ms and hooked it up properly in my audio graph (oops) the pops were gone. It's sounding great. Thanks!

I used 6 ChannelOuts and the default/JavaSound audio device with synth.start(44100, USE_DEFAULT_DEVICE, 2, USE_DEFAULT_DEVICE, 6) to see what would happen. On my Fedora laptop with stereo output it logs no errors and happily plays over stereo (presumably with the first two channels). On the macbook hooked up to the audio interface and 6 channel output, I get no audio, and this error is logged:

SEVERE: JavaSoundOutputStream - not supported.PCM_SIGNED 44100.0 Hz, 16 bit, 6 channels, 12 bytes/frame, little-endian

java.lang.NullPointerException: Cannot invoke "javax.sound.sampled.SourceDataLine.write(byte[], int, int)" because "this.line" is null

at com.jsyn.devices.javasound.JavaSoundAudioDevice$JavaSoundOutputStream.write(Unknown Source)

I also tried JSyn.createSynthesizer(JPortAudioDevice()). On Fedora I have portaudio-devel installed, but clearly that's not sufficient because I got this error:

Exception in thread "main" java.lang.NoClassDefFoundError: com/portaudio/PortAudio
        at com.jsyn.devices.jportaudio.JPortAudioDevice.<init>(Unknown Source)

Same error on the macbook, but I'm even less familiar with how to properly install PortAudio there.