JNAJack - Latency expectations and garbage collection

[Daniel Hams]

Hi Neil,

First comment - thank you very much for putting jnajack out there and making it available - it's much appreciated given the sorry state of regular Java audio!

I've been playing around a little using jnajack as the API for getting my floats and midi in and out of Java, but I've noticed a few things where I've perhaps not understood how things should work. If you have the time for a little discussion that'd be great.

My prototyping application is a little toy audio graph for playing with DSP (the real thing is in C++, but I find Java easier to refactor). I'll be releasing the Java source over the coming days but I've already got some points to mention below.

My issues:

(1) Achievable latencies seem a little high.

Using jack with a period size of 1024 X 2 @ 44100hz is as low as I can go (46ms) with Java without getting overruns - this seems a little high.

I'm using sun jdk1.8.0_25 on x86_64, jack2 (as jack1 boots the process out on overruns). 2.6Ghz Quad Core machine, plenty powerful enough.

Command line as follows:

java -XX:+UseG1GC -XX:InitiatingHeapOccupancyPercent=80 -XX:MaxGCPauseMillis=10 -Xms512m -Xmx1024 -XX:-UsePerfData -jar cd.jar

Now I tested under realtime linux too with similar results (unfortunately my current linux test machine has an older nvidia card and getting a realtime kernel and those drivers to co-operate is a pain). I've also played with the various different collectors and settings and this is so far the best I've found.

I've profiled with nanosecond timestamps under Java and my C++ implementation and get vaguely similar results in terms of processing times (its the processing time for nodes in an empty audio graph):

Java (varies a little bit of course, but I've show a "low" and "high" result):

Got rendering profile results - clockStart(18916865134381) clockEnd(18916865143985)
ProducerDuration(139) rpFetchDuration( 169) loopDuration(9296) totalDuration(9604)
JobLength(     214) JobThreadNum(0) JobName(Initial Fan)
JobLength(    6091) JobThreadNum(0) JobName(Master Out of type Master Out)
JobLength(     461) JobThreadNum(0) JobName(Master In of type Master In)
JobLength(     247) JobThreadNum(0) JobName(Final Sync)

Got rendering profile results - clockStart(19999825631986) clockEnd(19999825651186)
ProducerDuration(120) rpFetchDuration( 330) loopDuration(18750) totalDuration(19200)
JobLength(     705) JobThreadNum(0) JobName(Initial Fan)
JobLength(    1800) JobThreadNum(0) JobName(Master In of type Master In)
JobLength(   10965) JobThreadNum(0) JobName(Master Out of type Master Out)
JobLength(     379) JobThreadNum(0) JobName(Final Sync)

C++ (it includes the port_get_buffers() bits - doesn't vary as much, diffs mostly due to cache misses I guess):

PROFILE RESULTS - totalNanos   (00.000.007.815) loopNanos    (00.000.004.969) producerNanos(00.000.002.741)
PROFILE RESULTS - rpFetchNanos (00.000.000.105) clockStart   (09.378.461.903) clockEnd     (09.378.469.718)
JobLength(00.000.000.252) JobThreadNum(0) JobName(Initial Fan)
JobLength(00.000.000.604) JobThreadNum(0) JobName(Master In)
JobLength(00.000.002.389) JobThreadNum(0) JobName(Master Out)
JobLength(00.000.000.142) JobThreadNum(0) JobName(Final Sync)

PROFILE RESULTS - totalNanos   (00.000.008.940) loopNanos    (00.000.005.490) producerNanos(00.000.003.248)
PROFILE RESULTS - rpFetchNanos (00.000.000.202) clockStart   (35.883.790.539) clockEnd     (35.883.799.479)
JobLength(00.000.000.255) JobThreadNum(0) JobName(Initial Fan)
JobLength(00.000.000.506) JobThreadNum(0) JobName(Master In)
JobLength(00.000.002.798) JobThreadNum(0) JobName(Master Out)
JobLength(00.000.000.221) JobThreadNum(0) JobName(Final Sync)

So, total processing time is (9->20) microseconds under java, (8->9) under C++.

Unsurprisingly the C++ is faster, but the Java is within the same order of magnitude for the actual processing. The amount of time spent processing before pushing to jack should be well under the latency constraints for, say 10ms latencies - yet Java overruns - presumably due to garbage collection interferring with the processing threads.

I've verified with jprofiler and other tools there isn't any allocations on my hot-code path - although there are allocations inside jack_port_get_buffer and under there when constructing com.sun.jna.Pointers.

It's failing to meet a 23.2ms deadline every now and then (given use of two jack periods).

What kind of latencies are you able to achieve with jnajack?

Am I wasting my time looking deeper here as it's impossible with a regular java VM (maybe need RTSJ and realtime threads)?

(2) Use of jna vs jni

There was a mapped layer back in 2007 called jjack that (I think) used JNI - maybe you have previously tried with JNI too? The reason I mention this is the allocations I spotted above. In an ideal world we'd attempt to get to zero allocations during runtime use. Of course if an application developer creates garbage that's something else, but it would be nice to have the jnajack platform not be a factor in it.

(3) CallbackThreadInitializer and application exit

My application hangs attempting to shut down with the existing code base - and after I little investigation I discovered that changing the callback thread initialiser bits allows the shutdown.

Jack.java:96 change from

    setCTIMethod.invoke(null, callback, ctiConstructor.newInstance(true, false, "JNAJack"));
to
    setCTIMethod.invoke(null, callback, ctiConstructor.newInstance(false, false, "JNAJack"));

So calling the constructor with daemon=false allows my app VM to shutdown. I've attempted to close and shutdown the jack connection appropriately but I still get this issue.

Any pointers on what I'm probably missing? Maybe I should create a test case to show the issue ;-)

(3a) Now that I'm writing about this, should the callback thread initializer be setup for the other callbacks too? (Currently only setup for ProcessCallbackWrapper, probably should be added for XRunCallbackWrapper and others too).

Sorry for the rambling!

Cheers,

Dan

[Neil C Smith]

Hi Daniel,

On 17 January 2015 at 21:05, Daniel Hams <danie...@gmail.com> wrote:
> First comment - thank you very much for putting jnajack out there and making
> it available - it's much appreciated given the sorry state of regular Java
> audio!

Thanks! You may want to have a read of
https://praxisintermedia.wordpress.com/2013/11/06/jaudiolibs-audioservers-a-portaudio-esque-java-api/
I'm not that scathing of JavaSound myself, at least on Linux (Windows
is useless) - I'm able to get similar performance to JACK out of the
JAudioLibs JS backend, with a few little hacks in the background.

> I've been playing around a little using jnajack as the API for getting my
> floats and midi in and out of Java, but I've noticed a few things where I've
> perhaps not understood how things should work. If you have the time for a
> little discussion that'd be great.

Fire away! I'd recommend using the JAudioLibs AudioServer API for
audio - it's slightly simpler than using JNAJack directly, though
there's no access to MIDI without directly using JNAJack.

> (1) Achievable latencies seem a little high.
>
> Using jack with a period size of 1024 X 2 @ 44100hz is as low as I can go
> (46ms) with Java without getting overruns - this seems a little high.

...

> Command line as follows:
>
> java -XX:+UseG1GC -XX:InitiatingHeapOccupancyPercent=80
> -XX:MaxGCPauseMillis=10 -Xms512m -Xmx1024 -XX:-UsePerfData -jar cd.jar

OK, your achievable latency seems far too high, assuming otherwise
your system can achieve much better? My current laptop with stock
Ubuntu kernel (not even low latency kernel at the moment) and OpenJDK
7 can achieve a reliable 256 x 2 @ 48000hz with the internal soundcard
- the system won't go lower. I've achieved better in the past with a
realtime kernel.

First question - have you tried using

java -Xincgc -jar cd.jar

Does this give you better performance?

Generally, I've found the G1 garbage collector to be useless.
Unfortunately, I think that -Xincgc is deprecated in 8, though it
should still work - I really hope they change their mind! Likewise,
unless you need to be setting the heap size, don't! The smaller heap
you have, generally the better the performance because the GC has less
to do in one go - you want GC happening little and often.

> (2) Use of jna vs jni
>
> There was a mapped layer back in 2007 called jjack that (I think) used JNI -
> maybe you have previously tried with JNI too? The reason I mention this is
> the allocations I spotted above. In an ideal world we'd attempt to get to
> zero allocations during runtime use.

I did some benchmarking of JJack and JNAJack early on in the
development process. Generally, achievable latency was similar in my
experience, and this is back before JNA had direct native mapping.
CPU usage was initially much higher with JNAJack, until I helped get
the CallbackThreadInitializer API into JNA, since which it's generally
similar.

Offhand, I think JJack would have similar allocations to JNAJack - I
don't have the source to hand anymore, but I'm pretty sure it still
has to allocate the buffer reference in the JVM.

I'm happy to see a benchmark of JNI vs JNA using the JNAJack API -
I've just no intention of writing one! ;-)

> Of course if an application developer
> creates garbage that's something else, but it would be nice to have the
> jnajack platform not be a factor in it.

Not sure if you've seen Praxis LIVE (www.praxislive.org), which is the
software I wrote JNAJack for. In the current v2 alpha it's possible
to run the audio pipeline in a separate VM - this is an approach I'd
recommend for a Java JACK application if you want to get best
performance.

> (3) CallbackThreadInitializer and application exit
>
> My application hangs attempting to shut down with the existing code base -
> and after I little investigation I discovered that changing the callback
> thread initialiser bits allows the shutdown.
>
> Jack.java:96 change from
>
> setCTIMethod.invoke(null, callback, ctiConstructor.newInstance(true,
> false, "JNAJack"));
> to
> setCTIMethod.invoke(null, callback, ctiConstructor.newInstance(false,
> false, "JNAJack"));
>
> So calling the constructor with daemon=false allows my app VM to shutdown.
> I've attempted to close and shutdown the jack connection appropriately but I
> still get this issue.
>
> Any pointers on what I'm probably missing? Maybe I should create a test case
> to show the issue ;-)

A test case would be good. Your experience would appear to be the
exact opposite of what you should be seeing! :-\

> (3a) Now that I'm writing about this, should the callback thread initializer
> be setup for the other callbacks too? (Currently only setup for
> ProcessCallbackWrapper, probably should be added for XRunCallbackWrapper and
> others too).

That shouldn't be necessary. When a native thread calls into the JVM,
it must be attached to the JVM before it can call any Java methods.
Prior to the changes I requested in JNA, every callback would attach
and detach, creating a Thread proxy object each time - hence the high
CPU load. The CallbackThreadInitializer keeps the thread attached to
the JVM between process calls. IIRC, if the other callbacks happen in
the same JACK thread, then the existing process thread settings should
cover them too. Even if they're not covered, the overhead will be
minimal for infrequent callbacks.

> Sorry for the rambling!

No problem, I can ramble with the best of them! ;-)

btw - your GitHub profile says you're UK based - don't suppose you're
anywhere near Oxford?

Best wishes,

Neil


--
Neil C Smith
Artist : Technologist : Adviser
http://neilcsmith.net

Praxis LIVE - open-source intermedia development - www.praxislive.org
Digital Prisoners - interactive spaces and projections -
www.digitalprisoners.co.uk

[Daniel Hams]

Hi Neil,

Sorry for the sluggish reply. Been a smidgeon busy getting
a release of

(1) https://github.com/danielhams/mad-java

You've done releases, you know the pain :-)

Also sorry if the formatting of this is all screwey. Is there any interface other than the web one?

Firefox seems to give me an eight line window into a reply and no way to make it larger....
I ended up editing the reply in emacs and then copy-pasta'ing.

> You may want to have a read of
> https://praxisintermedia.wordpress.com/2013/11/06/jaudiolibs-audioservers-a-portaudio-esque-java-api/
> I'm not that scathing of JavaSound myself, at least on Linux
> (Windows is useless) - I'm able to get similar performance to JACK out of the
> JAudioLibs JS backend, with a few little hacks in the background.

I guess you've had more luck than I (with javasound). I got annoyed with the variance
in performance between different JDK versions and different platforms. And then on top
of that there was the annoyance of byte->float->byte conversions and no simplified
MIDI interface (dealing with running status is soooooo 90s .-)

Jack really does make all that go away :-)

> I'd recommend using the JAudioLibs AudioServer API for
> audio - it's slightly simpler than using JNAJack directly, though
> there's no access to MIDI without directly using JNAJack.

The AudioService API you mention seems nice but a little high level
for the kind of things I like to do (see #1)

Here's a fun little SVG diagram of it:

http://www.modular-audio.co.uk/blog/2015-01-31-003/cdbeans.svg

> OK, your achievable latency seems far too high, assuming otherwise
> your system can achieve much better?  My current laptop with stock
> Ubuntu kernel (not even low latency kernel at the moment) and OpenJDK
> 7 can achieve a reliable 256 x 2 @ 48000hz with the internal soundcard
> - the system won't go lower.  I've achieved better in the past with a
> realtime kernel.

I suspect I'm simply trying to do too much in one VM (see #1).

There is a Swing GUI and a raft of BufferedImages, plus 60 fps gui
updates and ring buffers between DSP code and GUI code....

Once I felt I had tried hard enough to get consistent latency I switched
to C++ (about 1.5 yrs ago) and left the Java bits as a prototyping tool.

Best thing about Java is the tooling maturity. I can't find robust refactoring
tools like Java has for C++. (I know there are some but I hate having my build
environment dictated to me by one tool).

So, back to the Java bits :-)

> Generally, I've found the G1 garbage collector to be useless.
> Unfortunately, I think that -Xincgc is deprecated in 8, though it
> should still work - I really hope they change their mind!   Likewise,
> unless you need to be setting the heap size, don't!  The smaller heap
> you have, generally the better the performance because the GC has less
> to do in one go - you want GC happening little and often.

For my use-case G1 is about the only one where I don't get occasional latency
spikes every five minutes or so once running.

Again, I'm sure this is related to the huge amount that's going on in the
VM in addition to the DSP processing.

> I did some benchmarking of JJack and JNAJack early on in the

> development process.....

> I'm happy to see a benchmark of JNI vs JNA using the JNAJack API -
> I've just no intention of writing one! ;-)

Yep fair enough, I figured you'd probably already put enough energy into it
to know if it offered any benefits.

> Not sure if you've seen Praxis LIVE (www.praxislive.org), which is the
> software I wrote JNAJack for.  In the current v2 alpha it's possible
> to run the audio pipeline in a separate VM -  this is an approach I'd
> recommend for a Java JACK application if you want to get best
> performance.

I'm a smidgeon jealous (of the side VM processing) and throw some snide
remarks your way.

Are you communicating via RMI between VM instances or good old sockets?

I'm wondering if (for example) passing audio data via ring buffers is practical
for the exchange from DSP <-> GUI. Parts of my GUI (e.g. spectral amp display)
pull over the raw samples so the FFT is happening GUI side instead of on
the realtime thread.

> CallbackThreadInitializer

> A test case would be good.  Your experience would appear to be the
> exact opposite of what you should be seeing! :-\

Gah, this ones an intermittent show-er. I have a suspicion it's some combination
of debugging settings and profiling settings that make it show up. I did get it to
show up once after our exchange here but it has since dissapeared again.

I've changed dev machine and JDK version since my original issue.

I did create a test case, and if it ever shows up again I'll try and nail it down.

> That shouldn't be necessary.....

> The CallbackThreadInitializer keeps the thread attached to

> the JVM between process calls.......

Yep, I was thinking that IIRC Jack2 uses a separate thread for event
notification (in addition to the realtime thread for DSP).

As you mention, this isn't high event load so isn't an issue.

> your GitHub profile says you're UK based - don't suppose you're
> anywhere near Oxford?

I'm in Colchester, Essex.

Although I imagine I'll be commuting up to London for work soon
(starting the contract hunting this week).

By the way, any objection to merging that change I suggested?

I currently have a custom JAR of JNAJack for my application
but would love to skip that and just pull an official release version.

Should you prefer any changes (for that pull) don't hesitate to let me know.

Cheers,

Dan

[Neil C Smith]

Hi,

On 1 February 2015 at 16:33, Daniel Hams <danie...@gmail.com> wrote:
> Sorry for the sluggish reply. Been a smidgeon busy getting
> a release of
>
> (1) https://github.com/danielhams/mad-java

That looks really interesting, and a nice UI too! (OT - what's the LAF?)

> Also sorry if the formatting of this is all screwey. Is there any interface
> other than the web one?

Yes, email! ;-) Don't use the web interface, it's horrible - just
make sure you're set to receive all emails (which I think you are) and
reply to them as any other mailing list.

> I guess you've had more luck than I (with javasound). I got annoyed with the
> variance
> in performance between different JDK versions and different platforms. And
> then on top
> of that there was the annoyance of byte->float->byte conversions

You should try out the JS server! There's a few things in the
background (cribbed from various places) that improve JS performance
(still not brilliant on Windows though) and it also handles the
byte->float stuff.

> and no
> simplified
> MIDI interface (dealing with running status is soooooo 90s .-)

but it doesn't deal with MIDI.

> The AudioService API you mention seems nice but a little high level
> for the kind of things I like to do (see #1)

Anything you can do with JNAJack you can still do with the AudioServer
API - you can get direct access to the JackClient, etc. It just
simplifies providing different audio backends - ie. you could also
support JS without any code changes on your side (except MIDI!) and
there is an ASIO impl. somewhere, though I don't think it's been
updated to the latest API.

> Here's a fun little SVG diagram of it:
>
> http://www.modular-audio.co.uk/blog/2015-01-31-003/cdbeans.svg

Fun, but none the wiser! ;-)

> For my use-case G1 is about the only one where I don't get occasional
> latency
> spikes every five minutes or so once running.

That sounds like the issue with the heap being too big I mentioned -
when it does a full compacting collection on a large heap it seems to
XRun, which is in the minutes frequency. Mind you, are you sure
that's Java only - my old laptop used to do the same when WiFi was on!
;-) You could try doing verbose GC output or watching in VisualVM for
a correlation, though both those can cause XRuns too so it's a bit hit
and miss!

In my experience, G1 just gives me XRuns every few seconds.

> I'm a smidgeon jealous (of the side VM processing) and throw some snide
> remarks your way.
>
> Are you communicating via RMI between VM instances or good old sockets?

Neither. The current unoptimised version is using OSC over TCP.
That's because it's got to work cross-platform and across a network as
well as locally. A bit more about it here -
https://praxisintermedia.wordpress.com/2014/11/06/distributed-hubs/
including a photo of me live-coding a Processing-based 3D video
component across laptops.

Running projects across a network was one reason for this feature, and
audio performance was the other - in v2, once I added the NetBeans
editor with full code completion, and javac rather than janino as the
in-process compiler, then GC went a bit nuts!

This is all easy to achieve, though, because Praxis LIVE is designed
from the ground up for separating pipelines (video, audio, GUI,
compiler, editor, etc.) - everything communicates asynchronously using
lock-free queues loosely based on the actor model. This sort of
architecture means that no component has to know whether messages are
VM local or not - the only code that needs changing is the dispatcher.
Old post - https://praxisintermedia.wordpress.com/2012/07/26/the-influence-of-the-actor-model/

So, yes, I'd recommend something similar. Of course, I'm not passing
around full audio data at the moment! ;-)

> I'm in Colchester, Essex.
>
> Although I imagine I'll be commuting up to London for work soon
> (starting the contract hunting this week).

Well, good luck with that.

I wondered because it's always interesting to meet up with other mad
people who like doing audio DSP in Java! ;-)

> By the way, any objection to merging that change I suggested?

...

> Should you prefer any changes (for that pull) don't hesitate to let me know.

Aren't there still a couple of changes pending? Just checked and
noticed it's been updated - not sure I got a notification about that.
Will have another look.

In general, I'd like to get it in sooner than later.

[Daniel Hams]

> make sure you're set to receive all emails (which I think you are) and
reply to them as any other mailing list.

I'm such an idiot. Thanks, that is indeed far easier.

> (OT - what's the LAF?)

It's gtk2 platform LAF using the Adwaita-Dark-3.14-rev2 gtk2 theme. I
have to admit it looks ok like this. The regular Java LAF, yeah, not
so nice :-)

Let's not talk about how it looks under OSX hehe.

> Anything you can do with JNAJack you can still do with the
> AudioServer API - you can get direct access to the JackClient, etc.

It has a certain attraction - but there are some things I'd prefer not
to compromise on - like latency timing info.
(Using JavaSound on top of Pulseaudio makes this is a tricky one)
In this I include access to some translation from DSP processing time
to GUI time.

My motivation is to have GUI updates synchronised with audio output -
if it works for looooong
period lengths, I know it's working properly for short ones too.

Jack provides methods that make this simple and allow for nice smooth
MIDI event processing.

Out of curiosity I recently had a look at how Mixxx handles MIDI and
timestamps (DJ toy is kinda where I'd like to take
my prototyping thing) - I gave them some (perhaps misguided) feedback here:

http://www.mixxx.org/forums/viewtopic.php?f=1&t=6879&sid=e0f4a506d81dbbf3cbac6097cefb3cba

I like the ability to consume events at the appropriate offset they
happened - so
timestamps and a natural mapping to the real world are quite important.

> You could try doing verbose GC output or watching in VisualVM for
> a correlation, though both those can cause XRuns too so it's a bit hit
> and miss!

It's been a while, but IIRC I profiled on a realtime Linux machine
using latencytop, sysprof, jprofiler as well as visualvm and spotted
some things I could avoid (like using Javolution non-allocating
iterators, turning off "in VM" profiling stats writing) but that
didn't remove the problem, just reduced it.

In addition to the expected GC pauses there were some other pauses in
there too for housekeeping and (presumably) hot code replace that
can't be avoided.

I have of course got a little loop to try and get things cache-hot
before using them, but there's only so far you can go when it's
happening under you.

After our discussion I'm more and more thinking I need to lower my
expectations a little.

In all honesty, if I can get vaguely ok latencies with the occasional
overrun that's
good enough for the toy as it stands.

> .. Some discussion of VM<->VM communication....

> Neither. The current unoptimised version is using OSC over TCP.

I guess I'll have to have a little play at some point and see what's
the practical way to get ring buffer / event transfers between VMs
that doesn't involve lots of garbage being generated.

> I wondered because it's always interesting to meet up with other mad
people who like doing audio DSP in Java! ;-)

Yeah, there's not many around is there. Shame really :-)

D

[Neil C Smith]

Hi,

On 7 February 2015 at 10:40, Daniel Hams <danie...@gmail.com> wrote:
>> make sure you're set to receive all emails (which I think you are) and
> reply to them as any other mailing list.
>
> I'm such an idiot. Thanks, that is indeed far easier.

You're far from the only one to be confused by it - I wish I'd chosen
something else. :-\

> It's gtk2 platform LAF using the Adwaita-Dark-3.14-rev2 gtk2 theme. I
> have to admit it looks ok like this. The regular Java LAF, yeah, not
> so nice :-)

Dang, I was hoping it was something cross-platform I hadn't seen
before. Nimbus isn't too bad, mind you, when you change all the
colours! ;-)

>... latency discussion...

> (like using Javolution non-allocating iterators,

I've not tried the Javolution library, but I tend to not use
Collections at all in audio-code, just plain old arrays (and various
variations on this code for adding and removing elements -
https://github.com/jaudiolibs/pipes/blob/master/src/main/java/org/jaudiolibs/pipes/impl/Utils.java
)

Mind you, for what you're doing I would expect the garbage produced in
Java 2D (a lot!) for your GUI updates is going to swamp anything you
can do to mitigate it in your code!

> After our discussion I'm more and more thinking I need to lower my
> expectations a little.

Well, I'd be interested to know how Praxis LIVE performs on your
system, both with audio in process and audio in a separate VM. If
performance is similar to your own tool then it's a pointer that it's
a VM thing - OTOH, if you can get better latency with Praxis LIVE it
might help point you to architectural changes that could give you
better performance, and whether a separate audio VM is worth the time
to pursue.

Mind you, until it's out of alpha, v2 documentation is a bit scarce,
so if you want to try it and get stuck, email me off-list or through
http://groups.google.com/d/forum/praxis-live

> I guess I'll have to have a little play at some point and see what's
> the practical way to get ring buffer / event transfers between VMs
> that doesn't involve lots of garbage being generated.

One thing I would say is that if you're passing blocks of audio data
around, it's time to look at that Java no-no and implement Object
pooling.