Calling into c from python, go, java and others

[Alex Newman]

Calling into c from the jvm is no low effort activity. The last time I measured parking and unparking was pretty slow. I know the contemporary jvm has some improvements but I am curious if there's a guide of all the overhead. How does the jvm compare to go or python? How about getting data to and fro? If I am running tons of threads when I make the call?

[Wojciech Kudla]

That's an interesting topic, especially with Java as it lacks some valuable features most of us will need at some point.
Whenever I needed to benchmark JNI call overhead, depending mostly on the kernel and the cpu, I got times from single digit to low tens of nanoseconds.
However, my experience is limited to working with primitive arguments as I always need to prevent native code to interact with Java heap.
I'm certain people on this list have had experience with more complex scenarios.

On Sun, Sep 25, 2016, 09:26 Alex Newman <pos...@gmail.com> wrote:

Calling into c from the jvm is no low effort activity. The last time I measured parking and unparking was pretty slow. I know the contemporary jvm has some improvements but I am curious if there's a guide of all the overhead. How does the jvm compare to go or python? How about getting data to and fro? If I am running tons of threads when I make the call?

--
You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dan Eloff]

To answer your question about Python and Go, CPython doesn't have a penalty for calling into C, since it's just a C interpreter with everything implemented in C anyway. That's assuming your calling a proper C extension and not using the ffi module. But because it's Python whatever you call is going to need to use the Python C api to extract arguments and do things with them, and the same for building the results, so that can very quickly end up costing more than the JNI overhead. Also if you don't do a lot of work in the call without touching the Python api, then you can't release the GIL and calling from many threads is going to be serialized. With PyPy the situation is very different and I expect the situation will change for CPython as well since it looks likely to (finally) get a JIT one of these days soon.

Go is somewhat slow at calling into C. At best about 120ns, but it got slightly better this week thanks to the improvements to defer: https://go-review.googlesource.com/#/c/29656/, now about 90ns for the simplest possible call. From what I've seen the JNI is a lot faster than that. It's possible to cheat with assembly in Go and call C functions manually (which requires deep knowledge of the x64 C call ABI on your platform.) Go itself does this when instrumenting code for use with the thread sanitizer - but you really have to know what you're doing and the C code shouldn't block or you could be in for a world of pain.

On Sun, Sep 25, 2016 at 1:26 AM, Alex Newman <pos...@gmail.com> wrote:

Calling into c from the jvm is no low effort activity. The last time I measured parking and unparking was pretty slow. I know the contemporary jvm has some improvements but I am curious if there's a guide of all the overhead. How does the jvm compare to go or python? How about getting data to and fro? If I am running tons of threads when I make the call?

--

You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-sympathy+unsub...@googlegroups.com.

[Alex Newman]

I had thought concurrency had something to do with this. What happens when I have a million green threads going on and I call into c in go?

To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-symp...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "mechanical-sympathy" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mechanical-sympathy/Kqh26EcKG8Q/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mechanical-symp...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Sent from my mobile 4045076749
Alex Newman

[Brett Kail]

Cliff Click has talked about the overhead of Java-to-C calls: https://www.youtube.com/watch?v=vzzABBxo44g#t=18m25s

- Arguming shuffling (including JNIEnv)

- Converting object reference arguments to handles (and vice versa for return values)

- synchronized keyword handling - I think this is the same for non-native but is included for a worst case demo

- Storing SP/PC info to allow stack walking for thread dump or GC

- Check for GC safepoint on method return