Erjang vs Jinterface

[Matthew Evans]

Hi Group,

We are currently developing a project where there is the need to
interface Erlang and Java.

In this system there is a central Java based application server, and a
mesh of distributed Erlang VMs that perform IPC and monitor /
interface with external devices.

The need has arisen to have the Erlang code and Java code communicate
with one another. The obvious "official" method is to use jinterface
for this purpose. However, I'm wondering if Erjang could be considered
a better alternative. I can see many advantages of using Erjang over
jinterface, the ability to spawn Erlang processes on the JVM to
perform blocking operations, the ability to allow Erlang developers
natively call Java code, and opening the resources of Erlang language
(mnesia etc) to the JVM. Certainly jinterface can do that, but Erjang
seems like a more natural fit.

I guess I'd like to know where one might consider Erjang vs jinterface
(there will not likely be the need to use NIFs in the JVM), and what
are the advantages of one over the other?

Thanks

Matt

[Matthew Evans]

Oh, and I meant will not be needing any BIFs on the JVM (if we do I
can offload that to another Erlang VM).

[Robert Virding]

How do you mean not needing any BIFs? Do you mean the standard Erlang
BIFs, which erjang already has most of them, or do you mean NIFs?

Robert

[Matthew Evans]

BIFs ... I won't be needing to use BIFs in Erjang (won't be needing
NIFs either).

My understanding is that Erjang has limited BIF support.

On Sep 11, 5:55 pm, Robert Virding <rvird...@gmail.com> wrote:
> How do you mean not needing any BIFs? Do you mean the standard Erlang
> BIFs, which erjang already has most of them, or do you mean NIFs?
>
> Robert
>

[Tony Arcieri]

Does Erjang have a mechanism to make a "NIF" call to arbitrary JVM code? I realize Kilim has limitations on what you're allowed to do...

--
Tony Arcieri

[Matthew Evans]

My understanding (might be wrong) is that Java code can be called
"natively" from Erjang.

Page 22 onwards of this seems to imply that

http://gotocon.com/dl/goto-cph-2011/slides/KrestenKrabThorup_ErjangAJVMBasedErlangVM.pdf

I can see great potential to marry the two languages in this way.
Using Erlang's strengths of concurrency, reliability and distribution
with Java's performance and wider user base.

On Sep 11, 6:11 pm, Tony Arcieri <tony.arci...@gmail.com> wrote:
> Does Erjang have a mechanism to make a "NIF" call to arbitrary JVM code? I
> realize Kilim has limitations on what you're allowed to do...
>

[Robert Virding]

From what I know erjang already has most Erlang BIFs so that should
probably not be a problem. Running distributed Erlang and erjang works
quite well, even mnesia works.

[Matthew Evans]

Thanks....I'll give it a go.

If it works, and is a success I'll go to an Erlang conference to
explain it.

On Sep 11, 6:27 pm, Robert Virding <rvird...@gmail.com> wrote:
> From what I know erjang already has most Erlang BIFs so that should
> probably not be a problem. Running distributed Erlang and erjang works
> quite well, even mnesia works.
>

[Matthew Evans]

Are there any downsides to Erjang vs. Jinterface?

[Erik Søe Sørensen]

Could you qualify "the ability to spawn Erlang processes on the JVM to
perform blocking operations" please?
Especially the "blocking" part. That can be understood as "operations that would block an Erlang process" - receive, timer usage, calls to other processes etc. - which will allow the scheduler/Java thread to service other processes in the meantime.
Or it can be understood as "operations that would block a Java thread" - i.e. actual Java operations - which would block the Java thread used as an Erlang-scheduler thread, of which there are a small, fixed number.

Also, which way are you planning to go to call Java from Erjang?

Calling Java from Erjang can be done in more than one way, as the slide you refer to suggests.
Regarding the "Plain Java: type conversions" bullet point:
My experience with calling Java code using erjang.m.java.JavaObject is that it isn't the most mature part of the codebase (as I understand it, it's mostly a proof of concept); it tries to guess how a Java object should be viewed by Erlang; the way this auto-conversion works may not be entirely healthy.
(Sorry I can't be more specific without some investigation; all I remember is taking JavaObject for a spin to find out whether it was easy to apply Triq to Java code that way, then concluding that there were some gotchas in JavaObject that would obscure the testing process.)
If you're going that way, I suggest not being too clever about it - treating the Java object as an opaque type, and returning only immutable data like strings and numbers (and other opaque types) would be the way to go.
The immutability point applies to any way of calling Java code from Erlang, of course.

Regarding the "BIFs: special calling convention" bullet point.
This is the better-tested of the two approaches on the slide, as all BIFs are implemented this way (well, duh).
This way is about creating a dummy Erlang module (say, mymod.erl), and a companion Java class with the name erjang.m.mymod.Native.
Each public method of this class which is annotated with erjang,BIF is then available as a function in the module with the same name as the method.
The method will need to take a number of EObjects as arguments and return an EObject or subclass thereof. It may also take an EProc (encapsulating the LWP state), as its first argument.
It should (I think) not throw anything but erjang.ErlangException, if you can help it. You may need to catch and wrap any unchecked exceptions. Also, don't block the thread - it's one of those Erlang scheduler threads.
(Depending on why you need to block, you may declare the function to throw Pausable, and then ensure that your class file is processed (weaved) with Kilim - this could be used e.g. for using timer:sleep() or erlang.yield() or other interesting BIFs; I won't go into details here though.)

Actually, there is a third option: implementing the Java part as an Erjang driver. I don't know of any clear advantages to this, though.

Without knowing more about your scenario, I'd recommend the BIF way, and doing any blocking stuff in separate Java threads.
I think maybe we should make this easier - just like you can make Java nodes talking to Erlang nodes, in Erjang there should be an easy way to have Java threads talking to Erjang threads. At the moment the Erjang LWP mailboxes are of bounded size, though, so 'send' is a potentially blocking operation, which complicates things somewhat.

Hoping this helps - let us know of any issues, should you encounter any.
Erik Søe Sørensen
Trifork

[Pavlo Baron]

I'm sorry for the maybe most unqualified answer containing alsmot
questions only, but:

- What exactly is the use case of your Java app server talking to the
Erlang nodes? What is the topology of the distributed system? Where and
how does the Java app server fit in?

- Is it necessary to wire at the lower level? Or would it be better to
go for REST services?

- Is it really necessary to map millions of lightweight Erlang processes
to just hundreds / thousands of Java threads? Or if these numbers aren't
relevant, why have a complex distributed Erlang node infrastructure?

- Which Java specifics would you like to use in your Erlang code? I can
think of library, e.g. calendar transformations and whatnot, but where
is the need on the Erlang side to make use of them?

- Is it more simple to run the whole thing on the JVM, including your
Java artefacts as well as a web application with an embedded Erjang and
your transformed Erlang code, which is also possible?

- Is it necessary to talk in both directions?

- Which practical value does have Mnesia usage outside Erlang, where
loads of distributed data stores with quality non-native interfaces
already exist?

When you have answered all these questions, you match your answers with
the possible Erlang/Java integration ways:

- jinterface means you have a pseudo Erlang node in your distributed
node topology, which is known by epmd. You can talk with and from it as
a normal Erlang node, but you are using Java threads in it, so you can't
bomb it with millions of parallel requests

- Erjang is here to run your Erlang code on the JVM. For things where
you have loads of C code behind your Erlang solution which is very
usual, it wouldn't help you much since it wouldn't auto-convert your C
code. When you have Erlang code only and are fixed to the JVM for
whatever operational reason, Erjang can help you

- There are still good old web services to integrate both worlds, and
neither jinterfacce nor Erjang compete with them - for the reasonable
use cases

Maybe it helps somehow.

pb

[Erik Søe Sørensen]

For completeness, I feel I should mention the "other official" method. It also uses jinterface, but you don't use it directly; instead, you code to IDL in both ends and let an IDL compiler handle the boilerplate (communication and data conversion).
That approach - as applied to Erlang nodes calling Java nodes - is described here: http://polymorphictypist.blogspot.com/2011/08/erlang-java-interop-demo.html
However, that is more a "jinterface is bothersome" solution than the "let's integrate Erlang and Java tightly" solution it sounds like you're after...

In any case, I hope we'll be hearing more about the scenario-in-case.
Regards,
Erik Søe Sørensen
Trifork

[Matthew Evans]

Thanks all for your replies.

Unfortunately I can't describe too much about the architecture other
than to say that the Erlang VMs will be running on embedded hardware
(still using Linux) and shall be monitoring traffic streams on those
devices as well as exchanging state information between them. The key
to the Erlang portion is "5 nines" reliability - that stuff can not
crash. The central Java application provides certain analytics and
application "smarts" to the whole system. There isn't a technical
reason why Java was chosen -- we just selected a language from the
available development pool (in fact there are stronger reasons for
doing that work in Erlang IMO).

Initially I'm just after an IPC, in which case jinterface should work
just fine (I've already got experience developing cnode interfaces, so
I know the pitfalls of that solution).

I can see a number of benefits of Erjang:

* Supervision of the Erjang VM (and by extension the JVM) from the
distributed Erlang VMs using the OTP supervision methods.

* No need to add call backs or other hooks in the interface as new
functionality is added.

* We need an Erlang VM on the same host as the JVM - using Erjang
would negate this need.

Mainly though, the ability to use features of both languages in a
"native" manner, has the greatest appeal.

I do however have enough stuff to get starting with.

Many thanks

Matt