Rust and Julia - Together? Advice

[Eric Forgy]

Hello,

I am building a web app for enterprise risk management. I've made some progress, but considering some significant changes now before I am too committed. My current platform is Java/Spring MVC on the back end (C) and Jsp, javascript, and d3 for the front end (V) and I was using Java for numerical modeling (M). Coming from a 20+ year background using Matlab, my experience trying to do any serious numerical modeling in Java is a little frustrating so far. While considering alternatives, e.g. Matlab Production Server, R, Python, etc, I was naturally led to Julia and am only now beginning to research it, but really like what I see so far.

Despite my ambitious goals, I am ignorant about most of the technologies I'm using and learning as I go. Do you have any suggestions about how I might be able to fit Julia into my Spring MVC framework? How can I call Julia from Java? I don't see anything obvious. How about setting up a Julia server and communicating with it via RESTful API from Java? Has anyone tried anything like that? I eventually want to run everything on AWS.

A bigger change would be to ditch Java all together and build everything either entirely in Julia or, I had the thought of building the plumbing in Rust and do the numerical modeling in Julia. Thus, capturing the best of both. Then, I'd be faced with the same question. How to call Julia from Rust? Is that crazy?

A possible solution for both scenarios might be to set up a Julia server and design a RESTful API. How does that sound? How about security? Could something like this be done if the data in sensitive?

I appreciate any thoughts and I really admire what you're doing with Julia. Reminds me of the glory days before I sold my soul to Wall Street (PhD computational electromagnetics, UIUC, 2002, MIT Lincoln Lab, 2002-2004). I'm considering trying to get back to academics to redeem myself before its too late though :)

Best regards,
Eric

[Viral Shah]

Offloading numerical computing to Julia with a RESTful interface, or something like zeromq is certainly the way to go. On JuliaBox, Tanmay has made it trivial to wrap computation in an API and serve it wherever you want. He should probably be announcing it shortly. If you are interested, we would love to have you kicking the tires. 

Other interfaces to look at could be IJulia and Jewel.jl.

-viral

[Stefan Karpinski]

Even though calling Julia through a REST API is certainly a viable approach, I think that loading the Julia runtime into Rust shouldn't be hard. See ui/repl.c for how the REPL loads and uses libjulia. This should be basically the same from Rust.

[Eric Forgy]

Thank you Viral and thank you Stefan.

The Rust solution would be an interesting and longer term effort because the person I would rely on to do most of the development is not yet ready to take the Rust plunge until it matures/stabilizes a bit more.

The other two ideas REST and zeromq look interesting and I'd like to learn more. I spent what little available time I had yesterday reading about message queueing with zeromq and others. That is very interesting stuff. I had a quick look at JuliaBox. That is pretty awesome :) I would love to understand how it works. Like I said, I am pretty much a blank slate and learning everything as I go, so looking at the JuliaBox code, it wasn't immediately obvious (.t? .lua? :)). Does there exist a simple document describing the process flow of how JuliaBox works?

In very general terms, could you help sketch out and point me to some reading material to help me make progress building a RESTful interface to Julia running on a server? Should it be built entirely in Julia? Is it already done in JuliaBox or elsewhere? I think initially, I will focus on REST. Another platform (OpenGamma) I'm working with also has a RESTful API so any effort I make will not be wasted even if I eventually start work with zeromq (which I probably will).

Thank you again.

Cheers,
Eric

[Steve Kelly]

I am currently building a path planner for 3D printers in Julia. We are also using a ZeroMQ interface to separate the web interface from the path planner. This is working very well for us now. We will also be using JuliaBox for packaging our application.

[Stefan Karpinski]

I can also attest that connecting a Julia process to other processes with ZeroMQ is a very good way to go – it's reliable and easy and quite efficient. ZMQ gives many side benefits too. Probably easier than developing a full-blown REST interface, imo.

[Eric Forgy]

Sounds good. I'll focus on zeromq first. Thank you :)

[Jerzy Głowacki]

For this purpose, I would deploy a web service in Julia, using e.g. http://juliawebstack.org/#HttpServer

[Stefan Karpinski]

My experience has been that using a full web stack is overkill for this kind of internal communication between processes. ZMQ let's you set up point-to-point, broadcast, anycast, pubsub, etc. messaging patterns really easily and Julia's built-in serialization and deserialization lets you send nearly arbitrary objects over the wire with no fuss.

[Eric Forgy]

Update:

I miraculously figured out how to use ZMQ in Julia, i.e. you type "using ZMQ" :)

More seriously, I managed to get jeromq working in Eclipse and started trying to get it to talk to Julia's ZMQ. I was trying to follow the examples in the zguide (which is awesome btw):

• http://zguide.zeromq.org/page:all
  

But had some difficulties. For example, it is not totally obvious how to implement the "Hello World" server in Julia. Here is the C version:

#include <zmq.h>

#include <stdio.h>

#include <unistd.h>

#include <string.h>

#include <assert.h>

int main (void)

{

// Socket to talk to clients

void *context = zmq_ctx_new ();

void *responder = zmq_socket (context, ZMQ_REP);

int rc = zmq_bind (responder, "tcp://*:5555");

assert (rc == 0);

while (1) {

char buffer [10];

zmq_recv (responder, buffer, 10, 0);

printf ("Received Hello\n");

sleep (1); // Do some 'work'

zmq_send (responder, "World", 5, 0);

}

return 0;

}

I got this far:

julia> using ZMQ

julia> ctx = Context(1)

Context(Ptr{Void} @0x00007fc8a25b8d60,[])

julia> s = Socket(ctx, REP)

Socket(Ptr{Void} @0x00007fc8a7ce2600)

julia> ZMQ.bind(s, "tcp://*:5555") 

This all seems to work. But I don't know how to implement the while loop properly. Any hints?

Best regards,

Eric

[Jake Bolewski]

I have some examples in my fork of ZMQ.jl here https://github.com/jakebolewski/ZMQ.jl.   I don't know  if they will work on the latest  ZMQ master.

[Eric Forgy]

Hi Jake,

That is awesome. Thank you for sharing. I'll give it a try next time I'm in my (second) office.

Assuming they work, would you consider adding them to the ZGuide? All examples in the ZGuide are given in many different languages, but there are no Julia examples. It would be a great exercise and a great contribution to implement all examples and add them there.

By the way, the Julia ZMQ and Java JeroMQ I have are based on different versioms of libzmq, i.e. 3.2.4 vs 3.2.2. I guess I'll find out next time I'm in front of my dev laptop, but should I be optimistic the two can still talk to each other?

Best regards,
Eric

[Uwe Fechner]

As far as I know they can talk to each other, as long as you use the same major version.

Uwe

[Eric Forgy]

After getting knocked over by the flu (or something heinous anyway), I managed to get my Java (JeroMQ) app to talk to Julia (ZMQ) using ZeroMQ. I needed to make some minor changes to the example code though, but it was still helpful. Thanks! Here is the revised "Hello World" server:

using ZMQ

# Hello world server

ctx = Context(1)

responder = Socket(ctx, REP)

ZMQ.bind(responder, "tcp://*:5555")

while true

  buffer = ZMQ.recv(responder)

  println("Received Hello")

  sleep(1)

  ZMQ.send(responder, "World")

end

[Job van der Zwan]

On Thursday, 18 December 2014 16:58:59 UTC+1, Eric Forgy wrote:

I miraculously figured out how to use ZMQ in Julia, i.e. you type "using ZMQ" :)

 
Well, the developers did say they want Julia to have all the good bits of Python