ZeroMQ and Mongrel interfaces

[Avik Sengupta]

I have been working on creating an interface to ZeroMQ and Mongrel2 in Julia. The code is here: https://github.com/aviks/julia-zmq-mongrel ... I doubt this should go into the standard library at this stage, hence a separate repo. 

ZeroMQ is a networking library provides what looks like a simple socket interface, but implements a series of higher order network functions over it. For example, you get pub/sub or request/response primitives built in, and it handles connections and reconnects automatically. 

Mongrel2 is a language agnostic web server, built on ZMQ, that allows one to easily write web service handlers in any supported language. You can therefore write a handler for a web request in a couple of lines of julia, and get a production quality web server to host it. You also get, for free, the ability to run a cluster of such handlers fronted by a single mongrel instance. 

I realise the web repl is done with an SCGI wrapper, but having never run scgi without problems in the past, I wanted a second path to the web for Julia code :). Eventually, I imagine someone will do a mod_julia for apache, and there'll be a full web server written in Julia itself (there's an effort at https://github.com/chzyer/JuliaWebServer I saw), but for the moment mongrel will provide a robust production quality web server for julia code. 

I'd appreciate any feedback on this. 

Regards

-

Avik

[Viral Shah]

Currently, the web repl is unmaintained. I feel that a bit of experimentation is needed before we settle on the right thing. Maybe Stephan will join us back sometime. :-)

Do you know if ZeroMQ is portable to mac, windows, and linux? The higher order features are really handy when we do more work on the distributed version. How does this compare to libuv?

-viral

[Avik Sengupta]

Yes, ZeroMQ (http://zeromq.org) is very portable. It comes with a Visual C++ build file. There is even an OpenVMS port, as well as android and iOS builds

Indeed, its the higher level primitives that are attractive. Real world networking code spends a lot of effort on these things... queuing, connecting, subscriptions,  etc, all of which you get cheaply or free with zmq.  One of the things I eventually want to get to is an RServe (http://stats.math.uni-augsburg.de/rserve/) like interface, for submitting julia jobs from other languages. 

I would imagine ZMQ is  complimentary to libuv, at different levels of abstraction. 

BTW, as part of this, I wrote a  pure julia JSON parser (the web ui uses a c++ library that I couldn't figure out how to access), which may make sense to add to the standard library at some point. 

Regards

-

Avik