Julia vs Clojure for Distributed Scientific Simulations

874 views
Skip to first unread message

George

unread,
Jan 26, 2016, 11:07:55 PM1/26/16
to julia-users
I'm working on a scientific simulation that is going to require a distributed environment.  There has been some discussion about whether to use Julia or Clojure for this project.  A few micro benchmarks seem to have different results for each language.  I'm not yet sold as to which language may be more expressive in this situation, but Lisp might be a preferred option mathematically for modeling purposes.

Does anyone have any practical experience in dealing with both of these languages and what your experiences were?  

Are there any meaningful benchmarks that compare the two, especially in a distributed environment?


Thanks!

-George

Ismael Venegas Castelló

unread,
Jan 27, 2016, 10:53:13 AM1/27/16
to julia-users
How are you benchmarking the Julia code? If you share the microbenchmarks you are doing, we can help you with that, the final decision is yours. Julia es actually a lot like lisp but with "normal" syntax, support for unicode (lots of mathematical symbols also) see: 


Julia has very good support for parallel and distributed computing: 

* http://julia.readthedocs.org/en/latest/manual/parallel-computing

True multithreading is coming as an experimental feature for v0.5:

* https://github.com/JuliaLang/julia/labels/multi-threading

As a side note Julia parser is implemented in Scheme; Femtolisp to be precise, try:

$ julia --lisp    # :D

And also there is LispSyntax.jl currently in alpha development, it may not be in time for your work, but suggestions, bug reports and pull requests are always welcome:

* Code: https://github.com/swadey/LispSyntax.jl

I haven't seen any distributed benchmark between Julia and Clojure, be the first to make one, we are here to help each other!

I'm still learning Lisp, Scheme, Clojure, Hy, Pixie, etc. in order for me to contribute to this project.

finally I'm not sure myself, but I have only seen reference to Clojure's Incanter and core.matrix as libraries geared towards scientific computing, in that regard the Julia community is very focused on scientific computing (altough julia is not a domain specific language, it's actually general purpose), here are some Julia videos (some are dated) related to distributed computing:

* Cluster managers and parallel Julia: https://www.youtube.com/watch?v=XJAQ24NS458
* Distributed memory direct linear algebra and optimization: https://www.youtube.com/watch?v=x-93-IDFdxM
* Distributed algorithms for full-waveform-inversion: https://www.youtube.com/watch?v=k71RYq2eK50
* Using Julia on a Cray Supercomputer: https://www.youtube.com/watch?v=NwyKz2KLdtY

Ismael Venegas Castelló

unread,
Jan 27, 2016, 10:57:55 AM1/27/16
to julia-users
Correction, none of those videos are dated "dated", they are from the las JuliaCon, I removed the old videos from the list but forgot to remove the comment of some being dated.

Also check out the Julia package ecosystem, there are lots of scientific computing libraries:

Mike Innes

unread,
Jan 27, 2016, 11:15:10 AM1/27/16
to julia...@googlegroups.com
With Clojure you're likely to get much better deployment / networking support, as well as the general robustness and tooling of the JVM. It's also really expressive for data manipulation (though not necessarily fast). Julia loses out on that but will blow Clojure out of the water for anything more computationally advanced; numerics, fiddly data structures etc.

To over simplify the decision a little, I'd probably use Clojure for something running indefinitely (e.g. web server) and Julia for something finite (e.g. a simulation). But I think you just have to look at what you expect the key pain points to be, and compare that to the strengths of each language.

Ismael VC

unread,
Jan 27, 2016, 1:11:22 PM1/27/16
to julia...@googlegroups.com
I dont know if you can call Julia from Java, but you can call Java from Julia, see:

* https://github.com/aviks/JavaCall.jl
* https://github.com/aviks/JavaCall.jl

Ismael Venegas Castelló

Data Analyst

Cel. 044 55 6434 0229

iven...@richit.com.mx

Cerro San Francisco 357, C.P. 04200

Campestre Churubusco, Coyoacán

Ciudad de México

  

Tel. 6718 1818

richit.com.mx

Joshua Ballanco

unread,
Jan 27, 2016, 2:34:13 PM1/27/16
to julia...@googlegroups.com
Last I checked, I think the only real option for calling Julia from Java is via JNA to the Julia C API. Not impossible, but not as convenient as JavaCall.jl.

As for Julia vs Clojure, I had a go at implementing a simple OCR nearest-neighbors algorithm in both Julia and Clojure. The Clojure code came out a bit shorter, and was moderately more expressive, but required almost 100x the runtime. Granted, I haven’t yet taken the time to add type-hints to the Clojure version, which could explain some of the difference. I suspect the biggest issue is the lack of a good dataframes library for Clojure, though you could pull in, for example, Spark’s dataframes via Clojure’s very convenient Java interop.

That actually brings up another point that some others have mentioned in this thread. Clojure benefits from the entire Java ecosystem. Of course, the more Java you bring into your Clojure, the less you benefit from the Lisp-y-ness of Clojure. You should check, though, as there is already a healthy ecosystem of wrappers that bring back some of that Lisp goodness (e.g. https://github.com/gorillalabs/sparkling for Spark). So it’s trade-offs all around.

One last point (something I’ve been meaning to look at but haven’t had the time): reading CSV files off disk was about 10x faster in Clojure than Julia. Something else to consider.

Mauro

unread,
Jan 27, 2016, 3:34:43 PM1/27/16
to julia...@googlegroups.com
On Wed, 2016-01-27 at 20:34, Joshua Ballanco <jbal...@gmail.com> wrote:
> One last point (something I’ve been meaning to look at but haven’t had
> the time): reading CSV files off disk was about 10x faster in Clojure
> than Julia. Something else to consider.

I think this package makes this much faster:
https://github.com/JuliaDB/CSV.jl

> On January 27, 2016 at 20:11:22, Ismael VC (ismael...@gmail.com) wrote:
>
> I dont know if you can call Julia from Java, but you can call Java from Julia, see:
>
> *https://github.com/aviks/JavaCall.jl
> *https://github.com/aviks/JavaCall.jl
>
> Ismael Venegas Castelló
> Data Analyst
> Cel. 044 55 6434 0229
> iven...@richit.com.mx
> Cerro San Francisco 357, C.P. 04200
> Campestre Churubusco, Coyoacán
> Ciudad de México
>
>
Reply all
Reply to author
Forward
0 new messages