Clojure targeting LLVM / C

[Jason Felice]

Hi!

I joined up here because I'd love to help on a Clojure that targets native (either via C or LLVM).  I heard there was an LLVM backend being thought about, but I haven't heard anything about it.

Is anyone working on this?  Has it been forgotten?

-Jason

[Linus Ericsson]

There is one version that compiles Clojure to scheme, which can be compiled to C which can be compiled to binary code:

https://github.com/takeoutweight/clojure-scheme

also checkout Mjolnir and Clojure-metal with similar aims but for LLVM.

https://github.com/halgari/mjolnir/
https://github.com/halgari/clojure-metal

There's also some projects that aims to compile clojure-code to CUDAs in

https://github.com/JulesGosnell/clumatra

and others.

Several of these are mentioned in:
https://groups.google.com/forum/#!topic/clojure/PsgKVlWZjOw

Beware however that many (but far from all) of the use cases for compiled-to-c-clojure is possible to address with ClojureScript + quick, possibly embedded, Javascript VM.

/Linus

--
You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure-dev...@googlegroups.com.
To post to this group, send email to cloju...@googlegroups.com.
Visit this group at http://groups.google.com/group/clojure-dev.
For more options, visit https://groups.google.com/d/optout.

[Paul deGrandis]

There's also:

ClojureC - https://github.com/schani/clojurec
Clojure-ObjC - https://github.com/galdolber/clojure-objc
An old but mostly functional ClojureScript-Lua - https://github.com/raph-amiard/clojurescript-lua
And my (mostly complete) ClojureScript-Terra - https://github.com/ohpauleez/cljs-terra

As stated before, it's not entirely desirable to compile down to native (embedding V8/Node or LuaJIT is an option, The JVM is rock solid with an HPC ecosystem, Mjolnir let's you generate LLVM stuff if you that's something you need)

What is the problem scenario you're facing?  Why is native compilation on the radar?

Cheers!
Paul

[Gary Trakhman]

I personally would be having a lot of fun with a native clojure if it offered an easy FFI.  But cljs+node-webkit is working well for my current use-case of native-packaged apps, and it seems like their FFI's decent if I ever actually needed it.

[Timothy Baldridge]

I've devoted more time to this than I care to admit, and it's my conclusion that two of the biggest barriers to this sort of things are: a) fast polymorphism b) fast GC. These are two things that the JVM excels at. 

I've spent a fair amount of time writing Clojure like programs in C++, and the performance was quite sub-par. Let's take (for example) a classic lazy seq:

(def numbers [val]
  (cons val (lazy-seq (numbers (inc val)))))

Assuming your system will be boxing integers, there are about 4-5 allocations that happen every for every "slot" of this sequence. 1 cons, 1 lazy seq, and about 2-3 boxes. 

Next let's say we're doing a simple "count" operation on this seq, now we have about 3-4 polymophic calls, for "first", "next" and calling "add" on the internal counter of count. All this means that for a single cell you're looking at about 4 allocations and 3 polymorphic calls. Implement this in C++ and you'll see how insanely slow it is. 

So the JVM (and LuaJIT) gives you quite a lot of leverage when it comes to this sort of thing. The JVM will remove unneeded polymorphisms, and inline functions where possible. The JVM GC is also very good at short term allocations (like our example produces). 

All that to say, in all my experiments, I have been unable to come within about 2x the performance of Clojure on the JVM. And most of my tests are barely hitting 10x slower than the JVM. 

So a "native clojure" for performance reasons has yet to be proven possible in my mind. Unless you plan on writing a custom JIT, and that's a whole other ball of wax. 

Personally I think Graal and Truffle are promising, but even then Java is wicked fast, so I'm not sure I'll ever be motivated to look into them much. 

Timothy

--
You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure-dev...@googlegroups.com.
To post to this group, send email to cloju...@googlegroups.com.
Visit this group at http://groups.google.com/group/clojure-dev.
For more options, visit https://groups.google.com/d/optout.

--
“One of the main causes of the fall of the Roman Empire was that–lacking zero–they had no way to indicate successful termination of their C programs.”
(Robert Firth)

[Jason Felice]

On Tue, Apr 29, 2014 at 11:17 AM, Paul deGrandis <paul.de...@gmail.com> wrote:

ClojureC - https://github.com/schani/clojurec
Clojure-ObjC - https://github.com/galdolber/clojure-objc
An old but mostly functional ClojureScript-Lua - https://github.com/raph-amiard/clojurescript-lua
And my (mostly complete) ClojureScript-Terra - https://github.com/ohpauleez/cljs-terra

Neat!

 

What is the problem scenario you're facing?  Why is native compilation on the radar?

 

Mostly because I'm very familiar with the environment and tools - there's an ecosystem of C libraries that I know how to leverage very well.  Some of them never migrated into Java land.

Example: I'm writing Avi (a lively vim-like editor).  I'm using JNI for access to the curses library, after pure Java solutions (lanterna and a few others) didn't pan out.  At the moment, the startup time isn't bad, but with the JVM, the Unix-y "many small tools" philosophy gets harder.  The time it takes for "lein" to print command help or start a REPL are both cases in point.  So I think a native backend is useful for light tools.

Another part was that handling of screen rendering in Avi required some pretty tight optimization to make it run faster than the keyboard repeat rate.  I've been able to get this working in the JVM by guessing that System.arraycopy() and Arrays.fill() use the hardware's underlying vector operations, but this is really deferring the pain because it relies on the screen content being a lot less complicated than I expect it to be.  While I expect _polymorphism_ to run better on the JVM, I expect to have more control over performance for pure byte manipulation on native.  For this project, I'd guess this will be the only thing that needs to perform well (at least for quite a while).

I'm also very interested in embedded environments - I'd love to have Arduino-type support.

[Akos Gyimesi]

Hi All,

 

I am also interested in a native compiler - in fact, just as Jason, I also signed up in the hope that one day I can help in such a project.

 

I work with a lot of C code, and it would be great to glue the pieces together with a high-level language. For example: we're writing a real-time video transcoding service in C. While it would be hard to imagine the transcoding part on the JVM or on V8, it makes perfect sense to write the controlling logic (starting/stopping jobs, reporting status, etc) in Clojure. In this case I don't care if this code is 10x slower. What I do care about is that it should be free from deadlocks/race conditions, and it should be easy to write and test.

 

Another use case is that I would like to write lightweight tools: log aggregators, monitoring tools, etc. This is also an area where CPU usage does not matter that much - but memory consumption does. I cannot start a JVM on an Amazon micro instance just to report the CPU usage to another service.

 

There is another reason why I would personally like to see a close-to-metal implementation: I admire Clojure because it is the first language that I know of that is actively used with very different programming paradigms: we have core.typed, core.logic, core.async - and they are not just toy projects, they are actively used in production. We conquered most of the "high-level" programming paradigms with Clojure. So what would happen if we conquered the low-level as well? Is it really the case that a video codec can be only written in C + Assembly? Wouldn't it be easier to translate a codec specification with the power of macros to a low-level language (e.g. a mjolnir-like code) than manually writing it in these languages?

 

Akos

[Colin Fleming]

cljs-terra looks really interesting, Paul. I loved the concept of Terra when it came out, it's a really nice mix of high and low level - a cljs wrapper over that could be really nice.

[jimduey]

Hey Jason,

I'm working on a Clojure for LLVM but it's not to a point I can go public with it yet. Hopefully it'll be self hosting soon and then I'll open it up.

As Timothy said, performance isn't a driving reason. It's more about not having to use the JVM and a chance to explore some other ideas in a Clojure-like language.

Jim