Clojure on CLR/DLR

[dmiller]

[I thought I'd slip this in while Rich has everyone distracted lazy
sequences.]


What do you do when you love Lisp, are intrigued by Clojure, but have
absolutely no projects at hand to test it out? Oh, and you have an
interest in how dynamic languages are being implemented in modern
virtual machine environments.

You might implement Clojure on the CLR, I guess. So I did. At least,
I have started.

The project has reached the point where I need input from the
community and especially Rich. Rich asked that I go public. So I
will. (Though most certainly I would prefer to wait until the code
is ... better.)

The code will go up on clojure-contrib ASAP. I need input from the
clojure-contrib project members on how they operate, where they want
to put it, etc.

This is definitely alpha-level, developer-only. Hard hats, goggles,
and heavy gloves recommended.


Goals:

(a) Implement a feature-complete Clojure on top of CLR and the DLR
(Dynamic Language Runtime).
(b) Stay as close as possible to the JVM implementation so that the
versions can stay in synch. That includes:
(c) To the extent possible, use exactly the same boostrap *.clj files
to define the environment.
(d) Try to use some of the more advanced features of the DLR, where it
makes sense to do so.
(e) Spawn a cottage industry of people making Visual Studio extensions
for Clojures. :)
(e) Have fun.


Status:

Let's call it alpha. Not for the casual user, but developers might
want to take a look.

Stop here unless you really want status details.


The basic runtime data structures--the persistent collections,
namespaces/symbols/keywords, vars, refs, atoms, agents, etc.--are 95%
complete. The remaining pieces are trivial.

The LispReader is feature-complete except for #=. Call this 96%
complete.

There are currently 500+ unit tests on the basic data structures and
the reader. Call this 34% complete.
There are no unit tests on compiler. That would be 0%.

This is not an interpreter. A compiler translates Clojure forms into
DLR expression trees. DLR does its magic, i.e. compiles to MSIL, and
computation happens. At the moment, there is not a single call to
Reflection.Emit in the compiler code. (Though that will have to
change soon.) It handles all special forms, can load most of
core.clj, deals with macros, inlines, tags, CLR-interop,

There is a basic REPL.

core.clj loads with minor edits (java.util.Collection =>
System.Collections.ICollection, for example). Of roughly 425 def
forms in core.clj, only 43 are commented out. 19 of those are for
specialized array access, 6 for regular expressions -- nothing
significant in terms of work. Of the 360 defs that load, most have
been tested at a rudimentary level. However, the roughly 20 defs
related to libs/loading/compiling definitely do not work. Call this
85% complete.

core-print.clj also loads and seems to work. ants.clj runs. (The sim
code is unchanged. The GUI was rewritten to Windows Forms.) 100%.

The code is consistent with revision 1279 of the JVM code (Feb 13).
It is running on release 10606 of the DLR (Feb 11).

It is slow. Roughly 4X slower than Clojure/JVM on one or two very
easy tests. (No need to comment on microbenchmarks.) This will be the
focus of the next round of work.


What's not there:

Libs and loading -- lots of design problems because of the differences
between JVM and CLR relating to classpaths, assemblies, etc. (Basic
file loads can be done.)

AOT/Compilation/gen-class

Proxies

Bootstrap *.clj files other than core.clj and core-print.clj -- Just
haven't had time yet.

Speed.


So there you have it. I'll post again when the code becomes available
on clojure-contrib.

-- David Miller

[Jeffrey Straszheim]

Awesome!

[Chouser]

On Mon, Feb 16, 2009 at 5:43 PM, dmiller <dmill...@gmail.com> wrote:
>

> The code will go up on clojure-contrib ASAP. I need input from the
> clojure-contrib project members on how they operate, where they want
> to put it, etc.

I don't know if you've looked at ClojureScript at all, but it's a
similar if noticeably less ambitious project to compile Clojure code to
JavaScript. It's in clojure-contrib already, but in
trunk/clojurescript instead of trunk/src. My reasons for this were
(1) I wasn't quite sure how to lay out the directory structure and
didn't want to mess up anyone else, and (2) it doesn't work with
Clojure trunk but instead requires a patch and rebuild of Clojure
itself. This patch is stored right there in contrib as well.

I think I've worked out (1) well enough, but (2) is still a factor.
As it is, I never get complaints from people trying to use it with an
unpatched Clojure and getting errors. Then again that may have more
to do with the inscrutability and lack of docs. :-) Once Rich has
applied the patch and ClojureScript can work with a regular build of
Clojure (or at least as well there as anywhere), I plan to move the
files into trunk/src/clojure/contrib/clojurescript

I don't know if that same idea would be appropriate for your project,
but it does make the code accessible without people using it
accidentally.

> The basic runtime data structures--the persistent collections,
> namespaces/symbols/keywords, vars, refs, atoms, agents, etc.--are 95%
> complete. The remaining pieces are trivial.

Fantastic! For ClojureScript, at least, that has been the bulk of the
work.

> core.clj loads with minor edits (java.util.Collection =>
> System.Collections.ICollection, for example). Of roughly 425 def
> forms in core.clj, only 43 are commented out. 19 of those are for
> specialized array access, 6 for regular expressions -- nothing
> significant in terms of work.

This is the majority of what the ClojureScript patch changes -- moving
explicit uses of non-Clojure Java class names out of .clj files and
into clojure.lang.RT (or other appropriate Clojure classes) so that
the .clj can be loaded as-is. RT and Numbers have to be ported by
hand anyway, so it's not significantly worse on that end.

I'd be very interested to compare notes and see if our needs have a
common solution.

> So there you have it. I'll post again when the code becomes available
> on clojure-contrib.

Sounds great! But there's one very important question you didn't
address. What are you going to call it? :-)

Seriously, though, since it seems likely that a majority of code
written to run on your port will not work on Clojure/JVM, because of
the runtime libs available (please correct me if I'm wrong), it's
important for a body of code to be able to clearly declare where it's
supposed to work. A name that is used consistently can help, I would
think.

--Chouser

[dmiller]

On Feb 16, 5:33 pm, Chouser <chou...@gmail.com> wrote:

> On Mon, Feb 16, 2009 at 5:43 PM, dmiller <dmiller2...@gmail.com> wrote:
>

> I don't know if you've looked at ClojureScript at all, but it's a
> similar if noticeably less ambitious project to compile Clojure code to
> JavaScript. It's in clojure-contrib already, but in
> trunk/clojurescript instead of trunk/src. My reasons for this were
> (1) I wasn't quite sure how to lay out the directory structure and
> didn't want to mess up anyone else, and (2) it doesn't work with
> Clojure trunk but instead requires a patch and rebuild of Clojure
> itself. This patch is stored right there in contrib as well.
>

I have looked briefly at ClojureScript.

Placement: I'm guessing a parallel off-trunk placement. This code is
completely independent of Clojure/JVM, except for the bootstrap *.clj
files. I have those included in the project, so I'm not broken by
Clojure/JVM changes.

Also, this code is not set up for casual play. You need to be in
Visual Studio, download the DLR, connect Tab A to Slot B, etc. I'm
thinking it should not be in trunk/src by the criteria you cite.

> This is the majority of what the ClojureScript patch changes -- moving
> explicit uses of non-Clojure Java class names out of .clj files and
> into clojure.lang.RT (or other appropriate Clojure classes) so that
> the .clj can be loaded as-is. RT and Numbers have to be ported by
> hand anyway, so it's not significantly worse on that end.
>
> I'd be very interested to compare notes and see if our needs have a
> common solution.
>

I need to make the same kinds of changes to the *.clj files. This has
not been automated yet, so being in synch is a matter of hand-
editing.

We most definitely need to compare notes.

> Sounds great! But there's one very important question you didn't
> address. What are you going to call it? :-)
>
> Seriously, though, since it seems likely that a majority of code
> written to run on your port will not work on Clojure/JVM, because of
> the runtime libs available (please correct me if I'm wrong), it's
> important for a body of code to be able to clearly declare where it's
> supposed to work. A name that is used consistently can help, I would
> think.
>

This is 100% C#/.NET.

I'm up for suggestions on the name. The obvious ones:

- Clojure.net
- ClojureCLR
- IronClojure (paralleling IronPython/IronRuby, unless MS has Iron
trademarked.)
- CLjR (too cute)

Perhaps Rich will have a preference. He'll have to live with it
longer than anyone and has branding/confusion issues to keep in mind.

-- David

[Rich Hickey]

I prefer ClojureCLR.

Rich

[Rayne]

Anything buy IronClojure.

[Dan]

On Mon, Feb 16, 2009 at 11:15 PM, Rayne <discip...@gmail.com> wrote:

Anything buy IronClojure.

There's already an IronLisp anyway:

http://www.codeplex.com/IronLisp

[Sean]

David,
You have a great idea here with porting clojure to the CLR. The .NET
shops are just a popular as Java shops, and something like this could
go a long way to improving software written by a lot of people. Your
initiative and hard work are to be commended.

How do you plan on solving the library problem? As you mentioned,
there are several discrepancies between .NET and Java libraries.
Also, there will be applications written for a certain library that
might only be available in one of the platforms. I know I have
several .JARs that work with legacy applications ( like SAP &
Oracle :) ) that I need to use. If some else needs to maintain them,
I wouldn't want them to accidently try to run it on the wrong
platform. If it can go wrong...

Of course, this is a problem open source systems have had for a while,
so it can be solved (apt,ASDF,rpm,rubygems...). It'll just take some
work.

Good work overall, I'm looking forward to what you come up with.

[dmiller]

ClojureCLR it shall be.

--dm

On Feb 16, 7:30 pm, Rich Hickey <richhic...@gmail.com> wrote:
>

> I prefer ClojureCLR.
>
> Rich

[dmiller]

Porting Clojure to the CLR is hardly an original idea. Rich started
with dual JVM/CLR implementations. And inquiries have been made on
this group any number of times.

Regarding the library problem, I'm not exactly sure which problem you
are referring to. Several things comes to mind.

At the implementation level, there is the problem of loading the *.clj
environment bootstrapping files. A common approach for Clojure,
ClojureScript and ClojureCLR would push some JVM library calls in
core.clj (for example) into RT. Type hints via tag metadata will
require a more creative solution. Also, some Clojure API functions
are JVM-specific and would need some modification. Those decisions
are above my pay grade. At any rate, these problems should not
surface to application programmers.

At the application level, interop from .clj code with Clojure library
classes should be portable because I've matched namespace/classname/
methodname pretty much across the board. If you have a lot of interop
with JVM/CLR class libraries, I would imagine you would pick one and
stick with it. In your example, if you have legacy JARs you need to
interact with, you're in Clojure on the JVM.

-- David

[Lucio Fulci]

I can see a minor problem with ClojureCLR, that is, "j" in clojure stands for JVM, right? So it's a bit messy.

[Laurent PETIT]

"Clonure" (n for dot *n*et), as in : "Clonure, a dot net clone of Clojure"

(ok, sorry ;-)

2009/2/17 Lucio Fulci <luciof...@gmail.com>

[Rayne]

Haha. I just noticed my typo in the previous post. Disregard that. :|

On Feb 17, 3:22 am, Laurent PETIT <laurent.pe...@gmail.com> wrote:
> "Clonure" (n for dot *n*et), as in : "Clonure, a dot net clone of Clojure"
>
> (ok, sorry ;-)
>

> 2009/2/17 Lucio Fulci <luciofulc...@gmail.com>

[AlamedaMike]

Fantastic news, David. This should help the spread of Clojure.

Although I like "Bonjure" as a name, and even though two syllable
names are generally considered best by marketers, I think ClojureCLR
is best for branding purposes. It helps spread the Clojure meme and it
linguistically supports the claim that the language is cross-VM.

The world needs a good Lisp that runs on popular VMs. Looks like we're
getting there. Thanks for your hard work.

[Timothy Pratley]

> "Clonure" (n for dot *n*et), as in : "Clonure, a dot net clone of Clojure"

hahaha nice :) In the same vein:
Clocure (c for clr) pronouced closer or cl oh cure (is that the cure
for common lisp or CLR?)
*chuckle* Ok ok I'm not being serious.

[Stefan Rusek]

I've been working on Xronos which is also a c# version of clojure (I
need to be careful to not use the work port, since it doesn't share
any code with clojure). It compiles to the DLR as well. It is located
here:

http://www.bitbucket.org/stefanrusek/xronos/wiki/Home

One big difference is that Xronos is released under the BSD license
instead of the Eclipse Public License.

--Stefan Rusek

[mikel]

On Feb 16, 11:35 pm, dmiller <dmiller2...@gmail.com> wrote:
> Porting Clojure to the CLR is hardly an original idea.  Rich started
> with dual JVM/CLR implementations.  And inquiries have been made on
> this group any number of times.

Yah, I posted one of those queries. My interest is future-proofing
Clojure, because I like it best of any programming language I've used
in 15 years, am staking the success of one of my projects on its
continued vigor, and have already suffered once through the untimely
extinction of a favorite programming language.

I have a tiny side project called coccl ("Clojure on Clozure Common
Lisp") which is nowhere near as far along as ClojureCLR; its purpose
was purely to test whether I could see how to implement Clojure on top
of some runtime other than the JVM--that is, I never had any
particular intention of actually making it work, I just wanted to
satisfy myself that it's possible, and I think I managed that. (Don't
ask for the coccl code because I won't give t to you; it's not far
enough developed to be useful for any purpose other than the one I
described).

The reason I bring up coccl at all is that the perspective of having
done a little work on it motivates me to ask something of you: that
you document your process of implementing Clojure CLR for the benefit
of future implementors. It would be very valuable to have notes one
where the biggest efforts have been, what parts were tricky (and which
ones were tricky because of the requirements of the language, versus
because of quirks of the runtime you chose to host it). It would also
be helpful to have notes that would help later implementors organize
their efforts--how do you break down the implementation of Clojure
into sensible subproject?

Obviously, notes from Rich along those lines would be extremely
valuable as well.

[Craig Andera]

> I'm up for suggestions on the name. The obvious ones:
>
> - Clojure.net
> - ClojureCLR
> - IronClojure (paralleling IronPython/IronRuby, unless MS has Iron
> trademarked.)
> - CLjR (too cute)
>
> Perhaps Rich will have a preference. He'll have to live with it
> longer than anyone and has branding/confusion issues to keep in mind.

So, as a long-time .NET guy, IronClojure seems like the best name, in
terms of making it obvious what it does: it's like IronRuby/Python,
but it's Clojure. Failing that, it seems like NClojure fits the
pattern of other JVM-ported efforts. I realize that there's already an
Enclojure.

Just to throw more chaff into the air:

* CoCLR: Clojure on the CLR. (Maybe pronounced cochlear?)
* Coc: Same as above.
* CoNET: Clojure on .NET.
* Icon: An Implementation of Clojure on .NET.
* Ichor: I can't think of an acronym here, but I want to. :)

[mikel]

There is an existing programming language named Icon, developed by
Ralph Griswold (the same guy who developed SNOBOL). Icon has been
around for decades, and is an interesting language in its own right.

[Dan]

> So, as a long-time .NET guy, IronClojure seems like the best name, in
> terms of making it obvious what it does: it's like IronRuby/Python,
> but it's Clojure. Failing that, it seems like NClojure fits the
> pattern of other JVM-ported efforts. I realize that there's already an
> Enclojure.
>
> Just to throw more chaff into the air:
>
> * CoCLR: Clojure on the CLR. (Maybe pronounced cochlear?)
> * Coc: Same as above.
> * CoNET: Clojure on .NET.
> * Icon: An Implementation of Clojure on .NET.

There is an existing programming language named Icon, developed by
Ralph Griswold (the same guy who developed SNOBOL). Icon has been
around for decades, and is an interesting language in its own right.

> * Ichor: I can't think of an acronym here, but I want to. :)

As far as I understood, the rules are that it should be derived from Clojure and sports either an N or a CLR. So I suggest Conjure

It looks like clojure, sounds pleasing, and sounds lispish (conj). And Lisp to me sounds like magic (in the Arthur C. Clarke meaning that it is a technology sufficiently advanced that it is indistinguishable from it).

[Matt Revelle]

+1

Does the following seem correct?
The language is Clojure, it's canonical implementation is on the JVM.
All other implementations are also Clojure.

Leaving out Clojure from the implementation name may encourage a "one
language, many implementations" notion. Not saying that's desirable,
only pointing it out.

-Matt

> >

[Craig Andera]

> As far as I understood, the rules are that it should be derived from Clojure and sports either an N or a CLR. So I suggest Conjure
>
> It looks like clojure, sounds pleasing, and sounds lispish (conj). And Lisp to me sounds like magic (in the Arthur C. Clarke meaning that it is a technology sufficiently advanced that it is indistinguishable from it).

Ooh - that's much better than mine. :)

+1!

[Rich Hickey]

The whole point of including David's work in contrib is to give people
confidence that it is, in fact, Clojure, derived from the original
sources, as is ClojureScript. Rebranding it is counterproductive.

This will be a supported effort, part of the Clojure project, and a
product of Clojure's community. CLR is a qualifier, Clojure is the
brand.

Rich

[Michel Salim]

Coc sounds similar to Coq, a theorem-prover system for Ocaml.

I'm actually fond of CLjR, but oh well :)

--
miʃel salim • http://hircus.jaiku.com/
IUCS • msa...@cs.indiana.edu
Fedora • sal...@fedoraproject.org
MacPorts • hir...@macports.org

[dmiller]

My thanks to Rich for the suggestion to go public and for agreeing to
include this as part of the Clojure community.

Thanks to all for the encouragement.

-- David

[Johan Berntsson]

On Feb 17, 9:17 am, dmiller <dmiller2...@gmail.com> wrote:
> Also, this code is not set up for casual play. You need to be in
> Visual Studio, download the DLR, connect Tab A to Slot B, etc.  I'm
> thinking it should not be in trunk/src by the criteria you cite.

I'm really looking forward to a CLR version of Clojure, but I don't
like the sound of this. I hope we will be able to use ClojureCLR
without Visual Studio later on, and mono support wouldn't hurt either
(the DLR has already been ported to mono, so it should be possible).

/Johan

[Marko Kocić]

If only there were C/C-- port of clojure which will keep all of
clojure ideas and instead of java use '.' for easy access to C/C++
libraries/functions. And it should also be able to compile to native
code and create native executables.

Just dreaming...

[dmiller]

When the rough edges are filed off, it should distributable as a set
of DLLs (and a console EXE) like any other .NET application. It
should be able to follow the DLR to Mono.

[Marko Kocić]

On 18 феб, 15:13, dmiller <dmiller2...@gmail.com> wrote:
> When the rough edges are filed off, it should distributable as a set
> of DLLs (and a console EXE)  like any other .NET application.   It
> should be able to follow the DLR to Mono.
>

You mean DLR can create executables that don't need .NET runtime?

[dmiller]

"like any other .NET application" means "needs the .NET runtime.", but
does not need Visual Studio or inserting Tab A into Slot B to get it
all to work.