Porting libraries to ClojureCLR

[David Jagoe]

G'day all,

I am using ClojureCLR (thanks everyone involved, clojure makes me
happy!) to develop a Windows application. I need a json library (and
don't really want to directly use the native C# one) so it seems like
the best option will be to port an existing Clojure library to the
CLR. Basically it should just mean replacing the Java stream reader
etc with the CLR equivalent. Has anybody done this or something
similar? Any advice on approach? Should I think about structuring
imports so that we can potentially merge my changes upstream to have a
library that supports JVM + CLR? Or just fork and have 2 libraries?

Thanks,
David

[Tatiana Racheva]

Yeah, I was thinking the other day about the divergence of the CLR and the JVM Clojures in general.

Let's keep as many things as possible in common. 

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

--

Thanks,

Tatiana

[Tatiana Racheva]

Not sure what "structuring imports" would entail. Do we need to use something like http://www.ikvm.net/devguide/net2java.html?

--

Thanks,

Tatiana

[David Jagoe]

On 24 November 2010 02:11, JMatt <jm...@jmatt.org> wrote:
> The easiest way to prevent divergence is to write and use native
> clojure libraries.

I totally agree. However because of clojure's excellent interop
capabilities it is extremely common (in fact encouraged) for
developers to directly use Java code. For example the json library I
am porting (https://github.com/danlarkin/clojure-json) directly uses
java.IO.StringReader etc.

I currently plan to do what you have suggested - implement a single
API using both Java and CLR components by means of defprotocol and
extend-type. It should work for this simple case. But actually I think
that the better solution would be for the json library to have no
VM-specific code and instead rely on lower-level clojure libraries to
provide the reading/writing/buffer capabilities. It is those libraries
which should switch implementation based on VM.

Another interesting problem is type hints. CLR equivalent to Integer
is Int32, so for my first stab I have broken java compatibility by
directly replacing #^Integer with #^Int32. Ultimately I will have to
create a new type (IntegerType) and have the implementation switch
based on the VM.

> I know I'd prefer to use clojure
> if I had to write .NET code some time in the future.

Yeah, it makes a huge difference!

For anyone that's interested, the current port is at
https://github.com/davidjagoe/clojure-json

but please note that I have temporarily broken java compatibility and
unicode is not supported at the moment because the .net readers don't
have an equivalent to java's read() which does the right thing wrt
chars... .net treats everything as a byte. Boo.

Cheers,
David

[nickik]

I thought about that a little too. The thing is that clojure is doing
something almost imposible. There is almost know way around writing
you json library backand in clr interop.

If we want to make clojure programm that run on the clr, the jvm and
possibly in the browser we have to write pure clojure but not using
native stuff is against the clojure way.

So how can we solve this?

I think we need a kind of tagsystem. With the tags you could indicate
if a library is pure clojure or if it uses lower level stuff. For
librarys that use jvm we have to define a interface with the clr
version can mirror. That way we would at least not need to change code
that runes on these librarys. That would need lots of communication
between the clr and jvm people. But there will always be code that
only runes on one host. You cant (trivially) call hadoop on the clr
for example.

The other possibility is to develop to diffrent ecosystems but that
would make it even harder to switch from one to the other.

In the long run the librarys that are writen in pure clojure will
hopfully become the standard.


---------------------------------------------

The same problem we have with the typesystem. No idea how to solve
that.

[Ken Wesson]

I gave some thought to the Integer vs. Int32 issue.

I think what would help here is a way to alias classes when :importing them.

It would require tweaking the ns macro and possibly parts of the
runtime. The outcome would be to allow something like

(ns foo
(:import (java.util.concurrent
AtomicBoolean (ConcurrentHashMap :as CHM) AtomicInteger)))

(defn make-default-chm []
(CHM. 12 12 12))

...

Handy so far for abbreviating some longer classnames, and the
application to CLR portability is apparent:

(ns foo
(:import (System (Int32 :as Integer))))

In theory, you'd just add this import to make code full of ^Integer
hints compile under ClojureCLR. You'd still need two versions of the
source file though, a CLR version with the import and a JVM version
without.

But then why not go even further and have the ns macro able to tell
what platform it's on?

(ns foo
(:import-jvm (package1.package2 BarBaz as BBaz))
(:import-clr (Package3.Package4.BooBaz as BBaz)))

(code-using-BBaz...)

Now in theory the one source file works under both.

Of course this doesn't help you if the BarBaz and BooBaz classes have
sufficiently different behavior that they can't be drop-in
replacements for each other. In that case the ideal would be to have
Clojure wrappers for the classes that translated into their semantics
from some common API -- an abstraction layer with separate JVM and CLR
versions (and normal imports). Now the problem becomes selecting
between these, but if the ns macro is going to learn to tell if it's
running on a JVM or the CLR anyway, why not add, also, :use-jvm,
:use-clr, :require-jvm, and :require-clr? Now the appropriate
abstraction layer can be used depending on the underlying runtime
without having to maintain two separate source trees mostly containing
copies of the same files but for a few platform-specific ones. Instead
there will be two differently-named versions of each platform-specific
file and one of everything else in a single source tree, and in some
cases fewer platform-specific files (e.g. where Int32 vs. Integer had
formerly required there be two versions of a file but :import-foo
allows there to be just one).

[dmiller]

On Nov 24, 1:24 pm, Ken Wesson <kwess...@gmail.com> wrote:
> I gave some thought to the Integer vs. Int32 issue.
>
> I think what would help here is a way to alias classes when :importing them.
>
> It would require tweaking the ns macro and possibly parts of the
> runtime. The outcome would be to allow something like
>
> (ns foo
>   (:import (java.util.concurrent
>              AtomicBoolean (ConcurrentHashMap :as CHM) AtomicInteger)))
>
> (defn make-default-chm []
>   (CHM. 12 12 12))
>
> ...
>
> Handy so far for abbreviating some longer classnames, and the
> application to CLR portability is apparent:
>
> (ns foo
>   (:import (System (Int32 :as Integer))))
>
> In theory, you'd just add this import to make code full of ^Integer
> hints compile under ClojureCLR. You'd still need two versions of the
> source file though, a CLR version with the import and a JVM version
> without.
>
> But then why not go even further and have the ns macro able to tell
> what platform it's on?
>
> (ns foo
>   (:import-jvm (package1.package2 BarBaz as BBaz))
>   (:import-clr (Package3.Package4.BooBaz as BBaz)))
>

Regarding enhancements to ns/import/etc.: Not to speak for Rich, but
he has in the past been resistant to modifications to these elements
to support CLR vs JVM.

Regarding Int32 vs Integer: They are really not equivalent. Int32 is
equivalent to Integer/TYPE or a ^int type hint, not to
java.lang.Integer. There is no CLR equivalent to the JVM Integer,
Long, Number, etc., a fact which is currently cause me no end of
problems as I try to incorporate into ClojureCLR all the compiler work
RIch &co have done for 1.3.

In fact, with 1.3, you will not be able to provide Int32 type hints on
fn args or return values. CLojureJVM in 1.3 disallows primitive type
hints in these places, wiping out ^int (JVM) and ^Int32 (CLR).

You might find this discussion relevant:
http://groups.google.com/group/clojure-dev/browse_thread/thread/6f9c9b7f36ae3134

-David