Java interface to the Clojure compiler
Stuart Sierra
Daniel Solano Gómez
> I've been looking into writing a leaner, meaner, CA-approved
> clojure-maven-plugin. I've got a rough prototype, but I've been hampered by
> the lack of a clean Java interface to the Clojure compiler. Right now we
> have clojure.lang.Compile, which only has a `main` method. It also calls
> `close` on System.out, which is bad when it's part of a longer process.
>
> Alternately, I could do something like this:
>
> RT.var("clojure.core", "compile").invoke(Symbol.intern(libraryName));
>
> but I don't like depending on that many implementation details. If the
> compiler gains more options, like stripping metadata, it becomes even harder
> to do that in a generic way.
>
> Others have asked for a documented API for interacting with Clojure from
To get the discussion moving forward, I see that you already added to
the Confluence page on build profiles. Another way to get
developer/user input would be to open feature requests for some of the
ideas, such as Java interface to compiler, stripping metadata from
compiled classes, separate compiler, etc. Based on responses, it would
be possible to prioritise the features, which should help inform the
decisions to make about how to implement it altogether.
Another option would be to put together a survey similar to the 1.3/2.0
debate and let people know about it on the general Clojure mailing list.
It could include questions such as:
1. How likely are you to use the following features, if implemented:
* Stripping metadata from AOT-compiled classes to reduce class size
* Clojure runtime that omits the compiler (AOT-only)
2. What types of environments are you currently using Clojure?
* Server (possibly implies better multicore support)
* Desktop
* Mobile (definitely implies a low profile)
3. How interested are you in seeing better support for the following
environments?
* Server
* Desktop
* Mobile
4. How often do you call Clojure from Java?
5. How important to you is a stable API for Java programs using Clojure?
---
These are just a few ideas off the top of my head. I am sure that a
better constructed set of questions can be put together with more
thought and input.
Just some brainstorming.
Sincerely,
Daniel Solano Gómez
Rich Hickey
On Mar 18, 7:39 pm, Stuart Sierra <the.stuart.sie...@gmail.com> wrote:
> I've been looking into writing a leaner, meaner, CA-approved
> clojure-maven-plugin. I've got a rough prototype, but I've been hampered by
> the lack of a clean Java interface to the Clojure compiler. Right now we
> have clojure.lang.Compile, which only has a `main` method. It also calls
> `close` on System.out, which is bad when it's part of a longer process.
>
> Alternately, I could do something like this:
>
> RT.var("clojure.core", "compile").invoke(Symbol.intern(libraryName));
>
> but I don't like depending on that many implementation details. If the
> compiler gains more options, like stripping metadata, it becomes even harder
> to do that in a generic way.
>
> Others have asked for a documented API for interacting with Clojure from
>
(clojure.core/compile (symbol "libname"))
would you feel you were relying on implementation details?
If not, the Java-ification seems no worse, and is bested in its
encapsulation of how clojure works only by something like:
RT.evalString("(clojure.core/compile 'libname)");
or perhaps an implementation of JSR 223:
http://download.oracle.com/javase/6/docs/technotes/guides/scripting/programmer_guide/index.html
Or, to flip it around, what would you rather write, that isn't a
redundant restatement of what the Clojure does in a Java wrapper that
will have to be kept in sync?
You say you're being hampered, where's the pain?
Rich
Stuart Sierra
Rich Hickey
As a general rule, using Clojure from Java should involve the
following process:
a) What would I call from Clojure?
b) Call that same exact thing from Java, using vars
So, RT.var("clojure.core",
"compile").invoke(Symbol.intern(libraryName)) should be your first
inclination, rather than trying to reuse what is clearly designed to
be a single-execution entry point (clojure.lang.Compile.main), and
therefor is not a library.
Var bindings are admittedly more involved, as there is no great way to
libraryize try/finally. You will need to bind *compile-path*,
*unchecked-math* and *warn-on-reflection* as does Compile.main _iff_
you aren't providing a way to bind them at some outer level. Your
utility, being intended to be part of some enclosing process, probably
shouldn't bind them to specific values unconditionally in any case,
only if they are not already bound, another way in which it will
differ from the single-execution model of Compile.main.
We could provide library fns (in Clojure!) that would enumerate the
vars that need to be bound for:
a) compilation
b) a REPL-like experience
since they are getting redundantly stated at several points. I imagine
the fns would return a map of var to default value.
Furthermore, we could try to wrap up the binding try/catch with fns
like:
(with-compilation-bindings ^Callable f)
(with-repl-bindings ^Callable f)
See main.clj with-bindings for a macro version of that. fn-based
versions would be semi-conveniently callable from Java using inner
classes.
Patch welcome for these.
Rich
Chas Emerick
On Mar 20, 2011, at 11:22 AM, Rich Hickey wrote:
On Mar 19, 2011, at 4:25 PM, Stuart Sierra wrote:The more general question is: at what level should Java tools (like Maven) interact with Clojure with the greatest likelihood of future compatibility? I've considered JSR 223 as well, although that's probably larger in scope.
As a general rule, using Clojure from Java should involve the following process:
a) What would I call from Clojure?
b) Call that same exact thing from Java, using vars
So, RT.var("clojure.core", "compile").invoke(Symbol.intern(libraryName)) should be your first inclination, rather than trying to reuse what is clearly designed to be a single-execution entry point (clojure.lang.Compile.main), and therefor is not a library.
Rich Hickey
There certainly isn't yet a supported public Java API for Clojure. I'm
amenable to a discussion of what might be in one. RT has too much else
in it. My point wasn't that there was a standard API, but that the
emphasis should be on enabling access to Clojure fns, not wrapping or
otherwise repackaging them - i.e. making it so the way to understand
how to do something is to understand how to do it in Clojure, then use
a minimal shim to access the same functions. There should be no
general hiding the fact that Clojure has vars and fns, while
minimizing the exposure to the specific classes.
So RT.var is the right idea. It would be nice to hide the Var class,
but unfortunately we can't make ClojureAPI.var() return both IFn and
IDeref without inventing a common subinterface. There are other
similar details.
In short, I'm in favor of pursuing http://dev.clojure.org/jira/browse/CLJ-452
as long as the scope is restricted to this model. I think you
understand that, given what's in 452 and other discussions we've had,
but general questions like 'what's the Java interface to the Clojure
compiler?' are off track. There should be a single, very small
interface to finding and dereferencing vars, and calling fns. Once you
have that, the rest of Clojure's public API is available.
IFn.invoke declaring Exception is the cause of much grief, but should
be considered separately. As a general rule I think that, while these
ideas interact, combining them should be limited to a Confluence
discussion, and we should keep the scope of tickets narrow.
Rich
Chas Emerick
On Mar 21, 2011, at 9:09 AM, Rich Hickey wrote:
>> Re: Stuart's mention of CLJ-452, I assume that RT and Var are implementation details; if I refer to them in Java code, I assume those references/calls will break in the future, potentially on the next point or alpha release.
>>
>> If that's not the case, and RT, Var, and so on should be considered part of Clojure's supported, public API, then I'd suggest we make them look like it (e.g. add proper documentation and publish javadoc of the public API classes as a peer to the autodoc).
>
> There certainly isn't yet a supported public Java API for Clojure. I'm amenable to a discussion of what might be in one. RT has too much else in it. My point wasn't that there was a standard API, but that the emphasis should be on enabling access to Clojure fns, not wrapping or otherwise repackaging them - i.e. making it so the way to understand how to do something is to understand how to do it in Clojure, then use a minimal shim to access the same functions. There should be no general hiding the fact that Clojure has vars and fns, while minimizing the exposure to the specific classes.
>
> So RT.var is the right idea. It would be nice to hide the Var class, but unfortunately we can't make ClojureAPI.var() return both IFn and IDeref without inventing a common subinterface. There are other similar details.
>
> In short, I'm in favor of pursuing http://dev.clojure.org/jira/browse/CLJ-452 as long as the scope is restricted to this model. I think you understand that, given what's in 452 and other discussions we've had, but general questions like 'what's the Java interface to the Clojure compiler?' are off track. There should be a single, very small interface to finding and dereferencing vars, and calling fns. Once you have that, the rest of Clojure's public API is available.
Thank you for the clarification re: public APIs. Sounds good to me. The necessary set of public API methods is likely very small, as you say.
> IFn.invoke declaring Exception is the cause of much grief, but should be considered separately. As a general rule I think that, while these ideas interact, combining them should be limited to a Confluence discussion, and we should keep the scope of tickets narrow.
Fair enough, though the outcome on that front might help inform the signatures of the eventual public Java API for calls.
I'll open up a discussion page in the wiki to get things percolating.
- Chas
Chas Emerick
On Mar 21, 2011, at 9:33 AM, Chas Emerick wrote:
> I'll open up a discussion page in the wiki to get things percolating.
Done, see http://dev.clojure.org/display/design/Improvements+to+interop+from+Java
- Chas
Sean Corfield
> So, RT.var("clojure.core", "compile").invoke(Symbol.intern(libraryName))
> should be your first inclination, rather than trying to reuse what is
> clearly designed to be a single-execution entry point
> (clojure.lang.Compile.main), and therefor is not a library.
FWIW, in my cfmljure project (which provides an easy bridge to call
Clojure code from CFML), I rely on:
* clojure.lang.RT
* RT.var( ns, fn )
* RT.loadResourceScript( filepath )
* Var.invoke( args )
I rely on loading source files (and having them automatically
compiled) because that seemed a more intuitive approach (esp. for my
audience). I'd need to track any breaking changes in this area and I'm
certainly interested in feedback on better ways to handle running .clj
code from inside a Java application (which is what CFML is,
essentially).
--
Sean A Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
World Singles, LLC. -- http://worldsingles.com/
Railo Technologies, Inc. -- http://www.getrailo.com/
"Perfection is the enemy of the good."
-- Gustave Flaubert, French realist novelist (1821-1880)
Chas Emerick
On Mar 21, 2011, at 8:24 PM, Sean Corfield wrote:
> FWIW, in my cfmljure project (which provides an easy bridge to call
> Clojure code from CFML), I rely on:
> * RT.loadResourceScript( filepath )
Good call IMO. I added load(File f) to [1].
- Chas
[1] http://dev.clojure.org/display/design/Improvements+to+interop+from+Java
Rich Hickey
On Mar 22, 2011, at 6:08 AM, Chas Emerick wrote:
>
> On Mar 21, 2011, at 8:24 PM, Sean Corfield wrote:
>
>> FWIW, in my cfmljure project (which provides an easy bridge to call
>> Clojure code from CFML), I rely on:
>
>> * RT.loadResourceScript( filepath )
>
> Good call IMO. I added load(File f) to [1].
>
Is there a good reason to call this rather than clojure.core/load (or
some other core fn)? If so, there should be a core fn for it. Then the
way to call it should be to get the var for that and invoke.
Repeat this process for anything else that seems like it needs to be
specially in this interface.
If, after that, you want to remove the two step process (get the var,
then invoke) for some common things, you can have a few static public
final Var members bound via static init.
Rich
Sean Corfield
> Is there a good reason to call this rather than clojure.core/load (or some
> other core fn)? If so, there should be a core fn for it. Then the way to
> call it should be to get the var for that and invoke.
Are you saying that it would be preferable to do something like:
RT.var( "clojure.core", "load" ).invoke( filepath );
(or use "load-file" - I'm not clear on the difference, based on
reading the docs?)
I'm certainly happy to reduce my dependency on specific methods in RT
if it means future-proofing my code. Thanx!
> If, after that, you want to remove the two step process (get the var, then
> invoke) for some common things, you can have a few static public final Var
> members bound via static init.
In Java perhaps, not in the dynamic scripting language I'm actually
using to load and invoke Clojure code...
Chas Emerick
On Mar 22, 2011, at 7:31 AM, Rich Hickey wrote:
>
> On Mar 22, 2011, at 6:08 AM, Chas Emerick wrote:
>
>>
>> On Mar 21, 2011, at 8:24 PM, Sean Corfield wrote:
>>
>>> FWIW, in my cfmljure project (which provides an easy bridge to call
>>> Clojure code from CFML), I rely on:
>>
>>> * RT.loadResourceScript( filepath )
>>
>> Good call IMO. I added load(File f) to [1].
>>
>
> Is there a good reason to call this rather than clojure.core/load (or some other core fn)? If so, there should be a core fn for it. Then the way to call it should be to get the var for that and invoke.
>
> Repeat this process for anything else that seems like it needs to be specially in this interface.
Thank you for the feedback here and on the wiki. I'll update the latter accordingly soon-ish.
> If, after that, you want to remove the two step process (get the var, then invoke) for some common things, you can have a few static public final Var members bound via static init.
I assume you mean bound via static init in the client application, *not* in the Clojure public API class.
- Chas
Rich Hickey
No, I meant in the API, if we feel it is compelling, and only for some
key entry points. If nothing else, it would serve as an example. I am
afraid of people repeatedly doing var("blah").invoke() for the same
var, not knowing the lookup cost.
We will need an interface unifying IDeref and IFn for the return type
of API.var()
Rich