Re: Calling Java from Clojure

558 views
Skip to first unread message
Message has been deleted

Alex Miller

unread,
Jun 10, 2019, 12:21:28 AM6/10/19
to Clojure
Looks like the title for this is backwards if I understand the intent, should be calling Clojure from Java, right?

Java code is going to invoke methods on classes (that's all it knows how to do). You have several options:

1) Use the Clojure Java API to invoke the Clojure runtime - here you're leveraging the Clojure runtime's Java impl core to call into Clojure (Stu's example)
2) Use protocols, records, and/or genclass to actually produce Java classes (your example, I think). 
3) Define your interface as a set of Java interfaces. Implement this interface in Clojure. You will need a small amount of glue code to provide the API impl (some kind of a factory to give you the impl of the Java interface - either written in Java or using #1). 

#1 is straightforward but tedious - it's really only worthwhile if you have a small amount of this code and hide the use of it.
#2 has significant downsides - needing to compile everything, all methods are just going to take/return Java Objects, no javadoc on apis,  etc. 
#3 has advantages in all those areas. You write your Java interface where you can best write it ... in Java (with Javadoc, and Java interfaces, and all the niceties Java users want). IDEs can autocomplete on the Java interfaces. You don't have to AOT - factories can reify or load protocol instances as needed as they return objects. You can even reload internal vars with a connected REPL without restarting the app.

The last does take a bit of planning to set up - you need API code (in Java), and Clojure code that relies on that Java code. lein makes it pretty trivial to put those in one project and compile in that order.

I pushed an example up here:


It defines a Java interface (java/cfj/Event.java), a static helper to hook the Clojure (java/cfj/Support.java), which uses the Clojure Java API to load the Clojure instance in clj/core/cfj.clj. Just `lein uberjar` and you're done. Use the Support API from Java as any other Java class. No AOT required.



On Sunday, June 9, 2019 at 9:53:56 PM UTC-5, eglue wrote:
I've been stalking Clojure for a while but now doing it professionally full-time (woo hoo!).

I saw a Stuart Halloway tweet responding to someone who'd found it a "soul-crushing, miserable experience."

I had a similar miserable experience and figured it was just me, but am now suspecting that's not the case. (Happy to be shown the light however.)

Stuart H posted https://github.com/stuarthalloway/clojure-from-java to demonstrate the ease of Java consuming Clojure, but I find it far more important to be able to *compile* Java against Clojure interfaces not invoke it dynamically from Java....because dynamic invocation from Java is unwieldy to the point of it being a likely deal breaker for any Java shop. Dynamically loading classes and invoking methods.... c'mon, no one's going to ask their Java devs to do this.

If Clojure doesn't have a good compile-time consumption story for Java consumers, I think that's a loss. (Even if it is just providing better docs or archetype/bootstrap examples in this regard.) Because otherwise Java code bases around the world could be eaten away by (compiled) Clojure from the inside out, giving Java dev teams enough time to be overtaken by the miracle that is Clojure.

Inspired by Stuart's example, I was successful in putting together a quick build to achieve this ideal: https://github.com/atdixon/clojure-from-java

However, I have a few open questions:

- when I tried to AOT only the public-facing clojure code that I needed to compile against, I found out at runtime that this wasn't going to work. Dynamic clojure code was loading my same types into DynamicClassLoader and when my statically-compiled, root-class-loaded code was getting executed the ClassCastExceptions of course were flying. So am I right to think that you have to AOT everything in order to do this right?
- IDE support (for me, Cursive) "works" but is non-ideal; you have to manually compile the dependee/Clojure jar, then Cursive will let you execute Java code against the manually aot-compiled Clojure code 
- AOT producing generics/generic types doens't seem to be part of any of this... is this a lacuna in the Clojure AOT space, are there libs that can help here?

This story ^^, if made easier, seems to me would boost Clojure adoption in Java/JVM shops.

What am I missing?


Message has been deleted

eglue

unread,
Jun 10, 2019, 1:04:18 AM6/10/19
to Clojure
This is great, Alex, thanks. (Sorry, I deleted from underneath you and reposted to fix the title before I saw your reply...)

The latter option, writing interfaces and classes in Java and calling w/ the glue code is a great option.

However one big motivator for people moving from Java to langs like Scala and Kotlin is being able to implement those bean/record types, eg, with such little fanfare & boilerplate (eg in Clojure, using defrecord is so concise). And you can sometimes end up with a handful of these, so the 'glue code' isn't quite so small. Also gen-class and definterface I find similarly nice. 

I wonder if there's a path to improve what you're calling option #2. Specifically the needing to AOT compile *everything*. It seems I should only have to AOT/precompile the surface area that my Java consumer wants to link with. At runtime perhaps there'd be some creative way to leave out the AOT classes (which would just have been used for static-time compiling) and when the JVM tries to load those classes some agent could interject and do the dynamic Clojure loading?

Alex Miller

unread,
Jun 10, 2019, 9:20:44 AM6/10/19
to clo...@googlegroups.com
On Mon, Jun 10, 2019 at 12:04 AM eglue <atd...@gmail.com> wrote:
This is great, Alex, thanks. (Sorry, I deleted from underneath you and reposted to fix the title before I saw your reply...)

The latter option, writing interfaces and classes in Java and calling w/ the glue code is a great option.

However one big motivator for people moving from Java to langs like Scala and Kotlin is being able to implement those bean/record types, eg, with such little fanfare & boilerplate (eg in Clojure, using defrecord is so concise).

The problem with wanting the Clojure-y version from Java is that those interfaces are generic and weird from the Java side. So either you get bean-like code familiar to Java users or generic collection/keyword interfaces that will be weird to use from Java. I heard your goal as "be friendly from Java", which means, making bean-like interfaces. On the Clojure side, it's pretty easy to macro-ize the creation of those over defrecords or whatever, so it could still be  a lot less macro work, just depends how far you want to go with it. You don't get all the benefits of Clojure from Java; you have to be in Clojure for the many related design decisions to compound together. There's no magic solution to that (other than "just use Clojure" :).
 
And you can sometimes end up with a handful of these, so the 'glue code' isn't quite so small. Also gen-class and definterface I find similarly nice. 

I find the glue code is actually small in practice (I've done a couple of real systems this way). This particular example is a little weird because it's just making a domain object, but generally you're writing the glue to provide a factory method for a facade. Below the facade, the Clojure code can make new objects just fine, so you're staying in Clojure for the rest of it.
 
I wonder if there's a path to improve what you're calling option #2. Specifically the needing to AOT compile *everything*. It seems I should only have to AOT/precompile the surface area that my Java consumer wants to link with. At runtime perhaps there'd be some creative way to leave out the AOT classes (which would just have been used for static-time compiling) and when the JVM tries to load those classes some agent could interject and do the dynamic Clojure loading?

Seems like you're taking the hardest path and trying to make it harder. I just don't see the point in running at this one, sorry. My experience is that the approach above works great and has none of these issues.
 
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/aejqMwraPk8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/07753636-3414-481f-8836-763ca85675ce%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

eglue

unread,
Jun 10, 2019, 11:34:25 AM6/10/19
to Clojure
> I find the glue code is actually small in practice (I've done a couple of real systems this way). This particular example is a little weird because it's just making a domain object, but generally you're writing the glue to provide a factory method for a facade. Below the facade, the Clojure code can make new objects just fine, so you're staying in Clojure for the rest of it.

Very helpful, thanks!

For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/aejqMwraPk8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clo...@googlegroups.com.

henrik42

unread,
Jun 11, 2019, 8:27:22 AM6/11/19
to Clojure
If you're willing to use Spring you can create Clojure-based Spring Beans [1] and let Spring inject (autowire) them into your Java-based Spring-Beans. And if you like you could use Spring just as a factory that you invoke. I would still define interfaces (that you want to use from Java) in Java, so that IDE autocompletion, generics etc. work as excepted.
HTH
Henrik


Example:
<bean id="load_script_code" 
	class="clojure.lang.Compiler" 
	factory-method="loadFile">
	<constructor-arg value="src/main/clojure/no-namespace-scripts/script-code.clj" />
</bean>

Alan Moore

unread,
Jun 11, 2019, 5:09:30 PM6/11/19
to Clojure
Maybe this is too old school but wouldn’t a dynamic proxy help here? E.g. java.lang.reflect.InvocationHandler & java.lang.reflect.Proxy.

Clearly you’d be relying on reflection but if your interface usage is large grained enough the overhead wouldn’t be too bad.

henrik42

unread,
Jun 12, 2019, 3:12:01 AM6/12/19
to Clojure
I hacked just that a few days ago to support Java development at work: https://github.com/henrik42/deeto Not released yet but could be a starter in that direction.
Henrik

Alan Moore

unread,
Jun 12, 2019, 2:09:17 PM6/12/19
to Clojure
Nice, this looks very handy. Thanks!

Alan

ru

unread,
Jun 17, 2019, 11:26:11 AM6/17/19
to Clojure
Excuse me, please, may be this is a naive question. As I understand, we want to call Clojure from Java when we want to use advantages of implementation of some functionality in Clojure. But, in this example I see implementation of the functionallity in Java. Am I right?

понедельник, 10 июня 2019 г., 7:21:28 UTC+3 пользователь Alex Miller написал:

Alex Miller

unread,
Jun 17, 2019, 4:35:30 PM6/17/19
to Clojure
No, there is a bit of Java code just to load the Clojure
code, but then it’s Clojure after that (the entity record impl).

ru

unread,
Jun 17, 2019, 5:07:41 PM6/17/19
to Clojure
core.clj defines only names "getTimestamp" and "getName", but meaning of these names defined in Support.java. For example, that "getTimestamp" means result of a new java.util.Date.

понедельник, 17 июня 2019 г., 23:35:30 UTC+3 пользователь Alex Miller написал:

Alex Miller

unread,
Jun 17, 2019, 5:33:23 PM6/17/19
to clo...@googlegroups.com
Yes, the whole point of this approach is to define the interface in Java and the implementation in Clojure.
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to

For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/aejqMwraPk8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/c7ef72fe-b062-49b6-a3bb-5183e1a73d6c%40googlegroups.com.

ru

unread,
Jun 17, 2019, 7:15:26 PM6/17/19
to Clojure
I understand it. And the approach seems promising and powerful. But the example seems to me not too successful. And I drew your attention to this fact in the hope of seeing a more convincing example. Thank you.

вторник, 18 июня 2019 г., 0:33:23 UTC+3 пользователь Alex Miller написал:
Yes, the whole point of this approach is to define the interface in Java and the implementation in Clojure.

On Jun 17, 2019, at 5:07 PM, ru <sor...@oogis.ru> wrote:

core.clj defines only names "getTimestamp" and "getName", but meaning of these names defined in Support.java. For example, that "getTimestamp" means result of a new java.util.Date.

понедельник, 17 июня 2019 г., 23:35:30 UTC+3 пользователь Alex Miller написал:
No, there is a bit of Java code just to load the Clojure
code, but then it’s Clojure after that (the entity record impl).

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to

For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/aejqMwraPk8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clo...@googlegroups.com.

Matching Socks

unread,
Jun 20, 2019, 9:06:53 PM6/20/19
to Clojure
Depends on what you seek an example of.  The cfj interop example provides 1-line Clojure implementations of the features the Java program consumes, and 1 line is plenty to demonstrate the interop.  On the other hand, those 1-liners are not an example of significant work you'd much rather do in Clojure than Java.  But such examples are legion.  Dump Github upside down and you would flood a city with code you would rather have written in Clojure.

Didier

unread,
Jun 21, 2019, 12:13:41 AM6/21/19
to Clojure
Option #1 and #3 are very much the same, just that Option #3 creates a Java layer on top, instead of having everything coupled to the Clojure Java APIs directly.

When you go with Option #2, you do not have to AOT everything, but AOT is transitive. So as you AOT the gen class, all code it requires and all code they in turn require will also be AOT. You can post-process the AOT when jaring, and only include the gen-class .class files in the Jar and not the dependencies, but that's more work.

Matching Socks

unread,
Jun 21, 2019, 5:26:37 AM6/21/19
to Clojure
Instead of deleting class files from a jar, you can use the :impl-ns option on gen-class to stop transitive AOT.  (gen-class makes a class file, but AOT does not reach the impl-ns.)  I don't remember seeing this in the manual, but I don't know why else impl-ns would exist...

eglue

unread,
Jun 21, 2019, 11:01:19 AM6/21/19
to Clojure
This is a good clarification.

I found that when I tried to exclude some AOT stuff from the jar you can get into a situation where Clojure will dynamically produce a class that is already statically-produced at the root classloader and then you'll hit ClassPathExceptions when you try to pass these things around.

For example I had dynamic (non-AOT'd) code paths using a defrecord, for which a .class file was generated dynamically. So if I kept the AOT'd defrecord, as well, I'd have two classes one in the root classloader and one from dynamic clojure (DynamicClassLoader). 

So if what you're saying is possible, I think you will have to show quite a bit of careful surgery to avoid these kinds of problems?

Alex Miller

unread,
Jun 21, 2019, 11:07:00 AM6/21/19
to clo...@googlegroups.com
With AOT, you generally shouldn't ever exclude part of the AOT'ed code. Most problems with AOT'ed code stem from having AOT'ed code call into non-AOT'ed code, so anything "downstream" should also be AOT'ed.

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to

For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/aejqMwraPk8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/02a72bb0-9a6f-4574-9134-2d8983db8b81%40googlegroups.com.

Didier

unread,
Jun 21, 2019, 11:25:22 AM6/21/19
to Clojure
Interesting about impl-ns, I'll have to look into that. If so, it would greatly simplify it all.

In my experience, if you only keep the gen-class .class, and delete everything else, it all works without issues. If you are actually doing AOT, not for the purpose of gen-class, I wouldn't recommend deleting parts of it. From what I understand, the gen-class creates a class which will load the Clojure code in the static initialize, and look for the Clojure code as a resource on the classpath. This seems to work fine if it is only there as a .clj, it will perform just in time compilation and everything will work as normal.

Does anyone know of edge cases here? Or has experienced issues with this strategy?

Sean Corfield

unread,
Jun 21, 2019, 3:35:53 PM6/21/19
to clo...@googlegroups.com

The approach I’ve taken around :gen-class has been to ensure that the namespace offering the Java class via :gen-class has no static dependencies at all – instead it requires/resolves any other namespaces/symbols it needs at run time. That way AOT creates the .class you need but doesn’t spread to anything else.

 

This is even easier in Clojure 1.10 with the addition of requiring-resolve (which is also thread safe, unlike regular require right now).

 

But yeah, as Alex indicated, you need to be careful if you have an AOT’d entry point and downstream code is not AOT’d. Just one more “here be dragons” aspect of AOT, IMO.

 

Sean Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood

 


From: clo...@googlegroups.com <clo...@googlegroups.com> on behalf of Didier <did...@gmail.com>
Sent: Thursday, June 20, 2019 9:13:40 PM
To: Clojure
Subject: Re: Calling Java from Clojure
 
Option #1 and #3 are very much the same, just that Option #3 creates a Java layer on top, instead of having everything coupled to the Clojure Java APIs directly.

When you go with Option #2, you do not have to AOT everything, but AOT is transitive. So as you AOT the gen class, all code it requires and all code they in turn require will also be AOT. You can post-process the AOT when jaring, and only include the gen-class .class files in the Jar and not the dependencies, but that's more work.

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to

For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/1dd84879-6855-4893-99ee-de15ea6a6329%40googlegroups.com.

Alex Miller

unread,
Jun 21, 2019, 4:54:01 PM6/21/19
to clo...@googlegroups.com
I just don’t understand why you would introduce aot or gen-class at all if you didn’t have to, which is one of the benefits of using the Clojure Java API.
You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/aejqMwraPk8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/CY4PR2201MB11266042416A0BEFDBB233C6F4E70%40CY4PR2201MB1126.namprd22.prod.outlook.com.

Didier

unread,
Jun 21, 2019, 5:57:59 PM6/21/19
to Clojure
Oh, not when you don't have too. I mean, you can always hand write a class in Java and have it call into Clojure. That's effectively gen-class but done manually. Otherwise, I favour the Clojure Java API.

But some code base are already using gen-class, and some people do use gen-class. Sometimes it works and it's easier then having to write Java and modify your build to also use javac and include Java built artifacts.

In those cases, the AOT is problematic, because it brings in dependencies, so if you package them as libraries, you get additional classes which can then conflict. So you need to find a way to only include the generated class and nothing else.

Alex Miller

unread,
Jun 21, 2019, 6:23:36 PM6/21/19
to clo...@googlegroups.com
In that case, I would try to isolate those gen-classes into as small a box as possible and make an artifact for just those.

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/aejqMwraPk8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.

Sean Corfield

unread,
Jun 21, 2019, 8:05:23 PM6/21/19
to clo...@googlegroups.com

Oh, you know me: I avoid AOT and gen-class at all costs _if I don’t have to use them_ 😊

 

Sean Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood

 


From: clo...@googlegroups.com <clo...@googlegroups.com> on behalf of Alex Miller <al...@puredanger.com>
Sent: Friday, June 21, 2019 1:53:43 PM
To: clo...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages