Promoting Gradle in Clojure Community

[Andrew Oberstar]

As noted in the 2014 State of Clojure, Gradle is a pretty low percentage of the build tool use within the Clojure community. As a long-time Gradle user and plugin author, I'd love to see Gradle be mentioned in the same breath as Lein and Boot in the Clojure community.

As Meikel mentioned in an April post, he doesn't consider the project "dead". However, since there hasn't been a commit to any of the repos in the last year, it's certainly not active.

I'm personally interested in working on Gradle support, though I'd like to target the new new model-space that's incubating in Gradle. Given that goal, I wouldn't really be able to contribute the effort back to Clojuresque, so it would be a "competing" Gradle plugin.

Basically, I wanted to float this discussion here before asking in a broader list (such as Clojure user).

What are other people's thoughts (including Meikel's) about Gradle's place in the Clojure community and how it can grow?

Andrew Oberstar

[Jason Zwolak]

Hi Andrew,

I'm a user of your Gradle Git plugin and you may remember I helped debug a problem with file permissions and access control lists that turned out to be an issue that was fixed in a newer version of jgit.

I'm disappointed no one has replied to your post, because I for one believe that Gradle has a special place in the Clojure community and lacks first class support for Clojure.  I'm using Clojuresque right now and running into difficulties using it in a mixed project (Java and Clojure).  I mean no harm in saying I'm running into difficulties... I just want to be clear that there's room for improvement and work in this area.

I believe Gradle's place is precisely in mixed projects where multiple languages are employed.  Especially in legacy Java projects (like mine) that want to bring in some of the power of Clojure for concurrency, state management, etc..

In my project for a small bio-tech company that contracts work to large pharmaceuticals, Gradle has proven to be really powerful in automating our build, deployment, and testing.  As we move forward to fix some really nasty state handling and concurrency bugs I've decided to employ Clojure and convert buggy Java code to Clojure.  There's no way we're switching build systems.  In our case it's primarily because we have a lot invested in our current highly functional build system developed on Gradle.  However, I'd be hard pressed to be convinced that leiningen or boot could do the sophisticated stuff we do in deployment which involves a third party framework called Tom Sawyer Perspectives and their custom licensing tool, ssh deployment code to an existing webstart server, versioning using your very own Gradle Git plugin, and more.

The power of Gradle is that it is general purpose, widely used, multi-lingual, with lots of plugins.

So I'm for continued work on a Clojure plugin for Java.  I'm not attached to starting a new project versus continued work on Clojuresque.  But I understand you want to move forward with the latest standards in Gradle.

You would likely see me as a user, tester, and occasional debugger, though I wouldn't likely invest to contribute large volumes of code.  Though, I have to say that's primarily because I can't always wrap my head around other people's code.  If the code was clean and well laid out then it is conceivable that I would contribute entire features or fill in sections on your todo list.

I'm glad to hear you're interested in this!

Jason

[Jason Zwolak]

I mean a "Clojure plugin for Gradle"

[Meikel Brandmeyer]

Hi everyone,

Am 15.09.2015 um 20:44 schrieb Jason Zwolak:

> I mean no harm in saying I'm running into difficulties... I
> just want to be clear that there's room for improvement and work in this
> area.

Can you explain your troubles there? In my experience mixed projects
work rather well. I even had Java, Clojure and Groovy in one project
without much trouble.

Build order might need some manual sorting, but clojuresque cannot know
that up-front. So I don't see another solution for that.

> So I'm for continued work on a Clojure plugin for Java. I'm not
> attached to starting a new project versus continued work on Clojuresque.
> But I understand you want to move forward with the latest standards in
> Gradle.

Gradle is really a pain at times, because it changes quite fast under
the hood. I'm a bit disconnected at the moment.

Anyone who wants to support getting clojuresque up to speed (test
integration is another problem area) is welcome. Let me know your
github/bitbucket account and you can get immediate access.

Meikel

[Andrew Oberstar]

Hi Jason,

Thanks again for the contribution to gradle-git and now for providing some good context from a Clojuresque user. I think we're on the same page as far as what Gradle brings to the table, so that's reassuring to me. Polygloy is probably the big driver, but the ecosystem of plugins is hard to give up for me (and sounds like you too) when using Lein or Boot.

Andy

--
Sie erhalten diese Nachricht, weil Sie in Google Groups E-Mails von der Gruppe "clojuresque" abonniert haben.
Wenn Sie sich von dieser Gruppe abmelden und keine E-Mails mehr von dieser Gruppe erhalten möchten, senden Sie eine E-Mail an clojuresque...@googlegroups.com.
Weitere Optionen finden Sie unter https://groups.google.com/d/optout.

[Andrew Oberstar]

Hi Meikel,

Thanks for responding!

On Tue, Sep 15, 2015 at 2:39 PM Meikel Brandmeyer <m...@kotka.de> wrote:

Hi everyone,

Am 15.09.2015 um 20:44 schrieb Jason Zwolak:

> I mean no harm in saying I'm running into difficulties... I
> just want to be clear that there's room for improvement and work in this
> area.

Can you explain your troubles there? In my experience mixed projects
work rather well. I even had Java, Clojure and Groovy in one project
without much trouble.

If I'm interpreting Jason correctly, he's only saying that one of Gradle's strong points is polyglot support. I don't think that was an example of where his difficulties lie with Clojuresque.

Personally, I haven't gotten much past the home page of the Clojuresque repositories for two reasons, which admittedly isn't great ground for me to be standing on when proposing starting a new plugin.

1. Having both Bitbucket and Github repos is a little confusing. It always takes me a bit to figure out which is the canonical source.

2. The documentation being on each repo feels disjointed. It's hard to get a picture of practical usage without a lot of jumping.

 

Build order might need some manual sorting, but clojuresque cannot know
that up-front. So I don't see another solution for that.

> So I'm for continued work on a Clojure plugin for Java.  I'm not
> attached to starting a new project versus continued work on Clojuresque.
>  But I understand you want to move forward with the latest standards in
> Gradle.

Gradle is really a pain at times, because it changes quite fast under
the hood. I'm a bit disconnected at the moment.

From the hype, their new model approach should address a lot of Gradle pains. Understanding how much of that pain is alleviated in practice is a large part of my motivation here. Granted, that's also the part least relevant to promoting Gradle to the Clojure community.

 

Anyone who wants to support getting clojuresque up to speed (test
integration is another problem area) is welcome. Let me know your
github/bitbucket account and you can get immediate access.

While I originally thought my goal of targeting the new model plugins implied that there couldn't be shared effort, I would expect the opposite is true. The tasks at least could be shared across both the current-style plugins and the model-style.

Before I get into anything on this front, I'll take a deeper look at Clojuresque's repositories to see how much of the effort seems shareable between those two plugin approaches. If it's significant, I may hit you up on that access offer.

 

Meikel

--
Sie erhalten diese Nachricht, weil Sie Mitglied der Google Groups-Gruppe "clojuresque" sind.

Wenn Sie sich von dieser Gruppe abmelden und keine E-Mails mehr von dieser Gruppe erhalten möchten, senden Sie eine E-Mail an clojuresque...@googlegroups.com.

Weitere Optionen: https://groups.google.com/d/optout

Andrew Oberstar 

[Jason Zwolak]

Hi Meikel,

Things are working well for me at the moment.  I just have a memory of it being difficult to get set up... I'll post in another thread if I have anything concrete to report.

Thanks for your concern!

Jason

--
Jason Zwolak

--
Sie erhalten diese Nachricht, weil Sie in Google Groups ein Thema der Gruppe "clojuresque" abonniert haben.
Wenn Sie sich von diesem Thema abmelden möchten, rufen Sie https://groups.google.com/d/topic/clojuresque/1j24yiOGa30/unsubscribe auf.
Wenn Sie sich von dieser Gruppe und allen Themen dieser Gruppe abmelden möchten, senden Sie eine E-Mail an clojuresque...@googlegroups.com.
Weitere Optionen: https://groups.google.com/d/optout

[Meikel Brandmeyer]

Hi,

Am 16.09.2015 um 01:47 schrieb Andrew Oberstar:

> 1. Having both Bitbucket and Github repos is a little confusing. It
> always takes me a bit to figure out which is the canonical source.

I have a deep distaste for git. With mercurial I just feel at home and
patch queues are awesome to work with. (So maybe I should switch to
darcs?) I also like the tracker and interface of bitbucket. So bitbucket
was up to now always authoritative.

That said: I'm pretty alone with using hg and since it works reasonably
well with git repos, that can be changed to github. In fact the repos
are already there.

> 2. The documentation being on each repo feels disjointed. It's hard to
> get a picture of practical usage without a lot of jumping.

Documentation is a sad story. A central site should be installed with
documentation and as single point of entry. Also I would prefer a
central bug tracker. The site could also host the documentation ebook -
should that ever be finished.

> From the hype, their new model approach should address a lot of Gradle
> pains. Understanding how much of that pain is alleviated in practice is
> a large part of my motivation here. Granted, that's also the part least
> relevant to promoting Gradle to the Clojure community.

In fact there is not much pain from the gradle's user point of view. At
least I never had big problems and quite a lot worked just as
advertised. Even with clojuresque.

The pain is more under the hoods. The APIs changed a lot actually and
the very sad thing is that the official plugins used gradle internal
APIs. After getting a myself a bloody nose of breaking-API-change I
reimplemented things. Pfff...

> While I originally thought my goal of targeting the new model plugins
> implied that there couldn't be shared effort, I would expect the
> opposite is true. The tasks at least could be shared across both the
> current-style plugins and the model-style.
>
> Before I get into anything on this front, I'll take a deeper look at
> Clojuresque's repositories to see how much of the effort seems shareable
> between those two plugin approaches. If it's significant, I may hit you
> up on that access offer.

The repo layout is as follows:

* common: common utility things, non-clojure specific, preparation for
scriptoresque (ClojureScript)
* base: minimal clojure plugin providing source sets, building, etc.
* nrepl: providing a repl task
* clojars: publication to clojars
* extra: some tasks (Watcher task, etc.) which are not clojuresque
specific and could be in their own plugin
* clojuresque: meta plugin which sucks in all of the above

Most repos are split into a plugin and a runtime part. The plugin is the
gradle side. The runtime is the part which has to be on the project's
classpath (via the clojuresque configuration) to be able to run the
different tasks.

The idea is to allow individual speed of development of the different
moving parts. Also in regard of a possible ClojureScript plugin. And to
allow to use just what is necessary. (I move to bintray for example. So
the clojars plugin becomes less interesting.)

I hope this gives a starting point to get an overview.

Feel free to fire away your questions. Vacation period is over, so
things should calm down now and I hope to be more responsive now.

Meikel

[Andrew Oberstar]

Hi again,

On Thu, Sep 17, 2015 at 1:36 PM Meikel Brandmeyer <m...@kotka.de> wrote:

Hi,

Am 16.09.2015 um 01:47 schrieb Andrew Oberstar:

> 1. Having both Bitbucket and Github repos is a little confusing. It
> always takes me a bit to figure out which is the canonical source.

I have a deep distaste for git. With mercurial I just feel at home and
patch queues are awesome to work with. (So maybe I should switch to
darcs?) I also like the tracker and interface of bitbucket. So bitbucket
was up to now always authoritative.

While I personally prefer Git and Github, that's out of ignorance of (lack of experience with) Mercurial and Bitbucket. So I really don't have any strong opinions there besides familiarity.

 

That said: I'm pretty alone with using hg and since it works reasonably
well with git repos, that can be changed to github. In fact the repos
are already there.

My main point is just that having two sets of repos is confusing, which specific one isn't as big of a concern for me. I do feel it would be clearer to either remove one or more clearly indicate on the mirror that it's not authoritative.

Pragmatically, Github gets better exposure (while, on principle, I don't like that argument).

 

> 2. The documentation being on each repo feels disjointed. It's hard to
> get a picture of practical usage without a lot of jumping.

Documentation is a sad story. A central site should be installed with
documentation and as single point of entry. Also I would prefer a
central bug tracker. The site could also host the documentation ebook -
should that ever be finished.

Both of those sound like good ideas to ease the learning curve for newcomers.

 

> From the hype, their new model approach should address a lot of Gradle
> pains. Understanding how much of that pain is alleviated in practice is
> a large part of my motivation here. Granted, that's also the part least
> relevant to promoting Gradle to the Clojure community.

In fact there is not much pain from the gradle's user point of view. At
least I never had big problems and quite a lot worked just as
advertised. Even with clojuresque.

The pain is more under the hoods. The APIs changed a lot actually and
the very sad thing is that the official plugins used gradle internal
APIs. After getting a myself a bloody nose of breaking-API-change I
reimplemented things. Pfff...

While the new model-space does bring some performance and consistency benefits to the user, it should address a lot of the laziness issues that are hard to deal with as a plugin implementer. Unfortunately, there are still a lot of internal classes being used in their new language plugins. I also feel that their source has generally gotten harder to understand since 1.0.

I've seen a few places where you reference the ClojureScript plugin as "possible". However, I also see the "scriptoresque" repos out there too. Should I take away that those were just prototypes and aren't in a working state?

 

I hope this gives a starting point to get an overview.

Feel free to fire away your questions. Vacation period is over, so
things should calm down now and I hope to be more responsive now.

I've given a brief look through your source (which was helped by these descriptions) and am now poking through the Gradle APIs for the new language plugins (using the model approach).

Unfortunately, there's going to be a huge learning curve for me to wrap my head around how these new plugins should work. There seems to be a significant amount of modelling effort (going beyond the simple use of extensions/conventions).

While I wrap my head around that, I'd like to avoid expending any extra cycles on backwards compatibility concerns. For now, I'm going to continue my original plan of building a parallel Clojure plugin. I feel like it will be easier to reconcile what can be shared afterwards. I'll certainly be using Clojuresque as inspiration as I get to the more Clojurey parts of this.

For those interested in following any progress, I created an organization on Github (Graclj, pronounced like grackle) before I got any responses to this thread, so I'll be working over there. I'll keep this list updated as I have questions or general updates.

https://github.com/graclj