Leiningen uberjar: excluding dev-dependencies; new minus-clojure task

[Perry Trolard]

Greetings,

Currently for the Leiningen uberjar task, all dev-dependencies are
included in the resulting standalone jar. In a comment in uberjar.clj,
there's a note that excluding these is on the todo list. I wonder if
there are any ideas on how best to do it?

I tried to tackle this in a naive way — finding jar files based on the
project name — but this doesn't catch dependecies futher down the
chain, e.g. for lein-clojars this'll find the lein-clojars.jar but
won't find jsch.jar, its dependency. More maven-aware folks may have
better ideas, but how does this solution sound: segregating dev-
dependencies in the <project>/lib/dev directory upon download? For
lein tasks that construct the project classpath, lib/dev would then be
included in it; for the uberjar task, only jars in the lib directory
would be zipped up.

I was working on uberjar because I want to write a similar task as a
plugin, which is an uberjar without clojure & clojure-contrib
included, so that I can drop a project & its dependencies in my local
Clojure development environment, e.g. my Clojure saxon wrapper library
as well as the Java Saxon jars it depends on. Anyone else have this
use case?

Perry

[Phil Hagelberg]

Perry Trolard <tro...@gmail.com> writes:

> Currently for the Leiningen uberjar task, all dev-dependencies are
> included in the resulting standalone jar. In a comment in uberjar.clj,
> there's a note that excluding these is on the todo list. I wonder if
> there are any ideas on how best to do it?

One simple way would be to simply invoke clean and deps before running
the main uberjar code, but that's wasteful.

> I tried to tackle this in a naive way — finding jar files based on the
> project name — but this doesn't catch dependecies futher down the
> chain, e.g. for lein-clojars this'll find the lein-clojars.jar but
> won't find jsch.jar, its dependency. More maven-aware folks may have
> better ideas, but how does this solution sound: segregating dev-
> dependencies in the <project>/lib/dev directory upon download? For
> lein tasks that construct the project classpath, lib/dev would then be
> included in it; for the uberjar task, only jars in the lib directory
> would be zipped up.

I'm a little reluctant to make calculating the classpath any more
complicated than it currently is since that logic needs to be duplicated
in a few places, but I agree that the current situation is not great.

Another idea would be to write a hidden blacklist file when fetching the
dev dependencies. The uberjar could then check that to know which jars
to skip.

I'm not sure which approach I prefer... thoughts?

> I was working on uberjar because I want to write a similar task as a
> plugin, which is an uberjar without clojure & clojure-contrib
> included, so that I can drop a project & its dependencies in my local
> Clojure development environment, e.g. my Clojure saxon wrapper library
> as well as the Java Saxon jars it depends on. Anyone else have this
> use case?

I'm not sure I understand. Why don't you just open your saxon wrapper
project in your IDE directly rather than adding an uberjar generated
from it? Plus if you put clojure and contrib on the classpath before
your uberjar, then the versions in your uberjar will be shadowed.

-Phil

[Perry Trolard]

On Jan 2, 12:35 am, Phil Hagelberg <p...@hagelb.org> wrote:

> I'm a little reluctant to make calculating the classpath any more
> complicated than it currently is since that logic needs to be duplicated
> in a few places, but I agree that the current situation is not great.
>
> Another idea would be to write a hidden blacklist file when fetching the
> dev dependencies. The uberjar could then check that to know which jars
> to skip.
>
> I'm not sure which approach I prefer... thoughts?

I thought about the latter, too. I don't currently understand the
maven stuff enough to know how dependencies get resolved into actual
jarfile names enough to do this (there's the DependenciesTask &c.
business in leiningen/deps.clj), but I'd be happy to investigate, esp.
if someone were willing to point me in the right direction.

> > I was working on uberjar because I want to write a similar task as a
> > plugin, which is an uberjar without clojure & clojure-contrib
> > included, so that I can drop a project & its dependencies in my local
> > Clojure development environment, e.g. my Clojure saxon wrapper library
> > as well as the Java Saxon jars it depends on. Anyone else have this
> > use case?
>
> I'm not sure I understand. Why don't you just open your saxon wrapper
> project in your IDE directly rather than adding an uberjar generated
> from it? Plus if you put clojure and contrib on the classpath before

> youruberjar, then the versions in your uberjar will be shadowed.

I wasn't clear. By "development environment," I meant my `clj` script,
not any particular project per se. In my clj script, I specify a place
on the filesystem to drop jars that I want on the classpath when its
invoked; I know others have setups like this. I always put clojure &
clojure-contrib on the classpath, so I simply want the Clojure library
& any of its dependencies zipped up in a sort of uberjar-minus-clojure
(to avoid shadowing, as you mention).

I suppose I could create a leiningen project for my clj script,
specifying libraries I want as dependencies & then just putting its
uberjar on the classpath, but this requires that the libraries are
available in maven repos. Have other leiningen users solved this?

Thanks,
Perry

[Saul]

On Jan 6, 6:07 pm, Perry Trolard <trol...@gmail.com> wrote:
> I suppose I could create a leiningen project for my clj script,
> specifying libraries I want as dependencies & then just putting its
> uberjar on the classpath, but this requires that the libraries are
> available in maven repos. Have other leiningen users solved this?
>

Have you looked at http://clojars.org ? It seems to have solved the
"libraries available in maven repos" problem.

Saul

[Phil Hagelberg]

Perry Trolard <tro...@gmail.com> writes:

> I wasn't clear. By "development environment," I meant my `clj` script,
> not any particular project per se. In my clj script, I specify a place
> on the filesystem to drop jars that I want on the classpath when its
> invoked; I know others have setups like this. I always put clojure &
> clojure-contrib on the classpath, so I simply want the Clojure library
> & any of its dependencies zipped up in a sort of uberjar-minus-clojure
> (to avoid shadowing, as you mention).
>
> I suppose I could create a leiningen project for my clj script,
> specifying libraries I want as dependencies & then just putting its
> uberjar on the classpath, but this requires that the libraries are
> available in maven repos. Have other leiningen users solved this?

Any time you have dependencies that aren't in repositories, it's going
to cause pain. It's much better to put them in a repository, even if
it's a private/local one, than to bend Leiningen to work with bare jars.

-Phil

[Perry Trolard]

On Jan 28, 12:09 am, Phil Hagelberg <p...@hagelb.org> wrote:

> Perry Trolard <trol...@gmail.com> writes:
> Any time you have dependencies that aren't in repositories, it's going
> to cause pain. It's much better to put them in a repository, even if
> it's a private/local one, than to bend Leiningen to work with bare jars.
>
> -Phil

Fair enough, though I think it was more a case of asking Leiningen to
produce bare jars for use outside of its environment — e.g. standalone
executables, the point of uberjar.

In case others are interested (& it isn't too obvious to mention), I
did end up using Leiningen to take care of my `clj` script setup. I
found that it greatly simplified things compared to the filesystem-
based configuration I used before. The project.clj looks like

(defproject clj "1.0.0-SNAPSHOT"
:description "Jar for `clj` CLI script."
:main clojure.main
:dependencies
[[org.clojure/clojure "1.2.0-master-SNAPSHOT"]
[org.clojure/clojure-contrib "1.1.0-new-SNAPSHOT"]
[clojure-saxon "0.9.0-SNAPSHOT"]
[clojure-http-client "1.0.0-SNAPSHOT"]])

& my `clj` script is now

-----
#!/bin/bash

jopts='-Xmx256m -server -Dfile.encoding=UTF-8'
cp=~/software/clojure/clj/clj-standalone.jar

# add to the classpath
if [[ $1 == "-cp" ]]; then
shift
cp=$cp:$1
shift
fi
# print the classpath
if [[ $1 == --print-c* ]]; then
echo $cp
exit 0
fi

exec rlwrap java $jopts -cp $cp clojure.main "$@"
----

Perry