I tried to make sense of things, but can't. I must be missing something
big here.
It seems that leiningen assumes that you are building an application. It
downloads all dependencies and can build an uberjar for you. That's
great, but what if what I have is a bunch of libraries, all being
developed and modified?
Also, I now have at least six clojure jars in ~/.m2 (huh?!), along with
a collection of other assorted stuff. I don't want to use any of these
clojure jars, I have a checked out git repo with clojure and this is the
only code I want to run. Also, swank-clojure insists on having its own
clojure jar in ~/.swank-clojure (why?).
I used to have a bunch of directories with git repos. I would update
those, run ant to gather code into jars, fiddle with the classpath in my
custom-crafted clj script, symlink things around. It was about as bad as
with Common Lisp. But now things got worse -- I no longer feel in
control of anything. Some things won't build, lein tells me about
"artifacts" (huh?!), things get installed in weird places. My clj script
that gets run by SLIME form Emacs can't find libraries that leiningen
downloaded.
How do people deal with this?
As a more general observation, I think that a large part of Perl's and
Python's success was a unified way of dealing with libraries. There
are certain directories where you can drop libraries and expect them to
work (be found). There is ONE way of running the VM (one script, blessed
by the language creator). Then on top of that there is CPAN, which plugs
into that, downloading dependencies and dropping libraries into those
well-known directories. It isn't perfect, but it works, and is
predictable.
Shouldn't we have a well-known Single Setup for Clojure as well?
I am very worried that Clojure is rapidly going the Common Lisp way,
with each developer having his own precious carefully crafted
setup. Every system is different, everyone uses a different script to
run clojure programs, there is no single place to drop your libraries
into, and libraries have no conventions on where their code is (what do
I add to classpath once I have a library checked out?). I thought
Leiningen was supposed to solve these issues, but either I am missing
something, or it is really a tool to manage dependencies for a single
application.
--J.
My 2c: Leiningen is an important step, but there is still plenty to do.
My workflow: I am developing multiple Clojure libraries and
applications, with some of the dependencies changing daily. Whenever I
want to make a change, I:
In the project being changed:
git tag a new version number
update the project.clj to a new version number
run "lein pom" and "lein jar"
copy the new files to a repos (clojars)
In projects that depend on changed project
update the dependency in project.clj (if it is hard coded and not >=)
run lein deps
This seems adequate to me (although there could be a bit more
automation). The problem of snapshot dependencies remains. The
solution that springs to mind here is to get projects off snapshots
(presumably by releasing more incremental versions). Library authors:
please release lots of versions! (or tell me why this is a bad idea).
Some needs I see:
(0) More point releases of key libs (or some other solution to the
snapshots problem)
(1) Unify around the project structure imposed by lein, and/or
(2) Improve the project structure imposed by lein, if it is broken in
some specific way. (Maybe add a well-known config file for adding
elements to the classpath?)
(3) "lein deps" (or some other task) to clean up old jars. Right now I
am deleting them by hand from the lib directory. (Is this already
solved?)
Stu
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient
> with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
"lein deps" copies dependencies into the project's lib directory.
Various other lein commands assume lib/*.jar is on the classpath, and
if you are writing scripts you should assume that classpath as well.
Is this good enough?
Stu
Oh yeah.
> Some needs I see:
>
> (0) More point releases of key libs (or some other solution to the
> snapshots problem)
Yes!
> (1) Unify around the project structure imposed by lein, and/or
As many of you know, I've always been tepid on lein. I'd rather go
with Maven whole-hog, because that offers the most robust model for
incorporating Java libraries.
The more we can leverage the Java tool set, the better. I'd rather
lein were just a thin POM generator for those who can't stand writing
XML.
-SS
> It seems that leiningen assumes that you are building an application. It
> downloads all dependencies and can build an uberjar for you. That's
> great, but what if what I have is a bunch of libraries, all being
> developed and modified?
That's my situation as well. I did start to use Leiningen, but only for producing releases for the outside world. On my development machine, it's the source code directories of all libraries that are on my classpath. That way I am sure which version of the code I am running.
> with Common Lisp. But now things got worse -- I no longer feel in
> control of anything. Some things won't build, lein tells me about
> "artifacts" (huh?!), things get installed in weird places. My clj script
> that gets run by SLIME form Emacs can't find libraries that leiningen
> downloaded.
>
> How do people deal with this?
Here's what works for me.
Each of my libraries is one "project" in the sense of swank-clojure. I use swank-clojure-project to start Clojure inside slime, and that takes care of my classpath. All I need to do is have the right stuff in the lib subdirectory of each project, so that's where are all the configuration is done.
Instead of doing something like "lein deps" to fill the lib subdirectories, I decided to manage them manually and have only soft links in there that point to the real jars and source directories. There's one soft link to the clojure jar (the one I rebuild after each "git pull" from github), one for clojure-contrib (same principle), and one for swank-clojure (same again). All other dependencies on Clojure libraries are soft links to the source code directory inside the respective project directories. Dependencies on external Java libs are soft links to the jar files, which reside in a single place on my hard disk.
> As a more general observation, I think that a large part of Perl's and
> Python's success was a unified way of dealing with libraries. There
> are certain directories where you can drop libraries and expect them to
> work (be found). There is ONE way of running the VM (one script, blessed
> by the language creator). Then on top of that there is CPAN, which plugs
> into that, downloading dependencies and dropping libraries into those
> well-known directories. It isn't perfect, but it works, and is
> predictable.
Right, but its imperfections have bothered me often enough. For once, if you have different applications that require different versions of the same library, you are in for a major headache. Python setuptools and its eggs are a partial solution, but add a lot of complexity of their own. Another source of endless frustration is maintaining a personal library installation on a machine where I don't have administrator rights and need to work with multiple Python versions. It's possible, but it's a big mess and I need to be very careful to be sure that I really use the version that I need.
I am not too unhappy with the JVM approach to configurability via classpath, but it requires a layer of management tools and/or conventions to be productive. What doesn't fit with my JVM world-view is Maven, and thus leiningen. Their dependency handling model is clearly geared towards application deployment rather than library development. Moreover, I think their model is too complicated, making it difficult to predict which jars will end up being used where. But perhaps that's just my lack of experience.
> Shouldn't we have a well-known Single Setup for Clojure as well?
It would perhaps be a good idea to have one default setup, with the option for expert users to roll their own. For newcomers the current situation is definitely too messy.
> I add to classpath once I have a library checked out?). I thought
> Leiningen was supposed to solve these issues, but either I am missing
> something, or it is really a tool to manage dependencies for a single
> application.
That's my view of what it is, but I can't claim much experience with it.
Konrad.
I have also seen a weird issue in my code base which has two classes
derived from java.lang.Exception. If I do a clean and try compile, I
get an error saying that my.package.MyException class not found. My
workaround is, set :namespaces to only those Exception derived
classes, compile and then set :namespaces to :all.
Maybe I am missing some cool lein thing here or lein needs quite a bit
of work in dependency handling. I am tempted to just stick to mvn
which is quite mature and well-documented. Any clojure-specific needs
could be addressed through a plug-in.
Comments, thoughts?
Thanks,
Praki
On Thu, Mar 04, 2010 at 07:11:59AM -0800, Praki Prakash wrote:
> I have also seen a weird issue in my code base which has two classes
> derived from java.lang.Exception. If I do a clean and try compile, I
> get an error saying that my.package.MyException class not found. My
> workaround is, set :namespaces to only those Exception derived
> classes, compile and then set :namespaces to :all.
You have to :require the namespaces implementing the exceptions in the
files were you use the Exceptions. This will resolve the dependency
problem. See here: http://tr.im/FNmt
Sincerely
Meikel
Out of interest, what scenarios does Maven account for that Leiningen
currently does not?
> The more we can leverage the Java tool set, the better. I'd rather
> lein were just a thin POM generator for those who can't stand writing
> XML.
I personally favour the opposite approach :). Whilst Java provides a
lot of useful functionality, I've never been particularly impressed
with the way most Java libraries are designed.
- James
Right, but its imperfections have bothered me often enough. For once, if you have different applications that require different versions of the same library, you are in for a major headache. Python setuptools and its eggs are a partial solution, but add a lot of complexity of their own. Another source of endless frustration is maintaining a personal library installation on a machine where I don't have administrator rights and need to work with multiple Python versions. It's possible, but it's a big mess and I need to be very careful to be sure that I really use the version that I need.
Also, I now have at least six clojure jars in ~/.m2 (huh?!), along with
a collection of other assorted stuff. I don't want to use any of these
clojure jars, I have a checked out git repo with clojure and this is the
only code I want to run. Also, swank-clojure insists on having its own
clojure jar in ~/.swank-clojure (why?).
I used to have a bunch of directories with git repos. I would update
those, run ant to gather code into jars, fiddle with the classpath in my
custom-crafted clj script, symlink things around. It was about as bad as
with Common Lisp. But now things got worse -- I no longer feel in
control of anything. Some things won't build, lein tells me about
"artifacts" (huh?!), things get installed in weird places. My clj script
that gets run by SLIME form Emacs can't find libraries that leiningen
downloaded.
As a more general observation, I think that a large part of Perl's and
Python's success was a unified way of dealing with libraries. There
are certain directories where you can drop libraries and expect them to
work (be found). There is ONE way of running the VM (one script, blessed
by the language creator). Then on top of that there is CPAN, which plugs
into that, downloading dependencies and dropping libraries into those
well-known directories. It isn't perfect, but it works, and is
predictable.
As many of you know, I've always been tepid on lein. I'd rather go
with Maven whole-hog, because that offers the most robust model for
incorporating Java libraries.
> Imperfections meaning it's perfectly broken :) That's why tools like
> virtualenv exist for Python.
I'd say it's "New Jersey style": good enough for 90% of the users, but
indeed fundamentally broken.
Konrad.
> How do people deal with this?
I don't have any good answers for you; I just want to say "I feel your
pain".
The situation is worse on Windows, where I can't even get leiningen to
work at all. Luckily I'm only working on one clojure project, so I
just copy clojure.jar and clojure-contrib.jar into my project's lib
directory, as well as a faked version of swank-clojure.jar (just a
zipfile of swank source, renamed to ".jar"). That works well enough
for me. Then I have a tiny little Rakefile that builds a classpath
string from the contents of lib/, and has two tasks: run tests, and
start a repl (which I rarely use anymore, since I got swank-clojure
working).
> As a more general observation, I think that a large part of Perl's and
> Python's success was a unified way of dealing with libraries. There
> are certain directories where you can drop libraries and expect them to
> work (be found). There is ONE way of running the VM (one script, blessed
> by the language creator). Then on top of that there is CPAN, which plugs
> into that, downloading dependencies and dropping libraries into those
> well-known directories. It isn't perfect, but it works, and is
> predictable.
Actually, I have noticed a move towards using tools like virtualenv
for Python projects, precisely because depending on system-wide
libraries makes it too non-obvious exactly what your external
dependencies are, as well as creating potential version conficts.
-- Chris
The situation is worse on Windows, where I can't even get leiningen to
work at all. Luckily I'm only working on one clojure project, so I
just copy clojure.jar and clojure-contrib.jar into my project's lib
directory, as well as a faked version of swank-clojure.jar (just a
zipfile of swank source, renamed to ".jar"). That works well enough
for me. Then I have a tiny little Rakefile that builds a classpath
string from the contents of lib/, and has two tasks: run tests, and
start a repl (which I rarely use anymore, since I got swank-clojure
working).
The tools (e.g. Archiva, Nexus et al), documentation, IDE support, integration with other languages e.g. mixed Java/Clojure/Scala/Groovy projects. The more clojure integrates with the existing ecosystem, the more likely it is to succeed, especially in a viral sense. This is the essence of clojure being a JVM language with great integration with Java.
Personally I wish that leiningen effort was instead put towards polyglot maven http://polyglot.sonatype.org/
Antony Blakey
-------------
CTO, Linkuistics Pty Ltd
Ph: 0438 840 787
Nothing is really work unless you would rather be doing something else.
-- J. M. Barre
>
> If Lein evolves to to handle dependencies of dependencies and intelligently generates the classpath based on these dependencies (instead of copying files around) what advantage does Maven really have?
The tools (e.g. Archiva, Nexus et al), documentation, IDE support, integration with other languages e.g. mixed Java/Clojure/Scala/Groovy projects. The more clojure integrates with the existing ecosystem, the more likely it is to succeed, especially in a viral sense. This is the essence of clojure being a JVM language with great integration with Java.
Felix
On 5 Mrz., 00:32, Glen Stampoultzis <gst...@gmail.com> wrote:
On Mar 5, 1:03 am, Felix Breuer <fe...@fbreuer.de> wrote:
> I agree that Windows is a second class citizen as far as clojure
> tools go.
Oh please stop that. I have a stable setup of Gradle + Clojuresque +
VimClojure on Windows. Granted setting up VimClojure on Windows is
tricky due to Vim and Windows specifics, but it's doable. However the
Gradle + Clojuresque part is as easy and trivial as it can get. And
this setup survived already a few updates of the different components
without blowing up a single time. I type "gradle runServer" in the
morning and get a running Jetty/Nailgun combination for dynamic
development with VimClojure - without thinking a single time about the
classpath. Which actually consists of several checked out other
projects (enlive, ring, fleetdb, ...). They were modified non-
intrusively to use clojuresque instead of maven/ant/lein/whatever as
build system. It checks the interdependencies and rebuilds the
required jars if necessary. A change in a dependency is immediately
picked up. If necessary, I can fix a known-to-work version in my local
repository with a few commands.
All without virtualbox, VMWare or whatever. So if one tool doesn't
fill your need, then choose another.
I apologise for sounding a little harsh.
Sincerely
Meikel
One huge drawback I've found with clojure (which it doubtless
inherited from Java) is that you need an actual jarfile in your
classpath, not just the directory containing the jarfile.
martin
On Mar 5, 7:47 am, Martin DeMello <martindeme...@gmail.com> wrote:
> One huge drawback I've found with clojure (which it doubtless
> inherited from Java) is that you need an actual jarfile in your
> classpath, not just the directory containing the jarfile.
With newer Java versions you can specify wildcards. java -cp lib/
\* ... will pick up everything in lib for the classpath.
Sincerely
Meikel
On 5 Mar, 01:03, Felix Breuer <fe...@fbreuer.de> wrote:
> I am also on Windows and have struggled a good deal with Leiningen (I
[...]
> On 5 Mrz., 00:32, Glen Stampoultzis <gst...@gmail.com> wrote:
>
>
>
> > On 5 March 2010 04:39, Chris Perkins <chrisperkin...@gmail.com> wrote:
>
> > > The situation is worse on Windows, where I can't even get leiningen to
> > > work at all. Luckily I'm only working on one clojure project, so I
[...]
> > Windows seems to be a second class citizen as far as clojure tools go. I
> > wasn't able to figure out how to get it going on Windows either which makes
> > things much harder considering so many clojure projects are now using it.
[...]
First of all have you ever tried this script?
http://github.com/technomancy/leiningen/blob/master/bin/lein.bat
Secondly have you seen this thread?
http://groups.google.com/group/leiningen/browse_thread/thread/2474d2e9019ee29c
There was almost no feedback at all.
Have you reported any issue with Leiningen on Windows?
http://github.com/technomancy/leiningen/issues
Leiningen is not perfect and I'd like to improve this tool (including
Windows version),
but so far I can see only some general complaining. Please report
concrete
issues and we will be able to make concrete improvements.
Br,
Rob
Neat! Never knew that. So now it *is* possible to have a standard
clojure lib directory the way ruby has its gems directory or python
its site-libs.
martin
Furthermore, all java tools play well on windows, so using ant, maven,
will work just as well on windows and on linux, mac, etc.
And you also have the Eclipse plugin for Clojure, the netbeans plugin
for clojure, the IntelliJ IDEA plugin for clojure, which work on
windows too !
> Granted setting up VimClojure on Windows is
> tricky due to Vim and Windows specifics, but it's doable. However the
> Gradle + Clojuresque part is as easy and trivial as it can get. And
> this setup survived already a few updates of the different components
> without blowing up a single time. I type "gradle runServer" in the
> morning and get a running Jetty/Nailgun combination for dynamic
> development with VimClojure - without thinking a single time about the
> classpath. Which actually consists of several checked out other
> projects (enlive, ring, fleetdb, ...). They were modified non-
> intrusively to use clojuresque instead of maven/ant/lein/whatever as
> build system. It checks the interdependencies and rebuilds the
> required jars if necessary. A change in a dependency is immediately
> picked up. If necessary, I can fix a known-to-work version in my local
> repository with a few commands.
>
> All without virtualbox, VMWare or whatever. So if one tool doesn't
> fill your need, then choose another.
>
> I apologise for sounding a little harsh.
>
> Sincerely
> Meikel
>
Maven supports a wide range of development/deployment models already.
Whatever you want to do (including compile/test/run Clojure code)
there's a plugin for that. Why recreate all this functionality in
Leiningen?
Yes, the Maven configuration model isn't pretty, but if you follow the
conventions it works. Polyglot Maven will obviate the need to write
XML. But frankly, I don't mind a little XML if it gets the job done.
Basically, I'm skeptical that any tool developed exclusively by/for
the Clojure community could ever achieve the scope and utility of a
tool used by thousands of people and companies. We're a small
community, we can't do it all alone.
-SS
I didn't know about Polyglot. Thanks. Looks like a very nice tool.
<http://polyglot.sonatype.org/clojure.html>
--
Ramakrishnan
Most of the time, I won't need the wide range of functionality that
Maven supports. For the most common tasks, such as compiling and
packaging, Leiningen is the better build tool. It's conceptually
simpler, more understandable, and easier to extend. I'd rather use a
build tool that covers 80% of tasks well, than a tool that covers 95%
poorly.
- James
Agreed.
> (1) Unify around the project structure imposed by lein, and/or
Yes, but it should be noted this isn't really the "leiningen" project
structure; it has been the conventional Clojure layout in the
community from long before leiningen was released.
> (2) Improve the project structure imposed by lein, if it is broken in some
> specific way. (Maybe add a well-known config file for adding elements to the
> classpath?)
People have asked for this, but I have yet to hear a good use case for
it. Mostly it's been requested by people who don't realize how easy it
is to push their own jars to clojars. But if someone wants to
implement it I will not refuse the patch.
> (3) "lein deps" (or some other task) to clean up old jars. Right now I am
> deleting them by hand from the lib directory. (Is this already solved?)
Should be able to "lein clean". =)
-Phil
I agree that lein+clojars is good enough for lib dependencies. My two
use cases so far are places where the classpath needs to point to
something other than outside libs:
(1) getting ./config on the classpath so I can keep properties files,
json config, whatever, separate from code and lib
(2) having more than one kind of tests, in different dirs (e.g. specs,
features) and having their classpaths work through lein.
Stu
The turning point was when I realized that the key to using Maven is
understanding the Maven model - understanding how Maven wants to
build. You don't tell Maven what to do. You just put the files in the
places Maven expects, and Maven then _knows_ what to do. My rule of
thumb is: if I'm spending a lot of time tweaking configuration and
overriding defaults in the pom file then I'm probably doing it wrong.
When I started programming in Clojure I immediately looked for a Maven
plugin to build Clojure projects. The first project I worked on I did
using the plugin. For my next Clojure project I decided to give
Leiningen a spin. Initially I was very impressed. It was easy to get a
project up and running, easy to build, easy to run tests, easy to run
a swank server etc. For a couple of weeks I was very happy. But.....
I finally hit a point where Leiningen was no longer working for me. My
current project has a Swing GUI, built using Matisse (the Netbeans GUI
builder). I couldn't find a way to build the Swing portion and the
Clojure portion together using Leiningen. I also couldn't find a way
to divide my project up into different sub-modules with one build
(maybe I'm guilty of trying to use Leiningen like I use Maven). In any
case, I ended up converting the project to use Maven.
Maven has a steep learning curve, but once you get it, it's incredibly
powerful.
"The philosophy of Tai Chi Chuan is that if one uses hardness to
resist violent force, then both sides are certain to be injured at
least to some degree. Such injury, according to tai chi theory, is a
natural consequence of meeting brute force with brute force. Instead,
students are taught not to directly fight or resist an incoming force,
but to meet it in softness and follow its motion while remaining in
physical contact until the incoming force of attack exhausts itself or
can be safely redirected, meeting yang with yin." (http://
en.wikipedia.org/wiki/Tai_chi_chuan)
If you fight with Maven, you will lose. If you meet it in softness and
follow its motion you will attain build enlightenment...
;-)
FInally, I agree with Stuart that polyglot Maven could / should be the
way forward. I don't really wrestle with pom files much manually
anyway since Netbeans does a very good job of automatically updating
them. But Leiningen has given me a taste of writing build files in
Clojure, so I will definitely be investigating the polyglot option
soon.
To learn more about Maven I would recommend
http://www.sonatype.com/books/mvnref-book/reference/public-book.html
and http://www.sonatype.com/books/mvnex-book/reference/public-book.html
As I'm one of the people that's requested this feature... I'll add my
reason too:
(3) Managing proprietary libraries and API's that you are developing
or using... Yes you can throw these into your own maven repo, but
then you also need to manage, configure and secure a server (when
working in a team).
R.
As I'm one of the people that's requested this feature... I'll add my
reason too:
(3) Managing proprietary libraries and API's that you are developing
or using... Yes you can throw these into your own maven repo, but
then you also need to manage, configure and secure a server (when
working in a team).
This is actually what the resources/ directory is for.
> (2) having more than one kind of tests, in different dirs (e.g. specs,
> features) and having their classpaths work through lein.
It would be best if this could be handled by plugins rather than
specified in every project.clj. Currently the plugin framework doesn't
expose the hooks that would be needed for this, but it's definitely on
the short list for version 1.2.0.
> (3) Managing proprietary libraries and API's that you are developing
> or using... Yes you can throw these into your own maven repo, but
> then you also need to manage, configure and secure a server (when
> working in a team).
I think it would be great if it were easy to run your own clojars
server. Not only for mirroring purposes (poor ato isn't going to be
able to eat the bandwidth bills forever) but also to deal with
situations like this.
But there may be other valid reasons that nobody has even encountered
yet, so I favour having total flexibility anyway.
-Phil
If you're using git, this should be avoided. git is remarkably bad at
storing binary data. A number of projects I've been watching have made
this mistake, and they've all had to correct it by rewriting history
down the line, which is very awkward.
-Phil
I seem to recall that recent versions of Git have improved
significantly in this area. However, there's still the fundamental
problem that changing the dependencies increases the size of the
project. New developers will not only have to download your source
files, but the binary diff of every dependency change.
So yep, it's a bad idea :)
- James
> If you're using git, this should be avoided. git is remarkably bad atI seem to recall that recent versions of Git have improved
> storing binary data.
significantly in this area. However, there's still the fundamental
problem that changing the dependencies increases the size of the
project. New developers will not only have to download your source
files, but the binary diff of every dependency change.
So yep, it's a bad idea :)