Libraries and build management hell
Michal B
To use libraries, you need to learn how to operate half a dozen build tools and git because each library author distributes their library differently. If figuring out how to install an IDE with clojure wasn't bad enough, now you need to figure out how to install and use each of the tools with it.
I'm not saying build tools are useless, on the contrary. It's just that most of the time, we want to sling two or three libraries together and code. Right? There is no need to start a project with a bunch of template files and an elaborate directory structure and to start configuring dependencies and to rely on some magic happening that makes your program run.
I think we over-engineered the build process to support the big projects and forgot the common case. Most projects are simple.
Let's remove this incidental complexity by returning to simplicity. Keep the build tools for the heavyweights and get back in touch with your libraries.
Instead of having a complicated installation guide for your library, have a Download section in your site. Have there a link to the latest stable version of your library as a jar file or, if necessary, a zip file with your jar and and all the necessary dependency jars (sane library authors won't mind). For a zip, shortly describe what's in it - library names and versions, and links to their sites. That's it.
I think most JVM users know or can quickly figure out how to take jars and put them in their project's classpath. It's simple to do with all IDEs (or without one) and there is no need to learn or install additional software or edit configuration files. Starter scripts should include in the classpath all jars in the current directory or jars/ directory by default.
Instead of managing libraries inside a dependencies file, you do it directly with the jar files. If the project gets too big, bring in the build tools.
What are your thoughts on this issue?
Timothy Baldridge
I used to think it was hard until I read up on lein. Can't get much
simpler than clojars and lein:
http://clojars.org/
http://alexott.net/en/clojure/ClojureLein.html
Now I'm starting to think that I actually like the lein method over
python easy_install or ruby gems.
Timothy
--
“One of the main causes of the fall of the Roman Empire was
that–lacking zero–they had no way to indicate successful termination
of their C programs.”
(Robert Firth)
gaz jones
for dependencies and a couple of shell scripts for building etc), but
leiningen makes it so easy for clojure that its more work _not_ to use
it... at least that has been my experience.
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your
> first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
Sean Corfield
> Why does it have to be so complicated to use libraries?
I can't imagine it being much simpler than using Leiningen...
> To use libraries, you need to learn how to operate half a dozen build tools
Just one: Leiningen. You can learn Cake or Maven instead if you want, I guess.
> and git because each library author distributes their library differently.
Hmm, I haven't needed git for anything I'm not actually contributing
to - with the exception of ClojureScript itself (and given the state
of that, I think a "build from source" approach is acceptable for
now).
> Right? There is no need to start a project with a bunch of template files
> and an elaborate directory structure and to start configuring dependencies
> and to rely on some magic happening that makes your program run.
Well, you could download JAR files and then use the java command with
the appropriate classpath and other arguments to run your script...
but that seems more work to me than telling Leiningen which libraries
I want and then telling Leiningen to run my -main function - with
Leiningen handling all of the dependencies and the classpath stuff
etc.
I don't think I'd choose to not use Leiningen, even on a single file
project that used no additional libraries:
lein new thing
cd thing
vi s[tab][tab][tab]
...sling together some code... ZZ
lein run -m thing.core
Done!
(I know, the heresy of using 'vi'... :)
--
Sean A Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
World Singles, LLC. -- http://worldsingles.com/
Railo Technologies, Inc. -- http://www.getrailo.com/
"Perfection is the enemy of the good."
-- Gustave Flaubert, French realist novelist (1821-1880)
Brent Millare
git://github.com/bmillare/dj.git
Acts more like a distro then a build tool though.
On Jul 28, 7:49 pm, Sean Corfield <seancorfi...@gmail.com> wrote:
Luc Prefontaine
We build a dozen projects here with a mix of Clojure and Java and we only use Leiningen.
The biggest effort was to consolidate the dependency libs pulled by Leiningen in our different
projects. You end up with multiple version of the same jar, a result of the dependency solving
done by Leiningen from the Maven world.
Copying the jar files pumped by Leiningen (cp */lib/*.jar) in a single folder and then pruning them
to retain the highest versions was the biggest challenge we faced.
All the libs we are referring to in dev, test and prod are stored in a single folder.
In dev we use Eclipse but have defined a number of user libraries to point to jars in this folder,
splitting them by purpose (Oracle, ...)
Since we want a stable slate we control the 75+ libs we monitor this closely and do not allow dependency resolution to
sc.. it up between different projects. If a newcomer is pulled by Leiningen in a project resulting from a change,
we just compare it with what we have and add it or not to it. It's definitively easier to do these days.
In Jan. 2009, when we went in prod, we we're not using Leiningen back then an we had around 140 libs,
all the dependencies were resolved/managed by hand... yurk !
Now the build process (we deliver our software AOT) is a piece of cake (no pun intended).
We defined a couple of generic scripts (lein-jar, lein-install) and call them from a simple
Bourne shell script to build the whole system.
We use Archiva internaly as a simple Maven repo so we can store the snapshots and stable
releases of our software and can refer to them in project.clj files.
It also caches all the external dependencies for us.
If we cannot find a maven artifact because an open source project did not push anything to Maven,
we build it from source and store it in Archiva et voilà ! We can then pull it from the Maven cloud
without changing anything.
Same thing if the jar exists apart from the Maven world. We just push it into Archiva.
Leiningen for us = less time to build, no more handcraft build scripts, ...
I cannot think about something simpler yet. If anyone has a suggestion, I am opened to it.
If I compare our build process and dependency management to other environments that I know (and I know a few),
it's still the most powerful thing I saw since sliced bread :)
BTWY, we use git for source control but that's a breeze compared to the hell we went through with the
build process before jumping in the Leiningen wagon.
Luc P.
--
Luc P.
================
The rabid Muppet
Mark Rathwell
Michal B
Michal B
Davi Santos
I don't understand what the Leiningen Intellij plugin is for.
josh rotenberg
> Why does it have to be so complicated to use libraries?
Heh. My current project is C/C++, and we are using gnu autotools for
the build. A few dependencies use pkgconfig, some have m4 macros (that
are out of date), one has its own backtick-config kind of thing, and
another has no real handy way to find its dependencies. In addition,
to get everything to play nice via unit tests takes our own fragile
voodoo. So for me, right now Leiningen, clojars, and even cloning
something from github is basically nirvana.
Josh
Alex Osborne
> To use libraries, you need to learn how to operate half a dozen build
> tools and git because each library author distributes their library
> differently.
There is a simple de facto standard, which in typical Clojure-style
we've inherited from the Java community for the sake of
interoperability. Library authors publish a jar file (which is just a
zip of the Clojure source code) to a HTTP repository which has a
directory structure following the Maven repository conventions.
People don't necessarily have to use Maven, often they use Leiningen or
Cake and sometimes they don't use a build tool at all. You can
certainly do it by hand or with a shell script easily enough.
A popular repository for open source community libraries is Clojars,
although some projects like to host their own or use Maven Central.
Often companies will have an internal repository which could just be
on a directory on a network share somewhere. Some use repository
software like Nexus or a local instance of the Clojars webapp, but
that's strictly optional.
> If figuring out how to install an IDE with clojure wasn't bad enough,
> now you need to figure out how to install and use each of the tools
> with it.
No you don't. Let's suppose we want to add the robert.hooke library to
our project without any build tools.
1. Goto http://clojars.org/.
2. Click "browse the repository" (down the bottom).
3. Click "robert".
4. Click "hooke".
5. Click "1.1.2".
6. Click "hooke-1.1.2.jar" and save the jar file to your project.
The repository is really nothing more than a bunch of directories
each containing a jar files and another file containing a list of
dependencies. I'm not sure how you could make it any simpler.
Admittedly Clojars' is not perfectly user-friendly and the search and
browse could do with improving. I'm not currently motivated to do this
work myself (due to lack of time, UI design not being my passion and
that it's currently "good enough" for me personally to live with) but
the code is open source and I'm happy to take patches that do so.
I'm actually kind of hoping that eventually someone with time and vision
will come along and slowly take it over. I created Clojars to scratch
an itch: I was just really unhappy with the existing options. I
maintain it in the sense of keeping it running, but it really is pretty
neglected.
I really hate thinking about build tools and dependencies, I just want
them to work and get out of my way. Unfortunately that means at work I
often end up being the "build/deployment guy" just because I'm the
first one to gets sufficiently annoyed to go and do something about it.
> Instead of having a complicated installation guide for your library,
> have a Download section in your site. Have there a link to the latest
> stable version of your library as a jar file or, if necessary, a zip
> file with your jar and and all the necessary dependency jars (sane
> library authors won't mind). For a zip, shortly describe what's in it
> - library names and versions, and links to their sites. That's it.
This actually is exactly how most Clojure libraries were distributed
before the creation of Leiningen. Maven of course existed and some
people who had a strong Java background were using it but many were
quickly put off by its verbosity, steep learning curve, poor
documentation and lack of transparency.
So what you describe is exactly what you'd get: ad-hoc heterogeneously
structured downloads with an accompanying zip or tarball full of jar
dependencies if you were lucky.
Said zip files could vary wildly, but could contain something like:
clojure.jar
contrib.jar
clj-http-2011.jar
Too bad if you want to know what version a dependency is. Often the
clojure and contrib jars were just some random snapshot of SVN (this was
before the move to Github). Maybe the library author even modified them
themselves so they didn't match a mainline version at all.
Even for small projects I would argue this is not simpler, it's more
complicated.
Now you could argue that it would be quite handy for people who want to
avoid tools and configuration files if there was a web site that
automatically generated a zip containing a jar and all of its
dependencies for download in one go. I can see myself using such a
thing from time to time but not for my primary programming.
There are some minor security and legal issues (eg license
compatibility), but I would be happy take a patch to Clojars that added
or integrated with it. It really is just moving the dependency
management tool from being client-side to server-side though.
> Instead of managing libraries inside a dependencies file, you do it
> directly with the jar files. If the project gets too big, bring in the
> build tools.
>
> What are your thoughts on this issue?
I think automated dependency management is a necessary evil unless you
basically don't reuse code. I don't particularly like dependency
management. I don't particularly like build tools in general. But I
really don't want to turn back the clock to the time before we (the
Clojure community) used it. That was even worse.
Hopefully some bright spark will come along and revolutionise dependency
management, like how Rich revolutionised the notion of state in
programming. As far as I'm aware nobody has so far, for any programming
language (or OS distro), I'm not completely happy with any of them.
It needs more hammock time.
Jonah Benton
>
> Hopefully some bright spark will come along and revolutionise dependency
> management, like how Rich revolutionised the notion of state in
> programming. As far as I'm aware nobody has so far, for any programming
> language (or OS distro), I'm not completely happy with any of them.
>
> It needs more hammock time.
What golang has done is interesting, insofar that a more complete
lifecycle is supported with the out of the box tools.
James Reeves
Why is it a burden?
Let's take a step back and consider a simple library with a few lines
of code and a couple of dependencies.
With Leiningen or Cake, the process would be:
1. lein new foobar
2. cd foobar
3. Add :dependencies to project.clj
4. Add :main namespace to project.clj
4. Write the code
5. lein run
I'm having trouble seeing how this could be much simpler, unless the
dependencies were somehow resolved automatically.
So what about if we forgo a build tool? Then the process would be:
1. mkdir foobar
2. cd foobar
3. Download dependencies manually as jars, including Clojure itself
4. mkdir lib
5. Place all dependencies in lib
6. mkdir -p src/foobar
7. Write code
8. java -cp 'lib/*.jar:src' foobar.core
I'm not seeing how this is any simpler. The act of going around and
downloading dependencies manually also seems particularly time
consuming.
And once you've written your application, how do you publish it? With
Leiningen and the Clojars plugin, it's:
1. lein push
But manually, it would be:
1. cd src
2. jar cf foobar-1.0.0.jar */*.clj
3. cd ..
4. mv src/foobar-1.0.0.jar lib
5. cd lib
6. zip cf foobar-1.0.0-standalone.zip *.jar
7. Upload resulting zip file to your project website
As far as I can see, even with the simplest of projects, there is no
reason not to use Cake or Leiningen. You'll have to do much more work
without them.
- James
Lee Spector
On Jul 29, 2011, at 8:14 AM, Alex Osborne wrote:
>
> 6. Click "hooke-1.1.2.jar" and save the jar file to your project.
>
> The repository is really nothing more than a bunch of directories
> each containing a jar files and another file containing a list of
> dependencies. I'm not sure how you could make it any simpler.
FWIW for some JVM newcomers (like me when I started) #6 involves some mysteries related to where exactly the jar should be saved and how exactly the other code can be told to find it. It varies depending on how you run your code (e.g. whether you have to figure out how to construct a -cp argument or do other non-obvious things in a complex IDE, etc.) and then there's the hierarchical namespace/directory structure correspondence, which I found initially surprising and confusing.
Bottom line is that I agree that leiningen makes all of this simpler, assuming that your editor and other tools play nice with leiningen. I'm just pointing out that the manual method is actually a little worse than you've implied, for people new to JVM programming.
-Lee
octopusgrabbus
On Jul 28, 10:06 pm, Mark Rathwell <mark.rathw...@gmail.com> wrote:
> The problem with jar downloads as the default distribution method is that
> non-Java people, and even plenty of Java people, seem to have problems
> consistently setting classpaths correctly. Seems much more straightforward
> to just have lein take care of that for you.
revolve around understanding the differences in how different packages
are built.
Also, the structure of different library's build directories that I
have seen on github varies quite a bit.
I am building an application and only need my source in ./project_name/
src/project_name.clj, but some libraries I have seen have a
project_name/project_name.clj under source.
For beginners, it is a roadblock. But because of this group, the
roadblock, at least for me, is temporary.
>
> As for complicated installation instructions, libraries meant to be used in
> lein have installation instructions that generally look like this:
>
> [compojure "0.6.4"]
>
> The other alternative is to have environment package managers like Python
> with easy_install and Ruby with gems. Then you need to bring virtualenv or
> Ruby version manager into the picture to use different versions of the same
> library in different apps. The Java world has tended towards dependencies
> being coupled with the application, rather than the environment. There are
> good arguments for both sides, but either way, I don't think it can get much
> easier than lein.
>
> - Mark
>
Stefan Kamphausen
On Thursday, July 28, 2011 11:23:58 PM UTC+2, Michal B wrote:
Why does it have to be so complicated to use libraries?
because it is a complicated topic.
Maybe I can compare it to other ecosystems.
* CPAN for Perl is a paradise for functionality. However, trying to use CPAN together (or against) the means of the operating system (sax Linux) is embarrassing (with Gentoo being the only exception I know of - different story). After 10+ years of professional programming in this field I only use standard installation plus pure Perl implementation libraries without dependencies for production today. Sucks.
* Ruby Gems? No way for a newcomer to understand this quickly. Used it primarily for Rails and gave up with setups for Rails 3.0 and 2.something on differents OSes.
* Python Eggs? A little less involved than Gems in my experience, but what do I know? Only used a few libs either for one-shot programs on my machine or slowly moving targets elsewhere.
* Emacs Lisp libraries? I've been doing this for 15+ years and it kinda works. But Clojure itself is a nice example, where it doesn't, what with the SLIME incompatibilities.
* Common Lisp? This community worked ages on trying to get past the "CL is cool, but libraries suck"-stage and only recently quicklisp came around, which may help (unless it gets to the stage of conflicting with the OS; see CPAN above).
* Java? Ho, humm-- as treebeard may say-- ho, humm, Java is ugly, Maven is even uglier, but in a way ... it works.
IMHO that's one big selling point of Clojure: you can use the Java-ecosystem. It's not nice, but it is definitely a huge success story. And thus, it is worth getting into it. Even more so, if Leiningen will help you for the first time.
Kind regards
Stefan
Alex Osborne
> FWIW for some JVM newcomers (like me when I started) #6 involves some
> mysteries related to where exactly the jar should be saved and how
> exactly the other code can be told to find it. It varies depending on
> how you run your code (e.g. whether you have to figure out how to
> construct a -cp argument or do other non-obvious things in a complex
> IDE, etc.) and then there's the hierarchical namespace/directory
> structure correspondence, which I found initially surprising and
> confusing.
>
> Bottom line is that I agree that leiningen makes all of this simpler,
> assuming that your editor and other tools play nice with
> leiningen. I'm just pointing out that the manual method is actually a
> little worse than you've implied, for people new to JVM programming.
Oh I totally agree. Perhaps I was being misleading there. I was
replying to Mikal's hypothesis that if library authors used a manual
release process instead of version control and build tools everything
would be considerably simpler for library consumers who know how to feed
jars to the JVM. It's definitely worth questioning your assumptions
from time to time and I'm glad he brought it up, it is giving me some
ideas. I don't think it's a good solution though because the tools at
the very least give us consistency.
I don't think the maven standard is perfect. The whole "group id" thing
in my opinion is confusing and not quite right (which is why with
Clojars I deviated from how Maven Central uses it). But at least we
have a way that all the tools (and people) can interoperate with each
other and I'm willing to live with some warts in order to get that.
The JVM does not make things fun. It was only in Java 6 that the
ability to use wildcards in the classpath (-cp lib/*) was added for
example, in earlier versions you had to specify every single jar file
one by one. But that's not the fault of build tools, the tools exist
precisely because the JVM does not ship with a good solution to this
problem.
Tools with a system install model like RubyGems and ez_install seem
deceptively easier when you're a beginner. I think they do work
well in a highly structured and well curated ecosystem (eg Debian) but
that takes serious commitment. Unfortunately once you get past the
beginner stage and start dealing with multiple versions of libraries and
such it quickly turns into a huge headache and you end up with awful
workarounds like RVM.
I work at a shop that uses both Ruby and Java. For our Ruby projects
after much frustration we've just resorted to chucking all the gems for
a project into the project's version control and pointing at them with
GEM_HOME. Some of our developers try to use Bundler but I find it to be
more trouble than it's worth (particularly when deploying to JRuby).
It's amazing how much time someone can waste messing around trying to
get Bundler to behave right.
Compared to that the Leiningen model as a default is downright blissful.
Clojure/core are pushing a third model. Get all the good stuff into
contrib. I guess you could compare that to something like FreeBSD and
in a sense the way the Linux kernel project works. Ultimately that
very likely will result in higher quality output and is not bad for
beginner users. But to get that quality it necessarily means a bigger
commitment and a higher barrier to entry for authors.
http://groups.google.com/group/clojure/browse_thread/thread/c40ff3e876b4b370
I'm glad to see that core are trying to make contrib model more
accessible. For a long time with Rich as the sole gatekeeper it was
a pretty select club. Even now though it could be better communicated,
the references to contrib on the website are pretty vague if you don't
already know what it is.
I've been fairly heavily following Clojure for a couple of years now, I
have signed the CA and I'm still not completely sure if I'd be welcome
to contribute to contrib. It's intimidating. Maybe that's just
timidity on my part though.
Michal B
Instead of iterating what's available today and how much better it makes things (I know), I hope we can question our assumptions about how things are and get a discussion going on how to make things better. If not for simplicity, do it for the new users who today when visiting the Getting Started pages are overwhelmed by rampant choice and endless technical procedures.
David Nolen
Instead of iterating what's available today and how much better it makes things (I know), I hope we can question our assumptions about how things are and get a discussion going on how to make things better. If not for simplicity, do it for the new users who today when visiting the Getting Started pages are overwhelmed by rampant choice and endless technical procedures.
Mark Rathwell
This discussion is getting a lot of "What are you talking about? Leiningen is what you're looking for!" replies. I know. I'm a happy lein user, and use it and clojars frequently with some of my larger projects. I'm very grateful for the work Phil and the other contributors did. It helped me tremendously. There's no need to get defensive. As I tried to emphasize several times in my post, this was not an attack on lein or other build tools - they certainly have their place in the development process. I was trying to get a discussion going on the current library situation, which I think if everyone took back a step they'd agree is far from ideal. Lein is a remedy, but not a solution. Is lein as good as it gets?
Instead of iterating what's available today and how much better it makes things (I know), I hope we can question our assumptions about how things are and get a discussion going on how to make things better. If not for simplicity, do it for the new users who today when visiting the Getting Started pages are overwhelmed by rampant choice and endless technical procedures.
Mark Engelberg
> One thing, though, is new users coming from Python/Ruby/etc really do seem
> to have a hard time adjusting to the fact the standard is not to have a
> Clojure installation, with a clj executable that you run scripts through,
> and with globally installed libraries. Maybe a Clojure installer for
> Mac/Windows/Linux, that installs a clj executable on the path, and that has
> its own directory that it adds to the classpath when it is run, where you
> could install jars (maybe using a lein plugin) that would be available
> globally, and not interfere with the current standard. But that is yet more
> choice to overwhelm user, so I don't know.
I think Mark Rathwell's post is on the right track. The notion of a
directory for globally installed libraries that you can set once and
automatically have all Clojure tools respect them would be a valuable
one. The issue I have with the build tools is that they all presume
you want to *build* something. 95% of the time, I just want to open a
file, write some Clojure code, and interactively test it in a REPL.
And yes, there are certain libraries I tend to use, and I want them to
always be available. Right now, for every file I want to tinker with,
I have to do lein new make-up-some-project-name. Then I have to go in
and edit the project.clj, trying to remember the names and version
numbers of every library I might use (and this has gotten harder to
remember now that contrib is split into separate modules), add some
development dependencies so I can connect from slime. Or I can copy
over the project structure from another lein project and go through
and delete all the irrelevant things. Then I have to lein deps and
wait 10 minutes for it to download the universe. Then I can lein
swank, and connect to it from emacs, all so that I can start editing a
file with the right classpaths, and play around with it in the REPL.
In Python, I pop open IDLE, start typing some code. Save it anywhere
I want, and hit F5, and then I'm in a a REPL where I can interact with
the code I just wrote. Any libraries I've downloaded are available.
In Racket, the story is much like Python, but even a bit better. Any
library from Racket's Planet repository (sort of analogous to
Clojars), you can refer to in your code, and it will automatically
download and install if you don't have it -- no need to install
libraries ahead of time, and you can email your Racket script to a
friend and trust that it will work on his system regardless of
dependencies.
I would love to have a more streamlined way in Clojure for my personal
common case -- writing a short script and using it interactively.
Lee Spector
On Jul 30, 2011, at 11:24 AM, Mark Engelberg wrote:
> I would love to have a more streamlined way in Clojure for my personal
> common case -- writing a short script and using it interactively.
+1 on the whole perspective presented here, most of which I cut.
-Lee
Michal B
The issue I have with the build tools is that they all presume
you want to *build* something. 95% of the time, I just want to open a
file, write some Clojure code, and interactively test it in a REPL.
I would love to have a more streamlined way in Clojure for my personal
common case -- writing a short script and using it interactively.
One problem with this approach is versions. Maybe a namespace should refer to a library's API version instead of referring a "versionless" library. I'm not sure if that's really an improvement, though.
Sean Corfield
> Mac/Windows/Linux, that installs a clj executable on the path
This was actually how I started with Clojure on Mac OS X (I think the
project was Clojure-MacOSX or something like that) but I quickly found
it was clunky and actually did _not_ match my normally way of working.
FWIW, Leiningen is essentially a central install with all the basic
scripts built in.
On Sat, Jul 30, 2011 at 8:24 AM, Mark Engelberg
<mark.en...@gmail.com> wrote:
> 95% of the time, I just want to open a
> file, write some Clojure code, and interactively test it in a REPL.
Really? Genuine question.
I spend 95% of my time maintaining a 70kloc project (of which ~1,500
is currently Clojure but that's growing and the overall project line
count is shrinking as the amount of Clojure grows). I always have the
project's REPL running in my IDE so I can interactively test code
anyway. The other 5% I work in one of the various REPLs I keep open in
terminal windows (at least one for each of Clojure 1.2 and 1.3).
> And yes, there are certain libraries I tend to use, and I want them to
> always be available.
Once you've used a library, it'll be in your local repo so just adding
the dependency to a scratch project - even if it ends up with a slew
of dependencies - is a quick edit and a very quick lein deps and
you're good to go.
> Then I have to lein deps and
> wait 10 minutes for it to download the universe.
Most new libraries download in a few seconds and once they're local
there's no download overhead (unless you're using snapshots and the
latest version check has to be performed).
> Then I can lein
> swank, and connect to it from emacs, all so that I can start editing a
> file with the right classpaths, and play around with it in the REPL.
I can only attest to Eclipse/CCW but running a REPL with the right
classpath is pretty straightforward.
I hear what folks are saying and I wonder if this all comes down to
expectations based on whether folks come from a JVM background? After
working on the JVM for about 14 years, Clojure, Leiningen and
Eclipse/CCW is about the simplest development setup I've ever had.
Perhaps this is just the price we pay for living on the JVM?
Michal, what's your language background? I see Mark R and Mark E
referring to Python so I can see their point of view on that
comparison.
--
Sean A Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
World Singles, LLC. -- http://worldsingles.com/
Railo Technologies, Inc. -- http://www.getrailo.com/
"Perfection is the enemy of the good."
-- Gustave Flaubert, French realist novelist (1821-1880)
> its own directory that it adds to the classpath when it is run, where you
--
Sean A Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
World Singles, LLC. -- http://worldsingles.com/
Railo Technologies, Inc. -- http://www.getrailo.com/
Phil Hagelberg
> always be available. Right now, for every file I want to tinker with,
> I have to do lein new make-up-some-project-name. Then I have to go in
> and edit the project.clj, trying to remember the names and version
> numbers of every library I might use (and this has gotten harder to
> remember now that contrib is split into separate modules), add some
> development dependencies so I can connect from slime. Or I can copy
> over the project structure from another lein project and go through
> and delete all the irrelevant things.
That sounds like a lot of unnecessary steps. Why delete the irrelevant
things? Or better yet, why start a new project at all? What you're
asking for seems to be "a collection of dependencies that's not
attached to a location on disk", but I fail to see why having it exist
in a certain place on disk is a problem to begin with. If you want a
scratch-pad, why not just re-use the same project?
> Then I have to lein deps and wait 10 minutes for it to download the universe.
If they actually are dependencies that you've used with other
projects, then they'll all be cached; no downloads necessary.
> I would love to have a more streamlined way in Clojure for my personal
> common case -- writing a short script and using it interactively.
On the other hand, maybe you don't want Leiningen at all. Why not use
Jark or cljr instead?
-Phil
Mark Engelberg
> On Sat, Jul 30, 2011 at 8:24 AM, Mark Engelberg
> <mark.en...@gmail.com> wrote:
>> 95% of the time, I just want to open a
>> file, write some Clojure code, and interactively test it in a REPL.
>
> Really? Genuine question.
You're probably right that by *time*, I'm mostly working in large
projects that I've already set up (although most of my work is at the
REPL, rather than with the focus of building a jar). I guess what I'm
thinking of is that 95% of the time when I go to start something new,
it ends up being for a short-lived task, so psychologically, the extra
hassle associated with creating a project infrastructure makes me want
to reach for something other than Clojure.
Phil makes a reasonable point that it is possible to create a single
"project" for one-off tasks. I'm not sure how well that would work
with the way I have things organized, and with my source control, but
it's something I can look into.
Ken Wesson
>
> I'm all for a better, easier solution that is better in most ways. What I'm
> saying is:
> 1. I don't want to go back to downloading jar files from the websites of all
> of the libraries I want to use in a project and tracking different versions,
> no matter how large or small the project is, as suggested in your OP. I
> think the centralized lein/clojars/github is an improvement over that in
> almost every way.
"Almost" being the operative word. One distinct disadvantage is that
it makes building your project require a working network connection
and depends on single points of failure in Clojars and Github. So,
network or site outages can delay your own work. Also, development
under this model would be difficult on a mobile platform (such as a
netbook) whose internet connection is flaky (e.g., up only when there
is a free WiFi hotspot within range) or expensive enough to turn off
most of the time (cellular data networks). (And both WiFi and cellular
are subject to flaky signal strength depending on local conditions,
too.)
> I really think incremental improvements to lein/cake/etc, over time, will
> yield better results than going back to square one.
This is probably true, even given the above issues that can arise with
a build process that presumes reliable 24/7 network connectivity.
--
Protege: What is this seething mass of parentheses?!
Master: Your father's Lisp REPL. This is the language of a true
hacker. Not as clumsy or random as C++; a language for a more
civilized age.
Sean Corfield
<mark.en...@gmail.com> wrote:
> I guess what I'm
> thinking of is that 95% of the time when I go to start something new,
> it ends up being for a short-lived task
Sounds like both Phil and I tackle that by having one or more scratch
projects that we just add new dependencies to as needed. Anything that
becomes longer-lived is "promoted" into its own project if/when it
makes sense.
> I'm not sure how well that would work
> with the way I have things organized, and with my source control, but
> it's something I can look into.
I guess our definitions of "one-off" might be different? If I'm just
doing a one-off experiment in the REPL, it's not likely to end up
under version control. If it's long-lived enough to go under version
control, it's probably worth a project :)
I definitely agree that compared to certain scripting languages,
there's more ceremony with Clojure (or any other JVM language) because
you can't just drop a source file in any old folder and run it. If you
don't mind a compilation step, ClojureScript -> JavaScript and then
running it with Node.js allows you to have scripts "anywhere" and
easily run them...
Alex Osborne
> Phil makes a reasonable point that it is possible to create a single
> "project" for one-off tasks. I'm not sure how well that would work
> with the way I have things organized, and with my source control, but
> it's something I can look into.
I have a "scratch" project which I use for that sort of thing. I still
use M-x slime (not lein swank or that new jack-in stuff) when doing
non-project stuff. I don't know if M-x slime is still supported in
newer versions, I haven't updated swank-clojure in ages. I hope it is,
it's really handy.
M-x slime just picks up jars from ~/.swank-clojure so I just turn that
into symlink for my scratch project's lib directory.
Here are the steps for setting it up (once off):
1. Create a scratch project.
$ cd ~/src && lein new scratch
2. Edit ~/src/scratch/project.clj with whatever libs you want. The
essentials:
:dependencies [[org.clojure/clojure "1.2.1"]
[org.clojure/clojure-contrib "1.2.0"]
[swank-clojure "1.2.1"]]
Newer swank-clojure jars don't work with the old elisp (1.1.0) I'm
running. Use whatever works for you.
3. Fetch 'em:
$ cd ~/src/scratch && lein deps
4. For M-x slime in Emacs:
$ ln -s ~/src/scratch/lib ~/.swank-clojure
5. Make up a ~/bin/clj launch script for CLI use and chmod a+x it:
#!/bin/sh
exec rlwrap java -cp "$HOME/src/scratch/lib/*" clojure.main "$@"
Now use M-x slime and "clj" as you would expect. If you want to install
some more jars, update the scratch project.clj, rerun lein deps and
restart emacs.
What's much nicer about this than the Python/Ruby model is that whatever
libs you put in your scratch area are not automatically on the load
path for all your proper projects. That keeps away the load path
pollution and version conflict issues that led to the creation of ugly
workarounds like RVM.
----------------------------------------------------------------------
A fun little project for someone might be to wrap this workflow into a
tiny lein plugin with a couple of convenience commands for setting it up
and updating project.clj. Perhaps something like:
leni scratch setup # init project, clj script, swank symlink
lein scratch install foo 1.5.0
lein scratch uninstall foo
lein scratch swank
lein scratch repl
----------------------------------------------------------------------
Tip: If you create a symlink from a source *directory* (not a jar) into
~/src/scratch/lib and just name it ".jar" swank and lein will both stick
that directory on the classpath. They don't actually check that it's a
jar file.
So if you've got lots of mini-projects scattered around that you want
on your scratch classpath you can just do something like:
$ ln -s ~/src/coolproject/src/ ~/src/scratch/lib/coolproject-src.jar
$ ln -s ~/src/junk-clj-files ~/src/scratch/lib/junk.jar
Brent Millare
script in the dependency context of a particular project. So lets say
you have a scratch project with many common dependencies. Then I run
anywhere, dj run foo.clj scratch-project. I'd imagine there is
something like this for cake or leiningen, or at least it would be
easy to write a plugin for.
On Jul 31, 2:20 am, Sean Corfield <seancorfi...@gmail.com> wrote:
> On Sat, Jul 30, 2011 at 5:27 PM, Mark Engelberg
>
yair
> "Almost" being the operative word. One distinct disadvantage is that
This is not correct. Once the jar has been downloaded after being
included in the dependencies, it stays in your local repository and no
network connection is required.
Cheers
Ken Wesson
If you add a new dependency, the network connection is needed.
And someone posted something recently about it balking if it's unable
to check for updated versions of some things.
Sean Corfield
> If you add a new dependency, the network connection is needed.
If you add a new dependency and you don't already have the JAR
downloaded, you need a network connection one way or another to go get
that JAR, yes? It doesn't matter what tool you use, including manual
downloading...
--
Sean A Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
World Singles, LLC. -- http://worldsingles.com/
Railo Technologies, Inc. -- http://www.getrailo.com/
Ken Wesson
> On Sun, Jul 31, 2011 at 7:49 PM, Ken Wesson <kwes...@gmail.com> wrote:
>> If you add a new dependency, the network connection is needed.
>
> If you add a new dependency and you don't already have the JAR
> downloaded, you need a network connection one way or another to go get
> that JAR, yes?
What if you have the JAR on a disk somewhere, for other reasons, but
until now it wasn't a dependency of that particular project?
pmbauer
until now it wasn't a dependency of that particular project?"
Ken Wesson
> "What if you have the JAR on a disk somewhere, for other reasons, but
> until now it wasn't a dependency of that particular project?"
> Your assertion that dependency management systems are in any way
> disadvantaged to manual dependency management in terms of SPOF or requiring
> a network connection is merely incorrect.
I might be able to disprove your scurrilous charge if I knew what the
heck you meant by "SPOF".
> See mvn install:install-file
>
> http://maven.apache.org/guides/mini/guide-3rd-party-jars-local.html
Not relevant. We were discussing use of lein deps.
Shantanu Kumar
On Aug 1, 9:59 am, Ken Wesson <kwess...@gmail.com> wrote:
> > "What if you have the JAR on a disk somewhere, for other reasons, but
> > until now it wasn't a dependency of that particular project?"
> > Your assertion that dependency management systems are in any way
> > disadvantaged to manual dependency management in terms of SPOF or requiring
> > a network connection is merely incorrect.
>
> I might be able to disprove your scurrilous charge if I knew what the
> heck you meant by "SPOF".
>
> > See mvn install:install-file
>
> >http://maven.apache.org/guides/mini/guide-3rd-party-jars-local.html
>
> Not relevant. We were discussing use of lein deps.
have the JAR on a disk somewhere..." - you can install the JAR to your
local repo that "lein deps" can fetch from.
Regards,
Shantanu
pmbauer
I might be able to disprove your scurrilous charge if ...
> See mvn install:install-file
>
> http://maven.apache.org/guides/mini/guide-3rd-party-jars-local.htmlNot relevant. We were discussing use of lein deps.
Ken Wesson
>> I might be able to disprove your scurrilous charge if ...
>
> I doubt that since your earlier assertion was factually incorrect.
If you have a personal problem with me, sort it out in private email
or keep it to yourself rather than badmouthing me on the list, please.
>> > See mvn install:install-file
>> >
>> > http://maven.apache.org/guides/mini/guide-3rd-party-jars-local.html
>>
>> Not relevant. We were discussing use of lein deps.
>
> Wrong again.
Not at all. We were, indeed, discussing lein deps.