clojure-contrib master now in submodules

91 views
Skip to first unread message

Stuart Sierra

unread,
Aug 20, 2010, 10:22:12 AM8/20/10
to Clojure
Hello, all,

As planned for some time, clojure-contrib has now been split into many
submodules on the "master" branch.


*** For users of clojure-contrib 1.2.0: nothing changes.


*** For users of clojure-contrib snapshots:

New builds of the master branch on github will be available as 1.3.0-
SNAPSHOT versions. Each major contrib library has its own module with
the groupId "org.clojure.contrib" and an artifactId which is the name
of the library.

For example, to use the clojure.contrib.macro-utils namespace in your
projects, add a dependency on group "org.clojure.contrib", artifact
"macro-utils", version "1.3.0-SNAPSHOT".

In Leiningen syntax, this looks like:

:dependencies [ ... [org.clojure.contrib/macro-utils "1.3.0-
SNAPSHOT"] ...]

In Maven syntax, this looks like:

<dependencies>
...
<dependency>
<groupId>org.clojure.contrib</groupId>
<artifactId>macro-utils</artifactId>
<version>1.3.0-SNAPSHOT</version>
</dependency>
...
</dependencies>

If you want to use ALL contrib libraries, add a dependency on group
"org.clojure.contrib", artifact "complete", version "1.3.0-SNAPSHOT".
This meta-library depends on all other contrib libraries.


*** For clojure-contrib developers:

Each library has its own directory under the "modules" directory at
the top level of clojure-contrib. Each module directory contains a
pom.xml file specifying the name, version number, and dependencies of
that library.

Every module pom.xml declares a "parent" located in the modules/parent
directory. The parent pom.xml file defines configuration settings
common to all clojure-contrib libraries. Currently the parent pom.xml
declares a dependency on Clojure 1.2.0 and sets up clojure-maven-
plugin to compile and test Clojure sources.

Individual libraries may override the parent configuration in their
own pom.xml files.

Building all of clojure-contrib (by running "mvn install" at the top
level) can take over 10 minutes. Fortunately, you do not need to
build all the modules most of the time. To build just one library, cd
to its directory under "modules" and run "mvn install" (or "mvn test"
to test). You will need to have already installed, at a minimum, the
parent module and any modules your library depends on.


*** For everyone:

There will doubtless be some breakage and difficulties during this
transition period. Please bear with us. Post your questions to the
list, and we will try to answer them as soon as possible.

Thanks,
Stuart Sierra

Tom Faulhaber

unread,
Aug 22, 2010, 2:11:56 AM8/22/10
to Clojure
A couple of questions:

1) Does use of clojure-contrib now require maven or leinigen as a
prerequisite or is there a place to go grab the jar files?
2) From my read of this, there is no longer a clojure-contrib.jar,
just a meta dependency that causes maven to grab all the modules. Is
that correct?

Tom

B Smith-Mannschott

unread,
Aug 22, 2010, 3:52:18 AM8/22/10
to clo...@googlegroups.com
Maven uses http, so one can download the various modules of clojure
contrib via a web browser. Snapshot builds are here:

http://build.clojure.org/snapshots/org/clojure/contrib/

i.e.

http://build.clojure.org/snapshots/org/clojure/contrib/MODULE/VERSION

The old monolithic clojure-contrib.jar is now
org.clojure.contrib/complete.jar you can find snapshot builds of it
here:

http://build.clojure.org/snapshots/org/clojure/contrib/complete/1.3.0-SNAPSHOT/

hth,
Ben

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en

B Smith-Mannschott

unread,
Aug 22, 2010, 4:00:27 AM8/22/10
to clo...@googlegroups.com
On Sun, Aug 22, 2010 at 09:52, B Smith-Mannschott <bsmit...@gmail.com> wrote:
> Maven uses http, so one can download the various modules of clojure
> contrib via a web browser. Snapshot builds are here:
>
> http://build.clojure.org/snapshots/org/clojure/contrib/
>
> i.e.
>
> http://build.clojure.org/snapshots/org/clojure/contrib/MODULE/VERSION
>
> The old monolithic clojure-contrib.jar is now
> org.clojure.contrib/complete.jar you can find snapshot builds of it
> here:
>
> http://build.clojure.org/snapshots/org/clojure/contrib/complete/1.3.0-SNAPSHOT/

Important detail: the one you want is the jar-with-dependencies variant.

Note: this jar contains not only all modules of clojure-contrib, but
also a complete copy of clojure 1.2.0.

I don't know if that's the intention. I could write a patch that
generates a 'complete' variant without clojure 1.2.0, if anyone's
interested.

Tim Daly

unread,
Aug 22, 2010, 4:43:51 AM8/22/10
to clo...@googlegroups.com
The fact that Maven uses http bit me today.
I was working offline in a coffee shop,
Maven tried to download something dynamically
and failed. End of my development work.

A git-based version of the system would be much
more useful (I know, I'm living in a backwater
country without proper internet but...).

Disk space is cheap. I have a 2T external drive.
Bandwith is not cheap and my time, at least to me,
is very expensive. I'd like to just do a git clone
of Clojure and be able to work anywhere.

Mavens use of http takes me all the way back to a
CVS/SVN situation where there are certain operations
I cannot do unless I'm connected. It's the late 90s
and this shouldn't be a blocking issue anymore.

Can I git-clone Maven so it will reach for a local repo?
Can I git-clone Clojure with a standalone build system?

Tim Daly

B Smith-Mannschott

unread,
Aug 22, 2010, 9:08:14 AM8/22/10
to clo...@googlegroups.com
On Sun, Aug 22, 2010 at 10:43, Tim Daly <da...@axiom-developer.org> wrote:
> The fact that Maven uses http bit me today.
> I was working offline in a coffee shop,
> Maven tried to download something dynamically
> and failed. End of my development work.

This smells a little like the problem we had when the amateurs at my
$JOB installed a company-wide proxy, which cheerfully responded to
attempts to GET inaccessible URLs, not with a 404, but with a
successful status code an an HTML page explaining that the resource
was not available. (Obviously this was configured by morons laboring
under the delusion that the only clients of the HTTP protocol are
humans sitting in front of IE6.) In any event, maven responded by
storing the bytes it was so given as a *.jar file in its repository
(though of course, it was no jar file.) Failed builds and much
hilarity ensued.

I've found two techniques useful in mitigating maven's hunger for an
omnipresent Internet connection.

(1) When I'm *really* not on the net, I tell maven that by calling it
in offline mode. This will only help if the dependencies required by
the build are already cached in the local ~/.m2/repository.

mvn -o

(2) I run a maven repository manager. Think of it as a caching proxy
for maven dependencies. I use the open source edition of Sonatype's
Nexus because it's dead-easy to get running.

At $JOB we have such a Nexus instance. I also have one at home running
on my server (a greying 1 GHz titanium PowerBook). I keep
~/.m2/settings.xml under version control with git, so it's easy enough
to 'cd ~/.m2 ; git checkout home' when I get home.

At home and at work, settings.xml is configured to route *all* maven
downloads through the appropriate nexus instance. I also keep a branch
around for when I'm truly out in the world without access to either of
the repository managers: 'git checkout nonexus'.

Having something like nexus around is nice because I do find it
occasionally useful to blow away ~/.m2/repository. It's nice not to
have to download everything from maven central all over again.

Though I do not currently do so, there's no reason why you couldn't
run a nexus on localhost. Bringing two laptops to the Cafe would seem
a tad excessive, after all. ;-)

I've set up a few Nexus instances and been fighting with Maven for
nearly five years now. I'll gladly share what experience I to help you
get things set up.

// Ben

Aaron Cohen

unread,
Aug 22, 2010, 9:05:53 PM8/22/10
to clo...@googlegroups.com
Maven isn't really analogous to git for the most part. Having a
complete history of a source code repository isn't a big deal thanks
to delta compression; keeping a complete history of every artifact
ever built would just be colossal.

One helpful tip though, is: mvn dependency:go-offline

That will download everything you need to build the current project,
including all dependencies.

lprefo...@softaddicts.ca

unread,
Aug 23, 2010, 12:28:07 AM8/23/10
to clo...@googlegroups.com
We use archiva to proxy accesses to external repos (central, clojar, ...)
Very simple to deploy and to use and it does the job.
We looked at nexus but found it more complex to use for our limited
needs.

I use an instance on my laptop to pull what I need from our central location
and keep it in cache.

Luc P.

Tim Daly <da...@axiom-developer.org> wrote ..

ataggart

unread,
Aug 23, 2010, 1:04:46 AM8/23/10
to Clojure
Yes, you can easily work offline. Simply recursively wget the entire
maven repo from http://repo1.maven.org/maven2/

It may take a while.

Before going into full-on DVCS evangelist mode, you should probably
step back and realize that maven is acting as a dependency management
system, not a source code repository. And yes, maven does cache
dependencies once they're pulled.

P.S. Please don't change the subject line; google groups is "smart"
enough to recognize that this is the same thread and just renames the
title displayed in the discussions list, thus many (myself included)
might have missed Stuart's announcement.



On Aug 22, 1:43 am, Tim Daly <d...@axiom-developer.org> wrote:
> The fact that Maven uses http bit me today.
> I was working offline in a coffee shop,
> Maven tried to download something dynamically
> and failed. End of my development work.
>
> A git-based version of the system would be much
> more useful (I know, I'm living in a backwater
> country without proper internet but...).
>
> Disk space is cheap. I have a 2T external drive.
> Bandwith is not cheap and my time, at least to me,
> is very expensive. I'd like to just do a git clone
> of Clojure and be able to work anywhere.
>
> Mavens use of http takes me all the way back to a
> CVS/SVN situation where there are certain operations
> I cannot do unless I'm connected. It's the late 90s
> and this shouldn't be a blocking issue anymore.
>
> Can I git-clone Maven so it will reach for a local repo?
> Can I git-clone Clojure with a standalone build system?
>
> Tim Daly
>
>
>
> B Smith-Mannschott wrote:
> > On Sun, Aug 22, 2010 at 09:52, B Smith-Mannschott <bsmith.o...@gmail.com> wrote:
>
> >> Maven uses http, so one can download the various modules of clojure
> >> contrib via a web browser. Snapshot builds are here:
>
> >>http://build.clojure.org/snapshots/org/clojure/contrib/
>
> >> i.e.
>
> >>http://build.clojure.org/snapshots/org/clojure/contrib/MODULE/VERSION
>
> >> The old monolithic clojure-contrib.jar is now
> >> org.clojure.contrib/complete.jar you can find snapshot builds of it
> >> here:
>
> >>http://build.clojure.org/snapshots/org/clojure/contrib/complete/1.3.0...
>
> > Important detail: the one you want is the jar-with-dependencies variant.
>
> > Note: this jar contains not only all modules of clojure-contrib, but
> > also a complete copy of clojure 1.2.0.
>
> > I don't know if that's the intention. I could write a patch that
> > generates a 'complete' variant without clojure 1.2.0, if anyone's
> > interested.
>
> >> hth,
> >> Ben
>

B Smith-Mannschott

unread,
Aug 23, 2010, 2:16:23 AM8/23/10
to clo...@googlegroups.com


On Mon, Aug 23, 2010 at 07:04, ataggart <alex.t...@gmail.com> wrote:
> Yes, you can easily work offline.  Simply recursively wget the entire
> maven repo from http://repo1.maven.org/maven2/
>
> It may take a while.

Whatever you do, please DO NOT DO THAT!

e.g.

http://maven.40175.n5.nabble.com/blacklisted-by-maven-central-repo-and-mirrors-td1045810.html

Jason van Zyl-2 Jul 10, 2010; 01:43pm:
# No typical usage pattern of Maven, even by a large number of developers, generally
# doesn't get you blocked. **We have heuristics, at least on Maven Central, that detects
# scrapers and that will get you instantly blacklisted.
** Sometimes it's a developers within
# an organization trying to grab the entire repository without anyone else in the
# organization knowing.
#
# You should definitely use a repository, like Nexus, and privately send me your IP
# and I can tell you the behavior that got you blocked from central, if indeed you have
# been blocked.

(emphasis mine)

Konrad Hinsen

unread,
Aug 23, 2010, 2:51:25 PM8/23/10
to clo...@googlegroups.com
On 20 Aug 2010, at 16:22, Stuart Sierra wrote:

> *** For clojure-contrib developers:
>
> Each library has its own directory under the "modules" directory at
> the top level of clojure-contrib. Each module directory contains a
> pom.xml file specifying the name, version number, and dependencies of
> that library.

I used to run the whole contrib library from source code, by putting
the source code directory on my classpath. It seems that now I need a
gigantic class path to do that. Is there any way around that?

Or is there any simple way to make a jar containing only the source
code files?

My goal is to be able to run either Clojure 1.2 or the current master
branch, with exactly the same code for clojure-contrib, without
constantly compiling and installing stuff.

Konrad.

Stuart Sierra

unread,
Aug 23, 2010, 3:22:02 PM8/23/10
to Clojure


On Aug 23, 2:51 pm, Konrad Hinsen <konrad.hin...@fastmail.net> wrote:
> Or is there any simple way to make a jar containing only the source code files?

Ys, the maven assembly plugin can do this. Ill work on it when I get
back later this week.
-S

Brian Carper

unread,
Sep 9, 2010, 3:47:16 PM9/9/10
to Clojure
On Aug 20, 7:22 am, Stuart Sierra <the.stuart.sie...@gmail.com> wrote:
> If you want to use ALL contrib libraries, add a dependency on group
> "org.clojure.contrib", artifact "complete", version "1.3.0-SNAPSHOT".
> This meta-library depends on all other contrib libraries.

This doesn't work because as was pointed out on IRC today, there's
currently a "bin" classifier. You have to use:

[org.clojure.contrib/complete "1.3.0-SNAPSHOT" :classifier "bin"]

--Brian

Stuart Sierra

unread,
Sep 9, 2010, 6:13:10 PM9/9/10
to Clojure
Can you clarify? Maven-aware build tools (e.g. Leiningen) should not
be trying to downlaod the "clojure-contrib:complete" JAR file.
Instead, referencing the "complete" project as a dependency should
transitively give you all its dependencies. Does that not work?

-S

Brian Carper

unread,
Sep 9, 2010, 6:43:51 PM9/9/10
to Clojure
On Sep 9, 3:13 pm, Stuart Sierra <the.stuart.sie...@gmail.com> wrote:
> Can you clarify?  Maven-aware build tools (e.g. Leiningen) should not
> be trying to downlaod the "clojure-contrib:complete" JAR file.
> Instead, referencing the "complete" project as a dependency should
> transitively give you all its dependencies.  Does that not work?
>
> -S

Nope. When I use [org.clojure.contrib/complete "1.3.0-SNAPSHOT"]
without :classifier "bin", Maven dies because it can't find the
"complete" JAR file. Like this: http://gist.github.com/572306

When I specify [... :classifier "bin"], it downloads all 50+ contrib
JAR files, as well as "complete-1.3.0-SNAPSHOT-bin.jar".

--Brian

Aaron Cohen

unread,
Sep 9, 2010, 7:00:29 PM9/9/10
to clo...@googlegroups.com

I think the problem is that the "complete" pom needs to have
<packaging>pom</packaging> to indicate there's not an important jar
file to download.

-- Aaron

Stuart Sierra

unread,
Sep 9, 2010, 7:03:08 PM9/9/10
to Clojure
On Sep 9, 7:00 pm, Aaron Cohen <aa...@assonance.org> wrote:
> I think the problem is that the "complete" pom needs to have
> <packaging>pom</packaging> to indicate there's not an important jar
> file to download.

It does. That's why I'm confused.
http://github.com/clojure/clojure-contrib/blob/master/modules/complete/pom.xml

Admittedly, I haven't tested this yet...

-S

Stuart Sierra

unread,
Sep 9, 2010, 7:04:40 PM9/9/10
to Clojure
Weird. I'll have to look into this some more. Thanks for the report.
-S

Sean Corfield

unread,
Sep 22, 2010, 2:02:44 AM9/22/10
to clo...@googlegroups.com
On Fri, Aug 20, 2010 at 7:22 AM, Stuart Sierra
<the.stua...@gmail.com> wrote:
> For example, to use the clojure.contrib.macro-utils namespace in your
> projects, add a dependency on group "org.clojure.contrib", artifact
> "macro-utils", version "1.3.0-SNAPSHOT".

Having built contrib against clojure 1.2.0 (see my note in another
thread), I can get individual module dependencies to work in lein like
this:

>    :dependencies [ ... [org.clojure.contrib/macro-utils "1.3.0-
> SNAPSHOT"] ...]

However, I can't get the all / complete dependency to work:

> If you want to use ALL contrib libraries, add a dependency on group
> "org.clojure.contrib", artifact "complete", version "1.3.0-SNAPSHOT".
> This meta-library depends on all other contrib libraries.

This doesn't work:

:dependencies [[org.clojure/clojure "1.3.0-master-SNAPSHOT"]
[org.clojure.contrib/complete "1.3.0-SNAPSHOT"]]

I've tried a few other dependencies based on what seems to be in my
local maven repo (caveat: I don't know maven), and couldn't find
anything that works.

Everything seems to be 1.3.1-SNAPSHOT but
[org.clojure.contrib/complete "1.3.1-SNAPSHOT"] doesn't work either.

Suggestions?
--
Sean A Corfield -- (904) 302-SEAN
Railo Technologies, Inc. -- http://getrailo.com/
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood

Justin Kramer

unread,
Sep 22, 2010, 12:35:58 PM9/22/10
to Clojure
Here's the magic incantation I've been using:

[org.clojure.contrib/complete "1.3.1-SNAPSHOT" :classifier "bin"]

I don't know how official or future-proof that is.

Justin

On Sep 22, 2:02 am, Sean Corfield <seancorfi...@gmail.com> wrote:
> On Fri, Aug 20, 2010 at 7:22 AM, Stuart Sierra
>
> <the.stuart.sie...@gmail.com> wrote:
> > For example, to use the clojure.contrib.macro-utils namespace in your
> > projects, add a dependency on group "org.clojure.contrib", artifact
> > "macro-utils", version "1.3.0-SNAPSHOT".
>
> Having built contrib against clojure 1.2.0 (see my note in another
> thread), I can get individual module dependencies to work in lein like
> this:
>
> >    :dependencies [ ... [org.clojure.contrib/macro-utils "1.3.0-
> > SNAPSHOT"] ...]
>
> However, I can't get the all / complete dependency to work:
>
> > If you want to use ALL contrib libraries, add a dependency on group
> > "org.clojure.contrib", artifact "complete", version "1.3.0-SNAPSHOT".
> > This meta-library depends on all other contrib libraries.
>
> This doesn't work:
>
>   :dependencies [[org.clojure/clojure "1.3.0-master-SNAPSHOT"]
>                         [org.clojure.contrib/complete "1.3.0-SNAPSHOT"]]
>
> I've tried a few other dependencies based on what seems to be in my
> local maven repo (caveat: I don't know maven), and couldn't find
> anything that works.
>
> Everything seems to be 1.3.1-SNAPSHOT but
> [org.clojure.contrib/complete "1.3.1-SNAPSHOT"] doesn't work either.
>
> Suggestions?
> --
> Sean A Corfield -- (904) 302-SEAN
> Railo Technologies, Inc. --http://getrailo.com/
> An Architect's View --http://corfield.org/

Sean Corfield

unread,
Sep 22, 2010, 3:28:06 PM9/22/10
to clo...@googlegroups.com
Awesome! That worked - thanx Justin!

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en

--

Sean A Corfield -- (904) 302-SEAN

Railo Technologies, Inc. -- http://getrailo.com/
An Architect's View -- http://corfield.org/

Stuart Sierra

unread,
Sep 22, 2010, 5:03:39 PM9/22/10
to Clojure
On Sep 22, 12:35 pm, Justin Kramer <jkkra...@gmail.com> wrote:
> Here's the magic incantation I've been using:
>
> [org.clojure.contrib/complete "1.3.1-SNAPSHOT" :classifier "bin"]
>
> I don't know how official or future-proof that is.

It's wrong, technically, but appears to work for now.

I think this may be a problem with Leiningen not understanding "pom"
packages. If somebody can confirm that, we can try to fix it.

-S

Brian Carper

unread,
Sep 23, 2010, 4:12:48 PM9/23/10
to Clojure
No, plain Maven has the same problem. If I add <classifier>bin</
classifier> to this pom it works, without it fails: http://gist.github.com/594252

--Brian
Reply all
Reply to author
Forward
0 new messages