What is Contrib?
Phil Hagelberg
Now that Clojure 1.0 is out, I think it's a good time to take a look at
contrib. I noticed it didn't get an official 1.0 release along with
Clojure core. I wonder if this is because its role is just not very
well-defined. Several people have expressed this opinion here on the
mailing list and on IRC.
My understanding is that contrib began as a "staging ground" for
core. If libraries or functions proved themselves useful, then they
could be promoted to core.
I get the feeling that things have changed since then. For one, core
doesn't seem to grow that way any more. There was talk of moving some
functions from contrib to core way back in November, but that doesn't
seem to have gone anywhere. I understand there's good motivation to keep
core small, but perhaps it's time to rethink what contrib is for.
The feeling I get now is that contrib is a bucket-o-code. Since we as a
community don't have any widely-accepted mechanisms for dependency
resolution, there's a quite understandable motivation to put useful
libraries in contrib so that other clojure hackers can make use of them
with minimal hassle.
Right now if you want to handle nontrivial dependencies in your project,
you need to use Maven or Ivy. Many are reluctant to try these out since
they're perceived as heavy-weight, and the idea of writing executable
XML is rightfully viewed with suspicion by folks who're familiar with
Greenspun's Tenth Law; it's certainly tacky. But these tools have the
advantage that they work right now.
I think things could be improved by creating a tool that wraps the
functionality that Maven provides in a more pleasant s-expression-based
interface and focuses just on the parts of Maven that are relevant to
building a Clojure project, but that's a discussion for another thread.
I wonder if improving the dependency management story of Clojure will
help to clear up the confusion surrounding contrib. I certainly think
there's room for a "standard library" that ships with Clojure, and
precedent for this seems to be there with the existence of clojure.xml
and clojure.zip, but we need some discussion about why these libraries
are included and what others might also be worth including.
It also may help to think about moving things out of contrib in two
ways: in one case functions would be moved into the core.clj namespace
(this was proposed of duck-streams' spit function) and in the other
whole libraries would get moved into the main Clojure distribution. It's
important to be very cautious with the first as these increase the
complexity of Clojure for everyone since they get loaded on startup, but
I suspect there's room for more growth in the second case.
Would love to hear what folks think about this.
thanks,
Phil
CuppoJava
contrib is. I am sharing my (possibly misguided) view because it's
likely a lot of other people think similarly.
Contrib seems to be a collection for user-created libraries for
Clojure. They all seem to be in various stages of completion, with
varying quality and support. I get the feeling that at any time, a
contrib library can suddenly die and be left without further support.
For this reason, I have avoided depending on any contrib code whenever
I could.
I do know of some very good and complete libraries in contrib. I just
feel that it's not reflective of the average.
This opinion is based on very little experience and research. But it's
my current opinion nonetheless. It's very likely that this opinion is
completely wrong, in which case I feel that just a little thoughtful
"advertising" would go a long way towards helping out Contrib.
-Patrick
e
So, if slurp is in the core, spit should be in the core (but why not just use python's way more typical convention of "read() and "write()".
Another example I recently saw is that "(nth)" is in the core, but "(pos)" (something I think of as nth's inverse) for "position" is something you need to go to java for (.indexOf). To me, get nth out, or put pos in.
So stuff people put in contrib to specifically fill a gap in core, should potentially have a special ticket category and be on the fast track for discussions.
This stuff might not seem like a big deal to the seasoned clojurian ... indeed, weathering such might be the battle scars of the persistent (to mix several metaphors and puns). But it seems like a type of barrier that can go away for future newbies (and current ones like me).
... and the answer that I hear a lot: "write a macro or function" only helps me. And where would I put it for others? In contrib? Ah ha! maybe for now, but what next?
- Eli
Daniel Lyons
On May 12, 2009, at 4:04 PM, Phil Hagelberg wrote:
> Would love to hear what folks think about this.
I am new here and new to Java and the JVM as well. Contrib strikes me
as an improvement over the situation with Ruby where tons of code was
dumped into the Ruby distribution and much of it rotted. Using it as a
staging area seems wise and has my vote. However I am not opposed to
having a rather large library, provided it is concise and well-
maintained. Haskell and Python are both testaments to the power of a
large standard library.
Dependency tracking and library discovery and installation are
extremely important to the success of any new language. It would be
wise to see where the predecessors have gone awry. For example, Ruby
Gems seem to fall down in a highly distributed setting in which the
same library exists in many different forks and there are no official
releases. I am not in favor of programming that way but if people are
going to (and they are likely to) then hopefully one has systems that
can cope with it. There are also system administration headaches that
arise from having two separate databases managing software on the same
system.
I don't know if Maven solves these same problems or not; I don't know
anything about Maven. But Clojure seems to have been well-served
during its short life by leveraging Java's strengths and building
orthogonally against its weaknesses. Utilizing whatever Java library
distribution/dependency tracking systems are available certainly
prevents Clojure from becoming a walled garden, which seems good. But
if Maven and friends are found to be inadequate, I would recommend
building something new, with heavy consideration after reviewing the
faults of our peers and predecessors.
—
Daniel Lyons
http://www.storytotell.org -- Tell It!
Laurent PETIT
For example, placing carefully designed general small additions such as spit in namespace clojure.contrib.core could facilitate the debates around the most probable incoming additions.
Konrad Hinsen
> Now that Clojure 1.0 is out, I think it's a good time to take a
> look at
> contrib. I noticed it didn't get an official 1.0 release along with
> Clojure core. I wonder if this is because its role is just not very
> well-defined. Several people have expressed this opinion here on the
> mailing list and on IRC.
I'd say the real question is not "what is contrib?" but "what kind of
library system should Clojure have?"
I think that it is clear to most of us that contrib started as
something which it isn't any more. At the moment, it's a bucket of
code whose only common point is the licence, which allows it to be
distributed under exactly the same terms as Clojure itself, whatever
those may become.
As I have said before, I strongly believe that Clojure should have a
standard library. Contrib could then become the staging ground for
both code and the standard library. But the important question is
about the standard library, not about the fate of contrib.
Alternatively, we could also envisage something like the Haskell
platform:
http://hackage.haskell.org/platform/
This is a collection of separately written libraries distributed as a
single package with a single installation procedure. I think an
officially labelled and maintained standard library is important, but
there could well be a collection of independent (and differently
licenced) libraries on top of that.
Konrad.
John Newman
(where dependencies are automatically pulled from a repository?)
Maybe it's not applicable here, but I thought it was pretty neat.
John
Rich Hickey
Clojure. Contrib is also a *process* we are all going through.
Clojure Libraries
Clojure shipped with no libraries other than itself. That is an
important sentence to consider. Clojure is in many ways a very tiny
Lisp used to build a DSL (if you will) for a set of libraries and
abstractions for sequencing, persistent collections and concurrency.
Those libraries took me a couple of years to write. They focused on
the critical things I was missing in my professional C#/Java work.
In spite of the dearth of its own libraries for I/O, DB, web etc,
Clojure was immediately useful, probably as useful as a nascent
language could be. That was (and is) a critical factor in its
adoption. This was of course due to its Java interop. Clojure had at
one and the same time almost no libraries, and libraries for
everything. No one is stuck unable to work until someone writes, say,
a Clojure database library, because they can use JDBC directly. A
Clojure DB library is a (welcome) luxury, one we can take time
defining and refining.
Library design is hard. It takes time, iteration, and use. We decided
early on - don't write wrappers merely to hide Java, but when they add
value. Don't write libraries because they are theoretically needed,
but because they are actually needed. Contrib isn't a free-for-all.
The people in it wrote libraries I decided had general utility. And
the contrib libraries still represent that - numerics, streams, data
structures, I/O, repl utilities, testing etc. All very general, and
defined only in terms of Clojure's core, Java 1.5, or each other. They
are all things that might become a useful part of Clojure proper (we
need to stop saying 'core' since it has namespace implications). Any
clojure.contrib.xxx could eventually become clojure.xxx and come in
clojure.jar or clojure-extra.jar. As to why xml and zip are in
Clojure: because I wrote them, and: zip is non-trivial and without it
people would have wondered how to manage these immutable trees, and
XML was simply to show, early on, how to "get into Clojure data
structures ASAP", an important model for all libraries.
Community
I vetted libraries and contributors on entry to contrib, and have
trusted them subsequently. I will vet any library on move from contrib
to clojure. However, I don't want to dominate contrib. I want it to
become a community. But each contributor doesn't necessarily know the
others. It will take time for them to gel as a community and act in
concert, develop internal leadership and processes etc.
License
What will drive libraries into Clojure itself? A surefire way is when
Clojure needs the facilities of the library in its implementation.
That's why datalog was accepted into contrib (rather, requested).
Because it could form the underpinnings of predicate dispatch or a
gradual logic-based type system, i.e. there is an anticipated need.
Similarly, many other parts of contrib could become useful for Clojure
itself, a reasonable metric for their utility at large. Or there could
be consensus (I know, spit :) on their general utility in being
included in Clojure proper. But if Clojure is going to consume from
contrib, it is critical that it leverage only libraries that can be
absorbed into Clojure's license, with the same provenance. Independent
libraries are unlikely to go through the hassle of maintaining
provenance, and thus become locked into their license du jour.
Contributing and maintenance
One current hindrance to moving things from contrib to clojure is that
it changes the maintenance model from direct to patch submission. I
understand the limits and hardships of SVN for the distributed model,
(I heard you all when I said "what do you want for 1.0?" and the
number one answer was "git") and am investigating alternatives.
Versioning and dependency management
Obviously matching versions is important. But Clojure *just* released
its first official version, and thus first viable target for making
any library version-nnn compatible. I'd like to see a clojure-
contrib.jar download that works with the release. We need to discuss
how best to produce stable views of contrib as each of its parts move
independently. Any discussion of dependencies must follow creation of
stable, named views.
Moving forward
These are all good questions, but their existence doesn't indicate
anything wrong with contrib other than that it is a work in progress.
Please be patient. Please help out. Please understand that the contrib
authors are volunteers. This is, and should remain, a conservative
process. Prematurely standardizing libraries or dependency management
isn't going to be good in the long run. Let's focus on immediate
problems and small steps.
Rich