Migration to 1.3 for mortals
Jan Rychter
to 1.3?
I thought I'd be able to at least estimate the effort involved for our
application in about an hour today, but failed. In spite of searching,
looking through various repos, reading the wiki and in general
googling around I was unable to find a migration path for the
clojure.contrib code we currently use. As a first step I thought I'd
fix project.clj so that it correctly points to newer clojure.contrib
library or modules.
A quick grep through our code shows we depend on:
clojure.contrib.combinatorics
clojure.contrib.command-line
clojure.contrib.core
clojure.contrib.def
clojure.contrib.except
clojure.contrib.io
clojure.contrib.java-utils
clojure.contrib.json
clojure.contrib.logging
clojure.contrib.map-utils
clojure.contrib.mmap
clojure.contrib.seq
clojure.contrib.seq-utils
clojure.contrib.str-utils
clojure.contrib.str-utils2
clojure.contrib.string
clojure.contrib.trace
clojure.contrib.zip-filter
clojure.contrib.zip-filter.xml
How do I go from this list to a list of new project.clj lines? After
an hour of poking and probing I only got to:
[org.clojure/data.json "0.1.1"]
[org.clojure/tools.cli "0.1.0"]
[org.clojure/tools.logging "0.2.0"]
I got some version numbers from github repo READMEs, some from
manually listing directories under https://oss.sonatype.org/content/groups/public/org/clojure/
There must surely be a better way — any hints would be very much
appreciated. Also, a suggestion for documentation maintainers: please
think about people who have code based on 1.2 and haven't followed
Clojure closely for the past 2 years or so — we lack a lot of context
information that might be obvious to you. As an example, I managed to
list lots of clojure.contrib modules before realizing that this
particular modularization is obsolete and has already been abandoned
in one of the alphas.
thanks,
--J.
Armando Blancas
http://dev.clojure.org/display/doc/Clojure+Contrib
Here's the main page for the new repos:
https://github.com/clojure
Sergey Didenko
Sean Corfield
> How do we mere mortals (that develop and maintain large apps) migrate
> to 1.3?
It's a good question - and it's being discussed right now on the
clojure-dev list because the biggest obstacle to folks moving to
Clojure 1.3 is what to do about contrib...
> A quick grep through our code shows we depend on:
A lot of those libraries have no active maintainer - although I
suspect many of the things you're relying on in several of them have
already migrated into Clojure's core namespaces?
> clojure.contrib.command-line
clojure.tools.cli - although it's actually a completely different
library now (based on clargon).
> clojure.contrib.core
Mostly moved to clojure.core.incubator.
> clojure.contrib.def
Mostly moved to clojure.core.incubator.
> clojure.contrib.except
Still under discussion:
http://dev.clojure.org/display/design/Contrib+Library+Names
Some of this moved to clojure.java.io I believe?
> clojure.contrib.java-utils
??
> clojure.contrib.json
clojure.data.json
> clojure.contrib.logging
clojure.tools.logging
> clojure.contrib.map-utils
??
> clojure.contrib.mmap
??
> clojure.contrib.seq
Looks like most of this was already promoted to clojure.core.
> clojure.contrib.seq-utils
(ClojureDocs shows the same contents for this as c.c.seq?)
> clojure.contrib.str-utils
> clojure.contrib.str-utils2
> clojure.contrib.string
Didn't most of those three get promoted to clojure.string a while back?
> clojure.contrib.trace
That just came up in discussion on clojure-dev but no volunteer has
stepped forward to maintain it yet I believe.
> clojure.contrib.zip-filter
Looks like it moved to clojure.data.zip?
> clojure.contrib.zip-filter.xml
Looks like it moved to clojure.data.zip.xml?
> I got some version numbers from github repo READMEs, some from
> manually listing directories under https://oss.sonatype.org/content/groups/public/org/clojure/
search.maven.org for org.clojure is probably the definitive word on
released versions:
http://search.maven.org/#search%7Cga%7C1%7Corg.clojure
> There must surely be a better way — any hints would be very much
> appreciated. Also, a suggestion for documentation maintainers: please
> think about people who have code based on 1.2 and haven't followed
> Clojure closely for the past 2 years or so — we lack a lot of context
> information that might be obvious to you. As an example, I managed to
> list lots of clojure.contrib modules before realizing that this
> particular modularization is obsolete and has already been abandoned
> in one of the alphas.
Definitely a big problem (lack of clear communication) and it seems
that Clojure/core has this on their radar now so hopefully things will
improve over the next few months.
--
Sean A Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
World Singles, LLC. -- http://worldsingles.com/
Railo Technologies, Inc. -- http://www.getrailo.com/
"Perfection is the enemy of the good."
-- Gustave Flaubert, French realist novelist (1821-1880)
Luc Prefontaine
I'll take care of c.c.trace. Just need to register on clojure-dev, my CA should be in by tomorrow.
Luc P.
--
Luc P.
================
The rabid Muppet
Jan Rychter
> There's this page:http://dev.clojure.org/display/doc/Clojure+Contrib
>
> Here's the main page for the new repos:https://github.com/clojure
a list of 1.2 clojure-contrib modules to a list of dependencies in the
new contrib. Plus the resources above are incomplete.
Take an example -- even in the relatively simple case of
clojure.contrib.combinatorics: note that it is NOT listed on
http://dev.clojure.org/display/doc/Clojure+Contrib, and if I follow
the github repos I'll end up in https://github.com/clojure/math.combinatorics
which tells me nothing about what I should add to project.clj (what is
the latest released version, for instance).
I have to say that from the point of view of application developers
the current state of contrib likely means I won't be migrating to 1.3
anytime soon. What's worse, I'll be left in a situation where there
are bugs in 1.2 contrib which only get fixed in the 1.3 version, which
I won't be able to use. I already have one such bug that I reported
(and provided a fix) for clojure.contrib.json. It never made it into
any contrib release, which means we have to use a locally-modified
version.
I think issues like that are fundamentally important if Clojure is to
be adopted for production work.
--J.
OGINO Masanori
> Problem is, none of those pages help with my problem
However,
> there are bugs in 1.2 contrib which only get fixed in the 1.3
> version, which I won't be able to use.
AFAIK new contrib is not only for 1.3 (for now).
In http://dev.clojure.org/display/doc/Clojure+Contrib :
> compatible with Clojure 1.2 and Clojure 1.3
So you can use new one when your version is 1.2 or later.
> I think issues like that are fundamentally important if Clojure is to
> be adopted for production work.
+1
--
Name: OGINO Masanori (荻野 雅紀)
E-mail: masanor...@gmail.com
Sean Corfield
> Take an example -- even in the relatively simple case of
> clojure.contrib.combinatorics: note that it is NOT listed on
> http://dev.clojure.org/display/doc/Clojure+Contrib, and if I follow
> the github repos I'll end up in https://github.com/clojure/math.combinatorics
> which tells me nothing about what I should add to project.clj (what is
> the latest released version, for instance).
That contrib has not been fully migrated yet. There is no build for it
(on build.clojure.org) and therefore it is not getting any Maven
releases yet.
> I have to say that from the point of view of application developers
> the current state of contrib likely means I won't be migrating to 1.3
> anytime soon. What's worse, I'll be left in a situation where there
> are bugs in 1.2 contrib which only get fixed in the 1.3 version, which
> I won't be able to use.
The new contrib modules are usable with Clojure 1.2 code by design
(and by mandate - the matrix tests are being run automatically against
Clojure 1.2, Clojure 1.2.1 and Clojure 1.3.0). See, for example, this
matrix for clojure.java.jdbc :
http://build.clojure.org/job/java.jdbc-test-matrix/
> I already have one such bug that I reported
> (and provided a fix) for clojure.contrib.json. It never made it into
> any contrib release, which means we have to use a locally-modified
> version.
If it hasn't been fixed in clojure.data.json, please create an issue
here : http://dev.clojure.org/jira/browse/DJSON
There should be a JIRA project for every active new contrib -
http://dev.clojure.org/jira
There should also be automated builds (both regular and matrix tests)
on build.clojure.org
> I think issues like that are fundamentally important if Clojure is to
> be adopted for production work.
I agree - which is why I'm pushing on clojure-dev for some serious
attention to be paid to the contrib migration issue.
Jan Rychter
> On Fri, Sep 9, 2011 at 1:09 AM, Jan Rychter <jrych...@gmail.com> wrote:
[...]
> > be adopted for production work.
>
> I agree - which is why I'm pushing on clojure-dev for some serious
> attention to be paid to the contrib migration issue.
reorganized. There is no clear migration path for applications that
use the monolithic 1.2 contrib. Not all of 1.2 contrib code made its
way into new modules yet. There is no migration guide for existing
applications.
Unless clojure 1.3 can be used with 1.2 monolithic contrib, I'd
strongly suggest that the release of clojure 1.3 should wait until the
contrib situation is resolved. I'd much rather see an announcement of
production-ready Clojure 1.3 than an announcement of very cool core
code that can't really be used in production systems.
Much as it sounds boring, it's the libraries that define the
applicability and usability of a programming language.
--J.
David Nolen
Luc Prefontaine
We have production code here and would like to see 1.3 out even if the contrib issue
resolve itself later. The first stable release of 1.3 needs to get used as widely
as possible.
If some issues emerge in 1.3 it would be better to know it as early as possible.
There are significant blocks of the old contrib available under the new modular
structure to create apps based on 1.3. 1.3 will not be left on the shelf waiting for the
outcome of this issue, it will be used. That's the ultimate test :)
Luc P.
--
Steve Purcell
> Waiting for contrib authors to sort out their libraries seems
> unrelated as to whether 1.3 should or should not be released. That's
> like delaying Python 3 because libraries don't support it yet. It
> still wouldn't be released.
Except that Python 3 comes bundled with many more libraries than
Clojure. I think the issue is that Clojure started off in the Python
mold as "Batteries Included" in the form of a coordinated Core +
Contrib, and has since transitioned to "Some Batteries Included",
jettisoning much of Contrib to fend for itself.
I respect the wisdom of those who've made the decision to decouple Core
and Contrib, so I'm not about to question that decision, but I guess I'm
not alone in having viewed Contrib + Core as "Clojure" up to this
point.
-Steve
Jan Rychter
launch and maintain, it is pretty much a given that you use contrib.
Lots of it, in fact. This might be different for people who hack on
Clojure itself, write libraries, or work on small (and fun) projects.
There is a disconnect that is very risky -- witness Common Lisp, where
for many years people would hack on nothing but language
implementations and library situation was abysmal. It's gotten much
better in recent years, but still isn't anywhere near languages like
Python or Ruby.
My worry is that the decoupling of "core Clojure" from contrib will
result in contrib authors not caring too much about sorting out the
current situation. Also, I think much of the fanfare of the 1.3
release will be shadowed by the contrib situation. And perhaps most
importantly, I worry that I won't be able to take advantage of Clojure
1.3 in any of my applications anytime soon.
I think contrib modularization should either be carried all the way in
time for 1.3, or stopped altogether (meaning I could continue to use
the same old monolithic contrib code with Clojure 1.3).
--J.
David Nolen
On Sep 10, 5:20 pm, Steve Purcell <st...@sanityinc.com> wrote:
> David Nolen <dnolen.li...@gmail.com> writes:
> > Waiting for contrib authors to sort out their libraries seems
> > unrelated as to whether 1.3 should or should not be released. That's
> > like delaying Python 3 because libraries don't support it yet. It
> > still wouldn't be released.
>
> Except that Python 3 comes bundled with many more libraries than
> Clojure. I think the issue is that Clojure started off in the Python
> mold as "Batteries Included" in the form of a coordinated Core +
> Contrib, and has since transitioned to "Some Batteries Included",
> jettisoning much of Contrib to fend for itself.
>
> I respect the wisdom of those who've made the decision to decouple Core
> and Contrib, so I'm not about to question that decision, but I guess I'm
> not alone in having viewed Contrib + Core as "Clojure" up to this
> point.
My worry is that the decoupling of "core Clojure" from contrib will
result in contrib authors not caring too much about sorting out the
current situation. Also, I think much of the fanfare of the 1.3
release will be shadowed by the contrib situation. And perhaps most
importantly, I worry that I won't be able to take advantage of Clojure
1.3 in any of my applications anytime soon.
Stuart Halloway
My worry is that the decoupling of "core Clojure" from contrib will
result in contrib authors not caring too much about sorting out the
current situation. Also, I think much of the fanfare of the 1.3
release will be shadowed by the contrib situation. And perhaps most
importantly, I worry that I won't be able to take advantage of Clojure
1.3 in any of my applications anytime soon.
http://clojure.com
Sean Corfield
> Thanks for the explanations. I'll summarize: clojure-contrib is being
> reorganized. There is no clear migration path for applications that
> use the monolithic 1.2 contrib. Not all of 1.2 contrib code made its
> way into new modules yet. There is no migration guide for existing
> applications.
I'm working on this page:
http://dev.clojure.org/display/design/Where+Did+Clojure.Contrib+Go
I need input from contrib library authors past and present.
One thing I discovered while ploughing through old contrib is that
there is duplication of functions across namespaces (sometimes two
namespaces are almost identical) and there's clearly a lot of unused
"cruft" in many old contrib modules! I see no point in dragging all
that baggage forward from release to release when it isn't being
maintained (or used). As each release of Clojure itself appears and
introduces backward compatibility issues, no matter how small, trying
to maintain the huge blog of legacy code that is the 1.2 monolithic
contrib makes absolutely no sense.
> I'd much rather see an announcement of
> production-ready Clojure 1.3 than an announcement of very cool core
> code that can't really be used in production systems.
World Singles is using Clojure 1.3 in production today - without
monolithic contrib. We're using several new contrib libraries and the
3rd party libraries we use have all been made 1.3 compatible
(sometimes that's taken me becoming a committer on those librares,
sometimes the author is only too happy to make the changes once
they've been pointed out).
Sean Corfield
> On Fri, Sep 9, 2011 at 1:09 AM, Jan Rychter <jryc...@gmail.com> wrote:
>> Take an example -- even in the relatively simple case of
>> clojure.contrib.combinatorics: note that it is NOT listed on
>> http://dev.clojure.org/display/doc/Clojure+Contrib, and if I follow
>> the github repos I'll end up in https://github.com/clojure/math.combinatorics
>> which tells me nothing about what I should add to project.clj (what is
>> the latest released version, for instance).
>
> That contrib has not been fully migrated yet. There is no build for it
> (on build.clojure.org) and therefore it is not getting any Maven
> releases yet.
FWIW, I've just worked with Mark Engelberg to Mavenize his three
contrib libraries (math.combinatorics, math.numeric-tower,
data.priority-map) and as soon as the Clojure/core folks get time to
do, there should be automated builds and the ability to release 0.0.1
versions to Maven Central.
If a contrib library you need has not been migrated to the new contrib
system, please contact the author and see if they're willing to
migrate and maintain the library. If they no longer care / support the
code, there's no point to it being in new contrib. The whole point of
new contrib is to ensure that Clojure's libraries are actually being
maintained and updated to work which each new version of Clojure.
But also look at what you're actually depending on and see whether
there's a better solution (an example: defnk in c.c.def has not been
migrated to clojure.core.incubator because Clojure supports keyword
arguments natively now thru map destructuring).
Sean Corfield
> If you use Clojure to build an application that you subsequently
> launch and maintain, it is pretty much a given that you use contrib.
> Lots of it, in fact.
I think that depends on when you started building stuff with Clojure.
I get the impression quite a bit of old contrib grew up to provide
functionality not in Clojure's core and I see quite a few pieces of
old contrib that are explicitly deprecated because functions moved to
Clojure 1.2 (c.c.string, for example, mostly moved to clojure.string
long before Clojure 1.3 got under way). I suspect that folks who
started with Clojure long ago and are relying heavily on old contrib,
are probably relying on those old deprecated namespaces and functions
and have simply avoided updating their code to use the newer,
supported namespaces. I think the break in contrib coming with 1.3
will be good for the Clojure ecosystem overall because it will force a
lot of folks to clean up their code :)
As noted in another one of my comments in this thread, World Singles
started with Clojure 1.2 but switched to Clojure 1.3 early on and so
we've essentially not been allowed to depend on that old, unsupported
contrib code which I view as a big positive for us (as well as getting
the performance boost of the updated numerics in Clojure 1.3).
One of the things World Singles needed early on was
clojure.contrib.sql so I pushed for that to be migrated and have ended
up being the maintainer now. If you can't find an author to migrate
and maintain an old contrib library, another option is for you to
volunteer to do that work and then you have the added benefit of being
able to enhance the library to better suit your needs (e.g., I added
the ability to return generated keys from insert operations in
clojure.java.jdbc).
> importantly, I worry that I won't be able to take advantage of Clojure
> 1.3 in any of my applications anytime soon.
A definite possibility - but changing your code to not rely on
outdated, unsupported libraries would have benefit to you in the long
run, yes?
> I think contrib modularization should either be carried all the way in
> time for 1.3, or stopped altogether (meaning I could continue to use
> the same old monolithic contrib code with Clojure 1.3).
Sorry, that ship done sailed a while back. I don't know how much work
it would be to make the modular contrib work on Clojure 1.3 (work
stopped on contrib around 1.3.0 Alpha 4) and given the number of
library authors who've stepped up to modernize and maintain new
contrib versions of old library modules, I can't imagine who'd put in
that work (to make old, unsupported code run on Clojure 1.3).
Luc Prefontaine
We were in production in Jan 2009 before Clojure 1.0 came out.
We face the issue of un-rooting ourselves from the "old" contrib stuff.
After analysis, 97% of the stuff we need from the "old" contrib is now available in 1.3
as separate libs.
The only thing we would like to keep is the trace module by Stuart Sierra and
we volunteer to move it to 1.3 and maintain it.
Nothing to panic about here. A new release always involves
some code rework anyway.
Luc P
--
Sean Corfield
<lprefo...@softaddicts.ca> wrote:
> We were in production in Jan 2009 before Clojure 1.0 came out.
> We face the issue of un-rooting ourselves from the "old" contrib stuff.
>
> After analysis, 97% of the stuff we need from the "old" contrib is now available in 1.3
> as separate libs.
>
> The only thing we would like to keep is the trace module by Stuart Sierra and
> we volunteer to move it to 1.3 and maintain it.
Excellent! I'm looking forward to trying that module once you have an
updated version of it!
Jan Rychter
> > If you use Clojure to build an application that you subsequently
> > launch and maintain, it is pretty much a given that you use contrib.
> > Lots of it, in fact.
>
> I think that depends on when you started building stuff with Clojure.
> I get the impression quite a bit of old contrib grew up to provide
> functionality not in Clojure's core and I see quite a few pieces of
> old contrib that are explicitly deprecated because functions moved to
> Clojure 1.2 (c.c.string, for example, mostly moved to clojure.string
> long before Clojure 1.3 got under way). I suspect that folks who
> started with Clojure long ago and are relying heavily on old contrib,
> are probably relying on those old deprecated namespaces and functions
> and have simply avoided updating their code to use the newer,
> supported namespaces. I think the break in contrib coming with 1.3
> will be good for the Clojure ecosystem overall because it will force a
> lot of folks to clean up their code :)
much rather see a quickly evolving Clojure ecosystem than one mired in
legacy code. I will gladly rework my code regularly so that it works
with newer releases. I'd just like to see the process somewhat
documented.
I think I've said all there was to be said in this discussion, so I'll
shut up now and wait for better documentation or a migration guide. In
the meantime, we'll stick to Clojure 1.2 together with our patched-up
monolithic contrib.
thanks,
--J.
Michael Wood
Since the new, separated contrib libraries are supposed to be
compatible with Clojure 1.2, you could perhaps also start migrating
one lib at a time at your leisure. This might even enable you to
contribute to a migration document.
Just a thought :)
--
Michael Wood <esio...@gmail.com>
Stuart Sierra
Since the new, separated contrib libraries are supposed to be
compatible with Clojure 1.2, you could perhaps also start migrating
one lib at a time at your leisure. This might even enable you to
contribute to a migration document.
Yes, this is what I am hoping people will do. Move to the new libraries first, then switch over to Clojure 1.3.
-Stuart Sierra
clojure.com
Jan Rychter
as the word "leisure" in the paragraph above doesn't quite apply to
our current situation :-)
thanks,
--J.