fallback namespaces

[Chas Emerick]

In the process of attempting to maintain 1.1 and 1.2 compatibility in
various libraries, I've found myself writing stuff like this a lot:
(try
(use '[clojure.contrib.reflect :only [call-method]])
(use '[clojure.contrib.io :only [as-file]])
(catch Exception e
(use '[clojure.contrib.java-utils
:only [file wall-hack-method]
:rename {wall-hack-method call-method, file as-file}])))
(from http://github.com/hugoduncan/clj-ssh/commit/93184ed9da4b21721bcf67a202ca36da93f9834d)

In larger projects, I'm resorting to putting such things into a
function that can be called, post-ns-declaration, from namespaces that
need access to libs that are under "compatibility management". ;-)

Of course, these tactics have been necessary in other compatibility
scenarios as well, but our recent shift towards basing everything we
do on 1.2 -- while much of the rest of the world is still very happy
with 1.1 -- has made the above much, much more common (and I can't
imagine such situations becomes less likely over time). Perhaps some
kind of fallback namespace declaration is needed? Something that
looks like this, perhaps:

(ns foo.bar
(or (:require ...)
(:require ...)))

...which implies short-circuiting behaviour as well as insensitivity
to FileNotFoundExceptions when attempting to load code.

Thoughts?

- Chas

--
You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
To post to this group, send email to cloju...@googlegroups.com.
To unsubscribe from this group, send email to clojure-dev...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/clojure-dev?hl=en.

[Matt Revelle]

On Apr 16, 2010, at 6:47 AM, Chas Emerick wrote:

> In the process of attempting to maintain 1.1 and 1.2 compatibility in various libraries, I've found myself writing stuff like this a lot:
> (try
> (use '[clojure.contrib.reflect :only [call-method]])
> (use '[clojure.contrib.io :only [as-file]])
> (catch Exception e
> (use '[clojure.contrib.java-utils
> :only [file wall-hack-method]
> :rename {wall-hack-method call-method, file as-file}])))
> (from http://github.com/hugoduncan/clj-ssh/commit/93184ed9da4b21721bcf67a202ca36da93f9834d)
>
> In larger projects, I'm resorting to putting such things into a function that can be called, post-ns-declaration, from namespaces that need access to libs that are under "compatibility management". ;-)
>
> Of course, these tactics have been necessary in other compatibility scenarios as well, but our recent shift towards basing everything we do on 1.2 -- while much of the rest of the world is still very happy with 1.1 -- has made the above much, much more common (and I can't imagine such situations becomes less likely over time). Perhaps some kind of fallback namespace declaration is needed? Something that looks like this, perhaps:
>
> (ns foo.bar
> (or (:require ...)
> (:require ...)))
>
> ...which implies short-circuiting behaviour as well as insensitivity to FileNotFoundExceptions when attempting to load code.
>
> Thoughts?

I wouldn't think contrib namespaces (at least the most commonly-used: io, seq, string) will be changing names again. Why do you think this problem will become more common?

The solution seems good, though it could be (mis?)used for reasons other than dealing with deprecated namespaces. Such as selecting a specific implementation of some functionality (e.g., choosing between linear algebra packages or XML parsers).

-Matt

[Daniel Solano Gomez]

As a transitional measure, why can't some of the renamed namespaces be
retained in such a way that they simply create aliases from the old
names to the new names? I don't know the size of the existing 1.0/1.1
code base, but generally other languages/APIs tend to keep deprecated
functions, at least for a few releases.

Perhaps some new sort of :deprecated metadata can be added such that
when someone uses the old API, a error is printed (similar to what
happens with the ^ reader macro). Such warnings could be turned off via
something ling *deprecation-warnings* or the like.

The main idea is to minimise breakage during the 1.[01]->1.2 transition,
but make it clear that in the future (1.3 or 1.4) the deprecated APIs
will be removed.

These are just some thoughts. I am not sure this would solve Chas
Emerick's problem, but it would definitely be helpful once 1.2 is
released.

Sincerely,

Daniel Solano Gómez

[Richard Newman]

> The solution seems good, though it could be (mis?)used for reasons
> other than dealing with deprecated namespaces. Such as selecting a
> specific implementation of some functionality (e.g., choosing
> between linear algebra packages or XML parsers).

I would use this kind of thing to target the same servlet code at
different containers. Real-world code often uses container-specific
classes, which often differ only by class name. Writing

(ns my.servlet
(or
(:import com.foo.bar.BazHelper)
(:import com.noo.goo.BazHelper)))

would be a wonderful thing.

[Chas Emerick]

On Apr 16, 2010, at 12:44 PM, Richard Newman wrote:

>> The solution seems good, though it could be (mis?)used for reasons
>> other than dealing with deprecated namespaces. Such as selecting a
>> specific implementation of some functionality (e.g., choosing
>> between linear algebra packages or XML parsers).
>
> I would use this kind of thing to target the same servlet code at
> different containers. Real-world code often uses container-specific
> classes, which often differ only by class name. Writing
>
> (ns my.servlet
> (or
> (:import com.foo.bar.BazHelper)
> (:import com.noo.goo.BazHelper)))
>
> would be a wonderful thing.

The same thing applies to logging frameworks (see c.c.logging) and
libraries for which one wants to provide wide compatibility (such as
lucene, for which we maintain compatibility for versions 1.2 all the
way up through 3.x (whatever the latest is)).

Thinking about it a little more though, there's a lot more that goes
into such compatibility efforts than just imports and namespaces -- as
a simple example, signatures have changed between c.c.str-utils2 in
1.1.0 and c.c.string in 1.2.0.

Maybe the "fallback namespaces" I suggested plus feature conditionals
in the reader would be the ultimate end goal? How do other platforms/
languages/etc handle this?

- Chas

[Phil Hagelberg]

On Fri, Apr 16, 2010 at 5:19 AM, Matt Revelle <mrev...@gmail.com> wrote:
>> Perhaps some kind of fallback namespace declaration is needed?  Something that looks like this, perhaps:
>>
>> (ns foo.bar
>>  (or (:require ...)
>>      (:require ...)))
>>
>> ...which implies short-circuiting behaviour as well as insensitivity to FileNotFoundExceptions when attempting to load code.

When clojure.contrib.test-is got moved to clojure.test, the old
namespace was kept around as a compatibility layer that simply wrapped
clojure.test. Perhaps a similar strategy would be wise for other
contrib namespaces that have moved?

> I wouldn't think contrib namespaces (at least the most commonly-used: io, seq, string) will be changing names again.  Why do you think this problem will become more common?

Hopefully some of them would be promoted out of contrib into Clojure
itself at some point.

-Phil

[Stuart Sierra]

On Apr 16, 6:47 am, Chas Emerick <cemer...@snowtide.com> wrote:
> In the process of attempting to maintain 1.1 and 1.2 compatibility in  
> various libraries, I've found myself writing stuff like this a lot:
> (try
>    (use '[clojure.contrib.reflect :only [call-method]])
>    (use '[clojure.contrib.io :only [as-file]])
>    (catch Exception e
>      (use '[clojure.contrib.java-utils
>             :only [file wall-hack-method]
>             :rename {wall-hack-method call-method, file as-file}])))

I don't like this; if everyone does this we'll have a mess. Instead,
maybe we should forward-port the old namespaces from contrib 1.1 to
1.2, marked "deprecated."

-SS

[Chas Emerick]

On Apr 22, 2010, at 11:01 AM, Stuart Sierra wrote:

> On Apr 16, 6:47 am, Chas Emerick <cemer...@snowtide.com> wrote:
>> In the process of attempting to maintain 1.1 and 1.2 compatibility in
>> various libraries, I've found myself writing stuff like this a lot:
>> (try
>> (use '[clojure.contrib.reflect :only [call-method]])
>> (use '[clojure.contrib.io :only [as-file]])
>> (catch Exception e
>> (use '[clojure.contrib.java-utils
>> :only [file wall-hack-method]
>> :rename {wall-hack-method call-method, file as-file}])))
>
> I don't like this; if everyone does this we'll have a mess. Instead,
> maybe we should forward-port the old namespaces from contrib 1.1 to
> 1.2, marked "deprecated."

Yeah, that's been suggested elsewhere -- I like that notion even less
than my hack. It delays the inevitable w.r.t. contrib, especially if
there's no policy around what "deprecated" means (i.e. how long do you
think test-is will linger?). Even worse, it means that bugs fixed in
the new namespaces (and presumably not backported to the old) will
continue to be encountered by applications using 1.2+.

FWIW, it's possible to quarantine the above hack, as done here:

http://github.com/hugoduncan/pallet/blob/master/src/pallet/compat.clj

Cheers,

- Chas

[Alex Osborne]

Chas Emerick <ceme...@snowtide.com> writes:

> Yeah, that's been suggested elsewhere -- I like that notion even less
> than my hack. It delays the inevitable w.r.t. contrib, especially if
> there's no policy around what "deprecated" means (i.e. how long do you
> think test-is will linger?). Even worse, it means that bugs fixed in
> the new namespaces (and presumably not backported to the old) will
> continue to be encountered by applications using 1.2+.

I'm with SS on this one as well, the try blocks (or even worse
making the ns macro even more complex and confusing) just seem like
trouble in the making. If the changes are just renames and thus just
your "or" syntax is good enough then can't the old name just immigrate
the appropriate vars from the new one, hence they benefit from any bug
fixes in the new namespace?

The other problem with doing it this way, is this only works for
code you own. What if you're using somebody else's library and they
haven't gotten around to putting in this try/catch block for when they
import contrib? You're stuffed.

Many other languages with large standard libraries do deprecation by leaving
a compatibility module which works but prints an ugly warning for at
least a version or two. They then remove it later if/when it becomes a
burden to maintain -- with major version number releases being a good
time to garbage collect anything that hasn't already become a burden.

The way things were just renamed/merged without leaving any trace has
already caused a lot of confusion as evidenced by the number of IRC
questions and mailing list posts. Even just throwing an error
"duck-streams has become io, please fix your code" when trying to
use/require it would be better than this. Particularly given that
these libraries (particularly duck-streams) are some of the most
frequently used Clojure libraries and the changes are likely to break a
very large amount of code.

[Chas Emerick]

Well, it's not just a matter of var immigration -- there have been a
bunch of signature changes, and potentially changes to semantics (the
proposed PrintWriter => Writer change for c.c.io/writer being one that
comes to mind from another recent thread).

(FWIW, I'm personally not concerned with 1.1 compatibility; that's
mostly a concession to the requirements of other open source projects
that I help out with, and in those circumstances, the eager/simple try-
catch approach has been good enough so far...along with a couple of
tweaks to accommodate the aforementioned signature changes.)

Anyway, I suppose the real question w.r.t. contrib is: is it a
standard library, or is it a proving ground and experimentation area
for code that might eventually find its way into core? I'd always
known it to be the latter. Those who have contributed the most to
contrib might feel otherwise, but I know I wouldn't be too keen on
having all of the expectations that come along with "standard library"
status placed at my feet after the fact.

That said, throwing an exception from the top level of the old
namespaces with a useful message seems like a good idea, and easily
within reach.

Cheers,

- Chas

[Stuart Halloway]

Rich has pretty strong opinions on this, which I will attempt to
channel here:

(1) Clojure itself has done well by being Java-like, in that things
don't just go away on you.

(2) While contrib is not a stdlib, we have hurt contrib users by
having things just disappear during the 1.2 alpha phase.

(3) There will be a joint 1.2 release of clojure and contrib.

(4) In the joint 1.2 release, we want to break people who never left
1.1 as little as possible. This probably means rehydrating some
contrib namespaces (e.g. duck-streams) that have vanished. Where that
doesn't work, a clear error message is the second best bet.

(5) We do not want to design and implement a language-level
deprecation feature at this time.

Putting my own hat back on, and disagreeing slightly with the straw
man Rich I just created :-) I am on the fence between rehydrating the
old namespaces and just adding the exceptions. The real question to
me: will the pain of the latter push people forward, or away?

Stu

> Well, it's not just a matter of var immigration -- there have been a
> bunch of signature changes, and potentially changes to semantics
> (the proposed PrintWriter => Writer change for c.c.io/writer being
> one that comes to mind from another recent thread).
>
> (FWIW, I'm personally not concerned with 1.1 compatibility; that's
> mostly a concession to the requirements of other open source
> projects that I help out with, and in those circumstances, the eager/

> simple try-catch approach has been good enough so far...along with a

[Alex Taggart]

I haven't kept up with the versioning of contrib, but am I incorrect in thinking that if some code references c.c.duck-streams, and some other code references c.c.io, then one simply needs to put 2 contrib jars on the classpath (e.g., v1.x and v2.x)? Isn't that kind of the point of namespaces?  Heck, you could even bundle them together in one release JAR for the transition.

- Alex

On Thu, Apr 22, 2010 at 6:35 PM, Stuart Halloway <stuart....@gmail.com> wrote:

Rich has pretty strong opinions on this, which I will attempt to channel here:

(1) Clojure itself has done well by being Java-like, in that things don't just go away on you.

(2) While contrib is not a stdlib, we have hurt contrib users by having things just disappear during the 1.2 alpha phase.

(3) There will be a joint 1.2 release of clojure and contrib.

(4) In the joint 1.2 release, we want to break people who never left 1.1 as little as possible. This probably means rehydrating some contrib namespaces (e.g. duck-streams) that have vanished. Where that doesn't work, a clear error message is the second best bet.

(5) We do not want to design and implement a language-level deprecation feature at this time.

Putting my own hat back on, and disagreeing slightly with the straw man Rich I just created :-)  I am on the fence between rehydrating the old namespaces and just adding the exceptions. The real question to me: will the pain of the latter push people forward, or away?

Stu

Well, it's not just a matter of var immigration -- there have been a bunch of signature changes, and potentially changes to semantics (the proposed PrintWriter => Writer change for c.c.io/writer being one that comes to mind from another recent thread).

(FWIW, I'm personally not concerned with 1.1 compatibility; that's mostly a concession to the requirements of other open source projects that I help out with, and in those circumstances, the eager/simple try-catch approach has been good enough so far...along with a couple of tweaks to accommodate the aforementioned signature changes.)

To unsubscribe from this group, send email to clojure-dev...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/clojure-dev?hl=en.

--
You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
To post to this group, send email to cloju...@googlegroups.com.
To unsubscribe from this group, send email to clojure-dev...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/clojure-dev?hl=en.

--
You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
To post to this group, send email to cloju...@googlegroups.com.
To unsubscribe from this group, send email to clojure-dev...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/clojure-dev?hl=en.

[Stuart Sierra]

On Apr 22, 9:35 pm, Stuart Halloway <stuart.hallo...@gmail.com> wrote:
> (4) In the joint 1.2 release, we want to break people who never left
> 1.1 as little as possible. This probably means rehydrating some
> contrib namespaces (e.g. duck-streams) that have vanished. Where that
> doesn't work, a clear error message is the second best bet.

Yes. I say just copy the old namespaces wholesale, so that their
behavior is identical.

In each deprecated namespace, put this at the top level:
(println "Namespace foo is deprecated, please use bar instead.")

As an alternative, create a sub-module of contrib containing the old
1.1 namespaces, but that's a lot more work for us maintainers.

-SS