Handling name collisions with clojure.core

[Mikera]

Hi all,

While building the API for core.matrix, I've fun into a few cases where the "best" name is a direct clash with clojure.core.

Examples are "+", "zero?", "vector?", "=="

In many of these cases, the core.matrix behaviour is a natural extension of the clojure.core function (i.e. it extends the same functionality to arbitrary N-dimensional arrays). 

I'm not very happy with any of the options I can see for handling this:

A) Use the good names in the "clojure.core.matrix" namespace. Problem: that gives you a ton of nasty warnings of the type "WARNING: + already refers to: #'clojure.core/+ in namespace: test.blank, being replaced by: #'clojure.core.matrix/+". Significant boilerplate must be maintained by the user in their ns declaration to prevent these warnings. I don't like forcing users to maintain boilerplate, and I think that normal idiomatic usage should be warning-free.

B) Separate the name-clashing functions into separate namespaces - e.g. "clojure.core.matrix.operators". Problem: that's something of an artificial division, and again it forces users to do extra ns-management work to access the functions they want.

C) Use different names. Problem: names would be worse, and this would be inconsistent and confusing, especially for functions that do effectively the same thing.

D) Encourage users to use aliases. Problem: that's horrendously ugly and inconvenient for numerical code. Users with any sense of elegance in their coding style would quite rightly throw their hands up in disgust. 

Currently we're doing B), I'd prefer to do A) but can't figure out a way to automatically suppress the warnings.

Any better ideas?

[Zach Tellman]

It is probably instructive to look at how (use-primitive-operators) works in primitive-math [1], though maybe not something you want to emulate.  The basic mechanism is pretty simple: use 'ns-unmap' to get rid of the operators you want to shadow, and bring in the operators from the alternate namespace.

Unfortunately, the next time you load the namespace, you'll get the same mess of collision warnings you were trying to avoid, because you haven't added the ':refer-clojure :exclude' clause to the ns declaration.  To get around this, use-primitive-operator hijacks 'ns' via alter-var-root so that this is automatically added if the alternate operators are detected [2], but otherwise leaves it unchanged.  This hijacking is undone if clojure.core is ever reloaded (usually by a :reload-all somewhere), however.  This could be fixed by adding a watcher to #'clojure.core/ns and forcing it back whenever it changes, but I haven't done that yet because it doesn't really affect me, and it feels hokey enough already.

Despite all that, though, it works pretty well.  Feel free to use this approach if you like, or create a less questionable variant if you can think of one.

Zach

[1] https://github.com/ztellman/primitive-math/blob/master/src/primitive_math.clj#L154

[2] https://github.com/ztellman/primitive-math/blob/master/src/primitive_math.clj#L135

[Mikera]

Hmmm clever trick. I hadn't thought about hijacking "ns" :-)

Still, it's a colossal hack. Monkey patching always makes me feel uneasy.

Perhaps Clojure itself needs patching to make this use case a bit more palatable? Some obvious options:

- Turn off all warnings for symbol replacement by default.

- Allow ^:ns-replace metadata for library functions that are intended to replace existing symbols (which would suppress warnings). You'd still get warnings on unintentional replacement, but it would allow library authors to make everything work smoothly when it is desired.

[Sean Corfield]

You only get the warning if you 'use' the namespace or 'refer all'
tho', correct?

And we've recently seen a lot of discussion that basically says "don't
do that" so it seems that either users of core.matric are going to
have two "approved" choices:
* require core.matrix with an alias, or choose to rename colliding
names however you want
* exclude the colliding symbols via refer-clojure and require them
from core.matrix as referred symbols

That's seems right to me: it is explicit and provides no surprises; it
gives the user control over how to manage things.

Sean

> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your
> first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+u...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.



--
Sean A Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
World Singles, LLC. -- http://worldsingles.com/

"Perfection is the enemy of the good."
-- Gustave Flaubert, French realist novelist (1821-1880)

[Dave Ray]

Maybe this is a dumb idea, but could you have a macro that rewrites code to use your ops?

  (require '[clojure.core.matrix :as m])

  (m/with-ops (+ ... (* ...) ...))

and then all the "special" symbols get rewritten/qualified with clojure.core.matrix?

Dave

[Alan Busby]

On Thu, Sep 5, 2013 at 2:35 PM, Dave Ray <dav...@gmail.com> wrote:

could you have a macro that rewrites code to use your ops?

  (require '[clojure.core.matrix :as m])

  (m/with-ops (+ ... (* ...) ...))

I wonder if there is value in a more general (with-ns clojure.core.matrix (foo (bar ...))) macro?

I can imagine a couple cases where I could use something like that.

[Mikera]

On Thursday, 5 September 2013 13:26:28 UTC+8, Sean Corfield wrote:

You only get the warning if you 'use' the namespace or 'refer all'
tho', correct?

And we've recently seen a lot of discussion that basically says "don't
do that" so it seems that either users of core.matric are going to
have two "approved" choices:
* require core.matrix with an alias, or choose to rename colliding
names however you want

* exclude the colliding symbols via refer-clojure and require them
from core.matrix as referred symbols

I remember the debates :-) and I don't think there was anything like a consensus that "don't do that" is the right answer. 

In particular note that neither "approved" choice is very nice for users:

- Aliases are pretty ugly for various DSL type situations. For numerical matrix code, it's really helpful to be able to write straightforward code without aliases e.g. (- R (+ M (* A x) (/ B y))) or similar. The equivalent with aliases is pretty unreadable by comparison.

- Manually excluding colliding symbols is just boilerplate to maintain. ns declarations are already complex/verbose enough, and we don't even have the tool support yet that makes this automatic in the way that Java does.

  

If there is a choice between improving the way that Clojure works or forcing extra inconveniences on users, I believe that we should choose the former every time.

 

That's seems right to me: it is explicit and provides no surprises; it
gives the user control over how to manage things.

I'm not against giving the user control. I just think the default behaviour should be super-friendly and not spit out warnings.

And in my view (:use clojure.core.matrix) is already being very explicit: our user wants to write numerical code in this namespace, and would like to have all the relevant functions available :-) 

[Colin Fleming]

This is actually probably not a bad solution. You wouldn't even need to rewrite, couldn't you just expand to a let?

(let [* clojure.core.matrix.*

      + clojure.core.matrix.+]
  (+ ... (* ...)))

Although thinking about it, you'd have to let-bind all possible operators every time, and the compiler doesn't do any dead-code elimination (although Hotspot might).

[Zach Tellman]

Not doing actual replacement via code-walking will prevent any functions with inline definitions actually being able to benefit from this.  I'm not sure if those are used in core.matrix, though.

You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/nar5vIAUo1c/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.

[Armando Blancas]

I just think the default behaviour should be super-friendly and not spit out warnings.

 

If other libs also redefine any of those operators or names, now or in later versions, I'd be glad to know. With last-one-in-wins some will lose. Maybe this will help: 

mvn clojure:repl 2> /dev/null

[Sean Corfield]

On Wed, Sep 4, 2013 at 11:25 PM, Mikera <mike.r.an...@gmail.com> wrote:
> I remember the debates :-) and I don't think there was anything like a
> consensus that "don't do that" is the right answer.

I didn't say there was consensus or that it was right :)

> I'm not against giving the user control. I just think the default behaviour
> should be super-friendly and not spit out warnings.

I don't think it's at all "friendly" to have standard functions
_silently_ replaced by some library I included. I want to know about
the collisions and I want to be able to control how those collisions
are resolved. Or I can explicitly decide to ignore the warnings. But
the warnings _should_ happen.

[Mikera]

On Friday, 6 September 2013 03:15:40 UTC+8, Sean Corfield wrote:

On Wed, Sep 4, 2013 at 11:25 PM, Mikera <mike.r.an...@gmail.com> wrote:
> I remember the debates :-) and I don't think there was anything like a
> consensus that "don't do that" is the right answer.

I didn't say there was consensus or that it was right :)

> I'm not against giving the user control. I just think the default behaviour
> should be super-friendly and not spit out warnings.

I don't think it's at all "friendly" to have standard functions
_silently_ replaced by some library I included. I want to know about
the collisions and I want to be able to control how those collisions
are resolved. Or I can explicitly decide to ignore the warnings. But
the warnings _should_ happen.

Nothing is being "silently replaced". We're talking about the case where the user has explicitly asked for this behaviour by their decision to "use" a whole namespace. Which is, as we know, something that some people like to do and others don't. If you don't want to use this double-edged sword, then you don't have to. Live and let live.

If you want exact control over how you import these symbols, then of course you still have it. But the default should IMHO be both a) convenient and b) warning free.

At most, this should be a warning that can be turned on in the same way as *warn-on-reflection*.

 

[Mikera]

Yeah, that could be tricky :-)

I think that this is best handled by something like a *warn-on-replace* option or similar that can be turned on to detect this case. 

Though hopefully this case is rather rare: I normally find that I only ever "use" one or two libraries per namespace, and these are carefully chosen because the namespace is all about expressing code in a specific DSL. Other libraries generally get required with an alias.

[John D. Hume]

I haven't tried this, so apologies if it couldn't even work, but have you considered providing a fn in your library intended to be used inside the ns macro?  The refer-clojure :exclude boilerplate could be replaced with something like this.

(ns my-thing
  (:require core.matrix.ns)
  (:core.matrix.ns/exclude-clojure-core-math-ops)
  (:use core.matrix))

It's not too much boilerplate, and probably explicit enough for anyone who was going to :refer :all anyway.

(Sorry for butchering your lib's name. Mobile keyboard bad for code.)