=> (defprotocol Foo
(a [x]))
Foo
=> (extend-protocol Foo
java.io.Serializable
(a [x] "*")
CharSequence
(a [x] "s"))
nil
=> (a (StringBuilder. "blah"))
"*"
Not "wrong" behaviour, but a sharp edge for sure.
Multimethods have `prefer-method`, so a `prefer-type` (or somesuch) for protocols would be most welcome.
Anyway, by now I remember some brief discussion about this on irc, and I see it's been considered to some degree; e.g. there is the "TBD" note on the original design docs here:
http://www.assembla.com/wiki/show/clojure/Protocols
and there's this ticket:
http://dev.clojure.org/jira/browse/CLJ-295
I've found that the above situation is so easy to get into that I almost wish that the extend-* fns for protocols only worked over concrete types, so I can certainly sympathize with Rich's thought (ultimately rejected, along with the ticket) that an exception should be thrown when a protocol fn is asked to dispatch in such cases.
----
Is there any more recent movement/thought in this area?
- Chas
To unsubscribe from this group, send email to clojure-dev...@googlegroups.com.
The problem is actually a little worse than that, as the output is not
predictable across JVM instances (at least in Clojure 1.2.0).
I encountered an issue with protocol that extended clojure.lang.IFn
and java.util.Map. When the protocol function was passed a
PersistentHashMap, then 90% of the time it would be treated as a Map,
but 10% of the time it would be treated as an IFn. The ancestor type
it chose to dispatch on seemed to be determined when the JVM is
started up.
Constantine Vetoshev describes the issue in more detail here:
http://groups.google.com/group/compojure/browse_thread/thread/113e0bc47d793d36/91def69b5a9444eb
- James
> On 10 December 2010 06:16, Chas Emerick <ceme...@snowtide.com> wrote:
>> Right now, there is no corollary to `prefer-method` for protocols.
>>
>> Not "wrong" behaviour, but a sharp edge for sure.
>
> The problem is actually a little worse than that, as the output is not
> predictable across JVM instances (at least in Clojure 1.2.0).
>
> I encountered an issue with protocol that extended clojure.lang.IFn
> and java.util.Map. When the protocol function was passed a
> PersistentHashMap, then 90% of the time it would be treated as a Map,
> but 10% of the time it would be treated as an IFn. The ancestor type
> it chose to dispatch on seemed to be determined when the JVM is
> started up.
>
> Constantine Vetoshev describes the issue in more detail here:
>
> http://groups.google.com/group/compojure/browse_thread/thread/113e0bc47d793d36/91def69b5a9444eb
>
> - James
"when the JVM is started up"? I presume it's flowing from hashmap entry order being undefined. In any case, your point is well taken.
- Chas
I mean that individual JVM processes behaved consistently, e.g. it
would always prefer a Map over an IFn for the life of the process. But
sometimes a JVM would start where IFn was preferred over Map.
> I presume it's flowing from hashmap entry order being undefined.
Yes, that's probably the cause, come to think of it. If the types are
ambiguous, I guess it must use whatever type comes first when
iterating over the map passed to `extend`.
- James
http://dev.clojure.org/jira/browse/CLJ-295
I've found that the above situation is so easy to get into that I almost wish that the extend-* fns for protocols only worked over concrete types, so I can certainly sympathize with Rich's thought (ultimately rejected, along with the ticket) that an exception should be thrown when a protocol fn is asked to dispatch in such cases.
----
Is there any more recent movement/thought in this area?
- Chas
> I kind of see this as a variant of programmer error similar to when you extend both java.lang.Object and java.lang.String to the same protocol. Extending a protocol to an interface sounds like it was only intended for interop.
I would doubt that -- there's no sense in not taking advantage of the type hierarchy that you find yourself within. Protocols and multimethods (via isa?) are entirely in lockstep in this regard.
That said, this raises in interesting tangential meta issue (and I'm probably putting words in your mouth here): do you really consider any use of interfaces to constitute interop? I would say just the opposite: Clojure seems to embrace interfaces wholeheartedly in a number of ways. I can see dimly though how one might characterize that embrace as entirely a result of practical concerns motivated by what the host offers (e.g. to support efficient polymorphic dispatch) rather than a proper endorsement of interfaces.
- Chas
I would doubt that -- there's no sense in not taking advantage of the type hierarchy that you find yourself within. Protocols and multimethods (via isa?) are entirely in lockstep in this regard.
On Dec 10, 2010, at 12:13 PM, David Nolen wrote:
> I kind of see this as a variant of programmer error similar to when you extend both java.lang.Object and java.lang.String to the same protocol. Extending a protocol to an interface sounds like it was only intended for interop.
That said, this raises in interesting tangential meta issue (and I'm probably putting words in your mouth here): do you really consider any use of interfaces to constitute interop? I would say just the opposite: Clojure seems to embrace interfaces wholeheartedly in a number of ways. I can see dimly though how one might characterize that embrace as entirely a result of practical concerns motivated by what the host offers (e.g. to support efficient polymorphic dispatch) rather than a proper endorsement of interfaces.
- Chas
In this case, it would probably be a bad idea (certainly no fun) to extend a protocol to all concrete types under ISeq to have the appropriate semantics, yet I think that's what you would need to do to make this unambiguous.
Sure, until clojure core adds a new seq impl in 1.3 and retroactively opens an edge case in my protocol... :) Seems like a bad idea to ever explicitly call out a Clojure core *concrete class* in your own code, much less try to specify a complete set of subtypes under an interface.
Why not leverage those inheritance relationships that already exist in those classes/interfaces in the core? You can't proactively query reflection for those (can't ask for all impls of a type) and register those. But you can leverage the upward (is my instance a foo?) question at runtime ala multimethods.
Am 10.12.2010 um 20:50 schrieb Alex Miller:
> In this case, it would probably be a bad idea (certainly no fun) to extend a protocol to all concrete types under ISeq to have the appropriate semantics, yet I think that's what you would need to do to make this unambiguous.
Hmm.. It is my understanding, that things like ISeq will turn into a PSeq – a protocol. Then you can't extend another protocol to a PSeq thing. Because PSeq then is not a type anymore. Then you would have to catch all concrete types.
Is that correct?
Sincerely
Meikel
Am 10.12.2010 um 21:42 schrieb Chas Emerick:
> I agree with Alex, though this is far from a future hypothetical. When extending protocols to types from third parties, you generally don't *want* to know about (nevermind directly reference) concrete classes, and often there's no sane way to determine what the full domain is of those concrete classes – e.g. in the case where there's a factory method / function somewhere that emits implementations of some interface. There, you absolutely want to extend to that interface.
Here again, we have the question: who defines the extension of a protocol. You should only extend a protocol to third-party types if you are the „owner“ of that protocol. You basically do it on behalf of the third party. And then you have to understand how these classes work inside.
> The fact we run into this in pure-Clojure code is largely a side effect of (a) the standard data structures having a large interface "footprint", and (b) the (correct, IMO) desire to extend protocol implementations to high-level interfaces rather than concrete types. The situation will not get prettier as defrecord gets more and more usage, especially in conjunction with third-party interfaces…
Again, I think protocols are taking over the part of interfaces. There will be the need to reference concrete types there. I think that is the reason, why Object and String are not a problem – there we have an hierarchy. This is not the case with interfaces. But this will go away (for clojure code at least) as protocols replace interfaces, since you can't extend a protocol to a protocol. (A protocol also defines an interface, but this is not enough, because the protocol could be extended to any existing type, which does not implement the interface.)
Please correct me in case I got something terribly wrong about protocols.
Sincerely
Meikel
I agree with Alex, though this is far from a future hypothetical. When extending protocols to types from third parties, you generally don't *want* to know about (nevermind directly reference) concrete classes, and often there's no sane way to determine what the full domain is of those concrete classes.
From: David Nolen <dnolen...@gmail.com>
Sent: Fri, December 10, 2010 2:39:44 PM
Subject: Re: prefer mechanism for protocols
On Fri, Dec 10, 2010 at 3:24 PM, Alex Miller <alexd...@yahoo.com> wrote:Sure, until clojure core adds a new seq impl in 1.3 and retroactively opens an edge case in my protocol... :) Seems like a bad idea to ever explicitly call out a Clojure core *concrete class* in your own code, much less try to specify a complete set of subtypes under an interface.I don't see why it's a bad idea to call out a Clojure concrete class if you have a protocol that needs to be extended to them.
Why not leverage those inheritance relationships that already exist in those classes/interfaces in the core? You can't proactively query reflection for those (can't ask for all impls of a type) and register those. But you can leverage the upward (is my instance a foo?) question at runtime ala multimethods.Bringing a type to a protocol is easy. Bringing a protocol to a open set of types which have a open set of interfaces seems to me like you're asking for a world of trouble.
In the first case if you miss a type, just add it to the list. In the second case ... not sure how to get out of that particular tarpit.
David
On Fri, Dec 10, 2010 at 3:24 PM, Alex Miller <alexd...@yahoo.com> wrote:Sure, until clojure core adds a new seq impl in 1.3 and retroactively opens an edge case in my protocol... :) Seems like a bad idea to ever explicitly call out a Clojure core *concrete class* in your own code, much less try to specify a complete set of subtypes under an interface.I don't see why it's a bad idea to call out a Clojure concrete class if you have a protocol that needs to be extended to them.
Why not leverage those inheritance relationships that already exist in those classes/interfaces in the core? You can't proactively query reflection for those (can't ask for all impls of a type) and register those. But you can leverage the upward (is my instance a foo?) question at runtime ala multimethods.Bringing a type to a protocol is easy. Bringing a protocol to a open set of types which have a open set of interfaces seems to me like you're asking for a world of trouble.
They're implementation details, subject to change. Surely intimate knowledge of them shouldn't be required in order to reliably get desired dispatch behaviour from a protocol.
I think it's instructive to look at where protocols need to go. Consider `seq` in the context of stringy things (defined by the CharSequence interface). Surely we wouldn't want `seq` to fail on previously-unknown implementations of CharSequence (of which there are many)?- Chas
In the second case, you either have conflict which you can resolve after the fact with preferences OR you have an unhandled type which you can handle via the trick Rich mentioned during the Sean's protocol talk at the conj. That is, extend Object and have that impl notice that you fell through the protocol dispatch and dynamically extend a protocol to the type you just encountered (after which you will route directly, not through Object). My colleague Dave McNeil did this recently and it worked great.In the first case if you miss a type, just add it to the list. In the second case ... not sure how to get out of that particular tarpit.
The key question is, what should be the basis of choosing one over the
other? Clearly, the ambiguity is inherent. Java allows multiple
inheritance of interfaces, and by hanging implementations off of
interfaces one gets multiple inheritance of implementation. It could
be resolved:
a) as an error - MI is bad, no MI for you
b) mechanically - I don't see a mechanical solution significantly more
meaningful than alphabetical order of classname, given the information
we have available, and it is unlikely to please anyone
c) using external information
Multimethods encountered the same problem and chose (c), using the
prefer system. While that usually enables you to proceed, proceed in
what is a good question. MI is inherently difficult to reason about.
Adding preferences ad hoc and dynamically begets a race to determine
what to prefer, and there is no good way to arbitrate conflicting
preferences. In addition, they are not at all declarative, so one has
to do dynamic querying to see what the relationships are (at the
moment). Finally, as Meikel has pointed out in this thread, moving
forward there will be fewer interfaces and more protocols, so there
won't necessarily be interfaces to hang things from. As we move away
from concrete static type hierarchy, to what should we attach any
categorization/disambiguation system (and can it be made fast)?
It seems weird to complain about having to connect the abstraction
(protocol) to a concrete class, when that is exactly what the author
of the concrete class had to do in choosing its interfaces (once and
for all). In one sense, all protocols do is open up that capability.
Defining concrete behavior for abstractions themselves (rather than
their concrete instances) is somewhat broken from a logic standpoint.
I see the appeal (define something once to cover an open set of
cases), but when combined with interface MI, the logic problem
remains. Preferences just patch up the logic by inserting sequentially
considered conditionals.
A is true of Xs
B is true of Ys
Fred is an X and a Y, but only A true of Fred.
In short, I think preferences are an ok idea, very much tied to
interfaces, and I'd like to see something better for protocols. Until
then, I've been punting and waiting to see what people really need.
Brilliant ideas welcome,
Rich
> b) mechanically - I don't see a mechanical solution significantly more meaningful than alphabetical order of classname, given the information we have available, and it is unlikely to please anyone
I'm sure you're familiar with how CLOS deals with multiple inheritance (of classes). I can't say I've investigated this deeply, but I remember that the Dylan folks came up with an improved method [1], and suggested another algorithm that Python implemented (C3 Linearization) [2].
I'm thinking the Dylan paper might inspire an approach that would work for Java interface multiple inheritance. The basic idea is that, for a given type, you compute a class/interface precedence list -- a linearization of the superclass and inherited interfaces. The precedence list is used to resolve conflicts from multiple inheritance such that the most "specific" one is preferred. The Dylan paper explains why a monotonic linearization is desirable.
Linearization is similar in spirit to the earlier suggestion that result of supers should be sorted. The C3 Linearization is a specific way of sorting the precedence list so that the preferences are in some sense "natural".
A Monotonic Superclass Linearization for Dylan
[1] http://www.webcom.com/haahr/dylan/linearization-oopsla96.html
> As described above, the Dylan linearization merges the local precedence order of a class with the linearizations of its direct superclasses. When there are several possible choices for the next element of the linearization, the class that has a direct subclass closest to the end of the output sequence is selected.
>
> It should be clear that the Dylan linearization is monotonic, because the merge procedure never reorders the linearizations of superclasses when producing the linearization. Similarly, it obeys local precedence order because the merge explicitly takes local precedence into account, and the local precedence orders of superclasses are propagated by the linearizations.
>
The Python 2.3 Method Resolution Order
[2] http://www.python.org/download/releases/2.3/mro/
Steve Miner
stev...@gmail.com
Am 01.03.2011 um 19:07 schrieb Steve Miner:
> I'm sure you're familiar with how CLOS deals with multiple inheritance (of classes). I can't say I've investigated this deeply, but I remember that the Dylan folks came up with an improved method [1], and suggested another algorithm that Python implemented (C3 Linearization) [2].
>
> I'm thinking the Dylan paper might inspire an approach that would work for Java interface multiple inheritance. The basic idea is that, for a given type, you compute a class/interface precedence list -- a linearization of the superclass and inherited interfaces. The precedence list is used to resolve conflicts from multiple inheritance such that the most "specific" one is preferred. The Dylan paper explains why a monotonic linearization is desirable.
>
> Linearization is similar in spirit to the earlier suggestion that result of supers should be sorted. The C3 Linearization is a specific way of sorting the precedence list so that the preferences are in some sense "natural".
I have no clue how a linearization works; let alone a monotonic or C3 one. I would prefer a more „obvious“ solution than some obscure algorithm however nifty and cunning it might be. Be it erroring out or some kind of preference declaration.
Consider the „sorting“ mentioned above. How to sort things? Alphabetically on the name? The Dylan link said, that the order of class names was significant. How do you control this with protocols? Think of the load order of namespaces. A change in a sublibrary might suddenly change your order of protocol extension; and hence your precedence list. So we have to come up with an even more cunning algorithm?
Whatever the final solution looks like: I hope, it's (relatively) easy to understand and to reason about.
Sincerely
Meikel