Protocols and default method implementations
Matthew
I've been looking at the new protocol (defprotocol, deftype,
defrecord, etc) feature in 1.2 and, while I love the core concepts,
it's been bothering me that there's no apparent way to provide
*automatic* default implementations for methods on a protocol.
I know from reading Rich Hickey's discussions on the design rationale
[1] and other threads on providing Scala Trait-like behaviour [2] that
you can choose to mix in a set of defaults when you implement the
protocol, but it seems important to me that a protocol author be able
to provide default implementations to enable protocols to evolve.
Allowing a function body in the defprotocol clause would seem to be
the obvious way, something like:
(defprotocol P
(foo [x] (default-foo x))
(bar-me [x] (bar-me [1 x])
[x y])) ; no default implementation
As I'm sure most of you know, a key problem with "pure" interface
features in environments like CORBA, COM, Java, etc has been that you
can't add new methods to published interfaces: you'd break all
existing implementations. Hence you see a history of evolving
interfaces indicated by names like IFoo, IFoo2, IFooEx, etc. In
contrast, a Scala-like trait system allows you to add a new method
*and* a default implementation (supposing that makes sense), so old
clients still work fine.
The problem, as I see it, with Clojure protocols is that you would
have to rely on clients mixing in a set of defaults in order to
happily extend a "public" protocol.
Am I missing something key here? I realise there are some very
experienced people contributing to Clojure, so am fully expecting to
be told I've missed something obvious ;)
Cheers,
Matthew.
[1] http://groups.google.com/group/clojure/msg/330c230e8dc857a9
[2] http://groups.google.com/group/clojure/msg/53228e04db4799a5
Laurent PETIT
Maybe have the library writer do this ? :
(defprotocol P
(foo [x] (default-foo x))
(bar-me [x] (bar-me [1 x])
[x y])) ; no default implementation
{:foo default-foo
:bar-me #(bar-me 1 %) })
and let the user pick the whole *P-defaults* for its mixins, or just parts of it ? (does this make sense ?)
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Shantanu Kumar
And Sean's comment here: http://programming-puzzler.blogspot.com/2010/08/racket-vs-clojure.html
Regards,
Shantanu
On Aug 12, 5:46 pm, Laurent PETIT <laurent.pe...@gmail.com> wrote:
> Hi,
>
> 2010/8/12 Matthew <mattp...@gmail.com>
Stuart Halloway
Take your example of a function that has a reasonable default for any object. Often such a function does not need to be part of the contract for implementers (i.e. the protocol) at all.
To make this more concrete, look at how protocols are used in Clojure itself. InternalReduce is a contract for implementers, consumers call reduce. Ditto for IOFactory: clients don't call it, they call the functions reader, writer, etc. instead.
Stu
Cameron
>Often such a function does not need to be part of the contract for implementers (i.e. the protocol) at all.
do you handle the case where you have a general function that works
for every type you'd like to implement a protocol for (thus not
technically needing to be in a protocol), but maybe 1 or 2 of the many
types have more efficient implementations possible? Do you just suck
it up and copy and paste the general function around? Or is there a
better way? Maybe the new case function?
After reading the docs on clojure.org/protocols, it seemed like you
could just define the general implementation of functions under Object
and then define your specific implementations on the other types, but
this didn't seem to work for me. I would get an error to the effect of
"foo is not defined for Type"
Stuart Halloway
Stu, (or anybody) I'd like to ask about a variation on this point. How
do you handle the case where you have a general function that works
for every type you'd like to implement a protocol for (thus not
technically needing to be in a protocol), but maybe 1 or 2 of the many
types have more efficient implementations possible? Do you just suck
it up and copy and paste the general function around? Or is there a
better way? Maybe the new case function?
After reading the docs on clojure.org/protocols, it seemed like you
could just define the general implementation of functions under Object
and then define your specific implementations on the other types, but
this didn't seem to work for me. I would get an error to the effect of
"foo is not defined for Type"
Nicolas Oury
The approach of extending with maps is great but lacks of access to
the content of the instance variables of the type.
(def Foo [bar]
)
(extend
clever-automatic-construction
)
As far as I know, clever-automatic-construction cannot use bar in its
implementation.
Which might be a bit annoying, because you have to rely on the
interface of the implementor of one thing
to implement another interface.
Maybe creating a getBar automatically could help?
Or there are other ways to work around that?
Best,
Nicolas.
Nicolas Oury
It could also be helpful to have an easy access (meta information, for
example) to the
fields (and their types) of a record type.
nickikt
problems. Protocol solve some of the Problems and Traits solve some
but the do not solve the same. Would be interesting to hear how
Protocols solves those problem, why the don't are problems or why the
don't need to be solved in Clojure because of whatever reason.
I couldn't use them in practices jet but I really like the idea.
Nicolas Oury
I just discovered (.bar (Foo. 5)) => 5
It is a bit embarrassing. Sorry about that. (I am not a Java person, I reckon)
As far as traits are concerned, I think they can be made with a couple
of macros.
I might try to have a go over the week-end.
Best,
Nicolas.
Sean Devlin
reuse. In today's article I discuss what I've termed partially
implemented protocols, which is geared toward providing a default
implementation. Granted, it's a bit ugly and I'll be the first to
admit that it starts to confuse the difference between a protocol &
implementation. Still, it cuts down on the bookkeeping required.
http://fulldisclojure.blogspot.com/2010/08/partially-implemented-protocols.html
Have fun
Sean
Tim Daly
protocol "is". Can someone use some other comp. sci. terms to
define this idea? I thought of them as Java interfaces with
default methods but clearly I'm wrong.
Randy Hudson
functions/methods without providing an implementation. The big
distinction is in more dynamic usage. Rich Hickey's description at
http://clojure.org/protocols is well written.
On Aug 12, 7:52 pm, Tim Daly <d...@axiom-developer.org> wrote:
> I find that I'm horribly confused at this point about what a
> protocol "is". Can someone use some other comp. sci. terms to
> define this idea? I thought of them as Java interfaces with
> default methods but clearly I'm wrong.
>
>
>
> Sean Devlin wrote:
> > I've posted a follow up to my article yesterday about protocols & code
> > reuse. In today's article I discuss what I've termed partially
> > implemented protocols, which is geared toward providing a default
> > implementation. Granted, it's a bit ugly and I'll be the first to
> > admit that it starts to confuse the difference between a protocol &
> > implementation. Still, it cuts down on the bookkeeping required.
>
Rich Hickey
On Aug 12, 2010, at 7:52 PM, Tim Daly wrote:
> I find that I'm horribly confused at this point about what a
> protocol "is". Can someone use some other comp. sci. terms to
> define this idea? I thought of them as Java interfaces with
> default methods but clearly I'm wrong.
>
Coming from CL, the best analogy is that a protocol is a *named* set
of generic functions that dispatch on the type of their first
argument. When you define a protocol with foo/bar/baz methods you get
generic functions in the protocol's namespace named foo/bar/baz.
Unlike interfaces, protocols require no derivation. You can extend a
protocol to a particular type by supplying a concrete map of method
names to function objects (extend), *or* the code for same (extend-
type), *or* by supplying definitions for the protocol methods inline
in a deftype/defrecord. The latter *looks* a lot like implementing an
interface, by design it is as easy to use, and might actually involve
implementing an interface under the hood (but that is an
implementation detail, enabled by the naming and first-arg dispatch
restriction).
Finally, from Java-land, you can get your type to extend a protocol by
deriving from the protocol's corresponding interface. This is what
seems to confuse people most, as if protocols are magic dynamic
interfaces. They are not. They *sometimes* use interfaces in the
implementation, but not always. At no point should one assume that
because a type supports a protocol it 'isA' that protocol (or its
interface).
Protocols are not about inheritance and that is the key distinction vs
interfaces.
Hope that helps,
Rich
Nicolas Oury
> not. They *sometimes* use interfaces in the implementation, but not always.
> At no point should one assume that because a type supports a protocol it
> 'isA' that protocol (or its interface).
>
That's very interesting. From a performance point of view, is there a
penalty involved in being in a case
where the implementation does not use interfaces?
Meikel Brandmeyer
On 13 Aug., 14:58, Nicolas Oury <nicolas.o...@gmail.com> wrote:
> That's very interesting. From a performance point of view, is there a
> penalty involved in being in a case
> where the implementation does not use interfaces?
vs. hard-wiring things with inline definition).
Hmm... Is it possible to provide some methods inline in a defrecord,
and some methods via extend?
Sincerely
Meikel
Nicolas Oury
> Using extend is slower, but more dynamic (read: mix-in via map merge
> vs. hard-wiring things with inline definition).
>
>
Interesting...
Is there a link explaining how it is implemented in the extend
situation and how much slower it is?
Cheers,
Nicolas
nickikt
What are your thoughts on the default implementation problem?
Matthew Phillips
> > Stu, (or anybody) I'd like to ask about a variation on this
> > point. How do you handle the case where you have a general
> > function that works for every type you'd like to implement a
> > protocol for (thus not technically needing to be in a protocol),
> > but maybe 1 or 2 of the many types have more efficient
> > implementations possible? Do you just suck it up and copy and
> > paste the general function around? Or is there a better way? Maybe
> > the new case function?
>
> This might be a job for a more granular protocol. Take a look at how
> granular Clojure's implementation abstractions are (even prior to
> the introduction of protocols). The Java interfaces often have 0, 1,
> or 2 methods.
protocol, and now want to extend it. Instead, you could just create a
new (granular) protocol, but that does seem to lead inexorably to the
IFoo, IFoo2, IFoo3, situation.
A more concrete example: say I've defined a protocol for AST nodes in
1.0 of a library, and later when developing 2.0 I discover it would
have been a good idea to have a "pretty-print" method on nodes to show
human-readable output. If the protocol had Trait-like characteristics
I could add pretty-print to the protocol, with a default
implementation that just prints the fields of the node, but override
that with a better implementation for some of the new record types I'm
including in 2.0.
But I can't do that because, unless I've had the foresight to make
sure clients always mix in a base set of (partial) defaults that I
provide (which may be initially empty), then things break down. Sean
Devlin's ideas (referenced earlier on the thread) on making it easy to
mix in defaults would be great for this, but they don't really address
the issue I'm looking at here.
Following Stuart's suggestion, I *could* just add a protocol called
"PrettyPrintable" with one method and implement it on some of the new
node types, but now I can't just call "pretty-print" on any node: I
need to write another function that checks if it's a PrettyPrintable
first and calls something default if not.
I've had to do this sort of thing many, many times in Java: one of the
reasons I got excited about multimethods in Clojure is that they allow
me to transparently extend the system after the fact with no such
ugliness.
Thanks to the superpower that is macro, I'm sure I could make a
defprotocol+ and a extend+ that do this, just wanted to check that
there's really no better way.
Cheers,
Matthew.
Stuart Halloway
> "PrettyPrintable" with one method and implement it on some of the new
> node types, but now I can't just call "pretty-print" on any node: I
> need to write another function that checks if it's a PrettyPrintable
> first and calls something default if not.
You can just extend PrettyPrintable to Object.
Stu
Armando Blancas
> 1.0 of a library, and later when developing 2.0 I discover it would
> have been a good idea to have a "pretty-print" method on nodes to show
> human-readable output. If the protocol had Trait-like characteristics
> I could add pretty-print to the protocol, with a default
> implementation that just prints the fields of the node, but override
> that with a better implementation for some of the new record types I'm
> including in 2.0.
interfaces? In Clojure I'd like to do something like:
(doseq [node (walk-tree)] (pretty-print node))
and have it work regardless of how each node is made. I'd expect
(pretty-print) to do more work if it can't just do (.pretty-print
node), but client code need not be affected and the API need not be
versioned in the sense IFoo, IFoo2, etc. That is, I'd think that
custom Clojure libraries would be modeled after the sequence library
and use protocol/types as implementation details, unless it's a "Java
API" done in Clojure (which I anticipate doing myself actually).
Kevin Downey
print-ast which then checks to see if PrettyPrintable has been
extended to the object and falls back to the default if it hasn't
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
--
And what is good, Phaedrus,
And what is not good—
Need we ask anyone to tell us these things?
Matthew Phillips
wrote:
below as a hypothetical (and obviously trivial) graph node library
evolves through several versions. I've used extend-type on Object to
retro-actively provide default implementations for older clients.
Adding to Object works, but doesn't feel right: as libraries grow,
they'll start bloating out the method sets on the global Object type.
One idea that I tried was to use extend-type on a protocol, say to
extend any Node to be a PrettyPrintableNode. Obviously this didn't
work, and I'm not sure it actually makes semantic sense, but it's
interesting that was my intuitive action.
Example below, interested in any thoughts.
Matthew.
---
;; A node has a data attachment and (possibly) children
(defprotocol Node
(data [n])
(children [n]))
(deftype SimpleNode [d]
Node
(data [n] d)
(children [n] []))
;; In version 2, I add want to add pretty-printing
(defprotocol PrettyPrintableNode
(pretty-print [n]))
;; Make anything potentially pretty-print'able
(extend-type Object
PrettyPrintableNode
(pretty-print [n] (str "A node: " (data n))))
;; Later, in version 3, I decide nodes also need a tag
(defprotocol TaggedNode
(tag [n]))
;; Default to node's data if no tag
(extend-type Object
TaggedNode
(tag [n] (data n)))
;; A new type of node that has a settable tag
(deftype SuperNode [d t]
Node
(data [n] d)
(children [n] [])
PrettyPrintableNode
(pretty-print [n] (str "Node: tag " (tag n)
", data: " (data n)))
TaggedNode
(tag [n] t))
---
Examples of use
> (def n (SimpleNode. "Matt"))
> (def n2 (SuperNode. "Matt" "a tag"))
> (tag n)
"Matt"
> (tag n2)
"a tag"
> (pretty-print n2)
"Node: tag a tag, data: Matt"
> (pretty-print n)
"A node: Matt"
Matthew Phillips
> > A more concrete example: say I've defined a protocol for AST nodes in
> > 1.0 of a library, and later when developing 2.0 I discover it would
> > have been a good idea to have a "pretty-print" method on nodes to show
> > human-readable output. If the protocol had Trait-like characteristics
> > I could add pretty-print to the protocol, with a default
> > implementation that just prints the fields of the node, but override
> > that with a better implementation for some of the new record types I'm
> > including in 2.0.
>
> Is this because you've got Java clients coding against your
> interfaces? In Clojure I'd like to do something like:
> (doseq [node (walk-tree)] (pretty-print node))
using Java interfaces in library design. Although I guess Java interop
will be important for me since I have a lot of existing Java to
support,
Matthew Phillips
> so clients don't directly call the protocol functions they call
> print-ast which then checks to see if PrettyPrintable has been
> extended to the object and falls back to the default if it hasn't
implement, not an API that might make use of protocols internally. If
the clients implement *and* call methods via protocols, then that kind
of approach doesn't really work.
Stuart Halloway
> they'll start bloating out the method sets on the global Object type.
No, you have this backwards. The protocol is not on Object, Object is on the protocol. Protocols live in namespaces. You can have 10,000 different protocols extended to Object with no bloat problems. You can have multiple methods with the same names but different semantics, and use any or all of them as you see fit.
This is a major benefit of decoupling polymorphism from inheritance. There is no bloat, and no sacred place (on Object) where you compete for names.
Stu
Nicolas Oury
>
> One idea that I tried was to use extend-type on a protocol, say to
> extend any Node to be a PrettyPrintableNode. Obviously this didn't
> work, and I'm not sure it actually makes semantic sense, but it's
> interesting that was my intuitive action.
>
alpha - library some time ago.
I called it type classes because it does a part of what Haskell's type
classes do.
You can define a rule:
Node => PrettyPrintableNode (implementation of PrettyPrintableNode using Node)
And it extends Object with a default implementatiuon of protocol
PrettyPrintable, that just takes the object it is called on,
looks at its type, and try to apply the rules it can apply to it (You
could have a rule NodeV2-25 => PrettyFormatable and another
PrettyFormatable, ... => PrettyPrintable).
If it finds a non-cyclic path to construct a PrettyPrintable instance,
it extend the types with the right protocols implementation, and
recalls the function.
I planned to improve on it, but it seems there were a really low
interest for this kind of thing on the list when I posted.
I wouldn't recommand to use it in this state, but if you want to have
a look at the code:
https://nicolasoury.repositoryhosting.com/trac/nicolasoury_type-classes
Best,
Nicolas.
Kevin Downey
regular functions while protocols provide a simple pluggable lower
bound that users can implement. and when users do implement the simple
protocols they suddenly get all the advanced features of the library
functions, like if you implement ISeq, suddenly all of clojure's
sequence functions work on your data.
Steven E. Harris
> ;; A node has a data attachment and (possibly) children
> (defprotocol Node
> (data [n])
> (children [n]))
>
> (deftype SimpleNode [d]
> Node
> (data [n] d)
> (children [n] []))
>
> ;; In version 2, I add want to add pretty-printing
>
> (defprotocol PrettyPrintableNode
> (pretty-print [n]))
>
> ;; Make anything potentially pretty-print'able
> (extend-type Object
> PrettyPrintableNode
> (pretty-print [n] (str "A node: " (data n))))
I'm confused by the mixture of formality and looseness here.
When you extend this way, binding this `pretty-print' implementation to
arguments of type Object, you're implicitly requiring that argument "n"
satisfies the protocol Node -- because you call the `data' function on
it. But if the function `data' would work on any kind of Object, then so
too would any call to `pretty-print', right? Should the call to `data'
within the `pretty-print' definition above require some mention of
protocol Node?
In other words, what's the difference between using `extend-type' here
on Object and just writing
,----
| (defn pretty-print
| [n]
| (str "A node: " (data n)))
`----
My experiments in the REPL don't show any difference in behavior.
Being able to define a normal `pretty-print' function above differs from
the generic function system in Common Lisp. There, if one has defined a
generic function with `defgeneric', and one later tries to define a
normal function with the same name, the normal function replaces the
generic function, and the compiler may emit a warning. Working the other
way, though, one may not define a generic function -- using `defgeneric'
or `defmethod' -- if the function name is already bound to a normal
function, macro, or special operator.
Is there supposed to be a difference between the normal function
`pretty-print' I wrote above and the function defined in the
`extend-type' form quoted above?
--
Steven E. Harris
Matthew Phillips
So, I think my original thesis was correct: I'm clearly misconstruing
something quite fundamental here ;)
And I can see now my original example was clumsy: for example
something like PrettyPrintable *should* be an orthogonal protocol to
Node. (Not to mention the example I had in mind was actually a tree
not a graph :/)
Anyway, to either clarify, or further muddy, the waters here's another
scenario, for an actual graph library this time, assuming you'll
indulge me further.
---
;; A node has a data attachment and (possibly) out-edges
(data [n])
;; A client's implementation of Node using library 1.0's protocol
Node
(data [n] d)
;; In version 2, I decide nodes may also report their in-edges. I'd
;; actually like to add "in-edges" to Node, but can't because that
;; would break existing clients.
(defprotocol Node2
(in-edges [n]))
;; A client's new type of node using library v 2.0's protocols
(deftype SimpleNode2 [d in out]
Node
(data [n] d)
(out-edges [n] out)
Node2
(in-edges [n] in))
;; Now, if I want any node's in-edges, I can't just call "in-edges"
;; because Node implementations won't have it, so I compute them in
;; that case.
(defn node-in-edges [n]
(if (isa? Node2)
(in-edges n)
(compute-node-in-edges n)))
;; OR, I could use extend-type to allow in-edges to be called on
;; anything, and use a default implementation for Node's.
(extend-type Object
Node2
(in-edges [n]
(if (isa? Node)
(compute-node-in-edges n)
(<throw an exception?>))))
---
In contrast, if I could have added in-edges to Node in v 2.0, with an
accompanying default implementation for existing clients, then I could
have saved quite a lot of messing about.
Am I still on the wrong track here?
Matthew.
Matthew Phillips
think. On a type class you can specify default implementations of any
or all of the functions, and they can even refer to themselves
cyclically (e.g the Eq class where == and /= are defined as each
other's complements): the instance just needs to provide whichever one
is most suitable.
Matthew.
On Aug 14, 9:30 pm, Nicolas Oury <nicolas.o...@gmail.com> wrote:
Nicolas Oury
> ;; Now, if I want any node's in-edges, I can't just call "in-edges"
> ;; because Node implementations won't have it, so I compute them in
> ;; that case.
> (defn node-in-edges [n]
> (if (isa? Node2)
> (in-edges n)
> (compute-node-in-edges n)))
>
isa? should be satisfies?, I think.
Matthew Phillips
> The other thing that you should consider is that protocols are the
> contract for implementers, not the contract for callers. If you
> change a contract for implementers, then the implementers *must*
> change.
>
> Take your example of a function that has a reasonable default for
> any object. Often such a function does not need to be part of the
> contract for implementers (i.e. the protocol) at all.
>
> To make this more concrete, look at how protocols are used in
> Clojure itself. InternalReduce is a contract for implementers,
> consumers call reduce. Ditto for IOFactory: clients don't call it,
> they call the functions reader, writer, etc. instead.
http://vimeo.com/11236603. Thanks for such a clear and informative
presentation -- it was especially illuminating to be shown how
InternalReduce is used.
I certainly take your point that you generally will require
implementors to provide all methods of a protocol -- because if you
can provide a default, why are you asking clients to provide it in the
protocol?
However, in my last message I have an example of where an extension to
the protocol of an optional method that is there for performance: it
*can* be implemented slowly by default, but new clients might want to
provide a faster version. It's kind of the same situation as
InternalReduce: you can always reduce by treating the item as a seq
(using the InternalReduce implementation for clojure.lang.ISeq), but
InternalReduce can be re-implemented for specific types like
java.lang.String as an optional accelerator.
A new method can always be implemented on a new protocol, and a
default provided for Object, but I guess I'm looking for the best of
both worlds.
In looking for examples in Java and COM of interface evolution, the
instances that immediately come up are actually examples where
implementors really do need to provide all methods,
e.g. LayoutManager, LayoutManager2 (AWT), IProvideClassInfo,
IProvideClassInfo2 (COM). So perhaps what I'm really asking for is
better seen as an equivalent to a :default multimethod.
Matthew.