re: oo
Raoul Duke
my take on it so far is that inheritance apparently sounds like a good
idea at first to some folks, but quickly turns into something of a
nightmare if one is actually concerned with keeping a coherent
semantics (so that (a) people can learn it (b) simulate it in their
heads and (c) so you can have tools to check things maybe and (d)
ultimately so the code using it sucks less -- things like LSP and Java
Generics show how fubar it can easily be). so people have come up with
mixins, and then traits. and interfaces. and stuff.
mainly, i just hope that whatever OO Clojurians finally coalesce
towards doesn't make the same mistakes that have already been made by
plenty of other languages, from old C++ through to new Scala. (not
that i have the Programming Language Theory chops to help much there,
myself.)
sincerely.
mikel
It's basically an automated facility for copying common code. On the
plus side, it's one approach to providing support for the principle of
"Don't Repeat Yourself". On the minus side, most systems with
inheritance combine the inheritance of behavior with the inheritance
of data structure, which are two totally different things. As always,
conflating orthogonal concerns leads to gratuitous duplication: you
inherit from a class because you want its data layout, but end up
writing gratuitous extra code because the inherited behavior is no
good to you (or vice versa). Also on the minus side, systems with
inheritance encourage you to scatter your API definition over lots of
places, which can be an obstacle to maintenance and judicious use.
I personally like Haskell's philosophy in this area: there is a
facility for defining data layouts, and there is a facility for
defining protocols, and they are completely separate. You can define a
new data layout without any reference to any behavior. Similarly, you
can define a new protocol and the behaviors it supports, without any
reference to any particular data layout. You can implement any
protocol you like at any time for any data layout you want, without
needeing to change the definitions of any data layouts. And,
conversely, you can add and extend data layouts any time you want,
without affecting existing implementations of behavior.
A Clojure system with a similar philosophy might consist of two
facilities: a facility for defining record types with named fields,
with record-extension that enables you to say "this record is that
other record over there plus these fields"; and a facility for
defining multifunctions that dispatch on some computable
characteristic of their arguments.
Clojure's built-in defstruct provides a way to specify record types,
but not to extend them. Konrad Hinsen is experimenting with a
typesystem that provides another way to define data layouts. Clojure's
built-in MultiFns provide one way to define functions that dispatch on
computable characteristics of their arguments. I'm experimenting with
implementing CLOS-style generic functions, which is another approach
(I have found Clojure MultiFns awkward to use thus far, especially
when trying to design extensible APIs around them; client code has to
know too much about the implementation details of MultiFns--e.g. you
have to know when and how to hand-tweak dispatching for the particular
MultiFns you are given).
Konrad Hinsen
> I personally like Haskell's philosophy in this area: there is a
> facility for defining data layouts, and there is a facility for
> defining protocols, and they are completely separate. You can define a
I like that as well, and I have been trying to do something similar
in Clojure with my types module (clojure.contrib.types) and my
collection of generic interfaces (clojure.contrib.generic). It's too
early to say if these will work out well, but I do think they are as
close to Haskell's philosophy as possible in a dynamically typed
language.
I haven't thought much about extending types yet. It could mean
opening the can of worms associated with inheritance and all that. I
am waiting for a concrete situation where extension would be useful
to think about how best to do it.
> Clojure's built-in defstruct provides a way to specify record types,
> but not to extend them.
It is not really meant to define types, the documentation clearly
says that struct maps are just an implementation of maps optimized
for a specific frequent use case.
> (I have found Clojure MultiFns awkward to use thus far, especially
> when trying to design extensible APIs around them; client code has to
> know too much about the implementation details of MultiFns--e.g. you
> have to know when and how to hand-tweak dispatching for the particular
> MultiFns you are given).
Could you elaborate a bit on this? I haven't met any major obstacles
with multimethods yet. The dispatch functions give quite a lot of
flexibility in practice. In what situation did you find them
inconvenient?
Konrad.
Mark Engelberg
<konrad...@laposte.net> wrote:
> Could you elaborate a bit on this? I haven't met any major obstacles
> with multimethods yet. The dispatch functions give quite a lot of
> flexibility in practice. In what situation did you find them
> inconvenient?
To summarize, I think the points that have been raised here and on
related threads are that:
1. Structs don't inherently have a type. If you want to dispatch on
type (a common use-case), you have to make a constructor that inserts
the type information as part of the struct. Some have expressed
concern that it may be too easy for this "type information" to be
altered, or worse, the data could be "changed" or "removed" in a way
that makes the struct inconsistent with the type label it carries
around.
2. No way to call "next-method" or "super", which limits the ability
to reuse related methods.
3. The dispatch mechanism requires a lot of explicit prefer-methods,
or else it may be hard to guarantee you won't get a run-time error
from a situation the dispatch system considers ambiguous. This also
makes code less extensible because to add a method, you must know
about all the other methods that have been implemented in order to
insert all the appropriate preferences.
Honestly, the one time I used Clojure's multimethods (contrib.math),
they suited my needs perfectly, but I definitely see where these
concerns are coming from. The existing mechanism is very flexible in
one respect: you can dispatch on anything you want, not just type. In
other respects, it seems like Clojure's multimethods are less
sophisticated than their CLOS/Dylan counterparts. If there is a way
to achieve the same things in Clojure, it's not readily apparent (see
recent thread about the next-method/super issue, for example).
mikel
On Mar 25, 4:13 am, Mark Engelberg <mark.engelb...@gmail.com> wrote:
> On Wed, Mar 25, 2009 at 1:44 AM, Konrad Hinsen
>
> > Could you elaborate a bit on this? I haven't met any major obstacles
> > with multimethods yet. The dispatch functions give quite a lot of
> > flexibility in practice. In what situation did you find them
> > inconvenient?
I agree with all the points you raise here.
> 3. The dispatch mechanism requires a lot of explicit prefer-methods,
> or else it may be hard to guarantee you won't get a run-time error
> from a situation the dispatch system considers ambiguous. This also
> makes code less extensible because to add a method, you must know
> about all the other methods that have been implemented in order to
> insert all the appropriate preferences.
> it seems like Clojure's multimethods are less
> sophisticated than their CLOS/Dylan counterparts
anything like prefer-method to prevent dispatching ambiguities in
either CLOS or Dylan. The requirement to hand-tweak dispatching
strikes me as particularly ugly. How can I design an extensible API on
MutiFns? I have no idea. Defining a new method can introduce an
ambiguity in dispatching that requires a prefer-method call to fix.
When I'm implementing the library, that's just a minor annoyance, but
when I contemplate delivering it to someone else for use and
extension, it suddenly turns into a major headache. Users of that
library can break existing code by defining new methods, because new
methods may introduce dispatching ambiguities. In order to fix the
breakage, they need to know the right prefer-method calls to make.
That means they need to understand esoteric details of the library
implementation for no good reason--for no reason at all, except that
if they don't know them, they won't be able to make the right prefer-
method calls.
Yuck.
Konrad Hinsen
> 1. Structs don't inherently have a type. If you want to dispatch on
> type (a common use-case), you have to make a constructor that inserts
> the type information as part of the struct. Some have expressed
And/or in the metadata.
> concern that it may be too easy for this "type information" to be
> altered, or worse, the data could be "changed" or "removed" in a way
> that makes the struct inconsistent with the type label it carries
> around.
I am one of those who have expressed concerns, but I must also say
that until now I have not encountered such a problem in real life. I
have come to the conclusion that interfaces and types work
differently in Clojure than in other languages that most of us are
more familiar with. So perhaps we are expecting problems that really
aren't there once you figure out how to do it "right".
> 2. No way to call "next-method" or "super", which limits the ability
> to reuse related methods.
Again, I'd like to see a real-life situation where this is an issue.
> 3. The dispatch mechanism requires a lot of explicit prefer-methods,
> or else it may be hard to guarantee you won't get a run-time error
> from a situation the dispatch system considers ambiguous. This also
And there as well.
> makes code less extensible because to add a method, you must know
> about all the other methods that have been implemented in order to
> insert all the appropriate preferences.
This would only be an issue in complex hierarchies. As long as each
library just adds its types to a hierarchy and implements methods for
them, there is no problem. That's why I'd like to see the real-life
situation where there is one.
Konrad.
Marko Kocić
portable?
I suppose that it contain base functions that are implemented in CL,
and the rest of clos built on top of that.
It might be interesting to try to adapt such a library to clojure.
I remember people mentioning PCL and closette as clos implementations,
but actualy never looked at them.
Regards,
Marko Kocić
mikel
write an extensible set of codecs for serializing and deserializing
game data, and for building a extensible set of data structures for
representing game state. The game in question is a relatively large
multiplayer RPG-style game whose data library needs to describe a
large number of different kinds of objects, support transfer of those
objects over networks and storage and retrieval on various kinds of
repositories, and which needs to be extended by a number of people on
an ongoing basis. I need to be able to clearly say what the protocols
are for creating existing objects, and for defining new kinds of
objects, and for serializing and deserializing them.
With a large number of types of objects, name collisions in field-
names becomes an issue. I can solve it by providing a mechanism to
define the names allowed in a particular kind of object, and a way to
declare requirements about the values allowed on those names. Storing
type tags in metadata is handy in that the type function then returns
the tag you want, and it's easy to make MultFns dispatch on those
values. On the other hand, not all objects that one might want to
dispatch on support metadata. That leaves storing type tags in the
object itself, which is fairly convenient, but as the disadvantages
Mark mentions. A partial workaround is to implement a feature for
decalring the fields allowed and required in maps, and to implement
consistency checking to ensure (to the extent that one can) that no
one uses the type-tag field for anything other than type-tags.
It's possible to provide a polymorphic API that covers each of the
various defined kinds of objects, but building that API involves a lot
of prefer-method calls. The inconvenience of that necessity increases,
seemingly without bound, as people add new kinds of objects and
methods to handle them. Each addition potentially requires its
implementor to examine an ever-increasing set of previously-defined
data descriptions and methods to determine what the correct prefer-
method calls are. There may not always be a correct set. When there
isn't, finding the data description and method definition that cause
the unsolvable problem isn't necessarily easy. I'm extremely
uncomfortable with any library that requires its user to be familiar
with all of its internals and all its dependencies; that's exactly
what libraries are meant to avoid. Therefore I am even more
uncomfortable with library-building tools that guarantee that's the
sort of library I'll be building.
The set of game data objects describes features of an imaginary world.
Those features (creatures, crafted items, factions, and so on)
naturally cluster into related taxonomies of shared and specialized
features. If you can say "this is just like those things except for
these features", you save a lot of duplicated code. Type extension is
one part of that ability; next-method is the other.
This is one "real-world" situation where all the issues Mark raised
are real and pressing issues for development, but it's not a special
situation. I've run into these issues many many times over the years.
They're why people keep on inventing mechanisms for type extension
over and over, and why they keep coming up with various different ways
to make protocols polymorphic. I ran into them almost twenty years
ago, working on a fairly large automated system for machine control,
written in a version of Common Lisp from before CLOS was widely
adopted. That project solved these issues by using a homegrown object
system.
If the language doesn't provide type extension and robust polymorphism
for protocols, its users will (assuming they keep using it).
mikel
example, it's a "standard" extension to Chicken Scheme (insofar as
anything to do with Chicken Scheme can be called "standard") and has
lots of enhancements provided by Felix.
The reason I didn't port Tinyclos to Clojure is that CLOS assumes a
specific model of classes, and I wanted to concentrate specifically on
generic functions, not add yet another typesystem to Clojure. You
could make a counterargument that the CLOS typesystem is specifically
designed to interoperate seamlessly with legacy typesystems, and you
would have a point.
Marko Kocić
On 25 мар, 21:41, mikel <mev...@mac.com> wrote:
> Tinyclos is decent, and lots of people have made good use of it. For
> example, it's a "standard" extension to Chicken Scheme (insofar as
> anything to do with Chicken Scheme can be called "standard") and has
> lots of enhancements provided by Felix.
> The reason I didn't port Tinyclos to Clojure is that CLOS assumes a
> specific model of classes, and I wanted to concentrate specifically on
> generic functions, not add yet another typesystem to Clojure. You
> could make a counterargument that the CLOS typesystem is specifically
> designed to interoperate seamlessly with legacy typesystems, and you
> would have a point.
Addition of clos like generic function to clojure that will work with
existing types will be excellent addition. But ability to define new
closs classes would make them even mose usefull. The hard part would
be to answer to the following questions:
Q1 - How to integrate existing java types, clojure types and clos
types into one consistent hierarchy wrt generic functions?
Q2 - How to support defining methods with the same name as existing
functions and let them be used seamlessly?
Q3 - How to handle arity overloads?
Q4 - How to approach immutability? Should clos class instances be
immutable maps or references to immutabe maps?
Qx - ...
Regards,
Marko Kocć
Rich Hickey
that break existing code implies both defining new methods on existing
super-types *and* some crosscutting multiple inheritance. I've seen it
happen only when first building method sets on existing hierarchies
not designed with this system in mind, e.g. when superimposing methods
on Java hierarchies. If you have an existing hierarchy from CLOS and
are disappointed that Clojure's multimethods don't work like CLOS,
well, ok.
However, all is not rosy with CLOS, and I've run into problems with
the left-to-right semantics of both arglists and superclass lists many
times, trying to game precedence lists etc. There's somethjing
inherently broken with not being able to derive from two superclasses
that have the same bases, just because they have different derivation
orders - see bottom of:
http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/Body/sec_4-3-5-2.html
Plus the inability to dispatch on other than class or eql, the
inability to superimpose another taxonomy without redefining the
class, the inability to have multiple independent taxonomies...
The reason Clojure doesn't have an elaborate system like CLOS, or even
like Python et al is that all of those systems have limits and quirks
and hardwiring one into a language is a bad idea, IMO. Clojure's
hierarchies and multimethods are just libraries - there's no reason
you couldn't define a CLOS-like object system for Clojure (although
any such system will have problems integrating Java's type hierarchy
because there is no definitive superclass ordering). I'm just trying
to tease out the useful ingredients of OO and make them available a la
carte.
Two things contribute to the sometimes-convenient default resolution
policy of CLOS:
One is left-to-right argument order precedence, which roughly
translates to vector precedence in Clojure (although the mapping to a
vector need not follow arg order). The other is default precedence of
classes based upon left-to-right declaration in defclass, and
topological sort based thereupon.
Right now, both matching and precedence follow isa?, but the latter
need not, or could be supplemented. Here are the characteristics I
think are important:
- It should be possible to superimpose a parent on an existing child
without modifying the child. This is a critical feature of Clojure's a
la carte hierarchies and one which I have desired in every OO language
(including CLOS) I have ever worked in. Requiring a client of
(independently developed) A and B to have to modify A in order to have
it work with B is unworkable.
- It should work transparently with Java hierarchies, which also
implies the above.
- It should support independent and order-independent assertions about
the hierarchy. Anything based upon canonic master definitions is
brittle and inextensible.
What I like about preference declarations is that you are
acknowledging a definite ambiguity in a pure derivation hierarchy
rather than poisoning the notion of hierarchy with directionality.
That said, perhaps the preference specifications at the method level
aren't as reusable/convenient as preferences declared at another level
or somewhere else.
I do care about making this easier to use, but arguments from
familiarity are insufficient - we have to acknowledge the tradeoffs
involved. I am not willing to repeat this behavior of CLOS for the
sake of convenience:
(defclass a () ())
(defclass b () ())
(defclass c (a b) ())
(defclass d (b a) ())
defclass e (c d) ())
(make-instance 'e)
Error: Error during finalization of class #<STANDARD-CLASS E
216B409B>: Cannot compute class precedence list for class: #<STANDARD-
CLASS E 216B409B>
Yuck.
Rich
Konrad Hinsen
> Plus the inability to dispatch on other than class or eql, the
> inability to superimpose another taxonomy without redefining the
> class, the inability to have multiple independent taxonomies...
The ability to have multiple taxonomies is indeed very useful in my
(limited) experience.
> One is left-to-right argument order precedence, which roughly
> translates to vector precedence in Clojure (although the mapping to a
> vector need not follow arg order).
I am not sure I'd want such a precedence. If I dispatch on several
arguments, their role is sometimes symmetric, sometimes not, and
where it is not, it is not necessarily the first argument that takes
precedence.
However, what I wished I had on some occasions is a way to specify a
sequence of dispatching values as a result of the dispatch function.
For example, in a two-argument dispatch, I'd like to be able to write
something like
(defmulti foo
(fn [x y]
(let [tx (type x) ty (type y)]
(list [tx ty] (cond (isa? tx ty) ty (isa? ty tx) tx :else #{tx
ty})))
with the result of calling a method for [tx ty] if available and one
of tx/ty/#{tx ty} otherwise. Without such an option, I sometimes have
to write several methods with an identical implementation but
different dispatch signatures.
One way to make this and other generalizations possible would be to
have access to the multimethod's implementation list (and perhaps
preference order) in the dispatch function.
> What I like about preference declarations is that you are
> acknowledging a definite ambiguity in a pure derivation hierarchy
> rather than poisoning the notion of hierarchy with directionality.
What I like about them is that an ambiguity often points to a design
problem that should better be fixed than circumvented.
> That said, perhaps the preference specifications at the method level
> aren't as reusable/convenient as preferences declared at another level
> or somewhere else.
How about a preference function on each multimethod, along with the
dispatch function? One could then easily use one such function for
several multimethods. But what I suggested above (access to the
dispatch signatures and preferences in the dispatch function) should
be even more general.
Konrad.
mikel
On Mar 26, 8:59 am, Rich Hickey <richhic...@gmail.com> wrote:
> On Mar 25, 6:47 am, mikel <mev...@mac.com> wrote:
> I wonder about the generality of this concern. Defining new methods
> that break existing code implies both defining new methods on existing
> super-types *and* some crosscutting multiple inheritance. I've seen it
> happen only when first building method sets on existing hierarchies
> not designed with this system in mind, e.g. when superimposing methods
> on Java hierarchies. If you have an existing hierarchy from CLOS and
> are disappointed that Clojure's multimethods don't work like CLOS,
> well, ok.
of data and a set of requirements for APIs. I know how to build a data
dictionary and API for these requirements in several languages; I
don't (yet) know how to do it in Clojure.
> However, all is not rosy with CLOS, and I've run into problems with
> the left-to-right semantics of both arglists and superclass lists many
> times, trying to game precedence lists etc. There's somethjing
> inherently broken with not being able to derive from two superclasses
> that have the same bases, just because they have different derivation
> orders - see bottom of:
>
I'll take your word for it that you've run into problems on this
point. I haven't.
The page that you cite explains what the potential conflicts are and
tells you what not to do, so that you can avoid those conflicts. Is
there an equally clear explanation of what not to do in Clojure, so as
to avoid visiting dispatch conflicts on users of my libraries? I
looked for such an explanation, but haven't found it yet. I hope I've
just missed it; I'd really rather use the built-in features of the
language.
> Plus the inability to dispatch on other than class or eql, the
> inability to superimpose another taxonomy without redefining the
> class, the inability to have multiple independent taxonomies...
Bauhaus and SK8 projects) did all of those things. If you mean, those
things are not ready to go out-of-the-box in CLOS, you're right. All
the CLOS implementations I've worked with, however, provide good
support for implementing those sorts of alternatives if you want them.
That's no slight on Clojure; I've found that Clojure also provides the
tools I need to build alternatives to the built-in features.
> The reason Clojure doesn't have an elaborate system like CLOS, or even
> like Python et al is that all of those systems have limits and quirks
> and hardwiring one into a language is a bad idea, IMO. Clojure's
> hierarchies and multimethods are just libraries - there's no reason
> you couldn't define a CLOS-like object system for Clojure (although
> any such system will have problems integrating Java's type hierarchy
> because there is no definitive superclass ordering). I'm just trying
> to tease out the useful ingredients of OO and make them available a la
> carte.
with MultiFns: as far as I can tell, they expose the internals of my
library implementations in unfortunate ways. Suppose you're using an
extensible codec API and you add support for a novel combination of
data formats. You write a new method and discover a brand new error
caused by a dispatch ambiguity. Now what? How do you find out what the
cause of the error is? What should the library provider have done to
ensure that you would not have this problem? What do you do to ensure
that your colleagues and customers won't have this problem?
The question isn't rhetorical; I actually encountered this exact
situation more than once, and it's the reason that I diverted time
from getting my main application done to writing a generic-function
library that makes that problem go away. The generic function library
works: the problem goes away, and all the relevant tests pass.
Go ahead and tell me that I failed to understand MultiFns correctly;
that would be good news. If you can tell me how to prevent the
aforementioned situation using MultiFns, then I can abandon generic
functions without regret, and get back to my actual goal. Note that I
am not asking for an estimate of how likely that sort of problem is.
Several people have mentioned that they don't think it very likely.
Their optimism aside, what interests me is a deterministic method of
preventing problems that have actually arisen.
> What I like about preference declarations is that you are
> acknowledging a definite ambiguity in a pure derivation hierarchy
> rather than poisoning the notion of hierarchy with directionality.
saying that you're not going to say what that order is. That's fine as
long as *I* can specify the traversal order, and I can, in part. But,
unless I've missed something important (always a possibility), prefer-
method only permits me to specify the order for graphs that are
already constructed. To make a safely-extensible library, I need to
specify the traversal order for subgraphs that are not yet
constructed. Again, if there's a good way to do that and I've missed
it, I'll orphan my generic-functions code in a heartbeat.
> That said, perhaps the preference specifications at the method level
> aren't as reusable/convenient as preferences declared at another level
> or somewhere else.
creating a hierarchy, to say, "hey, we're gonna traverse this one in
this order, okay?"
> I do care about making this easier to use, but arguments from
> familiarity are insufficient
Laurent PETIT
I wanted to write something along those lines, but you Konrad did this better than I could have done :-)
Though I don't have any concrete need currently:
Am I the only one to have the feeling that the current functionalities provided by multimethods are too tied with the idea that the dispatching will be based on type hierarchies ?
I think the current implementation closes the door to a lot of possibly interesting experiments/applications, because, as Konrad said, both the way the mechanism of discovering candidate methods, and the way to disambiguing them is somewhat hardcoded for a certain vision of type hierarchies.
The current implementation, while maybe a good default (I don't have strong arguments against or pro for saying it is the right default mechanism) could certainly be just an implementation of a redefinable more generic solution.
The way I understand it, the algorithm for discovering candidate methods is
1- compute a value from the arguments of the call,
2- and then determine for each method's declared dispatch-value if it is a relevant candidate
Why not make the behavior of 2- user redefinable in the defmulti declaration ?
Such a method could have the following definition :
(fn [computed-value dispatch-values])
This method could return either nil (then a method with a default dispatch-value would be searched as usual), either a list of candidate dispatch-values (no requirement on the order of the returned values, this would be the job of the disambuating method).
And the way I understand it, the algorithm for disambiguating candidate methods is
1- call a method that will apply an algorithm only based on deterministic relations between known-to-the-implementer-or-user existing methods.
Why not make this 1- behavior just a "default" behaviour, and let the possibility for the user to totally use another one, by passing it to the defmulti declaration as a function :
(fn [computed-value candidate-values])
(I'm not sure computed-value is interesting here, though, but maybe better have it than not)
This method could return an ordered list of candidates.
a) if the list is empty, then prefer-method will be called in a last resort
b) else the first item of the list is the method to dispatch to. And the rest of the list could be passed to the method in a non intrusive way (could it be via rebinding a global ?)
Thanks if you read past through this :-) And I'll be happy to see comments on this one. This really reflects what I had in mind for some time, so if I'm wrong in certain (or totally) areas, I'll be happy to learn from my own errors :-)
--
Laurent
Konrad.
Mark Engelberg
exactly which situations require prefer-method and which do not. One
thing that would help me understand the issues more deeply would be if
someone could post the simplest possible multimethod that requires
prefer-method to disambiguate.
In Rich's list of ideals, I particularly appreciate the desire to be
able impose a parent on a child without changing the child. This has
come up for me, and I was grateful Clojure had this ability.
I didn't see anything in the list of ideals that explains the lack of
a next-method, and it's not really clear to me how to reap some of the
benefits of inheritance without this.
On a somewhat related note, as my code grows, and I'm using more
complicated data structures, representing all of them as hash tables
is starting to feel a little too, well, unstructured. Can anyone else
report from the Clojure trenches on ways to manage this complexity?
Mark Engelberg
simplest ambiguous case I've found:
a b
| |
----------
|
c
user> (defmulti test-prefer :tag)
#'user/test-prefer
user> (defmethod test-prefer ::a [h] "a")
#<MultiFn clojure.lang.MultiFn@6551c1>
user> (defmethod test-prefer ::b [h] "b")
#<MultiFn clojure.lang.MultiFn@6551c1>
user> (derive ::c ::a)
nil
user> (derive ::c ::b)
nil
user> (test-prefer {:tag ::a})
"a"
user> (test-prefer {:tag ::b})
"b"
user> (test-prefer {:tag ::c})
; Evaluation aborted.
user> (prefer-method test-prefer ::a ::b)
#<MultiFn clojure.lang.MultiFn@6551c1>
user> (test-prefer {:tag ::c})
"a"
So if I understand correctly, the crux of the problem is that this
preference information is not part of the actual derive hierarchy.
This gives added flexibility to customize on a method by method basis,
but at a cost.
One cost seems to be that if an outside user wants to write a new
method on this very same type hierarchy, he can only do it if he knows
how the types are derived from one another inside the library so that
he can restate all the necessary precedences with appropriate
prefer-method calls on his own new method.
Another interesting question is whether at least test-prefer is safe
for extension. Here's a potential problem I see. Let's say that in
the above example, type ::c is what is intended to be publicly
visible, and the base classes ::a or ::b just provide the base
implementation for various methods, including prefer-method.
I may come along and want to extend test-prefer to a type ::d which
derives from ::c and ::e, where ::e provides an alternative
implementation.
a b (a and b are intended to be hidden from end-user)
| |
----------
|
c e
| |
------------
|
d
(derive ::d ::c)
(derive ::d ::e)
(defmethod test-prefer ::e [h] "e")
Now, as an external user, I know nothing about where test-prefer on
::c gets its behavior. Obviously, I have to disambiguate between
whether test-prefer chooses ::c over ::e, so I may try something like
this:
(prefer-method test-prefer ::c ::e)
But this will not work. I still get an error saying I need to
disambiguate between ::a and ::e. And not knowing anything about ::a,
I could be very confused, and not know how to provide this
information.
Considering how complex the situation can get with single dispatch, I
imagine it gets even more complex in multiple dispatch situations. In
the multimethods example at clojure.org/multimethods, an example is
given of disambiguating between [::shape ::rect] and [::rect ::shape].
But again, if you're writing the bar method from outside of the
library which defines ::shape and ::rect, you might not even know
which is more specific. It might be easier to choose a strategy, such
as more specific on the leftmost taking precedence, than to know the
details of how the various types in the library interact.
Anyway, this is my attempt to digest and restate what I've learned
from this thread and from tinkering around. Any other good examples
that illustrate some of the difficulties?
Konrad Hinsen
> I think I've answered at least part of my own question. This is the
> simplest ambiguous case I've found:
>
> a b
> | |
> ----------
> |
> c
Indeed. An ambiguous situation can occur whenever the type hierarchy
graph is not a tree. Since it takes at least three nodes to make a
graph that is not a tree, your example is the simplest one possible.
> So if I understand correctly, the crux of the problem is that this
> preference information is not part of the actual derive hierarchy.
That is one way to put it, but I think there are other possible
solutions than putting the preference information into the hierarchy.
> One cost seems to be that if an outside user wants to write a new
> method on this very same type hierarchy, he can only do it if he knows
> how the types are derived from one another inside the library so that
> he can restate all the necessary precedences with appropriate
> prefer-method calls on his own new method.
That could be avoided with a set of utility functions/macros for
placing types into a hierarchy and for defining multimethods. These
utility functions could take care of adding prefer-method calls as
necessary.
Of course, if this is a frequent need, explicitly language support
would be preferable, but at least that would be a way to experiment
with design options to figure out what works best.
> Now, as an external user, I know nothing about where test-prefer on
> ::c gets its behavior. Obviously, I have to disambiguate between
> whether test-prefer chooses ::c over ::e, so I may try something like
> this:
> (prefer-method test-prefer ::c ::e)
>
> But this will not work. I still get an error saying I need to
> disambiguate between ::a and ::e. And not knowing anything about ::a,
> I could be very confused, and not know how to provide this
> information.
True. It would be nice if prefer-method could itself figure out
that ::a provides the implementation for ::c. But again that
functionality can be added with utility functions, as the hierarchy
can be explored using the functions parents, ancestors, and descendants.
> Considering how complex the situation can get with single dispatch, I
> imagine it gets even more complex in multiple dispatch situations. In
> the multimethods example at clojure.org/multimethods, an example is
> given of disambiguating between [::shape ::rect] and [::rect ::shape].
> But again, if you're writing the bar method from outside of the
> library which defines ::shape and ::rect, you might not even know
> which is more specific. It might be easier to choose a strategy, such
> as more specific on the leftmost taking precedence, than to know the
> details of how the various types in the library interact.
I am not so sure about this. I wonder if there is a real-life use
case where a client library would need to solve dispatching issues on
types in another library about which it doesn't know anything. If
there isn't, the problem is not relevant, and if there is, I'd like
to see a demonstration that something like left-to-right precedence
is indeed a reasonable default.
On the other hand, I would definitely like to be able to implement
left-to-right precedence myself on top of Clojure's multimethods, and
it seems that at the moment this is not possible.
Konrad.
Laurent PETIT
On the other hand, I would definitely like to be able to implement
left-to-right precedence myself on top of Clojure's multimethods, and
it seems that at the moment this is not possible.
And also the ability to redefine the method that tries to determine candidate dispatch values, so that one could write polymorphic functions not only based on ahead of time defined data hierarchy graphs.
One solution could be to add more options to defmulti.
Another solution could be to have a *defmulti-flavor* global var that could have for value some hashmap : { :dispatch-value-matcher function1 :dispatch-value-narrower function2 } that could be bound to a different flavor for certain multifunctions by library creators.
Without that, for example, I don't know how one could prevent the combinatorial explosion of the prefer-method calls to make, even inside the boundaries of a library, even with simple hierarchies without multiple inheritence, for the following problem:
A, B, C, D, E, F all inherit from Z.
The library implementor wants to provide specialized functions such as:
(defmethod a-method [A B C D] [a b c d] ...)
(defmethod a-method [A B D C] [a b d c] ...)
If the implementor is able to concisely describe the resolution mechanism with an algorithm (a function), he still will have to implement the results of his algorithm in terms of prefer-method declarations.
For example, the classic above cited resolution mechanism "the leftmost specific wins" must be manually hard-coded. And in this special case it gets even worse for extension purpose as well.
--
Laurent
mikel
On Mar 27, 5:56 am, Konrad Hinsen <konrad.hin...@laposte.net> wrote:
> On Mar 27, 2009, at 9:25, Mark Engelberg wrote:
> > Considering how complex the situation can get with single dispatch, I
> > imagine it gets even more complex in multiple dispatch situations. In
> > the multimethods example at clojure.org/multimethods, an example is
> > given of disambiguating between [::shape ::rect] and [::rect ::shape].
> > But again, if you're writing the bar method from outside of the
> > library which defines ::shape and ::rect, you might not even know
> > which is more specific. It might be easier to choose a strategy, such
> > as more specific on the leftmost taking precedence, than to know the
> > details of how the various types in the library interact.
>
> I am not so sure about this. I wonder if there is a real-life use
> case where a client library would need to solve dispatching issues on
> types in another library about which it doesn't know anything.
library with the caveat that his people might run into unexpected
dispatch conflicts that could be solved simply by reading the library
sources, he'd say, "thanks, I'll pass."
> If
> there isn't, the problem is not relevant, and if there is, I'd like
> to see a demonstration that something like left-to-right precedence
> is indeed a reasonable default.
precedence worked well. "Demonstrate"? That's a lot of code to
recapitulate!
There is nothing magical about left-to-right precedence, except that
*you know in advance what it is*. You can reason about it, because you
know what it is. You can predict what kinds of uses will be a problem,
and therefore can avoid those problems. For purposes of writing
reusable and extensible subsystems, *any* defined order is better than
no defined order.
> On the other hand, I would definitely like to be able to implement
> left-to-right precedence myself on top of Clojure's multimethods, and
> it seems that at the moment this is not possible.
universal traversal order for MultiFn dispatch, as long as I can
define such an order when I need it. prefer-method isn't quite enough,
because it only works on dispatch values that are already defined. In
order to make a safely-extensible library, I need to be able to
prescribe a traversal order over dispatch values that haven't yet been
created.
I have a separate issue with prefer-method, in that it defines
traversal order implicitly, in a way that is hard to digest. That is,
you can't look in one place to find out what the traversal is; you
instead have to search out all the prefer-method calls and piece the
order together from examining each of them. That's tangential to the
present discussion, though.
If none of these considerations moves Rich much, that's okay. He gave
us the tools we need to write our own solutions.
Rich Hickey
There is prefers:
(prefers print-method)
-> {clojure.lang.ISeq #{clojure.lang.IPersistentCollection
java.util.Collection}, clojure.lang.IPersistentList
#{clojure.lang.ISeq}}
> If none of these considerations moves Rich much, that's okay. He gave
> us the tools we need to write our own solutions.
Hang tight - I hear your concerns and am thinking about them.
Rich
mikel
On Mar 25, 3:44 am, Konrad Hinsen <konrad.hin...@laposte.net> wrote:
> I haven't thought much about extending types yet. It could mean
> opening the can of worms associated with inheritance and all that. I
> am waiting for a concrete situation where extension would be useful
> to think about how best to do it.
dispatch and various approaches to dealing with that, but the topic
you touch on here is also interesting. It's probably about time I
experimented with your types library. I've been using my own extremely
simple model library, but I've been eyeballing clojure.contrib.types,
meaning to see if I like it better.
(Why would I use either one? Because I want a convenient way to write
definitions in source code that say what fields are to be expected and
allowed in application data.)
Michael Wood
>
> On 25 мар, 21:41, mikel <mev...@mac.com> wrote:
>> Tinyclos is decent, and lots of people have made good use of it. For
>> example, it's a "standard" extension to Chicken Scheme (insofar as
>> anything to do with Chicken Scheme can be called "standard") and has
>> lots of enhancements provided by Felix.
>
> I'll take a look at it. Haven't heard of if before.
I just saw this on Planet Lisp:
http://www.foldr.org/~michaelw/log/programming/lisp/clos
http://www.foldr.org/~michaelw/projects/jclos/
It's a port of Tiny-CLOS to Java by Michael Weber.
--
Michael Wood <esio...@gmail.com>
mikel
On Mar 28, 4:22 am, Michael Wood <esiot...@gmail.com> wrote:
>
> > On 25 мар, 21:41, mikel <mev...@mac.com> wrote:
> >> Tinyclos is decent, and lots of people have made good use of it. For
> >> example, it's a "standard" extension to Chicken Scheme (insofar as
> >> anything to do with Chicken Scheme can be called "standard") and has
> >> lots of enhancements provided by Felix.
>
> > I'll take a look at it. Haven't heard of if before.
>
> [...]
>
> I just saw this on Planet Lisp:
>
> It's a port of Tiny-CLOS to Java by Michael Weber.
for. You probably want to do some work on figuring out how to
integrate Clojure's various types. Scheme's system of types is simple
enough that it's not hard to fit CLOS to it. In the case of Common
Lisp it's a lot more work, but the work's already been done for you,
if you use any standard implementation of Common Lisp with a standard
CLOS in it.
The work to integrate Clojure is probably more akin to the work to
integrate Common Lisp, and it hasn't been done yet. In order to
compute slots and applicable methods, you need class precedence lists
for all possible values. That means you need a function that returns a
class for any input value. The clojure function named "class" doesn't
do the right thing, because the information it gives you for some
values is not correct; for example, if the input is an object with a
type tag in its metadata, the output of class is irrelevant. The
function named "type" doesn't work either, because it's possible to
construct perfectly good Clojure values for which that function is
undefined.
So, at minimum, to make a solid port, you need to add a function that
can return a sensible type value for any input, and you need to devise
a means of constructing some way of traversing any arbitrary set of
types to deterministically obtain a class-precedence list for it, so
that you can resolve dispatch. This can be done: doing it for Common
Lisp was no less troublesome than doing it for Clojure would be; but
it was significant work to do it, and you should expect that it would
be significant work to do it for Clojure, too.
And you should expect, as the designers of CLOS did, that not everyone
will be happy with your algorithm for constructing class-precedence
lists. Fortunately, the entire point of building CLOS on a MOP, as in
Tinyclos, is that anyone who doesn't like your solution has the tools
to make his own.
As an aside, Dylan was an example of a brand new Lisp that was
designed from the start to be built on CLOS. Dylan's fundamental model
of types was CLOS; every value (including classes and methods) was an
instance of a CLOS class. Of course, once the language was built, it
wasn't called "CLOS" anymore, because it wasn't Common Lisp. Nowadays
it doesn't look as much like a Lisp anymore, since s-expression syntax
was abandoned. It did contribute at least one thing of potential
relevance, though, for anyone considering implementing a CLOS-like
object system: the C3 class-linearization algorithm. Given any set of
classes, C3 gives you either a single, reliable ordering, or a clear
error that is capable of telling you what you need to fix.
David Nolen
So, at minimum, to make a solid port, you need to add a function that
can return a sensible type value for any input
mikel
(type (proxy [clojure.lang.IMeta clojure.lang.IRef][]))
java.lang.UnsupportedOperationException: meta (NO_SOURCE_FILE:0)
[Thrown class clojure.lang.Compiler$CompilerException]
No doubt someone is going to point out that the proxy object I created
there is useless; that's true, but beside the point. The point is that
it's straightforward to create some value v for which (type v) is
undefined. In order to make a Clojure-friendly version of CLOS, you
need some concept of object type such that you can define a function
that returns a sensible type for any value.
The way that Dylan handled situations similar to Clojure's proxy
objects was with objects called singletons; a singleton is an object
that represents a specific value as a class. For example, (singleton
5) returns an object that is a class, but is also the integer 5. You
can define methods that specialize on such objects.
For example, below is an excerpt from the UI libraries of Apple's long-
ago Bauhaus project. Notice the first line:
(define-method build-form ((key (singleton label:))
The equivalent syntax in Clojure (if clojure had these mechanisms)
would be:
(define-method build-form [[key (singleton :label)]
(There was no equivalent of defn in Dylan; there was only define-
method, because all values were instances of Dylan classes, and in
that environment a defn is no different from a define-method without
value constraints. Consequently, all functions were polymorphic; all
unsealed functions could be extended by adding new method
definitions.)
(define-method build-form ((key (singleton label:))
remaining-spec
form
current-x
current-y
layout-vector)
(ignore key)
(bind ((label-string (car remaining-spec))
(next (rest remaining-spec)))
(add-feature-to-form
(make <label-text>
string: label-string
top: current-y
left: current-x
style-array: $espy-10-bold-pointer)
next form current-x current-y layout-vector)))
David Nolen
(type (proxy [clojure.lang.IMeta clojure.lang.IRef][]))
java.lang.UnsupportedOperationException: meta (NO_SOURCE_FILE:0)
[Thrown class clojure.lang.Compiler$CompilerException]
No doubt someone is going to point out that the proxy object I created
there is useless; that's true, but beside the point. The point is that
it's straightforward to create some value v for which (type v) is
undefined. In order to make a Clojure-friendly version of CLOS, you
need some concept of object type such that you can define a function
that returns a sensible type for any value.
Meikel Brandmeyer
Am 29.03.2009 um 08:34 schrieb David Nolen:
> Not totally following you here as:
>
> (proxy [clojure.lang.IMeta clojure.lang.IRef][])
>
> immediately throws an error. I can't think of a situation in Clojure
> where the type function does not return a usable value. Let me know
> if I'm wrong, but your example is not a case as far as I can tell.
The point that Mikel wants to make is, that you can
easily provide a proxy, which implements clojure.lang.IMeta.
But when you don't provide a "meta" implementation,
the proxy will throw a UnsupportedOperation exception.
Hence type will not work.
I personally consider this a non-issue. What is the point
of providing an interface and then yell "I WONT DO IT" at
the system, when it tries to do the advertised action? In
my opinion, you should provide the advertised methods.
What's the point of the interface otherwise?
Sincerely
Meikel
Meikel Brandmeyer
Am 27.03.2009 um 09:25 schrieb Mark Engelberg:
> I may come along and want to extend test-prefer to a type ::d which
> derives from ::c and ::e, where ::e provides an alternative
> implementation.
>
> a b (a and b are intended to be hidden from
> end-user)
> | |
> ----------
> |
> c e
> | |
> ------------
> |
> d
>
>
> (derive ::d ::c)
> (derive ::d ::e)
> (defmethod test-prefer ::e [h] "e")
>
> Now, as an external user, I know nothing about where test-prefer on
> ::c gets its behavior. Obviously, I have to disambiguate between
> whether test-prefer chooses ::c over ::e, so I may try something like
> this:
> (prefer-method test-prefer ::c ::e)
>
> But this will not work. I still get an error saying I need to
> disambiguate between ::a and ::e. And not knowing anything about ::a,
> I could be very confused, and not know how to provide this
> information.
Is there some special reason, why choosing ::c is not enough in
prefer-method? As soon as I preferred ::c over ::e. Why should
I then need to go further up the tree?
Sincerely
Meikel
Konrad Hinsen
>> Enjoying the thread. Out of curiosity for which Clojure values is
>> the return
>> value of the type function undefined?
>
> Try this:
>
> (type (proxy [clojure.lang.IMeta clojure.lang.IRef][]))
>
> java.lang.UnsupportedOperationException: meta (NO_SOURCE_FILE:0)
> [Thrown class clojure.lang.Compiler$CompilerException]
>
>
> No doubt someone is going to point out that the proxy object I created
> there is useless; that's true, but beside the point.
Not entirely. Your example object is not only useless, it is as close
as possible to an object created intentionally to cause trouble. You
create an object that derives from IMeta, thus claiming that it
handles metadata, but then don't provide an implementation that would
actually make metadata work.
BTW, the function type could easily be fixed to handle your problem.
At the moment it does
(or (:type (meta x)) (class x))
Adding an exception handler that returns (class x) whenever (meta x)
fails would take care of your pathological object. I can't say if
this would add much runtime overhead, not being much of a JVM expert.
Konrad.
mikel
that calling type on it throws: because the print method for it calls
meta, which is not implemented.
David Nolen
mikel
On Mar 29, 7:22 am, Konrad Hinsen <konrad.hin...@laposte.net> wrote:
> On 29.03.2009, at 07:25, mikel wrote:
>
> >> Enjoying the thread. Out of curiosity for which Clojure values is
> >> the return
> >> value of the type function undefined?
>
> > Try this:
>
> > (type (proxy [clojure.lang.IMeta clojure.lang.IRef][]))
>
> > java.lang.UnsupportedOperationException: meta (NO_SOURCE_FILE:0)
> > [Thrown class clojure.lang.Compiler$CompilerException]
>
> > No doubt someone is going to point out that the proxy object I created
> > there is useless; that's true, but beside the point.
>
> Not entirely. Your example object is not only useless, it is as close
> as possible to an object created intentionally to cause trouble. You
> create an object that derives from IMeta, thus claiming that it
> handles metadata, but then don't provide an implementation that would
> actually make metadata work.
is an error.
The fact remains that it's a Clojure value. If you want to write a
Clojure implementation of CLOS, then you need your dispatching code to
be able to handle all Clojure values. If it doesn't, you have not
written a Clojure implementation of CLOS. This "pathological object"
is a Clojure value. Clojure's "type" function doesn't presently return
a type for such values, so, in order to implement CLOS, you're going
to need to write another type-returning function.
Discussions of whether such values make sense is beside the point. As
long as Clojure allows you to make them, any implementation of CLOS
for Clojure needs to deal with them.
mikel
On Mar 29, 4:40 pm, David Nolen <dnolen.li...@gmail.com> wrote:
> I see, thanks for the clarification. It does seem like you are purposefully
> creating a situation in which type will fail since something more likely:
> (type (proxy [clojure.lang.IRef][]))
>
> works fine.
>
> I'm wondering if a CLOS-like system would really have to handle such a
> degenerate (in the technical sense :) case as the one you've pointed out?
> I'm interested in hearing reasons if there is one.
which it's implemented. If Rich says that an object like the one I
created is not a Clojure value--that constructing such a value is an
error--then it makes sense to implement a version of CLOS that traps
them on input and raises an informative exception. Otherwise, we have
to accept that such values, no matter what we may think of them, are
Clojure values, and any implementation of CLOS for Clojure must do
something sensible with them.
Konrad Hinsen
> Sure, that's right. Maybe constructing such a value in the first place
> is an error.
I'd say so. If it were up to me to provide a fix for the situation
you describe, I'd fix proxy to make it impossible to create an object
that doesn't implement the interfaces it claims to implement.
However, it is well possible that such a fix would be difficult,
impossible, or imply a high run-time penalty. I don't know enough
about the JVM to judge.
However, I consider this case sufficiently pathological, and highly
unlikely to occur by mistake, that I'd accept functions raising
exceptions when presented with such an inconsistent object. In other
words, I'd declare such objects "not a valid Clojure object", without
necessarily enforcing that rule at object creation time.
Konrad.
Rich Hickey
method could prefer the path through ::c and not just ::c and its
descendants. I'm working on supporting this now.
Rich