Re: Multimethods and Scope
Joe Groff
> but I like that it solves the monkey-patching problem, and it's pretty
> simple. If you have ideas on how to improvement, I'm definitely interested.
Forgive me for digging up a stale thread, but I wanted to throw an
idea in here. I think it's a good idea to have different keywords for
"define generic function" and "add method to generic function", so
it's clear when you're intending to extend a method from another
module instead of defining one in the current module. For Magpie, you
could keep 'def' as the "define generic" form, and another keyword
such as "ext" (short for "extend") as the "extend generic with method"
form:
// ---
// a.mag
def exclaim // creates a.exclaim
ext (s is String) exclaim // gives it a method
print(s + "!")
end
// def with a block can remain as shorthand for the two above
definitions
def (s is String) exclaim2 // creates a.exclaim2 and gives it a
method
print(s + "!")
end
// ---
// b.mag
import a
ext (i is Integer) exclaim // extends a.exclaim with another
method
print(i toString + "!")
end
(If you don't like "ext" feel free to substitute your own bikeshed
color.) I know that Common Lisp and Factor, two other languages with
class-independent generic functions, both use similar schemes
(defgeneric/defmethod for Common Lisp, GENERIC:/M: for Factor), so
there's some precedent. The case of variables is a bit trickier ("ext
var" doesn't read well, and I think its meaning would be difficult to
internalize), but in general, I think this would be a good approach.
-Joe
Bob Nystrom
> I think it's a good idea to have different keywords for
> "define generic function" and "add method to generic function", so
> it's clear when you're intending to extend a method from another
> module instead of defining one in the current module.
Yeah, I've wondered about that off and on.
> For Magpie, you
> could keep 'def' as the "define generic" form, and another keyword
> such as "ext" (short for "extend") as the "extend generic with method"
> form:
>
> // ---
> // a.mag
> def exclaim // creates a.exclaim
> ext (s is String) exclaim // gives it a method
> print(s + "!")
> end
In general, I like being explicit over implicit, but it feels like
this punishes the common case. Most methods *aren't* multimethods:
they're just a single local method defined in a module. Given that it
feels strange to me to require all of them to have a separate
declaration and definition.
> // def with a block can remain as shorthand for the two above
> definitions
> def (s is String) exclaim2 // creates a.exclaim2 and gives it a
> method
> print(s + "!")
> end
>
> // ---
> // b.mag
> import a
> ext (i is Integer) exclaim // extends a.exclaim with another
> method
> print(i toString + "!")
> end
One way to look at this is to ask what problems it solves for a user.
This is more verbose, so there's a tax to it. Is there a benefit they
get in return?
The one I think of is that if you don't realize that you're extending
an existing method in scope (probably one you imported that you didn't
realize) then when you do "def", you'll get an error to alert you to
that fact.
It's hard to tell how beneficial that is, though. If you inadvertently
extend a method and redefine a new one with the same pattern, that
*is* an error already. (Though, heh, Magpie doesn't actually report
that error yet...) If you're extending a method accidentally but using
a different pattern, it's likely that that won't cause any problems
anyway. When you call it, you'll match yours and not the other ones
you didn't realize you're bringing in.
> (If you don't like "ext" feel free to substitute your own bikeshed
> color.) I know that Common Lisp and Factor, two other languages with
> class-independent generic functions, both use similar schemes
> (defgeneric/defmethod for Common Lisp, GENERIC:/M: for Factor), so
> there's some precedent.
I'm not a CLOS expert, but I believe CL will actually automatically
create a generic for you if you do defmethod and it doesn't exist. So
if you never use defgeneric, I think you'll get the same behavior as
Magpie. CL is a bit different here too because I believe a given
generic function has to have the same arity across all methods? Magpie
doesn't have that restriction, since it doesn't have the same concept
of arity.
> The case of variables is a bit trickier ("ext
> var" doesn't read well, and I think its meaning would be difficult to
> internalize),
Fortunately we wouldn't need it for variables. You can't overload
them, and they live in a different namespace from methods.
> but in general, I think this would be a good approach.
One alternative would be to have something like def/ext but allow both
to also define a concrete method. So:
def foo()
...
end
means "define a brand new method foo() and throw an error if one
already exists" while:
ext foo()
...
end
means "define a new specialization of the existing method foo() and
throw an error if one *doesn't* exist."
I'm not crazy about "ext", but otherwise this seems kind of nice. Thoughts?
- bob
Joe Groff
On Oct 7, 2011, at 9:43 PM, Bob Nystrom wrote:
> It's hard to tell how beneficial that is, though. If you inadvertently
> extend a method and redefine a new one with the same pattern, that
> *is* an error already. (Though, heh, Magpie doesn't actually report
> that error yet...) If you're extending a method accidentally but using
> a different pattern, it's likely that that won't cause any problems
> anyway. When you call it, you'll match yours and not the other ones
> you didn't realize you're bringing in.
Well, I think you start exposing yourself to the classic duck typing problem whenever your types start implicitly subscribing to protocols they can't live up to. I agree it's already much less likely to be a problem in practice with your design as-is compared to the anarchy of Ruby, but I think it's still worth squelching entirely if you can.
> I'm not a CLOS expert, but I believe CL will actually automatically
> create a generic for you if you do defmethod and it doesn't exist. So
> if you never use defgeneric, I think you'll get the same behavior as
> Magpie.
You're right about Common Lisp; I didn't know that.
> def foo()
> ...
> end
>
> means "define a brand new method foo() and throw an error if one
> already exists" while:
>
> ext foo()
> ...
> end
>
> means "define a new specialization of the existing method foo() and
> throw an error if one *doesn't* exist."
Yeah, I think that's what I was going for. You may still want a blockless "def foo" form to create a generic without giving it any methods.
> Fortunately we wouldn't need it for variables. You can't overload
> them, and they live in a different namespace from methods.
I'm sorry, I meant to refer to class member slots. In your previous email, you had examples like this:
// pet.mag
defclass Pet
var name
end
// friend.mag
defclass Friend
var name
end
and you indicated that the implicitly-created accessor methods for "name" would end up clashing or coexisting depending on where on the import graph "name" originates. Needing different forms for "def var name" or "ext var name" to carry this strict behavior over to accessor methods is kind of gross.
-Joe
Bob Nystrom
life for a bit...
On Fri, Oct 7, 2011 at 10:27 PM, Joe Groff <arc...@gmail.com> wrote:
> Well, I think you start exposing yourself to the classic duck typing problem whenever your types start implicitly subscribing to protocols they can't live up to. I agree it's already much less likely to be a problem in practice with your design as-is compared to the anarchy of Ruby, but I think it's still worth squelching entirely if you can.
A fair point. I don't think I'm as worried about it as you are (my
experience is that method signatures are generally precise enough to
avoid accidental matches) but it's a real concern.
> Yeah, I think that's what I was going for. You may still want a blockless "def foo" form to create a generic without giving it any methods.
Magpie actually has that now. :)
It's nice for associating a method with a module even when that module
doesn't provide any implementations of it. It also lets you document
the method as a whole outside of any specializations.
>
>> Fortunately we wouldn't need it for variables. You can't overload
>> them, and they live in a different namespace from methods.
>
> I'm sorry, I meant to refer to class member slots. In your previous email, you had examples like this:
>
> // pet.mag
> defclass Pet
> var name
> end
>
> // friend.mag
> defclass Friend
> var name
> end
>
> and you indicated that the implicitly-created accessor methods for "name" would end up clashing or coexisting depending on where on the import graph "name" originates. Needing different forms for "def var name" or "ext var name" to carry this strict behavior over to accessor methods is kind of gross.
Ah you're exactly right. How quickly I forget even my own language! :)
The more I use Magpie, the more this particular wart irritates me. I
have a little test script right now that tries to import two modules.
Both define classes that have fields named "file" so... import error.
:( It's no fun.
I need to think about it more, but I think you're idea of explicitly
extending may give a solution. Thinking out loud:
When you import two unrelated methods with the same name and different
patterns, we could just import them both and locally combine them into
a single mega-multimethod. The fact that their patterns differ is
enough to make sure you get the right one when you invoke it. The
reason it doesn't do that right now is because of extension.
Let's say I do:
import a
import b
def foo()
...
end
Both a and b define foo. Which of those multimethods should be
extended with the new foo() I'm defining here? Because there isn't a
simple right answer here, the current semantics are just that you
can't locally merge multimethods like this.
This feels like you're punishing the common case, though. Most methods
aren't extensions of existing multimethods and this prevents you from
being able to import different types with the same field name, which
is a painfully common use case.
If the syntax required you to specifically indicate which method you
were extending, (or at least to specifically annotate that you *are*
extending a method, and then only disallow merged methods in that
case) then you should be able to safely import overlapping methods
without a problem. This would address what to me is currently the most
unpleasant part of the language.
I'll probably have to try it out to see how well it works, but it at
least sounds promising.
Thanks for thinking about this!
- bob
Dave Cleaver
preference is towards a looser set of multimethod semantics that would
only cause problems if you try to import two multimethods with the
same pattern. Of course this does cause some issues with the ability
to document a multimethod as a whole without any specializations.
Dave
Bob Nystrom
> I seem to recall bumping into the field name problem as well. My
> preference is towards a looser set of multimethod semantics that would
> only cause problems if you try to import two multimethods with the
> same pattern.
Mine too. The nasty corner case comes from overriding. Consider this:
// a.mag
def foo
def bar(arg)
foo(arg)
end
// b.mag
import a
def foo(is String)
print("string")
end
bar("s") // "string"
Here, when you define "foo" in b, you also want module a to be able to
see that definition when it calls it from bar. The way that works now
is that when you import a multimethod from another module, you import
the exact same multimethod object. When new methods are added to it
using "def", the original module will see them too since it shares
that multimethod object.
That works fine if the multimethod you're adding too only comes from
one module, but what if you import two multimethods with the same name
and then try to override? Which one would get it?
Right now, to avoid that ambiguity, the rule is you can't import two
different multimethods with the same name. But that rule is what
causes the problem with field names. Thanks to Joe's prodding, the
alternative I'm considering is:
There are two ways to define a method, using "def" or "extend" (for
now at least). The semantics are roughly:
When you do "def":
- We check for pattern collisions with other methods of the same name.
- The method is defined in the current scope.
- If public, it is exported from the module.
When you import a method:
- We check for pattern collisions with other methods of the same name.
- The method is defined in the current scope.
When you do "extend":
- Same as for "def" and also...
- For each method in the module with the same name:
- Add this this method to the module where that method was originally defined
as if it was defined using "extend" in that module.
That would, I think, fix the field name problem while still allowing
you to override methods (which is critical). It also requires you to
be explicit about when you're overriding a method (which others
modules can see) and when you are just defining a method for your use
(which they can't) which might be a good thing.
There may be some problems with this, but we won't know for sure until
we try it out. I have next to know free time right now, but when I get
a chance, I'll try to hack it together and see how it goes.
> Of course this does cause some issues with the ability
> to document a multimethod as a whole without any specializations.
You can do that now just using def without a pattern on the left or right:
def foo
/// I am documenting the method as a whole.
end
- bob