[SIP 17] New SIP: type Dynamic

[martin odersky]

Here's a new SIP which describes the Dynamic feature we had as an
experimental feature since Scala 2.9. I made it into a SIP so that it
can be discussed and voted on formally for inclusion in 2.10. Since
the proposal reflects the consensus of the mailing list discussions
one year ago I do not expect it to be very controversial. But one can
always be surprised...

Cheers

-- Martin

[Eugene Burmako]

Here's the link to the SIP: https://docs.google.com/document/d/1XaNgZ06AR7bXJA9-jHrAiBVUwqReqG4-av6beoLaf3U/edit.

[Eugene Burmako]

How does Dynamic compare to dynamic in C#, namely:

1) What are the use cases it is supposed to enable?

2) Is it possible to be dynamic in argument position?

3) Is it possible to use "hardware acceleration" for dynamic from Java 1.7?

[Dave Griffith]

Technologically vapid.  Causes unnecessary breakage of existing code bases.  Not merely unlikely to fix the supposed issue, but actually likely to empower those who complain about the issue to amplify their complaining.   Patronising. Gutless.

We are all slightly stupider for having read this SIP.

[Daniel Spiewak]

I'm guessing this was intended for SIP-18?  :-)  Dynamic is many things, but "technologically vapid" is not one of them.

Daniel

[Josh Suereth]

Right. Dynamic is how we can play nicely with JRuby + Clojure and such.

[Daniel Sobral]

I'm surprised because I thought two new methods were added, for
getters and setters. They are gone, then? Not that I mind either way,
but... it does suprise me. :-)

Now, I do have one issue with it, that I did not bring up on the
document. This is a flexible tool, granted, but there's one particular
use case people want, which is the dynamic reference (like C#'s
dynamic). It can easily be implemented using Dynamic, and I wonder if
it cannot be further improved by extending AnyVal, which would avoid
the extra indirection. Let me give a brief example, just to make sure
we are all on the same page:

class DynamicRef(val underlying: Any) extends Dynamic {
def applyDynamic(m: String)(parms: Any*): Any = ??? // reflection
call on underlying
// def respondsTo? dynamic languages usually have a method like that
}

object DynamicRef {
implicit def apply(obj: Any): DynamicRef = new DynamicRef(obj)
}

def f(o: Any) {
val r: DynamicRef = o
r.doStuff()
}

So, it is simple to implement (at least I expect that to be the case
with Scala's own reflection library), but it is _not_ implemented by
default, and I think that's a mistake. This is a common enough use
case that it warrants inclusion in the standard library. Now, I'll
grant that I'm often asking for things in the standard library (though
there are things I think should be gone from there as well), but this
is one I don't particularly care for in my own code. I've seen plenty
evidence on mailing lists and Stack Overflow, however, that others do
want it.

--
Daniel C. Sobral

I travel to the future all the time.

[Eugene Burmako]

When resolving dynamic calls, C# injects the code that spawns a mini-C# compiler at runtime. This mini-compiler does overloading resolution, infers type arguments and so on. I'm not sure that this will be particularly easy to implement. However, with the new reflection and easy-to-use toolboxes (full-blown compilers at runtime), it's more or less doable, but there's another problem - scoped thingies.

Currently there's a way to reify code, but there's absolutely no way to reify its lexical scope (local names, owner classes, active imports). In C# they have a similar situation, but they are less affected (because only extension methods are not supported in dynamic situations, which is in itself annoying enough, since LINQ is based on extension methods). For us that would mean much more, since there won't be a way to make implicits work with DynamicRef.

All in all, I think that this SIP is very powerful for its minimalism (compare a single applyDynamic method with a dozen of BindXXX methods in System.Dynamic.DynamicMetaObject). However, I think, we should still address the most important use cases (one of them being type "dynamic" from C#), before moving forward.

[Eugene Burmako]

I've been looking into System.Dynamic in C# and found out several things that this SIP does not support. Let's go through them together (BindXXX methods are the one we're interested in): http://msdn.microsoft.com/en-us/library/system.dynamic.dynamicmetaobject.aspx.

Some background. BindXXX methods are mostly implemented by XXXBinders. For example, BindInvokeMember (method call) is serviced by InvokeMemberBinder. When compiling things that involve "dynamic" keyword, csc emits its own, internal, binders, named CSharpXXXBinder. A brief summary of the stuff going on can be found here: http://xeno-by.livejournal.com/37122.html.

1) First of all, methods. BindInvokeMember supports named arguments. CSharpInvokeMemberBinder (internal class emitted by C# compiler) also supports type arguments. I think, these two features should also be supported by applyDynamic (presumably by detecting applyDynamic calls in doTypedApply and augmenting it with type arguments and info about parameter names).

2) There are six more binders to represent various variations on methods. This category also includes BindBinaryOperation, BindUnaryOperation, BindGetMember, BindGetIndex, BindSetIndex, BindSetMember. These guys are mostly covered by applyDynamic as I understand it, but there's a minor nuisance. How do you distinguish between a field getter and a nullary method invocation from applyDynamic's PoV?

3) Casts. By the virtue of ConvertBinder C# virtualizes casts from dynamic to whatever. They need it to provide syntactic sugar to COM's QueryInterface, not sure whether we need that, but for sure this scenario is not supported by the SIP.

4) Applications. With InvokeBinder C# virtualizes direct applications of dynamic stuff (e.g. "dynamic x = blah; x(42)"). Again, not sure whether we need that (that's why I called for some use cases), but this is also something that's not currently supported.

5) Construction. Not sure under what circumstances CreateInstanceBinder can be used. This guy was always a mystery for me.

[Daniel Sobral]

On Sun, Mar 18, 2012 at 06:35, Eugene Burmako <eugene....@epfl.ch> wrote:

> 1) First of all, methods. BindInvokeMember supports named arguments.
> CSharpInvokeMemberBinder (internal class emitted by C# compiler) also
> supports type arguments. I think, these two features should also be
> supported by applyDynamic (presumably by detecting applyDynamic calls in
> doTypedApply and augmenting it with type arguments and info about parameter
> names).

Ah, yes. Named parameters. Another staple of today's dynamic languages.

> 2) There are six more binders to represent various variations on methods.
> This category also includes BindBinaryOperation, BindUnaryOperation,
> BindGetMember, BindGetIndex, BindSetIndex, BindSetMember. These guys are
> mostly covered by applyDynamic as I understand it, but there's a minor
> nuisance. How do you distinguish between a field getter and a nullary method
> invocation from applyDynamic's PoV?

I expected that was one reason why getters had their own methods, and
my surprising at finding out they don't (anymore?).

> 4) Applications. With InvokeBinder C# virtualizes direct applications of
> dynamic stuff (e.g. "dynamic x = blah; x(42)"). Again, not sure whether we
> need that (that's why I called for some use cases), but this is also
> something that's not currently supported.

I'm not sure I understand this. I'd expect Scala to translate that
into an "apply" call, which ought to suffice for any Scala or Java
object. Do you mean "blah" being a method invocation?

[martin odersky]

On Sun, Mar 18, 2012 at 10:35 AM, Eugene Burmako <eugene....@epfl.ch> wrote:
> I've been looking into System.Dynamic in C# and found out several things
> that this SIP does not support. Let's go through them together (BindXXX
> methods are the one we're interested in):
> http://msdn.microsoft.com/en-us/library/system.dynamic.dynamicmetaobject.aspx.
>
> Some background. BindXXX methods are mostly implemented by XXXBinders. For
> example, BindInvokeMember (method call) is serviced by InvokeMemberBinder.
> When compiling things that involve "dynamic" keyword, csc emits its own,
> internal, binders, named CSharpXXXBinder. A brief summary of the stuff going
> on can be found here: http://xeno-by.livejournal.com/37122.html.
>
> 1) First of all, methods. BindInvokeMember supports named arguments.
> CSharpInvokeMemberBinder (internal class emitted by C# compiler) also
> supports type arguments. I think, these two features should also be
> supported by applyDynamic (presumably by detecting applyDynamic calls in
> doTypedApply and augmenting it with type arguments and info about parameter
> names).
>

Let's say you have a call obj.meth("Bob", age = 13) where meth is not
defined in obj. Then that would be rewritten according to the SIP to

obj.applyDynamic("meth")("Bob", age = 13)

and then it would be up to applyDynamic to define the right-named
parameters. In all likelihood that would fail. What else can we do?
Package the parameter names somehow with the arguments? Then how to
pass them to the method? Postulate a
method

applyDynamicNamed

and rewrite to

obj.applyDynamicNamed("meth")(() -> "Bob", "age" -> 13)

It's a possibility. It does add to the complexity of the proposal but not much.

> 2) There are six more binders to represent various variations on methods.
> This category also includes BindBinaryOperation, BindUnaryOperation,
> BindGetMember, BindGetIndex, BindSetIndex, BindSetMember. These guys are
> mostly covered by applyDynamic as I understand it, but there's a minor
> nuisance. How do you distinguish between a field getter and a nullary method
> invocation from applyDynamic's PoV?

You don't. If you have a selection obj.meth you do not know whether
that's supposed to be a dynamic field selection or a dynamic method
call. So best best you can do is replace it with

obj.applyDynamic("meth")

I don't think we have a problem with setters, either. Pick:

obj.foo = 10

This expands first to

obj.foo_=(10)

and then to

obj.applyDynamic("foo_=")(10)

>
> 3) Casts. By the virtue of ConvertBinder C# virtualizes casts from dynamic
> to whatever. They need it to provide syntactic sugar to COM's
> QueryInterface, not sure whether we need that, but for sure this scenario is
> not supported by the SIP.
>

It is intentionally not supported because it opens up a can of worms.
Just to be clear: In terms of language surface area the C# proposal is
20 to 100 times more complicated than the Scala one.

Cheers

- Martin

[Daniel Sobral]

On Sun, Mar 18, 2012 at 14:40, martin odersky <martin....@epfl.ch> wrote:
>> 1) First of all, methods. BindInvokeMember supports named arguments.
>> CSharpInvokeMemberBinder (internal class emitted by C# compiler) also
>> supports type arguments. I think, these two features should also be
>> supported by applyDynamic (presumably by detecting applyDynamic calls in
>> doTypedApply and augmenting it with type arguments and info about parameter
>> names).
>>
> Let's say you have a call obj.meth("Bob", age = 13) where meth is not
> defined in obj. Then that would be rewritten according to the SIP to
>
>  obj.applyDynamic("meth")("Bob", age = 13)
>
> and then it would be up to applyDynamic to define the right-named
> parameters. In all likelihood that would fail. What else can we do?
> Package the parameter names somehow with the arguments? Then how to
> pass them to the method? Postulate a
> method
>
>  applyDynamicNamed
>
> and rewrite to
>
>  obj.applyDynamicNamed("meth")(() -> "Bob", "age" -> 13)
>
> It's a possibility. It does add to the complexity of the proposal but not much.

I assumed something like that. I actually thought of two parameter
lists, but tuples is way better. I'd just zip the parameter lists
anyway.

>
>> 2) There are six more binders to represent various variations on methods.
>> This category also includes BindBinaryOperation, BindUnaryOperation,
>> BindGetMember, BindGetIndex, BindSetIndex, BindSetMember. These guys are
>> mostly covered by applyDynamic as I understand it, but there's a minor
>> nuisance. How do you distinguish between a field getter and a nullary method
>> invocation from applyDynamic's PoV?
>
> You don't. If you have a selection obj.meth you do not know whether
> that's supposed to be a dynamic field selection or a dynamic method
> call. So best best you can do is replace it with
>
>  obj.applyDynamic("meth")

How can a non-macro implementationof applyDynamic distinguish between
these two calls?

object.meth
object.meth()

You have repeatedly emphasized those would be different methods, and
other languages *do* have a distinction between fields and methods,
which Dynamic would have to support somehow.

>
> I don't think we have a problem with setters, either. Pick:
>
>  obj.foo = 10
>
> This expands first to
>
>  obj.foo_=(10)
>
> and then to
>
>  obj.applyDynamic("foo_=")(10)

Yeah, that is trivial, but if getters get special treatment (for the
reason I explained above), then I'd like setters to get such special
treatment too. It would make it more elegant.

[Eugene Burmako]

>>I'm not sure I understand this. I'd expect Scala to translate that
>>into an "apply" call, which ought to suffice for any Scala or Java
>>object. Do you mean "blah" being a method invocation?

You're right, I overlooked this one. Apply will be enough.

[Eugene Burmako]

How about merging applyDynamic and applyDynamicNamed? Also, what about
type arguments?

On Mar 18, 6:40 pm, martin odersky <martin.oder...@epfl.ch> wrote:

> On Sun, Mar 18, 2012 at 10:35 AM, Eugene Burmako <eugene.burm...@epfl.ch> wrote:
> > I've been looking into System.Dynamic in C# and found out several things
> > that this SIP does not support. Let's go through them together (BindXXX
> > methods are the one we're interested in):

> >http://msdn.microsoft.com/en-us/library/system.dynamic.dynamicmetaobj....

[martin odersky]

On Sun, Mar 18, 2012 at 7:18 PM, Eugene Burmako <eugene....@epfl.ch> wrote:
> How about merging applyDynamic and applyDynamicNamed? Also, what about
> type arguments?
>

I'd erase type arguments because dynamic languages usually do not have
them. Not sure about merging applyDynamic and applyDynamicNamed. In a
lot of scenarios you might want to do without the names.

Cheers

- Martin

--
Martin Odersky
Prof., EPFL and Chairman, Typesafe
PSED, 1015 Lausanne, Switzerland
Tel. EPFL: +41 21 693 6863
Tel. Typesafe: +41 21 691 4967

[Eugene Burmako]

Well, to implement DynamicRef we need to reify the type arguments. Oh, good catch!! When writing this I actually realized that we now know how to reify types. Why not generate typetags for them?

Speaking of applyDynamicNamed, it's like a fly in the ointment. The original minimalistic concept is powerful enough to accomodate type arguments. It would be a crime against humanity to make it midimalistic just because of a petty corner case.

[Eugene Burmako]

In current proposal applyDynamic might have 1 or more parameter lists,
and the first parameter list must have 1 argument of type String (or
convertible).

I propose adding one more optional parameter to this parameter list,
that parameter would be of a new type, DynamicCallInfo. If the first
parameter list of applyDynamic contains 1 parameter, nothing is
changed in comparison with the current proposal. If there are 2
parameters, compiler generates an instance of DynamicCallInfo. We can
also make DynamicCallInfo an implicit value, then we won't hardcode
against parameter count.

DynamicCallInfo can contain names of the arguments, type tags that
correspond to type arguments, along with the number of pending
parameter lists (to support dynamic invocation of methods with
multiple parameter lists). With this augmentation in place, it becomes
possible to implement DynamicRef (except for macros).

On Mar 18, 10:07 pm, Eugene Burmako <eugene.burm...@epfl.ch> wrote:
> Well, to implement DynamicRef we need to reify the type arguments. Oh, good
> catch!! When writing this I actually realized that we now know how to reify
> types. Why not generate typetags for them?
>
> Speaking of applyDynamicNamed, it's like a fly in the ointment. The
> original minimalistic concept is powerful enough to accomodate type
> arguments. It would be a crime against humanity to make it midimalistic
> just because of a petty corner case.
>

> On 18 March 2012 21:47, martin odersky <martin.oder...@epfl.ch> wrote:
>
>
>
> > On Sun, Mar 18, 2012 at 7:18 PM, Eugene Burmako <eugene.burm...@epfl.ch>

[wang zaixiang]

the following code doesn't compile on Scala-2.10M3

package test

object TestDynamic {

  class MyDynamic extends Dynamic {
    def applyDynamic[T](n: String)(as: Any*) = { println("qual.applyDynamic(" + n + ")" + as.toList.mkString("(", ", ", ")")); }
    def applyDynamicNamed[T](n: String)(as: (String, Any)*) = { println("qual.applyDynamicNamed(" + n + ")" + as.toList.mkString("(", ", ", ")")); }
    def selectDynamic[T](n: String) = { println("qual.selectDynamic(" + n + ")"); }
    def updateDynamic(n: String)(x: Any): Unit = { println("qual.updateDynamic(" + n + ")(" + x + ")") }
  }

  def main(args: Array[String]) {
    val dynamic = new MyDynamic
    dynamic.name = "wangzx"   // this line does't pass compile, error: reassignment to val
    println(dynamic.name)

  }

  val dynamic = new MyDynamic
  dynamic.name = "wangzx"               // this line passwd compile
  println(dynamic.name)

}

Is there any problem on the code or other?

在 2012年3月18日星期日UTC+8上午3时32分15秒，Martin写道：

在 2012年3月18日星期日UTC+8上午3时32分15秒，Martin写道：

在 2012年3月18日星期日UTC+8上午3时32分15秒，Martin写道：

[Oleg Ilyenko]

I also had this problem. Here is Jira issue for it: https://issues.scala-lang.org/browse/SI-5726

On Tuesday, May 29, 2012 6:08:16 AM UTC+2, wang zaixiang wrote:
> the following code doesn&#39;t compile on Scala-2.10M3

>
> package test
>
> object TestDynamic {
>
>   class MyDynamic extends Dynamic {

>     def applyDynamic[T](n: String)(as: Any*) = { println(&quot;qual.applyDynamic(&quot; + n + &quot;)&quot; + as.toList.mkString(&quot;(&quot;, &quot;, &quot;, &quot;)&quot;)); }
>     def applyDynamicNamed[T](n: String)(as: (String, Any)*) = { println(&quot;qual.<WBR>applyDynamicNamed(&quot; + n + &quot;)&quot; + as.toList.mkString(&quot;(&quot;, &quot;, &quot;, &quot;)&quot;)); }
>     def selectDynamic[T](n: String) = { println(&quot;qual.selectDynamic(&quot; + n + &quot;)&quot;); }
>     def updateDynamic(n: String)(x: Any): Unit = { println(&quot;qual.updateDynamic(&quot; + n + &quot;)(&quot; + x + &quot;)&quot;) }

>   }
>
>   def main(args: Array[String]) {
>     val dynamic = new MyDynamic

>     <a href="http://dynamic.name" target="_blank">dynamic.name</a> = &quot;wangzx&quot;   // this line does&#39;t pass compile, error: reassignment to val
>     println(<a href="http://dynamic.name" target="_blank">dynamic.name</a>)

>
>   }
>
>   val dynamic = new MyDynamic

>   <a href="http://dynamic.name" target="_blank">dynamic.name</a> = &quot;wangzx&quot;               // this line passwd compile
>   println(<a href="http://dynamic.name" target="_blank">dynamic.name</a>)

>
> }
>
> Is there any problem on the code or other?
>

> 在 2012年3月18日星期日UTC+8上午3时32分15秒，<WBR>Martin写道：<blockquote class="gmail_quote" style="margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex">Here&#39;s a new SIP which describes the Dynamic feature we had as an

> experimental feature since Scala 2.9. I made it into a SIP so that it
> can be discussed and voted on formally for inclusion in 2.10. Since
> the proposal reflects the consensus of the mailing list discussions
> one year ago I do not expect it to be very controversial. But one can
> always be surprised...

> Cheers</p>
>  -- Martin
> </p>
> </p></blockquote>
> 在 2012年3月18日星期日UTC+8上午3时32分15秒，<WBR>Martin写道：<blockquote class="gmail_quote" style="margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex">Here&#39;s a new SIP which describes the Dynamic feature we had as an

> experimental feature since Scala 2.9. I made it into a SIP so that it
> can be discussed and voted on formally for inclusion in 2.10. Since
> the proposal reflects the consensus of the mailing list discussions
> one year ago I do not expect it to be very controversial. But one can
> always be surprised...

> Cheers</p>
>  -- Martin
> </p>
> </p></blockquote>
> 在 2012年3月18日星期日UTC+8上午3时32分15秒，<WBR>Martin写道：<blockquote class="gmail_quote" style="margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex">Here&#39;s a new SIP which describes the Dynamic feature we had as an

> experimental feature since Scala 2.9. I made it into a SIP so that it
> can be discussed and voted on formally for inclusion in 2.10. Since
> the proposal reflects the consensus of the mailing list discussions
> one year ago I do not expect it to be very controversial. But one can
> always be surprised...

> Cheers</p>
>  -- Martin
> </p>
> </p></blockquote>