What about adding extension methods to the language?

[Saxo]

Hello,

I know there is the pimp my library pattern with which extension methods can be defined. But this is kind of using a screw driver to hammer a nail into a wall. If there are extension methods, e.g. like in Kotlin (see this article), the overuse of implicit incurred by mimicking extension methods with its follow-up problems (code gets hard to read, you have to debug things to see what is happening) would go away. And implicit can fall back to fullfil the purpose it was initially conceived for.

Therefore I would propose to add extension methods to Scala. If done well as in Kotlin (very intuitive for the developer) it will not add clutter to the language.

Regards, Oliver

[Hanns Holger Rutz]

Scala 2.10's implicit classes provide a concise way of method extensions while being more powerful and generic than the mechanism in Kotlin AFAIK; e.g. they are properly scoped.

implicit class IntExt(val i: Int) extends AnyVal {
def squared = i * i
}

3.squared

[Rex Kerr]

Welllll, there is still an awful lot of boilerplate beyond

  extension def squared(i: Int) = i*i
  3.squared

and a huge amount of complexity in the compiler devoted to getting the implicit value class solution to not perform worse than the above.

The current system is adequate but not extraordinary IMO.

If value classes eventually can be passed unboxed, then I could see an argument that the overhead is worth it.  Right now, it's mostly a poor way to get extension methods.

  --Rex

[Hanns Holger Rutz]

Unless I'm missing something, they are already unboxed, that's the whole point of having value classes (which are orthogonal to implicit classes):

scala> class Test { 3.squared }
defined class Test

scala> :javap -v Test

{
public Test();
Code:
Stack=3, Locals=1, Args_size=1
0: aload_0
1: invokespecial #9; //Method java/lang/Object."<init>":()V
4: getstatic #15; //Field IntExt$.MODULE$:LIntExt$;
7: getstatic #20; //Field .MODULE$:L;
10: iconst_3
11: invokevirtual #24; //Method .IntExt:(I)I
14: invokevirtual #27; //Method IntExt$.squared$extension:(I)I
17: pop
18: return
LocalVariableTable:
Start Length Slot Name Signature
0 19 0 this LTest;

LineNumberTable:
line 8: 0

[Rex Kerr]

Er, I meant stored, not passed.  If you need to store a value class anywhere, a wrapper class is created (even for arrays).

  --Rex

[Eugene Burmako]

When we have macro annotations + if we have top-level functions [1],
then it'll be possible to write this as syntactic sugar for implicit
classes:

@extension def squared(i: Int) = i*i

[1] http://groups.google.com/group/scala-internals/browse_thread/thread/b9ba5dcadbb61452

On Dec 9, 2:17 pm, Rex Kerr <icho...@gmail.com> wrote:
> Er, I meant stored, not passed.  If you need to store a value class
> anywhere, a wrapper class is created (even for arrays).
>
>   --Rex
>

> > > On Sun, Dec 9, 2012 at 7:28 AM, Hanns Holger Rutz <cont...@sciss.de>

[Saxo]

It seems to me that the implicit search can can keep the compiler quite busy resulting in increased compile times. I'm not an expert in this field. It is just some observation. That would also be a reason to add extension methods to reduce the time the compiler spends doing implicit searches.

-- Oliver

[Daniel Sobral]

Em 09/12/2012 11:17, "Rex Kerr" <ich...@gmail.com> escreveu:
>
> Er, I meant stored, not passed.  If you need to store a value class anywhere, a wrapper class is created (even for arrays).

That doesn't make sense to me. An implicit value class used as extension method would never get stored. Why would you do that?

[Daniel Sobral]

Em 09/12/2012 10:21, "Saxo" <jet...@web.de> escreveu:
>
> Hello,
>
> I know there is the pimp my library pattern with which extension methods can be defined. But this is kind of using a screw driver to hammer a nail into a wall. If there are extension methods, e.g. like in Kotlin (see this article), the overuse of implicit incurred by mimicking extension methods with its follow-up problems (code gets hard to read, you have to debug things to see what is happening) would go away. And implicit can fall back to fullfil the purpose it was initially conceived for.

That *is* one of the purposes it was conceived for. Only it is a general mechanism with namespace and scope, as opposed to extension methods, and I'm betting the lack of these characteristics will be felt once extension methods become widely used.

[Saxo]

Am Sonntag, 9. Dezember 2012 16:14:23 UTC+1 schrieb Daniel Sobral:

Em 09/12/2012 10:21, "Saxo" <jet...@web.de> escreveu:
>
> Hello,
>
> I know there is the pimp my library pattern with which extension methods can be defined. But this is kind of using a screw driver to hammer a nail into a wall. If there are extension methods, e.g. like in Kotlin (see this article), the overuse of implicit incurred by mimicking extension methods with its follow-up problems (code gets hard to read, you have to debug things to see what is happening) would go away. And implicit can fall back to fullfil the purpose it was initially conceived for.

That *is* one of the purposes it was conceived for. Only it is a general mechanism with namespace and scope, as opposed to extension methods, and I'm betting the lack of these characteristics will be felt once extension methods become widely used.

The purpose of implicits looks to me to be some kind of type safe proxy. Extension methods are for something else. Here is an example from "Programming Scala":

val name: String = "scala"
println(name.capitalize.reverse)

This only works, because of this implicit:

implicit def stringWrapper(x: String) = new runtime.RichString(x)

You just add capitalize and reverse as extension method to String and you are done. There is nothing more to it and the implicit doesn't give you any additional bang here. Seems to me to be a workaround for missing extension methods only. Implicits are very powerful when you need some type safe proxy mechanism when implementating some generic library. But in the case above it only makes the compiler slow and complicates the language.

Question is whether those implicit classes added to Scala 2.10 (which I didn't know of) fixes the compiler performance problem (implicit search to make sure that for some specific implicit conversion only a single definition exists).
 

[Simon Ochsenreither]

I'd say that the current approach is superior to extension methods in pretty much every conceivable way. (I'd love to see a counter-example.)
Extension methods may look cute in made-up examples, but try to add the usual ~100 collection methods to Array or String. With extension methods, that will require 200 method implementations. Compare that with a few lines with implicits.

[Roland Kuhn]

Hi Oliver,

you might be interested in having a look at http://www.youtube.com/watch?v=hiurd7KaSEI&feature=plcp (starting ca. at 49:00), where Martin explains a few of the problems and how implicit value classes solve the problems without the drawbacks of extension methods.

Regards,

Roland

Dr. Roland Kuhn
Akka Tech Lead
Typesafe – The software stack for applications that scale.
twitter: @rolandkuhn

[Hanns Holger Rutz]

any other extension mechanism would have the same performance. you call .capitalize and the compiler needs to find the 'extension' that adds that method. naturally compile time resolution it will be slower the more extension points you add in separate units. what makes you think a syntactic sugar will speed things up?

[Saxo]

Am Sonntag, 9. Dezember 2012 17:27:16 UTC+1 schrieb Hanns Holger Rutz:

any other extension mechanism would have the same performance. you call .capitalize and the compiler needs to find the 'extension' that adds that method. naturally compile time resolution it will be slower the more extension points you add in separate units. what makes you think a syntactic sugar will speed things up?

No search for implicits if extension methods are used.

[Alec Zorab]

No. Then there's a search for the extension method instead...

[Saxo]

Okay, thanks!

[Daniel Sobral]

On Sun, Dec 9, 2012 at 2:16 PM, Saxo <jet...@web.de> wrote:

Am Sonntag, 9. Dezember 2012 16:14:23 UTC+1 schrieb Daniel Sobral:

Em 09/12/2012 10:21, "Saxo" <jet...@web.de> escreveu:

>
> Hello,
>
> I know there is the pimp my library pattern with which extension methods can be defined. But this is kind of using a screw driver to hammer a nail into a wall. If there are extension methods, e.g. like in Kotlin (see this article), the overuse of implicit incurred by mimicking extension methods with its follow-up problems (code gets hard to read, you have to debug things to see what is happening) would go away. And implicit can fall back to fullfil the purpose it was initially conceived for.

That *is* one of the purposes it was conceived for. Only it is a general mechanism with namespace and scope, as opposed to extension methods, and I'm betting the lack of these characteristics will be felt once extension methods become widely used.

 

The purpose of implicits looks to me to be some kind of type safe proxy. Extension methods are for something else. Here is an example from "Programming Scala":

I don't even know what that means. I suggest you look up "implicit" in computer science papers, as well as "view", and then look at when implicits were introduced in Scala, and what they replaced.

 

val name: String = "scala"
println(name.capitalize.reverse)

This only works, because of this implicit:

implicit def stringWrapper(x: String) = new runtime.RichString(x)

You just add capitalize and reverse as extension method to String and you are done. There is nothing more to it and the implicit doesn't give you any additional bang here. Seems to me to be a workaround for missing extension methods only. Implicits are very powerful when you need some type safe proxy mechanism when implementating some generic library. But in the case above it only makes the compiler slow and complicates the language.

Odersky is fond, recently, of pointing out you can't use extension methods to add an interface to a class. For example, in Scala String can be viewed as a Seq[Char] because of implicits.

That's one point, and I find it interesting you didn't even bother address the points I made. Specifically, you cannot control whether you want an extension method visible or not (scope), cannot choose *which* implementation of an extension method you want (namespace). Basically, extension methods are globals -- they are Java without packages. As long as no one is using them, everything is fine. When *every* library you use adds it's own extension methods to String, you won't find them so fine.

So let's give some additional examples:

In Specs, you can de-activate the implicits that add helper methods such as duration methods to Int.

In Scala, you can have both the method take that returns a String, and take that returns a Seq[Char]. Also, you don't need to write all these methods -- you can use the existing implementation of them all the way from GenTraversableOnce, shared by all scala collections.

And, you get all of that without introducing a language feature that only does one, limited, thing.

 

Question is whether those implicit classes added to Scala 2.10 (which I didn't know of) fixes the compiler performance problem (implicit search to make sure that for some specific implicit conversion only a single definition exists).
 

> Therefore I would propose to add extension methods to Scala. If done well as in Kotlin (very intuitive for the developer) it will not add clutter to the language.

>
> Regards, Oliver

--
Daniel C. Sobral

I travel to the future all the time.

[Rex Kerr]

On Sun, Dec 9, 2012 at 10:09 AM, Daniel Sobral <dcso...@gmail.com> wrote:

Em 09/12/2012 11:17, "Rex Kerr" <ich...@gmail.com> escreveu:

>
> Er, I meant stored, not passed.  If you need to store a value class anywhere, a wrapper class is created (even for arrays).

That doesn't make sense to me. An implicit value class used as extension method would never get stored. Why would you do that?

Because you want something that isn't just an extension method.  If you don't want anything more than an extension method, all that boilerplate is for nothing.

Right now it's an awkward compromise, though, because you'd be better off just passing the original.

I don't argue that extension methods can take the place of implicit conversions, just that they are way simpler yet do the same job as implicit value classes.  Implicit value classes as they stand now are great compared to what we had before, but nonetheless they have a large language complexity footprint (because of all the rules about what you can and can't do with them), a large compiler footprint, and a moderately large boilerplate footprint.

I expect some of these things to improve (maybe not enough to be worth the extra trouble over extension methods), but right now it's hard to understand the rationale (save for the rationale of building towards the future).

  --Rex

P.S. I still am glad we have *some* mechanism for this; I've just gone and switched a whole bunch of actual methods into "extension methods" via implicit value classes.  It really improves the use-site noticeably without losing performance.  But the library is kind of uglfied by it:

  // Before
  def nan(f: Float) = java.lang.Float.isNaN(f)

  // After
  implicit class RicherFloat(f: Float) extends AnyVal {
    def nan = java.lang.Float.isNaN(f)
  }

[Saxo]

I don't even know what that means. I suggest you look up "implicit" in computer science papers, as well as "view", and then look at when implicits were introduced in Scala, and what they replaced.

You may know the Binding class in Groovy. If a method is invoked from a subclass and cannot be dispatched, the invoke method in class Binding is called. You can redefine it in your subclass and then redirect the method send that could not be dispatched to some other object. This way you can implement a proxy.

With implicits you can do the same, but there is no runtime penalty caused by method lookup and dispatch at runtime and it's also type safe contrary to the solution in Groovy. In that way implicits can serve to implement type safe proxies.You would have understood if you had looked up "type safe" and "proxy" in the computer science papers.

[Simon Ochsenreither]

Because you want something that isn't just an extension method.  If you don't want anything more than an extension method, all that boilerplate is for nothing.

Right now it's an awkward compromise, though, because you'd be better off just passing the original.

I don't argue that extension methods can take the place of implicit conversions, just that they are way simpler yet do the same job as implicit value classes.  Implicit value classes as they stand now are great compared to what we had before, but nonetheless they have a large language complexity footprint (because of all the rules about what you can and can't do with them), a large compiler footprint, and a moderately large boilerplate footprint.

 

As soon as you need to define more than a single extension method, implicit classes have already paid of. I think that's pretty good.

Considering that some existing, important use-cases would require hundreds of extension methods per type and comparing that to the few lines of an implicit class inheriting the necessary methods (which currently solves the issue for good), I have to say that I don't see any value in extension methods.

I often disagree with Martin's decisions, but I think he (and the other people involved) really got thit thing right. After using implicit classes, extension methods look like a sad, amateurish joke.

[Rex Kerr]

On Sun, Dec 9, 2012 at 1:22 PM, Simon Ochsenreither <simon.och...@gmail.com> wrote:

Because you want something that isn't just an extension method.  If you don't want anything more than an extension method, all that boilerplate is for nothing.

Right now it's an awkward compromise, though, because you'd be better off just passing the original.

I don't argue that extension methods can take the place of implicit conversions, just that they are way simpler yet do the same job as implicit value classes.  Implicit value classes as they stand now are great compared to what we had before, but nonetheless they have a large language complexity footprint (because of all the rules about what you can and can't do with them), a large compiler footprint, and a moderately large boilerplate footprint.

 

As soon as you need to define more than a single extension method, implicit classes have already paid of. I think that's pretty good.

One could always

extension def (i: Int) {
  *(j: Vector[Int]) = j.map(_*i)
  +(j: Vector[Int]) = j.map(_+i)
  ...
}

as long-form syntax with

extension def (i: Int) * (j: Vector[Int]) = j.map(_*i)

as short form.  Don't imagine inefficient syntax and then complain that it's inefficient!

There is no regime in which adding "implicit class Whatever extends AnyVal" saves you typing.
 

Considering that some existing, important use-cases would require hundreds of extension methods per type and comparing that to the few lines of an implicit class inheriting the necessary methods (which currently solves the issue for good), I have to say that I don't see any value in extension methods.

This is true, to an extent.  But value classes can't inherit very easily--you _can_ inherit from traits that extend Any, but you then box when you call the trait's methods.  So while I completely agree about implicit classes _themselves_, I am only sold on value classes in the hope that they will continue to evolve (or as a much better fallback than not having extension methods at all).

I don't see any fundamental reason why implicit value classes can't evolve into something dramatically superior to extension methods, but we're not there yet.

  --Rex

[Paul Phillips]

On Sun, Dec 9, 2012 at 10:22 AM, Simon Ochsenreither <simon.och...@gmail.com> wrote:

After using implicit classes, extension methods look like a sad, amateurish joke.

Far be it for me to be the voice of moderation, but that's more than a bit overstated and unlikely to persuade anyone. I agree that the benefits of proper scoping are not yet adequately appreciated; let's not kill ourselves or anyone else getting the message out.

[Simon Ochsenreither]

Well, I really think that implicit classes, even in the current state, pull a lot of weight compared to the ad-hoc style of extension methods. They feel like they are properly integrated into the language and solve real-world issues compared to afterthoughts like classic extension methods, which seem to have their biggest strength as cute code examples on presentation slides.

I don't agree that often on various SIPs, but the points mentioned in the last Intel talk make a lot of sense and I haven't seen a design of the classic extension methods yet which comes close to solving the same issues.

[Rex Kerr]

On Sun, Dec 9, 2012 at 2:52 PM, Simon Ochsenreither <simon.och...@gmail.com> wrote:

Well, I really think that implicit classes, even in the current state, pull a lot of weight

I agree.  I was speaking only of value classes vs. extension methods.

Implicit conversions are great (with implicit classes as a special case of that).

  --Rex

[Daniel Sobral]

On Sun, Dec 9, 2012 at 3:55 PM, Rex Kerr <ich...@gmail.com> wrote:

On Sun, Dec 9, 2012 at 10:09 AM, Daniel Sobral <dcso...@gmail.com> wrote:

Em 09/12/2012 11:17, "Rex Kerr" <ich...@gmail.com> escreveu:

>
> Er, I meant stored, not passed.  If you need to store a value class anywhere, a wrapper class is created (even for arrays).

That doesn't make sense to me. An implicit value class used as extension method would never get stored. Why would you do that?

Because you want something that isn't just an extension method.  If you don't want anything more than an extension method, all that boilerplate is for nothing.

I get the suspicion you are arguing from incompatible two fronts here, but ok. I do agree with Martin, though, that boilerplate in language specification must also be taken into account.

 

Right now it's an awkward compromise, though, because you'd be better off just passing the original.

I don't argue that extension methods can take the place of implicit conversions, just that they are way simpler yet do the same job as implicit value classes.  Implicit value classes as they stand now are great compared to what we had before, but nonetheless they have a large language complexity footprint (because of all the rules about what you can and can't do with them), a large compiler footprint, and a moderately large boilerplate footprint.

I expect some of these things to improve (maybe not enough to be worth the extra trouble over extension methods), but right now it's hard to understand the rationale (save for the rationale of building towards the future).

  --Rex

P.S. I still am glad we have *some* mechanism for this; I've just gone and switched a whole bunch of actual methods into "extension methods" via implicit value classes.  It really improves the use-site noticeably without losing performance.  But the library is kind of uglfied by it:

  // Before
  def nan(f: Float) = java.lang.Float.isNaN(f)

  // After
  implicit class RicherFloat(f: Float) extends AnyVal {
    def nan = java.lang.Float.isNaN(f)
  }

A macro annotation would be helpful here, but it's difficult to drop the boilerplate and still be capable of doing things like Specs does, when it deactivates existing implicits to remove their methods from scope, or Scala does, when it provides two different conversions from String to IndexedSeq. Or being able to pass the conversion as a parameter to a function, like one does with Ordering throughout the libraries, making it possible to replace the behavior.

But that's just it. It's a general mechanism that does a lot of things, but being general makes it verbose. The trick is introducing general mechanisms to abstract repetition where such patterns appear, but I think the mechanism itself is superior to any alternative I'm aware of.

[Daniel Sobral]

On Sun, Dec 9, 2012 at 4:17 PM, Saxo <jet...@web.de> wrote:

I don't even know what that means. I suggest you look up "implicit" in computer science papers, as well as "view", and then look at when implicits were introduced in Scala, and what they replaced.

You may know the Binding class in Groovy. If a method is invoked from a subclass and cannot be dispatched, the invoke method in class Binding is called. You can redefine it in your subclass and then redirect the method send that could not be dispatched to some other object. This way you can implement a proxy.

That sounds a lot like how trait Dynamic works.

 

With implicits you can do the same, but there is no runtime penalty caused by method lookup and dispatch at runtime and it's also type safe contrary to the solution in Groovy. In that way implicits can serve to implement type safe proxies.You would have understood if you had looked up "type safe" and "proxy" in the computer science papers.

Look, I'm trying to help you understand why certain features in Scala exists, and how they fit in the grand scheme of things. I've told you something and, when you did not believe me, I told you how could you research your way to what I was saying -- or be in a position to refute what I was saying.

But if you neither believe what others who have much more experience in Scala than you say, nor avail yourself to the resources given, you'll have a hard time truly understanding Scala. And if you do not understand Scala, or Martin Odersky's goals for Scala, you don't stand much of a chance of changing anything at all about it, which is one the goals of this list, and the goal you seem to have with this whole thread.