Macros - tracking down a few things

[Paul Butcher]

Quite a few of the failures I'm getting when compiling ScalaMock3 under M7 are a result of things moving around within the API. I've managed to track quite a few of these down (e.g. isOverloaded has moved from Symbol to TermSymbol) but a few are still foxing me:

value flags is not a member of c.universe.Symbol

value typeParams is not a member of c.universe.Type

value resolveOverloaded is not a member of c.universe.TermSymbol

I'd be very grateful for pointers to where I should now find these :-)

Thanks!

--
paul.butcher->msgCount++

Snetterton, Castle Combe, Cadwell Park...
Who says I have a one track mind?

http://www.paulbutcher.com/
LinkedIn: http://www.linkedin.com/in/paulbutcher
MSN: pa...@paulbutcher.com
AIM: paulrabutcher
Skype: paulrabutcher

[Eugene Burmako]

1) resolveOverloaded is gone, because its implementation was duplicating existing compiler code, and after some discussion we discussed that its value doesn't overweight the technical debt we're imposing on ourselves. Quite likely it will reappear in 2.10.1.

2) typeParams is gone, it's supposed to be replaced by pattern matching.

3) flags are gone as well. Since we introduced a comprehensive set of isXXX methods, flags were deemed to be redundant in the API.

Please let me know if this breaks your use case.

[Paul Butcher]

On 21 Aug 2012, at 15:21, Eugene Burmako <eugene....@epfl.ch> wrote:

1) resolveOverloaded is gone, because its implementation was duplicating existing compiler code, and after some discussion we discussed that its value doesn't overweight the technical debt we're imposing on ourselves. Quite likely it will reappear in 2.10.1.

I'm currently using this to tell which instance of an overloaded method should be used to set expectations.

This is probably best illustrated by example. If the user is mocking a type of the form:

trait Foo { def m(x: Int): Unit; def m(x: String): Unit }

Then ScalaMock generates a mock which looks something like this:

val mock = new Foo {

  def m(x: Int) = mock$m$0(x)

  def m(x: String) = mock$m$1(x)

  val mock$m$0 = new MockFunction1[Int, Unit]

  val mock$m$1 = new Mock?Function1[String, Unit]

}

When setting expectations on one of these overloaded methods with, for example:

(mock.m(_: Int)).expects(42)

(mock.m(_: String)).expects("foo"))

I need to work out which instance of the method is in use to know whether the expectations should be set on mock$m$0 or mock$m$1.

Specifically, this is the method that makes use of resolveOverloaded:

https://github.com/paulbutcher/ScalaMock/blob/develop/core/src/main/scala/org/scalamock/Mock.scala#L332

I'm not sure how I would implement this (or something equivalent) without resolveOverloaded?

2) typeParams is gone, it's supposed to be replaced by pattern matching.

I'm currently using this when defining a new DefDef for the mocked version of a method. Here's the relevant method:

https://github.com/paulbutcher/ScalaMock/blob/develop/core/src/main/scala/org/scalamock/Mock.scala#L196

How I could replace this with pattern matching?

3) flags are gone as well. Since we introduced a comprehensive set of isXXX methods, flags were deemed to be redundant in the API.

The only time I use hasFlag is m.hasFlag(DEFERRED), but there doesn't appear to be an isDeferred method?

See our e-mail conversation on scala-user with the subject line "Macros - how do I tell if a method needs an "override" modifier?" for a discussion of why I need this.

Regarding flags, I'm using this in order to pass flags through to the mock method:

https://github.com/paulbutcher/ScalaMock/blob/develop/core/src/main/scala/org/scalamock/Mock.scala#L185

Without having access to the flags variable, I'm going to have to reconstruct it by calling each isXXX method in turn, I fear?

Please let me know if this breaks your use case.

See above :-)

--
paul.butcher->msgCount++

Snetterton, Castle Combe, Cadwell Park...
Who says I have a one track mind?

http://www.paulbutcher.com/
LinkedIn: http://www.linkedin.com/in/paulbutcher
MSN: pa...@paulbutcher.com
AIM: paulrabutcher
Skype: paulrabutcher

[Jason Zaugg]

You would copy/paste the deleted implementation [1] [2] into your project.

-jason

[1] https://github.com/scala/scala/blob/v2.10.0-M6/src/reflect/scala/reflect/api/Symbols.scala#L235
[2] https://github.com/scala/scala/blob/v2.10.0-M6/src/reflect/scala/reflect/internal/Symbols.scala#L87

[Paul Butcher]

On 21 Aug 2012, at 16:00, Jason Zaugg <jza...@gmail.com> wrote:

You would copy/paste the deleted implementation [1] [2] into your project.

-jason

[1] https://github.com/scala/scala/blob/v2.10.0-M6/src/reflect/scala/reflect/api/Symbols.scala#L235
[2] https://github.com/scala/scala/blob/v2.10.0-M6/src/reflect/scala/reflect/internal/Symbols.scala#L87

Are you confident that this would work? Given that significant portions of it are within reflect.internal (and therefore might be accessing things that aren't available in the public API)?

[Jason Zaugg]

On Tue, Aug 21, 2012 at 5:12 PM, Paul Butcher <pa...@paulbutcher.com> wrote:
> On 21 Aug 2012, at 16:00, Jason Zaugg <jza...@gmail.com> wrote:
>
> You would copy/paste the deleted implementation [1] [2] into your project.
>
> -jason
>
> [1]
> https://github.com/scala/scala/blob/v2.10.0-M6/src/reflect/scala/reflect/api/Symbols.scala#L235
> [2]
> https://github.com/scala/scala/blob/v2.10.0-M6/src/reflect/scala/reflect/internal/Symbols.scala#L87
>
>
> Are you confident that this would work? Given that significant portions of
> it are within reflect.internal (and therefore might be accessing things that
> aren't available in the public API)?

I think quite a lot is available in the public API, but there are a
few things missing, such as weak_<:<. (It's actually an interesting
exercise to see how far you can get with the public API.)

But nothing is stopping you, technically, from using the API in
reflect.internal; you just need a few creative casts. Here's how to
call weak_<:< this way:

https://gist.github.com/3416688

So you can copy the implementation as-is, and write a wrapper that
casts the parameters and results.

Obviously you run the risk that reflect.internal will change without
notice in a future release. But hopefully by that point,
resolveOverloaded will be provided for you anyway.

-jason

[Eugene Burmako]

1) Yep, just copy/paste. I've submitted a pull request that adds
weak_<:<: https://github.com/scala/scala/pull/1178, and the rest
should be okay. Stuff like isByNameParamType and same for varargs will
need to be reimplemented, but that's two oneliners that use public
API: definitions.ByNameParamClass and definitions.RepeatedParamClass/
definitions.JavaRepeatedParamClass.

2) If you have a MethodSymbol, you can use its newly introduced
typeParams method.

3a) Use TypeSymbol.isAbstractType instead of hasFlag DEFERRED.

3b) Yeah flags needs some love, that's true. For now you could cast to
reflect.internal.Symbols#Symbol and use `flags` from there. I've also
created an issue in tracker: https://issues.scala-lang.org/browse/SI-6267.

On Aug 21, 5:48 pm, Jason Zaugg <jza...@gmail.com> wrote:

> On Tue, Aug 21, 2012 at 5:12 PM, Paul Butcher <p...@paulbutcher.com> wrote:
> > On 21 Aug 2012, at 16:00, Jason Zaugg <jza...@gmail.com> wrote:
>
> > You would copy/paste the deleted implementation [1] [2] into your project.
>
> > -jason
>
> > [1]

> >https://github.com/scala/scala/blob/v2.10.0-M6/src/reflect/scala/refl...
> > [2]
> >https://github.com/scala/scala/blob/v2.10.0-M6/src/reflect/scala/refl...

[Paul Butcher]

Great - thanks.

I'm not sure that I'll have something compiling in time for Josh's 48 hour deadline though :-)

--
paul.butcher->msgCount++

Snetterton, Castle Combe, Cadwell Park...
Who says I have a one track mind?

http://www.paulbutcher.com/
LinkedIn: http://www.linkedin.com/in/paulbutcher
MSN: pa...@paulbutcher.com
AIM: paulrabutcher
Skype: paulrabutcher

[Eugene Burmako]

Sorry I broke so much stuff, though I believe it was for the best.

[Chris Hodapp]

One thing to note about that resolveOverloaded implementation: I never fixed https://issues.scala-lang.org/browse/SI-6066, since it was decided to remove it. When I attempt to get resolveOverloaded put back in for 2.10.1, it will be by moving a bit of the compiler into reflection, so it will not have this issue.

[Chris Hodapp]

2) typeParams is gone, it's supposed to be replaced by pattern matching.

I'm not so sure about the "opaque members" theory of API design... it feels... wrong to have extractors be the only way to get at core object properties like this... am I weird?

[Paul Butcher]

On 22 Aug 2012, at 01:07, Chris Hodapp <clho...@gmail.com> wrote:

2) typeParams is gone, it's supposed to be replaced by pattern matching.

I'm not so sure about the "opaque members" theory of API design... it feels... wrong to have extractors be the only way to get at core object properties like this... am I weird?

No - I share your unease.

[Eugene Burmako]

The problem with things like that is that they aren't core properties of api.Type. For example, to the best of my knowledge, typeParams only makes sense for PolyTypes (1 out of 15 publicly exposed types flavors).

As to extractors vs regular methods for core functionality, I also agree that it's quite incovenient to live without methods. As a result, since M3 we've added a bunch of accessors in api that mirror extractors in base.

[Eugene Burmako]

Also note that we now have MethodSymbol.typeParams that should cover the most common use case.

[Paul Butcher]

On 21 Aug 2012, at 18:06, Eugene Burmako <eugene....@epfl.ch> wrote:

Sorry I broke so much stuff, though I believe it was for the best.

Don't worry - I knew when I decided to start playing with macros that I would be chasing a work in progress.

I guess the moral of the story is that I should have been building daily against master rather than waiting for the next snapshot :-)

[Daniel Sobral]

On Tue, Aug 21, 2012 at 1:38 PM, Eugene Burmako <eugene....@epfl.ch> wrote:
> 1) Yep, just copy/paste. I've submitted a pull request that adds
> weak_<:<: https://github.com/scala/scala/pull/1178, and the rest
> should be okay. Stuff like isByNameParamType and same for varargs will
> need to be reimplemented, but that's two oneliners that use public
> API: definitions.ByNameParamClass and definitions.RepeatedParamClass/
> definitions.JavaRepeatedParamClass.
>
> 2) If you have a MethodSymbol, you can use its newly introduced
> typeParams method.

I actually tried this on my serialization code, but I could not get
"Int" out of "GenTraversable[Int]" through that path. Once it's a
symbol, it becomes "A", and typeSignatureIn doesn't restore its
Intness.

--
Daniel C. Sobral

I travel to the future all the time.

[Paul Phillips]

On Sun, Aug 26, 2012 at 3:53 AM, Daniel Sobral <dcso...@gmail.com> wrote:

> 2) If you have a MethodSymbol, you can use its newly introduced
> typeParams method.

I actually tried this on my serialization code, but I could not get
"Int" out of "GenTraversable[Int]" through that path. Once it's a
symbol, it becomes "A", and typeSignatureIn doesn't restore its
Intness.

It's not clear what you mean here - typeParams of a method symbol are method type parameters, which will remain as "A"s even in GenTraversable[Int], because they are independent.  Maybe a specific example of what you couldn't accomplish.

[Daniel Sobral]

Well, that's the point. With the method that has been removed
(typeArguments), and for which typeParams was put foward as an
alternative, I could get the Int out of GenTraversable[Int].

[Eugene Burmako]

Umm typeArguments and typeParams serve different purposes:
1) typeArguments for a TypeRef(_, _, args) returns args: List[Type]
2) typeParams for a PolyType(tparams, _) returns tparams: List[Symbol]

For example:
1) typeOf[List[Int]].typeArguments used to return List(typeOf[Int]).
Now you need to pattern match against TypeRef.
2) m.typeSignature.typeParams (for m: MethodSymbol standing for def
foo[T] = ???) used to return List(<symbol of T>). Now you can either
pattern match the typeSignature against PolyType or use m.typeParams.

On Aug 26, 4:51 pm, Daniel Sobral <dcsob...@gmail.com> wrote:
> On Sun, Aug 26, 2012 at 12:53 AM, Paul Phillips <pa...@improving.org> wrote:
>

[Paul Phillips]

On Sun, Aug 26, 2012 at 7:51 AM, Daniel Sobral <dcso...@gmail.com> wrote:

Well, that's the point. With the method that has been removed
(typeArguments), and for which typeParams was put foward as an
alternative, I could get the Int out of GenTraversable[Int].

typeParams cannot possibly be an alternative to typeArguments.  I don't know what logic lies behind the choices of what is and is not in the api and why it's there, but there isn't any scenario where type parameters are a replacement for type arguments.

Again, a specific example of what it is you can't do would clear it up a lot faster.  I don't know if these are the right way to get these, but for illustrative purposes:

scala> import scala.reflect.runtime.universe._

import scala.reflect.runtime.universe._

scala> import collection.GenTraversable

import collection.GenTraversable

scala> typeOf[GenTraversable[Int]]

res0: reflect.runtime.universe.Type = scala.collection.GenTraversable[Int]

scala> res0 match { case t: TypeRef => t.args }

res1: List[reflect.runtime.universe.Type] = List(Int)

scala> res0.typeSymbol match { case s: ClassSymbol => s.typeParams }

res2: List[reflect.runtime.universe.Symbol] = List(type A)

scala> 

[Daniel Sobral]

I'm saying the same thing you are saying, but from a position of much
more ignorance. Three problems were raised in this thread:

value flags is not a member of c.universe.Symbol

value typeParams is not a member of c.universe.Type

value resolveOverloaded is not a member of c.universe.TermSymbol

To which Eugene replied:

1) resolveOverloaded is gone, because its implementation was
duplicating existing compiler code, and after some discussion we
discussed that its value doesn't overweight the technical debt we're
imposing on ourselves. Quite likely it will reappear in 2.10.1.

2) typeParams is gone, it's supposed to be replaced by pattern matching.

3) flags are gone as well. Since we introduced a comprehensive set of
isXXX methods, flags were deemed to be redundant in the API.

So far so good, but then Eugene followed up with:

1) Yep, just copy/paste. I've submitted a pull request that adds
weak_<:<: https://github.com/scala/scala/pull/1178, and the rest
should be okay. Stuff like isByNameParamType and same for varargs will
need to be reimplemented, but that's two oneliners that use public
API: definitions.ByNameParamClass and definitions.RepeatedParamClass/
definitions.JavaRepeatedParamClass.

2) If you have a MethodSymbol, you can use its newly introduced
typeParams method.

3a) Use TypeSymbol.isAbstractType instead of hasFlag DEFERRED.

3b) Yeah flags needs some love, that's true. For now you could cast to
reflect.internal.Symbols#Symbol and use `flags` from there. I've also
created an issue in tracker: https://issues.scala-lang.org/browse/SI-6267.

So, there was I fixing a code that relied on typeArguments, and I
decided to follow up with the above suggestion, and discovered that it
didn't really work. So I reported here, in this thread, that it
doesn't really work, for those that might stumble upon it in the
future.

[Paul Phillips]

On Sun, Aug 26, 2012 at 5:01 PM, Daniel Sobral <dcso...@gmail.com> wrote:

So, there was I fixing a code that relied on typeArguments, and I
decided to follow up with the above suggestion, and discovered that it
didn't really work. So I reported here, in this thread, that it
doesn't really work, for those that might stumble upon it in the
future.

As nearly as I can tell, nobody has ever mentioned typeArguments except you.  The method which I see people talking about having been removed is typeParams.

[Daniel Sobral]

Oh... Ok, I'll shut up now.