Problems with Reflection 2.10M4

[Antoras]

I came in some trouble with the new Milestone 4:


First some imports:
--------------------------------------------------------------------------
import scala.reflect.runtime.{universe => u, currentMirror => m}
import scala.tools.reflect.ToolBox
--------------------------------------------------------------------------





In my opinion some types have an inconsistent implementation of
toString. Some types return only their short names. `scala.Product` for
example returns "Product" instead of "scala.Product".
--------------------------------------------------------------------------
scala> u.typeOf[scala.Product]
res31: reflect.runtime.universe.Type = Product // wrong?

scala> u.typeOf[java.lang.Object]
res32: reflect.runtime.universe.Type = Object // wrong?

scala> u.typeOf[scala.Function1[_,_]]
res33: reflect.runtime.universe.Type = scala.Function1[_, _] // correct?
--------------------------------------------------------------------------





ToolBox potentially has a bug in the implementation of `parseExpr`. A
call with a string containing a syntax error, produces a ToolBoxError. A
subsequent call with a string containing correct code results in the
same error:
--------------------------------------------------------------------------
scala> val tb = m.mkToolBox()
tb: scala.tools.reflect.ToolBox[reflect.runtime.universe.type] =
scala.tools.reflect.ToolBoxFactory$ToolBoxImpl@1f521f12

scala> tb.parseExpr("def x = {}") // works fine
res34: tb.u.Tree = def x = ()

scala> tb.parseExpr("def x = {")
scala.tools.reflect.ToolBoxError: reflective compilation has failed:

'}' expected but eof found.
<strack-trace>

scala> tb.parseExpr("def x = {}") // doesn't work any more
scala.tools.reflect.ToolBoxError: reflective compilation has failed:

'}' expected but eof found.
<strack-trace>
--------------------------------------------------------------------------





How can one get a reference to a self reference or better how to get
information that a class is based on a trait?
--------------------------------------------------------------------------
trait T {
def m: Int
}
class A {
self: T => // how to get reference to this? Is it possible?
def a = m*2
}
--------------------------------------------------------------------------





How can one call a method of an instance? The way I did it, an
AssertionError is thrown. What does it mean and how to avoid it?
--------------------------------------------------------------------------
scala> class X { def meth(i: Int) = i*2 }
defined class X

scala> val x = new X
x: X = X@4ba39183

scala> val insMirr = m.reflect(x)
insMirr: reflect.runtime.universe.InstanceMirror =
scala.reflect.runtime.JavaMirrors$JavaMirror$JavaInstanceMirror@155cd305

scala> val methName = u.newTermName("meth")
methName: reflect.runtime.universe.TermName = meth

scala> val methSym = insMirr.symbol.newMethodSymbol(methName)
methSym: reflect.runtime.universe.MethodSymbol = method meth

scala> val methMirr = insMirr.reflectMethod(methSym)
methMirr: reflect.runtime.universe.MethodMirror =
scala.reflect.runtime.JavaMirrors$JavaMirror$JavaMethodMirror@33220792

scala> methMirr(10)
java.lang.AssertionError: assertion failed: meth
at scala.Predef$.assert(Predef.scala:162)
<more stack-trace>
--------------------------------------------------------------------------





How to use implicit conversions with ToolBox.typeCheck?
--------------------------------------------------------------------------
scala> val tb = m.mkToolBox()
tb: scala.tools.reflect.ToolBox[reflect.runtime.universe.type] =
scala.tools.reflect.ToolBoxFactory$ToolBoxImpl@5849e989

scala> val expr = tb.parseExpr("1 to 3 map (_+1)")
expr: tb.u.Tree = 1.to(3).map(((x$1) => x$1.$plus(1)))

scala> tb.typeCheck(expr)
scala.tools.reflect.ToolBoxError: reflective typecheck has failed: value
to is not a member of Int
at
scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$ToolBoxGlobal$$anonfun$typeCheckExpr$1.apply(ToolBoxFactory.scala:145)
<more stack-trac>
--------------------------------------------------------------------------
I found ToolBox.inferImplicit{Value,View} but I didn't find a way to use
them.





Is there a built-in way to get an easier to read representation of the
FlagSets.FlagSet?
--------------------------------------------------------------------------
scala> case class X(i: Int)
defined class X

scala> u.typeOf[X]
res60: reflect.runtime.universe.Type = X

scala> res60.typeSymbol.flags
res62: reflect.runtime.universe.FlagSet = 2048
--------------------------------------------------------------------------
Currently there is only a Long returned. A Set[NameOfFlag] may be more
useful.


Ok, that's it for now.
Cheers, Antoras

[Eugene Burmako]

Hey Antoras,

Thanks for the very well presented feedback!

1) This happens because TypeRefs and ExistentialTypes are printed
differently, though, this is an inconsistency: https://issues.scala-lang.org/browse/SI-5941.

2) https://issues.scala-lang.org/browse/SI-5942

3) Use Template.self:
https://github.com/scalamacros/kepler/blob/c4bed54df50c339482a8f0e1fb1421c66f52090a/src/library/scala/reflect/base/Trees.scala#L464.

C:\Projects\Kepler\sandbox @ topic/showraw>scalac -Yreify-copypaste
Test.scala
// produced from source-C:\Projects\Kepler\sandbox
\Test.scala,line-4,offset=86
object Test extends App {
val $u: scala.reflect.runtime.`package`.universe.type =
scala.reflect.runtime.`package`.universe;
val $m: $u.Mirror =
scala.reflect.runtime.`package`.universe.runtimeMirror(this.getClass.getClassLoader);
import $u._, $m._
val tpe = staticClass("scala.Unit").asTypeSymbol.asTypeConstructor
println(tpe)
}
// produced from source-C:\Projects\Kepler\sandbox
\Test.scala,line-4,offset=86
object Test extends App {
val $u: scala.reflect.runtime.`package`.universe.type =
scala.reflect.runtime.`package`.universe;
val $m: $u.Mirror =
scala.reflect.runtime.`package`.universe.runtimeMirror(this.getClass.getClassLoader);
import $u._, $m._, Flag._
val tree = Block(List(ClassDef(Modifiers(TRAIT | ABSTRACT |
DEFAULTPARAM | INTERFACE, newTypeName(""), List()), newTypeName("T"),
List(), Template(List(build.Ident(staticClass("java.lang.Object"))),
build.emptyValDef, List(DefDef(Modifiers(DEFERRED, newTypeName(""),
List()), newTermName("m"), List(), List(),
build.Ident(staticClass("scala.Int")), EmptyTree)))),
ClassDef(Modifiers(NoFlags, newTypeName(""), List()),
newTypeName("A"), List(),
Template(List(build.Ident(staticClass("java.lang.Object"))),
ValDef(Modifiers(PRIVATE, newTypeName(""), List()),
newTermName("self"), Ident(newTypeName("T")), EmptyTree),
List(DefDef(Modifiers(NoFlags, newTypeName(""), List()),
newTermName("<init>"), List(), List(List()), TypeTree(),
Block(List(Apply(Select(Super(This(newTypeName("")), newTypeName("")),
newTermName("<init>")), List())), Literal(Constant(())))),
DefDef(Modifiers(NoFlags, newTypeName(""), List()), newTermName("a"),
List(), List(), TypeTree(),
Apply(Select(Select(This(newTypeName("A")), newTermName("m")),
newTermName("$times")), List(Literal(Constant(2))))))))),
Literal(Constant(())))
import scala.tools.reflect.ToolBox
println($m.mkToolBox().runExpr(tree))
}

4) Creating new symbols won't help here, you need to look up an
existing method symbol by using
`insMirror.symbol.typeSignature.declaration(methName).asMethodSymbol`.

5) Toolboxes don't auto-import standard packages. Until I fix
https://issues.scala-lang.org/browse/SI-5943, you'll need to
explicitly import them.

6) I have a pull request for that: https://github.com/scala/scala/pull/729.
In particular, there will be a show(FlagSet) defined in a runtime
universe:
https://github.com/scalamacros/kepler/blob/c4bed54df50c339482a8f0e1fb1421c66f52090a/src/reflect/scala/reflect/api/Printers.scala#L89.

Thanks again for your testing and feedback. That's very much
appreciated!

> scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$ToolBoxGlobal$$anonfun$typeC­heckExpr$1.apply(ToolBoxFactory.scala:145)

[Daniel Sobral]

On Mon, Jun 18, 2012 at 12:21 PM, Eugene Burmako <xen...@gmail.com> wrote:
> Hey Antoras,

>
> 4) Creating new symbols won't help here, you need to look up an
> existing method symbol by using
> `insMirror.symbol.typeSignature.declaration(methName).asMethodSymbol`.

I've seen that suggested before. Declaration will ignore inherited,
but not overridden, members. Just replace "declaration" with "member"
to correct for it.

Also, this works for me:

m.reflect(new X).reflectMethod(u.typeOf[X].member(u.newTermName("f")).asMethodSymbol).apply()

The main difference seems to be one goes through the class type, and
the other through the class mirror.

--
Daniel C. Sobral

I travel to the future all the time.

[Antoras]

Hi Eugene,

I see, a lot of work still need be done.

3) Template is a member of the AST. How can I get access to it with
reflection? The only thing I found is the extractor
universe.Template(parents, self, body). But how to get the instance to
extract?

[Antoras]

The type-path does not work if the corresponding class is a member of a def:

def x = {

class X { def meth(i: Int) = i*2 }

val insMirr = m.reflect(new X)
val mn = u.newTermName("meth")
val ms1 = insMirr.symbol.typeSignature.member(mn).asMethodSymbol // works
val ms2 = u.typeOf[X].member(mn).asMethodSymbol // does not work
insMirr.reflectMethod(ms2)(10)
}

A call of x results in:

java.lang.ClassNotFoundException: no Java class corresponding to class X
found
<stack-trace>

[Jason Zaugg]

On Mon, Jun 18, 2012 at 10:00 PM, Antoras <ma...@antoras.de> wrote:

> 3) Template is a member of the AST. How can I get access to it with
> reflection? The only thing I found is the extractor
> universe.Template(parents, self, body). But how to get the instance to
> extract?

Is this what you're after?

scala> val u = reflect.runtime.universe
u: scala.reflect.api.JavaUniverse = scala.reflect.runtime.JavaUniverse@74bf8935

scala> trait T { self: Ordered[T] => }
defined trait T

scala> u.typeOf[T].typeSymbol.asClassSymbol.selfType
res0: u.Type = T with scala.Ordered[T]

-jason

[Eugene Burmako]

The problem here arises because information about local classes
doesn't get persisted. Therefore reification (which implements typeOf)
reifies local classes using dummy symbols, which are meaningless
reflection-wise. Hence the crash.

insMirror-based variant works, because it treats everything that
doesn't have type information as plain Java classes, and this is quite
enough to look up simple symbols and invoke their methods.

I'm not sure we can do anything here, but current error message
definitely leaves much to be desired. I opened a bug for that:
https://issues.scala-lang.org/browse/SI-5944

On Jun 18, 10:03 pm, Antoras <m...@antoras.de> wrote:
> The type-path does not work if the corresponding class is a member of a def:
>
> def x = {
>    class X { def meth(i: Int) = i*2 }
>    val insMirr = m.reflect(new X)
>    val mn = u.newTermName("meth")
>    val ms1 = insMirr.symbol.typeSignature.member(mn).asMethodSymbol // works
>    val ms2 = u.typeOf[X].member(mn).asMethodSymbol // does not work
>    insMirr.reflectMethod(ms2)(10)
>
> }
>
> A call of x results in:
>
> java.lang.ClassNotFoundException: no Java class corresponding to class X
> found
> <stack-trace>
>
> On 06/18/2012 09:34 PM, Daniel Sobral wrote:
>
>
>
>
>
>
>
> > On Mon, Jun 18, 2012 at 12:21 PM, Eugene Burmako<xeno...@gmail.com>  wrote:
> >> Hey Antoras,
>
> >> 4) Creating new symbols won't help here, you need to look up an
> >> existing method symbol by using
> >> `insMirror.symbol.typeSignature.declaration(methName).asMethodSymbol`.
> > I've seen that suggested before. Declaration will ignore inherited,
> > but not overridden, members. Just replace "declaration" with "member"
> > to correct for it.
>
> > Also, this works for me:
>

> > m.reflect(new X).reflectMethod(u.typeOf[X].member(u.newTermName("f")).asMethodSymbol).app­ly()

[Antoras]

Yes, that looks fine. Thanks.

[Daniel Sobral]

On Mon, Jun 18, 2012 at 5:23 PM, Eugene Burmako <xen...@gmail.com> wrote:
> The problem here arises because information about local classes
> doesn't get persisted. Therefore reification (which implements typeOf)
> reifies local classes using dummy symbols, which are meaningless
> reflection-wise. Hence the crash.

Can you make more clear what exactly is not possible?

Are "local classes" any nested class, or is this specific of run-time
loaded classes, such as on REPL?

Does the problem happens solely when a symbol derived from a
typeOf-returned object is used with a mirror, or are there other
circumstances which might trigger it?

[Eugene Burmako]

By "local class" I meant a class that has an enclosing method. Inner and nested classes are fine as long as they are not defined inside a method (or a block inside a method). Whether a class is defined in REPL or in "normal" environment doesn't really matter here.

The problem with local classes is that Scala doesn't persist semantic information about them (probably for optimization). By persisting semantic information I mean serializing a signature along with referenced symbols and types into ScalaSignature annotations. This is also referred to as "pickling".

Unlike with reflection, which does the best thing possible with what it has (it tries to infer semantic information from Java signatures), reification bails from the very beginning. Whenever it sees a non-locatable symbol (by "non-locatable" I mean a symbol that doesn't get pickled, "local class" is a subset of "non-locatable"), it reifies it as a so-called "free type", a synthetic dummy symbol that remembers the original name and owner and has a surrogate type signature that closely follows the original.

Free type symbols look like type symbols and try to quack like type symbols, but they are actually not. Hence whenever you try to do something meaningful with a free type symbol, the result will be less than satisfactory. You can check whether a symbol is a free type by calling `sym.isFreeType`. You can also get a list of all free types referenced by a tree and its children by calling `tree.freeTypes`. Finally, you can get warnings when reification produces free types by using -Xlog-free-types.