Scala 2.10 macros hack: fast for loops that look and feel like real ones

[Olivier Chafik]

Hi all,

I've rolled my sleeves tonight and ported *the most useful* subset of
ScalaCL's loop optimizations to the new 2.10.0-M4 macros.

Try it while it's hot:

https://github.com/ochafik/optimized-loops-macros

Hope this gets reviewed, amended and... integrated to
scala.collection.immutable.Range :-D

Current syntax uses `start to_/until_ end [by step]` with a special
InlinableRange (implicitly convertible to a Range anyway). I'll
probably add filters in the coming days.

I must say the transition path to M4 was much smoother than I expected
(infinitely easier than M2 -> M3).
Congratulations to the Scala team for that great milestone!

Cheers
--
Olivier
http://ochafik.com
http://twitter.com/ochafik

[Eugene Burmako]

The concept looks good! Now we only need a pull request that incorporates this into trunk :)

Some comments:

1) I've seen you needed some casts to go between a macro impl and your macro helper. You can try the approach employed in reification macros to get rid of those casts [1, 2].

2) What's the justification for resetAllAttrs? If it's the necessity to remove all references to the lambda's parameter, then a transformer that does exactly that would be more robust. Sure, typically if you erase symbols and re-typecheck the tree, but sometimes it won't work: https://issues.scala-lang.org/browse/SI-5464 (for one, do nested inlined loops work? labeldefs used to be especially unfriendly to re-typechecking). Also, if the body of a foreach contains expansions of macros, which establish non-standard bindings by manually assigning symbols, then these bindings will be wiped out and won't be able to be restored.

3) (a minor one) Instead of definitions.IntClass.asType you can use IntTpe. Same for booleans and some other core classes [3].

4) Nice trick with transform. Saves quite a bit of typing.

[1] https://github.com/scalamacros/kepler/blob/691aa0981e701c73df01ad9d5c6dd8aafb289980/src/compiler/scala/reflect/reify/Taggers.scala#L7

[2] https://github.com/scalamacros/kepler/blob/691aa0981e701c73df01ad9d5c6dd8aafb289980/src/compiler/scala/tools/reflect/FastTrack.scala#L18

[3] https://github.com/scalamacros/kepler/blob/691aa0981e701c73df01ad9d5c6dd8aafb289980/src/library/scala/reflect/base/StandardDefinitions.scala#L1

[Olivier Chafik]

Hi Eugene,

Thanks for the fast feedback!

2012/7/4 Eugene Burmako <eugene....@epfl.ch>:

> The concept looks good! Now we only need a pull request that incorporates
> this into trunk :)

Hehe, that's probably for my next "free night" slot :-)

> Some comments:
> 1) I've seen you needed some casts to go between a macro impl and your macro
> helper. You can try the approach employed in reification macros to get rid
> of those casts [1, 2].

Nice hack! I've gone a bit too caky though, so it would need some
tweaks to work in my case (I refer to dependent singletons such as
matcher objects: not sure I can implicitly convert them, and cannot
really import them accross the existential types...). Also, I'm
concerned about introducing implicit conversions to and from
existential types everywhere (<shiver> reflection... compilation
time...)

> 2) What's the justification for resetAllAttrs? If it's the necessity to
> remove all references to the lambda's parameter, then a transformer that
> does exactly that would be more robust.

I pulled it out of my hat when all other hope was gone... My guess is
that it gives the typer a chance to set a symbol to my shiny new "i"
reference (replacement of lambda parameter references). I believe
typer stops when it finds a typed (/ symbolized?) subtree, and I
didn't want to set the symbol myself because I thought we weren't
supposed to tamper with them too much them in macros.

> Sure, typically if you erase symbols
> and re-typecheck the tree, but sometimes it won't work:
> https://issues.scala-lang.org/browse/SI-5464 (for one, do nested inlined
> loops work? labeldefs used to be especially unfriendly to re-typechecking).
> Also, if the body of a foreach contains expansions of macros, which
> establish non-standard bindings by manually assigning symbols, then these
> bindings will be wiped out and won't be able to be restored.

Oooh... so can we set create / symbols in macros ? (I just need to
create vals and vars, no fancy new methods or classes which I think I
read somewhere could not be created within macros)

> 3) (a minor one) Instead of definitions.IntClass.asType you can use IntTpe.
> Same for booleans and some other core classes [3].

Cool!

> 4) Nice trick with transform. Saves quite a bit of typing.

Every char matters ;-)

[Olivier Chafik]

2012/7/4 Eugene Burmako <eugene....@epfl.ch>:
> [...]

> (for one, do nested inlined
> loops work? labeldefs used to be especially unfriendly to re-typechecking).

Yes they do:

https://github.com/ochafik/optimized-loops-macros/blob/master/src/test/scala/scala/inlinable/InlinableRangeTest.scala#L113

Cheers
~Olivier

[Eugene Burmako]

2) This is not exactly supported, but I think you can try to create your own symbols. I believe I've also seen someone create a temporary ValDef tree, typecheck it with c.typecheck, discard the result, but remember the symbol of the result and then use it. If this works, let me know :)

[Olivier Chafik]

Hi Eugene,

2012/7/4 Eugene Burmako <eugene....@epfl.ch>:

> 2) This is not exactly supported, but I think you can try to create your own
> symbols. I believe I've also seen someone create a temporary ValDef tree,
> typecheck it with c.typecheck, discard the result, but remember the symbol
> of the result and then use it. If this works, let me know :)

Thank you, it did the trick!

https://github.com/ochafik/optimized-loops-macros/blob/master/src/main/scala/scala/inlinable/RangeLoops.scala#L32

The code should now be more robust (+ ironed a few typos out), even in
the presence of other misbehaving macros :-)

I've also updated the README with instructions on how to build and use
in a simple sbt project (+ renamed its artifact to "scala-inlinable"):

https://github.com/ochafik/optimized-loops-macros