Scalaxy/MacroExtensions: a DSL / compiler plugin to write macro-based DSLs (extension methods with no runtime dependency!)

[Olivier Chafik]

Hi all,

In the past days I've started playing with pre-typer compiler plugins, and came up with a simple syntax to define macro-based enrichments:

@extend(Any) def quoted(quote: String) = quote + self + quote

Gets expanded by the plugin into:

    import scala.language.experimental.macros

    implicit class scalaxy$extensions$quoted$1(self: Any) {

      def quoted(quote: String) = macro scalaxy$extensions$quoted$1.quoted

    }

    object scalaxy$extensions$quoted$1 {

      def quoted(c: scala.reflect.macros.Context)

                (quote: c.Expr[String]): c.Expr[String] = {

        import c.universe._

        val Apply(_, List(selfTree$1)) = c.prefix.tree

        val self = c.Expr[Any](selfTree$1)

        {

          reify(quote.splice + self.splice + quote.splice)

        }

      }

    }

Here's the post where I describe it, an where to find it on GitHub.

Any feedback / suggestions would be greatly appreciated!

Cheers

--
Olivier
http://ochafik.com
http://twitter.com/ochafik

[Eugene Burmako]

Oh cool! Now we have a use case for macro annotations :)

--
You received this message because you are subscribed to the Google Groups "scala-internals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-interna...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Eugene Burmako]

Allright, I became really curious about how you did annotation
resolution before namer. Apparently you didn't: if
annotationName.toString == "extend". What if I define an annotation
named @extend myself?

On Feb 20, 11:40 am, Eugene Burmako <eugene.burm...@epfl.ch> wrote:
> Oh cool! Now we have a use case for macro annotations :)
>

> On 20 February 2013 11:37, Olivier Chafik <olivier.cha...@gmail.com> wrote:
>
>
>
>
>
>
>
> > Hi all,
>
> > In the past days I've started playing with pre-typer compiler plugins, and
> > came up with a simple syntax to define macro-based enrichments:
>
> >  @extend(Any) def quoted(quote: String) = quote + self + quote
>
> > Gets expanded by the plugin into:
>
> >     import scala.language.experimental.macros
>
> >     implicit class scalaxy$extensions$quoted$1(self: Any) {
>
> >       def quoted(quote: String) = macro scalaxy$extensions$quoted$1.quoted
>
> >     }
>
> >     object scalaxy$extensions$quoted$1 {
>
> >       def quoted(c: scala.reflect.macros.Context)
>
> >                 (quote: c.Expr[String]): c.Expr[String] = {
>
> >         import c.universe._
>
> >         val Apply(_, List(selfTree$1)) = c.prefix.tree
>
> >         val self = c.Expr[Any](selfTree$1)
>
> >         {
>
> >           reify(quote.splice + self.splice + quote.splice)
>
> >         }
>
> >       }
>
> >     }
>

> > Here's the post where I describe it <http://ochafik.com/blog/?p=872>, an where
> > to find it on GitHub<https://github.com/ochafik/Scalaxy/tree/master/MacroExtensions/>
> > .

[Pascal Voitot Dev]

Cool idea but please do not transform Scala in Compile-time AOP ;)
Till now, Scala globally escaped from the annotation pollution...

[Olivier Chafik]

Absolutely, @extend is only resolved by name (see this comment).

Ideally, some basic 'namer' pass or a non-ambiguous annotation name should be used.

Also I didn't pay much attention to checking the def was in a module that's itself statically accessible (although the compiler will throw errors soon enough if that's not true).

--
Olivier
http://ochafik.com
http://twitter.com/ochafik

2013/2/20 Eugene Burmako <eugene....@epfl.ch>

[Olivier Chafik]

Hehe, I know this is kind of a Pandora box, but it's just meant to be an experiment.

If people really want to make their code unmaintainable (say, to keep their job  for life), I think it's important they have the freedom to use many such compiler plugins throughout their codebase ;-)

--
Olivier
http://ochafik.com
http://twitter.com/ochafik

2013/2/20 Pascal Voitot Dev <pascal.v...@gmail.com>

[Pascal Voitot Dev]

On Wed, Feb 20, 2013 at 11:56 AM, Olivier Chafik <olivier...@gmail.com> wrote:

Hehe, I know this is kind of a Pandora box, but it's just meant to be an experiment.

If people really want to make their code unmaintainable (say, to keep their job  for life), I think it's important they have the freedom to use many such compiler plugins throughout their codebase ;-)

New ideas and experiments are always welcome and I love macro concept (I wrote Play Json macros and Datomisca macros) because I find they are really useful in these cases.

When I see the monsters created with runtime bytecode enhancement in Java, I'm still expecting the first huge coding horror based on macros :D:D:D

[Olivier Chafik]

Update: I've changed the annotation from @extend to @scalaxy.extend, to minimize collisions.

Also, I've added partial support for type parameters (for extended type and method parameter types) and made the whole thing a bit more hygienic (and more tested ;-)):

@scalaxy.extend(Array[A]) def tup[A, B](b: B): (A, B) = (self.head, b)

https://github.com/ochafik/Scalaxy/tree/master/MacroExtensions

I've got some questions, though:

• Is there any way to pass implicits (such as Numeric[T] or ClassTag[T]) through reify?
  (seems to be SI-5747, although it dates back to M3)
• Is it possible / planned to have params with default values in methods implemented as macros?

Cheers

--
Olivier
http://ochafik.com
http://twitter.com/ochafik

[Alois Cochard]

On Friday, 22 February 2013 10:16:35 UTC, Olivier Chafik wrote:

Update: I've changed the annotation from @extend to @scalaxy.extend, to minimize collisions.

Also, I've added partial support for type parameters (for extended type and method parameter types) and made the whole thing a bit more hygienic (and more tested ;-)):

@scalaxy.extend(Array[A]) def tup[A, B](b: B): (A, B) = (self.head, b)

https://github.com/ochafik/Scalaxy/tree/master/MacroExtensions

I've got some questions, though:

• Is there any way to pass implicits (such as Numeric[T] or ClassTag[T]) through reify?
  (seems to be SI-5747, although it dates back to M3)

You can use Context.inferImplicit[Value/View] -> http://www.scala-lang.org/api/current/index.html#scala.reflect.macros.Context 

Which make you able to lookup for implicits in the caller context.

[Eugene Burmako]

1) Could you show an example of what you would like to achieve?

2) Currently that's not implemented: https://issues.scala-lang.org/browse/SI-5920. If this is critical for you, please ping me in a week (now I have a lot of things to handle), and I'll look into it.

[Olivier Chafik]

Hi Eugene, Alois,

Thanks for your replies :-)

I would just like to be able to define macro extensions such as the following:

  @scalaxy.extend(T) 

  def squared[T : Numeric]: T = self * self

  

  @scalaxy.extend(T) 

  def fill[T](length: Int): Array[T] = {

    val v = self

    Array.fill(length)(v)

  }

  

  @scalaxy.extend(Any) 

  def quoted(quote: String = "'"): String = 

    quote + self + quote

Regarding the Numeric[T] example, I managed to make it work with this:

  implicit class NumericExtensions2[T](self: T) {

    def squared2(implicit ev: Numeric[T]): T = macro squaredImpl2[T]

  }

  

  def squaredImpl2[T : c.WeakTypeTag](c: Context)(ev: c.Expr[Numeric[T]]): c.Expr[T] = {

    import c.universe._

    val Apply(_, List(selfTree)) = c.prefix.tree

    val self = c.Expr[T](selfTree)

    reify {

      implicit val n = ev.splice

      self.splice * self.splice

    }

  }

But using inferImplicitValue didn't seem to work (although the macro compiles fine):

  implicit class NumericExtensions[T : Numeric](self: T) {

    def squared: T = macro squaredImpl[T]

  }

  

  def squaredImpl[T : c.WeakTypeTag](c: Context): c.Expr[T] = {

    import c.universe._

    val Apply(_, List(selfTree)) = c.prefix.tree

    val self = c.Expr[T](selfTree)

    val ev = c.Expr[Numeric[T]](c.inferImplicitValue(weakTypeTag[Numeric[T]].tpe))

    reify {

      implicit val n = ev.splice

      self.splice * self.splice

    }

  }

(

  got a cryptic:

  error: type mismatch;

  [error]  found   : scala.math.Numeric.IntIsIntegral.type

  [error]  required: scala.math.Numeric[scala.math.Numeric.IntIsIntegral.type]

  and using typeOf[Numeric[T]] instead of weakTypeTag[Numeric[T]].tpe yields a:
  error: No TypeTag available for scala.math.Numeric[T]

)

Will definitely re-ping you about the default param thingie ;-)

Cheers

--
Olivier
http://ochafik.com
http://twitter.com/ochafik

2013/2/22 Eugene Burmako <eugene....@epfl.ch>

[Eugene Burmako]

Here's the error I'm witnessing when trying to reproduce your bug report: https://gist.github.com/xeno-by/5017950. That's kind of different from the error you get. Am I doing something wrong?

[Olivier Chafik]

Sorry, I forgot the following import in my previous snippet:

  import scala.math.Numeric.Implicits._

It will make that macro to compile, and any code like the following snippet will cause the aforementioned error:

  object Run extends App {

    import Macros._ 

    println(10.squared)

  }

Meanwhile, I've clarified the syntax and semantics of my experimental macro extensions: by-name parameters are now supported (and by-value ones are handled properly, i.e. evaluated only once), implicits are passed through transparently:

  @scalaxy.extension[Int] 

  def copiesOf[T : ClassTag](generator: => T): Array[T] = 

    Array.fill[T](self)(generator)

(which macro-expand `10 copiesOf new Entity` into `Array.fill(10)(new Entity)`)

Cheers

--
Olivier
http://ochafik.com
http://twitter.com/ochafik

2013/2/23 Eugene Burmako <eugene....@epfl.ch>

[Eugene Burmako]

First of all, I think c.inferImplicitValue won't work correctly here,
because its scope is the point of macro expansion, not the template of
NumericExtensions. I'd use Select(c.prefix, evidence$blah) instead.

Secondly, selfTree is equal to math.this.Numeric.IntIsIntegral. So
you're essentially multiplying two instances of type classes, also
deceiving the compiler about their actual type by wrapping them into
c.Expr[T], whereas they have type Numeric[T]. I think if you fix this,
the macro will work.

On Feb 23, 7:28 pm, Olivier Chafik <olivier.cha...@gmail.com> wrote:
> Sorry, I forgot the following import in my previous snippet:
>
>   import scala.math.Numeric.Implicits._
>
> It will make that macro to compile, and any code like the following snippet
> will cause the aforementioned error:
>
>   object Run extends App {
>     import Macros._
>     println(10.squared)
>   }
>
> Meanwhile, I've clarified the syntax and semantics of my experimental macro

> extensions <https://github.com/ochafik/Scalaxy/tree/master/MacroExtensions>:

> by-name parameters are now supported (and by-value ones are handled
> properly, i.e. evaluated only once), implicits are passed through
> transparently:
>
>   @scalaxy.extension[Int]
>   def copiesOf[T : ClassTag](generator: => T): Array[T] =
>     Array.fill[T](self)(generator)
>
> (which macro-expand `10 copiesOf new Entity` into `Array.fill(10)(new
> Entity)`)
>
> Cheers
> --
> Olivierhttp://ochafik.comhttp://twitter.com/ochafik
>

> 2013/2/23 Eugene Burmako <eugene.burm...@epfl.ch>

>
>
>
>
>
>
>
> > Here's the error I'm witnessing when trying to reproduce your bug report:
> >https://gist.github.com/xeno-by/5017950. That's kind of different from
> > the error you get. Am I doing something wrong?
>

> >> 2013/2/22 Eugene Burmako <eugene.burm...@epfl.ch>

>
> >>> 1) Could you show an example of what you would like to achieve?
> >>> 2) Currently that's not implemented:
> >>>https://issues.scala-lang.org/browse/SI-5920. If this is critical for
> >>> you, please ping me in a week (now I have a lot of things to handle), and
> >>> I'll look into it.
>

> >>> On 22 February 2013 11:16, Olivier Chafik <olivier.cha...@gmail.com>wrote:
>
> >>>> *Update*: I've changed the annotation from @extend to @scalaxy.extend,

> >>>> to minimize collisions.
>
> >>>> Also, I've added partial support for type parameters (for extended type
> >>>> and method parameter types) and made the whole thing a bit more hygienic
> >>>> (and more tested ;-)):
>
> >>>> @scalaxy.extend(Array[A]) def tup[A, B](b: B): (A, B) = (self.head, b)
>
> >>>>https://github.com/ochafik/Scalaxy/tree/master/MacroExtensions
>
> >>>> I've got some questions, though:
>

> >>>>    - Is there any way to pass implicits (such as Numeric[T] or

> >>>>    ClassTag[T]) through reify?

> >>>>    (seems to be SI-5747 <https://issues.scala-lang.org/browse/SI-5747>,

> >>>>    although it dates back to M3)

> >>>>    - Is it possible / planned to have params with default values in

> >>>>    methods implemented as macros?
>
> >>>> Cheers
> >>>> --
> >>>> Olivier
> >>>>http://ochafik.com
> >>>>http://twitter.com/ochafik
>

> >>>> 2013/2/20 Pascal Voitot Dev <pascal.voitot....@gmail.com>

>
> >>>>> On Wed, Feb 20, 2013 at 11:56 AM, Olivier Chafik <
> >>>>> olivier.cha...@gmail.com> wrote:
>
> >>>>>> Hehe, I know this is kind of a Pandora box, but it's just meant to be
> >>>>>> an experiment.
> >>>>>> If people really want to make their code unmaintainable (say, to keep

> >>>>>> their job  for life <http://thc.org/root/phun/unmaintain.html>), I

> >>>>>> think it's important they have the freedom to use many such compiler
> >>>>>> plugins throughout their codebase ;-)
>
> >>>>> New ideas and experiments are always welcome and I love macro concept
> >>>>> (I wrote Play Json macros and Datomisca macros) because I find they are
> >>>>> really useful in these cases.
>
> >>>>> When I see the monsters created with runtime bytecode enhancement in
> >>>>> Java, I'm still expecting the first huge coding horror based on macros
> >>>>> :D:D:D
>
> >>>>>> --
> >>>>>> Olivier
> >>>>>>http://ochafik.com
> >>>>>>http://twitter.com/ochafik
>

> >>>>>> 2013/2/20 Pascal Voitot Dev <pascal.voitot....@gmail.com>

>
> >>>>>>> Cool idea but please do not transform Scala in Compile-time AOP ;)
> >>>>>>> Till now, Scala globally escaped from the annotation pollution...
>
> >>>>>>> On Wed, Feb 20, 2013 at 11:40 AM, Eugene Burmako <
> >>>>>>> eugene.burm...@epfl.ch> wrote:
>
> >>>>>>>> Oh cool! Now we have a use case for macro annotations :)
>

> >>>>>>>> On 20 February 2013 11:37, Olivier Chafik <olivier.cha...@gmail.com

> >>>>>>>> > wrote:
>
> >>>>>>>>> Hi all,
>
> >>>>>>>>> In the past days I've started playing with pre-typer compiler
> >>>>>>>>> plugins, and came up with a simple syntax to define macro-based enrichments:
>
> >>>>>>>>>  @extend(Any) def quoted(quote: String) = quote + self + quote
>
> >>>>>>>>> Gets expanded by the plugin into:
>
> >>>>>>>>>     import scala.language.experimental.macros
>
> >>>>>>>>>     implicit class scalaxy$extensions$quoted$1(self: Any) {
>
> >>>>>>>>>       def quoted(quote: String) = macro scalaxy$extensions$quoted$1.quoted
>
> >>>>>>>>>     }
>
> >>>>>>>>>     object scalaxy$extensions$quoted$1 {
>
> >>>>>>>>>       def quoted(c: scala.reflect.macros.Context)
>
> >>>>>>>>>                 (quote: c.Expr[String]): c.Expr[String] = {
>
> >>>>>>>>>         import c.universe._
>
> >>>>>>>>>         val Apply(_, List(selfTree$1)) = c.prefix.tree
>
> >>>>>>>>>         val self = c.Expr[Any](selfTree$1)
>
> >>>>>>>>>         {
>
> >>>>>>>>>           reify(quote.splice + self.splice + quote.splice)
>
> >>>>>>>>>         }
>
> >>>>>>>>>       }
>
> >>>>>>>>>     }
>
> >>>>>>>>> Here's the post
>

> ...
>
> read more »