meta-programming in Dotty?

[Klaus Havelund]

I would like to perform meta programming on Scala, including:

  1. read the AST of a program - including type information

  2. produce some other output from this, think of it as static analysis, can also be for visualization etc.

  3. define macros 

The question is whether Dotty is useful for this or is it better to use something like ScalaMeta that is directly targeted towards

this problem.

PS: I am familiar with the fact that Dotty is a somewhat moving target so that is not a problem. I am more thinking

of longer term here, but I would need to get started right away.

Klaus

[Dmitry Petrashko]

Hello Klaus,

The question is whether Dotty is useful for this or is it better to use something like ScalaMeta that is directly targeted towards this problem.

I think the answer is "it depends". If you are doing something that is very close to syntax, such as what scalafmt is doing, ScalaMeta would be the best choice. For example, Dotty does not track token-level information and ScalaMeta has put substantial effort in doing it right.

If you are doing static analysis for optimizations\semantics\proofs, better place might be Dotty.
The main reason is that you can decide to run at a later phase, where the compiled files has been substantially simplified.
For example:

• if you do static analysis that is interprocedural, you may want to run after Mixin in Dotty, as otherwise you will have to re-implement most of it;
  
• you may want to run after pattern matching, as figuring out semantics of patterns is quite involved;
  
• Do you want to have complex types? if not, you may want to run after Erasure;
• Do you want functions to be closed? if yes, you want to run after Lambalift;
  
• Do you want to have inner classes? if not, you may want to run after Flatten.

In case you decide to re-implement patter-matching\mixin\flatten\lambdalift\etc in your analysis, you'll need to make sure they behave the same as the implementation in Dotty. It's actually worse, as you'll need to track changes in Dotty in case bugs are fixed or behavior is changed.

If you do transformations\optimizations, you may want to run in Dotty because compiler can create expressions that users can't write.

For example, it would be hard to make pattern matching implemented outside of Dotty efficient.

Best,

Dmitry

[Klaus Havelund]

Many thanks for your useful answer!

Is there documentation on how to interfere with the compiler for this kind of meta-programming work, or

is it a "read-the-code" kind of activity at this early stage?

Klaus

[Dmitry Petrashko]

Is there documentation on how to interfere with the compiler for this kind of meta-programming work, or

is it a "read-the-code" kind of activity at this early stage?

The documentation that we currently have is:

• the "Internals" section on http://dotty.epfl.ch/docs/index.html . I will not be surprised if the details are outdated(e.g. http://dotty.epfl.ch/docs/internals/core-data-structures.html says that stdlib does not compile we should get IDE & sbt support... we have all of this already!). Those docs are stored in our main github repo and improvements would be very welcome;
  
• a series of recordings about internals: https://www.youtube.com/watch?v=yYd-zuDd3S8&list=PLiSA_JXqRietNhqpQAikiRX7ocm9FyDnv
  
• and the source.

Note that Dotty currently does not have plugin infrastructure or public API that would be useful for you. You'll have to either fork dotty or create your own `main`. At the same time, the internal design of Dotty has been quite stable for last several years. E.g. the phases & trees remain unchanged for last 2.5 years.

Best,

Dmitry

On Tuesday, 8 August 2017 22:32:16 UTC+2, Klaus Havelund wrote:

Many thanks for your useful answer!

[Klaus Havelund]

Great!! Thanks.

Klaus

On Tuesday, August 8, 2017 at 12:59:16 PM UTC-7, Klaus Havelund wrote: