ASTs of complete modules/packages?

[Andreas Lobinger]

Hello colleagues,

i'm playing around with some ideas for testing and i searching for something like this:

* read-in (from the original file or via name) a complete module (or package)
* transform/parse to AST
* insert additional Expr/code in dedicated places (to count calls etc.)
* eval, so the module exists with the orginal name, but with more functionality

Any pointer to read?

Wishing a happy day,
      Andreas

[Cedric St-Jean]

I faced very similar issues with ClobberingReload.jl. https://github.com/cstjean/ClobberingReload.jl/blob/master/src/ClobberingReload.jl Check out parse_file (courtesy of @stevengj), parse_module, and creload. I haven't "expanded" the includes, but it seems straight-forward to do with a comprehension.

[Andreas Lobinger]

Hello colleague,

On Monday, September 26, 2016 at 12:50:44 AM UTC+2, Cedric St-Jean wrote:

I faced very similar issues with ClobberingReload.jl. https://github.com/cstjean/ClobberingReload.jl/blob/master/src/ClobberingReload.jl Check out parse_file (courtesy of @stevengj), parse_module, and creload. I haven't "expanded" the includes, but it seems straight-forward to do with a comprehension.

Thank you. I shortlisted the 'reload' mechanisms anyway, but ClobberingReload seems new. Actually i was wondering, if i'm the first one to look into code instrumentation. This should be really straight forward in julia (i did it once in f77 and that clearly not straight forward...).

[Cedric St-Jean]

It would make sense to put .jl file-parsing code in a separate, community-maintained module, because the rules for finding which file a module corresponds to are not trivial, and can change over time (they became case-sensitive in 0.5)

It's too bad that 

macro module_parser(m)

    m

end

@module_parser module X

a = 10

end

is a syntax error, because it would be cleaner for instrumentation, and for augmenting Julia's syntax.

Cédric

[Yichao Yu]

On Mon, Sep 26, 2016 at 8:31 AM, Cedric St-Jean <cedric...@gmail.com> wrote:
> It would make sense to put .jl file-parsing code in a separate,
> community-maintained module, because the rules for finding which file a
> module corresponds to are not trivial, and can change over time (they became
> case-sensitive in 0.5)
>
> It's too bad that
>
> macro module_parser(m)
> m
> end
>
> @module_parser module X
> a = 10
> end

julia> macro m(m)
Expr(:toplevel, m)
end
@m (macro with 1 method)

julia> @m module A
end
A

[Cedric St-Jean]

Cool, thank you.

[Jameson]

This also sounds similar to some of the issues I dealt with in some experimental code I wrote to simulate the result of precompiling modules. In `eval_expr`, I dealt with the ability to handle some expressions specially (such as `=`) and to recurse into `module` declarations. (The net goal of this code was to serialize the result of each statement so that deserialize could replay the same results, but faster).

https://github.com/vtjnash/Speed.jl/blob/master/src/Speed.jl#L104