[erlang-questions] in-flight module transforms - request for comments

[Ulf Wiger]

From those of you who enjoy modifying existing code at run-time, I would like to ask for some input.

Inspired by Joe Norton's use of Meck to introduce support for asciiedoc syntax in EDoc [1], I started playing with Meck to do similar things with e.g. tweaking epp on the fly for alternative syntax support.

I found that Meck was a bit limited in this regard, and ultimately, it's a slight abuse of the tool anyway. Since I maintain the parse_trans application [2], I thought I'd continue my experimentation there (in the module-transforms branch).

First, I copied meck_mod.erl (into parse_trans_mod.erl) and made a slight addition to it: 

  parse_trans_mod:transform_module(Module, Transforms, Options) -> CompileAndLoadResult

which fetches the abstract code of a module, transforms it, then compiles and loads the result.

I then added two helper functions (replace_function/3 and export_function/3), for starters, to help make easy transforms. 

Here's an example [3]:

-module(test_transform_mod).

-export([ex1/0]).

-include("codegen.hrl").

ex1() ->

    parse_trans_mod:transform_module(

      ex1, [fun(Fs, _Os) ->

   parse_trans:export_function(int, 0, Fs)

   end,

   fun transform_ex1/2], [{pt_pp_src,true}]).

transform_ex1(Forms, _Opts) ->

    NewF = codegen:gen_function(add, fun(A, B) ->

    A - B

    end),

    parse_trans:replace_function(add, 2, NewF, Forms).

A shell dialogue to illustrate:

Eshell V5.8.4  (abort with ^G)

1> ex1:add(5,3).

8

2> test_transform_mod:ex1().

Pretty-printed in "./ex1.xfm"

ok

3> ex1:add(5,3).

2

This changes the example module ex1 [4], by exporting a previously internal function, int/0, and changing the semantics of the add/2 function.

The idea is hardly new. Yariv Sadan did something similar with smerl. I wanted to add a few functions that would draw on the stuff already supported by parse_trans.

For those of you out there who have used similar things before, what kind of support would you like to see?

BR,

Ulf W

[1] https://github.com/norton/asciiedoc/blob/master/src/asciiedoc_lib.erl#L39

[2] http://github.com/esl/parse_trans

[3] https://github.com/esl/parse_trans/blob/module-transforms/examples/test_transform_mod.erl

[4] In ex1, add(X, Y) -> X+Y.

     After transformation, add(A, B) -> A - B.

Ulf Wiger, CTO, Erlang Solutions, Ltd.

http://erlang-solutions.com

[Ulf Wiger]

On 18 Oct 2011, at 12:06, Ulf Wiger wrote:

> From those of you who enjoy modifying existing code at run-time, I would like to ask for some input.
>
> Inspired by Joe Norton's use of Meck to introduce support for asciiedoc syntax in EDoc [1], I started playing with Meck to do similar things with e.g. tweaking epp on the fly for alternative syntax support.

So, just to illustrate what tweaking epp could look like:

Say I want to abbreviate the Erlang fun … end syntax, using something shorter:

-module(toker_test).
-export([double/1, i2l/1]).

double(L) ->
lists:map(`(X) -> X*2`, L).

i2l(L) ->
lists:map(`integer_to_list/1, L).

A sample shell dialogue:

Eshell V5.8.4 (abort with ^G)

1> c(toker_test,[{outdir,"../ebin"}]).
./toker_test.erl:5: syntax error before: '`'
./toker_test.erl:8: syntax error before: '`'
./toker_test.erl:2: function double/1 undefined
./toker_test.erl:2: function i2l/1 undefined
error
2> c(toker_c,[{outdir,"../ebin"}]).
{ok,toker_c}
3> toker_c:c(toker_test,[{outdir,"../ebin"}]).
{ok,toker_test}
4> toker_test:double([1,2,3]).
[2,4,6]
5> toker_test:i2l([1,2,3]).
["1","2","3"]

The actual tweak is almost exactly like the example I showed earlier, but replacing epp:parse_erl_form/1 with a version that calls my own, modified version of erl_parse. The other thing needed is to export epp:epp_request/2.

Not that I'm recommending doing this for real! ;-)

BR,
Ulf W

Ulf Wiger, CTO, Erlang Solutions, Ltd.
http://erlang-solutions.com

_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[Tim Watson]

On 18 October 2011 11:06, Ulf Wiger <ulf....@erlang-solutions.com> wrote:

For those of you out there who have used similar things before, what kind of support would you like to see?

I'm interesting in writing up some simple code instrumentation tools so I can do stuff like declaratively inserting performance counters and/or logging statements into my code. I've already used parse_trans and love it, but doing this without forcing users into using a parse_transform sounds really good - a post compile step provided by a rebar plugin could transform the beam code afterwards, or as you say this can even be done at runtime.

One thing that would be useful would be some kind of qlc-esque selection mechanism for finding code that matches certain conditions. Whether these are match specifications or something else, they would make a very useful addition. Matching on type/spec information would be even better, although I suspect that's a little outside of the scope you're looking at right now.

Cheers,

Tim

[Tim Watson]

On 18 October 2011 12:12, Ulf Wiger <ulf....@erlang-solutions.com> wrote:

The actual tweak is almost exactly like the example I showed earlier, but replacing epp:parse_erl_form/1 with a version that calls my own, modified version of erl_parse. The other thing needed is to export epp:epp_request/2.

Not that I'm recommending doing this for real!  ;-)

Actually I think that making epp a bit more extensible would be a really superb thing. When will we be able to do quai-quotations eh!? :)