Erlang R16A and pmods

[Evan Miller]

Erlang R16A (beta) is out:

http://www.erlang.org/news/41

Ericsson has release a parse transform implementing parameterized
modules. It can be found here:

https://github.com/erlang/pmod_transform

I tried using it but it didn't work out of the box, so I'll need to
dig in some more. If you want to play around with it yourself, just
add pmod_pt.erl to the boss_db src/ directory and change this line:

https://github.com/evanmiller/boss_db/blob/master/src/boss_compiler.erl#L26

To

ParseTransforms = [pmod_pt, boss_db_pt| ... ]

Hopefully we'll figure it out soon.

Besides that, lots of good stuff in the R16 release; check the README.
Kudos to the Erlang team!

Evan

[rambocoder]

Evan,

I am trying to get the pmod transform to work to help everybody out, and here is what I am noticing:

If I bypass the boss_record_compiler:compile("puppy.erl") and just do

boss_compiler:compile("puppy.erl") then the parse transform gets applied just fine, but your boss_record_compiler is where the magical ORM functions get added.

The strack trace for the exception points at the pmod_pt transform, as:

** exception error: no function clause matching pmod_pt:add_new_funcs([],
                                                                      [{function,0,new,1,
                                                                           [{clause,0,
                                                                                [{var,0,'Id'}],
                                                                                [],
                                                                                [{call,0,{atom,0,instance},[{var,0,'Id'}]}]}]},
                                                                       {function,0,instance,1,
                                                                           [{clause,0,
                                                                                [{var,0,'Id'}],
                                                                                [],
                                                                                [{tuple,0,[{atom,0,puppy},{var,0,'Id'}]}]}]}]) (src/pmod_pt.erl, line 109)
     in function  pmod_pt:add_new_funcs/2 (src/pmod_pt.erl, line 112)
     in call from pmod_pt:add_new_funcs/2 (src/pmod_pt.erl, line 112)
     in call from pmod_pt:pmod_expand/5 (src/pmod_pt.erl, line 100)
     in call from pmod_pt:transform/1 (src/pmod_pt.erl, line 79)
     in call from pmod_pt:parse_transform/2 (src/pmod_pt.erl, line 48)
     in call from lists:foldl/3 (lists.erl, line 1248)
     in call from boss_compiler:compile/2 (src/boss_compiler.erl, line 27)

So I am thinking that either the boss_record_compiler is spitting out some funky erlang parsed tree or the pmod_pt just has a bug in it.

I do notice that when just doing boss_compiler:compile("puppy.erl"), NewForms variable has {eof, 0} as its last tupple in the list, but that tupple does not exist when using the boss_record_compiler:compile("puppy.erl")

So I added {eof, 0} tupple to NewForms, to make lines 25-28 be:

            NewForms1 = NewForms ++ [{eof,0}],
            ParseTransforms = [pmod_pt, boss_db_pt|proplists:get_value(parse_transforms, Options, [])],
            RevertedForms = lists:foldl(fun(Mod, Acc) ->
                       Mod:parse_transform(Acc, CompilerOptions)
               end, erl_syntax:revert(NewForms1), ParseTransforms),

and now I get passed the pmod_pt parse transform, however the compilation blows up with:

{error,[{"puppy.erl",
         [{0,erl_lint,{unbound_var,'Id'}},
          {0,erl_lint,{unbound_var,'Id'}},
          {0,erl_lint,{unbound_var,'Id'}},
          {0,erl_lint,{unbound_var,'THIS'}},
          {0,erl_lint,{unbound_var,'THIS'}},
          {0,erl_lint,{unbound_var,'THIS'}},
          {0,erl_lint,{unbound_var,'THIS'}},
          {0,erl_lint,{unbound_var,'THIS'}}]}],
       []}

due to Line 47 and thats where I am stuck.
https://github.com/evanmiller/boss_db/blob/master/src/boss_compiler.erl#L47

I suspect it is due to the RevertedForms having such function definitions as:

{function,0,id,0,[{clause,0,[],[],[{var,0,'Id'}]}]}  <-- In this case, Id is not bound, so that mean that the pmod parse transform did not work?

Evan, naturally you know your compiler code better and maybe you know how to get it going.

-rambocoder

[Evan Miller]

Looks like you got exactly to the point where I gave up... I think
you're right that the pmod parse transform fails to transform those
variables for whatever reason. It could be some issue with the two
types of ASTs that sometimes can intermingle and sometimes not (one
has terms like {tree, function, ...} and the other just looks like
{function ...}). I'm not sure though.

Anyway thanks for taking a look at it. Let me know if you make any
more progress and I'll do likewise.

Evan

> --
> You received this message because you are subscribed to the Google Groups
> "ChicagoBoss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to chicagoboss...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>



--
Evan Miller
http://www.evanmiller.org/

[rambocoder]

Is there any way in erlang to fire up a debuger and step thru the code line by line and watch variables?

That would help a lot when debugging the boss_record_compiler.

Also, what if we take out {tree, from the preprocessors here https://github.com/evanmiller/boss_db/blob/master/src/boss_record_compiler.erl#L115

because I just dont see a tupple that starts with {tree here http://www.erlang.org/doc/man/erl_syntax_lib.html#analyze_forms-1

-rambocoder

[Kai Janson]

debugger:start(). From within the erl shell.

Sent from my tricorder

[Evan Miller]

On Tue, Feb 5, 2013 at 7:42 PM, rambocoder <erl...@rambocoder.com> wrote:
> Is there any way in erlang to fire up a debuger and step thru the code line
> by line and watch variables?
>
> That would help a lot when debugging the boss_record_compiler.
>
> Also, what if we take out {tree, from the preprocessors here
> https://github.com/evanmiller/boss_db/blob/master/src/boss_record_compiler.erl#L115
>
> because I just dont see a tupple that starts with {tree here
> http://www.erlang.org/doc/man/erl_syntax_lib.html#analyze_forms-1

These aren't the same data structures. {tree, ...} is returned from
the erl_syntax library (or the reverted form -- I forget which).

[rambocoder]

Kai, thank you for the debugger:start() it really made a difference :)

Evan,

What if we throw this in:

            NewNewForm = erl_syntax:revert_forms(erl_syntax:revert(NewForms)),

            ParseTransforms = [pmod_pt, boss_db_pt|proplists:get_value(parse_transforms, Options, [])],
            RevertedForms = lists:foldl(fun(Mod, Acc) ->
                       Mod:parse_transform(Acc, CompilerOptions)

               end, NewNewForm, ParseTransforms),

erl_syntax:revert_forms(erl_syntax:revert(NewForms)) converts code that looks like this:

[{attribute,1,file,{"puppy.erl",1}},
              {attribute,{1,2},module,{puppy,['Id']}},
              {tree,attribute,
               {attr,0,[],none},
               {attribute,
                {tree,atom,{attr,0,[],none},export},
                [{tree,list,
                  {attr,0,[],none},
                  {list,
                   [{tree,arity_qualifier,
                     {attr,0,[],none},
                     {arity_qualifier,
                      {tree,atom,{attr,0,[],none},attribute_names},
                      {tree,integer,{attr,0,[],none},0}}}],

Into:

[{attribute,1,file,{"puppy.erl",1}},
 {attribute,{1,2},module,{puppy,['Id']}},
 {attribute,0,export,[{attribute_names,0}]},
 {attribute,0,export,[{attribute_types,0}]},
 {attribute,0,export,[{database_columns,0}]},
 {attribute,0,export,[{database_table,0}]},
 {attribute,0,export,[{validate_types,0}]},
 {attribute,0,export,[{validate,0}]},
 {attribute,0,export,[{save,0}]},
 {attribute,0,export,[{set,1}]},
 {attribute,0,export,[{set,2}]},
 {attribute,0,export,[{attributes,0}]},
 {attribute,0,export,[{belongs_to_names,0}]},
 {attribute,0,export,[{belongs_to,0}]},
 {attribute,0,export,[{id,0}]},
 {attribute,0,export,[{get,1}]},
 {attribute,{2,2},export,[{blah,0}]},
 {function,{4,1},blah,0,[{clause,{4,1},[],[],[{atom,{4,11},ok}]}]},
 {function,0,attribute_names,0,
     [{clause,0,[],[],[{cons,0,{atom,0,id},{nil,0}}]}]},
 {function,0,attribute_types,0,[{clause,0,[],[],[{nil,0}]}]},
 {function,0,database_columns,0,
     [{clause,0,[],[],
          [{cons,0,{tuple,0,[{atom,0,id},{string,0,"id"}]},{nil,0}}]}]},
 {function,0,database_table,0,[{clause,0,[],[],[{string,0,"puppies"}]}]},
 {function,0,validate_types,0,
     [{clause,0,[],[],

and that is what the parse transform from OTP expects?

Because when I used http://www.erlang.org/doc/man/erl_syntax.html#is_tree-1 on the result of just erl_syntax:revert(NewForms) which is what is in boss_db right now, it returned 'false' which means that it is in "old style" parse transform tree, and the pmod_pt expects "new" style? That's what I gather from erlang docs and eye balling this line https://github.com/erlang/pmod_transform/blob/master/src/pmod_pt.erl#L186

Still tinkering with it...

-rambocoder

[Evan Miller]

On Tue, Feb 5, 2013 at 10:26 PM, rambocoder <erl...@rambocoder.com> wrote:
> Kai, thank you for the debugger:start() it really made a difference :)
>
> Evan,
>
> What if we throw this in:

Sounds fine to me. Let me know if it works...

[rambocoder]

Evan and group,

I think I got it to work using the erl_syntax:revert_forms approach

I pushed the code to https://github.com/rambocoder/boss_db/tree/pmods

Can you comment on my commits and if it's fine, I will send a pull request

-rambocoder

[Evan Miller]

I try to avoid "if" statements and would probably write

{NewNewForms, BossDBParseTransforms} = case Version of
Version when Version >= 16 -> ....
_ -> ....
end,
ParseTransforms = BossDBParseTransforms ++ ...,
RevertedForms = ...,

Other than that, I'm very excited to get this integrated!

Evan

[rambocoder]

Thank you for the tip, I changed the code to use case and sent the pull request.

boss_db_test_app.erl needs boss_test.erl to run, what do you think is the best way to not have boss_test.erl in both BossDB and ChicagoBoss?

Btw, even with my changes, I cannot get "make test_db_mock" to run, it still just hangs there.

But with couple fixes included in the pull request you can run the mock adapter tests if you copy boss_test.erl to boss_db/src , compile everything and just go into the shell with:

erl -pa ebin -pa deps/*/ebin

and when inside the erlang shell:

boss_db_test_app:start(a,b).

I am just not sure how to update the Makefile to have it run the different adapter tests.

-rambocoder