PEG Progress

[Graeme Defty]

Tony,

I am at the point where some of the builtins parse correctly (boolean, tuple, range, numeric, object)

I have not yet checked whether they are generating the right parse tree, but it is good enough to fool the rest of the compiler into accepting the output anyway.

However, when I get to module.re, despite completing the parse, I get the following error:

----------Parse finished : Result--------

[{module,8,'Module',
     [{function,9,
          {identifier,9,class},
          [],
          {var,1,'_'},
          [{module_name,9,'Module'}]},
      {function,11,call,
          [{var,11,fake_self},{atom,11,to_s},{var,11,args},{var,11,block}],
          {var,11,'_'},
          [{match,12,
               {tuple,12,[{atom,12,reia_module},{var,12,name}]},
               {var,12,fake_self}},
           {'case',13,
               {native_call,13,code,ensure_loaded,[{var,13,name}]},
               [{clause,14,
                    [{tuple,14,[{atom,14,module},{var,14,name}]}],
                    [{remote_call,15,{var,15,name},to_string,[],{nil,1}}]},
                {clause,16,
                    [{var,16,'_'}],
                    [{throw,17,'NameError',
                         {dstring,17,
                             [{string,17,"undefined module "},{var,17,name}]}},
                     {atom,18,error}]}]}]},
      {function,22,call,
          [{var,22,fake_self},{atom,22,inspect},{var,22,args},{var,22,block}],
          {var,22,'_'},
          [{remote_call,23,{var,23,fake_self},to_s,[],{nil,1}}]},
      {function,26,call,
          [{var,26,fake_self},{var,26,method},{var,26,args},{var,26,block}],
          {var,26,'_'},
          [{match,27,
               {tuple,27,[{atom,27,reia_module},{var,27,name}]},
               {var,27,fake_self}},
           {native_call,28,erlang,apply,
               [{var,28,name},
                {var,28,method},
                {cons,28,
                    {var,28,args},
                    {cons,28,{var,28,block},{empty,28}}}]}]}]}]

----------- End of Parse Result ---------

.:none: internal error in lint_module;
crash reason: {function_clause,
                  [{erl_internal,bif,
                       [{identifier,
                            [{line,{'src/builtins/module.re',9}}],
                            class},
                        2]},
                   {erl_lint,'-bif_clashes/2-lc$^0/1-0-',1},
                   {erl_lint,bif_clashes,2},
                   {erl_lint,forms,2},
                   {erl_lint,module,3},
                   {compile,lint_module,1},
                   {compile,'-internal_comp/4-anonymous-1-',2},
                   {compile,fold_comp,3}]}
{"init terminating in do_boot",{{badmatch,error},[{reia_bytecode,compile_submodule,3},{reia_bytecode,'-compile_submodules/3-lc$^0/1-0-',3},{reia_bytecode,compile_expressions,3},{reia_bytecode,compile,3},{reia_internal,compile,2},{init,start_it,1},{init,start_em,1}]}}

Crash dump was written to: erl_crash.dump
init terminating in do_boot ()
rake aborted!
Command failed with status (1): [bin/reiac -o ebin/module.reb src/builtins/...]

The complaint sems to be about the definition of the 'class' method on line 9, but submitting a much simplified version of this module to the reia parser ("module Module \n  def class; Module; end \n end".parse()) gives the following parse tree:

[(:module,1,:Module,[(:function,2,:class,[],(:var,1,:_),[(:module_name,2,:Module)])])]

which looks to me like the first few lines of my parse tree.

Any thoughts?

As I mentioned, I have not compared the output from the parse of the whole module with the standard compiler (an automated way to do that may be a good next tool to build) so I guess that would be my next step, but wondered if you had any insights (read 'shortcuts') ((*really* read 'ways of saving me work')).

Cheers,

g

[Tony Arcieri]

It looks like it's breaking on {identifier, 9, class}

My parser strips "identifier" nodes and instead simply includes the name.

An element(3, Identifier) should be all you need to extract the name from the identifier node. Substitute that in your parse tree instead and cross your fingers, everything should world.

--
You received this message because you are subscribed to the Google Groups "Reia" group.
To post to this group, send email to re...@googlegroups.com.
To unsubscribe from this group, send email to reia+uns...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/reia?hl=en.

--
Tony Arcieri
Medioh! Kudelski

[Graeme Defty]

Duh!

Well that's what I get for working too late at night. It was obviously just past stupid o'clock.

The next one I have hit is in json_parser.re. There is a method defined thus:

  def transform(term)
    case term.class()
    when List
      [transform(element) for element in term]
    when Tuple
      [(transform(key), transform(value)) for (key, value) in term[0]].to_dict()
    when Binary
      (:reia_string, term)
    when Atom
      throw("unexpected atom from JSON parser: #{term}") unless term == :null
      nil
    else term
    end
  end

Because of my more liberal view on whitespace, the line "when Binary" is joined with the following to produce a class instantiation, rather than the module reference + tuple actually intended.

I do not like it much, but I can see no other solution than to insist that the parameter list of a class instantiation begins on the same line as the class name (and that the tuple in a case like this does not) to avoid this ambiguity.

Any better ideas or other thoughts?

g
____________________________________________

[Tony Arcieri]

On Tue, Feb 1, 2011 at 1:19 PM, Graeme Defty <graeme...@gmail.com> wrote:

Because of my more liberal view on whitespace, the line "when Binary" is joined with the following to produce a class instantiation, rather than the module reference + tuple actually intended.

I do not like it much, but I can see no other solution than to insist that the parameter list of a class instantiation begins on the same line as the class name (and that the tuple in a case like this does not) to avoid this ambiguity.

Any better ideas or other thoughts?

That's how it works in Ruby... the expression is considered complete when the newline is encountered unless there's an open paren, at which point the newline is considered optional.

Python has "implicit line joining" for everything inside (...), [...], {...} delimiters. I haven't researched how Ruby handles this but I expect that sort of approach isn't going to work (particularly in a PEG).

Instead optional end-of-line tokens vs. mandatory end-of-line tokens must be flagged as such on a case-by-case basis. Unfortunately I failed to do this when implementing (and reimplementing) the LALR grammar, and worse, attempting to add it now would require me to completely reconstruct the grammar again (for the third time)

I've seriously considered doing this... but held off in hopes that we could just switch to the PEG.

[Graeme Defty]

OK - I have tried to allow line-ends where it is clear that a line cannot end (as per Ruby).  If anything I have probably been a bit liberal in this area, so we may need to tighten up a bit as instances arise (such as this one).

In the meantime, as I indicated, once the thing is working well enough to use, I will start to document the places where lines are forbidden, optional and mandatory.

g
_________________________________________

--

[Graeme Defty]

Ha ha ha

>>   ...I have tried to allow line-ends where it is clear that a line cannot end ...

Make that "... where it is clear that a *statement* cannot end..."  (e.g. after a binary operator)

More of that late-night stuff creeping in :-(

_______________________________________________________