Action or Scanless::G accessors for separator in quantified rules

[Thomas Weigert]

Dear all,

this may be a "newbie cannot find the documentation" issue, but I cannot see how I can get, when defining an action, to the separator defined for a quantified rule.

It would be most convenient, if there were an array descriptor action available that provides me the separator, so I can construct an appropriate semantic action.

That not being the case, I tried to work my way through the grammar starting from the rule that triggered the action. However, it does not seem that the separator is represented in the stuff returned by rule_expand().

So my suggestion is (unless I missed the right place in the docu) to provide a predefined action to yield the separator, if such was defined as part of the array descriptor actions.

Thoughts?

Along these lines it would be nice to be able to pass additional parameters to the actions by putting this into the DSL.

Th.

[Jeffrey Kegler]

Quantified rules are semantic sugar for BNF, and you can write BNF rules and have access to all the symbols in them, including those you think of as separators.  I say "think of", because if separators are given a semantics, they are (conceptually) more than just separators.  Sequences can be written as recursions -- internally, that is what Marpa::R2 actually does.  Use a left recursion, because that is easier on Marpa's internals.

Ideally, we would follow the TIMTOADI principle, so that for those who want to use quantified rules, with separators, but who also see their separators as having semantics, then they can do that too.  Maybe Kollos will get that fancy.

--
You received this message because you are subscribed to the Google Groups "marpa parser" group.
To unsubscribe from this group and stop receiving emails from it, send an email to marpa-parser...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Thomas Weigert]

Jeff,

thanks. I understand that I can write the list explicitly, but that means I also have to write the semantics to turn the spine into a list. Not difficult, but not as clear. My suggestion was just that since we have the DSL and this information is available also, why not provide this information.

I have another suggestion along these lines: When I write custom actions, I don't get all the nice information that I can send down into actions when I use the array descriptor actions. In particular, I don't get position information. As this information is available, it would be nice to have as parameters that are being handed to the custom actions whatever the user had defined as array descriptors for the default action. That way, we can even control whether and how values is passed to custom actions.

A related suggestion was mentioned below: Allow to mention symbols (of the grammar or just Perl names) that are passed as parameters to custom actions. Then I could have an action
   separator => COMMA action => insert_separator(COMMA)
which passes the separator down this way which would be another way to do above, albeit this gives additional flexibility.

Th.

[Jeffrey Kegler]

In custom actions you can access context variables: http://search.cpan.org/~jkegl/Marpa-R2-2.102000/pod/Semantics.pod#Action_context

Right now there are only a few of these.  Which others would you like?

[Jeffrey Kegler]

I actually did not understand what this syntax would be supposed to do.

[Thomas Weigert]

Why don't I write this up with more detailed examples... Th.

[Jeffrey Kegler]

I'm trying to minimize the new features added to Marpa::R2, but your write up will help with the planning for Kollos, in any case.

[Andreas Kupries]

On Thu, Mar 5, 2015 at 4:51 PM, Jeffrey Kegler <jeffre...@jeffreykegler.com> wrote:

Quantified rules are semantic sugar for BNF, and you can write BNF rules and have access to all the symbols in them, including those you think of as separators.  I say "think of", because if separators are given a semantics, they are (conceptually) more than just separators.  Sequences can be written as recursions -- internally, that is what Marpa::R2 actually does.

Is there any documentation around on how Marpa::R2 rewrites the sugar into basic BNF ?

Ok, I know how it can be done for the quantified rules, via left recursion and additional non-terminals. However for the priority rules I am still not fully clear how these would be done.

If there is no documentation, then is the rewrite engine in some central/singular place in the sources, or is its  activity distributed through it, and where would I have to look ?

[Jeffrey Kegler]

For the precedenced rules, you create new symbols for each precedence,
proceeding with associativity in the obvious way. Much of it is done
in this routine:
https://github.com/jeffreykegler/Marpa--R2/blob/master/cpan/lib/Marpa/R2/MetaAST.pm#L568

One thing to note is that it is *not* operator precedence -- the rule
have precedence, and the operators play no special role.

It's not dissimilar to rewrites you'll see elsewhere, but I might be
its inventor, if only because nobody previously had a parser that
could be expected to parse the rewritten rules.

[Andreas Kupries]

On Fri, Mar 6, 2015 at 11:33 AM, Jeffrey Kegler
<jeffre...@jeffreykegler.com> wrote:
> For the precedenced rules, you create new symbols for each precedence,
> proceeding with associativity in the obvious way. Much of it is done
> in this routine:
> https://github.com/jeffreykegler/Marpa--R2/blob/master/cpan/lib/Marpa/R2/MetaAST.pm#L568

Thank you.
It reminds me that I have not read perl code since Perl4.
I believe it will take me a while to decipher that.

> One thing to note is that it is *not* operator precedence -- the rule
> have precedence, and the operators play no special role.

So, in essence the rewrite generates an ambiguous grammar which the
engine has no problems with, parsing them all in parallel, right ?

And at the end the rule priorities are then used to collapse
ambiguity, by keeping only the highest priority at each node ?!

And I further guess that the only ambiguities which survive to the
parse forest are those which have the same highest priority at a node.

> It's not dissimilar to rewrites you'll see elsewhere, but I might be
> its inventor, if only because nobody previously had a parser that
> could be expected to parse the rewritten rules.
>
> On Fri, Mar 6, 2015 at 11:22 AM, Andreas Kupries
> <andreas...@gmail.com> wrote:
>>
>>
>> On Thu, Mar 5, 2015 at 4:51 PM, Jeffrey Kegler
>> <jeffre...@jeffreykegler.com> wrote:
>>>
>>> Quantified rules are semantic sugar for BNF, and you can write BNF rules
>>> and have access to all the symbols in them, including those you think of as
>>> separators. I say "think of", because if separators are given a semantics,
>>> they are (conceptually) more than just separators. Sequences can be written
>>> as recursions -- internally, that is what Marpa::R2 actually does.
>>
>>
>> Is there any documentation around on how Marpa::R2 rewrites the sugar into
>> basic BNF ?
>>
>> Ok, I know how it can be done for the quantified rules, via left recursion
>> and additional non-terminals. However for the priority rules I am still not
>> fully clear how these would be done.
>>
>> If there is no documentation, then is the rewrite engine in some
>> central/singular place in the sources, or is its activity distributed
>> through it, and where would I have to look ?

--
Andreas Kupries
Senior Tcl Developer
Code to Cloud: Smarter, Safer, Faster™
F: 778.786.1133
andr...@activestate.com, http://www.activestate.com
Learn about Stackato for Private PaaS: http://www.activestate.com/stackato

[Jeffrey Kegler]

No. Sorry if I was not clear.

The rewrite for precedenced rules removes the ambiguity of the pure
BNF in the SLIF DSL, and in this way is very much like the one you see
described elsewhere, particularly for dealing with expressions using
top-down parsers. You add a new symbol for the expression at each
level of precedence, and the rewritten BNF will be unambiguous.

[Andreas Kupries]

On Fri, Mar 6, 2015 at 12:08 PM, Jeffrey Kegler
<jeffre...@jeffreykegler.com> wrote:
> No. Sorry if I was not clear.

No problem.

>
> The rewrite for precedenced rules removes the ambiguity of the pure
> BNF in the SLIF DSL, and in this way is very much like the one you see
> described elsewhere, particularly for dealing with expressions using
> top-down parsers. You add a new symbol for the expression at each
> level of precedence, and the rewritten BNF will be unambiguous.

Ok. Guess I have to google then for bnf rewriting.
Thanks for your time.

[Jeffrey Kegler]

Andreas: Actually, it occurs to me that I can work up an example from
Marpa's internal trace mechanisms, showing the internal rules. I'm
away from my development environment now, but I'll do that in a bit.

On Fri, Mar 6, 2015 at 12:30 PM, Andreas Kupries

[Andreas Kupries]

That should really help with understanding this. Thanks.

On Fri, Mar 6, 2015 at 2:38 PM, Jeffrey Kegler

[Jeffrey Kegler]

Andreas: Your example is in this Github gist:
https://gist.github.com/jeffreykegler/1dd56ce50c1dd2395856

On Fri, Mar 6, 2015 at 2:44 PM, Andreas Kupries

[Andreas Kupries]

Thank you very much. Saved now.


On Fri, Mar 6, 2015 at 5:10 PM, Jeffrey Kegler