Adding lhs and name to array item descriptors of action adverb

[Ruslan Shvedov]

lhs for G1 rules and name for L0 so that to enable

:default ::= action => [lhs, values]

lexeme default = action => [name, value]

instead of

:default ::= action => [values] bless => ::lhs

lexeme default = action => [start,length,value] bless => ::name

My use case:

In https://metacpan.org/source/JKEGL/Marpa-R2-2.078000/t/sl_timeflies.t, I had to do this

my ($tag, $contents) = ( ref $_[0], $_[0] );
$tag =~ s/^PennTags:://;

to remove bless_package that was added only to have the lhs of G1 and L0 rules in the parse value.

With lhs and name in array item descriptors, I'd be able to do just

my ($tag, $contents) = @_;

and achieve the same.

Perhaps removing the need to handle bless_package option in favor of array descriptor can be more efficient.

If that's ok and possible, I'd start working on a patch.

[Jeffrey Kegler]

This is in response to the comment in the IRC group, right?

The bless adverb is confusing, but so is the action adverb, which is also not used for ASF's.  Of course a comment to that effect in the grammar might be sufficient to elminate that confusion.

By the way, (as you may have already noticed) this code is used as one of the displays in the POD, so it has to remain suitable for that.  And (of course) that makes the current confusion in it something of an issue.

Yes, your approach looks promising and I look forward to the patch.

-- jeffrey

--
You received this message because you are subscribed to the Google Groups "marpa parser" group.
To unsubscribe from this group and stop receiving emails from it, send an email to marpa-parser...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Ruslan Shvedov]

On Mon, Jan 13, 2014 at 6:58 PM, Jeffrey Kegler <jeffre...@jeffreykegler.com> wrote:

This is in response to the comment in the IRC group, right?

No, I'm afraid. :)

The bless adverb is confusing, but so is the action adverb, which is also not used for ASF's. 

Well, from sl_panda.t I thought that ASF's are created from evaluated parse after all actions are applied, otherwise PennTags package wouldn't be needed. Or am I missing something?

 

Of course a comment to that effect in the grammar might be sufficient to elminate that confusion.

By the way, (as you may have already noticed) this code is used as one of the displays in the POD, so it has to remain suitable for that.  And (of course) that makes the current confusion in it something of an issue.

I really need some more information here, sorry.

Yes, your approach looks promising and I look forward to the patch.

Ok, I'll give it a shot on the weekend ($work breaks in)

[Jeffrey Kegler]

No, the ASF's do not see any effect from the "bless" or "action" adverbs.  They're there so I can do a dual test -- one with the ASF's, and once using repeated calls to $slr->value().  I compare the two results to make sure they're the same, but they use a different mechanism.

I think you can see why I did this, but apparently it has very much confused issues.  LLamaRider on the IRC channel was also confused about this.

-- jeffrey

[Ruslan Shvedov]

On Mon, Jan 13, 2014 at 7:33 PM, Jeffrey Kegler <jeffre...@jeffreykegler.com> wrote:

No, the ASF's do not see any effect from the "bless" or "action" adverbs.  They're there so I can do a dual test -- one with the ASF's, and once using repeated calls to $slr->value().  I compare the two results to make sure they're the same, but they use a different mechanism.

Yes I see now, I was confused by rh_value() call and that value is produced by applying actions, hence the confusion. Comparison with $recce->value() with just looked like a separate test of how the grammar works and I didn't go deeper even if I was recalling that ASF's don't apply actions.

I think you can see why I did this, but apparently it has very much confused issues.  LLamaRider on the IRC channel was also confused about this.

Just didn't read it carefully enough, I think. :) 

Sure, comparing $asf->traverse( {}, \&full_traverser ) and $recce->value() would be more straightforward, but no problem in fact.

BTW, today I was reading the code for Marpa::R2::ASF::traverse() and was plainly fascinated by its design. Absolutely brilliant piece of code I've never seen before, whether in books or in the wild.

[Jeffrey Kegler]

Re: $asf->traverse().  Thanks for noticing.  Yes, I'm pretty sure this is new work, and publishable when I get time to write it up.

Re the bless/action adverb confusion: This is a real mistake on my part in the presentation.  Even careful readers like yourself look at the code to form their general idea of "what is this all about".  And as things stand, the action/bless adverbs cause a serious misdirection.  They create the framework in which you'll read the documentation, and a bad framework invites a misreading.

I think you can see why I did it.  Testing is important and a test which you can run two different ways and compare is really good for catching problems.  It was too good an opportunity to pass up.  But it would be nice to both keep the dual test and eliminate the misdirection of the reader.

-- jeffrey

[Ron Savage]

I have to say that if traverse() is called such that if the first argument is a per-traversal object, then the /first/ argument of the callbacks (i.e full_traverser() and pruning_traverser() ) should also be that object. The current code is unnecessarily confusing.

[Ron Savage]

On Tuesday, 14 January 2014 03:58:18 UTC+11, Jeffrey Kegler wrote:

This is in response to the comment in the IRC group, right?

Am I on the wrong group? I'm in irc.perl.org#marpa. I don't see any conversations. 

[Deyan Ginev]

The channel is on irc.freenode.net rather than irc.perl.org. See you there! :)

--

[Ron Savage]

Aaaaagggggghhhhhh. And I /did/ warn about splitting comm channels. Hoist by my own petard (more-or-less) :-).

[Jeffrey Kegler]

It is, in a sense, the first argument.  In the Perl style, the args unpack as

    my ($object, arg0, ...) = @_;

The glade is the base object, and the per-traversal object is arg0.

--jeffrey

On 01/13/2014 01:37 PM, Ron Savage wrote:

I have to say that if traverse() is called such that if the first argument is a per-traversal object, then the /first/ argument of the callbacks (i.e full_traverser() and pruning_traverser() ) should also be that object. The current code is unnecessarily confusing.

[Ron Savage]

Hmmmm.

[Ruslan Shvedov]

I think I need some good advice to make sure that I'm on the right track.

I started from cpan\lib\Marpa\R2\Value.pm and Value-overview.html and thought of adding op_push_lhs by analogy to op_push_values (MARPA_OP_PUSH_VALUES) command and getting lhs via marpa_rule_lhs after marpa_v_rule at MARPA_STEP_RULE.

Schematic as it is, is this the right way of adding an array descriptor?

On Mon, Jan 13, 2014 at 6:58 PM, Jeffrey Kegler <jeffre...@jeffreykegler.com> wrote:

[Jeffrey Kegler]

I'm not sure I have the context here.  MARPA_OP_PUSH_VALUES is from the undocumented (and unsupported) SLIF interface to my XS code.  MARPA_STEP_RULE and marpa_v_rule() are from the documented (and supported) Libmarpa interface.

That undocumented SLIF XS interface has the advantage of speeding up the SLIF.  It has the disadvantage of making my Perl code very hard to use as a template or starting point for your own.  This was a major step backward from the NAIF, whose Perl-to-XS interface was the THIF, 100% documented and supported.  The SLIF still relies on the 100% documented Libmarpa interface, but that interface has been pushed down inside the XS code, and the SLIF's Perl-to-XS interface is not documented or supported.

Since MARPA_OP_PUSH_VALUES and MARPA_STEP_RULE are from two different interfaces, one of which is not supported, I don't think you'll be able to mix the two.  I am not at all sure that I am addressing your question, or that I understood it, but I hope this clarifies things a bit.

-- jeffrey

[Ruslan Shvedov]

What happened was I started looking how the action adverb's array descriptors are handled and found that code in cpan\lib\Marpa\R2\Value.pm 

                    if (   $result_descriptor eq 'values'

                        or $result_descriptor eq 'value' )

                    {

                        if ( defined $lexeme_id ) {

                            push @push_ops, $op_push_values;

...

and then, above 

        state $op_push_values     = Marpa::R2::Thin::op('push_values');

and then, in R2.xs, 

static int

v_do_stack_ops (V_Wrapper * v_wrapper, SV ** stack_results)

{

...

        case MARPA_OP_PUSH_VALUES:

        case MARPA_OP_PUSH_SEQUENCE:

          {

            if (!values_av)

              {

                values_av = (AV *) sv_2mortal ((SV *) newAV ());

              }

            switch (step_type)

              {

              case MARPA_STEP_TOKEN:

                {

...

                }

                break;

              case MARPA_STEP_RULE:

                {

                  int stack_ix;

                  const int arg_n = marpa_v_arg_n (v);

...

So, I thought, that's the way array descriptors are currently handled in Marpa::R2. Am I missing something important?

[Ruslan Shvedov]

Now I see that I took the wrong path with R2.xs, sorry. 

I now see push_values is created via create_ops.pl; need to take a closer look, thanks and sorry for the mess.

On Tue, Jan 14, 2014 at 10:31 AM, Jeffrey Kegler <jeffre...@jeffreykegler.com> wrote:

[Jeffrey Kegler]

Yes, that's how I do array descriptors in the SLIF -- but I didn't really have the context.  Is the about how to add a "lhs" array descriptor item to the array descriptor actions?  If so, the answer is yes, that you're on the right track.

My apologies for being dense.  Nobody else that I know of has looked at the code quite so deeply (at least that has told me).  You're talking about patching the SLIF's internals, then?  I didn't really understand that.  But, yes, if that's what you're trying to do, so far so good.

That explains why you're so interesting in the C langauge warning flags. :-)

-- jeffrey

[Ruslan Shvedov]

On Tue, Jan 14, 2014 at 11:08 AM, Jeffrey Kegler <jeffre...@jeffreykegler.com> wrote:

Yes, that's how I do array descriptors in the SLIF -- but I didn't really have the context.  Is the about how to add a "lhs" array descriptor item to the array descriptor actions?  If so, the answer is yes, that you're on the right track.

I'm very glad to hear that. :) 

My apologies for being dense.  Nobody else that I know of has looked at the code quite so deeply (at least that has told me).  You're talking about patching the SLIF's internals, then?  I didn't really understand that.  But, yes, if that's what you're trying to do, so far so good.

Sure; with the right guidance I hope to do that.

 

That explains why you're so interesting in the C langauge warning flags. :-)

Oh, yes. :)).

[Jeffrey Kegler]

My confusion stems from not realizing that you were getting into the code as deeply as you are. -- jeffrey

[Ruslan Shvedov]

On Tue, Jan 14, 2014 at 11:15 AM, Jeffrey Kegler <jeffre...@jeffreykegler.com> wrote:

My confusion stems from not realizing that you were getting into the code as deeply as you are. -- jeffrey

No problem, I've been somewhat mystified and hardly able to explain myself clearly. So, it's all good I think.

[Jeffrey Kegler]

@rns: An additional frustration that you may experience is that I am heavily refactoring the Libmarpa internals at this point, so that when your patch comes, it may have to wait in order to keep my problems with merge issues down.

I haven't talked about what I'm doing much, because I'm still not 100% sure it will work.  But I think I've found a way to simplify the Marpa algorithm.  This simplification will, if what I hope is correct, preserve the theoretical time complexity results of Marpa.  In practice, it will speed it up a bit.  It will also make future enhancements to Marpa and the SLIF easier.  It will even make Marpa's theory paper simpler and shorter, which would be an excellent thing.  :-)  But it's a major rewrite of Marpa's innermost internals and is some of the trickiest and most difficult coding I've done in a while.  So it very much affects my priorities.

-- jeffrey

[Ruslan Shvedov]

On Tue, Jan 14, 2014 at 11:27 AM, Jeffrey Kegler <jeffre...@jeffreykegler.com> wrote:

@rns: An additional frustration that you may experience is that I am heavily refactoring the Libmarpa internals at this point, so that when your patch comes, it may have to wait in order to keep my problems with merge issues down.

No frustration at all, I have to note, just a feel of living at a cutting edge, so to say. :) 

Seriously, absolutely no worries in this dept. — I still need time to become comfortable with command creation and have some translation work to do, so it's absolutely ok. 

Some great things just take time. :)

 

I haven't talked about what I'm doing much, because I'm still not 100% sure it will work.  But I think I've found a way to simplify the Marpa algorithm.  This simplification will, if what I hope is correct, preserve the theoretical time complexity results of Marpa.  In practice, it will speed it up a bit.  It will also make future enhancements to Marpa and the SLIF easier.  It will even make Marpa's theory paper simpler and shorter, which would be an excellent thing.  :-)  But it's a major rewrite of Marpa's innermost internals and is some of the trickiest and most difficult coding I've done in a while.  So it very much affects my priorities.

I've been following your commits for quite some time so far and has had a feeling that you're on something really big, so wishing you every success is the least I can do. 

If I can help with something, I'd gladly will. The two or three coming days are going to be pretty busy, but I'll hopefully have some spare time at the end of the week.

 

-- jeffrey