Would anyone miss table file output in PLY?

[David Beazley]

I've been thinking about some cleanup to PLY lately and have a question. Would anyone actually miss the table output in PLY (parsetab.py, lextab.py) if it were removed entirely? Here's some context:

The original version of PLY was developed on a 200 MHZ home computer about 12 years ago. At that time, generation of the LALR tables was relatively expensive so creating table files was a way of caching the result and reusing it (much like Python uses .pyc files). Today, it's a bit different. For instance, making the LALR tables for the ANSI C grammar (over 200 rules and nearly 350 states) on a modern machine only takes around a half a second. Thus, I'm honestly wondering if I could just ditch all of that table reading/writing code and not worry about it.

Does anyone have any particular thoughts about this?

Cheers,
Dave

[Erez Sh]

Half a second is quite a while, imho. ply can be useful for many types
of software, not just compilers, and a delay in the loading time can
really affect their usability.
As an alternative, perhaps you could make the table object pickle-able
(and accessible), so that the caller can implement caching himself if
he finds it important. As a bonus, it could be stored elsewhere, like
in a database. Just an idea.

[Bart Whiteley]

I agree with Erez. I would like to see this optimization remain in some form.

> --
> You received this message because you are subscribed to the Google Groups "ply-hack" group.
> To post to this group, send email to ply-...@googlegroups.com.
> To unsubscribe from this group, send email to ply-hack+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/ply-hack?hl=en.
>

[David Beazley]

I agree that focusing on pickling might be a sensible way to go about it. Specifically, make it so that the instance returned by lex() and yacc() can be pickled as desired by the user.

I'm not planning any changes along these lines for Ply-3.5, but will consider it in future releases later.

Cheers,
Dave


On Apr 27, 2012, at 4:49 AM, David Cournapeau wrote:

> I use ply to parse the package declaration in my packaging tool bento, and the related command line tools would be significantly slower without the ply caching.
>
> OTOH, the current caching solution is a bit difficult to use in those situation: I think having a pickable object, and extracting the caching policy away from the main code would cover most usecases in that context.
>
> David

[Gašper Ažman]

Hm.

I'm not using that functionality personally - as far as I'm concerned,
you can ditch it.

Gasper

> --
> You received this message because you are subscribed to the Google Groups "ply-hack" group.
> To post to this group, send email to ply-...@googlegroups.com.
> To unsubscribe from this group, send email to ply-hack+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/ply-hack?hl=en.
>

--


Gašper Ažman

[Rob Fowler]

Pickling sounds like a great idea, if possible. Personally the table system works ok for my compiler as well.

[David Cournapeau]

On Thu, Apr 26, 2012 at 7:27 PM, David Beazley <da...@dabeaz.com> wrote:

[Dave Benjamin]

I think making it so that the user can decide how best to store the precalculated data, or not to store it all, would be nicest. Dropping files in the current directory is not really ideal as a factory setting, though there are surely still use cases that benefit from that behavior.

[Joseph S. Tate]

Another vote for leaving the optimization in some form.

> --
> You received this message because you are subscribed to the Google Groups "ply-hack" group.
> To post to this group, send email to ply-...@googlegroups.com.
> To unsubscribe from this group, send email to ply-hack+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/ply-hack?hl=en.
>

--
Joseph Tate
Personal e-mail: jtate AT dragonstrider DOT com
Web: http://www.dragonstrider.com

[John Szakmeister]

One project I worked on was almost as big as the ANSI C grammar. I
don't think it would have been quite as useful had we needed to wait
for tables every time. OTOH, processors have come a long way since
then, and I haven't worked on that project in a while. Personally, I
think I'd want to keep it from doing that work over again. But I
certainly wouldn't cry about it leaving either. :-)

-John

[Boriel]

I agree with Erez and Bart.

BTW, I currently notice some delay when starting ply (my compiler, http://zxbasicand) for the first time it generates LALR tables, and they might be useful for other purposes.

On the other hand, I understand you want to clean up PLY (e.g. this makes it simpler and more maintainable). Maybe you could move that code out, to a different module or something?

Regards,
Jose

2012/4/27 Bart Whiteley <bwhi...@gmail.com>

--

Boriel      http://www.boriel.com