Anyone extended MAXTABLE in yacc parsers?
C. H. Seet
a parse table with about 130000 elements (grammar is large).
Byacc's limit (defined by MAXTABLE) is 32500. I've tried extending this
number to 150000, but the resultant table does not work.
I then tried a combination of the following:
- extend MAXTABLE to 150000
- redefine BITS_PER_WORD from 32 to 64
- redefine SETBIT and BIT macros to cope with 64 bit "word"
- replace "unsigned" by "unsigned __int64" (I'm using Visual C++)
- replace short by int
- redefine MAXSHORT from 32767 to 2147483647
I still could not generate a table that works. Does anyone have a
solution, or has anyone successfully extended some variant of yacc
(with source code in public domain) to cope with large grammars? The
source code is necessary as I would like to port over some extensions
I've made.
Looking forward to some help,
C.H. Seet
[Sorry, I've never used a grammar that didn't fit in the byacc defaults.
Maybe Bob C. will have some suggestions. -John]
--
Send compilers articles to comp...@iecc.com, meta-mail to
compiler...@iecc.com. Archives at http://www.iecc.com/compilers
Vladimir Makarov
COCOM toolset on
MSTA supports SYSTEM V YACC and Posix YACC and has no constraints on tables
size. The single constraint is only your virtual memory size. The
important difference MSTA from bison and byacc which I know - there is no
implicit attribute propagation.
C. H. Seet wrote:
> I'm using byacc 1.9 on Win32 platform, and I've been trying to generate
> a parse table with about 130000 elements (grammar is large).
[Bison is a mutant version of byacc, the internal structures are the same
the last I checked. -John]
Robert Corbett
>Byacc's limit (defined by MAXTABLE) is 32500. I've tried extending this
>number to 150000, but the resultant table does not work.
The reason the limit is 32500 is that the offsets into the table
are shorts. To make tables larger than 32767 work, you will have
to change a bunch of shorts to ints or longs.
At the time I wrote Berkeley Yacc, machines had such small memories
that using shorts instead of ints was a big plus. Times change.
Sincerely,
Bob Corbett
Robert Corbett
>the last I checked. -John]
Both Bison and Berkeley Yacc are mutant variants of the parser
generator I wrote as part of my research on error recovery.
That parser generator was named Bison. It was based on the
brilliant (though hard to read) paper by Pennello and DeRemer
in TOPLAS 4:4.
Max TenEyck Woodbury
>
> The reason the limit is 32500 is that the offsets into the table
> are shorts. To make tables larger than 32767 work, you will have
> to change a bunch of shorts to ints or longs.
>
> At the time I wrote Berkeley Yacc, machines had such small memories
> that using shorts instead of ints was a big plus. Times change.
I don't know if this is common practice, but I have found it useful to
'typedef' a new type when I think I need certain properties. The
typedef gives me a place to document the properties required and lets
me tailor the exact type used to the environment. This looks like a
place where that practice would be very useful.
I have recently started including macros for output modifiers
to match the typedefs, but that's another topic.
Vadim Maslov
>
> I'm using byacc 1.9 on Win32 platform, and I've been trying to generate
> a parse table with about 130000 elements (grammar is large).
> number to 150000, but the resultant table does not work.
> - extend MAXTABLE to 150000
> - redefine BITS_PER_WORD from 32 to 64
> - redefine SETBIT and BIT macros to cope with 64 bit "word"
> - replace "unsigned" by "unsigned __int64" (I'm using Visual C++)
> - replace short by int
> - redefine MAXSHORT from 32767 to 2147483647
>
We extended BtYacc (which is different from byacc) to work with more
than 32,767 elements in the table.
It requires more than just adjusting constants. Our tables go up to
80,000 elements which is more than 65,536.
You can get it at http://www.siber.com/btyacc/ for free.
It is based on Berkeley Yacc and it can do:
- backtracking
- %include and %define statement in the grammar.
Regards,
Vadim Maslov