[Boost-users] [preprocessor] extending loop limit >> 256

[Martin Pasdzierny]

Hi boost experts,

using BOOST_PP_LOCAL_ITERATE() and friends I found the maximum number
of iterations to be limited to 256.

The according values seemed to be defined in
.../preprocessor/config/limits.hpp

But changing these values does *not* have any effect on the maximum
number of iterations

...
#define BOOST_PP_LOCAL_MACRO(n) g(n);
#define BOOST_PP_LOCAL_LIMITS (1, 300 )
#include BOOST_PP_LOCAL_ITERATE()
#undef BOOST_PP_LOCAL_MACRO
#undef BOOST_PP_LOCAL_LIMITS
...

still creates:

g(1);
g(2);
g(3);
...
g(255);
g(256);

the expected:
g(257);
g(258);
...
g(299);
g(300);
are *not* generated.

Stepping through the libraries code I found the mechanisms far more
complicated than expected.
Now my question: is there any recommended procedure to increase the
iteration limitation to values like 10000 ?

I found a couple of code locations explicitly repeating code with hard
coded values from 1 ... 256.
I guess at least this code needs to be extended, but probably consistent
with some other changes.

I am thinking of a perl script to generate stuff like that, but before
working on that I'd like to get some qualified recommendations ;)

Thanks in advance,
Martin.
_______________________________________________
Boost-users mailing list
Boost...@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users

[Joel Falcou]

What is your use case ? If it's loop unrolling consider using a small
template recursive-function. it'll be inlined in the same way

[Martin Pasdzierny]

C++ template based MPL is very beautiful and would be favored if
supported by the compilers we have to support. This works quite nice
with e.g. gcc 4.x, we successfully used template lists with up to 8000
items.

But: try to make some proprietary and outdated compilers (as required by
our customer) compile a template list of 10000 tokens. You may get:
- compiler does not support template recursion deeper than e.g. 16
- compiler crashes without any message (as our AS400 IBM compiler)
- compiler signals 'out of memory'
- compiler succeeds after some days of compilation time and creates
quite big binaries even in case of pure compile time statements.
All that is not acceptable.

We have to generate code supporting a large data model we can't change
(up to 10000 columns per table). The idea is to generate e.g. SQL
statements from within the C++ tool chain by appropriately #including
the text files describing the data model. If the proprietary
preprocessors fail we think of using Hartmut Kaisers 'wave' instead.

I already have some preprocessor-based code in place, but it became very
complicated and hard to maintain. I'd prefer to use a well tested and
documented framework like boost.preprocessor instead.
My question really is about the limitations of boost-preprocessor.

Thanks in advance, Martin.

[Paul Mensonides]

On Wed, 29 Dec 2010 13:01:22 +0100, Martin Pasdzierny wrote:

> using BOOST_PP_LOCAL_ITERATE() and friends I found the maximum number
> of iterations to be limited to 256.
>
> The according values seemed to be defined in
> .../preprocessor/config/limits.hpp
>
> But changing these values does *not* have any effect on the maximum
> number of iterations

No, it won't. Those definitions are informational.

> Stepping through the libraries code I found the mechanisms far more
> complicated than expected.
> Now my question: is there any recommended procedure to increase the
> iteration limitation to values like 10000 ?
>
> I found a couple of code locations explicitly repeating code with hard
> coded values from 1 ... 256.
> I guess at least this code needs to be extended, but probably consistent
> with some other changes.

Besides that, the main thing would be increasing how many digits are put
together on the bounding limits. I believe that the values in
LOCAL_LIMITS are evaluated (to avoid repetitive use of TUPLE_ELEM) by the
innards of the slot mechanism. That mechanism does 10 digit (base 10)
resolution. The local iteration mechanism only pulls the last three
digits (see boost/preprocessor/iteration/detail/{start,finish}.hpp at the
ends).

I haven't specifically looked at what it would take with the file-
iteration mechanism, but it would be similar (but in more places).

-Paul