Skip to first unread message
aytem...@gmail.com
Mar 8, 2016, 10:23:47 PM3/8/16
to deap-users
Hi,
I am seeding individuals from a file and initializing the population from them. I am able to initialize the population just fine but when I run the GP algorithm, it crashes with this error:
MemoryError: DEAP : Error in tree evaluation : Python cannot evaluate a tree higher than 90. To avoid this problem, you should use bloat control on your operators. See the DEAP documentation for more information. DEAP will now abort.
To be fair, one of the individuals is really long, like this:
add(multiply(1.1, 4461.2419), multiply(safeDiv(subtract(ARG0, 297805.5), 475.5), add(negative(0.004993), add(multiply(safeDiv(subtract(ARG1, 661.5), 294.5), 6142.2189), multiply(safeDiv(subtract(ARG2, 16493.0), 10305.0), add(negative(5832.6553), add(multiply(safeDiv(subtract(ARG3, 2366.3995), 41.2835), 1.0387), multiply(safeDiv(subtract(ARG4, 723.504), 33.446), add(negative(2.5521), multiply(safeDiv(subtract(ARG5, 1837.08), 1748.055), add(negative(0.099004), add(multiply(safeDiv(subtract(ARG6, 931.0525), 717.3595), 0.5513), add(multiply(safeDiv(subtract(ARG7, 0.458), 0.502), 1.014), add(multiply(safeDiv(subtract(ARG8, 36.85), 29.625), 11083.688), multiply(safeDiv(subtract(ARG1, 661.5), 294.5), add(multiply(safeDiv(subtract(ARG8, 36.85), 29.625), 4934.4353), multiply(safeDiv(subtract(ARG2, 16493.0), 10305.0), add(multiply(safeDiv(subtract(ARG3, 2366.3995), 41.2835), 0.55848), multiply(safeDiv(subtract(ARG2, 16493.0), 10305.0), add(multiply(safeDiv(subtract(ARG8, 36.85), 29.625), 5.7479), multiply(safeDiv(subtract(ARG3, 2366.3995), 41.2835), add(multiply(safeDiv(subtract(ARG8, 36.85), 29.625), 0.51928), multiply(safeDiv(subtract(ARG4, 723.504), 33.446), multiply(safeDiv(subtract(ARG8, 36.85), 29.625), add(negative(3.3441), multiply(safeDiv(subtract(ARG5, 1837.08), 1748.055), add(multiply(safeDiv(subtract(ARG8, 36.85), 29.625), 0.10886), multiply(safeDiv(subtract(ARG6, 931.0525), 717.3595), add(multiply(safeDiv(subtract(ARG8, 36.85), 29.625), 4.8525), multiply(safeDiv(subtract(ARG7, 0.458), 0.502), add(multiply(safeDiv(subtract(ARG8, 36.85), 29.625), 2.4708), multiply(safeDiv(subtract(ARG8, 36.85), 29.625), add(negative(8.2013), multiply(safeDiv(subtract(ARG2, 16493.0), 10305.0), multiply(safeDiv(subtract(ARG8, 36.85), 29.625), add(negative(2.628), multiply(safeDiv(subtract(ARG3, 2366.3995), 41.2835), multiply(safeDiv(subtract(ARG8, 36.85), 29.625), add(negative(0.57294), multiply(safeDiv(subtract(ARG4, 723.504), 33.446), multiply(safeDiv(subtract(ARG8, 36.85), 29.625), add(negative(0.04801), multiply(safeDiv(subtract(ARG5, 1837.08), 1748.055), add(multiply(safeDiv(subtract(ARG8, 36.85), 29.625), 0.94035), multiply(safeDiv(subtract(ARG6, 931.0525), 717.3595), add(multiply(safeDiv(subtract(ARG8, 36.85), 29.625), 4.6713), multiply(safeDiv(subtract(ARG7, 0.458), 0.502), add(multiply(safeDiv(subtract(ARG8, 36.85), 29.625), 1.5136), multiply(safeDiv(subtract(ARG8, 36.85), 29.625), 10.22433)))))))))))))))))))))))))))))))))))))))))))))))))
If I use smaller individuals, the code runs fine. How do I get around this error?
Thanks!
Marc-André Gardner
Mar 8, 2016, 10:35:54 PM3/8/16
to deap-users
Hi,
This error is actually issued by DEAP to avoid a quite more cryptic error :
This is because the Python interpreter as a limited amount of memory where it can put the ongoing parenthesis (in this case the function calls). This limit is 100, but around 90, errors begin to show in DEAP, the difference being explained by the wrappers added by DEAP for the evaluation of the tree.
It is simply impossible to evaluate such large individuals using Python. The only way to avoid this error is to use bloat control algorithms, like the one implemented here : http://deap.readthedocs.org/en/master/api/tools.html#bloat-control In particular, simply put a static limit of 17 or 20 is "enough" most of the time, even if there exist better bloat control approaches.
Fortunately, most of the time, such a deep tree is not very useful, so a limit of 90 in the tree height (remember, we're not talking about the tree size here) is not really detrimental.
Do not hesitate if you have any other questions,
Marc-André
This error is actually issued by DEAP to avoid a quite more cryptic error :
s_push: parser stack overflow
MemoryError
This is because the Python interpreter as a limited amount of memory where it can put the ongoing parenthesis (in this case the function calls). This limit is 100, but around 90, errors begin to show in DEAP, the difference being explained by the wrappers added by DEAP for the evaluation of the tree.
It is simply impossible to evaluate such large individuals using Python. The only way to avoid this error is to use bloat control algorithms, like the one implemented here : http://deap.readthedocs.org/en/master/api/tools.html#bloat-control In particular, simply put a static limit of 17 or 20 is "enough" most of the time, even if there exist better bloat control approaches.
Fortunately, most of the time, such a deep tree is not very useful, so a limit of 90 in the tree height (remember, we're not talking about the tree size here) is not really detrimental.
Do not hesitate if you have any other questions,
Marc-André
aytem...@gmail.com
Mar 10, 2016, 10:36:40 PM3/10/16
to deap-users
Great, thanks a lot for your help! :)
Reply all
Reply to author
Forward
0 new messages