New Issues with Ephemerals and Scoop?

[Thomas Macrina]

Hey guys,

I've run into a couple issues with ephemerals and scoop in v1.0.1 . Here's a simple example to help me explain: https://gist.github.com/macrintr/10751834

When I run this code without scoop, it works without a problem. When I run it with scoop, it fails on the exception: "Ephemerals with different functions should be named differently, even between psets." When I comment out that section of gp.py (line 377), I run into the scoop exception for not being able to pickle an individual. My guess is that the pickling fails because of the staticmethod that's included in addEphemeralConstants (gp.py, line 372) or when it's a decorator in the Ephemeral class (gp.py, line 233).

Any advice on how I could work around this? Thanks again for all the help, guys.

Thomas

[Yannick Hold-Geoffroy]

Hello Thomas,

I am curious as to what "deaper" refers to in your code?

The problem arises because SCOOP imports your program to check for errors before executing it. Since your program is executed twice on the node containing the root task, the ephemeral function is defined twice (so ends up as different definitions) which DEAP flags as an attempt to re-define an ephemeral with another function. I have fixed this issue in a recent commit which prevents the process containing the root task to import your module beforehand. You can either install SCOOP manually using the googlecode repository or using "pip install hg+https://code.google.com/p/scoop/" to get a version of SCOOP with this fix.

There is a second problem in your code: you try to pass your pset as an argument of your evaluation function. Sadly, the pset objet is not picklable.

Here is a modified version of your code that works using the latest development version of SCOOP: https://gist.github.com/soravux/10786149

Have a nice day,

Yannick Hold

[Thomas Macrina]

Thanks, Yannick! The patch and removing the pset parameter from the evaluate function worked like a charm.

I forgot to clean up my code - "deaper" was a copy of the deap repo where I had been directly modifying the code to see if I could bend ephemerals to my will.

Thanks again,

Thomas

[George Coles]

I ran into a problem with this - I am creating my Primtive Set multiple times in a loop, in my program. Because the PrimitiveSet class is an object, I just didnt expect that it would be assigning things into globals!

I would have thought that the Primitive set would act as a portable context, and would essentially define its own scope - may I ask why it needs to put variables into the global scope of the enclosing program? 

thx

George

[Yannick Hold-Geoffroy]

Hello Mr. Coles,

Sorry, but I fail to see the link between your question and the current thread. I guess you would have a better chance creating a new thread for your new question.

While I can't vouch for the reasoning behind this logic, I guess the main usage of a module using deap will be to perform evolutionary algorithms. If you need to import this module, don't use "from my_module import *" but specify the names of the objects you want to import. This is common practice to prevent the cluttering of your namespace. With that in mind, deap's globals won't pollute the rest of your applications. On a more technical point of view, putting the primitive set in the global variables of the module is quite handy. It prevents the need to pass references everywhere and also simplifies the use of parallel modules such as multiprocessing. Since the primitive set is already available from the global scope, there is no problem to initialize the rest of the program on a single worker and perform evaluations on multiple threads (which need the primitives to compile the individuals).

Have you got an example where this behavior is problematic?

Have a nice day,

Yannick Hold

--
You received this message because you are subscribed to the Google Groups "deap-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to deap-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.