Parallelization of cma_minfct.py example

[Giovanni Pellegrini]

I am trying to use cma for a continuous optimization problem. The cma_minfct.py example seems to be and ideal template for my case, where I basically need to change just a few parameters and the evaluation function to get a working script. Yet the evaluation of my objective functions is rather time consuming, therefore I would like to perform a parallel evaluation via toolbox.register("map",pool.map). As I first step I tried to parallelize the cma_minfct.py example itself, yet I get the following error:

Process ForkPoolWorker-1:

Traceback (most recent call last):

  File "/home/giovi/anaconda3/lib/python3.4/multiprocessing/process.py", line 254, in _bootstrap

    self.run()

  File "/home/giovi/anaconda3/lib/python3.4/multiprocessing/process.py", line 93, in run

    self._target(*self._args, **self._kwargs)

  File "/home/giovi/anaconda3/lib/python3.4/multiprocessing/pool.py", line 108, in worker

    task = get()

  File "/home/giovi/anaconda3/lib/python3.4/multiprocessing/queues.py", line 357, in get

    return ForkingPickler.loads(res)

AttributeError: Can't get attribute 'Individual' on <module 'deap.creator' from '/home/giovi/anaconda3/lib/python3.4/site-packages/deap-1.1.0-py3.4-linux-x86_64.egg/deap/creator.py'>

It is my understanding the workers are unable to see the creator generated with

creator.create("Individual", list, fitness=creator.FitnessMin)

but I really do not understand how to fix the example in order to execute it in a parallel fashion.

Best and thanks for deap

Giovanni

[François-Michel De Rainville]

Do you create your Individual class in the global scope of the main script? This is mandatory because the workers have to build the classes as well.

Best regards,

François-Michel

--
You received this message because you are subscribed to the Google Groups "deap-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to deap-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Giovanni Pellegrini]

My script is identical to cma_minfct.py with the addition of the only line toolbox.register("map",pool.map). Here is the gist of my script so you can check. Anyway, to answer your question, I think it is in the global scope, but I could be mistaken.


Best


Giovanni

[Marc-André Gardner]

Hi Giovanni,

Just move the line :

pool = Pool()

Right before the line :

toolbox.register("map",pool.map)

And after all the other preamble declarations. Else, the pool is created before the Individual class is made, and the workers can not access it, so you get an error.

Do not hesitate if there's any other issue and have fun with DEAP,

Marc-André

[François-Michel De Rainville]

Good catch!

--

[Giovanni Pellegrini]

Awesome!!! Thanks a lot!!!