Re: [inspyred] multiprocessing applies threading instead of multiple processors

[Aaron Garrett]

The multiprocessing in inspyred just uses the Python standard multiprocessing library. So my guess would be that your question is really more general about how to get multiprocessing to use all cores. My quick search on that turned up several possibilities, some of which depend on what other libraries you have installed. 

My first thought about the "while" loop that makes things work "correctly" is that maybe your evaluator executes so quickly that the multiprocessing library (and, one presumes, the OS) doesn't send any of those processes to a different core. But then you put an infinite loop in there, which makes the process run indefinitely, and now using multiple cores is warranted. I would search for "force python multiprocessing to use all cores" or something like that and see if any of the first results match something like your situation. I think, though, that you will find that the core use will go up correspondingly as your evaluation function becomes more intensive.

On Thu, Sep 13, 2018 at 1:04 PM Ferdi Zoet <apf...@gmail.com> wrote:

Hi,

I am working on a Vehicle Routing Problem. I have the following config:

//////////////////////////////////////////////////////////////////////////////////////////////////////////////
def main():

  data = DataProblem()

  prng = Random()
  prng.seed(time.time())

  problem = inspyred.benchmarks.TSP(data.distances)
  ea = inspyred.ec.EvolutionaryComputation(prng)
  ea.selector = inspyred.ec.selectors.tournament_selection
  ea.variator = [custom_partially_matched_crossover, custom_inversion_mutation]
  ea.replacer = inspyred.ec.replacers.generational_replacement
  ea.terminator = inspyred.ec.terminators.generation_termination
  final_pop = ea.evolve(generator=generator,
                        bounder=problem.bounder,
                        maximize=True,
                        pop_size=100,
                        evaluator= inspyred.ec.evaluators.parallel_evaluation_mp,
                        mp_evaluator=evaluator,
                        data=data,
                        max_generations=1000,
                        tournament_size=5,
                        num_selected=100,
                        num_elites=1)

  best = max(ea.population)
  print('Best Solution: {0}: {1}'.format(str(best.candidate), 1/best.fitness))
  return ea


if __name__ == '__main__':

  main()
//////////////////////////////////////////////////////////////////////////////////////////////////////////////


However, it appears that the library is using 4 threads instead of 4 cores. I am kinda stuck and tried various things.

Here is my evaluator:

//////////////////////////////////////////////////////////////////////////////////////////////////////////////
def evaluator(candidates, args):

    data = args['data']
    distances = data.distances

    fitness = []

    for candidate in candidates:
        tot_distance = 0
        for index, route in enumerate(candidate):
            tot_distance += distance(route, distances)

        fitness.append(1/tot_distance)
    # while True:
    #   pass

    return fitness
//////////////////////////////////////////////////////////////////////////////////////////////////////////////

Its a pretty straight forward calculation. For the VRP a candidate is a list of lists. In the evaluator im just calculating the total distance for the candidate.

When I uncomment "while True: pass", the 4 cores are being enabled. Could someone point me in the right direction?

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups "Inspyred" group.
To unsubscribe from this group and stop receiving emails from it, send an email to inspyred+u...@googlegroups.com.
To post to this group, send email to insp...@googlegroups.com.
Visit this group at https://groups.google.com/group/inspyred.
For more options, visit https://groups.google.com/d/optout.

--

Aaron Garrett, Ph.D.

Assistant Professor

Computer Science

Wofford College

429 North Church Street

Spartanburg, SC 29303

garr...@wofford.edu

(864) 597-4529