How to obtain all the population individuals at each generation?

[Ning An]

Hi all,

I am trying to use history class to get all the population individuals at each generation, but found an error as follows: AttributeError: 'Toolbox' object has no attribute 'population'

The code I am using is here:

      hist = tools.History()

      toolbox.decorate("mate", hist.decorator)

      toolbox.decorate("mutate", hist.decorator)

      population = toolbox.population(n=300)

      hist.update(population)

[Ning An]

I know the final best individual at the end of the run can be easily obtained by using the function "hof.items[0]", but how can I obtain the best individuals at each generation? This is an old question and I found many discussions on google about it but did not find the solution yet. Thanks in advance!

--
You received this message because you are subscribed to the Google Groups "deap-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to deap-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/deap-users/61ee6d6a-c7bf-46f2-8d21-58f9186e6aeeo%40googlegroups.com.

--

Dr. Ning An

School of Aerospace

Xi'an Jiaotong University

Personal Web: www.anning003.com

[Derek Tishler]

I was able to apply the history example(genealogy tree figure isnt working for me, makes a chaotic circle vs expected tree on symbreg and onemax example...perhaps my networkx is messed up). But the history part appears to be working. 

Perhaps there is an issue in the order you applied the history items? I placed '############' around my changes so you can see where they can go to get History working.

From: https://deap.readthedocs.io/en/master/api/tools.html#history

History examples(scroll down on gist for second example): 
https://gist.github.com/DMTSource/7babce7f0269390d9d827443ce2231f5

[Derek Tishler]

However...reading the source of History, it seems to be genealogy specific?I am unsure how to print the individuals per gen(that info does not appear to be stored?).

https://github.com/DEAP/deap/blob/master/deap/tools/support.py#L21

[serguey maximov]

Hi,

I got this answer last year: https://stackoverflow.com/questions/58999820/python-deap-getting-the-pareto-front-for-every-generation

It seems to work.

To view this discussion on the web visit https://groups.google.com/d/msgid/deap-users/CACiDVdF2ZuJaRsoY63qVMU%2Bd4SOCNq%3DzdPZs25sQaY7qh4T-6A%40mail.gmail.com.

[Derek Tishler]

Serguey;s method seems easy to implement! That may be the way to go.

I was going to suggest(a more complicated but perhaps desirable method) using the algorithm locally in order to have full access to its inner working. You can copy easimple, or other algos, to your local code, and then do anything you like in the loop such as using toolbox.clone (dont just store individuals, as they may change) each individuals each gen, or take from them the info you need and save to a dataframe or dict or something.

Location of easiimple for example(don't forget to change varand to algorithms.varand or it'll break):

https://github.com/DEAP/deap/blob/master/deap/algorithms.py#L85

On Friday, July 24, 2020 at 6:24:23 AM UTC-4, serguey maximov wrote:

Hi,

I got this answer last year: https://stackoverflow.com/questions/58999820/python-deap-getting-the-pareto-front-for-every-generation

It seems to work.

On Fri, Jul 24, 2020 at 10:26 AM Ning An <anin...@gmail.com> wrote:

I know the final best individual at the end of the run can be easily obtained by using the function "hof.items[0]", but how can I obtain the best individuals at each generation? This is an old question and I found many discussions on google about it but did not find the solution yet. Thanks in advance!

On Fri, Jul 24, 2020 at 5:05 PM Ning An <anin...@gmail.com> wrote:

Hi all,

I am trying to use history class to get all the population individuals at each generation, but found an error as follows: AttributeError: 'Toolbox' object has no attribute 'population'

The code I am using is here:

      hist = tools.History()

      toolbox.decorate("mate", hist.decorator)

      toolbox.decorate("mutate", hist.decorator)

      population = toolbox.population(n=300)

      hist.update(population)

--
You received this message because you are subscribed to the Google Groups "deap-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to deap-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/deap-users/61ee6d6a-c7bf-46f2-8d21-58f9186e6aeeo%40googlegroups.com.

--

Dr. Ning An

School of Aerospace

Xi'an Jiaotong University

Personal Web: www.anning003.com

--
You received this message because you are subscribed to the Google Groups "deap-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to deap-...@googlegroups.com.

[Georges FARINA]

Hi, 

I'm reviving this conversation... 

I am implementing a knapsack EA almost exactly like the ga-knapsack example for a real life geographical example. My knapsack items is a dataframe of 300 points. 

I am trying to only store my last population so as to create a panda dataframe that features them. I am only interested in this last population. 

I want this dataframe to features in rows each individuals of my last population. 

And in columns : a first column that shows arrays or lists of the items that constitute each individuals (each knapsack)

Three more columns that have the fitness values of the individuals for my 3 objective functions.

This is for plotting purpose as well as exporting my results in Q-GIS.

Thing is, i don't know how to store properly my last population and all the data (fitness values, "inside" items) that are associated with it., hence why I'm commenting here after quite some unsuccessful hear pulling.  

For instance, if I create an array from my last pop in the following manner after the algorithm call :

bestPop = np.array([ind for ind in pop])

It ultimately is useless for storing it in a dataframe with other values, as I get "can't get attribute 'individual' errors.

I guess my begginer question is just : how to retrieve a list, array or whatever of the items inside each of my best individuals ? 

Many many thanks

Georges

--------------------------------------------------------------------------------------------

This e-mail transmission (message and any attached files) may contain information that is proprietary, privileged and/or confidential to Veolia Environnement and/or its affiliates and is intended exclusively for the person(s) to whom it is addressed. If you are not the intended recipient, please notify the sender by return e-mail and delete all copies of this e-mail, including all attachments. Unless expressly authorized, any use, disclosure, publication, retransmission or dissemination of this e-mail and/or of its attachments is strictly prohibited. 

Ce message electronique et ses fichiers attaches sont strictement confidentiels et peuvent contenir des elements dont Veolia Environnement et/ou l'une de ses entites affiliees sont proprietaires. Ils sont donc destines a l'usage de leurs seuls destinataires. Si vous avez recu ce message par erreur, merci de le retourner a son emetteur et de le detruire ainsi que toutes les pieces attachees. L'utilisation, la divulgation, la publication, la distribution, ou la reproduction non expressement autorisees de ce message et de ses pieces attachees sont interdites.

--------------------------------------------------------------------------------------------

[Derek Tishler]

Georges,

Can you try taking the final pop(output from the algo) and doing something like:
# Form each column for the DataFrame

pop_dict = {

    "individual": [ind for ind in pop]

    "ind_size" : [len(ind) for ind in pop],

    "fitness1" : [ind.fitness.values[0] for ind in pop],

    "fitness2" : [ind.fitness.values[1] for ind in pop],

    "fitness3" : [ind.fitness.values[2] for ind in pop],

}
# Create a DataFrame from the data dict where keys are the columns

df = pd.DataFrame(data=pop_dict)

It works like:
IN

 pd.DataFrame(data={"B" : [1,2,3], "A" : [4,5,6]})
OUT[note index automatically assigned as range(n)]

   B  A

0  1  4

1  2  5

2  3  6

[Georges FARINA]

Many thanks, worked like a charm ! 

Just a tiny last question if you please.. 

I've been struggling with this : 

My solution individuals are lists of items. I've stored them in a dataframe with all my 50 best individuals, their fitness values, etc

 Yet i cannot access the item inside each of these individuals. It all appears as  " individual{6,8,1,...,etc}", where the numbers are the id of each of the items inside those individuals.
I wish to make standard arrays of those, so as to have like

bestIndividual1 = [6,8,1,...]

bestIndividual2 = ...

etc

It may be very trivial and i apologize about this; 

Georges

[Derek Tishler]

Georges,

I'm glad things are moving forward.
Your individual sounds like a list so In the DataFrame forming stage, specifically the dict step I shared above, can you try recasting the individual from a Deap Individual Object to a list or array object using something like:
"individual": [list(ind) for ind in pop]
or
"individual": [np.array(ind) for ind in pop]