run a function on all workers (setup)

14 views
Skip to first unread message

William Heymann

unread,
Oct 24, 2017, 5:32:49 AM10/24/17
to scoop-users
I am trying to figure out how to run a single function on all workers once for setup. 

I have a system where after the simulation has reached a certain point there are some parameters that are changed and I need to instruct all workers to rebuild their local caches.

I know I can map the function and use a large enough sequence that all the workers will probably run the code to update the cache at least once but that does not guarantee all of them run it and running it multiple times is not efficient. 

The other idea I had was for the master process to generate a token and hand it with every simulation to all workers and if the token is ever different then they rebuild their caches but that would require a lot of changes to the system and I would prefer not do something like that. 

Any suggestions for how to do this?

Thanks
Bill

Derek Tishler

unread,
Oct 28, 2017, 3:19:06 AM10/28/17
to scoop-users
I had to do this with stochastic/live data that changed with each new n_generations. To ensure this happened, I also passed the generation number into evaluate, and then into a function in the class that would check for a change in gen number and update to the next dataset before compiling and using the up-to-date data sourced from the Class object. 

I use a globally defined Class just like the ant example:
https://github.com/DEAP/deap/blob/master/examples/gp/ant.py
Reply all
Reply to author
Forward
0 new messages