Netlogo on HPC cluster(s)

[Dave Olsthoorn]

Hi all.
 
The researchers at the department of 'Engineering Systems and Services' at the
TU Delft sometimes use NetLogo for simulating complex systems. These simulations
can take quite log to run because of the complex interactions and big
BehaviorSpace parameters. In the past people have had success using scripts to 
split the BehaviorSpace in to parts to speed it up by running them in parallel 
on a HPC cluster, the problem with these scripts is that they are too complex
for most users in our department.
 
Because of this, I was tasked to figure out a way to make NetLogo easier to use

on HPC clusters. Using the control API to read and distribute the BehaviorSpace,

submitting jobs on the (SLURM based) HPC cluster and combining/collecting the

data of the runs and writing them to one csv file. The current plan is that this would

essentially be a variation of netlogo-headless with remote workers that run as jobs

in the cluster.

Do you, the developers of NetLogo, have opinions about design or architecture

and is this something that upstream NetLogo itself wants?

Place of future code/documentation: https://github.com/daveol/NetLogo-HPC

[Michael Tamillow]

I may be mistaken, but I think if you just use the BehaviorSpace option "parallel runs", it will utilize all cores to run multiple simulations (at most one core per run). I believe the HPC cluster should act as a single machine and run across processors. 

https://ccl.northwestern.edu/netlogo/docs/behaviorspace.html

On the HPC side, there may be some wrangling if you are using special hardware or have some unusual architecture. Collecting and aggregating the data after runs is up to you, and will always be a little extra work.

--
You received this message because you are subscribed to the Google Groups "netlogo-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to netlogo-devel+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dave Olsthoorn]

Hi,

While some clusters combine all processors (like beowulf), however we at the TU Delft use a job based cluster (slurm). So "parrallel runs" is still the idea, just across different "jobs" which might run on different nodes in the cluster.

Op woensdag 6 juni 2018 20:09:25 UTC+2 schreef Michael Tamillow: