Parallel computation for Monte Carlo simulation

[Jairo Arturo Torres M.]

I would like to use Itzï in a server which has available 80 cores (40 virtual) for processing. I am trying to deploy a Monte Carlo simulation for propagating input uncertainty through Itzï. I wonder if:

1. Can I establish a setup were several instances of Itzï run in parallel with different ensembles of precipitation?
2. Is it possible to define the maximum number of cores that Itzï uses for computation?
3. Should I expect any drawback while accessing GRASS GIS in my parallel setup?

Any help would be appreciated.

Cheers,

Arturo

[Laurent C.]

Hi Jairo,

1. It might be possible, but I believe it might be limited by GRASS

2. The number of threads used can be changed by setting the environment variable OMP_NUM_THREAD. But I do not know if it impacts individual instances, or if it is global.

3. Only if you try to write to the same file in parallel, I believe. You might try the GRASS mailing list for that answer.

However, running all your ensemble members in series might not be necessarily slower than running them in parallel (and you avoid some potential headaches). Itzï will use all available threads anyway. I never checked how it scales to that number of cores, but it might be interesting to test.

Cheers,

Laurent

--
You received this message because you are subscribed to the Google Groups "Itzï" group.
To unsubscribe from this group and stop receiving emails from it, send an email to itzi+uns...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/itzi/f1336b64-c1aa-492a-ae51-781ccf0caab1%40googlegroups.com.

[Jairo Arturo Torres M.]

Hi Laurent,

Thanks for your reply and advice. I will try two approaches: the first, running the simulation in series; the second, trying to access GRASS in parallel mode. I will keep you informed of this trial.

Cheers,

Arturo

To unsubscribe from this group and stop receiving emails from it, send an email to it...@googlegroups.com.

[Jairo Arturo Torres M.]

Hi Laurent,

I am trying to run in the server (with 40+40 cores) my workflow that runs fine in my laptop (4 + 4 cores). I got the next execution message:

WARNING: Error during execution: Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/itzi/itzi.py", line 210, in sim_runner_worker
    sim_runner.run()
  File "/usr/local/lib/python3.6/dist-packages/itzi/itzi.py", line 80, in run
    gis.check_output_files(self.conf.output_map_names.itervalues())
AttributeError: 'dict' object has no attribute 'itervalues'
                   
Simulations complete. Elapsed times:                   
itzi_HSure_carve50cm_BCtype4_0_extended_inf0_n0_03_hpc1_parameters.ini: 0:00:00                   
Total: 0:00:00                   
Average: 0:00:00                   
character(0)

Do you have any hint from where the problem comes from? Do you know the maximum number of cores that itzi can handle or require?

Cheers,

Arturo

El miércoles, 9 de octubre de 2019, 16:23:49 (UTC+2), Laurent C. escribió:

To unsubscribe from this group and stop receiving emails from it, send an email to it...@googlegroups.com.

[Laurent C.]

Hi Arturo,

I see you try to run itzi with python3.6. Itzï was developed at a time when GRASS was not compatible with python3, so itzi is written for python2.

Please try with python2, I hope it might solve your issue.

By the way, any help to make itzi compatible with python3 is more than welcome!

Regards,

Laurent

To unsubscribe from this group and stop receiving emails from it, send an email to itzi+uns...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/itzi/7339052b-e214-4256-933a-aa425e515dae%40googlegroups.com.