parallel computation with openquake
Pierre Arroucau
I am using openquake 2.9.0 on Linux, I used openquake-setup-linux-2.9.0.run to install it.
It is running well so far, and now I would be interested in running it on a cluster in order to run larger calculations by distributing tasks on several nodes. I guess it should not be too complicated given that the code already runs automatically on all available cpus on my laptop. Is there a tutorial that explains how to proceed? The best I have found for now is the following page: https://github.com/gem/oq-engine/blob/master/doc/installing/cluster.md but it is not super-clear to me. For instance I'm not sure if the package I installed is enough or if I need to install additional components ; if so is there anything that needs root privilege or can I do it on my own?
Thanks in advance for any help.
Cheers,
Pierre
Daniele Viganò
Dear Pierre,
the most easy way to install the OpenQuake Engine on multiple
nodes is either using packages (Ubuntu
or RedHat/CentOS)
or using Docker.
This requires root access (or an access to a Docker-like cluster
where to deploy containers), a master node (where users have
access via CLI or WebUI/API) and multiple workers (where most of
the calculation is performed). These nodes must communicate via a
network link (see here
for more info about protocols used).
Packages will do the hard part of the job for you; in theory you can also run a cluster having the OpenQuake Engine installed from sources (like we do on our physical clusters); root access isn't strictly required when installing the Engine this way even if some tricks are needed, like configuring the Engine to use non-reserved ports (if your firewall allows that) and having a custom installation of RabbitMQ. This is anyway not recommended unless you already have a very good knowledge of Linux, Python and how Celery works.
If your organization/company is looking for a customized installation of the Engine on a complex environment I would suggest you to get in touch with us at engine....@openquake.org providing more information about your current needs or plans; eventually we can also discuss about a collaboration/partnership.
Hope that helped.
Cheers,
Daniele
--
You received this message because you are subscribed to the Google Groups "OpenQuake Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openquake-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Pierre Arroucau
I will pass the message to the cluster administrators and see.
Cheers,
Pierre
Pierre Arroucau
The cluster is actually running under Debian, do you think the Ubuntu package should work?
Cheers,
Pierre
Daniele Viganò
Hi Daniele,
The cluster is actually running under Debian, do you think the Ubuntu package should work?
Cheers,
Daniele
Pierre Arroucau
Cheers,
Pierre