Configuring aiida for remote clusters with ssh restrictions

[Kayahan Saritas]

Hello, 

I am a new user interested in using aiida in my daily work. I have read through most introductory material on the website, but couldn't find specific info about some of the questions I have. I would appreciate it if you could answer when you have the time. My questions are mainly related to how to work with aiida when we need to work with an access restricted remote cluster. Maybe we can discuss several scenarios:

1. Remote cluster that requires an entry from a physical/digital keyfob (securID for example)  in addition to the ssh password to access the cluster. 

I would think that if the local installation (on laptop) of aiida will repeatedly need to access the remote computer, it will be difficult to do so in a setup like this. What would your recommendation be for this case in terms of configuring aiida? Can I still use a local installation of aiida on my laptop efficiently, or should I install aiida on the remote computer?

2. SSH connection would need to be handled through a proxy such as:

<ssh -o ProxyCommand="ssh -W %h:%p jumphost" server>.

I think this should be easier to resolve. Mainly I am wondering how to get a list of options when configuring a computer. 

https://aiida.readthedocs.io/projects/aiida-core/en/latest/howto/run_codes.html?highlight=computer#computer-setup

I am assuming there is already a solution for this in that list of options, but I would appreciate it if you could guide me. 

Thanks,

Kayahan Saritas

ORNL

[Sebastiaan Huber]

Hi Kayahan,

Hello, 

I am a new user interested in using aiida in my daily work. I have read through most introductory material on the website, but couldn't find specific info about some of the questions I have. I would appreciate it if you could answer when you have the time. My questions are mainly related to how to work with aiida when we need to work with an access restricted remote cluster. Maybe we can discuss several scenarios:

1. Remote cluster that requires an entry from a physical/digital keyfob (securID for example)  in addition to the ssh password to access the cluster. 

I would think that if the local installation (on laptop) of aiida will repeatedly need to access the remote computer, it will be difficult to do so in a setup like this. What would your recommendation be for this case in terms of configuring aiida? Can I still use a local installation of aiida on my laptop efficiently, or should I install aiida on the remote computer?

If the server really requests two-factor authentication (2FA) each time you connect, then this would indeed be not feasible as a setup for AiiDA, as the daemon will connect quite often.
If it is possible to generate a key through 2FA that is then valid for some amount of time (let's say 24 hours or more) then this could be a possibility.
You would simply have to manually regenerate the key each time and restart the daemon.
AiiDA has built in functionality to pause active jobs if the connection with the cluster is interrupted and can be easily resumed when it is reestablished.

Since more and more centers are moving to 2FA and removing the possibility for services to interact directly over SSH, we are working on addressing this problem.
One example is the FirecREST REST API (https://github.com/eth-cscs/firecrest).
This is being developed by the Swiss Supercomputing Center (CSCS) but I have heard that NERSC is considering adopting it as well.
We are working on providing direct support in AiiDA for this protocol, but this will take some time before it is production ready.

The other solution would indeed be to have AiiDA installed on the same network as the cluster.
This is the solution used by your colleagues at Lawrence Livermore National Laboratory (LLNL).
They have AiiDA, PostgreSQL and RabbitMQ running each on dedicated machines inside the cluster network and so AiiDA can then easily submit and control jobs to SLURM on the cluster without problems.

2. SSH connection would need to be handled through a proxy such as:

<ssh -o ProxyCommand="ssh -W %h:%p jumphost" server>.

I think this should be easier to resolve. Mainly I am wondering how to get a list of options when configuring a computer. 

https://aiida.readthedocs.io/projects/aiida-core/en/latest/howto/run_codes.html?highlight=computer#computer-setup

I am assuming there is already a solution for this in that list of options, but I would appreciate it if you could guide me.

In `verdi computer setup` simply choose `core.ssh` for the transport type option.
When the computer is created, you configure it using `verdi computer configure core.ssh COMPUTER_LABEL`.
Here you should probably use the `--proxy-jump` and/or `--proxy-command` options:

  --proxy-jump TEXT               SSH proxy jump for tunneling through other
                                  SSH hosts. Use a comma-separated list of
                                  hosts of the form [user@]host[:port]. If
                                  user or port are not specified for a host,
                                  the user & port values from the target host
                                  are used. This option must be provided
                                  explicitly and is not parsed from the SSH
                                  config file when left empty.
  --proxy-command TEXT            SSH proxy command for tunneling through a
                                  proxy server. For tunneling through another
                                  SSH host, consider using the "SSH proxy
                                  jump" option instead! Leave empty to parse
                                  the proxy command from the SSH config file.

It looks though that you might just be able to configure the connection in your `~/.ssh/config` file and AiiDA will parse this (make sure the hostname in the config and the AiiDA setup match).
Use `verdi computer configure core.ssh --help` to show detailed help for all options.

I have not ever configured myself a computer with a proxy jump, so I am afraid I cannot give more details than that.

Hope that helps,

Regards,

Sebastiaan

Thanks,

Kayahan Saritas

ORNL

--
AiiDA is supported by the NCCR MARVEL (http://nccr-marvel.ch/), funded by the Swiss National Science Foundation, and by the European H2020 MaX Centre of Excellence (http://www.max-centre.eu/).
 
Before posting your first question, please see the posting guidelines at http://www.aiida.net/?page_id=356 .
---
You received this message because you are subscribed to the Google Groups "aiidausers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aiidausers+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/aiidausers/1f8e89e8-610c-4afc-8e7a-7019e4909df0n%40googlegroups.com.