Work Queue firewall issues

[Gutenkunst, Ryan N - (rgutenk)]

Hello,

We are running into some firewall issues with Work Queue, and we hope you can help somewhat. Briefly, for our application the manager does trivial computation and mainly collects results, so we envisioned that the manager could run locally, communicating with Work Queue works on AWS or an HPC system.

What we’re finding is that communication between the manager process and factory workers seems to impeded if we run the manager on a local computer at our university or home with the workers on AWS. Putting the workers on the local computer and the manager on AWS works fine as a test. Our hypothesis is that the manager isn’t getting the unsolicited incoming communications from the workers, due to a university or ISP firewall. We’ve tried switching the port to 80 or 8080, and that doesn’t work either.

Do you have any suggested workarounds?

Thanks,
Ryan

--
Ryan Gutenkunst
Associate Professor and Associate Department Head
Department of Molecular and Cellular Biology, University of Arizona
phone: (520) 626-0569, office: LSS 325, web: http://gutengroup.arizona.edu

[Ben Tovar]

Ryan,

It is not uncommon for universities to have firewalls that block incoming connections for all but a small curated set of machines. One quick way you can test if this is what is happening to you is to launch the manager as usual in your local machine, and in one of the AWS machines use nmap to see if the port is being filtered, like:

# from a aws machine

nmap -Pn ip-machine-with-manager

If you don't see your selected port as "open", then this indeed points to a firewall issue. The first thing you would want to do is ask your local sysadmin about which policies do they have, as the easiest solution would be for your selected port to be whitelisted. If this is not possible, you can use ssh tunnels to redirect connections from AWS. In this case, all workers connect to an AWS machine as if the manager was running there, and this machine connects to your local machine using ssh.

Ben

--
You received this message because you are subscribed to the Google Groups "Cooperative Computing Tools" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cctools-nd+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cctools-nd/4E00D0FF-EBF1-4204-88DF-F33B82C9B7DB%40arizona.edu.

[Ryan Gutenkunst]

Thanks Ben!

We’ll explore these options.

But, wow, working around firewall issues adds a ton of complexity for our potential users. Not your fault what security looks like nowadays, but very frustrating and it’s likely deal-breaking for cloud usage in many cases. :-/

Best,
Ryan

> To view this discussion on the web visit https://groups.google.com/d/msgid/cctools-nd/CAMik99VbB-qFF1if1qDa6kPif%3DT3knunN%3DHZYky-Ugv1sKZ9iw%40mail.gmail.com.

[Douglas Thain]

Hello Ryan -

For better or worse, this is the reality of building anything on the Internet today.  You need to ask the network operators for "permission" before you can make connections from place to  place.  In our experience, most facilities have designated specific head nodes, virtual machines, or other locations where network activity is more encouraged.  Suggest talking to your local facility people to sort things out.

Sincerely,

Doug