wait_for port isn't waiting

[Brian Williams]

Hi everyone,

I have a playbook where I am waiting for a port to come up. On certain machines it sometimes doesn't waits and immediately fails. I'm not certain what is causing this, and I can't easily reproduce.

Here is the task in the role:

- name: wait for api to come back up (120 minutes)

  wait_for: port=7778 timeout=7200 delay=61

Here is an example of the task with timestamps:

16:37:16 TASK [act_appliance : wait for api to come back up (120 minutes)] *******
16:38:12 ok: [gin]
16:38:13 fatal: [aregsky1]: FAILED! => {"changed": false, "elapsed": 18280, "failed": true, "msg": "Timeout when waiting for 127.0.0.1:7778"}

As you can see from the timestamps it is failing out of the gate and then not waiting.

The only time it passes is when it succeeds on the first try (This was run with 'strategy: free' unlike the above failure):

14:34:43 TASK [act_appliance : wait for api to come back up (120 minutes)] *******
14:43:50 changed: [aregsky1]

Afterwards I check and the port came up fine within 120 minutes, but the play didn't wait for it:

$ date
Fri Apr 21 17:06:55 EDT 2017

$ nmap -p 7778 aregsky1

Starting Nmap 7.40 ( https://nmap.org ) at 2017-04-21 17:06 EDT
Nmap scan report for aregsky1 (<omitted>)
Host is up (0.028s latency).
PORT     STATE    SERVICE
7778/tcp filtered interwise

Nmap done: 1 IP address (1 host up) scanned in 0.41 seconds

I'm not really sure what could possibly do this. The system may reboot if the upgrade required it, but immediately before that task is:

- name: wait for system to be up
  local_action: wait_for host="{{ inventory_hostname }}" port=26 timeout=3600

Any help would be appreciated,

Brian