host_add uniqueness: name, ansible_host & ansible_ssh_extra_args ?

[alex...@delvelabs.ca]

Hi,

I use docker to run (network namespaced) VPN...
I have to use a custom ProxyCommand, per VPN, to access the other side with Ansible, so far so good...

So I'm using Ansible to find the right container and then add_host to set it up (as everything is dynamic) with:

  - name: Add VPN host
    add_host:
      name: "{{ item }}"
      groups: vpn
      ansible_ssh_extra_args: "-o StrictHostKeyChecking=no -o ProxyCommand=\"ssh -q user@{{ inventory_hostname }} docker exec -i {{ item }} nc %h %p\""
      ansible_ssh_user: "user"
      ansible_host: "10.0.0.1"
    with_items: "{{ vpn_container_names }}"

This works great with ssh directly and also "seems" to work with ansible, but I only get to access the first container with subsequent loops.
It is as if adding them with the same ansible_host did nothing, even though the name (container_xyz_1) was different...

I would have thought that "name" + "ansible_host" + ssh options would be used as the unique pair, but it seems something like this is not sufficiently unique in a dynamic inventory context:

# name= & ansible_host= (what it would look like in a file from my understanding)
container_001122334455_1 ansible_host=10.0.0.1 ansible_ssh_extra_args=... ansible_ssh_user=user
container_667788990011_1 ansible_host=10.0.0.1 ansible_ssh_extra_args=... ansible_ssh_user=user
etc.

Even though they all have the ProxyCommand that IS unique and works great on its own:
ssh -o StrictHostKeyChecking=no -o ProxyCommand="ssh -q us...@machine.example.com docker exec -i container_001122334455_1 nc %h %p" us...@10.0.0.1
ssh -o StrictHostKeyChecking=no -o ProxyCommand="ssh -q us...@machine.example.com docker exec -i container_667788990011_1 nc %h %p" us...@10.0.0.1
...

So am I correct in assuming that host uniqueness is only based on ansible_host and doesn't care about the host's "name" if ansible_host is set?
if so, should it be based on all the relevant variables: name, host & options (ansible_ssh_extra_args, etc.) considering different virtualization cases or remote network namespaces with jump hosts with similar IP space?

Or do I simply have a bug in my playbook?
(which would actually be great news as I'm not sure how to go around this issue otherwise!)

Thanks a lot, :)
Alex

[alex...@delvelabs.ca]

digging deeper, things get stranger...
With -vvv I see this, running a shell command to get the IP (anonymized):

ssh -C -q -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=user -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o 'ProxyCommand=ssh -q us...@machine.example.com docker exec -i container_001122334455_1 nc %h %p' -o ControlPath=/home/user/.ansible/cp/ansible-ssh-%h-%p-%r -tt 10.0.0.1

if I use it myself, I get to the right machine, but clearly ansible is outputing the ip of the first machine in the list which isn't this one. So -vvv is lying about what it's actually using to connect.

There is some ssh persistance that could be at play here, digging deeper I see:
~/.ansible/cp/ansible-ssh-10.0.0.1-22-user

which isn't unique enough, and by the look of it there isn't a random argument available directly in ssh...
If I can pin it per host added with add_host, I could prepend the container name to this which would save the day.

This really sound like a major bug with Ansible for anyone having overlapping address space in different networks...

(currently trying with 2.1.2.0)

Alex

[alex...@delvelabs.ca]

Indeed, this can fix it for now (not pretty) in my add_host loop by prepending the ControlPath with the unique container ID (hopefully good enough for now):

...
      ansible_ssh_extra_args: "-o StrictHostKeyChecking=no -o ProxyCommand=\"ssh -q user@{{ inventory_hostname }} docker exec -i {{ item }} nc %h %p\" -o ControlPath=~/.ansible/cp/{{ item }}-ssh-%h-%p-%r"
      ansible_scp_extra_args: "-o StrictHostKeyChecking=no -o ProxyCommand=\"ssh -q user@{{ inventory_hostname }} docker exec -i {{ item }} nc %h %p\" -o ControlPath=~/.ansible/cp/{{ item }}-ssh-%h-%p-%r"
      ansible_sftp_extra_args: "-o StrictHostKeyChecking=no -o ProxyCommand=\"ssh -q user@{{ inventory_hostname }} docker exec -i {{ item }} nc %h %p\" -o ControlPath=~/.ansible/cp/{{ item }}-ssh-%h-%p-%r"
...

Feels like the hash should be based on all the options, not just ip-port-user :)