ansible setup subcall to lsblk timing out on Xen mount

[Michael Bubb]

Using this version of Ansible

ansible --version
ansible 2.1.1.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
[

On a Softlayer VM 

Linux staging 2.6.32-573.8.1.el6.x86_64 #1 SMP Tue Nov 10 18:01:38 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

with these mount points 

[root@staging ~]# lsblk
NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
xvdc    202:32   0  150G  0 disk
└─xvdc1 202:33   0  150G  0 part /opt
xvda    202:0    0   25G  0 disk
├─xvda1 202:1    0  256M  0 part /boot
└─xvda2 202:2    0 24.8G  0 part /
xvdb    202:16   0    2G  0 disk
└─xvdb1 202:17   0    2G  0 part [SWAP]

The setup play in a previously working playbook failed like this

[setup]
fatal: [staging]: FAILED! => {"changed": false, "cmd": "/bin/lsblk -ln --output UUID /dev/xvda1", "failed": true, "msg": "Traceback (most recent call last):\n  File \"/tmp/ansible_a4NBhA/ansible_modlib.zip/ansible/module_utils/basic.py\", line 2093, in run_command\n    cmd = subprocess.Popen(args, **kwargs)\n  File \"/usr/lib64/python2.6/subprocess.py\", line 642, in __init__\n    errread, errwrite)\n  File \"/usr/lib64/python2.6/subprocess.py\", line 1223, in _execute_child\n    data = _eintr_retry_call(os.read, errpipe_read, 1048576)\n  File \"/usr/lib64/python2.6/subprocess.py\", line 462, in _eintr_retry_call\n    return func(*args)\n  File \"/tmp/ansible_a4NBhA/ansible_modlib.zip/ansible/module_utils/facts.py\", line 101, in _handle_timeout\n    raise TimeoutError(error_message)\nTimeoutError: Timer expired\n", "rc": 257}

I confirmed with an adhoc ansible setup

ansible staging -m setup
staging | FAILED! => {
    "changed": false,
    "cmd": "/bin/lsblk -ln --output UUID /dev/xvda1",
    "failed": true,
    "msg": "Traceback (most recent call last):\n  File \"/tmp/ansible_HuNPPI/ansible_modlib.zip/ansible/module_utils/basic.py\", line 2093, in run_command\n    cmd = subprocess.Popen(args, **kwargs)\n  File \"/usr/lib64/python2.6/subprocess.py\", line 642, in __init__\n    errread, errwrite)\n  File \"/usr/lib64/python2.6/subprocess.py\", line 1223, in _execute_child\n    data = _eintr_retry_call(os.read, errpipe_read, 1048576)\n  File \"/usr/lib64/python2.6/subprocess.py\", line 462, in _eintr_retry_call\n    return func(*args)\n  File \"/tmp/ansible_HuNPPI/ansible_modlib.zip/ansible/module_utils/facts.py\", line 101, in _handle_timeout\n    raise TimeoutError(error_message)\nTimeoutError: Timer expired\n",
    "rc": 257
}

On the server itself this command worked fine

/bin/lsblk -ln --output UUID /dev/xvda1

And there was no obvious issues in syslogs

I then ran this and it worked

ansible staging -m setup -a 'gather_subset=network'
staging | SUCCESS => {
    "ansible_facts": {

Then I unmouned and remounted /boot (/dev/xvda1). The subcall then  failed on /dev/xvdc1

A reboot of the machine fixed the issue.

I am curious why the Ansible call to lsblk did not work but the native call did.