SemLock / No space left on device

[David Blackburn]

This is not a Ansible problem but its stopping me from using Ansible on my Mac, if I reboot the system the problem goes away for about a day.

I am unable to find much information about this problem this is the most relevant information I could find.  https://stackoverflow.com/questions/28788319/no-space-left-on-device-when-creating-semaphore

Any help would be great!

The error I get from Ansible is....

ansible-playbook playbook.yml -vvv

Using /etc/ansible/ansible.cfg as config file

 [WARNING]: provided hosts list is empty, only localhost is available

 

ERROR! Unexpected Exception: [Errno 28] No space left on device

the full traceback was:

 

Traceback (most recent call last):

  File "/usr/local/bin/ansible-playbook", line 103, in <module>

    exit_code = cli.run()

  File "/Library/Python/2.7/site-packages/ansible/cli/playbook.py", line 154, in run

    pbex = PlaybookExecutor(playbooks=self.args, inventory=inventory, variable_manager=variable_manager, loader=loader, options=self.options, passwords=passwords)

  File "/Library/Python/2.7/site-packages/ansible/executor/playbook_executor.py", line 59, in __init__

    self._tqm = TaskQueueManager(inventory=inventory, variable_manager=variable_manager, loader=loader, options=options, passwords=self.passwords)

  File "/Library/Python/2.7/site-packages/ansible/executor/task_queue_manager.py", line 101, in __init__

    self._final_q = multiprocessing.Queue()

  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/__init__.py", line 218, in Queue

    return Queue(maxsize)

  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/queues.py", line 63, in __init__

    self._rlock = Lock()

  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/synchronize.py", line 147, in __init__

    SemLock.__init__(self, SEMAPHORE, 1, 1)

  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/synchronize.py", line 75, in __init__

    sl = self._semlock = _multiprocessing.SemLock(kind, value, maxvalue)

OSError: [Errno 28] No space left on device

Running the following using any version of Python returns the following error.

>python -c 'import multiprocessing; multiprocessing.Semaphore()'

Traceback (most recent call last):

  File "<string>", line 1, in <module>

  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/__init__.py", line 197, in Semaphore

    return Semaphore(value)

  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/synchronize.py", line 111, in __init__

    SemLock.__init__(self, SEMAPHORE, value, SEM_VALUE_MAX)

  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/synchronize.py", line 75, in __init__

    sl = self._semlock = _multiprocessing.SemLock(kind, value, maxvalue)

OSError: [Errno 28] No space left on device

[David Blackburn]

I partially found the answer to my problem and hopefully the answer will be of use to somebody.  The problem is I am running out of semaphores and I was using at the system V tools to view semaphores and not the POSIX command set.

Unfortunately I am still unable to figure out what is using the semaphores and the usage continues to grow the default is 1000 and I have had to raise the value twice.

sudo sysctl -w kern.posix.sem.max=20000

sudo zprint semaphores

Dave