threading.Lock in gevent.wsgi.WSGIServer, threading.Semaphore vs gevent.lock.Semaphore

[Sergey]

Hello,

I'm developing Flask app, processed by gevent.wsgi.WSGIServer.
If I need to isolate some pieces of code inside request handlers, can I use threading.Lock?
If I need semaphore, must I use threading.Semaphore or gevent.lock.Semaphore?
What is the difference between them?

Thank you!

[Jason Madden]

If you use `threading.Lock` in a gevent application, you will stop all other greenlets when one greenlet tries to take a lock already held by some other greenlet. This is Bad because it reduces the throughput of the application. Worse, it can actually completely deadlock (freeze) the application because the greenlet that holds the lock won't be able to run to release it! (The standard library versions only work at the thread level, but all greenlets run in a single thread.)

Long story short, you either need to explicitly use the gevent equivalents, *or* you need to use gevent.monkey to patch the system very early on so that the standard versions are transparently replaced with their gevent equivalents.

Try this code:

import gevent
import threading

lock = threading.Lock()

def task1():
lock.acquire()
print("Task 1: locked the lock, going to sleep.")
gevent.sleep(5)
print("Task 1: wake from sleep, unlocking the lock")
lock.release()

def task2():
gevent.sleep(1)
print("Task 2: Wake from sleep, locking the lock")
lock.acquire()
print("Task 2: Locked the lock")
print("Task 2: Unlocking the lock")
lock.release()

t1 = gevent.spawn(task1)
t2 = gevent.spawn(task2)

gevent.joinall([t1, t2])

Notice how it never finishes. I have to kill it with Ctrl-C:

$ python /tmp/threading_lock_test.py
Task 1: locked the lock, going to sleep.
Task 2: Wake from sleep, locking the lock
^CTraceback (most recent call last):
File "//src/gevent/greenlet.py", line 536, in run
result = self._run(*self.args, **self.kwargs)
File "/tmp/threading_lock_test.py", line 17, in task2
lock.acquire()
KeyboardInterrupt

Now change threading.Lock to gevent.threading.Lock. Notice how everything cooperates as expected:

$ python /tmp/gevent_lock_test.py
Task 1: locked the lock, going to sleep.
Task 2: Wake from sleep, locking the lock
Task 1: wake from sleep, unlocking the lock
Task 2: Locked the lock
Task 2: Unlocking the lock