Thread safety

[Victor Stinner]

Hi,

On IRC, someone told me that it took him hours to understand that
asyncio.Queue is not thread-safe and he expected asyncio.Queue to be
thread-safe. I modified the asyncio documentation to mention in almost
all classes that asyncio classes are not thread-safe.

I didn't touch the doc of lock classes (ex: asyncio.Lock), because I
don't know if they are thread safe or not. Since locks should be used
with "yield from lock", it's not easy to combine them with
loop.call_soon_threadsafe().

Maybe we should modify asyncio.Queue and asyncio.Lock to make them thread-safe?

What do you think?

Victor

[Ludovic Gasc]

On Wednesday, February 25, 2015 at 2:43:54 PM UTC+1, Victor Stinner wrote:

Maybe we should modify asyncio.Queue and asyncio.Lock to make them thread-safe?

For me, it's the ideal scenario, but:

1. I have no idea if it's complicated

2. If it should reduce performances when you don't use threads

[Andrew Svetlov]

-1

I'm OK with current asyncio locks and queue implementation, they are
not-threadsafe by design explicitly.

For locks you can use `yield from lock.acquire()` (requires nested
function definition for passing into `loop.call_soon_threadsafe()`)
and `lock.release()`.

I personally don't like to add strictness for asyncio implementations:
that may lead to performance degradations and may make problems for
writing non-standard event loops.

--
Thanks,
Andrew Svetlov

[Ben Darnell]

On Wed, Feb 25, 2015 at 8:43 AM, Victor Stinner <victor....@gmail.com> wrote:

Hi,

On IRC, someone told me that it took him hours to understand that
asyncio.Queue is not thread-safe and he expected asyncio.Queue to be
thread-safe. I modified the asyncio documentation to mention in almost
all classes that asyncio classes are not thread-safe.

I didn't touch the doc of lock classes (ex: asyncio.Lock), because I
don't know if they are thread safe or not.

Whether the current implementations happen to be thread-safe or not, I don't think they are intended to guarantee thread-safety.

 

Since locks should be used
with "yield from lock", it's not easy to combine them with
loop.call_soon_threadsafe().

Queues are also called with "yield from" in many cases (put() to a bounded queue, or get() from any queue); the only queue methods that could trivially be made thread-safe are put_nowait() and get_nowait() (and the latter is not very useful without some way to wait for updates).

Supporting both "yield from" and threaded-style use of a queue would require either separate methods for use from other threads or a more complicated Future implementation that could be either yielded from or blocked on (this would have performance implications and also remove one of the safeguards that prevents misuse of asyncio.Future today).

 

Maybe we should modify asyncio.Queue and asyncio.Lock to make them thread-safe?

-1 on modifying asyncio.Lock. You can use a threading.Lock in an asyncio program to protect data that is used by other threads (and if you're concerned about blocking, make your critical sections smaller).

A hybrid thread-safe and asynchronous queue is more interesting. I'm on the fence about whether it makes sense to modify asyncio.Queue or make it a separate class, but it does seem like something that would be useful to have. It might be as simple as adding a couple of methods like this:

  def blocking_get(self):

    """Remove and return an item from the queue. If queue is empty, wait until an item is available.

    This method is **not** a coroutine and should not be called from the event loop thread.

    It is safe to call from any other thread.

    """"

    future = concurrent.futures.Future()

    @asyncio.coroutine

    def cb():

      result = yield from self.get()

      future.set_result(result)

    self._loop.call_soon_threadsafe(cb)

    return future.result()

In fact, this wrapper could be made generic to let you call any asyncio coroutine from another thread and block waiting for the results. 

-Ben

[Victor Stinner]

2015-02-25 16:23 GMT+01:00 Andrew Svetlov <andrew....@gmail.com>:
> I'm OK with current asyncio locks and queue implementation, they are
> not-threadsafe by design explicitly.

Ok, it expected this answer. It is a more a documentation issue than a
design issue :-)

> For locks you can use `yield from lock.acquire()` (requires nested
> function definition for passing into `loop.call_soon_threadsafe()`)
> and `lock.release()`.

If you use a nested function and call_soon_threadsafe(), you can use
"with (yield from lock):".

I will add "This class is not thread safe." to all lock classes.

But it would help to have a short examples showing how to:
- share a queue between multiple coroutines
- use a lock in coroutines

And maybe also examples with multiple threads:

- "share" a queue between multiple coroutines running in different
threads (in different event loops)

Victor

[Guido van Rossum]

Yes, this is all by design. The *only* operation in all of asyncio that's thread-safe is call_soon_threadsafe(). I don't see a reason to add more hybrid methods, but I do think we should provide an example about how a thread and an event loop can communicate using some kind of queue. (There are a few approaches here -- e.g. you could use a regular thread-safe queue.Queue and use run_in_executor() to wrap the q.get() call, or you could use an asyncio Queue and make a thread-safe wrapper to put something into it.)

--

--Guido van Rossum (python.org/~guido)

[Antoine Pitrou]

On Wed, 25 Feb 2015 10:42:23 -0500
Ben Darnell <b...@bendarnell.com> wrote:
> > Maybe we should modify asyncio.Queue and asyncio.Lock to make them
> > thread-safe?
> >
>
> -1 on modifying asyncio.Lock. You can use a threading.Lock in an asyncio
> program to protect data that is used by other threads (and if you're
> concerned about blocking, make your critical sections smaller).

That will stop all coroutines, though, not only the one waiting for the
lock.
(not that I think trying to synchronize between threads and coroutines
is a terrific idea)

Regards

Antoine.

[Ben Darnell]

On Wed, Feb 25, 2015 at 12:17 PM, Antoine Pitrou <soli...@pitrou.net> wrote:

On Wed, 25 Feb 2015 10:42:23 -0500
Ben Darnell <b...@bendarnell.com> wrote:
> > Maybe we should modify asyncio.Queue and asyncio.Lock to make them
> > thread-safe?
> >
>
> -1 on modifying asyncio.Lock. You can use a threading.Lock in an asyncio
> program to protect data that is used by other threads (and if you're
> concerned about blocking, make your critical sections smaller).

That will stop all coroutines, though, not only the one waiting for the
lock.

Correct; my claim is that this is acceptable for those cases where sharing mutable state between threads and coroutines is appropriate (which I agree should be discouraged in general - I'd like for queues to be the most attractive solution to this problem). Coroutines by their nature block each other when they do anything at all; as long as you don't do too much while holding the lock I would expect to incur less blocking by waiting on a lock than by going through another iteration of the coroutine machinery.

-Ben

[Victor Stinner]

Hi,

2015-02-25 16:23 GMT+01:00 Andrew Svetlov <andrew....@gmail.com>:

> I'm OK with current asyncio locks and queue implementation, they are
> not-threadsafe by design explicitly.

Ok, so it's just a documentation issue. It's now fixed:
https://hg.python.org/cpython/rev/366e3ad5e3bd

Victor

[Tin Tvrtković]

I was contemplating this just now.

Let's say you want to stream a (huge) file from disk. You can fire off a reader in a separate thread, using call_in_executor. Now this reader needs to be able to feed the client chunks of the file as they are read, and also you'd want backpressure. A thread-safe and async-aware queue (with limited capacity) seems like the simplest solution by far. It'd be great if something like this was available from the standard library; I'm not entirely sure how to create one myself.

What would the API of this queue look like? Would you need a put/get version that blocks the thread, and a separate put/get version that blocks a task?

On Wednesday, February 25, 2015 at 2:43:54 PM UTC+1, Victor Stinner wrote:

[Andrew Svetlov]

Published [janus](https://pypi.python.org/pypi/janus/0.0.1) -- thread-safe and async-aware queue with limited capacity, as you requested :)

[Ludovic Gasc]

Thank you Andrew for that, interesting piece of code ;-)

--

Ludovic Gasc (GMLudo)

http://www.gmludo.eu/