Asynchronous programming

[Steven D'Aprano]

The latest versions of Python are introducing new keywords for asynchronous
programming, async and await. See PEP 492:

https://www.python.org/dev/peps/pep-0492/

Is there a good beginner's tutorial introducing the basics of asynchronous
programming? Starting with, why and where would you use it?

I've read "What colour is your function" and it isn't quite jelling for me.

http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/


How is this the same as, or different from, event-based programming? I'm
familiar with programming in an event-based language where the interpreter
itself provides an event loop and dispatches messages to your objects:

- I define objects such as buttons, text fields, etc., and give them methods
("handlers") which handle certain messages such as "mouseUp", etc.;

- the interpreter runs in a loop, firing off messages in response to the
user's actions;

- there's a message passing hierarchy, whereby messages are first received
by (let's say) the button the user clicked on, if not handled by a mouseUp
method it is passed on to the next object in the hierarchy (say, the
window), and then finally to the interpreter itself, at which point it
either ignores the message or raises an exception.


If I'm using async and await in Python, where's the event loop? What are the
messages, and where are they sent? Or am I on the wrong track altogether?




--
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

[Terry Reedy]

You might be able to glean something from the succession of files I
uploaded to
https://bugs.python.org/issue27546
Integrate tkinter and asyncio (and async)

I started with just mixing tk and asyncio callbacks. After some
struggle with similar question as you ask above, I ended up with more or
less callback free code using async def and async for. Once I got over
the hump, I rather like it.

--
Terry Jan Reedy

[Chris Angelico]

On Thu, Aug 11, 2016 at 1:53 PM, Steven D'Aprano
<steve+...@pearwood.info> wrote:
> How is this the same as, or different from, event-based programming? I'm
> familiar with programming in an event-based language where the interpreter
> itself provides an event loop and dispatches messages to your objects:
>
> - I define objects such as buttons, text fields, etc., and give them methods
> ("handlers") which handle certain messages such as "mouseUp", etc.;
>
> - the interpreter runs in a loop, firing off messages in response to the
> user's actions;
>
> - there's a message passing hierarchy, whereby messages are first received
> by (let's say) the button the user clicked on, if not handled by a mouseUp
> method it is passed on to the next object in the hierarchy (say, the
> window), and then finally to the interpreter itself, at which point it
> either ignores the message or raises an exception.
>
>
> If I'm using async and await in Python, where's the event loop? What are the
> messages, and where are they sent? Or am I on the wrong track altogether?

There'll be one somewhere, probably at some top-level main loop. Async
programming isn't usually compared with events *as such*, but with
callbacks. Consider these three ways of doing a database transaction:

def synchronous(id):
trn = conn.begin_transaction()
trn.execute("select name from people where id=%d", (id,))
name, = trn.fetchone()
trn.execute("update people set last_seen=now() where id=%d", (id,))
trn.commit()
return name

def callbacks_1(cb, id):
conn.begin_transaction(callbacks_2, cb, id)
def callbacks_2(trn, cb, id):
trn.execute("select name from people where id=%d", (id,),
callbacks_3, cb, id)
def callbacks_3(trn, cb, id):
trn.fetchone(callbacks_4, cb, id)
def callbacks_4(trn, data, cb, id):
name, = data
trn.execute("update people set last_seen=now() where id=%d",
(id,), callbacks_5, cb, name)
def callbacks_5(trn, cb, name):
trn.commit(callbacks_6, cb, name)
def callbacks_6(trn, cb, name):
cb(name)

def asynchronous(id):
trn = yield from conn.begin_transaction()
yield from trn.execute("select name from people where id=%d", (id,))
name, = yield from trn.fetchone()
yield from trn.execute("update people set last_seen=now() where
id=%d", (id,))
yield from trn.commit()
return name

Change this last one to "async def" and change the "yield from"s into
"await", and you get the new syntax, but it's still built on top of
generators. Basically, async code works by stopping execution in the
middle of a logical operation, and then resuming it later. (The
callbacks example could be improved on some by using closures, and in
JavaScript, where this is really common, you can use anonymous
functions. I still prefer the last form though.) Every time you hit an
"await", you call into that function and see what it says - in this
case, it's going to "yield" somewhere, sending back some sort of
awaitable thing, eg a half-done database operation. When that
operation completes, you go into the next step of the process.

How does this align with event handling? Not very well. Normally event
handling means non-linear execution - for instance, button clicks on a
GUI aren't going to step nicely through a progression. But the two
work well together; if you have a primary event loop that starts out
by looking for GUI events, you could have something like the above
asynchronous function, and it'll give control back to the main loop
while the database is working - thus making the GUI responsive again.
You can mess around with the front end while the back end does its
work.

Hope that helps some.

ChrisA

[Paul Rubin]

Steven D'Aprano <steve+...@pearwood.info> writes:
> Is there a good beginner's tutorial introducing the basics of asynchronous
> programming? Starting with, why and where would you use it?

You might look at some node.js tutorials since there are about a
gazillion of them and some of them must be good. Also twistedmatrix.org
for an older Python package of that sort.

The basic reason to do it is it lets you serve a lot of concurrent i/o
channels (network connections, say) without using threads. If you want
to read a packet, you launch a non-blocking read that returns
immediately, and then transfer control (often through some
behind-the-scenes magic) to an event loop that dispatches back to you
after your read request completes, through either a callback or a
coroutine jump depending on which framework you're using. In Python
that gets you much higher performance than threads, plus you avoid the
usual bunch of hazards that parents used to scare their kids with about
multi-threaded programming.

> How is this the same as, or different from, event-based programming?

The basic characteristic of asynchronous programming is that it involves
changing all your usual blocking i/o calls to non-blocking ones, so your
program can keep running as soon as your request is started. That's not
always the case with event-based programming in general, where event
handlers might do i/o. The basic problem you face is the inversion of
control needed to get in and out of the event loop, compared to the
sequential blocking i/o that you'd use in a threaded or non-concurrent
program.

The new thing in Python is asyncio, which uses coroutines instead of
callbacks to switch between event handlers. It's asynchronous
programming either way, but coroutines can be seen as a cleaner
implementation technique depending on your POV. Coroutines in the form
of "yield from" first appeared in Python 3.??? and became more fully
integrated with the new async and await keywords. Before that, async
concurrency in Python was done using callback schemes. Twisted used to
work like that (I don't know if it still does). I hated it but I was in
a minority.

> If I'm using async and await in Python, where's the event loop? What
> are the messages, and where are they sent? Or am I on the wrong track
> altogether?

It's supplied by the asyncio package. Btw, David Beazely's article
about Python coroutines is very good, though it was written before the
new keywords were added.

[Chris Angelico]

On Thu, Aug 11, 2016 at 3:06 PM, Paul Rubin <no.e...@nospam.invalid> wrote:
> The basic reason to do it is it lets you serve a lot of concurrent i/o
> channels (network connections, say) without using threads. If you want
> to read a packet, you launch a non-blocking read that returns
> immediately, and then transfer control (often through some
> behind-the-scenes magic) to an event loop that dispatches back to you
> after your read request completes, through either a callback or a
> coroutine jump depending on which framework you're using. In Python
> that gets you much higher performance than threads, plus you avoid the
> usual bunch of hazards that parents used to scare their kids with about
> multi-threaded programming.
>

Hmm. I'm not sure about that last bit. In order to properly support
asynchronous I/O and the concurrency and reentrancy that that implies,
you basically need all the same disciplines that you would for
threaded programming - no shared mutable state, keep your locals
local, etc. The only difference is that context switches are coarser
than they are in threading (and remember, CPython already refuses to
switch threads in the middle of one bytecode operation, so most of the
worst horrors can't happen even there). But maybe I'm too comfortable
with threads. It's entirely possible; my first taste of reentrancy was
interrupt handling in real mode 80x86 assembly language, and in
comparison to that, threads are pretty benign!

ChrisA

[Paul Rudin]

Steven D'Aprano <steve+...@pearwood.info> writes:

>

> Is there a good beginner's tutorial introducing the basics of asynchronous
> programming? Starting with, why and where would you use it?

You could do worse than watch Dave Beazley's pycon talk:
https://www.youtube.com/watch?v=lYe8W04ERnY

[Christian Gollwitzer]

Am 11.08.16 um 05:53 schrieb Steven D'Aprano:

> How is this the same as, or different from, event-based programming? I'm
> familiar with programming in an event-based language where the interpreter
> itself provides an event loop and dispatches messages to your objects

I'll just comment on that one. Asynchronous programming is event based
programming. In fact if you drop down to the C level, it works the same
- you get callbacks when new data arives over a socket, or when a timer
triggers. asyncio is just a way that makes writing stateful callback
handlers easier, in the same way that generators are easier to write
using "yield" than as a function with a global variable.

In typical GUI code, there are usually not that many places qhere ou
have sequential code. A simple exmaple might be a counter. Using asyncio
and a properly integrated GUI toolkit, you could write it as (pseudo-code)

async def counter():
for i in range(10):
button.settext("click me %d"%i)
await button.click()

button.disable()
messageBox("You reached the end!")

Writing that same thing in callback code to button.on_click() is
obviously less fun and feels inverted.

Christian

[Christian Gollwitzer]

Am 11.08.16 um 06:38 schrieb Terry Reedy:

> You might be able to glean something from the succession of files I
> uploaded to
> https://bugs.python.org/issue27546
> Integrate tkinter and asyncio (and async)
>
> I started with just mixing tk and asyncio callbacks. After some
> struggle with similar question as you ask above, I ended up with more or
> less callback free code using async def and async for. Once I got over
> the hump, I rather like it.

Interesting discussion! Tcl itself acquired coroutines some years ago
(in 8.6) and people in this community are very excited with async based
programming.

I'm convinced that it is possible to integrate Tcl's event loop with
asyncio's loop without regular update polling. This might require a
patch to Tkinter at the C level. For example, an easy way is to put
Tcl/Tk in it's own thread. The Tcl event loop can be woken up from a
different thread by the TclThreadQueueEvent() and TclTheradAlert() API
calls. Basically, you would spawn a new thread (from the C level), when
loading Tkinter, which loads Tk and sits in a Tcl event loop ("vwait
forever"). tk.eval() and tk.call() would be redefined to post events
using the aforementioned functions. When a Tcl callback arrives, it
would send an event to asyncio in the main thread - no idea how this
works, though. In the end, it would also allow to use Tkinter from
different Python threads, because all calls would be rooted to the one
instance only.

Christian

[Paul Rubin]

Christian Gollwitzer <auri...@gmx.de> writes:
> I'm convinced that it is possible to integrate Tcl's event loop with
> asyncio's loop without regular update polling. This might require a
> patch to Tkinter at the C level. For example, an easy way is to put

> Tcl/Tk in it's own thread. ...

I did something like that a while back, putting tkinter in its own
thread but also using its .after() method to wake up the tkinter loop
every 50 msec or something like that. I didn't notice any significant
cpu load or UI lag from that, and it was on a fairly slow machine (300
mhz ARM, I think). The tkinter thread received callbacks from tkinter
and passed them on to the application's main dispatch loop through a
normal Python queue. So there was no need to patch the tkinter C code.

[Marko Rauhamaa]

Chris Angelico <ros...@gmail.com>:

> Hmm. I'm not sure about that last bit. In order to properly support
> asynchronous I/O and the concurrency and reentrancy that that implies,
> you basically need all the same disciplines that you would for
> threaded programming

Correct. Python's asyncio closely mirrors threads in its API and thought
model. An async is like a thread except that you know the control will
only be relinquished at an "await" statement. Thus, you probably need
far less locking.

One visible difference between asyncs and threads is the fact that
threads don't need a special "await" construct. Instead, ordinary
function calls support context switching deep down in the Python VM and
the operating system. Each "await" explicitly marks a state in a state
machine. Whenever you turn a nonblocking function into a blocking one,
you will need to change the function call to an await statement.

Asyncs have a couple of distinct advantages over threads:

* You can await multiple events in a single statement allowing for
multiplexing. Threads can only wait for a single event.

* Asyncs can be canceled; in general, threads can't.

> But maybe I'm too comfortable with threads. It's entirely possible; my
> first taste of reentrancy was interrupt handling in real mode 80x86
> assembly language, and in comparison to that, threads are pretty
> benign!

I steer clear of threads if I can because:

* No multiplexing.

* No cancellation.

* Deadlocks and/or race conditions are virtually guaranteed.

* Error symptoms are highly nonlocal making debugging very tricky.

Instead, I strongly prefer asynchronous programming and multiprocessing.

However, I'm afraid Python is following an unfortunate trend with
asyncio. Deep down concurrent programming involves finite state
machines. State machines can be expressed in many ways; traditionally,
you have SDL diagrams, state/event matrices etc. Asyncio marks states
with inline "await" keywords. Toy examples look really enticing and
readable compared with the more traditional implementation methods.
However, real-world code can quickly become spaghetti as the dozens of
cells of the state/event matrix are implemented.

My favorite asynchronous development model is the "callback hell," where
each cell of the state/event matrix is represented by a method (or at
least a switch case in C) and each state has an explicit name in the
source code. The resulting code is still hard to read and hard to get
right, but that complexity is unavoidable because reality just is that
complex.


Marko

[Chris Angelico]

On Thu, Aug 11, 2016 at 5:55 PM, Marko Rauhamaa <ma...@pacujo.net> wrote:
> Chris Angelico <ros...@gmail.com>:
>
>> Hmm. I'm not sure about that last bit. In order to properly support
>> asynchronous I/O and the concurrency and reentrancy that that implies,
>> you basically need all the same disciplines that you would for
>> threaded programming
>
> Correct. Python's asyncio closely mirrors threads in its API and thought
> model. An async is like a thread except that you know the control will
> only be relinquished at an "await" statement. Thus, you probably need
> far less locking.

I've never needed ANY explicit locking in a threaded Python or Pike
program, and only rarely in C. But then, I'm one of those people who
finds threaded programming relatively easy.

> Asyncs have a couple of distinct advantages over threads:
>
> * You can await multiple events in a single statement allowing for
> multiplexing. Threads can only wait for a single event.

Err, that's kinda the point of select() and friends. There are some
types of event that you can't select() on, but those also tend not to
work perfectly with async programming either - because, guess what,
event loops for async programs are usually built on top of select.

> * Asyncs can be canceled; in general, threads can't.

Sadly, it's not that simple. Some async actions can be cancelled,
others cannot. At best, what you're saying is that *more* things can
be cancelled if they were started asynchronously.

>> But maybe I'm too comfortable with threads. It's entirely possible; my
>> first taste of reentrancy was interrupt handling in real mode 80x86
>> assembly language, and in comparison to that, threads are pretty
>> benign!
>
> I steer clear of threads if I can because:
>
> * No multiplexing.
>
> * No cancellation.

See above

> * Deadlocks and/or race conditions are virtually guaranteed.

Definitely not. There are EASY ways to avoid deadlocks, and race
conditions are only a problem when you have some other bug. Yes,
threading bugs can be harder to debug; but only when you've violated
the other principles.

> * Error symptoms are highly nonlocal making debugging very tricky.

But at least you have consistent and simple tracebacks. Callback model
code generally lacks that. (That's one of the advantages of Python's
generator-based async model - although it's not perfect.)

> Instead, I strongly prefer asynchronous programming and multiprocessing.

Multiprocessing is *exactly* as complicated as multithreading, with
the additional overhead of having to pass state around explicitly. And
if you're not getting the overhead of passing state around, you
wouldn't have the complexity of shared state in threading, ergo both
models would be identical.

> My favorite asynchronous development model is the "callback hell," where
> each cell of the state/event matrix is represented by a method (or at
> least a switch case in C) and each state has an explicit name in the
> source code. The resulting code is still hard to read and hard to get
> right, but that complexity is unavoidable because reality just is that
> complex.

Wow. Are you for real, or are you trolling us? You actually *enjoy*
callback hell?

Give me yield-based asyncio any day.

ChrisA

[Steven D'Aprano]

On Thursday 11 August 2016 16:21, Christian Gollwitzer wrote:

> In typical GUI code, there are usually not that many places qhere ou
> have sequential code. A simple exmaple might be a counter. Using asyncio
> and a properly integrated GUI toolkit, you could write it as (pseudo-code)
>
> async def counter():
> for i in range(10):
> button.settext("click me %d"%i)
> await button.click()
> button.disable()
> messageBox("You reached the end!")
>
> Writing that same thing in callback code to button.on_click() is
> obviously less fun and feels inverted.

I don't know how you would write it in callback fashion, but the above seems
inside out to me. It feels like COMEFROM instead of GOTO, if you understand the
analogy. Rather than

execute code in a linear fashion
until you reach a yield (GOTO), then transfer control *out*

it's

execute code in a linear fashion
control transferred *in* from somewhere else (COMEFROM)


This is how I would expect to write something like that in an event-driven way:

def mouseUp(self):
# assume self.counter is initially set to 0
self.settext("Click Me %d" % self.counter)
self.counter += 1
if self.counter == 10:
self.disable()
messageBox("You have reached the end!")


which feels natural to me. Processing of each click is independent of any other
click, and there's no non-linear transfer of control. The user clicks, the
mouseUp method is called, it runs, and it is done. Whatever persistent state
there is is part of the button itself.

I don't know whether you would call that a callback. I suppose it could be, in
the sense that you might say:

button.set_mouseup_function(mouseUp)

but I'm used to thinking of it as a property of the button.



--
Steve

[Chris Angelico]

On Thu, Aug 11, 2016 at 7:45 PM, Steven D'Aprano
<steve+comp....@pearwood.info> wrote:
> I don't know whether you would call that a callback. I suppose it could be, in
> the sense that you might say:
>
> button.set_mouseup_function(mouseUp)
>
> but I'm used to thinking of it as a property of the button.

"Callback" simply means "function that someone else calls". In fact,
most dunder methods could be described as callbacks.

ChrisA

[Paul Rudin]

A callback is normally a callable you pass to some other function with
the expectation that it will be invoked in certain circumstances.


That's not really the same thing as something someone else calls: any
function that's part of a published api is intended to be called by
someone else; but that doesn't really make it a callback.

[Marko Rauhamaa]

Chris Angelico <ros...@gmail.com>:

To me, the key is that the code should closely mirror the classic FSM
formalisms:

<URL: http://images.slideplayer.com/14/4228155/slides/slide_21.jpg>

<URL: http://www.detewe.ru/q931/image/10.gif>

<URL: https://people.ece.cornell.edu/land/courses/ece4760/FinalPr
ojects/s2011/pss83/Website/Time%20System/instruction_matrix.PNG>


It *would* be possible to write asyncio code the same way where you
would represent states as asyncs and express state transitions as:

await self.ESTABLISHING()

Alas, that would lead to unlimited async recursion, which, I'm pretty
sure, would overflow Python's execution stack.


Marko

[Steven D'Aprano]

Instructions unclear, poked myself in the eye with a sharp stick.

I will, thanks for the reference, but in general I find that videos are good
for giving the illusion of understanding. It all seems so very clear at the
time, and then twenty minutes later it's all "Now did he say to rotate the
shifter or shift the rotator? How far into the talk was that bit again?"

Between programmers taking screenshots of code instead of copying and
pasting text, and the popularity of video over written examples, I'm
feeling like Master Li at the end of "The Story Of The Stone".

Bonus points to anyone who gets the reference without having to google it.

[Steven D'Aprano]

On Thu, 11 Aug 2016 07:33 pm, Chris Angelico wrote:

> Yes,
> threading bugs can be harder to debug; but only when you've violated
> the other principles.

Especially the principle "Avoid threaded programming".

Some people, when faced with a problem, Now they think "I know, I'll have
use two threading." problems.

[Marko Rauhamaa]

Steven D'Aprano <steve+...@pearwood.info>:

> Instructions unclear, poked myself in the eye with a sharp stick.

I have updated my Dining Philosophers example for Python 3.5:

<URL: http://pacujo.net/~marko/philosophers.py>

It demonstrates how to get an event loop and start a number of asyncs
concurrently.


Marko

[Terry Reedy]

On 8/11/2016 2:34 AM, Christian Gollwitzer wrote:
> Am 11.08.16 um 06:38 schrieb Terry Reedy:
>> You might be able to glean something from the succession of files I
>> uploaded to
>> https://bugs.python.org/issue27546
>> Integrate tkinter and asyncio (and async)
>>
>> I started with just mixing tk and asyncio callbacks. After some
>> struggle with similar question as you ask above, I ended up with more or
>> less callback free code using async def and async for. Once I got over
>> the hump, I rather like it.

My next experiment, not uploaded, is adapting tk mainloop so it can be
used instead of asyncio.loop.runforever as the driver for async def and
async for and await. There is either a simple bug I overlooked or I am
missing some detail of how the future and task interact.

> Interesting discussion! Tcl itself acquired coroutines some years ago
> (in 8.6) and people in this community are very excited with async based
> programming.

When I read something like "Python finally acquired an event loop in
3.4" I wonder where people have been. The tk event loop has been in
Python perhaps for 2 decades, and one actually could call select every
xx milleseconds. The main problem is that tcl/tk does not do file
events on Windows. The asyncio people did tackle that, but ended up
with two loops, Selector and Proactor, neither of which are as complete
as Selector on *nix. Tcl/tk could try to borrow than work.

> I'm convinced that it is possible to integrate Tcl's event loop with
> asyncio's loop without regular update polling. This might require a
> patch to Tkinter at the C level. For example, an easy way is to put
> Tcl/Tk in it's own thread. The Tcl event loop can be woken up from a
> different thread by the TclThreadQueueEvent() and TclTheradAlert() API
> calls. Basically, you would spawn a new thread (from the C level), when
> loading Tkinter, which loads Tk and sits in a Tcl event loop ("vwait
> forever"). tk.eval() and tk.call() would be redefined to post events
> using the aforementioned functions. When a Tcl callback arrives, it
> would send an event to asyncio in the main thread - no idea how this
> works, though. In the end, it would also allow to use Tkinter from
> different Python threads, because all calls would be rooted to the one
> instance only.

--
Terry Jan Reedy

[Steven D'Aprano]

On Thu, 11 Aug 2016 03:06 pm, Paul Rubin wrote:

> The basic characteristic of asynchronous programming is that it involves
> changing all your usual blocking i/o calls to non-blocking ones, so your
> program can keep running as soon as your request is started.

That's the bit that confuses me. I understand about parallel processing, but
that's not what this is about.

Say I want to download data from a network, and it will take a long time. If
I can do the read in parallel to something else, that makes sense:

begin downloading in another thread/process
make a coffee
process download


But what's the point in doing it asynchronously if I have to just wait for
it to complete?

begin downloading in an async thread
twiddle thumbs, doing nothing
process download

Take Chris' "yield from" example, rewritten for await:


def asynchronous(id):
trn = await conn.begin_transaction()
await trn.execute("select name from people where id=%d", (id,))
name, = await trn.fetchone()
await trn.execute("update people set last_seen=now() where id=%d",(id,))
await trn.commit()
return name

Since I'm spending all my time waiting for the other things to return
control, why don't I just write it synchronously?

def synchronous(id):
trn = conn.begin_transaction()
trn.execute("select name from people where id=%d", (id,))
name, = trn.fetchone()
trn.execute("update people set last_seen=now() where id=%d", (id,))
trn.commit()
return name

[Marko Rauhamaa]

Steven D'Aprano <steve+...@pearwood.info>:

> Say I want to download data from a network, and it will take a long
> time. If I can do the read in parallel to something else, that makes
> sense:
>
> begin downloading in another thread/process
> make a coffee
> process download
>
> But what's the point in doing it asynchronously if I have to just wait for
> it to complete?

You can do many things concurrently, as the philosophers in my previous
example.

You will need two asyncs, one for downloading, another one for making
coffee, just like you would with threads. Then, run both asyncs
concurrently with asyncio.wait().


Marko

[Steven D'Aprano]

On Thu, 11 Aug 2016 02:41 pm, Chris Angelico wrote:

> Consider these three ways of doing a database transaction:
>
> def synchronous(id):
> trn = conn.begin_transaction()
> trn.execute("select name from people where id=%d", (id,))
> name, = trn.fetchone()
> trn.execute("update people set last_seen=now() where id=%d", (id,))
> trn.commit()
> return name

That makes perfect sense. Good old fashioned synchronous programming.

> def callbacks_1(cb, id):
> conn.begin_transaction(callbacks_2, cb, id)
> def callbacks_2(trn, cb, id):
> trn.execute("select name from people where id=%d", (id,),
> callbacks_3, cb, id)
> def callbacks_3(trn, cb, id):
> trn.fetchone(callbacks_4, cb, id)
> def callbacks_4(trn, data, cb, id):
> name, = data
> trn.execute("update people set last_seen=now() where id=%d",
> (id,), callbacks_5, cb, name)
> def callbacks_5(trn, cb, name):
> trn.commit(callbacks_6, cb, name)
> def callbacks_6(trn, cb, name):
> cb(name)

Now you're surely pulling my leg. Your conn.begin_transaction has a
completely different signature! (No arguments in the first case, three in
this case.)

> def asynchronous(id):
> trn = yield from conn.begin_transaction()
> yield from trn.execute("select name from people where id=%d", (id,))
> name, = yield from trn.fetchone()
> yield from trn.execute("update people set last_seen=now() where
> id=%d", (id,))
> yield from trn.commit()
> return name

That ... looks wrong. You're taking something which looks like a procedure
in the first case (trn.execute), so it probably returns None, and yielding
over it. Even it that's not wrong, and it actually returned something which
you ignored in the first case, it looks like you're mixing two distinct
ways of using generators:

- Generator as iterator ("yield x" or "yield from subiterator");
something which *sends* values out for the purpose of iteration.

- Generator as coroutine ("y = yield x"); something which *receives*
values from the called using the send() method.

Another way of putting it is, "a bogus generator that generates and receives
values (not a recommended coding style)". (David Beazly, per url below.)

When I say coroutine here, I mean in the *old* sense, as described by David
Beasley's A Curious Course on Coroutines and Concurrency:

http://www.dabeaz.com/coroutines/

So there's something deeply suspicious to me about the coding style.

[Michael Selik]

On Thu, Aug 11, 2016 at 11:01 AM Steven D'Aprano <steve+...@pearwood.info>
wrote:

> That ... looks wrong. You're taking something which looks like a procedure
> in the first case (trn.execute), so it probably returns None, and yielding
> over it. Even it that's not wrong, and it actually returned something which
> you ignored in the first case
>

It's a standard, perhaps a mistaken standard, but nonetheless database
cursors tend to have that feature: execute returns the mutated self. I
agree that execute looks like it should return None instead. The return
self pattern feels Rubyish to me (or Rubic?).

[Michael Selik]

On Thu, Aug 11, 2016 at 11:46 AM Michael Selik <michae...@gmail.com>
wrote:

> On Thu, Aug 11, 2016 at 11:01 AM Steven D'Aprano <
> steve+...@pearwood.info> wrote:
>
>> That ... looks wrong. You're taking something which looks like a procedure
>> in the first case (trn.execute), so it probably returns None, and yielding
>> over it. Even it that's not wrong, and it actually returned something
>> which

>> you ignored in the first case
>>
>
> It's a standard, perhaps a mistaken standard, but nonetheless database
> cursors tend to have that feature: execute returns the mutated self. I
> agree that execute looks like it should return None instead. The return
> self pattern feels Rubyish to me (or Rubic?).
>

Contradicting myself:
yield from c.execute(query).fetchall() # looks good
yield from c.execute(query) # looks bad

[Jussi Piitulainen]

The first looks like it allocates a list to hold all the results at once
only to yield each result one by one.

How about:
c.execute(query)
yield from c

[Paul Rudin]

Steven D'Aprano <steve+...@pearwood.info> writes:

>

> But what's the point in doing it asynchronously if I have to just wait for
> it to complete?
>
> begin downloading in an async thread
> twiddle thumbs, doing nothing
> process download

If you have nothing else to do, then there's no point.

But suppose you're implementing a web server. A request comes in - in
order to respond you need some data from a database. So you do the
database bit async. Whilst you're waiting for that you can get on with
processing other incoming requests.

[Steven D'Aprano]

On Thu, 11 Aug 2016 03:34 pm, Paul Rudin wrote:

Thanks. I'm not sure that it helped. I kind of gibbered a bit when Dave said
that threaded code was good because you can easily reason about it. Most of
his talk is about how asyncio shouldn't be in the standard library, and the
rest didn't really answer my questions.

What I'm taking from this is that asynchronous programming is mindboggling
difficult to reason about or understand, and its the best thing ever.

*half-a-wink*


Having-flashbacks-to-when-I-was-first-introduced-to-OOP-ly y'rs,



--
Steve
If anyone mentions monads, I'm out of here.

[Chris Angelico]

On Fri, Aug 12, 2016 at 12:55 AM, Steven D'Aprano
<steve+...@pearwood.info> wrote:
> On Thu, 11 Aug 2016 02:41 pm, Chris Angelico wrote:
>
>> Consider these three ways of doing a database transaction:
>>
>> def synchronous(id):
>> trn = conn.begin_transaction()
>> trn.execute("select name from people where id=%d", (id,))
>> name, = trn.fetchone()
>> trn.execute("update people set last_seen=now() where id=%d", (id,))
>> trn.commit()
>> return name
>
> That makes perfect sense. Good old fashioned synchronous programming.

Let's assume in this case that we started with:

conn = synchronous_database_connection()

>> def callbacks_1(cb, id):
>> conn.begin_transaction(callbacks_2, cb, id)
>> def callbacks_2(trn, cb, id):
>> trn.execute("select name from people where id=%d", (id,),
>> callbacks_3, cb, id)
>> def callbacks_3(trn, cb, id):
>> trn.fetchone(callbacks_4, cb, id)
>> def callbacks_4(trn, data, cb, id):
>> name, = data
>> trn.execute("update people set last_seen=now() where id=%d",
>> (id,), callbacks_5, cb, name)
>> def callbacks_5(trn, cb, name):
>> trn.commit(callbacks_6, cb, name)
>> def callbacks_6(trn, cb, name):
>> cb(name)
>
> Now you're surely pulling my leg. Your conn.begin_transaction has a
> completely different signature! (No arguments in the first case, three in
> this case.)

Let's assume that this one started with:

conn = callback_database_connection()

It's doing the same job as the 'conn' in the first example, but it's a
completely different API to cater to the fact that it has to handle
callbacks. You could use this API for synchronous calls by doing
something like this:

def begin_transaction(callback, *args):
real_conn.begin_transaction()
callback(*args)

The asynchronous version would end up saving callback and args
somewhere, triggering the operation, and having code somewhere that
processes the response. Supposing we're talking to PostgreSQL over a
socket (TCP or Unix domain), the response handler would be triggered
any time that socket becomes readable (ie via select() on the socket),
and it would decode the response, figure out which transaction is
being responded to (if there are multiple in flight), and send the
response on its way. Most likely the transaction would have some kind
of "current in-flight query" attribute (and would reject reentrant
calls - see, any form of async programming has to cope with
reentrancy), so that's where the callback would be stored.

>> def asynchronous(id):
>> trn = yield from conn.begin_transaction()
>> yield from trn.execute("select name from people where id=%d", (id,))
>> name, = yield from trn.fetchone()
>> yield from trn.execute("update people set last_seen=now() where
>> id=%d", (id,))
>> yield from trn.commit()
>> return name
>
> That ... looks wrong. You're taking something which looks like a procedure
> in the first case (trn.execute), so it probably returns None, and yielding
> over it. Even it that's not wrong, and it actually returned something which
> you ignored in the first case, it looks like you're mixing two distinct
> ways of using generators:
>
> - Generator as iterator ("yield x" or "yield from subiterator");
> something which *sends* values out for the purpose of iteration.
>
> - Generator as coroutine ("y = yield x"); something which *receives*
> values from the called using the send() method.

Yeah, generators as coroutines are a bit weird. That's another good
reason for using the new async and await "keywords" (not technically
keywords yet), as it doesn't look as weird. But ultimately, it's doing
the same thing - the methods would look something like this:

def begin_transaction():
# trigger the "begin transaction" query
yield Awaitable("waiting for transaction...")
# verify that the query was successful

The event loop attempts to step the "asynchronous" generator. It
yields from begin_transaction, which yields an Awaitable. The event
loop thus receives, from the generator, an object to be placed on the
queue. It's that simple.

Here's a very VERY simple, but complete, example of yield-based coroutines.

# Partially borrowed from example in Python docs:
# https://docs.python.org/3/library/selectors.html#examples
import selectors
import socket
import time

sel = selectors.DefaultSelector()
def eventloop():
while "loop forever":
for key, mask in sel.select():
sel.unregister(key.fileobj)
run_task(key.data)

def run_task(gen):
try:
waitfor = next(gen)
sel.register(waitfor, selectors.EVENT_READ, gen)
except StopIteration:
pass

def mainsock():
sock = socket.socket()
sock.bind(('localhost', 1234))
sock.listen(100)
sock.setblocking(False)
print("Listening on port 1234.")
while "moar sockets":
yield sock
conn, addr = sock.accept() # Should be ready
print('accepted', conn, 'from', addr)
conn.setblocking(False)
run_task(client(conn))

def client(conn):
while "moar data":
yield conn
data = conn.recv(1000) # Should be ready
if not data: break
print("Got data")
# At this point, you'd do something smart with the data.
# But we don't. We just echo back.
conn.send(data) # Hope it won't block
if b"quit" in data: break
print('closing', conn)
conn.close()

run_task(mainsock())
eventloop()


Aside from slapping a "yield sock" before accepting or reading from a
socket, it's exactly like synchronous code. Obviously a real example
would be able to yield other types of events too (most common would be
the clock, to handle an asynchronous time.sleep() equivalent), but
this is fully functional and potentially even useful.

ChrisA

[Paul Rubin]

Steven D'Aprano <steve+...@pearwood.info> writes:

> But what's the point in doing it asynchronously if I have to just wait for
> it to complete?
> begin downloading in an async thread
> twiddle thumbs, doing nothing
> process download

Suppose the remote server is overloaded so it sends files much slower
than your internet connection can receive them. And you want to
download 10 different files from 10 different such servers. If you do
them synchronously (wait for each download to finish before starting the
next) it takes much longer than necessary. What you want is to download
the 10 files simultaneously, using 10 different network connections.
The download procedure has to speak some network protocol over a socket
for each file.

How do you deal with the concurrency? There are many possibilities:

- use 10 different computers
- open 10 windows on 1 computer, and start a download in each window
(i.e. this means use multiple processes).
- single, multi-threaded download client (oh nooo! the thread
monsters will eat you if you try to do that!!!!)
- single threaded client with asynchronous i/o, so it can have
10 network requests "in the air" simultaneously, using select() or
epoll() to handle each piece of incoming data as soon as it arrives.

The multi-thread and multi-process approaches are conceptually simple
since each connection appears to be synchronous and blocking. They are
both actually async under the covers, but the async i/o and dispatch is
abstracted away by the OS, so the user program doesn't have to worry
about it.

The in-client async approach is generally the most efficient (OS
processes and threads are expensive), but imposes complexity on the
client protocol by making it juggle what each connection is doing,
i.e. where it is in the network protocol at any moment.

A lot of ways have developed over the years to organize client-side
async programs and keep them from getting too confusing:

- explicit state machines (a struct with a state tag for each
connection, and a big event loop with a switch statement),
frequently seen in C programs
- Chained callbacks ("callback hell") seen in node.js
- Callbacks on objects ("reactor pattern"), used in Twisted Matrix
- explicit cooperative multitasking (used in RTOS's, classic Forth,
etc.)
- lightweight processes or threads handled by the language runtime
(GHC, Erlang). This means the user program thinks it's doing
blocking i/o but it's really not.
- coroutines (Lua and now Python's asyncio)
- continuation-based hackery (various Haskell enumeratee libraries)
- probably more that I don't know about or am forgetting.

I like the Erlang/GHC approach best, but it basically means building a
miniature OS into the language runtime and making sure all the
user-visible i/o calls actually use this "OS" instead of actual system
calls to the underlying kernel. The Erlang and GHC implementations are
quite complicated while Python is basically a fairly simple interpreter
wrapped around the standard C libraries.

In Python, the async discussion is basically between various forms of
callbacks, and a few different forms of coroutines. I think that the
conception of best practices is still not completely settled.

I ignore all this and use threads and take the performance hit. I find
it simpler and I haven't been eaten by any thread monsters yet (though
there's always a first time). I figure if I need high performance,
Python isn't the way to do it in the first place: Python is more about
convenience and productivity than performance. I've had 1000 or so
Python threads on a midsized EC2 instance and it's worked ok.

If you really want to do crazy fast async i/o and you use C++, check out
http://www.seastar-project.org/

I haven't tried it yet but want to.

Here's a cool paper about the current GHC I/O system:
http://haskell.cs.yale.edu/wp-content/uploads/2013/08/hask035-voellmy.pdf

[Steven D'Aprano]

Thanks to everyone who has answered, I think I'm slowly starting to get it
now. Let's see if we can come up with a toy example that doesn't involve
low-level socket programming :-)

Let me simulate a slow function call:


import random, time

def work(id):
print("starting with id", id)
workload = random.randint(5, 15)
for i in range(workload):
time.sleep(0.2) # pretend to do some real work
print("processing id", id) # let the user see some progress
print("done with id", id)
return 10 + id


pending = [1, 2, 3, 4]

for i, n in enumerate(pending):
pending[i] = work(n)


How do I write work() so that it cooperatively multi-tasks with other ...
threads? processes? what the hell do we call these things? What does this
example become in the asynchronous world?

In this case, all the work is pure computation, so I don't expect to save
any time by doing this, because the work is still all being done in the
same process/thread, not in parallel. It may even take a little bit longer,
due to the overhead of switching from one to another.

(I presume that actual disk or network I/O may be better, because the OS
will run the I/O in parallel if possible, but I don't expect that in this
case.)

Am I getting close?

[Chris Angelico]

Well, let's see. I can quickly tweak my select() demo to support time delays.

# Partially borrowed from example in Python docs:
# https://docs.python.org/3/library/selectors.html#examples
import selectors
import socket
import time

sel = selectors.DefaultSelector()

sleepers = {} # In a non-toy, this would probably be a heap, not a dict

def eventloop():
while "loop forever":

t = time.time()
for gen, tm in list(sleepers.items()):
if tm <= t:
del sleepers[gen]
run_task(gen)
delay = min(sleepers.values(), default=t+3600) - t
if delay < 0: continue
for key, mask in sel.select(timeout=delay):

sel.unregister(key.fileobj)
run_task(key.data)

def run_task(gen):
try:
waitfor = next(gen)

if isinstance(waitfor, float):
sleepers[gen] = waitfor
else:

sel.register(waitfor, selectors.EVENT_READ, gen)
except StopIteration:
pass

def sleep(tm):
yield time.time() + tm

def mainsock():
sock = socket.socket()
sock.bind(('localhost', 1234))
sock.listen(100)
sock.setblocking(False)
print("Listening on port 1234.")
while "moar sockets":
yield sock
conn, addr = sock.accept() # Should be ready
print('accepted', conn, 'from', addr)
conn.setblocking(False)
run_task(client(conn))

def client(conn):
while "moar data":
yield conn
data = conn.recv(1000) # Should be ready
if not data: break
print("Got data")
# At this point, you'd do something smart with the data.

# But we don't. We just echo back, after a delay.
yield from sleep(3)

conn.send(data) # Hope it won't block
if b"quit" in data: break
print('closing', conn)
conn.close()

if __name__ == '__main__':
run_task(mainsock())
eventloop()



So, if it used this tiny event loop, your code would look like this:

def work(id):
print("starting with id", id)
workload = random.randint(5, 15)
for i in range(workload):

yield from sleep(0.2) # pretend to do some real work

print("processing id", id) # let the user see some progress
print("done with id", id)
return 10 + id

for n in [1, 2, 3, 4]:
run_task(work(n))
eventloop()

But crucially, this depends on having some kind of waitable work. That
generally means either non-blocking I/O (of some sort - a lot of
things in a Unix system end up being reads and writes to some
file-like thing, eg a socket, pipe, or device), or a time delay, or
maybe waiting on a signal. If the 'work' is a blocking CPU-bound
operation, this will never be able to multiplex.

ChrisA

[Lawrence D’Oliveiro]

On Friday, August 12, 2016 at 2:25:05 AM UTC+12, Terry Reedy wrote:

> When I read something like "Python finally acquired an event loop in
> 3.4" I wonder where people have been. The tk event loop has been in

> Python perhaps for 2 decades...

As was pointed out to me just a few days ago, that’s Tcl, not Python.

[Paul Rubin]

Steven D'Aprano <steve+...@pearwood.info> writes:

> How do I write work() so that it cooperatively multi-tasks with other ...
> threads? processes? what the hell do we call these things? What does this
> example become in the asynchronous world?

If it's heavily computational then you have to yield to the scheduler
frequently so that other tasks have a chance to run. So if your 0.2
second computation involves 100 iterations of something, you could yield
once per iteration (every 2 msec).

[Paul Rudin]

Steven D'Aprano <steve+...@pearwood.info> writes:

> Thanks to everyone who has answered, I think I'm slowly starting to get it
> now. Let's see if we can come up with a toy example that doesn't involve
> low-level socket programming :-)
>
> Let me simulate a slow function call:
>
>
> import random, time
>
> def work(id):
> print("starting with id", id)
> workload = random.randint(5, 15)
> for i in range(workload):
> time.sleep(0.2) # pretend to do some real work
> print("processing id", id) # let the user see some progress
> print("done with id", id)
> return 10 + id
>
>
> pending = [1, 2, 3, 4]
>
> for i, n in enumerate(pending):
> pending[i] = work(n)
>
>
> How do I write work() so that it cooperatively multi-tasks with other ...
> threads? processes? what the hell do we call these things? What does this
> example become in the asynchronous world?
>

They're not separate processes or threads, just think tasks that suspend
and restart cooperatively. You need some kind of event loop to
orchestrate running the tasks. Asyncio provides one.

You can do your example like this:

import asyncio, random

async def work(id):

print("starting with id", id)
workload = random.randint(5, 15)
for i in range(workload):

await asyncio.sleep(0.2) # pretend to do some real work

print("processing id", id) # let the user see some progress
print("done with id", id)
return 10 + id

loop = asyncio.get_event_loop()

pending = [1, 2, 3, 4]

jobs = [asyncio.ensure_future(work(n)) for n in pending]

loop.run_until_complete(asyncio.gather(*jobs))

loop.close()

> In this case, all the work is pure computation, so I don't expect to save
> any time by doing this, because the work is still all being done in the
> same process/thread, not in parallel. It may even take a little bit longer,
> due to the overhead of switching from one to another.
>

Yeah - you're not gaining anything here. All (?) the interesting use
cases involve waiting for something (e.g. disk access, network
communication, separate process) before you can proceed with your
computation.

[Steven D'Aprano]

On Fri, 12 Aug 2016 03:47 pm, Paul Rudin wrote:

> Steven D'Aprano <steve+...@pearwood.info> writes:
>
>> Thanks to everyone who has answered, I think I'm slowly starting to get
>> it now. Let's see if we can come up with a toy example that doesn't
>> involve low-level socket programming :-)
>>
>> Let me simulate a slow function call:

[...]

> They're not separate processes or threads, just think tasks that suspend
> and restart cooperatively. You need some kind of event loop to
> orchestrate running the tasks. Asyncio provides one.
>
> You can do your example like this:
>
> import asyncio, random
>
> async def work(id):
> print("starting with id", id)
> workload = random.randint(5, 15)
> for i in range(workload):
> await asyncio.sleep(0.2) # pretend to do some real work
> print("processing id", id) # let the user see some progress
> print("done with id", id)
> return 10 + id
>
> loop = asyncio.get_event_loop()
>
> pending = [1, 2, 3, 4]
>
> jobs = [asyncio.ensure_future(work(n)) for n in pending]
>
> loop.run_until_complete(asyncio.gather(*jobs))
>
> loop.close()

That is *awesome*. Thank you for the example!

[Paul Rubin]

Steven D'Aprano <steve+...@pearwood.info> writes:
>> await asyncio.sleep(0.2) # pretend to do some real work

> That is *awesome*. Thank you for the example!

Keep in mind that the above basically takes the task off the list of
runnables for 0.2 seconds, so it sits doing nothing and doesn't
interfere with other tasks running. In the case where you're doing
actual computation, you want to yield (await) much more often than 0.2
sec, so other tasks can stay responsive. This is cooperative
multitasking, where each task has to be nice to the others, in contrast
with preemptive multitasking where the task switches happen behind the
scenes.

[Marko Rauhamaa]

Paul Rubin <no.e...@nospam.invalid>:

> Keep in mind that the above basically takes the task off the list of
> runnables for 0.2 seconds, so it sits doing nothing and doesn't
> interfere with other tasks running. In the case where you're doing
> actual computation, you want to yield (await) much more often than 0.2
> sec, so other tasks can stay responsive. This is cooperative
> multitasking, where each task has to be nice to the others, in
> contrast with preemptive multitasking where the task switches happen
> behind the scenes.

I don't suppose the Python database API has yet been ported to asyncio,
has it? That means database access is not yet quite usable with asyncio
(or any other form of asynchronous programming).

Also, one must be careful with file access, which is necessarily
blocking on linux (unless Python takes Linux's AIO API into use, which
would be groundbreaking).


Marko

[Paul Rubin]

Marko Rauhamaa <ma...@pacujo.net> writes:
> Also, one must be careful with file access, which is necessarily
> blocking on linux (unless Python takes Linux's AIO API into use, which
> would be groundbreaking).

AIO is a possibility for i/o on an already-open file and I think there
may be other ways to do it too. But there appears to be no way to open
a file without potentially blocking, potentially for a long time on a
busy hard disk or a remote file system. GHC and Erlang both use a
thread pool to deal with opening disk files.

[Ethan Furman]

On 08/11/2016 07:14 AM, Steven D'Aprano wrote:
> On Thu, 11 Aug 2016 03:06 pm, Paul Rubin wrote:
>
>> The basic characteristic of asynchronous programming is that it involves
>> changing all your usual blocking i/o calls to non-blocking ones, so your
>> program can keep running as soon as your request is started.
>
> That's the bit that confuses me. I understand about parallel processing, but
> that's not what this is about.
>
> Say I want to download data from a network, and it will take a long time. If
> I can do the read in parallel to something else, that makes sense:
>
> begin downloading in another thread/process
> make a coffee
> process download
>
>
> But what's the point in doing it asynchronously if I have to just wait for
> it to complete?
>
> begin downloading in an async thread
> twiddle thumbs, doing nothing
> process download

async, threads, multiprocessing, etc., all only make sense if you have more than one thing to do. The advantage of the new async stuff is it allows us to write in a synchronous manner (making async calls instead of blocking calls) and the underlying framework takes care of the hassle of using threads or multiprocessing or whatever.

--
~Ethan~

[Lawrence D’Oliveiro]

On Saturday, August 13, 2016 at 8:26:05 PM UTC+12, Marko Rauhamaa wrote:

> ... (unless Python takes Linux's AIO API into use, which
> would be groundbreaking).

Last I looked, asynchronous I/O on Linux was still a work in progress. The whole POSIX API has been grounded in the assumption that all I/O is blocking--in other words, I/O requests and process scheduling have been inextricably linked. And that has been incorporated into the design of Linux, which they now have to disentangle.

Before I was introduced to Unix/Linux, I was using VMS, where all I/O was inherently asynchronous and decoupled from process scheduling: all the synchronous I/O calls were just userland wrappers that did “queue I/O request” followed by “wait for completion”.

[Ethan Furman]

On 08/11/2016 10:47 PM, Paul Rudin wrote:
> Steven D'Aprano writes:

[...]

>> In this case, all the work is pure computation, so I don't expect to save
>> any time by doing this, because the work is still all being done in the
>> same process/thread, not in parallel. It may even take a little bit longer,
>> due to the overhead of switching from one to another.
>
> Yeah - you're not gaining anything here. All (?) the interesting use
> cases involve waiting for something (e.g. disk access, network
> communication, separate process) before you can proceed with your
> computation.

The obvious exception being a UI remaining responsive to the user while the calculation is processing.

--
~Ethan~

[Terry Reedy]

Tk is part of Tcl/Tk. Tk's mainloop() calls level tcl loop functions.

The tkinter and _tkinter modules are Python stdlib modules. For
timer-based callbacks, there is little difference between using tkinter
and async modules. The fact that async uses a Python-coded event loop
while tkinter uses a Tcl-coded loop is nearly invisible. Several other
stdlib modules wrap 'foreign' code too.

--
Terry Jan Reedy