How asyncio works? and event loop vs exceptions

[Marco S.]

I'm developing a web app based on aiohttp, and I find the event loop
concept very interesting. I never programmed with it before, but I
know that node.js and GUIs are based on it.

What I can't understand is how asyncio make it possible to run
multiple tasks concurrently, since it's single threaded (if you don't
use loop.run_in_executor()). I just tried to imagine that it should
act as a cpu scheduler. Is this true? If so, how and when asyncio
decide to switch to another task?

Furthermore I have a question about exceptions in asyncio. If I
understand well how it works, tasks exceptions can be caught only if
you wait for task completion, with yield from, await or
loop.run_until_complete(future). But in this case the coroutine blocks
the execution of the program until it returns. On the contrary you can
execute the coroutine inside an asyncio task and it will be
non-blocking, but in this case exceptions can't be caught in a try

statement.

Is this correct? If so, asyncio programming style can't be a little
divergent from what was the philosophy of Python until now?

[Terry Reedy]

On 7/22/2016 8:27 PM, Marco S. via Python-list wrote:
> I'm developing a web app based on aiohttp, and I find the event loop
> concept very interesting. I never programmed with it before, but I
> know that node.js and GUIs are based on it.
>
> What I can't understand is how asyncio make it possible to run
> multiple tasks concurrently, since it's single threaded (if you don't
> use loop.run_in_executor()). I just tried to imagine that it should
> act as a cpu scheduler. Is this true?

The code for BaseEventLoop is in asyncio.base_events.py. In particular,
see def run_forever. Very simplified,

def run_forever():
while True:
handle_ready_io_events()
call_ready_scheduled_functions()

> If so, how and when asyncio decide to switch to another task?

It does not decide. Asyncio does *cooperative* multi-tasking. This
depends on handlers running for a short time and quitting, even if there
is more work to do. The OS does *pre-emptive* multi-tasking; it
switches processes that are still running.

It is possible to add other events into the mix. Adding these lines

def tk_update():
root.update()
loop.call_later(.01, tk_update)
tk_update()

to an asyncio program before loop.run_forever and the loop will drive a
tkinter gui. A day ago, I experimentally patched IDLE to run off of an
asyncio loop instead of the tk loop. Everything I tried worked. See
https://bugs.python.org/issue27546.

> Furthermore I have a question about exceptions in asyncio.

I am too ignorant about this to even ask.

--
Terry Jan Reedy

[Ian Kelly]

On Fri, Jul 22, 2016 at 6:27 PM, Marco S. via Python-list
<pytho...@python.org> wrote:
> Furthermore I have a question about exceptions in asyncio. If I
> understand well how it works, tasks exceptions can be caught only if
> you wait for task completion, with yield from, await or
> loop.run_until_complete(future). But in this case the coroutine blocks
> the execution of the program until it returns. On the contrary you can
> execute the coroutine inside an asyncio task and it will be
> non-blocking, but in this case exceptions can't be caught in a try
> statement.

If you don't want to block the current function on the task, then spin
off another task to do the error handling. Instead of this:

async def do_something():
try:
await do_something_else()
except DidNothingError as e:
handle_error(e)
...

Consider this:

async def do_something():
get_event_loop().create_task(await_and_handle(do_something_else()))
...

async def await_and_handle(awaitable):
try:
await awaitable
except DidNothingError as e:
handle_error(e)

If you want, you could generalize that further by passing in the
exception class and error handler as well:

async def do_something():
get_event_loop().create_task(await_and_handle(
do_something_else(), DidNothingError, handle_error))
...

async def await_and_handle(awaitable, error_class, handler):
try:
await awaitable
except error_class as e:
handler(e)

[Marco Sulla]

Really good suggestion, thank you.