Function naming in asyncio

[Martin Teichmann]

Hi list,

first of all, I wanted to thank all the authors of asyncio. It's just

great, and makes my life as a programmer so much easier!

There is just one thing I have problems with: the naming of

the functions is, well, awkward. For long I thought that it's

just my poor English, but recently I have been introducing

asyncio to a coworker who is native english speaker, and

when I told him: if you want to run two coroutines ham and 

spam in parallel, just write

   await gather(ham(), spam())

the only result was laughter. Await gather? What should

that mean? I know what the authors meant, gather the futures,

but this is very theoretical, it's has nothing to do with what

one wants to do in the code.

The most weird of them is ensure_future. It used to be

called async, which made some sense to me. But telling

someone: if you want to run coroutine spam in the background,

write

   await ensure_future(spam())

will also gain raised eyebrows at best. Sure, I can use

get_event_loop().create_task (finally a good name!), or

the Task constructor, but this makes the API very

non-systematic, why do we have sometimes functions

in the asyncio module, sometimes you have to call the

event loop?

run_in_executor also exists only in the event loop, and

has a weird calling syntax. Sure, it should not be used that

often, unfortunately I have to work with a C library that

exposes only a blocking API, so my code is full of lines

like

    await get_event_loop().run_in_executor(None, spam())

which also doesn't look to nice to me.

Then there is a large overloading in the word wait:

    await wait([ham(), spam())

    await wait_for(spam(), timeout=3)

at least wait_for now has a similar context manager called

timeout, that is good wording.

So, I arbitrarily propose to introduce new names. Maybe

some native speaker comes up with even better names.

    gather -> parallel

    wait -> first_completed, first_exception and all_completed

    wait_for -> timed

    ensure_future -> create_task

and make run_in_executor also a function, but with the

first parameter put at the end.

Another naming problem is TimeoutError: it shadows

the builtin TimeoutError. This leads to awful bugs. As timeouts

often don't occur (and have a tendency that they lack

test cases, as no one likes long-running tests), once they

do occur, something bad happens. And even the typical

code cleaning utilities (like pyflakes) don't cry "you forgot

to import TimeoutError!" because it is built in. This is

why I propose to simply define TimeoutError to be the

builtin TimeoutError, or at least inherit from it.

Greetings

Martin

[Stefan Scherfke]

HI Martin,

> Am 2016-02-25 um 09:44 schrieb Martin Teichmann <martin.t...@gmail.com>:
>
> Hi list,
>
> first of all, I wanted to thank all the authors of asyncio. It's just
> great, and makes my life as a programmer so much easier!

Yep. :)

>
> There is just one thing I have problems with: the naming of
> the functions is, well, awkward. For long I thought that it's
> just my poor English, but recently I have been introducing
> asyncio to a coworker who is native english speaker, and
> when I told him: if you want to run two coroutines ham and
> spam in parallel, just write
>
> await gather(ham(), spam())

“There are only two hard things in Computer Science: cache invalidation
and naming things.”

-- Phil Karlton

I guess some of the confusion comes from replacing "yield from" with
"await". To me, "yield from gather(...)" doesn’t sound as strange as
"await gather(...)".

> So, I arbitrarily propose to introduce new names. Maybe
> some native speaker comes up with even better names.
>
> gather -> parallel
> wait -> first_completed, first_exception and all_completed
> wait_for -> timed
> ensure_future -> create_task

In SimPy, a descrete-event simulation lib with similar concepts as
asyncio, we also had long discussions about naming stuff when we wrote
version 3.

For stuff like "gather()" and "wait_for()", we introduced so called
"Condition Events" and created "any_of()" and "all_of()" methods around
it. "await all_of(fut1, fut2)" or "await any_of(fut1, fut2)" reads a
lot better then "gather()", imho.

Instead of a timeout parameter, we simply would do a "await any_of(fut1,
sleep(x))" (actually, in SimPy, "sleep()" is called "timeout()").

And, even better, you could use the "&" and "|" operators to concatenate
futures/events: "await fut | timeout" or "await fut1 & fut2".

Instead of "create_task()" or "async()" we use "process()" to indicate
that we start a new "sub process", but "create_task()" would be fine to
if it was in the "asyncio" namespace and not a method of loop.

That some methods (e.g., "gather()") are in the "asyncio" namespace and
require a loop argument and some methods are methods of a loop instance
itself also baffles me.

Unfortunately, the problem with asyncio is, that it is in the stdlib
and you cannot just start renaming stuff, or only with long deprecation
periods.

Cheers,
Stefan

[Andrew Svetlov]

Why do you hesitate to use loop members?

I found passing explicit loop is very important and should be done everywhere.

asyncio.TimeoutError cannot be inherited from TimeoutError builtin:
the former has no errno and technically it is not an OSError

--
Thanks,
Andrew Svetlov

[Stefan Scherfke]

> Am 2016-02-25 um 12:33 schrieb Andrew Svetlov <andrew....@gmail.com>:
>
> Why do you hesitate to use loop members?

No, I don’t hesitate. I’m just wondering why some functions are exposed
as loop members and some are normal functions taking a loop argument.
Why the different namespaces?

> I found passing explicit loop is very important and should be done everywhere.

I agree. In SimPy, we even went so far to not have an implicit, global
default loop (we call it Environment). Thus, you *must* pass the
environment explicitly around.

> asyncio.TimeoutError cannot be inherited from TimeoutError builtin:
> the former has no errno and technically it is not an OSError

Actually, the asyncio TimeoutError is not really an error, but imho
rather something like an interrupt (like "KeyboardInterrupt", which is
also not called "KeyboardInterruptError"), so just "asyncio.Timeout" (or
maybe "asyncio.TimeoutInterrupt") would maybe we more appropriate.

Cheers,
Stefan

[Andrew Svetlov]

On Fri, Feb 26, 2016 at 3:11 AM, Stefan Scherfke
<ste...@sofa-rockers.org> wrote:
>
>> Am 2016-02-25 um 12:33 schrieb Andrew Svetlov <andrew....@gmail.com>:
>>
>> Why do you hesitate to use loop members?
>
> No, I don’t hesitate. I’m just wondering why some functions are exposed
> as loop members and some are normal functions taking a loop argument.
> Why the different namespaces?
>

It's question of code coupling. Functions that require access to loop
internal state are loop members.
Ones that can be built on top of existing loop members are just first
class functions/coroutines.

>> I found passing explicit loop is very important and should be done everywhere.
>
> I agree. In SimPy, we even went so far to not have an implicit, global
> default loop (we call it Environment). Thus, you *must* pass the
> environment explicitly around.
>
>> asyncio.TimeoutError cannot be inherited from TimeoutError builtin:
>> the former has no errno and technically it is not an OSError
>
> Actually, the asyncio TimeoutError is not really an error, but imho
> rather something like an interrupt (like "KeyboardInterrupt", which is
> also not called "KeyboardInterruptError"), so just "asyncio.Timeout" (or
> maybe "asyncio.TimeoutInterrupt") would maybe we more appropriate.
>

Historically Guido van Rossum made asyncio.TimeoutError as just alias
for concurrent.Future.TimeoutError.
That makes sense.

Unfortunately it clashes with builtin TimeoutError, but looks like the
ship has sailed two years ago.
Renaming concurrent.Future.TimeoutError and asyncio.TimeoutError to
TimeoutInterrupt will make a mess (we still need to support old names
for several Python releases).

--
Thanks,
Andrew Svetlov

[Martin Teichmann]

Hi,

> Unfortunately, the problem with asyncio is, that it is in the stdlib
> and you cannot just start renaming stuff, or only with long deprecation
> periods.

I just checked: asyncio is still on a provisional status. So one may still change things. One should use the provisional status to make asyncio shiny.

As to loop members vs. module functions: I think all loop members susceptible to be used in a task should have a module function counterpart, to be consistent. That does not apply to run_forever and friends as they are not called inside a task.

Greetings

Martin

[Andrew Svetlov]

On Fri, Feb 26, 2016 at 3:39 AM, Martin Teichmann
<lkb.te...@gmail.com> wrote:
> Hi,
>
>> Unfortunately, the problem with asyncio is, that it is in the stdlib
>> and you cannot just start renaming stuff, or only with long deprecation
>> periods.
>
> I just checked: asyncio is still on a provisional status. So one may still
> change things. One should use the provisional status to make asyncio shiny.

I believe provisional status assumes relative small backward
compatible changes that may appear in minor python releases also (at
least it is the policy for aiohttp right now).
Massive renaming is not an option.

The final word belongs to Guido anyway. Personally I can live with
current naming.

`asyncio.create_task` as alias for `loop.create_task` may be slightly
useful though.

--
Thanks,
Andrew Svetlov