Coverage.py support for await and yield-from: how should it work?

[Ned Batchelder]

Hi all,

I've just released Coverage.py 4.1 beta 1: http://nedbatchelder.com/blog/201601/coveragepy_41b1.html

It has a new branch analysis implementation, partly to deal with the async keyword.  But I need some help deciding how it should work with the new features.

A yield-from statement is possibly an exit from the function, as it yields values.  If you use a yield-from, and no values are yielded, coverage.py 4.1b1 will mark it as partially uncovered because the branch to the function exit isn't taken.  But yield-from is also used to call functions that won't produce a value.  So we have a choice: Should coverage.py could insist on a branch to the function exit?  Pro: this could alert you to code you thought produced values, but doesn't.  Con: the places you know you aren't getting any values, you'll have to use a pragma comment to shut up coverage.py.

What would you prefer?

My understanding is that "await" behaves the same, so I'm assuming the same decision would apply there.  BTW: the 4.1b1 code is definitely wrong in that it doesn't grok the difference between "yield from x" and "v = yield from x".  The latter is clearly not a function exit, but 4.1b1 also insists that it should branch to the function exit, which it never will, so at the very least that has to get fixed.

Any other thoughts about what's needed are also welcome.  Thanks,

--Ned.

[Yury Selivanov]

Hi Ned,

On 2016-01-11 7:39 PM, Ned Batchelder wrote:
> [...]

> What would you prefer?
>
> My understanding is that "await" behaves the same, so I'm assuming the
> same decision would apply there.

I haven't used coverage with generator-based coroutines, but here's my
understanding on how it should behave with 'async def' ones:

In coroutines world, 'await foo()' is analogous to a function call. So
if the 'await' expression was executed, it doesn't matter if 'foo()'
awaited on something or not (or if any 'YIELD' opcode was ever run);
from the coverage standpoint, the code was covered.

Thanks,
Yury

[Aymeric Augustin]

Hello Ned,

On 12 janv. 2016, at 01:39, Ned Batchelder <n...@nedbatchelder.com> wrote:

> So we have a choice: Should coverage.py could insist on a branch to the function exit? Pro: this could alert you to code you thought produced values, but doesn't. Con: the places you know you aren't getting any values, you'll have to use a pragma comment to shut up coverage.py.

For developers who are only using `yield from` in the context of asyncio, this question is equivalent to: “should coverage.py mark a branch as not covered if a function never returns a result?”. The answer to that is “no”.

For developers who are using `yield from` to simplify passing values from iterators across function call chains, there’s a good chance that something at one end of the chain will have an uncovered branch if no value is yielded.

As a consequence, I believe the practical policy for coverage.py is to ignore the function exit branch in `yield from`.

If you wanted to be really smart, you could try to tell apart `yield from generator` from `yield from coroutine`. As of Python 3.5, you could look at the CO_COROUTINE flag (https://www.python.org/dev/peps/pep-0492/#coroutine-objects). Unfortunately, it looks like this doesn't works for pre-3.5 coroutines decorated with @asyncio.coroutine — which is the only option for asyncio libraries that wish to support Python 3.4 and 3.3.

--
Aymeric.

[Ned Batchelder]

FWIW, I've just posted coverage.py 4.1b2, which no longer treats await or yield-from as an exit from the function: https://pypi.python.org/pypi/coverage/4.1b2

Any feedback appreciated :)

--Ned.

[Andrew Svetlov]

Coverage report from 4.1b2 is perfect!
At least I don't see false positives on run over aiohttp test suite.

--
Thanks,
Andrew Svetlov

[Samuel Colvin]

Just found this thread as I was about to submit an issue about this.

I would agree with Andrew that 4.1b2 looks right to me.

Thanks a lot.

Samuel