Re: [gevent] gevent fix for code coverage

[Denis Bilenko]

On Wed, Jul 4, 2012 at 1:19 AM, ggw <jonathan....@gyregroup.com> wrote:
> I am fairly new to python and gevent, so I'm not sure if this might break
> anything for anyone else, but it seems to restore coverage reporting.

Did you check if spawned greenlets are also covered by coverage.py
with this patch? From the code it seems that trace function is None
most of the time, except when the MAIN greenlet is running, so event
loop callbacks and spawned greenlets are excluded.

>The issue seems to be that sys.settrace() used by coverage.py doesn't like to
> see the gevent frames without expecting them.

I'm not sure what do you mean by 'gevent frame', so it's hard to
comment on it and everything below.

[ggw]

...event loop callbacks and spawned greenlets are excluded...

That helps!  I can now handle spawned greenlets, but I'm still a little unclear on the event loop callbacks.  Is there an example you can point me to?

 

I'm not sure what do you mean by 'gevent frame'

Not sure if python uses the same terms, but I'm used to describing my location with the terms thread, stack and frame.  A thread is a path of execution through the program (possibly simultaneous with other threads), a stack  describes the nested calls made by the thread, a frame is a single layer on the stack (a function call, with associated local variables and arguments).

The issue I was raising with "gevent frame" was that the coverage trace function kept a representation of the stack and expected it to remain the same.  So if we are running coverage on a function with a blocking call in the middle, the stack before and after that blocking call may look different.  After the blocking call, the stack starts with gevent's Hub.switch() but before the blocking call the stack looked different -- the trace function found a frame there that hadn't been there before the blocking function.
 

[ggw]

The latest implementation works for our project's unit tests but requires that both coverage and gevent have slight changes.  Here are public repositories in case anyone wants to try out the combination.  One caveat -- so far, this works for line coverage metrics, but disables branch coverage.  There is certainly work to do, but for now I'm just hoping anyone out there can point out cases that won't show correct coverage metrics or places I might break gevent.

Each project for now just has two commits.  The first is the untouched original code from the gevent or coverage repository.  The next commit is my changes.  Just a few lines in a couple files.

coverage source - edited trace function to ignore the unexpected change to the stack after a blocking call
gevent source - edited Hub.switch to save and restore the trace function; Greenlet.__init__ to apply the global trace function set with threading.settrace