Is there a way to tell if a generator has been exhausted using pure
Python code? I've looked at CPython sources and it seems that
something like "active"/"exhausted" attribute on genobject is missing
from the API. For the time being I am using a simple C extension to
look at f_stacktop pointer of the generator frame, which seems to
differentiate active generators from exhausted ones. See
http://bazaar.launchpad.net/~ruby/pythoscope/support-python2.3/annota...
for complete source code.

I may be missing something obvious here. Is there a better way to tell
if a given generator object is still active or not?

In other words, don't LBYL, because it's EAFP. Whatever you need to do
that requires the next item from the generator, do that; you'll get a
specific exception if the generator is exhausted.

I'm working on the Pythoscope project (http://pythoscope.org) and I
use tracing mechanisms of CPython (sys.settrace) to capture function
calls and values passed to and from them. Now, the problem with
generators is that when they are ending (i.e. returning instead of
yielding) they return a None, which is in fact indistinguishable from
"yield None". That means I can't tell if the last captured None was in
fact yielded or is a bogus value which should be rejected. Let me show
you on an example.

import sys

def trace(frame, event, arg):
    if event != 'line':
        print frame, event, arg
    return trace

def gen1():
    yield 1
    yield None

def gen2():
    yield 1

sys.settrace(trace)
print "gen1"
g1 = gen1()
g1.next()
g1.next()
print "gen2"
g2 = gen2()
[x for x in g2]
sys.settrace(None)

The first generator isn't finished, it yielded 1 and None. Second one
is exhausted after yielding a single value (1). The problem is that,
under Python 2.4 or 2.3 both invocations will generate the same trace
output. So, to know whether the last None was actually a yielded value
I need to know if a generator is active or not.

Your solution, while gives me an answer, is not acceptable because
generators can cause side effects (imagine call to launch_rockets()
before the next yield statement ;).

You could put in a feature request on python-dev
concerning this.

When it comes to side-effects, timing matters. For example, a generator
which cleans up after it has run (deleting temporary files, closing
sockets, etc.) will leave the environment in a different state if run to
exhaustion than just before exhaustion. Even if you store the final
result in a one-ahead class, you haven't saved the environment, and that
may be significant.

(Of course, it's possible that it isn't significant. Not all differences
make a difference.)

The best advice is, try to avoid side-effects, especially in generators.

Via "What’s New in Python 2.5" (http://docs.python.org/whatsnew/
2.5.html):

"""
Another even more esoteric effect of this change: previously, the
gi_frame attribute of a generator was always a frame object. It’s now
possible for gi_frame to be None once the generator has been
exhausted.
"""

There are two possible definition of 'exhausted': 1) will raise
StopIteration on the next next() call; 2) has raised StopIteration at
least once. The wrapper converts 2) to 1), which is to say, it obeys
definition 1 once the underlying iteration has obeyed definition 2.

Since it is trivial to set 'exhausted=True' in the generator user code
once StopIteration has been raised (meaning 2), I presume the OP wants
the predictive meaning 1).

Without a iterator class wrapper, I see no way to predict what a
generator will do (expecially, raise StopIteration the first time)
without analyzing its code and local variable state.

I said in response to YOU that once exhaustion has occurred, then the
same number of side effects would have occurred.

  For example, a generator

Assuming that is not the case, the cleanup will not happen until the
what turns out to be the final item is requested from the wrapper. Once
cleanup has happened, .exhausted will be set to True. If proper
processing of even the last item requires that cleanup not have
happened, then that and prediction of exhaustion are incompatible. One
who wants both should write an iterator class instead of generator function.

While, as you showed, it is trivial to create a generator that will
have the "exhausted" flag, in my specific case I have no control over
the user code. I have to use what the Python genobject API gives me
plus the context of the trace function.