Event loop error handler

167 views
Skip to first unread message

Yury Selivanov

unread,
Feb 6, 2014, 11:53:04 AM2/6/14
to python-tulip
As discussed with Guido in issue #80
(https://code.google.com/p/tulip/issues/detail?id=80),
I'm proposing a strawman design for having a user definable event loop
handler for unhandled exceptions.

Two new API points:

- "loop.set_exception_handler(callback)"
Sets 'callback' as a new unhandled error handler for the event loop
'loop'. The signature of callback should be '(loop, exception,
context)'
where: 'loop' parameter is the current event loop running; 'exception'
is the unhandled exception object; 'context' is a string (str),
that will be the basic context information, like
"unhandled exception in add_reader callback" (the kind of information
that usually goes to the log title).

It's OK to call 'loop.stop()' or 'loop.call_*' methods from the
handler.

If a user set handler raises an exception, that exception will be
handled by the default event loop exception handler (that will
simply log it)

- "loop.restore_default_exception_handler()"
Restores exception handler to the default one.
Default exception handler just logs the error with
'loop.logger.exception()'

Let's discuss.

Thanks,
Yury

Guido van Rossum

unread,
Feb 6, 2014, 1:24:38 PM2/6/14
to Yury Selivanov, python-tulip
On Thu, Feb 6, 2014 at 8:53 AM, Yury Selivanov <yseliv...@gmail.com> wrote:
As discussed with Guido in issue #80
(https://code.google.com/p/tulip/issues/detail?id=80),
I'm proposing a strawman design for having a user definable event loop
handler for unhandled exceptions.

Two new API points:

  - "loop.set_exception_handler(callback)"
    Sets 'callback' as a new unhandled error handler for the event loop
    'loop'. The signature of callback should be '(loop, exception, context)'
    where: 'loop' parameter is the current event loop running; 'exception'
    is the unhandled exception object; 'context' is a string (str),
    that will be the basic context information, like
    "unhandled exception in add_reader callback" (the kind of information
    that usually goes to the log title).

I'd rename the context arg to 'message', then you don't have to explain what it is. :-)

Also I'd steer clear from calling the handler 'callback' -- maybe just name it 'exception_handler'?
 
    It's OK to call 'loop.stop()' or 'loop.call_*' methods from the handler.

Cool.
 
    If a user set handler raises an exception, that exception will be
    handled by the default event loop exception handler (that will
    simply log it)

Nice.
 
  - "loop.restore_default_exception_handler()"
    Restores exception handler to the default one.
    Default exception handler just logs the error with
    'loop.logger.exception()'

Maybe instead of having a separate API for this rarely-used action, this could be spelled as loop.set_exception_handler(None)?

A few questions:

- Do we need an API to ask the loop for its current exception handler? If so, what should it return if no handler is set explicitly?

- Should we allow multiple exception handlers, or just one? (I'd prefer just one.)

- Should we have a way to invoke the default handler explicitly? E.g. if the user's handler doesn't want to do anything special, they could pass it to the default handler instead of having to figure out how the default handler logs things.

- Should we have a way to invoke the current handler explicitly? E.g. if there's user code that has encountered an exception and it wants that exception to be treated the same as the loop handles other exceptions.

--
--Guido van Rossum (python.org/~guido)

Yury Selivanov

unread,
Feb 6, 2014, 3:09:55 PM2/6/14
to python...@googlegroups.com

On 2/6/2014, 1:24 PM, Guido van Rossum wrote:
> On Thu, Feb 6, 2014 at 8:53 AM, Yury Selivanov <yseliv...@gmail.com>wrote:
>
>> As discussed with Guido in issue #80
>> (https://code.google.com/p/tulip/issues/detail?id=80),
>> I'm proposing a strawman design for having a user definable event loop
>> handler for unhandled exceptions.
>>
>> Two new API points:
>>
>> - "loop.set_exception_handler(callback)"
>> Sets 'callback' as a new unhandled error handler for the event loop
>> 'loop'. The signature of callback should be '(loop, exception,
>> context)'
>> where: 'loop' parameter is the current event loop running; 'exception'
>> is the unhandled exception object; 'context' is a string (str),
>> that will be the basic context information, like
>> "unhandled exception in add_reader callback" (the kind of information
>> that usually goes to the log title).
>>
> I'd rename the context arg to 'message', then you don't have to explain
> what it is. :-)
Agree. Need to make it clear, though, that 'message' is not
necessarily the same as 'exception.args[0]'

>
> Also I'd steer clear from calling the handler 'callback' -- maybe just name
> it 'exception_handler'?
>
>
>> It's OK to call 'loop.stop()' or 'loop.call_*' methods from the
>> handler.
>>
> Cool.
>
>
>> If a user set handler raises an exception, that exception will be
>> handled by the default event loop exception handler (that will
>> simply log it)
>>
> Nice.
>
>
>> - "loop.restore_default_exception_handler()"
>> Restores exception handler to the default one.
>> Default exception handler just logs the error with
>> 'loop.logger.exception()'
>>
> Maybe instead of having a separate API for this rarely-used action, this
> could be spelled as loop.set_exception_handler(None)?
Good. Since we always need to have at least default exception
handler, using None in this context seems very logical.
>
> A few questions:
>
> - Do we need an API to ask the loop for its current exception handler?
I thought about this, and my reasoning for not proposing
'loop.get_exception_handler()' was to keep the API simple.

Besides, let's pretend we have this API in place, and you
can get the current handler. The getter will return the handler
to the caller, and that handler, or callback object will make
sense only to the application/code that set it in the first
place. And I think in this case, it's better to force the client
code to carefully use and track the calls to
'set_exception_handler()'.
> If
> so, what should it return if no handler is set explicitly?
If we make the default handler a public API, then we can return
a bound method. If we decide to hide it -- then 'None'.

>
> - Should we allow multiple exception handlers, or just one? (I'd prefer
> just one.)
The API will be more complex then, as we'll need functions to
remove exception handlers, at least. Applications can always
set one handler that will call other functions (as many as they
need).

>
> - Should we have a way to invoke the default handler explicitly? E.g. if
> the user's handler doesn't want to do anything special, they could pass it
> to the default handler instead of having to figure out how the default
> handler logs things.
I think it's a good idea to let users trigger the default handler
somehow.

One option is to add it to the public API. But it may cause some
confusion about when you should use it. Basically, the only place
it should be used in, is a custom exception handler.

Another option would be to let the custom handler simply re-raise
the exception, but this way we'll mix exceptions originated *in*
the handler with other exceptions, so I'd be -1 on this approach.

And another option is to establish a protocol: if a custom handler
returns an exception object, that object is passed to the default
handler.

>
> - Should we have a way to invoke the current handler explicitly? E.g. if
> there's user code that has encountered an exception and it wants that
> exception to be treated the same as the loop handles other exceptions.
>
-1 on this. Non-sophisticated code should just stick to the logging
module. The kind of code that uses its custom error handlers, can
develop its own facilities for error reporting.

Yury

Guido van Rossum

unread,
Feb 6, 2014, 3:44:28 PM2/6/14
to Yury Selivanov, python-tulip
On Thu, Feb 6, 2014 at 12:09 PM, Yury Selivanov <yseliv...@gmail.com> wrote:

On 2/6/2014, 1:24 PM, Guido van Rossum wrote:
On Thu, Feb 6, 2014 at 8:53 AM, Yury Selivanov <yseliv...@gmail.com>wrote:

As discussed with Guido in issue #80
(https://code.google.com/p/tulip/issues/detail?id=80),
I'm proposing a strawman design for having a user definable event loop
handler for unhandled exceptions.

Two new API points:

   - "loop.set_exception_handler(callback)"
     Sets 'callback' as a new unhandled error handler for the event loop
     'loop'. The signature of callback should be '(loop, exception,
context)'
     where: 'loop' parameter is the current event loop running; 'exception'
     is the unhandled exception object; 'context' is a string (str),
     that will be the basic context information, like
     "unhandled exception in add_reader callback" (the kind of information
     that usually goes to the log title).

I'd rename the context arg to 'message', then you don't have to explain
what it is. :-)
Agree. Need to make it clear, though, that 'message' is not
necessarily the same as 'exception.args[0]'

If it was we wouldn't need an extra argument for it. :-)
 


Also I'd steer clear from calling the handler 'callback' -- maybe just name
it 'exception_handler'?


     It's OK to call 'loop.stop()' or 'loop.call_*' methods from the
handler.

Cool.


     If a user set handler raises an exception, that exception will be
     handled by the default event loop exception handler (that will
     simply log it)

Nice.


   - "loop.restore_default_exception_handler()"
     Restores exception handler to the default one.
     Default exception handler just logs the error with
     'loop.logger.exception()'

Maybe instead of having a separate API for this rarely-used action, this
could be spelled as loop.set_exception_handler(None)?
Good. Since we always need to have at least default exception
handler, using None in this context seems very logical.


A few questions:

- Do we need an API to ask the loop for its current exception handler?
I thought about this, and my reasoning for not proposing
'loop.get_exception_handler()' was to keep the API simple.

Sure.
 
Besides, let's pretend we have this API in place, and you
can get the current handler. The getter will return the handler
to the caller, and that handler, or callback object will make
sense only to the application/code that set it in the first
place.

Not necessarily, you know its signature.
 
And I think in this case, it's better to force the client
code to carefully use and track the calls to
'set_exception_handler()'.

Not necessarily. But I agree to leave this out unless and until an actual real use case presents itself.

 
If
so, what should it return if no handler is set explicitly?
If we make the default handler a public API, then we can return
a bound method. If we decide to hide it -- then 'None'.



- Should we allow multiple exception handlers, or just one? (I'd prefer
just one.)
The API will be more complex then, as we'll need functions to
remove exception handlers, at least. Applications can always
set one handler that will call other functions (as many as they
need).

Right.


- Should we have a way to invoke the default handler explicitly? E.g. if
the user's handler doesn't want to do anything special, they could pass it
to the default handler instead of having to figure out how the default
handler logs things.
I think it's a good idea to let users trigger the default handler
somehow.

One option is to add it to the public API. But it may cause some
confusion about when you should use it. Basically, the only place
it should be used in, is a custom exception handler.

I suppose so. But this doesn't bother me much.
 
Another option would be to let the custom handler simply re-raise
the exception, but this way we'll mix exceptions originated *in*
the handler with other exceptions, so I'd be -1 on this approach.

Agreed.
 
And another option is to establish a protocol: if a custom handler
returns an exception object, that object is passed to the default
handler.

Eew. Special return values are very fragile in my experience.


- Should we have a way to invoke the current handler explicitly? E.g. if
there's user code that has encountered an exception and it wants that
exception to be treated the same as the loop handles other exceptions.

-1 on this. Non-sophisticated code should just stick to the logging
module. The kind of code that uses its custom error handlers, can
develop its own facilities for error reporting.

I'm not sure. The default exception handler may, over time, develop some useful custom behavior, and it's not like its signature is very complicated or using it is fraught with problems. So I don't see much downside to letting users call loop.default_exception_handler().
 
My suggestion would be two extra event loop methods:

loop.default_exception_handler(exception, message)
loop.call_exception_handler(exception, message)

The former is the default handler.

The latter calls the handler you set, if you set one, otherwise the default handler.

Yury Selivanov

unread,
Feb 6, 2014, 4:15:36 PM2/6/14
to gu...@python.org, python-tulip
On 2/6/2014, 3:44 PM, Guido van Rossum wrote:
> [snip]
> I'm not sure. The default exception handler may, over time, develop
> some useful custom behavior, and it's not like its signature is very
> complicated or using it is fraught with problems. So I don't see much
> downside to letting users call loop.default_exception_handler(). My
> suggestion would be two extra event loop methods:
> loop.default_exception_handler(exception, message)
> loop.call_exception_handler(exception, message) The former is the
> default handler. The latter calls the handler you set, if you set one,
> otherwise the default handler.

So let me reiterate on the proposed design:


loop.set_exception_handler(handler)
Sets 'handler' as a new unhandled exceptions handler.

If 'handler' is a callable object, then it should
have the following signature (or compatible one):
"(loop, exception, message)".

If 'callback' is None, default event loop exceptions
handler will be set.


loop.default_exception_handler(exception, message)
Triggers default exceptions handler of the event loop.


loop.call_exception_handler(exception, message)
Triggers the exceptions handler set with
'set_exception_handler'.

If there was no custom exceptions handler set, the
default one is triggered.


Yury

Victor Stinner

unread,
Feb 6, 2014, 5:06:38 PM2/6/14
to Yury Selivanov, python-tulip
Hi,

2014-02-06 17:53 GMT+01:00 Yury Selivanov <yseliv...@gmail.com>:
> I'm proposing a strawman design for having a user definable event loop
> handler for unhandled exceptions.

With my short experience of asyncio, I can say that it's a pain to
reconnect an "unhandled exception" to the future and the original
source code.

> - "loop.set_exception_handler(callback)"
> Sets 'callback' as a new unhandled error handler for the event loop
> 'loop'. The signature of callback should be '(loop, exception, context)'

... so it would be nice to give something to retrieve the source of
the exception. The minimum would be the future, task, or handle
object. The new parameter may be None if it doesn't apply.

I don't know why what's the best option to recover the original source
code from a task. An option is to use tracemalloc to get the traceback
where a task has been created. Another option is to always save the
traceback where a future has been created, directly in the future.
That's why I'm in favor of a "debug mode" for asyncio, to record the
traceback in debug mode, but don't record it in production to not add
any overhead.

The same problem applies from callbacks (handles, call_soon/call_at/etc.).

Victor

Guido van Rossum

unread,
Feb 6, 2014, 5:10:01 PM2/6/14
to Yury Selivanov, python-tulip
On Thu, Feb 6, 2014 at 1:15 PM, Yury Selivanov <yseliv...@gmail.com> wrote:
So let me reiterate on the proposed design:

This design looks good to me.

loop.set_exception_handler(handler)
   Sets 'handler' as a new unhandled exceptions handler.

I'd sy as "the" [...] handler. There is at most one handler, and setting it irrevocably overrides the previous handler.
 
   If 'handler' is a callable object, then it should
   have the following signature (or compatible one):
   "(loop, exception, message)".

   If 'callback' is None, default event loop exceptions
   handler will be set.


loop.default_exception_handler(exception, message)
   Triggers default exceptions handler of the event loop.


loop.call_exception_handler(exception, message)
   Triggers the exceptions handler set with
   'set_exception_handler'.

   If there was no custom exceptions handler set, the
   default one is triggered.

I guess the next step is specifying when this is called. Or just a patch. :-)

Yury Selivanov

unread,
Feb 6, 2014, 5:29:21 PM2/6/14
to Victor Stinner, python-tulip

On 2/6/2014, 5:06 PM, Victor Stinner wrote:
> Hi,
>
> 2014-02-06 17:53 GMT+01:00 Yury Selivanov <yseliv...@gmail.com>:
>> I'm proposing a strawman design for having a user definable event loop
>> handler for unhandled exceptions.
> With my short experience of asyncio, I can say that it's a pain to
> reconnect an "unhandled exception" to the future and the original
> source code.
>
>> - "loop.set_exception_handler(callback)"
>> Sets 'callback' as a new unhandled error handler for the event loop
>> 'loop'. The signature of callback should be '(loop, exception, context)'
> ... so it would be nice to give something to retrieve the source of
> the exception. The minimum would be the future, task, or handle
> object. The new parameter may be None if it doesn't apply.
I feel your pain ;)

However, I'm not sure how passing the future/handle to the
handler may help. You have the name of the failed callback
in the traceback of the exception. And, in many places all
you have is just the failed callback function, and no
related future or task.

Maybe you can show some real examples? (and implementation
suggestions)

Yury

Yury Selivanov

unread,
Feb 6, 2014, 5:33:41 PM2/6/14
to python...@googlegroups.com
I'll start working on the patch then..

Yury

Victor Stinner

unread,
Feb 6, 2014, 5:34:38 PM2/6/14
to python-tulip

Install an handler for unhandled exceptions is not something new: there is already sys.displayhook and sys.excepthook.

I'm not sure that it's useful to get the "default" handler. Just give access to the current handler. So when you setup a new handler, just call the previous if you don't know what to do with it. It is the design chosen for the new malloc API, PEP 445. So you just need two function: get and set.

For sys.displayhook/excepthook, the default is stored in sys.__displayhook/excepthook__.

Victor

Victor Stinner

unread,
Feb 6, 2014, 5:46:53 PM2/6/14
to Yury Selivanov, python-tulip

> However, I'm not sure how passing the future/handle to the
> handler may help.  You have the name of the failed callback
> in the traceback of the exception.  And, in many places all
> you have is just the failed callback function, and no
> related future or task.

I'm talking about Future destructor which logs the unhandled exception. The future is just self.

Do you know the new function tracemalloc.get_object_traceback()?

Victor

Yury Selivanov

unread,
Feb 6, 2014, 5:54:16 PM2/6/14
to Victor Stinner, python-tulip
Saw it briefly in the what's new, but haven't had a chance
to try it myself yet (looking forward to!)

A question to you and Guido: should the callback receive
three arguments: loop, exception, and context, where
context is a namedtuple (so we can add new fields to it
in future releases). For now, 'context' can have two fields:
'message' and 'callback'. The former is the log message, and
the latter is the failed callback function (or None).

Another option is to have a loop method to get the current/last
executed callback.

What do you think?

Yury

Yury Selivanov

unread,
Feb 6, 2014, 5:58:46 PM2/6/14
to python...@googlegroups.com
You're right, it's nothing new. However, I always hated the need
to save old hook, attach new one, then at some point restore the
old one. This explicit management of the old hook is almost never
gives you anything good. Yes, you have an option of stacking the
hooks on top of each other, but this is also rarely used. To me,
having an option to reset to or get the default hook looks
simpler. And, if needed, we can always add 'get_exception_handler'
to the spec later, right?

Yury

Guido van Rossum

unread,
Feb 6, 2014, 11:02:24 PM2/6/14
to Yury Selivanov, Victor Stinner, python-tulip
On Thu, Feb 6, 2014 at 2:54 PM, Yury Selivanov <yseliv...@gmail.com> wrote:
A question to you and Guido: should the callback receive
three arguments: loop, exception, and context, where
context is a namedtuple (so we can add new fields to it
in future releases). For now, 'context' can have two fields:
'message' and 'callback'. The former is the log message, and
the latter is the failed callback function (or None).

Hm... I don't like to have this much structure, but I do know that it's a pain to extend a callback API with new information, so I think it's a good idea. But maybe apart from the loop everything should be passed this way (i.e. also the exception)? And may I suggest to make it a dict instead of a named tuple? It's easier to check a dict for the presence of something that may or may not exist.
 
Another option is to have a loop method to get the current/last
executed callback.

Much less flexible.

BTW I don't like the excepthook/displayhook API much -- the sys module has lots of objects that you are allowed to assign to, but that's a bit of a historical accident (see e.g. the recent request to make sys.std{in,out,err} thread-local). I would like to go ahead with the proposed API (set_exception_handler, default_exception_handler, call_exception_handler). I don't see much of a use case for saving/restoring these outside unittests (where the better approach is to just create a new event loop). The expected usage pattern is to set a custom handler in main() right after creating the loop. A framework that wants some kind of "pass the buck" behavior can build that on top of the basic API combined with a convention.

Yury Selivanov

unread,
Feb 7, 2014, 2:17:02 PM2/7/14
to gu...@python.org, Victor Stinner, python-tulip

On 2/6/2014, 11:02 PM, Guido van Rossum wrote:
> On Thu, Feb 6, 2014 at 2:54 PM, Yury Selivanov<yseliv...@gmail.com>wrote:
>
>> >A question to you and Guido: should the callback receive
>> >three arguments: loop, exception, and context, where
>> >context is a namedtuple (so we can add new fields to it
>> >in future releases). For now, 'context' can have two fields:
>> >'message' and 'callback'. The former is the log message, and
>> >the latter is the failed callback function (or None).
>> >
> Hm... I don't like to have this much structure, but I do know that it's a
> pain to extend a callback API with new information, so I think it's a good
> idea. But maybe apart from the loop everything should be passed this way
> (i.e. also the exception)? And may I suggest to make it a dict instead of a
> named tuple? It's easier to check a dict for the presence of something that
> may or may not exist.
>
>
While working on the patch I encountered the following problem:
event loop may not always be available, if we decide to use
the new error handling API for "futures._TracebackLogger.__del__"
and 'futures.Future.__del__". And I think that those errors
should go through the new API.

Should we move the 'loop' callback parameter to the 'context'
dict? And make it optional too?

Yury

Guido van Rossum

unread,
Feb 7, 2014, 3:52:02 PM2/7/14
to Yury Selivanov, Victor Stinner, python-tulip
Can't you add a reference to the loop to the tb logger object? The loop should outlive any futures anyway (since the future has a reference to the loop) and it shouldn't be a ref cycle.


On Fri, Feb 7, 2014 at 11:17 AM, Yury Selivanov <yseliv...@gmail.com> wrote:

On 2/6/2014, 11:02 PM, Guido van Rossum wrote:

Yury Selivanov

unread,
Feb 7, 2014, 4:37:22 PM2/7/14
to gu...@python.org, Victor Stinner, python-tulip

On 2/7/2014, 3:52 PM, Guido van Rossum wrote:
> Can't you add a reference to the loop to the tb logger object? The loop
> should outlive any futures anyway (since the future has a reference to the
> loop) and it shouldn't be a ref cycle.
>
Sure.

Another question: "logger.exception" is also used in:

- selector_events.py: in _accept_connection, in case of errors in
pause_writing/resume_writing and _fatal_error

- proactor_events.py: in case of failed accept, _fatal_error and errors
in pause/resume writing

- unix_events.py: In pipe transport's _fatal_error, in case of
exception in SIGCHLD handler

- windows_events.py: pipe accept failed

All of the above sites are logging exceptions (typically OSErrors).
Should we use the loop exception API there, or you want to keep using
loggers directly?

And one more, aesthetic question: the currently agreed on signature of
exception handlers is '(loop, context)'. I have a method
"BaseEventLoop.default_exception_handler(self, context)", but when the
method is bound, its signature is just '(context)', hence, this bound
method cannot be passed to 'set_exception_handler'. Should the signature
be "BaseEventLoop.default_exception_handler(self, loop, context)", or
can I just make it a staticmethod with '(loop, context)'?

Yury

Guido van Rossum

unread,
Feb 7, 2014, 4:47:03 PM2/7/14
to Yury Selivanov, glyph.lefkowitz, Victor Stinner, python-tulip
On Fri, Feb 7, 2014 at 1:37 PM, Yury Selivanov <yseliv...@gmail.com> wrote:

On 2/7/2014, 3:52 PM, Guido van Rossum wrote:
Can't you add a reference to the loop to the tb logger object? The loop
should outlive any futures anyway (since the future has a reference to the
loop) and it shouldn't be a ref cycle.

Sure.

Another question: "logger.exception" is also used in:

- selector_events.py: in _accept_connection, in case of errors in pause_writing/resume_writing and _fatal_error

- proactor_events.py: in case of failed accept, _fatal_error and errors in pause/resume writing

- unix_events.py: In pipe transport's _fatal_error,  in case of exception in SIGCHLD handler

- windows_events.py: pipe accept failed

All of the above sites are logging exceptions (typically OSErrors). Should we use the loop exception API there, or you want to keep using loggers directly?

I've got a hunch saying that every place where we log an exception we should use the new handler, but TBH I don't have much of a use case -- I've been very happy with the default logging and I would probably just be configuring the logger rather than overriding the exception handler.

I think it's time to ask Glyph for a use case that can't be dealt with by overriding the logger.
 
And one more, aesthetic question: the currently agreed on signature of exception handlers is '(loop, context)'. I have a method "BaseEventLoop.default_exception_handler(self, context)", but when the method is bound, its signature is just '(context)', hence, this bound method cannot be passed to 'set_exception_handler'. Should the signature be "BaseEventLoop.default_exception_handler(self, loop, context)", or can I just make it a staticmethod with '(loop, context)'?

But why would you want to pass the default handler to set_exception_handler()? I had expected that there would be something like this:

def call_exception_handler(self, exception, context):
    if self._exception_handler is not None:
        try:
            self._exception_handler(self, exception, context)
            return
        except Exception as exc:
            self.default_exception_handler(exc, {'message': 'custom exception handler failed'})
        self.default_exception_handler(exception, context)

Yury Selivanov

unread,
Feb 7, 2014, 4:53:55 PM2/7/14
to gu...@python.org, glyph.lefkowitz, Victor Stinner, python-tulip

On 2/7/2014, 4:47 PM, Guido van Rossum wrote:
>> >And one more, aesthetic question: the currently agreed on signature of
>> >exception handlers is '(loop, context)'. I have a method
>> >"BaseEventLoop.default_exception_handler(self, context)", but when the
>> >method is bound, its signature is just '(context)', hence, this bound
>> >method cannot be passed to 'set_exception_handler'. Should the signature be
>> >"BaseEventLoop.default_exception_handler(self, loop, context)", or can I
>> >just make it a staticmethod with '(loop, context)'?
>> >
> But why would you want to pass the default handler to
> set_exception_handler()? I had expected that there would be something like
> this:
>
> def call_exception_handler(self, exception, context):
> if self._exception_handler is not None:
> try:
> self._exception_handler(self, exception, context)
> return
> except Exception as exc:
> self.default_exception_handler(exc, {'message': 'custom
> exception handler failed'})
> self.default_exception_handler(exception, context)
That's almost exactly the code I currently have.

And the question is rather rhetorical: i.e. if it's OK to pass
"loop.default_exception_handler" to "loop.set_exception_handler" [1]
or it shouldn't be used that way, always use
"loop.set_exception_handler(None) [2]". I like the [2], and that's
how I have it implemented now, but wanted to confirm it with the list.

Yury

Guido van Rossum

unread,
Feb 7, 2014, 5:00:21 PM2/7/14
to Yury Selivanov, glyph.lefkowitz, Victor Stinner, python-tulip
OK, don't change it. :-)

Yury Selivanov

unread,
Feb 7, 2014, 5:03:02 PM2/7/14
to gu...@python.org, glyph.lefkowitz, Victor Stinner, python-tulip
On 2/7/2014, 4:47 PM, Guido van Rossum wrote:
> On Fri, Feb 7, 2014 at 1:37 PM, Yury Selivanov<yseliv...@gmail.com>wrote:
>
>> >
>> >On 2/7/2014, 3:52 PM, Guido van Rossum wrote:
>> >
>>> >>Can't you add a reference to the loop to the tb logger object? The loop
>>> >>should outlive any futures anyway (since the future has a reference to the
>>> >>loop) and it shouldn't be a ref cycle.
>>> >>
>>> >> Sure.
>> >
>> >Another question: "logger.exception" is also used in:
>> >
>> >- selector_events.py: in _accept_connection, in case of errors in
>> >pause_writing/resume_writing and _fatal_error
>> >
>> >- proactor_events.py: in case of failed accept, _fatal_error and errors in
>> >pause/resume writing
>> >
>> >- unix_events.py: In pipe transport's _fatal_error, in case of exception
>> >in SIGCHLD handler
>> >
>> >- windows_events.py: pipe accept failed
>> >
>> >All of the above sites are logging exceptions (typically OSErrors). Should
>> >we use the loop exception API there, or you want to keep using loggers
>> >directly?
>> >
> I've got a hunch saying that every place where we log an exception we
> should use the new handler, but TBH I don't have much of a use case -- I've
> been very happy with the default logging and I would probably just be
> configuring the logger rather than overriding the exception handler.
>
> I think it's time to ask Glyph for a use case that can't be dealt with by
> overriding the logger.
>
>
I'm +1 on using the loop API everywhere. It just gives us more
control and flexibility in improving/working with error reporting
later down the road. Using logger directly feels inconsistent.

Yury

Guido van Rossum

unread,
Feb 7, 2014, 5:18:02 PM2/7/14
to Yury Selivanov, glyph.lefkowitz, Victor Stinner, python-tulip
OK, go for it unless Glyph pipes up. :-)


On Fri, Feb 7, 2014 at 2:03 PM, Yury Selivanov <yseliv...@gmail.com> wrote:
On 2/7/2014, 4:47 PM, Guido van Rossum wrote:

Glyph

unread,
Feb 10, 2014, 6:12:55 PM2/10/14
to Guido van Rossum, Yury Selivanov, Victor Stinner, python-tulip
I have a use-case, which is to direct Twisted's logging to Tulip's (or vice versa) in a way which behaves consistently.  Using the logger directly is, as I said in a message a long time ago, a really wide interface for alternate logging systems to try to use.

(And stdlib logging.py is slow enough that it can seriously start to impact your frame rate if you're trying to do logging from a GUI, so there's still a pretty substantial use-case for Twisted's logging client-side.  This is probably less true than it was 10 years ago when I really cared deeply about this for my day job, but nevertheless, I still dream of making some cool games in Python ;-).)

Glyph

unread,
Feb 10, 2014, 6:37:56 PM2/10/14
to Guido van Rossum, Yury Selivanov, Victor Stinner, python-tulip
Oops, it was pointed out to me that this message isn't super clear.

To put it more simply:

Please go ahead and do this, using the loop API everywhere would be a lot better for my uses :-).

-glyph
Reply all
Reply to author
Forward
0 new messages