Python 3.9 asyncio: Task cancel() throws asyncio.exceptions.CancelledError instead of asyncio.CancelledError

[Clint Olsen]

I'm having some issues with task cancellation inside a signal handler. My tasks do get cancelled, but I see odd behavior:

Traceback (most recent call last):
File "/home/utils/release/sw/tools/python-3.9.7/lib/python3.9/site-packages/grpc/aio/_call.py", line 406, in _consume_request_iterator
async for request in request_iterator:
File "/home/foo/lib/python/rush/client.py", line 105, in send_status
status = await sendq.get()
File "/home/utils/Python/3.9/3.9.7-20211101/lib/python3.9/asyncio/queues.py", line 166, in get
await getter
asyncio.exceptions.CancelledError: Client killed with signal 2

Attempting to catch asyncio.CancelledError or asyncio.CancelledError does not work. The function in question looks like:

101 async def send_status():
102 while True:
103 logging.info('Waiting for sendq')
104 try:
105 status = await sendq.get()
106
107 if not status:
108 break
109
110 logging.info(f'Sendq status {status}')
111
112 message = rush_pb2.Status(**status)
113 yield message
114 except asyncio.exceptions.CancelledError as err:
115 logging.error(f'Weird error {err}')
116 #raise asyncio.CancelledError
117 break

If I use "except:" it does catch this exception.

What's going on here?

Thanks,

-Clint

[Chris Angelico]

>>> asyncio.exceptions.CancelledError is asyncio.CancelledError
True

Does that answer the question?

ChrisA

[Clint Olsen]

On Friday, March 31, 2023 at 3:23:24 PM UTC-7, Chris Angelico wrote:

> On Sat, 1 Apr 2023 at 09:19, Clint Olsen <xxx...@gmail.com> wrote:
> > Attempting to catch asyncio.CancelledError or asyncio.CancelledError does not work. The function in question looks like:

> >>> asyncio.exceptions.CancelledError is asyncio.CancelledError
> True
>
> Does that answer the question?

No, I couldn't catch either exception even though they are the same.

-Clint

[Chris Angelico]

Okay, so that deals with the part from the subject line, leaving a
slightly different problem: The caught exception is not of the same
type as you were expecting. First question: Can you reproduce the
issue on command? If so, I would recommend trying this:

except BaseException as e:
print("Got an exception!", type(e))
print(id(type(e)))
print(id(asyncio.CancelledError)
except:
print("Weird things are happening")
import sys
print(sys.exc_info())
print(id(sys.exc_info()[0]))
print(id(asyncio.CancelledError))

Basically, I want to know whether (a) BaseException has changed, which
would be a nightmare; and (b) whether asyncio.CancelledError has
changed.

This is the kind of bizarre behaviour that can happen if a module is
reloaded, or if there are multiple versions loaded for some reason.
But if that ISN'T what's happening, there'll have to be some other
explanation.

ChrisA

[Clint Olsen]

On Friday, March 31, 2023 at 4:14:51 PM UTC-7, Chris Angelico wrote:
> Okay, so that deals with the part from the subject line, leaving a
> slightly different problem: The caught exception is not of the same
> type as you were expecting. First question: Can you reproduce the
> issue on command? If so, I would recommend trying this:
>
> except BaseException as e:
> print("Got an exception!", type(e))
> print(id(type(e)))
> print(id(asyncio.CancelledError)
> except:
> print("Weird things are happening")
> import sys
> print(sys.exc_info())
> print(id(sys.exc_info()[0]))
> print(id(asyncio.CancelledError))
>
> Basically, I want to know whether (a) BaseException has changed, which
> would be a nightmare; and (b) whether asyncio.CancelledError has
> changed.
>
> This is the kind of bizarre behaviour that can happen if a module is
> reloaded, or if there are multiple versions loaded for some reason.
> But if that ISN'T what's happening, there'll have to be some other
> explanation.

Here's what I see:

Got an exception! <class 'asyncio.exceptions.CancelledError'>
8204064
8204064

Thanks,

-Clint

[Chris Angelico]

Can you confirm that it is indeed failing to catch the exception? Try this:

except asyncio.CancelledError:
print("Cancelled correctly")

followed by the same type checking from above. Since the ID is the
same, I would expect it to match!

Can you post a full runnable example that exhibits the problem?

ChrisA

[Clint Olsen]

On Friday, March 31, 2023 at 5:51:33 PM UTC-7, Chris Angelico wrote:
> Can you confirm that it is indeed failing to catch the exception? Try this:
>
> except asyncio.CancelledError:
> print("Cancelled correctly")
>
> followed by the same type checking from above. Since the ID is the
> same, I would expect it to match!

Well, now this works, so it's difficult to say how that was happening. If I can reproduce it I'll update this conversation. Thanks for the help!

-Clint