Check failed: !IsQuiesced()

[Matthias Verstraete]

Hi all,

We are currently experiencing problems with a grpc setup.

Client and server are both running on the same system. Each is running in a separate docker container. The server is implemented in c++ and is using the ubuntu system package (1.30.2-3build6). The client is implemented in python and is using grpcio 1.67.1 (originally we used 1.71.1, but some online threads suggested to rollback to 1.67.1 which unfortunately didn't help).

On the client side we see the following log line appear regularly:

fork_posix.cc:77] Other threads are currently calling into gRPC, skipping fork() handlers

And after a while the following log line shows up

work_stealing_thread_pool.cc:241] Check failed: !IsQuiesced()

At that point our python process crashes.

We are currently still trying to find more information about what is happening, but it's hard to reproduce this issue with a minimal setup. What we tried already:

* We tried reproducing the issue on another system with same OS, docker images and test setup and failed to reproduce it.

* We tried to run a simple python script which does grpc calls (without our client code) but failed to reproduce it then.

* Our server communicates to external devices via a different protocol. When we disabled this communication, we also could reproduce the problem anymore.

If anyone might have an insight of what might be happening or in which direction we should look, that would be greatly appreciated.

Thanks and kind regards,

Matthias

[Matthias Verstraete]

It turned out to be a bug in another library we were using.

It used grpc in the background as well.

Kind regards,

Matthias

Op dinsdag 29 april 2025 om 16:53:07 UTC+2 schreef Matthias Verstraete: