Hi,
I have been troubleshooting an issue with gevent's
WSGIServer where performing a graceful shutdown (via
server.stop()) prevents new connections but does not close existing connections, and also does not prevent processing of new requests from those existing connections. The result is that the server takes a long time to shut down (as it's waiting for connections to close, but doing nothing to close them) and requests are often killed mid-processing at the end of the timeout period.
I've attached simple reproduction code below in [1]. The results look like this:
$ venv/bin/python main.py
Waiting 5 seconds before calling server.stop()...
127.0.0.1 - - [2024-10-27 14:11:35] "GET / HTTP/1.1" 200 87 2.007094
Stopping...
127.0.0.1 - - [2024-10-27 14:11:37] "GET / HTTP/1.1" 200 87 2.005282
127.0.0.1 - - [2024-10-27 14:11:39] "GET / HTTP/1.1" 200 87 2.004275
127.0.0.1 - - [2024-10-27 14:11:41] "GET / HTTP/1.1" 200 87 2.005276
127.0.0.1 - - [2024-10-27 14:11:43] "GET / HTTP/1.1" 200 87 2.003820
127.0.0.1 - - [2024-10-27 14:11:45] "GET / HTTP/1.1" 200 87 2.002957
127.0.0.1 - - [2024-10-27 14:11:46] "GET / HTTP/1.1" 500 161 1.112180
Stopped after 10.00 seconds
In my curl output:
* Re-using existing connection with host localhost
> GET / HTTP/1.1
> Host: localhost:8000
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 500 Internal Server Error
< Content-Type: text/plain
< Connection: close
< Content-Length: 21
< Date: Sun, 27 Oct 2024 19:11:46 GMT
<
* Closing connection
Internal Server Error
From the above, you can see that it takes 10 seconds to shut down (the full
stop_timeout) and that the last request was killed mid-processing.
I am wondering if there's a way to have gevent proactively close connections in the pool when shutdown is triggered in order to improve shutdown speed and avoid killing requests mid-processing.
I've been reading through gevent source code but can't find a way to achieve this without subclassing
WSGIServer and/or
WSGIHandler. I landed on the workaround provided below in [2]. After that change the server shuts down as soon as it finishes processing the current request:
$ venv/bin/python main.py
Waiting 5 seconds before calling server.stop()...
127.0.0.1 - - [2024-10-27 14:20:42] "GET / HTTP/1.1" 200 87 2.006489
Stopping...
127.0.0.1 - - [2024-10-27 14:20:44] "GET / HTTP/1.1" 200 87 2.004722
Stopped after 0.61 seconds
This seems to work although I am not sure if this solution introduces new issues. Does that seem like a reasonable approach, or is there something I'm missing?
If you think this is worth addressing upstream, I would be happy to try submitting a patch.
Thanks,
Chris
[1] Simple reproduction:
# Tested with Python 3.12.7, gevent 24.10.3
import gevent.pool
import gevent.pywsgi
import time
def app(environ, start_response):
gevent.sleep(2)
start_response("200 OK", [])
return [b"hello world"]
def main():
server = gevent.pywsgi.WSGIServer(
("127.0.0.1", 8000),
application=app,
spawn=gevent.pool.Pool(),
)
server.stop_timeout = 10
gevent.spawn(server.serve_forever)
print("Waiting 5 seconds before calling server.stop()...")
gevent.sleep(5)
print("Stopping...")
start = time.time()
server.stop()
print(f"Stopped after {time.time() - start:.2f} seconds")
if __name__ == '__main__':
main()To send several requests over a single connection, you can use curl with multiple URLs:
curl -v localhost:8000 localhost:8000 localhost:8000 localhost:8000 localhost:8000 localhost:8000 localhost:8000[2] Workaround is adding these two classes:
class GracefulShutdownWSGIServer(gevent.pywsgi.WSGIServer):
shutdown_event: gevent.event.Event
def __init__(self, *args, **kwargs):
self.shutdown_event = gevent.event.Event()
super().__init__(*args, **kwargs)
def stop(self, *args, **kwargs):
self.shutdown_event.set()
super().stop(*args, **kwargs)
class GracefulShutdownWSGIHandler(gevent.pywsgi.WSGIHandler):
"""WSGI handler which proactively closes connections when the server is in shutdown."""
_shutdown_event: gevent.event.Event
# Flag representing whether the base class thinks the connection should be
# closed. The base class sets `self.close_connection` based on the HTTP
# version and headers, which we intercept using a property setter into this
# attribute.
_close_connection: bool = False
def __init__(self, sock, address, server):
self._shutdown_event = server.shutdown_event
super().__init__(sock, address, server)
@property
def close_connection(self):
# This property overrides `close_connection` in the base class which is
# used to control keepalive behavior.
return self._close_connection or self._shutdown_event.is_set()
@close_connection.setter
def close_connection(self, value):
# This setter allows the base class to set `self.close_connection`
# directly, while still allowing us to override the value when we know
# the server is in shutdown.
self._close_connection = value
def read_requestline(self):
real_read_requestline = gevent.spawn(super().read_requestline)
ready = gevent.wait([self._shutdown_event, real_read_requestline], count=1)
if self._shutdown_event in ready:
real_read_requestline.kill()
# None triggers the base class to close the connection.
return None
ret = real_read_requestline.get()
if isinstance(ret, BaseException):
raise ret
return ret
def handle_one_request(self):
ret = super().handle_one_request()
if ret is True and self._shutdown_event.is_set():
return None
return ret...and then constructing the server like this:
server = GracefulShutdownWSGIServer(
("127.0.0.1", 8000),
application=app,
spawn=gevent.pool.Pool(),
handler_class=GracefulShutdownWSGIHandler,
)