Groups keyboard shortcuts have been updated
Dismiss
See shortcuts

Graceful shutdown of pywsgi.WSGIServer with keepalive connections

4 views
Skip to first unread message

Chris Kuehl

unread,
Oct 28, 2024, 6:29:52 AM10/28/24
to gev...@googlegroups.com
Hi,

I have been troubleshooting an issue with gevent's WSGIServer where performing a graceful shutdown (via server.stop()) prevents new connections but does not close existing connections, and also does not prevent processing of new requests from those existing connections. The result is that the server takes a long time to shut down (as it's waiting for connections to close, but doing nothing to close them) and requests are often killed mid-processing at the end of the timeout period.

I've attached simple reproduction code below in [1]. The results look like this:

$ venv/bin/python main.py
Waiting 5 seconds before calling server.stop()...
127.0.0.1 - - [2024-10-27 14:11:35] "GET / HTTP/1.1" 200 87 2.007094
Stopping...
127.0.0.1 - - [2024-10-27 14:11:37] "GET / HTTP/1.1" 200 87 2.005282
127.0.0.1 - - [2024-10-27 14:11:39] "GET / HTTP/1.1" 200 87 2.004275
127.0.0.1 - - [2024-10-27 14:11:41] "GET / HTTP/1.1" 200 87 2.005276
127.0.0.1 - - [2024-10-27 14:11:43] "GET / HTTP/1.1" 200 87 2.003820
127.0.0.1 - - [2024-10-27 14:11:45] "GET / HTTP/1.1" 200 87 2.002957
127.0.0.1 - - [2024-10-27 14:11:46] "GET / HTTP/1.1" 500 161 1.112180
Stopped after 10.00 seconds

In my curl output:

* Re-using existing connection with host localhost
> GET / HTTP/1.1
> Host: localhost:8000
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 500 Internal Server Error
< Content-Type: text/plain
< Connection: close
< Content-Length: 21
< Date: Sun, 27 Oct 2024 19:11:46 GMT
<
* Closing connection
Internal Server Error

From the above, you can see that it takes 10 seconds to shut down (the full stop_timeout) and that the last request was killed mid-processing.

I am wondering if there's a way to have gevent proactively close connections in the pool when shutdown is triggered in order to improve shutdown speed and avoid killing requests mid-processing.

I've been reading through gevent source code but can't find a way to achieve this without subclassing WSGIServer and/or WSGIHandler. I landed on the workaround provided below in [2]. After that change the server shuts down as soon as it finishes processing the current request:

$ venv/bin/python main.py
Waiting 5 seconds before calling server.stop()...
127.0.0.1 - - [2024-10-27 14:20:42] "GET / HTTP/1.1" 200 87 2.006489
Stopping...
127.0.0.1 - - [2024-10-27 14:20:44] "GET / HTTP/1.1" 200 87 2.004722
Stopped after 0.61 seconds

This seems to work although I am not sure if this solution introduces new issues. Does that seem like a reasonable approach, or is there something I'm missing?

If you think this is worth addressing upstream, I would be happy to try submitting a patch.

Thanks,
Chris


[1] Simple reproduction:

# Tested with Python 3.12.7, gevent 24.10.3
import gevent.pool
import gevent.pywsgi
import time


def app(environ, start_response):
    gevent.sleep(2)
    start_response("200 OK", [])
    return [b"hello world"]


def main():
    server = gevent.pywsgi.WSGIServer(
        ("127.0.0.1", 8000),
        application=app,
        spawn=gevent.pool.Pool(),
    )
    server.stop_timeout = 10
    gevent.spawn(server.serve_forever)

    print("Waiting 5 seconds before calling server.stop()...")
    gevent.sleep(5)

    print("Stopping...")
    start = time.time()
    server.stop()

    print(f"Stopped after {time.time() - start:.2f} seconds")


if __name__ == '__main__':
    main()


To send several requests over a single connection, you can use curl with multiple URLs:

curl -v localhost:8000 localhost:8000 localhost:8000 localhost:8000 localhost:8000 localhost:8000 localhost:8000

[2] Workaround is adding these two classes:

class GracefulShutdownWSGIServer(gevent.pywsgi.WSGIServer):
    shutdown_event: gevent.event.Event

    def __init__(self, *args, **kwargs):
        self.shutdown_event = gevent.event.Event()
        super().__init__(*args, **kwargs)

    def stop(self, *args, **kwargs):
        self.shutdown_event.set()
        super().stop(*args, **kwargs)


class GracefulShutdownWSGIHandler(gevent.pywsgi.WSGIHandler):
    """WSGI handler which proactively closes connections when the server is in shutdown."""

    _shutdown_event: gevent.event.Event

    # Flag representing whether the base class thinks the connection should be
    # closed. The base class sets `self.close_connection` based on the HTTP
    # version and headers, which we intercept using a property setter into this
    # attribute.
    _close_connection: bool = False

    def __init__(self, sock, address, server):
        self._shutdown_event = server.shutdown_event
        super().__init__(sock, address, server)

    @property
    def close_connection(self):
        # This property overrides `close_connection` in the base class which is
        # used to control keepalive behavior.
        return self._close_connection or self._shutdown_event.is_set()

    @close_connection.setter
    def close_connection(self, value):
        # This setter allows the base class to set `self.close_connection`
        # directly, while still allowing us to override the value when we know
        # the server is in shutdown.
        self._close_connection = value

    def read_requestline(self):
        real_read_requestline = gevent.spawn(super().read_requestline)
        ready = gevent.wait([self._shutdown_event, real_read_requestline], count=1)

        if self._shutdown_event in ready:
            real_read_requestline.kill()
            # None triggers the base class to close the connection.
            return None

        ret = real_read_requestline.get()
        if isinstance(ret, BaseException):
            raise ret
        return ret

    def handle_one_request(self):
        ret = super().handle_one_request()
        if ret is True and self._shutdown_event.is_set():
            return None
        return ret


...and then constructing the server like this:

server = GracefulShutdownWSGIServer(
    ("127.0.0.1", 8000),
    application=app,
    spawn=gevent.pool.Pool(),
    handler_class=GracefulShutdownWSGIHandler,
)

Reply all
Reply to author
Forward
0 new messages