Server-Sent Events (SEE) blocking all threads

[Orestis Zambounis]

My flask application implements SSE to push updates to each of my users. I.e. each of my users gets individual updates and therefore unique SSE connections. I deploy the application to mod_wsgi. Now mod_wsgi allows me to use a fixed number of threads (default 15) which are quickly all blocked when the number of users grows. This happens as the SSE connection are kept open and are therefore blocking the thread. Subsequent requests can't get in anymore and I get a Script timed out before returning headers: wsgi.pyerror.

My current workaround is to put a timer in my SSE generator and break it after a certain threshold. Still, with many users a fixed number of threads will still lead to blocked requests.

def stream():
...
queue = memory.get(...)
@stream_with_context
def eventStream():
try:
start = time.time()
while True:
delta = start - time.time()
if delta > app.config["SSE_TIMEOUT"]:
break
# wait for source data to be available, then push it
try:
entry = queue.get( True, timeout=app.config["SSE_TIMEOUT"] - delta)
except Empty:
break
ev = ServerSentEvent(json.dumps(entry))
yield ev.encode()
finally:
...
return Response(eventStream(), mimetype="text/event-stream")

I'd like top find a clean solution to be able to handle an arbitrary amount of users which are able to receive updates through SSE without blocking all threads in wsgi.


I posted the issue also on stackoverflow: https://stackoverflow.com/questions/57345375/server-sent-events-sse-blocking-threads-in-mod-wsgi

[Graham Dumpleton]

The only WSGI servers where this would work is those which are implemented using greenlets. Thus, eventlet and gevent modes of gunicorn. Any WSGI server which uses normal processes/threads, SSE will not really work well because of restrictions on size of workers available.

> --
> You received this message because you are subscribed to the Google Groups "modwsgi" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/7528d624-f52b-4520-8509-0db74d31a2cd%40googlegroups.com.

[Orestis Zambounis]

Many thanks. I'm going to dive into gunicorn, gevents etc.
As I am deploying to Elastic beanstalk which is set up with Apache/mod_wsgi by default would you recommend to look into switching the server to gunicorn or use some other strategy for real-time communication such as sockets in terms of effort (given that I have no knowledge about setting up gunicorn or websockets) instead? Do websockets present the same problems as SSE in my usecase for mod_wsgi?

[Graham Dumpleton]

> On 6 Aug 2019, at 21:03, Orestis Zambounis <orestis....@gmail.com> wrote:
>
> Many thanks. I'm going to dive into gunicorn, gevents etc.
> As I am deploying to Elastic beanstalk which is set up with Apache/mod_wsgi by default would you recommend to look into switching the server to gunicorn or use some other strategy for real-time communication such as sockets in terms of effort (given that I have no knowledge about setting up gunicorn or websockets) instead? Do websockets present the same problems as SSE in my usecase for mod_wsgi?

Yes they do and websockets likely will not even work at all.

You are better off using an async server and not wsgi for this use case.

Graham

>
> --
> You received this message because you are subscribed to the Google Groups "modwsgi" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/d98597b5-fce7-42ec-ae01-6f0d154ac810%40googlegroups.com.