Waitress and multiprocess pooling

645 views
Skip to first unread message

Mikko Ohtamaa

unread,
May 12, 2022, 4:46:55 PM5/12/22
to pylons-...@googlegroups.com
Hi,

I am running a Pyramid based website and now traffic is picking up. I feel I might bump the problems of Waitress scalability soon. As far as I understand, Waitress does not offer multiple process pooling modes, only threading. This, combined with Python GIL, might cause problems if requests start to be more and more CPU bound.

- Am I correct and Waitress is limited by Python threading?

- What's the recommended modern multiprocess enabled web server to do more scaleable Pyramid hosting?

Thank you,
Mikko

Ps. Site is here, also we are hiring if anyone is looking for gigs or jobs in Pyramid + SQLAlchemy:


Chris McDonough

unread,
May 12, 2022, 5:09:30 PM5/12/22
to pylons-...@googlegroups.com

> - Am I correct and Waitress is limited by Python threading?
> - What's the recommended modern multiprocess enabled web server to do more scaleable Pyramid hosting?

Apologies for a "pop up" visit here, I have been AWOL forever, but I think I do know an answer to at least the first question.

If you choose to stick with Waitress, and you are indeed bottlenecked by the GIL, and you have a single-host-single-instance deployment, you could change it in a way that looks a lot like a solution for running on multiple hosts. For example, if you now run a single Waitress instance on TCP port 8080, you could spin one up on 8081, one on 8082, etc. It's the same scaling solution as might exist across multiple hosts, except just on one host.

Then you'd need some sort of proxy that is willing to round-robin requests (or whatever scheduling you want) to each of those instances.

I don't know the current state of the world wrt new coolness in WSGI servers is. That said, the set of changes to your codebase that are required to support a multiprocessing model rather than a multithreading WSGI model are the same regardless of which server you choose, so I'd suggest just trying it first with Waitress, find the stuff that doesn't work or is awkward and fix it. If you decide to use another WSGI server, that work won't be wasted.

- C

Jonathan Vanasco

unread,
May 12, 2022, 6:14:06 PM5/12/22
to pylons-discuss
- What's the recommended modern multiprocess enabled web server to do more scaleable Pyramid hosting?

I like uWSGI, but others like gunicorn and I think there is another popular option. 

Regardless of the server you choose, please be aware they may (though I am pretty sure they all will) all cause issues with your current codebase due to the characteristics of copy-on-write memory optimization.  Usually the issues I've seen are with database connections and random seed generators.  Most of the libraries you are probably using will support correcting this with specific calls to make after the process forks.  If you're using SqlAlchemy, look at Engine.dispose() and most crypto libraries have an `atfork` function. 

I just wanted to put that on your radar now, so you can audit your code for this stuff if you decide to switch your deployment.

Mikko Ohtamaa

unread,
May 13, 2022, 6:28:17 AM5/13/22
to pylons-...@googlegroups.com

Apologies for a "pop up" visit here, I have been AWOL forever, but I think I do know an answer to at least the first question.


You can check out any time you like But you can never leave.



Mikko Ohtamaa

unread,
May 13, 2022, 6:29:47 AM5/13/22
to pylons-...@googlegroups.com
 

Regardless of the server you choose, please be aware they may (though I am pretty sure they all will) all cause issues with your current codebase due to the characteristics of copy-on-write memory optimization.  Usually the issues I've seen are with database connections and random seed generators.  Most of the libraries you are probably using will support correcting this with specific calls to make after the process forks.  If you're using SqlAlchemy, look at Engine.dispose() and most crypto libraries have an `atfork` function. 

Thank you Jonathan. This is golden, rarely seen, insight.

Br,
Mikko


Chris McDonough

unread,
May 13, 2022, 10:57:04 AM5/13/22
to pylons-...@googlegroups.com

> You can check out any time you like But you can never leave.
> https://www.youtube.com/watch?v=EqPtz5qN7HM

https://www.youtube.com/watch?v=_xRg-r2SohE

- C

Thierry Florac

unread,
May 16, 2022, 9:13:08 PM5/16/22
to pylons-...@googlegroups.com
I personally use Apache with mod_wsgi, in multi-processes mode (from 1 to 16, based on vCPU count).
I know it's probably not the most "modern" option, but I know it, it works pretty well and I didn't notice any problem (I use a PostgreSQL ZODB with RelStorage and SQLAlchemy together).
I also used GUnicorn for a while, but only for async-based applications...

Best regards,
Thierry


--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/20f1a129-1853-412b-a707-ebeb11db10e2n%40googlegroups.com.

Mikko Ohtamaa

unread,
May 18, 2022, 3:03:26 AM5/18/22
to pylons-...@googlegroups.com
On Fri, 13 May 2022 at 00:14, Jonathan Vanasco <jvan...@gmail.com> wrote:
- What's the recommended modern multiprocess enabled web server to do more scaleable Pyramid hosting?

I like uWSGI, but others like gunicorn and I think there is another popular option.

I decided to go with Gunicorn, because I used uWSGI extensively. Gunicorn is somewhat easier to manage with modern Python package management tools like Poetry, plus you do not encounter the potential issues raising from compiling C.

So far so good, only option I missing from Gunicorn is Waitress's "url_prefix". Is there a generic way for setting the WSGI application root /  URL prefix that is not specific to Waitress?


Br,
Mikko

Michael Merickel

unread,
May 19, 2022, 8:14:54 PM5/19/22
to pylons-...@googlegroups.com
Mikko,

The best agnostic impl I know for url prefix is the PasteDeploy prefix middleware.


You can mount it in your ini pipeline pretty easily:

[pipeline:main]
pipeline =
    prefix
    myapp

[filter:prefix]
use = egg:PasteDeploy#prefix
prefix = /foo


--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.

Mikko Ohtamaa

unread,
May 23, 2022, 5:52:12 AM5/23/22
to pylons-...@googlegroups.com
Hi all,

Thank you for your kind support.

We have been running a Gunicorn for half a week now and you can clearly where we switched over, as the request latency is much more stable none. We also managed to solve all configure problems with Gunicorn thanks to your kind help.

Backend latency:

image.png

Even PostgreSQL latency is more controlled now, don't though not sure if I understand why - probably something to do with PSQL, Waitress, psycopg2 and SQLAlchemy connection pooling.

image.png


Br,
Mikko


Michael Merickel

unread,
May 23, 2022, 10:12:53 AM5/23/22
to pylons-...@googlegroups.com
Mikko,

What is your gunicorn config?


On May 23, 2022, at 04:52, Mikko Ohtamaa <mi...@redinnovation.com> wrote:


Hi all,

Thank you for your kind support.

We have been running a Gunicorn for half a week now and you can clearly where we switched over, as the request latency is much more stable none. We also managed to solve all configure problems with Gunicorn thanks to your kind help.

Backend latency:

<image.png>

Even PostgreSQL latency is more controlled now, don't though not sure if I understand why - probably something to do with PSQL, Waitress, psycopg2 and SQLAlchemy connection pooling.

<image.png>



Br,
Mikko


--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.

Mikko Ohtamaa

unread,
May 23, 2022, 10:18:01 AM5/23/22
to pylons-...@googlegroups.com
Hi,


What is your gunicorn config?

The config is below. Just to clarify, the performance issues (latency) were with Waitress and now after switching to gunicorn all the issues have pretty much disappeared.

gunicorn \
  --name backend \
  --bind 127.0.0.1:3456 \
  --workers 8 \
  --threads 2 \
  --env "SCRIPT_NAME=/api" \
  --statsd-host "localhost:8125" \
  --access-logfile logs/gunicorn_access.log \
  --error-logfile logs/gunicorn_error.log \
  "backend.server.server:gunicorn_entry_point()"  
Reply all
Reply to author
Forward
0 new messages