Non-multithread compatible modules and waitress

[Zsolt Ero]

I had a few questions in
https://github.com/Pylons/waitress/issues/253, which were not really
an issue and I'd like to ask them here instead.

My question mostly is that if I'm using modules which are
non-multithread compatible in pserve / waitress, how can I make sure
they work correctly? Shouldn't I reset / reinitialize them at some
point of the startup process?

For example, Google client Python libraries, which are not thread
safe, as they are based on httplib2.
https://developers.google.com/api-client-library/python/guide/thread_safety

Zsolt

[Theron Luhn]

By reset/reinit on startup, I assume you’re thinking of how you handle safely forking?  E.g. SQLAlchemy’s engine.dispose() post-fork

With forked workers, everything is kept separate, but forking breaks file handles (which includes network connections), which is why you need to reset things like SQLAlchemy. 

Threaded workers are a different beast, anything application-level will be shared among all threads. 

Generally I use non-thread-safe things as reified properties in the request object. 

config.add_request_method(lambda _: Http(), ‘ghttp’, reified=True)

That will create a new object per request and avoid any threading issues.

If per-request is too expensive, you could use a threadlocal, but that’s much more complicated. 

https://stackoverflow.com/a/13240093

— Theron

--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
To post to this group, send email to pylons-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAKw-smCwEmXb-qyh%3D0hfSHTjqt%3DOMV_BzSaAaSn3FKGYYbOkhQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.