os.environ and threadsafe: true

[Jason Collins]

The top answer in this SO thread suggests that os.environ should be generally avoided because it's not threadsafe in the threadsafe: true Python 2.7 environment.

  http://stackoverflow.com/questions/5901653/name-of-current-app-in-google-app-engine-python  

I'm dubious because the WSGI spec explicitly uses os.environ to construct its request objects, etc.

Can anyone confirm/deny the safety of using os.environ in a threadsafe: true deployment?

Thanks,

j

[Kyle Finley]

According to Nick Johnson (a former App Engine team member) os.environ is threadsafe:

But what if my app makes use of 'os.environ' or other CGI-isms, you ask? All is not lost. The smart folks on the Python 2.7 runtime team foresaw that some apps will inevitably do this, and built in a workaround for them. 'os.environ' and a few other bits and pieces are now "thread local" on the 2.7 runtime, meaning that each thread (and hence each request) see a different copy of them, containing only the data relevant to the current request. Apps that expect to get request information from os.environ can thus continue to work fine. Bear in mind that this really is a workaround, though - it's definitely cleaner to rewrite your apps to rely only on the WSGI environment, if you have the opportunity. The WSGI environment can be accessed as self.request.environ if you're using webapp.

http://blog.notdot.net/2011/10/Migrating-to-Python-2-7-part-1-Threadsafe

I don't know if it is documented anywhere else, I suspected it is.

- Kyle

[Anand Mistry]

In the python27 runtime, os.environ is thread safe since it is implemented as request/thread-local. However, this means that changes in one request will not be visible by other requests. Also, since os.environ is conceptually a global dictionary, I would recommend against writing to it whenever possible. If you need to access request data, you should be reading from the first positional parameter passed to the WSGI application when it is invoked.

Regarding the relationship between the WSGI environ and os.environ, there is no direct relationship specified in PEP333. What PEP333 says is:

The environ parameter is a dictionary object, containing CGI-style environment variables....

In addition to the CGI-defined variables, the environ dictionary may also contain arbitrary operating-system "environment variables"...

Finally, some applications, frameworks, and middleware may wish to use the environ dictionary to receive simple string configuration options. Servers and gateways should support this by allowing an application's deployer to specify name-value pairs to be placed in environ. In the simplest case, this support can consist merely of copying all operating system-supplied environment variables from os.environ into the environ dictionary, since the deployer in principle can configure these externally to the server, or in the CGI case they may be able to be set via the server's configuration files.

This last case we handle by supporting specifying environment variables in app.yaml.