Re: Local thread variable (attempting to create a connection pool)

[Chris Cogdon]

What you essentially want is a common pattern called a "singleton": something that there is only one of, like "None" and "Elipsis". You can do something workable, but ugly, using module-level variables, but there are known patterns for creating singletons

Here's some starters:

http://stackoverflow.com/questions/42558/python-and-the-singleton-pattern

On Friday, December 14, 2012 9:33:19 AM UTC-8, Rafael Almeida wrote:

Hello,

I have a django application which needs to connect to some backend services. Problem is I don't want to create a new connection everytime, but to use a connection pool. One connection per thread would be fine, but I don't know how to use django in order to achive that. I'd like to perhaps save those connections on cache subsystem, but I don't know how to create one key per thread, per worker.

Thank you for your help,

Rafael

[Rafael Almeida]

I'm not sure it's really a singleton that I need because I need a different value for each thread. I'm using threading.local for it, so far I haven't run into any issues, but I'd like to hear the opinion of a more experienced django developer.

I save in cache the possible IPs of the backend system and I open connections to it when needed inside a threading.local subclass. I use an object from such class in order to retrieve a connection.

The threading.local subclass is instantiated in a module variable, I could use the singleton class you pointed out as well. But I don't see the need for it in my case.

[Russell Keith-Magee]

Can a thread local work? Yes.

Are they a good idea? IMHO - Generally speaking, no.

From a purely architectural point of view, it doesn't matter what you call them -- you're talking about a global variable. Any argument that holds for why global variables are bad also holds for thread locals. They make everything harder to test. They make problems harder to isolate. If you can avoid using them, you should.

The other big problem is that you need to be *really* careful. In the context of a web server, a thread can last for multiple requests. This means you need to be absolutely certainly that you're not leaking *any* state between requests -- especially user state. If you leak *any* state between requests, you run the risk of introducing a security problem, because a request made by one user can leak sensitive information (account session IDs, credentials, and so on) to another user. Of course, what gets leaked depends entirely on what you're putting in the thread local -- but if you don't put it in a thread local in the first place, it can't leak :-)

Making matters worse, the problems won't generally reveal themselves during development; it's only when you have high volume of requests and a number of different threads being reused extensively that you start to see any problems with leaking state. 

So - can they be used? Yes. Django uses thread locals in a couple of places. If we had our time over (and none of the architectural constraints), would we avoid using them? Absolutely. 

Yours,

Russ Magee %-)

--
You received this message because you are subscribed to the Google Groups "Django users" group.

To view this discussion on the web visit https://groups.google.com/d/msg/django-users/-/hsdCWcl-2owJ.

To post to this group, send email to django...@googlegroups.com.
To unsubscribe from this group, send email to django-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.