pymongo connection pool and thread safety

[Alx Peter]

We are analyzing MongoDB with Python in our next web project.
I've read that with pymongo, we can share connection and database instances across different threads. Also pymongo gives a connection pooling mechanism as well.
Now, if we can share DB instance (from a global connection instance) then, is there a case where connection pooling of any usage ? we could just reuse the DB instance for every request (one request per thread) and leaving the single connection open through out the life time of app. Whats the use case to 'end_request()' on connection after every operation/end-of current thread and get it back from pool next time ?
Any performance benefit ? between switching different connection Vs locking on same connection etc... Seems I'm missing something.
Please correct me if I'm wrong, Sorry for the basic noob question.

Rgds
Pet.

[Bernie Hackett]

> we could just reuse the DB instance for every request (one request per thread) and leaving the single connection open through out the life time of app.

Yes, PyMongo uses a socket per thread to connect. You can share one
instance of connection.Connection (or database.Database) among
multiple threads.

> Whats the use case to 'end_request()' on connection after every operation/end-of current thread and get it back from pool next time ?

PyMongo does not automatically return connections to the pool for
reuse by other threads. You have to call end_request to do that.
PyMongo also does not limit the number of connections it will make. If
there is no connection in the pool to reuse when your application
spawns a new thread PyMongo will just create a new connection. Re-
using connections by calling end_request will limit the total number
of connections to mongod.

[Alx Peter]

Thanks, Any idea if there is any performance difference between these three setup:-

1) Initialize with just 1 connection and share it across multiple threads and create one DB instance for each thread.
2) Initialize with just 1 connection; create 1 DB instance; and share this DB instance across multiple threads.
3) Using connection pool, initialize it with, say, 10 connections. Get one connection per request-thread and create DB instance for each thread and return connection (end_request) once the processing is completed.

[Bernie Hackett]

There really isn't any difference between 1 and 2 since all the
instances of database.Database are using the same instance of
connection.Connection.

For 3, if you are using a static size pool of worker threads that pull
jobs off a queue calling end_request isn't as important. If you are
always spawning new threads you always want to make sure you call
end_request before a thread dies.