Pymongo is much slower than client mongo shell?

[Brandt]

I'm relatively new to mongodb, and having a performance problem in pymongo. I have a collection that's 50 GBs (uncompressed) with about 39 million documents. Querying it over indexed fields gives a result that's about 125,000 documents and 150 MBs. When I do the following in the mongo shell, it takes about a second.

  var result = db.my_collection.find(my_query).toArray()

However, when I do the same thing in pymongo, it takes over 7 seconds.

  db = pymongo.MongoClient()['my_db']

  result = list(db['my_collection'].find(my_query)) 

Some extra info:

- I'm using Ubuntu 14.04, python 2.7.6, pymongo 3.2, and mongo 3.2.

- I think my pymongo is configured to use C because I installed python-dev and both pymongo.has_c() and bson.has_c() show True.

- Everything is run locally.

I find it hard to believe that pymongo is 7x slower than the mongo shell. What am I missing?

Thanks!

[Kevin Adistambha]

Hi Brandt,

I tried to reproduce your tests using the same versions of PyMongo, MongoDB, and the mongo shell but didn’t see any significant speed difference.

To help us investigate, could you provide more details, such as:

• How did you perform the benchmark? For example, did you run both tests with a shell script or directly in an interactive mongo/python session?

• How did you measure the time it took for the scripts to run, and how many times did you run the scripts to determine the average execution time?

• What sort of hardware did you run the tests on? (CPU, total/available memory, virtual machine or bare metal)

Best regards,
Kevin