PyMongo seems not to honor ReadPreference for a command

[ACh]

Dear All,
if I am not completely wrong, the PyMongo 3.0.3 documentation claims:

"

Note that since most database commands can only be sent to the primary of a replica set, the command() method does not obey the Database’s read_preference, but you can pass an explicit read preference to the method:

>>> db.command('dbstats', read_preference=ReadPreference.NEAREST)
{...}

"

Nevertheless at least dbstats always return a result from the primary member with PyMongo, while it works correctly in the Mongo shell. This is a bit of an issue when working with mixed storage engines (Primary on WT, Secondaries with MMAPv1).

In my case I was running the command on a sharded cluster while connected to a Mongo router (mongos).

Please check the following, trivial, excerpts:

• Via Mongo shell:
  

mongo --username superuser --password xxxxxxxx admin
mongos> db.getMongo().setReadPref('primary')
mongos> db.getSisterDB('afumigatus_cb_v3').runCommand({ dbStats: 1, scale: 1 }).storageSize
30728192
mongos> db.getMongo().setReadPref('secondary')
mongos> db.getSisterDB('afumigatus_cb_v3').runCommand({ dbStats: 1, scale: 1 }).storageSize
124157952

• Using PyMongo:

In [1]: import pymongo

In [2]: import urllib

In [3]: mongo_host = 'mongos-hxvm-001'

In [4]: mongodb_user = 'superuser'

In [5]: mongodb_pwd  = 'xxxxxxxx'

In [6]: username = urllib.quote_plus(mongodb_user)

In [7]: password = urllib.quote_plus(mongodb_pwd)

In [8]: uri = 'mongodb://' + username + ':' + password + '@' + mongo_host + '/admin?authMechanism=MONGODB-CR'

In [9]: client = pymongo.MongoClient(uri)

In [10]: client
Out[10]: MongoClient('mongos-hxvm-001', 27017)

In [11]: client.read_preference
Out[11]: Primary()

In [12]: db = client.afumigatus_cb_v3

In [13]: from pymongo import ReadPreference

In [14]: db.command('dbstats', read_preference=ReadPreference.PRIMARY)['storageSize']
Out[14]: 30728192

In [15]: db.command('dbstats', read_preference=ReadPreference.SECONDARY)['storageSize']
Out[15]: 30728192

Having a look around I found this bug report: https://jira.mongodb.org/browse/PYTHON-691, which is about a similar topic and has been closed. In the comments the list read_preferences.secondary_ok_commands is cited, but I can't find any sign of that.

I have tried with PyMongo 3.0.3 and PyMongo 3.0.2 without any luck.

Could someone please give me a clue?

Thanks a lot in advance
Regards,
ACh

[ACh]

Hi All,
I have run another couple of tests and the issue is only experienced when connecting via the mongos. If I directly connect to one of the shards, listing the replica set members in the URI, I get the correct result:

In [32]: uri = 'mongodb://' + username + ':' + password + '@' + mongo_host + 'hx-mongo-002:27018,hh-mongo-003:27018,hh-mongo-004:27018' + '/admin?authMechanism=MONGODB-CR'

In [33]: client = pymongo.MongoClient(uri)

In [34]: client
Out[34]: MongoClient([u'hx-mongo-002.ebi.ac.uk:27018', u'hh-mongo-003.ebi.ac.uk:27018', u'hh-mongo-004.ebi.ac.uk:27018'])

In [35]: db = client.afumigatus_cb_v3

In [36]: db.command('dbstats', read_preference=ReadPreference.PRIMARY)['storageSize']
Out[36]: 30728192.0

In [37]: db.command('dbstats', read_preference=ReadPreference.SECONDARY)['storageSize']
Out[37]: 124157952.0

Any idea about why the ReadPreference is not honoured when the query is routed by mongos?

Thanks
Regards,
ACh

[Bernie Hackett]

What version of MongoDB? When you run the command from pymongo against mongos, what do you see in the mongos log? The command and readPreference should be logged.

[ACh]

Hi Bernie,
the MongoDB cluster is on version 3.0.6.
I have browsed the mongos log file, but can't find any trace of the command or the related read preference. Should I increase the default verbosity to catch that?

Thanks
Regards,
Alessio