Pymongo, Replication, Slaves, Round-robin queries
James Gregory
Quick question: we've just turned on replication for our database, for
reasons of reliability, however it occurs to me that we should be able
to use these replicas to increase the read-performance (or at least
service more parallel queries) on queries where consistency is not a
requirement. We are lucky enough to have several such situations, but
I'm unclear from the docs on how to do this.
This page:
http://www.mongodb.org/display/DOCS/Querying#Querying-slaveOk
Seems to suggest that setting slaveOk on the connection will yield a
connection that will connect to an arbitrary slave. I'm unclear as to
whether that means it will connect to a single, arbitrary slave and
issue all subsequent requests to that one server, or whether it will
connect to a different one each time (round-robin, random, something
like that).
It also notes that some drivers allow a way to specify this per-query.
This would be ideal, but pymongo seems not to be one of these drivers,
unless I'm missing something.
PyMongo's docs seem to make it look like by specifying slave_okay on the
Connection, I can connect to a single slave and issue queries against
it. That makes it sound like I'm meant to open a "writer" connection (to
the master), and n "reader" connections, one to each of my replicas, and
then manage distributing queries amongst them all myself.
How is this meant to work? I'm ok with any of the above options (or even
something completely different), but am having trouble understanding the
specifics of this stuff from the docs.
Thanks for your help.
Cheers,
James.
Michael Dirolf
> Hi,
>
> Quick question: we've just turned on replication for our database, for
> reasons of reliability, however it occurs to me that we should be able to
> use these replicas to increase the read-performance (or at least service
> more parallel queries) on queries where consistency is not a requirement. We
> are lucky enough to have several such situations, but I'm unclear from the
> docs on how to do this.
>
> This page:
>
> http://www.mongodb.org/display/DOCS/Querying#Querying-slaveOk
>
> Seems to suggest that setting slaveOk on the connection will yield a
> connection that will connect to an arbitrary slave. I'm unclear as to
> whether that means it will connect to a single, arbitrary slave and issue
> all subsequent requests to that one server, or whether it will connect to a
> different one each time (round-robin, random, something like that).
>
> It also notes that some drivers allow a way to specify this per-query. This
> would be ideal, but pymongo seems not to be one of these drivers, unless I'm
> missing something.
>
> PyMongo's docs seem to make it look like by specifying slave_okay on the
> Connection, I can connect to a single slave and issue queries against it.
> That makes it sound like I'm meant to open a "writer" connection (to the
> master), and n "reader" connections, one to each of my replicas, and then
> manage distributing queries amongst them all myself.
This is one approach that will work. There is also a
MasterSlaveConnection class in PyMongo, which will handle some of this
automatically, but I expect that that might change somewhat as we work
to standardize support for this sort of thing across drivers.
> How is this meant to work? I'm ok with any of the above options (or even
> something completely different), but am having trouble understanding the
> specifics of this stuff from the docs.
>
> Thanks for your help.
>
> Cheers,
>
> James.
>
> --
> You received this message because you are subscribed to the Google Groups
> "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to
> mongodb-user...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/mongodb-user?hl=en.
>
>
James Gregory
>> PyMongo's docs seem to make it look like by specifying slave_okay on the
>> Connection, I can connect to a single slave and issue queries against it.
>> That makes it sound like I'm meant to open a "writer" connection (to the
>> master), and n "reader" connections, one to each of my replicas, and then
>> manage distributing queries amongst them all myself.
>>
> This is one approach that will work. There is also a
> MasterSlaveConnection class in PyMongo, which will handle some of this
> automatically, but I expect that that might change somewhat as we work
> to standardize support for this sort of thing across drivers.
>
>
Ok. Thanks for the clarification. This then sounds like I'd need to
implement a layer to handle failover, as if one of my replicated
machines went down, I'd need to catch that and try another server. Is
that right?
Cheers,
James.
Michael Dirolf
Correct. My guess is that driver support for doing this sort of thing
for you will probably improve with time, though. There's also been
some discussion of talking to replica sets through a mongos instance
to get some of this behavior for free. Currently, though, the best bet
is handling it in your application code.