Using replica sets with mongokit - mongodb-user

The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Message from discussion Using replica sets with mongokit
Date: Fri, 27 Apr 2012 16:06:39 -0700 (PDT)
From: =?UTF-8?Q?=C3=89ric_Araujo?= <eara...@socialludia.com>
To: mongodb-user@googlegroups.com
Message-ID: <32674627.100.1335567999119.JavaMail.geo-discussion-forums@vbbfk16>
Subject: Using replica sets with mongokit
MIME-Version: 1.0
Content-Type: multipart/mixed; 
	boundary="----=_Part_98_22764519.1335567999117"

------=_Part_98_22764519.1335567999117
Content-Type: multipart/alternative; 
	boundary="----=_Part_99_15644175.1335567999117"

------=_Part_99_15644175.1335567999117
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hello,

I=E2=80=99ve been learning how to use MongoDB with Python for a few days in=
 order=20
to update some code so that reads go to a secondary replica.
After much reading and trying I have hit a wall and would like to confirm a=
=20
few things to see what direction I should take. I don=E2=80=99t mind gettin=
g
rtfm if you give me the address of the specific friendly manual, or being=
=20
redirected to the mongokit mailing list if you tell me the Mongo setup
is fine and the problem is on the Python side.

The code has many classes that subclass mongokit.Document and share a=20
common connection for find, save and remove.  In my test
suite I start three mongod nodes, give them 30 seconds to create their=20
files and connect to the repl set primary with this idiom:
mongokit.Connection('mongodb://127.0.0.1:27031/?replicaSet=3Dtstest').  Aft=
er=20
configuring the repl set and waiting 30 more seconds I can see
that writes propagate to all replicas, but I can=E2=80=99t get reads to go =
to the=20
secondaries.  From the pymongo docs I thought that I would be able to
use the same connection object and pass read_preference=3DSECONDARY (this=
=20
supersedes slave_ok) to the find method, but that does not
work.  I get no error but the setting seems to have no effect (I check with=
=20
system.profile.find; more on that later).  Is that expected?

There is also a ReplicaSetConnection in pymongo but I don=E2=80=99t underst=
and when=20
one should use Connection or ReplicaSetConnection, and
anyway I have to use mongokit.Connection which inherits from=20
pymongo.Connection.  I tried testing with a subclass of
pymongo.ReplicaSetConnection and mongokit.Connection but quickly stopped=20
after seeing they did not work together.

I finally read on the mongokit ML and a blog that one needs to open a=20
second connection to a secondary in order to send reads to it.  Is
this the only way?  It would be a bit inconvenient for the code I=E2=80=99m=
 working=20
on.  Currently the Mongo configuration is one string in a config
file, i.e. a full URI to the repl set primary with the repl set name=20
embedded, which works great for different environments (dev/stage/etc)
and automatically handles changes in the repl set config.  If I had to put=
=20
the URI of a secondary server in the config files it would lose
the auto-discovery and config changes adaptation, which are great features=
=20
of replica sets.  Then in the code I would have to open a
second connection in addition to the existing one, and handle failover=20
myself.

Last, I=E2=80=99m not sure that my test code is right.  To check that the r=
eads go=20
to the secondaries, I wanted to use profiling, but again I can=E2=80=99t ge=
t
information just for a secondary.  If I use my main connection object with=
=20
read_preference=3DSECONDARY I get the same info than that from
the primary, and if I try to make a second connection and get profiling=20
info it fails with AutoReconnect('master has changed').  Is profiling the
right call for what I want to do?  Is it normal that I can=E2=80=99t work w=
ith two=20
parallel connections?  I could of course change the code, run the
application on a test server and check the logs manually, but if possible I=
=20
would prefer to have an automated test to make sure that reads go
to secondaries.

Versions: mongod 2.0.4, pymongo 2.1.1, mongokit 0.7.2.

Thanks in advance for any idea; I=E2=80=99ll check email for some time toda=
y and=20
then on Monday.
Best regards

------=_Part_99_15644175.1335567999117
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Hello,<br><br>I=E2=80=99ve been learning how to use MongoDB with Python for=
 a few days in order to update some code so that reads go to a secondary re=
plica.<br>After much reading and trying I have hit a wall and would like to=
 confirm a few things to see what direction I should take. I don=E2=80=99t =
mind getting<br>rtfm if you give me the address of the specific friendly ma=
nual, or being redirected to the mongokit mailing list if you tell me the M=
ongo setup<br>is fine and the problem is on the Python side.<br><br>The cod=
e has many classes that subclass mongokit.Document and share a common conne=
ction for find, save and remove.&nbsp; In my test<br>suite I start three mo=
ngod nodes, give them 30 seconds to create their files and connect to the r=
epl set primary with this idiom:<br>mongokit.Connection('mongodb://127.0.0.=
1:27031/?replicaSet=3Dtstest').&nbsp; After configuring the repl set and wa=
iting 30 more seconds I can see<br>that writes propagate to all replicas, b=
ut I can=E2=80=99t get reads to go to the secondaries.&nbsp; From the pymon=
go docs I thought that I would be able to<br>use the same connection object=
 and pass read_preference=3DSECONDARY (this supersedes slave_ok) to the fin=
d method, but that does not<br>work.&nbsp; I get no error but the setting s=
eems to have no effect (I check with system.profile.find; more on that late=
r).&nbsp; Is that expected?<br><br>There is also a ReplicaSetConnection in =
pymongo but I don=E2=80=99t understand when one should use Connection or Re=
plicaSetConnection, and<br>anyway I have to use mongokit.Connection which i=
nherits from pymongo.Connection.&nbsp; I tried testing with a subclass of<b=
r>pymongo.ReplicaSetConnection and mongokit.Connection but quickly stopped =
after seeing they did not work together.<br><br>I finally read on the mongo=
kit ML and a blog that one needs to open a second connection to a secondary=
 in order to send reads to it.&nbsp; Is<br>this the only way?&nbsp; It woul=
d be a bit inconvenient for the code I=E2=80=99m working on.&nbsp; Currentl=
y the Mongo configuration is one string in a config<br>file, i.e. a full UR=
I to the repl set primary with the repl set name embedded, which works grea=
t for different environments (dev/stage/etc)<br>and automatically handles c=
hanges in the repl set config.&nbsp; If I had to put the URI of a secondary=
 server in the config files it would lose<br>the auto-discovery and config =
changes adaptation, which are great features of replica sets.&nbsp; Then in=
 the code I would have to open a<br>second connection in addition to the ex=
isting one, and handle failover myself.<br><br>Last, I=E2=80=99m not sure t=
hat my test code is right.&nbsp; To check that the reads go to the secondar=
ies, I wanted to use profiling, but again I can=E2=80=99t get<br>informatio=
n just for a secondary.&nbsp; If I use my main connection object with read_=
preference=3DSECONDARY I get the same info than that from<br>the primary, a=
nd if I try to make a second connection and get profiling info it fails wit=
h AutoReconnect('master has changed').&nbsp; Is profiling the<br>right call=
 for what I want to do?&nbsp; Is it normal that I can=E2=80=99t work with t=
wo parallel connections?&nbsp; I could of course change the code, run the<b=
r>application on a test server and check the logs manually, but if possible=
 I would prefer to have an automated test to make sure that reads go<br>to =
secondaries.<br><br>Versions: mongod 2.0.4, pymongo 2.1.1, mongokit 0.7.2.<=
br><br>Thanks in advance for any idea; I=E2=80=99ll check email for some ti=
me today and then on Monday.<br>Best regards<br>
------=_Part_99_15644175.1335567999117--

------=_Part_98_22764519.1335567999117--
Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
Account Options
