Temporary errors on mongo findOne (null response for existing keys) - mongodb-user

The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Message from discussion Temporary errors on mongo findOne (null response for existing keys)
Received-SPF: neutral (google.com: 209.85.217.175 is neither permitted nor denied by best guess record for domain of da...@guzik.com.ar) client-ip=209.85.217.175;
MIME-Version: 1.0
In-Reply-To: <8404403.157.1335221760961.JavaMail.geo-discussion-forums@ynjb15>
References: <29de63fe-2411-4d22-a0ff-3a1562fe5...@v22g2000yqm.googlegroups.com>
 <CA+Y0caMcfvnoyyJoYdSyq4d3fk31wiWAopwOAZFsh0dbvck...@mail.gmail.com>
 <CA+Y0caNbFQpx=OyJDVsvbXR1ToW25VrHkMG9g825WW1v37_...@mail.gmail.com>
 <17541497.72.1335217435759.JavaMail.geo-discussion-forums@ynnn9>
 <CA+Y0caN3MP=K3KZbT9_+2e4gQjzpdheOG-a7Fgir02p4em=...@mail.gmail.com> <8404403.157.1335221760961.JavaMail.geo-discussion-forums@ynjb15>
From: Dario Alejandro Guzik <da...@guzik.com.ar>
Date: Tue, 24 Apr 2012 12:24:04 -0300
Message-ID: <CA+Y0caM5Dr8yKMaY3eKM6JCSYiHcK_fxxq0i2=mtg42YBD5...@mail.gmail.com>
Subject: Re: [mongodb-user] Temporary errors on mongo findOne (null response
 for existing keys)
To: mongodb-user@googlegroups.com
Content-Type: multipart/alternative; boundary=f46d04083ad7927a4904be6e5996

--f46d04083ad7927a4904be6e5996
Content-Type: text/plain; charset=ISO-8859-1

I am checking right now if the misses corresponds to new documents to check
if it is a replication timing issue.

Just to check: findAndModify is slower than update? (leaving aside other
aspects like concurrency handling, etc)

2012/4/23 Randolph Tan <rando...@10gen.com>

> Are you using slaveOk? It could just be that the document was not fully
> replicated to the secondaries yet.
>
>
> On Monday, April 23, 2012 5:53:28 PM UTC-4, Dario Guzik wrote:
>>
>>
>>
>> 2012/4/23 Randolph Tan <rando...@10gen.com>
>>
>>> Hi,
>>>
>>> I have a couple questions.
>>>
>>> 1. Can you explain what you meant by "key that is allocated in the
>>> default shard (the same that holds the queues) ". Specifically, what is a
>>> key? Are you referring to the actual document?
>>>
>> Sorry, a little confusing. The key I'm referring to is actually a
>> document on an unsharded collection. I am doing findOne(field:something)
>> using the Java API 2.7.2 and the result is null. I know the object is there
>> and no exception is thrown. If I retry the findOne 100ms later I get the
>> document.
>>
>>
>>> 2. Did you drop the collection and created a new one with the same name?
>>>
>> No, actually It;s a kind of difficult task since the DB is already in
>> production stage and has 500K documents and it's being used
>>
>>
>>> 3. Can you attach the sample docs of these queries?
>>>
>> yep, rather simple documents:
>>
>> { "_id" : ObjectId("**4f7330d6487c95ae7196eab5"), "value" : "**
>> AAACfZA9IiSpcBABYpw3pXXXXXXXXX**XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX**XXXXXXXXXX",
>> "client_id" : "175268255910551", "expires" : ISODate("2012-05-27T15:40:04.
>> **978Z"), "scope" : [       "scope1",      "scope2",   "scope3" ],
>> "status" : "valid", "next_update" : ISODate("2012-04-27T15:40:53.**676Z"),
>> "inner_id" : "13XXXXX773" }
>>
>>  the query is over inner_id (not the real name because of privacy issues)
>> indexes are
>> [
>>         {
>>                 "v" : 1,
>>                 "key" : {
>>                         "_id" : 1
>>                 },
>>                 "ns" : "mybase.data_collection",
>>                 "name" : "_id_"
>>         },
>>         {
>>                 "v" : 1,
>>                 "key" : {
>>                         "inner_id" : 1
>>                 },
>>                  "unique" : true,
>>                 "ns" : "mybase.data_collection",
>>                 "name" : "inner_id_1"
>>         },
>>         {
>>                 "v" : 1,
>>                 "key" : {
>>                         "next_update" : 1
>>                 },
>>                 "ns" : "mybase.data_collection",
>>                 "name" : "next_update_1"
>>         }
>>
>>
>>
>>
>>> Increasing the replica set members can make write performance worse if
>>> you use safe writes. However, adding a shard can help distribute the write
>>> load.
>>>
>>> On Monday, April 23, 2012 5:24:10 PM UTC-4, Dario Guzik wrote:
>>>>
>>>> What bothers me most is that many read operations that should be super
>>>> fast are taking 100+ ms and still no clue of what might be happening
>>>>
>>>> 2012/4/23 Dario Alejandro Guzik <da...@guzik.com.ar>
>>>>
>>>>> Here you can find stats:
>>>>> PRIMARY> db.stats()
>>>>> {
>>>>>         "db" : "social",
>>>>>         "collections" : 23,
>>>>>         "objects" : 56243345,
>>>>>         "avgObjSize" : 614.3731572864309,
>>>>>          "dataSize" : 34554401444,
>>>>>         "storageSize" : 37738672016,
>>>>>         "numExtents" : 313,
>>>>>         "indexes" : 34,
>>>>>         "indexSize" : 4673139968,
>>>>>         "fileSize" : 47155511296,
>>>>>         "nsSizeMB" : 16,
>>>>>         "ok" : 1
>>>>> }
>>>>>
>>>>>
>>>>> 2012/4/23 Dario Guzik <da...@guzik.com.ar>
>>>>>
>>>>>> Hi!
>>>>>>
>>>>>> Some context first: We are working on a project using mongoDB as our
>>>>>> main data storage. There are three shards which are recplica sets of 3
>>>>>> instances. Each one of them with enough RAM an CPU (quad cores and
>>>>>> 16GB RAM)
>>>>>> Most collections are sharded but some processing queues that are not
>>>>>> sharded.
>>>>>> We are experiencing some problems when using this queues. When We
>>>>>> start the process that reads these queues and does a lot of writting
>>>>>> sometimes the mongos asking for a key that is allocated in the default
>>>>>> shard (the same that holds the queues) gets a null response, but then
>>>>>> you can manually check that key and it is there in all three instances
>>>>>> of the rs. I assume it has something to do with the heavy writting
>>>>>> load that the rs is coping with.
>>>>>> I need help to find out which path should We take to solve this
>>>>>> errors. Expanding the rs to 5 members is a good idea? moving some
>>>>>> queues elsewhere?
>>>>>>
>>>>>> I have db.stats() from the shard and some mongostats log if it helps.
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "mongodb-user" group.
>>>>>> To post to this group, send email to mongodb-user@googlegroups.com.
>>>>>> To unsubscribe from this group, send email to
>>>>>> mongodb-user+unsubscribe@**googl**egroups.com<mongodb-user%2Bunsubscribe@googlegroups.com>
>>>>>> .
>>>>>> For more options, visit this group at http://groups.google.com/**
>>>>>> group**/mongodb-user?hl=en<http://groups.google.com/group/mongodb-user?hl=en>
>>>>>> .
>>>>>>
>>>>>>
>>>>>
>>>>  --
>>> You received this message because you are subscribed to the Google
>>> Groups "mongodb-user" group.
>>> To view this discussion on the web visit https://groups.google.com/d/**
>>> msg/mongodb-user/-/**d5nC5I89DSAJ<https://groups.google.com/d/msg/mongodb-user/-/d5nC5I89DSAJ>
>>> .
>>>
>>> To post to this group, send email to mongodb-user@googlegroups.com.
>>> To unsubscribe from this group, send email to mongodb-user+unsubscribe@*
>>> *googlegroups.com <mongodb-user%2Bunsubscribe@googlegroups.com>.
>>> For more options, visit this group at http://groups.google.com/**
>>> group/mongodb-user?hl=en<http://groups.google.com/group/mongodb-user?hl=en>
>>> .
>>>
>>
>>  --
> You received this message because you are subscribed to the Google Groups
> "mongodb-user" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/mongodb-user/-/asi0puyd_3UJ.
>
> To post to this group, send email to mongodb-user@googlegroups.com.
> To unsubscribe from this group, send email to
> mongodb-user+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/mongodb-user?hl=en.
>

--f46d04083ad7927a4904be6e5996
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div class=3D"gmail_extra">I am checking right now if the misses correspond=
s to new documents to check if it is a replication timing issue.</div><div =
class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">Just to check: f=
indAndModify is slower than update? (leaving aside other aspects like concu=
rrency handling, etc)<br>

<br><div class=3D"gmail_quote">2012/4/23 Randolph Tan <span dir=3D"ltr">&lt=
;<a href=3D"mailto:rando...@10gen.com" target=3D"_blank">rando...@10gen.com=
</a>&gt;</span><br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 =
.8ex;border-left:1px #ccc solid;padding-left:1ex">

Are you using slaveOk? It could just be that the document was not fully rep=
licated to the secondaries yet.<div><div class=3D"h5"><br><br>On Monday, Ap=
ril 23, 2012 5:53:28 PM UTC-4, Dario Guzik wrote:<blockquote class=3D"gmail=
_quote" style=3D"margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padd=
ing-left:1ex">

<div><br><br><div class=3D"gmail_quote">2012/4/23 Randolph Tan <span dir=3D=
"ltr">&lt;<a href=3D"mailto:rando...@10gen.com" target=3D"_blank">randolph@=
10gen.com</a>&gt;</span><br><blockquote class=3D"gmail_quote" style=3D"marg=
in:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">



Hi,<div><br></div><div>I have a couple questions.</div><div><br></div><div>=
1. Can you explain what you meant by &quot;key that is allocated in the def=
ault=A0shard (the same that holds the queues)=A0&quot;. Specifically, what =
is a key? Are you referring to the actual document?</div>



</blockquote><div>Sorry, a little confusing. The key I&#39;m referring to i=
s actually a document on an unsharded collection. I am doing findOne(field:=
something) using the Java API 2.7.2 and the result is null. I know the obje=
ct is there and no exception is thrown. If I retry the findOne 100ms later =
I get the document.</div>



<div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;=
border-left:1px #ccc solid;padding-left:1ex"><div>2. Did you drop the colle=
ction and created a new one with the same name?</div></blockquote><div>No, =
actually It;s a kind of difficult task since the DB is already in productio=
n stage and has 500K documents and it&#39;s being used</div>



<div>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;=
border-left:1px #ccc solid;padding-left:1ex"><div>3. Can you attach the sam=
ple docs of these queries?</div></blockquote><div>yep, rather simple docume=
nts:</div>



<div><br></div><div><div>{ &quot;_id&quot; : ObjectId(&quot;<u></u>4f7330d6=
487c95ae7196eab5&quot;), &quot;value&quot; : &quot;<u></u>AAACfZA9IiSpcBABY=
pw3pXXXXXXXXX<u></u>XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX<u></u>XXXXXXXXXX&quot;, =
&quot;client_id&quot; : &quot;175268255910551&quot;, &quot;expires&quot; : =
ISODate(&quot;2012-05-27T15:40:04.<u></u>978Z&quot;), &quot;scope&quot; : [=
 =A0 =A0 =A0 &quot;scope1&quot;, =A0 =A0 =A0&quot;scope2&quot;, =A0 &quot;s=
cope3&quot; ], &quot;status&quot; : &quot;valid&quot;, &quot;next_update&qu=
ot; : ISODate(&quot;2012-04-27T15:40:53.<u></u>676Z&quot;), &quot;inner_id&=
quot; : &quot;13XXXXX773&quot; }</div>



</div><div><br></div><div>=A0the query is over inner_id (not the real name =
because of privacy issues)</div><div>indexes are</div><div><div>[</div><div=
>=A0 =A0 =A0 =A0 {</div><div>=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 &quot;v&quot; =
: 1,</div><div>=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 &quot;key&quot; : {</div>



<div>=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 &quot;_id&quot; : 1</d=
iv><div>=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 },</div><div>=A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 &quot;ns&quot; : &quot;mybase.data_collection&quot;,</div><div>=
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 &quot;name&quot; : &quot;_id_&quot;</div>



<div>=A0 =A0 =A0 =A0 },</div><div>=A0 =A0 =A0 =A0 {</div><div>=A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 &quot;v&quot; : 1,</div><div>=A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 &quot;key&quot; : {</div><div>=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 &quot;inner_id&quot; : 1</div><div>=A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 },</div>


<div>
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 &quot;unique&quot; : true,</div><div>=A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0 &quot;ns&quot; : &quot;mybase.data_collection&q=
uot;,</div><div>=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 &quot;name&quot; : &quot;in=
ner_id_1&quot;</div><div>=A0 =A0 =A0 =A0 },</div><div>



=A0 =A0 =A0 =A0 {</div><div>=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 &quot;v&quot; :=
 1,</div><div>=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 &quot;key&quot; : {</div><div=
>=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 &quot;next_update&quot; : =
1</div><div>=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 },</div><div>=A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 &quot;ns&quot; : &quot;mybase.data_collection&quot;,</div>



<div>=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 &quot;name&quot; : &quot;next_update_1=
&quot;</div><div>=A0 =A0 =A0 =A0 }</div></div><div><br></div><div><br></div=
><div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8e=
x;border-left:1px #ccc solid;padding-left:1ex">



<div><br></div><div>Increasing the replica set members can make write perfo=
rmance worse if you use safe writes. However, adding a shard can help distr=
ibute the write load.</div><div><div><br>On Monday, April 23, 2012 5:24:10 =
PM UTC-4, Dario Guzik wrote:<blockquote class=3D"gmail_quote" style=3D"marg=
in:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex">



<div>What bothers me most is that many read operations that should be super=
 fast are taking 100+ ms and still no clue of what might be happening<br><b=
r><div class=3D"gmail_quote">2012/4/23 Dario Alejandro Guzik <span dir=3D"l=
tr">&lt;<a href=3D"mailto:da...@guzik.com.ar" target=3D"_blank">dario@guzik=
.com.ar</a>&gt;</span><br>





<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div>Here you can find stats:</div><div><div=
>PRIMARY&gt; db.stats()</div>

<div>{</div><div>=A0 =A0 =A0 =A0 &quot;db&quot; : &quot;social&quot;,</div>
<div>=A0 =A0 =A0 =A0 &quot;collections&quot; : 23,</div><div>=A0 =A0 =A0 =
=A0 &quot;objects&quot; : 56243345,</div><div>=A0 =A0 =A0 =A0 &quot;avgObjS=
ize&quot; : 614.3731572864309,</div>

<div>
=A0 =A0 =A0 =A0 &quot;dataSize&quot; : 34554401444,</div><div>=A0 =A0 =A0 =
=A0 &quot;storageSize&quot; : 37738672016,</div><div>=A0 =A0 =A0 =A0 &quot;=
numExtents&quot; : 313,</div><div>=A0 =A0 =A0 =A0 &quot;indexes&quot; : 34,=
</div>


<div>=A0 =A0 =A0 =A0 &quot;indexSize&quot; : 4673139968,</div><div>=A0 =A0 =
=A0 =A0 &quot;fileSize&quot; : 47155511296,</div><div>=A0 =A0 =A0 =A0 &quot=
;nsSizeMB&quot; : 16,</div><div>


=A0 =A0 =A0 =A0 &quot;ok&quot; : 1</div><div><div><div>}</div><div><br></di=
v><br><div class=3D"gmail_quote">2012/4/23 Dario Guzik <span dir=3D"ltr">&l=
t;<a href=3D"mailto:da...@guzik.com.ar" target=3D"_blank">da...@guzik.com.a=
r</a>&gt;</span><br>






<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">Hi!<br>
<br>
Some context first: We are working on a project using mongoDB as our<br>
main data storage. There are three shards which are recplica sets of 3<br>
instances. Each one of them with enough RAM an CPU (quad cores and<br>
16GB RAM)<br>
Most collections are sharded but some processing queues that are not<br>
sharded.<br>
We are experiencing some problems when using this queues. When We<br>
start the process that reads these queues and does a lot of writting<br>
sometimes the mongos asking for a key that is allocated in the default<br>
shard (the same that holds the queues) gets a null response, but then<br>
you can manually check that key and it is there in all three instances<br>
of the rs. I assume it has something to do with the heavy writting<br>
load that the rs is coping with.<br>
I need help to find out which path should We take to solve this<br>
errors. Expanding the rs to 5 members is a good idea? moving some<br>
queues elsewhere?<br>
<br>
I have db.stats() from the shard and some mongostats log if it helps.<br>
<span><font color=3D"#888888"><br>
--<br>
You received this message because you are subscribed to the Google Groups &=
quot;mongodb-user&quot; group.<br>
To post to this group, send email to <a href=3D"mailto:mongodb-user@googleg=
roups.com" target=3D"_blank">mongodb-user@googlegroups.com</a>.<br>
To unsubscribe from this group, send email to <a href=3D"mailto:mongodb-use=
r%2Bunsubscribe@googlegroups.com" target=3D"_blank">mongodb-user+unsubscrib=
e@<u></u>googl<u></u>egroups.com</a>.<br>
For more options, visit this group at <a href=3D"http://groups.google.com/g=
roup/mongodb-user?hl=3Den" target=3D"_blank">http://groups.google.com/<u></=
u>group<u></u>/mongodb-user?hl=3Den</a>.<br>
<br>
</font></span></blockquote></div><br></div></div></div>
</blockquote></div><br></div>
</blockquote>

<p></p>

-- <br>
You received this message because you are subscribed to the Google Groups &=
quot;mongodb-user&quot; group.<br></div></div>
To view this discussion on the web visit <a href=3D"https://groups.google.c=
om/d/msg/mongodb-user/-/d5nC5I89DSAJ" target=3D"_blank">https://groups.goog=
le.com/d/<u></u>msg/mongodb-user/-/<u></u>d5nC5I89DSAJ</a>.<div><div>

<br>=20
To post to this group, send email to <a href=3D"mailto:mongodb-user@googleg=
roups.com" target=3D"_blank">mongodb-user@googlegroups.com</a>.<br>
To unsubscribe from this group, send email to <a href=3D"mailto:mongodb-use=
r%2Bunsubscribe@googlegroups.com" target=3D"_blank">mongodb-user+unsubscrib=
e@<u></u>googlegroups.com</a>.<br>

For more options, visit this group at <a href=3D"http://groups.google.com/g=
roup/mongodb-user?hl=3Den" target=3D"_blank">http://groups.google.com/<u></=
u>group/mongodb-user?hl=3Den</a>.<br>


</div></div></blockquote></div><br></div>
</blockquote>

<p></p>

-- <br>
You received this message because you are subscribed to the Google Groups &=
quot;mongodb-user&quot; group.<br></div></div>
To view this discussion on the web visit <a href=3D"https://groups.google.c=
om/d/msg/mongodb-user/-/asi0puyd_3UJ" target=3D"_blank">https://groups.goog=
le.com/d/msg/mongodb-user/-/asi0puyd_3UJ</a>.<div class=3D"HOEnZb"><div cla=
ss=3D"h5">

<br>=20
To post to this group, send email to <a href=3D"mailto:mongodb-user@googleg=
roups.com" target=3D"_blank">mongodb-user@googlegroups.com</a>.<br>
To unsubscribe from this group, send email to <a href=3D"mailto:mongodb-use=
r%2Bunsubscribe@googlegroups.com" target=3D"_blank">mongodb-user+unsubscrib=
e@googlegroups.com</a>.<br>

For more options, visit this group at <a href=3D"http://groups.google.com/g=
roup/mongodb-user?hl=3Den" target=3D"_blank">http://groups.google.com/group=
/mongodb-user?hl=3Den</a>.<br>


</div></div></blockquote></div><br></div>

--f46d04083ad7927a4904be6e5996--
Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
Account Options
