Date: Tue, 11 Sep 2012 00:45:42 -0700 (PDT)
From: marcandre <malegr...@gmail.com>
To: mongodb-user@googlegroups.com
Message-Id: <c3762ec6-775f-460d-9962-208b03c4f565@googlegroups.com>
In-Reply-To: <CALKyTE7hs2QVDLuV1KRDTc5K-OCbMFUwofgLgFa=aE5-4_FCGQ@mail.gmail.com>
References: <2fb72d20-018d-4970-82a1-865bc7b4e6ca@googlegroups.com>
<e547d433-6347-481f-856b-7b742298cd7c@googlegroups.com>
<76aa74a0-9a97-4077-ba25-1541bf40b8be@googlegroups.com>
<5b16a4e2-fee2-4405-a1e8-38bcf8202cff@googlegroups.com>
<5bc3ef1e-2cf6-421f-83a4-fe5f18378b82@googlegroups.com>
<5c6efae1-d14e-4fa9-b44e-ab68263a1a81@googlegroups.com>
<1f5c8e83-fc39-4256-a5d3-8ec77b5fa20f@googlegroups.com>
<c8b00800-c1f8-4562-b985-506be8c71997@googlegroups.com>
<CALKyTE5MgUEX1VYxx+k7kpVporqT0YH7E+CrYgTDL58fT5Mizg@mail.gmail.com>
<50c469a3-8dcb-4c58-bf40-808ba22dea5c@googlegroups.com>
<CALKyTE7pjJJBuRHmLSLMx9rNf2nC4-0Em0YYYDDTWHJv99Gbzw@mail.gmail.com>
<CALKyTE6kWojV4AcN4uOkY18-nTd7soWZ++Ji5h2ixL_S5VbZ3A@mail.gmail.com>
<CALKyTE7hs2QVDLuV1KRDTc5K-OCbMFUwofgLgFa=aE5-4_FCGQ@mail.gmail.com>
Subject: Re: [mongodb-user] Re: Very slow query
MIME-Version: 1.0
Content-Type: multipart/mixed;
boundary="----=_Part_42_23605758.1347349542222"
------=_Part_42_23605758.1347349542222
Content-Type: multipart/alternative;
boundary="----=_Part_43_27939798.1347349542222"
------=_Part_43_27939798.1347349542222
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Hellow everybody ,
=20
=20
When I update my code I get with explain cursor :=20
=20
array(16) { ["cursor"]=3D> string(24) "BtreeCursor tags_1 multi"=20
["isMultiKey"]=3D> bool(true) ["n"]=3D> int(30) ["nscannedObjects"]=3D> int=
(330)=20
["nscanned"]=3D> int(330) ["nscannedObjectsAllPlans"]=3D> int(330)=20
["nscannedAllPlans"]=3D> int(330) ["scanAndOrder"]=3D> bool(false)=20
["indexOnly"]=3D> bool(false) ["nYields"]=3D> int(0) ["nChunkSkips"]=3D> in=
t(0)=20
["millis"]=3D> int(8) ["indexBounds"]=3D> array(1) { ["tags"]=3D> array(2) =
{=20
[0]=3D> array(2) { [0]=3D> string(7) "windows" [1]=3D> string(7) "windowt" =
}=20
[1]=3D> array(2) { [0]=3D> object(MongoRegex)#7 (2) { ["regex"]=3D> string(=
8)=20
"^windows" ["flags"]=3D> string(0) "" } [1]=3D> object(MongoRegex)#8 (2) {=
=20
["regex"]=3D> string(8) "^windows" ["flags"]=3D> string(0) "" } } } }=20
["allPlans"]=3D> array(1) { [0]=3D> array(5) { ["cursor"]=3D> string(24)=20
"BtreeCursor tags_1 multi" ["n"]=3D> int(30) ["nscannedObjects"]=3D> int(33=
0)=20
["nscanned"]=3D> int(330) ["indexBounds"]=3D> array(1) { ["tags"]=3D> array=
(2) {=20
[0]=3D> array(2) { [0]=3D> string(7) "windows" [1]=3D> string(7) "windowt" =
}=20
[1]=3D> array(2) { [0]=3D> object(MongoRegex)#9 (2) { ["regex"]=3D> string(=
8)=20
"^windows" ["flags"]=3D> string(0) "" } [1]=3D> object(MongoRegex)#10 (2) {=
=20
["regex"]=3D> string(8) "^windows" ["flags"]=3D> string(0) "" } } } } } }=
=20
["oldPlan"]=3D> array(2) { ["cursor"]=3D> string(24) "BtreeCursor tags_1 mu=
lti"=20
["indexBounds"]=3D> array(1) { ["tags"]=3D> array(2) { [0]=3D> array(2) { [=
0]=3D>=20
string(7) "windows" [1]=3D> string(7) "windowt" } [1]=3D> array(2) { [0]=3D=
>=20
object(MongoRegex)#11 (2) { ["regex"]=3D> string(8) "^windows" ["flags"]=3D=
>=20
string(0) "" } [1]=3D> object(MongoRegex)#12 (2) { ["regex"]=3D> string(8)=
=20
"^windows" ["flags"]=3D> string(0) "" } } } } } ["server"]=3D> string(12)=
=20
"host:27017" }=20
=20
For this moment my code look for one key word . I use research by keyword=
=20
prefix like for example /^windows/ . But when I type the word (windows for=
=20
example), the first time the web page is very slow (It take 1 or 2=20
minutes to display results), and the second time the web page responses=20
very quikly (less than 1 second) .
=20
Thank you very much
Le lundi 10 septembre 2012 22:03:35 UTC+2, Sammaye a =C3=A9crit :
> Though thinking twice here, you can prefix since win would match windows=
=20
> with /^win/ and r would match r2 with /^r/. That might work.
>
> On 10 September 2012 20:52, Sam Millman <sam.m...@gmail.com <javascript:>=
>wrote:
>
>> Ok my last message was written a bit quickly.
>>
>> Basically I understand what your trying to do, essentially an FTS.=20
>> Problem is the infixing on your fields is 2 characters min so "win 2008=
=20
>> Server R Data" won't match without regex. I strongely suggest you either=
=20
>> deal with out regex or you split up your words further, but then they co=
uld=20
>> lose their meaning and return superflious amount of documents.
>>
>>
>> On 10 September 2012 20:50, Sam Millman <sam.m...@gmail.com <javascript:=
>
>> > wrote:
>>
>>> If it is an array of words then you don't need the regex, try taking=20
>>> that out.
>>>
>>>
>>> On 10 September 2012 20:44, marcandre <male...@gmail.com <javascript:>>=
wrote:
>>>
>>>> In this code I am trying is to research only documents that have all=
=20
>>>> keywords asked by the client,=20
>>>> =20
>>>> For example a user who is looking for documents, when he types "window=
s=20
>>>> 2008 Server R2 Datacenter" or "win 2008 Server R Data" the research e=
ngin=20
>>>> have to
>>>> display documents that contains the following keywords : array(0 =3D>=
=20
>>>> windows, 2008, Server, R2, Datacenter) contained in the "tags" field =
(It=20
>>>> is an array)=20
>>>> =20
>>>>
>>>> Le lundi 10 septembre 2012 21:14:11 UTC+2, Sammaye a =C3=A9crit :
>>>>
>>>>> Non-prefixed regexes do not use indexes would be one of your problems=
:
>>>>>
>>>>> $who =3D array();
>>>>> if(count($ArrayWord) > 1) {=20
>>>>> $tmp =3D array();=20
>>>>> foreach ($ArrayWord as $q) {=20
>>>>> $tmp[] =3D new MongoRegex( "/". strtolower($q) ."/" );=20
>>>>> }
>>>>> $who['tags'] =3D array('$all' =3D> $tmp);=20
>>>>> }else{=20
>>>>> $who['tags'] =3D new MongoRegex( "/". strtolower($mot) ."/" );=20
>>>>> }=20
>>>>>
>>>>> What exactly are you trying to do here?
>>>>>
>>>>> On 10 September 2012 20:10, marcandre <male...@gmail.com> wrote:
>>>>>
>>>>>> Thank you very much for these informations,
>>>>>> =20
>>>>>> But, how can I get a BtreeCursor with ensureIndex() function ?
>>>>>> Whitch arguments can I put in this function, I know that I have to=
=20
>>>>>> give the field name in the first argument .
>>>>>> =20
>>>>>>
>>>>>> Le lundi 10 septembre 2012 18:01:33 UTC+2, Gianfranco a =C3=A9crit :
>>>>>> =20
>>>>>>> Yes, as Gelin said.
>>>>>>> That query is not using an index, that's why is taking longer, goin=
g=20
>>>>>>> through the entire collection.
>>>>>>>
>>>>>>> Make sure you ensureIndex() on the correct fields.
>>>>>>> Then try doing an explain again to check if it worked
>>>>>>>
>>>>>>> On Monday, September 10, 2012 4:52:28 PM UTC+1, gelin yan wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> =E5=9C=A8 2012=E5=B9=B49=E6=9C=8810=E6=97=A5=E6=98=9F=E6=9C=9F=E4=
=B8=80UTC+8=E4=B8=8B=E5=8D=8811=E6=97=B648=E5=88=8611=E7=A7=92=EF=BC=8C****=
marcandre=E5=86=99=E9=81=93=EF=BC=9A
>>>>>>>>>
>>>>>>>>> I get this reponse when I call var_dump($cursor->explain()); :=
=20
>>>>>>>>> =20
>>>>>>>>> =20
>>>>>>>>> array(15) { ["cursor"]=3D> string(11) "BasicCursor" ["isMultiKey"=
]=3D>=20
>>>>>>>>> bool(false) ["n"]=3D> int(30) ["nscannedObjects"]=3D> int(1148) [=
"nscanned"]=3D>=20
>>>>>>>>> int(1148) ["nscannedObjectsAllPlans"]=3D> int(1148) ["nscannedAll=
Plans"]=3D>=20
>>>>>>>>> int(1148) ["scanAndOrder"]=3D> bool(false) ["indexOnly"]=3D> bool=
(false)=20
>>>>>>>>> ["nYields"]=3D> int(0) ["nChunkSkips"]=3D> int(0) ["millis"]=3D> =
int(2)=20
>>>>>>>>> ["indexBounds"]=3D> array(0) { } ["allPlans"]=3D> array(1) { [0]=
=3D> array(5) {=20
>>>>>>>>> ["cursor"]=3D> string(11) "BasicCursor" ["n"]=3D> int(30) ["nscan=
nedObjects"]=3D>=20
>>>>>>>>> int(1148) ["nscanned"]=3D> int(1148) ["indexBounds"]=3D> array(0)=
{ } } }=20
>>>>>>>>> ["server"]=3D> string(12) "host:27017" }=20
>>>>>>>>> =20
>>>>>>>>> I am limiting the find to avoid display every documents in the=20
>>>>>>>>> same page .
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Hi=20
>>>>>>>> "BasicCursor" implies your index didn't work for your query.=
=20
>>>>>>>>
>>>>>>>>> Le lundi 10 septembre 2012 17:31:40 UTC+2, Gianfranco a =C3=A9cri=
t :
>>>>>>>>>
>>>>>>>>>> If you run explain() on the cursor you can see why is taking tha=
t=20
>>>>>>>>>> long.
>>>>>>>>>>
>>>>>>>>>> var_dump($cursor->explain());
>>>>>>>>>>
>>>>>>>>>> Do that on one and see if it's using an index (BtreeCursor)
>>>>>>>>>>
>>>>>>>>>> Another problem could be that you are limiting the find() result=
s=20
>>>>>>>>>> to 30 and then doing a skip.
>>>>>>>>>> This is not every efficient, and I don't understand from your=20
>>>>>>>>>> code exactly why are you doing that.
>>>>>>>>>>
>>>>>>>>>> You can also try to increase the timeout of the cursor and try i=
f=20
>>>>>>>>>> that helps
>>>>>>>>>>
>>>>>>>>>> On Monday, September 10, 2012 4:07:30 PM UTC+1, marcandre wrote:
>>>>>>>>>>>
>>>>>>>>>>> I don't think that Postgres causes the bach being slow, because=
=20
>>>>>>>>>>> when I use MongoDB as a reseach engin to research documents b=
y keywords ,=20
>>>>>>>>>>> =20
>>>>>>>>>>> my reseach engin is very slow or ends with an error of timeout =
:=20
>>>>>>>>>>> *Fatal error: Uncaught exception 'MongoCursorTimeoutException'=
=20
>>>>>>>>>>> with message 'cursor timed out (timeout: 30000, time left: 0:0,=
status: 0)'
>>>>>>>>>>> * .
>>>>>>>>>>> =20
>>>>>>>>>>> This my code for research douments :
>>>>>>>>>>> =20
>>>>>>>>>>> $cnx_mongo =3D new Mongo("127.0.0.1", array("persist" =3D> "x")=
);
>>>>>>>>>>> $db =3D $cnx_mongo->my_database;
>>>>>>>>>>> $ma_collection =3D $db->my_collection;
>>>>>>>>>>> =20
>>>>>>>>>>> $who =3D array();
>>>>>>>>>>> if(count($ArrayWord) > 1) {=20
>>>>>>>>>>> $tmp =3D array();=20
>>>>>>>>>>> foreach ($ArrayWord as $q) {=20
>>>>>>>>>>> $tmp[] =3D new MongoRegex( "/". strtolower($q) ."/" );=20
>>>>>>>>>>> }
>>>>>>>>>>> $who['tags'] =3D array('$all' =3D> $tmp);=20
>>>>>>>>>>> }else{=20
>>>>>>>>>>> $who['tags'] =3D new MongoRegex( "/". strtolower($mot) ."/" )=
;=20
>>>>>>>>>>> }=20
>>>>>>>>>>> =20
>>>>>>>>>>> =20
>>>>>>>>>>> =20
>>>>>>>>>>> $cursor =3D $ma_collection->find($who)->**li**mit(30)->skip(($o=
ff_set=20
>>>>>>>>>>> - 1)*30);
>>>>>>>>>>> =20
>>>>>>>>>>> $cursor->timeout(1000);
>>>>>>>>>>> =20
>>>>>>>>>>> print($cursor->count());
>>>>>>>>>>> =20
>>>>>>>>>>> foreach($cursor as $obj){
>>>>>>>>>>> =20
>>>>>>>>>>> $id_prod =3D $obj["id_prod"];
>>>>>>>>>>> $ref_constr =3D $obj["ref_constr"];
>>>>>>>>>>> $img_prod =3D $obj["img_prod_petit"];
>>>>>>>>>>> $zoom =3D $obj["img_prod_zoom"];
>>>>>>>>>>> $titre =3D utf8_decode($obj["titre_prod"]****);
>>>>>>>>>>> =20
>>>>>>>>>>> $prix_max =3D $obj["prix_max"];
>>>>>>>>>>> $prix_min =3D $obj["prix_min"];
>>>>>>>>>>> $nb_offres =3D $obj["nb_offres"];
>>>>>>>>>>> $max_dispo =3D $obj["max_dispo"];
>>>>>>>>>>> $min_dispo =3D $obj["min_dispo"];
>>>>>>>>>>> $id_assoc =3D $obj["id_assoc"];
>>>>>>>>>>> }
>>>>>>>>>>> =20
>>>>>>>>>>>
>>>>>>>>>>> Le lundi 10 septembre 2012 16:42:35 UTC+2, Gianfranco a =C3=A9c=
rit :
>>>>>>>>>>>
>>>>>>>>>>>> As you did correctly, if this is not a live production=20
>>>>>>>>>>>> database, you can ensureIndex() after you finish the batch.
>>>>>>>>>>>> Because if there's an index, the database needs to update the=
=20
>>>>>>>>>>>> index every time there is new document inserted.
>>>>>>>>>>>>
>>>>>>>>>>>> Is it possible, and i'm just wondering here, that PostgreSQL i=
s=20
>>>>>>>>>>>> one of the causes of the batch being slow?
>>>>>>>>>>>> If not do you have any more information on why is not?
>>>>>>>>>>>>
>>>>>>>>>>>> On Monday, September 10, 2012 1:50:17 PM UTC+1, marcandre wrot=
e:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Good afternoon,
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> I'am using MongoDB with 350 000 documents and php. The proble=
m=20
>>>>>>>>>>>>> is the php reads documents very slowly .
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> When I'm updating documents I drop the database and I insert=
=20
>>>>>>>>>>>>> the documents : during this processus the inserts are very lo=
ng .
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> I don't understand, I putted indexes .
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> I show you a piece of my php code when I updating documents :=
=20
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> $cnx_mongo =3D new Mongo("127.0.0.1", array("persist" =3D> "x=
"));
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> $db =3D $cnx_mongo->my_database;
>>>>>>>>>>>>> $ma_collection =3D $db->my_collection;
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> $response =3D $ma_collection->drop();
>>>>>>>>>>>>> $response =3D $db->drop();
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> $db =3D $cnx_mongo->my_database;
>>>>>>>>>>>>> $ma_collection =3D $db->my_collection;
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> ///I Import data from a database Postgres=20
>>>>>>>>>>>>> =20
>>>>>>>>>>>>>
>>>>>>>>>>>>> ///I put index in a fiel named tags=20
>>>>>>>>>>>>> $ma_collection->ensureIndex("****tags");
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> This is the code when php reads documents :
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> ///Research of documents who containts key words
>>>>>>>>>>>>> $cnx_mongo =3D new Mongo("127.0.0.1", array("persist" =3D> "x=
"));
>>>>>>>>>>>>> $db =3D $cnx_mongo->my_database;
>>>>>>>>>>>>> $ma_collection =3D $db->my_collection;
>>>>>>>>>>>>>
>>>>>>>>>>>>> //Get back keywords researching by tags fields
>>>>>>>>>>>>> $who =3D array();
>>>>>>>>>>>>> if(count($ArrayWord) > 1) {=20
>>>>>>>>>>>>> $tmp =3D array();=20
>>>>>>>>>>>>> foreach ($ArrayWord as $q) {=20
>>>>>>>>>>>>> $tmp[] =3D new MongoRegex( "/". strtolower($q) ."/" );=20
>>>>>>>>>>>>> }
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> $who['tags'] =3D array('$all' =3D> $tmp);=20
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> }else{=20
>>>>>>>>>>>>> $who['tags'] =3D new MongoRegex( "/". strtolower($mot) ."/"=
);=20
>>>>>>>>>>>>> }=20
>>>>>>>>>>>>>
>>>>>>>>>>>>> $cursor =3D $ma_collection->find($who)->**li**
>>>>>>>>>>>>> mit(30)->skip(60);
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> $cursor->timeout(1000);
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> print($cursor->count()); // I display the number of document=
s=20
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> foreach($cursor as $obj){
>>>>>>>>>>>>> ///I display the results of document =20
>>>>>>>>>>>>> }
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> The tags field containts array keywords .
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> What should I do to resolve this problem ?
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> I checked the RAM (6 Go of physical RAM and 4 Go of available=
=20
>>>>>>>>>>>>> memory) .
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> How can I check the cause of this slow reads and inserts ?
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> I there is something to configure in mongodb ?
>>>>>>>>>>>>> =20
>>>>>>>>>>>>> Best regards .
>>>>>>>>>>>>>
>>>>>>>>>>>> --=20
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "mongodb-user" group.
>>>>>> To post to this group, send email to mongod...@googlegroups.com
>>>>>>
>>>>>> To unsubscribe from this group, send email to
>>>>>> mongodb-user...@**googlegroups.com
>>>>>>
>>>>>> See also the IRC channel -- freenode.net#mongodb
>>>>>>
>>>>>
>>>>> --=20
>>>> You received this message because you are subscribed to the Google
>>>> Groups "mongodb-user" group.
>>>> To post to this group, send email to mongod...@googlegroups.com<javasc=
ript:>
>>>> To unsubscribe from this group, send email to
>>>> mongodb-user...@googlegroups.com <javascript:>
>>>> See also the IRC channel -- freenode.net#mongodb
>>>>
>>>
>>>
>>
>
------=_Part_43_27939798.1347349542222
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable
<div>Hellow everybody ,</div><div> </div><div> </div><div>When I =
update my code I get with explain cursor : </div><div> </div><div=
>array(16) { ["cursor"]=3D> string(24) "BtreeCursor tags_1 multi"=20
["isMultiKey"]=3D> bool(true) ["n"]=3D> int(30) ["nscannedObjects"]=
=3D>=20
int(330) ["nscanned"]=3D> int(330) ["nscannedObjectsAllPlans"]=3D> in=
t(330)=20
["nscannedAllPlans"]=3D> int(330) ["scanAndOrder"]=3D> bool(false)=20
["indexOnly"]=3D> bool(false) ["nYields"]=3D> int(0) ["nChunkSkips"]=
=3D>=20
int(0) ["millis"]=3D> int(8) ["indexBounds"]=3D> array(1) { ["tags"]=
=3D>=20
array(2) { [0]=3D> array(2) { [0]=3D> string(7) "windows" [1]=3D> =
string(7)=20
"windowt" } [1]=3D> array(2) { [0]=3D> object(MongoRegex)#7 (2) {=20
["regex"]=3D> string(8) "^windows" ["flags"]=3D> string(0) "" } [1]=
=3D>=20
object(MongoRegex)#8 (2) { ["regex"]=3D> string(8) "^windows" ["flags"]=
=3D>=20
string(0) "" } } } } ["allPlans"]=3D> array(1) { [0]=3D> array(5) {=
=20
["cursor"]=3D> string(24) "BtreeCursor tags_1 multi" ["n"]=3D> int(30=
)=20
["nscannedObjects"]=3D> int(330) ["nscanned"]=3D> int(330)=20
["indexBounds"]=3D> array(1) { ["tags"]=3D> array(2) { [0]=3D> arr=
ay(2) {=20
[0]=3D> string(7) "windows" [1]=3D> string(7) "windowt" } [1]=3D> =
array(2) {=20
[0]=3D> object(MongoRegex)#9 (2) { ["regex"]=3D> string(8) "^windows"=
=20
["flags"]=3D> string(0) "" } [1]=3D> object(MongoRegex)#10 (2) {=20
["regex"]=3D> string(8) "^windows" ["flags"]=3D> string(0) "" } } } }=
} }=20
["oldPlan"]=3D> array(2) { ["cursor"]=3D> string(24) "BtreeCursor tag=
s_1=20
multi" ["indexBounds"]=3D> array(1) { ["tags"]=3D> array(2) { [0]=3D&=
gt;=20
array(2) { [0]=3D> string(7) "windows" [1]=3D> string(7) "windowt" } =
[1]=3D>=20
array(2) { [0]=3D> object(MongoRegex)#11 (2) { ["regex"]=3D> string(8=
)=20
"^windows" ["flags"]=3D> string(0) "" } [1]=3D> object(MongoRegex)#12=
(2) {=20
["regex"]=3D> string(8) "^windows" ["flags"]=3D> string(0) "" } } } }=
}=20
["server"]=3D> string(12) "host:27017" } </div><div> </div><di=
v>For this moment my code look for one key word . I use research by keyword=
prefix like for example /^windows/ . But when I type the word (windo=
ws for example), the first time the web page is very slow (It take 1 o=
r 2 minutes to display results), and the second time the web page resp=
onses very quikly (less than 1 second) .</div><div> </div><div>Thank y=
ou very much</div><div><br>Le lundi 10 septembre 2012 22:03:35 UTC+2, Samma=
ye a =C3=A9crit :</div><blockquote style=3D"margin: 0px 0px 0px 0.8ex;=
padding-left: 1ex; border-left-color: rgb(204, 204, 204); border-left-widt=
h: 1px; border-left-style: solid;" class=3D"gmail_quote">Though thinking tw=
ice here, you can prefix since win would match windows with /^win/ and r wo=
uld match r2 with /^r/. That might work.<br><br><div class=3D"gmail_quote">=
On 10 September 2012 20:52, Sam Millman <span dir=3D"ltr"><<a href=3D"ja=
vascript:" target=3D"_blank" gdf-obfuscated-mailto=3D"_g0mzdk2gacJ">sam.m..=
....@gmail.com</a>></span> wrote:<br>
<blockquote style=3D"margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-l=
eft-color: rgb(204, 204, 204); border-left-width: 1px; border-left-style: s=
olid;" class=3D"gmail_quote">Ok my last message was written a bit quickly.<=
br><br>Basically I understand what your trying to do, essentially an FTS. P=
roblem is the infixing on your fields is 2 characters min so "win 2008 Serv=
er R Data" won't match without regex. I strongely suggest you either deal w=
ith out regex or you split up your words further, but then they could lose =
their meaning and return superflious amount of documents.<div>
<div><br>
<br><div class=3D"gmail_quote">On 10 September 2012 20:50, Sam Millman <spa=
n dir=3D"ltr"><<a href=3D"javascript:" target=3D"_blank" gdf-obfuscated-=
mailto=3D"_g0mzdk2gacJ">sam.m...@gmail.com</a>></span> wrote:<br><blockq=
uote style=3D"margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-left-col=
or: rgb(204, 204, 204); border-left-width: 1px; border-left-style: solid;" =
class=3D"gmail_quote">
If it is an array of words then you don't need the regex, try taking that o=
ut.<div><div><br><br><div class=3D"gmail_quote">On 10 September 2012 20:44,=
marcandre <span dir=3D"ltr"><<a href=3D"javascript:" target=3D"_blank" =
gdf-obfuscated-mailto=3D"_g0mzdk2gacJ">male...@gmail.com</a>></span> wro=
te:<br>
<blockquote style=3D"margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-l=
eft-color: rgb(204, 204, 204); border-left-width: 1px; border-left-style: s=
olid;" class=3D"gmail_quote"><div>In this code I am trying is to =
research only documents that have all keywords asked by the client, </=
div><div> </div>
<div>For example a user who is looking for documents, when he typ=
es "windows 2008 Server R2 Datacenter" or "win 2008 Server R Data" &nb=
sp;the research engin have to</div><div>display documents that co=
ntains the following keywords : array(0 =3D> windows, 2008, Se=
rver, R2, Datacenter) contained in the "tags" field (It is =
an array) </div>
<div> </div><div><br>Le lundi 10 septembre 2012 21:14:11 UTC+2, Sammay=
e a =C3=A9crit :</div><blockquote style=3D"margin: 0px 0px 0px 0.8ex; =
padding-left: 1ex; border-left-color: rgb(204, 204, 204); border-left-width=
: 1px; border-left-style: solid;" class=3D"gmail_quote">
<div>Non-prefixed regexes do not use indexes would be one of your problems:=
<br><br>$who =3D array();<br> if(count($ArrayWord) > 1) { <br> =
; $tmp =3D array(); <br> foreach ($ArrayWord as $q) { <br>&=
nbsp; $tmp[] =3D new MongoRegex( "/". strtolower($q) ."/" ); <br=
>
}<br> $who['tags'] =3D array('$all' =3D> $tmp);&n=
bsp;<br> }else{ <br> $who['tags'] =3D new MongoRegex( =
"/". strtolower($mot) ."/" ); <br> } <br><br>What exactly are you tryi=
ng to do here?<br>
<br></div><div class=3D"gmail_quote"><div><div>On 10 September 2012 20:10, =
marcandre <span dir=3D"ltr"><<a>male...@gmail.com</a>></span> wrote:<=
br></div></div><blockquote style=3D"margin: 0px 0px 0px 0.8ex; padding-left=
: 1ex; border-left-color: rgb(204, 204, 204); border-left-width: 1px; borde=
r-left-style: solid;" class=3D"gmail_quote">
<div><div>
<div>Thank you very much for these informations,</div><div> </div><div=
>But, how can I get a <span style=3D'font-family: Consolas,"Andale Mon=
o WT","Andale Mono","Lucida Console",Monaco,"Courier New",Courier,monospace=
; font-size: 14px; background-color: rgb(225, 225, 225);'>BtreeCursor =
</span> with ensureIndex() function ?</div>
<div>Whitch arguments can I put in this function, I know that I have to giv=
e the field name in the first argument .</div><div> </div><div><b=
r>Le lundi 10 septembre 2012 18:01:33 UTC+2, Gianfranco a =C3=A9crit :=
</div></div></div>
<div>
<div><div><div><blockquote style=3D"margin: 0px 0px 0px 0.8ex; padding-left=
: 1ex; border-left-color: rgb(204, 204, 204); border-left-width: 1px; borde=
r-left-style: solid;" class=3D"gmail_quote">Yes, as Gelin said.<div>That qu=
ery is not using an index, that's why is taking longer, going through the e=
ntire collection.</div>
<div><br></div><div>Make sure you ensureIndex() on the correct fields.</div=
><div>Then try doing an explain again to check if it worked</div><div><br>O=
n Monday, September 10, 2012 4:52:28 PM UTC+1, gelin yan wrote:<blockquote =
style=3D"margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-left-color: r=
gb(204, 204, 204); border-left-width: 1px; border-left-style: solid;" class=
=3D"gmail_quote">
<br><br>=E5=9C=A8 2012=E5=B9=B49=E6=9C=8810=E6=97=A5=E6=98=9F=E6=9C=9F=E4=
=B8=80UTC+8=E4=B8=8B=E5=8D=8811=E6=97=B648=E5=88=8611=E7=A7=92=EF=BC=8C<u><=
/u><u></u><wbr>marcandre=E5=86=99=E9=81=93=EF=BC=9A<blockquote style=3D"mar=
gin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-left-color: rgb(204, 204,=
204); border-left-width: 1px; border-left-style: solid;" class=3D"gmail_qu=
ote">
<div>I get this reponse when I call var_dump($cursor->explain()); =
: </div><div> </div><div> </div><div>array(15) { ["cursor"]=3D>=
; string(11) "BasicCursor" ["isMultiKey"]=3D>=20
bool(false) ["n"]=3D> int(30) ["nscannedObjects"]=3D> int(1148)=20
["nscanned"]=3D> int(1148) ["nscannedObjectsAllPlans"]=3D> int(1148)=
=20
["nscannedAllPlans"]=3D> int(1148) ["scanAndOrder"]=3D> bool(false)=
=20
["indexOnly"]=3D> bool(false) ["nYields"]=3D> int(0) ["nChunkSkips"]=
=3D>=20
int(0) ["millis"]=3D> int(2) ["indexBounds"]=3D> array(0) { }=20
["allPlans"]=3D> array(1) { [0]=3D> array(5) { ["cursor"]=3D> stri=
ng(11)=20
"BasicCursor" ["n"]=3D> int(30) ["nscannedObjects"]=3D> int(1148)=20
["nscanned"]=3D> int(1148) ["indexBounds"]=3D> array(0) { } } }=20
["server"]=3D> string(12) "host:27017" } </div><div> </div><div>I&n=
bsp;am limiting the find to avoid display every documents in the same page =
.</div><div><br></div></blockquote><div><br>Hi <br>  =
; "BasicCursor" implies your index didn't work for your query. <br>
</div><blockquote style=3D"margin: 0px 0px 0px 0.8ex; padding-left: 1ex; bo=
rder-left-color: rgb(204, 204, 204); border-left-width: 1px; border-left-st=
yle: solid;" class=3D"gmail_quote"><div>Le lundi 10 septembre 2012 17:31:40=
UTC+2, Gianfranco a =C3=A9crit :</div>
<blockquote style=3D"margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-l=
eft-color: rgb(204, 204, 204); border-left-width: 1px; border-left-style: s=
olid;" class=3D"gmail_quote">If you run explain() on the cursor you can see=
why is taking that long.<div>
<br></div><div><span style=3D'color: rgb(0, 0, 187); font-family: Consolas,=
"Andale Mono WT","Andale Mono","Lucida Console",Monaco,"Courier New",Courie=
r,monospace; font-size: 14px; background-color: rgb(225, 225, 225);'>var_du=
mp</span><span style=3D'color: rgb(0, 119, 0); font-family: Consolas,"Andal=
e Mono WT","Andale Mono","Lucida Console",Monaco,"Courier New",Courier,mono=
space; font-size: 14px; background-color: rgb(225, 225, 225);'>(</span><spa=
n style=3D'color: rgb(0, 0, 187); font-family: Consolas,"Andale Mono WT","A=
ndale Mono","Lucida Console",Monaco,"Courier New",Courier,monospace; font-s=
ize: 14px; background-color: rgb(225, 225, 225);'>$cursor</span><span style=
=3D'color: rgb(0, 119, 0); font-family: Consolas,"Andale Mono WT","Andale M=
ono","Lucida Console",Monaco,"Courier New",Courier,monospace; font-size: 14=
px; background-color: rgb(225, 225, 225);'>-></span><span style=3D'color=
: rgb(0, 0, 187); font-family: Consolas,"Andale Mono WT","Andale Mono","Luc=
ida Console",Monaco,"Courier New",Courier,monospace; font-size: 14px; backg=
round-color: rgb(225, 225, 225);'>explain</span><span style=3D'color: rgb(0=
, 119, 0); font-family: Consolas,"Andale Mono WT","Andale Mono","Lucida Con=
sole",Monaco,"Courier New",Courier,monospace; font-size: 14px; background-c=
olor: rgb(225, 225, 225);'>());</span><br>
<br>Do that on one and see if it's using an index (<span style=3D'font-fami=
ly: Consolas,"Andale Mono WT","Andale Mono","Lucida Console",Monaco,"Courie=
r New",Courier,monospace; font-size: 14px; background-color: rgb(225, 225, =
225);'>BtreeCursor</span>)</div>
<div><br></div><div>Another problem could be that you are limiting the find=
() results to 30 and then doing a skip.</div><div>This is not every efficie=
nt, and I don't understand from your code exactly why are you doing that.</=
div>
<div><br></div><div>You can also try to increase the timeout of the cursor =
and try if that helps</div><br>On Monday, September 10, 2012 4:07:30 PM UTC=
+1, marcandre wrote:<blockquote style=3D"margin: 0px 0px 0px 0.8ex; padding=
-left: 1ex; border-left-color: rgb(204, 204, 204); border-left-width: 1px; =
border-left-style: solid;" class=3D"gmail_quote">
<div>I don't think that Postgres causes the bach being slow, because w=
hen I use MongoDB as a reseach engin to research documents by k=
eywords , </div><div> </div><div>my reseach engin is very slow or=
ends with an error of timeout : <b>Fatal error: Uncaught exception 'MongoC=
ursorTimeoutException' with=20
message 'cursor timed out (timeout: 30000, time left: 0:0, status: 0)'</b>&=
nbsp; .</div><div> </div><div>This my code for research douments =
:</div><div> </div><div>$cnx_mongo =3D new Mongo("127.0.0.1", array("p=
ersist" =3D> "x"));<br>
$db =3D $cnx_mongo->my_database;<br>$ma_collection =3D $db->my_collec=
tion;</div><div> </div><div>$who =3D array();<br> if(count($Array=
Word) > 1) { <br> $tmp =3D array(); <br> foreach (=
$ArrayWord as $q) { <br> $tmp[] =3D new MongoRegex( "/". s=
trtolower($q) ."/" ); <br>
}<br> $who['tags'] =3D array('$all' =3D> $tmp);&n=
bsp;<br> }else{ <br> $who['tags'] =3D new MongoRegex( =
"/". strtolower($mot) ."/" ); <br> } <br> <br> <br>&nbs=
p;<br>$cursor =3D $ma_collection->find($who)-><u></u>li<u></u><wbr>mi=
t(30)->skip(($off_set - 1)*30);<br>
<br>$cursor->timeout(1000);</div><div> </div><div>print($curs=
or->count());</div><div> </div><div>foreach($cursor as $obj){</div>=
<div> </div><div>$id_prod =3D $obj["id_prod"];</div><div>$ref_constr =
=3D $obj["ref_constr"];<br>
$img_prod =3D $obj["img_prod_petit"];<br>$zoom =3D $obj["img_prod_zoom"];</=
div><div>$titre =3D utf8_decode($obj["titre_prod"]<u></u><u></u><wbr>);<br>=
<br>$prix_max =3D $obj["prix_max"];<br>$prix_min =3D $obj["prix_min"]=
;<br>
$nb_offres =3D $obj["nb_offres"];<br>$max_dispo =3D $obj["max_dispo"];<br>$=
min_dispo =3D $obj["min_dispo"];</div><div>$id_assoc =3D $obj["id_assoc"];<=
/div><div>}</div><div> </div><div><br>
Le lundi 10 septembre 2012 16:42:35 UTC+2, Gianfranco a =C3=A9crit :</=
div>
<blockquote style=3D"margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-l=
eft-color: rgb(204, 204, 204); border-left-width: 1px; border-left-style: s=
olid;" class=3D"gmail_quote">As you did correctly, if this is not a live pr=
oduction database, you can ensureIndex() after you finish the batch.<d=
iv>
Because if there's an index, the database needs to update the index every t=
ime there is new document inserted.</div><div><br></div><div>Is it possible=
, and i'm just wondering here, that PostgreSQL is one of the causes of=
the batch being slow?</div>
<div>If not do you have any more information on why is not?</div><div><br>O=
n Monday, September 10, 2012 1:50:17 PM UTC+1, marcandre wrote:<blockquote =
style=3D"margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-left-color: r=
gb(204, 204, 204); border-left-width: 1px; border-left-style: solid;" class=
=3D"gmail_quote">
<div>Good afternoon,</div><div> </div><div>I'am using MongoDB with 350=
000 documents and php. The problem is the php reads documents very sl=
owly .</div><div> </div><div>When I'm updating documents&nbs=
p;I drop the database and I insert the documents : during this processus th=
e inserts are very long .</div>
<div> </div><div>I don't understand, I putted indexes .</div><div>&nbs=
p;</div><div>I show you a piece of my php code when I updating do=
cuments : </div><div> </div><div>$cnx_mongo =3D new Mongo("127.0.0.1",=
array("persist" =3D> "x"));<br>
<br> $db =3D $cnx_mongo->my_database;<br> $ma_collection=
=3D $db->my_collection;<br> <br> $response =3D $ma_collection=
->drop();<br> $response =3D $db->drop();<br> <br> $db =
=3D $cnx_mongo->my_database;<br> $ma_collection =3D $db->my_coll=
ection;<br>
<br> </div><div> ///I Import data from a database Postgres =
<br> </div><div><br> ///I put index in a fiel named tags </div><d=
iv> $ma_collection->ensureIndex("<u></u><u></u><wbr>tags");</div><d=
iv> </div><div>This is the code when php reads documents :</div>
<div> </div><div>///Research of documents who containts key words</div=
><div>$cnx_mongo =3D new Mongo("127.0.0.1", array("persist" =3D> "x"));<=
br>$db =3D $cnx_mongo->my_database;<br>$ma_collection =3D $db->my_col=
lection;</div>
<div><br> //Get back keywords researching by tags fields</div><div>&nb=
sp;$who =3D array();<br> if(count($ArrayWord) > 1) { <br> &nbs=
p;$tmp =3D array(); <br> foreach ($ArrayWord as $q) { <br> =
$tmp[] =3D new MongoRegex( "/". strtolower($q) ."/" ); <br>
}<br> <br> $who['tags'] =3D array('$all' =
=3D> $tmp); <br> <br> }else{ <br> $who['tags'] =3D=
new MongoRegex( "/". strtolower($mot) ."/" ); <br> } </div><div><br>&=
nbsp;$cursor =3D $ma_collection->find($who)-><u></u>li<u></u><wbr>mit=
(30)->skip(60);<br>
<br> $cursor->timeout(1000);</div><div> </div><div> =
;print($cursor->count()); // I display the number of documents </div><di=
v> </div><div> foreach($cursor as $obj){</div><div> //=
/I display the results of document <br>
}</div><div> </div><div> </div><div>The tags field containt=
s array keywords .</div><div> </div><div>What should I do to resolve t=
his problem ?</div><div> </div><div>I checked the RAM (6 Go of physica=
l RAM and 4 Go of available memory) .</div>
<div> </div><div>How can I check the cause of this slow reads and inse=
rts ?</div><div> </div><div>I there is something to configure in mongo=
db ?</div><div> </div><div>Best regards .</div></blockquote></div></bl=
ockquote></blockquote>
</blockquote></blockquote></blockquote></div></blockquote>
<p></p>
-- <br>
You received this message because you are subscribed to the Google<br>
Groups "mongodb-user" group.<br></div></div>
To post to this group, send email to <a>mongod...@googlegroups.com</a><div>=
<br>
To unsubscribe from this group, send email to<br>
</div><a>mongodb-user...@<u></u>googlegroups.<wbr>com</a><div><br>
See also the IRC channel -- <a href=3D"http://freenode.net#mongodb" target=
=3D"_blank">freenode.net#mongodb</a><br>
</div></div></div></blockquote></div><br>
</blockquote><div><div>
<p></p>
-- <br>
You received this message because you are subscribed to the Google<br>
Groups "mongodb-user" group.<br>
To post to this group, send email to <a href=3D"javascript:" target=3D"_bla=
nk" gdf-obfuscated-mailto=3D"_g0mzdk2gacJ">mongod...@googlegroups.com</a><b=
r>
To unsubscribe from this group, send email to<br>
<a href=3D"javascript:" target=3D"_blank" gdf-obfuscated-mailto=3D"_g0mzdk2=
gacJ">mongodb-user...@<wbr>googlegroups.com</a><br>
See also the IRC channel -- <a href=3D"http://freenode.net#mongodb" target=
=3D"_blank">freenode.net#mongodb</a><br>
</div></div></blockquote></div><br>
</div></div></blockquote></div><br>
</div></div></blockquote></div><br>
</blockquote>
------=_Part_43_27939798.1347349542222--
------=_Part_42_23605758.1347349542222--
Message from discussion Very slow query
| Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy |
| ©2013 Google |