80k writes/s via php cli - NoSQL Databases

The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Message from discussion 80k writes/s via php cli
Received-SPF: pass (google.com: domain of the.mindstorm.mailinglist+caf_=nosql-databases=googlegroups....@gmail.com designates 74.125.83.45 as permitted sender) client-ip=74.125.83.45;
Received-SPF: pass (google.com: domain of mongodb-user+bncCKDs2tekDRDkgpD7BBoEs4lekg@googlegroups.com designates 209.85.220.187 as permitted sender)
Date: Fri, 16 Mar 2012 20:13:39 -0700 (PDT)
From: MKN Web Solutions <mich...@mknwebsolutions.com>
To: mongodb-user@googlegroups.com
Message-ID: <14990647.1878.1331954019489.JavaMail.geo-discussion-forums@ynjc20>
In-Reply-To: <26174410.571.1331949566977.JavaMail.geo-discussion-forums@vbux23>
References: <12802862.551.1331874935151.JavaMail.geo-discussion-forums@ynbq18>
 <AA13299C-3CB6-46BA-80B6-4EEE569F1...@gmail.com>
 <1159704.28.1331901684645.JavaMail.geo-discussion-forums@yncd14>
 <D3E7F90C-AAE9-43C5-80EF-6377AC389...@mac.com>
 <26174410.571.1331949566977.JavaMail.geo-discussion-forums@vbux23>
Subject: Re: [mongodb-user] 80k writes/s via php cli
MIME-Version: 1.0
Reply-To: mongodb-user@googlegroups.com
Precedence: list
Mailing-list: list mongodb-user@googlegroups.com; contact mongodb-user+owners@googlegroups.com
Sender: mongodb-user@googlegroups.com
Content-Type: multipart/alternative; 
	boundary="----=_Part_1877_16565934.1331954019487"

------=_Part_1877_16565934.1331954019487
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Using the batch script I was actually able to hit 90k w/s.  My lock ratio=
=20
is now 100%. =20



On Friday, March 16, 2012 9:59:26 PM UTC-4, MKN Web Solutions wrote:
>
> I'm sure the batch insert would work best on this.  However I wanted to=
=20
> stress the max # of connections / writes in a potential scenario. =20
>
> I'm running PHP 5.3 with this test.  I do have APC enabled, but remember=
=20
> this is a script that's looping internally and not outside.  So the scrip=
t=20
> is already being compiled.
>
> The script is as follows:
>
> <?php
> $db =3D new Mongo("mongodb://localhost");=20
> $c =3D $db->winner;
>
> while(true){
> $d =3D "test 1 2 3";
> $insert =3D $c->ppl->insert( array("x"=3D>$d) ); //single write
> }
>
> ?>
>
>
> On Friday, March 16, 2012 8:48:29 AM UTC-4, Tim Hawkins wrote:
>>
>> Also, have you checked out
>>
>> http://php.net/manual/en/mongocollection.batchinsert.php
>>
>> Sent from my iPad
>>
>> On 16 Mar 2012, at 20:41, MKN Web Solutions <mich...@mknwebsolutions.com=
>=20
>> wrote:
>>
>> That's what I figured.  Yea, I was running multiple php cli's with the=
=20
>> same script.=20
>>
>> On Friday, March 16, 2012 6:50:42 AM UTC-4, SCO wrote:
>>>
>>> I'm not a mongo guru, but given your lock level around 80%, you still=
=20
>>> have room, enough to reach 100kins/s.
>>>
>>> Did you try paralellizing php cli scripts ?
>>>
>>>
>>>
>>> Le 16 mars 2012 =E0 06:15, MKN Web Solutions <mich...@mknwebsolutions.c=
om>=20
>>> a =E9crit :
>>>
>>> I'm able to achieve 80k writes per second via CLI php.  The script is=
=20
>>> basic, it simply injects a 10 char string into "x" (in this case it isn=
't=20
>>> indexed, nor does it need to be since this is a write test only).
>>>
>>> The machine specs are amd phenom II x4 2.2ghz @ 64bit, 4GB ram, 250GB=
=20
>>> 7200 rpm hd, running ubuntu.
>>>
>>> CPU usage spiked around 50-70% usage by MongoDB and 20-30% by php cli.=
=20
>>>  Mongo running as a single instance, master set on, no repl set nor sla=
ves.
>>>
>>> My question is, what's the next bottleneck.  The HD utilization was als=
o=20
>>> bouncing around 10-35%.  I want to get the max writes possible that the=
=20
>>> machine can produce. =20
>>>
>>>
>>> Thanks for any tips / help.
>>>
>>>
>>> <https://lh3.googleusercontent.com/-JKp03qVaVS4/T2LLsmoGwoI/AAAAAAAAAAM=
/9cgwA3Dsxy4/s1600/Screen+Shot+2012-03-15+at+11.17.03+PM.png>
>>>
>>>  --=20
>>> You received this message because you are subscribed to the Google=20
>>> Groups "mongodb-user" group.
>>> To view this discussion on the web visit=20
>>> <https://groups.google.com/d/msg/mongodb-user/-/QTuYHv-zm3IJ>
>>> https://groups.google.com/d/msg/mongodb-user/-/QTuYHv-zm3IJ.
>>> To post to this group, send email to mongodb-user@googlegroups.com.
>>> To unsubscribe from this group, send email to=20
>>> mongodb-user+unsubscribe@googlegroups.com.
>>> For more options, visit this group at=20
>>> http://groups.google.com/group/mongodb-user?hl=3Den.
>>>
>>>  --=20
>> You received this message because you are subscribed to the Google Group=
s=20
>> "mongodb-user" group.
>> To view this discussion on the web visit=20
>> https://groups.google.com/d/msg/mongodb-user/-/CHOl72aoJtQJ.
>> To post to this group, send email to mongodb-user@googlegroups.com.
>> To unsubscribe from this group, send email to=20
>> mongodb-user+unsubscribe@googlegroups.com.
>> For more options, visit this group at=20
>> http://groups.google.com/group/mongodb-user?hl=3Den.
>>
>>

--=20
You received this message because you are subscribed to the Google Groups "=
mongodb-user" group.
To view this discussion on the web visit https://groups.google.com/d/msg/mo=
ngodb-user/-/zL5uLClG86MJ.
To post to this group, send email to mongodb-user@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user+unsubscribe@goog=
legroups.com.
For more options, visit this group at http://groups.google.com/group/mongod=
b-user?hl=3Den.


------=_Part_1877_16565934.1331954019487
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Using the batch script I was actually able to hit 90k w/s. &nbsp;My lock ra=
tio is now 100%. &nbsp;<div><br></div><div><br><br>On Friday, March 16, 201=
2 9:59:26 PM UTC-4, MKN Web Solutions wrote:<blockquote class=3D"gmail_quot=
e" style=3D"margin: 0;margin-left: 0.8ex;border-left: 1px #ccc solid;paddin=
g-left: 1ex;">I'm sure the batch insert would work best on this. &nbsp;Howe=
ver I wanted to stress the max # of connections / writes in a potential&nbs=
p;scenario. &nbsp;<div><br></div><div>I'm running PHP 5.3 with this test. &=
nbsp;I do have APC enabled, but remember this is a script that's looping in=
ternally and not outside. &nbsp;So the script is already being compiled.</d=
iv><div><br></div><div>The script is as follows:</div><div><br></div><div>&=
lt;?php</div><div><div>$db =3D new Mongo("mongodb://localhost");&nbsp;</div=
><div>$c =3D $db-&gt;winner;</div><div><br></div><div>while(true){</div><di=
v><span style=3D"white-space:pre">	</span>$d =3D "test 1 2 3";</div><div><s=
pan style=3D"white-space:pre">	</span>$insert =3D $c-&gt;ppl-&gt;insert( ar=
ray("x"=3D&gt;$d) ); //single write</div><div>}</div></div><div><br></div><=
div>?&gt;</div><div><br><br>On Friday, March 16, 2012 8:48:29 AM UTC-4, Tim=
 Hawkins wrote:<blockquote class=3D"gmail_quote" style=3D"margin:0;margin-l=
eft:0.8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor=3D"#FFF=
FFF"><div>Also, have you checked out</div><div><br></div><div><a href=3D"ht=
tp://php.net/manual/en/mongocollection.batchinsert.php" target=3D"_blank">h=
ttp://php.net/manual/en/<wbr>mongocollection.batchinsert.<wbr>php</a><br><b=
r>Sent from my iPad</div><div><br>On 16 Mar 2012, at 20:41, MKN Web Solutio=
ns &lt;<a href=3D"mailto:mich...@mknwebsolutions.com" target=3D"_blank">mic=
h...@mknwebsolutions.com</a>&gt; wrote:<br><br></div><div></div><blockquote=
 type=3D"cite"><div>That's what I figured. &nbsp;Yea, I was running multipl=
e php cli's with the same script.&nbsp;<br><br>On Friday, March 16, 2012 6:=
50:42 AM UTC-4, SCO wrote:<blockquote class=3D"gmail_quote" style=3D"margin=
:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgco=
lor=3D"#FFFFFF"><div>I'm not a mongo guru, but given your lock level around=
 80%, you still have room, enough to reach 100kins/s.</div><div><br></div><=
div>Did you try paralellizing php cli scripts ?</div><div><br><br></div><di=
v><br>Le 16 mars 2012 =E0 06:15, MKN Web Solutions &lt;<a href=3D"mailto:mi=
ch...@mknwebsolutions.com" target=3D"_blank">mich...@mknwebsolutions.com</a=
>&gt; a =E9crit&nbsp;:<br><br></div><div></div><blockquote type=3D"cite"><d=
iv>I'm able to achieve 80k writes per second via CLI php. &nbsp;The script =
is basic, it simply injects a 10 char string into "x" (in this case it isn'=
t indexed, nor does it need to be since this is a write test only).<div><br=
></div><div>The machine specs are amd phenom II x4 2.2ghz @ 64bit, 4GB ram,=
 250GB 7200 rpm hd, running ubuntu.</div><div><br></div><div>CPU usage spik=
ed around 50-70% usage by MongoDB and 20-30% by php cli. &nbsp;Mongo runnin=
g as a single instance, master set on, no repl set nor slaves.</div><div><b=
r></div><div>My question is, what's the next bottleneck. &nbsp;The HD utili=
zation was also bouncing around 10-35%. &nbsp;I want to get the max writes =
possible that the machine can produce. &nbsp;</div><div><br></div><div><br>=
</div><div>Thanks for any tips / help.</div><div><br></div><p style=3D"text=
-align:center;clear:both"><a href=3D"https://lh3.googleusercontent.com/-JKp=
03qVaVS4/T2LLsmoGwoI/AAAAAAAAAAM/9cgwA3Dsxy4/s1600/Screen+Shot+2012-03-15+a=
t+11.17.03+PM.png" style=3D"margin-left:1em;margin-right:1em" target=3D"_bl=
ank"><img border=3D"0"></a></p><div><br></div>

<p></p>

-- <br>
You received this message because you are subscribed to the Google Groups "=
mongodb-user" group.<br>
To view this discussion on the web visit <a href=3D"https://groups.google.c=
om/d/msg/mongodb-user/-/QTuYHv-zm3IJ" target=3D"_blank"></a><a href=3D"http=
s://groups.google.com/d/msg/mongodb-user/-/QTuYHv-zm3IJ" target=3D"_blank">=
https://groups.google.com/d/<wbr>msg/mongodb-user/-/QTuYHv-<wbr>zm3IJ</a>.<=
br>=20
To post to this group, send email to <a href=3D"mailto:mongodb-user@googleg=
roups.com" target=3D"_blank">mongodb-user@googlegroups.com</a>.<br>
To unsubscribe from this group, send email to <a href=3D"mailto:mongodb-use=
r+unsubscribe@googlegroups.com" target=3D"_blank">mongodb-user+unsubscribe@=
<wbr>googlegroups.com</a>.<br>

For more options, visit this group at <a href=3D"http://groups.google.com/g=
roup/mongodb-user?hl=3Den" target=3D"_blank">http://groups.google.com/<wbr>=
group/mongodb-user?hl=3Den</a>.<br>


</div></blockquote></div></blockquote>

<p></p>

-- <br>
You received this message because you are subscribed to the Google Groups "=
mongodb-user" group.<br>
To view this discussion on the web visit <a href=3D"https://groups.google.c=
om/d/msg/mongodb-user/-/CHOl72aoJtQJ" target=3D"_blank">https://groups.goog=
le.com/d/<wbr>msg/mongodb-user/-/<wbr>CHOl72aoJtQJ</a>.<br>=20
To post to this group, send email to <a href=3D"mailto:mongodb-user@googleg=
roups.com" target=3D"_blank">mongodb-user@googlegroups.com</a>.<br>
To unsubscribe from this group, send email to <a href=3D"mailto:mongodb-use=
r+unsubscribe@googlegroups.com" target=3D"_blank">mongodb-user+unsubscribe@=
<wbr>googlegroups.com</a>.<br>

For more options, visit this group at <a href=3D"http://groups.google.com/g=
roup/mongodb-user?hl=3Den" target=3D"_blank">http://groups.google.com/<wbr>=
group/mongodb-user?hl=3Den</a>.<br>


</div></blockquote></div></blockquote></div></blockquote></div>

<p></p>

-- <br />
You received this message because you are subscribed to the Google Groups "=
mongodb-user" group.<br />
To view this discussion on the web visit <a href=3D"https://groups.google.c=
om/d/msg/mongodb-user/-/zL5uLClG86MJ">https://groups.google.com/d/msg/mongo=
db-user/-/zL5uLClG86MJ</a>.<br />=20
To post to this group, send email to mongodb-user@googlegroups.com.<br />
To unsubscribe from this group, send email to mongodb-user+unsubscribe@goog=
legroups.com.<br />

For more options, visit this group at http://groups.google.com/group/mongod=
b-user?hl=3Den.<br />



------=_Part_1877_16565934.1331954019487--
Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
Account Options
