Find performance

20 views
Skip to first unread message

Matic

unread,
Nov 24, 2009, 9:49:41 AM11/24/09
to mongodb-user
Hey!

Since MongoDB is great functionality wise for my project, I've decided
to benchmark it with PHP. While insert performance is great at ~27k/
sec (more than Memcached and MemcacheDB), the read (find) performance
is not so great (at ~8k/sec) using '_id' (basic K/V functionality)
compared with 14k/sec and 12k/sec for Memcached and MemcacheDB. The
benchmark was conducted on Linux 2.6.27, 64-bit, PHP 5.3.0 and MongoDB
1.1.3. The benchmark results are consistent with the benchmark on
official MongoDB website.

I have the following questions now:
1) What is being done to speed up the find (read) functionality in the
near future (3 months)?
2) Are reads faster if I read from several different collections
concurrently?
3) How can Memcached be faster if MongoDB is using native OS cache?

Have a happy day!
Matic

Eliot Horowitz

unread,
Nov 24, 2009, 9:51:10 AM11/24/09
to mongod...@googlegroups.com
Can you do a top while running the test and see which process is maxed?
Right now we're often client bound on simple things like that.
> --
>
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>
>
>

Matic

unread,
Nov 24, 2009, 10:08:50 AM11/24/09
to mongodb-user
The CPU is dual core (Core2).
PHP process is 30%, Mongo is 70%, CPU idle time (average between
cores) is about 50%, which means neither of the processes are maxing
the CPU out. I'm using OS level virtualization (OpenVZ) so PHP and
MongoDB are on different VPS, but the same goes for Memcached and
MemcacheDB. CPU measurements were taken on the host node that sees the
processes of all VPS.

On Nov 24, 3:51 pm, Eliot Horowitz <eliothorow...@gmail.com> wrote:
> Can you do a top while running the test and see which process is maxed?
> Right now we're often client bound on simple things like that.
>

Eliot Horowitz

unread,
Nov 24, 2009, 10:20:44 AM11/24/09
to mongod...@googlegroups.com, mongodb-user
What form are you storing the data in memcache? Wondering if it's a
network bottleneck... Can you confirm disk is idle during the test?

On Nov 24, 2009, at 10:08 AM, Matic <matrix...@alpha-force.net>
wrote:

diegomsana

unread,
Nov 24, 2009, 10:28:07 AM11/24/09
to mongodb-user
depending on documents size, it may be a network bottleneck. Most
memcached clients gzip strings above 100 bytes.

If you're using mongodb as a K/V store, you might want to compress
values on the fly using php before storing them. CPU overhead is worth
the IO/network savings.

On 24 nov, 13:20, Eliot Horowitz <eliothorow...@gmail.com> wrote:
> What form are you storing the data in memcache?  Wondering if it's a  
> network bottleneck...  Can you confirm disk is idle during the test?
>
> On Nov 24, 2009, at 10:08 AM, Matic <matrix.goo...@alpha-force.net>  

Matic

unread,
Nov 24, 2009, 10:36:41 AM11/24/09
to mongodb-user
The disk is idle as there are no 'wa' times, so no process is waiting
for IO. I cannot see how network could be the bottleneck if both VPS
are on the same physical machine. For benchmarking I'm using a simple
array with 5 keys and values. I'm doing K/V benchmark because it's the
simplest to make. My primary use case for MongoDB is as a replacement
for MySQL. It would be great if I could replace MemcacheDB and
Memcached with MongoDB in the architecture as well.

What kind of read performance are you guys getting? The read
performance on official site is similar to mine.

Eliot Horowitz

unread,
Nov 24, 2009, 10:53:09 AM11/24/09
to mongod...@googlegroups.com
Is memcache using udp or tcp?
Doing a quick test locally its definitely network bound per socket.
Using 2 connections in the same client i double my speed.

Matic

unread,
Nov 24, 2009, 11:19:31 AM11/24/09
to mongodb-user
Both Memcached and MemcacheDB tests are using tcp. If I disable
compression for Memcached, I get even faster read (and write) results
(17k/sec uncompressed, 14.4k/sec compressed).

If I use find instead of findOne, it's way faster (200k/sec), but that
only returns the cursor. Once I add iteration and store the result in
an array, the performance drops back to ~7k/sec which is slightly
worse than using findOne directly.


On Nov 24, 4:53 pm, Eliot Horowitz <eliothorow...@gmail.com> wrote:
> Is memcache using udp or tcp?
> Doing a quick test locally its definitely network bound per socket.
> Using 2 connections in the same client i double my speed.
>

Matic

unread,
Nov 24, 2009, 11:33:08 AM11/24/09
to mongodb-user
If I use 2 PHP clients to read the data from MongoDB, I get the same
speed.

On Nov 24, 4:53 pm, Eliot Horowitz <eliothorow...@gmail.com> wrote:
> Is memcache using udp or tcp?
> Doing a quick test locally its definitely network bound per socket.
> Using 2 connections in the same client i double my speed.
>

Eliot Horowitz

unread,
Nov 24, 2009, 11:35:09 AM11/24/09
to mongod...@googlegroups.com
So the speed of each client is half?
i just did the same thing and each process was about 10% slower, so
80% faster overall

Matic

unread,
Nov 24, 2009, 11:51:43 AM11/24/09
to mongodb-user
Exactly, the individual speed for each client halved while the
combined speed stayed the same. What kind of speed do you get?

On Nov 24, 5:35 pm, Eliot Horowitz <eliothorow...@gmail.com> wrote:
> So the speed of each client is half?
> i just did the same thing and each process was about 10% slower, so
> 80% faster overall
>

Mathias Stearn

unread,
Nov 24, 2009, 11:57:51 AM11/24/09
to mongod...@googlegroups.com
What's your ping time? On my computer localhost gets 0.025ms so that gives a hard limit of 40k round-trips/sec. The reason you can insert more than that is because inserts are asynchronous so they do not wait for a round trip to send the next one. When I enable safe inserts (calls getlasterror after each one) the ops/second drops to about the level of finds.



On Tue, Nov 24, 2009 at 11:33 AM, Matic <matrix...@alpha-force.net> wrote:

Kristina Chodorow

unread,
Nov 24, 2009, 12:11:46 PM11/24/09
to mongod...@googlegroups.com
If memcached is getting 14k results/sec, how can it be a round trips/sec limit? 

Matic: Optimization is an ongoing project for the drivers.  If you can provide some sample documents, I can see if there's anything I can do to make your use case faster on the driver end.

(By the way, Matic, thanks for helping us figure this out.)

Matic

unread,
Nov 24, 2009, 12:44:59 PM11/24/09
to mongodb-user
Enabling safe mode on inserts reduces the insert speed from ~25k/sec
to ~4,5k/sec. Using 2 clients to insert concurrently, halves the speed
per client, while the combined speed stays the same.

By benchmark use case is a simple key/value example.

My simple benchmark script is available at http://pastebin.com/f34a840b4
.

I will benchmark tomorrow without the OS virtualization layer and with
a newer kernel.

My average RTT between the two VPS is 0.044 msec.

On Nov 24, 5:57 pm, Mathias Stearn <math...@10gen.com> wrote:
> What's your ping time? On my computer localhost gets 0.025ms so that gives a
> hard limit of 40k round-trips/sec. The reason you can insert more than that
> is because inserts are asynchronous so they do not wait for a round trip to
> send the next one. When I enable safe inserts (calls getlasterror after each
> one) the ops/second drops to about the level of finds.
>
> > mongodb-user...@googlegroups.com<mongodb-user%2Bunsu...@googlegroups.com>
> > > >> > >>> .
> > > >> > >>> For more options, visit this group athttp://groups.google.com/
> > > >> > >>> group/mongodb-user?hl=en.
>
> > > >> > > --
>
> > > >> > > You received this message because you are subscribed to the Google
> > > >> > > Groups "mongodb-user" group.
> > > >> > > To post to this group, send email to
> > mongod...@googlegroups.com.
> > > >> > > To unsubscribe from this group, send email to
> > mongodb-user...@googlegroups.com<mongodb-user%2Bunsu...@googlegroups.com>
> > > >> > > .
> > > >> > > For more options, visit this group athttp://
> > groups.google.com/group/mongodb-user?hl=en
> > > >> > > .
>
> > > > --
>
> > > > You received this message because you are subscribed to the Google
> > Groups "mongodb-user" group.
> > > > To post to this group, send email to mongod...@googlegroups.com.
> > > > To unsubscribe from this group, send email to
> > mongodb-user...@googlegroups.com<mongodb-user%2Bunsu...@googlegroups.com>
> > .
> > > > For more options, visit this group athttp://
> > groups.google.com/group/mongodb-user?hl=en.
>
> > --
>
> > You received this message because you are subscribed to the Google Groups
> > "mongodb-user" group.
> > To post to this group, send email to mongod...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > mongodb-user...@googlegroups.com<mongodb-user%2Bunsu...@googlegroups.com>

Matic

unread,
Nov 25, 2009, 5:00:32 AM11/25/09
to mongodb-user
I have now run benchmarks on Ubuntu 9.10, 64-bit, with 2.6.31 kernel
and without any virtualization layers. Results were as following:
RTT (ping): 0,008 ms (very fast)
Single threaded read: ~8300/sec
Dual threaded read: ~14100/sec
Quad threaded read: ~14500/sec
Single insert: ~39000/sec
Safe insert: ~4500/sec

The CPU was at the same frequency, but with 4 cores instead of 2. But
like with the dual core CPU, none of the benchmarks used the CPU
fully.

I still don't know what kind of results you guys get, but I do hope
you improve the read performance soon.

Have a happy day,
Matic

On Nov 24, 6:44 pm, Matic <matrix.goo...@alpha-force.net> wrote:
> Enabling safe mode on inserts reduces the insert speed from ~25k/sec
> to ~4,5k/sec. Using 2 clients to insert concurrently, halves the speed
> per client, while the combined speed stays the same.
>
> By benchmark use case is a simple key/value example.
>
> My simple benchmark script is available athttp://pastebin.com/f34a840b4

Geir Magnusson Jr.

unread,
Nov 25, 2009, 5:14:06 AM11/25/09
to mongod...@googlegroups.com
I'm confused - you expect better read performance? How fast do you expect it to be?

geir
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.

Tomasz Drobiszewski

unread,
Nov 25, 2009, 5:34:01 AM11/25/09
to mongod...@googlegroups.com

> still don't know what kind of results you guys get, but I do hope
>you improve the read performance soon.
- Mongodb is NOT a simple cache engine like memcache(DB), tokyo and is slover but ...
a) try to catch collection from memcache, tokyo
b) try  to make range query on memcache, tokyo
c) try agregate data in memcached
...

Sorry if my English is not perfect
-- 
Tomasz Drobiszewski

Matic

unread,
Nov 25, 2009, 8:00:21 AM11/25/09
to mongodb-user
I want the K/V read performance to be similar to MemcacheDB so I can
remove a layer of complexity from the architecture.

It would be great if MongoDB would have good enough caching to remove
the need for a separate caching engine like Memcached, TokyoTyrant,
MemcacheDB or Redis. That way people in high performance/traffic
environments won't have to run Memcached, MemcacheDB and some kind of
queriable persistent storage like MongoDB or MySQL.

MongoDB has great potential to replace all of the above mentioned
complexity with one server software. We all agree that it's easier to
maintain one software package than three or even four. Then there is
monitoring, security updates, different code bases, different
bugs, ... Everything becomes simpler with one excellent hybrid
software. MongoDB can be that excellent software.

Imagine MongoDB replacing MySQL for majority of new web applications
one year from now. Lower server resource usage, faster websites and
happier programmers. I'm not CEO of 10gen, but if I was, that would be
our goal for the next year.

Happy day,
Matic

Michael Lugassy

unread,
Nov 25, 2009, 8:08:37 AM11/25/09
to mongod...@googlegroups.com
+1 for Tomasz - you cannot compare the flexibility of MongoDB to the
basic KV store of memcache, especially with expiring cache and
pre-warming speed. How fast do you really need it to go? Will you
really max out Mongo's read performance?

Mathias Stearn

unread,
Nov 25, 2009, 8:53:45 AM11/25/09
to mongod...@googlegroups.com
Maybe I'm missing something, but aren't you getting the same numbers as memcached now?  There are some limits to how much you can get out of a single server. If you need more speed, I'd suggest you look into sharding.

To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.

Matic

unread,
Nov 25, 2009, 10:18:56 AM11/25/09
to mongodb-user
The benchmarks in the first post ran in a VPS (OS virtualization). The
MongoDB benchmarks a few posts above are from Ubuntu, without
virtualization. There, I forgot to add Memcached/MemcacheDB
benchmarks. The complete benchmarks are as following:

MongoDB:
Single threaded read: ~8300/sec
Dual threaded read: ~14100/sec
Quad threaded read: ~14500/sec
Single insert: ~39000/sec
Safe insert: ~4500/sec

Memcached:
Single threaded read: ~20000/sec
Quad threaded read: ~80000/sec
Single insert: ~20500/sec

MemcacheDB:
Single threaded read: ~18300/sec
Quad threaded read: ~39000/sec
Single insert: ~14000/sec

Server and client were on the same host and connection was done via
loopback interface with TCP.

I'm not saying MongoDB needs to be as fast as Memcached, but at least
35000 reads/sec would be nice.

On Nov 25, 2:53 pm, Mathias Stearn <math...@10gen.com> wrote:
> Maybe I'm missing something, but aren't you getting the same numbers as
> memcached now?  There are some limits to how much you can get out of a
> single server. If you need more speed, I'd suggest you look into sharding.
>
> > <mongodb-user%2Bunsu...@googlegroups.com<mongodb-user%252Buns...@googlegroups.com>
>
> > > > > > >> > >>> .
> > > > > > >> > >>> For more options, visit this group athttp://
> > groups.google.com/
> > > > > > >> > >>> group/mongodb-user?hl=en.
>
> > > > > > >> > > --
>
> > > > > > >> > > You received this message because you are subscribed to the
> > Google
> > > > > > >> > > Groups "mongodb-user" group.
> > > > > > >> > > To post to this group, send email to
> > > > > mongod...@googlegroups.com.
> > > > > > >> > > To unsubscribe from this group, send email to
> > > > > mongodb-user...@googlegroups.com<mongodb-user%2Bunsu...@googlegroups.com>
> > <mongodb-user%2Bunsu...@googlegroups.com<mongodb-user%252Buns...@googlegroups.com>
>
> > > > > > >> > > .
> > > > > > >> > > For more options, visit this group athttp://
> > > > > groups.google.com/group/mongodb-user?hl=en
> > > > > > >> > > .
>
> > > > > > > --
>
> > > > > > > You received this message because you are subscribed to the
> > Google
> > > > > Groups "mongodb-user" group.
> > > > > > > To post to this group, send email to
> > mongod...@googlegroups.com.
> > > > > > > To unsubscribe from this group, send email to
> > > > > mongodb-user...@googlegroups.com<mongodb-user%2Bunsu...@googlegroups.com>
> > <mongodb-user%2Bunsu...@googlegroups.com<mongodb-user%252Buns...@googlegroups.com>
>
> > > > > .
> > > > > > > For more options, visit this group athttp://
> > > > > groups.google.com/group/mongodb-user?hl=en.
>
> > > > > --
>
> > > > > You received this message because you are subscribed to the Google
> > Groups
> > > > > "mongodb-user" group.
> > > > > To post to this group, send email to mongod...@googlegroups.com.
> > > > > To unsubscribe from this group, send email to
> > > > > mongodb-user...@googlegroups.com<mongodb-user%2Bunsu...@googlegroups.com>
> > <mongodb-user%2Bunsu...@googlegroups.com<mongodb-user%252Buns...@googlegroups.com>

Eliot Horowitz

unread,
Nov 25, 2009, 2:59:05 PM11/25/09
to mongod...@googlegroups.com
Made a few improvements.
If you could try your tests with current master on github or the next
nightly ( will be built around 3 am)
Seeing a fair amount of speedup, especially multi-threaded tests.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.

Matic

unread,
Nov 26, 2009, 5:36:55 AM11/26/09
to mongodb-user
MongoDB just became even better!

Benchmark results from the latest build:
Single threaded read: ~9400/sec, 13% improvement
Dual threaded read: ~23000/sec, 63% improvement
Quad threaded read: ~25000/sec, 72% improvement

Excellent work Eliot! Thank you.

Happy day,
Matic

On Nov 25, 8:59 pm, Eliot Horowitz <eliothorow...@gmail.com> wrote:
> Made a few improvements.
> If you could try your tests with current master on github or the next
> nightly ( will be built around 3 am)
> Seeing a fair amount of speedup, especially multi-threaded tests.
>
> ...
>
> read more »

Michael Lugassy

unread,
Nov 26, 2009, 4:36:45 PM11/26/09
to mongod...@googlegroups.com
Matic -- I'll be happy to learn more on your experience with using
MongoDB as a replacement for memcached.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages