wiredTiger poor performance when compared with MMAPv1

92 views
Skip to first unread message

edgardi...@gmail.com

unread,
Jun 28, 2016, 1:39:40 AM6/28/16
to mongodb-user
Hello,

I have encountered an example where wiredTiger performs extremely poorly when compared to MMAPv1
6min vs 181ms (see below).

One can address this by adding an index for that specific query, however that is unpractical and I believe that there should be a better way to improve wiredTiger's performance.
Do you have any suggestions how one should address this?


Here is a comparison, in both cases these were done on a fresh DB where we restored the collection from the same bson. 

mmapv1::
config:
engine: 'mmapv1'
result:
from pymongo import MongoClient
conn = MongoClient(host='ms1:34567')
db = conn.hmfs
time db.forms.find({'dimension':int(2),'degree':int(2)}).count()
CPU times: user 4 ms, sys: 0 ns, total: 4 ms
Wall time: 185 ms
0
log entry:
2016-06-28T05:10:12.412+0000 I COMMAND  [conn15] command hmfs.forms command: count { count: "forms", query: { dimension: 2, degree: 2 } } planSummary: IXSCAN { dimension: -1, field_label: 1, level_norm: -1 } keyUpdates:0 writeConflicts:0 numYields:447 reslen:62 locks:{ Global: { acquireCount: { r: 896 } }, MMAPV1Journal: { acquireCount: { r: 448 } }, Database: { acquireCount: { r: 448 } }, Collection: { acquireCount: { R: 448 } } } protocol:op_query 181ms 

> db.stats(1024*1024)

{

        "db" : "hmfs",

        "collections" : 3,

        "objects" : 368392,

        "avgObjSize" : 248291.20771352254,

        "dataSize" : 87231.15405273438,

        "storageSize" : 88116.09313964844,

        "numExtents" : 67,

        "indexes" : 17,

        "indexSize" : 205.6210479736328,

        "fileSize" : 90005,

        "nsSizeMB" : 16,

        "extentFreeList" : {

                "num" : 0,

                "totalSize" : 0

        },

        "dataFileVersion" : {

                "major" : 4,

                "minor" : 22

        },

        "ok" : 1

} 


wiredTiger:
config:
  engine: 'wiredTiger'
  wiredTiger:
    collectionConfig: 
      blockCompressor: none
result:
from pymongo import MongoClient
conn = MongoClient(host='ms1:23456')
db = conn.hmfs
time db.forms.find({'dimension':int(2),'degree':int(2)}).count()
CPU times: user 200 ms, sys: 48 ms, total: 248 ms
Wall time: 6min 9s
0
log entry:
2016-06-28T05:09:22.454+0000 I COMMAND  [conn4] command hmfs.forms command: count { count: "forms", query: { dimension: 2, degree: 2 } } planSummary: IXSCAN { dimension: -1, field_label: 1, level_norm: -1 } fromMultiPlanner:1 keyUpdates:0 writeConflicts:0 numYields:13831 reslen:62 locks:{ Global: { acquireCount: { r: 27664 } }, Database: { acquireCount: { r: 13832 } }, Collection: { acquireCount: { r: 13832 } } } protocol:op_query 369464ms
>

> db.stats(1024*1024)

{

        "db" : "hmfs",

        "collections" : 1,

        "objects" : 368356,

        "avgObjSize" : 205668.8590086764,

        "dataSize" : 72249.75417041779,

        "storageSize" : 72618.22265625,

        "numExtents" : 0,

        "indexes" : 17,

        "indexSize" : 43.0703125,

        "ok" : 1

} 
 
Best,
Edgar


Asya Kamsky

unread,
Jul 1, 2016, 1:54:23 AM7/1/16
to mongodb-user
You don't mention the version you ran this with but even aside from that, I don't understand what you mean by "One can address this by adding an index for that specific query, however that is unpractical and I believe that there should be a better way to improve wiredTiger's performance."

Indexes *is* how we improve database performance.

Asya


--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: https://docs.mongodb.org/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/a8d4b536-f5ad-4f07-98b3-3c245de205b6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Asya Kamsky
Lead Product Manager
MongoDB
Download MongoDB - mongodb.org/downloads
Free MongoDB Monitoring - cloud.mongodb.com
Free Online Education - university.mongodb.com
Get Involved - mongodb.org/community
We're Hiring! - https://www.mongodb.com/careers

Yann Simon

unread,
Jul 1, 2016, 2:27:03 AM7/1/16
to mongodb-user
Of course we should use an index to optimize queries.
But there are cases where it is not always possible to use an index (for ex: user-defined queries)
In that cases, I also have the feeling that MMAPv1 is performing better than wiredTiger (no numbers sorry).

In one project, it is a blocking issue to migrate to wiredTiger.

Edgar Costa

unread,
Jul 4, 2016, 1:20:39 AM7/4/16
to mongod...@googlegroups.com
I issued a bug on JIRA: https://jira.mongodb.org/browse/SERVER-24833

Asya Kamsky, if I search for an entry with a key that doesn't not exist, there is no index that will help with that search.
Further, Yann Simon guessed correctly, here we are dealing with user-defined queries.

Best,
Edgar



You received this message because you are subscribed to a topic in the Google Groups "mongodb-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mongodb-user/4IW3rvgeGh4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mongodb-user...@googlegroups.com.

To post to this group, send email to mongod...@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
Reply all
Reply to author
Forward
0 new messages