> I observed that some (very few) queries are extremely slow, even when
> they use only indexed fields.
> Read queries are almost always executed in slaveOk mode.

> During the weekend I profiled all nodes by storing queries which took
> more than 3 seconds.
> Nearly 7000 queries were filtered during this time. Given the total
> number of queries, this seems not too much to me but I'm concerned
> about the slowest queries which took several minutes up to an hour!

> Here is the slowest query:
> {
> "_id" : ObjectId("4f795741559a0435df3b2cc0"),
> "ts" : "2012-04-02T08:11:37Z",
> "op" : "query",
> "ns" : "offerStore.offer",
> "query" : {
>         "shopId" : 222377,
>         "onlineProductIds" : {
>                 "$ne" : null
>         },
>         "smallPicture" : {
>                 "$ne" : null
>         },
>         "bokey" : {
>                 "$ne" : null
>         },
>         "_id" : {
>                 "$gt" : 0
>         },
>         "lastShopDataChange" : {
>                 "$gt" : "2012-03-02T00:00:00Z"
>         }},

> "ntoreturn" : 2000,
> "nscanned" : 107068,
> "nreturned" : 152,
> "responseLength" : 30897,
> "millis" : 3571537,
> "client" : "#.#.#.#",
> "user" : ""

> }

> How can a query that slow (nearly 1 hour!)? A query that returns only
> 152 documents by scanning a total of only 107068 documents.

> Another type of query, which executed very slowly, was this one:
> {
> "_id" : ObjectId("4f795741559a0435df3b2cb6"),
> "ts" : "2012-04-02T04:56:45Z",
> "op" : "getmore",
> "ns" : "offerStore.offer",
> "query" : {
>         "shopId" : 83769},

> "cursorid" : NumberLong("625703795491758659"),
> "nreturned" : 12803,
> "responseLength" : 4194466,
> "millis" : 864696,
> "client" : "#.#.#.#",
> "user" : ""

> }

> The field "shopId" is an indexed field.

> Here are the stats of my database:

> db.stats()
> {
> "raw" : {
>         "offerStoreDE2/s197:27018,s209:27018,s198:27018" : {
>                 "db" : "offerStore",
>                 "collections" : 5,
>                 "objects" : 29862952,
>                 "avgObjSize" : 2137.5809535507406,
>                 "dataSize" : 63834477412,
>                 "storageSize" : 114947460352,
>                 "numExtents" : 88,
>                 "indexes" : 9,
>                 "indexSize" : 11303606160,
>                 "fileSize" : 130866479104,
>                 "nsSizeMB" : 16,
>                 "ok" : 1
>         },
>         "offerStoreDE3/s117:27018,s129:27018,s118:27018" : {
>                 "db" : "offerStore",
>                 "collections" : 6,
>                 "objects" : 20098901,
>                 "avgObjSize" : 2126.199923866484,
>                 "dataSize" : 42734281776,
>                 "storageSize" : 69905829456,
>                 "numExtents" : 75,
>                 "indexes" : 10,
>                 "indexSize" : 6773039280,
>                 "fileSize" : 81498472448,
>                 "nsSizeMB" : 16,
>                 "ok" : 1
>         },
>         "offerStoreDE4/s115:27018,s132:27018,s125:27018" : {
>                 "db" : "offerStore",
>                 "collections" : 6,
>                 "objects" : 23699669,
>                 "avgObjSize" : 1902.8760452308427,
>                 "dataSize" : 45097532420,
>                 "storageSize" : 59188194976,
>                 "numExtents" : 71,
>                 "indexes" : 10,
>                 "indexSize" : 7836336256,
>                 "fileSize" : 70766297088,
>                 "nsSizeMB" : 16,
>                 "ok" : 1
>         },
>         "offerstore/s142:27018,s144:27018,s143:27018" : {
>                 "db" : "offerStore",
>                 "collections" : 8,
>                 "objects" : 18242133,
>                 "avgObjSize" : 1987.735232277936,
>                 "dataSize" : 36260530476,
>                 "storageSize" : 48505167600,
>                 "numExtents" : 75,
>                 "indexes" : 14,
>                 "indexSize" : 6151573344,
>                 "fileSize" : 57887686656,
>                 "nsSizeMB" : 16,
>                 "ok" : 1
>         }},

> "objects" : 91903655,
> "avgObjSize" : 2044.8242464785542,
> "dataSize" : 187926822084,
> "storageSize" : 292546652384,
> "numExtents" : 309,
> "indexes" : 43,
> "indexSize" : 32064555040,
> "fileSize" : 341018935296,
> "ok" : 1

> }

> How to proceed to find the bottleneck and make my queries quicker?
> Thank you!