need help understanding write locks

[arin]

Hey guys,

From time to time my app will exhibit, what seems to me, high lock times.

Here's an example from my mongodb.log (i'm profiling everything while I troubleshoot this)

Sat Sep  8 00:38:36 [conn69541] insert jeraff.notification keyUpdates:0 locks(micros) w:121485 121ms

Sat Sep  8 00:40:46 [conn69259] insert jeraff.activityStream keyUpdates:0 locks(micros) w:3100666 3105ms

My notification docs' size is:

jeraffprod:PRIMARY> Object.bsonsize(db.notification.findOne());

232

And my activityStream docs' size is:

jeraffprod:PRIMARY> Object.bsonsize(db.activityStream.findOne());

226

My entire DB's index size is about 1.5GB

The DB's data size: is a hair under 2.8GB

the DB's file size is 14GB

The machine has 8GB RAM.

Mongo has 22.2g mapped & 5.75g resident.

Now - my questions is:

What should I be looking into to find out why seemingly "simple" inserts are locking for so long? 

I'm sure my code's doing something to make mongo unhappy - I just want to know what procedures to employ to get down to the bottom of these problems.

Thanks

[Kenny Gorman]

Can you post the code that you are using for the inserts?

Thanks
Kenny

> --
> You received this message because you are subscribed to the Google
> Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com
> To unsubscribe from this group, send email to
> mongodb-user...@googlegroups.com
> See also the IRC channel -- freenode.net#mongodb

[Arin Sarkissian]

Hi Kenny,

We're using Java w/ Morphia.
The code for the inserts is super boring…

https://gist.github.com/3671210

using mongo-java-driver v 2.5
morphia version .99

- Arin

Arin Sarkissian
650-759-9272
http://well.io/phatduckk

> > To post to this group, send email to mongod...@googlegroups.com (mailto:mongod...@googlegroups.com)

> > To unsubscribe from this group, send email to

> > mongodb-user...@googlegroups.com (mailto:mongodb-user...@googlegroups.com)
> > See also the IRC channel -- freenode.net (http://freenode.net)#mongodb

>
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "mongodb-user" group.

> To post to this group, send email to mongod...@googlegroups.com (mailto:mongod...@googlegroups.com)

> To unsubscribe from this group, send email to

> mongodb-user...@googlegroups.com (mailto:mongodb-user...@googlegroups.com)
> See also the IRC channel -- freenode.net (http://freenode.net)#mongodb

[Jenna deBoisblanc]

Hi Arin,

There could be a handful of reasons why inserts are taking a while, especially since your data appears to exceed RAM.  It would be nice to see additional metrics- page faults, lock percentage, etc.  Are you servers in MMS?

https://mms.10gen.com/user/login

> > mongodb-user...@googlegroups.com (mailto:mongodb-user+unsub...@googlegroups.com)

> > See also the IRC channel -- freenode.net (http://freenode.net)#mongodb
>  
>  
>  
> --  
> You received this message because you are subscribed to the Google
> Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com (mailto:mongod...@googlegroups.com)
> To unsubscribe from this group, send email to

> mongodb-user...@googlegroups.com (mailto:mongodb-user+unsub...@googlegroups.com)

[David K Storrs]

Arin,

What version of Mongo are you using? The <2.0 versions had real
problems with global write locking. 2.0 greatly improved this, and
2.2 (the current stable) goes much farther. You may see significant
improvement simply upgrading. http://docs.mongodb.org/manual/release-notes/2.2/#upgrading

Also, how many threads / processes do you have doing the writes?
About how many writes / s do you expect to see, and how many do you
typically see / do you see during a slowdown?

Do you have any externalities that might be impacting you? (e.g. cron
jobs that run stats over the DB)

Dave

On Sep 7, 6:39 pm, Arin Sarkissian <a...@toodo.com> wrote:
> Hi Kenny,
>
> We're using Java w/ Morphia.
> The code for the inserts is super boring…
>
> https://gist.github.com/3671210
>
> using mongo-java-driver v 2.5
> morphia version .99
>
> - Arin
>
> Arin Sarkissian

> 650-759-9272http://well.io/phatduckk