Mongo is optimised for write once read many that is true however that is a
bit of a crazy time for some deletes.
What exactly are you deleting? How many are you deleting? What is the size
of each document you are deleting? Maybe it is the find query on the remove
taking the time?
On 25 September 2012 08:01, Moore <lyx...@gmail.com> wrote:
> --
> You received this message because you are subscribed to the Google
> Groups "mongodb-user" group.
> To post to this group, send email to mongodb-user@googlegroups.com
> To unsubscribe from this group, send email to
> mongodb-user+unsubscribe@googlegroups.com
> See also the IRC channel -- freenode.net#mongodb
> On Tuesday, September 25, 2012 10:08:13 PM UTC+8, Stephen Lee wrote:
>> Per Sammaye's question, could you provide the .explain() for
>> online.online { fullad: "ad54060064" }? eg.
>> use online
>> db.online.find( { fullad: "ad54060064" } ).explain()
>> ?
>> On Tuesday, September 25, 2012 3:43:50 AM UTC-4, Moore wrote:
>>> there are more than 4 millions documents, from the begining , operation
>>> as lighting.
>>> one portal do modify documents, write/delete something like that;
>>> one python module as a tiny web-server do mongodb-query , 20 connections
>>> in concurre. hundreds of mongo-query per seconds.
>>> sometime do have a full tables scan for another app.
>>> for the mongodb server , box running centos-6.2 whith 32G ram and 300G
>>> lvm-hd. mongod process occupied 3G and a bits more.
>>> that's all the running-environment.
>> --
> You received this message because you are subscribed to the Google
> Groups "mongodb-user" group.
> To post to this group, send email to mongodb-user@googlegroups.com
> To unsubscribe from this group, send email to
> mongodb-user+unsubscribe@googlegroups.com
> See also the IRC channel -- freenode.net#mongodb
Could the slow operations have been during the "full table scan" you mentioned? That could cause a backup of operations and the latency you are seeing.
The mongostat output you provided looked like things were running OK at that time. Can you let that run and correlate a slow insert/delete log entry to the mongostat time?
If you are churning the documents in the database a lot and they are not already of a fairly uniform size you may want to look at using the "power of 2 allocator" with the collection. That will cut down the amount of fragmentation you will see with MongoDB under heavy write/update/delete workloads.
you might want to try the power of two space allocation option in 2.2.
It helps a lot in many cases with frequent deletes. I have not looked
at all the details of this situation so I can't confirm it will
resolve the issues.
> --
> You received this message because you are subscribed to the Google
> Groups "mongodb-user" group.
> To post to this group, send email to mongodb-user@googlegroups.com
> To unsubscribe from this group, send email to
> mongodb-user+unsubscribe@googlegroups.com
> See also the IRC channel -- freenode.net#mongodb
Somewhat tunning operations fixed this problem but I don't know which one was.
1. upgrade to 2.2, for that OS is CentOS6.2 2. power of two space allocation setting. 3. tuned OS parameter from tuned.noarch parameters included in file named throughput-performance/sysctl.ktune 4. ulimit set OS Max open files more than default 1024 and others such as Max msgqueue size etc.
It is crucial that we know why we are doing what we are doing,
In God we trust , i don't know why i doing and i don't know how i doing.
On Sunday, September 30, 2012 4:29:34 AM UTC+8, Max Schireson wrote:
> you might want to try the power of two space allocation option in 2.2. > It helps a lot in many cases with frequent deletes. I have not looked > at all the details of this situation so I can't confirm it will > resolve the issues.
When a document grows, it needs to be moved to a larger space. The old
space it used becomes free. The system keeps track of these spaces and
tries to reuse them. Depending on the exact pattern of document sizes,
sometimes spaces can pile up and take a while to get reused. The movement,
possible pileup, and associated bookkeeping his can slow the system down.
Power of two allocation allocates storage to each document in fixed sizes
of eg 512 bytes, 1024 bytes, etc. This greatly reduces the potential
pileup/fragmentation as well as the move frequency. It does make the
database somewhat less space efficient in the absence of frequent deletes,
but from the quick description of your use case seemed like a better fit.
It is a new option with 2.2 to address just this type of issue, hence my
recommendation to upgrade to take advantage of it.
Hope this helps,
-- Max
On Sep 29, 2012 4:21 PM, "Moore" <lyx...@gmail.com> wrote:
> Somewhat tunning operations fixed this problem but I don't know which one
> was.
> 1. upgrade to 2.2, for that OS is CentOS6.2
> 2. power of two space allocation setting.
> 3. tuned OS parameter from tuned.noarch parameters included in file named
> throughput-performance/sysctl.ktune
> 4. ulimit set OS Max open files more than default 1024 and others such as
> Max msgqueue size etc.
> It is crucial that we know why we are doing what we are doing,
> In God we trust , i don't know why i doing and i don't know how i doing.
> On Sunday, September 30, 2012 4:29:34 AM UTC+8, Max Schireson wrote:
>> you might want to try the power of two space allocation option in 2.2.
>> It helps a lot in many cases with frequent deletes. I have not looked
>> at all the details of this situation so I can't confirm it will
>> resolve the issues.
>> --
> You received this message because you are subscribed to the Google
> Groups "mongodb-user" group.
> To post to this group, send email to mongodb-user@googlegroups.com
> To unsubscribe from this group, send email to
> mongodb-user+unsubscribe@googlegroups.com
> See also the IRC channel -- freenode.net#mongodb
