Remove performance

[Michael]

I have a collection, which i clean every 10 minutes (it has around
70-100 inserts per second). When ever i do a mass remove, I'm getting
sluggish performance for read queries from other collections (response
time can be as high as several seconds).

Is it possible to tune MongoDB, so removes wouldn't affect read
performance?
I'm running 1.7.6 with -dur option.

I know i should use capped collections, but unfortunately i update
records after insert, and they can grow in size.

[sumit]

I posted a similar question yesterday, see -
http://groups.google.com/group/mongodb-user/browse_thread/thread/5d5dd12e37382b5b

[Scott Hernandez]

How are you doing the removes? Is it possible for you to do the
removes in smaller batches over more time?

Do you have any mongostat numbers from when the removes are happening?
Are the faults high during that period of removing?

> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>
>

[Michael]

I do removes like this:

val trailTm = new ObjectId( new Date(System.currentTimeMillis -
600*1000))
Work.delete(
("_id" -> ("$lt" -> trailTm.toStringMongod))
)

Yes, it's possible to do it more frequently, I did it once per minute,
and it's working good now, thanks.

[Michael]

It seems that i still get slow queries sometimes, during flushes.
Should i do flushes more frequent? At 17:50:38 i had 14 slow
requests, which took 200ms according to mongod log. Here is
output from mongostat.


63 69 13 0 0 1 0 0.7
0 0|0 0|0 23k 18k 26 17:50:36
50 56 13 0 0 1 0 0.6
0 0|0 0|0 19k 15k 26 17:50:37
insert query update delete getmore command flushes locked % idx miss
% qr|qw ar|aw netIn netOut conn time
59 70 23 0 0 1 1 39.6
0 0|0 0|1 24k 18k 26 17:50:38
53 59 14 0 0 1 0 0.7
0 0|1 0|1 20k 16k 26 17:50:39
76 91 34 0 0 1 0 1
0 0|0 0|0 32k 24k 26 17:50:40

[Eliot Horowitz]

Is your disk saturated at that point ( iostat -x 2 )?
If so, then you could increase fsync frequency and see if it smooths
performance out.

[Michael]

No, disk is not saturated.
I found, that it's probably because of FreeBSD (i have 7.1):
http://jira.mongodb.org/browse/SERVER-663

MongoDB still have that problem?

[Eliot Horowitz]

Yes, there are definitely still some issues with FreeBSD, so that's
likely what you're running into.
Are you tied to that or can you move to linux?