MongoDB slow writes causes socket time out exception

[Mark de Jong]

I've written a daemon which removes and inserts records async. Each
hour most of the collections are cleared and they'll get new inserted
data (10-12 million deletes and 10-12 million inserts). The daemon
uses ~60-80 of the CPU while inserting the data (due calculating 1+
million knapsack problems). When I fire up the daemon it can do it's
job about 1-2 mins till it crashes due a socket time out (writing data
to the MongoDB server).

When I look in the logs I see it takes about 30 seconds to remove data
in the collection. It seems it has something to do with the CPU load
and memory usage.., because when I run the daemon on a different PC
everything goes fine.

I already made some modifications to my daemon. I've decreased the
concurrent number of writes. However the daemon still crashes when the
memory is getting full (11.8GB of 12GB) and receives more load
(loading data into the frontend).

Ofcourse there should be try/catch statements to catch such
exceptions, but it should not happen in the first place. I'm looking
for a solution to solve this issue instead of working around it.

- Total storage size is: 8,1 GB
- Index size is: 2,1 GB

I guess the problem lies in that the working set + indexes are too
large to store in memory and MongoDB needs to access the HDD (which is
slow 5400 rpm).. However why would this be a problem? Aren't there
other strategies to store the collections (e.g. in seperate files
instead of large chunks of 2GB). If an Relational database can read/
write data in an acceptable amount of time from the disk, why can't
MongoDB?

00:02:46 [conn3] insert bargains.auction-history-eu-bloodhoof-horde
421ms
00:02:47 [conn6] insert bargains.auction-history-eu-blackhand-horde
1357ms
00:02:48 [conn3] insert bargains.auction-history-eu-bloodhoof-alliance
577ms
00:02:48 [conn6] insert bargains.auction-history-eu-blackhand-alliance
499ms
00:02:49 [conn4] remove bargains.crafts-eu-agamaggan-horde 34881ms
00:02:49 [conn5] remove bargains.crafts-eu-aggramar-horde 3135ms
00:02:49 [conn5] insert bargains.crafts-eu-aggramar-horde 234ms
00:02:50 [conn2] remove bargains.auctions-eu-aerie-peak-horde 36223ms
00:02:52 [conn5] remove bargains.auctions-eu-aegwynn-horde 1700ms


Posted the question Stackoverflow aswell:

http://stackoverflow.com/questions/8093679/mongodb-slow-writes-causes-socket-time-out-exception

[Eliot Horowitz]

Are you doing fire and forget removes or safe?
If you're doing fire and forget, the query after that will appear slow.
What are you setting the socket timeout to?

> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>
>

[Mark de Jong]

Additional Info:

Running on:

MongoDB 2.0.1
Windows 2008 R2
12 GB RAM
2 TB HDD (5400 rpm)
mongocsharpdriver version 1.3

Actually I am dropping the collection and then inserting stuff again:

public void ClearAndAdd(IEnumerable<T> items)
{
collection.Drop();
collection.InsertBatch(items);
}

The collections vary from size (crafts 3630 to auctions 20000-30000).
If you look at the log above, you'll see both take 30 to drop..

I haven't set any timeout as I was expecting this to be no issue :)

On Nov 15, 7:58 pm, Eliot Horowitz <el...@10gen.com> wrote:
> Are you doing fire and forget removes or safe?
> If you're doing fire and forget, the query after that will appear slow.
> What are you setting the socket timeout to?
>

> >http://stackoverflow.com/questions/8093679/mongodb-slow-writes-causes...

[Eliot Horowitz]

You're not doing a drop (which is very fast) you're doing a remove,
which is very different.

If you just want to remove all data from a collection, you should call
.drop() not .remove( {} )

[Mark de Jong]

I see, Strange that collection.Drop() does not issue the "drop"
command, must be something wrong in my code :) I'll check it tonight,
we'll keep in touch!

On Nov 16, 4:13 am, Eliot Horowitz <el...@10gen.com> wrote:
> You're not doing a drop (which is very fast) you're doing a remove,
> which is very different.
>
> If you just want to remove all data from a collection, you should call
> .drop() not .remove( {} )
>

[Robert Stam]

Using collection.Drop() in the C# driver will send a drop command to the server, which is different from removing all the documents. Not sure why your server log shows a remove instead of a drop. Did your code used to look different? Did you call remove from the mongo shell?

[Mark de Jong]

The daemon is quite stable now. I am using this now:

public void Clear()
{
if (collection.Exists())
{
var command = new CommandDocument {
{ "drop", collectionName },
};

collection.Database.RunCommand(command);
}
}

collection.Drop does not seem to issue the "drop" command.. I am using
1.3 mongocsharpdriver from nuget.

Reading the source from github:
https://github.com/mongodb/mongo-csharp-driver/blob/master/Driver/Core/MongoDatabase.cs#L392.
It should issue a drop. I was using 1.2 first, maybe some assembly
caching issue?

On Nov 16, 5:18 pm, Robert Stam <rob...@10gen.com> wrote:
> Using collection.Drop() in the C# driver will send a drop command to the
> server, which is different from removing all the documents. Not sure why
> your server log shows a remove instead of a drop. Did your code used to
> look different? Did you call remove from the mongo shell?
>

[Robert Stam]

The implementation of DropCollection has always been to send a drop command to the server, so it shouldn't make any difference what version of the driver you were using. Your code is basically equivalent to the driver's implementation of DropCollection.

[Mark de Jong]

Just cleaned my projects and the problem seems to be gone :)

Thanks for all the help !

On Nov 17, 2:55 pm, Robert Stam <rob...@10gen.com> wrote:
> The implementation of DropCollection has always been to send a drop command
> to the server, so it shouldn't make any difference what version of the
> driver you were using. Your code is basically equivalent to the driver's
> implementation of DropCollection.
>

> On Thu, Nov 17, 2011 at 5:04 AM, Mark de Jong <av3n...@gmail.com> wrote:
>
>
>
>
>
>
>
> > The daemon is quite stable now. I am using this now:
>
> >        public void Clear()
> >        {
> >            if (collection.Exists())
> >            {
> >                var command = new CommandDocument {
> >                    { "drop", collectionName },
> >                };
>
> >                collection.Database.RunCommand(command);
> >            }
> >        }
>
> > collection.Drop does not seem to issue the "drop" command.. I am using
> > 1.3 mongocsharpdriver from nuget.
>
> > Reading the source from github:
>

> >https://github.com/mongodb/mongo-csharp-driver/blob/master/Driver/Cor...

[Robert Stam]

You're welcome.