Mongodb c# driver 1.6.1, now getting a "An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full" error

[Sam Martin]

H there,

Previously I was using a source version of the driver from a month or so ago to get round an authentication issue. This fixed was officially released in 1.6.1 so I updated the driver in my app.

The only other change to the config is that now the database is held on an SSD.

The problem I have now is that about 2/3 of the way through a 3-4M record import process (fetching, transforming and saving to new collection) the so runs out of sockets.

Getting an error "An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full"

After this I cannot create any new connections to anything and I have to restart.

Do you have any idea of what might cause this? Do I need to change the db connection timeout? Or manually manage connections?

For each record, I'm obviously calling on a number of connections, but they'd normally be coming from a pool the driver manages, is that still correct?

Thanks in advance,

Sam

[Robert Stam]

I'm hoping Sam will follow up with some more information, but apparently if you write data fast enough in fire and forget mode (safe=false), which happens to be the default, you can write data faster than the network can transport it out and you can overwhelm the operating system buffers with too much data.

An easy solution is to put "safe=true" on the connection string. If acknowledging every single Insert slows you down too much, you can also consider using InsertBatch to insert documents in batches, and there will only be one acknowledgement per batch. Even small batch sizes (say 10-100 documents) make a big difference in speed.

--
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com
To unsubscribe from this group, send email to
mongodb-user...@googlegroups.com
See also the IRC channel -- freenode.net#mongodb