Mongdb C# Driver Object ID Duplicate Key Error

[Mark Lambie]

Has anyone Run into a duplicate ObjectID key error after inserting 13 million records?

Setup - I have a Intel Nuc i7, with 16G RAM, running Windows 10, MongoDB 3.2.4, wireTiger (not replcaiting).
On the Same Machine I'M running Visual Studio's and the program I created in debug mode.
Program - use PCAP.Net reads a pcap file - for each packet it creates a packetBson object, when a 1 million objects are added to my list, it calls insertMany for the collection.

Issue: Around the 13 Million records inserted I get _id - duplicate key in index Exception.

I've tried using just the ObjectId.GenerateNewId(), ObjectId.GenerateNewId(System.DateTime.Now), and IIdGenerator _generator = new AscendingGuidGenerator()
all three methods - I still get _id duplicate key in index around the 13 million mark.

Thanks in advance for any help.
Mark

[Robert Stam]

It is possible to end up with duplicate ObjectIds but only if you generate them fast enough. But you would have to generate more than 16,777,216 ObjectIds per second, which is pretty hard to do. On my laptop I can only generate about 12 million per second even when the only thing in the loop is the call to ObjectId.GenerateNewId().

In order to understand how you could end up with a duplicate ObjectId consider how an ObjectId is structured. An ObjectId consists of four components: timestamp, machine, PID and increment. The timestamp increments once per second, the machine and PID are fixed for your application, and the increment in incremented by one each time a new ObjectId is generated. The increment is 3 bytes long and rolls over to 0 when it overflows. 3 bytes is enough to hold 16,777,216 unique values, and therefore at most this many unique ObjectIds can be generated per second.

It's possible your duplicate ObjectIds are due to some other cause.

Can you reproduce this in a standalone program you would be willing to share?

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: https://docs.mongodb.org/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/ba670361-6121-44fc-afca-a78c68b3d0e3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.