Re: E11000 duplicate key error index

[Gianfranco]

This means the index has two entries for the same _id

If the collection is empty you can drop() the collection causing the default index ( _id ) to be removed as well.

This is not related to this issue but note that mongodb 2.0.4 is quite old. Not the last version of the 2.0.x branch and the latest one is 2.4.1

Cheers,

Gianfranco

On Wednesday, 10 April 2013 13:00:32 UTC+1, Diego Woitasen wrote:

Hi,

 I have really weird issue with Mongo or PyMongo not sure. When I try to insert a document in a collection I get:

pymongo.errors.DuplicateKeyError: E11000 duplicate key error index: shared_db.trackvars.$_id_  dup key: { : ObjectId('5165480d95cc645e2ead9309') }

Always. The collections is empty and the error appears on the first insert.

I have two machines, testing and production. Production with more than 6M of documents in the collections doesn't have any problem. This problems appears in testing. The code of both machines is the same, pymongo and mongo version the same. Both are virtual machines.

Pymongo 2.5

Mongo 2.0.4 (from Ubuntu's repos).

Any hint to discover what's going on?

Thanks!

Regards,

  Diego

[Diego Woitasen]

I upgraded to the latest Mongo version and have the same problem. I started with an empty BD (rm -fR /var/lib/mongodb) and the problem.

About the code, I'm inserting the documents in a loop. But same code works in the production machine.

Any issues related with clocking and Virtual Machine that affects the generation of IDs?

[Gianfranco]

How are you generating the _id(s)? Manually or are you letting MongoDB create them?

There are no issues as there is a incrementing counter included when generating object_ids

If you post your code here it might be more helpful to see how it behaves.

[Bernie Hackett]

I'm guessing your code looks something like this (greatly simplified)?:

>>> doc = {}
>>> for i in xrange(2):
... doc['i'] = i
... c.shard_db.trackvars.insert(doc)
...
ObjectId('5166d479fa5bd835199a0638')
Traceback (most recent call last):
File "<stdin>", line 3, in <module>
File "pymongo/collection.py", line 357, in insert
continue_on_error, self.__uuid_subtype), safe)
File "pymongo/mongo_client.py", line 916, in _send_message
rv = self.__check_response_to_last_error(response)
File "pymongo/mongo_client.py", line 857, in __check_response_to_last_error
raise DuplicateKeyError(details["err"])

pymongo.errors.DuplicateKeyError: E11000 duplicate key error index:

shard_db.trackvars.$_id_ dup key: { :
ObjectId('5166d479fa5bd835199a0638') }

The problem is that PyMongo injects an _id field into the document,
*if the _id field does not exist*, before inserting it (_id is always
generated client side with 10gen drivers). That means that the first
time through the loop _id is added by the insert method. Since "doc"
is defined outside the loop, each subsequent pass through the loop
uses the *same* value for _id.

If you want to modify some globally defined document in a loop this
way make sure you also chnage the _id value in the same loop. You can
create instances of bson.objectid.ObjectId in your code for this:

>>> from bson.objectid import ObjectId
>>> for i in xrange(2):
... doc['i'] = i
... doc['_id'] = ObjectId()
... c.shard_db.trackvars.insert(doc)
...
ObjectId('5166d4d5fa5bd835199a0639')
ObjectId('5166d4d5fa5bd835199a063a')

> --
> --
> You received this message because you are subscribed to the Google
> Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com
> To unsubscribe from this group, send email to
> mongodb-user...@googlegroups.com
> See also the IRC channel -- freenode.net#mongodb
>
> ---
> You received this message because you are subscribed to the Google Groups
> "mongodb-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mongodb-user...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Diego Woitasen]

That's right... my code looks like that...

But... it works in another machine, same code... verified :)

Let me research a little more...

> You received this message because you are subscribed to a topic in the Google Groups "mongodb-user" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/mongodb-user/APLpTS4iIuw/unsubscribe?hl=en.
> To unsubscribe from this group and all its topics, send an email to mongodb-user...@googlegroups.com.

> For more options, visit https://groups.google.com/groups/opt_out.
>
>

--
Diego Woitasen

[Diego Woitasen]

Fixed, thanks.

Now I'm doing

>>> for i in xrange(2):
... doc = {}

... doc['i'] = i
... c.shard_db.trackvars.insert(doc)

And works.... Anyway I don't understand how this is working in production :P

--
Diego Woitasen

[Bernie Hackett]

Depending on whether you use MongoClient or Connection, and what write
concern is in use, it may not be working in production. Check the logs
on your mongod instances for duplicate key errors.

[Diego Woitasen]

I'm updating production right now :P

[Sergio Sandino]

"name":"MongoError","code":11000,"err":"insertDocument :: caused by :: 11000 E11000 duplicate key error index this error is in node.js 

[Yaroslav Kyrpych]

 

Hi,

 

I tried Diego's solution although my case have several nested ifs and whiles:

 

while True:

     doc={}

     doc['a']=1

     db.collection.insert(doc)

 

but still getting "DuplicateKeyError: insertDocument :: caused by :: 11000 E11000 duplicate key error index: ".

Is using objectIds generator within the loop only solution?

How does write concern affect this? I have w=1, and journaling enabled.

 

Thank you,

 

Yaroslav

[Yaroslav Kyrpych]

As follow-up: it works via bson.objectid generation, however, could you let me know if it's possible to restructure code in such a way so that I don't need to use id generator? My attempts to restructure so far have not yielded any results.

 

Thank you in advance,

 

Yaroslav

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.

---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.

To post to this group, send email to mongod...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/620de523-aeb9-4dff-83b7-4bd24cec84c5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Asya Kamsky]

It'll probably be easier to understand what is going on if you start a new thread rather than replying to an old one, and provide the details of versions you are using, your code, the error, etc.

Asya

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CABSeBixUUimBFSuZBAEj08OkHtqyAwH6S6q9hvmoXb6ZH8P1ig%40mail.gmail.com.