using upsert=True with queryset.update()

[Nathan]

I am trying to atomically upsert several documents and am not having
much success. I am doing the following:

ids = ["id1", "id2", "id3"]
update_params = {"inc__field": 1, "inc__field2": 1}
ClassName.objects(_id__in=ids).update(upsert=True, **update_params)

When an id is not yet present in mongo, a new document is created/
stored with 'field' set to 1 and 'field2' set to 1 as expected.
However, _id is not being set correctly (I set _id in __init__) and
other mongoengine boilerplate is not present (there is not "_cls"
field).

I was expecting new documents to be upserted as if I had done the
following (assuming "id3" was not yet present in mongo):

obj = ClassName(_id="id3", field=1,field2=1)
obj.save()

Any help would be greatly appreciated!

[Dan Crosta]

Upsert will create at most one document at a time in MongoDB, so this won't be possible. You can, of course, issue single-document upserts in a python loop:

    for id in ids:

        ClassName.objects(_id=id).update(upsert=True, **update_params)

- Dan

[Nathan]

Hey Dan,

Thanks for the help. Unfortunately I am still running into the same
problems. Namely, _id is set to a randomly generated id, not the id
passed to .objects() and there is no mongoengine boilerplate saved in
mongo (ex. no "_cls" field). Further, I dont think I can pass in _id
to the update command as a hack to get around this?

Is there a way to atomically upsert using mongoengine such that any
object that must be created is identical to an object created via:

obj = ClassName(_id="id3", field=1,field2=1)
obj.save()

For example, assume I had a simple counter class:

class Counter(mdb.Document):
_id = StringField(primary_key=True) #counter name
count = IntField(default=0)

and I want to be able to atomically increment/create it. i could try
the following:

counter_name=...
Counter.objects(_id=counter_name).update(upsert=True, inc__count=1)

works great if the _id is alread there. However, this will write a doc
with a randomly generated id if not

Further, I could try:
obj, created = Counter.objects.get_or_create(_id=counter_name,
defaults={'count': 1})
if not created:
update....

However, that seems inefficient as i have to hit the db twice?

Further, I could modify the class to use a mongo generated id, but
then each doc would have to store 3 fields instead of 2

Let me know if I'm missing something or thinking about this the wrong
way. Any help is greatly appreciated!