Multiple modifications within a single update

[AndrewO]

Hi,

Over on the MongoDB list, I asked about querying based on the size of
an embedded array. One of the people over there recommended that I
keep a counter on the same document and increment/decrement it
whenever I changed the array.

When I looked into doing this with MongoMapper, it looked like there
was no way to have this happen in one operation. I don't know enough
about how Mongo's concurrency works, but it seemed like this would be
bad from atomicity standpoint, and possibly from a performance one as
well. Is this right?

I went ahead and implemented a MongoMapper plugin that would allow you
to queue up multiple modifications using a block and then execute them
after the block is done:

https://github.com/AndrewO/mongo_mapper_multi_modifiers

I'd appreciate any feedback.

Thanks,
Andrew

[Abel Tamayo]

Can't that be done with a model callback like this:

class MyClass

  include MongoMapper::Document

  key anArray, Array

  key arrayCounter, Integer

  before_save { self.arrayCounter = anArray.length }

end

--
You received this message because you are subscribed to the Google
Groups "MongoMapper" group.
For more options, visit this group at
http://groups.google.com/group/mongomapper?hl=en?hl=en

[AndrewO]

Hi Abel,

I hadn't thought of using a callback. But doesn't that introduce a
race condition on the value of arrayCounter? If I retrieve a document,
someone else removes an embedded item (and changes the counter) and I
add a value, then I've overwritten their change (side question: does
MongoMapper optimistically lock? I don't see anything from a cursory
glance at the code). Based on my (albeit limited) understanding of
MongoDB, I think I'd prefer to use the atomic operations in
modifications like this.

Also, I'm pretty sure I'd want to make other changes to an object that
aren't necessarily determinable from a single key's value (e.g.
pushing into two embedded arrays at the same time), so from what I can
tell, I'd still need the functionality from the plugin.

Thanks,
Andrew

[Abel Tamayo]

Yeah, I didn't think about races and I'm not sure about how MongoDB manages blocking right now... I'll have to look into it.

[Jamie Orchard-Hays]

Mongo is fire and forget. There is a safe mode you can look into.

You're not going to have a race condition any different on setting your counter than any other field. I use the callback strategy for a number of things. It's a great tool.

Setting a counter for your array is probably your most pragmatic solution until Mongodb adds support for querying by array length.

[AndrewO]

Hi Jamie,

By "fire and forget" do you mean that it won't return error messages
unless you enable safe mode? Thanks for reminding me of that.

Still, if I understand correctly, multiple atomic modifiers within an
update statement are atomic. It seems we'd want to use those wherever
possible, especially since operations between multiple docs and
collections are not.

Moving on from the array/counter example to something more general,
here's an example of what I'm worried about:

https://gist.github.com/702039

(This hasn't actually been run, so there may be errors. Lines with ----
> are what I imagine the driver is doing.)

In the promote! method, if we fire off two updates (lines 9-12),
there's a chance that another object could read between them and make
a decision based off of data in the middle of a change. But if we use
the atomic operators in a single update (lines 46-51—the modify method
is from my plugin), we don't have this problem, right?

–Andrew

[John Nunemaker]

The syntax for this looks really interesting. I like the block idea. I will look through the code and see if it is something worth pulling into the MM modifier plugin. Nice work.

[AndrewO]

Thanks, John. I'd love it if it were pulled in. I tried to stick with
the tools and style in MM as much as possible (although had to go with
T::U assertions in the test).

I whipped it together in one night of hacking, so I'm sure it can be
improved. Let me know if I can help with anything.

[Jon Kern]

you forgot to implement

def sooooooooooooooo
schedule_weekend_work
end

jon
blog: http://technicaldebt.com
twitter: http://twitter.com/JonKernPA

AndrewO said the following on 11/16/10 12:29 PM:

> Hi Jamie,
>
> By "fire and forget" do you mean that it won't return error messages
> unless you enable safe mode? Thanks for reminding me of that.
>
> Still, if I understand correctly, multiple atomic modifiers within an
> update statement are atomic. It seems we'd want to use those wherever
> possible, especially since operations between multiple docs and
> collections are not.
>
> Moving on from the array/counter example to something more general,
> here's an example of what I'm worried about:
>
> https://gist.github.com/702039
>
> (This hasn't actually been run, so there may be errors. Lines with ----
>> are what I imagine the driver is doing.)
> In the promote! method, if we fire off two updates (lines 9-12),
> there's a chance that another object could read between them and make
> a decision based off of data in the middle of a change. But if we use

> the atomic operators in a single update (lines 46-51�the modify method

> is from my plugin), we don't have this problem, right?
>

> �Andrew