findAndModify multiple documents

[clarity]

Hello,

Will findAndModify ever support modifying multiple documents at once?

Thanks

[Andreas Jung]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

clarity wrote:
> Hello,
>
> Will findAndModify ever support modifying multiple documents at once?

Please read

http://www.mongodb.org/display/DOCS/findAndModify+Command

- -aj
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQGUBAEBAgAGBQJNh4/pAAoJEADcfz7u4AZjol4Lv0jJsqN8WUumTVBinlcS3RAr
4p2XBNauIZpUiqwMqT+n+0uJWQPHLJBfvj4whpZWXkDFQgl5CzzKlPPXJCBzB7BC
0TbkceJVuEhCp+qmhpWgmAxFrJOHVQjBjuUCdKOXpmTv6GQ5HcTeBl9lcs+T52wQ
F9aRlSMx0cI/SlVwCSp2IKOWSaf9WqVuS+atq1vBOUr4I8NTu9nVzc/VA2CgoNmp
YmPGPi9TD9DYn/2fPcer/XQkx/8IXzE1qioykZyUuz73eETZiCs4tx6lG7uSS7Ke
UTHtNu9u1dDlajDGgBQgia96f9RrSSrnAbgxtQ1zpORyRIvxhHysFQMiHwNhEKpF
3NKZpEWpsck0lqnbpUu+n+eLZxZ+oqfLSAWElyePIztgKINAwRTXS6CbvKF/U9Gy
CavctGFw2Kuje/AMMQJNYvS+5cg1OrrFdh5uNIhs6GhAEDeZftmDMELD0ugCe21i
0dLI7kiwLMO0GlEiOOPILuEl3LGmq/8=
=YIxr
-----END PGP SIGNATURE-----

[Gates]

@clarity:

What is the purpose the multi-document findAndModify?

Right now the findAndModify interface (or behavior) is pretty clear.
- inputs: query, update operations, flags
- output: document that was modified
- result: atomic modification of a single document

If you want to support the modification of multiple documents, then
you're asking to change both the output and the result. That's a
pretty significant change.

Again, is there a specific reason for this change?

- Gates

[clarity]

I built a game server utilizing mongodb, and I need to send messages
to users on a different server, so I'm writing the data to a
listeners[] array.

On every server, I'm going to have an algorithm that runs every second
that finds all the documents where numListeners (the number of values
in the listeners array) is greater than 0 and the user is on the local
server. Then, it pushes out the data in the listeners array.

I'm going to use findAndModify, but I don't like the idea of having to
run findAndModify for every user logged in (up to 10k) every second...
sounds too slow.

--Ryan

[Andreas Jung]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

clarity wrote:
> I built a game server utilizing mongodb, and I need to send messages
> to users on a different server, so I'm writing the data to a
> listeners[] array.
>
> On every server, I'm going to have an algorithm that runs every second
> that finds all the documents where numListeners (the number of values
> in the listeners array) is greater than 0 and the user is on the local
> server. Then, it pushes out the data in the listeners array.

Why won't a simple update() work here?

- -aj
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQGUBAEBAgAGBQJNh5kAAAoJEADcfz7u4AZjYnkLwMkdJYzmARYXFiOWsOc9rMbp
Md7pqXV5jIOafp2lxINFhH5cQBAZLdDM//dplxcGyij/gZSSQfOck5Ti9VaWs4bw
34SZwP60t36HhNtLLG6hjowP9E1CY//jDu9xIZGkgXiO2uc58JcUbXWsIiY22wj+
6fYSZX180/ObiW8TOFdxPjBp4hdzx6hFkfsdZGzkyn97RssEmNZGnkluzUr0xtz/
xok5XiNHp8NtH0aO7Au1CznjJ88ktTMGxGhETdQMVSlJCJSD6XjdO02/0pUZEP4o
tic81I/J2Cw135JshcOw2T1vMvz+s52agAY7UrV0qZFb0t1LV37NvuVgvb/pRzt8
AWxHbII4KHgMrvaI4Uoh2r2abSdNDaWeIv8ZCwPGq/qBFTLKemweQ43uEQVHfcsc
TZUOdAM+qZnYhtf3eJo8GJM1ZacNI73sMk9VqSsF/c5rmhLotlaIEF+ELWFwfOQw
e6MUPWFi7yqdKLWIkqveJ1K1VY+o0zY=
=Qwc9
-----END PGP SIGNATURE-----

[clarity]

well, I'd have to find it, process all of the listeners, and then
update it while making sure that the listeners haven't changed..
findAndModify works better because I can update it and get the
listeners at the same time.

--Ryan

On Mar 21, 11:29 am, Andreas Jung <li...@zopyx.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> clarity wrote:
> > I built a game server utilizing mongodb, and I need to send messages
> > to users on a different server, so I'm writing the data to a
> > listeners[] array.
>
> > On every server, I'm going to have an algorithm that runs every second
> > that finds all the documents where numListeners (the number of values
> > in the listeners array) is greater than 0 and the user is on the local
> > server. Then, it pushes out the data in the listeners array.
>
> Why won't a simple update() work here?
>
> - -aj
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (Darwin)

> Comment: Using GnuPG with Mozilla -http://enigmail.mozdev.org/

>
> iQGUBAEBAgAGBQJNh5kAAAoJEADcfz7u4AZjYnkLwMkdJYzmARYXFiOWsOc9rMbp
> Md7pqXV5jIOafp2lxINFhH5cQBAZLdDM//dplxcGyij/gZSSQfOck5Ti9VaWs4bw
> 34SZwP60t36HhNtLLG6hjowP9E1CY//jDu9xIZGkgXiO2uc58JcUbXWsIiY22wj+
> 6fYSZX180/ObiW8TOFdxPjBp4hdzx6hFkfsdZGzkyn97RssEmNZGnkluzUr0xtz/
> xok5XiNHp8NtH0aO7Au1CznjJ88ktTMGxGhETdQMVSlJCJSD6XjdO02/0pUZEP4o
> tic81I/J2Cw135JshcOw2T1vMvz+s52agAY7UrV0qZFb0t1LV37NvuVgvb/pRzt8
> AWxHbII4KHgMrvaI4Uoh2r2abSdNDaWeIv8ZCwPGq/qBFTLKemweQ43uEQVHfcsc
> TZUOdAM+qZnYhtf3eJo8GJM1ZacNI73sMk9VqSsF/c5rmhLotlaIEF+ELWFwfOQw
> e6MUPWFi7yqdKLWIkqveJ1K1VY+o0zY=
> =Qwc9
> -----END PGP SIGNATURE-----
>

>  lists.vcf
> < 1KViewDownload

[Gaetan Voyer-Perrault]

@clarity: based on your description, what you are building here is a basic variant of a Message Queue.

You want to grab an object, clear the listeners array and then process the data that was in the listener. 

This makes sense, but it still doesn't clarify why you need to a "multi" version of findAndModify. At some point you have to process each item individually. If you did a "multi-findAndModify" and returned the 10k changed items, then for some period of time, all of those items would exist only in memory on your app. At this point, your data is no more secure than if you were just using a regular update().

Back to your original point here, the "multi-findAndModify" doesn't really fix this issue here.

> but I don't like the idea of having to run findAndModify for every user logged in (up to 10k) every second... sounds too slow.

This is a problem we can handle. 

Have you tried it for 10k / second?

What type of hardware are you throwing at this problem?

Given that you're doing "findAndModify", following by "update other box", you'll probably need to test this with multiple threads b/c "update other box" will introduce network latency. 

MongoDB has several deployments operating above 10k writes / second. So it's probably worth investigating the available tools.

- Gates

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.