Next and previous document

[Roger]

Hi all

What is the best concept for get a next or previous document
from mongodb?

I have a document and need to get a next or previous document.
The best whould be to use limit=1 and size=1 but this requires
that I start from a known (batch) position or from the first
document.

Do I need to get all documents and find the position from
the current document in the given cursor and then use the
size and limit methods?

btw,
next or previous should depend on insert order.

I'm using pymongo 1.9.0 and the newest mongodb.

Any hints?

Regards
Roger Ineichen
_____________________________
END OF MESSAGE

[Andreas Jung]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Roger wrote:
> Hi all
>
> What is the best concept for get a next or previous document
> from mongodb?
>
> I have a document and need to get a next or previous document.
> The best whould be to use limit=1 and size=1 but this requires
> that I start from a known (batch) position or from the first
> document.
>
> Do I need to get all documents and find the position from
> the current document in the given cursor and then use the
> size and limit methods?
>
> btw,
> next or previous should depend on insert order.
>
>

Why don't you use a simple counter as _id?
Navigating to the previous or next document based on the current _id
is then trivial.

- -aj
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQGUBAEBAgAGBQJMzZ69AAoJEADcfz7u4AZjUVwLwKqf3Fu2TZ0wikK9k+3PGFoL
a/qcqBP5YKkDBqiXnP1/qmwrS5xvHPZC4BZE83DZlQiTqhUUqGvqalDDcJ85IQoE
gXNMKTKDO3IGXzlFrLQYMowA7NLO+URuCBzkkFwnZeCotcTtCSNL8/Bs+wIuKKiw
6DahtFY1iei5Uxqm3869DTxI4vYpm/AUexKO0G4h9kWzZBgBMQco9GdbYQhAP9kQ
WODaoaQ/IdpgNGzWr3D5UVlZcAFLV49nWgIttk5NfHTFZgiRGptiG+afgxXo+Wd+
YkEcAuG9NkUjPNrJwZG8z0XS8qJMqKUnFOJmuC6j4dOGBXqPrQOK6pZ9sw5cb9Z5
GaPa0f5DBDumS5ae4BtQGj0BxqF/q4illfTvUXGVVYqs+eQL4TnjiBKUQL12iMnp
9jIlDS7msJCcJDYKWjV+aAaoTIWX4bpQ06buUWIjorFIbfO6FRQyL701QnsiRF+t
DvHRGlBVRLviwJGQ6PNFCA9yRxRGOyU=
=zCau
-----END PGP SIGNATURE-----

[Roger]

Hi Andreas

> Betreff: Re: [mongodb-user] Next and previous document

>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Roger wrote:
> > Hi all
> >
> > What is the best concept for get a next or previous document from
> > mongodb?
> >
> > I have a document and need to get a next or previous document.
> > The best whould be to use limit=1 and size=1 but this
> requires that I
> > start from a known (batch) position or from the first document.
> >
> > Do I need to get all documents and find the position from
> the current
> > document in the given cursor and then use the size and
> limit methods?
> >
> > btw,
> > next or previous should depend on insert order.
> >
> >
> Why don't you use a simple counter as _id?
> Navigating to the previous or next document based on the
> current _id is then trivial.

This whould be my last option. But this requires a
global counter and a lock mechanism if I write from
more then one client to the mongodb.

Or can this be done in a simple way without to lock
or sync the global counter with more then one client
using this counter?

btw,
Nice to meet some Zope geeks on this list ;-)

Regards
Roger Ineichen

> - -aj
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (Darwin)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iQGUBAEBAgAGBQJMzZ69AAoJEADcfz7u4AZjUVwLwKqf3Fu2TZ0wikK9k+3PGFoL
> a/qcqBP5YKkDBqiXnP1/qmwrS5xvHPZC4BZE83DZlQiTqhUUqGvqalDDcJ85IQoE
> gXNMKTKDO3IGXzlFrLQYMowA7NLO+URuCBzkkFwnZeCotcTtCSNL8/Bs+wIuKKiw
> 6DahtFY1iei5Uxqm3869DTxI4vYpm/AUexKO0G4h9kWzZBgBMQco9GdbYQhAP9kQ
> WODaoaQ/IdpgNGzWr3D5UVlZcAFLV49nWgIttk5NfHTFZgiRGptiG+afgxXo+Wd+
> YkEcAuG9NkUjPNrJwZG8z0XS8qJMqKUnFOJmuC6j4dOGBXqPrQOK6pZ9sw5cb9Z5
> GaPa0f5DBDumS5ae4BtQGj0BxqF/q4illfTvUXGVVYqs+eQL4TnjiBKUQL12iMnp
> 9jIlDS7msJCcJDYKWjV+aAaoTIWX4bpQ06buUWIjorFIbfO6FRQyL701QnsiRF+t
> DvHRGlBVRLviwJGQ6PNFCA9yRxRGOyU=
> =zCau
> -----END PGP SIGNATURE-----
>

> --
> You received this message because you are subscribed to the
> Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to
> mongodb-user...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/mongodb-user?hl=en.
>
>

[Andreas Jung]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Roger wrote:

>
>> This whould be my last option. But this requires a
>> global counter and a lock mechanism if I write from
>> more then one client to the mongodb.

Well what determines the ordering? The insertion time?

If yes, why can't you use a created timestamp, search on items
larger than the timestamp of the current object and limiting to the
result set to one row (same for previous)?

>
>> btw,
>> Nice to meet some Zope geeks on this list ;-)
>
>

Yeah - just reminds me that we had no-SQL 10 years ago - before
MongoDB :-)

- -aj
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQGUBAEBAgAGBQJMzaOAAAoJEADcfz7u4AZjy0ALwNs8hjJBmd/r1G3OdTp4/lX/
CpjouevNjW51sNCSNdTkhC7Xv80TPPV0bbowrrS4paSdEzVax3uFcVIjfOP1voxh
4tj+4ARdEXusVDfkM+xsSMR0ag5TkG7wiqriU7aL/Mz7p0M9WNtclq7cN3Dfp40+
QFN74XpSK8uXm3SshjnENuagZ/ocFEyU4ZrH5qcgY/tZaWarq7I1zFdDbt/GRHt9
YHvf9I5fCOpFg+8ftTsqny/tDItd6YfUMJQBNyXII1IPIQ6DgMld5k8JabgPD9s9
GZ9lgNyQQwe5OWXrduQvQh9dWVytw0+9R/Nwrki2tyeh3jzoIsNuCdQ/4Hm2RnGE
QhTS+CnVL6//URzfeYgFZtyC2CHWqF+UsgFqB5zoLcN39vhLSm9HJi/HHGZgJga+
Bqtb4/umuqfCnk6VQmVvkPwR9nC795ugNZXHddA2TNtPHF80nDHiICq6jDe2XfgS
RXcl0TBGVFL/GcbW4YBCbGYzZSt2VYA=
=zgEt
-----END PGP SIGNATURE-----

[Andreas Jung]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Andreas Jung wrote:
> Roger wrote:
>
>>> This whould be my last option. But this requires a
>>> global counter and a lock mechanism if I write from
>>> more then one client to the mongodb.
>
> Well what determines the ordering? The insertion time?
>
> If yes, why can't you use a created timestamp, search on items
> larger than the timestamp of the current object and limiting to the
> result set to one row (same for previous)?

Of course you need to sort first before applying the limit.

- -aj
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQGUBAEBAgAGBQJMzbC5AAoJEADcfz7u4AZjivALv1ie+HZKs4NzAr29ZsHnvVA1
Xd0CWR3SX8HHU4AFqxX8lsCL/AeScWRtvuPQcDsEm/w1195PThd1avX1myBVLKhF
/vR7zjVsmirqlwIG8xiCo9K/uX2JEICR2LdeBSd35G1MY2U9AEaUwxQ2GcLC2olK
pZDF6drINoyuusx0S+eFoyFqTPO3SnS24D1AOpTOV9OrmsWgMczdUxQJLwpfdfq6
vIbOUmwttWdXr8fhtapqZ8CzOOYu5dBycAzuv7qPbiaWv3IXMEiaSH9Nr9wtuBd1
vIYnWxBBF5gZA8A3Ryeeoru1kNKYFQz4AftWKJ9jrPLSRlJBAyEcvW7llFtv6UzE
xoIk/3InB+QHPk7OIkMFPOnMcyGpu6bPFGIBi+4F6mg5zBO6P6H4IZtCXXmhmkCK
YHDlz3KqpKmtO3nhTi568CA0IpCDIMmLIpaqzBTzA4Enti/EROAGrLboigl67xT5
4SsbnVbVC9V0t5f9f2qXsXUnB8mBkaE=
=l2qM
-----END PGP SIGNATURE-----

[Roger]

Hi Andreas

> Betreff: Re: AW: [mongodb-user] Next and previous document

>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Roger wrote:
>
> >
> >> This whould be my last option. But this requires a global
> counter and
> >> a lock mechanism if I write from more then one client to
> the mongodb.
>
> Well what determines the ordering? The insertion time?

Yes, I could add an explicit timestamp.

> If yes, why can't you use a created timestamp, search on
> items larger than the timestamp of the current object and
> limiting to the result set to one row (same for previous)?

Good idea! I think this could work. It whould also
prevent some problem if documents get removed compared
to an integer counter.

Thanks a lot

Regards
Roger Ineichen

> >
> >> btw,
> >> Nice to meet some Zope geeks on this list ;-)
> >
> >
>
> Yeah - just reminds me that we had no-SQL 10 years ago -
> before MongoDB :-)
>
> - -aj
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (Darwin)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iQGUBAEBAgAGBQJMzaOAAAoJEADcfz7u4AZjy0ALwNs8hjJBmd/r1G3OdTp4/lX/
> CpjouevNjW51sNCSNdTkhC7Xv80TPPV0bbowrrS4paSdEzVax3uFcVIjfOP1voxh
> 4tj+4ARdEXusVDfkM+xsSMR0ag5TkG7wiqriU7aL/Mz7p0M9WNtclq7cN3Dfp40+
> QFN74XpSK8uXm3SshjnENuagZ/ocFEyU4ZrH5qcgY/tZaWarq7I1zFdDbt/GRHt9
> YHvf9I5fCOpFg+8ftTsqny/tDItd6YfUMJQBNyXII1IPIQ6DgMld5k8JabgPD9s9
> GZ9lgNyQQwe5OWXrduQvQh9dWVytw0+9R/Nwrki2tyeh3jzoIsNuCdQ/4Hm2RnGE
> QhTS+CnVL6//URzfeYgFZtyC2CHWqF+UsgFqB5zoLcN39vhLSm9HJi/HHGZgJga+
> Bqtb4/umuqfCnk6VQmVvkPwR9nC795ugNZXHddA2TNtPHF80nDHiICq6jDe2XfgS
> RXcl0TBGVFL/GcbW4YBCbGYzZSt2VYA=
> =zgEt
> -----END PGP SIGNATURE-----
>

[jdill]

Just watch out. If documents are inserted at the same time as they are
being read, you could end up with partial results during newly
inserted queries, and thus its not a perfect way to ensure you get all
next set of records, as more could have been inserted with the same
timestamp in the interim.

For this, ended up needing to use capped collections and natural
order with skip.. it was the only way to guarantee that pagination
works exactly the same in every case, even while new data is pouring
in from multiple clients.

Jeremy

> > Comment: Using GnuPG with Mozilla -http://enigmail.mozdev.org/

[Dwight Merriman]

if you can use a capped collection they naturally maintain insertion order.  but that isn't always applicable.

if you want to use monotonically increasing ids a way to do it is documented here : 

http://www.mongodb.org/display/DOCS/Atomic+Operations#AtomicOperations-%22InsertifNotPresent%22

[Markus Gattol]

> Just watch out. If documents are inserted at the same time as they are
> being read, you could end up with partial results during newly
> inserted queries

This is why you can snapshot queries i.e. a query will have a
consistent view of the data set even if the data set changes while the
query is still being run agains the database.