pymongo handling autoreconnect

[sebest]

Hello,

Is there a way to handle the AutoReconnect exception without wrappring
every query in a try/catch ?

thanx

[Dwight Merriman]

what language/driver?

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

[Kyle Banker]

The short answer is no. So far the philosophy on this has been to let the application handle this. That said, it's not difficult to build a very simple wrapper for pymongo to to handle this for you.

[Dwight Merriman]

the question is what do you want to do when an error occurs?  

one approach is put a try/catch top level.  say top level of generation of a page:

try {

  dowork();

}

catch { 

  return error page;

}

then you have a bunch of queries underneath below doWork(), down the call stack, and none of them do any catches.

On Sun, May 30, 2010 at 11:20 AM, sebest <sebastien...@gmail.com> wrote:

[Andreas Jung]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dwight Merriman wrote:
> the question is what do you want to do when an error occurs?
>
> one approach is put a try/catch top level. say top level of generation
> of a page:
>
> try {
> dowork();
> }
> catch {
> return error page;

Here is some example code how app-level reconnection could be handled:

from ym.db import getCollection
from ym.db.database import setupDatabases
import time

from pymongo.errors import AutoReconnect

coll = getCollection('profiles')

while 1:
try:
coll.save(dict(a=2))
print 'written'
except AutoReconnect:
count = 0
while count < 10:
try:
print 'trying to reconnect'
setupDatabases()
print 'reconnected'
break
except AutoReconnect:
count += 1
time.sleep(1)

getCollection() and setupDatabases are some high-level methods
for setting up database connections and for getting hold of a collection
by their unique name (specific to our project).

This approach is work in theory. But there is a major problem in
reality. Having to use try..except or try...catch (depending on the
language) will make your code very ugly...
Ok in Python 2.6 we have context managers where we can deal with
reconnections issues in a better way than using the standard exception
handling (no idea about Java or other languages).

I think there should be better reconnection support on the driver level.
Either it should be handled completely transparently to the application
(e.g. a save() operation would fail with an explict error after N tries
to reconnect within M seconds) - or the application should be able to
register a callback method that would be called by the driver in case of
a disconnect. The callback method would get the related parameters in
order to reconnect on the application level and the driver could re-try
the operation....this is only a very rough sketch...one might check how
other (Python) database adapters deal with that.

Speaking for Zope: in Zope we have the standard Python database bindings
(specific to each database type) and a database adapter on top handling
transaction integration with Zope, providing a connection pool and
dealing with connection handling.

Andreas

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkwCljUACgkQCJIWIbr9KYzaYgCgsHU7TLeeFBEJRInI8H0fyf50
3OoAn0f0CJqWA3ZiSb88HL/5Mp6mK8l4
=pvzw
-----END PGP SIGNATURE-----

[Dwight Merriman]

good question what should or shouldn't be in the driver

for now i think best is to write your own save() and query() methods that do reconnect attempts.  then you won't have to put that everywhere.

--

[sebest]

Sorry for the late answer, i'm using pymongo and mongokit.

I currently use a similar solution, using decorators, but maybe the
context manager approach could be better.

Do you have any example of it?

Maybe pymongo could have such a context manager builtin?

thanx

> Comment: Using GnuPG with Mozilla -http://enigmail.mozdev.org/

>
> iEYEARECAAYFAkwCljUACgkQCJIWIbr9KYzaYgCgsHU7TLeeFBEJRInI8H0fyf50
> 3OoAn0f0CJqWA3ZiSb88HL/5Mp6mK8l4
> =pvzw
> -----END PGP SIGNATURE-----
>

>  lists.vcf
> < 1 000AfficherTélécharger

[sebest]

Here is some example code to handle transparent reconnection easily
using monkeypatching:
http://paste.pocoo.org/show/224441/

works great with replica pairs and django

you should adjust the number of retry and the delay between each retry
if you are using this from a webapp