MongoEngine with MongoHQ, can save but not query.

[Matthew]

I was testing out MongoHQ using MongoEngine but it seems that
everytime I query something, I get this error:

database error: unauthorized db:testdb lock type:-1

I've no problem adding new entries to the db.

My connection:
mongoengine.connect('testdb', username='testuser',
password='testpass', host='flame.mongohq.com', port=27034)

I've tested with a local mongodb (taht req username and password), it
works fine.

I'm not sure if this has to do with MongoEngine, PyMongo or MongoHQ.

Any help would be greatly appreciated.

[JAM]

Hi Matthew,

Try this:

connect("dbname", host="usermongohq:passwor...@flame.mongohq.com:
27034/dbname")

before attempting to connect your application to mongohq, create the
database and look the server name and the port number

in this example my server name is: flame.mongohq.com
and my port is: 27034

I hope to serve you :D

Greetings
------------------------------------------
sorry for my bad English, I only speak Spanish but I read English
well LOL

[Matthew]

Hi JAM,

Thanks for that. But I'm still getting the same error. I don't think
my connection has problem because I can save new documents to MongoHQ,
just that I can't query.

On Jan 11, 2:24 pm, JAM <jesusang...@gmail.com> wrote:
> Hi Matthew,
>
> Try this:
>

> connect("dbname", host="usermongohq:passwordmong...@flame.mongohq.com:

[Derek Payton]

I'm also getting the same error with MongoEngine+MongoHQ. What I
noticed is that it only happens when running as a web server (either
Django's runserver or gunicorn). If I drop into a python shell,
everything works as expected and I can connect no problem.

Any ideas?

[Matthew]

I think I've got the solution for this.

It seems that Django/mongoengine "disconnect" the connection after
certain period of inactivity. This happens with any remote MongoDB
(not just MongoHQ). So the workaround is to tell mongoengine to
reconnect to the db everytime a query is made.

In QuerySetManager class in mongoengine.queryset

Make sure to add reconnect=True to db = _get_db() as such:

db = _get_db(reconnect=True)

Hope this helps.

matt

[Derek Payton]

Setting up a connection on a per-query basis is ... gnarly. The
problem seems to be a threading one. Try this out, seems to work fine
for me:

https://gist.github.com/783993

Mongoengine stores the connection in a global object that doesn't
transfer nicely from the master thread to children (this would also
explain why the error pops up in runserver/gunicorn, but not the
shell). We need to establish a separate connection for every thread,
and this middleware takes care of connecting to mongodb on the first
request of each thread. It's not my preferred solution, but it seems
to work.

--Derek

[Derek Payton]

I've updated the middleware to connect to mongoengine in the __init__
method and raise a MiddlewareNotUsed exception. This will create the
connection to MongoDB when the thread is started, but removes the
overhead of processing the middleware and checking threadlocals on
every request.

https://gist.github.com/783993

--Derek

[Brent]

Those of you who've had this error, do you have a test that can
consistently reproduce it, or could point me in the right direction on
how to write one?

I'm pushing to use MongoEngine for a project at work, but this
threading issue makes me (and other devs) uneasy. I'm investigating
what it would take to make mongoengine thread safe, and starting with
a test that can cause this error seems like a good place to start.

Thanks!
Brent