NDB "Deadlock waiting for..." on datastore get/put/query in 1.6 pre-release?

[Steve Sherrie]

I'm testing the 1.6 pre-release SDK (which has ndb included) and getting the following error on all gets, puts and queries.

RuntimeError: Deadlock waiting for <Future 4982090 created by tasklet_wrapper(tasklets.py:906) for tasklet positional_wrapper(datastore_rpc.py:84); pending>

It's late, so I'll look into this more tomorrow, but has anyone else tested in 1.6, and if so, was this also the case for you?

Steven Sherrie

[Guido van Rossum]

AFAICT the NDB in the SDK prerelease is fully functional. Does your app work with the NDB from the NDB code site? Did you adjust all your imports to use google.appengine.ext.ndb? Maybe using dev_appserver.py --debug will shed some light on your issue.

--Guido

--
--Guido van Rossum (python.org/~guido)

[Steve Sherrie]

I just tried the 2.7 hello world example, with the same result (whether importing ndb from google.appengine.ext or from the separately downloaded package).

import webapp2

from google.appengine.ext.ndb.model import Model, StringProperty

#from ndb.model import Model, StringProperty

class TestModel(Model):

  a_name = StringProperty()

class MainPage(webapp2.RequestHandler):

  def get(self):

    self.response.headers['Content-Type'] = 'text/plain'

    self.response.out.write('Hello, webapp2 World!')

    TestModel(a_name='String Name').put()

app = webapp2.WSGIApplication([('/', MainPage)],

                              debug=True)

Also, using the --debug flag on the dev_appserver.py did not seem to provide any extra details.

Steven Sherrie

[Guido van Rossum]

Hm... Apparently webapp2 and NDB don't like each other. I'll look into
this more.

[Steve Sherrie]

Thanks!

[Steve Sherrie]

Oh, the other important point is that the problem vanishes when deployed.

Steven Sherrie

[Beech Horn]

Steve in the example you posted I cannot see a @context.toplevel
decorator on MainPage.get().

IIUC without such a decorator ndb cannot progress the eventloop and
hence you'll get deadlock issues (no progress == deadlock).

FYI I tried a couple of py25 apps on 1.6.0 dev_appserver using the
built in ndb version without any troubles, but this doesn't mean that
there aren't any.

[Guido van Rossum]

On Wed, Nov 2, 2011 at 13:57, Beech Horn <beec...@gmail.com> wrote:
> Steve in the example you posted I cannot see a @context.toplevel
> decorator on MainPage.get().
>
> IIUC without such a decorator ndb cannot progress the eventloop and
> hence you'll get deadlock issues (no progress == deadlock).

Actually, as long as you don't use any async_*() calls or yield, you
don't need that decorator -- the synchronous calls run the event loop
until they see progress. I see the same problem with or without it,
but only in the Python 2.7 runtime. My hunch is that the event loop
state gets reset however by some shenanigans with os.environ, which is
not a simple dict in the 2.7 App Engine runtime.

I'll get to the bottom of this yet.

> FYI I tried a couple of py25 apps on 1.6.0 dev_appserver using the
> built in ndb version without any troubles, but this doesn't mean that
> there aren't any.

It works consistently with py25 for me too.

--Guido

[Guido van Rossum]

I think I have a fix. You can see it here:
http://codereview.appspot.com/5331059 You can apply it either to the
copy of NDB you got from the NDB code site or in the prerelease SDK.

The problem is that when using the Py27 SDK, somehow os.getenv()
doesn't act the same way as os.environ.get().

I'll see if I can get this fixed in the final 1.6 SDK that goes out
next week. Hopefully the fix is to make os.getenv() work properly.

--Guido

[Steve Sherrie]

Works for me. Thanks Guido!

Steven Sherrie