NDB support for GAE

[Quint]

Hi,

I think web2py should support NDB for the Google Datastore.

NDB a newer datastore api which has automatic caching features built in.

https://developers.google.com/appengine/docs/python/ndb/

I think it would be great for GAE users to be able to use NDB because it potentially has better performance and can reduce the costs of your app (dependent of the type of app off course).

In fact, i think it wouldn't be that hard to integrate it.

I had a go at it and i only had to change a couple of lines in dal.py and it seems to work.

You can switch between NDB and DB because they both should store exactly the same data in the datastore.

I attached a patch i made based on version 2.6.4

Maybe one of the developers could have a look.

I'm not a web2py expert so this does need to be reviewed by someone with more in depth knowledge.

But maybe this could serve as a starting point.

resources:

https://docs.google.com/document/d/1AefylbadN456_Z7BZOpZEXDq8cR8LYu7QgI7bt5V0Iw/mobilebasic

http://dylanv.org/2012/08/22/a-hitchhikers-guide-to-upgrading-app-engine-models-to-ndb/

Thanks!

[Massimo Di Pierro]

Thank you I will review this asap!

[Saisha D]

Hi,

 Just curious, what's the status of this patch?

I'm eagerly waiting for NDB support in web2py.

S

[Saisha D]

btw dumb question

when db.py says DAL(google:datastore)

Is it referring to the DB datastore in GAE

https://developers.google.com/appengine/docs/python/datastore/

Asking because there a few options for storage mentioned in the GAE docs.

[Massimo Di Pierro]

Thank you for the patch. I guess my question is why do we need a new adapter? Is this for backward compatibility because otherwise it would break apps that use Field('...',type='decimal(...)')?

[Christian Foster Howes]

interesting....  i as skim the NDB docs very quickly and the patch above i see that it is *very* similar to the "DB Datastore"/Big Table.  if the APIs are the same then we should just use the same DAL adapter with a parameter to use NDB.  Though because of the caching nature of NDB there may be a whole set of additional/different methods that need to be supported.

Saisha D: yes, the google:datastore in web2py currently is using the type of DB storage that you linked to (the only type of storage supported when web2py first integrated with GAE).

cfh

[Quint]

Yes, using the same adapter with a parameter is also a possibility i thought of. Don't really know anymore why i choose this. Maybe because initially i wanted to create a plugable thingy without changing the DAL code.

[Massimo Di Pierro]

If you send me a revised patch to do this, I will include. It may be better than using a different adapter.

[Quint]

Ok,

 

I will look at it this weekend.

[Quint]

Hi,

I attached a new patch.

I also attached a file with a couple of tests I ran. (based on test_dal.py)

Needs more testing by other GAE users though.

Quint

On Monday, October 28, 2013 2:49:37 PM UTC+1, Massimo Di Pierro wrote:

[Cato Gonzalez]

I am just starting a project on GAE using web2py and would certainly prefer to start with ndb from the beginning. Is this patch going to be "official"? Can I start with the regular db and then switch to ndb once it's "official"?

I am new to GAE and web2py but after reading all docs I believe they are both the technologies I choose for this new project.

Thanks.

Cato

On Tuesday, November 5, 2013 4:24:14 PM UTC-5, Quint wrote:

Hi,

I attached a new patch.

I also attached a file with a couple of tests I ran. (based on test_dal.py)

Needs more testing by other GAE users though.

Quint

On Monday, October 28, 2013 2:49:37 PM UTC+1, Massimo Di Pierro wrote:

[Quint van den Muijsenberg]

You can always switch between DB and NDB. Thats the idea of the patch.

You can also grab the trunk version and help testing it.

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to a topic in the Google Groups "web2py-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/web2py/Tek9hreXC3U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to web2py+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.