Expando and Polymodel on GAE?

[Dave]

I'm new to Web2py and trying to get a sense for its GAE support,
beyond the basic "RDBMS-like" functions (which Web2py seems to support
very well!)

Expando models and PolyModel: Is there any way to use either of these
with the DAL currently? Are there plans to, if not? PolyModel
especially seems like it wouldn't be too difficult to add, as it can
be used identically to the default Model class (all of its magic is
handled under the hood I think.)

Thanks for the information, and the great work so far. I'm really
impressed with how easy web2py is to develop with.

[mdipierro]

I agree that polymodel would be trivial. We would only need to decide
how to pass parameters to db.define_table to determine if a table is a
polymodel or extends an existing one. This would not have an
equivalent in SQL.

[mdipierro]

Can you help testing polymodel? In trunk:

db=DAL('gae')
db.define_table('contact',Field('address'), polymodel=True)
db.define_table('person',Field('first_name'), polymodel=db.contact)
db.define_table('company',Field('business_name'),
polymodel=db.contact)

db.person.insert(first_name="John", address="here')
db.company.insert(business_name="John Inc", address="there')
contacts = db(db.contact.id>0).select() # should lists both persons
and conpanies

Massimo

[Dave]

I tested this, and it does give all contacts when using
> contacts = db(db.contact.id>0).select()

Unfortunately, using
> person = db(db.person.id>0).select()

also returns all the contacts. I think the intended behavior is to
just return the contacts that are also a person (as well as those that
are a sub-class of person.) Additionally, trying to get at the person-
specific fields fails in that case, e.g.:
> for contact in person
> response.flash= contact.first_name
doesn't work.

Also, the syntax is great for supporting polymodel, but if you want to
later add support for expando classes, it might be a bit awkward.

~Dave

[mdipierro]

Can you please try again using latest trunk?

Massimo

On Sep 4, 7:13 pm, Dave <thefe...@gmail.com> wrote:
> I tested this, and it does give all contacts when using
>
> > contacts = db(db.contact.id>0).select()
>
> Unfortunately, using
>
> > person = db(db.person.id>0).select()
>
> also returns all the contacts.  I think the intended behavior is to
> just return the contacts that are also a person (as well as those that
> are a sub-class of person.)  Additionally, trying to get at the person-
> specific fields fails in that case, e.g.:> for contact in person
> >    response.flash= contact.first_name
>
> doesn't work.
>

> Also, the syntax is great for supportingpolymodel, but if you want to

> later add support for expando classes, it might be a bit awkward.
>
> ~Dave
>
> On Sep 4, 10:34 am, mdipierro <mdipie...@cs.depaul.edu> wrote:
>

> > Can you help testingpolymodel? In trunk:

>
> > db=DAL('gae')
> > db.define_table('contact',Field('address'),polymodel=True)
> > db.define_table('person',Field('first_name'),polymodel=db.contact)
> > db.define_table('company',Field('business_name'),
> >polymodel=db.contact)
>
> > db.person.insert(first_name="John", address="here')
> > db.company.insert(business_name="John Inc", address="there')
> > contacts = db(db.contact.id>0).select() # should lists both persons
> > and conpanies
>
> > Massimo
>
> > On Sep 2, 11:07 am, mdipierro <mdipie...@cs.depaul.edu> wrote:
>

> > > I agree thatpolymodelwould be trivial. We would only need to decide

> > > how to pass parameters to db.define_table to determine if a table is a

> > >polymodelor extends an existing one. This would not have an

> > > equivalent in SQL.
>
> > > On Sep 2, 10:40 am, Dave <thefe...@gmail.com> wrote:
>
> > > > I'm new to Web2py and trying to get a sense for its GAE support,
> > > > beyond the basic "RDBMS-like" functions (which Web2py seems to support
> > > > very well!)
>

> > > > Expando models andPolyModel:  Is there any way to use either of these

[Dave]

Using the test code you provided before, I now get this error:

---------------------------------------------------------------
Traceback (most recent call last):
File "C:\Users\Dave\Documents\Python\web2py\gluon\restricted.py",
line 188, in restricted
exec ccode in environment
File "C:\Users\Dave\Documents\Python\web2py\applications\welcome/
models/db.py", line 79, in <module>

db.define_table('person',Field('first_name'),
polymodel=db.contact)

File "C:\Users\Dave\Documents\Python\web2py\gluon\contrib\gql.py",
line 128, in define_table
fields.insert(0,args['polymodel'])
AttributeError: 'tuple' object has no attribute 'insert'
---------------------------------------------------------------

That points to one of the new lines of code you added in rev 854.

~Dave

On Sep 8, 9:36 am, mdipierro <mdipie...@cs.depaul.edu> wrote:
> Can you please try again using latest trunk?
>
> Massimo
>

[mdipierro]

I think this is fixed now but needs another test.

[Dave]

Line 128 of gql.py gives a syntax error:

fields=[args['polymodel']]+[field fof field in fields]

I took a guess that this was supposed to give a list of fields from
both the base table and the table being created in, and replaced it
with this instead:

fields = map(lambda x: args['polymodel'][x], args['polymodel'].fields)
+list(fields)

This change allowed the table to be constructed with fields from both
itself and its base class. Unfortunately, this gives an error when
_create is called on the table (line 137), due to duplicate properties
from the base table. Here's the trace:

Traceback (most recent call last):
File "C:\Users\Dave\Documents\Python\web2py\gluon\restricted.py",
line 188, in restricted
exec ccode in environment
File "C:\Users\Dave\Documents\Python\web2py\applications\welcome/

models/db.py", line 80, in <module>

db.define_table('person',Field('first_name'),
polymodel=db.contact)
File "C:\Users\Dave\Documents\Python\web2py\gluon\contrib\gql.py",

line 137, in define_table
t._create(polymodel=args.get('polymodel',None))

File "C:\Users\Dave\Documents\Python\web2py\gluon\contrib\gql.py",

line 250, in _create
self._tableobj = classobj(self._tablename,
(polymodel._tableobj, ), myfields)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine
\ext\db\polymodel.py", line 125, in __init__
super(PolymorphicClass, cls).__init__(name, bases, dct,
map_kind=False)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine
\ext\db\__init__.py", line 418, in __init__
_initialize_properties(cls, name, bases, dct)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine
\ext\db\__init__.py", line 336, in _initialize_properties
raise DuplicatePropertyError('Duplicate property: %s' % attr_name)
DuplicatePropertyError: Duplicate property: address

[mdipierro]

OK. we are getting closer... I think I can fix this tonight.

[Massimo Di Pierro]

would you do one more test using this gql.py?

[Dave]

It still has both the original problems. Here's what I'm testing so
far:

>contacts = db(db.contact.id>0).select()

Returns all contacts, as it should

>people = db(db.person.id>0).select()

Also returns all contacts, instead of just the contacts that are of
type "person"

>address = people[0].address

Works, and returns the address of the first person.

>name = people[0].first_name

Crashes, with the following trace:

Traceback (most recent call last):
File "C:\Users\Dave\Documents\Python\web2py\gluon\restricted.py",
line 188, in restricted
exec ccode in environment
File "C:\Users\Dave\Documents\Python\web2py\applications\welcome/

controllers/default.py:index", line 66, in <module>
File "C:\Users\Dave\Documents\Python\web2py\gluon\globals.py", line
96, in <lambda>
self._caller = lambda f: f()

File "C:\Users\Dave\Documents\Python\web2py\applications\welcome/

controllers/default.py:index", line 23, in index
File "C:\Users\Dave\Documents\Python\web2py\gluon\sql.py", line 729,
in __getattr__
return dict.__getitem__(self,key)
KeyError: 'first_name'

[geoff]

http://web2py.com/book/default/chapter/06#Indexes

The example appears to be missing a quote.
Adding the missing single-quote I am still getting a syntax
error:

Error traceback

File "/web2py/web2py/gluon/restricted.py", line 186, in

restricted
exec ccode in environment

File "/web2py/web2py/applications/ewc/models/db.py", line 108,
in <module>
db.executesql('CREATE INDEX IF NOT EXISTS idx_filemaker_id
ON recipe filemaker_id;')
File "/web2py/web2py/gluon/sql.py", line 1428, in executesql
self._execute(query)
File "/web2py/web2py/gluon/sql.py", line 947, in <lambda>
self._execute = lambda *a, **b: self._cursor.execute(*a,
**b)
OperationalError: near "filemaker_id": syntax error


it is for the line:
db.executesql('CREATE INDEX IF NOT EXISTS idx_filemaker_id ON
recipe
filemaker_id;')

I have a db.recipe.filemaker_id of type 'integer'


Generally, I am importing some information from a few different
sources
into what will be a unified database. Because of this, there
are
multiple keys to manage.

I have a db.recipes table which contains a field, filemaker_id.

I am importing into db.nutrition_information from a large csv
file, which has as its key a column that
maps to db.recipe.filemaker_id

I am linking db.nutrition_information to db.recipe via
db.nutrition_information.external_key

(things other than recipes will be linked to
db.nutrition_information)

The problem is, for entry into db.nutrition_information, I have
to look up db.recipe.id via db.recipe.filemaker_id

This is why I want to create an index on db.recipe.filemaker_id
as the import process is very slow right now. Although I will
only have to do it once, it is nice to do as many test imports
as possible.

I think the code I am using may be inefficient, I hope someone
will take a look at it:

recipes = db(db.recipe.filemaker_id==rows[0]).select()
external_key = recipes[0].id #filemaker_id is unique

[mdipierro]

This is now fixed. Here is a final example of usage:

db=DAL('gae')
db.define_table('contact',Field('address'), polymodel=True)

db.define_table('person',Field('first_name'), db.contact,
polymodel=db.contact)
db.define_table('company',Field('business_name'),db.contact,polymodel=db.contact)

if not db(db.contact.id>0).count():

db.person.insert(first_name="John", address='here')
db.company.insert(business_name="John Inc", address='there')

persons = db(db.person.id>0).select()
companies = db(db.company.id>0).select()
contacts = db(db.contact.id>0).select()
# latter includes both people and companies


Mind that you can do:

db.define_table('person',Field('first_name'), db.contact,
polymodel=db.contact)

or

db.define_table('person',db.contact, Field('first_name'),
polymodel=db.contact)

I.e. you must choose explicitly whether the fields of the extended
class should appear after or before the new fields.

[Dave]

Sounds great! I'm on vacation this week, but I look forward to trying
it out next week.

~Dave

[mdipierro]

This is not in web2py 1.84.1

[Alvin SW]

Hi Pierro,

About the polymodel, I think that it may have an equivalence in SQL. I
know pgsql has inheritance feature for its tables.
http://www.postgresql.org/docs/8.1/static/ddl-inherit.html
Would it be similar to polymodel?

On Sep 3, 2:07 am, mdipierro <mdipie...@cs.depaul.edu> wrote:
> I agree thatpolymodelwould be trivial. We would only need to decide
> how to pass parameters to db.define_table to determine if a table is apolymodelor extends an existing one. This would not have an
> equivalent in SQL.
>

[mdipierro]

Looks the same. We could add support for it but I think it is a
PostgreSQL specific feature.

Massimo

On Sep 18, 7:38 pm, Alvin SW <a...@nuansa.org> wrote:
> Hi Pierro,
>
> About the polymodel, I think that it may have an equivalence in SQL. I

> know pgsql has inheritance feature for its tables.http://www.postgresql.org/docs/8.1/static/ddl-inherit.html

[Dave]

I gave it a quick test, and it seems to work for all the simple tests
I tried this time. Great work! I'll see how it holds up once I start
using it for more than simple tests. Thanks!

~Dave