StringListProperty validates on read?!

[Bryan Donlan]

Hi,

I've run into a problem where the SDK reports a BadValueError on a /
read/ operation - but not the corresponding write.

My model looks like:
class PingSummary(db.Model):
dest = db.StringProperty(required = True)
successpings = db.StringListProperty()
failurepings = db.StringListProperty()
timestart = db.IntegerProperty(required = True)

order = db.IntegerProperty(required = True)
subsumed = db.BooleanProperty(default = False)
included = db.StringListProperty()

and I uploaded my (very incomplete) code to http://fushizen.net/~bd/sinemon.tbz2

The following set of requests will trigger the error:
curl -v 'http://localhost:8080/ping?
pingtime=0&monitorhost=test&dest=sine&success=1'
curl -v 'http://localhost:8080/summarize'

As you can see, it dies on a query:
Traceback (most recent call last):
File "/home/bd/opt/google_appengine/google/appengine/ext/webapp/
__init__.py", line 499, in __call__
handler.get(*groups)
File "/home/bd/opt/google_appengine/sinemon/main.py", line 34, in
get
models.updatesummaries()
File "/home/bd/opt/google_appengine/sinemon/sinemon/models.py", line
101, in updatesummaries
updateorder(order)
File "/home/bd/opt/google_appengine/sinemon/sinemon/models.py", line
90, in updateorder
for item in PingSummary.all().filter('order = ',
order).filter('subsumed =', False):
File "/home/bd/opt/google_appengine/google/appengine/ext/db/
__init__.py", line 1379, in next
return self.__model_class.from_entity(self.__iterator.next())
File "/home/bd/opt/google_appengine/google/appengine/ext/db/
__init__.py", line 927, in from_entity
instance = cls(None, **entity_values)
File "/home/bd/opt/google_appengine/google/appengine/ext/db/
__init__.py", line 555, in __init__
prop.__set__(self, value)
File "/home/bd/opt/google_appengine/google/appengine/ext/db/
__init__.py", line 372, in __set__
value = self.validate(value)
File "/home/bd/opt/google_appengine/google/appengine/ext/db/
__init__.py", line 2117, in validate
(self.name, self.item_type.__name__))
BadValueError: Items in the included list must all be basestring
instances

At this point, requests to the (debug dump) root will also return an
error:
Traceback (most recent call last):
File "/home/bd/opt/google_appengine/google/appengine/ext/webapp/
__init__.py", line 499, in __call__
handler.get(*groups)
File "/home/bd/opt/google_appengine/sinemon/main.py", line 19, in
get
for m in models.PingSummary.all():
File "/home/bd/opt/google_appengine/google/appengine/ext/db/
__init__.py", line 1379, in next
return self.__model_class.from_entity(self.__iterator.next())
File "/home/bd/opt/google_appengine/google/appengine/ext/db/
__init__.py", line 927, in from_entity
instance = cls(None, **entity_values)
File "/home/bd/opt/google_appengine/google/appengine/ext/db/
__init__.py", line 555, in __init__
prop.__set__(self, value)
File "/home/bd/opt/google_appengine/google/appengine/ext/db/
__init__.py", line 372, in __set__
value = self.validate(value)
File "/home/bd/opt/google_appengine/google/appengine/ext/db/
__init__.py", line 2117, in validate
(self.name, self.item_type.__name__))

BadValueError: Items in the included list must all be basestring
instances

Is this a bug in the SDK, and/or am I doing something wrong?

Thanks,

Bryan Donlan

[Marzia Niccolai]

Hi Bryan,
 It should do the validation, but this seems to be an issue with appending to a ListProperty:

obj.included.append(item.key())

item.key() is not a string, and must be cast to a string.  Changing it to this should work:

obj.included.append(str(item.key()))

I have filed an issue concerning lack of validation when calling append on a list property:
http://code.google.com/p/googleappengine/issues/detail?id=523

Thanks,
Marzia

[ryan]

it's actually not clear that append() should validate, and even if we
wanted it to, it'd be difficult to do. we do validation on property
assignment, but here, obj.included is a python list, and we're
modifying it in place, not assigning to it. the append() code path is
provided by python's list class, so no there's app engine code that
runs when that line is evaluated. we could provide our own list
subclass that does, but then you'd lose all of the built-in list
operators, like the above use of [1] for list creation.

given all that, it's doubtful that we'd actually change anything here.

[Bryan Donlan]

It'd be nice to validate on put - if it'd be too much of a hit in the
real thing (doubtful, as it validated on load), at least do it in the
SDK...

Also, in my code I'm just replacing the list entirely rather than
using .append - in such a case, would it be possible to hook the
update (again, only in the SDK as a debugging aid, if it'd be too
slow)?

[Blixt]

I figured I'd chime in here too... I wrote a simple wrapper class for
the list class that'll validate the type of values being added to the
list. As has been mentioned, this will have an impact on performance,
although I'd wager that impact is dwarfed by everything else going on
in the datastore when storing a list. The best solution might be to
implement your own property class that'll do the validating so you can
choose whether you want to use a couple of extra CPU cycles for the
sake of validation.

Here's my wrapper class along with an example of how it could be used
by the db module:
http://paste.blixt.org/501

Regards,
Andreas

On Jun 26, 4:32 am, "Bryan Donlan" <bdon...@gmail.com> wrote: