GAE Python DB change Broke My App!

[Steve]

Revision 262 comitted on May 22 shows a change to " /trunk/python/google/appengine/ext/db/__init__.py" which added a default_kind=Expando inside  model_from_protobuf. This has broken my application for the last couple weeks. The change is not something I can override through exposed parameters. How am I supposed to fix this without significantly rewriting my app?

[Barry Hunter]

Maybe it would help to actually explain how you are using it - what
exactly is broken?

Just saying 'its broken' doesnt go very far to explain the issue you
facing. Let alone being enough for anyone to figure out a solution
that could help you.

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/bVd8KrYiaxwJ.
> To post to this group, send email to google-a...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengi...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.

[Steve]

I'm trying to dig in to what exactly is crashing.  It looks like I'm now getting an Expando instance where one is not expected.  I'm not sure what was expected in these crash cases.  I haven't looked at this code in 2 years, so it's taking a lot to re-familiarize myself with what's going on.  I didn't expect backward incompatible changes.

--Steve

On Saturday, June 2, 2012 2:33:11 PM UTC-10, barryhunter wrote:

Maybe it would help to actually explain how you are using it - what
exactly is broken?

Just saying 'its broken' doesnt go very far to explain the issue you
facing. Let alone being enough for anyone to figure out a solution
that could help you.


On Sun, Jun 3, 2012 at 12:48 AM, Steve <unetright...@xoxy.net> wrote:
> Revision 262 comitted on May 22 shows a change to "
> /trunk/python/google/appengine/ext/db/__init__.py" which added a
> default_kind=Expando inside  model_from_protobuf. This has broken my
> application for the last couple weeks. The change is not something I can
> override through exposed parameters. How am I supposed to fix this without
> significantly rewriting my app?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/bVd8KrYiaxwJ.

> To post to this group, send email to google-appengine@googlegroups.com.

> To unsubscribe from this group, send email to

> google-appengine+unsubscribe@googlegroups.com.

[Steve]

The problem seems to be that previously I could encode "None" to a protobuf and get a "None" back from decoding.  Now instead of "None", I get the new default_kind Expando.

--Steve

On Saturday, June 2, 2012 2:33:11 PM UTC-10, barryhunter wrote:

Maybe it would help to actually explain how you are using it - what
exactly is broken?

Just saying 'its broken' doesnt go very far to explain the issue you
facing. Let alone being enough for anyone to figure out a solution
that could help you.


On Sun, Jun 3, 2012 at 12:48 AM, Steve <unetright...@xoxy.net> wrote:
> Revision 262 comitted on May 22 shows a change to "
> /trunk/python/google/appengine/ext/db/__init__.py" which added a
> default_kind=Expando inside  model_from_protobuf. This has broken my
> application for the last couple weeks. The change is not something I can
> override through exposed parameters. How am I supposed to fix this without
> significantly rewriting my app?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/bVd8KrYiaxwJ.

> To post to this group, send email to google-appengine@googlegroups.com.

> To unsubscribe from this group, send email to

> google-appengine+unsubscribe@googlegroups.com.

[Steve]

I'm mistaken.  It's not a problem with Nones.  I still can't figure out why sometimes I'm getting an Exapndo and my app crashes.  My app doesn't use any Expandos.  Google, why u make backwards incompatible change??

--Steve

[Alfred Fuller]

Hi Steve,

We try to avoid backwards incompatible changes as much as possible.

You will get an expando when the decoded proto does not have a key (as in this case the correct model class cannot be determined, see http://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/api/datastore.py#1086). Are you passing in an empty string to the decode function? Anything that produces an expando should have failed to encode and decode before (with an exception). Do you have a byte string ("print '%r' % encoded_proto") you can send me for testing?

Thanks,

Alfred

To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/CCbrNcjrfvYJ.

To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.

[Steve]

Thanks for the reply Alfred.  Sorry for the delayed response.  I'm only able to work on this on the weekend and I spent all last weekend porting to a different framework to debug the problem.

TL;DR - Despite crashing more often than not, I could never identify where the keyless encoded entities were being created.

My app was last deployed around Thanksgiving 2010 using whatever was the then current sdk.  It was created on top of the Kay framework.  When I installed the latest SDK (to get this code breaking change), Kay spit out errors.  Even after I upgraded to the latest Kay release the dev appserver still wouldn't run.  So I couldn't reproduce the problem in an environment where I could dig in to the entities causing the crash.

On the production servers, I could track down that the problems were from decoding memcache'd entities.  But since you can't browse the memcache entries (like the datastore) without knowing specific keys, I couldn't dig very deep there either.

Finally I gave up and started porting my app off of Kay and on to the latest webob & webapp2 releases that still support python 2.5.  When I was able to redeploy on that foundation, the problem disappeared.  I was pretty aggravated to have my app crashy for so long and then to lose a weekend to porting it, but at least the update has gotten me 75% of the way towards getting the app ported to the 2.7 runtime.

Thanks again for the reply with the source link.  If the problem rears it's head again I'll be better prepared to know where to look.

Regards,

Steve